Reuters is reporting that OpenAI achieved an advance with a technique called Q* (pronounced Q-Star).
So what is Q*?
I asked around the AI researcher campfire and…
It’s probably Q Learning MCTS, a Monte Carlo tree search reinforcement learning algorithm.
Which is right in line with the strategy DeepMind (vaguely) said they’re taking with Gemini.
Another corroborating data-point: an early GPT-4 tester mentioned on a podcast that they are working on ways to trade inference compute for smarter output. MCTS is probably the most promising method in the literature for doing that.
So how do we do it? Well, the closest thing I know of presently available is Weave, within a concise / readable Apache licensed MCTS lRL fine-tuning package called minihf.
https://github.com/JD-P/minihf/blob/main/weave.py
I’ll update the post with more info when I have it about q-learning in particular, and what the deltas are from Weave.
You must log in or register to comment.
Yeah I think its MCTS reinforcement learning algorithm. I think DeepMind is the best lab when it comes to depeloping strategy and planning capable agents, given how good AlphaZero and AlphaGo is, and if they integrate it with the “Gemini” project, they really might just “ecliplse” GPT-4. I don’t know how scalable it would be in terms of inference given the amount of compute required.
Have DeepMind released any leading-edge tools recently? MuZero was quite a few years ago now, and AlphaGo is ancient in AI terms.
DeepMind seem to have promised an awful lot, come up with a lot of clever announcements, but been very sparse on actual delivery of much at all.
Has to be a mix of Q-learning and A* right?
I know you’re joking, but it’s hilarious how many random things in science just got given letters.
A* is the algorithm your phone uses to help you drive home…and the supermassive black hole in the centre of the galaxy.
It’s also a star.
What is a star?
…and the supermassive black hole in the centre of the galaxy.
What did you think they were gonna use for that? Djikstra’s?
I was going to say it seems like it was just yesterday I was learning A* and now I find out that they are already up to Q*
Can you please ELI-idiot?
Mayve an A* search in vector space
Let the co-founder of OpenAI John Schulman explain it to you
This should be higher up.
explain it to you
lol, might as wel spoken mandarin, thhis is so far away from my math skills. I have no clue what this guy is saying
What’s so special about Q*
A marketing piece by OpenAI to lie to people to hype product
I think there a few llms that incorporate mcts on github
Q* is just a reinforcement learning technique.
Perhaps they scaled it up and combined it with LLMs
Given their recently published paper, they probably figured out a way to get GPT to learn their own reward function somehow.
Perhaps some chicken little board members believe this would be the philosophical trigger towards machine intelligence deciding upon its own alignment.
Given their recently published paper, they probably figured out a way to get GPT to learn their own reward function somehow.
you just need 2 GPTs talking with each other. the seconds acts as a critic and guides the first
Which paper are you referring to?
We can infer that any such advance by OpenAI that follows the naming convention of “Q*” would likely be a significant development in the field of reinforcement learning, possibly expanding upon or enhancing traditional Q-Learning methodologies.
Thanks, ChatGPT
Wasn’t there a big thing about tree search just a few months ago? haven’t been keeping up too much.
If it teaches itself to learn it’s just a matter of time until it teaches itself to code
I’m wondering if Q-Star is a recursive self improvement mechanism? Perhaps the in house model they have can innovate and consistently learn on top of what it’s been trained on?
I heard they have an even bigger breakthrough up their sleeve… Rumor is that it’s called GPT2, and it’s too dangerous to even release to the public 👀
It’s a silicon based version of Qanon. I will be terminated by telling you but wait 'till they launch MAGA (Machine Augmented General AI) !!!
FREEDUM
gguf when
Calling it Q was a terrible idea. The cookers are going to go crazier
I’m not a fan of tech companies in general but Amazon is definitely one of most disliked.