Could you please share a citation for the mentioned research papers?

I’m interested in seeing this as well.

He probably means that, although scaling might still deliver better loss reduction, this won’t necessarily cash out to better performance “on the ground”.

Subjectively, GPT4 does feel like a smaller step than GPT3 and GPT2 were. Those had crazy novel abilities that the previous one lacked, like GPT3’s in-context learning. GPT4 displays no new abilities.* Yes, it’s smarter, but everything it does was possible, to some limited degree, with GPT3. Maybe this just reflects test saturation. GPT4 performs so well that there’s nowhere trivial left to go. But returns do seem to be diminishing.

(*You might think of multimodality, but they had to hack that into GPT4. It didn’t naturally emerge with scale, like, say, math ability.)

I think you have to use a reasonably smart human as a baseline, otherwise literally any computer is AGI. Babbage’s Analytical Engine from 1830 was more intelligent than a human in a coma.

Hi ChatGPT.

Greg Brockman has quit:

https://twitter.com/gdb/status/1725667410387378559