And they’d lose out on every single thing they built in-house.
Don’t assume Google, Inflection, Anthropic etc. are more than 1 yr behind.
I wonder how real this threat really is.
Sam Altman is not the only guy who knows how to monetize things. Also these employees are sitting on 10M+ bonuses that they will NEVER get from Microsoft. At most they will get 250K-500K TC.
Nationalized infrastructure built by megacorp contractors is my predicted future for AI research the 2030’s.
I don’t doubt there are multi-trillion parameter multi-modal models being dreamed up right now by the US DoD and OpenAI to run psyops online, detect and recruit agents, push against foreign propaganda, etc…
And that’s okay I’d rather that stable organizations had the reins rather than some l33Tcode bro (and in US’s case the org is controlled by a democratically ellected person).
As a guy in his mid 30s doing masters level courses at Stanford in prep for a possible part time masters this advice to stick with the basics for risk mitigation is much needed.
Measuring and improving quality of NLP datasets in a comprehensive way is probably the main migraine there.
You can measure and improve quality by many dimensions that practitioners disagree on… ( accuracy, completeness, consistency, timeliness, validity, and uniqueness are common ways to slice data quality) and there’s no consistent single measure for some of those either.
So do engineering innovations make sense for discussions?
I would love for someone to do extensive experimentation on best practices for RAG- including implications on cost, latency and performance (precision), etc.
It’s not the very latest per se, but knowing the differences and tricks to reduce wall time would be amazing.