One of the prevalent doomerist ideas about AI is that big tech will control it all in the future. Yet reality is behaving totally differently. Open-source AI seems only a few months behind big tech.
Which is why big tech is currently trying to convince world leadership that only they can be trusted to properly align powerful AI models. It’s almost time to do the ol’ ladder pull.
Corporations would love open source Ai models since it saves them the money that they’d have to pay to someone like openai.
Can this be easily self hosted?
The problem is most of these models need like a terabyte of VRAM… And consumers have about 8-24GB.
Holy shit a terabyte?
This specific one says it’ll run on 24GB actually. But some are just crazy big.
There are smaller models that can run on most laptops.
In benchmarks this looks like it is not far off Chat-GPT 3.5.
It’s not even close, less than half of 3.5’s 85.5% in ARC. Some larger Open models are competitive in Hellaswag, TruthfulQA and MMLU but ARC is still a major struggle for small models.
3Bs are kind of pointless right now because the machines with processors capable of running them at a usable speed probably have enough memory to run a 7B anyway.
It’s a solid model but comparing to gpt4 immediately removes any credibility that they had.
