I’m a data scientist working in consulting (EU) and I would like to push my company to delve more into open-source LLMs over closed ones (OpenAI…).

I’m constantly amazed by the progress shown by open-sourced LLMs in this sub. The increasing efficiency of models coupled with the expanding open-source LLM ecosystem make these models more and more competitive.

I keep thinking that there’s a market for custom LLMs, notably for private-sensitive companies like banks and insurances.

Any insights?

  • LuluViBritannia@alien.topBEnglish
    1·
    1 year ago

    The entire market will eventually use local LLMs, it’s simply as that.

    Online services are never an ideal solution for any business. It’s not just about privacy.

    - The owners can do whatever they want, so if they change settings or even simply shut down, you’re screwed.

    - Online services are a public traffick, so in case of high-density, it bottlenecks. Just like highroads, expect major slowdowns of your job if you use an online service that happens to be saturated. And slowdown means financiary loss.

    - In case of internet issues, you’re screwed.

    - you have to pay for the service, which can get freaking expensive depending on how much you use it.

    ​

    Local LLMs have none of these issues. And more than that:

    - While general intelligence like ChatGPT or Claude is incredible, it will never be enough for every use case. There will always be cases where you need a more specialized alternative, even if less intelligent.

    - The gap between the big ones and local LLMs is frankly not that high. I’m not going to say “they’re as intelligent as ChatGPT!”, but as a matter of fact, everything I was able to make with ChatGPT, I succeeded with a local LLM as well or even better. Analysing code and rewriting it/completing it? Managed with a 7B. Writing creative short stories? Easy even with a 7B.

    - An online service has its own abilities, and the devs can update it but you have no guarantee they will. In the case of LLMs, context length matters so much! OpenAI did raise GPT’s context length regularly, but what if they don’t?

    - Intelligence isn’t the only point about an AI! A local LLM has its own default language style, and even the big ones are hard to steer away from it. ChatGPT’s answers, for example, are very lengthy, constantly. With a Local LLM, it’s easier to steer. You can even force it to adopt a certain format.