I’m a data scientist working in consulting (EU) and I would like to push my company to delve more into open-source LLMs over closed ones (OpenAI…).

I’m constantly amazed by the progress shown by open-sourced LLMs in this sub. The increasing efficiency of models coupled with the expanding open-source LLM ecosystem make these models more and more competitive.

I keep thinking that there’s a market for custom LLMs, notably for private-sensitive companies like banks and insurances.

Any insights?

  • noco-ai@alien.topBEnglish
    1·
    1 year ago

    OpenAI is also so overwhelmed with demand that their API “limits” are a complete joke. I have a tier 4 account and have always hit server error issues when running any kind of single threaded application with their API well before hitting any of their published limits. I get more TPS from my local systems than I was ever able to get from the OpenAI API.