It seems to me that the next big boom for cloud computing will be offering to train and host models that understand the unique business domains it serves.
Are the smart corporations already training local LLMs to understand and answer questions about their business, or is this space too new to accommodate them?
I feel like some of you may be missing a huge business opportunity. You may not realize the value of what you have already researched.
Yes.
Maybe it’s just where I work, but our documentation is just not good enough to train a model. It’s a bunch of esoteric knowledge passed down verbally by the elders (tech leads).
One day, when they leave, it will be lost forever.
Yeah record every zoom call and meeting going forward do those transcripts can be converted into sops
My perspective as a Fortune 500 IT solutions architect… why would I spend a few million dollars and a year of project time to build out local infrastructure that’ll already be outdated by the time it’s installed, when I can just hand my developers and data team permissions on Azure to be able to immediately access the same or better resources for a fraction of the cost? Scale is value, and cloud service providers will always have far greater scale.
Yeah might fit in the US, but not in Europe. Dependencies can lead to problems. Especially when there might be a conflict. I would not want to run important infrastructure that is dependent on US services only.
All major cloud providers have data centers in Europe.
You are right. But if you have a chinese customer for example, there might come up different problems like with NVIDIA and GPUs. Independency is key for a lot of players.
Someone has not gone and sat down with the legal department.
- Who owns LLM generated work? IANAL but based on recent cases it looks like “not you” and “no one” because there is no human hand in the output. It seems that this puts it directly in the public domain.
- What does the TOS say at OpenAI and MS right now regarding these items. It doesn’t seem very clear and it looks like they are gonna use the OUTPUT for whatever they want. https://foundation.mozilla.org/en/campaigns/microsoft-ai/ Mozilla’s making it personal, it might just be a clear statement on “unowned” things that might be “owned” in the future.
Depending on your business, a LLM that does not tell stories, do porn, math, answer logic problems might be wasteful if you want to supercharge customer service by shoving in your own documentation. A thinner model might be cheaper to run at the scale of Fortune 500 CS than say anything azure is offering.
Without doing the math, and you need to have local hardware enough to do the math it’s nearly impossible to make any sort of cost benefit analysis.
Cloud for scale is NOT value, cloud for scale is COST… Value is an asset and deprecation on said asset. If you aren’t tracking your revenue vs expenses in cloud on a week over week basis, if you dont know your cloud costs per user, or customer (and those are going to be different depending on what you do and how you do it) there is zero corollary between cloud and value. The free money is gone, the belt is only gonna get tighter money needs to be in every metric…
why would I spend a few million dollars and a year of project time to build out local infrastructure that’ll already be outdated by the time it’s installed
That’s probably the argument for all cloud architecture.
Long-term cost and risk might be persuasive, but that hasn’t swayed IT managers thus far for non-LLM specific infrastructure. I am guessing it won’t do much to sway future IT managers.
I’m also assuming Azure will let you get very custom with the LLMs you can train via their services.
This gives me something to think about.
That’s probably the argument for all cloud architecture.
Long-term cost and risk might be persuasive, that hasn’t swayed IT managers thus far for non-LLM specific infrastructure
It’s 2023. What are you talking about? Where have you been?
Not everyone still uses the cloud, I still know people who run and manage physical clusters. This is mostly true for institutions such as hospitals, universities, etc. Using cloud solutions on these cases not just would add external dependencies but also much higher costs, for instance, handling and processing hundreds or thousands of terabytes of critical or scientific data.
This is the right answer. Unless LLM infrastructure and the model itself is your competitive advantage, time to market and simplicity is going to win every time. The good thing from the OpenAI fiasco is that abstraction layers are likely going to become more important.
The state-of-the-art on training and architecture is likely to improve over the next year alone, certainly over the next 2 or 3. It’s also reasonable to expect cheaper hardware for running LLMs, since all the chip makers are working on it.
If you don’t need a local LLM now but think it might save money only in the long run, it probably makes sense to wait and build one once we’re better at it
Collating training data in the mean time probably makes sense. Recording as much as you can, encouraging employees to document more, etc. That data will be useful even in the absence of AI, and with improving AI technology it is likely to become more and more valuable every year. It also takes time to produce that data, and no one else can do it for you
What’s the state of licensing anyway? Aren’t pretty much all the open source models currently available to run locally supposed to be for non-commercial use?
Most small businesses don’t know hours to use easy (commercial) AI let alone something as refined as this.
A lot of them don’t believe in it either.
Fall behind they will.
Really depends on how they are currently prioritising it and how are the dividing the resources. A lot of small businesses don’t have this on their roadmap for the near future and that’s a huge factor