Title says it all. Why spend so much effort finetuning and serving models locally when any closed-source model will do the same for cheaper in the long run. Is it a philosophical argument? (As in freedom vs free beer) Or are there practical cases where a local model does better.
Where I’m coming from is the requirement of a copilot, primarily for code but maybe for automating personal tasks as well, and wondering whether to put down the $20/mo for GPT4 or roll out my own personal assistant and run it locally (have an M2 max, compute wouldn’t be a huge issue)
I use it for development. All the things mentioned are nice, but there’s no way I could afford to do development using a paid service. I pass/generate way too many tokens and my company hasn’t really sponsored my work yet.
Having chatgpt write a pirate poem hardly costs a thing. Getting an llm to summarize a bunch of search results, or read an email inbox flagging certain scenarios, or parse through a codebase looking for specific features gets very, very expensive fast.