Using and losing lots of money on gpt-4 ATM, it works great but for the amount of code I’m generating I’d rather have a self hosted model. What should I look into?
Using and losing lots of money on gpt-4 ATM, it works great but for the amount of code I’m generating I’d rather have a self hosted model. What should I look into?
As far as self hosted models go, deepseek-coder-33B-instruct is the best model I have found for coding. Anecdotally it seems more coherent and gives better results than Phind-CodeLlama-34B-v2.
Think this would be good-enough/suitable to use with AutoGPT/BabyAGI type situations? This is my main use-case, for bulk inspiration if not productivity. The API’s can get expensive if left on full-automatic overnight.
I wanna do something similar, please let me know what conclusion you reach to
What environment do you use to interact with self-hosted code models when coding? I’ve been using and enjoying Cursor for the way it’s integrated into the IDE, but I’ve been exploring options for going self-hosted just to feel freer from whatever record I’m putting on someone else’s server.
My code editor of choice (Helix) doesn’t support integrations or plugins so I haven’t tried Cursor or Copilot. I’m building my own UI right now that focuses on first-class support for models served by llama.cpp.