Looking to spend between $300 to $800 max for a gpu that will run ai models efficiently. The 1080 TI has 11 GB of ram, but no tensor cores, so it seems like not the best choice.
Just got a new rig, with a 3080 super, which I thought would be good, but it only has 8 GB of ram, big bummer, so I want to replace it with something that will do a better job. What should I get?
If you have a good PSU( above 850w and recent ), a used 3090 is probably your best bet.
I am not sure what you call “run ai models”
I train on GPU but then i move to prod and it runs on CPU. The inference is not near as computation intensive as training so CPU might be enough.
I converted my model to onnx format, wrote code in C# and packaged it like AWS lambda function. My AI model is not called constantly and is only a small part of my project. So my AWS bill is literally couple dollars a month.
I got a membership on google colab and I think it’s really worth it compared to buying your own GPU. The GPUs that you can use there are so much faster than what you can buy with that budget.
Depends on what your requirements for AI models are, both now and in the future.