Looking to spend between $300 to $800 max for a gpu that will run ai models efficiently. The 1080 TI has 11 GB of ram, but no tensor cores, so it seems like not the best choice.
Just got a new rig, with a 3080 super, which I thought would be good, but it only has 8 GB of ram, big bummer, so I want to replace it with something that will do a better job. What should I get?
I am not sure what you call “run ai models”
I train on GPU but then i move to prod and it runs on CPU. The inference is not near as computation intensive as training so CPU might be enough.
I converted my model to onnx format, wrote code in C# and packaged it like AWS lambda function. My AI model is not called constantly and is only a small part of my project. So my AWS bill is literally couple dollars a month.