Amazon has the Acer A770 on sale for $250. That’s a lot of compute with 16GB of VRAM for $250. There is no better value. It does have it’s challenges. Somethings like MLC Chat run with no fuss just like on any other card. Other things need some effort like Oob, Fastchat and BigDL. But support for it is getting better and better everyday. At this price, I’m tempted to get another. I have seen some reports of running multi-GPU setups with the A770.
It also comes with Assassins Mirage for those people that still use their GPUs to game.
3090’s draw 350W as per google. Arc a770 draws 225.
I guess people here normally go with a dual 3090 setup. That’s 700 watt for 48gb of VRAM. This comes out to 14.58 watt per gigabyte of VRAM.
Lets assume you manage to cool properly, you could probably run 4 A770 for 64gb of VRAM which sounds pretty nice. This watt to VRAM gigabyte ratio is 14.06 which is actually better than the nvidia cards. Also noteworthy, the pins are 1x8 pin & a 1.6 pin if I’m reading correctly. So you would have to be careful what mobo you went with but I’m pretty sure that’s doable.
It all comes down to driver support, but it could be a really nice rig that would you to run more complex models on the cheap. I’m not taking into account driver support which Nvidia obviously has an advantage in, this would affect the processing speeds of the A770. I would say however, the addition of cheap VRAM would probably be worth the extra processing times so long as things actually worked.
My 3090’s don’t draw 350W for inference, more like ~200 tops.
I did manage to draw 350W from one by running whisper to subtitle something though.
There is also the issue of pcie slots. Currently running a second card in a x4 slot and it’s noticeably slower. Getting four full speed x16 slots is going to be some pretty specialised equipment. All the crypto rigs are slow slots to my knowledge since it doesn’t matter there
It is good to see more competitive cards in this space though. Dual 770 could be very accessible