Basically, that’s my question. The caveat is that I would like to avoid a Mac Mini and I wonder if some of Minisforum’s mini PCs can handle LLM.
Basically, that’s my question. The caveat is that I would like to avoid a Mac Mini and I wonder if some of Minisforum’s mini PCs can handle LLM.
How about a Zotac zbox? ZBOX QRP7N3500
Max supported memory is 64GB and it has an RTX3050 with 12GB VRAM. I expect that it could run 7b models easily.