Basically, that’s my question. The caveat is that I would like to avoid a Mac Mini and I wonder if some of Minisforum’s mini PCs can handle LLM.
Basically, that’s my question. The caveat is that I would like to avoid a Mac Mini and I wonder if some of Minisforum’s mini PCs can handle LLM.
I don’t know what specs mini PCs tend to have, but I can run OpenHermes 2.5 on my laptop with an Intel Core i5-8365U CPU and 24 gigs of ram. Even without a graphics card, I get responses in less than a minute with a bunch of memory tokens.