Basically, that’s my question. The caveat is that I would like to avoid a Mac Mini and I wonder if some of Minisforum’s mini PCs can handle LLM.
Basically, that’s my question. The caveat is that I would like to avoid a Mac Mini and I wonder if some of Minisforum’s mini PCs can handle LLM.
How mini do you want? I plugged a llama2 7b into an N100 w/16gb and ran it, speed was not very good.
Real question is what are you trying to accomplish and is this the best route to do so?