I was looking into Galactica the other day, and it’s available as a “pip install”, that on first use downloads the model (260GB download LOL) and sets up everything. I got a slight headache by looking for hours through different small models, what I need to download etc, has any other model, a really small one for my ssh-only server without CUDA, been packaged like that? Though I wouldn’t mind a model that fits in my 8GB laptop RTX either.
You must log in or register to comment.
You can use hf hub, see: https://huggingface.co/docs/hub/models-downloading
I doubt 1GB will give you much in the way of conversational skills from the AI.
I suspect that 7B is about usable with 13B getting there.