I have a good PC, but the problem is with my RAM. The model is LLM, which makes the memory requirement understandable, but I can’t afford to get that much RAM right now so I’m looking for a roundabout. Any help?
I have a good PC, but the problem is with my RAM. The model is LLM, which makes the memory requirement understandable, but I can’t afford to get that much RAM right now so I’m looking for a roundabout. Any help?
Paste your model name in this HF space, https://huggingface.co/spaces/Vokturz/can-it-run-llm
https://imgur.com/a/Ednemii (the result)
It seems you need less than 32 GiBs vram
And thanks to you, it now works. I knew exactly what to do and what I needed to get it to work. Thanks, man!