Does anyone have a setup that works with a Yi34b model on 12GB vram and 32GB ram?
I have tried using GGUF but I always end up with the <s> answer.
I’m on Oobabooga and any specific settings to overcome this hurdle with GGUF would be greatly appreciated!
I</s>
Does anyone have a setup that works with a Yi34b model on 12GB vram and 32GB ram? I have tried using GGUF but I always end up with the <s> answer. I’m on Oobabooga and any specific settings to overcome this hurdle with GGUF would be greatly appreciated! I</s>