Couldn’t wait for the great TheBloke to release it so I’ve uploaded a Q5_K_M GGUF of Intel/neural-chat-7b-v3-1.
From some preliminary test on PISA sample questions it seems at least on par with OpenHermers-2.5-Mistral-7B
Couldn’t wait for the great TheBloke to release it so I’ve uploaded a Q5_K_M GGUF of Intel/neural-chat-7b-v3-1.
From some preliminary test on PISA sample questions it seems at least on par with OpenHermers-2.5-Mistral-7B
For anyone wondering, you can actually rent Gaudi from Intel’s Dev Cloud to finetune like this:
https://eduand-alvarez.medium.com/llama2-fine-tuning-with-low-rank-adaptations-lora-on-gaudi-2-processors-52cf1ee6ce11
https://developer.habana.ai/intel-developer-cloud/
The blog cites $10/hour for 8 HPUs.