mcmoose1900@alien.topB to LocalLLaMA@poweruser.forumEnglish · 2 years agoNouse-Capybara-34B 200Khuggingface.coexternal-linkmessage-square12linkfedilinkarrow-up11arrow-down10
arrow-up11arrow-down1external-linkNouse-Capybara-34B 200Khuggingface.comcmoose1900@alien.topB to LocalLLaMA@poweruser.forumEnglish · 2 years agomessage-square12linkfedilink
minus-squareCombinatorilliance@alien.topBlinkfedilinkEnglisharrow-up1·2 years agoI believe these are TheBloke’s GGUF quants if anyone’s interested: https://huggingface.co/TheBloke/Nous-Capybara-34B-GGUF
minus-squareWolframRavenwolf@alien.topBlinkfedilinkEnglisharrow-up1·2 years agoAlso note this important issue that affects this and all other Yi-based models: BOS token as 1 seriously hurts these GGUF Yi models
minus-squarea_beautiful_rhind@alien.topBlinkfedilinkEnglisharrow-up1·2 years agoSo we can just skip BOS token on all these models?
minus-squareambient_temp_xeno@alien.topBlinkfedilinkEnglisharrow-up1·2 years agoI did the gguf-py/scripts/gguf-set-metadata.py some-yi-model.gguf tokenizer.ggml.bos_token_id 144 and it’s changed the outputs a lot from yesterday.
I believe these are TheBloke’s GGUF quants if anyone’s interested: https://huggingface.co/TheBloke/Nous-Capybara-34B-GGUF
Also note this important issue that affects this and all other Yi-based models:
BOS token as 1 seriously hurts these GGUF Yi models
So we can just skip BOS token on all these models?
I did the gguf-py/scripts/gguf-set-metadata.py some-yi-model.gguf tokenizer.ggml.bos_token_id 144
and it’s changed the outputs a lot from yesterday.