I just recently wrote my own pure python/chromadb program but before i had great success in oogabooga and this model. I think maybe there is a setting that is overlooked that maybe i enabled in oobabooga or maybe its one of the generation kwargs that just seems to work flawlessly. The model has issues with keeping its self separate from the user so take care in your wording in the system message too.

having seen the model’s tokenizer.default_chat_template that isnt unbelievable, its a real mess with impossible conditions.

My health is keeping me from making a better response but If you’re dead set on using it message me and we’ll work it out together. I like this model the most.

Im using this and its shockingly great:
https://huggingface.co/TheBloke/Xwin-MLewd-7B-V0.2-GPTQ

Just discovering TheBloke/Xwin-MLewd-13B-v0.2-GPTQ

Can this query my docs too?