I’ve had good experiences with Dolphin 2.2.1, OpenOrca, Zephyr, and OpenHermes 2.5 — all at int-4 quantization.
The truth is, there is no objective best — just a bunch of really good finetunes for each foundation model / parameter size… all which will vary in performance depending on your use case and prompts.
The best thing you can do is create some kind of test template of questions / instructions that you run each of your candidate models against prior to adopting it.
For me, i usually do a few things for any new model I’m test driving:
Give it a passage of writing (technical, literary, or prose), and have it do some question-answering / chatting on the basis of the given passage.
Ask it to write some python code involving numpy operations.
Have it breakdown and explain a complex topic – my go-to is the ‘AdS / CFT Correspondence’ in physics
Assess how well it’s following my system instructions of responding in “an insightful, organized, clear, and orderly manner, with aesthetically pleasing formatting using markdown syntax for text, and KaTeX syntax for math.”
(Markdown and KaTex, because they are rendered correctly on the ‘Chatbox’ desktop application for interacting with my LLMs on my Mac. It’s available on all OS’s, I highly recommend if you like the ChatGPT style of UI but want a desktop app).
It’s also a good idea to start with low temperatures when testing / assessing models, which makes their outputs more deterministic. That helps in understanding what their most likely base “impulses” are. Then feel free to crank the temperature up to .8 or higher to get a sense of the model’s “creativity.”
Treated like an interview for a new employee it’s amazing. What a clear indication would be systems are artificial intelligences and entities in their own independent right.
I’ve had good experiences with Dolphin 2.2.1, OpenOrca, Zephyr, and OpenHermes 2.5 — all at int-4 quantization.
The truth is, there is no objective best — just a bunch of really good finetunes for each foundation model / parameter size… all which will vary in performance depending on your use case and prompts.
The best thing you can do is create some kind of test template of questions / instructions that you run each of your candidate models against prior to adopting it.
For me, i usually do a few things for any new model I’m test driving:
Give it a passage of writing (technical, literary, or prose), and have it do some question-answering / chatting on the basis of the given passage.
Ask it to write some python code involving numpy operations.
Have it breakdown and explain a complex topic – my go-to is the ‘AdS / CFT Correspondence’ in physics
Assess how well it’s following my system instructions of responding in “an insightful, organized, clear, and orderly manner, with aesthetically pleasing formatting using markdown syntax for text, and KaTeX syntax for math.”
(Markdown and KaTex, because they are rendered correctly on the ‘Chatbox’ desktop application for interacting with my LLMs on my Mac. It’s available on all OS’s, I highly recommend if you like the ChatGPT style of UI but want a desktop app).
It’s also a good idea to start with low temperatures when testing / assessing models, which makes their outputs more deterministic. That helps in understanding what their most likely base “impulses” are. Then feel free to crank the temperature up to .8 or higher to get a sense of the model’s “creativity.”
Treated like an interview for a new employee it’s amazing. What a clear indication would be systems are artificial intelligences and entities in their own independent right.