VITS2 was published recently by the authors of VITS. If I understand correctly, it implements the use of transformers, runs more efficiently than VITS, and is capable of better voices too, provided the dataset. some folks make an open source implementation of it, with the help of the authors of the paper. See the GitHub repo

I find running an OpenAI style API endpoint (using llama.cpp directly when I want fine control, or StudioLM when I need something quick and easy) is the best way to go in combination with a good chat UI designed to interface with OpenAI models.

To that end, I redirect Chatbox to my local LLM server, and I LOVE IT. Clean but powerful interface, support for markdown, ability to save different agents for quick recall, and more. Highly, HIGHLY recommend it.

It’s open source and available on pretty much every platform – and you can use it to interface with both local LLM and with OpenAI LLM’s.

I’ve had good experiences with Dolphin 2.2.1, OpenOrca, Zephyr, and OpenHermes 2.5 — all at int-4 quantization.

The truth is, there is no objective best — just a bunch of really good finetunes for each foundation model / parameter size… all which will vary in performance depending on your use case and prompts.

The best thing you can do is create some kind of test template of questions / instructions that you run each of your candidate models against prior to adopting it.

For me, i usually do a few things for any new model I’m test driving:

1. Give it a passage of writing (technical, literary, or prose), and have it do some question-answering / chatting on the basis of the given passage.

2. Ask it to write some python code involving numpy operations.

3. Have it breakdown and explain a complex topic – my go-to is the ‘AdS / CFT Correspondence’ in physics

4. Assess how well it’s following my system instructions of responding in “an insightful, organized, clear, and orderly manner, with aesthetically pleasing formatting using markdown syntax for text, and KaTeX syntax for math.”

(Markdown and KaTex, because they are rendered correctly on the ‘Chatbox’ desktop application for interacting with my LLMs on my Mac. It’s available on all OS’s, I highly recommend if you like the ChatGPT style of UI but want a desktop app).

It’s also a good idea to start with low temperatures when testing / assessing models, which makes their outputs more deterministic. That helps in understanding what their most likely base “impulses” are. Then feel free to crank the temperature up to .8 or higher to get a sense of the model’s “creativity.”

Please do train on mistral! very much looking forward to seeing how that works, I’m LOVING the mistral models.