Goliath easily kicks lzlv 70b to the crib. But it’s like an unruly horse, completely ignoring my prompts and directions in favor of whatever direction it wants to head too. Haven’t found any temps yet that make it as intelligent as lzlv, but sometimes it does shit that there’s no way lzlv would accomplish so it feels as if it’s finrtuning just need some more logic implemented.
Goliath 120b is the only model I tried so far that is not infested with GTP prose.
Goliath 120b is the only model so far I’ve tried that doesn’t ChatGTP out on me. The 3.0 quant fits on 2x 3900rtx.
I don’t think more context is actually the way to go for now. Most of the longer context models I found became very unreliable at higher contexts. And they become so slow too! Instead I use context injections trough Sillytavern linked to keywords that activate the entry in the lorebook. That way, you can punch far above your weigh by having context activate and deactivate depending on the circumstances.
I pretty much gave up trying to make Yi based models actually use more then 4k context. And at that point I rather just use Lzlv 70b which is much smarter with better prose and knowledge.
The repetition issue pretty much makes the models unusable past the context where it breaks.
Lzlv 70b still the best model by a mile for story writing.
It is, the 3bpw quant is noticably better then lzlv 70b. Goliath is an unruly horse. It will allow itself to be controlled until it doesn’t a s just goes and does its own thing. But it’s prose is so much better then lzlv that I’m never going back. It’s the first model that doesn’t speak like ChatGPT.