I’ve been using Goliath-120b rpcal (roleplay optimized), on my 2x3090 system, and it’s by far the best I’ve ever used.
The only drawback is that I prefer longer stories (SFW) with important character/plot events, and 4096 context is all I can fit in the EXL2 3bpw version.
I wish there was a 2.xx version that could fit 8192 context or even 10240. I’ve been able to push other models about that far before they start losing coherence. (It might be suboptimal alpha values in exllamav2?)
Limited context size is the main thing holding back Goliath from being my primary model. It’s amazing in every other way.
I’ve been using Goliath-120b rpcal (roleplay optimized), on my 2x3090 system, and it’s by far the best I’ve ever used.
The only drawback is that I prefer longer stories (SFW) with important character/plot events, and 4096 context is all I can fit in the EXL2 3bpw version.
I wish there was a 2.xx version that could fit 8192 context or even 10240. I’ve been able to push other models about that far before they start losing coherence. (It might be suboptimal alpha values in exllamav2?)
Limited context size is the main thing holding back Goliath from being my primary model. It’s amazing in every other way.