OverallBit9@alien.topOPBtoLocalLLaMA@poweruser.forum•Is there any way to speed up the MythoMax-L2-13B on a 6GB GPU?English
1·
11 months agoTesting Q5 seems like the best at least for this GPU I use, but only on mythomax I’m not sure if other models would be the same.
In my tests Q4 is giving me the same amount of tokens as Q5 so I decided to use Q5, first time tesint text gen locally with models, thank you very much for explaining I am getting used to it now and understanding what the settings do.