You must log in or register to comment.
Amazing! And they published the code.
Also, the omniquant paper they linked is amazing! They hooked some super quantization into MLC, apparently?
Alright, these numbers are kinda wild. u/The-Bloke got any spare compute to quant an LQ Nous-Cappy70B or Goliath-120B?