China needs to start pumping out their own dedicated AI accelerator cards. I’m sick of Nvidia’s VRAM business model. Having to run multiple giant GPUs in parallel instead of simply soldering more RAM chips onto the board is extremely wasteful.

So it turns out you just need to train on GPT output for better benchmarks lol. Not to say there’s a chance GPT models are contaminated with benchmark test data too. “Distillation” went a little too far. Easy VC money though, I would do the same.