There was a paper which I couldn’t find at the moment, which says in the early stages of the gpt, when they added code into its knowledge base, it got better at reasoning. I think that math might help in some other ways, but code can be used to solve math problems and do more than math in anycase.
It’s strange that coding is so much better than math given that they are theoretically equivalent, and the the difference is only in the distribution of things we find interesting. I guess a lot more code is repetitive/mundane compared to harder math?
Code can do things like launch severs, build out system architectures, read in a file, write pixels to the screen, call system calls, call a calculator app or talk to a quantum computer, move a robot, etc… significantly more than math.
Coding is not Important to make better LLM, it’s all about math
There was a paper which I couldn’t find at the moment, which says in the early stages of the gpt, when they added code into its knowledge base, it got better at reasoning. I think that math might help in some other ways, but code can be used to solve math problems and do more than math in anycase.
I think OP is responding to (without commenting on correctness…)
Yes, it’s all about math problems, code is just tool to express them
It’s strange that coding is so much better than math given that they are theoretically equivalent, and the the difference is only in the distribution of things we find interesting. I guess a lot more code is repetitive/mundane compared to harder math?
Code can do things like launch severs, build out system architectures, read in a file, write pixels to the screen, call system calls, call a calculator app or talk to a quantum computer, move a robot, etc… significantly more than math.