I’d listen to this recent talk by Bill Dally (chief scientist @ NVIDIA) who talks a bit about the underlying math operation primitives from a computer architecture-level:

https://www.youtube.com/watch?v=kLiwvnr4L80

He cuts through some of the marketing language like CUDA and tensor cores to focus on the complex instructions from 16 mins on or so.

Brockman has real dev (not research) experience and has a lot of street cred in that direction. Don’t forget that ChatGPT is as much an app as it is a LLM.