This big difference with tensor cores is that they use a 16-bit float multiply combined with a 32-bit float accumulate. This makes them much more efficient in terms of transistors required… but not a swap in replacement for CUDA.

Libraries like Pytorch can do matrix multiply (MM) on both CUDA cores and Tensor cores (and CPU, too, if you like). Typically Tensor cores are ~1.5-2x faster (in theory they’re much faster, in practice we’re often memory bandwidth limited so it doesn’t matter). The current default in Pytorch is to perform MM on CUDA, and convolutions on Tensor cores. The reason being that MM sometimes requires extra precision, and in vision models, most of the work is in the convolutions anyway.

The star in Q* traditionally refers to a policy which is optimal.