CUDA cores (or shader cores in general) have long been used to compute graphics. A very often used operation in computer graphics are matrix multiplications, just like in deep learning. Back in the days (AlexNet) NNs were computed using shader cores, but now have completely moved to be computed on Tensor cores. My question are:
-
Why have these workloads been seperated? (Yes obviously the tensor cores are more specialized and leave out a bunch of unnecessary operations, but how and why not integrate it into the CUDA cores to boost MM operations for computer graphics?)
-
Why isn’t the workload offloaded to the other cores when the mathematical operations are the same
-
What makes tensor cores so much more efficient and faster?
Shader cores and tensor core only make sense when they are in a gaming card. Shader core carries the work of shading, while tensor core carries the upscaling, especially in case of Ray Tracing. In terms of operands, shading units works with vector as the base data structure, while in tensor core it is matrix (4x4 matrix) In ML card like A100, the amount of shading unit:tensor units is much lower than the ratio in gaming card.