Why are GPUs fundamental to AI? And AI training?

Deep learning algorithms require linear algebra (matrix multiplications) and GPUs are very functional for linear algebra and matrices (as per their original design to compute graphics transformations effectively).

The matrix multiplications are used to update weights, and multiple cycles of updating weights, often referred to as epochs are required to adequately train these neural network models.

GPUs are basically the "free weights" of the AI training world.  More GPUS means more "reps" of AI training moves.

Geoffrey Hinton was one of the early pioneers who harnessed the power of GPUs to achieve AI training speeds hitherto unknown.