Optimizing a Metal Matmul kernel for 5TFLOPs+ performance on my M1 Max

References

https://andrewkchan.dev/posts/things-april-2024.html

https://www.nuss-and-bolts.com/p/optimizing-a-webgpu-matmul-kernel

https://siboehm.com/articles/22/CUDA-MMM

https://github.com/gpu-mode/lectures/tree/main/lecture_031

https://www.youtube.com/watch?v=h1mOkhFOSPU

https://developer.apple.com/documentation/metal/performing-calculations-on-a-gpu

https://explosion.ai/blog/metal-performance-shaders

https://github.com/pytorch/pytorch/tree/e48ee2cf50d86d87ef7c7d0839267dbed4903ebf/aten/src/ATen/native/mps/operations