Apple didn't have to go this hard...

RDMA over Thunderbolt support landed in macOS 26.2.

Is it good for HPC? For Exo and LLMs? Find out in this video.

Resources (some links are affiliate links):

- Mac Studio: https://www.apple.com/mac-studio/
- Exo 1.0: https://exolabs.net
- Apple MLX PR for Thunderbolt RDMA: https://github.com/ml-explore/mlx/pull/2808
- M5 Neural Accelerator speedup: https://machinelearning.apple.com/research/exploring-llms-mlx-m5
- DeskPi Rackmate TL1: https://amzn.to/4p5ek94
- Tupavco mini-rack PDU: https://amzn.to/4j0DpAq

Disclaimer: The Mac Studios used in this cluster are on loan from Apple for testing. They are not sponsoring this video, nor had any say in the contents. DeskPi sent the Rackmate TL1 used in the video for my testing, but they also had no say in the video contents. I always add the "sponsored" flag on my videos if I did not purchase the equipment being shown.

Support me on Patreon: https://www.patreon.com/geerlingguy
Sponsor me on GitHub: https://github.com/sponsors/geerlingguy
Merch: https://www.redshirtjeff.com
2nd Channel: https://www.youtube.com/@GeerlingEngineering
3rd Channel: https://www.youtube.com/@Level2Jeff

Contents:

00:00 - A rack-full of Macs
02:01 - Putting together a cluster
03:16 - The downsides of Thunderbolt
05:09 - One is enough
07:45 - Managing macOS in a cluster (vs Linux)
08:37 - HPL and llama.cpp cluster performance
09:30 - Enabling RDMA
09:51 - Massive LLMs
11:34 - RDMA over TB5 is great... when it works
12:10 - EXO community and OSS
12:51 - Apple could go harder
13:30 - Jack of all trades, not just AI