TinyGPU¤

TinyGPU app lets you use AMD and NVIDIA GPUs on macOS over USB4/Thunderbolt with tinygrad.

Requirements¤

• macOS (12.1+)
• USB4/Thunderbolt port
• A supported GPU (AMD RDNA3+ or NVIDIA Ampere+)

Setup¤

1. Connect your GPU¤

Plug the supported GPU into your Mac over USB4/Thunderbolt.

2. Initiate the driver install¤

Note: If tinygrad is cloned but not installed, run commands with PYTHONPATH=.

curl-fsSLhttps://raw.githubusercontent.com/tinygrad/tinygrad/master/extra/setup_tinygpu_osx.sh|sh

This downloads TinyGPU.app and triggers a system prompt to install the driver extension.

3. Enable the driver¤

You should see a system prompt: "TinyGPU" would like to use a new driver extension. Click Open System Settings and toggle TinyGPU on.

If you missed the prompt, go to System Settings > General > Login Items & Extensions > Driver Extensions and toggle TinyGPU on.

4. Compiler Setup¤

AMD¤

curl-fsSLhttps://raw.githubusercontent.com/tinygrad/tinygrad/master/extra/setup_hipcomgr_osx.sh|sh

NV¤

Install Docker Desktop if you don't have it.

curl-fsSLhttps://raw.githubusercontent.com/tinygrad/tinygrad/master/extra/setup_nvcc_osx.sh|sh

Make sure ~/.local/bin is on your PATH:

exportPATH="$HOME/.local/bin:$PATH"

5. Use it!¤

DEV={AMD|NV}python3tinygrad/apps/llm.py