They use the GPU but an Apple Silicon GPU has the same high speed access to all the RAM on the machine as the CPU does, rather than having its own walled-off maybe 16 GB VRAM in mainstream gaming GPUs or 24 GB in RTX 4090 or RTX 5090 (MSRP $1999 but in practice $3000-$4000 at the moment). Nvidia A100 (80GB VRAM) apparently cost $15,000 or so.
Not only does Apple's unified memory give the GPU more RAM to use, but it also eliminates copying things between CPU RAM and GPU RAM.
A Mac Mini with 48 GB RAM costs $1799. A Mac Studio with 96 GB RAM is $3999 — until March you could get a Mac Studio with 512 GB RAM for $3999, all of which could be used for your AI model.
i believe you meant something like US$9999 for the 512GB. otherwise, i'm going to feel like QUITE the fool for choosing the 96GB variant at the same price
... confirms that, but I'm not clear on which config the $4000 upgrade was starting from, but I think it's from the M3 Ultra's standard 96 GB.
Both articles say that previously 256 GB RAM was a $1600 upgrade from 96 GB but "now" is a $2000 upgrade.
Now "now" even 256 GB isn't available and 96 GB is the only option with the M3 Ultra.
M3 Ultra is currently $3999 with 96 GB and 28 core CPU 60 core GPU, or +$1500 with 32 core CPU 80 core GPU. If the base price was the same prior to March then yeah $3999+$1500+$4000 = $9499 would have been the price for a maxed out CPU/GPU/RAM config with a 1 TB disk.
I wasn't looking at Mac Studio prices back then so unfortunately articles such as these are my only clue. I'm still more than happy with my original M1 Mini with 16 GB RAM as I don't work with local LLMs — only on my little NUC-like RISC-V "K3" machine with 32 GB RAM and regular RISC-V cores with 1024 bit vectors as the "NPU", doing around 7 tok/s on 32B models while using 14W of power.
Not only does Apple's unified memory give the GPU more RAM to use, but it also eliminates copying things between CPU RAM and GPU RAM.
A Mac Mini with 48 GB RAM costs $1799. A Mac Studio with 96 GB RAM is $3999 — until March you could get a Mac Studio with 512 GB RAM for $3999, all of which could be used for your AI model.
https://www.tomshardware.com/tech-industry/apple-pulls-512-m...
Some are coming up used at silly prices.
https://www.trademe.co.nz/a/marketplace/computers/desktops/a...
NB NZ$44,999 is "only" US$25,772.