Guide
How much VRAM do you need to run LLMs locally?
A language model has to fit in memory to run at full speed. The rule of thumb: memory ≈ parameters × bytes per parameter, plus about 20% for the context cache and runtime. Longer contexts need more.
Estimated memory by model size
| Model size | 4-bit (Q4) | 8-bit | 16-bit (FP16/BF16) |
|---|---|---|---|
| 3B | ~3 GB | ~4 GB | ~8 GB |
| 7–8B | ~6 GB | ~11 GB | ~20 GB |
| 13–14B | ~10 GB | ~18 GB | ~34 GB |
| 27–32B | ~22 GB | ~41 GB | ~77 GB |
| 70B | ~48 GB | ~90 GB | ~168 GB |
| ~120B | ~81 GB | ~153 GB | ~288 GB |
| 405B | ~273 GB | ~516 GB | ~972 GB |
Estimates: 4-bit uses about 0.56 bytes per parameter (typical Q4 formats), 8-bit about 1.06, 16-bit 2. Mixture-of-experts models must still fit all their parameters in memory, even though only some are active per token.
Current GPUs and Macs that fit
| Model | Needs | Smallest current options |
|---|---|---|
| 3B at 4-bit | ~3 GB | RTX 5050 8 GB, RX 9050 8 GB, RTX 5060 8 GB, RX 9060 8 GB |
| 7–8B at 4-bit | ~6 GB | RTX 5050 8 GB, RX 9050 8 GB, RTX 5060 8 GB, RX 9060 8 GB |
| 13–14B at 4-bit | ~10 GB | Arc B570 10 GB, Arc B580 12 GB, RTX 5070 12 GB, RX 9070 GRE 12 GB |
| 27–32B at 4-bit | ~22 GB | RTX PRO 4000 Blackwell 24 GB, RTX PRO 4000 Blackwell SFF Edition 24 GB, Arc Pro B60 24 GB, Arc Pro B70 32 GB |
| 70B at 4-bit | ~48 GB | RTX PRO 5000 Blackwell 48 GB, M5 Pro 64 GB, RTX PRO 6000 Blackwell Workstation Edition 96 GB, DGX Spark (128 GB) 128 GB |
| ~120B at 4-bit | ~81 GB | RTX PRO 6000 Blackwell Workstation Edition 96 GB, DGX Spark (128 GB) 128 GB, RTX Spark Superchip (20-core) 128 GB, Ryzen AI Max+ 395 (Radeon 8060S) 128 GB |
Apple Macs share one pool of unified memory between CPU and GPU, so a large share of it is available to models; leave room for the system. Memory bandwidth decides how fast text is generated: compare it on each spec sheet.
Want a shortlist for your budget? Choose "Local AI" in the GPU finder.
Launch prices are official prices at release. Specifications link to each product's spec sheet and its sources.