Guide

How much VRAM do you need to run LLMs locally?

Updated 21 November 2026Built from the GPU Atlas database

A language model has to fit in memory to run at full speed. The rule of thumb: memory ≈ parameters × bytes per parameter, plus about 20% for the context cache and runtime. Longer contexts need more.

Estimated memory by model size

Model size4-bit (Q4)8-bit16-bit (FP16/BF16)
3B~3 GB~4 GB~8 GB
7–8B~6 GB~11 GB~20 GB
13–14B~10 GB~18 GB~34 GB
27–32B~22 GB~41 GB~77 GB
70B~48 GB~90 GB~168 GB
~120B~81 GB~153 GB~288 GB
405B~273 GB~516 GB~972 GB

Estimates: 4-bit uses about 0.56 bytes per parameter (typical Q4 formats), 8-bit about 1.06, 16-bit 2. Mixture-of-experts models must still fit all their parameters in memory, even though only some are active per token.

Current GPUs and Macs that fit

ModelNeedsSmallest current options
3B at 4-bit~3 GBRTX 5050 8 GB, RX 9050 8 GB, RTX 5060 8 GB, RX 9060 8 GB
7–8B at 4-bit~6 GBRTX 5050 8 GB, RX 9050 8 GB, RTX 5060 8 GB, RX 9060 8 GB
13–14B at 4-bit~10 GBArc B570 10 GB, Arc B580 12 GB, RTX 5070 12 GB, RX 9070 GRE 12 GB
27–32B at 4-bit~22 GBRTX PRO 4000 Blackwell 24 GB, RTX PRO 4000 Blackwell SFF Edition 24 GB, Arc Pro B60 24 GB, Arc Pro B70 32 GB
70B at 4-bit~48 GBRTX PRO 5000 Blackwell 48 GB, M5 Pro 64 GB, RTX PRO 6000 Blackwell Workstation Edition 96 GB, DGX Spark (128 GB) 128 GB
~120B at 4-bit~81 GBRTX PRO 6000 Blackwell Workstation Edition 96 GB, DGX Spark (128 GB) 128 GB, RTX Spark Superchip (20-core) 128 GB, Ryzen AI Max+ 395 (Radeon 8060S) 128 GB

Apple Macs share one pool of unified memory between CPU and GPU, so a large share of it is available to models; leave room for the system. Memory bandwidth decides how fast text is generated: compare it on each spec sheet.

Want a shortlist for your budget? Choose "Local AI" in the GPU finder.

Launch prices are official prices at release. Specifications link to each product's spec sheet and its sources.