What is VRAM?

VRAM, short for video RAM, is the fast memory built into a graphics card (GPU). When you run an AI model on your own machine, VRAM is usually the first limit you hit.

Why it decides what you can run

A GPU can only work quickly on data that sits in its own memory. To run a model, the card has to hold the model's weights there, plus working space for the conversation. If they do not fit, the model will not load, or it spills into slower system memory and crawls.

A rough way to estimate

The weights take about the number of parameters times the bytes used for each one:

• 16-bit (FP16 or BF16): 2 bytes per parameter, so about 2 GB per billion parameters.
• 8-bit: 1 byte, about 1 GB per billion.
• 4-bit: about 0.5 GB per billion, a bit more in real files.

So a 7 billion parameter model needs roughly 14 GB at 16-bit and around 4 GB at 4-bit. Shrinking the numbers this way is called quantization, and it is the main reason big models can run on small cards.

The part people forget

Weights are not the whole bill. While a model generates text it stores a running record of the conversation called the KV cache, and it grows with the length of your context window and the number of simultaneous users. Guides to LLM memory sizing report that a long context can add tens of gigabytes on top of the weights for a large model. Software overhead takes a little more. Treat any estimate as a planning figure and check real usage in your runtime.

Consumer cards versus data center chips

Gaming cards commonly carry 8 to 24 GB. Data center chips use stacked high-bandwidth memory with far more capacity and speed. Some computers, like recent Macs, share one pool of "unified memory" between the processor and graphics, which is why they can load very large models.

Why it matters to you

Before you download an open-weight model, check its size and the quantized options against your VRAM. Smaller image models often fit in 8 GB, while the largest chat models need far more than one consumer card.