Definition
GGUF is GGUF is a single-file format for storing an AI model's weights and settings, used by llama.cpp and tools built on it to run quantized models locally on ordinary computers.
GGUF is a file format for packaging an AI model so it can run on your own computer. It bundles the model's numbers, its settings and the information needed to read text into one file, which you download and open with a compatible app.
A language model is more than its model weights. To run it you also need details such as the model's architecture, its tokenizer and configuration values. GGUF stores these together as tensors plus metadata in one binary file. That is the main appeal: one file, no scattered folders of loose parts.
The format comes from the llama.cpp project, which runs models on regular CPUs and consumer graphics cards. GGUF is that project's native format, and tools such as Ollama and LM Studio load it too. It replaced older formats from the same family, GGML among them.
Most GGUF files you will meet are quantized. Quantization stores a model's numbers at lower precision, such as 8-bit or 4-bit instead of 16-bit, which shrinks the file and the memory it needs. A model that would not fit on your hardware at full precision may fit once quantized. You trade a small amount of quality for a big drop in size.
File names usually signal the level, with labels such as Q4 or Q8. Lower numbers mean smaller files and more quality loss. Pick the biggest one your VRAM or system memory can hold, leaving room for the context window, because longer conversations use extra memory through the KV cache.
GGUF is the practical doorway to on-device AI. When a lab releases an open-weight model, the community often publishes GGUF builds soon after, so you can run it privately on a laptop without sending data to a cloud service. Machines with unified memory are especially good at this.
You want to try an open model on your laptop. You download a 4-bit GGUF file of about several gigabytes, point a local app at it, and start chatting offline. The same model at full precision might need a data center card.
It is a container, not a model and not a license. The rules for using the model still come from the model's own license, and one family of models may ship in many GGUF versions of different quality. Always check where a file came from, since a file from an unknown uploader is something you are choosing to trust.
GGUF connects to inference, the act of running a model, and to fine-tuning and LoRA, which change a model before it is packaged.
GGML Universal File, according to the llama.cpp project's naming. Some sites give other expansions, but this is the one that matches the project.
Often yes. llama.cpp runs on CPUs, though a GPU makes responses faster.
Mistral released the weights of Large 4 on Oct. 29, a trillion-parameter model it calls the best open-weight model for coding, priced at $2.10 per million output tokens on its API. It still trails the closed frontier. You can own it, and that is the point.