Definition
Parameter is a parameter is one of the learned numbers inside an AI model, adjusted during training, and the count of them, often billions, is a rough measure of the model's size.
A parameter is a single number a model learns during training. A neural network is a huge web of these numbers. Together they decide how the model turns an input, such as your prompt, into an output, such as its reply. The collection of all of them is also called the model's weights.
Think of a mixing board with billions of tiny dials. Training turns each dial a little at a time until the board produces good results. The final dial positions are the model.
Before training, the numbers are essentially random. The model reads huge amounts of data, makes predictions, and is nudged to be less wrong. Each nudge adjusts many parameters slightly. After enough rounds, the numbers encode patterns of language, code and images.
When you chat with a model, none of the parameters change. The model uses them as they are. That is inference, as opposed to training.
Labs often describe a model by its parameter count, such as 8 billion or hundreds of billions. More parameters generally give a model more capacity to store patterns, but it is only a rough guide. Training data, training method and design matter a great deal, so a smaller, well-trained model can beat a larger one.
Size also sets cost. More parameters mean more memory to hold the model and more computation per answer.
Some models do not use every parameter for every token. In a mixture of experts design, only a few specialist sub-networks run at a time. Labs then quote two numbers: total parameters, which is the full size, and active parameters, which is how many do the work on each token. Active count is closer to the per-answer compute, while total count is closer to the memory needed.
A model with 8 billion parameters stored at two bytes each needs roughly 16 gigabytes just to hold its weights, before any working memory. That is why small models fit on a laptop and very large ones need server hardware.
Parameters are what post-training and fine-tuning adjust after the first round of training. They are not the same as the context window, which limits how much text the model reads at once.
It is the parameter count in billions: 7B means about seven billion learned numbers in the model.
No. More parameters add capacity and cost, but training data and method matter too, so a smaller well-trained model can beat a larger one.
Mistral opened a public preview of Mistral Large 4 on Tuesday, a mixture-of-experts model with about 1 trillion parameters and 49 billion active, and promised weights by the end of October. The API is a preview. The weights are the launch.
Reflection AI introduced Beam on Monday, a 501-billion-parameter mixture-of-experts model with 23 billion active, and promised Apache 2.0 weights later in October. Until the weights ship, it is a waitlist and a claim about efficiency.
Microsoft made its agent containment layer generally available on Windows 11 and opened preorders for Nvidia RTX Spark PCs, from $2,599.99 for the Surface Laptop Ultra. The laptop gets the headlines, but the fence around your agents is the bigger change.
Meta released Muse Wonder and Muse Wonder Full on Oct. 12, calling them its most capable models, available now through the Meta Model API and Muse Code. The post gives no scores, no prices and no safety report yet, so the frontier claim is Meta's until you test it.
Nvidia is in talks to acquire open-weight model startup Reflection AI or invest more in it, the Financial Times reported, days after Reflection announced its 501B-parameter Beam model. Nothing changes for you until Beam's weights ship under Apache 2.0.