Definition
Elo rating is an Elo rating is a score built from head-to-head matchups, and AI leaderboards use it to rank models by how often people prefer one model's answer over another's in blind comparisons.
An Elo rating is a way to turn many one-on-one contests into a single ranking number. It was invented for chess, and AI leaderboards borrowed it to rank models by how often people pick one model's answer over another's.
In a typical AI arena, you type a prompt and receive answers from two anonymous models. You pick the better one. Each vote is a "match." Every model starts with a baseline score. When a model wins, its score goes up, and the loser's goes down.
The size of the change depends on how surprising the result was. Beating a higher-rated model earns a big gain. Beating a much weaker one earns almost nothing. After thousands of votes, the scores settle into a ranking that reflects how often each model wins.
The scores also have a useful meaning: the gap between two ratings predicts how often the higher-rated one wins. A small gap means near coin-flip results. A large gap means one model wins most of the time. Many leaderboards also show a confidence range, since scores are estimates from limited votes. Some use closely related statistical methods rather than pure Elo, but the idea is the same.
Elo-style leaderboards capture something an AI benchmark with fixed questions often cannot: which answer real people actually like. They are common for chat, writing, image generation and editing, where there is no single correct answer to score.
When a lab says its model "gained 70 Elo points," it means the model now wins more head-to-head votes than before. That is a relative measure, not a percentage of anything.
The best habit is to use Elo as a starting list, then compare finalists on your own prompts. A model that leads the arena can still lose on your specific task.
Elo ratings are one way to evaluate models. Fixed tests such as SWE-bench measure specific skills, while Elo measures preference. Model calibration is a different question again, about whether a model's confidence matches its accuracy.
It means people choose that model's answers more often in blind head-to-head comparisons. The bigger the gap, the more often it wins.
No. A benchmark score measures performance on fixed tasks. Elo measures relative preference from many head-to-head votes.
Google released Nano Banana 2.1 on October 6, an update to its Flash-tier image model that, by Google's own evaluations, holds multi-character scenes together far better. If you make images with people in them or infographics with real text, test it this week.