Definition
GSM8K is GSM8K is a benchmark of about 8,500 grade-school math word problems, written by people, that tests whether an AI model can work through a multi-step problem and reach the right answer.
GSM8K stands for "Grade School Math 8K." It is one of the most widely quoted math tests for AI models, and when you see a score on it, the number is the percentage of problems the model answered correctly.
OpenAI researchers released GSM8K in 2021 alongside the paper "Training Verifiers to Solve Math Word Problems." The dataset holds roughly 8,500 problems written by human workers. The authors split it into about 7,500 training problems and 1,000 test problems.
A typical question is a short story with numbers: someone buys some items, spends part of their money, and you must work out what is left. The math is simple. The challenge for a model is to chain several steps together without slipping.
Each problem has a single numeric answer. The model writes its working, and its final number is compared with the key. Scoring is the share it gets right.
In the original paper the authors found that even the largest models of that time struggled, and they proposed training a separate "verifier" to pick the best of several candidate solutions. That idea helped shape later work on reasoning.
GSM8K helped make chain of thought famous, because asking a model to write out steps raised scores sharply. It is still a quick check that a model, especially a small one, can follow multi-step arithmetic logic.
Because of this, GSM8K is most useful today for small models and for before-and-after checks, such as seeing whether fine-tuning hurt a model's basic math. For harder problems, labs turn to MATH-500 and other tests.
As with any AI benchmark, a high score tells you the model is good at this kind of problem. It does not tell you the model will do well on your own tasks, which is what evals are for.
Grade School Math 8K, a set of about 8,500 grade-school math word problems released by OpenAI in 2021.
MIT CSAIL researchers say a frontier LLM, used as is, read and rewrote neural network weights without gradient descent, lifting a small Qwen model's MATH-500 score from 7.9% to 48.1%. It is an early blog post, not a peer-reviewed paper, and still the strangest result of the week.