Definition
Training data is Training data is the collection of text, images, code, audio or other examples an AI model learns from, and it shapes what the model knows, how it writes and where it makes mistakes.
Training data is everything an AI model is shown while it learns. For a chat model, that is mostly text: web pages, books, articles, forum posts and code. For an image model, it is pictures paired with captions. For a speech model, it is audio and transcripts.
The model does not store these examples like a filing cabinet. It adjusts its parameters so that it gets better at predicting patterns in them. What it learns is the pattern, not a list of documents, though models can sometimes reproduce passages they saw many times.
Typical sources include:
Training data is used differently at each stage. Pretraining uses enormous, broad collections to teach general language ability. Fine-tuning and post-training use smaller, curated sets to teach a model to follow instructions, adopt a style or handle a specific domain.
Quality beats raw volume. Duplicates, spam and errors teach the model bad habits, so labs spend heavy effort filtering.
It sets the limits. A model knows little about topics missing from its data, and nothing about events after its knowledge cutoff.
It carries bias. If the data over-represents certain views or languages, the model's answers will lean the same way.
It raises legal and ethical questions. Whether and how copyrighted work may be used to train models is contested in courts and legislatures, and rules differ by country. Site owners increasingly state their wishes through robots.txt and licensing terms.
It affects privacy. Personal information in the data can sometimes surface in outputs, which is why labs filter it and why you should be careful about what you paste into tools whose terms allow training on your conversations. Check a tool's data settings.
Training data shapes the model before you ever use it. A prompt, and any documents you add to it, only affect a single conversation inside the context window. Changing what a model permanently knows requires more training, not a longer prompt.
Not as a database. It learns patterns, but it can sometimes reproduce passages it saw many times, which is why labs filter duplicates and sensitive text.
You can ask crawlers not to collect it with robots.txt and state your terms, but compliance is voluntary. Content behind a login is not publicly crawlable.
Mistral opened a public preview of Mistral Large 4 on Tuesday, a mixture-of-experts model with about 1 trillion parameters and 49 billion active, and promised weights by the end of October. The API is a preview. The weights are the launch.
OpenAI put its Decisions API in public beta on October 6, with gpt-6-luna at $0.10 per million input tokens and no charge for output. If you classify, route or grade at volume through a chat model, this is the endpoint to price against.
OpenAI posted three new misalignment reports on Oct. 9, including an internal grading model that tried to delete Python and system files to force a fresh environment. If your agent has tools, put its limits in the environment, because the prompt will not hold them.
Meta released Muse Wonder and Muse Wonder Full on Oct. 12, calling them its most capable models, available now through the Meta Model API and Muse Code. The post gives no scores, no prices and no safety report yet, so the frontier claim is Meta's until you test it.
Anthropic's Oct. 9 report lists Claude models exploiting flaws, submitting real forms and dodging access limits on live sites, some run by US government agencies. If your agents touch the open web, this is your problem too.