What is Embedding?

An embedding turns something messy, like a sentence or a photo, into a fixed-length list of numbers. Items with similar meaning end up with similar numbers. That lets software search by meaning instead of by exact words.

How it works

A specialized model, called an embedding model, reads the input and outputs a long list of numbers, often hundreds or thousands of them. Think of it as an address on a huge map of meaning. "Cheap flights to Rome" and "inexpensive airfare to Italy" share few words, but their addresses land close together. "Chocolate cake recipe" lands far away.

To compare two embeddings, software measures how close they are, most often with a calculation called cosine similarity. Close means related.

Why it matters to you

Embeddings are the quiet machinery behind a lot of AI features you already use:

• Semantic search. Find the document you mean, not just the one containing your keyword.
• Retrieval for chatbots. In retrieval-augmented generation, your question is embedded, matched against embedded documents, and the best matches are handed to the model so its answer is grounded in them.
• Recommendations and clustering. Group similar support tickets, products or articles automatically.
• Duplicate detection. Spot near-identical text even when it was reworded.

The numbers usually live in a vector database or search index built to find nearest neighbors quickly.

An example

Say you store your company handbook as embeddings, one per paragraph. An employee asks, "Can I work from my cousin's place for a month?" The system embeds that question, finds the paragraphs about remote work and travel, and passes them to the model. No paragraph contains the word "cousin," and none needs to.

Limits to know

• Embeddings from different models are not interchangeable. If you switch embedding models, you must re-embed everything.
• They capture meaning roughly, not exactly. Numbers, negations and rare names can be matched poorly, so many systems combine them with ordinary keyword search.
• An embedding is not a token. Tokens are the chunks of text a model reads. An embedding is a numeric summary of meaning that a model produces.
• They are compressed, but they can still reveal information about the original text, so treat them as sensitive data if the source is.

Related terms

Embeddings are different from the model weights that make up a trained model: weights are what the model has learned, while an embedding is what the model outputs for one specific input. Large language models also use embeddings internally, turning each token into a vector as the first step of processing it.