09 Embeddings
We keep saying "vector" and "embedding." Here's the whole idea, made concrete: an embedding represents meaning as a long list of numbers, and the magic is that similar meanings get similar numbers. "Puppy" and "dog" land right next to each other; "car" ends up far away in a different direction. Once meaning is math, you can measure how related two things are.
The idea, visualized
Imagine squashing those long vectors down to a 2D map. Words with related meanings cluster together; unrelated words drift apart. The model was never told these categories — it learned to organize concepts by meaning, just from reading. Click any word to light up its nearest neighbors.
Because all meaning lives in one shared space, you can find things that are conceptually related to a question even when they share no words. That's semantic search — and it's fundamentally more powerful than matching keywords.
The measure
So how do you turn "close in space" into a number? For text it's almost always cosine similarity — it measures the angle between two vectors. Point in nearly the same direction (small angle) → score near 1 (very similar). Point at right angles → near 0 (unrelated). Pick a comparison and watch the angle.
Why it beats keyword search
Query "how do I get my money back?" → misses a doc titled "Refund policy," because they share no words. Literal, brittle.
Same query lands right on "Refund policy" — their vectors are close because their meaning is close. This is what powers RAG retrieval.
An embedding's length (its dimensionality) varies by model: OpenAI's text-embedding-3-large uses 3,072 numbers, Cohere's around 1,024, and small open-source models like all-MiniLM just 384 — small enough to run free on your laptop.
Always use the same embedding model for indexing and for querying. Index with OpenAI and search with Cohere and the numbers are incompatible — your results turn to garbage. It's like looking up a Japanese word in a French–English dictionary.
Chapter 9 in one breath
An embedding turns text into a vector so that related meanings sit close together. Cosine similarity measures how close by the angle between vectors — and that's the engine under semantic search and RAG retrieval. Just remember: same model to index and to query.
A matching slide deck with speaker notes — press S for notes, F for fullscreen.