We keep saying "vector" and "embedding." Here's the whole idea, made concrete — and once meaning is math, you can measure how related two things are.
The idea
An embedding is meaning as a list of numbers.
Take a word, a sentence, a whole document — the model turns it into a long list of numbers called a vector. That vector is a coordinate: a single point in a huge "meaning space."
Text in, numbers out.
Nothing mystical yet — it's just a fixed-length list of numbers that stands in for the meaning of the text.
The magic
Similar meanings get similar numbers.
The whole payoff is in how those numbers land. Words that mean related things end up with vectors that sit close together; unrelated words end up far apart.
🐶 "puppy" ↔ "dog"
Almost the same meaning → their vectors land right next to each other.
🚗 "car" ↔ "dog"
Totally different meaning → its vector ends up far away, in another direction.
Closeness in numbers is closeness in meaning. That's the one idea to hold onto.
Visualized
A map where distance = meaning.
Squash those long vectors down to 2D and related words cluster together. Pick dog — its nearest neighbors light up green; car sits far away.
Where it comes from
Nobody drew those clusters.
The model was never handed the categories "animals" or "vehicles." It learned to organize concepts by meaning entirely on its own — just from reading enormous amounts of text and noticing which words show up in similar contexts.
Meaning, self-organized.
Words used the same way drift together; words used differently drift apart. The map is a side effect of that.
The measure
Cosine similarity: the angle between vectors.
How do you turn "close in space" into a number? Measure the angle between two vectors. Small angle → same direction → high score.
Reading the number
Same direction → ~1. Right angle → ~0.
Cosine similarity lands on a simple scale you can read at a glance:
~1.0
Point the same way. Nearly identical meaning.
~0.5
A moderate angle. Loosely related.
~0.0
Right angle. Unrelated meaning.
It's the angle that matters, not the length of the vector.
Concrete
Dog vs. three words.
Same reference vector — dog — compared against three others. Watch the score fall as the meaning drifts:
dog · puppy
0.94
Same direction — very related.
dog · cat
0.72
Still animals — a moderate angle.
dog · car
0.14
Different world — a wide angle.
Why it wins
Match meaning, not spelling.
🔤 Keyword search
Literal · brittle
Query "how do I get my money back?" misses a doc titled "Refund policy" — they share no words.
🧭 Semantic search
Meaning-aware
Same query lands right on "Refund policy" — their vectors are close because their meaning is close.
Find things related to a question even when they share no words — the engine under RAG retrieval.
One detail
Different models, different dimensions.
An embedding's length — its number of dimensions — depends on the model you use. Bigger isn't automatically better; it's a tradeoff of nuance vs. cost and speed.
text-embedding-3-large
3,072
Lots of nuance, higher cost.
Cohere embed
1,024
A common middle ground.
all-MiniLM
384
Tiny — runs free on your laptop.
The one rule to remember
Index and query with the same model.
⚠️ THE GOLDEN RULE
Always embed with the same model for indexing and for querying. Index with one model and search with another and the numbers are incompatible — your results turn to garbage.
It's like looking up a Japanese word in a French–English dictionary. Different space, meaningless answer.
Chapter 9 in one breath
Similar meaning, similar numbers.
An embedding turns text into a vector so related meanings sit close together. Cosine similarity reads that closeness as the angle between vectors — and that's the engine under semantic search and RAG.
Just remember: same model to index and to query.
Next up
Chapter 10 · MCP.
We've turned meaning into numbers and measured how related two things are. Next we look at how agents plug into the outside world through a shared standard — the Model Context Protocol.