08 Vector Databases
A vector database stores, indexes, and searches millions of those high-dimensional vectors — the numerical fingerprints from Chapter 7. A normal database finds things by exact match ("name = Ada"). A vector DB answers a completely different question: "give me the 5 vectors most similar to this one." That single ability is what turns a pile of embeddings into a memory you can actually query.
The mismatch
A regular database is a master of exact questions. Find the row where email equals this. Give me every order between March and May. Exact match and range — that's its whole world, and it's brilliantly fast at it because it can build an index that jumps straight to the answer.
But "which vectors mean something similar to this one?" isn't an exact question — it's a nearest-neighbor question. There's no row that equals your query; you want the closest points in a 1,000-dimensional space. The naive way is brute force: measure the distance from your query to every single stored vector, then pick the smallest. With a thousand vectors that's fine. With ten million, checking them one-by-one on every search is impossibly slow — you'd wait seconds for a single lookup. A vector database exists to make that search feel instant.
Exact-match indexes are useless here — no vector is equal to your query. And comparing against every vector one at a time doesn't scale. We need a smarter map through the space.
The big idea
Here's the whole point of a vector database in one picture. The grey dots are stored vectors; the pink dot is your query. Run brute force and watch it exhaustively measure the distance to every dot — the comparison counter climbs to 40. Then run HNSW: the dots are pre-wired into a sparse graph, so the search just hops a few edges toward the query and lands on the answer in a handful of steps. Same result, a fraction of the work.
Measuring "close"
To find the "nearest" vectors, the database needs a way to score how close two of them are. For text, that score is almost always cosine similarity — it measures the angle between two vectors rather than the raw distance between their tips. Two fingerprints pointing in the same direction are treated as similar even if one is "longer" than the other, which is exactly what you want when comparing meaning: a short phrase and a long paragraph about the same idea should still count as close.
We'll unpack cosine similarity properly in Chapter 9 · Embeddings — the geometry, the intuition, and why the angle matters more than the length. For now, just hold onto the one-liner: closer angle = more similar meaning.
The landscape
You almost never build the HNSW index yourself — you reach for a vector database that does it for you. They all do the same core job; they differ in how they're run, what extra tricks they add, and how far they scale. Here's the field, with a one-line "reach for this when…" for each.
Cloud-native and fully managed — no servers to run, it just works. Reach for it when you want production similarity search without operating anything yourself.
Vector and keyword search fused into one engine. Reach for it when you want hybrid retrieval — meaning plus exact terms — from a single system.
Written in Rust, blazingly fast, with excellent metadata filtering. Reach for it when performance and rich "search-within-a-subset" filters matter.
The easiest to start with — runs locally, zero setup. Reach for it for prototypes and local development when you just want to build.
Built for billions of vectors and enterprise scale. Reach for it when your dataset is enormous and you need serious horizontal scale.
An extension that adds vector search to PostgreSQL. Reach for it when you already run Postgres and want vectors living right beside your existing data.
Don't overthink the choice. Prototype with Chroma — it runs locally and you'll be doing similarity search in about five minutes. When you go to production, move to Pinecone or Qdrant. That covers the vast majority of real projects.
A vector database is the agent's long-term memory bank. Fast, approximate similarity search is the machinery that makes RAG and agent memory practical at scale — without it, "remember everything and recall what's relevant" would simply be too slow to use.
Chapter 8 in one breath
A vector database stores millions of vectors and answers one question fast: which are most similar to this one? Brute force checks everything; HNSW hops a smart graph and checks a handful. Cosine similarity is the yardstick. Prototype on Chroma, ship on Pinecone or Qdrant — and you've built your agent a memory it can actually search.
A matching slide deck with speaker notes — press S for notes, F for fullscreen.