Vector Databases and Embeddings: How Semantic Search Works

The rise of large language models (LLMs) has made a once-niche technology a central component of modern software architecture: vector search.^[23] Systems like question-answering bots, recommendation engines, and multimodal search now depend on the ability to find data based on its conceptual meaning, not just its keywords.^[24] This is semantic search, and it's powered by two core components: embeddings, which convert complex data into numerical representations, and vector databases, which are specialized systems designed to search through those representations at incredible speed.^[25]

Understanding this technology requires looking past the marketing and into the mechanics of high-dimensional geometry and probabilistic algorithms. For engineers and technical leaders, a solid grasp of how these systems work is critical for making sound architectural decisions. It's a world of tradeoffs between speed, accuracy, cost, and complexity. Knowing the principles of vector search allows you to decide when to use it, what it will cost, and how it can fail.

From Words to Numbers: What Are Embeddings?

Computers do not process text, images, or sound. They process numbers. The first step in any semantic system is to translate unstructured data into a numerical format that a machine can understand. This format is a vector, which is essentially a list of numbers. The process of creating this vector is called embedding.^[26]

An embedding is a high-dimensional vector that captures the semantic essence of a piece of data. Early techniques for representing text numerically included counting word occurrences, such as Term Frequency-Inverse Document Frequency (TF-IDF)^[1], which assigns weights to words based on their rarity and frequency. While useful, these methods are lexical. They don't capture context or meaning. The word "bank" has the same representation whether it's a river bank or a financial bank.

Modern embeddings are generated by deep learning models. A key breakthrough came in 2013 with the development of Word2Vec^[2] by a team at Google led by Tomas Mikolov. This model learned vector representations of words from vast amounts of text. It demonstrated that semantic relationships could be encoded as mathematical operations. The classic example is that the vector for "King" minus the vector for "Man" plus the vector for "Woman" results in a vector very close to "Queen".

Today, sophisticated transformer-based models, like those available from OpenAI, Google, or open-source providers on Hugging Face^[3], create embeddings for sentences, paragraphs, or entire documents. When you send a piece of text to an embedding model's API, it processes the text through its neural network and returns a vector. For example, OpenAI's popular text-embedding-3-small model generates a vector with 1536 dimensions^[4]. The crucial property of these embeddings is that semantic similarity corresponds to spatial proximity. Two pieces of text that mean similar things will have vectors that are close to each other in this 1536-dimensional space.

The Search Problem: Finding Needles in a High-Dimensional Haystack

Once you have converted your dataset (perhaps millions of product descriptions, support articles, or user comments) into vectors, you face a new challenge. A user submits a query, you generate an embedding for that query, and now you must find the vectors in your database that are "closest" to the query vector. "Closeness" is typically measured using a distance metric like Euclidean distance^[5] (the straight-line distance between two points) or cosine similarity^[6] (the cosine of the angle between two vectors, which measures orientation rather than magnitude).

A naive approach would be to perform an exhaustive search, also known as a k-nearest neighbor (k-NN) search.^[27] You would compare the query vector to every single vector in your database, calculate the distance for each, keep track of the top k closest ones you've seen, and return the final list. This brute-force method is perfectly accurate. It guarantees finding the true nearest neighbors. It is also computationally disastrous at scale. The complexity is O(N*d), where N is the number of vectors and d is the number of dimensions. For a million 1536-dimension vectors, a single query requires billions of calculations, making it too slow for any real-time application.

This is where the core innovation of vector databases comes in: Approximate Nearest Neighbor (ANN) search. ANN algorithms trade a small, acceptable amount of accuracy for an enormous gain in speed. Instead of guaranteeing the absolute closest results, they provide results that are very likely to be among the closest, but do so orders of magnitude faster. For most semantic search applications, this tradeoff is excellent. A user doesn't care if the 5th result should have been the 4th, as long as all the top results are highly relevant.

The Core Mechanism: How ANN Search Works

ANN algorithms pre-process the vector data into an indexed data structure that enables fast searching. There are many different ANN algorithms, but most modern vector databases use graph-based or clustering-based methods. Understanding two of the most popular, HNSW and IVF, provides a solid foundation.

HNSW: Hierarchical Navigable Small World

Hierarchical Navigable Small World (HNSW)^[7] is one of the most performant and widely used ANN algorithms. It's a graph-based method that builds a structure analogous to a multi-level highway system for your vector space.

Imagine a social network where people are connected to their immediate neighbors and also to a few acquaintances in distant locations. This "small-world" property allows you to navigate from any point to any other point in a small number of hops. HNSW applies this concept to vectors.

Here's how it works step-by-step:

1. Index Construction: HNSW builds a layered graph. At the very bottom (layer 0), every vector is a node, connected to its nearest neighbors. This creates a dense graph. A probabilistic process then promotes a small fraction of these nodes to the layer above (layer 1). In this higher layer, another graph is built connecting these promoted nodes. This process repeats, with each successive layer becoming sparser than the one below. The top layer is a very sparse graph that serves as a fast entry point for searches.

2. Search Execution: A search begins at an entry point in the topmost, sparsest layer.

• The algorithm greedily traverses the graph, always moving to the neighbor node that is closest to the query vector.
• When it reaches a local minimum (a node where no neighbor is closer to the query), it drops down to the next layer below.
• It resumes the greedy search in this denser layer, starting from the point it dropped into.
• This process continues, navigating through progressively denser layers of the graph, until it reaches the bottom layer (layer 0).
• A final refinement search is performed in the bottom layer to find the best candidates. The closest nodes found during this traversal are returned as the approximate nearest neighbors.

HNSW's performance comes from its ability to make long jumps across the vector space in the upper layers and perform fine-grained searches in the lower layers. Its search time complexity is close to O(log N), a massive improvement over the O(N) of a linear scan.

IVF: Inverted File Index

Another popular method is the Inverted File Index (IVF), often used in libraries like Facebook AI Similarity Search (Faiss)^[8]. The intuition behind IVF is to partition the vector space into cells and only search a small subset of those cells, similar to using an index in a book instead of reading the entire book.

1. Training and Partitioning: Before indexing, the algorithm uses a clustering method like k-means^[9] to partition the vector space. It identifies k representative vectors called "centroids". Each centroid defines the center of a cell or region in the vector space.

2. Indexing: For each vector in your dataset, the algorithm determines which of the k centroids is closest. It then adds the vector (or its ID) to an "inverted list" associated with that centroid. The final index is essentially a map from centroid IDs to lists of vectors that belong to that cell.

3. Search Execution: When a query arrives:

• The query vector is compared to all k centroids to find the closest ones.
• The algorithm selects the nprobe nearest centroids (nprobe is a tunable parameter). A higher nprobe value increases accuracy at the cost of speed.
• Instead of searching the entire dataset, the search is restricted to only the vectors contained in the inverted lists of those nprobe cells.

The effectiveness of IVF depends heavily on the choice of k and nprobe. If k is too small, the cells are too large and searches are slow. If k is too large, the cells are too small and you might miss relevant vectors in adjacent cells unless nprobe is high. Often, IVF is combined with Product Quantization (PQ), a vector compression technique, to reduce memory usage. This combination is known as IVF-PQ.

A Concrete Example: Building a Simple Semantic Search

A simple Python example can clarify these concepts. This code uses the sentence-transformers library to generate embeddings and faiss-cpu to build an IVF index.

import faiss
import numpy as np
from sentence_transformers import SentenceTransformer

# 1. Define a corpus of documents
data = [
 "The cat sat on the mat.",
 "A feline was resting on the rug.",
 "Dogs are loyal companions.",
 "The sun is the center of our solar system.",
 "Our galaxy is called the Milky Way."
]

# 2. Load a pre-trained model to create embeddings
model = SentenceTransformer('all-MiniLM-L6-v2')
embeddings = model.encode(data)

# The model 'all-MiniLM-L6-v2' creates 384-dimensional vectors
dimension = embeddings.shape[1] # Should be 384

# 3. Build an IVF index using Faiss
num_cells = 2 # Number of cells to partition the data into
quantizer = faiss.IndexFlatL2(dimension) # The quantizer defines the cells
index = faiss.IndexIVFFlat(quantizer, dimension, num_cells)

# 4. Train the index on the data and add the embeddings
index.train(embeddings)
index.add(embeddings)

print(f"Index is trained: {index.is_trained}")
print(f"Total vectors in index: {index.ntotal}")

# 5. Define a query and embed it
query_text = "Where did the kitty sit?"
query_vector = model.encode([query_text])

# 6. Search the index
k = 2 # Number of nearest neighbors to retrieve
index.nprobe = 2 # Search both cells for higher accuracy
distances, indices = index.search(query_vector, k)

# 7. Print the results
print(f"\nQuery: '{query_text}'")
print(f"Top {k} most similar sentences:")
for i, idx in enumerate(indices[0]):
 print(f" - Result {i+1}: '{data[idx]}' (Distance: {distances[0][i]:.4f})")

Running this code will show that the query "Where did the kitty sit?" correctly identifies "The cat sat on the mat." and "A feline was resting on the rug." as the most similar sentences, even though they don't share many keywords with the query.

The Role of the Vector Database

While libraries like Faiss or Hnswlib^[10] provide the core ANN indexing algorithms, a full-fledged vector database provides a complete, production-ready system around them.^[26] A vector database combines an ANN index with the features expected of any modern database.

Key features include:

• Data and Vector Management: A vector database stores the vectors themselves alongside their source data IDs and any associated metadata. This is crucial for real-world applications.
• CRUD and Filtering: Production systems need to add, update, and delete data. A vector database handles these operations without requiring a full index rebuild, which is a complex task for some ANN structures. It also allows for metadata filtering, a killer feature. For example, a query might be: "Find products similar to this red dress, but only show items that are in stock and under $50." This combines a vector search with a traditional structured data filter.
• Scalability and Reliability: Vector databases are designed as distributed systems. They handle sharding data across multiple nodes, replicating for fault tolerance, and providing backups. These are standard database features, but they are non-trivial to implement for vector workloads.
• Developer Experience: They provide stable APIs, client SDKs for various languages, and operational tooling for monitoring and management.

Popular options in this space include managed services like Pinecone^[11], open-source projects you can host yourself like Weaviate^[12], Milvus, and Qdrant^[13], and extensions that add vector capabilities to traditional databases, such as pgvector for PostgreSQL^[14].

Real-World Tradeoffs and Decision Lens

As a technical leader, your decision to adopt vector search should be based on a clear understanding of its costs and limitations.

Cost and Performance

• Storage Cost: High-dimensional vectors are large. A single 1536-dimension vector using 32-bit floats takes up 6.144 KB. One billion of these vectors requires over 6 TB of storage, just for the vectors. Because ANN indexes need to be in memory (RAM) for fast querying, this translates directly to high infrastructure costs.
• Compute Cost: Embedding generation can be expensive, especially for large datasets or if you are running the models yourself. Indexing is a one-time (or periodic) CPU-intensive process. Querying is fast but consumes CPU per query.
• The Latency vs. Accuracy Tradeoff: This is the fundamental compromise of ANN. You must tune your index parameters (ef_construction and ef_search in HNSW, nprobe in IVF) to match your application's needs. A system for a chatbot needs low latency, while an offline data analysis task might prioritize higher accuracy (recall). You must benchmark to find the right balance for your specific data and query patterns.

Failure Modes and Common Misconceptions

• Misconception: It's magic. Vector search is just applied mathematics. Its results are only as good as the embedding model that created the vectors. If the model was trained on biased data, or its understanding of a topic is poor, the search results will reflect that. This is a classic "garbage in, garbage out" problem.
• Misconception: It replaces keyword search. It complements it. The best search systems often use a hybrid approach. Vector search excels at finding conceptually similar items.^[28] Keyword search, using algorithms like Okapi BM25^[15], is superior for finding exact matches for specific terms, product SKUs, or error codes. A hybrid system can run both types of queries and fuse the results for the best of both worlds.
• Failure Mode: Domain Mismatch. Using a general-purpose embedding model trained on Wikipedia and the web will produce poor results for a highly specialized domain like genomics, legal precedent, or internal financial reports. The model lacks the specific vocabulary and context. In these cases, you may need to fine-tune an embedding model on your own data, which is a significant machine learning project.
• Failure Mode: The Curse of Dimensionality. While ANN algorithms are designed to fight the curse of dimensionality^[16], where space becomes vast and sparse as dimensions increase, performance still degrades with extremely high dimensions. Modern models balance dimensionality and information content, but it remains a fundamental constraint.

When to use it:

• Your primary need is searching by meaning or concept, not exact keywords.
• Applications include recommendation engines, question-answering systems, image or audio similarity searches, and plagiarism detection.
• It's a core component of the Retrieval-Augmented Generation (RAG)^[17] pattern, where you retrieve relevant documents from a vector database to provide context to an LLM before it generates an answer.

When not to use it:

• You require 100% accurate recall. In this case, you need an exact k-NN search, and you must be prepared for the performance cost.
• Your searches are based on exact, known identifiers. A standard database with a B-tree^[18] index is far more efficient for this.
• Your dataset is small (e.g., a few thousand items). A brute-force linear scan might be fast enough and avoids the complexity of setting up a dedicated vector search system.

The State of the Art

The vector database landscape is maturing rapidly. Embedding models continue to improve, with leaderboards like the Massive Text Embedding Benchmark (MTEB)^[19] providing objective measures of performance across various tasks. The market is seeing both purpose-built vector databases and the integration of vector search capabilities into established database platforms. Major cloud providers now offer integrated solutions, such as Amazon OpenSearch Service's k-NN support and Google Cloud's Vertex AI Vector Search^[20].

Currently, there is no formal ISO^[21] or IETF^[22] standard for vector database APIs, so each product has its own interface. However, the underlying concepts of indexing, searching, and filtering are becoming common across all of them. What was once a specialized tool for machine learning researchers is now a foundational piece of the modern technology stack, driven by the simple, powerful idea of turning meaning into math.