How to Get Started with Vector Databases: Embeddings, Similarity Search, and RAG

Prerequisites

  • Python 3.11+
  • Basic understanding of LLMs and embeddings
  • chromadb or pgvector installed

What is a Vector Database?

Traditional databases search by exact keyword matches. A vector database searches by semantic meaning. You store text as mathematical vectors (embeddings) and query with “find documents similar in meaning to this text.”

Traditional: "Find all rows WHERE name = 'John'"
Vector:      "Find documents similar to 'how to debug a memory leak'"

Step 1: Generate Embeddings

Embeddings are dense vectors (lists of floats) that represent meaning. OpenAI’s text-embedding-3-small produces 1536-dimensional vectors.

from openai import OpenAI

client = OpenAI()

def get_embedding(text: str) -> list[float]:
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=text
    )
    return response.data[0].embedding

# Example
embedding = get_embedding("How to configure PostgreSQL replication")
# [0.0123, -0.0456, 0.0789, ...]  # 1536 numbers

Step 2: Set Up ChromaDB

ChromaDB is a lightweight open-source vector database:

pip install chromadb

import chromadb

client = chromadb.PersistentClient(path="./chroma_data")

collection = client.get_or_create_collection(
    name="docs",
    metadata={"hnsw:space": "cosine"}  # Cosine similarity
)

Step 3: Chunk Documents

Don’t embed entire documents. Split into small chunks for precise retrieval:

def chunk_text(text: str, chunk_size: int = 500, overlap: int = 50) -> list[str]:
    """Split text into overlapping chunks."""
    words = text.split()
    chunks = []
    start = 0
    while start < len(words):
        chunk = " ".join(words[start:start + chunk_size])
        chunks.append(chunk)
        start += chunk_size - overlap  # Overlap maintains context across chunks
    return chunks

document = "Your long document text here..."
chunks = chunk_text(document)

Chunk size guidelines:

  • 256-512 tokens: best for precise Q&A
  • 1000-2000 tokens: better for summarisation
  • Always include 10-20% overlap between chunks

Step 4: Store Chunks with Metadata

for i, chunk in enumerate(chunks):
    embedding = get_embedding(chunk)
    collection.add(
        ids=[f"doc_{i}"],
        embeddings=[embedding],
        documents=[chunk],
        metadatas=[{
            "source": "postgresql_guide.pdf",
            "page": i,
            "topic": "database"
        }]
    )

Metadata filtering enables hybrid search:

# Semantic search filtered by metadata
results = collection.query(
    query_embeddings=[get_embedding("replication setup")],
    n_results=3,
    where={"topic": "database"}
)

Query the vector database:

query = "How do I set up PostgreSQL replication?"
query_embedding = get_embedding(query)

results = collection.query(
    query_embeddings=[query_embedding],
    n_results=5
)

for i, doc in enumerate(results["documents"][0]):
    distance = results["distances"][0][i]
    print(f"Distance: {distance:.4f} | {doc[:100]}...")

Similarity metrics:

MetricRangeBest for
Cosine[-1, 1]Text similarity (default)
Euclidean[0, ∞)Image/sensor data
Dot Product[-∞, ∞]Normalized embeddings

Step 6: Basic RAG Pipeline

RAG (Retrieval-Augmented Generation) retrieves relevant documents and feeds them to an LLM:

def rag_query(query: str, collection, llm_client) -> str:
    # 1. Get query embedding
    query_embedding = get_embedding(query)

    # 2. Retrieve relevant chunks
    results = collection.query(
        query_embeddings=[query_embedding],
        n_results=3
    )

    # 3. Build context from retrieved chunks
    context = "\n\n".join(results["documents"][0])

    # 4. Generate response with context
    response = llm_client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {
                "role": "system",
                "content": "Answer the question using ONLY the provided context. If the answer isn't in the context, say 'I don't have enough information.'"
            },
            {
                "role": "user",
                "content": f"Context:\n{context}\n\nQuestion: {query}"
            }
        ]
    )
    return response.choices[0].message.content

Using pgvector Instead

If you already use PostgreSQL, add vector search with pgvector:

CREATE EXTENSION vector;

CREATE TABLE documents (
    id SERIAL PRIMARY KEY,
    content TEXT,
    embedding vector(1536)
);

-- Create an index for fast similarity search
CREATE INDEX ON documents USING ivfflat (embedding vector_cosine_ops);

-- Query
SELECT content, 1 - (embedding <=> '[0.1, 0.2, ...]') AS similarity
FROM documents
ORDER BY embedding <=> '[0.1, 0.2, ...]'
LIMIT 5;

pgvector is production-ready and avoids running a separate vector DB.

Verification

# Add test documents
collection.add(
    ids=["test1"],
    embeddings=[get_embedding("PostgreSQL replication uses WAL shipping.")],
    documents=["PostgreSQL replication uses WAL shipping."]
)

collection.add(
    ids=["test2"],
    embeddings=[get_embedding("Python virtual environments isolate dependencies.")],
    documents=["Python virtual environments isolate dependencies."]
)

# Query about databases
results = collection.query(
    query_embeddings=[get_embedding("How does database replication work?")],
    n_results=2
)

print(results["documents"])
# [['PostgreSQL replication uses WAL shipping.', 'Python virtual environments isolate dependencies.']]
# First result is correct (database topic)

Summary

  • Embeddings convert text into semantic vectors
  • Chunking splits documents into retrievable pieces with overlap
  • ChromaDB is great for prototyping; pgvector for production
  • Metadata filtering enables hybrid search
  • RAG = retrieve relevant chunks + feed them to LLM for grounded answers

References

Advertisement