ChromaDB: Getting Started with Embedded Vector Stores

When building prototypes, local development environments, or lightweight serverless applications, managing a distributed cloud database cluster adds unnecessary operational overhead.

ChromaDB is an open-source, AI-native embedded vector database designed for Python. It requires zero cloud infrastructure, runs directly inside your application process, and persists vector indices locally to disk using DuckDB, SQLite, and Parquet.

1. ChromaDB Architecture

flowchart TD
    subgraph ChromaEngine["ChromaDB In-Memory & Local Architecture"]
        Client["Python Application / FastAPI / Airflow"] --> API["ChromaDB Client Interface"]
        API --> Index["HNSW Vector Index (Fast C++ ANN Search)"]
        API --> MetaStore["Metadata & Document Store (SQLite / DuckDB)"]

        Index --> Disk[("Local Storage Directory<br/>Parquet & HNSW Binaries")]
        MetaStore --> Disk
    end

    style Client fill:#1e293b,color:#fff
    style API fill:#2563eb,color:#fff
    style Index fill:#059669,color:#fff
    style MetaStore fill:#7c3aed,color:#fff
    style Disk fill:#d97706,color:#fff

Key Advantages:

1. Zero Deployment: No Docker containers or cloud accounts needed—simply pip install chromadb.
2. Built-in Embeddings: Integrates automatically with Sentence-Transformers or OpenAI; if you pass raw strings, ChromaDB can embed them for you automatically.
3. Rich Metadata Filtering: Query by vectors while filtering by business attributes (where={"environment": "production"}).

2. Core API Operations

ChromaDB organizes vectors into Collections (equivalent to tables in SQL):

3. Hands-on: Building a Technical Documentation Search Engine

Here is a complete, runnable script using ChromaDB with persistent storage and metadata filtering:

import os
import chromadb
from chromadb.utils import embedding_functions

# Step 1: Initialize persistent client
CHROMA_DATA_DIR = "./scratch/chroma_db_demo"
os.makedirs(CHROMA_DATA_DIR, exist_ok=True)

client = chromadb.PersistentClient(path=CHROMA_DATA_DIR)

# Step 2: Create or retrieve a collection
# Default embedding function: all-MiniLM-L6-v2 (runs locally on CPU)
default_ef = embedding_functions.DefaultEmbeddingFunction()
collection = client.get_or_create_collection(
    name="data_platform_docs",
    embedding_function=default_ef,
    metadata={"description": "Internal Data Platform architecture documentation"}
)

# Step 3: Upsert technical passages with rich metadata
documents = [
    "Snowflake dynamic tables automatically refresh incremental data based on target lag specifications.",
    "PySpark broadcast join should be used when joining a massive fact table with a dimension table under 50MB.",
    "Airflow CeleryExecutor distributes task execution across a worker farm using Redis as the message broker.",
    "Databricks Unity Catalog provides fine-grained row and column-level security across multi-cloud lakehouses."
]

metadatas = [
    {"system": "snowflake", "layer": "storage", "author": "analytics_eng"},
    {"system": "spark", "layer": "compute", "author": "data_platform"},
    {"system": "airflow", "layer": "orchestration", "author": "sre"},
    {"system": "databricks", "layer": "governance", "author": "security"}
]

ids = ["doc_sf_01", "doc_spark_01", "doc_af_01", "doc_dbx_01"]

collection.upsert(
    documents=documents,
    metadatas=metadatas,
    ids=ids
)
print(f"Collection count: {collection.count()} documents indexed.")

# Step 4: Semantic Query with Metadata Filtering
query_text = "How do we speed up joining small tables in distributed compute?"

results = collection.query(
    query_texts=[query_text],
    n_results=2,
    where={"layer": "compute"}  # Metadata filter!
)

print("\n=== Search Results (Filtered by layer=compute) ===")
for i in range(len(results["documents"][0])):
    doc_id = results["ids"][0][i]
    doc_text = results["documents"][0][i]
    meta = results["metadatas"][0][i]
    distance = results["distances"][0][i]
    print(f"\nRank [{i+1}] (ID: {doc_id}, Distance: {distance:.4f}):")
    print(f"System: {meta['system'].upper()} | Metadata: {meta}")
    print(f"Content: {doc_text}")

Key Takeaways

• ChromaDB is ideal for local development, CI/CD testing, and embedded Python applications.
• PersistentClient saves vector indices to disk so they survive application restarts.
• Metadata filtering (where) enables high-precision scoping before running vector similarity searches.