Home › Learn › Databricks interview prep › Vector Search
Databricks interview prep · data engineering
Vector Search: internals, limits, production and maintenance
How Databricks Vector Search works inside, its limits, how to ingest, production and maintenance challenges, and when not to use it.
Defaults, limits and preview status change often. Check the current Databricks docs before quoting a number.
Documents become vectors; a query finds the nearest ones.
The idea
What it is
A managed service for similarity search: you store embeddings and, given a query embedding, it returns the nearest items. It is the retrieval step in RAG and semantic search. Indexes are Unity Catalog objects; a Delta table is the source of truth.
Internals
How it works
Exact search compares a query against every vector, which is too slow at millions. Production systems use approximate nearest neighbour (ANN) search. Databricks' service uses an HNSW-style graph: each vector links to nearby ones; a search enters at a sparse top layer, hops toward the query, drops to denser layers and finishes with a local search. Knobs trade recall, latency, memory and build time; the managed service mostly sets them for you. The distance measure (cosine, dot product, L2) must match the embedding model.
| Piece | Role |
|---|---|
| Source Delta table | Text, primary key and optionally the embeddings |
| Embedding model | Managed (a hosted endpoint the index calls) or self-managed (you supply vectors) |
| Endpoint | The compute that hosts indexes and serves queries |
| Index | ANN structure plus stored metadata columns |
| Sync process | For Delta Sync indexes, reads changes from the table (needs Change Data Feed) |
| Query path | Embed the query, ANN search, filters, optional hybrid and rerank, return top-k with metadata |
| Index type | How it is fed | Use when |
|---|---|---|
| Delta Sync Index | Automatic from a Delta table; triggered or continuous sync | Default |
| Direct Vector Access | You upsert and delete through the API | Source not Delta, or you need control |
Search features: metadata filtering; hybrid search (vector plus keyword, merged by reciprocal rank fusion) helps with names and codes; reranking rescoring the top candidates for precision at extra latency.
Limitations
What it cannot do
Numbers change. Check current limits for index size, dimension, throughput and endpoint types (a standard type for low latency and a storage-optimized type for larger scale).
| Area | Limitation |
|---|---|
| Scale caps | Maximum vectors per index and endpoint, embedding dimension, query and ingest throughput |
| Eventual consistency | New rows appear after the sync completes |
| Change Data Feed | Required on the source; breaking its history disrupts sync |
| Schema | Primary key must be stable; changing the embedding column or model usually means a rebuild |
| Filter behaviour | Syntax narrower than SQL; very selective filters can reduce recall or add latency |
| One model per index | Cannot mix vectors from different models |
| No automatic row-level security | Access is at index and endpoint level through Unity Catalog; per-user rows need filters your application enforces (check current state) |
| Approximate | Recall below 100 percent; results can shift after updates |
| Quality is yours | Chunking and embedding quality are not decided by the service |
| Cost floor | A running endpoint costs money when idle |
| Availability | Not every feature in every region or cloud |
Ingestion
How to get data in
- Land raw files in a Unity Catalog volume with Auto Loader.
- Parse to text and clean (remove boilerplate and duplicates).
- Chunk into passages of a few hundred tokens with some overlap, respecting headings. Make chunk IDs deterministic (document ID plus position) and keep metadata (source, section, date, access group, version).
- Write chunks to a Delta table with Change Data Feed on and a primary key.
- Create the Delta Sync index with managed or self-managed embeddings.
- Sync after each pipeline run, or continuously if freshness demands.
- Query through the SDK, REST or SQL, with filters and optional hybrid search.
- Batch embedding calls to respect model rate limits.
- Updates and deletes: delete all old chunks of a changed document, then write the new ones; otherwise stale chunks stay searchable.
- Keep parsing separate from embedding so you can re-embed without re-parsing.
- Store the embedding model name and version.
- Direct Vector Access: you own retries, idempotency and ordering.
Production
Production-grade challenges
| Challenge | What to do |
|---|---|
| Retrieval quality | Build an evaluation set of real questions with known relevant chunks; measure recall at k and answer quality; change one variable at a time |
| Chunking | Structure-aware chunks, overlap, parent-document metadata |
| Latency | Cache frequent query embeddings, limit k, rerank only the top candidates, measure each stage |
| Throughput | Load test before launch, scale or split indexes, rate-limit callers |
| Filtered recall | Test filter-heavy queries; consider separate indexes for very different partitions |
| Multi-tenancy | Enforce the tenant filter on the server, never trusting the client; separate indexes for sensitive tenants; restrict permissions |
| Freshness | Continuous or short triggered syncs; monitor lag |
| Hybrid needs | Pure vector search misses exact identifiers; test hybrid on your queries |
| Model dependence | Pin versions, plan migration, monitor endpoint health and quotas |
| Cost | Right-size the endpoint, drop unused indexes, avoid re-embedding unchanged content |
| Observability | Log queries, filters, retrieved chunk IDs, scores and feedback, with privacy controls |
| Sensitive data | Vectors can leak information about the source text; apply the source's classification and access rules |
Maintenance
Maintenance challenges
| Task | Detail |
|---|---|
| Changing the embedding model | Re-embed the whole corpus and rebuild; run the new index in parallel, compare quality, then switch traffic |
| Changing chunking | Reprocess and rebuild |
| Sync failures and lag | Causes: CDF disabled or history vacuumed, schema change, embedding endpoint limits, permissions |
| Source table changes | Renaming or retyping the key or text column can break the index; treat the table as a contract |
| Deletes and compliance | Remove chunks from the source and confirm they leave the index; test it |
| Index drift | Re-run evaluation as data grows or shifts |
| Orphaned or duplicate chunks | From non-deterministic IDs; reconcile source and index counts regularly |
| Endpoint lifecycle | Upgrades and scaling may need parallel indexes |
| Disaster recovery | The index can be rebuilt from the Delta table; know the rebuild time |
| Regression tests | Run the evaluation set in CI whenever pipeline, model or chunking changes |
Choose
When it is not the right tool
| Situation | Alternative |
|---|---|
| Exact keyword matching is enough | SQL, full-text or a search engine |
| A few thousand vectors | An in-memory library (governance may still favour the service) |
| Transactional updates with vectors | A database with vector support, or Lakebase (check status) |
| Exotic ANN tuning or distance measures | A dedicated vector database |
| Relational queries mixed with similarity | SQL on Delta for the structured part plus Vector Search for the semantic part |
Practice
Questions to rehearse
- Why approximate rather than exact search, and what do you give up?
- Delta Sync versus Direct Vector Access?
- Why does Delta Sync need Change Data Feed?
- How do you handle updates and deletes so stale chunks never appear?
- A user says the answer is wrong. Is retrieval or generation at fault?
- How do you implement multi-tenant isolation?
- Describe a no-downtime embedding model migration.
- Why can strong filters reduce recall?
- How would you size 50 million chunks at 100 queries per second?
FAQ
Questions people ask
Delta Sync or Direct Vector Access?
Delta Sync when the data lives in Delta. Direct access when you need control of every upsert.
Do the source table's row filters apply to the index?
Do not assume so. Enforce tenant filters in your application and check the current behaviour.
Next
Where to go next
All Databricks interview topics · All data engineering · Databricks on this site