Home › Learn › Databricks interview prep › Vector Search

Databricks interview prep · data engineering

Vector Search: internals, limits, production and maintenance

How Databricks Vector Search works inside, its limits, how to ingest, production and maintenance challenges, and when not to use it.

Maintenance commandsRow-level securityCompute choiceJoinsSkew and shuffleAQE, Photon, POCost and performance leversDeclarative and semanticAsset BundlesLakeflow ConnectUniFormVector Searchdbt and ODCSEgress and ingressMaintenance in picturesClustering and compactionMaintenance strategySlow writesBackup and recoveryTaggingQuestion bank

Defaults, limits and preview status change often. Check the current Databricks docs before quoting a number.

Source Delta table (text,key, optional vectors)Embedding modelIndex on a Vector SearchendpointQuery: filters, hybrid,rerank, top-k

Documents become vectors; a query finds the nearest ones.

The idea

What it is

A managed service for similarity search: you store embeddings and, given a query embedding, it returns the nearest items. It is the retrieval step in RAG and semantic search. Indexes are Unity Catalog objects; a Delta table is the source of truth.

Internals

How it works

Exact search compares a query against every vector, which is too slow at millions. Production systems use approximate nearest neighbour (ANN) search. Databricks' service uses an HNSW-style graph: each vector links to nearby ones; a search enters at a sparse top layer, hops toward the query, drops to denser layers and finishes with a local search. Knobs trade recall, latency, memory and build time; the managed service mostly sets them for you. The distance measure (cosine, dot product, L2) must match the embedding model.

PieceRole
Source Delta tableText, primary key and optionally the embeddings
Embedding modelManaged (a hosted endpoint the index calls) or self-managed (you supply vectors)
EndpointThe compute that hosts indexes and serves queries
IndexANN structure plus stored metadata columns
Sync processFor Delta Sync indexes, reads changes from the table (needs Change Data Feed)
Query pathEmbed the query, ANN search, filters, optional hybrid and rerank, return top-k with metadata
Index typeHow it is fedUse when
Delta Sync IndexAutomatic from a Delta table; triggered or continuous syncDefault
Direct Vector AccessYou upsert and delete through the APISource not Delta, or you need control

Search features: metadata filtering; hybrid search (vector plus keyword, merged by reciprocal rank fusion) helps with names and codes; reranking rescoring the top candidates for precision at extra latency.

Limitations

What it cannot do

Numbers change. Check current limits for index size, dimension, throughput and endpoint types (a standard type for low latency and a storage-optimized type for larger scale).

AreaLimitation
Scale capsMaximum vectors per index and endpoint, embedding dimension, query and ingest throughput
Eventual consistencyNew rows appear after the sync completes
Change Data FeedRequired on the source; breaking its history disrupts sync
SchemaPrimary key must be stable; changing the embedding column or model usually means a rebuild
Filter behaviourSyntax narrower than SQL; very selective filters can reduce recall or add latency
One model per indexCannot mix vectors from different models
No automatic row-level securityAccess is at index and endpoint level through Unity Catalog; per-user rows need filters your application enforces (check current state)
ApproximateRecall below 100 percent; results can shift after updates
Quality is yoursChunking and embedding quality are not decided by the service
Cost floorA running endpoint costs money when idle
AvailabilityNot every feature in every region or cloud

Ingestion

How to get data in

  1. Land raw files in a Unity Catalog volume with Auto Loader.
  2. Parse to text and clean (remove boilerplate and duplicates).
  3. Chunk into passages of a few hundred tokens with some overlap, respecting headings. Make chunk IDs deterministic (document ID plus position) and keep metadata (source, section, date, access group, version).
  4. Write chunks to a Delta table with Change Data Feed on and a primary key.
  5. Create the Delta Sync index with managed or self-managed embeddings.
  6. Sync after each pipeline run, or continuously if freshness demands.
  7. Query through the SDK, REST or SQL, with filters and optional hybrid search.

Production

Production-grade challenges

ChallengeWhat to do
Retrieval qualityBuild an evaluation set of real questions with known relevant chunks; measure recall at k and answer quality; change one variable at a time
ChunkingStructure-aware chunks, overlap, parent-document metadata
LatencyCache frequent query embeddings, limit k, rerank only the top candidates, measure each stage
ThroughputLoad test before launch, scale or split indexes, rate-limit callers
Filtered recallTest filter-heavy queries; consider separate indexes for very different partitions
Multi-tenancyEnforce the tenant filter on the server, never trusting the client; separate indexes for sensitive tenants; restrict permissions
FreshnessContinuous or short triggered syncs; monitor lag
Hybrid needsPure vector search misses exact identifiers; test hybrid on your queries
Model dependencePin versions, plan migration, monitor endpoint health and quotas
CostRight-size the endpoint, drop unused indexes, avoid re-embedding unchanged content
ObservabilityLog queries, filters, retrieved chunk IDs, scores and feedback, with privacy controls
Sensitive dataVectors can leak information about the source text; apply the source's classification and access rules

Maintenance

Maintenance challenges

TaskDetail
Changing the embedding modelRe-embed the whole corpus and rebuild; run the new index in parallel, compare quality, then switch traffic
Changing chunkingReprocess and rebuild
Sync failures and lagCauses: CDF disabled or history vacuumed, schema change, embedding endpoint limits, permissions
Source table changesRenaming or retyping the key or text column can break the index; treat the table as a contract
Deletes and complianceRemove chunks from the source and confirm they leave the index; test it
Index driftRe-run evaluation as data grows or shifts
Orphaned or duplicate chunksFrom non-deterministic IDs; reconcile source and index counts regularly
Endpoint lifecycleUpgrades and scaling may need parallel indexes
Disaster recoveryThe index can be rebuilt from the Delta table; know the rebuild time
Regression testsRun the evaluation set in CI whenever pipeline, model or chunking changes

Choose

When it is not the right tool

SituationAlternative
Exact keyword matching is enoughSQL, full-text or a search engine
A few thousand vectorsAn in-memory library (governance may still favour the service)
Transactional updates with vectorsA database with vector support, or Lakebase (check status)
Exotic ANN tuning or distance measuresA dedicated vector database
Relational queries mixed with similaritySQL on Delta for the structured part plus Vector Search for the semantic part

Practice

Questions to rehearse

  1. Why approximate rather than exact search, and what do you give up?
  2. Delta Sync versus Direct Vector Access?
  3. Why does Delta Sync need Change Data Feed?
  4. How do you handle updates and deletes so stale chunks never appear?
  5. A user says the answer is wrong. Is retrieval or generation at fault?
  6. How do you implement multi-tenant isolation?
  7. Describe a no-downtime embedding model migration.
  8. Why can strong filters reduce recall?
  9. How would you size 50 million chunks at 100 queries per second?
In an interview, say: Vector Search is an approximate nearest-neighbour index kept in sync from a Delta table. The hard parts are chunking, evaluation, tenant isolation, updates and deletes, freshness, and the migration whenever the embedding model changes.

FAQ

Questions people ask

Delta Sync or Direct Vector Access?

Delta Sync when the data lives in Delta. Direct access when you need control of every upsert.

Do the source table's row filters apply to the index?

Do not assume so. Enforce tenant filters in your application and check the current behaviour.

Next

Where to go next

All Databricks interview topics · All data engineering · Databricks on this site