Home › AWS › AWS data services › OpenSearch

AWS data services · Operational stores · about 9 min read

Amazon OpenSearch Service (and Serverless)

Managed OpenSearch for full-text search, log analytics and vector search, as domains or Serverless; not on this platform's main path.

RoleOperational stores
StatusGenerally available. The next generation of OpenSearch Serverless (GA May 2026) separates compute from storage and can scale to zero after 10 idle minutes; collection groups (Feb 2026) share capacity across collections.
Verdict on this siteKeep if you have it
AzureAzure AI Search, or Elastic on Azure
Google CloudVertex AI Search, or Elastic Cloud on Google Cloud
DatabricksDatabricks Vector Search for embeddings; SQL warehouse for analytics
Open sourceOpenSearch, Elasticsearch, Apache Solr
Verdict: Keep if you have it. The reference platform does not need it: Databricks Vector Search serves retrieval for AI, CloudWatch holds operational logs, and Delta tables answer analytics. Keep an existing OpenSearch estate for what it does well, full-text search in an application and log search, and feed it from gold tables; do not add it as a second analytics store.

In plain words

What it is

Amazon OpenSearch Service runs OpenSearch, the open-source search and analytics engine that began as a fork of Elasticsearch. You send it JSON documents, it builds an inverted index, which is a list of every word pointing to the documents that contain it, and then it can answer text searches in milliseconds, rank results by relevance, handle typos and synonyms, and count results by category. It also stores vectors, the lists of numbers that AI models produce to represent meaning, and finds the nearest ones, which is the core of semantic search.

It comes in two shapes. A managed domain is a cluster of nodes you size: data nodes hold shards of each index, dedicated manager nodes keep the cluster in order, and warm and cold tiers hold older data more cheaply. OpenSearch Serverless has no nodes to choose. You create collections for search, vector search or time series, and AWS adds and removes OpenSearch Compute Units (OCUs) as load changes. The next generation of Serverless, generally available since May 2026, keeps data in a shared storage layer and can scale compute to zero after ten idle minutes, waking in about ten seconds.

A retail example: an online store with 200,000 products needs a search box that understands "waterproof hiking boots size 9", forgives a typo, and shows filters for brand and price with counts. That is what OpenSearch is built for. The product data itself is mastered elsewhere; a job takes the gold product table from Databricks every hour and indexes it into OpenSearch for the website.

The index at the back of a very large book, kept up to date as pages change, that can also find pages that mean the same thing as your words.

Big picture

Where it fits

SourcesMove inStreamStoreProcessServeUseAcross everything: catalog · security · orchestration · observability and cost · infrastructure as codegets data fromhands data toGold tables in Databricks (scheduled export)Data FirehoseApplication logsOpenSearchOperational storesWebsite or app searchOpenSearch DashboardsBedrock

The lit lane is where OpenSearch works. Boxes on the left feed it; boxes on the right use what it produces.

OpenSearch is not on the reference platform's main path. Ingestion runs through DMS, AppFlow, Transfer Family and Kinesis into S3; Databricks builds bronze, silver and gold; serving uses the Databricks SQL warehouse, Athena for ad hoc queries and Redshift where it is kept for existing reports. Each common reason to add OpenSearch is already covered: retrieval for AI assistants uses Databricks Vector Search on governed tables, operational logs and metrics live in CloudWatch, and analytics runs on Delta tables.

Where it does fit is at the edge, as a consumer of gold. An application team that needs full-text search for a product catalog, a spare-parts catalogue for maintenance engineers or a knowledge base of quality reports can index from gold tables on a schedule. The gold table stays the record; the index is a derived copy that can be rebuilt. If the company already runs OpenSearch for log analytics or security events, keep it for that and do not move platform analytics into it.

The governance gap is the main reason for caution. OpenSearch indices are outside Unity Catalog, so masks, lineage and access reviews do not follow the data there. Anything indexed must be reviewed for personal data before it leaves the platform.

How it works

The ideas you need

IdeaWhat it means
IndexA named collection of documents with a mapping that says how each field is stored and searched, much like a table.
Shard and replicaAn index is split into shards spread across nodes; replicas are copies for availability and read throughput.
DomainA managed OpenSearch cluster you size, with data nodes, optional dedicated manager nodes and storage tiers.
CollectionThe OpenSearch Serverless unit: a group of indices of one type (search, vector search or time series) with capacity managed by AWS.
OCUOpenSearch Compute Unit, the Serverless measure of indexing and search capacity, billed per hour.
MappingThe schema of an index; a field mapped as text is analysed into words, a keyword field is matched exactly. Changing it usually needs a reindex.
k-NN vector searchFinding the stored vectors nearest to a query vector, used for semantic search and retrieval for AI assistants.
UltraWarm and cold storageCheaper tiers on domains that keep older, read-only indices on S3-backed storage with slower queries.

Choosing

When to use it, and when not

Use it when

  • An application needs full-text search with relevance ranking, typo tolerance, synonyms and facet counts, such as a product or parts catalog.
  • You already run OpenSearch or Elasticsearch for log or security analytics and want AWS to manage it.
  • You need hybrid search, combining keyword and vector similarity, inside an application outside Databricks.
  • Search load is spiky or idle most of the time, and next-generation Serverless can scale to zero between uses.

Avoid it when

  • The question is analytics over tables, such as sales by store and month; use the Databricks SQL warehouse.
  • You want retrieval for an AI assistant over governed platform data; Databricks Vector Search keeps it inside Unity Catalog.
  • It would be the record for any data; indices should be rebuildable copies of gold.
  • You need joins across entities; OpenSearch works on denormalised documents and has no real joins.
  • The team has no one to tune shards, mappings and memory on a domain, and Serverless does not fit the workload.

Production grade

What goes wrong in production

ProblemWhat breaksWhat to do
Too many small shardsEach index gets the default shard count, daily indices multiply them, and the cluster spends memory managing thousands of tiny shards.Aim for shards of roughly 10 to 50 GB, use index templates and rollover, and watch the per-node shard limit.
Mapping explosionDynamic mapping turns every new JSON key into a field until the mapping hits its field limit and indexing fails.Define explicit mappings, set dynamic to strict or false for untrusted data, and flatten free-form attributes.
Heap pressureLarge aggregations or many vectors fill the Java heap, garbage collection stalls and nodes drop out.Alarm on JVMMemoryPressure, size nodes for vectors, and limit aggregation sizes.
Write rejections at peakBulk indexing faster than the cluster can absorb leads to rejected requests and missing documents.Batch bulk requests, retry with backoff, and alarm on rejected thread-pool counts. Index from gold on a schedule, not row by row.
Stale or drifting indexThe index drifts from gold because deletes were never propagated.Rebuild the index from gold into a new index and switch an alias, or propagate deletes from the change feed of the gold table.
Serverless minimum costClassic Serverless collections keep a minimum of OCUs running all month, even for a small test.Use next-generation collections with minimum capacity set to zero, or group collections to share OCUs.
Data outside governancePersonal data indexed for search escapes Unity Catalog masks and access reviews.Index only the fields search needs, mask or drop personal data in the export job, and use fine-grained access control.

Alternatives

What to compare it with

AlternativePick the alternative whenPick OpenSearch when
Databricks Vector SearchYou need retrieval for AI assistants over platform data, governed in Unity Catalog and synced from Delta tables.An external application needs full-text search with facets and typo tolerance.
Databricks SQL warehouseThe question is analytics: totals, trends and joins over tables.You need relevance-ranked text search.
Amazon CloudWatch LogsYou need operational logs and alarms for AWS services and jobs, with Logs Insights queries.You need long-term, heavily searched log analytics at large scale with custom dashboards.
Elastic CloudYou want Elasticsearch features and support from Elastic itself.You prefer an AWS-managed open-source engine.
Azure AI SearchYou are building application search on Azure.You are on AWS.
Amazon S3 Vectors or Bedrock Knowledge BasesYou want cheap vector storage or managed retrieval for a Bedrock application.You need full-text relevance and facets.

Cost

How you pay

Managed domains bill per instance-hour for every data, manager and warm node, plus EBS storage per GB-month, UltraWarm and cold storage, snapshots beyond the free automated ones, and data transfer. Reserved instances lower the rate. OpenSearch Serverless bills OCU-hours for indexing and search separately, plus storage per GB-month in S3-backed managed storage. Classic Serverless collections keep a minimum number of OCUs running; next-generation collections can drop compute to zero when idle and bill only storage. OpenSearch Ingestion pipelines bill their own capacity units. List prices, check your region.

What drives the bill

  • Node count and size on domains, including dedicated manager nodes and replicas.
  • OCU-hours on Serverless, driven by indexing rate and query load.
  • Data volume and retention, split across hot, warm and cold tiers.
  • Vector count and dimensions, which need memory.

Cost traps

  • Classic Serverless minimum OCUs billed around the clock for a small or test collection.
  • Clusters sized for peak with three zones of replicas that idle most of the day.
  • Keeping months of logs in hot storage instead of warm or cold tiers.
  • Re-indexing the full catalog every few minutes instead of only changes.

Prices change; check the AWS pricing page for your region before deciding.

Security and governance

Keeping it safe

  • Create domains inside a VPC, or Serverless collections with VPC endpoints, and disable public access.
  • Turn on fine-grained access control and map IAM roles or SAML groups to OpenSearch roles with index- and field-level permissions.
  • Encrypt at rest with KMS, require node-to-node encryption and HTTPS with a modern TLS policy.
  • Turn on audit logs for domains and send them to CloudWatch Logs; CloudTrail records configuration calls.
  • Index only fields search needs; mask or drop personal data in the export job, because Unity Catalog policies do not follow data into OpenSearch.

Operating it

What to watch, and the limits

WatchWhy
ClusterStatus.redOne or more primary shards are unassigned; some data cannot be searched or written.
JVMMemoryPressureHeap use on nodes; sustained values above about 80 percent precede garbage collection stalls and node loss.
FreeStorageSpaceDisk left per node; low values block writes when the cluster sets indices to read-only.
ThreadpoolWriteRejectedBulk write requests rejected because the cluster is overloaded.
SearchLatencyTime to answer searches; the measure users feel.
SearchOCU and IndexingOCUCapacity used by a Serverless collection, which is what you pay for.
LimitValue and notes
Shards per node1,000 per 16 GB of heap, up to 4,000, on OpenSearch 2.17 and later
Data nodes per domainUp to 1,002 hot nodes across three zones, by instance family and request
HTTP request payload10 MiB on small instance types, 100 MiB on larger ones
Shard size65 GiB maximum per shard for Multi-AZ with Standby on most instance families
Domain access policy100 KiB

Limits and quotas checked 2026-11-11; many can be raised through Service Quotas.

Decisions

Situations and verdicts

SituationChoiceVerdict
Website product search for 200,000 productsOpenSearch indexed hourly from the gold product tablePursue with care: the right tool for full-text search, owned by the app team, with no personal data indexed.
Retrieval for an internal assistant over quality reportsOpenSearch vector searchDrop: use Databricks Vector Search so access and lineage stay in Unity Catalog.
Existing OpenSearch cluster for application and security logsKeep itKeep if you have it: it does that job well; do not move platform analytics into it.
Ad hoc analysis of sales by storeOpenSearch aggregationsDrop: use the Databricks SQL warehouse or Athena on the tables.

FAQ

Questions people ask

Is OpenSearch the same as Elasticsearch?

OpenSearch began as a fork of Elasticsearch 7.10 in 2021 and is now governed by the OpenSearch Software Foundation. The APIs are similar but have diverged; Amazon OpenSearch Service also still runs some older Elasticsearch versions.

Does OpenSearch Serverless still have a minimum cost?

Classic collections keep a minimum number of OCUs running. The next generation, generally available since May 2026, can scale compute to zero after ten idle minutes, while storage is still billed.

Should I use OpenSearch or Databricks Vector Search for RAG?

On this platform, Databricks Vector Search, because it syncs from Delta tables and stays governed by Unity Catalog. OpenSearch fits when an application outside Databricks needs hybrid keyword and vector search.

How do I load gold tables into OpenSearch?

Run a scheduled Databricks job that reads the gold table, or only its changes, and bulk-indexes documents into OpenSearch, ideally into a new index switched in with an alias for full rebuilds.

Sources

Checked against

Checked 2026-11-11.

Related

Where it connects

AWS and Databricks reference platform · Amazon CloudWatch · Amazon Bedrock · Amazon Athena · AWS on this site · All AWS data services

← ElastiCache · RDS PostgreSQL →