Home › AWS › AWS data services › Bedrock
AWS data services · ML and AI · about 11 min read
Amazon Bedrock (models, Knowledge Bases, Guardrails, AgentCore)
One AWS API to call many large language models in your account, with retrieval, guardrails and a platform for running agents.
In plain words
What it is
A large language model, or LLM, is a model trained on a huge amount of text that can read and write language: summarise a document, answer a question, pull fields out of a note or draft code. Companies such as Anthropic, Meta, Mistral and Amazon build these models. Amazon Bedrock lets you call many of them through one AWS API, billed on your AWS bill, with requests that stay inside AWS and are not used to train the models.
Around the model call, Bedrock adds several building blocks. Knowledge Bases do retrieval-augmented generation, or RAG: they split your documents into chunks, turn them into vectors (lists of numbers that capture meaning), store them in a vector store and fetch the most relevant chunks for each question so the model answers from your content. Guardrails filter harmful content, block topics, mask personal data and check whether an answer is grounded in the sources. Batch inference runs large jobs at a lower price. AgentCore is a platform for running agents: programs where a model decides which tools to call, in a loop, to finish a task.
A concrete example: a tyre maker receives thousands of warranty claims a month, each with a free-text note from a dealer. A nightly Databricks job sends each tokenised note to a Bedrock model with a prompt that asks for the failure mode, the tyre part and a confidence score, and writes the answers to a silver table. A Guardrail checks every response. Quality engineers then see failure modes by plant and by batch in a dashboard, instead of reading notes by hand.
Names and status change quickly here. The original Bedrock Agents feature was renamed Agents Classic and closed to new customers on 30 July 2026. AgentCore, launched in October 2025, now covers a managed agent loop called Harness, a serverless Runtime, Memory, Gateway for turning APIs into tools, Identity, Policy, Evaluations, Observability and more.
A switchboard in your own building that connects you to many expert writers, with a security desk that checks what goes in and what comes out.
Big picture
Where it fits
The lit lane is where Bedrock works. Boxes on the left feed it; boxes on the right use what it produces.
Bedrock sits in the Use lane, next to BI and ML. On the reference platform its role is narrow and clear: it is one of two allowed homes for models. The rule is that models run inside the platform or the company's AWS account, either Databricks-hosted models or Amazon Bedrock, and no data is sent to a public chat service. Where the company already has a Bedrock agreement, quotas and approved models, Databricks jobs and apps call Bedrock instead of standing up a second model contract.
The model sits behind a port. That means pipelines and agents call a small internal interface such as 'classify this text' or 'draft this rule', and an adapter decides whether Bedrock or a Databricks model serves it. Swapping models or providers is then a configuration change, not a rewrite. Databricks AI Gateway can sit in front of external models, including Bedrock, to add rate limits, usage logging and guardrails in one place.
Retrieval over governed data stays in Databricks: Vector Search indexes built from gold tables, with Unity Catalog grants. Bedrock Knowledge Bases suit document collections that live only on the AWS side. The platform's own agents, which draft manifest entries, rules and documentation as pull requests, and the triage agent that reads failed runs, can use Bedrock models through the same port. Every call is logged with the run id so its cost and output are traceable.
How it works
The ideas you need
| Idea | What it means |
|---|---|
| Foundation model | A large model trained on broad data that you use as-is or adapt, such as Claude, Llama, Mistral or Amazon Nova. |
| Converse API | Bedrock's common request format for chat-style calls across models, including tool use. |
| Inference profile | A named route for a model that can spread requests across Regions (cross-Region inference) or track cost per application. |
| Knowledge Base | Managed retrieval: Bedrock chunks documents from S3 or other sources, embeds them, stores vectors and retrieves relevant chunks. |
| Guardrail | A policy applied to inputs and outputs: content filters, denied topics, word filters, personal-data masking and grounding checks. |
| Batch inference | Submit many requests as a file in S3 and get results back later, at a lower price than on-demand calls. |
| Provisioned throughput | Reserved model capacity bought for a term, for steady high volume or for custom models. |
| AgentCore | A set of services for running agents: Harness, Runtime, Memory, Gateway, Identity, Policy, Evaluations, Observability and more. |
| Prompt caching | Reusing a long, repeated prompt prefix across calls so it is charged and processed at a lower rate. |
Choosing
When to use it, and when not
Use it when
- The company already has Bedrock approved, with model access, quotas and a security review done, and wants Databricks to call it.
- You must keep model traffic inside the AWS account and Region, with VPC endpoints and IAM controlling who may call which model.
- You need models Databricks does not host in your region, or a model provider the company has contracted through AWS.
- You extract fields from large volumes of free text overnight, where batch inference cuts cost.
- You build an agent that calls AWS-side tools and needs managed sessions, memory, tool gateway and identity through AgentCore.
Avoid it when
- You plan to build a new agent on Bedrock Agents Classic; it is closed to new customers and gets no new models.
- Retrieval is over gold tables governed in Unity Catalog; Databricks Vector Search keeps the same grants and lineage.
- You would send untokenised personal data or secret recipes in prompts because it is convenient.
- A rule, a lookup table or a regular expression solves the problem; a model adds cost and uncertainty for no gain.
Production grade
What goes wrong in production
| Problem | What breaks | What to do |
|---|---|---|
| Throttling | A nightly batch of ten thousand calls hits the tokens-per-minute quota and fails half way, with ThrottlingException errors. | Use batch inference for big jobs, retry with backoff, request quota increases ahead of time, and use cross-Region inference profiles where data rules allow. |
| Cost per run unknown | Model calls are billed by tokens, and nobody can say which pipeline or agent spent what. | Use application inference profiles or tags per data product, log tokens per call with the run id, and join with the cost view. |
| Wrong or made-up answers | The model invents a failure mode that is not in the note, and it lands in silver as if it were fact. | Constrain output to an allowed list, ask for evidence spans, use Guardrails grounding checks, sample results for human review, and keep a confidence column. |
| Personal data in prompts | A dealer's note contains a customer's phone number and it is sent to the model. | Tokenise at the door before bronze, mask with Guardrails' personal-data filter as a second line, and log only prompt ids, not prompt text. |
| Model versions retire | A model version reaches end of life and calls start failing, or a new version changes answers silently. | Pin model ids in config, watch Bedrock's model lifecycle notices, and re-run an evaluation set before switching versions. |
| Agents Classic in maintenance | Teams copy old tutorials that use Bedrock Agents, which new accounts can no longer create. | Use the AgentCore Harness for declared agents, or your own loop on AgentCore Runtime; AWS offers a migration path for existing agents. |
| Two retrieval stacks | Some answers come from a Bedrock Knowledge Base and others from Databricks Vector Search, with different freshness and grants. | Decide by where the content lives: governed tables in Vector Search, AWS-only document sets in Knowledge Bases, both behind the same retrieval port. |
Alternatives
What to compare it with
| Alternative | Pick the alternative when | Pick Bedrock when |
|---|---|---|
| Databricks Foundation Model APIs and AI Gateway | You want models next to the data, governed in Unity Catalog, with usage logged in system tables. | The company standard and contract are on Bedrock, or the model you need is only on Bedrock. |
| Databricks Vector Search | Retrieval is over gold tables and must follow Unity Catalog grants. | Documents live only in S3 or SharePoint and the app is AWS-side. |
| Azure AI Foundry | You are on Azure. | You are on AWS. |
| Google Vertex AI | You are on Google Cloud or need Gemini natively. | You are on AWS and want one bill and IAM. |
| Self-hosted open-weight models with vLLM on EKS or SageMaker | You need full control of the model, custom weights or steady very high volume. | You do not want to run GPU fleets and patch serving stacks. |
| Provider API directly, such as the Claude API | You want the newest features first and have approved that provider directly. | Data must stay within the AWS account boundary and Region. |
Cost
How you pay
Model calls on demand are charged per input and output token, at a different price for each model; images and other media have their own units. Batch inference is charged at a lower rate per token than on-demand. Provisioned throughput is charged per model unit per hour for a term. Knowledge Bases charge for embedding calls and for the vector store you choose. Guardrails charge per text unit checked by each policy. AgentCore services charge by consumption, such as runtime seconds, memory events and gateway calls. Prices vary by model and Region, so check the Bedrock and AgentCore pricing pages.
What drives the bill
- Tokens per call, mostly long prompts with many retrieved chunks.
- Choice of model, since large models cost many times more per token than small ones.
- Number of calls, including retries and agent loops that call the model many times per task.
- Vector store size and query rate for Knowledge Bases.
- Guardrail policies applied to every input and output.
Cost traps
- Agent loops that run many steps per question multiply token cost without anyone noticing.
- Using the largest model for simple extraction that a small model does as well.
- On-demand calls for an overnight backlog that batch inference would do for less.
- A vector store with an always-on minimum charge left running for a pilot.
Prices change; check the AWS pricing page for your region before deciding.
Security and governance
Keeping it safe
- Call Bedrock through a VPC interface endpoint so traffic stays private, and grant InvokeModel only on approved model ids per role.
- Use service control policies to block models or Regions the company has not approved.
- Tokenise personal data before it reaches bronze, and add Guardrails' personal-data filters as a second line for free text.
- Turn on model invocation logging to an encrypted S3 bucket or CloudWatch Logs only where policy allows, because logs contain prompts and answers.
- Encrypt Knowledge Base data sources and vector stores with customer managed KMS keys, and limit which roles can query each base.
- CloudTrail records Bedrock management calls; join model access with the platform's access review.
Operating it
What to watch, and the limits
| Watch | Why |
|---|---|
| Invocations and InvocationThrottles | Calls and throttled calls per model; throttles show you are near quota. |
| InputTokenCount and OutputTokenCount | Tokens per model, which drive cost. |
| InvocationLatency | Time per call; long tails slow batch pipelines and agents. |
| InvocationClientErrors and InvocationServerErrors | Bad requests and service-side failures. |
| Guardrail interventions | How often a guardrail blocked or masked content, by policy. |
| Answer quality on an evaluation set | Scores from a fixed set of test notes, re-run before any model or prompt change. |
| Limit | Value and notes |
|---|---|
| Tokens and requests per minute | Set per model and Region in Service Quotas; many can be raised on request |
| Bedrock Agents Classic | CreateAgent and InvokeInlineAgent unavailable to accounts without use in the prior 12 months, since 30 July 2026 |
| Agents Classic models | Model catalogue frozen at 30 July 2026; new models go to AgentCore and Bedrock inference |
| Model availability | Each model is offered only in some Regions; cross-Region inference may route to other Regions in the same geography |
Limits and quotas checked 2026-11-11; many can be raised through Service Quotas.
Decisions
Situations and verdicts
| Situation | Choice | Verdict |
|---|---|---|
| Extract failure modes from warranty notes overnight | Bedrock batch inference called from a Databricks job through the model port, Guardrail on outputs, results to silver | Pursue with care: good value, but needs tokenised input, fixed output lists and an evaluation set. |
| Chat over gold sales metrics for managers | Genie over metric views in Databricks | Drop Bedrock here: the governed semantic layer and grants are in Databricks. |
| New support agent that calls order and stock APIs | AgentCore Harness or Runtime with Gateway tools and Identity, model behind the port | Pursue with care: AgentCore is the current AWS path; keep the agent loop portable. |
| Existing Bedrock Agents Classic agent in production | Keep it running, plan migration to AgentCore when it next needs changes | Keep if you have it: it works, but gets no new features or models. |
FAQ
Questions people ask
Is Amazon Bedrock Agents deprecated?
It was renamed Bedrock Agents Classic and closed to new customers on 30 July 2026. Accounts that used it in the prior 12 months keep access. AWS recommends Bedrock AgentCore for new agents; Bedrock models, Knowledge Bases and Guardrails are unaffected.
Can Databricks call Bedrock models?
Yes. A Databricks job or app can call Bedrock with the AWS SDK using an IAM role, or register Bedrock as an external model behind Databricks AI Gateway so calls are rate-limited and logged in one place.
Does Bedrock use my prompts to train models?
AWS states that Bedrock does not use your prompts and outputs to train the base models or share them with model providers. Check the current data privacy terms and any logging you turn on yourself.
Bedrock Knowledge Bases or Databricks Vector Search?
Use Vector Search when the content is in Unity Catalog tables and must follow the same grants. Use Knowledge Bases for document sets that live on the AWS side and serve AWS-side apps.
How do I stop Bedrock throttling errors?
Use batch inference for big jobs, add retries with backoff, request quota increases, and consider cross-Region inference profiles if your data rules allow the traffic to move between Regions.
Sources
Checked against
- AWS docs: What is Amazon Bedrock
- AWS docs: Bedrock Agents Classic maintenance mode
- AWS docs: What is Amazon Bedrock AgentCore
- AWS docs: Guardrails for Amazon Bedrock
- AWS: Bedrock pricing
Checked 2026-11-11.
Related
Where it connects
AWS and Databricks reference platform · Amazon SageMaker AI · Amazon Comprehend · AWS PrivateLink · Platforms: AWS · All AWS data services
← Quick Sight · Comprehend →