Home › AWS › AWS data services › Bedrock

AWS data services · ML and AI · about 11 min read

Amazon Bedrock (models, Knowledge Bases, Guardrails, AgentCore)

One AWS API to call many large language models in your account, with retrieval, guardrails and a platform for running agents.

RoleML and AI
StatusGenerally available. Amazon Bedrock Agents (now called Agents Classic) is closed to new customers since 30 Jul 2026; AWS points to Bedrock AgentCore. Models, Knowledge Bases and Guardrails are unaffected. AgentCore has grown many parts since its October 2025 launch; check each part's status in your region.
Verdict on this sitePursue with care
AzureAzure AI Foundry models and Foundry Agent Service
Google CloudVertex AI (Gemini and Model Garden) and Agent Builder
DatabricksFoundation Model APIs, AI Gateway, Vector Search and Mosaic AI Agent Framework
Open sourceOpen-weight models served with vLLM, LangGraph, pgvector
Verdict: Pursue with care. Call Bedrock models from Databricks where the company already uses Bedrock, so models run inside the AWS account and no data goes to a public chat service. Put the model behind one port, keep retrieval over governed tables, and do not start new agents on Agents Classic.

In plain words

What it is

A large language model, or LLM, is a model trained on a huge amount of text that can read and write language: summarise a document, answer a question, pull fields out of a note or draft code. Companies such as Anthropic, Meta, Mistral and Amazon build these models. Amazon Bedrock lets you call many of them through one AWS API, billed on your AWS bill, with requests that stay inside AWS and are not used to train the models.

Around the model call, Bedrock adds several building blocks. Knowledge Bases do retrieval-augmented generation, or RAG: they split your documents into chunks, turn them into vectors (lists of numbers that capture meaning), store them in a vector store and fetch the most relevant chunks for each question so the model answers from your content. Guardrails filter harmful content, block topics, mask personal data and check whether an answer is grounded in the sources. Batch inference runs large jobs at a lower price. AgentCore is a platform for running agents: programs where a model decides which tools to call, in a loop, to finish a task.

A concrete example: a tyre maker receives thousands of warranty claims a month, each with a free-text note from a dealer. A nightly Databricks job sends each tokenised note to a Bedrock model with a prompt that asks for the failure mode, the tyre part and a confidence score, and writes the answers to a silver table. A Guardrail checks every response. Quality engineers then see failure modes by plant and by batch in a dashboard, instead of reading notes by hand.

Names and status change quickly here. The original Bedrock Agents feature was renamed Agents Classic and closed to new customers on 30 July 2026. AgentCore, launched in October 2025, now covers a managed agent loop called Harness, a serverless Runtime, Memory, Gateway for turning APIs into tools, Identity, Policy, Evaluations, Observability and more.

A switchboard in your own building that connects you to many expert writers, with a security desk that checks what goes in and what comes out.

Big picture

Where it fits

SourcesMove inStreamStoreProcessServeUseAcross everything: catalog · security · orchestration · observability and cost · infrastructure as codegets data fromhands data toDatabricks jobs and apps through the model portGold tables and documents for retrievalS3BedrockML and AISilver tables with extracted fieldsAgents and apps that act on answersCloudWatch

The lit lane is where Bedrock works. Boxes on the left feed it; boxes on the right use what it produces.

Bedrock sits in the Use lane, next to BI and ML. On the reference platform its role is narrow and clear: it is one of two allowed homes for models. The rule is that models run inside the platform or the company's AWS account, either Databricks-hosted models or Amazon Bedrock, and no data is sent to a public chat service. Where the company already has a Bedrock agreement, quotas and approved models, Databricks jobs and apps call Bedrock instead of standing up a second model contract.

The model sits behind a port. That means pipelines and agents call a small internal interface such as 'classify this text' or 'draft this rule', and an adapter decides whether Bedrock or a Databricks model serves it. Swapping models or providers is then a configuration change, not a rewrite. Databricks AI Gateway can sit in front of external models, including Bedrock, to add rate limits, usage logging and guardrails in one place.

Retrieval over governed data stays in Databricks: Vector Search indexes built from gold tables, with Unity Catalog grants. Bedrock Knowledge Bases suit document collections that live only on the AWS side. The platform's own agents, which draft manifest entries, rules and documentation as pull requests, and the triage agent that reads failed runs, can use Bedrock models through the same port. Every call is logged with the run id so its cost and output are traceable.

How it works

The ideas you need

IdeaWhat it means
Foundation modelA large model trained on broad data that you use as-is or adapt, such as Claude, Llama, Mistral or Amazon Nova.
Converse APIBedrock's common request format for chat-style calls across models, including tool use.
Inference profileA named route for a model that can spread requests across Regions (cross-Region inference) or track cost per application.
Knowledge BaseManaged retrieval: Bedrock chunks documents from S3 or other sources, embeds them, stores vectors and retrieves relevant chunks.
GuardrailA policy applied to inputs and outputs: content filters, denied topics, word filters, personal-data masking and grounding checks.
Batch inferenceSubmit many requests as a file in S3 and get results back later, at a lower price than on-demand calls.
Provisioned throughputReserved model capacity bought for a term, for steady high volume or for custom models.
AgentCoreA set of services for running agents: Harness, Runtime, Memory, Gateway, Identity, Policy, Evaluations, Observability and more.
Prompt cachingReusing a long, repeated prompt prefix across calls so it is charged and processed at a lower rate.

Choosing

When to use it, and when not

Use it when

  • The company already has Bedrock approved, with model access, quotas and a security review done, and wants Databricks to call it.
  • You must keep model traffic inside the AWS account and Region, with VPC endpoints and IAM controlling who may call which model.
  • You need models Databricks does not host in your region, or a model provider the company has contracted through AWS.
  • You extract fields from large volumes of free text overnight, where batch inference cuts cost.
  • You build an agent that calls AWS-side tools and needs managed sessions, memory, tool gateway and identity through AgentCore.

Avoid it when

  • You plan to build a new agent on Bedrock Agents Classic; it is closed to new customers and gets no new models.
  • Retrieval is over gold tables governed in Unity Catalog; Databricks Vector Search keeps the same grants and lineage.
  • You would send untokenised personal data or secret recipes in prompts because it is convenient.
  • A rule, a lookup table or a regular expression solves the problem; a model adds cost and uncertainty for no gain.

Production grade

What goes wrong in production

ProblemWhat breaksWhat to do
ThrottlingA nightly batch of ten thousand calls hits the tokens-per-minute quota and fails half way, with ThrottlingException errors.Use batch inference for big jobs, retry with backoff, request quota increases ahead of time, and use cross-Region inference profiles where data rules allow.
Cost per run unknownModel calls are billed by tokens, and nobody can say which pipeline or agent spent what.Use application inference profiles or tags per data product, log tokens per call with the run id, and join with the cost view.
Wrong or made-up answersThe model invents a failure mode that is not in the note, and it lands in silver as if it were fact.Constrain output to an allowed list, ask for evidence spans, use Guardrails grounding checks, sample results for human review, and keep a confidence column.
Personal data in promptsA dealer's note contains a customer's phone number and it is sent to the model.Tokenise at the door before bronze, mask with Guardrails' personal-data filter as a second line, and log only prompt ids, not prompt text.
Model versions retireA model version reaches end of life and calls start failing, or a new version changes answers silently.Pin model ids in config, watch Bedrock's model lifecycle notices, and re-run an evaluation set before switching versions.
Agents Classic in maintenanceTeams copy old tutorials that use Bedrock Agents, which new accounts can no longer create.Use the AgentCore Harness for declared agents, or your own loop on AgentCore Runtime; AWS offers a migration path for existing agents.
Two retrieval stacksSome answers come from a Bedrock Knowledge Base and others from Databricks Vector Search, with different freshness and grants.Decide by where the content lives: governed tables in Vector Search, AWS-only document sets in Knowledge Bases, both behind the same retrieval port.

Alternatives

What to compare it with

AlternativePick the alternative whenPick Bedrock when
Databricks Foundation Model APIs and AI GatewayYou want models next to the data, governed in Unity Catalog, with usage logged in system tables.The company standard and contract are on Bedrock, or the model you need is only on Bedrock.
Databricks Vector SearchRetrieval is over gold tables and must follow Unity Catalog grants.Documents live only in S3 or SharePoint and the app is AWS-side.
Azure AI FoundryYou are on Azure.You are on AWS.
Google Vertex AIYou are on Google Cloud or need Gemini natively.You are on AWS and want one bill and IAM.
Self-hosted open-weight models with vLLM on EKS or SageMakerYou need full control of the model, custom weights or steady very high volume.You do not want to run GPU fleets and patch serving stacks.
Provider API directly, such as the Claude APIYou want the newest features first and have approved that provider directly.Data must stay within the AWS account boundary and Region.

Cost

How you pay

Model calls on demand are charged per input and output token, at a different price for each model; images and other media have their own units. Batch inference is charged at a lower rate per token than on-demand. Provisioned throughput is charged per model unit per hour for a term. Knowledge Bases charge for embedding calls and for the vector store you choose. Guardrails charge per text unit checked by each policy. AgentCore services charge by consumption, such as runtime seconds, memory events and gateway calls. Prices vary by model and Region, so check the Bedrock and AgentCore pricing pages.

What drives the bill

  • Tokens per call, mostly long prompts with many retrieved chunks.
  • Choice of model, since large models cost many times more per token than small ones.
  • Number of calls, including retries and agent loops that call the model many times per task.
  • Vector store size and query rate for Knowledge Bases.
  • Guardrail policies applied to every input and output.

Cost traps

  • Agent loops that run many steps per question multiply token cost without anyone noticing.
  • Using the largest model for simple extraction that a small model does as well.
  • On-demand calls for an overnight backlog that batch inference would do for less.
  • A vector store with an always-on minimum charge left running for a pilot.

Prices change; check the AWS pricing page for your region before deciding.

Security and governance

Keeping it safe

  • Call Bedrock through a VPC interface endpoint so traffic stays private, and grant InvokeModel only on approved model ids per role.
  • Use service control policies to block models or Regions the company has not approved.
  • Tokenise personal data before it reaches bronze, and add Guardrails' personal-data filters as a second line for free text.
  • Turn on model invocation logging to an encrypted S3 bucket or CloudWatch Logs only where policy allows, because logs contain prompts and answers.
  • Encrypt Knowledge Base data sources and vector stores with customer managed KMS keys, and limit which roles can query each base.
  • CloudTrail records Bedrock management calls; join model access with the platform's access review.

Operating it

What to watch, and the limits

WatchWhy
Invocations and InvocationThrottlesCalls and throttled calls per model; throttles show you are near quota.
InputTokenCount and OutputTokenCountTokens per model, which drive cost.
InvocationLatencyTime per call; long tails slow batch pipelines and agents.
InvocationClientErrors and InvocationServerErrorsBad requests and service-side failures.
Guardrail interventionsHow often a guardrail blocked or masked content, by policy.
Answer quality on an evaluation setScores from a fixed set of test notes, re-run before any model or prompt change.
LimitValue and notes
Tokens and requests per minuteSet per model and Region in Service Quotas; many can be raised on request
Bedrock Agents ClassicCreateAgent and InvokeInlineAgent unavailable to accounts without use in the prior 12 months, since 30 July 2026
Agents Classic modelsModel catalogue frozen at 30 July 2026; new models go to AgentCore and Bedrock inference
Model availabilityEach model is offered only in some Regions; cross-Region inference may route to other Regions in the same geography

Limits and quotas checked 2026-11-11; many can be raised through Service Quotas.

Decisions

Situations and verdicts

SituationChoiceVerdict
Extract failure modes from warranty notes overnightBedrock batch inference called from a Databricks job through the model port, Guardrail on outputs, results to silverPursue with care: good value, but needs tokenised input, fixed output lists and an evaluation set.
Chat over gold sales metrics for managersGenie over metric views in DatabricksDrop Bedrock here: the governed semantic layer and grants are in Databricks.
New support agent that calls order and stock APIsAgentCore Harness or Runtime with Gateway tools and Identity, model behind the portPursue with care: AgentCore is the current AWS path; keep the agent loop portable.
Existing Bedrock Agents Classic agent in productionKeep it running, plan migration to AgentCore when it next needs changesKeep if you have it: it works, but gets no new features or models.

FAQ

Questions people ask

Is Amazon Bedrock Agents deprecated?

It was renamed Bedrock Agents Classic and closed to new customers on 30 July 2026. Accounts that used it in the prior 12 months keep access. AWS recommends Bedrock AgentCore for new agents; Bedrock models, Knowledge Bases and Guardrails are unaffected.

Can Databricks call Bedrock models?

Yes. A Databricks job or app can call Bedrock with the AWS SDK using an IAM role, or register Bedrock as an external model behind Databricks AI Gateway so calls are rate-limited and logged in one place.

Does Bedrock use my prompts to train models?

AWS states that Bedrock does not use your prompts and outputs to train the base models or share them with model providers. Check the current data privacy terms and any logging you turn on yourself.

Bedrock Knowledge Bases or Databricks Vector Search?

Use Vector Search when the content is in Unity Catalog tables and must follow the same grants. Use Knowledge Bases for document sets that live on the AWS side and serve AWS-side apps.

How do I stop Bedrock throttling errors?

Use batch inference for big jobs, add retries with backoff, request quota increases, and consider cross-Region inference profiles if your data rules allow the traffic to move between Regions.

Sources

Checked against

Checked 2026-11-11.

Related

Where it connects

AWS and Databricks reference platform · Amazon SageMaker AI · Amazon Comprehend · AWS PrivateLink · Platforms: AWS · All AWS data services

← Quick Sight · Comprehend →