AWS Bedrock Explained: How to Build AI Apps with Amazon's Foundation Models

I have built production AI applications on AWS Bedrock for federal clients in GovCloud — this is a practitioner guide, not a marketing overview. If you are building AI applications on AWS, you have a choice: wire together your own model hosting infrastructure, navigate separate API keys for every AI provider, and manage embedding pipelines, vector stores, and safety filters yourself — or use AWS Bedrock and let Amazon handle all of it.

Bedrock is Amazon's answer to the enterprise AI infrastructure problem. It gives you serverless, pay-per-token access to the most capable foundation models on the market — Claude from Anthropic, Llama from Meta, Titan from Amazon, Mistral, and Stability AI image models — all through a single, unified API that integrates natively with IAM, CloudWatch, VPC endpoints, and every other AWS service you already use.

This guide covers everything you need to know to actually build with it: the architecture, the pricing model, the managed services built on top of it, code examples, and a clear framework for deciding when Bedrock is the right choice versus self-hosting or the OpenAI API.

What Is AWS Bedrock?

AWS Bedrock is a fully managed foundation model service. The core value proposition is simple: you get API access to powerful AI models without managing any servers, GPU clusters, or model weights. You do not deploy anything. You do not configure CUDA drivers. You call an API and pay for what you use.

What separates Bedrock from simply calling the Anthropic API or OpenAI API directly is the integration layer. Bedrock is deeply embedded in the AWS ecosystem:

• IAM authentication — No API keys floating around in environment variables. Access is controlled by AWS IAM roles and policies, the same way you control access to S3, DynamoDB, or any other AWS service.
• VPC endpoint support — Traffic never leaves AWS. You can call Bedrock from a private subnet with no internet gateway.
• CloudWatch integration — Logging, metrics, and tracing for every model invocation, built in.
• Managed higher-order services — Knowledge Bases (RAG), Agents (tool use and orchestration), and Guardrails (safety and content controls) are all first-party services built on top of the base model access.

The result is a platform where you can move from "I want an AI feature" to "this AI feature is running in production with logging, access controls, content filtering, and a RAG pipeline" significantly faster than assembling those components yourself.

Models Available in Bedrock

AWS Bedrock gives you access to six model families — Anthropic Claude, Meta Llama, Amazon Titan, Mistral, Cohere, and Stability AI — through a single unified API, all within the AWS compliance boundary, with no separate accounts or API keys per provider required. The lineup changes as new models are released, but the major families as of early 2026 are:

Anthropic — Claude

The Claude model family is one of Bedrock's most capable offerings. Claude 3.5 Sonnet and Claude 3.5 Haiku are available for production workloads, with Claude offering strong performance on reasoning, long-context tasks, coding, and instruction following. Claude 3 Opus remains available for the most demanding tasks. Anthropic's models are particularly strong for enterprise applications requiring nuanced instruction following and safety-conscious outputs.

Meta — Llama

Llama 3.1 and Llama 3.2 variants are available, including the 8B, 70B, and 405B parameter versions. Llama is the leading open-weights model family and gives you a strong open-source option within the managed Bedrock environment — useful when you want model transparency or need to demonstrate that you are not sending data to a proprietary provider.

Amazon — Titan

Amazon's own Titan Text models (Express, Lite, Premier) cover general text generation and summarization. Titan Embeddings (v1 and v2) are the recommended embedding models for Bedrock Knowledge Bases. Amazon also offers Titan Image Generator for image synthesis tasks.

Mistral AI

Mistral Large, Mistral Small, and Mixtral 8x7B are available. Mistral's models are known for strong performance per dollar and European data residency provenance — useful for workloads with EU compliance requirements.

Stability AI

Stable Diffusion XL and Stable Image Core for image generation. Available through the same InvokeModel API as text models.

Bedrock vs. OpenAI API vs. Azure OpenAI

All three platforms give you API access to large language models. The differences matter most at the enterprise and government level, where compliance, multi-model flexibility, and infrastructure integration drive decisions.

The headline difference: if you are already on AWS, Bedrock is almost always the right foundation. If you are a small team building a consumer-facing product and speed to market is everything, OpenAI's API has the fastest onboarding path. If your organization is Microsoft-first and needs FedRAMP compliance with GPT-4o specifically, Azure OpenAI is the answer.

Bedrock Architecture: APIs and Core Primitives

Bedrock exposes two primary invocation APIs plus the higher-order services built on top of them.

InvokeModel

The InvokeModel API is the low-level primitive. You send a raw request body that matches the model provider's schema and get back a raw response. The request body format varies by model family — Anthropic Claude uses the Messages API format, while Amazon Titan uses its own schema. This is the most flexible option but requires you to handle model-specific formatting.

import boto3
import json

client = boto3.client("bedrock-runtime", region_name="us-east-1")

body = json.dumps({
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 1024,
    "messages": [
        {
            "role": "user",
            "content": "Summarize the key risks in this contract."
        }
    ]
})

response = client.invoke_model(
    modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
    body=body,
    contentType="application/json",
    accept="application/json"
)

result = json.loads(response["body"].read())
print(result["content"][0]["text"])

Converse API

The Converse API is Bedrock's unified, model-agnostic interface introduced in 2024. Instead of formatting requests differently for each model, you use a single consistent schema and Bedrock translates it to whatever the underlying model expects. This is the recommended API for most new applications because it makes model switching trivially easy — swap the modelId and nothing else changes.

import boto3

client = boto3.client("bedrock-runtime", region_name="us-east-1")

response = client.converse(
    modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
    # Swap to "meta.llama3-70b-instruct-v1:0" — same code, different model
    messages=[
        {
            "role": "user",
            "content": [{"text": "What are the main use cases for AWS Bedrock?"}]
        }
    ],
    inferenceConfig={
        "maxTokens": 1024,
        "temperature": 0.3
    }
)

output = response["output"]["message"]["content"][0]["text"]
print(output)

For streaming responses — useful for chat interfaces where you want text to appear progressively — use converse_stream instead of converse. The interface is identical; Bedrock handles the server-sent event stream.

Knowledge Bases: Managed RAG Without the Pipeline

Knowledge Bases for Bedrock is AWS's fully managed RAG service — you point it at an S3 bucket, choose an embedding model, and Bedrock handles chunking, embedding, vector storage, and retrieval automatically, eliminating the need to build and maintain a custom pipeline. Retrieval-Augmented Generation (RAG) is the most common pattern for building AI applications over proprietary data — internal documents, policy manuals, product catalogs, case files. The standard self-built RAG pipeline involves: chunking documents, running them through an embedding model, storing vectors in a vector database, retrieving relevant chunks at query time, and injecting them into the model prompt.

That pipeline takes real engineering effort to build and maintain. Knowledge Bases for Bedrock does all of it for you, managed and serverless.

How It Works

Agents for Bedrock: Autonomous AI with Tool Use

An Agent for Bedrock is an AI system that can take multi-step actions by calling external APIs and AWS services autonomously, based on a user goal. Where a standard model invocation answers a question, an Agent breaks down a task, decides what tools to use, calls those tools, observes the results, and continues until the task is complete.

You define an Agent by specifying:

• A foundation model — which model drives the reasoning (Claude is the most common choice)
• An instruction set — a system-level prompt describing the agent's role and behavior
• Action groups — collections of API operations the agent can call, defined as OpenAPI schemas backed by Lambda functions
• Knowledge Base associations — optional; gives the agent access to a document corpus for retrieval

A real example: an HR document agent that can look up an employee record from an API, retrieve the relevant policy from a Knowledge Base, and generate a personalized response. The agent reasons about which actions to take, calls the Lambda behind your HR API, fetches the policy document, and synthesizes a complete answer — without you writing any orchestration logic.

Guardrails: Content Filtering and Safety Controls

Guardrails for Bedrock is a configurable safety layer that sits between your application and any model on Bedrock. It applies on both the input (user prompt) and the output (model response), and it works with every model on the platform — not just Amazon's own Titan models.

Guardrails give you four main control surfaces:

• Content filters — Block or flag content across categories: hate speech, violence, sexual content, misconduct, and prompt injection attacks. Each category has adjustable sensitivity thresholds (LOW, MEDIUM, HIGH).
• Topic denial — Block specific topics by natural language description. Example: "Do not discuss competitor products." The model will redirect or decline any prompt that falls into that topic.
• PII detection and redaction — Automatically detect and redact personally identifiable information (names, SSNs, email addresses, phone numbers, credit card numbers) from both inputs and outputs.
• Grounding checks — For RAG applications, Guardrails can verify that the model's response is factually grounded in the retrieved source documents, reducing hallucinations.

AWS Bedrock Pricing: On-Demand vs. Provisioned Throughput

Bedrock offers two pricing modes. Understanding them matters for cost management at scale.

On-Demand Pricing

You pay per token — input tokens and output tokens are priced separately. There is no minimum commitment. This is ideal for applications with variable or unpredictable traffic. Pricing varies significantly by model: as of early 2026, Claude 3.5 Haiku is roughly $0.80 per million input tokens and $4 per million output tokens, while Claude 3.5 Sonnet is roughly $3 per million input tokens and $15 per million output tokens. Llama 3.1 70B is cheaper at roughly $0.99 per million input/output tokens. Exact pricing changes — check the AWS Bedrock pricing page for current rates.

Provisioned Throughput

Provisioned Throughput lets you purchase a guaranteed number of model units (MUs) — essentially reserved capacity. You pay a fixed hourly rate regardless of whether you use the capacity. This is appropriate when you have:

• Consistent, high-volume workloads where you need predictable latency
• Compliance or SLA requirements that on-demand's best-effort model cannot guarantee
• Custom fine-tuned models (fine-tuned models on Bedrock require Provisioned Throughput — they cannot run on-demand)

For most teams building internal tools, prototypes, or moderate-traffic applications, on-demand is the right starting point. Revisit Provisioned Throughput when you have real usage data and can quantify whether the reserved capacity math works in your favor.

Setting Up Bedrock: IAM, boto3, and SDK Basics

Getting your first Bedrock call working takes about 15 minutes. Here is the setup sequence:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "bedrock:InvokeModel",
        "bedrock:InvokeModelWithResponseStream",
        "bedrock:Converse",
        "bedrock:ConverseStream"
      ],
      "Resource": [
        "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0",
        "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-haiku-20241022-v1:0"
      ]
    }
  ]
}

Building a Document Q&A System: Conceptual Walkthrough

A production document Q&A system on Bedrock requires four AWS services: S3 for document storage, a Bedrock Knowledge Base for managed RAG, Lambda for the API handler, and API Gateway for the HTTP endpoint — with optional Guardrails for PII redaction. You can have this architecture running in under a day. The most common enterprise AI application is exactly this: "Ask questions about our policy manual," "Search the contract library," "Query our internal knowledge base." Here is how you build one on Bedrock end-to-end.

Architecture

• S3 bucket — Store your documents (PDFs, Word files, text files)
• Bedrock Knowledge Base — Automated chunking, embedding (Titan Embeddings v2), and vector storage (OpenSearch Serverless managed by Bedrock)
• Lambda function — API handler that receives user questions and calls the Knowledge Base
• API Gateway — HTTP endpoint for your front-end or integration layer
• Guardrails — PII redaction on both input and output, content filtering enabled

The Query Flow

When a user submits a question, your Lambda calls the RetrieveAndGenerate API with the user's question and the Knowledge Base ID. Bedrock automatically embeds the question, searches the vector store for the top-K relevant chunks, injects those chunks into a prompt, calls the generation model (your choice — Claude is common), and returns the answer along with citations showing which source documents were used.

import boto3

agent_client = boto3.client("bedrock-agent-runtime", region_name="us-east-1")

response = agent_client.retrieve_and_generate(
    input={"text": "What is the company's remote work policy for international travel?"},
    retrieveAndGenerateConfiguration={
        "type": "KNOWLEDGE_BASE",
        "knowledgeBaseConfiguration": {
            "knowledgeBaseId": "YOUR_KB_ID",
            "modelArn": "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-5-sonnet-20241022-v2:0",
            "retrievalConfiguration": {
                "vectorSearchConfiguration": {"numberOfResults": 5}
            }
        }
    }
)

answer = response["output"]["text"]
citations = response["citations"]  # Source documents used
print(answer)

The citations object contains the exact text passages retrieved, the S3 URIs of the source files, and relevance scores. This is critical for enterprise applications where users need to verify answers against original documents — you can surface "This answer was drawn from PolicyManual_v3.pdf, pages 12-14" directly in your UI.

Bedrock for Government: FedRAMP and GovCloud

AWS Bedrock holds FedRAMP High authorization and is available in AWS GovCloud (US-East) and GovCloud (US-West). This makes it one of the few managed foundation model services that federal agencies can use for workloads involving sensitive, mission-critical, or Controlled Unclassified Information (CUI) data.

GovCloud Considerations

Running Bedrock in GovCloud means your model invocation traffic, request/response data, and any stored embeddings stay physically within AWS infrastructure restricted to U.S. persons. Key operational differences from commercial regions:

• Model availability in GovCloud lags the commercial regions — not every model available in us-east-1 is available in GovCloud. Check the current GovCloud Bedrock model list before designing your architecture.
• GovCloud requires a separate AWS account linked to your commercial account. Management console access and IAM are separate from your commercial environment.
• Knowledge Bases and Agents for Bedrock are available in GovCloud but may have feature parity delays versus commercial regions.

Authority to Operate (ATO)

FedRAMP authorization for Bedrock covers AWS's responsibility layer — the infrastructure, the service endpoints, the encryption at rest and in transit. Your application running on top of Bedrock still requires its own ATO from the relevant agency. The AWS FedRAMP package simplifies the ATO process significantly by providing pre-authorized controls you can inherit, but it does not eliminate the agency authorization requirement for your application.

When to Use Bedrock vs. Self-Hosting vs. OpenAI

Use Bedrock if you are already on AWS or need FedRAMP compliance. Use the OpenAI API directly if speed of onboarding is the priority and compliance is not a factor. Self-host only if you have air-gap requirements or the GPU infrastructure to run models efficiently at scale. The choice depends on your compliance requirements, infrastructure preferences, team skills, and the specific application you are building.

Use Bedrock When:

• You are already on AWS and want infrastructure consistency — IAM, VPC, CloudWatch, billing all in one place
• You need FedRAMP authorization or GovCloud for federal or regulated industry workloads
• You want multi-model flexibility without managing separate provider relationships
• You want managed RAG (Knowledge Bases) or managed Agents without building the orchestration layer yourself
• You need enterprise-grade content safety controls (Guardrails) that apply consistently across models
• You are building for an organization that requires no data to leave AWS infrastructure

Use the OpenAI API (Direct) When:

• You need the fastest possible onboarding — OpenAI's API requires an API key and nothing else
• You are building a consumer product where GPT-4o or o1 is a specific functional requirement
• You do not have existing AWS infrastructure and the setup overhead of Bedrock is not justified
• Your use case is low-sensitivity and compliance overhead is pure cost with no benefit

Self-Host Models When:

• You need air-gapped deployment — models running on hardware that has never touched the internet
• You have specific data sovereignty requirements that prohibit any data leaving your own infrastructure
• You need to fine-tune heavily and run custom model variants at scale (though Bedrock supports fine-tuning with Provisioned Throughput)
• You have the GPU infrastructure and MLOps expertise to manage model serving efficiently

For most enterprise teams building internal AI tools on AWS, Bedrock is the path of least resistance to production-quality AI with enterprise-grade compliance. The managed RAG, Agents, and Guardrails alone save months of engineering compared to building equivalent functionality yourself.

The bottom line: AWS Bedrock is the right foundation model platform for teams already on AWS — it trades the simplicity of a single-provider API key for enterprise-grade IAM integration, FedRAMP High authorization, multi-model flexibility, and managed services (Knowledge Bases, Agents, Guardrails) that would take months to build yourself. If you are building AI applications in a regulated or AWS-native environment, Bedrock eliminates more infrastructure work than any other option on the market.

Frequently Asked Questions

Do I need to manage servers to use AWS Bedrock?

No. Bedrock is fully serverless. You call an API endpoint and pay per token. There are no EC2 instances to manage, no GPU clusters to configure, no Docker containers to deploy for the base model access. If you use Agents for Bedrock with Lambda-backed action groups, you do manage those Lambda functions — but the model inference layer itself is entirely managed by AWS.

Can I use my own fine-tuned model on Bedrock?

Yes. Bedrock supports fine-tuning for select model families (Amazon Titan, Llama, Cohere Command) using your own training data stored in S3. Fine-tuned models must be deployed on Provisioned Throughput — they cannot run on the on-demand tier. You can also import custom model weights (for supported architectures) if you have fine-tuned externally and want to run them through the Bedrock interface.

How does Bedrock handle data privacy?

AWS commits that data sent to Bedrock for inference is not used to train the underlying foundation models. Your prompts and completions are not shared with model providers (Anthropic, Meta, etc.). Data is encrypted in transit with TLS and at rest. For GovCloud deployments, data stays within the GovCloud boundary. AWS's data processing addendum for Bedrock covers these commitments in detail — review it with your legal team for regulated industry use cases.

Is Bedrock supported in the AWS CDK and Terraform?

Yes. AWS CDK has L1 constructs for Bedrock resources (Knowledge Bases, Agents, Guardrails) and the community has contributed higher-level L2 constructs. The official AWS CDK Bedrock alpha module provides production-ready constructs for common patterns. Terraform's AWS provider includes Bedrock resources for infrastructure-as-code deployments. Both are actively maintained and generally stable for the core service features.

Note: AWS Bedrock pricing, model availability, and feature set change frequently. Verify current details at aws.amazon.com/bedrock before architectural decisions. Code examples reflect boto3 SDK patterns as of early 2026 and should be validated against the current SDK documentation.

Sources: AWS Documentation, Gartner Cloud Strategy, CNCF Annual Survey