Tactical Edge
Contact Us
Back to Insights

How to Migrate OpenAI Workloads to AWS with Amazon Bedrock

OpenAI models now run on Amazon Bedrock behind OpenAI-compatible APIs. A step-by-step playbook for inventory, feature parity, endpoint and auth changes, evaluation gates, and cutover.

Cloud & Infrastructure11 min
By Marcus Rivera, Cloud Architecture Lead ยท November 16, 2026
OpenAIAmazon BedrockMigrationResponses APIAI GovernanceAWS

If your product calls the OpenAI API today, you can now run the same models inside your AWS account. AWS and OpenAI made OpenAI models and Codex generally available on Amazon Bedrock on June 1, 2026, added the GPT-5.6 Sol, Terra, and Luna models in July, and made GPT-6 Astra generally available on September 8, 2026 [4][6].

For most teams, the code change is small. Bedrock exposes OpenAI-compatible Responses and Chat Completions APIs, and AWS documents that you can bring an existing OpenAI SDK codebase over by changing the base URL and API key [2]. The harder work is everything around that change: confirming that the features you depend on exist on Bedrock, proving that quality holds, and switching traffic in a way you can reverse.

This guide is a practical playbook for that move. It covers what changes, what stays the same, and the gates we recommend before production traffic leaves your current provider.

2
Bedrock endpoints that serve the OpenAI-compatible APIs
[2]
1M+
Token context windows on GPT-5.4 and newer models on Bedrock
[1]
1:1
Commercial-Region pricing compared with OpenAI direct
[1]

Why Teams Move OpenAI Workloads to AWS

The reasons are rarely about the model. They are about where the workload runs and who governs it.

  • One security and governance boundary. OpenAI models on Bedrock inherit IAM permissions, VPC and PrivateLink isolation, KMS encryption, and AWS CloudTrail audit logging [4]. The model is governed like the rest of your AWS estate.
  • Data handling you configure. AWS states that prompts and responses are not used to train models and are not shared with model providers [4]. Bedrock also gives you an explicit, per-Region data retention setting that you can enforce with Service Control Policies [3].
  • Consolidated billing. Bedrock pricing in commercial Regions matches OpenAI direct pricing for equivalent services, billing goes through AWS, and usage counts toward existing AWS commitments [1][5].
  • One platform for several model families. The same endpoints that serve OpenAI models also serve the Anthropic Messages API, so a team can evaluate and route across providers without a second integration stack [2].
Same model, different platform
Moving to Bedrock does not change the model you call. It changes the endpoint, the identity system, the quota model, the retention controls, and which platform features surround the model. Plan the migration around those differences, not around the model itself.

The Migration Playbook

The seven steps below move one workload at a time. Run them in order; each step produces the input for the next.

Step 1: Inventory the Workload

Before you change a line of code, write down what the workload actually uses. Teams often discover that a "simple chat feature" relies on stored conversations, a hosted tool, or a batch job that nobody listed.

For each application, record:

  • APIs called. Responses, Chat Completions, Embeddings, Batch, Files, Images, Audio, Realtime, Assistants, or fine-tuning.
  • Models and parameters. Model names, reasoning effort, temperature, structured output schemas, and token limits.
  • Tools. Client-side function calls, hosted tools such as file search or web search, remote MCP servers, and computer use.
  • State. Whether you use store: true, previous_response_id, or background mode.
  • Traffic shape. Peak requests and tokens per minute, latency targets, and streaming usage.
  • Data obligations. Retention requirements, residency constraints, and which teams may see prompts and outputs.

That inventory becomes the input to every later step: the parity check, the endpoint choice, the quota request, and the evaluation set.

Step 2: Map Feature Parity

OpenAI publishes a feature matrix for its models on Bedrock. Use it as the source of truth, and treat anything not listed as unconfirmed until you test it [1].

CapabilityStatus on BedrockNotes
Text, image, and file inputSupportedResponses API [1]
Structured outputs and function callingSupportedClient-side tools work on both endpoints [1][2]
StreamingSupportedRequires the streaming IAM action on bedrock-runtime [2]
Reasoning effort, persisted reasoning, prompt cachingSupportedCached input is billed at a 90% discount [1][5]
Custom tools, client-side tool search, computer useSupportedResponses API [1]
Hosted web searchbedrock-mantle onlyServer-side tools are not available on bedrock-runtime [1][2]
Background modebedrock-mantle onlyNot available when retention mode is none [1][3]
Hosted file search, shell tool, image generation toolNot supportedReplace with your own retrieval or tool services [1]
Remote MCP serversNot supportedCall MCP servers from your application instead [1]
WebSockets and mid-turn steeringNot supportedNot available on either endpoint [1]
Asynchronous tool calling, pro mode, multi-agentNot supportedRedesign flows that depend on them [1]

OpenAI's guide covers the Responses API. It does not describe parity for Assistants, Batch, Embeddings, Images, Audio, fine-tuning, Realtime, or Files, so check the AWS model cards before assuming any of these work [1]. A workload built on the Assistants API or hosted file search needs a design change, not just a configuration change.

Step 3: Choose an Endpoint

Bedrock has two inference endpoints. Both run on the same underlying inference engine and both serve the OpenAI-compatible Responses and Chat Completions APIs, but their features differ [2].

Decision factorbedrock-runtimebedrock-mantle
Base hostbedrock-runtime.{region}.amazonaws.com (OpenAI APIs on /openai/v1)bedrock-mantle.{region}.api.aws
AWS recommendationDefault for most new applicationsWhen you need features only it offers
Bedrock GuardrailsYesNo
Cross-Region inference profilesYesNo
Server-side and hosted tools, including web searchNoYes
Background (long-running) ResponsesNo, background=true returns 400Yes
Projects for workload isolation and cost trackingDefault project onlyYes
IAM action for inferencebedrock:InvokeModelbedrock-mantle:CreateInference
Per-token priceSameSame

Source: AWS endpoint documentation [2].

Our default recommendation follows AWS: start on bedrock-runtime, especially if you want Guardrails or cross-Region inference, and use bedrock-mantle for the specific workloads that need hosted web search, background mode, or Projects [2]. You can use both from the same application.

Two details catch teams during migration. On bedrock-runtime, the Responses API requires model on every request, including follow-ups that use previous_response_id [1]. And a stored response belongs to the AWS Region that served it, so retrieval and continuation must go back to that Region [2].

Step 4: Change the Endpoint, Authentication, and Model

For OpenAI SDK code, the core change is three values: the base URL, the credential, and the model ID. This Python example follows the pattern AWS publishes for the GPT-5.6 models [5]:

python
import os
from openai import OpenAI

# For example: bedrock-mantle.us-east-1.api.aws/openai/v1 over HTTPS
client = OpenAI(
    base_url=os.environ["BEDROCK_OPENAI_BASE_URL"],
    api_key=os.environ["AWS_BEARER_TOKEN_BEDROCK"],
)

response = client.responses.create(
    model="openai.gpt-5.6-terra",
    input="Summarize the attached incident report in five bullet points.",
)
print(response.output_text)

Work through the details behind those three values:

  • Model IDs. Bedrock model IDs use an openai. prefix, such as openai.gpt-5.6-terra. On bedrock-runtime you can also call cross-Region inference profiles with a us. or global. prefix [1].
  • Regions. Availability differs by model and endpoint. At launch, GPT-5.6 Sol was in US East (N. Virginia) and US East (Ohio), while Terra and Luna added US West (Oregon) [5]. Changing the model can mean changing the Region and base URL [1].
  • Credentials. A Bedrock API key works with the OpenAI SDK. For long-running applications, OpenAI recommends the standard AWS credential chain with SigV4 signing over a static bearer token. The TypeScript, Python, Go, Java, and Ruby providers support it; .NET does not have a Bedrock provider yet [1].
  • Permissions. Authentication does not grant access by itself. The calling IAM principal needs the inference action for the endpoint you chose [2].
Keep the switch in configuration
Put the base URL, credential source, and model ID behind configuration rather than code. That one decision makes the evaluation gate, the canary rollout, and the rollback in the following steps a configuration change instead of a redeploy.

Step 5: Configure Data Retention Before First Traffic

Retention is the setting teams most often skip, and it is the one compliance reviewers ask about first.

Bedrock controls retention with a mode, set per Region at the account level or per project on bedrock-mantle [3]:

  • none gives zero data retention. On the Responses API, store defaults to false, store=true is rejected, and background mode is unavailable [3].
  • default applies the model's own retention policy. AWS may retain data for safety and abuse prevention, and the provider does not receive it. Setting store=false alone does not guarantee zero retention [3].

In both modes, AWS does not share request or response content with OpenAI [1]. If your workload requires zero retention, set the mode to none explicitly and enforce it with an SCP that uses the bedrock-mantle:DataRetentionMode or bedrock:DataRetentionMode condition key [3]. Some models require retention for abuse detection; if your effective mode is below what a model requires, Bedrock blocks the request, so confirm the model's allowed modes during testing, not on cutover day [3].

Also note that an AWS Region is not an OpenAI data residency jurisdiction. If you use cross-Region inference profiles, review their destination Regions against your residency requirements [1].

Step 6: Set an Evaluation Gate

"It returned a sensible answer" is not a migration test. Before any production traffic moves, run the same evaluation set against your current OpenAI setup and the Bedrock configuration, and require the new path to match the baseline.

A practical gate covers four areas:

GateWhat to measurePass condition
Task qualityScores on a fixed set of real, anonymized requests, graded by rubric, reference answers, or an LLM judgeWithin an agreed tolerance of the current baseline
Contract fidelityStructured output schema validity, tool call arguments, refusal behaviorNo new schema failures or malformed tool calls
PerformanceTime to first token, total latency, and throughput at expected peakMeets your service level at peak load
CostTokens per task, including cached inputWithin the forecast for the same volume

Amazon Bedrock evaluations support automatic, human, and judge-model evaluation jobs, and can also score models outside Amazon Bedrock, so one tool can grade both the current and the new path [7]. Whatever tool you use, keep the dataset and rubric fixed between runs so the comparison means something. Our guide to agent release gates and evaluation harnesses covers how to build a gate that catches regressions beyond demo accuracy.

Load-test before cutover as well. Each endpoint manages throughput differently: bedrock-runtime uses fixed per-model RPM and TPM quotas that you can request increases for, while bedrock-mantle uses scheduling and queueing that can briefly queue requests under load [2]. Request quota increases based on the peak numbers from your inventory.

Step 7: Cut Over with Rollback and Observability

Treat the provider switch like any other production release.

  1. 1Shadow traffic. Send a copy of live requests to Bedrock without returning its responses to users, and compare outputs and latency.
  2. 2Canary. Route a small share of real traffic to Bedrock behind a feature flag, and watch error rates, latency, and quality signals.
  3. 3Ramp. Increase the share in steps, holding at each step long enough to see peak traffic.
  4. 4Keep the old path warm. Leave your OpenAI configuration in place until the Bedrock path has run at full traffic through a complete business cycle, so rollback is a flag change.

Observability should be in place before the canary, not after. Use CloudTrail for the audit record of API activity [4], track usage by IAM principal on bedrock-runtime or by Project on bedrock-mantle [2], and keep application-level traces that record the model, endpoint, latency, token counts, and evaluation signals per request.

Plan for features you removed
If the parity check forced you to replace a hosted tool, such as file search, with your own service, that service is now part of the migration. Give it its own tests, quotas, and on-call ownership before cutover.

What Changes and What Stays the Same

Stays the sameChanges
The OpenAI models and their behaviorBase URL, credentials, and model IDs
Responses and Chat Completions request shapes for supported featuresIdentity moves to IAM, API keys, and SigV4
Per-token pricing in commercial RegionsBilling goes through AWS
Your prompts, schemas, and client-side toolsHosted tools and some agentic features need replacements
Quotas, Regions, and retention are configured in AWS

A Migration Checklist

  • Inventory APIs, models, tools, state, traffic, and data obligations for each workload.
  • Check every feature against the OpenAI on Bedrock feature matrix and AWS model cards.
  • Choose bedrock-runtime or bedrock-mantle per workload.
  • Confirm Region and model ID availability for the endpoint.
  • Move base URL, credential source, and model ID into configuration.
  • Grant least-privilege IAM permissions and prefer the AWS credential chain.
  • Set and enforce the data retention mode in every Region you use.
  • Request quota increases from measured peak traffic.
  • Pass the evaluation gate against the current baseline.
  • Shadow, canary, and ramp with a flag-based rollback and observability in place.

How Tactical Edge Helps

Tactical Edge is an AWS Advanced Tier Services Partner that designs and runs generative AI workloads on AWS. Our AWS AI consulting team can run the inventory and parity assessment, build the evaluation gate, and plan the cutover with you. For teams that want ongoing support after the move, our managed AI operations service covers monitoring, evaluation, and cost tracking in production. We work alongside the support that AWS and OpenAI provide for these migrations, with clear ownership at each step.

Talk with Tactical Edge about migrating your OpenAI workloads to AWS.

Frequently Asked Questions

Can I use the OpenAI SDK with Amazon Bedrock?

Yes. Both Bedrock endpoints support the OpenAI-compatible Responses and Chat Completions APIs, and AWS documents that an existing OpenAI SDK codebase can move by changing the base URL and API key [2]. OpenAI recommends the AWS credential chain with SigV4 signing for long-running applications in the languages that support it [1].

Does running OpenAI models on Bedrock cost more?

No, for commercial Regions. OpenAI states that Bedrock pricing in commercial Regions matches OpenAI direct pricing for equivalent services, and AWS states the per-token price is the same on both Bedrock endpoints [1][2].

Which OpenAI features are not available on Bedrock?

OpenAI lists hosted file search, the shell tool, the image generation tool, remote MCP servers, WebSockets, asynchronous tool calling, pro mode, Programmatic Tool Calling, and multi-agent as not supported. Hosted web search and background mode are available only on the bedrock-mantle endpoint [1].

Should I use bedrock-runtime or bedrock-mantle?

AWS recommends bedrock-runtime for most new applications, and it is the endpoint that supports Bedrock Guardrails and cross-Region inference. Use bedrock-mantle when you need server-side tools such as web search, background inference, or Projects [2].

Does AWS share my prompts with OpenAI?

AWS states that prompts and responses are not used to train models and are not shared with model providers [4]. OpenAI's guide adds that with the default or none retention mode, AWS does not share request or response content with OpenAI [1]. Set the retention mode explicitly for each Region you use [3].

References

[1]OpenAI. "OpenAI models in Amazon Bedrock." https://developers.openai.com/api/docs/guides/amazon-bedrock

[2]Amazon Web Services. "Inference endpoints supported by Amazon Bedrock." https://docs.aws.amazon.com/bedrock/latest/userguide/endpoints.html

[3]Amazon Web Services. "Data retention," Amazon Bedrock User Guide. https://docs.aws.amazon.com/bedrock/latest/userguide/data-retention.html

[4]Amazon Web Services. "OpenAI models and Codex on Amazon Bedrock are now generally available," June 1, 2026. https://aws.amazon.com/blogs/machine-learning/openai-models-and-codex-on-amazon-bedrock-are-now-generally-available/

[5]Amazon Web Services. "Get started with OpenAI GPT-5.6 Sol, Terra, and Luna on Amazon Bedrock." https://aws.amazon.com/blogs/machine-learning/get-started-with-openai-gpt-5-6-sol-terra-and-luna-on-amazon-bedrock/

[6]Amazon. "OpenAI models on Amazon Bedrock." https://www.aboutamazon.com/news/aws/bedrock-openai-models

[7]Amazon Web Services. "Amazon Bedrock User Guide: Evaluation." https://docs.aws.amazon.com/bedrock/latest/userguide/evaluation.html

Article Summary

  1. 1OpenAI models, including the GPT-5.x family and GPT-6 Astra, are generally available on Amazon Bedrock, with commercial-Region pricing that matches OpenAI first-party rates.
  2. 2Bedrock serves the OpenAI-compatible Responses and Chat Completions APIs on both of its endpoints, so most code moves by changing the base URL, the credential, and the model ID.
  3. 3Feature parity is not complete: hosted file search, remote MCP servers, WebSockets, and some agentic features are not available, and several features depend on which endpoint you choose.
  4. 4Treat the switch as a governed release: inventory the workload, run an evaluation gate against your current baseline, then cut over behind a flag with rollback and observability in place.

Ready to discuss this for your organization?

Talk to our team about implementing these approaches in your environment.

Get in Touch
Tactical Edge

AI workflows connected to the data, tools, and systems your teams use.

Seattle, WA ยท Washington, DC ยท Paris, France

AWS PartnerAWS Advanced Tier Services Partner

AWS AI Services Competency

AWS Migration and Modernization Competency

AWS Business Value Realization Competency

Migration Services

Solutions

  • Agentic AI Systems
  • Agent Protocols (MCP/A2A)
  • AgentOps
  • Agent Governance
  • Moonshot Migrations
  • Cloud & Data
  • Amazon Quick
  • Amazon Connect
  • Document Automation
  • Industry Solutions
  • AI Infrastructure Go-to-Market
  • ISV Freedom Program

Platforms

  • All Platforms
  • Prospectory โ†—
  • Projectory โ†—
  • Monitory โ†—
  • Connectory โ†—
  • Greenway โ†—
  • Detectory โ†—

Services

  • Advisory & Strategy
  • Design & Engineering
  • Implementation
  • PoC & Pilot Programs
  • Agent Programs
  • Managed AI Operations
  • Governance & Compliance
  • AI Consulting

Company

  • About Us
  • Our Approach
  • AWS Partnership
  • Security
  • Demo Library
  • Events
  • Workshops
  • Insights & Resources
  • Careers
  • Contact

ยฉ 2026 Tactical Edge. All rights reserved.

Privacy PolicyTerms of ServiceAI PolicyCookie Policy