AWS Bedrock vs Self-Hosted LLMs: Choose the Operating Model, Not the Model

Updated: August 2026 · AWS · LLM infrastructure · FinOps · AI operations

Decision summary

Amazon Bedrock reduces the infrastructure you must operate.

Self-hosting increases the infrastructure and model controls you own.

Neither option is automatically faster, cheaper or safer. The right choice depends on workload shape and the team's ability to run it.

The question “Bedrock or self-hosted?” is often framed as a model comparison. That is the wrong level of abstraction.

The same model family can behave differently depending on serving stack, quantisation, hardware, batching and runtime settings. A managed service can also offer more customisation than the word managed implies.

The decision is really about the operating model:

Who owns capacity, patching and failure recovery?

How much runtime and model control is required?

Can the team attribute cost to a feature or customer?

Where may prompts and outputs be processed?

What happens when demand or a model version changes?

Answer those questions before comparing headline token and GPU prices.

What Amazon Bedrock removes and what it does not Amazon Bedrock provides managed access to foundation models through AWS APIs. Teams do not provision GPU instances or operate the underlying inference servers for on-demand use.

That can shorten the path to a production experiment because AWS handles much of the serving layer. Bedrock also integrates with AWS Identity and Access Management (IAM), logging, encryption and regional controls.

What it does not remove is application engineering. Your team still owns:

prompt and context design;

evaluation and output validation;

retrieval quality and data permissions;

timeouts, retries and fallbacks;

abuse controls and human escalation;

latency and cost budgets; and

monitoring the business outcome, not just the model call.

Managed inference transfers a category of operational work to AWS. It does not transfer accountability for the application.

Bedrock's cost profile is variable, not invisible The original mistake many teams make is multiplying a prompt by a published token price and calling that the unit cost. Real agentic requests may include tool definitions, retrieved documents, conversation history, retries and multiple model calls.

A more useful cost unit is the completed business operation:

cost per completed operation = inference calls

• retrieval and storage

• orchestration compute

• observability

• failed and retried work

• human review Pricing varies by provider, model, Region and inference mode, so a single “cost per 1,000 tokens” range is not a reliable architecture assumption. Use the current AWS pricing page and the actual input, output, cache and batch behaviour of the selected model.

Bedrock supports native attribution if you configure it It is inaccurate to say Bedrock has no native cost attribution. AWS documents several mechanisms:

IAM principal attribution for user or team visibility;

application inference profiles for application or workload tagging on supported runtime APIs;

projects or workspaces for supported Bedrock Mantle workloads; and

per-request metadata with model invocation logs for prompt-level token detail.

The caveat is granularity. Native billing data is generally aggregated, while per-request logs provide token counts that you must convert into cost. A shared gateway can also obscure the original user unless you propagate identity or request metadata deliberately.

So the real failure pattern is not “Bedrock cannot report unit economics.” It is launching without an attribution design.

Optimise the request graph before changing the platform Costs often grow because the application does more inference than the product team realises:

every user action fans out to several agents;

large static instructions are resent on every call;

failed tools cause full-chain retries;

an expensive model handles tasks a smaller model could complete; or

a long conversation is retained without compaction.

Bedrock supports prompt caching for compatible models, which can reduce repeated-input cost and latency. It also offers batch inference for suitable asynchronous workloads and provisioned options for predictable demand on supported models. These features change the cost comparison, so include them in the benchmark.

What self-hosting actually means Self-hosting is not simply “run an open-source model on EC2.” Production ownership can include:

selecting and validating model weights and licences;

choosing accelerators, quantisation and serving software;

capacity planning and autoscaling;

batching, queueing and admission control;

model and runtime patching;

observability for latency, memory and output quality;

multi-zone or multi-Region recovery where required;

security hardening and access control; and

on-call response when the serving layer fails.

Some teams already operate this platform. For them, another model may be an incremental workload. For a team without machine-learning infrastructure, it is a new production system with a specialised failure surface.

That is the hidden comparison: not token price versus GPU price, but managed inference versus the fully loaded cost of operating inference.

When self-hosting can be the better choice Self-hosting deserves serious consideration when the requirement cannot be met economically or technically through a managed API.

1. Sustained utilisation is high and predictable Owned capacity becomes easier to justify when demand keeps accelerators productively occupied. Low utilisation reverses the economics because the organisation pays for idle hardware or instances.

High volume alone is not proof that self-hosting is cheaper. Benchmark the actual model, hardware, concurrency, context length, batching policy and service-level objective. Include engineers, monitoring, failover capacity and upgrade work in the total cost.

You need runtime controls Bedrock does not expose for the chosen model Self-hosting can provide control over quantisation, kernels, scheduling, batching, speculative decoding, model files and serving topology. It can also make version pinning and reproducibility more explicit.

Do not confuse this with “Bedrock cannot customise models.” AWS currently documents supervised fine-tuning, reinforcement fine-tuning and distillation for supported models, and separately supports importing certain customised models. Availability depends on the model, method and Region.

The comparison must therefore name the required control. “More customisation” is too vague to drive an architecture decision.

The organisation already runs ML infrastructure An existing platform team, accelerator estate, deployment pipeline and on-call rotation change the economics. The marginal cost of adding a model may be reasonable because the operating capability already exists.

Without that capability, self-hosting can divert senior engineers from the product that is meant to create value.

A verified data-handling requirement demands it Some organisations require processing within a specific environment or need controls unavailable through a selected managed configuration. Self-hosting can be appropriate, but only after translating “data residency” into precise requirements: storage location, processing location, support access, telemetry, backups and cross-border transfer.

Bedrock offers regional endpoints and both geographic and global cross-Region inference profiles. A geographic profile keeps requests within its defined geography; a global profile may route to any supported commercial Region. That routing choice must be explicit. For strict processing-location requirements, verify the selected model, Region and inference profile rather than assuming every Bedrock request behaves identically.

When Bedrock is usually the better first move Bedrock is often the pragmatic default when:

the team needs to test several model providers quickly;

demand is uncertain or bursty;

the use case is not worth a new GPU operations function;

AWS-native identity, logging and network controls are valuable;

the team wants model customisation without operating the serving fleet; or

engineering time is more constrained than inference spend.

“First move” does not mean “temporary prototype.” A managed service can remain the correct long-term platform. The decision should be revisited only when evidence shows a constraint: cost per completed operation, latency, control, capacity or compliance.

Compare the two options with one benchmark Architecture debates become useful when both options are tested against the same workload.

Create a representative evaluation set and measure:

Dimension What to measure

Quality Task success, groundedness and policy violations

Latency Median and tail latency at expected concurrency

Throughput Completed operations per minute, not raw tokens alone

Reliability Timeouts, throttling, queue delay and recovery behaviour

Cost Cost per successful operation at low, normal and peak load

Operations Deployment work, incidents, upgrades and on-call burden

Control Required runtime, model, network and data-handling controls

Change Effort to adopt a new model or roll back a version Use at least three demand scenarios.

A self-hosted system may look efficient at peak utilisation and poor during normal traffic. On-demand Bedrock may look attractive at low volume while provisioned capacity or self-hosting changes the result at a stable baseline.

Include failure tests. What happens when the model endpoint throttles, a tool call times out or traffic doubles? A platform that is marginally cheaper during a clean benchmark may be more expensive if it fails badly under load.

A sensible hybrid is designed, not accumulated Many mature systems use more than one inference path. Bedrock may handle experimentation, lower-volume tasks or models unavailable internally. A self-hosted model may serve a stable, high-volume path with a justified control requirement.

Hybrid becomes dangerous when routing emerges by accident.

Define:

which workload goes to which model and why;

the data permitted on each path;

common evaluation and observability standards;

fallback behaviour and quality thresholds; and

cost ownership for every route.

Without those rules, the organisation inherits two platforms and understands neither.

The decision framework Use these questions in order:

What must the model accomplish? Define quality, latency and reliability targets.

What controls are genuinely required? Name the runtime, model and data controls, not a preference for “flexibility”.

What is the workload shape? Measure context length, call graph, concurrency, repetition and peak-to-average demand.

What capability already exists? Count the engineers, infrastructure and on-call processes available now.

What is the full cost per successful outcome? Include failed work and operations.

What would trigger a change? Establish thresholds before the first production deployment.

Final reality check Bedrock and self-hosting optimise different ownership boundaries, but neither eliminates architecture work.

Choose Bedrock when transferring inference operations to AWS lets the team ship and learn faster without violating a real requirement. Choose self-hosting when measured workload economics or a named control justifies owning the platform.

The most expensive option is the one selected from a slogan:

“managed is always expensive”;

“open source is free”;

“self-hosting gives complete control”; or

“the cloud handles everything”.

The correct question is simpler:

Which responsibilities create product value for us — and which responsibilities should we pay someone else to operate?

Primary references AWS — Amazon Bedrock pricing

AWS — Track usage and costs in Amazon Bedrock

AWS — Application inference profiles

AWS — Model customisation in Amazon Bedrock

AWS — Import a customised model

AWS — Prompt caching

AWS — Batch inference

AWS — Geographic cross-Region inference

AWS — Data protection in Amazon Bedrock

Features, supported models, Regions and pricing change. Verify the linked AWS documentation for the intended deployment before making a production decision.