The AI landscape has handed enterprises a genuinely difficult architectural question: do you call a vendor API (OpenAI, Anthropic, Google), or do you self-host an open-weight model (Llama 3, Mistral, Qwen)?
This is not a question of convenience. It touches data privacy, compliance obligations, long-term economics, and how much operational risk you’re willing to carry. The right answer varies by organisation. This guide maps the trade-offs so you can make the call with clarity.
Data Privacy, Security, and Compliance: Where the Decision Often Starts
For many enterprises, data sovereignty ends the conversation early.
When you call a vendor API, your prompts and data travel to a third-party server. Even with enterprise agreements that prohibit training on your inputs, the data physically leaves your network. For healthcare, financial services, and government, that exposure creates real legal and compliance risk.
A self-hosted model eliminates that exposure entirely. Prompts stay inside your infrastructure making compliance audits simpler and intellectual property does not pass through external systems.
If your data classification policies, regulatory obligations, or legal counsel would flag external data transmission as a problem, self-hosting is the baseline requirement.
Cost: The Maths Depends on How Much You Use It
The cost comparison between vendor APIs and self-hosted models is heavily dependent on utilisation rates.
Vendor APIs charge per token. That structure works well for prototyping, low-volume applications, and irregular usage. You pay for what you consume and carry no idle infrastructure costs.
Self-hosting flips the model. You pay for compute uptime, not token generation. Costs are fixed and predictable, regardless of whether the model is processing requests or sitting idle.
| Cost Factor | Vendor API (e.g., OpenAI) | Self-Hosted (e.g., Llama 3) |
|---|---|---|
| Pricing Model | Usage-based (per token) | Time-based (per hour of compute or power use) |
| Upfront Investment | Zero to minimal | High (cloud instance setup or hardware purchase) |
| Predictability | Variable, scales with usage | Fixed, predictable monthly infrastructure costs |
| High Utilisation Cost | Expensive at scale | Highly cost-effective |
| Low Utilisation Cost | Very cheap | Expensive (paying for idle compute) |
The crossover point is utilisation. High, continuous workloads favour self-hosting because the fixed cost per token drops as throughput increases. Sporadic or unpredictable workloads favour vendor APIs as you avoid paying for idle capacity.
Self-hosting also provides protection against vendor price changes and model deprecations, which add a different kind of cost: the engineering time to migrate when a vendor discontinues a model you’ve built on.
Uptime, Maintenance, and Operational Burden
Vendor APIs abstract away the infrastructure entirely. The provider manages scaling, load balancing, hardware failures, and security patching. Your team consumes a service.
Self-hosting transfers that burden internally. Running a production-grade LLM requires people who can manage GPU memory constraints, configure inference servers (such as vLLM or Ollama), handle hardware failures, and maintain network security. If you don’t have that capability today, building it takes time and money.
Speed is also a practical consideration. Vendor APIs are purpose-built for throughput, delivering hundreds of tokens per second. Self-hosted models struggle to match that performance without significant infrastructure investment. For latency-sensitive applications, that gap matters.
Version Control, Model Updates, and Stability
Vendor APIs give you access to the latest models the moment they release. New capabilities, longer context windows, and multimodal features arrive without any action on your part.
The trade-off is stability. Vendors deprecate older models and sometimes alter model behaviour in ways that affect existing applications. An output that worked reliably on one version may behave differently after an update, and you may not get much notice.
Self-hosting locks you to a specific version. Behaviour is consistent and reproducible for as long as you need it. Updating to a newer model is a deliberate choice, not something imposed on you. The open-source community releases capable models frequently, and quantisation techniques now allow large models to run on hardware that would have been insufficient even a year ago.
Guardrails, Customisation, and Model Behaviour
Vendor models ship with built-in safety guardrails. These are designed for general audiences and broad use cases. For most applications, they’re appropriate. For specialised enterprise use cases, they can be too restrictive and refusing prompts that are benign in context, or producing outputs that don’t match your organisation’s voice.
The US government’s June 2026 export control directive suspending access to Anthropic’s Claude Fable 5 for all foreign nationals, citing a jailbreak vulnerability, illustrated a different kind of risk: vendor-side disruption that enterprises have no control over. Organisations that had built workflows on Fable 5 found access cut without warning. That event accelerated a shift in enterprise thinking toward models that no government directive can reach.
Self-hosted models give you complete control over behaviour. You can fine-tune on proprietary data, define your own guardrails, and build an AI capability that reflects your specific domain. Competitors using generic vendor APIs cannot replicate that. The model becomes an asset, not a commodity.
Conclusion: Match the Architecture to the Actual Risk
The vendor vs. self-host decision comes down to where your risks sit and what you’re optimising for.
Choose a Vendor API when:
- Speed to market is the primary goal.
- Usage is low or unpredictable.
- Data leaving your network is not a compliance concern.
- Your organisation lacks the MLOps capability to run production infrastructure.
Choose to Self-Host when:
- Data privacy or regulatory requirements prohibit external data transmission.
- Utilisation is high and continuous, making per-token pricing prohibitive.
- Deep customisation or proprietary fine-tuning is required.
- The application must operate in offline or air-gapped environments.
- Resilience against third-party disruption such as regulatory, commercial, or otherwise, is a business requirement.
For many organisations, neither option alone is the right answer. A hybrid architecture routes sensitive, high-volume, or compliance-constrained work to self-hosted models, while vendor APIs handle complex reasoning tasks or public-facing applications where frontier capability matters more than data control. Simple, ad-hoc queries with no sensitive context go to the vendor. Anything carrying proprietary data stays internal.
The architecture question is really a risk question. Map your actual risks first. The right infrastructure follows from that.