Open-weights vs closed models
Open-weights models publish their parameters for self-hosting; closed models are API-only. The choice trades control and residency for convenience.
An open-weights model is one whose trained Parameters & weights you can download and run on your own computers or cloud account. A closed model is only available through the provider's API: you send text, they run it, you get an answer.
Open weights give you control, privacy and the ability to fine-tune deeply, but you run the infrastructure. Closed models are easier to start with, and the frontier ones are usually the most capable.
Buying a car versus using a ride service. Owning means you choose modifications and where it goes, but you handle fuel and maintenance; the ride service just arrives, on their terms.
Assuming open weights are free or automatically cheaper. At low or spiky volume, idle GPUs and engineering time cost more than API tokens; self-hosting pays off at steady high utilisation or when data rules require it.
'Open weights' is not open source: most releases publish weights but not training data or full training code, and the OSI's Open Source AI Definition 1.0 (2024) requires more. Licences differ materially as of Oct 2026: Qwen3 and gpt-oss (Apache-2.0), DeepSeek-V3/R1 (MIT), Mistral (Apache-2.0 for several models), Gemma (Gemma terms of use), Llama 3.x (Llama Community Licence: a 700M monthly-active-user threshold, an acceptable-use policy, 'Built with Llama' attribution, and derived model names beginning with 'Llama'). Legal review is part of model selection.
Open weights buy: data residency, no per-token vendor pricing, no forced Model deprecation & upgrades, full fine-tuning and LoRA & parameter-efficient fine-tuning, logit access, Model quantization control. Costs: GPU capacity and utilisation risk, an inference stack to run, your own safety layers and Guardrails, and a capability gap to the strongest closed models on the hardest tasks. Closed APIs give frontier capability, managed scaling and features such as Prompt caching and batch discounts, with vendor terms and version churn. Most production systems mix both behind an LLM gateway that routes by task, cost and data sensitivity.
Start on a closed API behind an LLM gateway. Move a route to open weights when data rules require it or when sustained volume beats the break-even with 2× margin. Shortlist Apache-2.0/MIT models first, get legal sign-off on the licence, and run the same Golden set on both before switching.
Clinical documentation company: PHI-heavy note summarisation runs on a self-hosted open-weights 70B-class model in FP8 on 2×H100 inside its own VPC (data never leaves), at a steady ~1,000 tokens/s. Non-sensitive analytics questions go to a closed frontier API. The gateway logs cost per route, and Model fallback & routing sends overflow to a second hosted provider of the same open model under a BAA.
Lead with requirements, not ideology: data residency, cost at your volume and utilisation, capability needed, licence. Staff line: 'Open weights trade API convenience for control; I decide per workload, keep both behind a gateway, and re-run the break-even every quarter because prices move.' Expect: 'when is self-hosting cheaper?' — steady high utilisation; spiky low volume almost never.
Open weights
- Download + self-host
- Data stays in your VPC
- Deep fine-tuning
- You run GPUs + safety
Closed / API
- Call provider API
- Frontier capability
- Managed scaling, caching
- Vendor terms + churn
Numbers worth knowing
- Llama licence MAU threshold
- 700M monthly active usersLlama 3.x Community Licence
- Self-host break-even
- ~280 tokens/s sustained2×H100 at $6/hour vs $6/M blended API, illustrative
- gpt-oss-120b footprint
- one 80 GB GPUMXFP4, Apache-2.0
How to decide
| When | Do this |
|---|---|
| Data cannot leave your environment | Open weights in your VPC (or a provider with the right residency contract). |
| Spiky or low volume | Closed or hosted-open API; do not run idle GPUs. |
| You need deep fine-tuning or logit access | Open weights. |
Cost maths
Assume 2×H100 at $3/GPU-hour = $6/hour = $4,380/month, and an API blended price of $6/M tokens ($3 in / $15 out at 3:1). Self-host $/M = $6 ÷ (tokens/s × 3,600) × 1e6: at 1,000 tok/s sustained → $1.67/M (wins); at 100 tok/s → $16.7/M (loses). Add ~0.5 FTE of platform engineering (~$10k/month illustrative) and the break-even moves to ~900 tok/s.
Operating it
- API spend on one route > 2× the self-host cost at measured utilisation for 3 months
- A new data-residency or contract requirement
- Provider deprecates a model your product depends on
- Self-hosted GPU utilisation < 30% averaged over a week → reconsider API
- Cost per M tokens per route, self-host vs API, recomputed monthly
- Models in production without a recorded licence review → 0
When it goes wrong
Two weeks before GA, legal blocked the launch of a product built on a fine-tuned open model.
Pre-launch licence review of the model card and product branding.
The base model's licence required specific attribution and naming for derivatives; the team had named and marketed the fine-tune without either, and no one had read the licence at selection time.
Renamed the model and added attribution to meet the licence, then added licence review to the model-selection checklist and an allowlist of pre-approved licences in the gateway.
Related terms
Where it shows up
Sources
- The Open Source AI Definition 1.0 — Open Source Initiative, 2024spec
- Llama 3.1 Community License Agreement — Meta, 2024docs
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — DeepSeek-AI, 2025paper
- DeepSeek-V3 Technical Report — DeepSeek-AI, 2024paper
Check yourself
Three questions. Every option is explained once you answer.