Find the best AI / LLM gateway for 2026 — the only list with a reproducible cost benchmark, a 5-axis scorecard, and independently-measured proxy overhead, not just links.
🧭 Which gateway should I use? · 💰 Cost calculator · ⚖️ LiteLLM vs OpenRouter vs Portkey — pick by need, size the cost, or compare the most-asked options.
USD per task, computed from models.json by a unit-tested script. Click a header to sort. → Cost calculator for your own token mix.
| Model | Provider | 100K report | Summarize | Coding | 1M chat |
|---|
Scores ★1–5 across five axes from gateways_scorecard.csv per the published rubric. 🏠 = self-hosted, you own compliance. Click a header to sort.
| Gateway | Type | Compliance | Markup | Security | Stability | Obsv |
|---|
What actually breaks once a gateway is in production — incidents, CVEs, acquisitions — from BENCHMARKS Part 5. Every event is dated and links to a primary source; stars measure popularity, this measures the 3am pager.
| Gateway | Praised for | Recurring gripe | Dated event worth knowing |
|---|
Data-backed head-to-heads and guides for the questions people actually search:
Browse all comparisons → · 🧭 Gateway picker · 💰 Cost calculator
A proxy between your code and LLM providers: one OpenAI-compatible endpoint and key for many models, adding routing, failover, caching, rate limits, cost tracking and guardrails.
A router decides which model gets each request (cheap vs flagship); a gateway is the full proxy layer (auth, caching, observability, guardrails) that usually includes routing.
LiteLLM is the default for breadth (Python, 100+ providers). For raw performance pick Bifrost (Go); for enterprise Kubernetes pick Kong or Higress.
OpenRouter is hosted (zero ops, ~5.5% fee, 400+ models); LiteLLM is self-hosted (your keys, your infra, 0% markup). Start hosted, self-host when volume justifies it.
For zero ops, use Vercel AI Gateway or Cloudflare AI Gateway (0% markup). For lowest token cost, route bulk work to cheap models — a 100K-token report runs ~$0.03 on DeepSeek vs $3.01 on GPT-5.5, a 106× spread.
Every gateway sees your prompts, but retention varies enormously — some routers are zero-data-retention by default, others (e.g. Martian) license your prompts to train their own models by ToS, and OpenAI is currently preserving API logs under a court order. See the primary-source data-retention matrix; for sensitive data get ZDR in the contract and check the specific upstream, not just the router.
Verified July 2026 against each provider's own docs: OpenRouter :free 50 req/day (1,000/day after a one-time $10 top-up) · Groq Llama-3.3-70B 30 req/min, 100K tok/day · GitHub Models free with any GitHub account (experimentation only) · Cerebras 1M tok/day at 5 req/min · SambaNova 20 req/day. Google hides Gemini free-tier numbers behind a login now; Mistral's free mode trains on your data by default; Together AI's free models are gone. Full row-by-row table with sources: the verified free-tier table.