LiteLLM Alternatives (2026): 8 AI Gateways Compared by Cost, Security & Self-Hosting
Last updated 2026-07-07 · Part of Awesome AI Gateway — the only AI-gateway list with a reproducible cost benchmark and a security-honest scorecard. ⭐ Star it.
📊 Key numbers · LiteLLM is the most-starred OSS gateway (⭐ 54.9k) but shipped two serious 2026 CVEs — a pre-auth SQL injection and an unauthenticated RCE on CISA's KEV list (both fixed in current stable; pin it and don't expose the admin panel) (security scorecard). On cost the model dominates, not the gateway: a 100K-token report ranges $0.03–$3.01 (a ~106× spread, computed 2026-06). Whatever alternative you pick, configure fallback: rate limits alone caused ~⅓ of all LLM errors in production telemetry (Datadog, Mar 2026).
LiteLLM is the default self-hosted LLM gateway — one OpenAI-compatible proxy in front of 100+ providers, virtual keys, budgets, load balancing. It's popular for good reason. But people go looking for alternatives for three honest reasons:
- Security posture. LiteLLM had two serious 2026 CVEs — a pre-auth SQL injection (CVE-2026-42208) and an unauthenticated RCE that landed on CISA's Known-Exploited-Vulnerabilities list (CVE-2026-42271). Both are fixed in
v1.83.7-stable— if you run LiteLLM, pin to current stable and never expose the admin panel — but the track record sends some teams looking. - They don't want to run a server at all → a hosted gateway.
- They want lower latency or a different governance model → a Go/Rust-native proxy, or an enterprise guardrail platform.
Here's the honest, data-backed map. Scores are ★1–5 from the scorecard rubric (snapshot 2026-06).
TL;DR — pick by your actual constraint
| Alternative | Type | Markup | Security | Reach for it when |
|---|---|---|---|---|
| Bifrost | Self-hosted (Go) | $0 | ★3.5 · no known CVEs | You want LiteLLM's job but faster and with a clean CVE record |
| Portkey Gateway (OSS) | Self-hosted (MIT) | $0 | ★4.0 | You want guardrails + circuit breakers + MCP OAuth built in |
| Kong AI Gateway | Self-hosted | $0 core | ★4.0 | You already run Kong (PII sanitization + RBAC need the Enterprise tier) |
| Envoy AI Gateway | Self-hosted (K8s) | $0 | ★4.0 | You're Kubernetes/Istio-native and want a CNCF-aligned proxy |
| OpenRouter | Hosted | ~5.5% | ★3.0 | You want zero ops and ~340 models in five minutes |
| Cloudflare AI Gateway | Hosted | 0% | ★4.0 | You want a free, 0-markup hosted gateway with DLP/PII scanning |
| Vercel AI Gateway | Hosted | 0% | ★3.5 | You're on Vercel and want true 0% markup incl. BYOK |
| new-api / one-api | Self-hosted | $0 | ★1.5–2.0 | You need a China-friendly relay panel — and will patch aggressively |
Same task, the model behind the gateway can cost 100× more ($0.03 vs $3.01 for one 100K-token report — a 106× spread). Every alternative below lets you route cheap-by-default and escalate only when needed — that, not the gateway's own fee, is where the money is.
If you want to stay self-hosted (just better than LiteLLM)
- Bifrost — the closest drop-in upgrade. Go-native, 0.56 ms measured added latency per request — ~10× lighter than LiteLLM's 5.41 ms (independent harness; the vendor's ~11µs claim did not reproduce there) — adaptive load balancing, cluster mode, 1000+ models, and no known CVEs. If your reason for leaving LiteLLM is performance or security hygiene, start here.
- Portkey Gateway (OSS) — MIT, 2.69 ms measured overhead (marketed "<1ms" did not reproduce independently), with guardrails, circuit breakers, fallbacks and MCP OAuth 2.1 free to self-hosters. The richest governance feature set of the open-source options; upgrade path to the managed cloud if you scale.
- Kong AI Gateway — if you already run Kong or APISIX, AI Prompt Guard plus Kong's mature auth/mTLS/rate-limit plugins bolt onto infrastructure you already operate (★4.0 security, tied for the self-hosted top; note PII sanitization, Model Armor and RBAC sit in the Enterprise tier, not OSS).
- Envoy AI Gateway — built on Envoy, native to Kubernetes/Istio, with multi-provider routing and an MCP gateway (OAuth + CEL authz). No semantic cache or per-key budgets yet, but the cleanest fit for a CNCF-aligned platform team.
If you'd rather not run a server (hosted alternatives)
- OpenRouter — the fastest start: change
base_url, get ~340 models behind one key, auto-failover, free zero-data-retention, EU region-lock. ~5.5% credit fee; no public SLA outside enterprise. - Cloudflare AI Gateway — 0% markup, SOC 2 II / ISO 27001 / PCI / GDPR, with free DLP + PII scanning, guardrails and fallback. 100% SLA on Business+. The strongest free hosted option on compliance.
- Vercel AI Gateway — true 0% markup including BYOK, SOC 2 II, 99.99% SLA (Enterprise), ZDR option. The obvious choice if you already deploy on Vercel.
If you need a China-friendly relay panel
- new-api (⭐ 43.6k) is the most active successor to one-api (⭐ 36k), the MIT original. They're great for multi-key, multi-provider reselling panels — but be clear-eyed: new-api carries a cluster of 2026 CVEs (IDOR auth-bypass, two SSRF, a SQLi/DoS), so sandbox it, restrict egress, and patch aggressively. See the dedicated one-api vs new-api vs LiteLLM (Chinese) breakdown.
So, should you actually leave LiteLLM?
Often, no. Patched to v1.83.7-stable and kept off the public internet, LiteLLM is a healthy project with weekly releases and the broadest provider coverage. Leave it when you have a specific reason:
- Performance / clean CVE record → Bifrost
- Built-in guardrails & governance → Portkey
- Already on Kong/APISIX/Envoy/K8s → the matching native gateway
- Zero ops → OpenRouter (fastest) or Cloudflare/Vercel (0% markup)
Full scorecard (compliance / markup / security / stability for 20+ gateways) and the reproducible cost tables are in the evaluation set →. Browse all gateways by need in Awesome AI Gateway →.
Found this useful? ⭐ Star the list — it's how the next engineer choosing a gateway finds it. CC0, no signup, no tracking, no vendor money.