Why do concurrent requests bypass an LLM budget limit?

Because the limit is checked before a request runs and the cost is recorded after it finishes. Requests that overlap all read the same pre-request total, all pass the check, and all execute. LiteLLM #18730 reports this going roughly 6x over a 100 TPM cap.

How do I test whether my spend limit actually holds?

Create a key with a small budget you are willing to lose, then fire 20 requests concurrently rather than in sequence. Compare final recorded spend against the limit. A sequential test will pass even on a system that has this bug, because the race never occurs.

What is reserve-before-execute?

Debit the estimated maximum cost from the wallet atomically before the request runs, then settle the difference against actual cost afterwards. It turns budget enforcement into an admission-control decision, so a second concurrent request sees the first one's reservation.

Does a provider-side billing limit fix this?

It fixes the catastrophic case, because the provider enforces it and holds the money. It does not give you per-customer attribution, and most providers enforce it with a delay, so treat it as a backstop rather than a precise cap.

Is my dashboard number wrong too?

Possibly. Reported spend and actual billed spend are separate numbers that can drift. Claude Code #83048 reports budget.spent() reading 72x under actual consumption, so compare your platform's figure against the provider's billing console before trusting either.

Why Your LLM Spend Limit Doesn't Actually Stop Spending

Short answer: most budget systems check spend before a request and record it after, with no lock in between. Concurrent requests all read the same pre-request total, all pass the check, and all run. The limit is enforced against stale numbers, so it is a dashboard rather than a control. Fix it in this order: set a hard limit in the provider console, cap per-key spend at your gateway, and adopt reserve-before-execute only if you need per-customer attribution.

Symptoms

You probably have this problem if any of these are true.

Recorded spend on a key exceeds its configured max_budget and the key keeps serving.
Your platform’s cost figure and the provider’s billing console disagree by more than a rounding error.
Overspend appears in bursts that line up with traffic spikes, not with steady usage.
A sequential test of your budget cap passes, but real workloads still overshoot.

That last one is the tell. A cap that holds at concurrency one and fails at concurrency ten is not a misconfiguration — it is a race.

Why it happens

The failing sequence is check → execute → record. It is correct for one request at a time and wrong the moment two overlap, because nothing holds a lock across the three steps.

Two overlapping requests both read the same pre-request total. Both compare it against the limit. Both pass. Both execute. Neither was wrong at the moment it checked; the total was simply stale by the time it mattered.

This is the classic check-then-act race applied to money. It does not require unusual load — two requests landing in the same window is enough, and agent workloads generate exactly that pattern because an agent decides its own request volume.

The evidence, across four layers

The reason to treat this as a pattern rather than one vendor’s bug is that it appears at four layers that share no implementation. Every title below was fetched and confirmed verbatim on 2026-08-22. These are user reports and open issues, not confirmed vendor behaviour.

Layer	Issue	What it reports
Model vendor CLI	#83048	`budget.spent()` reports 72x under actual consumption, filed SEV-1
Model vendor CLI	#85022	Drained a prepaid balance without consent
Gateway	#18730	Concurrent requests bypass TPM limits, ~6x over a 100 TPM cap
Gateway	#26672	A key with `max_budget: 0.05` reached `spend: 0.540408` and kept serving
Gateway	#35524	`reserve_budget_for_request()` returns without reserving when cost cannot be estimated
Sandbox provider	daytona #4791	A 2 vCPU sandbox burned ~$5 of credit in one hour
Protocol	MCP #3229	RFC asking for token metering and session budgets, because the protocol has none

Four more sit behind these: Claude Code #77095, #77373, #85400, and LiteLLM #12905 on team-key budgets going unenforced.

#35524 is the one worth reading twice. The reservation mechanism exists; the report says it declines to engage precisely when a request’s maximum cost is hardest to predict, falling back to read-time enforcement instead of admission control.

Test your own setup

Two tests, about twenty minutes. The first checks whether your numbers are honest, the second whether your cap is real.

Check meter drift

Record your platform’s reported spend and the provider’s billing console figure for the same 24-hour window, then divide.

# LiteLLM example: spend recorded for a key over the last day.
# Prerequisite: LITELLM_URL and LITELLM_MASTER_KEY set; key_name is the key you are auditing.
curl -s "$LITELLM_URL/spend/logs?start_date=2026-08-21&end_date=2026-08-22" \
  -H "Authorization: Bearer $LITELLM_MASTER_KEY" \
| python3 -c 'import json,sys; print(round(sum(r["spend"] for r in json.load(sys.stdin)), 6))'

Expected output is a single number. Compare it against the same window in your provider’s billing console. A ratio near 1.0 means your meter is honest; anything above 1.05 is drift worth explaining before you trust any cap built on it.

Force the race

Sequential tests pass on broken systems. You have to overlap the requests.

# Fire 20 concurrent requests against a key with a deliberately small budget.
# Prerequisite: TEST_KEY has a max_budget you are willing to lose (e.g. $0.05).
seq 20 | xargs -P 20 -I{} curl -s -o /dev/null -w '%{http_code}\n' \
  "$LITELLM_URL/v1/chat/completions" \
  -H "Authorization: Bearer $TEST_KEY" \
  -H 'Content-Type: application/json' \
  -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"hi"}],"max_tokens":16}' \
| sort | uniq -c

-P 20 is the load-bearing flag: it runs the requests in parallel. Expect a mix of 200 and 429. Then read the key’s final recorded spend and compare it against the budget. If final spend exceeds the budget, your cap is advisory. If every request returns 200, it is not enforcing at all.

Fixes, cheapest first

1. Hard limit at the provider

Set a billing limit in the provider console. It is enforced by the party holding the money, needs no code, and covers the catastrophic case. Most providers apply it with some delay, so treat it as a backstop rather than a precise cap.

This is the correct stopping point for a solo builder on one key.

2. Per-key caps plus alerting at the gateway

Useful for isolating tenants from each other, and honest about what it is: the cap will still overshoot under concurrency, per #18730 and #26672. Pair it with an alert on the drift ratio from the test above so you find out in minutes rather than at invoice time.

3. Reserve-before-execute

Invert the ordering so the wallet is debited before the request runs:

authenticate → authorize → reserve → execute → meter → settle

reserve debits an estimated maximum atomically, so a second concurrent request sees the first one’s reservation and is refused admission. settle returns the unused difference and writes both to an append-only ledger.

Two properties carry the whole design. The reservation must be atomic — a read followed by a write is the same race in different clothing. And estimation must fail closed: when maximum cost cannot be estimated, refuse or charge a conservative ceiling. #35524 reports that path going the other way.

This is the fix the reporters propose themselves. #18730’s submitter names AWS Bedrock’s token-reservation pattern explicitly; MCP #3229 asks the protocol to carry metering because server operators absorb inference costs with no standard way to bill per transaction. It also shows up in the market: BricksLLM, Bifrost, agentgateway, and any-llm-gateway each independently built their own proxy citing stolen keys, missing per-key metrics, and config sprawl.

KitDev’s gateway runs this sequence against an append-only ledger, with Hermes and Pi executing inside Firecracker sandboxes. Those are development proofs on self-hosted infrastructure, not a production availability claim; cost-per-request figures from that setup are to be captured from a live test.

What none of this fixes

Estimation error sets your ceiling. Reserving a maximum means reserving pessimistically. Long-context requests reserve large amounts and release most at settle, so a customer near their limit gets refused for capacity they will not use.

The ledger becomes critical-path. Atomic reservation puts a write in front of every request. Decide in advance whether a wallet-store outage refuses traffic or disables enforcement.

Non-token meters stay invisible. Sandbox time, egress, and storage are separate meters. Daytona #4791 is a compute-time burn, and a gateway that only meters model calls will never see it.

Enforcement is not attribution. You can enforce a global cap perfectly and still not know which customer caused the spend. Attribution requires tenant identity carried through to the ledger — a data-model change, not a limit setting.

When it’s actually something else

If your numbers are wrong but your cap holds under the concurrency test, you have a reporting problem, not an enforcement one. Start from the drift ratio and the provider’s billing export, not from your gateway config.

If spend is high but every cap behaves correctly, the problem is upstream in workload design — retry storms and context resends are the usual causes. A tool-call failure that silently triggers retries is one of them; see fixing broken tool calls in self-hosted inference.

Reference

Check-then-act race: LiteLLM #18730, #26672
Reservation skipped when cost is unestimable: LiteLLM #35524
Reported-versus-actual drift: Claude Code #83048, #77373
Non-token meters: Daytona #4791
Protocol-level gap: MCP #3229
Containment, the other half of runaway-agent risk: how to sandbox an AI coding agent