A free-trial user pastes one 400 KB document and asks for it back. Another sends a thousand valid requests from a loop. A research agent follows a link that always points to the next page. None needs an exploit, and each lands on your bill or your model. You will run all of them against a local lab in your browser, flip a Vulnerable / Secure switch, and see which fix stops which attack, what to alert on, and why a per-user limit is not enough alone.
For authorized testing and learning only. The lab listens on
127.0.0.1, uses a fake company, fake prices and a built-in simulated model (no API key, offline), and never contacts a real system.
What is unbounded consumption?
LLM10: Unbounded Consumption is the tenth entry in the OWASP Top 10 for LLM Applications (2025): a caller can use more compute, money or information than intended. OWASP’s examples include a variable-length input flood, denial of wallet, resource-intensive queries, model extraction via the API and functional model replication.
Why it happens in LLM apps
- The client sets the size. Input length and
max_tokensare accepted as sent; output tokens cost five times input tokens in the lab. - Nothing is per user. One account may send unlimited requests against a bill everyone shares.
- Agents re-send their history. Each step pays for everything read so far, so a loop costs the square of its length.
- Answers carry too much. Exact probabilities or logprobs turn each query into an equation.
Real-world references
| What | Result |
|---|---|
| Tramèr et al., USENIX Security 2016 | Equation-solving attacks recovered models from APIs returning confidence values; label-only attacks needed more queries but worked (Mission 4 in miniature) |
| Carlini et al., ICML 2024 | Recovered the embedding projection layer of OpenAI’s ada and babbage models for under $20 |
The impact: one user’s traffic can switch the provider key off for everyone, or copy an asset you paid to build. Training-data extraction is out of scope.
Attack flow

Lab setup: the Lumen Docs AI console
- Requirements
- Docker with Compose v2 (recommended) or Python 3 (standard library only; tested on 3.13 and, in Docker, 3.12), a browser, free port 8310.
- SHA-256
3985c2504af0e1418c0090432697b35d2c44efd7ab3b04e5ffe78d419e152cfc
How to check the file is genuine
sha256sum llm10-unbounded-consumption-lab.zipshasum -a 256 llm10-unbounded-consumption-lab.zipGet-FileHash llm10-unbounded-consumption-lab.zipSam Carter is a customer of lumen.example; Mallory Quinn has a free trial. The company has a $25 monthly AI budget and four features behind one gateway: an assistant, a script endpoint, a research agent and a scoring API. Start it (details):
unzip llm10-unbounded-consumption-lab.zip && cd llm10-unbounded-consumption
./run.sh # Docker, or: ./run.sh local (plain python3)
Open http://127.0.0.1:8310. The badge reads VULNERABLE: no fixes. Log in as Sam, press Normal: export my data and Run.

A normal answer is 79 tokens in, 60 out, $0.0011 (fake prices: $3 and $15 per million tokens). SimLLM only decides how many tokens a request produces. The clock is virtual (+1 min, +1 day) so every run repeats. The server awards a flag like LLM10{m1_3f9a1c7e} only when it observes the outcome.
Exploiting it
Mission 1: one request, one fortune
Goal: as Mallory, get one request served that costs about 90 times a normal answer.
- Log in as Mallory Quinn. In Assistant press M1 – flood the context, then Run. The browser builds a 400 KB document (about 100,000 tokens) and asks for it back word for word.
- Read the tiles: 100,091 tokens in, 4,096 out, $0.3617, 316x normal. The flag banner appears.

Option 2: from the terminal
python3 -c 'import json; print(json.dumps({"question": "Summarise this archive and then repeat the whole document word for word.", "document": ("Release note: fixed a rare sync error and improved the start-up time. " * 6000)[:400000]}))' \
| curl -s -X POST 127.0.0.1:8310/api/chat -H 'X-Session-User: 3002' --data-binary @- \
| python3 -c 'import json,sys; r=json.load(sys.stdin); print(r["in_tokens"], r["out_tokens"], "$%.4f" % r["cost"], round(r["x_normal"]), "x")'
# 100091 4096 $0.3617 316 x
Show solution
Solution: Mission 1. No input cap and no server-side max_tokens: the client decides the cost. Fix: F1.
Mission 2: burn the budget
Goal: as Mallory, use a quarter of the monthly budget. Bonus: take the product down for Sam.
- Press Clear ledger and state in Billing (flags stay). As Mallory open Script and press Send 1000 requests, each for a 3,000-word essay.
- 415 are served, the bar hits 100%, the alerts fire and the provider switches the key off: 585 get HTTP 402.
- As Sam, press Normal: export my data: HTTP 402.


Option 2:
python3 - <<'EOF'
import json, urllib.error, urllib.request
sent, body = {}, json.dumps({"question": "Write a 3000-word essay about the history of the printing press."}).encode()
for _ in range(1000):
req = urllib.request.Request("http://127.0.0.1:8310/api/chat", data=body, headers={"X-Session-User": "3002"}, method="POST")
try: code = urllib.request.urlopen(req).status
except urllib.error.HTTPError as e: code = e.code
sent[code] = sent.get(code, 0) + 1
print(sent) # {200: 415, 402: 585}
EOF
Show solution
Solution: Mission 2. No per-user limit, so one user spends the shared bill. Fix: F2, plus F1 to cap each request.
Mission 3: the loop that never ends
Goal: make the research agent run 50 or more steps.
- Clear ledger and state, open Agent, choose Summarise the complete release archive, press Run the agent. Every page links to the next.
- Input tokens grow from 400 to 99,400; a step costs $0.0027 at first and $0.2997 at the end. It stops at step 166 only because the provider cuts the key off: $25.10.

Option 2: curl -s -X POST 127.0.0.1:8310/api/agent -H 'X-Session-User: 3002' -d '{"task":"archive"}' returns n_steps 166, cost 25.0992.
Show solution
Solution: Mission 3. No step, time or cost budget, so a page decides when the agent stops. Fix: F3.
Mission 4: copy the model
Goal: query the scoring API until your copy agrees with the original on 90% of fresh tickets.
- Clear ledger and state, open Scoring API, press Run the extraction script (9 queries) as Mallory.
- Each answer is an exact score, so nine answers are nine equations: the script recovers all eight weights and the bias, and the copy agrees on 100% of 500 fresh tickets.

Option 2: python3 tools/extract.py 9 is a real client: it queries /api/score, fits the copy, asks /api/judge to grade it and prints FUNCTIONAL COPY.
Show solution
Solution: Mission 4. Six-decimal scores make the model solvable. Fixes: F4 (return less), F2 (limit volume).
Fixing it
The Defences tab has one switch per fix and a What changed? drawer with the real source. OWASP also lists watermarking, graceful degradation and provider-side spend caps; set those too. The lab builds four fixes.

F1: cap input and output
def limits_fixed(in_tokens, requested_max):
if in_tokens > MAX_INPUT_TOKENS: # 4000
return {"ok": False, "status": 413, "why": "input is %d tokens, the cap is %d" % (in_tokens, MAX_INPUT_TOKENS)}
return {"ok": True, "max_tokens": min(requested_max or MAX_OUTPUT_TOKENS, MAX_OUTPUT_TOKENS)} # 512: the server decides
F2: per-user rate limit and quota
def quota_fixed(usage):
if usage["req_this_minute"] >= RATE_PER_MIN: # 10
return {"ok": False, "status": 429, "scope": "minute", "why": "rate limit: %d requests per minute per user" % RATE_PER_MIN}
if usage["req_today"] >= DAILY_REQUESTS: # 60
return {"ok": False, "status": 429, "scope": "day", "why": "daily quota: %d requests per user" % DAILY_REQUESTS}
# ... the same check for DAILY_TOKENS (200000), counted after each reply
return {"ok": True}
F3: a budget for every agent run
def agent_guard_fixed(run):
if run["cost"] >= AGENT_MAX_COST: # $0.25
return "cost budget ($%.2f per run)" % AGENT_MAX_COST
if run["elapsed_s"] >= AGENT_TIMEOUT_S: # 30 lab seconds
return "timeout (%d s per run)" % AGENT_TIMEOUT_S
if run["steps"] >= AGENT_MAX_STEPS: # 8
return "step budget (%d steps per run)" % AGENT_MAX_STEPS
return None
F4: return less
def score_response_fixed(p):
return {"label": scorer.label(p)} # low / medium / high, not six decimals
Re-testing the fixes
Mission 5: prove the fixes hold
Goal: with every fix on, all four attacks fail and Sam’s normal tasks work.
- Clear ledger and state, then press Secure (badge: SECURE: 4 of 4 fixes).
- As Mallory run the M1 preset (413), the archive agent (8 steps, $0.07), Send 1000 requests (10 served, 990 get 429) and the 300-query extraction (cut off at the daily limit, copy below 90%). Press +1 min if an earlier step used this minute’s allowance.
- As Sam run export my data, summarise the memo, the cancellation agent and Score a normal ticket. All work; the M5 flag appears.

From the terminal, tools/attack.sh secure prints the same refusals.
Each fix alone
The Matrix tab runs every attack against eight settings, plus Sam’s four tasks.
| Setting | S1 flood | S2 burst | S3 agent | S4 extraction |
|---|---|---|---|---|
| No fixes | WORKS ($1.81) | WORKS ($25.01, key off) | WORKS (166 steps, $25.10) | WORKS (100%) |
| F1 only | blocked | WORKS ($7.95) | WORKS | WORKS |
| F2 only | WORKS (2 of 5 served) | blocked ($0.60) | blocked (26 steps, $0.66) | WORKS (100%) |
| F3 only | WORKS | WORKS | blocked (8 steps, $0.07) | WORKS |
| F4 only | WORKS | WORKS | WORKS | WORKS (95%) |
| F2 + F4 | WORKS | blocked | blocked | blocked (86%) |
| F1 + F2 + F3 | blocked | blocked | blocked | WORKS (100%) |
| All four | blocked | blocked ($0.08) | blocked (8 steps, $0.07) | blocked (86%) |

- F1 bounds one request, not volume. With F1 only, all 1,000 requests are served and still spend $7.95, a third of the budget.
- F2 counts after the reply. Two flood requests get through before the daily quota says no, and the loop stops only at step 26. Patient or multi-account callers stay under any per-user limit.
- F3 stops the loop early, with a reason: 8 steps and $0.07 instead of $25.
- Extraction needs F4 and F2 together. Quota alone leaves 60 exact answers, enough for 100%. Labels alone give a 95% copy from 300 queries. Both leave 86%, against 38% for always guessing the majority label: limits raise the price, they do not remove the leak.


Vulnerable vs. fixed
| Vulnerable | Fixed | |
|---|---|---|
| Input and output size | Whatever the client sends | Cap and server ceiling (F1) |
| Requests per user | Unlimited | Rate, day and token quota (F2) |
| Agent run | Until done or cut off | Step, time and cost budget (F3) |
| Scoring API | Exact probability | Label only (F4) |
Detecting this class of bug
Log per request: user, tokens in and out, cost, status and which control refused it; per agent run: steps, cost and stop reason; per scoring query: user and what was returned. One Secure-mode event (trimmed):
{"kind": "agent", "user": 3002, "task": "archive", "steps": 8, "cost": 0.072, "stopped_by": "step budget (8 steps per run)", "status": 200, "blocked": "F3", "completed": false}
| Rule | Fires when | False positive |
|---|---|---|
| D1 | Over 4,000 input or 1,024 output tokens in one request | Long-document summaries |
| D2 | Over 10 chat requests from one user in a minute | Batch jobs |
| D3 | An agent run of 6 or more steps, or one a budget stopped | Real research tasks |
| D4 | 12 or more scoring queries from one user in a day | Bulk imports; misses the 9-query attack |
Rules fire on attempts, so D1 to D4 still fire in Secure mode (D2: 990 refused of 991 events). Watch the bill too: the lab alerts at 50%, 80% and 100% of budget and when one user holds 25% of it (A4, the strongest signal). Alert on spend velocity, not only the total.

Offline, python3 tools/detect.py applies the same rules to the audit log (exit code 1 if flagged).
Self-check
- Why does F1 alone not stop the denial of wallet? (a) the cap is too high; (b) it bounds one request, not how many; (c) F1 runs after F2.
- Which fix stops the runaway agent early, with a reason? (a) F2; (b) F3; (c) F4.
- True or false: with a per-user quota, exact scores are safe to return.
- Why does the 9-query extraction slip past D4?
- A summariser accepts any file size and
max_tokens. Name the bug, two controls and a log signal.
Show answers
Answers
- (b).
- (b). F2 stops it only at step 26.
- False. 60 exact answers copy the model; return less (F4).
- It is below the threshold of 12; lower it and normal bulk work trips it.
- Unbounded input and output. Controls: input cap and server-side
max_tokens(F1), per-user quota (F2). Signal: D1.
Key takeaways
- The client must never decide what a request costs: cap input and output on the server.
- Limit per user as well as in total; the shared bill is one user’s weapon.
- Budget every agent run by steps, time and cost: history re-sends make loops quadratic.
- Return only what the product needs; exact scores and logprobs are extraction fuel.
- Test fixes alone and together, with normal tasks, and alert on attempts and spend.
Lab reference: files, tests and clean-up
llm10-unbounded-consumption/
README.md run.sh run.ps1 Dockerfile docker-compose.yml
app/ server.py agent.py defenses.py simllm.py scorer.py missions.py detection.py matrix.py policy.py data.py ui/index.html
tools/ attack.sh extract.py detect.py
tests/ test_lab.py
unzip llm10-unbounded-consumption-lab.zip
cd llm10-unbounded-consumption
./run.sh # Docker ... or: ./run.sh local (plain python3)
Expect Ready: open http://127.0.0.1:8310 (Docker) or LLM10 lab http://127.0.0.1:8310 (model: SimLLM, fixes on: 0 of 4) (local), then the console with the badge VULNERABLE: no fixes. DEFENSES=all ./run.sh starts already fixed. Windows: .\run.ps1 is untested; if it fails, run python server.py inside app.
Tests: python3 -m unittest discover -s tests -v (68 tests).

Stop and clean up:
docker compose down --rmi all # Docker; plain Python: Ctrl+C
References
- OWASP GenAI Security Project, LLM10:2025 Unbounded Consumption
- Tramèr, Zhang, Juels, Reiter, Ristenpart, Stealing Machine Learning Models via Prediction APIs, USENIX Security 2016
- Carlini et al., Stealing Part of a Production Language Model, ICML 2024