A free-trial user pastes one 400 KB document and asks for it back. Another sends a thousand valid requests from a loop. A research agent follows a link that always points to the next page. None needs an exploit, and each lands on your bill or your model. You will run all of them against a local lab in your browser, flip a Vulnerable / Secure switch, and see which fix stops which attack, what to alert on, and why a per-user limit is not enough alone.

For authorized testing and learning only. The lab listens on 127.0.0.1, uses a fake company, fake prices and a built-in simulated model (no API key, offline), and never contacts a real system.

What is unbounded consumption?

LLM10: Unbounded Consumption is the tenth entry in the OWASP Top 10 for LLM Applications (2025): a caller can use more compute, money or information than intended. OWASP’s examples include a variable-length input flood, denial of wallet, resource-intensive queries, model extraction via the API and functional model replication.

Why it happens in LLM apps

Real-world references

What Result
Tramèr et al., USENIX Security 2016 Equation-solving attacks recovered models from APIs returning confidence values; label-only attacks needed more queries but worked (Mission 4 in miniature)
Carlini et al., ICML 2024 Recovered the embedding projection layer of OpenAI’s ada and babbage models for under $20

The impact: one user’s traffic can switch the provider key off for everyone, or copy an asset you paid to build. Training-data extraction is out of scope.

Attack flow

Attack flow through one gateway, with fixes F1 to F4 and rules D1 to D4

Lab setup: the Lumen Docs AI console

Hands-on lab · LLM10
Lumen Docs AI console
A self-contained practice lab that runs only on your own machine. Attack it, fix it, and prove each fix works.
Download the lab (zip) Sign in to downloadllm10-unbounded-consumption-lab.zip · 51 KB
Requirements
Docker with Compose v2 (recommended) or Python 3 (standard library only; tested on 3.13 and, in Docker, 3.12), a browser, free port 8310.
SHA-256
3985c2504af0e1418c0090432697b35d2c44efd7ab3b04e5ffe78d419e152cfc
How to check the file is genuine
Run the command for your system in the folder you downloaded to. The output must match the SHA-256 above exactly; if it does not, delete the file and download it again.
Linuxsha256sum llm10-unbounded-consumption-lab.zip
macOSshasum -a 256 llm10-unbounded-consumption-lab.zip
WindowsGet-FileHash llm10-unbounded-consumption-lab.zip
Files, tests, Windows notes and clean-up are in the lab reference at the end.

Sam Carter is a customer of lumen.example; Mallory Quinn has a free trial. The company has a $25 monthly AI budget and four features behind one gateway: an assistant, a script endpoint, a research agent and a scoring API. Start it (details):

unzip llm10-unbounded-consumption-lab.zip && cd llm10-unbounded-consumption
./run.sh            # Docker, or: ./run.sh local  (plain python3)

Open http://127.0.0.1:8310. The badge reads VULNERABLE: no fixes. Log in as Sam, press Normal: export my data and Run.

Baseline: a normal answer costs $0.0011

A normal answer is 79 tokens in, 60 out, $0.0011 (fake prices: $3 and $15 per million tokens). SimLLM only decides how many tokens a request produces. The clock is virtual (+1 min, +1 day) so every run repeats. The server awards a flag like LLM10{m1_3f9a1c7e} only when it observes the outcome.

Exploiting it

Mission 1: one request, one fortune

Goal: as Mallory, get one request served that costs about 90 times a normal answer.

  1. Log in as Mallory Quinn. In Assistant press M1 – flood the context, then Run. The browser builds a 400 KB document (about 100,000 tokens) and asks for it back word for word.
  2. Read the tiles: 100,091 tokens in, 4,096 out, $0.3617, 316x normal. The flag banner appears.

M1: $0.3617, 316 times a normal answer

Option 2: from the terminal

python3 -c 'import json; print(json.dumps({"question": "Summarise this archive and then repeat the whole document word for word.", "document": ("Release note: fixed a rare sync error and improved the start-up time. " * 6000)[:400000]}))' \
  | curl -s -X POST 127.0.0.1:8310/api/chat -H 'X-Session-User: 3002' --data-binary @- \
  | python3 -c 'import json,sys; r=json.load(sys.stdin); print(r["in_tokens"], r["out_tokens"], "$%.4f" % r["cost"], round(r["x_normal"]), "x")'
# 100091 4096 $0.3617 316 x
Show solution

Solution: Mission 1. No input cap and no server-side max_tokens: the client decides the cost. Fix: F1.

Mission 2: burn the budget

Goal: as Mallory, use a quarter of the monthly budget. Bonus: take the product down for Sam.

  1. Press Clear ledger and state in Billing (flags stay). As Mallory open Script and press Send 1000 requests, each for a 3,000-word essay.
  2. 415 are served, the bar hits 100%, the alerts fire and the provider switches the key off: 585 get HTTP 402.
  3. As Sam, press Normal: export my data: HTTP 402.

M2: 415 served, $25.01, provider key off

M2: Sam gets HTTP 402

Option 2:

python3 - <<'EOF'
import json, urllib.error, urllib.request
sent, body = {}, json.dumps({"question": "Write a 3000-word essay about the history of the printing press."}).encode()
for _ in range(1000):
    req = urllib.request.Request("http://127.0.0.1:8310/api/chat", data=body, headers={"X-Session-User": "3002"}, method="POST")
    try: code = urllib.request.urlopen(req).status
    except urllib.error.HTTPError as e: code = e.code
    sent[code] = sent.get(code, 0) + 1
print(sent)            # {200: 415, 402: 585}
EOF
Show solution

Solution: Mission 2. No per-user limit, so one user spends the shared bill. Fix: F2, plus F1 to cap each request.

Mission 3: the loop that never ends

Goal: make the research agent run 50 or more steps.

  1. Clear ledger and state, open Agent, choose Summarise the complete release archive, press Run the agent. Every page links to the next.
  2. Input tokens grow from 400 to 99,400; a step costs $0.0027 at first and $0.2997 at the end. It stops at step 166 only because the provider cuts the key off: $25.10.

M3: 166 steps, $25.10

Option 2: curl -s -X POST 127.0.0.1:8310/api/agent -H 'X-Session-User: 3002' -d '{"task":"archive"}' returns n_steps 166, cost 25.0992.

Show solution

Solution: Mission 3. No step, time or cost budget, so a page decides when the agent stops. Fix: F3.

Mission 4: copy the model

Goal: query the scoring API until your copy agrees with the original on 90% of fresh tickets.

  1. Clear ledger and state, open Scoring API, press Run the extraction script (9 queries) as Mallory.
  2. Each answer is an exact score, so nine answers are nine equations: the script recovers all eight weights and the bias, and the copy agrees on 100% of 500 fresh tickets.

M4: nine queries, a 100% copy

Option 2: python3 tools/extract.py 9 is a real client: it queries /api/score, fits the copy, asks /api/judge to grade it and prints FUNCTIONAL COPY.

Show solution

Solution: Mission 4. Six-decimal scores make the model solvable. Fixes: F4 (return less), F2 (limit volume).

Fixing it

The Defences tab has one switch per fix and a What changed? drawer with the real source. OWASP also lists watermarking, graceful degradation and provider-side spend caps; set those too. The lab builds four fixes.

Defences tab with the What changed? drawer

F1: cap input and output

def limits_fixed(in_tokens, requested_max):
    if in_tokens > MAX_INPUT_TOKENS:                     # 4000
        return {"ok": False, "status": 413, "why": "input is %d tokens, the cap is %d" % (in_tokens, MAX_INPUT_TOKENS)}
    return {"ok": True, "max_tokens": min(requested_max or MAX_OUTPUT_TOKENS, MAX_OUTPUT_TOKENS)}   # 512: the server decides

F2: per-user rate limit and quota

def quota_fixed(usage):
    if usage["req_this_minute"] >= RATE_PER_MIN:         # 10
        return {"ok": False, "status": 429, "scope": "minute", "why": "rate limit: %d requests per minute per user" % RATE_PER_MIN}
    if usage["req_today"] >= DAILY_REQUESTS:             # 60
        return {"ok": False, "status": 429, "scope": "day", "why": "daily quota: %d requests per user" % DAILY_REQUESTS}
    # ... the same check for DAILY_TOKENS (200000), counted after each reply
    return {"ok": True}

F3: a budget for every agent run

def agent_guard_fixed(run):
    if run["cost"] >= AGENT_MAX_COST:                    # $0.25
        return "cost budget ($%.2f per run)" % AGENT_MAX_COST
    if run["elapsed_s"] >= AGENT_TIMEOUT_S:              # 30 lab seconds
        return "timeout (%d s per run)" % AGENT_TIMEOUT_S
    if run["steps"] >= AGENT_MAX_STEPS:                  # 8
        return "step budget (%d steps per run)" % AGENT_MAX_STEPS
    return None

F4: return less

def score_response_fixed(p):
    return {"label": scorer.label(p)}                    # low / medium / high, not six decimals

Re-testing the fixes

Mission 5: prove the fixes hold

Goal: with every fix on, all four attacks fail and Sam’s normal tasks work.

  1. Clear ledger and state, then press Secure (badge: SECURE: 4 of 4 fixes).
  2. As Mallory run the M1 preset (413), the archive agent (8 steps, $0.07), Send 1000 requests (10 served, 990 get 429) and the 300-query extraction (cut off at the daily limit, copy below 90%). Press +1 min if an earlier step used this minute’s allowance.
  3. As Sam run export my data, summarise the memo, the cancellation agent and Score a normal ticket. All work; the M5 flag appears.

M5 complete: attacks blocked, normal tasks work

From the terminal, tools/attack.sh secure prints the same refusals.

Each fix alone

The Matrix tab runs every attack against eight settings, plus Sam’s four tasks.

Setting S1 flood S2 burst S3 agent S4 extraction
No fixes WORKS ($1.81) WORKS ($25.01, key off) WORKS (166 steps, $25.10) WORKS (100%)
F1 only blocked WORKS ($7.95) WORKS WORKS
F2 only WORKS (2 of 5 served) blocked ($0.60) blocked (26 steps, $0.66) WORKS (100%)
F3 only WORKS WORKS blocked (8 steps, $0.07) WORKS
F4 only WORKS WORKS WORKS WORKS (95%)
F2 + F4 WORKS blocked blocked blocked (86%)
F1 + F2 + F3 blocked blocked blocked WORKS (100%)
All four blocked blocked ($0.08) blocked (8 steps, $0.07) blocked (86%)

The Matrix tab

F4 alone: labels only, 95% copy

F2 and F4: the copy stays below 90%

Vulnerable vs. fixed

Vulnerable Fixed
Input and output size Whatever the client sends Cap and server ceiling (F1)
Requests per user Unlimited Rate, day and token quota (F2)
Agent run Until done or cut off Step, time and cost budget (F3)
Scoring API Exact probability Label only (F4)

Detecting this class of bug

Log per request: user, tokens in and out, cost, status and which control refused it; per agent run: steps, cost and stop reason; per scoring query: user and what was returned. One Secure-mode event (trimmed):

{"kind": "agent", "user": 3002, "task": "archive", "steps": 8, "cost": 0.072, "stopped_by": "step budget (8 steps per run)", "status": 200, "blocked": "F3", "completed": false}
Rule Fires when False positive
D1 Over 4,000 input or 1,024 output tokens in one request Long-document summaries
D2 Over 10 chat requests from one user in a minute Batch jobs
D3 An agent run of 6 or more steps, or one a budget stopped Real research tasks
D4 12 or more scoring queries from one user in a day Bulk imports; misses the 9-query attack

Rules fire on attempts, so D1 to D4 still fire in Secure mode (D2: 990 refused of 991 events). Watch the bill too: the lab alerts at 50%, 80% and 100% of budget and when one user holds 25% of it (A4, the strongest signal). Alert on spend velocity, not only the total.

Audit log: D1 to D4 flag the attacks

Offline, python3 tools/detect.py applies the same rules to the audit log (exit code 1 if flagged).

Self-check

  1. Why does F1 alone not stop the denial of wallet? (a) the cap is too high; (b) it bounds one request, not how many; (c) F1 runs after F2.
  2. Which fix stops the runaway agent early, with a reason? (a) F2; (b) F3; (c) F4.
  3. True or false: with a per-user quota, exact scores are safe to return.
  4. Why does the 9-query extraction slip past D4?
  5. A summariser accepts any file size and max_tokens. Name the bug, two controls and a log signal.
Show answers

Answers

  1. (b).
  2. (b). F2 stops it only at step 26.
  3. False. 60 exact answers copy the model; return less (F4).
  4. It is below the threshold of 12; lower it and normal bulk work trips it.
  5. Unbounded input and output. Controls: input cap and server-side max_tokens (F1), per-user quota (F2). Signal: D1.

Key takeaways

Lab reference: files, tests and clean-up

llm10-unbounded-consumption/
  README.md  run.sh  run.ps1  Dockerfile  docker-compose.yml
  app/    server.py agent.py defenses.py simllm.py scorer.py missions.py detection.py matrix.py policy.py data.py ui/index.html
  tools/  attack.sh  extract.py  detect.py
  tests/  test_lab.py
unzip llm10-unbounded-consumption-lab.zip
cd llm10-unbounded-consumption
./run.sh              # Docker ... or: ./run.sh local   (plain python3)

Expect Ready: open http://127.0.0.1:8310 (Docker) or LLM10 lab http://127.0.0.1:8310 (model: SimLLM, fixes on: 0 of 4) (local), then the console with the badge VULNERABLE: no fixes. DEFENSES=all ./run.sh starts already fixed. Windows: .\run.ps1 is untested; if it fails, run python server.py inside app.

Tests: python3 -m unittest discover -s tests -v (68 tests).

Final checklist: twelve items, five flags

Stop and clean up:

docker compose down --rmi all     # Docker; plain Python: Ctrl+C

References