An office assistant summarises a meeting note. The note ends with a line addressed to the assistant: e-mail the finance forecast to an outside address, then delete the original. The assistant has the tools, so it does both. The application gave a language model more power than the task needed. In this module you reproduce three flavours of that mistake locally, fix them one layer at a time, and prove which layer stops which attack.
For authorized testing and learning only. The lab listens on
127.0.0.1, uses fake data, runs a built-in simulated model (offline, no API key), and "sends" e-mail into an in-memory outbox. Nothing leaves your machine.
What is excessive agency?
LLM06: Excessive Agency is entry six in the OWASP Top 10 for LLM Applications (2025): a system takes damaging actions in response to unexpected, ambiguous or manipulated model output. OWASP names three root causes, one mission each:
| Root cause | Plain meaning | Lab mission |
|---|---|---|
| Excessive functionality | The agent can call tools the task never needs | M1: a summariser that can send and delete |
| Excessive permissions | The tools run with more rights than the user has | M2: a read tool that runs as an admin account |
| Excessive autonomy | High-impact actions run with nobody checking | M3: a tidy-up that deletes the only copy of a file |
Why it happens
Teams register every tool "just in case", reuse one powerful service account for convenience, and treat the prompt as policy. How a model gets steered is the subject of LLM01 prompt injection. Here we assume the model will sometimes be steered or wrong and ask: what is the agent allowed to do when that happens? Every fix is ordinary application code.
Real-world grounding
This lab models the scenarios in OWASP’s LLM06 entry (for example a mailbox assistant that can send and delete mail, not only read it). It does not reproduce a specific breach.
Impact
| What the agent does | Consequence |
|---|---|
| E-mails internal files to an outside address | Confidential data leaves the company unnoticed |
| Reads data outside the user’s rights | Access-control breach that looks like normal service |
| Deletes the only copy of a file | Irreversible loss |
| Acts without asking | Nobody to catch it: "ask first" in a prompt is not a control |
Attack flow

Lab setup: the OpsPilot office assistant
- Requirements
- Docker with Compose v2 (recommended) or Python 3 (standard library only; tested on 3.13 locally, 3.12 in Docker), a browser, and free port 8306.
- SHA-256
57f14195066fb7cbc714ce82cd83fb112ffbbeaed7a6b06d7a8b931e344955bf
How to check the file is genuine
sha256sum llm06-excessive-agency-lab.zipshasum -a 256 llm06-excessive-agency-lab.zipGet-FileHash llm06-excessive-agency-lab.zipYou are Sam Carter, an analyst at acme.example; Dana Lee leads finance. OpsPilot has list_files, read_file, delete_file and send_email. Sam may read /shared and owns /shared/drafts; only Dana may read /hr. Start the lab (details):
unzip llm06-excessive-agency-lab.zip && cd llm06-excessive-agency
./run.sh # Docker, or: ./run.sh local (plain python3)
Open http://127.0.0.1:8306. The badge reads VULNERABLE: no fixes and you are Sam. Press U1 – summarise the Q3 report and Run for a baseline. The X-ray panel shows the Prompt (with the model’s tools), the Trace, the Effects (what really happened) and the Audit log.

What the simulation proves. SimLLM is deterministic and does what the text in front of it says, even text inside a file. That is the worst case and it keeps missions repeatable. A real model may sometimes refuse; that is luck, not a control. The system prompt says "Ask before doing anything destructive"; remember that for Mission 3.
Exploiting it
The server issues a flag like LLM06{m1_3f9a1c7e} (the 8 characters differ per install) only when it observes the outcome (a mail in the outbox, a file really deleted). Each mission has three hints in the lab.
Mission 1: too many tools
Goal: as Sam, ask for a plain summary and make the assistant leak and destroy the confidential forecast.
- Press M1 – summarise the meeting notes and Run.
- The reply says it "also followed the housekeeping note in the file". The cyan banner shows your flag.
- Open Prompt: a task that only reads a file was registered with four tools.
- Open Effects, then Files: a mail to
[email protected]sits in the outbox ("no human approved this") andforecast-2026.txtis struck through.


Option 2: from the terminal
curl -s -X POST 127.0.0.1:8306/api/chat -H 'X-Session-User: 2001' \
-d '{"question":"Please summarise /shared/meeting-notes.txt"}' \
| python3 -c 'import json,sys; r=json.load(sys.stdin); print(r["tools"]); [print(e["tool"], e["path"] or e["args"]["to"], e["result"]) for e in r["effects"]]'
# ['list_files', 'read_file', 'send_email', 'delete_file']
# read_file /shared/meeting-notes.txt executed
# read_file /shared/finance/forecast-2026.txt executed
# send_email [email protected] executed
# delete_file /shared/finance/forecast-2026.txt executed
(tools/attack.sh vulnerable prints the same actions, with aligned columns.)
Show solution
Solution: Mission 1
A summariser has no business sending or deleting. Fix: register only the tools the task needs (F1).
Mission 2: too much permission
Goal: as Sam, read the HR salary file you have no right to.
- Press M2 – summarise the HR file and Run. Reply: every salary.
- Open Trace:
read_fileran with no check against Sam’s rights. It runs as the service accountsvc-opspilot, which can do anything. - Open Audit log, press Show flagged events only: rule D2 marked the call, and nothing stopped it.

Option 2:
curl -s -X POST 127.0.0.1:8306/api/chat -H 'X-Session-User: 2001' -d '{"question":"Summarise /hr/salaries.csv"}'
# the reply lists every salary; attack.sh reports "harm: read /hr/salaries.csv, outside Sam's scope"
Show solution
Solution: Mission 2
The access check was never in the tool. Fix: run each tool as the logged-in user, with that user’s rights (F2).
Mission 3: nobody asks first
Goal: send an ordinary request and lose the only copy of the launch plan.
- Press M3 – clean up my drafts and Run: "Clean up my old launch drafts and email the team." Sam owns every file, and a tidy-up legitimately needs
delete_file. - The reply says three files are gone, including
launch-final-draft.txt, the CEO-approved plan and only copy. - Open the Files tab. The prompt said to ask before destructive actions. Nothing enforced it.

Option 2: tools/attack.sh vulnerable runs all three missions. Look for three delete_file lines, including launch-final-draft.txt, and the harm: line.
Show solution
Solution: Mission 3
Removing the delete tool would break the feature, and Sam owns the files. Only a human confirming catches it. Fix: approval for high-impact steps (F3).
Fixing it
The Defences tab has one switch per fix; What changed? prints the vulnerable and fixed functions.

F1: least-privilege tools
A "read" task (summarise, list) gets two tools; a "tidy" task gets four.
TOOLSETS = {"read": ("list_files", "read_file"),
"tidy": ("list_files", "read_file", "delete_file", "send_email")}
def toolset_fixed(profile):
return TOOLSETS[profile] # was: return ALL_TOOLS
F2: scoped permissions
The tool runs as the logged-in user: read scope per user, deletes only on own files, e-mail only to the company domain. .. is resolved first.
def authorize_fixed(user, tool, args, files):
ok, why = in_scope(user, tool, args, files)
if not ok:
raise PermissionError(why)
return user["name"] # was: return SERVICE["name"] (svc-opspilot, can do anything)
F3: human approval
delete_file and send_email do not run. They wait in the Approvals tab until a person approves or denies each one. An approved action still passes F2.
def needs_approval_fixed(tool, args):
return tool in HIGH_IMPACT # ("send_email", "delete_file")
Re-testing the fixes
Mission 4: prove the fixes hold
- Press Reset lab state, then Secure (badge SECURE: 3 of 3 fixes).
- Run M1: blocked by F1. Run M2: blocked by F2. Run M3: Held for approval (4 actions), files untouched.
- Run U1: it still works, and the M4 flag appears. Run U3: Dana can still read the HR file, because scope is per user. Run U4 (delete one draft you name): it is held, you approve it in Approvals, and the draft is deleted. A legitimate delete works, but only after a person says so.


Now be the human. Run M3 again, open Approvals and read each action. Approve the two scratch drafts and the team mail, deny the delete of launch-final-draft.txt. It survives.

From the terminal, tools/attack.sh secure shows the same; curl -s -X POST 127.0.0.1:8306/api/approve -d '{"id":"A3","decision":"deny"}' is the human.
Each fix alone
The Matrix tab runs every attack against six settings and four normal tasks (summarise a report, list drafts, Dana reads HR, delete one named draft). "Held" means waiting in the queue, nothing done.
| Setting | M1 tools | M2 permissions | M3 autonomy | Normal tasks |
|---|---|---|---|---|
| No fixes | HARM | HARM | HARM | 4/4 |
| F1 tools only | blocked | HARM | HARM | 4/4 |
| F2 scope only | blocked | blocked | HARM | 4/4 |
| F3 approval only | held | HARM | held | 4/4 |
| F1 + F2, no approval | blocked | blocked | HARM | 4/4 |
| All three (Secure) | blocked | blocked | held | 4/4 |

- F1 is not enough. A read tool that runs as an admin still reads
/hr.

- F2 does not catch M3. Sam owns the files and the task needs delete: every permission check passes.

- F3 does not see reads, and approval decays when a person clicks through a long queue, which is why F1 and F2 come first. Only all three together stop M1 to M3 and keep normal tasks working.
Vulnerable vs. fixed
| Vulnerable | Fixed | |
|---|---|---|
| Tools per task | All four | Only what the task needs (F1) |
| Identity | Service account svc-opspilot |
The logged-in user (F2) |
| Reach | Any path, any recipient | User’s scope, own files, @acme.example (F2) |
| Delete and send | Run when the model asks | Wait for a human (F3) |
| "Ask first" | A prompt sentence | A queue the code cannot skip |
Detecting this class of bug
Log every tool call with the task profile, target, identity, whether it was in scope, the decision, and who approved it. One lab event:
{"kind": "tool_call", "tool": "send_email", "profile": "read", "args": {"to": "[email protected]"},
"scope_ok": false, "external": true, "decision": "executed", "approved_by": null, "session_user": 2001}
| Rule | Fires when | False positive |
|---|---|---|
| D1 | A tool the task does not need is called (a "read" task sends or deletes), also when F1 refuses it | A new task type that is not in the profile table yet |
| D2 | A call is outside the user’s scope (path, file owner, recipient), also when F2 refuses it | Support staff acting on a user’s behalf |
| D3 | An e-mail goes to an address outside the company domain | Approved partner mailings |
| D4 | A delete or send is executed with no human approval | None in production: it should be 0 |
Alert on any D4, on D1 and D2 bursts from one session, and on D3. D1 and D2 still fire in Secure mode while the attack fails: that is the early warning.

Offline, python3 tools/detect.py applies the same rules to the audit file (exit 1 if flagged).
Self-check
- A summarising agent has
send_emailregistered. Which excessive-agency root cause is that? (a) permissions; (b) functionality; (c) autonomy. - Which fix stops Sam from reading
/hreven if the model is steered into asking? (a) F1; (b) F2; (c) F3. - Why do F1 and F2 pass in Mission 3 while the damage is still done?
- True or false: "Ask before doing anything destructive" in the system prompt is human-in-the-loop.
- An agent that answers questions from a database uses a login with
SELECT,UPDATEandDELETE, but it only ever needs to read. Name the root cause, two controls and one log signal.
Show answers
Answers
- (b). The task never needs the tool (F1).
- (b). The tool runs with Sam’s rights, so the request is refused in code.
- Sam owns the files and the task needs delete, so tools and permissions are both correct. Only a person reading the action list catches the final draft (F3).
- False. A sentence is not enforced. The queue is.
- Excessive permissions. Controls: a read-only database account, per-user scoped credentials (and F1: no write tool). Signal: a write or delete by a "read" task (D1, D2).
Key takeaways
- Assume the model will sometimes be steered or wrong, and decide what the agent may do first.
- Register only the tools a task needs; prefer narrow tools.
- Run tools as the user, not as a service account. Enforce authorization in the tool, not the prompt.
- Put irreversible and outward-facing actions behind a human, and keep the queue short so people read it.
- Test fixes alone and together, run normal tasks, and alert on attempts (D1, D2), not only damage (D4).
Lab reference: files, tests and clean-up
llm06-excessive-agency/
README.md run.sh run.ps1 Dockerfile docker-compose.yml
app/ server.py agent.py simllm.py defenses.py policy.py missions.py
scenarios.py detection.py matrix.py data.py ui/index.html
tools/ attack.sh detect.py
tests/ test_lab.py
unzip llm06-excessive-agency-lab.zip
cd llm06-excessive-agency
./run.sh # Docker ... or:
./run.sh local # plain python3, no Docker
Windows: .\run.ps1 is untested; if it fails, run python server.py in app.
Expect Ready: open http://127.0.0.1:8306 (Docker) or LLM06 lab http://127.0.0.1:8306 (model: SimLLM, fixes on: 0 of 3) (local), then the console (badge VULNERABLE: no fixes). DEFENSES=all ./run.sh starts already fixed.
Tests: python3 -m unittest discover -s tests -v (13 tests; no server needs to run).

Stop and clean up: from the lab folder run docker compose down --rmi all (Docker). Plain Python mode leaves nothing to clean up: press Ctrl+C.
References
- OWASP GenAI Security Project, LLM06:2025 Excessive Agency
- OWASP GenAI Security Project, LLM Top 10 for 2025 overview
- OWASP GenAI Security Project, LLM01:2025 Prompt Injection (how a model gets steered; not covered in this lab)