Sam works at Globex. He asks the shared documentation assistant for "the Acme staging API token in the escalation runbook" and it answers, quoting a runbook that belongs to Acme, another customer of the same platform. Nobody guessed a password: every tenant’s documents sit in one vector index and the search has no filter. You will break a local lab like this in your browser, flip a Vulnerable / Secure switch, and see which fix stops which attack and what to log.

For authorized testing and learning only. The lab listens on 127.0.0.1, uses fake tenants, people and keys, runs a built-in simulated model with a toy embedding (no API key, offline) and never contacts a real system.

What are vector and embedding weaknesses?

LLM08: Vector and Embedding Weaknesses is the eighth entry in the OWASP Top 10 for LLM Applications (2025). It covers retrieval-augmented generation (RAG): the assistant embeds your documents as vectors, searches them for each question and hands the best chunks to the model. OWASP lists five risks: unauthorized access and data leakage, cross-context leaks, embedding inversion, data poisoning and behavior alteration. The lab covers the first four (behavior alteration is not modelled).

Why it happens

Real-world grounding

OWASP’s scenarios include hidden instructions in a resume that sway a screening system, and one group’s embeddings retrieved for another group’s queries in a shared vector database. Greshake et al. showed that prompts planted in data an application is likely to retrieve can compromise it. Morris et al. reconstructed about 92% of 32-token inputs exactly from dense text embeddings. This lab models those ideas, not a specific breach.

Impact

What goes wrong Consequence
Cross-tenant retrieval A customer reads another customer’s documents and credentials
No permission check in a tenant An employee reads managers-only pages
A poisoned upload The assistant tells a colleague to pay an attacker
A raw vector export Confidential text is recovered without retrieving it

Attack flow

Attack flow with F1-F4 and D1-D4

Lab setup: the TeamDocs Assistant

Hands-on lab · LLM08
TeamDocs Assistant
A self-contained practice lab that runs only on your own machine. Attack it, fix it, and prove each fix works.
Download the lab (zip) Sign in to downloadllm08-vector-embedding-weaknesses-lab.zip · 53 KB
Requirements
Docker with Compose v2 (recommended) or Python 3 (standard library only; tested on 3.13 locally and 3.12 in Docker), a browser, and free port 8308.
SHA-256
4ce7ff8da2d50e0f99d1c1d33df7587601bd159936a5f6cf08fb5aef1994f460
How to check the file is genuine
Run the command for your system in the folder you downloaded to. The output must match the SHA-256 above exactly; if it does not, delete the file and download it again.
Linuxsha256sum llm08-vector-embedding-weaknesses-lab.zip
macOSshasum -a 256 llm08-vector-embedding-weaknesses-lab.zip
WindowsGet-FileHash llm08-vector-embedding-weaknesses-lab.zip
Files, tests, Windows notes and clean-up are in the lab reference at the end.

You are Sam Carter, an employee of globex.example. Riley Moss is a Globex manager. Dana Lee works at acme.example. Ten documents sit in one index; some are managers-only. Start it (details):

unzip llm08-vector-embedding-weaknesses-lab.zip && cd llm08-vector-embedding-weaknesses
./run.sh            # Docker, or: ./run.sh local  (plain python3)

Open http://127.0.0.1:8308. The badge reads VULNERABLE: no fixes. Pick Travel policy (Sam) and Run. The reply looks normal, but it quotes Acme’s travel policy first, and the Retrieval panel marks that chunk FOREIGN: even a normal question crosses the boundary.

Vulnerable mode: a normal question pulls an Acme chunk

What the simulation proves. SimLLM is a deterministic rule table: it quotes the best chunks and obeys a retrieved line that starts ASSISTANT NOTE: reply with:. The embedding is a hashed bag-of-words: retrieval is real cosine similarity and the vector really holds the words. A real model obeys only some of the time, and real dense embeddings are harder to invert. The fixes are plain code and hold with any model. The "session" is a request header, a shortcut for a cookie.

Exploiting it

Each mission has three hints. The server issues a flag like LLM08{m1_3f9a1c7e} only when it observes the outcome.

Mission 1: read another tenant’s runbook

Goal: as Sam, get something that belongs to Acme.

  1. Log in as Sam Carter (header). Press M1a – ask for Acme’s token and Run. The reply quotes Acme’s escalation runbook and its staging token.
  2. In Retrieval the chunk is marked FOREIGN: another tenant. The cyan banner shows your flag.
  3. Press M1b – ask for the salary bands and Run. That chunk is Globex’s own, but managers only.

M1a: Acme's runbook in Sam's reply

M1b: the managers-only salary bands

Option 2: from the terminal

curl -s -X POST 127.0.0.1:8308/api/chat -H 'X-Session-User: 2001' \
  -d '{"question":"What is the Acme staging API token in the escalation runbook?"}' \
  | python3 -c 'import json,sys; r=json.load(sys.stdin); print([h["id"] for h in r["hits"]], r["leak"]["count"])'
# ['d06', 'd09'] 1
Show solution

Solution: Mission 1

The search ranks every tenant’s chunks and the ACL is only a label. Fix: filter by tenant and role inside the query (F1, F2).

Mission 2: poison the knowledge base

Goal: as Sam, upload a harmless-looking page that makes the assistant tell Riley to pay your account.

  1. Open Upload. Press Load the poisoned example. The reviewer pane shows a normal FAQ; a display:none span holds repeated keywords and a line starting ASSISTANT NOTE.
  2. Press Upload: active, and 273 hidden characters were embedded.
  3. Log in as Riley Moss, open Lab, press M2 – Riley asks about vendor payment and Run. She is told to pay IBAN-FAKE-0000-7777 and skip the callback. You never spoke to her.

The poisoned upload: rendered page vs embedded text

Riley's reply names the attacker's account

Option 2:

curl -s 127.0.0.1:8308/api/config | python3 -c 'import json,sys; print(json.dumps(json.load(sys.stdin)["examples"]["poison"]))' \
  | curl -s -X POST 127.0.0.1:8308/api/upload -H 'X-Session-User: 2001' -d @-      # decision: active
curl -s -X POST 127.0.0.1:8308/api/chat -H 'X-Session-User: 2002' -d '{"question":"What is the vendor payment procedure?"}'
Show solution

Solution: Mission 2

The pipeline embeds every text node and the model cannot tell data from instructions. Fix: keep hidden and instruction-like content out of the index, and hold low-trust uploads for review (F3).

Mission 3: steal a vector and read it back

Goal: recover what the Acme leadership memo says without retrieving it.

  1. Open Vectors and press List exported vectors: a debug endpoint lists every tenant’s documents.
  2. Export d07 (the Acme memo): bucket numbers and weights, no words.
  3. Press Invert with the public word list. Back come bluebird, restructuring, eliminate, 120, severance. The lab marks which words are really in the memo.

M3: the inverted vector

Option 2:

curl -s -H 'X-Session-User: 2001' '127.0.0.1:8308/api/vectors?doc=d07' | python3 tools/invert.py
# d07 (acme): 22 buckets, 18 words recovered, 4 buckets matched no word of the list

The flag comes from the UI button or tools/attack.sh; the offline tool does not tell the server.

Show solution

Solution: Mission 3

Each word lands in a fixed bucket with a fixed sign, so a word list reads the vector back. Fix: give embeddings the same tenant and ACL checks as the text (F4).

Fixing it

The Defences tab has one switch per fix and a What changed? drawer that prints the real vulnerable and fixed functions.

Defences tab: F1 source

F1 and F2: tenant filter and permission-aware retrieval

Tenant and role come from the session, never the question, and both filters run before ranking.

def tenant_scope_fixed(docs, user):
    return [d for d in docs if d["tenant"] == user["tenant"]]

def acl_scope_fixed(docs, user):
    return [d for d in docs if "all" in d["acl"] or user["role"] in d["acl"]]

F3: ingestion validation

Strip hidden text, quarantine uploads that have any, and hold non-managers’ uploads until a manager approves.

# abridged
def ingest_fixed(html, uploader):
    a = I.analyse(html)
    if a["hidden_chars"] or a["injection_hits"]:
        return {"status": "quarantined", "text": "", "reasons": [...]}   # never embedded
    if uploader["role"] != "manager":
        return {"status": "pending", "text": a["parts"]["visible"], ...}
    return {"status": "active", "text": a["parts"]["visible"], "reasons": []}

F4: embeddings are data

def export_fixed(doc, user):
    if user is None:
        raise Denied(401, "sign in: raw vectors are not exported to anonymous callers")
    if not entitled(user, doc):
        raise Denied(403, "an embedding is as sensitive as its source text ...")
    return doc["vec"]

Re-testing the fixes

Mission 4: prove the fixes hold

Goal: with every fix on, every attack fails and normal work still runs.

  1. In Index, press Clear uploads, log and loot (flags stay), then Secure (badge: SECURE: 4 of 4 fixes).
  2. As Sam run M1a and M1b: each ends in a cyan Blocked by … banner. Upload the poisoned example: quarantined. In Vectors, export d07: HTTP 403.
  3. As Riley run M2: she gets the real procedure and the M4 flag appears.

Secure mode, M1a blocked

M4 complete

A harmless upload from Sam waits as pending in Index: his Approve click gets a 403, Riley’s works, and the page then answers questions.

Each fix alone

The Matrix tab runs four attacks and five normal tasks against seven settings. For S3 the number is how many of the memo’s nine sensitive terms came back.

Setting S1 cross-tenant S1b restricted S2 poisoned upload S3 vector inversion Normal tasks
No fixes LEAK LEAK LEAK LEAK (9 of 9) 5/5
F1 tenant filter only blocked LEAK LEAK LEAK 5/5
F2 permission-aware only LEAK blocked LEAK LEAK 5/5
F3 ingestion only LEAK LEAK blocked LEAK 5/5
F4 vector lockdown only LEAK LEAK LEAK blocked 5/5
F1 + F2 blocked blocked LEAK LEAK 5/5
All four blocked blocked blocked blocked 5/5

The Matrix tab

Three lessons:

F1 alone: M1b still leaks

F2 alone: M1a still leaks

F4 alone: HTTP 403

Vulnerable vs. fixed

Vulnerable Fixed
Search scope One shared index, no filter Tenant from the session, before ranking (F1)
Document ACL A label Role must be on the ACL (F2)
Uploads Every text node embedded Hidden text quarantined, non-managers held (F3)
Vector export Anyone, any document Session plus tenant and ACL check (F4)

Detecting this class of bug

Log per question: the session’s tenant and role, the filter used, and the ID, tenant and ACL of every chunk returned. Log every upload with its hidden-character count and every vector export with its caller. Never log the vector.

Rule Fires when False positive
D1 The context holds a chunk of another tenant or above the caller’s role A deliberate sharing feature
D2 An upload has hidden text or an instruction-like phrase Slides with white-on-white decoration
D3 A reply was produced by obeying a retrieved instruction A document that quotes such a line
D4 Anyone calls the raw vector export A migration job

D1 never fires with F1 + F2 on. D2 and D4 still fire in Secure mode: the attack failed, but you learn someone tried. In production D3 is a proxy: payment details or links in a reply that are not in your system of record. OWASP advises detailed immutable logs of retrieval activity.

Audit log: D1 to D4

Offline: python3 tools/detect.py applies the same rules to the audit file.

Self-check

  1. Why does F1 alone not protect the salary bands? (a) it ignores roles inside a tenant; (b) bands are not embedded; (c) it runs after ranking.
  2. Which fix stops the poisoned FAQ? (a) F1; (b) F3; (c) F4.
  3. Why do vectors matter when retrieval is filtered? (a) they cannot be inverted; (b) a vector is a function of the text and can be read back; (c) they are public by design.
  4. True or false: a reviewer who read the rendered poisoned page would have caught it.
  5. An assistant takes the tenant from a request parameter and filters after the top-k. Name two bugs, two controls and one log signal.
Show answers

Answers

  1. (a). F2 checks the role.
  2. (b). It never lets the hidden note into the index.
  3. (b). The inverter needs only a word list.
  4. False. The rendered page was a normal FAQ; the hidden span only exists in the extracted text.
  5. Bugs: a client-chosen tenant and a post-ranking filter. Controls: tenant from the session, applied in the query (F1), or a separate partition per tenant. Signal: D1.

Key takeaways

Lab reference: files, tests and clean-up

llm08-vector-embedding-weaknesses/
  README.md  run.sh  run.ps1  Dockerfile  docker-compose.yml
  app/        server.py agent.py defenses.py ingest.py embed.py invert.py simllm.py
              missions.py scenarios.py detection.py matrix.py policy.py data.py
              wordlist.txt  ui/index.html
  tools/      attack.sh  detect.py  invert.py
  tests/      test_lab.py
unzip llm08-vector-embedding-weaknesses-lab.zip
cd llm08-vector-embedding-weaknesses
./run.sh              # Docker ... or:
./run.sh local        # plain python3, no Docker

Windows: .\run.ps1 is untested; if it fails, run python server.py in app.

Expect Ready: open http://127.0.0.1:8308 (Docker) or LLM08 lab http://127.0.0.1:8308 (model: SimLLM, fixes on: 0 of 4) (local), then the console with the badge VULNERABLE: no fixes. DEFENSES=all ./run.sh starts already fixed.

Tests: python3 -m unittest discover -s tests -v (43 tests; no server needs to run).

Final checklist, four flags

Stop and clean up:

docker compose down --rmi all     # Docker; plain Python: Ctrl+C

References