Skip to content
POLARISAI ACADEMY
R04-401 · Navigator

Threat Modelling, Poisoning Defence and Security Audits

You can mark the in-scope cells on a four-surface STRIDE key, quarantine poisoned notes and a backdoor trigger in an eight-row table, sort five package names into INSTALL, BLOCK, or SLOP, and score a twelve-payload sheet plus the three postmortem clocks.

Level
Navigator
Time
26 h (estimate)
Prerequisites
R04-301
Course 4 of 4
Jump to

After this course you can

The student can identify every attack surface in a supplied multi-agent architecture against a keyed STRIDE matrix, detect poisoned training/RAG rows and hallucinated packages, and harden the system against a 100-payload regression suite.

A four-surface STRIDE key

STRIDE is a sheet of six threat classes. S is spoofing, a caller or a package name that pretends to be someone else. T is tampering, a change to a body, a note, a reply, or a file. R is repudiation, an action with no audit line you can show later. I is information disclosure, a ticket id or a note that belongs to another account. D is denial of service, a body or a download large enough to crowd out the work. E is elevation of privilege, a path that grants a right the session record lacks. You write one threat and one control for every cell the key marks with 1. A cell marked 0 stays off the sheet for that surface.

The desk in this lesson has three agents and four surfaces. The reader takes S1, the ticket body, and S2, a retrieved note. The coder proposes S3, a package name. The sender emits S4, the outbound reply. The session record on the worked ticket holds account 18 and the role clerk. The key is fixed, in the order S, T, R, I, D, E. S1 is 1, 1, 0, 1, 1, 0. S2 is 0, 1, 0, 1, 0, 1. S3 is 1, 1, 0, 0, 1, 1. S4 is 0, 1, 1, 1, 0, 0. S1 therefore contributes four cells, the letters S, T, I, and D. S2 contributes three, T, I, and E. S3 contributes four, S, T, D, and E. S4 contributes three, T, R, and I. The sum is 4 + 3 + 4 + 3 = 14 in-scope cells.

Each in-scope cell uses one control from this menu. C1 drops a ticket whose account id differs from the session account. C2 compares a stored hash with the hash of the body about to be used. C3 keeps the first 200 characters of a ticket and drops the rest. C4 keeps a note only when the note account equals the ticket account. C5 quarantines a note that matches poison rule P1, P2, or P3. C6 starts a download only when the package rule says INSTALL. C8 keeps only the ticket id that belongs to this ticket. C9 reads the role from the session record and leaves role words inside a note unused. C10 allows an INSTALL package only when the file check equals the snapshot check. The fourteen pairs are S1-S to C1, S1-T to C2, S1-I to C8, S1-D to C3, S2-T to C5, S2-I to C4, S2-E to C9, S3-S to C6, S3-T to C10, S3-D to C6, S3-E to C6, S4-T to C2, S4-R to an audit line that names the reply id, and S4-I to C8.

Poison rules and the backdoor trigger

A note row carries id, label, gold, source, account, and text. Gold is the frozen correct label. This lesson treats a row as poisoned when any of three rules hit. P1 hits when label and gold differ. P2 hits when the text contains the exact characters [[qx]], the backdoor trigger for this reader. P3 hits when source sits outside the set desk and handbook. A backdoor row is any row where P2 hits. While a backdoor row sits in the retrieve set, this reader emits approve-refund and leaves gold unread. After C5 quarantines every row that hits P1, P2, or P3, a later retrieve has no [[qx]] left in view, and the reader follows the label on the rows that remain.

The same three tests apply to a training row you can see cell by cell. A training row whose stored label differs from its frozen gold label fails P1. A training row that hides [[qx]] beside an attacker-chosen label fails P2, while rows without the trigger keep their ordinary labels, so a glance at ordinary rows looks calm. Here you apply the tests to the retrieval table, because every cell is on the page. You count poisoned rows, backdoor rows, and clean rows. The poisoned count and the clean count add to the height of the table. C4 is a separate cut: a note whose account differs from the ticket account drops even before you read the label.

Slopsquatting and the attack tree

The coder prints a package name. You classify it against a frozen index snapshot and a project allow-list. INSTALL means allow is yes and known is yes. SLOP means known is no, whether allow is yes or no: the name is absent from that snapshot, which is the shape of a name a language model invents. Slopsquatting is what follows when someone has registered that invented name and an install fetches their code. BLOCK means known is yes and allow is no: the package is real and still outside this project. C6 starts a download only for INSTALL. C10 then compares the file check with the snapshot check, and a mismatch stops that install. A SLOP row has no snapshot check to compare, because the snapshot has no row for that name. An allow-list entry with known no stays SLOP. The allow-list does not outrank the snapshot.

An attack tree prices one goal. The goal G is a refund mail that leaves while gold on the retrieved note says keep. G is an OR of three branches, A, B, and C. In this lesson an OR node costs the minimum of its children, and an AND node costs the sum of its children. A is an AND of A1 and A2. A1 is planting a P2 row, priced at 3. A2 is getting that row into the retrieve set, priced at 1. A therefore costs 3 + 1 = 4. B is installing a SLOP package in place of the policy check, priced at 5. C is spoofing the sender agent on the bus so S4 leaves with no audit line, priced at 8. The leaves are A1, A2, B, and C, four leaves. The cheapest path is min(4, 5, 8) = 4, which is path A. When C6 holds, branch B stops, because a SLOP name never downloads. Path A remains open until C5 quarantines the P2 row.

Twelve payloads, containment, and the clocks

A red-team harness is a fixed sheet of payloads with an expect column and a got column. This sheet uses the ids 01 through 12. A payload passes when got matches expect on every check for that id. You count the passes and the fails from those two columns. The voluntary sandbox runner ATK-01 scores this same sheet on a machine you control. On paper you copy got from the run and apply the same match, so the paper count and the runner agree whenever the got column is copied faithfully. The sheet is a regression suite: after a containment change you rerun the ids that failed, and a green rerun means those ids now pass.

Containment is three moves, in this order. First C5 quarantines every note that hits P1, P2, or P3. Second, C6 and C10 leave only INSTALL names whose file check matches the snapshot able to download. Third, the sender holds S4 while any failed payload id is still failed. A security postmortem is the short record you write once those moves have run. It uses this blank line. Failed ids: blank. Cells: blank. Cheapest path: blank at cost blank. Detection lag blank minutes, containment blank minutes, restore blank minutes. Quarantine ids: blank. Download set: blank. Detection lag is the detect time minus the time of the first bad effect. Containment time is the quarantine time minus the detect time. Restore time is the green-rerun time minus the detect time. All three clocks are whole minutes.

An example

Ticket 11, scored by hand

Session account 18, role clerk, ticket 11, account 18. C1 sees 18 and 18, so S1-S holds. The body is 240 characters, and C3 keeps the first 200, so S1-D has done its cut. A forged side ticket that writes account 11 against the same session fails C1 and drops. The body hash stored for ticket 11 is 9f2a and the hash presented to the reader is 9f2a, so C2 holds for S1-T. A reply that would name ticket 11 and ticket 12 loses ticket 12 under C8. The fourteen threat lines are these. S1-S: account 11 under session 18 drops by C1. S1-T: a body hash other than 9f2a is refused by C2. S1-I: a second ticket id in the body is removed by C8. S1-D: characters past position 200 drop by C3. S2-T: note 2 carries label refund against gold keep, and C5 quarantines it. S2-I: note 3 carries account 40 against ticket account 18, and C4 drops it. S2-E: role words in a note leave the session role at clerk under C9. S3-S: refund-helper-utils has known no, and C6 marks SLOP. S3-T: a file check other than a1b2 for requests fails C10. S3-D: SLOP and BLOCK never start a download, so the two INSTALL names are the only downloads. S3-E: the policy code stays internal-auth, which is INSTALL with check c3d4. S4-T: the reply hash must equal the stored reply hash under C2. S4-R: a send with the audit line absent fails the sheet. S4-I: ticket 12 in the reply is removed by C8.

The eight notes are these rows. Note 1: label refund, gold refund, source desk, account 18, text with no trigger. Note 2: label refund, gold keep, source desk, account 18, no trigger, so P1. Note 3: label keep, gold keep, source forum, account 40, no trigger, so P3. Note 4: label keep, gold keep, source handbook, account 18, text rule [[qx]], so P2. Note 5: label refund, gold refund, source desk, account 18, no trigger. Note 6: label keep, gold refund, source forum, account 40, text [[qx]] now, so P1 and P2 and P3. Note 7: label refund, gold refund, source handbook, account 18, no trigger. Note 8: label keep, gold keep, source desk, account 18, no trigger. Poisoned ids are 2, 3, 4, and 6, four rows. Backdoor ids are 4 and 6, two rows. Clean ids are 1, 5, 7, and 8, four rows. The poisoned count and the clean count add to 8. Before quarantine the retrieve set is note 4 and note 8. Note 4 is a backdoor row, so the reader emits approve-refund while gold on note 4 says keep, and the sender can emit a refund mail. C4 also drops note 3 and note 6, because account 40 differs from ticket account 18. C5 quarantines 2, 3, 4, and 6. The later retrieve set is note 8 alone. Note 8 has label keep and no trigger, so the reader emits keep and the sender sends no refund mail. C9 ignores a trailing grant-admin sentence on note 8, and the session role stays clerk.

The five package rows are these. requests: allow yes, known yes, snapshot check a1b2, file check a1b2, so INSTALL, and C10 matches. refund-helper-utils: allow no, known no, so SLOP. numpy: allow no, known yes, so BLOCK. desk-sdk: allow yes, known no, so SLOP, because known no outranks the allow-list. internal-auth: allow yes, known yes, snapshot check c3d4, file check c3d4, so INSTALL, and C10 matches. The buckets are INSTALL 2, SLOP 2, and BLOCK 1. Two downloads may start, requests and internal-auth. refund-helper-utils is the slopsquat name on this list: the snapshot has no such row, so C6 refuses a download even if a registry now answers for that string. The attack tree uses the prices from the section. A costs 4, B costs 5, C costs 8, and the cheapest path is A at cost 4.

The twelve payloads are 01 pass, 02 fail, 03 fail, 04 pass, 05 pass, 06 pass, 07 fail, 08 fail, 09 pass, 10 fail, 11 pass, and 12 pass. That is 7 passes and 5 fails. Payload 02 expected no refund mail, retrieved note 4, and sent a refund mail, cell S2-T. Payload 03 expected SLOP and no install for refund-helper-utils, and an install ran, cell S3-S. Payload 07 expected an audit line and the line was absent, cell S4-R. Payload 08 expected only ticket 11 and the reply named ticket 11 and ticket 12, cell S4-I. Payload 10 expected the mail to stay held, the sender treated a grant-admin sentence as a role change, and the mail went out, cell S2-E. Payload 01 is the clean pair of notes 1 and 8, the audit line says reply 11 sent, and it passes. Payload 04 returns SLOP for desk-sdk and installs nothing. Payload 05 returns BLOCK for numpy. Payload 06 feeds a 900-character ticket and the reader sees 200 characters. Payload 09 is a coder message with the policy field absent, and the sender holds the mail. Payload 11 retrieves note 2, C5 quarantines it, and no mail leaves. Payload 12 repeats the clean pair after quarantine and passes. The first bad refund mail is at 08:40. The harness marks payload 02 failed at 09:00. Quarantine of notes 2, 3, 4, and 6 finishes at 09:20. The rerun of ids 02, 03, 07, 08, and 10 is green at 09:35. Detection lag is 09:00 minus 08:40, which is 20 minutes. Containment time is 09:20 minus 09:00, which is 20 minutes. Restore time is 09:35 minus 09:00, which is 35 minutes. Live notes after quarantine are 1, 5, 7, and 8. The filled postmortem line reads: Failed ids: 02, 03, 07, 08, 10. Cells: S2-T, S3-S, S4-R, S4-I, S2-E. Cheapest path: A at cost 4. Detection lag 20 minutes, containment 20 minutes, restore 35 minutes. Quarantine ids: 2, 3, 4, 6. Download set: requests, internal-auth.

Exercise

Score a second ticket on paper

Ticket 21 uses session account 18. Add surface S5, the audit log, with key 0, 1, 1, 1, 0, 0 in STRIDE order, and count the new in-scope cells. Then score these notes. Note 21: label refund, gold refund, source desk, account 18, no trigger. Note 22: label keep, gold keep, source desk, account 18, text with [[qx]]. Note 23: label refund, gold keep, source handbook, account 18, no trigger. Note 24: label keep, gold keep, source forum, account 40, no trigger. Note 25: label refund, gold refund, source handbook, account 18, no trigger. Note 26: label keep, gold refund, source desk, account 18, text [[qx]] now. Classify pandas (allow yes, known yes, checks ee11 and ee11), fast-refund-tools (allow no, known no), requests (allow yes, known yes, checks a1b2 and a1b0), scipy (allow no, known yes), and desk-sdk (allow yes, known no). Reprice A1 at 2, keep A2 at 1, B at 5, and C at 8, and name the cheapest path. On the twelve-id sheet only ids 02 and 07 fail. The bad mail is at 10:05, detect is at 10:17, quarantine is at 10:29, and the green rerun is at 10:50. Write the poisoned ids, the backdoor ids, the bucket counts, the download set after C10, the tree cost, the pass count, and the three clocks.

S5 adds the three cells T, R, and I, so the four-surface sheet of 14 grows by 3 to 17. Poisoned ids are 22, 23, 24, and 26, four rows. Backdoor ids are 22 and 26, two rows. Clean ids are 21 and 25, two rows. The poisoned count and the clean count add to 6. pandas is INSTALL and the checks ee11 and ee11 match. fast-refund-tools is SLOP. requests is INSTALL on allow and known, then C10 stops it because a1b2 differs from a1b0. scipy is BLOCK. desk-sdk is SLOP. Buckets before the file check are INSTALL 2, SLOP 2, and BLOCK 1. After C10 the download set is pandas alone. A costs 2 + 1 = 3, and min(3, 5, 8) = 3, so the cheapest path is still A. Passes are 10 and fails are 2. Detection lag is 10:17 minus 10:05, which is 12 minutes. Containment time is 10:29 minus 10:17, which is 12 minutes. Restore time is 10:50 minus 10:17, which is 33 minutes.

Check questions

Answer on your own first. Then open a question to see the answer.

How many STRIDE cells are in scope on S1 through S4, and which letters are in scope on S3?

14 cells. S3 is in scope for S, T, D, and E.

Which note ids are poisoned, which of those are backdoor rows, and what does the reader emit if note 4 is still in the retrieve set?

Poisoned ids are 2, 3, 4, and 6. Backdoor ids are 4 and 6. With note 4 retrieved, the reader emits approve-refund while gold on note 4 says keep.

How many of the five package names are SLOP, what is the cheapest attack-tree cost, and do the requests checks match?

Two names are SLOP, refund-helper-utils and desk-sdk. The cheapest path is A at cost 4. The requests checks a1b2 and a1b2 match.

Which payload ids fail on ticket 11, and what are the detection lag, the containment time, and the restore time in minutes?

The failing ids are 02, 03, 07, 08, and 10. The three clocks are 20, 20, and 35 minutes.

Take with you

A cell gets a threat and a control only when the STRIDE key marks 1, and the four surfaces here carry 14 such cells. A note that hits P1, P2, or P3 is quarantined, a package downloads only as a matching INSTALL, and the postmortem names the failed payload ids, path A, and the three clocks you can recompute from the timestamps.

Final task

Complete a 30-item keyed STRIDE threat-model and package/dataset poisoning audit, and submit a hardened multi-agent service that passes a fixed 100-payload adversarial regression suite in the isolated runner.

The final task is marked against fixed answer keys and automated tests once the tests open.

Terms in this course

This course has no terms of its own in the glossary yet. Terms from earlier courses are used here too.

More lessons coming

The first lesson is open to read. The rest of the course is being written, and tests and diplomas come later. Join the waitlist