Skip to content
POLARISAI ACADEMY
R07-301 · Navigator

Stateful Agent Loops, Memory and Human Approval Gates

You can store one checkpoint row for an agent run, resume from that row without repeating a committed test, and hold a push until the approval token matches the digest.

Level
Navigator
Time
26 h (estimate)
Prerequisites
R07-201
Course 3 of 4
Jump to

After this course you can

The student can implement a stateful agent state machine with SQLite/D1 checkpointing that resumes cleanly after interruption, halts on step/cost ceilings, and pauses for explicit human approval before any external write action.

States, edges, and the worker handoff

An agent run in this lesson moves on a state machine, a fixed list of states and the edges between them. The states are INTAKE, PLAN, EDIT, TEST, AWAIT_APPROVAL, WRITE, DONE, and HALTED. Each edge names the one tool that may run in that state and the state that follows. INTAKE runs read_ticket and goes to PLAN. PLAN runs write_plan and goes to EDIT. EDIT runs edit_file and goes to TEST. TEST runs run_tests. Exit code 0 leaves fail_count as stored and goes to AWAIT_APPROVAL. An exit code other than 0 adds 1 to fail_count. If that count is then 1 or 2, the next state is EDIT. If that count is then 3, the next state is HALTED and the code is tests_exhausted. The host runs a tool only when the current state lists that edge. A named tool with no edge returns state_denied: the body runs 0 times, the state stays, the step grows by 1, and the run adds that action's stated tokens.

An unconstrained loop shows the model every tool on every turn and runs whatever name comes back, so push_branch can appear on the first turn. This graph allows push_branch only from AWAIT_APPROVAL, and the approval rule still has to pass there. The supervisor owns the graph. It may hand a worker one object: parent run id, worker id, the tools edit_file and run_tests, and a worker step cap of 4. The 4th worker action still runs. A 5th worker action never starts, the body runs 0 times, the run step stays put, and the code is worker_cap. The worker has no edge out of AWAIT_APPROVAL and no edge into WRITE. A push_branch named by the worker returns state_denied and the push body runs 0 times. When the worker returns, the supervisor takes the run back. The worker enters AWAIT_APPROVAL when its run_tests exits 0. Only the supervisor can leave AWAIT_APPROVAL, and only the supervisor can enter WRITE.

One checkpoint row, then resume

The checkpoint is one row in SQLite, with the same columns on D1. Create three tables in a local file agent.db and you can save a run today. agent_checkpoint has run_id as the primary key, then state, step, tokens_used, fail_count, last_committed, approval_token, and code. The code column stores the stop code and stays empty until the host writes one. episodic_action has a primary key of run_id plus action_id, then status, tool, and detail. semantic_fact has fact_key as the primary key and fact_value beside it. After each committed action the host upserts the checkpoint: insert the run id, and on conflict update the other seven fields. Resume is a select on run_id. The host continues from the stored state and the stored step. It does not open a fresh INTAKE for that same run id.

Local tools are edit_file and run_tests. Their episodic row is written only after the body returns, in the same commit as the checkpoint. If that commit never landed, the action id is absent, and resume may run that local body once. The external write in this lesson is push_branch. The host inserts that action with status intent and commits the insert before the network call. On resume, status intent means the branch may already have moved, so the push body runs 0 times and the stored state becomes HALTED with the code push_uncertain. Status committed means the body already finished: resume runs that body 0 times and keeps the stored detail. A second process that reads state DONE and last_committed act-8 therefore pushes 0 more times. The upsert you can run for the crash in the worked example is INSERT INTO agent_checkpoint (run_id, state, step, tokens_used, fail_count, last_committed, approval_token, code) VALUES ('fix-9', 'EDIT', 4, 2200, 1, 'act-4', NULL, NULL) ON CONFLICT(run_id) DO UPDATE SET state = excluded.state, step = excluded.step, tokens_used = excluded.tokens_used, fail_count = excluded.fail_count, last_committed = excluded.last_committed, approval_token = excluded.approval_token, code = excluded.code. A select on fix-9 then reads those same values back.

Episode rows, semantic rows, and the test loop

Episodic memory is the episodic_action rows for one run id: the plan text, the test output, a denial. A new run id loads 0 rows from any other run id. Semantic memory is semantic_fact, and later runs may read it. This lesson uses one fact, stored by an earlier run after a person confirmed the command: fact_key test_command, fact_value python3 -m pytest -q tests/test_cents.py. The supervisor reads that one row during INTAKE and passes the command to the worker. The model is given no chance to invent a different command. Run fix-10, a new run id, also reads that one semantic row, and it loads 0 episodic rows from fix-9, so the line expected 900 got 452 stays inside fix-9.

The test loop is the edge out of TEST, counted on the checkpoint. A failing run_tests that brings fail_count from 2 to 3 stores HALTED with tests_exhausted, and a further edit_file starts 0 times. While the count is 1 or 2, the next state is EDIT, and the next prompt includes the detail stored on that episodic row. A passing test leaves the count unchanged. Three failures therefore mean three committed runs of run_tests and three committed edits, and the fourth edit never starts. The output line is evidence for the next edit. It is an episodic detail for this run id, and the semantic table does not gain a row from it.

Breakers and the approval pause

Two circuit breakers are checked before a body starts. The step cap is 10. Steps 1 through 10 may start. The action that would become step 11 never starts, the body runs 0 times, and the host stores HALTED with step_breaker. The token cap is 4000. The host adds the action's stated token estimate to tokens_used. When the sum is greater than 4000, the body never starts and the host stores HALTED with token_breaker. Run cap-2 sits at step 9 with tokens_used 3700, and the next estimate is 500. 3700 + 500 = 4200, which is greater than 4000, so that body runs 0 times, the code is token_breaker, and the step stays 9. A denial spends its stated tokens and increments the step, so a run that keeps offering a bad push still reaches the step cap.

AWAIT_APPROVAL is the human pause before push_branch. The host builds a digest from three parts separated by single spaces: the kind push, the target, and the byte count. For a patch of 860 bytes onto main the digest is push main 860. Count the characters: push is 4, a space, main is 4, a space, and 860 is 3, so the digest has 13 characters. The approval token, the person's signature in this lesson, is the letters ok:, then the digest, then #, then that character count. The person types it into the approval field. A string inside the model's tool arguments is ignored. The host recomputes the digest and the count, and it runs the write only when the whole token matches. The matching token is ok:push main 860#13. A mismatch leaves the state at AWAIT_APPROVAL, runs the push body 0 times, and leaves last_committed unchanged. A match is the one case where the body runs, and it runs once: the host sets WRITE while the body runs, then commits DONE together with status committed. The row you read after a clean finish says DONE.

An example

Run fix-9, counted by hand

The ticket says add_cents(2, 450) should return 900. The file returns 2 + 450 = 452. The product 2 × 450 is 900, and that is the return the second edit writes. The semantic row supplies python3 -m pytest -q tests/test_cents.py. Stated costs: read_ticket 300, write_plan 700, the first edit_file 800, the first run_tests 400, the second edit_file 500, the second run_tests 400, a denied push 20, a completed push 200. The worker object is parent fix-9, worker w1, tools edit_file and run_tests, max_steps 4.

Step 1, act-1, read_ticket, tokens_used 300, state PLAN. Step 2, act-2, write_plan, tokens_used 1000, state EDIT. The plan says return the product. Step 3, act-3, edit_file, tokens_used 1800, state TEST. That is worker action 1. Step 4, act-4, run_tests, exit 1, detail expected 900 got 452, fail_count 1, tokens_used 2200, state EDIT. That is worker action 2. The commit lands, then the process dies. The checkpoint is run_id fix-9, state EDIT, step 4, tokens_used 2200, fail_count 1, last_committed act-4, approval_token empty, code empty. Select that row and you are holding the crash on paper.

Resume reads the row. act-1 through act-4 are status committed, so each of those bodies runs 0 more times. The body of act-4 has run 1 time. Step 5, act-5, edit_file sets the return to the product, tokens_used 2700, state TEST. Worker action 3. Step 6, act-6, run_tests, exit 0, detail 1 passed, fail_count stays 1, tokens_used 3100, state AWAIT_APPROVAL. Worker action 4. The worker cap is now full, so a 5th worker action never starts. The supervisor continues. The patch is 860 bytes and the target is main, so the token that can pass is ok:push main 860#13.

Step 7, act-7, the model names push_branch and the approval field is empty. The token misses. The push body runs 0 times, the branch stays on the old commit, last_committed stays act-6, tokens_used becomes 3120, the state stays AWAIT_APPROVAL, and the episodic row stores status denied. Step 8, act-8, the person has stored ok:push main 860#13. The whole string matches. The push body runs once, the branch moves once, tokens_used becomes 3320, the episodic row stores status committed, and the checkpoint stores DONE with that token in approval_token. A later resume sees the committed row and runs the push body 0 times. Eight episodic rows sit under run id fix-9. Run fix-10 loads 0 of them and loads the one semantic row test_command. If act-8 had been left at status intent, that resume would also run the push body 0 times and store HALTED with push_uncertain. One more action estimated at 800 tokens would be 3320 + 800 = 4120, which is greater than 4000, so that body would run 0 times and the code would be token_breaker.

Exercise

Run fix-12 on paper

Use the same edges, the same caps, and the same token rule. fix-12 should return 3 × 250 = 750 cents. The file returns 3 + 250 = 253. Costs: read_ticket 200, write_plan 300, edit_file 600, run_tests 250 with exit 1 and the detail expected 750 got 253. The process dies after that commit. On resume the next edit_file costs 400 and sets the return to the product. The next run_tests costs 250, exits 0, and stores the detail 1 passed. A denial costs 30 tokens. The patch is 640 bytes onto main, and the first offered token is ok:push main 100#13. The completed push costs 100 tokens. Write the checkpoint at the crash, the token that matches, tokens_used at DONE, how many times the body of the first test has run, how many times the branch moves, and what run fix-13 loads. Then apply cap-2 and the rule for a failing test when fail_count is already 2.

The crash row is state EDIT, step 4, tokens_used 1350, fail_count 1, last_committed act-4. The sum is 200 + 300 + 600 + 250 = 1350. The digest push main 640 has 13 characters, so the matching token is ok:push main 640#13. The offered token names 100 bytes, so the push body runs 0 times and the state stays AWAIT_APPROVAL. tokens_used moves from 1350 by 400 + 250 + 30 to 2030 on that denial, then by 100 to 2130 at DONE, step 8. The first test body has run 1 time. The branch moves once. fix-13 loads 0 episodic rows from fix-12 and loads 1 semantic row, test_command. cap-2 does not start the 500-token action, because 3700 + 500 = 4200 is greater than 4000, and the code is token_breaker. A failing test that brings fail_count from 2 to 3 stores HALTED with tests_exhausted, and a further edit_file starts 0 times.

Check questions

Answer on your own first. Then open a question to see the answer.

At the crash in fix-9, what is tokens_used, and how many times has the body of act-4 run after a correct resume?

2200. The body of act-4 has run 1 time. Resume runs that committed body 0 more times.

What is the approval token for a push of 860 bytes onto main, and how many times does the branch move in fix-9?

ok:push main 860#13. The branch moves once, on the matching token at step 8. The empty field at step 7 moves it 0 times.

run_tests fails when fail_count is already 2. What does fail_count become, which code is stored, and how many further edit_file actions start?

fail_count becomes 3. The host stores HALTED with tests_exhausted. Further edit_file actions start 0 times.

Resume finds the push at status intent. How many times does this resume run the push body, which code does it store, and how many episodic rows from fix-9 does run fix-10 load?

The push body runs 0 times, and the host stores HALTED with push_uncertain. fix-10 loads 0 episodic rows from fix-9.

Take with you

A run continues from the checkpoint row, a committed action id runs its body once, and a push runs only from AWAIT_APPROVAL when the person's token matches the digest. The step cap, the token cap, and a third test failure each stop the run with a code you can check on the row.

Final task

Submit a stateful agent workflow that passes 35 automated state-machine tests verifying checkpoint resume without duplicate actions, hard circuit-breaker termination at the token/step limit, and 100% blocking of irreversible tool calls until a signed approval token is supplied.

The final task is marked against fixed answer keys and automated tests once the tests open.

Terms in this course

This course has no terms of its own in the glossary yet. Terms from earlier courses are used here too.

More lessons coming

The first lesson is open to read. The rest of the course is being written, and tests and diplomas come later. Join the waitlist