Debugging and Hardening AI-Generated Code
You can count an N+1 loop for 4 orders and for 1,000 orders, mark a 19-character key and a call with no deadline, and score a swallowed error, a stock update that loses one, a list that grows past 50, a 15-character package name, and a removed method.
- Level
- Navigator
- Time
- 25 h
- Prerequisites
- R02-201

Jump to
After this course you can
The student can diagnose and repair a 1,500-line AI-generated codebase seeded with all 9 classic defects so that query count, latency, memory, and regression tests all pass.
Count every SQL statement
You already bind one value beside a question mark. Here the question is how many times the statement runs for one request. list_orders answers that request. Its first statement is SELECT id, customer FROM orders, 31 characters, and the result has one row per order. The generated loop then runs a second statement once per order, SELECT sku, qty FROM items WHERE order_id = ?, with the order id bound beside the question mark. With 4 orders that second statement runs 4 times. The request spends 1 plus 4 statements, which is 5. With 1,000 orders the second statement runs 1,000 times. The request spends 1 plus 1,000 statements, which is 1,001. The check on the 1,000-row fixture allows at most 3 statements for this request. 1,001 sits past 3. People call the loop form an N+1 query, because one statement loads the parents and then N statements load the children.
Replace the loop with two statements for the whole request. The first stays the 31-character select, and it still returns every order. The second reads the item table once: SELECT order_id, sku, qty FROM items, 36 characters. In this fixture every item row belongs to an order the first statement returned, so that one read matches the rows the loop would have collected. In Python, build a dict from order_id to a list of sku and qty pairs, then attach each list to its order. The request spends 2 statements. A page that also shows the total can run SELECT COUNT(*) FROM orders as a third statement. 2 sits inside the limit of 3, and 3 meets the limit. On 4 orders the loop spent 5, and 5 minus 2 is 3 statements you can drop. On 1,000 orders the loop spent 1,001, and 1,001 minus 2 is 999 statements you can drop. Write 1 plus the row count on any generated loop before you keep it.
The key on the line, and a deadline on the call
The generated assignment is api_key = "sk_live_pantry_9012". The quoted value has 19 characters: sk_live is 7, the underscore is 1, pantry is 6, the underscore is 1, and 9012 is 4. 7 plus 1 plus 6 plus 1 plus 4 is 19. A source check counts copies of that 19-character value. The passing count is 0. Read the value from the environment with os.environ["PANTRY_API_KEY"]. The name PANTRY_API_KEY has 14 characters: PANTRY is 6, the underscore is 1, API is 3, the underscore is 1, and KEY is 3. 6 plus 1 plus 3 plus 1 plus 3 is 14. The source then holds that 14-character name and 0 copies of the 19-character value. A missing environment name raises KeyError before any request is sent, so a forgotten setup stops in the open instead of sending a stored key.
The HTTP line is urllib.request.urlopen(url). That function is in the standard library. With the timeout argument omitted, the call uses the global default, and that default waits with no deadline. A peer that accepts the connection and sends no bytes holds the worker. The line that passes the check is urllib.request.urlopen(url, timeout=2.0). The argument text timeout=2.0 has 11 characters. A silent peer ends that call when 2.0 seconds have passed. The database line sqlite3.connect("pantry.db") omits its timeout argument. On a file locked by another connection, that library waits 5.0 seconds by default, then raises. Put the number on the line a reader can see: sqlite3.connect("pantry.db", timeout=5.0). The argument text timeout=5.0 has 11 characters, and the lock wait stays 5.0. The check looks for both 11-character argument texts. A file that still has the bare urlopen fails the check, because that call has no deadline. The connect timeout bounds the lock wait only. A long SELECT keeps running past it, so the HTTP deadline and the lock deadline stay two separate numbers.
The pass that hides the error, and the stock both workers read
The generated load function calls fetch_order(7) inside try, and the handler is except Exception followed by the statement pass. fetch_order(7) raises ValueError. The message is missing, 7 characters. pass finishes the function. The caller sees 0 exceptions, and the return value is None. An outer branch that treats None as an empty success sends status 200 and the body {}, 2 characters. Delete the pass and let the ValueError leave load. The caller then sees 1 exception, and the message is still the 7-character missing. The outer branch can answer status 404 and the body {"error":"missing"}, 19 characters: the braces and the colon take 3, "error" takes 7, and "missing" takes 9. 3 plus 7 plus 9 is 19. A test that expects ValueError reports 1 failed on the pass version, because 0 exceptions arrive, and the same test reports 1 passed when the exception leaves the function.
Stock starts at 10. Write this interleaving on paper, the one where neither worker waits. Worker A reads 10. Worker B reads 10. A writes 10 minus 1, which is 9. B writes 10 minus 1, which is 9. The stored stock is 9. Two sales from 10 leave 8 when each sale waits for the write ahead of it. 9 minus 8 is 1, the unit that was sold and still sits in the stored count. That lost update is a race condition: both reads happened, then both writes stored the same successor. Put a lock around the read, the subtraction, and the write, and hold it until the write has landed. B's read then waits until A has stored 9. B reads 9 and writes 8. The locked interleaving stores 8. The integer you score is the stock after the two sales. 9 marks the unlocked interleaving. 8 marks the locked one.
A list with no cap, a name to look up, a method the notes removed
CACHE is a list at module scope, and it starts at length 0. Each request runs CACHE.append(row) and never deletes. After 1,000 requests the length is 1,000. In this fixture the row is a string of 200 characters, so the list holds 1,000 times 200 characters, which is 200,000. The process keeps those strings for the life of the run, because nothing pops them. Cap the list at 50. When the length is already 50, delete index 0, then append. After the first 50 requests the length is 50. After 1,000 requests the length is still 50, and the strings hold 50 times 200 characters, which is 10,000. 200,000 minus 10,000 is 190,000 characters the uncapped list would have kept past the cap. The check reads the length. 1,000 fails a cap of 50. 50 meets it. A fresh process starts at length 0 again, so the cap is a bound on one run, and you still measure it after the 1,000-row fixture.
The requirements line is pantry-utils-x7==1.2.0, 22 characters. The project name pantry-utils-x7 has 15 characters: pantry is 6, the hyphen is 1, utils is 5, the hyphen is 1, and x7 is 2. 6 plus 1 plus 5 plus 1 plus 2 is 15. Slopsquatting is the later event in which a package index gains a project under a name like that, a name a generator wrote into a requirements file, and a future install fetches that publisher's files. In this lesson the lookup for the 15-character name comes back empty, so you delete the line. The copy count for that name becomes 0. The imports you keep are json, os, sqlite3, and urllib, which ship with Python. The client notes for this file remove the method list_items_v1, 13 characters, and they name list_items, 10 characters. The generated line items = client.list_items_v1() raises AttributeError, because the notes have taken that method off the object. After the rename the file holds 0 calls to the 13-character name and 1 call to the 10-character name.
Nine marks on one file
Start from the generated file and score nine marks. On 4 orders the loop spends 5 statements. On 1,000 orders the same loop spends 1,001 statements, past the limit of 3. The 19-character key sk_live_pantry_9012 appears 1 time. urllib.request.urlopen(url) has no timeout argument, so that call has no deadline. sqlite3.connect("pantry.db") omits timeout, so the 5.0 second lock wait is the library default and the line does not show it. fetch_order(7) raises ValueError inside pass, the caller sees 0 exceptions, and the return value is None. The unlocked interleaving from stock 10 stores 9. After 1,000 appends the CACHE length is 1,000, and the row strings hold 200,000 characters. The 15-character name pantry-utils-x7 appears 1 time. list_items_v1 appears 1 time, and that call raises AttributeError.
Apply the edits and score the same nine marks again. The request uses the 31-character select and the 36-character item select, so it spends 2 statements, inside the limit of 3. The 19-character key appears 0 times, and the source reads the 14-character name PANTRY_API_KEY. The HTTP call carries the 11-character argument timeout=2.0, and the connect call carries the 11-character argument timeout=5.0. The ValueError leaves load, so the caller sees 1 exception and the message missing. The outer branch can send status 404 and the 19-character body. The locked interleaving stores 8. The capped CACHE length is 50, and the strings hold 10,000 characters. The package name appears 0 times. list_items_v1 appears 0 times, and list_items appears 1 time.
Write the nine marks as one row, generated then edited. Statements go from 1,001 to 2 on the 1,000-row fixture. Key copies go from 1 to 0. The HTTP deadline goes from absent to 2.0 seconds. The connect argument goes from absent to 5.0. Exceptions at the caller go from 0 to 1. Stock after the two sales goes from 9 to 8. CACHE length goes from 1,000 to 50. Package-name copies go from 1 to 0. Calls to the removed method go from 1 to 0. A file is ready for a larger run when that edited column is the one you can recompute from the lines.
Check it on paper
The fixture has 6 orders and the handler still runs one item statement per order, plus the opening order statement. Write the statement count, then write the statement count after you switch to the two-statement form. Say which of those two counts sits inside the limit of 3. The quoted key is sk_test_box_55. Write its character count, and write the passing copy count. The call is urllib.request.urlopen(url). Write the argument text that puts a 2.0 second deadline on it, and write how many characters that argument text has. Stock starts at 6. Three workers each read 6 before any write, and each then writes 5. Write the stored stock for that interleaving. Then write the stored stock when each worker holds the lock through the read and the write. CACHE has a cap of 50, each row string has 20 characters, and 80 requests append. Write the length and the character total with the cap, and write them again with no cap. The requirements name is box-tools-qq. Write its character count.
The loop spends 7 statements, because 1 plus 6 is 7. The two-statement form spends 2. Only 2 sits inside the limit of 3. sk_test_box_55 has 14 characters: sk_test is 7, the underscore is 1, box is 3, the underscore is 1, and 55 is 2. The passing copy count is 0. The deadline argument is timeout=2.0, and that text has 11 characters. The unlocked interleaving stores 5, because every worker writes 6 minus 1. The locked interleaving stores 3, because the stock steps 6, then 5, then 4, then 3. With the cap, the length is 50 and the strings hold 1,000 characters, because 50 times 20 is 1,000. With no cap, the length is 80 and the strings hold 1,600 characters, because 80 times 20 is 1,600. box-tools-qq has 12 characters: box is 3, the hyphen is 1, tools is 5, the hyphen is 1, and qq is 2.
Check questions
On 1,000 orders, how many SQL statements does the per-order loop spend, and how many does the two-statement form spend?
The loop spends 1,001 statements, which is 1 plus 1,000. The two-statement form spends 2. The check allows at most 3, so 2 meets it and 1,001 sits past it.
How many characters are in the quoted value sk_live_pantry_9012, and how many copies of that value does a passing source contain?
19 characters, and the passing copy count is 0. The source reads the environment name PANTRY_API_KEY, which has 14 characters.
Stock starts at 10. In the unlocked interleaving both workers read 10 and both write 9. What is stored, and what does the locked interleaving store?
The unlocked interleaving stores 9. The locked interleaving stores 8. The gap is 1, the unit sold and still counted.
fetch_order(7) raises ValueError with the message missing inside except Exception and pass. How many exceptions does the caller see, and how many calls to list_items_v1 remain after the rename?
The caller sees 0 exceptions, and load returns None. After the rename, 0 calls to the 13-character name list_items_v1 remain, and list_items is the call that stays.
Score the file with nine marks: statements as 1 plus the rows, copies of the 19-character key, a deadline on urlopen, the lock wait written on the connect, exceptions that reach the caller, stock after the interleaving, the capped list length, copies of the invented package name, and calls to the removed method.
Final task
Submit a patched version of the seeded defective repository; the automated test suite verifies 0 hardcoded secrets, 0 N+1 queries on a 1,000-row fixture (≤ 3 SQL queries per request), enforced timeouts, and 100% regression pass rate.
Terms in this course
This course has no terms of its own in the glossary yet. Terms from earlier courses are used here too.
The first lesson is open to read. The rest of the course is being written, and tests and diplomas come later. Join the waitlist