Node Pipelines, ControlNet and Custom LoRA Training
You can count the links in a 13-node API graph, count the sampler steps a control map covers, accept or reject a captioned training row, and compute a LoRA card from rank, alpha, and the number of API calls.
- Level
- Navigator
- Time
- 25 h
- Prerequisites
- R06-101

Jump to
After this course you can
The student can build a dataset validation and captioning pipeline that filters out unlicensed or malformed images and exports a reproducible ComfyUI API workflow with verified structural conditioning nodes.
Thirteen nodes and seventeen links
The call you send is one JSON object with two keys this lesson requires: prompt and client_id. prompt is a dictionary of nodes. The key is the node id, and the worked graph uses the ids 1 through 13. Each value holds class_type and inputs. A wired input is a two-item list: the source node id, then the source slot. Each such list counts as one link. A number or a file name typed straight onto the node is a widget, and the link count leaves widgets out.
Nodes 1, 2, 3, 4, 5, 6, 7, 8, 10, 11, and 12 use the usual API class names. Node 1 is CheckpointLoaderSimple. Slot 0 is the model, slot 1 is the clip, and slot 2 is the vae. Node 2 is LoraLoader. Its model input reads node 1 slot 0, its clip input reads node 1 slot 1, and the widgets strength_model and strength_clip are both 1. Those two strengths scale the trained file at run time. Nodes 3 and 4 are CLIPTextEncode, positive then negative, and each takes clip from node 2 slot 1. Node 5 is EmptyLatentImage. Node 6 is ControlNetLoader. Node 7 is LoadImage for the structure map. Node 8 is ControlNetApplyAdvanced. Node 9 is this lesson's image-token node, class name IPAdapterApply, with inputs model, image, and weight, and with a model output on slot 0. Grade node 9 by that list. Node 10 is KSampler. Node 11 is VAEDecode. Node 12 is SaveImage. Node 13 is a second LoadImage, the style image that feeds node 9.
Count the wired inputs only. Node 2 has 2, node 3 has 1, node 4 has 1, node 8 has 4 (positive, negative, control_net, image), node 9 has 2 (model from node 2 slot 0, image from node 13 slot 0), node 10 has 4 (model from node 9 slot 0, positive from node 8 slot 0, negative from node 8 slot 1, latent_image from node 5 slot 0), node 11 has 2 (samples from node 10 slot 0, vae from node 1 slot 2), and node 12 has 1. The sum is 2 + 1 + 1 + 4 + 2 + 4 + 2 + 1 = 17. Nodes 1, 5, 6, 7, and 13 hold widgets only, so they add zero links. Four checks pass before a send. Check 1 is 13 nodes and 17 links. Check 2 says KSampler positive comes from node 8 slot 0 and KSampler negative comes from node 8 slot 1. Check 3 says the model path is node 1 slot 0, then node 2 slot 0, then node 9 slot 0, then node 10. Check 4 says the width and the height on node 5 equal the width and the height of the structure file on node 7. Wiring node 3 straight into KSampler positive, in place of the wire from node 8, keeps the link count at 17 and fails check 2.
Control maps, the step window, and four image tokens
Node 8, ControlNetApplyAdvanced, takes the positive conditioning, the negative conditioning, a control model, and a structure image. This lesson uses three kinds of structure image. A canny map is an edge picture. The hand rule marks a pixel as an edge when the absolute difference with the pixel on its right is at least 40. The last pixel in the strip has no right-hand neighbor, so the rule leaves it unmarked. A production detector uses a gradient and two thresholds. You grade the hand rule. The worked strip reads 10, 50, 55, 120, 30. The four differences are 40, 5, 65, and 90. Three of those four meet 40, so the strip holds 3 edge pixels. A depth map stores an integer from 0 through 255 at each pixel, and in this lesson a larger integer means nearer. A pose map stores 18 body keypoints. A keypoint is valid when x runs from 0 through width minus 1 and y runs from 0 through height minus 1. On a 512 frame the valid span is 0 through 511. The worked pose file has 18 keypoints and 4 of them carry x = 520. Valid keypoints are 18 minus 4, which is 14. This lesson refuses a pose map with fewer than 5 valid keypoints. 14 meets that gate.
The worked KSampler uses 20 sampler steps, numbered 0 through 19, with start_percent 0 and end_percent 0.8. Step k sits inside the window when start_percent is less than or equal to k/20 and k/20 is below end_percent. Here that is k/20 below 0.8, so k is below 16. Steps 0 through 15 are inside, and that is 16 steps. Steps 16 through 19 are outside, and that is 4 steps. On an inside step the applied residual equals the text residual plus strength times the control residual. The worked numbers are text 0.25, strength 0.80, and control 1.00, so 0.25 plus 0.80 times 1.00 equals 1.05. On an outside step the control term is dropped and the residual stays 0.25.
Node 9 appends image tokens beside the positive prompt. The control map stays pixel-aligned with the empty latent. The image tokens have a fixed bundle size in this lesson: 4 tokens for the one style image on node 13. The worked positive prompt is given as 12 tokens, so the attention total you check is 12 plus 4, which is 16. The negative prompt is a second conditioning and stays out of that total. The weight multiplies the image-token vectors. Weight 0.50 leaves the total at 16.
A row that may enter the training set
Each candidate is one image, one hash, one license, and one caption line. This lesson's clearance list is own, cc0, and cc-by. The marks nc and nd stay out of the set. nc is a non-commercial license, and nd is a no-derivatives license. Both sides of the image have to be at least 512 and divisible by 64. The hash is a SHA-256 digest written in lowercase hex. The digest is 32 bytes, and 32 times 2 hex characters is 64, so the stored string has length 64 and each character is a digit 0 through 9 or a letter a through f. The caption splits on commas, and spaces around a field come off. Five fields stand in this order: subject, count, pose, background, license tag. The subject is mug, chair, or lamp. The count is an integer from 1 through 4. The pose is front, side, or three-quarter. The background is studio, street, or room. The license tag equals the row license. A matching tag cannot rescue a license that sits outside the clearance list.
Eight rows go through those rules. Row a is own, 512 by 512, hash length 64, caption mug, 1, front, studio, own, and it passes. Row b is cc-by, 768 by 512, hash length 64, caption chair, 2, side, room, cc-by, and it passes. Row c is nc, 512 by 512, hash length 64, caption lamp, 1, front, studio, nc, and the license fails. Row d is cc0, 500 by 512, hash length 64, caption mug, 1, side, street, cc0. The side 500 is below 512, and 500 divided by 64 is 7.8125, so the side fails the minimum and the grid. Row e is own, 512 by 512, hash length 63, caption mug, 1, front, studio, own, and the hash length fails. Row f is cc-by, 1024 by 1024, hash length 64, caption mug, 1, front, studio, cc0. The tag cc0 disagrees with the row license cc-by, so the row fails. 1024 divided by 64 is 16, so the size itself would have passed. Row g is own, 768 by 768, hash length 64, caption lamp, 3, three-quarter, room, own, and it passes. Row h is cc0, 512 by 512, hash length 64, caption vase, 1, front, studio, cc0, and the subject vase fails. Three rows pass: a, b, and g. Five rows fail.
The manifest keeps one line for each passing row, in id order. Six fields sit between vertical bars: id, width, height, license, hash length, and the caption. The three lines are a | 512 | 512 | own | 64 | mug, 1, front, studio, own, then b | 768 | 512 | cc-by | 64 | chair, 2, side, room, cc-by, then g | 768 | 768 | own | 64 | lamp, 3, three-quarter, room, own. Latent numbers follow the grid you already use: divide each side by 8, then multiply by 4. Row a is 64 times 64 times 4, which is 16384. Row b is 96 times 64 times 4, which is 24576. Row g is 96 times 96 times 4, which is 36864. The empty latent on that call uses the same width and height, so check 4 passes for each of the three lines.
Rank, alpha, and one call per row
A LoRA on one linear layer adds two small matrices while the checkpoint weights stay frozen. This lesson's practice network adapts 4 layers, and each layer has 320 inputs and 320 outputs. Trainable numbers on one layer equal rank times (320 plus 320), which is rank times 640. Rank 8 gives 8 times 640, which is 5120 on one layer, and 4 times 5120 is 20480 on the network. Rank 4 gives 10240 on the network. Rank 16 gives 40960. The allowed ranks on a card are 4, 8, and 16. The training scale is alpha divided by rank. The card allows scale 1 and scale 0.5, so alpha equals the rank, or half the rank. The stand-in product of the two small matrices, before that scale, is 0.20. Scale 1 adds 0.20. Scale 0.5 adds 0.10. Scale 2 is off the card. Alpha changes the scale and leaves the trainable count unchanged: alpha 4 with rank 8 still stores 20480 numbers, at scale 0.5. This lesson stores each trainable number in 2 bytes, so the rank-8 network occupies 20480 times 2, which is 40960 bytes.
Allowed training-step counts are 100, 150, 200, 250, 300, 350, and 400. The worked card is rank 8, alpha 8, and 200 training steps. The scale is 8 divided by 8, which is 1. The trainable count is 20480 and the stored size is 40960 bytes, so the card passes. Rank 32 sits outside the allowed ranks. Alpha 16 with rank 8 gives scale 2, so that card fails while the rank itself is allowed. Loader strengths on node 2 stay at 1 and 1, so the run-time multiplier is 1. The training scale already lives in the file. Sampler steps stay 20. Training steps and sampler steps are two counters.
The batch sends one POST to /prompt for each manifest line, and batch_size on the empty latent stays 1, because the three pictures have three sizes. The worked batch therefore sends 3 calls. client_id is the string batch-06 on every call. Each call overrides 4 widgets inside the prompt: the structure file name on node 7, the seed on node 10, the width on node 5, and the height on node 5. Override writes equal 3 times 4, which is 12. Seeds follow id order and start at 100. Row a gets 100, row b gets 101, and row g gets 102. Row b writes width 768 and height 512. Row g writes width 768 and height 768.
From eight rows to three calls
Keep rows a, b, and g. Latent numbers are 16384 for a, 24576 for b, and 36864 for g.
The graph that rides with those lines has 13 nodes. The link count runs 2 plus 1 plus 1 plus 4 plus 2 plus 4 plus 2 plus 1, which is 17. KSampler positive comes from node 8 slot 0 and KSampler negative from node 8 slot 1, so check 2 passes. The model path runs from node 1 through node 2 and node 9 into node 10, so check 3 passes. For row a the empty latent is 512 by 512, matching the structure file, so check 4 passes on that call. The same match sets row b to 768 by 512 and row g to 768 by 768.
Sampler steps are 20, start_percent is 0, and end_percent is 0.8, so 16 steps sit inside the window and 4 sit outside. Text residual 0.25, strength 0.80, and control residual 1.00 give 1.05 inside the window and 0.25 outside it. The positive prompt contributes 12 tokens and node 9 contributes 4, so the attention total is 16 at weight 0.50. The LoRA card is rank 8, alpha 8, and 200 training steps. The scale is 1, the trainable count is 20480, and at 2 bytes each the file stores 40960 bytes. Both loader strengths stay at 1.
POST /prompt goes out three times, with client_id batch-06 and batch_size 1. Seeds are 100, 101, and 102. Each call overrides the file name, the seed, the width, and the height, so the override writes are 12. The line you can tick before the first POST is: nodes 13, links 17, checks 1 through 4 passed for that row's size, inside steps 16, residual 1.05, attention tokens 16, trainable 20480, calls 3.
Mark a graph, a window, and two new rows
A twin of the worked graph still has 17 links, and KSampler positive now comes from node 3 instead of node 8. Name the check that fails. Then take 30 sampler steps, start_percent 0, end_percent 0.5, strength 0.60, text residual 0.20, and control residual 1.00. Write the inside step count, the inside residual, and the outside residual. On a canny strip 0, 10, 80, count the edge pixels under the hand rule. On a 512 frame, a pose file has 18 keypoints and 3 of them carry x = 600. Write the valid count and say whether the gate of 5 passes. Row i is cc0, 640 by 512, with a 64-character lowercase hex hash and the caption chair, 1, front, street, cc0. Row j is nd, 512 by 512, with a 64-character lowercase hex hash and the caption mug, 1, front, studio, nd. Mark each row pass or fail, and write the latent count for a row that passes. The LoRA card under test is rank 16, alpha 8, 250 training steps, on the same 4 layers of 320, stored at 2 bytes, with stand-in product 0.20. Write the scale, the trainable count, the byte count, the added amount, and whether the card passes. A positive prompt of 10 tokens goes through one image-token node at weight 0.25. Write the attention total. Row i joins the worked passing manifest, and row j stays out. Write the API call count, the override writes, and the seed on each passing id.
Check 2 fails, because KSampler positive has to come from node 8. The link count can stay 17. Steps 0 through 14 sit inside the window, which is 15 steps, and steps 15 through 29 sit outside. The inside residual is 0.20 plus 0.60 times 1.00, which is 0.80. The outside residual is 0.20. The canny differences are 10 and 70, so one difference meets 40 and the strip holds 1 edge pixel. Valid pose keypoints are 18 minus 3, which is 15, and 15 meets the gate of 5. Row i passes. Its latent count is 80 times 64 times 4, which is 20480. Row j fails because nd sits outside own, cc0, and cc-by, even though the caption tag matches the row. The LoRA scale is 8 divided by 16, which is 0.5. Trainable numbers are 16 times 640 times 4, which is 40960. Bytes are 40960 times 2, which is 81920. The added amount is 0.20 times 0.5, which is 0.10. Rank 16, scale 0.5, and 250 training steps are all allowed, so the card passes. The attention total is 10 plus 4, which is 14, and weight 0.25 leaves that 14 in place. Passing ids in order are a, b, g, and i, so the batch sends 4 calls and 16 override writes. The seeds are 100, 101, 102, and 103.
Check questions
How many nodes and how many links does the worked API graph contain, and which check fails when KSampler positive is wired from node 3 while the link count stays 17?
13 nodes and 17 links. Check 2 fails, because KSampler positive and KSampler negative both have to come from node 8.
The worked sampler runs 20 steps with start_percent 0 and end_percent 0.8. How many steps sit inside the window, what is the applied residual inside it, and what is the residual outside it, using text residual 0.25, strength 0.80, and control residual 1.00?
16 steps sit inside, steps 0 through 15, and the applied residual there is 1.05. The other 4 steps stay at 0.25.
How many of the eight rows pass, which ids are they, and why does row f fail when both sides are 1024?
Three rows pass: a, b, and g. Row f fails because the caption tag is cc0 and the row license is cc-by. 1024 divided by 64 is 16, so the size itself clears.
On the worked LoRA card, rank 8 and alpha 8, how many trainable numbers are stored, what is the scale, how many bytes do they occupy at 2 bytes each, and how many API calls and override writes does the passing manifest produce?
20480 trainable numbers, scale 1, and 40960 bytes. The manifest produces 3 API calls and 12 override writes.
Tick the graph before the first POST: 13 nodes, 17 links, KSampler fed by the control node, and the empty latent copied from that row's width and height. A training row stays in only when the license, both sides, the 64-character hash, and the five caption fields all pass, and the LoRA card shows an allowed rank with scale 1 or 0.5. Each passing row is one API call.
Final task
Submit a dataset validator script and ComfyUI API graph inspected by automated tests that verify: 100% valid image/caption pairs, rejection of seeded corrupt/unlicensed files, valid ControlNet/LoRA graph topology, and batch API execution. (Optional Portfolio item: custom LoRA showcase).
Terms in this course
This course has no terms of its own in the glossary yet. Terms from earlier courses are used here too.
The first lesson is open to read. The rest of the course is being written, and tests and diplomas come later. Join the waitlist