Fine-Tuning with LoRA and QLoRA
You can count the trainable weights in a LoRA pair, run one scaled forward pass and the matching merge by hand, read a learning rate off a warmup-then-decay schedule, and keep the checkpoint with the lowest validation loss whose old-task score still passes.
- Level
- Navigator
- Time
- 26 h
- Prerequisites
- R08-201

Jump to
After this course you can
The student can implement a reproducible LoRA training and evaluation script with fixed random seeds that reduces validation loss on a toy task while passing catastrophic-forgetting regression checks.
Full updates and the adapter budget
Full fine-tuning updates every weight in the matrices you chose. Parameter-efficient fine-tuning, PEFT, freezes those weights and trains a small added set. LoRA, low-rank adaptation, is the PEFT form in this lesson. For a matrix W with d_in columns and d_out rows, the adapter is a pair: A has shape r by d_in, and B has shape d_out by r. The trainable count is r times d_in plus r times d_out. On a square matrix that is 2 times r times d. The forward pass adds a scaled product to the frozen result: y equals W x plus (alpha divided by r) times B times (A x).
Take one square matrix with d = 1024 and rank r = 8. The full matrix holds 1024 times 1024 = 1,048,576 weights. A holds 8 times 1024 = 8,192 weights, B holds the same 8,192, and the pair holds 16,384. That is 1/64 of the full matrix. A count that stops at r times d gives 8,192 and leaves B out. In this lesson 1 MiB means 1,048,576 bytes. Each trainable weight costs 12 bytes: 2 for the BF16 value, 2 for the BF16 gradient, 4 for the first Adam moment, and 4 for the second. A frozen BF16 weight costs 2 bytes and has no moments. Full fine-tuning of this one matrix costs 1,048,576 times 12 = 12,582,912 bytes, which is 12 MiB. LoRA stores the frozen base at 1,048,576 times 2 = 2,097,152 bytes and the adapter at 16,384 times 12 = 196,608 bytes. The sum is 2,293,760 bytes, which is 2.1875 MiB.
Rank, alpha, targets, and dropout
Alpha sets the scale in front of B (A x). With alpha = 16 and r = 8 the scale is 16/8 = 2. Raising r to 16 while alpha stays 16 cuts the scale to 1, and it doubles the trainable count on this square matrix from 16,384 to 32,768. The target list names the matrices that receive a pair. A small block here has two layers, and each layer has q, k, v, and o, eight matrices in all. Targeting q and v attaches 4 pairs: 4 times 16,384 = 65,536 trainable weights. Targeting all eight attaches 8 times 16,384 = 131,072. The frozen base is still all eight matrices either way, because a matrix with no pair is used as W and left unchanged.
Dropout sits only on the adapter branch, and only while training. This lesson uses inverted dropout with p = 0.5: each kept coordinate of x is divided by 1 minus p, a multiply by 2, and each dropped coordinate becomes 0. At evaluation the mask is off and x enters A unchanged. B is stored as zeros at step 0, and A is drawn once with the seed fixed at 7, so a second run with that seed draws the same A. With B at zero the product B (A x) is the zero vector and y equals W x. The numbers in the worked pass are a later B, the one saved with a checkpoint, so that product has nonzero entries.
The 4-bit base, the schedule, and one accumulated step
QLoRA keeps the same A, the same B, and the same scale. What changes is the storage of W. Each frozen weight is a 4-bit code, half a byte, plus one 2-byte scale shared by 64 weights. The scale adds 2/64 = 1/32 byte per weight. For the 1024 matrix the codes take 524,288 bytes and the scales take 1,048,576 divided by 64, times 2, which is 32,768 bytes. The adapter still takes 196,608 bytes, because A and B stay in BF16 with Adam moments. The stored sum is 524,288 + 32,768 + 196,608 = 753,664 bytes, which is 0.71875 MiB. During the multiply for that matrix the run also builds a temporary BF16 copy of W, 2,097,152 bytes, and releases that copy when the multiply finishes. Gradients update A and B. The 4-bit codes stay as they were loaded.
The schedule has 80 steps and a peak learning rate of 0.0002. Steps 1 through 8 are warmup: the rate is 0.0002 times s divided by 8. Step 4 is 0.0001. Step 8 is 0.0002. Steps 9 through 80 fall in a straight line to zero: the rate is 0.0002 times (80 minus s) divided by 72. Step 44 gives (80 minus 44) divided by 72 = 36/72 = 1/2, so the rate is 0.0001. Step 80 gives 0. Gradient accumulation gathers several micro-batches before one update. Here each micro-batch holds 2 rows and the accumulation count is 4, so the effective batch is 8. The optimizer sees the mean of the four micro-batch gradients and takes one step.
The checkpoint you keep, then the merge
Read train loss, validation loss, and an old-task score at the same steps. The old task is four prompts the frozen model already answered correctly, and the score is how many of the four are still correct. The rule has two parts. Keep the step whose validation loss is the lowest. Then require the old-task score at that step to be at least 3 out of 4. On the recorded run the rows are step 20: train 1.60, validation 1.70, old task 4; step 40: train 0.90, validation 1.00, old task 4; step 60: train 0.40, validation 1.20, old task 2; step 80: train 0.10, validation 1.80, old task 1. The lowest validation loss is 1.00 at step 40, and the old-task score there is 4, so step 40 is the one you keep. Step 80 has the lowest train loss and fails both parts.
A merge writes the adapter into the base: W prime equals W plus (alpha divided by r) times B A, with the product formed in BF16. Inference then loads W prime and leaves A and B unloaded. Loading the merged matrix and the pair together would add the same product a second time. The adapter file for the 1024 matrix, A and B in FP16 with no moments, is 16,384 times 2 = 32,768 bytes. The merged file packed with the 4-bit rule is the codes plus the scales, 524,288 + 32,768 = 557,056 bytes. Seed 7, set before A is drawn and before the rows are shuffled, makes the loss at step 40 repeat on a second run of the same script.
One pass, one merge, and one step, by hand
Use the 2 by 2 matrix W = [[1, 0], [0, 1]] and the input x = [3, 1]. Then W x = [3, 1]. The rank is r = 1 and alpha = 2, so the scale alpha divided by r is 2. A = [[1, -1]] and the checkpoint B = [[2], [1]]. A x = 1 times 3 plus negative 1 times 1 = 2. B times that scalar is [4, 2]. Times the scale that is [8, 4]. Add W x: y = [3 plus 8, 1 plus 4] = [11, 5]. The trainable count on this tiny pair is 1 times (2 plus 2) = 4, the same size as the full matrix. The 1024 matrix is where the fraction becomes 1/64. The tiny pair is where each product stays visible.
The product B A is [[2, -2], [1, -1]]. Times the scale that is [[4, -4], [2, -2]]. Add W: W prime = [[5, -4], [2, -1]]. Check with the same x: 5 times 3 plus negative 4 times 1 = 11, and 2 times 3 plus negative 1 times 1 = 5. The merged multiply matches y. At step 0, B is [[0], [0]], the added vector is [0, 0], and y equals [3, 1].
Training dropout uses p = 0.5 and the given mask that drops the second coordinate of x. The kept coordinate is multiplied by 2, so the adapter sees [6, 0]. A times that vector is 6. B times 6 is [12, 6]. Times the scale that is [24, 12]. Then y = [3 plus 24, 1 plus 12] = [27, 13]. Evaluation uses the undropped pass, y = [11, 5].
One coordinate of B starts the accumulation at 0.010. The four micro-batch gradients are 4, 0, 2, and 2. Their mean is 8/4 = 2. The step runs at schedule step 4, where the rate is 0.0001. The change is negative 0.0001 times 2 = negative 0.0002, and the coordinate becomes 0.010 minus 0.0002 = 0.0098. Four micro-batches produced one update. The checkpoint rule on the recorded rows keeps step 40.
Check it on paper
For r = 4 on one 1024 by 1024 matrix, write the trainable count and the fraction of 1,048,576. Using W, x, and A from the worked pass, replace B with [[1], [0]], keep alpha = 2 and dropout off, and write y and W prime. Write the learning rate at step 8 and at step 62. Start the same coordinate at 0.010, take four micro-batch gradients of 1, 1, 1, and 1, step at the step-4 rate, and write the new value. On the recorded rows, write the step you keep and its old-task score. Write the stored QLoRA byte total for one 1024 matrix at r = 8, and the merged 4-bit file size for that matrix.
The trainable count is 4 times 1024 times 2 = 8,192, and 8,192 divided by 1,048,576 is 1/128. A x stays 2, B times 2 is [2, 0], the scale makes [4, 0], and y = [7, 1]. B A = [[1, -1], [0, 0]], times the scale is [[2, -2], [0, 0]], and W prime = [[3, -2], [0, 1]]. The rate at step 8 is 0.0002. At step 62, (80 minus 62) divided by 72 = 18/72 = 1/4, so the rate is 0.00005. The mean gradient is 1, the change is negative 0.0001, and the coordinate becomes 0.0099. You keep step 40, with old-task score 4 out of 4. The stored QLoRA total is 753,664 bytes. The merged 4-bit file is 557,056 bytes.
Check questions
On one 1024 by 1024 matrix with r = 8, how many weights are trainable, and what fraction of the full matrix is that?
16,384, which is 1/64 of the full matrix.
In the worked pass with dropout off, what is y, and what is the merged matrix W prime?
y is [11, 5]. W prime is [[5, -4], [2, -1]].
What is the learning rate at step 4 and at step 44, and what is the adapter coordinate after the worked accumulation that started at 0.010?
The rate is 0.0001 at both steps. The coordinate becomes 0.0098.
Which recorded step do you keep, what is its old-task score, and how many bytes does stored QLoRA use for one 1024 matrix at r = 8?
Keep step 40, with old-task score 4 out of 4. Stored QLoRA uses 753,664 bytes.
A LoRA pair on a d by d matrix trains 2 r d weights and adds (alpha divided by r) times B A x to the frozen product. QLoRA stores that frozen matrix as 4-bit codes plus scales while A and B stay in BF16. Keep the step with the lowest validation loss whose old-task score is at least 3 out of 4, and merge with W prime equal to W plus (alpha divided by r) times B A before you pack the file.
Final task
Submit a PyTorch/PEFT LoRA training and evaluation script executed on a tiny CPU/GPU fixture model in the isolated runner; automated tests verify trainable parameter count matches exact LoRA math (r * d), loss decreases deterministically, and adapter export succeeds.
Terms in this course
This course has no terms of its own in the glossary yet. Terms from earlier courses are used here too.
The first lesson is open to read. The rest of the course is being written, and tests and diplomas come later. Join the waitlist