Image 1 of 1: ‘Diagram of the lesson's AWS components: S3 buckets at the top, a notebook instance on the left launching SageMaker training, tuning and processing jobs in the center, and Bedrock on the right called as an API’
The four pieces the lesson uses, and what
moves between them
Image 1 of 1: ‘Diagram from the overview episode: S3 buckets at the top, the notebook instance on the left launching SageMaker training, tuning and processing jobs, and Bedrock on the right’
The notebook instance launches jobs and
calls Bedrock; S3 holds the data and the outputs
Figure 2
Image 1 of 1: ‘Screenshot showing the three required tags on a SageMaker notebook instance’
Image 1 of 1: ‘Timeline of a SageMaker training job showing the unbilled wait for an instance and the billed phases from container pull to model upload’
describe_training_job returns
BillableTimeInSeconds: the seconds the instance was yours,
from container pull to model upload, per instance. The minute or so
before that, while SageMaker finds an instance, is not billed. (The
job’s console page also shows a “Training time”; on an on-demand
instance it is the same number, and neither one is your script’s own
training time, which only the script’s log has. The two lines the SDK
prints when fit() returns, Training seconds
and Billable seconds, are the same pair: they differ only
on spot instances, which this lesson does not use, so read either one.)
SageMaker bills per second, with no rounding, so cost is billable
seconds x instance count x hourly rate. We’ll wrap this in a small
helper and call it after every job in this lesson.
Image 1 of 1: ‘Two panels comparing instance_count=2 without distribution, where each instance trains on the full data, with data parallelism, where batches are split and gradients synced’
Two instances without a distribution
strategy run the whole job twice; with data parallelism they split the
batches and share gradients
Image 1 of 1: ‘Dot plot of benchmark scores for three task-specific models against a much larger general model on the same benchmark; the specialist is ahead in each row by 6.5, 2.2 and 1.2 points’
Image 1 of 1: ‘Left: a d by d weight matrix, all trained, 16 bytes per weight. Right: the same matrix frozen in bf16 plus two thin trained matrices B (d by r) and A (r by d); output is W x plus B A x.’
Full fine-tuning updates the whole weight
matrix; LoRA freezes it and trains two thin matrices beside
it
Figure 3
Image 1 of 1: ‘Horizontal bar chart on a logarithmic GPU-hours axis from 0.1 to 10 million. Training from scratch, every weight from random: 460,000 GPU-hours. Full fine-tuning, every weight from a pretrained model: 800 GPU-hours. LoRA fine-tuning, an adapter under one percent of the weights: 0.5 GPU-hours. Each bar is labeled with the GPUs, the time and the electricity. Energy is GPU electricity at the rated draw, a lower bound.’
GPU-hours, on a log scale, to train a 3B
model from scratch, to fully fine-tune it, and to LoRA fine-tune
it
Figure 4
Image 1 of 1: ‘Stacked bar chart of bytes per parameter for inference versus LoRA versus full fine-tuning’
GPU memory per parameter: inference, a LoRA
fine-tune, and a full fine-tune
Figure 5
Image 1 of 1: ‘A layer W with an input arrow, an output arrow, an error arrow going back and an update arrow into W, each labeled 2 FLOPs; a box totals 6 for full fine-tuning and 4 for LoRA’
The three matrix products per token; a full
fine-tune does all three, LoRA skips the weight update
Figure 6
Image 1 of 1: ‘Horizontal bar chart on a logarithmic dollar axis: this episode 0.75, homework 1.50, the full Badger Scribe train set 1.20, 7B full fine-tune 140, 70B LoRA 600, CHURRO-3B 6,300, 70B full fine-tune 12,000, with the ten-run sweep shown as a lighter extension of each bar’
Cost per run, on a log scale, from this
episode’s LoRA run to a 70B full fine-tune; the lighter extension is a
ten-run sweep
Image 1 of 1: ‘Diagram of the RAG pipeline with the embedder and generator highlighted as the two model calls’
The RAG pipeline; the embedder and the
generator are the only parts that change between episodes
Figure 2
Image 1 of 1: ‘Comparison table of four ways to run inference on AWS: Bedrock API (default, pay per token, nothing to shut down), Processing job (batch with open models, bills per second while the job runs, terminates itself), GPU notebook (learning and debugging only, bills per hour until stopped, about $90 if forgotten over a weekend) and Inference endpoint (steady traffic only, bills 24/7, about $1,000 if idle for a month).’
Image 1 of 1: ‘Paired horizontal bars comparing a 27B open-weight model on one GPU with a frontier commercial model; within a few points on agent tasks and science questions, ahead on chart reasoning’
Qwen3.8-27B against Claude Opus 4.8 on three
benchmarks, as reported by each model’s authors
