All Images

Overview: ML on AWS

Figure 1

Diagram of the lesson's AWS components: S3 buckets at the top, a notebook instance on the left launching SageMaker training, tuning and processing jobs in the center, and Bedrock on the right called as an API
S3 holds data and outputs; a small notebook instance launches SageMaker jobs that read from and write to S3 and end on their own; Bedrock is called as an API per token

Data Storage: Setting up S3

Figure 1

Screenshot showing required tags for an S3 bucket
Example of Tags for an S3 Bucket

Notebooks as Controllers

Figure 1

Diagram from the overview episode: S3 buckets at the top, the notebook instance on the left launching SageMaker training, tuning and processing jobs, and Bedrock on the right
The notebook instance launches SageMaker jobs that read from and write to S3 and end on their own, and calls Bedrock as an API

Figure 2

Screenshot showing the three required tags on a SageMaker notebook instance
Tag Setup Example

Accessing and Managing Data in S3 with SageMaker Notebooks

(Optional) Pushing Your Notebooks to GitHub from SageMaker

Training Models in SageMaker: Intro

Figure 1

Timeline of a SageMaker training job showing the unbilled wait for an instance and the billed phases from container pull to model uploaddescribe_training_job returns BillableTimeInSeconds: the seconds the instance was yours, from container pull to model upload, per instance. It also returns TrainingTimeInSeconds, which on on-demand instances is the same number (it only differs on spot instances), so do not expect it to isolate your script. The minute or so before that, while SageMaker finds an instance, is not billed. SageMaker bills per second, with no rounding, so cost is billable seconds x instance count x hourly rate. We’ll wrap this in a small helper and call it after every job in this lesson.

Training Models in SageMaker: PyTorch Example

Figure 1

Two panels comparing instance_count=2 without distribution, where each instance trains on the full data, with data parallelism, where batches are split and gradients synced
Left: two instances each receive the full dataset and run the whole script independently, same model and twice the bill. Right: data parallelism splits the batches across instances and syncs gradients each step.

Hyperparameter Tuning in SageMaker: Neural Network Example

Fine-tuning a Vision-Language Model with Training Jobs

Figure 1

Dot plot of benchmark scores for three task-specific models against a much larger general model on the same benchmark; the specialist is ahead in each row by 6.5, 2.2 and 1.2 points
Three research tasks where a small specialist model scores above a much larger general one: CHURRO (3B) over Gemini 2.5 Pro on historical handwriting, Goedel-Prover-V2 (8B) over DeepSeek-Prover-V2 on formal proofs, MedGemma (27B) over GPT-4o on medical questions

Figure 2

Left: a d by d weight matrix, all trained, 16 bytes per weight. Right: the same matrix frozen in bf16 plus two thin trained matrices B (d by r) and A (r by d); output is W x plus B A x.
Full fine-tuning updates every entry of a d × d weight matrix; LoRA freezes it and trains two thin matrices of rank r whose product is added to its output

Figure 3

Horizontal bar chart on a logarithmic dollar axis: this episode 0.95, homework 1.90, the full Badger Scribe train set 2.25, 7B full fine-tune 140, 70B LoRA 600, CHURRO-3B 6,300, 70B full fine-tune 12,000, with the ten-run sweep shown as a lighter extension of each bar
Cost per run on a log scale, from about a dollar for this episode’s LoRA run to twelve thousand dollars for a 70B full fine-tune; a ten-run sweep shifts every bar up by one decade

Figure 4

Stacked bar chart of bytes per parameter for inference versus LoRA versus full fine-tuning
Bars of GPU memory per parameter: 1 byte for FP8 inference, 2 for bf16 inference, about 2.5 for a LoRA fine-tune, 16 for a full fine-tune made of the weight, its gradient, a 32-bit copy of the weight and the two averages Adam keeps

Figure 5

A layer W with an input arrow, an output arrow, an error arrow going back and an update arrow into W, each labelled 2 FLOPs; a box totals 6 for full fine-tuning and 4 for LoRA
Three matrix products per token: forward (2 FLOPs per weight), error passed back to the inputs (2), and the weight update (2); full fine-tuning does all three for 6, LoRA skips the update for 4

Overview of RAG Workflows on AWS

Figure 1

Diagram of the RAG pipeline with the embedder and generator highlighted as the two model calls
RAG pipeline: PDFs are chunked and embedded once; per question, the question is embedded, the top-k chunks are retrieved by cosine similarity, and a generator answers from question plus chunks. The embedder and generator are the only parts that change between episodes.

Figure 2

Comparison table of four ways to run RAG on AWS: Bedrock API (default, pay per token, nothing to shut down), Processing job (batch with open models, bills per second while the job runs, terminates itself), GPU notebook (learning and debugging only, bills per hour until stopped, about $90 if forgotten over a weekend) and Inference endpoint (steady traffic only, bills 24/7, about $1,000 if idle for a month).
Four columns, Bedrock API, Processing job, GPU notebook and Inference endpoint, compared on where the model runs, what runs between calls, billing unit, latency, model choice, what you must remember, a cost example and which episode covers it; Bedrock is marked as the default and Processing jobs as the safe way to use a GPU

RAG with Bedrock

RAG with Processing Jobs

Figure 1

Paired horizontal bars comparing a 27B open-weight model on one GPU with a frontier commercial model; within a few points on agent tasks and science questions, ahead on chart reasoning
Qwen3.8-27B against Claude Opus 4.8 on three benchmarks: 73.0 vs 74.6 on Terminal-Bench 2.1, 89.2 vs 93.6 on GPQA Diamond, 83.7 vs 80.5 on CharXiv reasoning

(Optional) RAG with a Notebook GPU

(Optional) The Same Jobs from the Terminal

Resource Management and Monitoring