Summary and Setup

Workshop Overview

---

This workshop teaches researchers how to run machine learning and AI work on AWS. It works through the practical decisions, where to store the data, which compute to use, how to launch and monitor a job, and how to estimate and manage the cost.

We use Amazon SageMaker to train, fine-tune and run our own models, and Amazon Bedrock to call models AWS hosts through an API. Throughout, we work from a small SageMaker notebook instance that acts as the controller. The demanding work runs on separate instances that SageMaker starts for each job and releases when the job finishes.

The workshop is in four parts, from AWS basics to training, inference and resource management.

1 Orientation

2 SageMaker

3 Bedrock

4 Cleanup

What cloud computing offers a researcher, when AWS makes sense next to a laptop or a campus cluster, and how its pricing works. Then the setup every later episode uses.

• AWS services and pricing. What SageMaker, Bedrock and S3 each do, and the difference between paying for compute time and paying per token.
• Working environment. Create an S3 bucket for data and results, launch a small SageMaker notebook instance, and read and write files in S3 from Python. Optionally, put your notebooks under version control.
• Cost management. Choose compute to fit the job, estimate the cost before launching, and track spending across the resources you create.

Training, fine-tuning and hyperparameter tuning. SageMaker runs the training scripts on separate compute instances, starting with small models and ending with a vision-language model of billions of parameters. Along the way we compare hardware choices and see how training time, model size and the number of experiments set the cost.

• Model training. Train XGBoost and PyTorch models on the Titanic data. Launch the training jobs from the notebook, compare CPU and GPU, and see when more than one instance helps.
• Hyperparameter tuning. Set up a search over learning rates and epochs, let SageMaker run the training jobs, and compare the results under a cost ceiling.
• Fine-tuning. Adapt CHURRO-3B, an open-weight vision-language model, to transcribe historical German handwriting from the Badger Scribe challenge. Use LoRA to cut the GPU memory needed, estimate the run’s cost, and measure transcription accuracy before and after.

Inference and retrieval-augmented generation (RAG). Training changes a model’s parameters. Inference uses a model that is already trained. We build a RAG pipeline that retrieves passages from research papers and has a language model answer questions from that evidence, then build it again with open-weight models on SageMaker, on the same problem, to compare cost, performance and flexibility.

• RAG with Bedrock. Build the pipeline for WattBot, a challenge of questions about the environmental impact of AI. Split research papers into searchable passages, embed them, retrieve the relevant evidence, and produce answers with citations. The models are Bedrock’s, with usage tracked through tagged inference profiles.
• Multimodal retrieval. Extend the pipeline to retrieve from figures and tables as well as text, and measure whether that improves the answers.
• RAG with SageMaker. Run the embedding and generation models yourself, as GPU Processing jobs that shut down when they finish, and compare the results and the costs with Bedrock. An optional episode runs the same pipeline interactively on a GPU notebook, to see model loading and inference step by step.

Resource management and cost tracking. Cloud resources keep billing after you stop using them, notebook instances and stored data above all. We finish by reviewing what was created, checking what it cost, and cleaning up the account.

• Find the running resources and stored data, stop the notebook instance, and remove what is no longer needed, including S3 buckets and Bedrock inference profiles.
• Use Cost Explorer and the resource tags to see where the workshop’s spending went.
• Review the account rules that kept costs in check during the workshop, and how to set the same safeguards on an AWS account of your own.
• Optionally, launch and manage the same resources from a terminal with the AWS CLI.

The lesson uses four AWS pieces. The overview episode introduces each one in more detail.

The four pieces the lesson uses, and what moves between them

Who this workshop is for

Researchers and students who already use machine learning or AI and want to run that work in the cloud. You should be comfortable with Python and Jupyter and know the basics of model training, hyperparameter tuning, fine-tuning, embeddings and RAG. The emphasis is on using AWS, not on teaching those methods. Instructors may review some of them briefly depending on the audience, but the workshop assumes the background. No previous AWS experience is needed.

Cost controls

One of the workshop’s main points is that cloud computing is affordable for research when the resources are chosen and managed with care. Run as written, the exercises cost each learner a few dollars, CPU and GPU training jobs, hyperparameter tuning, a LoRA fine-tune and RAG inference included.

We estimate the cost before launching a job, set a limit on how long it can run, and check its billable time afterwards. Resources are tagged so that spending can be attributed to each learner and project. We also look at how fast the cost grows when a GPU instance is left running or an experiment is scaled up without checking what it needs.

The workshop uses a shared AWS account with restrictions that block the expensive mistakes. The final part looks at those safeguards and how to set the same ones on an account of your own.

What this workshop does not cover

• General ML/AI instruction. The focus is on running existing ML and AI workflows on AWS, not on teaching the algorithms.
• Model deployment. We do not deploy trained or fine-tuned models as always-on inference endpoints. A model trained elsewhere, on a campus cluster say, can still be used here. Upload its weights to S3 and run inference over a dataset as a Processing job, which is what the RAG with Processing Jobs episode does with an open-weight model. Serving it to an application behind an endpoint is the part the lesson leaves out, and the RAG overview says when that is the right tool.
• AI agents and tool use. We do not cover agent frameworks or applications that let a language model call external tools.

If there is another ML or AI workflow you would like to see covered, we welcome suggestions. Open an issue on the lesson GitHub or email endemann@wisc.edu.

Setup (Complete Before The Workshop)

---

Before the workshop:

1. GitHub Account (optional) – Only needed for the optional episode on pushing your own code to GitHub.
2. AWS Access – Use a shared AWS account (UW-Madison) or an AWS account of your own.
3. Data – Download one small zip.
4. Glossary — Skim the workshop glossary.
5. Episodes 1 to 4 first — They create the bucket and notebook instance every later episode uses.

Details on each step are outlined below.

1. GitHub Account (optional)

Only needed for the optional episode on pushing your notebooks to GitHub. Sign up if you plan to do it.

2. AWS Account

There are two ways to get access to AWS for this lesson. Please wait for a pre-workshop email from the instructor to confirm which option to choose.

Shared account (UW-Madison)

Your own AWS account

If you are attending this lesson at UW-Madison (ML Marathon or Research Bazaar), the instructors will provide a shared AWS account for all attendees. You do not need to set up your own AWS account. What to expect:

• Before the workshop, the instructor adds you to the shared account and emails you a sign-in link. It opens the AWS console directly in the workshop account and role after you sign in with your UW–Madison NetID. Bookmark it; it is the link you will use every session. The link is sent by email and not posted here on purpose: treat it like any other sign-in page and only use the one your instructor sent.
• Verify your access before the workshop: open the link and sign in. If you see a “no access” or “not assigned” message, email the instructor with your username. Check that the account name and region in the top-right of the console match the email.
• Three rules on the shared account. Stick to the lesson materials, so that expensive pipelines such as training or tuning an LLM do not burn the credits. Do not use any tools the lesson does not cover without asking the instructors first. Stop, and at the end of the workshop delete, every notebook instance you create. A GPU notebook left running costs about $60 a day.
• The account enforces a short list of rules on every launch (the three cost tags with values from fixed lists, the lesson’s instance types, a 48-hour cap, no endpoints). A job that breaks one is refused with an AccessDenied message that does not say why; If a Launch Is Refused does, and the lesson’s helper module can tell you before and after the launch. Check there before emailing.

If you are not attending through the shared account, follow these steps:

1. Create an account at aws.amazon.com on the paid plan. Since July 2025 new accounts start on a free plan limited to a set of free-tier services, which does not cover GPU instances.
2. Everything in the workshop is billed. Following the materials end to end costs about $10: the Titanic episodes a few dollars, the Bedrock episode cents, the Processing Jobs episode under a dollar, the fine-tuning episode about $0.75 in class plus about $1.50 for the optional homework. The one way to spend much more is to leave a GPU notebook instance running: about $60 a day. Set a Budgets alert at $50 before you start.
3. Request GPU quotas three to four weeks before the workshop. New accounts have a quota of 0 for GPU instances, and increases can take weeks. In the console, go to Service Quotas → Amazon SageMaker and request: ml.g5.xlarge for training job usage (1), ml.g4dn.xlarge for training job usage (1), ml.g5.xlarge for processing job usage (2), and ml.g6e.xlarge for notebook instance usage (1, optional GPU-notebook episode only). A justification like “self-paced machine learning workshop” is enough.
4. Check under Amazon Bedrock → Model access that the RAG episode’s models, DeepSeek V3 and Titan Text Embeddings V2, show as available in your region. Bedrock enables its serverless models automatically; only Anthropic’s need a use-case form.
5. Create the notebook role. Every notebook and job in the lesson runs as an IAM role named ml-sagemaker-use, which the controllers episode tells you to pick. On your own account you create it, with the Bedrock permissions the RAG episodes need. The commands are under The notebook role and Bedrock on the Account preparation checklist. The guard rails on that page are optional on an account of your own, and worth setting if other people will use it.
6. Stage the shared datasets yourself, since there is no instructor bucket: the WattBot files for the RAG episodes, and the Badger Scribe data and the CHURRO-3B weights for the fine-tuning episode. The commands are on the Shared data bucket page. Set shared_bucket in those episodes to your own bucket.

Steps 3 to 6 need administrator rights on the account, which you have on an account you created. On an institutional account, ask its administrator.

Callout

Instructors, before the workshop

The launch rules the shared-account tab describes are not part of SageMaker. They are policies you attach to the account, and the shared data bucket has to be staged. Work through the instructor pages before the first session: the Account preparation checklist sets the quotas, Bedrock model access, the cost tags and the guard rails, and the Shared data bucket page stages the datasets and model weights the fine-tuning and RAG episodes read. The Instructor Notes index has the rest, including the end-of-workshop cleanup.

Once your AWS account is set up, log in to the AWS Management Console. For the shared account that means the sign-in link from your instructor’s email, of the form https://<org>.awsapps.com/start/#/console?account_id=...&role_name=...; for your own account, the regular console sign-in at https://console.aws.amazon.com/.

3. Download the Data

1. Download data.zip (Right-click -> Save as).
2. Extract it (Right-click -> Extract all on Windows; double-click on Mac).
3. Keep the two Titanic files somewhere you can find them during the S3 episode, e.g.

• ~/Downloads/data/titanic/titanic_train.csv
• ~/Downloads/data/titanic/titanic_test.csv

4. Review the Workshop Glossary Page

Skim the Glossary before the workshop (it is also in the top menu of every lesson page); the terms come up from the first episode.

5. Episodes 1 to 4 come first

Do episodes 1 to 4 before anything else. Episode 2 creates your S3 bucket, episode 3 creates the notebook instance and clones the lesson onto it, which is where every later episode’s notebook comes from, and episode 4 is the first read of your bucket from the notebook.

After that, do episodes 6 to 8 in order, since 8 needs the files 7 saves. Episode 9, the fine-tune, stands on its own. Episodes 10 to 13 also go in order, since each reuses what the one before cached in your bucket.