Output and Uncertainty
Last updated on 2026-11-05 | Edit this page
Estimated time: 30 minutes
Overview
Questions
- What is the role of randomness in the outputs generated by a large language model?
- Why is it important to be aware of this randomness when working with a chatbot?
Objectives
- Explain how tokens are selected when model output is generated
- Identify implications of the unpredictability of LLM output
Random output generation
The previous episode explained that model output is influenced by user and system prompt, other context, and the weights that result from the training process. Output is generated one token at a time.
The Transformer Explainer tool illustrates the process leading to token selection. Each time, a list is produced of most probable next tokens and one token selected from this list. The tokens in that list each have an associated probability of being chosen as the next in the sequence. Sometimes there will be a single token with a very high chance of being selected. Other times the probabilities will be roughly equal between multiple different options. This randomness means that different output can be generated from multiple repeats of the process, even if the starting conditions are the same each time. (There is a way to avoid this, by reducing the value of a setting on the model called its temperature, but that discussion is beyond the scope of this lesson.)
The purpose of the activity below is to help learners appreciate the role of randomness in the generation of output, and how random outcomes can compound to produce highly divergent results. It will be most effective if you make use of shared notes in the workshop etherpad and encourage learners to compare the results they are getting.
To get things started: * Choose a starting sentence fragment from the Examples dropdown (the lesson authors have had success with “As the spaceship was approaching the”). * Or, re-use “On the way to the meeting, I picked up a ____” from the previous episode. * Or, provide your own/ask for a suggestion from the learners.
Refer to the additional guidance provided by the inline Instructor Notes in the body of the exercise.
Exploring random token selection
Visit the Transformer Explainer page https://poloclub.github.io/transformer-explainer/. Your instructor will provide you with a starting phrase to type into the box at the top of the page or to choose from the Examples dropdown menu beside it. Press the Generate button. After a moment, you will see a list of possible next tokens appear on the right-hand side of the page, accompanied by their relative probabilities of being chosen. One of those tokens (typically a word or punctuation mark) will have been chosen and added to the sentence at the top of the page.
- Was the token chosen for you the one with the highest probability? If not, how many other tokens had a higher probability of being chosen next?
Work through the list of most likely choices and ask learners to raise their hands if each one was the token chosen on their screen. The numbers of hands raised for each option should roughly correspond to the probabilities of each token. Take a tally of these different options on a white board or in the etherpad.
Exploring random token selection (continued)
Press the Generate button again. Consider this next token and the list of other possible options.
- How do you think these options differ from those produced for people who got a different token in the first iteration?
Referring to the list/tally of different words chosen in the first iteration, ask learners what word they got next and what the top three options were in their list of possible next tokens. This is a good opportunity to get learners thinking about how quickly outputs can diverge from a shared origin.
Exploring random token selection (continued)
Now press Generate several more times, until you reach the tool’s length limit (12 words at the time of writing). Copy and paste the sentence you received into the shared notes document.
- Review the sentences shared by other learners in the workshop. Are there any duplicates? Did anybody else get exactly the same sentence as you?
How does this compare to human communication?
How does the way that large language model output is generated compare to the way that people communicate? How does your interpretation of a piece of text change if you know it was AI-generated, vs human-generated?
The randomness in chatbot response generation is intentional. It reduces the predictability of the chatbot’s responses and makes them feel less “robotic”. But the unpredictability also has some disadvantages. Randomness explains why responses vary from one attempt to the next, but it does not account for why LLMs sometimes generate incorrect output. To understand why LLMs can be wrong we need to examine an issue that has been called ‘hallucination’. ## ‘Hallucinations’ The information in large language model output does not always correspond with observable reality: the text might include incorrect information that could be misleading without verification. These non-factual outputs have become known as hallucinations, although some objections have been raised to this terminology. The model itself does not know whether the words and sentences in the output it is generating are factually accurate. Nor can it be held accountable for the consequences of any inaccuracies.
It’s important to remember that an LLM does not understand the
content of any text or code it generates.
For example, a model that can write a convincing paragraph about quantum
mechanics has not ‘understood’ quantum mechanics. It has learned what
such paragraphs tend to look like. Verification is the responsibility of
the human on whose behalf the text is being generated.
This responsibility and other risks of using AI-generated outputs are discussed in more detail later in the workshop, in the Considering Implications of GenAI Use lesson.
How reliable is model output?
Ask your chatbot to generate the script of a short conference presenter introduction about you. An example prompt:
I am chairing a session at an upcoming [YOUR RESEARCH DOMAIN] conference. One of the presenters is [YOUR NAME]. Please draft the script of a short biography that I can use to introduce the presenter. Maximum four sentences.
Review the response you receive.
- Is the information accurate?
- If there are factual inaccuracies, how serious are they?
- If you were really presenting at a conference, how would you feel if the session chair read this out to introduce you?
As the world’s foremost expert in you (presumably), you are uniquely well positioned to review this output. Reflecting on your assessment of this script, how confident would you feel about using AI to generate the script for an introduction of another researcher in your field?
If you have the option to switch off web searching in your chatbot session, turn it off and try running the prompt again. How does this affect the output?
What does this change?
Imagine that you need to cite a key research paper in a grant application that has to be submitted in the next few hours. You accidentally deleted the paper from your reference manager and need to find its details as quickly as possible. There is a good chance that one of the paper’s authors will review your grant proposal so you also need to make sure that those details are correct!
Discuss and compare the relative merits of taking the following approaches to solve your problem:
- Ask a colleague with comprehensive knowledge of the research literature and a strong memory for details, but who does not answer their phone and has been known to take days to read and respond to messages.
- Search for the paper on the internet.
- Ask a chatbot.
Tools and Models are Changing Fast
Chatbots running in a browser or an app have become more sophisticated in recent years. They can search the internet and have access to a computing environment where they can install and run programs and process the results. The core technology is still the large language model, with additional layers constructed around it. The aim of these additional layers is to augment the capabilities of the model and to mitigate the risks of its output not correlating with reality, e.g. by searching the internet for authoritative information rather than relying on the accuracy of the generated output, or triggering a program to run calculations instead of generating the results itself.
- LLM outputs are generated from tokens randomly selected based on probabilities calculated by the model. The same prompt provided multiple times will usually produce different results even if the starting conditions are identical.
- Bear this randomness in mind as you work with chatbots: responses that appear authoritative, from tools whose responses are often accurate and helpful, can include inaccuracies.