Validating What We Have Made

Last updated on 2026-10-22 | Edit this page

Overview

Questions

Why do we need to check code that a chatbot has generated?
What kinds of errors can be hidden in AI-generated code?
How can we test whether code is doing what we intended?
Which parts of our code deserve the most scrutiny?

Objectives

After following this episode, learners will be able to…

Recognise that evaluation of generated code and validation of results is essential.
Identify ways to test the code generated by a chatbot.
Evaluate the importance of testing different parts of the code they generate.

But Is It True?

AI-generated code can be wrong. Perhaps not often, and especially not when working with common libraries, functions, and tasks similar to those that many people have done in the past.

A 2023 study evaluating the code quality of AI-assisted coding tools found that ChatGPT generated correct code 65.2% of the time and GitHub Copilot generated correct code only 46.3% of the time. Given the rapid improvements to generative AI over the past few years, it may not be fair to suggest these figures are representative of outputs generated by the current models used by ChatGPT and GitHub Copilot. Nonetheless, this study underscores the potential danger of depending solely on AI tools for critical research tasks without carefully reviewing the outputs.

It has been found that LLMs perform worse on unusual and niche tasks, especially those that are not well-represented in their training data. Since research computing tasks are often concerned with questions/data at the limits of current human understanding, you should be prepared to encounter more problems than somebody generating code for more routine tasks in well-trodden territory.

Recall from Episode 3 that outputs from chatbots often have an authoritative tone, regardless of their relevance or factual accuracy. This tone can make us inclined to accept the output without critically evaluating it, but it’s very important that we do.

Whether you wrote the code from scratch or generated it with AI, you are responsible for the results it produces. Therefore, it is up to you to determine whether or not the output generated is accurate and/or does what you need it to do.

In the context of research, some of this validation might take the form of follow-up experiments to confirm hypotheses produced from a computational analysis. But there are many things you can do to detect and correct inaccuracies in the implementation of research software, which might otherwise result in the publication of erroneous claims and resources wasted on further research in the wrong direction.

Types of Errors

There are a few common types of errors to bear in mind when assessing whether your code is correct:

Syntax errors: A syntax error occurs when code breaks the grammatical rules of the programming language such as a missing colon, an unclosed bracket or a misspelled keyword. AI-generated code is generally quite reliable when it comes to syntax because LLMs are good at reproducing the surface-level structure of a programming language. However, syntax errors can still occur when AI stitches together code from different sources or versions of a package.
Logical errors: A logical error is more dangerous because the code runs without crashing, but it doesn’t do what you actually intended. This is possibly the greatest risk when working code, regardless of whether the code was written by an AI or a human. Some examples of logical errors include using the wrong formula, being off by one when indexing a variable, or making a subtly incorrect assumption about your data. Code containing these errors may look plausible and well-structured but will give give you an incorrect output.
Runtime errors: A runtime error occurs when code is syntactically valid and starts running, but fails partway through. This may be due to problems such as trying to divide by zero or access a file that doesn’t exist. You may only notice them with specific inputs or edge cases, meaning AI-generated code can pass initial testing and still fail later on unfamiliar data.

The Code We Will Be Validating

For the challenges in this episode we will extend the life expectancy plotting code from the previous episode. In addition to the plotting function we wrote before, we have asked a chatbot for code that calculates how much life expectancy changed in each country of a continent between the first and last years in the data, and plots the result.

Copy the code below into a new R script and run it. You will need the gapminder_data.csv file from the previous episode in your working directory.

R

library(ggplot2)
 
gapminder <- read.csv("gapminder_data.csv")
 
summary(gapminder)
 
# Calculate the change in life expectancy for each country in a continent
calculate_life_expectancy_change <- function(data, continent_name) {
 
  continent_data <- data[data$continent == continent_name, ]
  countries <- unique(continent_data$country)
 
  results <- data.frame()
 
  for (country_name in countries) {
 
    country_data <- continent_data[continent_data$country == country_name, ]
    country_data <- country_data[order(country_data$year), ]
    n <- nrow(country_data)
 
    # Life expectancy in the earliest and most recent years
    first_life_exp <- country_data$lifeExp[1]
    last_life_exp <- country_data$lifeExp[n - 1]
 
    total_change <- last_life_exp - first_life_exp
    years_covered <- max(country_data$year) - min(country_data$year)
    change_per_year <- total_change / years_covered
 
    results <- rbind(
      results,
      data.frame(
        country = country_name,
        continent = continent_name,
        total_change = total_change,
        change_per_year = change_per_year
      )
    )
  }
 
  return(results)
}
 
# Plot the total change in life expectancy for each country
plot_life_expectancy_change <- function(change_data) {
 
  ggplot(
    data = change_data,
    mapping = aes(x = reorder(country, total_change), y = total_change)
  ) +
    geom_col() +
    coord_flip() +
    labs(
      x = "Country",
      y = "Change in life expectancy (years)"
    )
}
 
# Plot life expectancy over time (from the previous episode)
plot_continent_life_expectancy <- function(data, continent_name) {
 
  continent_data <- data[data$continent == continent_name, ]
 
  ggplot(
    data = continent_data,
    mapping = aes(x = year, y = lifeExp, color = continent)
  ) +
    geom_line() +
    facet_wrap(~ country) +
    labs(
      x = "Year",
      y = "Life expectancy"
    ) +
    theme(
      axis.text.x = element_text(angle = 90, hjust = 1)
    )
}
 
americas_change <- calculate_life_expectancy_change(gapminder, "Americas")
plot_life_expectancy_change(americas_change)
 
plot_continent_life_expectancy(gapminder, "Americas")

The code runs without any error messages and produces sensible-looking plots. But is it correct?

For teaching purposes, a subtle error has been deliberately introduced into this code.

In real use you would not know whether, or where, an error was hiding, which is exactly why it’s important to conduct the checks in this episode.

Challenge

Spot the logical error

Read through the definition of the calculate_life_expectancy_change function carefully, without running anything new.

What does each comment say the code is doing?
Does the code beneath each comment actually do that?
Can you spot anything that looks wrong? How confident are you?

Show me the solution

The line

R

last_life_exp <- country_data$lifeExp[n - 1]

has an off-by-one error. The comment says this is the life expectancy in the most recent year, but because n is the number of rows, [n - 1] selects the second-to-last row. On the gapminder data that is 2002 rather than 2007. It should be [n].

Notice that:

the code runs without complaint,
the results are plausible (every country just shows a slightly smaller improvement than it should), and
the plots look perfectly reasonable.

Did you spot it by reading? If not, don’t worry. This kind of error is easy to miss by eye, which is why the next challenges look at ways of detecting problems by running the code.

How can we detect any errors?

Some errors, such as syntax errors, will stop the script from running and produce an error message. The debugging techniques from previous episodes will be helpful for detecting and fixing these.
However, logical or runtime errors are less straightforward to detect.

‘AI-generated code may introduce plausible but incorrect logic’ is rated as ‘High likelihood’, ‘High severity’ and ‘Very high reach’ on the Responsible AI Risk Register developed by the Institute for Research Software’s Responsible AI in RSE study group https://jshng-glasgow.github.io/Responsible-AI-Risk-Register/

To find these errors, looking at the code is often not enough on its own because:

Once the script/code base gets large, it becomes difficult for one person to keep a complete mental model of how it works, and how any given change will affect the functioning of the whole.
It can also be difficult to consider the code that isn’t there. Changes generated by a model might all seem sensible by themselves, but be missing something that later turns out to be important.

Callout

Technical and Comprehension Debt

Two concepts to bear in mind when deciding whether to use AI assistance for coding are technical debt and comprehension debt.

Technical debt: The cost of maintaining code that was written quickly, without much thought given to its stucture.
Comprehension debt: The cost of relying on code that you don’t fully understand, often because it was written by somebody else and/or generated by AI, leaving you vulnerable to undetected errors and unable to make changes or fixes without assistance.

One risk of AI-assisted coding is the accumulation of comprehension debt. UCL Advanced Research Computing Centre’s policy on AI-assisted software development sets out the expectation that staff remain able to make unassisted changes to the codebases they are responsible for.

So how can we detect these more subtle errors and validate that our code is doing what it should be?

We can test it!

This could be informal, formal, or both.

Informal testing

Minimally, run your code on a small test dataset. Check results carefully. Think first about what you expect to see, and compare the outcome with that expectation.

Challenge

Run code on a small dataset

Create a very small dataset where you can work out the answer by hand:

R

test_data <- data.frame(
  country   = c("A", "A", "B", "B"),
  continent = "Testland",
  year      = c(2000, 2010, 2000, 2010),
  lifeExp   = c(40, 60, 50, 75)
)

In country A, life expectancy was 40 in 2000 and 60 in 2010. In country B, life expectancy was 50 in 2000 and 75 in 2010.

Before running any code, calculate by hand what total_change and change_per_year should be for countries A and B.
Now run calculate_life_expectancy_change(test_data, "Testland").
Do the results match your expectations? If not, what might explain the difference?

Show me the solution

By hand:

Country	Total change	Change per year
A	60 - 40 = 20	20 / 10 = 2
B	75 - 50 = 25	25 / 10 = 2.5

Running the function gives a total_change of 0 for both countries.

The mismatch tells us something is wrong with how the last value is chosen, even though we haven’t yet found the exact line.

Note that it was important to calculate the answer before running the code.

If we had looked at the output first, it would be much easier to convince ourselves that it looked reasonable.

Challenge

Print statements

When looking for logical errors, it can be useful to print out the value of a variable at different stages throughout the code or at each iteration of the loop.

Add the following line inside the loop in calculate_life_expectancy_change(), immediately after last_life_exp is assigned:

R

print(paste(country_name, "| first:", first_life_exp, "| last:", last_life_exp, "| n rows:", n))

Re-create the calculate_life_expectancy_change() function with the print statement included.

Run the calculate_life_expectancy_change() function on the Testland data again. What is unexpected in the output?
Using the information from the output, edit the function so that the variable last_life_exp is actually using the last row in the dataframe.

Show me the solution

1 and 2. The printed last value matches the first row of the data. n appears to refer to the last row of the dataset and therefore n - 1 will refer to the row before the last row. 3. Change n - 1 to n to accurately refer to the last row in the dataset.

Callout

Would an AI really make this mistake?

Not very often. We planted this bug for teaching purposes, and a current chatbot would usually get this line right.

However, there is a plausible reason an off-by-one slip like this could happen. Python counts positions from 0, so the last item in a list of n items is at position n - 1. R counts from 1, so the last item is at position n. A model trained on a lot of Python code can occasionally carry the Python habit over into R.

The same kind of mistake can appear whenever code is translated between languages with different conventions, so it is worth checking even when the code looks sensible. Indeed, noticing mistakes in code can be difficult precisely because they are unpredictable: you should get into the habit of checking for them and validating the output of your code because manually reviewing large volumes of code in detail is very difficult.

Formal testing

Formal testing can be performed at different code levels, each serving a distinct purpose to ensure software behaves correctly at various stages of execution. Together, these testing levels provide a structured approach to improving software quality and reliability. Unit testing is the most granular level, where individual components like functions are tested in isolation to confirm they behave correctly under a variety of inputs. This makes it easier to identify and fix bugs early in the development process.

Programming languages typically have at least one framework to create unit tests, and automated code generation has made it much easier and faster to do. Of course, you will also need to check the tests so that you can be confident that they are correct and/or testing the right things.

Callout

Learn more about Unit Testing

If you’re interested in learning more about unit testing in R you could investigate the testthat package.

For some training materials on unit testing in Python have a look at the Byte-sized RSE lesson on Unit Testing Code

Bear in mind that scientific code may be more difficult to test: a lot of unit tests are concerned with things that are quite easily and logically testable, e.g. whether the output of a function is a positive integer, whether the function fails gracefully when provided with unexpected values, etc. But results produced by scientific code may need to be tested for their plausibility, e.g. whether they respect the laws of thermodynamics, whether a number fits within a biologically feasible range, etc. You will need to use your own domain expertise to inform this kind of evaluation, and it is worth noting that LLMs lack the kind of mental model of the world and how it works that you apply in your daily work, making them less likely to provide useful output in these cases.

Which parts of the code are most important to test?

Allocate your rigour: consider how crucial each part of the code is, and how likely it is that an AI model would get it right, and treat those parts accordingly. For example, routine plotting functions and other standard boilerplate are more likely to be generated correctly and, in the case of the plotting code, errors might be detected quite easily. You might choose to spend less effort reviewing that code. But complex and/or crucial processing and calculations that are particularly important for the results you will be reporting from your work merit a much closer look.

Challenge

Which parts of the code are most important to test?

Look again at the three functions in our script: calculate_life_expectancy_change(), plot_life_expectancy_change() and plot_continent_life_expectancy().

Imagine you will report the results of this analysis in a paper. Which function would you scrutinise most closely? Which least? Why?
Look at the plot produced by plot_life_expectancy_change() for the Americas. Could you have told from the plot alone that the calculation behind it contained an error?
What kinds of errors in the plotting functions would be easy to notice, and what kinds might slip through?

Show me the solution

calculate_life_expectancy_change() deserves the closest scrutiny because its output is what would be reported as a result, and it contains the logic where subtle errors can hide. The two plotting functions contain mostly boilerplate and deserve less effort.
No. The plot of total_change looks perfectly reasonable even with the bug, because every bar is shifted by a similar small amount. A plot can display wrong numbers just as clearly as correct ones, so it can’t validate the calculation behind it.
Errors such as missing axis labels, a wrong colour scale, or an empty or garbled plot are easy to see. Errors in what data is being plotted (for example, plotting the wrong column or a filtered subset by mistake) are much harder to spot, as the plot will still look polished.

Challenge

Ask the chatbot to mark its own homework

Ask the chatbot to review the code and provide constructive feedback on how it could be improved. For example:

Please review the following R code. Check that it does what its comments say it does, and point out any bugs or edge cases that could produce incorrect results.

Then paste in the code from this episode.

To get a different perspective, you could also try starting a new chat session or even opening up a different chatbot and providing some context for the project.

Did the chatbot find the off-by-one error?
Did it suggest any other changes? Were they all genuine improvements?
Were you able to understand these additional suggestions? If not, how could you adjust the prompt to guide the chatbot to produce a more suitable response?
Did you get the same feedback from the second chat session or chatbot?

Show me the solution

Results will vary between chatbots and between runs, so there’s no single right answer. Many chatbots will spot the n - 1 error when asked to compare the comments with the code, but not all will, and some will flag “problems” that aren’t really problems or suggest unnecessary changes. Responses may include suggestions pitched at a greater level of expertise than learners could yet be expected to possess.

A chatbot reviewing code can be a useful additional check, but it is not a substitute for testing.
A review that finds nothing wrong is not proof that the code is correct.
Any suggested fixes still need to be checked. Re-run your small test dataset after making a change to confirm the problem has gone away and no new problem has been introduced.

Errors outside Code Logic

As well as validating the code itself, check packages are legitimate before importing them.

AI tools sometimes hallucinate non-existent coding packages in their outputs. A study by the security company Vulcan identified a cyberattack technique where criminals would publish a malicious package under the name of the hallucinated package, hoping that AI would suggest the same package to other users, who would then install the cyber criminals’ malicious package based on the AI tool’s recommendation.

This practice has become known as ‘slopsquatting’, a combination of ‘AI Slop’ and ‘typosquatting’ (the practice of registering domain names or software package names that are slightly misspelled versions of popular ones to trick users into visiting them or downloading malicious content).

Check that packages exist

Does the package exist?
- Search on pypi.org/CRAN
- Check GitHub for the repository
Is it the official package?
- Check author/maintainer
- Look for signs of legitimacy (stars, downloads, history)
Is the name suspiciously similar to a popular package?
- flask-auth vs flask_auth vs flask-authentication

Challenge

Check that packages exist

Suppose you ask the chatbot how to improve the plots in our script. It replies that you could make them interactive, so that learners can hover over a line to see the exact values, by using a different plotting package called plotly. It suggests adding the following to your script:

R

library(plotly)

americas_change_plot <- plot_life_expectancy_change(americas_change)
ggplotly(americas_change_plot)

Before you install this package, run the checks:

Does the package exist? Search for it on CRAN. Is there also a GitHub repository for it?
Is it the official package? Who is the author/maintainer? Are there signs of legitimacy, such as a long release history, a link to a project website, documentation, or many downloads?
Is the name suspiciously similar to a popular package? Could it be a misspelling or variation of a package you already know? Based on your findings, would you be happy to install it? What else would you want to know before you do?

Show me the solution

It exists. plotly is listed on CRAN, and its CRAN page links to a GitHub repository (plotly/plotly.R).
It appears to be legitimate. The CRAN page names a maintainer and the organisation behind it (Plotly Technologies Inc.), links to a project website and documentation (plotly-r.com), lists a bug-report page, and shows a long history of versions. Its dependencies include ggplot2, which fits with the chatbot’s suggestion of converting an existing ggplot with ggplotly().
The name isn’t a lookalike of something else. plotly is the original, well-known package, not a variation on the name of another one.

On this evidence it is reasonable to install plotly.

AI-generated code that runs cleanly and looks well-structured can still be wrong, and you are accountable for the results it produces. No validation technique can replace your own understanding of the code, so one of the best things you can do now is keep building your ability to read, run and explore code. The better you understand what your code is doing, the sooner you will notice when something is wrong and the more confidently you can judge whether its outputs are correct. This will give you a deeper understanding of the code your research relies on and help you use an AI tool as an assistant rather than an authority.

Key Points

Code that runs without errors and produces plausible-looking output can still contain logical errors.
It’s important to check AI-generated code and in order to sport logical errors.
Work out the expected answer by hand on a small dataset before running the code, then compare.
Print statements let you inspect intermediate values and see what the code is actually doing.
Focus the most scrutiny on the calculations behind the results you will report, and less scrutiny on routine boilerplate such as plotting code.
Asking a chatbot to review code can be a useful extra check, but a review that finds nothing wrong is not proof of correctness, and any suggested fixes need to be re-tested.
It’s not only the code itself that needs validating. Check that suggested packages exist, are the official ones, and are not lookalikes of popular names.