Minimal reproducible data

Last updated on 2025-08-26 | Edit this page

Overview

Questions

What is a minimal reproducible dataset, and why do I need it?
How do I create a minimal reproducible dataset?
Can I just use my own data?

Objectives

Describe a minimal reproducible dataset
Appreciate the different approaches and viewpoints on providing example data
Identify the relevant aspects of your data
Create a suitable reprex dataset from scratch
Share your own dataset in a way that is minimal, accessible, and reproducible
Subset a built-in dataset to use in your reprex

4.1 What is a minimal reproducible dataset and why do I need it?

Now that Mickey has narrowed down their problem area and stripped down their code to make it minimal, they need to ensure it is reproducible; this means it needs to be accessible and executable such that anyone else can simoly copy-paste the code into their system and replicate their issue. Importantly, a code snippet will usually require data objects in order to run! Therefore, every reprex requires a minimal reproducible dataset to use with the code.

Furthermore, as we have seen previously, sometimes the source of the problem isn’t actually the code, but rather the data! Providing an appropriate example, or mock dataset allows a helper to better investigate and manipulate that data to fix the problem.

Callout

Remember: your helper may not have access to your computer and files!

You might be used to always uploading data from separate files, but helpers can’t access those files. Even if you sent someone a file, they would need to put it in the right directory, make sure to load it in exactly the same way, make sure their computer can open it, etc. Since the goal is to make everyone’s lives as easy as possible, we need to think about our data in a different way–as a mock object created in the script itself.

As with the code, a reprex dataset should also be minimal–free of unnecessary information. By removing extraneous information and only keeping what is required to replicate the issue, we can ensure our helper will be able to easily see how the data is structured and where the problem arises. While it may sometimes feel like unnecessary effort, the process of creating a minimal dataset will not only help others help you, but also allow you to better understand your data and often discover the source of the problem without the need for external help.

In summary, just like with the code, a minimal reproducible dataset must be:

minimal: it only contains information required to run your minimal code. You can also think of this as being relevant to the problem (keep only what is necessary).
reproducible: it must be accessible to someone without your computer, and it must consistently replicate your problem. This means it also needs to be complete, meaning there are no dependencies that have been omitted (e.g., packages).

Callout

Pro-tip

An example of what minimal reproducible examples look like can also be found in the ?help section, in R Studio. Just scroll all the way down to where there are examples listed. These will usually be minimal and reproducible, since they are intended to be directly copy-pasted and run by anyone.

For example, let’s look at the function mean:

R

?mean

We see examples that can be run directly on the console, with no additional code.

R

x <- c(0:10, 50)
xm <- mean(x)
c(xm, mean(x, trim = 0.10))

OUTPUT

[1] 8.75 5.50

In this case, x is the mock dataset consisting of just 1 variable. Notice how it was created as part of the example. This will be your goal with your reprex.

Challenge

Exercise 1: Test your knowledge! (3 mins)

The datasets listed below are not well suited for use in a reprex. Can you explain why? Copy each one onto your own R script to check whether they are reproducible. What did you find?

sample_data <- read_csv(“/Users/kaija/Desktop/RProjects/ResearchProject/data/sample_dataset.csv”)
sample_data <- data.frame(species = species_vector, weight = c(10, 25, 14, 26, 30, 17))

Show me the solution

Not reproducible because it is a screenshot.
Not reproducible because it is a path to a file that only exists on someone else’s computer and therefore you do not have access to it using that path.
Not reproducible because we are not given the source for species_vector.

Challenge

Extra Practice (optional)

Let’s say we want to know the average weight of all the species in our rodents dataset. We try to use the following code…

R

mean(rodents$weight)

OUTPUT

[1] NA

…but it returns NA! We don’t know why that is happening and we want to ask for help.

Which of the following represents a minimal reproducible dataset for this code? Can you describe why the other ones are not?

sample_data <- data.frame(month = rep(7:9, each = 2), hindfoot_length = c(10, 25, 14, 26, 30, 17))
sample_data <- data.frame(weight = rnorm(10))
sample_data <- data.frame(weight = c(100, NA, 30, 60, 40, NA))
sample_data <- sample(rodents$weight, 10)
sample_data <- rodents_modified[1:20,]

Show me the solution

The correct answer is C!

does not include the variable of interest (weight).
does not produce the same problem (NA result with a warning message)–the code runs just fine.
minimal and reproducible.
is not reproducible. Sample randomly samples 10 items; sometimes it may include NAs, sometime it may not (not guaranteed to reproduce the error). It can be used if a seed is set (see next section for more info).
uses a dataset that isn’t accessible without previous data wrangling code–the object rodents_modified doesn’t exist.

4.2 Can I just use my own data?

While Mickey is grateful to Remy for providing them with a roadmap to follow when they need help, they still feel it would be much easier and faster to just send Remy their whole dataset rather than creating a different one. If they give Remy access to everything, can’t he still help?

Challenge

Exercise 2: Reflect (3 mins)

When Mickey feels like sharing their own data would be easier, for whom do you think they are referring? Who would find it easier, Mickey or Remy?
Can you think of any reasons why sharing the original data may not be possible or recommended?

Show me the solution

Mickey is thinking that it would be easier for themselves, not necessarily for Remy.

Remember: one of the goals of creating a reprex is to help the helpers. They don’t have to help, they are volunteering their time. As such, they deserve to be treated with kindness and respect. If you find yourself getting frustrated at how much time and effort creating a reprex might be taking, remember that (1) trusting the process may reveal the solution along the way; and (2) being kind, clear, and helpful will reward you with a quicker, more accurate solution.
There are several reasons why sharing the original data may not be possible or recommended. The original dataset may be:

too large - the Portal dataset is ~35,000 rows with 13 columns and contains data for decades. That’s a lot!
private - the dataset might not be published yet, it may not be yours to share, or maybe it includes protected information such as personal medical information or the location of endangered species.
hard to send - on most online forums, you can’t attach supplemental files (more on this later). Even if you are just sending data to a colleague, file paths can get complicated, the data might be too large to attach, etc.
complicated - it would be hard to locate the relevant information. One example to steer away from are needing a ‘data dictionary’ to understand all the meanings of the columns (e.g. what is “plot type” in ratdat?) We don’t our helper to waste valuable time to figure out what everything means.
highly derived/modified from the original file. You may have already done a bunch of preliminary data wrangling you don’t want to include when you send the example, so you would need to provide the intermediate dataset directly to your helper.

Mickey is not entirely wrong. While there are instances when it is not possible or advisable to share original data, there are also many ways around such challenges and some instances may indeed benefit from keeping the original data. However, it is still important to provide helpers with data that is minimal and reproducible. Therefore, while Mickey does not have to create a brand new example dataset, they should at least work to make their original data minimal and accessible (see the above reflection exercise), and this may not end up being easier or faster than creating a mock dataset from scratch.

In summary, there are multiple ways to provide a minimal, reproducible dataset for a reprex, including using a simplified version of the original dataset. In the following section we will highlight 3 common approaches.

4.3 Three different approaches

In general, there are 3 common ways to provide minimal reproducible data for a reprex.

Add a few lines of code to create a mock dataset with the same key characteristics as the original data.
Subset the original data to be minimal and reproducible.
Subset a dataset that is already provided by R (e.g., cars, npk, penguins, etc.). For a complete list, use library(help = "datasets").

Callout

Pros and Cons

The developers of this lesson believe everyone is entitled to use any option they prefer, therefore the rest of this episode will expand on each of the 3 approaches listed above. However, within the data science community, opinions differ on which method is best recommended. Below is a summary table of advantages and disadvantages of each approach based on many conversations with several data science groups.

	Advantages	Disadvantages
Data from Scratch	Often the most concise Easiest for helpers Helps problem-solve along the way (e.g., identify what data aspects are generating the problem) More universally applicable: Easy to share, collaborate, teach, and understand Avoids privacy/security concerns Lets you clearly illustrate the sought outcome Uses important-to-learn skills Easier with practice	Can be intimidating Requires good understanding of your data Harder to generate if the error is idiosyncratic or dependent on having a large dataset Time-consuming if unpracticed Iterative (you may need to trial and error a few times to replicate the problem) More likely to require analogies, less interpretable/concrete, more likely to require greater context Risks generating XY problems–make sure you are asking the right question/replicating the right problem
R-built Data	Simple and easy to share–no need to provide additional data Familiar–helpers already know the data structure Potentially faster than starting from scratch (i.e. faster than generating rows/columns de novo). No privacy/security concerns Can easily be manipulated or simplified Generalizes the problem	May require a good mental model of the problem Harder if the error is idiosyncratic Greatest risk of generating XY problems–make sure you are asking the right question/replicating the right problem Iterative (you may need to trial and error a few times to replicate the problem) Need to re-think the question so it matches a different dataset and context–mental gymnastics Still need to choose which dataset and variables are more appropriate
Your Data	Can require the least mental effort Problem is easy to replicate even if you don’t understand it at all Accurately represents the actual problem; avoids XY problems. Richer context may intrigue/motivate helpers Can be quicker if dataset is small May be required for idiosyncratic problems that are based on aspects of the data itself that you don’t know about (e.g. when the data itself, rather than the code per se, is central to the problem) Captures data structures that are difficult or time-consuming to replicate if you are a novice	Less growth-minded if chosen as an “easy way out”–skips the learning process and any insights you could have gained. Easy to do poorly and think that you’re doing it well; perceived “easiness” leads to overcomplication/confusion Leaves all the work to the helpers if you don’t also work to minimize it–less motivation for harder work Could obfuscate the problem if not minimized–less likely to find a quick answer More difficult to share–may be large/messy Security/privacy/sanitizeation problems

4.4 Creating a mock dataset from scratch

While starting from scratch can be daunting at first, it becomes easier and faster with practice. Once you are familiar with the basic building blocks, it is a very straight-forward method of creating a minimal reproducible dataset. This is also the preferred method for other activities that require a reprex (e.g., teaching, collaborating, developing, etc.), and it often provides valuable problem-solving insights. So let’s breakdown this process to be more digestible!

Mickey is still new at this and has 2 pressing questions:

How do I create a dataset from scratch?
How do I know which key aspects of my data to recreate?

Let’s start with the first.

There are many ways one can create a dataset in R (these should be familiar if you took the Carpentries lesson Data Analysis and Visualization in R for Ecologists).

You can start by creating vectors using c()

R

vector <- c(1,2,3,4) 
vector

OUTPUT

[1] 1 2 3 4

You can also add some randomness by sampling from a vector using sample().

For example you can sample numbers 1 through 10 in a random order

R

x <- sample(1:10)
x

OUTPUT

 [1]  7  8  4  6  1 10  3  5  2  9

Or you can randomly sample from a normal distribution

R

x <- rnorm(10)
x

OUTPUT

 [1]  0.2513454  0.5913126 -1.3471443  0.8039914  0.1522019  0.5335401
 [7] -0.6697545  0.6394825 -0.3165660 -0.8280896

You can also use letters to create factors.

R

x <- sample(letters[1:4], 20, replace=T)
x

OUTPUT

 [1] "c" "d" "a" "c" "d" "c" "a" "a" "a" "b" "b" "b" "d" "b" "b" "d" "a" "a" "b"
[20] "b"

Remember that a data frame is just a collection of vectors. You can create a data frame using data.frame (or tibble in the dplyr package), and then define a vector for each variable.

R

data <- data.frame (x = sample(letters[1:3], 20, replace=T), 
                    y = rnorm(1:20))
head(data)

OUTPUT

  x          y
1 a -0.3781020
2 c -0.2510934
3 c -0.6544835
4 c  1.9972867
5 c -0.2187458
6 b -1.7033189

However, when sampling at random you must remember to set.seed() before sending it to someone to make sure you both get the same numbers!

Callout

For more handy functions for creating data frames and variables, see the [cheatsheet]. Some questions may require specific formats. For these, you can use any of the provided as.someType functions: as.factor, as.integer, as.numeric, as.character, as.Date, as.xts.

Discussion

Exercise 3: You try! (5 mins)

Create a data frame with:

A. One categorical variable with 2 levels and one continuous variable.

B. One continuous variable that is normally distributed.

C. Name, sex, age, and treatment type.

4.5 Identifying the relevant aspects of your data

No matter which approach you choose to take for providing a dataset, they key is always to identify which elements of the original data are necessary, or relevant, to replicate the problem. To do so, here are a few guiding questions:

Which variables are necessary to replicate the problem?
What data type (discrete or continuous) is each variable?
How many levels and/or observations are necessary?
Do the values need to be distributed in a specific way?
Are there any NAs that could be relevant?

Let’s check back with Mickey and the minimal code they settled on:

R

# Mickey's minimal code [ UPDATE AS NEEDED ]

library(dplyr)
library(ggplot2)

rodents<-read.csv('data/surveys_complete_77_89.csv')

rodents_subset <- rodents %>%
  filter(species == c("ordii", "spectabilis"),
         sex == c("F", "M"))

table(rodents_subset$sex, rodents_subset$species)

OUTPUT


    ordii spectabilis
  F   333           0
  M     0         610

R

table(rodents$sex, rodents$species)

OUTPUT


    albigula audubonii bilineata brunneicapillus chlorurus clarki eremicus
          62        69       223              23        11      1       14
  F      474         0         0               0         0      0      372
  M      368         0         0               0         0      0      468

    flavus fulvescens fulviventer fuscus gramineus harrisi hispidus leucogaster
        15          0           0      2         3     136        2          16
  F    222         46           3      0         0       0       68         373
  M    302         16           2      0         0       0       42         397

    leucophrys maniculatus megalotis melanocorys merriami ordii penicillatus
             2           9        33          13       45     3            6
  F          0         160       637           0     2522   690          221
  M          0         248       680           0     3108   792          155

    scutalatus  sp. spectabilis spilosoma squamata taylori torridus viridis
             1   18          42       149       16       0       28       1
  F          0    4        1135         1        0       0      390       0
  M          0    5        1232         1        0       3      441       0

To make sure Remy can work on this anywhere, Mickey needs to ensure he has the required dataset to run the code.

Challenge

Exercise 4: Quick, think! (2 mins)

Based on the current minimal code, which dataset does Mickey need to recreate? Hint: they currently have two datasets, rodents and rodents_subset.

Show me the solution

Mickey needs to provide a mock dataset to replace the original rodents dataset.

Let’s take a closer look at the dataset we need to substitute and then answer the questions outlined earlier.

R

head(rodents)

OUTPUT

  record_id month day year plot_id species_id sex hindfoot_length weight
1         1     7  16 1977       2         NL   M              32     NA
2         2     7  16 1977       3         NL   M              33     NA
3         3     7  16 1977       2         DM   F              37     NA
4         4     7  16 1977       7         DM   M              36     NA
5         5     7  16 1977       3         DM   M              35     NA
6         6     7  16 1977       1         PF   M              14     NA
        genus  species   taxa                plot_type
1     Neotoma albigula Rodent                  Control
2     Neotoma albigula Rodent Long-term Krat Exclosure
3   Dipodomys merriami Rodent                  Control
4   Dipodomys merriami Rodent         Rodent Exclosure
5   Dipodomys merriami Rodent Long-term Krat Exclosure
6 Perognathus   flavus Rodent        Spectab exclosure

Discussion

Exercise 5: Your turn! (5 mins)

Try to answer the following questions on your own to determine what we need to include in our minimal reproducible dataset:

Which variables does Mickey need to replicate their problem?
What data type (discrete or continuous) is each variable?
How many levels and/or observations are necessary?
Do the values need to be distributed in a specific way?
Are there any NAs that could be relevant?

Let’s go over the answers together and help Mickey build a dataset as we go along!

How many variables does Mickey need to reproduce their problem?

They need species, sex, and maybe a third identifier like record_id. This means they potentially need 3 vectors (remember, each column in a dataframe is essentially a vector, and in “tidy data” should correspond with a variable; each row is then an observation).

R

# create 3 variables: species, sex, and maybe record_id

# a vector for species:

# a vector for sex:

# a vector for record_id:

What data type (discrete or continuous) is each variable?

Species and sex are both discrete (categorical) variables, while record ID would be more continuous.

R

# create 3 variables: species, sex, and maybe record_id

# a vector for species: categorical 

# a vector for sex: categorical

# a vector for record_id: continuous

How many levels and/or observations are necessary?

Since Mickey is filtering their dataset down to 2 categories for both species and sex, that means they need at least 3 levels in each to start with. In terms of number of observations there don’t seem to be specific restrictions other than they probably want at least 1 observation per original category, so 2*3=6, or they can just pick a generally nice number like 10. This is where creating a reprex dataset becomes a bit more of an art than a science; it is common to use trial and error until the problem is replicated accurately.

R

# create 3 variables: species, sex, and maybe record_id

# a vector for species: categorical with 3 levels 

# a vector for sex: categorical with 3 levels 

# a vector for record_id: continuous, ~10

Do the values need to be distributed in a specific way?

This question probably isn’t going to be relevant most of the time, but certainly worth considering. If Mickey needed a longer dataset of measurements they may have wanted to make sure it was normally distributed. If they needed a longer dataset of counts they may have wanted to make sure it was Poisson distributed. Or maybe they had binary data. But in this case, Mickey has a fairly short dataset and the code doesn’t include anything that should vary depending on the distribution, so it probably doesn’t matter. Again, this process can be one of trial and error. They can always come back to this question if they are unable to replicate their problem (hint: in which case the distribution may be related to the problem they are having!).

Are there any NAs that could be relevant?

Mickey’s data does have NAs for the sex variable. It might not matter or it could be important, so let’s have them put in NAs in the mock dataset just in case.

R

# create 3 variables: species, sex, and maybe record_id

# a vector for species: categorical with 3 levels
species <- sample(letters, 3, replace=F) 
          # or name 3 categories like we do with sex below
species

OUTPUT

[1] "p" "a" "g"

R

# a vector for sex: categorical with 3 levels, one of which is NA 
sex <- c('M','F',NA)
sex

OUTPUT

[1] "M" "F" NA

R

# a vector for record_id: continuous, ~10 
record_id <- 1:10 
record_id

OUTPUT

 [1]  1  2  3  4  5  6  7  8  9 10

R

# Now let's go "sampling" and put our "obervations" in a dataframe
sample_data <- data.frame(
  # record_id stays the same, since these are our 10 "observations"
  record_id = record_id,
  # randomly select 10 observations from our list of species
  species = sample(species, 10, replace=T),
  # randomly select 10 observations from our list of sexes
  sex = sample(sex, 10, replace=T)
)

# Look at our new dataset
sample_data

OUTPUT

   record_id species  sex
1          1       p    F
2          2       a    M
3          3       p    F
4          4       a    F
5          5       a    M
6          6       p    M
7          7       g    F
8          8       p    F
9          9       p <NA>
10        10       a    M

And just like that we helped Mickey create a mock dataset from scratch! Notice that they could also have compiled the same type of dataset in a single line by creating each vector within data.frame()

R

sample2_data <- data.frame(
  record_id = 1:10,
  species = sample(letters[1:3], 10, replace=T),
  sex = sample(c('M','F', NA), 10, replace=T)
)

sample2_data

OUTPUT

   record_id species  sex
1          1       b    M
2          2       a    F
3          3       a    M
4          4       b    F
5          5       c    F
6          6       a    F
7          7       a    M
8          8       c    M
9          9       a    M
10        10       c <NA>

Important: Notice that the outputs of the two datasets are not the same. If you want the outputs to be EXACTLY the same each time, but you are using sample() which is an inherently random process, you must first use set.seed() and share that with your helper too.

R

set.seed(1) # set seed before recreating the sample
sample_data <- data.frame(
  record_id = 1:10,
  species = sample(letters[1:3], 10, replace=T),
  sex = sample(c('M','F', NA), 10, replace=T)
)
sample_data

OUTPUT

   record_id species  sex
1          1       a <NA>
2          2       c    M
3          3       a    M
4          4       b    M
5          5       a    F
6          6       c    F
7          7       c    F
8          8       b    F
9          9       b <NA>
10        10       c    M

Callout

Adding a set.seed() at the start of your reprex will ensure anyone else who runs the same code in the same order will always get the same results. However, if using it more generally, you may want to read more about it. For example, in the example below we set a seed of 2 and then run sample(10) twice. You will notice that the output of each sample run is not the same. However, if you run the whole code again, you will see that each of the outputs actually do stay the same.

R

set.seed(2)
sample(10)

OUTPUT

 [1]  5  6  9  1 10  7  4  8  3  2

R

sample(10)

OUTPUT

 [1]  1  3  6  2  9 10  7  5  4  8

Great! Now we need to check whether the mock dataset works with the minimal code Mickey created earlier. Does it run? Does it replicate the problem they were having?

R

# Minimal code [or whatever we end up with] 
sample_subset <- sample_data %>% # replace rodents with our sample dataset
  filter(species == c("a", "b"), # replace species with those from our sample dataset
         sex == c("F", "M")) # this can stay the same because we recreated it the same

table(sample_subset$sex, sample_subset$species)

OUTPUT


    a b
  F 1 0
  M 0 1

It works! The sample size has unexpectedly been reduced to just 2 observations, when we would have expected a sample of 8, based on the sample_data output above. Wherever the issue may lie, we were able to successfully replicate it in this minimal reproducible example.

4.6 Using the original data set

Even if you master the art of creating mock datasets, there may be occasions in which your data or problem is too complex and you can’t seem to replicate the issue. Or maybe you still think using your original data would just be easier.

In cases when you want to make your own data minimal and reproducible, you will want to take a similar approach to what we did in Episode 3 when making the code minimal. Keep what is essential, get rid of the rest. In other words, we will want to subset our data into a smaller, more digestible chunk.

The question still arises: how do I know what is essential?

Use the same guiding questions that we used earlier!

Which variables are necessary to replicate the problem?
What data type (discrete or continuous) is each variable? (perhaps less necessary, since you are keeping the original variables)
How many levels and/or observations are necessary? (we don’t want to get rid of more than we need)
Do the values need to be distributed in a specific way? (worth keeping in mind in terms of how we are removing observations)
Are there any NAs that could be relevant?

Based on our previous answers we end up with:

We need species, sex, and maybe record_id
Species and sex are categorical, record_id is a continuous count of our observations.
As we said earlier, we want 3 each for species and sex, which happens to already be the case. And we could reduce our record_id size to ~10.
Not really, but we want to make sure that when we reduce the number of observations we still have observations in each of the 3 levels in species and sex.
NA’s are present in the sex variable, so let’s make sure we keep at least one.

Now that we have a clearer goal, let’s subset the data.

Useful functions for subsetting a dataset include subset(), head(), tail(), and indexing with [] (e.g., iris[1:4,]). Alternatively, you can use tidyverse functions like select(), and filter() from the tidyverse. You can also use the same sample() functions we covered earlier.

Note: you should already have an understanding of how to subset or wrangle data using the tidyverse from the Data Analysis and Visualization in R for Ecologists. If not, go check it out!

R

# Mickey's minimal code [ UPDATE AS NEEDED ]

library(dplyr)
library(ggplot2)

rodents<-read.csv('data/surveys_complete_77_89.csv')

rodents_subset <- rodents %>%
  filter(species == c("ordii", "spectabilis"),
         sex == c("F", "M"))

table(rodents_subset$sex, rodents_subset$species)

OUTPUT


    ordii spectabilis
  F   333           0
  M     0         610

R

table(rodents$sex, rodents$species)

OUTPUT


    albigula audubonii bilineata brunneicapillus chlorurus clarki eremicus
          62        69       223              23        11      1       14
  F      474         0         0               0         0      0      372
  M      368         0         0               0         0      0      468

    flavus fulvescens fulviventer fuscus gramineus harrisi hispidus leucogaster
        15          0           0      2         3     136        2          16
  F    222         46           3      0         0       0       68         373
  M    302         16           2      0         0       0       42         397

    leucophrys maniculatus megalotis melanocorys merriami ordii penicillatus
             2           9        33          13       45     3            6
  F          0         160       637           0     2522   690          221
  M          0         248       680           0     3108   792          155

    scutalatus  sp. spectabilis spilosoma squamata taylori torridus viridis
             1   18          42       149       16       0       28       1
  F          0    4        1135         1        0       0      390       0
  M          0    5        1232         1        0       3      441       0

Given that the code that is going wrong is that which creates rodents_subset, we need to create a minimal reproducible version of rodents! We can then insert our new_rodents dataset in place of the original rodents one.

Step 1: select the variables of interest

R

# subset rodent into new_rodent to make it minimal
# Note: there are many ways you could do this!
new_rodents <- rodents %>% 
  # 1. select the variables of interest
  select(record_id, species, sex)
  # PAUSE. Does this work so far?
new_rodents

OUTPUT

   record_id      species sex
1          1     albigula   M
2          2     albigula   M
3          3     merriami   F
4          4     merriami   M
5          5     merriami   M
6          6       flavus   M
7          7     eremicus   F
8          8     merriami   M
9          9     merriami   F
10        10       flavus   F
11        11  spectabilis   F
12        12     merriami   M
13        13     merriami   M
14        14     merriami
15        15     merriami   F
16        16     merriami   F
17        17  spectabilis   F
18        18 penicillatus   M
19        19       flavus
20        20  spectabilis   F
21        21     merriami   F
22        22     albigula   F
23        23     merriami   M
24        24     hispidus   M
25        25     merriami   M
26        26     merriami   M
27        27     merriami   M
28        28     merriami   M
29        29 penicillatus   M
30        30  spectabilis   F
31        31     merriami   F
32        32     merriami   F
33        33     merriami   F
 [ reached 'max' / getOption("max.print") -- omitted 16845 rows ]

Step 2-5: reduce the number of observations to ~10 while making sure the dataset still contains at least 3 species and at least 3 sexes

While the rest is just one step, it is the trickiest, because this is where we want to ensure the key elements of our original dataset, as defined earlier, are preserved.

Discussion

Exercise 6: Your Turn! (5 mins)

How would you continue the subsetting pipeline? How could you reduce the number of observations while making sure you still have at least 3 species and 3 sexes left? Hint: there is no single right answer! Trial and error works wonders.

R

set.seed(1)
new_rodents <- rodents %>% 
  # 1. select the variables of interest
  select(record_id, species, sex) %>%
  slice_sample(n=4, replace = F, by='sex')
new_rodents

OUTPUT

   record_id     species sex
1       2359    merriami   M
2      16335    albigula   M
3       9910       ordii   M
4       8278       ordii   M
5      12038    merriami   F
6       7862   megalotis   F
7       9221    albigula   F
8       1335 spectabilis   F
9       3320 melanocorys
10       343      flavus
11     14482        <NA>
12      9376   spilosoma

The code ran without issues, hooray! But do we end up with what we were looking for?

Do we have ~10 observations? Yes! 9 seems good enough
Do we have at least 3 species? Yes! We have 7 (we could choose to reduce this further)
Do we have at least 3 sexes? Yes! M, F, and blank

Great! All of our requirements are fulfilled. Now let’s see if it replicates Mickey’s problem when we add it to their minimal code.

Note: slice_sample() and similar functions allow you to specify and customize how exactly you want that sample to be taken (check the documentation!). For example, you can specify a proportion of rows to select, specify how to order variables, whether ties [may require more explanation] should be kept together, or even whether to weigh certain variables. All of this allows you to keep aspects of your dataset that may be relevant and hard to replicate otherwise.

Remember the minimal code:

R

rodents_subset <- rodents %>%
  filter(species == c("ordii", "spectabilis"),
         sex == c("F", "M"))

table(rodents_subset$sex, rodents_subset$species)

OUTPUT


    ordii spectabilis
  F   333           0
  M     0         610

R

table(rodents$sex, rodents$species)

OUTPUT


    albigula audubonii bilineata brunneicapillus chlorurus clarki eremicus
          62        69       223              23        11      1       14
  F      474         0         0               0         0      0      372
  M      368         0         0               0         0      0      468

    flavus fulvescens fulviventer fuscus gramineus harrisi hispidus leucogaster
        15          0           0      2         3     136        2          16
  F    222         46           3      0         0       0       68         373
  M    302         16           2      0         0       0       42         397

    leucophrys maniculatus megalotis melanocorys merriami ordii penicillatus
             2           9        33          13       45     3            6
  F          0         160       637           0     2522   690          221
  M          0         248       680           0     3108   792          155

    scutalatus  sp. spectabilis spilosoma squamata taylori torridus viridis
             1   18          42       149       16       0       28       1
  F          0    4        1135         1        0       0      390       0
  M          0    5        1232         1        0       3      441       0

We now want to replace rodents with our new_rodents. Do we need to change anything else?

We actually still have ordii and spectabilis as species in our list, so we can keep it as is. Same for sex. So we’re all set!

R

new_subset <- new_rodents %>%
  filter(species == c("ordii", "spectabilis"),
         sex == c("F", "M"))

The code ran without any errors! But does it replicate Mickey’s problem?

Take a step back to remind yourself of what you are looking for. What was the problem we had identified?

The number of rows that remain after the filter is lower than expected.

So what would we expect to see with this new dataset? Since it is nice and short, this makes it a lot easier to predict the outcome.

We are asking for the 2 ordii rows, both males, and the 1 spectabilis row, which is female.

R

table(new_subset$sex, new_subset$species)

OUTPUT

< table of extent 0 x 0 >

Instead we end up with nothing! Why aren’t we getting the rows we are asking for?

Maybe our table is just wrong, let’s look at the actual dataset we end up with

R

new_subset

OUTPUT

[1] record_id species   sex
<0 rows> (or 0-length row.names)

Still nothing! What is going on?? We don’t have an answer, but we certainly replicated a problem that occurs when we filter the data. Time to ask for help!

But wait, Mickey’s dataset is now minimal and relevant, but is it reproducible (accessible outside their device)? Not yet. We created a subset of their original dataset rodents but this came from a file on their computer. They could share the csv file and add a line of code that uploads it… but we already said this is not good practice for a reprex, and it makes it hard to ask for help on a community website. Remember, the more steps required, the less likely someone will want to help.

Thankfully, there is a nifty function dput() that can help us out. Let’s try it and see what happens.

R

dput(new_rodents)

OUTPUT

structure(list(record_id = c(2359L, 16335L, 9910L, 8278L, 12038L,
7862L, 9221L, 1335L, 3320L, 343L, 14482L, 9376L), species = c("merriami",
"albigula", "ordii", "ordii", "merriami", "megalotis", "albigula",
"spectabilis", "melanocorys", "flavus", NA, "spilosoma"), sex = c("M",
"M", "M", "M", "F", "F", "F", "F", "", "", "", "")), class = "data.frame", row.names = c(NA,
-12L))

It spit out a hard-to-read but not excessively long chunk of code. This code, when run, will recreate the new_rodents dataset! We can also break it down and label it further to help the reader.

R

reprex_data <- structure(list(
  
# a unique identifier
record_id = c(2359L, 16335L, 9910L, 8278L, 12038L, 7862L, 9221L, 1335L, 9862L, 14979L, 11333L, 351L), 

# a list of species
species = c("merriami", "albigula", "ordii", "ordii", "merriami", "megalotis", "albigula", "spectabilis", "harrisi", "merriami", "spilosoma", "leucogaster"), 

# a list of sexes. Note: this includes some blanks!
sex = c("M", "M", "M", "M", "F", "F", "F", "F", "", "", "", "")),

class = "data.frame", row.names = c(NA, -12L))

print(reprex_data)

OUTPUT

   record_id     species sex
1       2359    merriami   M
2      16335    albigula   M
3       9910       ordii   M
4       8278       ordii   M
5      12038    merriami   F
6       7862   megalotis   F
7       9221    albigula   F
8       1335 spectabilis   F
9       9862     harrisi
10     14979    merriami
11     11333   spilosoma
12       351 leucogaster

Ta-da! Now anyone can easily recreate Mickey’s minimal dataset and use it to run the minimal code. Now, was that really easier than creating a dataset from scratch?

Some of you may still be thinking that you could just use dput() on the original dataset. And it would work. But that wouldn’t be very considerate to those who are trying to help.

Discussion

Exercise 7: Try it! (1 min)

Try running dput(rodents) in your script.

It becomes a huge chunk of code! When clearly we don’t need all of that.

Remember, we want to keep everything minimal for many reasons:

to make it easy for our helpers to understand our data and code
to allow helpers to quickly focus their efforts on the right factors
to make the problem-solving process as easy and painless as possible
bonus: to help us better understand and zero-in on the source of our problem, often stumbling upon a solution along the way

Nevertheless, it remains an option for when your data appears too complex or you are not quite sure where your problem lies and therefore are not sure what minimal components are needed to reproduce the example. In other words, when you don’t have a good mental model of what the problem is even after going through the initial steps we outlined earlier in the lesson.

4.7 Using an R-built dataset

The last approach we mentioned is to build a minimal reproducible dataset based on the datasets that already exist within R (and therefore everyone would have access to).

A list of readily available datasets can be found using library(help="datasets"). You can then use ? in front of the dataset name to get more information about the contents of the dataset.

For a more detailed discussion of the benefits of using this approach see the Pros and Cons callout in section 4.3.

This approach essentially blends the skills you already learned in the first two. You need to identify a dataset with appropriate variables that match the “key elements” of the original dataset. You then need to further reduce that dataset to a minimal, relevant, number or rows. Once again, you can use the previously learned functions such as select(), filter(), or sample().

Since you already learned everything you need, why not try it yourself?

Challenge

Exercise 8: Extra Challenge (10 mins)

Using the “HairEyeColor” dataset, create a minimal reproducible dataset for the same issue and minimal code we have been exploring.

Start by using ?HairEyeColor to read a description of the dataset and View(HairEyeColor) to see the actual dataset.
Which variables would be a good match for our situation? What are our requirements?
How can we subset this dataset to make it minimal and still replicate our issue?

Show me the solution

Remember, there are many possible solutions! The most important feature is that the example dataset can replicate the issue when used within our minimal code.

The following is 1 possible solution:

We selected Hair and Eye as replacements for species and sex because they are both categorical and have at least 3 levels. We don’t strictly need anything else. We will call our new rodents replacement hyc. We set a seed because we want a random sample.

R

set.seed(1)

# the dummy dataset
hyc <- as.data.frame(HairEyeColor) %>% # oh no! Needs to be converted to df -- might need to change example or have them figure that one out... or we can give them this first line.
  select(Hair, Eye) %>%
  slice_sample(n=10)
print(hyc)

OUTPUT

    Hair   Eye
1  Black Hazel
2  Blond Brown
3    Red  Blue
4  Black Brown
5  Brown Brown
6    Red  Blue
7    Red Hazel
8  Brown Green
9  Brown Brown
10   Red Brown

R

# the minimal code
hyc_subset <- hyc %>%
  filter(Hair == c('Red','Blonde'),
         Eye == c('Blue', 'Brown'))

# illustrating the issue
table(hyc_subset$Hair, hyc_subset$Eye)

OUTPUT


        Brown Blue Hazel Green
  Black     0    0     0     0
  Brown     0    0     0     0
  Red       0    1     0     0
  Blond     0    0     0     0

R

# the whole subset
print(hyc_subset)

OUTPUT

  Hair  Eye
1  Red Blue

R

# But we know there are more!
table(hyc$Hair, hyc$Eye) # Reds have 2 Blue and 1 Brown, and Blonds have 1 Brown!

OUTPUT


        Brown Blue Hazel Green
  Black     1    0     1     0
  Brown     2    0     0     1
  Red       1    2     1     0
  Blond     1    0     0     0

Callout

What about NAs?

If your data has NAs and they may be causing the problem, it is important to include them in your example dataset. You can find where there are NAs in your dataset by using is.na, for example: is.na(krats$weight). This will return a logical vector or TRUE if the cell contains an NA and FALSE if not. The simplest way to include NAs in your dummy dataset is to directly include it in vectors: x <- c(1,2,3,NA). You can also subset a dataset that already contains NAs, or change some of the values to NAs using mutate(ifelse()) or substitute all the values in a column by sampling from within a vector that contains NAs.

One important thing to note when subsetting a dataset with NAs is that subsetting methods that use a condition to match rows won’t necessarily match NA values in the way you expect. For example

R

test <- data.frame(x = c(NA, NA, 3, 1), y = rnorm(4))
test %>% filter(x != 3)

OUTPUT

  x          y
1 1 -0.3053884

R

# you might expect that the NA values would be included, since “NA is not equal to 3”. But actually, the expression NA != 3 evaluates to NA, not TRUE. So the NA rows will be dropped!
# Instead you should use is.na() to match NAs
test %>% filter(x != 3 | is.na(x))

OUTPUT

   x          y
1 NA  0.4874291
2 NA  0.7383247
3  1 -0.3053884

Great work! You created a minimal reproducible example. In the next episode, you will learn about reprex, a package that can help you double-check that your example is properly reproducible by running it in a clean environment. (As an added bonus, reprex will format your example nicely so it’s easy to post to places like Slack, GitHub, and StackOverflow.)

Key Points

A minimal reproducible dataset (a) contains the minimum number of lines, variables, and categories, in the correct format, to replicate your problem; and (b) must be fully reproducible, meaning that someone else can run the same code from anywhere without additional steps.
To make it accessible, you can create a dataset from scratch using as.data.frame, you can use an R-built dataset like cars, or you can use a subset of your own dataset and then use dput() to generate reproducible code.

Bonus: Additional Practice

Here are some more practice exercises if you wish to test your knowledge

Discussion

Extra Practice? Would need to change from mpg, since that’s from ggplot

For each of the following, identify which data are necessary to create a minimal reproducible dataset using mpg.

We want to know how the highway mpg has changed over the years
We need a list of all “types” of cars and their fuel type for each manufacturer
We want to compare the average city mpg for a compact car from each manufacturer

(I copied these from excercise 6 in the google doc… but I’m not sure that they are getting at the point of the lesson…)

Challenge

Another Excercise

Each of the following examples needs your help to create a dataset that will correctly reproduce the given result and/or warning message when the code is run. Fix the dataset shown or fill in the blanks so it reproduces the problem.

set.seed(1) sample_data <- data.frame(fruit = rep(c(“apple”, “banana”), 6), weight = rnorm(12)) ggplot(sample_data, aes(x = fruit, y = weight)) + geom_boxplot() HELP: how do I insert an image from clipboard?? Is it even possible?
bodyweight <- c(12, 13, 14, , ) max(bodyweight) [1] NA
sample_data <- data.frame(x = 1:3, y = 4:6) mean(sample_data$x) [1] NA Warning message: In mean.default(sample_data$x): argument is not numeric or logical: returning NA
sample_data <- ____ dim(sample_data) NULL

Show me the solution

“fruit” needs to be a factor and the order of the levels must be specified: sample_data <- data.frame(fruit = factor(rep(c("apple", "banana"), 6), levels = c("banana", "apple")), weight = rnorm(12))
one of the blanks must be an NA
?? + what’s really the point of this one?
sample_data <- data.frame(x = factor(1:3), y = 4:6)