Minimal reproducible data
Last updated on 2025-08-05 | Edit this page
Overview
Questions
- What is a minimal reproducible dataset, and why do I need it?
- How do I create a minimal reproducible dataset?
- Can I just use my own data?
Objectives
- Describe a minimal reproducible dataset
- Appreciate the different approaches and viewpoints on providing example data
- Identify the relevant aspects of your data
- Create a suitable reprex dataset from scratch
- Share your own dataset in a way that is minimal, accessible, and reproducible
- Subset a built-in dataset to use in your reprex
4.1 What is a minimal reproducible dataset and why do I need it?
[INSERT ROADMAP]
Now that Mickey has narrowed down their problem area and stripped down their code to make it minimal, they need to ensure it is reproducible; this means it needs to be accessible and executable such that anyone else can simoly copy-paste the code into their system and replicate their issue. Importantly, a code snippet will usually require data objects in order to run! Therefore, every reprex requires a minimal reproducible dataset to use with the code.
Furthermore, as we have seen previously, sometimes the source of the problem isn’t actually the code, but rather the data! Providing an appropriate example, or mock dataset allows a helper to better investigate and manipulate that data to fix the problem.
Remember: your helper may not have access to your computer and files!
You might be used to always uploading data from separate files, but helpers can’t access those files. Even if you sent someone a file, they would need to put it in the right directory, make sure to load it in exactly the same way, make sure their computer can open it, etc. Since the goal is to make everyone’s lives as easy as possible, we need to think about our data in a different way–as a mock object created in the script itself.
As with the code, a reprex dataset should also be minimal–free of unnecessary information. By removing extraneous information and only keeping what is required to replicate the issue, we can ensure our helper will be able to easily see how the data is structured and where the problem arises. While it may sometimes feel like unnecessary effort, the process of creating a minimal dataset will not only help others help you, but also allow you to better understand your data and often discover the source of the problem without the need for external help.
In summary, just like with the code, a minimal reproducible dataset must be:
- minimal: it only contains information required to run your minimal code. You can also think of this as being relevant to the problem (keep only what is necessary).
- reproducible: it must be accessible to someone without your computer, and it must consistently replicate your problem. This means it also needs to be complete, meaning there are no dependencies that have been omitted (e.g., packages).
Pro-tip
An example of what minimal reproducible examples look like can also
be found in the ?help section, in R Studio. Just scroll all
the way down to where there are examples listed. These will usually be
minimal and reproducible, since they are intended to be directly
copy-pasted and run by anyone.
For example, let’s look at the function mean:
R
?mean
We see examples that can be run directly on the console, with no additional code.
R
x <- c(0:10, 50)
xm <- mean(x)
c(xm, mean(x, trim = 0.10))
OUTPUT
[1] 8.75 5.50In this case, x is the mock dataset consisting of just 1 variable. Notice how it was created as part of the example. This will be your goal with your reprex.
Exercise 1: Test your knowledge! (3 mins)
The datasets listed below are not well suited for use in a reprex. Can you explain why? Copy each one onto your own R script to check whether they are reproducible. What did you find?
sample_data <- read_csv(“/Users/kaija/Desktop/RProjects/ResearchProject/data/sample_dataset.csv”)sample_data <- data.frame(species = species_vector, weight = c(10, 25, 14, 26, 30, 17))
- Not reproducible because it is a screenshot.
- Not reproducible because it is a path to a file that only exists on someone else’s computer and therefore you do not have access to it using that path.
- Not reproducible because we are not given the source for
species_vector.
Extra Practice (optional)
Let’s say we want to know the average weight of all the species in
our rodents dataset. We try to use the following code…
R
mean(rodents$weight)
OUTPUT
[1] NA…but it returns NA! We don’t know why that is happening and we want to ask for help.
Which of the following represents a minimal reproducible dataset for this code? Can you describe why the other ones are not?
sample_data <- data.frame(month = rep(7:9, each = 2), hindfoot_length = c(10, 25, 14, 26, 30, 17))sample_data <- data.frame(weight = rnorm(10))sample_data <- data.frame(weight = c(100, NA, 30, 60, 40, NA))sample_data <- sample(rodents$weight, 10)sample_data <- rodents_modified[1:20,]
The correct answer is C!
- does not include the variable of interest (weight).
- does not produce the same problem (NA result with a warning message)–the code runs just fine.
- minimal and reproducible.
- is not reproducible. Sample randomly samples 10 items; sometimes it may include NAs, sometime it may not (not guaranteed to reproduce the error). It can be used if a seed is set (see next section for more info).
- uses a dataset that isn’t accessible without previous data wrangling code–the object rodents_modified doesn’t exist.
4.2 Can I just use my own data?
While Mickey is grateful to Remy for providing them with a roadmap to follow when they need help, they still feel it would be much easier and faster to just send Remy their whole dataset rather than creating a different one. If they give Remy access to everything, can’t he still help?
Exercise 2: Reflect (3 mins)
When Mickey feels like sharing their own data would be easier, for whom do you think they are referring? Who would find it easier, Mickey or Remy?
Can you think of any reasons why sharing the original data may not be possible or recommended?
-
Mickey is thinking that it would be easier for themselves, not necessarily for Remy.
Remember: one of the goals of creating a reprex is to help the helpers. They don’t have to help, they are volunteering their time. As such, they deserve to be treated with kindness and respect. If you find yourself getting frustrated at how much time and effort creating a reprex might be taking, remember that (1) trusting the process may reveal the solution along the way; and (2) being kind, clear, and helpful will reward you with a quicker, more accurate solution.
There are several reasons why sharing the original data may not be possible or recommended. The original dataset may be:
- too large - the Portal dataset is ~35,000 rows with 13 columns and contains data for decades. That’s a lot!
- private - the dataset might not be published yet, it may not be yours to share, or maybe it includes protected information such as personal medical information or the location of endangered species.
- hard to send - on most online forums, you can’t attach supplemental files (more on this later). Even if you are just sending data to a colleague, file paths can get complicated, the data might be too large to attach, etc.
- complicated - it would be hard to locate the relevant information.
One example to steer away from are needing a ‘data dictionary’ to
understand all the meanings of the columns (e.g. what is “plot type” in
ratdat?) We don’t our helper to waste valuable time to figure out what everything means. - highly derived/modified from the original file. You may have already done a bunch of preliminary data wrangling you don’t want to include when you send the example, so you would need to provide the intermediate dataset directly to your helper.
Mickey is not entirely wrong. While there are instances when it is not possible or advisable to share original data, there are also many ways around such challenges and some instances may indeed benefit from keeping the original data. However, it is still important to provide helpers with data that is minimal and reproducible. Therefore, while Mickey does not have to create a brand new example dataset, they should at least work to make their original data minimal and accessible (see the above reflection exercise), and this may not end up being easier or faster than creating a mock dataset from scratch.
In summary, there are multiple ways to provide a minimal, reproducible dataset for a reprex, including using a simplified version of the original dataset. In the following section we will highlight 3 common approaches.
4.3 Three different approaches
In general, there are 3 common ways to provide minimal reproducible data for a reprex.
Add a few lines of code to create a mock dataset with the same key characteristics as the original data.
Subset the original data to be minimal and reproducible.
Subset a dataset that is already provided by R (e.g.,
cars,npk,penguins, etc.). For a complete list, uselibrary(help = "datasets").
Pros and Cons
The developers of this lesson believe everyone is entitled to use any option they prefer, therefore the rest of this episode will expand on each of the 3 approaches listed above. However, within the data science community, opinions differ on which method is best recommended. Below is a summary table of advantages and disadvantages of each approach based on many conversations with several data science groups.
| Advantages | Disadvantages | |
| Data from Scratch |
|
|
| R-built Data |
|
|
| Your Data |
|
|
4.4 Creating a mock dataset from scratch
While starting from scratch can be daunting at first, it becomes easier and faster with practice. Once you are familiar with the basic building blocks, it is a very straight-forward method of creating a minimal reproducible dataset. This is also the preferred method for other activities that require a reprex (e.g., teaching, collaborating, developing, etc.), and it often provides valuable problem-solving insights. So let’s breakdown this process to be more digestible!
Mickey is still new at this and has 2 pressing questions:
- How do I create a dataset from scratch?
- How do I know which key aspects of my data to recreate?
Let’s start with the first.
There are many ways one can create a dataset in R (these should be familiar if you took the Carpentries lesson Data Analysis and Visualization in R for Ecologists).
You can start by creating vectors using c()
R
vector <- c(1,2,3,4)
vector
OUTPUT
[1] 1 2 3 4You can also add some randomness by sampling from a vector using
sample().
For example you can sample numbers 1 through 10 in a random order
R
x <- sample(1:10)
x
OUTPUT
[1] 2 4 3 8 6 10 5 9 7 1Or you can randomly sample from a normal distribution
R
x <- rnorm(10)
x
OUTPUT
[1] 0.4067538 -0.6560523 -0.2841006 -0.4533571 -1.2286705 -0.6140703
[7] -0.2668842 0.1158522 0.3218414 0.1549816You can also use letters to create factors.
R
x <- sample(letters[1:4], 20, replace=T)
x
OUTPUT
[1] "a" "d" "a" "d" "d" "d" "c" "a" "c" "d" "c" "b" "a" "b" "c" "d" "a" "a" "d"
[20] "a"Remember that a data frame is just a collection of
vectors. You can create a data frame using
data.frame (or tibble in the
dplyr package), and then define a vector for each
variable.
R
data <- data.frame (x = sample(letters[1:3], 20, replace=T),
y = rnorm(1:20))
head(data)
OUTPUT
x y
1 c 1.0122365
2 b 1.1777645
3 a -0.8310678
4 b -0.9053038
5 c 0.7169477
6 c 0.5596631However, when sampling at random you must remember
to set.seed() before sending it to someone to make sure you
both get the same numbers!
Callout
For more handy functions for creating data frames and variables, see
the [cheatsheet]. Some questions may require specific
formats. For these, you can use any of the provided
as.someType functions: as.factor,
as.integer, as.numeric,
as.character, as.Date,
as.xts.
Exercise 3: You try! (5 mins)
Create a data frame with:
A. One categorical variable with 2 levels and one continuous variable.
B. One continuous variable that is normally distributed.
C. Name, sex, age, and treatment type.
4.5 Identifying the relevant aspects of your data
No matter which approach you choose to take for providing a dataset, they key is always to identify which elements of the original data are necessary, or relevant, to replicate the problem. To do so, here are a few guiding questions:
- Which variables are necessary to replicate the problem?
- What data type (discrete or continuous) is each variable?
- How many levels and/or observations are necessary?
- Do the values need to be distributed in a specific way?
- Are there any NAs that could be relevant?
Let’s check back with Mickey and the minimal code they settled on:
R
# Mickey's minimal code [ UPDATE AS NEEDED ]
library(dplyr)
library(ggplot2)
rodents<-read.csv('data/surveys_complete_77_89.csv')
rodents_subset <- rodents %>%
filter(species == c("ordii", "spectabilis"),
sex == c("F", "M"))
table(rodents_subset$sex, rodents_subset$species)
OUTPUT
ordii spectabilis
F 333 0
M 0 610R
table(rodents$sex, rodents$species)
OUTPUT
albigula audubonii bilineata brunneicapillus chlorurus clarki eremicus
62 69 223 23 11 1 14
F 474 0 0 0 0 0 372
M 368 0 0 0 0 0 468
flavus fulvescens fulviventer fuscus gramineus harrisi hispidus leucogaster
15 0 0 2 3 136 2 16
F 222 46 3 0 0 0 68 373
M 302 16 2 0 0 0 42 397
leucophrys maniculatus megalotis melanocorys merriami ordii penicillatus
2 9 33 13 45 3 6
F 0 160 637 0 2522 690 221
M 0 248 680 0 3108 792 155
scutalatus sp. spectabilis spilosoma squamata taylori torridus viridis
1 18 42 149 16 0 28 1
F 0 4 1135 1 0 0 390 0
M 0 5 1232 1 0 3 441 0To make sure Remy can work on this anywhere, Mickey needs to ensure he has the required dataset to run the code.
Exercise 4: Quick, think! (2 mins)
Based on the current minimal code, which dataset does Mickey need to
recreate? Hint: they currently have two datasets, rodents
and rodents_subset.
Mickey needs to provide a mock dataset to replace the original
rodents dataset.
Let’s take a closer look at the dataset we need to substitute and then answer the questions outlined earlier.
R
head(rodents)
OUTPUT
record_id month day year plot_id species_id sex hindfoot_length weight
1 1 7 16 1977 2 NL M 32 NA
2 2 7 16 1977 3 NL M 33 NA
3 3 7 16 1977 2 DM F 37 NA
4 4 7 16 1977 7 DM M 36 NA
5 5 7 16 1977 3 DM M 35 NA
6 6 7 16 1977 1 PF M 14 NA
genus species taxa plot_type
1 Neotoma albigula Rodent Control
2 Neotoma albigula Rodent Long-term Krat Exclosure
3 Dipodomys merriami Rodent Control
4 Dipodomys merriami Rodent Rodent Exclosure
5 Dipodomys merriami Rodent Long-term Krat Exclosure
6 Perognathus flavus Rodent Spectab exclosureExercise 5: Your turn! (5 mins)
Try to answer the following questions on your own to determine what we need to include in our minimal reproducible dataset:
- Which variables does Mickey need to replicate their problem?
- What data type (discrete or continuous) is each variable?
- How many levels and/or observations are necessary?
- Do the values need to be distributed in a specific way?
- Are there any NAs that could be relevant?
Let’s go over the answers together and help Mickey build a dataset as we go along!
- How many variables does Mickey need to reproduce their problem?
They need species, sex, and maybe a third identifier like record_id. This means they potentially need 3 vectors (remember, each column in a dataframe is essentially a vector, and in “tidy data” should correspond with a variable; each row is then an observation).
R
# create 3 variables: species, sex, and maybe record_id
# a vector for species:
# a vector for sex:
# a vector for record_id:
- What data type (discrete or continuous) is each variable?
Species and sex are both discrete (categorical) variables, while record ID would be more continuous.
R
# create 3 variables: species, sex, and maybe record_id
# a vector for species: categorical
# a vector for sex: categorical
# a vector for record_id: continuous
- How many levels and/or observations are necessary?
Since Mickey is filtering their dataset down to 2 categories for both species and sex, that means they need at least 3 levels in each to start with. In terms of number of observations there don’t seem to be specific restrictions other than they probably want at least 1 observation per original category, so 2*3=6, or they can just pick a generally nice number like 10. This is where creating a reprex dataset becomes a bit more of an art than a science; it is common to use trial and error until the problem is replicated accurately.
R
# create 3 variables: species, sex, and maybe record_id
# a vector for species: categorical with 3 levels
# a vector for sex: categorical with 3 levels
# a vector for record_id: continuous, ~10
- Do the values need to be distributed in a specific way?
This question probably isn’t going to be relevant most of the time, but certainly worth considering. If Mickey needed a longer dataset of measurements they may have wanted to make sure it was normally distributed. If they needed a longer dataset of counts they may have wanted to make sure it was Poisson distributed. Or maybe they had binary data. But in this case, Mickey has a fairly short dataset and the code doesn’t include anything that should vary depending on the distribution, so it probably doesn’t matter. Again, this process can be one of trial and error. They can always come back to this question if they are unable to replicate their problem (hint: in which case the distribution may be related to the problem they are having!).
- Are there any NAs that could be relevant?
Mickey’s data does have NAs for the sex variable. It might not matter or it could be important, so let’s have them put in NAs in the mock dataset just in case.
R
# create 3 variables: species, sex, and maybe record_id
# a vector for species: categorical with 3 levels
species <- sample(letters, 3, replace=F)
# or name 3 categories like we do with sex below
species
OUTPUT
[1] "h" "u" "b"R
# a vector for sex: categorical with 3 levels, one of which is NA
sex <- c('M','F',NA)
sex
OUTPUT
[1] "M" "F" NA R
# a vector for record_id: continuous, ~10
record_id <- 1:10
record_id
OUTPUT
[1] 1 2 3 4 5 6 7 8 9 10R
# Now let's go "sampling" and put our "obervations" in a dataframe
sample_data <- data.frame(
# record_id stays the same, since these are our 10 "observations"
record_id = record_id,
# randomly select 10 observations from our list of species
species = sample(species, 10, replace=T),
# randomly select 10 observations from our list of sexes
sex = sample(sex, 10, replace=T)
)
# Look at our new dataset
sample_data
OUTPUT
record_id species sex
1 1 u F
2 2 u F
3 3 b <NA>
4 4 b <NA>
5 5 u F
6 6 h <NA>
7 7 h <NA>
8 8 h M
9 9 u M
10 10 h FAnd just like that we helped Mickey create a mock dataset from
scratch! Notice that they could also have compiled the same type of
dataset in a single line by creating each vector within
data.frame()
R
sample2_data <- data.frame(
record_id = 1:10,
species = sample(letters[1:3], 10, replace=T),
sex = sample(c('M','F', NA), 10, replace=T)
)
sample2_data
OUTPUT
record_id species sex
1 1 a F
2 2 c <NA>
3 3 b <NA>
4 4 a F
5 5 a F
6 6 c <NA>
7 7 c <NA>
8 8 c <NA>
9 9 a <NA>
10 10 a FImportant: Notice that the outputs of the two
datasets are not the same. If you want the outputs to be EXACTLY the
same each time, but you are using sample() which is an
inherently random process, you must first use set.seed()
and share that with your helper too.
R
set.seed(1) # set seed before recreating the sample
sample_data <- data.frame(
record_id = 1:10,
species = sample(letters[1:3], 10, replace=T),
sex = sample(c('M','F', NA), 10, replace=T)
)
sample_data
OUTPUT
record_id species sex
1 1 a <NA>
2 2 c M
3 3 a M
4 4 b M
5 5 a F
6 6 c F
7 7 c F
8 8 b F
9 9 b <NA>
10 10 c MCallout
Adding a set.seed() at the start of your reprex will
ensure anyone else who runs the same code in the same
order will always get the same results. However, if using it
more generally, you may want to read more about it. For example, in the
example below we set a seed of 2 and then run sample(10)
twice. You will notice that the output of each sample run is not the
same. However, if you run the whole code again, you will see that each
of the outputs actually do stay the same.
R
set.seed(2)
sample(10)
OUTPUT
[1] 5 6 9 1 10 7 4 8 3 2R
sample(10)
OUTPUT
[1] 1 3 6 2 9 10 7 5 4 8Great! Now we need to check whether the mock dataset works with the minimal code Mickey created earlier. Does it run? Does it replicate the problem they were having?
R
# Minimal code [or whatever we end up with]
sample_subset <- sample_data %>% # replace rodents with our sample dataset
filter(species == c("a", "b"), # replace species with those from our sample dataset
sex == c("F", "M")) # this can stay the same because we recreated it the same
table(sample_subset$sex, sample_subset$species)
OUTPUT
a b
F 1 0
M 0 1It works! The sample size has unexpectedly been reduced to just 2 observations, when we would have expected a sample of 8, based on the sample_data output above. Wherever the issue may lie, we were able to successfully replicate it in this minimal reproducible example.
4.6 Using the original data set
Even if you master the art of creating mock datasets, there may be occasions in which your data or problem is too complex and you can’t seem to replicate the issue. Or maybe you still think using your original data would just be easier.
In cases when you want to make your own data minimal and reproducible, you will want to take a similar approach to what we did in Episode 3 when making the code minimal. Keep what is essential, get rid of the rest. In other words, we will want to subset our data into a smaller, more digestible chunk.
The question still arises: how do I know what is essential?
Use the same guiding questions that we used earlier!
- Which variables are necessary to replicate the problem?
- What data type (discrete or continuous) is each variable? (perhaps less necessary, since you are keeping the original variables)
- How many levels and/or observations are necessary? (we don’t want to get rid of more than we need)
- Do the values need to be distributed in a specific way? (worth keeping in mind in terms of how we are removing observations)
- Are there any NAs that could be relevant?
Based on our previous answers we end up with:
- We need species, sex, and maybe record_id
- Species and sex are categorical, record_id is a continuous count of our observations.
- As we said earlier, we want 3 each for species and sex, which happens to already be the case. And we could reduce our record_id size to ~10.
- Not really, but we want to make sure that when we reduce the number of observations we still have observations in each of the 3 levels in species and sex.
- NA’s are present in the sex variable, so let’s make sure we keep at least one.
Now that we have a clearer goal, let’s subset the data.
Useful functions for subsetting a dataset include
subset(), head(), tail(), and
indexing with [] (e.g., iris[1:4,]). Alternatively, you can use
tidyverse functions like select(), and
filter() from the tidyverse. You can also use the same
sample() functions we covered earlier.
Note: you should already have an understanding of how to subset or wrangle data using the tidyverse from the Data Analysis and Visualization in R for Ecologists. If not, go check it out!
R
# Mickey's minimal code [ UPDATE AS NEEDED ]
library(dplyr)
library(ggplot2)
rodents<-read.csv('data/surveys_complete_77_89.csv')
rodents_subset <- rodents %>%
filter(species == c("ordii", "spectabilis"),
sex == c("F", "M"))
table(rodents_subset$sex, rodents_subset$species)
OUTPUT
ordii spectabilis
F 333 0
M 0 610R
table(rodents$sex, rodents$species)
OUTPUT
albigula audubonii bilineata brunneicapillus chlorurus clarki eremicus
62 69 223 23 11 1 14
F 474 0 0 0 0 0 372
M 368 0 0 0 0 0 468
flavus fulvescens fulviventer fuscus gramineus harrisi hispidus leucogaster
15 0 0 2 3 136 2 16
F 222 46 3 0 0 0 68 373
M 302 16 2 0 0 0 42 397
leucophrys maniculatus megalotis melanocorys merriami ordii penicillatus
2 9 33 13 45 3 6
F 0 160 637 0 2522 690 221
M 0 248 680 0 3108 792 155
scutalatus sp. spectabilis spilosoma squamata taylori torridus viridis
1 18 42 149 16 0 28 1
F 0 4 1135 1 0 0 390 0
M 0 5 1232 1 0 3 441 0Given that the code that is going wrong is that which creates rodents_subset, we need to create a minimal reproducible version of rodents! We can then insert our new_rodents dataset in place of the original rodents one.
Step 1: select the variables of interest
R
# subset rodent into new_rodent to make it minimal
# Note: there are many ways you could do this!
new_rodents <- rodents %>%
# 1. select the variables of interest
select(record_id, species, sex)
# PAUSE. Does this work so far?
new_rodents
OUTPUT
record_id species sex
1 1 albigula M
2 2 albigula M
3 3 merriami F
4 4 merriami M
5 5 merriami M
6 6 flavus M
7 7 eremicus F
8 8 merriami M
9 9 merriami F
10 10 flavus F
11 11 spectabilis F
12 12 merriami M
13 13 merriami M
14 14 merriami
15 15 merriami F
16 16 merriami F
17 17 spectabilis F
18 18 penicillatus M
19 19 flavus
20 20 spectabilis F
21 21 merriami F
22 22 albigula F
23 23 merriami M
24 24 hispidus M
25 25 merriami M
26 26 merriami M
27 27 merriami M
28 28 merriami M
29 29 penicillatus M
30 30 spectabilis F
31 31 merriami F
32 32 merriami F
33 33 merriami F
[ reached 'max' / getOption("max.print") -- omitted 16845 rows ]Step 2-5: reduce the number of observations to ~10 while making sure the dataset still contains at least 3 species and at least 3 sexes
While the rest is just one step, it is the trickiest, because this is where we want to ensure the key elements of our original dataset, as defined earlier, are preserved.
Exercise 6: Your Turn! (5 mins)
How would you continue the subsetting pipeline? How could you reduce the number of observations while making sure you still have at least 3 species and 3 sexes left? Hint: there is no single right answer! Trial and error works wonders.
R
set.seed(1)
new_rodents <- rodents %>%
# 1. select the variables of interest
select(record_id, species, sex) %>%
slice_sample(n=4, replace = F, by='sex')
new_rodents
OUTPUT
record_id species sex
1 2359 merriami M
2 16335 albigula M
3 9910 ordii M
4 8278 ordii M
5 12038 merriami F
6 7862 megalotis F
7 9221 albigula F
8 1335 spectabilis F
9 3320 melanocorys
10 343 flavus
11 14482 <NA>
12 9376 spilosoma The code ran without issues, hooray! But do we end up with what we were looking for?
- Do we have ~10 observations? Yes! 9 seems good enough
- Do we have at least 3 species? Yes! We have 7 (we could choose to reduce this further)
- Do we have at least 3 sexes? Yes! M, F, and blank
Great! All of our requirements are fulfilled. Now let’s see if it replicates Mickey’s problem when we add it to their minimal code.
Note: slice_sample() and
similar functions allow you to specify and customize how exactly you
want that sample to be taken (check the documentation!). For example,
you can specify a proportion of rows to select, specify how to order
variables, whether ties [may require more explanation]
should be kept together, or even whether to weigh certain variables. All
of this allows you to keep aspects of your dataset that may be relevant
and hard to replicate otherwise.
Remember the minimal code:
R
rodents_subset <- rodents %>%
filter(species == c("ordii", "spectabilis"),
sex == c("F", "M"))
table(rodents_subset$sex, rodents_subset$species)
OUTPUT
ordii spectabilis
F 333 0
M 0 610R
table(rodents$sex, rodents$species)
OUTPUT
albigula audubonii bilineata brunneicapillus chlorurus clarki eremicus
62 69 223 23 11 1 14
F 474 0 0 0 0 0 372
M 368 0 0 0 0 0 468
flavus fulvescens fulviventer fuscus gramineus harrisi hispidus leucogaster
15 0 0 2 3 136 2 16
F 222 46 3 0 0 0 68 373
M 302 16 2 0 0 0 42 397
leucophrys maniculatus megalotis melanocorys merriami ordii penicillatus
2 9 33 13 45 3 6
F 0 160 637 0 2522 690 221
M 0 248 680 0 3108 792 155
scutalatus sp. spectabilis spilosoma squamata taylori torridus viridis
1 18 42 149 16 0 28 1
F 0 4 1135 1 0 0 390 0
M 0 5 1232 1 0 3 441 0We now want to replace rodents with our
new_rodents. Do we need to change anything else?
We actually still have ordii and spectabilis as species in our list, so we can keep it as is. Same for sex. So we’re all set!
R
new_subset <- new_rodents %>%
filter(species == c("ordii", "spectabilis"),
sex == c("F", "M"))
The code ran without any errors! But does it replicate Mickey’s problem?
Take a step back to remind yourself of what you are looking for. What was the problem we had identified?
- The number of rows that remain after the filter is lower than expected.
So what would we expect to see with this new dataset? Since it is nice and short, this makes it a lot easier to predict the outcome.
- We are asking for the 2 ordii rows, both males, and the 1 spectabilis row, which is female.
R
table(new_subset$sex, new_subset$species)
OUTPUT
< table of extent 0 x 0 >Instead we end up with nothing! Why aren’t we getting the rows we are asking for?
Maybe our table is just wrong, let’s look at the actual dataset we end up with
R
new_subset
OUTPUT
[1] record_id species sex
<0 rows> (or 0-length row.names)Still nothing! What is going on?? We don’t have an answer, but we certainly replicated a problem that occurs when we filter the data. Time to ask for help!
But wait, Mickey’s dataset is now minimal and relevant, but is it
reproducible (accessible outside their device)? Not yet. We created a
subset of their original dataset rodents but this came from
a file on their computer. They could share the csv file and add a line
of code that uploads it… but we already said this is not good practice
for a reprex, and it makes it hard to ask for help on a community
website. Remember, the more steps required, the less likely someone will
want to help.
Thankfully, there is a nifty function dput() that can
help us out. Let’s try it and see what happens.
R
dput(new_rodents)
OUTPUT
structure(list(record_id = c(2359L, 16335L, 9910L, 8278L, 12038L,
7862L, 9221L, 1335L, 3320L, 343L, 14482L, 9376L), species = c("merriami",
"albigula", "ordii", "ordii", "merriami", "megalotis", "albigula",
"spectabilis", "melanocorys", "flavus", NA, "spilosoma"), sex = c("M",
"M", "M", "M", "F", "F", "F", "F", "", "", "", "")), class = "data.frame", row.names = c(NA,
-12L))It spit out a hard-to-read but not excessively long chunk of code.
This code, when run, will recreate the new_rodents dataset!
We can also break it down and label it further to help the reader.
R
reprex_data <- structure(list(
# a unique identifier
record_id = c(2359L, 16335L, 9910L, 8278L, 12038L, 7862L, 9221L, 1335L, 9862L, 14979L, 11333L, 351L),
# a list of species
species = c("merriami", "albigula", "ordii", "ordii", "merriami", "megalotis", "albigula", "spectabilis", "harrisi", "merriami", "spilosoma", "leucogaster"),
# a list of sexes. Note: this includes some blanks!
sex = c("M", "M", "M", "M", "F", "F", "F", "F", "", "", "", "")),
class = "data.frame", row.names = c(NA, -12L))
print(reprex_data)
OUTPUT
record_id species sex
1 2359 merriami M
2 16335 albigula M
3 9910 ordii M
4 8278 ordii M
5 12038 merriami F
6 7862 megalotis F
7 9221 albigula F
8 1335 spectabilis F
9 9862 harrisi
10 14979 merriami
11 11333 spilosoma
12 351 leucogaster Ta-da! Now anyone can easily recreate Mickey’s minimal dataset and use it to run the minimal code. Now, was that really easier than creating a dataset from scratch?
Some of you may still be thinking that you could just use
dput() on the original dataset. And it would work. But that
wouldn’t be very considerate to those who are trying to help.
Exercise 7: Try it! (1 min)
Try running dput(rodents) in your script.
It becomes a huge chunk of code! When clearly we don’t need all of that.
Remember, we want to keep everything minimal for many reasons:
- to make it easy for our helpers to understand our data and code
- to allow helpers to quickly focus their efforts on the right factors
- to make the problem-solving process as easy and painless as possible
- bonus: to help us better understand and zero-in on the source of our problem, often stumbling upon a solution along the way
Nevertheless, it remains an option for when your data appears too complex or you are not quite sure where your problem lies and therefore are not sure what minimal components are needed to reproduce the example. In other words, when you don’t have a good mental model of what the problem is even after going through the initial steps we outlined earlier in the lesson.
4.7 Using an R-built dataset
The last approach we mentioned is to build a minimal reproducible dataset based on the datasets that already exist within R (and therefore everyone would have access to).
A list of readily available datasets can be found using
library(help="datasets"). You can then use ?
in front of the dataset name to get more information about the contents
of the dataset.
For a more detailed discussion of the benefits of using this approach see the Pros and Cons callout in section 4.3.
This approach essentially blends the skills you already learned in
the first two. You need to identify a dataset with appropriate variables
that match the “key elements” of the original dataset. You then need to
further reduce that dataset to a minimal, relevant, number or rows. Once
again, you can use the previously learned functions such as
select(), filter(), or
sample().
Since you already learned everything you need, why not try it yourself?
Exercise 8: Extra Challenge (10 mins)
Using the “HairEyeColor” dataset, create a minimal reproducible dataset for the same issue and minimal code we have been exploring.
Start by using
?HairEyeColorto read a description of the dataset andView(HairEyeColor)to see the actual dataset.Which variables would be a good match for our situation? What are our requirements?
How can we subset this dataset to make it minimal and still replicate our issue?
Remember, there are many possible solutions! The most important feature is that the example dataset can replicate the issue when used within our minimal code.
The following is 1 possible solution:
We selected Hair and Eye as replacements for species and sex because
they are both categorical and have at least 3 levels. We don’t strictly
need anything else. We will call our new rodents
replacement hyc. We set a seed because we want a random
sample.
R
set.seed(1)
# the dummy dataset
hyc <- as.data.frame(HairEyeColor) %>% # oh no! Needs to be converted to df -- might need to change example or have them figure that one out... or we can give them this first line.
select(Hair, Eye) %>%
slice_sample(n=10)
print(hyc)
OUTPUT
Hair Eye
1 Black Hazel
2 Blond Brown
3 Red Blue
4 Black Brown
5 Brown Brown
6 Red Blue
7 Red Hazel
8 Brown Green
9 Brown Brown
10 Red BrownR
# the minimal code
hyc_subset <- hyc %>%
filter(Hair == c('Red','Blonde'),
Eye == c('Blue', 'Brown'))
# illustrating the issue
table(hyc_subset$Hair, hyc_subset$Eye)
OUTPUT
Brown Blue Hazel Green
Black 0 0 0 0
Brown 0 0 0 0
Red 0 1 0 0
Blond 0 0 0 0R
# the whole subset
print(hyc_subset)
OUTPUT
Hair Eye
1 Red BlueR
# But we know there are more!
table(hyc$Hair, hyc$Eye) # Reds have 2 Blue and 1 Brown, and Blonds have 1 Brown!
OUTPUT
Brown Blue Hazel Green
Black 1 0 1 0
Brown 2 0 0 1
Red 1 2 1 0
Blond 1 0 0 0What about NAs?
If your data has NAs and they may be causing the problem, it is
important to include them in your example dataset. You can find where
there are NAs in your dataset by using is.na, for example:
is.na(krats$weight). This will return a logical vector or
TRUE if the cell contains an NA and FALSE if not. The simplest way to
include NAs in your dummy dataset is to directly include it in vectors:
x <- c(1,2,3,NA). You can also subset a dataset that
already contains NAs, or change some of the values to NAs using
mutate(ifelse()) or substitute all the values in a column
by sampling from within a vector that contains NAs.
One important thing to note when subsetting a dataset with NAs is that subsetting methods that use a condition to match rows won’t necessarily match NA values in the way you expect. For example
R
test <- data.frame(x = c(NA, NA, 3, 1), y = rnorm(4))
test %>% filter(x != 3)
OUTPUT
x y
1 1 -0.3053884R
# you might expect that the NA values would be included, since “NA is not equal to 3”. But actually, the expression NA != 3 evaluates to NA, not TRUE. So the NA rows will be dropped!
# Instead you should use is.na() to match NAs
test %>% filter(x != 3 | is.na(x))
OUTPUT
x y
1 NA 0.4874291
2 NA 0.7383247
3 1 -0.3053884Great work! You created a minimal reproducible example. In the next episode, you will learn about reprex, a package that can help you double-check that your example is properly reproducible by running it in a clean environment. (As an added bonus, reprex will format your example nicely so it’s easy to post to places like Slack, GitHub, and StackOverflow.)
Key Points
- A minimal reproducible dataset (a) contains the minimum number of lines, variables, and categories, in the correct format, to replicate your problem; and (b) must be fully reproducible, meaning that someone else can run the same code from anywhere without additional steps.
- To make it accessible, you can create a dataset from scratch using
as.data.frame, you can use an R-built dataset likecars, or you can use a subset of your own dataset and then usedput()to generate reproducible code.
Bonus: Additional Practice
Here are some more practice exercises if you wish to test your knowledge
Extra Practice? Would need to change from mpg, since that’s from ggplot
For each of the following, identify which data are necessary to
create a minimal reproducible dataset using mpg.
- We want to know how the highway mpg has changed over the years
- We need a list of all “types” of cars and their fuel type for each
manufacturer
- We want to compare the average city mpg for a compact car from each
manufacturer
(I copied these from excercise 6 in the google doc… but I’m not sure that they are getting at the point of the lesson…)
Another Excercise
Each of the following examples needs your help to create a dataset that will correctly reproduce the given result and/or warning message when the code is run. Fix the dataset shown or fill in the blanks so it reproduces the problem.
-
set.seed(1)sample_data <- data.frame(fruit = rep(c(“apple”, “banana”), 6), weight = rnorm(12))ggplot(sample_data, aes(x = fruit, y = weight)) + geom_boxplot()HELP: how do I insert an image from clipboard?? Is it even possible? - bodyweight <- c(12, 13, 14, , ) max(bodyweight) [1] NA
- sample_data <- data.frame(x = 1:3, y = 4:6) mean(sample_data\(x) [1] NA Warning message: In mean.default(sample_data\)x): argument is not numeric or logical: returning NA
- sample_data <- ____ dim(sample_data) NULL
- “fruit” needs to be a factor and the order of the levels must be
specified:
sample_data <- data.frame(fruit = factor(rep(c("apple", "banana"), 6), levels = c("banana", "apple")), weight = rnorm(12)) - one of the blanks must be an NA
- ?? + what’s really the point of this one?
sample_data <- data.frame(x = factor(1:3), y = 4:6)