Key Points

What is Reproducible Research?

In this lesson, we have learnt:

  • What is reproducible research?

  • The different terms around reproducibility

Why does Reproducibility matter?

In this lesson, we have learnt:

  • About the reproducibility crisis

  • How reproducibility can be improved broadly

  • How by changing our work to be more reproducible, it can also benefit ourselves

Introducing 7 Steps towards Reproducible Research

Step 1 - Planning to be organisedWhat is your next step?

Step 2 - Keeping your files Tidy and Organised

In this lesson, we have learnt:

  • How to tidy the data inside our files

  • How to organise columns and rows

  • How to handle dates

  • How to handle missing data

We build trust in our knowledge by:

  • Having clarity in our processes for handling missing data

  • Having well structured data

We retain knowledge by:

Minimising the risk of data loss by being well organised

We build business continuity by:

By being organised, others can quickly gain an understanding of the data

Step 3 - Methodology and Protocols

In this lesson, we have learnt:

  • What details to keep about our literature reviews

  • About Preregistrations and where to submit

  • About Registered Reports formats and when they are appropriate

  • What details to include about your methodologies and protocols

  • Why and where to publish your protocols

We build trust in our knowledge by:

  • By being open on our plans in our registered reports and preregistrations

  • Publishing our protocols

  • Being detailed in our methodologies

We retain knowledge using:

  • Detailed recording of our literature review, so we can understand why those papers were reviewed

  • By saving our protocols for use later

We build business continuity by:

  • By saving our protocols for use later

  • By sharing our registered reports and preregistrations

  • By sharing with the team why those papers were reviewed in your literature review

Step 4 - Documentation and writing it down

In this lesson, we have learnt:

  • What documentation is and how it can help us

  • How to consider the audience you are writing for

  • What to document for staff onboarding and offboarding

  • The curse of knowledge and how that skews our perception

  • What to include for documentation

We build trust in our knowledge by:

When asked, we can refer to our notes for further information on what we did, how we did it and why

We retain knowledge using:

Having notes on what we did, how we did it and why saved for our past projects

We build business continuity by:

  • Having proper onboarding and offboarding

  • Sharing segments of documentation across the team

Step 5 - Testing and Controls

In this lesson, we have learnt:

  • Why we should be checking our data for validity and integrity during processing

  • What we should be looking for when inspecting our data

  • Tools for inspecting data

  • Physical testing and hardware QA plays an important part too

  • Our data may have a lineage of origin, and we need to be aware and document the provenance of our data

  • It is important to track our analysis history (and how to record it)

  • That version control is a way to track changes over time

We build trust in our knowledge by:

  • We are testing our data for validity and integrity - and being able to show how we are testing!

  • We are tracking our versions of software, hardware and analysis pipelines, so that it is easier to reproduce later

We retain knowledge using:

  • Tracking metadata about our data (for example, where did an image come from? Who originally made the dataset?) for later reference

  • Recording the different versions of software and hardware, so we can go back to previous versions for reproducibility.

  • Tracking the different versions of our analysis pipelines

We build business continuity by:

  • Keeping versions of our analysis pipeline, so that it is clear what the latest version of the analysis pipeline was

  • Recording the different version hardware and software

  • Sharing where the data originated from via its data lineage information

Step 6 - Automation

In this lesson, we have learnt:

  • Why automation can be beneficial

  • What tools are useful for automation

  • Ideas on what to automate

We build trust in our knowledge by:

  • Showing how we automated work, to eliminate human error

We retain knowledge using:

  • Keeping a copy of these automations

We build business continuity by:

  • Having an automation, so that you are not relying on a staff member to perform this job.

Step 7 - Publishing, Persistent Identifiers and Preparing for Reuse

In this lesson, we have learnt:

  • What the difference is between an identifier and a persistent identifier

  • What a DOI and ORCID is

  • How to get a DOI minted for your articles and datasets

  • If and how to share your datasets

  • What FAIR sharing is, and how mediated sharing works

  • What to consider for licensing

  • Where you can deposit your datasets or grey materials

  • Negative results and how this still can be important to publish

We build trust in our knowledge by:

  • Publishing our data with a license, so that others can reuse it

  • Sharing your data as FAIR, so that people can find you and request your data in a safe way

  • Helping people track who you are via an ORCID id

We retain knowledge using:

  • Having our datasets saved with persistent URL links, so that they are never lost.

  • Attaching our outputs and papers to our ORCID id, so that people can always find us as authors and our scope of work

We build business continuity by:

  • Attaching persistent identifiers to our data when publishing datasets, so that others in the team can reference them

Wrapping up