Open Science at CMOR

CMOR Lunch’n’Learn

29 June 2023

Ross Wilson

Introduction to Open Science

What is Open Science?


Open science is a set of principles and practices that aim to make scientific research from all fields accessible to everyone for the benefits of scientists and society as a whole.

— UNESCO Recommendation on Open Science

“Closed” Science


  • Research published only in paywalled journals that not everyone can access

  • Data that supports scientific results being unavailable

  • Software, code, protocols, etc. being unknown, undocumented, or inaccessible

  • Science not being accessible to communities that would benefit from it

Principles of Open Science


  1. Transparency, scrutiny, critique, and responsibility

  2. Equality of opportunities

  3. Responsibility, respect, and accountability

  4. Collaboration, participation, and inclusion

  5. Flexibility

  6. Sustainability

Principles of Open Science

  • Today we’ll mostly focus on the first principle

    • Transparency, or what we might call ‘open scientific knowledge’


  • Are our publications, data, metadata, protocols, educational resources, software, source code freely available to all?


(This is one end of a continuum of ‘openness’ – depending on the context, nature of the data, etc., it may not be desirable to make all of these completely open in all circumstances)

The Research Workflow


  1. We design a study to answer a research question


  1. We collect some data


  1. We analyse the data and write a paper


  1. The paper gets published

The Research Workflow


  • Usually, only

4. The paper gets published

results in any accessible output


(And often then it is behind a journal paywall and not accessible to all)


  • 1., 2., and 3. are often a black box, accessible only to the researcher themselves

Components of Open Scientific Knowledge


  • Open protocols/documentation
  • Open research data
  • Open software/source code
  • Open access publication

Open protocols/documentation


  • RCTs and systematic reviews now generally require registration and pre-specified protocols, e.g.


  • Usually not required for other study types, but still a good idea

Open research data


  • Open data allows other researchers to:
    • check the correctness of your analysis
    • combine your data with other sources (e.g. IPD meta-analysis)
    • use your data to answer questions you hadn’t even thought of…
  • For health data, it is very important to consider the appropriateness of data sharing:
    • is additional data preprocessing needed to ensure participant anonymity?
    • should there be restrictions on who can access data and for what purpose?
  • If data is being shared, it needs to be documented clearly

Data sharing – a cautionary tale

Open software/source code


  • As with open data, sharing source code allows others to check the correctness of your analysis


  • Wherever possible, the use of open source or freely-available software allows others (or yourself later) to replicate your work

Open access publication


  • Research is of no use if no one can access it…

  • To ensure our research can reach the widest possible audience, aim for open access publishing whenever possible

  • Otherwise you can usually post either a ‘pre-print’ or ‘post-print’ (accepted manuscript) on an institutional (e.g. OURArchive) or other repository, sometimes after an embargo period.

    • Sherpa Services provides an easy search function to look up journals’ sharing policies

Open Science Framework

Open Science Framework


  • At CMOR, we have started using the Open Science Framework (OSF) for collaborative open science



  • The OSF allows both private work (within the research team) and public sharing for dissemination and accessibility

Project organisation on the OSF


  • Work on the OSF is organised into ‘Projects’, which can be nested to create a structured organisation for all of your files, data, code, protocols, and outputs


  • Within a projected, nested sub-projects are referred to as ‘Components’
    • The advantage of nesting components within a project is that each component can have its own permission settings (contributors, public/private access, etc)

CMOR @ OSF

  • We have set up such a nested structure for research projects at CMOR:

Centre for Musculoskeletal Outcomes Research

  • Reducing the burden of osteoarthritis in Aotearoa New Zealand
    • Discrete Event Simulation modelling
    • Risks, Impacts, Equity, and Cost-effectiveness
      • Long-term impacts of ligament injury
        • Analysis plan
      • Impacts of opioid use around TJR surgery
      • Cost-effectiveness of TJR surgery
    • He Oranga mō te whanau
    • Mātauranga
  • Measuring the health state preferences of New Zealanders
    • DCE
      • Data management plan
      • Protocols
      • Data
      • Analysis
      • Publications
  • Graduate students
    • Richelle Caya
      • Systematic review
    • …

Preregistration

  • Preregistration is the practice of documenting your research plan at the beginning of your study and storing that plan in a read-only public repository (such as the OSF)

  • Preregistration, including a detailed analysis plan, helps to address issues of reproducibility through reducing the potential for bias in analysis and reporting

  • See Create a preregistration for information on creating and submitting a preregistration for a project on the OSF

    • Note that once a preregistration is submitted and approved, you cannot make any changes (by design!), so check everything carefully

Resources

Return to CMOR Lunch’n’Learn presentations