Policies for Researchers

This page provides important instructions that must be read before the sharing and publication of any OpenSAFELY project results released from the Level 4 results server.

If you have any questions, in the first instance contact your co-pilot; if you do not have a co-pilot, please contact [email protected].

Permitted Study Results Policy

All outputs from the NHS England OpenSAFELY COVID-19 service must be aggregated data with small number suppression applied.

The service operates as a trusted research platform where no patient record level data is permitted to be extracted from the platform.

You MUST NOT request the release of any information (e.g. name, listsize) that identifies, or could identify, ICBs, Local Authorities (including MSOA identifiers), Primary Care Networks (PCNs) and individual GP practices from the Level 4 results server.

Larger geographic / regional outputs can be released, such as NHS England operating regions, which are listed in relevant data tables in the OpenSAFELY platform. An example use of these regions is in Table 1, p.3 of this paper.

Refer to the Datasets Used heading regarding the general rules around the sharing of results and the publication of results. Some datasets have their own additional rules for the sharing and publication of results. Make sure you read the information for each dataset carefully.

OpenSAFELY Analytic Methods Policy

Introduction

This policy outlines the analytic methods that are currently supported and not supported within OpenSAFELY due to resource for compute and output checking. The policy is approved by NHS England, and managed in collaboration with the OpenSAFELY team.

The development of this policy takes into account a variety of constraints, such as: the available and funded OpenSAFELY compute; the capacity within the output-checking service, which oversees statistical disclosure controls (SDC), to satisfy contractual, legal and ethical obligations; the unknown privacy risks associated with some aspects of some analysis methods (eg generative and risk prediction models).

In addition, we must ensure we bring the public, profession and other stakeholders with us if and when there are analytic approaches that have, or are perceived to have, additional challenges around interpretability, explainability, safety, efficacy, cost-effectiveness¹ and privacy (such as machine learning / AI models) are being run against GP and NHS data; maintaining the trust of the public and profession is a key priority for teams running the OpenSAFELY service.

The OpenSAFELY output-checking service supports most established analytical methods. The following table is an indicative and non-exhaustive list:

The following table lists the analytic methods currently not-supported by the OpenSAFELY output-checking service which oversees statistical disclosure controls (SDC).

Your OpenSAFELY study must not use any libraries, scripts, or run any code that uses these analytic methods, without exceptional prior permission (see below):

Your responsibility as OpenSAFELY users (researchers, analysts, data scientists)

1. Study lead(s) must familiarise themself with the not SDC-supported methods.
2. The study lead(s) must ensure only outputs from SDC-supported methods are requested for release.

This policy will also be named in the data access agreement that all approved users sign.

Important notes

1. Users are not permitted to run methods on the Non SDC-Supported list even if they are not planning to release the model for external querying or inspection, without exceptional prior permission (see below).
   • It might be acceptable for a non-SDC support method to be used for an internal project analysis need (for example, using such models for propensity matching, inverse propensity weighting) with no requirement to release the outputs, model or performance statistics. Discuss this with your co-pilot: explain the need and purpose and we will let you know.
2. We do not support the release of models using Non SDC-Supported methods because (not exhaustive list):
   • It is extremely labour intensive to evaluate large volumes of potentially disclosive data for release outside a TRE/SDE; and for some types of model that might be proposed for export there are no currently recognised means to evaluate disclosivity risk in any setting.
   • Development of models that cannot be used as intended (e.g., for implementing risk prediction tools outside of OpenSAFELY) will utilise compute and resources unnecessarily.
3. If you are planning to conduct analyses to produce a risk prediction model using the SDC-Supported methods, you must detail this on your application form “Study Information” section, so that NHS England and others can consider any regulatory or liability issues; in particular you must specify if you intend to implement the model in clinical practice. However, any risk prediction model files generated (using the SDC-Supported methods) may still not be approved for release if the model files are unreasonably large such that output-checkers cannot effectively assess for disclosivity risks.
4. It is the responsibility of users to ensure outputs requested for release are clearly explained to output-checkers; only the minimal required; clearly in line with the project purpose; in line with this policy.

How this policy will be updated

As OpenSAFELY’s features and resources change over time, it is possible that various methods on the not SDC-supported list will be supported. To help us prioritise how the platform develops, email us ([email protected], subject: analytic methods suggestions) and share the specific method(s) of interest you would like us to support, including:

1. Why is the method necessary for a future your project?
2. Can the analysis not be conducted using supported methods?
3. What is the intended benefit of using the method?
4. What have you considered to be the risks of using the method?
5. How do you intend to mitigate these risks? Specifically, cite how you think the statistical disclosure control process could work.

We encourage users to also talk with their co-pilots as part of this feedback.

Requests for non SDC-supported methods will be logged and reviewed on a quarterly basis by NHS England, the OpenSAFELY team and shared with our governance board to determine how this policy will evolve over time.²

Authorship Policy

Our team is strongly committed to “team science”, and to recognising the deep technical and methodological contribution of research software engineers to research outputs. We have a strong preference, specifically during the pilot phase when all projects are delivered in close collaboration, for members of the OpenSAFELY team who materially contribute to your study and/or to the iterative development of the platform and analytic pipelines to be offered authorship on outputs. This is likely to change over time as the platform expands, and as external teams become more “customers” than “collaborators”. For clarity, this relates to platform contributions, and there is never any expectation of authorship for individual researchers involved in OpenSAFELY who are not involved in a research project. Read our authorship policy for further details.

Plan S

We ask that academic outputs comply with Wellcome’s Plan S requirements for journal publication.

Acknowledgment and Data Sharing / Publication Policy

NHS England oversees the final approval for all publication ready papers, reports or presentations, principally to check that the outputs align with the stated application purpose; NHS England has been extremely supportive of all research and analyses to date. The usual response time for approval is 1-2 weeks.

The acknowledgment and sharing/publication of results guidelines are dependent on the datasets used for your project. The acknowledgement content must be used in all published papers, official reports and presentations given outside of your research team/collaborators.

You MUST NOT share any results that have not been released through the official output checking process. This includes:

- verbal sharing

- allowing someone to look over your shoulder

- transcribing (e.g., to paper or email)

- using screen sharing software or any recording device/software

Datasets used

For the Datasets listed below

The following additional acknowledgement and publication of results guidelines must be followed if your study uses data from ICNARC, ISARIC, ONS-CIS, PHOSP.

Information Governance, Ethics, and funding content policy

For published papers, official reports and presentations you must use the following content for the relevant section headings.

Note: If a study uses both EMIS and TPP, please reference them both as data processors in sections below.

Abstract

• Must add: “With the approval of NHS England we…”

Methods - Data Sharing or Data Source headings

• Must add (note additional comment in paragraph regarding Type-1 Opt outs): All data were linked, stored and analysed securely using the OpenSAFELY platform, https://www.opensafely.org/, as part of the NHS England OpenSAFELY COVID-19 service. Data include pseudonymised data such as coded diagnoses, medications and physiological parameters. No free text data are included. [If your project is ID 156 or above, also add: No GP data from patients who have registered a Type-1 Opt out with their GP surgery were included in this study]. All code is shared openly for review and re-use under MIT open license [LINK TO GITHUB REPO OF PAPER BEING SUBMITTED]. Detailed pseudonymised patient data is potentially re-identifiable and therefore not shared.
• When listing data sources, suggested phrase: Primary care records managed by the GP software provider, TPP/EMIS were linked to [ONS death data, etc.] through OpenSAFELY.

Software and Reproducibility

• If required use: Data management was performed using Python [XX], with analysis carried out using [Stata 16.1/Python/R]. Code for data management and analysis, as well as codelists, are archived online [link your project github repo]. [All iterations of the pre-specified study protocol are archived with version control https://github.com/opensafely/xxxxxx/protocol].
• For any federated analyses use: This was an analysis delivered using federated analysis through the OpenSAFELY platform. A federated analysis involves carrying out patient level analysis in multiple secure datasets, then later combining them: codelists and code for data management and data analysis were specified once using the OpenSAFELY tools; then transmitted securely from the OpenSAFELY jobs server to the OpenSAFELY-TPP platform within TPP’s secure environment, and separately to the OpenSAFELY-EMIS platform within EMIS’s secure environment, where they were each executed separately against local patient data; summary results were then reviewed for disclosiveness, released, and combined for the final outputs. All code for the OpenSAFELY platform for data management, analysis and secure code execution is shared for review and re-use under open licences on GitHub: https://github.com/OpenSAFELY.

Patient and Public Involvement and Engagement (PPIE)

• Where relevant: Insert any project specific PPIE.
• Consider: OpenSAFELY has involved patients and the public in various ways: we developed a public website that provides a detailed description of the platform in language suitable for a lay audience (https://opensafely.org); we have participated in two citizen juries exploring public trust in OpenSAFELY; we have co-developed an explainer video (https://www.opensafely.org/about/); we have patient representation who are experts by experience on our OpenSAFELY Oversight Board; we have partnered with Understanding Patient Data to produce lay explainers on the importance of large datasets for research; we have presented at various online public engagement events to key communities (e.g., Healthcare Excellence Through Technology; Faculty of Clinical Informatics annual conference; NHS Assembly; HDRUK symposium); and more. To ensure the patient voice is represented, we are working closely to decide on language choices with appropriate medical research charities (e.g., Association of Medical Research Charities). We will share information and interpretation of our findings through press releases, social media channels, and plain language summaries.

Funding

• Must add:

  The OpenSAFELY platform is principally funded by grants from:

  • NHS England [2023-2025];
  • The Wellcome Trust (222097/Z/20/Z) [2020-2024];
  • MRC (MR/V015737/1) [2020-2021].

  Additional contributions to OpenSAFELY have been funded by grants from:

  • MRC via the National Core Study programme, Longitudinal Health and Wellbeing strand (MC_PC_20030, MC_PC_20059) [2020-2022] and the Data and Connectivity strand (MC_PC_20058) [2021-2022];
  • NIHR and MRC via the CONVALESCENCE programme (COV-LT-0009, MC_PC_20051) [2021-2024];
  • NHS England via the Primary Care Medicines Analytics Unit [2021-2024].

  The views expressed are those of the authors and not necessarily those of the NIHR, NHS England, UK Health Security Agency (UKHSA), the Department of Health and Social Care, or other funders. Funders had no role in the study design, collection, analysis, and interpretation of data; in the writing of the report; and in the decision to submit the article for publication.

• Where relevant: Insert any additional grants specific to the work or authors.

Information governance and ethical approval

• Must add: NHS England is the data controller of the NHS England OpenSAFELY COVID-19 Service; [TPP is the data processor] [EMIS is the data processor] [EMIS and TPP are the data processors]; all study authors using OpenSAFELY have the approval of NHS England.³ This implementation of OpenSAFELY is hosted within the [EMIS environment which is] [TPP environment which is] [EMIS and TPP environments which are] accredited to the ISO 27001 information security standard and [is][are] NHS IG Toolkit compliant;⁴

  Patient data has been pseudonymised for analysis and linkage using industry standard cryptographic hashing techniques; all pseudonymised datasets transmitted for linkage onto OpenSAFELY are encrypted; access to the NHS England OpenSAFELY COVID-19 service is via a virtual private network (VPN) connection; the researchers hold contracts with NHS England and only access the platform to initiate database queries and statistical models; all database activity is logged; only aggregate statistical outputs leave the platform environment following best practice for anonymisation of results such as statistical disclosure control for low cell counts.⁵

  The service adheres to the obligations of the UK General Data Protection Regulation (UK GDPR) and the Data Protection Act 2018. The service previously operated under notices initially issued in February 2020 by the the Secretary of State under Regulation 3(4) of the Health Service (Control of Patient Information) Regulations 2002 (COPI Regulations), which required organisations to process confidential patient information for COVID-19 purposes; this set aside the requirement for patient consent.⁶ As of 1 July 2023, the Secretary of State has requested that NHS England continue to operate the Service under the COVID-19 Directions 2020.⁷ In some cases of data sharing, the common law duty of confidence is met using, for example, patient consent or support from the Health Research Authority Confidentiality Advisory Group.⁸

  Taken together, these provide the legal bases to link patient datasets using the service. GP practices, which provide access to the primary care data, are required to share relevant health information to support the public health response to the pandemic, and have been informed of how the service operates.

• For RESEARCH, you must add: This study was approved by the Health Research Authority [REC reference XXX] and by the XXX Ethics Board [reference XXX].

• For SERVICE EVALUATION/AUDIT, you must add: This study was supported by [NAME + OFFICIAL ROLE] as senior sponsor, and approved by the XXX Ethics Board [reference XXX]. (NHS England service evaluations/audits are currently not required to have Ethics approval.)

• NOTE: remember to add additional governance and ethical content pertaining to data not processed within OpenSAFELY.

Data access and verification

• If requested, use the following: Access to the underlying identifiable and potentially re-identifiable pseudonymised electronic health record data is tightly governed by various legislative and regulatory frameworks, and restricted by best practice. The data in the NHS England OpenSAFELY COVID-19 service is drawn from General Practice data across England where [EMIS is the data processor][TPP is the data processor][EMIS and TPP are the data processors].

  [EMIS][TPP][EMIS and TPP] developers initiate an automated process to create pseudonymised records in the core OpenSAFELY database, which are copies of key structured data tables in the identifiable records. These pseudonymised records are linked onto key external data resources that have also been pseudonymised via SHA-512 one-way hashing of NHS numbers using a shared salt. University of Oxford, Bennett Institute for Applied Data Science developers and PIs, who hold contracts with NHS England, have access to the OpenSAFELY pseudonymised data tables to develop the OpenSAFELY tools.

  These tools in turn enable researchers with OpenSAFELY data access agreements to write and execute code for data management and data analysis without direct access to the underlying raw pseudonymised patient data, and to review the outputs of this code. All code for the full data management pipeline — from raw data to completed results for this analysis — and for the OpenSAFELY platform as a whole is available for review at github.com/OpenSAFELY.

  The data management and analysis code for this paper was led by (XX) and contributed to by (XX).