HARmonized Protocol Template to Enhance Reproducibility (HARPER)

Clinical trials have detailed study protocols and are registered on ClinicalTrials.gov. What level of details are needed for real-world data (RWD) analyses that aim to estimate treatment effects? In Europe, the European Medicines Agency (EMA) requires registration of many study protocols using a template for observational post-authorization safety studies (PASS) conducted by marketing authorization holders. Other efforts include ISPE’s guidelines for good pharmacoepidemiology practice (GPP) section on protocol development, National Evaluation System for health Technology (NEST) protocol guidance, and the Structured Template and Reporting Tool for Real World Evidence (STaRT-RWE).

The International Society for Pharmacoepidemiology (ISPE) and the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) convened a joint task force to provide standards for RWD protocols. They developed the HARmonized Protocol Template to Enhance Reproducibility (HARPER). The Table below compares HARPER against PASS, GPP, NEST and STaRT-RWE.

The authors wisely note a few limitations of HARPER.

Structured vs. Flexibility. HARPER’s structured approach enhances clarity for stakeholders and consistency across studies. However, some complex study designs may be completely reasonable, but may not fit within the HARPER structure. Many of the sections, however, contain free text sections. Minimum, not maximum transparency. HARPER does not cover every aspect of transparency over the lifecycle of a research study, which may involve sharing of protocol, code, data, as well as results. Thus, HARPER should be seen as the minimum requirements for study protocol transparency. Data evolution. As data collection evolves and new methods are developed, HARPER’s approach may need to adapt over time.

The full HARPER Table of Contents is listed below.

1. Title Page 2. Abstract 3. Amendments and updates 4. Timeline Table 1 Milestones and Timeline 5. Rationale and background 6. Research question and objectives Table 2 Primary and secondary research questions and objective 7. Research methods 7.1. Study design 7.2. Study design diagram 7.3. Setting 7.3.1 Context and rationale for definition of time 0 (and other primary time anchors) for entry to the study population Table 3 Operational Definition of Time 0 (index date) and other primary time anchors 7.3.2 Context and rationale for study inclusion criteria: Table 4. Operational Definitions of Inclusion Criteria 7.3.3 Context and rationale for study exclusion criteria Table 5. Operational Definitions of Exclusion Criteria 7.4. Variables 7.4.1 Context and rationale for exposure(s) of interest Table 6. Operational Definitions of Exposure 7.4.2 Context and rationale for outcome(s) of interest Table 7. Operational Definitions of Outcome 7.4.3 Context and rationale for follow up Table 8. Operational Definitions of Follow-Up 7.4.4 Context and rationale for covariates (confounding variables and effect modifiers, e.g. risk factors, comorbidities, comedications) Table 9. Operational Definitions of Covariates 7.5. Data analysis 7.5.1 Context and rationale for analysis plan Table 10. Primary, secondary, and subgroup analysis specification Table 11. Sensitivity analyses – rationale, strengths and limitations 7.6. Data sources 7.6.1 Context and rationale for data sources Table 12. Metadata about data sources and software7.7. Data management 7.8. Quality control 7.9. Study size and feasibilityTable 13. Power and sample size 8. Limitation of the methods 9. Protection of human subjects 10. Reporting of adverse events 11. References 12. Appendices

You can read more details on the HARPER approach here and at the Open Science Foundation here.