Implementing external controls

The Challenge

University College London Hospital (UCLH) and The Francis Crick Institute wished to investigate repurposing an existing cystic fibrosis treatment (nebulised dornase alfa) in hospitalised patients with severe COVID-19. They believed that the agent’s mechanism of action could improve the survival of these patients by reducing excess inflammation in the lungs (1). However, this study faced key constraints due to the ongoing COVID-19 pandemic including:

• Limited knowledge and time – Little was known about COVID-19 while there was an urgent need to address the medical emergency.
• Changing landscape – The medical and clinical environment was rapidly changing with regards to case numbers and vaccines available.
• Competing studies – Numerous clinical studies and platform trials were competing for investigator sites and to recruit patients.
• Incomplete information – Limited data was available to inform the study design regarding uncertainty on appropriate outcome variables.

To overcome these issues the Sponsors wished to:

• assess the feasibility of using real-world data (RWD) in a synthetic or external control arm to replace enrolling patients.
• rapidly design a study using a synthetic control arm if this proved robust and feasible.

The Approach

Due to pandemic recruitment and logistical constraints, it was not possible to randomise patients for a planned study with an equal allocation to the investigational treatment and control, but a 3:1 ratio could be used instead. However, due to the small control arm, this would decrease the study’s probability of success (PoS). One way to compensate for this and improve PoS would be to increase the size of the control arm by augmenting it with existing historical data.

Assessment of synthetic control arm feasibility

Historical data for use in a synthetic control arm was collated from electronic health records by the UCL team. To evaluate the feasibility of using this data as part of a synthetic control arm, in line with FDA guidance (2), our Statistical Consultants:

• Examined the historical data to ensure comparability between the historical data and the clinical trial data. This included assessing if there could be reasonable comparison between the:
  • endpoints
  • frequency of follow-up visits
  • time period of follow-up
  • geographic region of the populations

• Carried out propensity score matching to select matched control patients who were similar to the clinical trial participants. Propensity score matching is used to balance baseline characteristics between groups as best as possible.

Study design using synthetic control data

A suitable study design was required quickly due to the rapid planned initiation of the COVASE trial (1 month from first meeting to grant approval and a further 1.5 months to first patient).

Our Statistical Consultants provided key advice regarding a hybrid design and protocol for a randomised controlled single-centre Phase 2 proof of concept trial where:

• Patients were randomized 3:1 Dornase alpha + best available care: best available care (BAC) as shown in Figure 1.
• The BAC arm was supplemented with 60 historical controls using electronic health records.
• C-reactive protein (CRP) was chosen as primary endpoint because:
  • Its variability was well understood.
  • It’s collected routinely in electronic health records.
  • It’s a clinically important marker of inflammation.
  • Its role in COVID-19 was supported by emerging early data from China and Italy

The Results

Our Statistical Consultants were able to:

• Match historical controls based on age, gender, BMI, baseline CRP and key comorbidity.
• Show that use of synthetic controls in this study was feasible.
• Give key advice on the design of a single-site, randomised, controlled, parallel, open-label investigation with primary endpoint change in CRP using synthetic controls.
• Conduct the primary analysis which compared the investigational treatment to a control arm which combined historical and randomised controls and carry out multiple sensitivity analyses to help evaluate efficacy of the treatment.
• Show via analysis of the data collected that nebulised dornase alfa:
  • was safe in hospitalized patients with severe COVID-19 pneumonia.
  • resulted in a significant reduction in inflammation, markers of immune pathology and time to discharge.

The Impact

Using this design, UCLH and the Frances Crick Institute was able to:

• Implement a novel study that provided proof-of-concept evidence under challenging and constantly changing circumstances.
• Motivate participants to enrol into study as they had a 75% chance of receiving an active investigational treatment.
• Accelerate study timelines as using historical controls recruitment for the study took 14 months, as opposed to 21 months if 1:1 randomisation was chosen, saving approximately 7 months.
• Publish the study in a peer-reviewed journal (3).

“Exploristics provided exemplary support for the COVASE study, from conception and fundraising to the design, execution and statistical analysis – often working to tight timelines. They consistently looked for a smarter way to overcome challenges such as the use of external control data to deal with the likely recruitment issues during the height of the pandemic. They worked to very high standards and inspired great confidence in their work. They really are wonderful, I can highly recommend and would love to work with them again.”

Professor Joanna Porter, Consultant in Respiratory Medicine and Academic Lead for NHS National Interstitial Lung Disease Centre – University College Hospital London, UK

References

1. Cystic fibrosis drug trialled to fight inflammation caused by COVID-19 | Crick
2. https://www.fda.gov/media/164960/download
3. https://elifesciences.org/reviewed-preprints/87030