By Exploristics Chief Data Scientific Officer – Kimberley Hacquoil
In clinical development, clear, precise and transparent specification of the research question and associated treatment effects of interest are vital. This is especially crucial as trials often involve multiple stakeholders who need to make different decisions. Estimands clarify “what” is to be estimated and link trial objectives with trial designs and analysis to describe the “how”.
Estimands were first discussed in an ICH concept paper in 2014 before they were implemented in ICH E9 (R1) addendum in 2019. However, people may still get scared by the word “estimand” and find it daunting when approaching this for the first time – they may think that it is something radically different or new. I don’t believe it is drastically different, and as a statistician I very much welcome the adjustment away from certain aspects of trial design being a statistical analysis problem to solve. It takes these considerations out of the shadows of the statistical analysis plan and puts it right at the heart of the objectives and motivation for the trial.
In recent years, since working outside of clinical studies in the strategy and portfolio space, I have learnt how decision analysis frameworks provide a vital approach to ensuring there is proper framing of a situation/problem. Doing this up front and as a multi-disciplinary team means true alignment can be achieved and everyone can get behind the decision-making framework. This is also true for the estimands framework. It removes the potential for misalignment between analysis methods (especially dealing with missing data) and the treatment effects of interest.
Estimands Framework
This framework presents a structured approach to facilitate and support the discussions between project and study teams when establishing clinical trial objectives, design, conduct, analysis and interpretation. It also enables valuable exchanges between sponsors, regulators and payors regarding the treatment effects of interest for a trial. When designing trials and clinical development plans, different people may want to understand different things from the same study. Payors and regulators may not be requiring the same thing, and what the sponsor needs to consider for patients may be different again. Understanding and articulating these different needs is key for success.
Estimands need to be defined before deciding on study design and primary analysis. This means before power, probability of success, and sample size decisions since they are linked to study objectives and study design. An estimand has multiple elements and is made up of the endpoint of interest, target patient population, treatment, strategy for intercurrent events and the summary measure. All these elements are linked and combined to describe the estimand. Each element is vital and the estimand doesn’t exist if one of these is not articulated. The estimand with the trial objective defines the “what” to estimate.
Once a team understands and are clear on “what” they want to estimate, the main estimator and estimate can be defined, and this describes the “how” to estimate. In the past statisticians (including myself) have been so focussed on the “how” that they forget to look up and check it aligns with the “what”. Using the estimands framework results in clearer answers to well specified questions, and so in turn improves both regulatory and clinical decision-making and may ultimately lead to a higher probability of technical and registrational success for studies.
Intercurrent Events and Missing Data
Intercurrent events are events which occur after treatment is initiated that affect either the interpretation or existence of the measurements associated with the clinical question of interest. Intercurrent events need to be addressed when the clinical question of interest is described so that the treatment effect can be defined and estimated. There are a number of different ways to deal with intercurrent events and the most common ones are treatment policy strategy, hypothetical strategy, composite strategy, while on treatment strategy and principal stratum.
The estimand framework will impact how missing data are defined and the methodology used to deal with missing data. For example, if a patient stops medication (intercurrent event), then under the treatment policy strategy, the measurements after the event are required, and if they are not collected would be considered missing. Under the hypothetical strategy the measurements (whether collected or not) should be replaced with the hypothetical measurements expected if that patient had not had the event. However, under the while on treatment strategy, the measurements post the event would not be of interest, and so are not considered missing. Any missing data that are imputed with assumptions still need to be clearly defined in the analysis plan and any sensitivity analyses to assess these.
Simulation for Estimands – the way forward.
At the design stage of a protocol, there may be different estimands which are being considered. For example, there may be a number of endpoints of interest, different patient populations to consider, multiple treatment options, different ways to deal with intercurrent events and various summary measures. By simulating realistic patient-level data which addresses these different things, we can then create in-silico trials to assess the impact on the probability of success.
More importantly, by having patient-level data we can artificially make our data “messy”. By this I mean we can remove or change data based on intercurrent events and explore different methods for dealing with such events to provide the most unbiased analysis of the data and optimal study design. This might mean removing data and then adding some back in through imputation methods, making it all the more real! Simulation is the only way to deal with and plan for intercurrent events that do not happen “at random”. Simulating data to mimic these events and exploring different analysis options will provide teams with transparent information of what might happen in the actual trials before any patient is recruited. We can then address any potential issues and adapt accordingly at the design stage of the trial not at the end of the study through ad hoc or multiple sensitivity analyses.
Simulation of lifelike in-silico trials using an estimands approach will provide teams with a data driven and quantifiable impact on the probability of success of a study and allow controllable factors such as different sample sizes, decision rules and adaptive designs to be assessed and compared.
The estimands framework drives alignment between the “what” and the “how” of clinical development and requires statisticians to also come out of the shadows and be at the heart of the discussions when designing trials. Simulation is already a powerful tool in the statistician’s toolbox, and the estimands framework will allow this tool to shine in all its glory. We need to exploit this opportunity and maximise the impact we can make to drug development.
Read more:
Estimands, opportunity or risk for drug developers?
Statistical Consulting Services
Watch more:
Matching trial objectives to design via the estimands framework