This month Darren Jolliffe, Director of S-cubed Biometrics Ltd, explains how S-cubed can assist you in applying the forthcoming regulatory guidance ICH E9(R1) which will have a direct impact on how protocols are developed.
The draft addendum R1 to ICH E9 “Estimands and Sensitivity Analysis in Clinical Trials” is due to be finalised before the end of 2019. When this guidance is approved, it will then have a direct impact on how protocols are written. Currently there is a lack of alignment between the objective(s) written into a protocol and the subsequent quantification of the “treatment effect” that is then reported. This ICH E9 addendum (currently at a draft stage) has been prepared to help companies designing and reporting clinical studies provide additional, required information upfront in the protocol, to also then fit with their study design expectations. It is written to promote the alignment between trial objectives, study design, conduct, analysis and inference.
Planning a clinical trial? What’s in the study protocol?
The following is a brief summary of some of the items that need to be considered at the time of planning a clinical trial, and what additional information should be included within the study protocol to address the estimand specification.
Let’s say a specified protocol primary endpoint is the change (from baseline) in value of a specific type of clinical parameter assessment which is taken on each patient at week x (say week 24) in the study. And let’s also consider what might happen to a subject in practice prior to that week 24 assessment:
- Some patients will take the treatment as planned.
- Some patients will need to take additional concomitant mediations or rescue medication, or some may switch treatment
- Some patients may discontinue treatment (e.g. due to an adverse event or lack of efficacy)
- Some patients may experience a terminal event that prevents follow-up for the endpoint of interest, e.g. death, or for example tooth extraction when the endpoint is assessing tooth pain
These “Intercurrent events” occur after the treatment initiation and either preclude observation of the variable or affect its interpretation.
What happens to patients during the trial is potentially going to have an impact on the treatment effect observed. For instance, are we interested in the benefit of treatment in the absence of other specific concomitant/rescue medication being taken, or are we interested in the benefit of the treatment policy: planned treatment and any concomitant/rescue medication taken as required?
Of course multiple types of intercurrent events could occur in a clinical trial and the ICH draft guidance sets out five different strategies for handling them. A strategy should be chosen for each (relevant) intercurrent event to reflect the scientific question of interest and the same strategy does not need to apply to all types of (relevant) intercurrent events. It may be appropriate to use different strategies for different events. However not all strategies will be equally acceptable for regulatory consideration and certain strategies may not be relevant to the study design being considered.
Types of Strategies Specified in the Draft Guidance
- Treatment-policy: the data collected for the variable of interest is used regardless of whether or not the intercurrent event occurs
- Composite strategy: The intercurrent event is taken to be a component of the variable, e.g. subject is a non-responder if they use rescue medication.
- Hypothetical strategy: The hypothetical scenario of what is envisaged to happen to the patient if the intercurrent event did not occur.
- Principal stratum strategy: subset of the population based on a stratum definition, e.g. in a vaccine trial, only the population of patients who get infected.
- While on treatment strategy: consider response to treatment prior to the occurrence of the intercurrent event.
The choice of strategies to be used must be the object of a multidisciplinary discussion, in particular between sponsors and regulators and the preferred strategy for handling each (relevant) intercurrent event should be clearly defined in the protocol with reasons for that choice.
Note: Study discontinuation / loss to follow-up (missing data) is not an intercurrent event. Missing data is a different issue, which requires separate consideration for the statistical analysis.
In the protocol, intercurrent events are to be described under a broader heading of the “Estimand”.
Definition: Estimand (taken from the ICH E(R1) draft addendum)
Is the target of estimation to address the scientific question of interest posed by the trial objective. Attributes of an estimand include the:
- Population of interest
- Variable (or endpoint) of interest
- Specification of how intercurrent events are reflected in the scientific question of interest
- Population level summary for the variable
- Typically characterised through inclusion/exclusion criteria in the study protocol
- Measurements taken, e.g. Eosinophil count. Expressed as a function: change from baseline to Week 12
- Intercurrent event: whether patients complete 12 weeks of treatment, adopting a treatment policy strategy, i.e. use result regardless of whether patients complete treatment.
- A basis for the comparison between treatments, e.g. the difference in mean change from baseline to Week 12
A second part of the addendum covers “Sensitivity analysis”. Within the statistical analysis of some previously performed clinical trials, analyses labelled as sensitivity analysis may in fact have had different targets of estimation (estimands). When now considered in this new setting (estimand framework) and if this happened, it would not be surprising when inconsistent results between the main and sensitivity analyses are achieved (i.e. going against the intention of performing them).
The guidance clarifies why sensitivity analysis should be performed, to address that when changing assumptions (within the same estimand) it does not impact the observed treatment effect.
Definition: Sensitivity Analysis (taken from the ICH E9(R1) draft addendum)
Is a series of analyses targeting the same estimand, with differing assumptions to explore the robustness of inferences from the main estimator to deviations from its underlying modelling assumptions and limitations in the data.
Consider what assumptions have been made in the main analysis of an endpoint, including those associated with the use of statistical modelling to address missing data or replace not applicable observed data (due to the estimand approach used). Then amend those assumptions to assess if there is an impact on the observed treatment effect. Then discuss these findings to explain and justify (if possible) what has been observed.
Who writes the Estimand?
It is not the purpose of the statistician to solely write the “Estimand” section of a protocol. It is important that all departments involved in contributing to the development of the study protocol work together to promote a more meaningful description of treatment effects (through development of estimand(s)) that fit with licensing plans, and have clinical trials designed to fit with the agreed trial objectives. This will give increased transparency at the outset with respect to the subsequent data analysis and inference.
It is acknowledged within the wider pharmaceutical community that adoption of this new guidance is likely to be a learning curve for everyone. There is no guidance on the specifics of where to put this detail (i.e. Section) within the protocol, nor a requirement for specific standard wording to use to address it. Rather, it simply needs to be thought about upfront at the study design stage, and the required details (described above) should be addressed to provide clear and complete estimand definitions.
Look out for the finalised guidance. Over 1000 comments were received on the draft document, so the guideline could result in some significant additions/changes. In addition, it is expected that the ICH related material will be updated with useful examples to help with preparing protocol related estimand sections.