Epidemiology Director IQVIA Amsterdam, Netherlands
Background: Single-arm trials (SATs) are increasingly utilised and augmented by compiling a Real World (RW) External Comparator Arm (ECA) to provide better context and interpretability of results. A simulation study was performed to test the ECA design performance in case of covariate missingness and unmeasured confounding.
Objectives: To determine the bias, type I error and 95%CI coverage of statistical approaches in case of covariate missingness and unmeasured confounding.
Methods: •
Design: Simulation study, applying different hazard ratios, sample sizes, sample size ratios, covariate missingness percentages (up to 100%) and numbers of (partially) missing covariates as input parameters. • Setting: 2*900 scenarios were simulated based on 2 real case studies (using treatment arm data from completed RCTs and ECAs derived from a RWD source), allowing for pattern recognition of findings dependent on changes in input parameters • Exposures or interventions: Treatment versus Comparator (generic setting) • Main outcome measures: Bias, type I error and 95%CI coverage for the endpoint Overall Survival (OS) • Statistical analysis: Inverse propensity score (PS) weighted Cox regressions were applied using the Average Treatment effect (ATE), the Average Treatment Effect on the Treated (ATT) and the Average Treatment Effect in the Overlap Population (ATO) estimands. Missing data was handled by multiple imputation (MI).
Results: Depending on the degree of covariate missingness and unmeasured confounding, the ECA design led to biased point estimates and increased type I errors and decreased 95%CI coverage. However, even with considerable missingness, the bias correction using MI in combination with PS weighting was substantial when comparing with the unadjusted analysis. Overall, the ATE performed best and the ATT performed worst when averaging the performance over all investigated scenarios. This is considered to be an important finding. However, this finding is not strictly generalizable to all possible settings, and there were scenarios where either the ATO or the ATT performed best.
Conclusions: This study was able to quantify bias and other performance characteristics for ECA study scenarios. This quantification helped to enhance understanding regarding the magnitude of expected bias and the influence of varying degree of missingness and unmeasured confounding. As a conclusion, the ECA study design should only be applied when anticipating large treatment effects, as suggested by the FDA.
