A methodology study to evaluate external comparator arm study results versus randomised controlled trial results: metastatic Hormone-Sensitive Prostate Cancer case study

Background: Single-arm trials (SATs) with an external comparator arm (ECA) have been introduced as a design approach when randomised controlled trials (RCTs) are infeasible and/or unethical.

Objectives: To compare estimated effect sizes for the endpoints overall survival (OS) and time to next treatment or death (TTND) utilising data from a completed RCT and real-world data (RWD) forming an ECA.

Methods: Estimated effect sizes from a completed RCT in patients with metastatic Hormone-Sensitive Prostate Cancer (mHSPC) were compared with an approach using an ECA. RCT data were obtained via Yale University Open Data Access (YODA). In the RCT all patients were on a stable regimen of androgen deprivation therapy. Patients receiving abiraterone acetate and prednisone were compared with patients receiving placebo. RWD for mHSPC was obtained from a network of regional community health systems in the US. RWD were harmonised with RCT data, considering eligibility criteria, treatments compared, covariates and the outcomes OS and TTND. The data from the experimental arm of the RCT was then used as if it was a SAT and analysed with the RWD ECA using propensity score methods to account for the imbalance of potential confounders between the SAT and the ECA. Multiple imputation was used to derive a complete set of baseline covariates for each patient.

Results: The mHSPC RCT dataset included 597 and 602 patients in the treatment and control arms, respectively. After applying prioritized eligibility (and treatment-related) criteria, there were 92 patients in the ECA dataset. With OS as outcome the Hazard Ratio (HR) from the original RCT was 0.66 (95% confidence interval [CI]=0.57-0.78). The unweighted HR comparing the SAT with the ECA was 0.72 (95% CI=0.53-0.99). The HR was 0.84 (95% CI=0.55-1.26) by average treatment effect (ATE) weighting. An HR of 0.86 and 0.81, respectively, was observed by average treatment effect on the treated (ATT, 95 % CI=0.55-1.34) and average treatment effect on the overlap population (ATO, 95% CI=0.58-1.13) weighting. With TTND as outcome the HR for the RCT data was 0.47 (95% CI=0.41-0.55). The unweighted HR comparing the SAT with the ECA was 0.40 (95% CI=0.31-0.52). The HR was 0.41 (95% CI=0.30-0.58) by ATE weighting.

Conclusions: The sample size and number of OS events were low in the ECA, leading to a low precision and to inconclusive OS results. In contrast, in a TTND analysis the power was sufficiently high to obtain high precision results, which were very similar to those of the mHSPC RCT. Missing data was clearly seen as the key challenge when utilising RWD which has not been collected for the specific purpose of the study. This case study is part of a larger project that also includes a multiple myeloma case study and simulations.