Assessing comparability of osteoporosis treatment groups using real world healthcare data from the UK and Denmark: evaluating propensity score and negative control outcome methods to address confounding by indication

Background: Evidence on the comparative effectiveness of osteoporosis treatments is heterogeneous. This may be attributed to different populations and clinical practice, but also to differing methodologies ensuring comparability of treatment groups before treatment effect estimation and the amount of residual confounding by indication.

Objectives: To assess the comparability of denosumab vs oral bisphosphonate (OBP) groups using propensity score (PS) methods and applying different rules for evidence of residual bias in negative control outcome (NCO) analysis.

Methods: New user cohort study using data from the UK Clinical Practice Research Datalink (CPRD) and the linked Danish National Registries (DNR) included women aged ≥50 years initiating denosumab or OBP in 2011-2018. Follow up was from treatment initiation until treatment switch or discontinuation, NCO occurrence, end of enrolment or study period, or a maximum of three years.
Balance of observed covariates was assessed using absolute standardised mean difference (ASMD) before and after PS weighting, matching, and stratification, with ASMD >0.1 indicating imbalance. Residual confounding was assessed using NCOs with ≥100 events. The Cox model estimated the hazard ratio (HR) and 95% confidence interval (CI) between treatment and NCO. Presence of residual confounding was evaluated with two approaches: (1) >5% of NCOs with 95% CI excluding 1, (2) >5% of NCOs with an upper CI < 0.75 or lower CI >1.3.

Results: Differences between treatment groups within CPRD (6528 denosumab, 194191 OBP users) and DNR (4276 denosumab, 75293 OBP users) were observed; denosumab users were older, had greater proportions of fracture history and higher cumulative OBP use.
The number of imbalanced covariates before adjustment (CPRD 22/87; DNR 18/83) decreased, with imbalance remaining after weighting (CPRD 10; DNR 2), matching (CPRD 3; DNR 4) and stratification (CPRD 3; DNR 9).
Using approach 1, residual confounding was present for all PS methods in both databases (≥8% of NCOs). Using approach 2, residual confounding was present in CPRD with PS matching (1/19, 5.3%) and stratification (3/47, 6.4%), but not with weighting (2/47, 4.3%). Within DNR, no NCOs had HR estimates with upper or lower CI limits beyond the specified bounds indicating residual confounding for any PS method.

Conclusions: Achievement of covariate balance and determination of residual bias were dependent upon several factors including the population under study, PS method, prevalence of NCO, and the threshold indicating residual confounding. Future studies should consider all covariate balancing PS methods and account for the stringency of residual bias assessments to determine whether treatment groups are comparable.