(223) A simulation-based approach to assess the potential impact of unmeasured confounding on study validity at the design stage of non-interventional studies

Publication Number: 1214

Background: Hypothetical unmeasured confounding is often raised as a source of potential bias at the design stage of non-interventional studies, but evaluating such concerns quantitatively is challenging.

Objectives: To develop and assess a simulation-based approach for unmeasured confounding in non-interventional studies at the design stage.

Methods: We generated hypothetical cohorts involving a binary treatment (prevalence 25%), a time to event outcome (incidence 5%), 13 measured covariates (6 confounders, 4 treatment-only determinants, and 3 outcome risk factors) and a binary unmeasured confounder (U, prevalence 10%). We further generated a binary measured ‘proxy’ variable (p) correlated with the generated unmeasured confounder. In simulation scenarios involving 500 cohorts of 5000 observations each, we varied the strength of unmeasured confounding (strong U, coefficients of 1.0 in the treatment and outcome models and moderate U, coefficients of 0.5) and strength of correlation between U and p (0.7, 0.5, 0.3). Following data generation, three levels of adjustments were pursued to estimate treatment effects, a) no adjustment, b) Level 1: adjustment for only measured confounders through propensity score matching (PSM), c) Level 2: adjustment for measured confounders and proxy (p) through PSM. We evaluated the achievement of balance through absolute standardized mean differences (ASMD) in U and p with each level of adjustment in all scenarios. We also evaluated relative bias with each level of adjustment.

Results: Across all scenarios, including proxies in PSM (Level 2) led to improvement in balance of U across the two treatment groups, but this improvement was highly dependent on the correlation between U and p (Strong U scenario, mean ASMD 0.31 with no adjustment, 0.16, 0.23, and 0.29 with Level 2 adjustments with correlations of 0.7, 0.5, and 0.3 respectively; Moderate U scenario, mean ASMD 0.15 with no adjustment, 0.08, 0.12, and 0.14 with Level 2 adjustments with correlations of 0.7, 0.5, and 0.3 respectively). In the strong U scenario, relative bias was lower with Level 2 adjustments (8.4%, 11.3%, 12.6% at correlations 0.7, 0.5, and 0.3, respectively) compared to Level 1 (15.6%, 14.9%, 14.1%). In the moderate U scenario, relative bias was limited after Level 1 adjustment (3% on average) and moved little with Level 2 adjustment.

Conclusions: A simple simulation-based approach that takes into consideration the hypothetical unmeasured confounding structure and correlation with measured confounders may be helpful in identifying whether real unmeasured confounders with those characteristics are likely to exist and can lead to informed study design.