Principled Approaches To Handle Partially Observed Confounder Data From Electronic Health Records: A Plasmode Simulation Study

Background: Electronic health records (EHR) may enhance confounding adjustment for variables usually not measured in claims data. However, these data are often only partially observed.

Objectives: To investigate the performance of principled diagnostic and analytic approaches for partially observed EHR confounders in various missingness scenarios.

Methods: We identified a cohort of diabetic SGLT2 or DPP4 inhibitor initiators from Medicare claims linked to EHR from Mass General Brigham in Boston and derived 3 sub-cohorts with complete information on HbA1c, BMI and smoking as confounders of interest (COI). Using a plasmode framework, we simulated 100 datasets each, including the COI in the outcome generation model and a true null treatment effect. The COIs were set to missing completely at random (MCAR), at random (MAR), depending on an unmeasured confounder (MNARU), or on the value of the COI itself (MNARV). Aiming to differentiate between missingness mechanisms, we applied 3 diagnostics: 1) compared distributions between patients with or without the observed COI (e.g. averaged standardized mean differences [ASMD]), 2) assessed the ability to predict missingness based on observed covariates, and 3) estimated if missingness was associated with the outcome. Lastly, we evaluated complete case (CC), inverse probability weighting (IPW), missing indicator and (non)parametric multiple imputation (MI) analyses by computing the root mean squared error (RMSE) between the true and estimated log hazard ratio (HR) of the treatment effect. We varied numerous simulation parameters, such as the proportion missing and presence of heterogeneous treatment effects (HTE).

Results: The diagnostics successfully differentiated between MCAR and MAR. The distribution of patient characteristics indicated few differences under MCAR but notable differences under MAR (ASMD 0.05 vs 0.20). Missingness was easiest to predict under MAR and hardest under MCAR (area under curve [AUC] 0.58 and 0.50). While missingness was not associated with the outcome under MCAR, MAR showed a spurious association (HR crude 0.53 [95% CI 0.23-0.83]). In comparison, MNARU also showed an association in the crude HR (0.43 [0.13-0.74]) but was the only mechanism that kept this tendency after adjustment (HR 0.31 [-0.03-0.66]), while MNARV was nearly indistinguishable from MCAR. MI using a random forest model overall resulted in the lowest RMSE (0.239 [0.238-0.241]), which was driven by better performance in scenarios with HTE. CC and IPW performed worst.

Conclusions: Principled diagnostics provided insights into underlying missingness structures and may be routinely used. MCAR and MNARV were difficult to distinguish. Depending on the diagnosed mechanism, MI with nonparametric models could help reduce bias.