Background: The disease risk score (DRS) is an alternative to the propensity score (PS) for confounding control. Literature has shown that DRS has comparable performance to PS in many settings. Most studies examined the performance of adjusting, stratifying or matching on PS or DRS, but very few used standardization, and few assessed the impact of misspecifying effect measure modification terms (EMMs) in the DRS model. In addition, machine learning models have been widely used for PS, but not for DRS.
Objectives: We aimed to use random forests for all model fitting, and investigate the performance of standardization over PS vs. DRS when EMMs are misspecified.
Methods: Design & Setting: We conducted a simulation study. We randomly generated exposure (A), outcome (Y), measured covariates (C1–C4), and unmeasured covariate (U), for the study population (N = 3,000) for 1,000 times. C1, C2 and U are confounders, and C3 and C4 are outcome predictors. The effect measure modifiers are C1 and C4. In the data generating process, the true exposure and outcome models are complex, with polynomial and multi-way interaction terms. For DRS model fitting, we generated an external population (N0 = 2,000) with different model forms and parameterizations, as suggested in the literature.
Exposure & Outcome Measures: Both exposure and outcome were binary. The prevalence of exposure and outcome was 42.7% and 16.2% in the study cohort, respectively. The prevalence of exposure and outcome was 17.2% and 13.7% in the external population, respectively. We used risk difference as the measure of effect.
Statistical Analysis: Since it is impossible to know the functional form of exposure and outcome models, in each of the 1000 iterations, we fit random forests for the PS, DRS, and the outcome models. We compared the performances of (1) correctly specifying EMMs, (2) omitting EMMs, (3) adjusting for A×PS or A×DRS, in the outcome model. We also checked the balance of PS in the entire population and the balance of DRS in the unexposed population, before adjusting for PS and DRS in the final outcome model.
Results: Risk difference estimates using PS were all nearly unbiased, with similar empirical standard error (SE), whether EMMs were misspecified or not. For DRS method, only correctly specifying EMMs gave unbiased estimate. Either omitting EMMs or adjusting for A×DRS gave biased estimates, although the SEs were smaller.
Conclusions: The results show that when using random forests, PS method is robust to EMMs omission, while DRS method is not. Future research may explore using these differences the potential to detect unknown EMMs using DRS through a data-driven approach by examining including which EMMs give the closest estimate to that using PS method.
