Background: Pharmacoepidemiologists frequently restrict study populations based on covariates (or summary measures of multiple covariates) to reduce confounding and improve the exchangeability of study populations. When estimates change after restriction it is difficult to evaluate whether changes are because of better confounding control or different distributions of measured effect measure modifiers (EMMs) within the restricted population.
Objectives: Establish a heuristic to draw conclusions about measured EMM and unmeasured confounding or EMM based on three values: the change in estimate before and after restriction; the change in estimate using inverse odds weights (IOW) to standardize the restricted population to resemble the full population in EMMs; and the change in estimate using IOW to standardize the full population to resemble the restricted population in EMMs.
Methods: We simulated data with a binary treatment X, a binary outcome Y, and two associated binary confounders C1 (e.g., age < 80 vs 80+) and C2 (e.g., male vs female), with C2 acting as a possible risk difference (RD) EMM. We created four scenarios with distinct data generating mechanisms: 1) no unmeasured confounders or EMMs and a constant RD of 10% for X; 2) an unmeasured confounder C3 present only in those with C1 but no EMMs, with the same RD; 3) no unmeasured confounders, but EMM by C2 resulting in a variable RD for X; and 4) an unmeasured confounder C3 present only in those with C1 in addition to EMM by C2. In each scenario, we estimated the average treatment effect (ATE) RD adjusting for C1 and C2; the ATE RD restricting to those without C1; the ATE RD reweighting those without C1 to resemble the full population with respect to C2; and the ATE RD reweighting the full population to resemble those without C1 with respect to C2. We examined differences between scenarios with respect to the three changes in estimate discussed in the objectives.
Results: In scenario 1 all analyses yielded identical RD estimates of 10%. In scenario 2, the RD changed from 11% to 10% after restriction, but IOW did not change the restricted RD estimate. In scenario 3, RD estimates changed from 15% to 13% after restriction and IOW reversed these changes in estimate by correcting for the imbalance in C2 (a measured EMM). In scenario 4, RD estimates changed from 16% to 13% after restriction, but IOW did not completely reverse these changes in estimate.
Conclusions: The four scenarios of no unmeasured confounders or EMMs, unmeasured confounders but no EMMs, measured EMMs but no confounders, and unmeasured confounders and measured EMMs resulted in unique patterns for the three changes in estimate we examined. Under some assumptions, weighting tools can identify if estimates change after restriction due to changes in measured EMMs or unmeasured factors.
.jpg "Michael Andrew Webster-Clark, PharmD, PhD photo")
