Background: Use of electronic health record (EHR) data is often prone to misclassification of study variables in comparative effectiveness research, due to data incompleteness. A previously developed algorithm to identify those with high EHR-continuity was shown to reduce such misclassification. Whether the effect estimates in real-world comparative studies are dependent on levels of EHR-continuity is unclear.
Objectives: To compare the effect estimates obtained from patients with different levels of EHR-continuity in two studies: 1) non-user comparator design: effect of pump inhibitors (PPI) vs. non-PPI users on 1-year risk of pneumonia; and 2) active comparator design: PPI vs. H2 receptor antagonists (H2RA) on 1-year risk of pneumonia.
Methods: Patients aged ≥65 years were identified in 2 US EHR systems (MA and NC) linked with Medicare claims data (2007/1/1 – 2017/12/31). Eligible patients were required to have ≥365-day continuous enrollment in Medicare (Parts A, B, D), with ≥1 overlapping encounter in the study EHR. We compared the effect estimates in the total linked study population and by excluding patients with < 25%, 50%, or 75% EHR-continuity score. We used Cox regression weighted by stabilized inverse probability of treatment weights to estimate the hazard ratio (HR). We calculated % change in the effect estimate (expected = 0%) as the % difference between HR based on EHR only vs. linked EHR-claims data since claims data capture medical information recorded outside of study EHR. We only used EHR to define user cohorts to ensure comparison was done in the same study population.
Results: In MA data study 1 (N=102,409), the outcome capture rate (sensitivity) by the study EHR was 30% in PPI user vs. 24% in non-PPI users in the total cohort, leading to 10.8% change in the HR. After excluding the lower 75% EHR-continuity patients, the outcome misclassification was comparable in 2 groups (56% in PPI and 58% in non-PPI), and the % change the HR was improved to 0.9%. In study 2 (N=65,546), comparing PPI vs. H2RA in the total cohort, the sensitivity of outcome measure was 30% in PPI vs. 34% in H2RA, and the change of the HR was 3.6%. After excluding the lower 75% EHR-continuity patients, the outcome misclassification was 57% in both arms, and the change of the HR estimate was 2.4%. A similar pattern was observed with NC data.
Conclusions: In active-comparator studies, the effect estimates on the relative scale may be less impacted by EHR-continuity levels. In contrast, cautions are needed in non-user comparator study designs as differential misclassification of outcomes may be present.
