Background: Pharmacovigilance studies often use restricted reference sets, such as an ‘active comparator’, for the data mining of reports in spontaneous reporting systems (SRS). This concept is borrowed from pharmacoepidemiologic study designs and aims to mitigate classical epidemiological biases such as confounding by indication. However, the spontaneous reporting of suspected adverse drug reactions (ADRs) introduces additional reporting biases which may depend on external factors, influencing reporting rates in an intractable manner. Currently, it is unclear which reference sets are generally the most appropriate for use in pharmacovigilance.
Objectives: To assess the impact of reference set selection on signal identification in SRS.
Methods: Using the FDA Adverse Event Reporting System (FAERS), we studied two drug-event combinations (DECs) with a propensity for false positive signals (direct oral anticoagulants [DOACs] and acute liver failure [ALF]; sodium-glucose cotransporter-2 inhibitors [SGLT2i] and acute kidney injury [AKI]) due to reporting biases brought on by initial safety concerns (hepatotoxicity of ximelagatran for DOACs; early reports of nephrotoxicity for SGLT2i) that were later refuted by pharmacoepidemiologic studies or randomized trials. For each DEC, we computed estimates of the proportional reporting ratio, reporting odds ratio and information component using four reference sets (active compactor, class exclusion, active comparator with class exclusion, full data; the latter as ‘benchmark’). Analyses were conducted within two calendar time periods, defined based on external events (market approval for DOACs, FDA safety warning for AKI/SGLT2i) hypothesized to alter reporting rates.
Results: Overall, we could not identify a predictable pattern in the direction or magnitude of disproportionality analysis estimates when comparing reference sets to benchmark across different signal detection study settings and between DECs. For example, the active comparator reference set attenuated the SGLT2i/AKI false positive signal, whereas it enhanced the DOAC/ALF false positive signal relative to benchmark. Restricting to the initial post-approval period augmented the false positive signal for DOACs/ALF across reference sets. Restricting to the period before the FDA warning weakened the false positive signal for SGLT2i/AKI across reference sets.
Conclusions: Our work showed that restricting reference sets for signal detection may not influence estimates in a consistent manner. It also highlighted the influence of external events such as regulatory decisions. When data mining suspected ADRs in SRS, we recommend implementing multiple reference sets and considering regulatory drug history to strengthen the generated hypotheses.
