Building Consensus on RWE Evaluation: Standardized Tool for Assessing Validity in RWE Studies of Medication Safety and Effectiveness

Background: Non-randomized RWE studies are subject to multiple sources of bias; however, the complexity of data, study design, and analytical methods makes evaluation of such studies challenging. As part of the International Society for Pharmacoepidemiology’s (ISPE) effort to improve the quality of RWE evaluation globally, an ISPE-led group developed a standardized assessment tool to support Health Technology Assessment (HTA) and payer decision-makers in evaluating RWE studies of medication safety and effectiveness. This tool could also be used in any other setting where evaluation of RWE study validity is needed.

Objectives: To introduce the ISPE community to the tool using a real-time evaluation of a published RWE study. This interactive workshop will be valuable to any ISPE member who needs to critically evaluate RWE on drug safety and effectiveness. The secondary objective is to provide an overview of study design features and major sources of bias in observational drug effectiveness and safety studies for trainees and junior ISPE members.

Description: Dr. Girman will introduce the need for a standardized tool to evaluate non-randomized comparative effectiveness and safety studies of medications for bias and will describe how the tool was created and validated (10 minutes). Using a published RWE study as an example, the presenters will guide attendees through a real-time, interactive evaluation of the study using 6 domains of the tool. Dr. Girman will introduce the study. Dr. Lund will then discuss study designs (new vs prevalent user; type of comparator) and walk through the misclassification domain (15 minutes). Dr. Bykov will guide the audience through sources of bias related to study design, covering time-related bias, bias due to the depletion of outcome-susceptible individuals, informative censoring, reverse causation, and detection bias (15 minutes). Dr. Patorno will cover confounding (15 minutes). Dr. Franklin will present an assessment of analytic methods, including propensity scores, and missing data domain (15 minutes). Dr. Girman will then discuss the summary of the evaluation and the caveats of relying on a standardized tool to evaluate RWE (5 minutes). Lastly, the presenters will take questions and feedback from the audience on the tool and provide final remarks on evaluation and interpretation (15 minutes). This interactive workshop will be valuable to any ISPE member who needs to critically evaluate RWE on drug safety and effectiveness and will indirectly serve as a primer on major sources of bias in RWE studies.