Bias Introduced by COVID-19 Related Healthcare Disruptions in Safety and Effectiveness Studies: Proposing Methodological Solutions

Background: Regulators, pharmaceutical industry, payers, and healthcare delivery systems continue to need timely, unbiased, actionable information derived from real-world data (RWD). However, the ability to draw robust causal inference is hampered when significant healthcare disruptions, such as those observed with the COVID-19 pandemic, occur. Although potential pandemic related disruption bias is well-recognized for studies spanning the pandemic period, there is currently no guidance on best practice to minimize this bias.

Objectives: This symposium uses case studies based on RWD on COPD and asthma from the United States and Europe to:
1) Illustrate how changes in treatment and healthcare seeking patterns during the early stages of the pandemic introduced distinct bias and confounding in comparative real-world studies; beyond those that are normally expected;
2) Discuss the resulting modifications needed for existing exposure and outcome measurement algorithms;
3) Introduce analytical strategies, including natural cubic spline regression and multilevel modelling, to adjust for COVID-19 associated healthcare disruptions.
This session will benefit investigators from industry, regulatory, consulting and academic teams involved in generating comparative RWE from the COVID-19 pandemic period, as well as the decision makers using those data.

Description: The symposium will begin by reviewing the drivers of healthcare disruption in the early stages of the COVID-19 pandemic. Parameters, including real world treatment patterns, healthcare utilization and adverse disease outcomes will be described using COPD and asthma examples. Potential bias introduced into comparative RWE studies spanning periods before and during the pandemic will be discussed with a focus on exposure misclassification and pandemic associated confounding. Methodological solutions will be explored including adjustments to existing treatment exposure algorithms to reduce exposure misclassification, as well as the use of natural cubic spline regression and multilevel modelling to adjust for pandemic associated confounding. Approaches will be discussed using a treatment effectiveness example with methods performance quantified against a benchmark treatment effect measured during the pre-pandemic period. Finally, the regulators’ perspective will be explored discussing how pandemic associated methodological challenges are considered for regulatory decision-making. An interactive discussion with the audience and panel will further inform methodological approaches to strengthen the validity of RWE generated during the COVID-19 pandemic period and potential application for future health threat crisis.