Senior Director Epidemiology The Janssen Pharmaceutical Companies of Johnson & Johnson Beerse, Belgium
Background: Weighing benefits and risks of vaccination is a crucial part of public health decision-making for immunization recommendations and vaccine use. These decisions increasingly rely on quantitative benefit-risk assessments (qBRA), which compare numeric estimates of averted health outcomes and associated adverse events for a vaccine.
Objectives: Our aim was to systematically review and critically appraise research articles and publicly available reports on COVID-19 vaccine qBRAs.
Methods: We used defined terms to identify potential qBRAs (search dates: 1 January 2019 - 1 April 2022) in 1) a systematic literature review, 2) a targeted review of public health authority (HA) websites, 3) a general internet search, and 4) by reviewing citations in identified reports and publications. After de-duplication and restriction to unique qBRAs, standard details from each source were extracted including study details; geographical location; target population; comparison group for the vaccine of interest; analytical approach including techniques, measures, stratifications, and sensitivity analyses; main findings; and stated limitations.
Results: Of the 1637 published articles, 8 reports from HA websites, 5 general internet items, and 2 sources found in reference lists, we identified 22 unique qBRAs. Eleven qBRAs were publications from academic researchers, 9 were made by HAs, and 2 were industry-led. All qBRAs were based in high income countries except one conducted for 13 middle income European countries. Nearly all qBRAs employed age-stratifications (n=19) and/or sensitivity analyses (n=16) on vaccine effectiveness, viral transmission rates, rates of risk outcomes, or population vaccine coverage levels. Most assessments focused on severe outcomes for both benefits and risks, such as hospitalizations, ICU admissions, and deaths. Identified sources varied in the level of detail provided for data sources, methods employed, and reported results.
Conclusions: Like previous assessments of publicly available qBRAs, we found that COVID-19 vaccine qBRAs lacked consistently reported information necessary to make comparisons across qBRAs. Emerging guidance and best practices including templates, examples, and checklists could promote improvement in the design and reporting of future qBRAs.
Open data sources used for qBRAs expanded greatly during the COVID-19 pandemic. Better standardization of these sources (definitions, categorizations, formats, and update frequencies) could improve comparability across qBRAs in the future.
