PhD Student University of North Carolina Chapel Hill Chapel Hill, United States
Background: Trial participants often differ from patients in real-world settings (i.e., the target population). When these differences are related to the efficacy of an intervention, applying randomized control trial (RCT) estimates to real-world settings without accounting for these differences can lead to misleading treatment effect estimates. We apply a novel visualization tool that allows researchers to understand systematic differences in the distribution of individual characteristics between trial and target populations.
Objectives: Apply a novel visualization that shows how a set of RCTs differ from a target population with respect to specified characteristics, the association of that characteristic with the outcome, and the sample size of the RCTs using COVID-19 as a case example.
Methods: We collected clinical and demographic profiles of 149 COVID-19 related RCTs from January 2020 to June 2021 and used data from the North Carolina (NC) Department of Health and Human Services that reports age and race distributions for case and death data attributable to COVID-19 for our target population. With this data, we created bubble plots assessing how age and race differed between the RCT participants and COVID-19 diagnosed patients in NC through July 2021. Each RCT with age or race information contributed a dot to the bubble plot. The X axis position was determined by the relative prevalence of the value of the covariate (e.g., Black race) in the trial compared to the NC target population. The y-axis position was determined by the relative risk of death in the NC target population for the individuals with that value of the covariate compared to the rest of the NC population. This allowed us to identify whether some covariate levels were underrepresented, the extent to which those covariates were related to COVID-19 mortality, and whether the lack of representation was more common in larger or smaller trials.
Results: There were clear patterns showing under-enrollment of Black individuals in COVID-19 RCTs compared with the NC target population. These differences persisted when we limited to studies enrolling patients within the US. Treatment trials appeared to underrepresent older adults compared to the NC target population of individuals hospitalized for COVID-19. This is likely due to the greater comorbidity burden among older adults hospitalized with COVID-19, which may disqualify them from trials.
Conclusions: By simultaneously displaying (1) differences in patient characteristics between RCTs and a target population, (2) the associations between these characteristics and outcomes in the target population, and (3) the size of each RCT, these bubble plots can help researchers identify potential threats to the generalizability of RCTs for target populations of interest.
