(070) Medical chart review of selected adverse event of special interest (AESI) to support rapid safety assessment of the Janssen COVID-19 vaccine (JCOVDEN): a validation study using US electronic health records and claims data

Publication Number: 407

Background: Regulatory guidance highlights the need to conduct validation studies to evaluate the performance of algorithm-based computable phenotypes.

Objectives: Rapidly conduct case validation across multiple data partners of select AESIs JCOVDEN and mRNA COVID-19 vaccine recipients in the US.

Methods: We conducted rapid safety surveillance across 3 data partners: Regenstrief Institute (RI) with access to the Indiana Network for Patient Care (a health information exchange), Department of Veterans Affairs (VA) with nationwide VA electronic medical records, and Optum Life Science with access to Optum’s pre-adjudicated medical claims and adjudicated pharmacy claims. Suspected cases of hemorrhagic stroke, immune thrombocytopenia (ITP), arterial co-occurrence of thrombosis and thrombocytopenia, co-occurrence of venous thrombosis and thrombocytopenia and transverse myelitis (TM) that occurred following vaccination were identified using the same predefined structured data algorithms across partners. Each data partner independently conducted medical chart adjudication of AESIs by trained clinicians and estimated the positive predictive value (PPV). The number of charts varied by partner and AESI.

Results: For hemorrhagic stroke PPV ranged from 10.0% (1 chart adjudicated case / 10 structured data identified suspected cases) in Optum to 50.0% (25/50) in VA to 87.4% (180/206) in RI. For ITP events PPV ranged from 15.4% (2/13) in Optum to 31.5% (76/286) in VA to 71.6% (78/109) in RI. For co-occurrence of arterial thrombosis and thrombocytopenia events PPV ranged from 12.5% (3/24) in Optum to 50.6% (125/247) in RI to 66.0% (136/206) in VA. For venous co-occurrence of thrombosis and thrombocytopenia events PPV ranged from 36.8% (7/19) in Optum to 77.6% (142/183) in RI to 71.7% (76/106) in VA. For TM events PPV ranged from 10.5% (2/19) in VA to 50.0% (1/2) in Optum to 52.9% (9/17) in RI.

Conclusions: We identified and validated events with PPVs ranging from 10% to 87% within 3-12 months of implementation with each data partner. PPVs varied by AESIs and database. The ability to determine differential PPV across events occurring before and after vaccine exposure was not possible given the low number of events. Future work will continue to explore best methods to conduct rapid validation of RWE.