Research Scientist PPD, part of Thermo Fisher Scientific, United States
Background: The need to develop SARS-CoV-2 diagnostic tests in response to the COVID-19 public health emergency contributed to the use of real-world (RW) evidence for regulatory decision making. Since the COVID-19 pandemic started, rapid antigen tests (RATs) and reverse transcription polymerase chain reaction (RT-PCR) tests have been submitted to the United States (US) Food and Drug Administration (FDA) for Emergency Use Authorization (EUA), and in Europe under the In Vitro Diagnostic Medical Device Directive (IVDD) or the In Vitro Diagnostic Medical Device Regulation (IVDR) for approval. The standards for validation studies under regulatory guidance have now been widely applied.
Objectives: To describe the methodological approach to clinical performance validation studies (CPVS) for SARS-CoV-2 RATs conducted in the context of EUA, IVDD, and IVDR submissions.
Methods: We reviewed the results of 7 CPVS for SARS-CoV-2 RATs that we conducted across 33 US point-of-care facilities. We present findings and summarize methodologic approaches for design and implementation.
Results: CPVS are conducted in a RW setting, prior to market authorization and this poses unique challenges. Study operators need to be representative of target operators under RW conditions with no more instructions than they would have routinely received. Our 7 studies included 72 operators who collected nasal or nasopharyngeal samples after self-review of the device’s instructions-for-use.
Test positivity rate is an important metric, which can signal a test performance issue, a test misuse, or an epidemiological trend. Our results showed that 54.8% of excluded samples were due to operator errors. User accuracy tests with low positive ( <2x Limit of Detection) and negative samples showed a large variation in test operator agreement (positive: 30.0-97.9%; negative: 86.5-100.0%).
In our studies, 4 FDA-cleared or EUA-approved SARS-CoV-2 RT-PCR tests were used as comparators. Compared to RT-PCR results, RATs showed moderate sensitivity [Positive Percent Agreement (PPA): 70.8-90.0%] and high specificity [Negative Percent Agreement (NPA): 98.4- 100.0%].
Conclusions: In our review, RATs had moderate PPA and high NPA, making them attractive assays that are easy-to-use with results that are easy to interpret, allowing testing capacity to be scaled up at lower cost. In our experience, the same validation study approach could be used to support clinical performance evaluation for molecular diagnostics. These studies, which have a large RW component, require careful methodological planning, gold standard choice, and monitoring. The learnings from the numerous EUA of SARS-CoV-2 diagnostic tests validation studies can be applied to validation of diagnostic devices for different intended use.
