Vice President, Research OM1, Inc. Tufts University School of Medicine Boston, United States
Background: Many patients with major depressive disorder (MDD) suffer from treatment-resistant depression (TRD), but there is no consensus on how to define and identify TRD in practice. Existing definitions are challenging to implement in real-world datasets and lead to wide differences in estimated TRD prevalence, impeding research on TRD outcomes. Machine learning methods have demonstrated strong performance in identifying patients with specified characteristics at scale, but their potential to reliably identify TRD cases in real-world data is unexplored.
Objectives: To evaluate the performance of a machine learning algorithm in identifying physician-attested cases of TRD within a broader MDD population using structured real-world data.
Methods: This study was conducted using data from the PremiOM™ MDD Dataset, a large, de-identified multi-source RWD network with claims and specialty EMR data on US adult MDD patients. Within this dataset, clinical notes from mental health specialist providers recorded between January 1, 2013 and December 31, 2021 were examined for TRD attestation to identify a TRD-positive cohort. A TRD-negative group was sampled among MDD patients with no TRD attestation, at a ratio of 4:1. The study dataset was divided into training (80%), testing (10%), and validation (10%) sets. A machine learning-based classification tool (OM1 Patient Finder™) was calibrated using the training set to distinguish labeled TRD cases using structured health history data (coded diagnoses, procedures, lab tests, medication history, and demographic factors) from within one year prior to the prediction index date. Classification performance was evaluated using the validation set.
Results: There were 3,771 patients that met attestation criteria and formed the TRD-positive cohort; 15,084 MDD patients with no TRD attestation formed the TRD-negative cohort. The machine learning model performed well in identifying TRD-positive cases (validation set AUROC: 0.87). Performance remained strong when evaluated only on men (33.1% of study dataset, AUROC: 0.88); only on women (66.9%, AUROC: 0.87); and within selected age bands. Performance was driven by a range of factors, including indications of MDD severity, patterns of prior treatment, and evidence of sex hormone imbalances.
Conclusions: A machine learning model trained using a physician attestation-labeled TRD cohort successfully identified positive TRD cases using only structured medical record data. This performance demonstrates that a machine learning tool can label TRD patients similar to those considered TRD-positive by expert clinicians. Future research should focus on further validation of this approach and its utility in capturing TRD patient cohorts relative to other methods of identifying TRD.
