Background: In the management of chronic conditions (e.g., Diabetes, HIV, and Parkinson's disease), clinicians must treat patients at multiple time-points for achieving the best long-term outcomes, involving dynamic treatment regimes (DTRs). Different types of methods (e.g., mathematical, statistical, machine learning) have been adapted to build evidence-based tools for helping clinicians choose the best treatments.
Objectives: To provide insights into which methods can be applied under certain conditions (e.g, types of medical conditions, short or long-term outcomes, and characteristics of the dataset) for calculating DTRs utilizing observational data.
Methods: A systematic review was conducted including studies using observational data to calculate DTRs published in PubMed or EMBASE between January 1950 and until January 2022. Only peer-reviewed full research articles were included. Two independent reviewers screened the titles and abstracts using the tool, ASReview, and the relevant articles were full-text screened for inclusion. Information from the articles was extracted in duplicate using a predefined data extraction form. Extracted data included, for example, the name of the method, therapeutic area, and validation method.
Results: In total, 54341 articles were identified. Of those, 100 were considered eligible for inclusion after the full-text review. Most articles (77.0%; 77/100) were published after 2016 and involved the therapeutic areas critical care (21.0%; 21/100; of which 15 were on sepsis), and cardiovascular diseases (16.0%; 16/100; of which six were on bleeding or thrombosis). We found that reinforcement learning (RL) methods (42.0%; 42/100) and causal inference (CI) based methods (21.0%; 21/100) were the most common for computing DTRs. Specifically, Q-learning and dynamic programming were the common RL methods, while marginal structural models and g-estimation were the common CI methods. 79.0% (79/100) of the articles validated their methods, where 73.4% (58/79) were based on observational data, and 6.3% (5/79) consulted with clinicians. In the former validation category, 22.4% (13/58) estimated the error of the model using a variety of measures (e.g., accuracy, root mean square error, square error, absolute error, precision, and area under the curve), and 48.3% (28/58) estimated the patients’ expected outcomes.
Conclusions: The use of observational data for calculating DTRs has increased in the past years; RL and CI based methods are the most commonly used. More than two-thirds of articles validated their methods using different types of evaluation methods. Further research is required to assess which of these algorithms and validation methods are most appropriate in specific research settings.
