Instructor Brigham and Women's Hospital Division of Pharmacoepidemiology and Pharmacoeconomics, United States
Background: Inferring causal effects in matched cohort database studies requires balance between compared groups on relevant baseline characteristics. In pharmacoepidemiology, balance is often sought by developing a propensity score (PS) model that includes potential confounders, then matching each exposed patient with non-exposed patients with a similar PS value. To assess balance, standardized mean differences (SMDs) are calculated for each of the confounders in the matched population. The cohort is considered well-balanced if SMDs are < 0.1. However, SMDs only reflect marginal balance in confounder distributions. Higher-order interactions between confounding factors can still lead to bias. However, imbalances in the joint distribution of confounders are typically not evaluated. These interactions may be particularly relevant in the context of proxy confounders, where several interacting covariates collectively act as a proxy for an unobserved confounding factor.
Objectives: Develop a non-parametric Bayesian test to assess if balance has been reached at different scales, including higher-order interactions of the confounding factors. We focus on the common case where there are many categorical potential confounders.
Methods: We assess balance by computing the posterior probability of the null hypothesis of equality in distribution of confounders between exposure groups, using a flexible Bayesian model. Unlike p-values, this can be directly interpreted as the probability that balance has been achieved. When the null hypothesis is rejected, we use functionals of the posterior to identify the confounders and/or the higher-order interactions that are responsible for the imbalance. We tested our approach in several simulations that realistically mimic cohort studies. The metric of interest was the power of the Bayesian test at detecting imbalance when type-I error was fixed at 5%. We compared the performance of the Bayesian test to the classification permutation, CrossNN, and CrossMST tests.
Results: In challenging simulation scenarios where marginal balance was achieved but higher-order interactions were responsible for residual imbalance between groups, power ranged from 5% (under the null) to 60%. In the same scenarios, the competitors had power ranging from 5% (under the null) to 20%.
Conclusions: The proposed approach is effective in identifying discrepancies in the joint distribution of multiple potential confounders, including high-order interactions, in several realistic simulations. When such discrepancies are identified, they can be addressed to improve the balance between exposure groups or ignored if deemed unimportant for the study.
