Is the sequence ratio an unbiased estimate of the incidence rate ratio? A simulation study

Background: The sequence symmetry analysis (SSA) is an increasingly used method for drug safety signal detection and the quantification of prescription cascades. The effect estimate obtained from a SSA is the sequence ratio (SR), which is assumed to be an estimate of an underlying incidence rate ratio (IRR), but this has not been evaluated formally.

Objectives: To evaluate under which conditions the SR is an unbiased estimate of the true IRR.

Methods: We simulated cohorts of 100,000 individuals who were followed for five years during which individuals could initiate an exposure drug and experience an outcome of interest. Times of medication initiation and the outcome were drawn from an exponential distribution. The rate of exposure was fixed at 1 event per 50 person years (PY) and the rate of the outcome varied between 1 event per 10 000 PY, 200 PY, 50 PY and 10 PY to simulate very rare, rare, common, and frequent events. The outcome rate among exposed individuals was further modified by a true IRR of 0.2, 0.5, 1.0, 2.0 and 5.0, yielding 20 different scenarios.
We evaluated additional scenarios where the outcome was fatal and led to immediate censoring with a probability of 0.01, 0.05, 0.1, 0.5, and 0.9, and a scenario where the outcome reduced the rate of exposure with rate ratios of 0.75, 0.5, 0.25, and 0.1 corresponding to the event of interest being a contraindication for treatment with the exposure drug.
For all scenarios, we calculated observed IRRs using a person time based cohort study and SRs using an SSA-design with an observation window of 365 days before and after initiation of the exposure drug. We simulated 2500 cohorts for each scenario and calculated the mean SR and IRR, and bias as log(estimate) – log(true IRR).

Results: We found the cohort estimator to be unbiased in all scenarios, except for the scenario with a very rare outcome (outcome rate 1/10 000 PY) with bias between −0.11 (true IRR 5.0, observed IRR 4.76) and +0.91 (true IRR 0.2, observed IRR 0.50). The SR was close to unbiased for rare, common and frequent events, except when the true IRR was 5.0 (mean SR 4.73 and 3.75, bias -0.06 and -0.29 for common and frequent events). The SR was strongly biased towards 1.0 when the outcome was very rare (bias −0.46, −0.84, +0.58, and +1.42 for true IRRs of 2.0, 5.0, 0.5 and 0.2). When the outcome was potentially fatal, the SR was upwards biased for censoring probabilities of 0.5 or greater (bias +0.68 and +2.33), independent of the true IRR. When the outcome reduced the probability of future exposure, the SR was biased for all simulated exposure rate ratios (+0.27, +0.67, +1.37, +2.30).

Conclusions: The SR is an unbiased estimate of the true incidence rate ratio, except when the outcome is very rare, the true IRR high, the outcome has a high mortality, or when the outcome reduces the probability of future exposure.