You are here

Aim High: The Case for Non-Inferiority!

By: Richard Bermudes, MD

What is non-inferiority? I don’t think we are used to thinking in these terms. Do you remember the “my dad is better than your dad” conversations on the playground growing up?  No one ever argued their dad was non-inferior (figure 1).

What is a non-inferiority trial? How does this differ from a superiority trial? In some ways this is a difficult concept for doctors and patients. We are used to thinking in terms of better or wanting the “superior” treatment. Why would we want a non-inferior treatment? I don’t think I have ever been asked by a patient, “which treatment is non-inferior?” However, patients want treatments that are effective, tolerable and convenient.

In psychiatry, the norm is to compare an active intervention or drug to placebo. The typical study is designed to show that drug A is superior to placebo by a certain margin measured on a depression rating scale. We have a number of two-arm studies for several treatments, but there are very few multiple arm superiority trials and there are even fewer non-inferiority trials. In clinical research, we start to think of a non-inferiority when the treatments may only be marginally different.

The THREE-D study found intermittent theta burst (iTBS) was non-inferior to 10 Hz rTMS in patients with treatment-resistant depression (Blumberger et al 2018). Noninferiority trials test whether a new experimental treatment is not unacceptably less efficacious than an active control treatment already in use. Non-inferiority trials may be performed when the new treatment may offer important advantages over currently available standard treatments, in terms of improved safety, convenience, better compliance or cost (D'Agostino et al 2003).

How do we determine if a new form of TMS (iTBS) is not inferior to an established form of TMS (10Hz TMS)? We have to prospectively determine the maximum acceptable extent of clinical non-inferiority of an experimental treatment or the margin (M). In the THREE-D study, a margin of 15% was chosen to compare the response rates in each group, and a margin of 10% was chosen to compare the remission rates in each group.

Should you start treating patients with theta burst? That depends on a number of factors that may be unique to your community, the patients as well as payors or other regulations. If you do incorporate theta burst I recommend running an “internal non-inferiority trial” with the goal of continuous quality improvement (CQI).

Here is how it might work:

Group 1: MDD Canadian patients treated with theta burst per the THREE-D study.

Group 2: MDD commercially Insured patients from Northern California treated September 2018 through March 2019 (time period after launching theta burst program).

If the difference between remission rates is 10% or less, then the groups are not inferior (figure 2).” If the difference between Group 1 and Group 2 remission rates is greater than 10%, then the experimental treatment is inferior by the equivalence margin.

Group 1 remission rate- Group 2 remission rate < 10% (non-inferiority)

Group 1 remission rate- Group 2 remission rate > 10% (inferior effectiveness in Group 2)

What would contribute to inferior effectiveness? In this scenario we might look at patient selection or the patient population, number of treatments received, physician/technician training or other factors to decrease the difference in remission rates. “Inferiority” would be a call to action to look for correctable factors to decrease the margin.

What does the future hold for non-inferiority studies? Will this study design gain broader acceptance with payors, providers and regulators (e.g. FDA)? As researchers embrace this design, clinical providers should consider incorporating some form of non-inferiority testing so that the quality of TMS is maintained as different protocols transition from research to clinical Practice.


Blumberger, D.M., Vila-Rodriguez, F., Thorpe, K.E., Feffer, K., Noda, Y., Giacobbe, P., Knyahnytska, Y., Kennedy, S.H., Lam, R.W., Daskalakis, Z.J. and Downar, J., 2018. Effectiveness of theta burst versus high-frequency repetitive transcranial magnetic stimulation in patients with depression (THREE-D): a randomised non-inferiority trial. The Lancet, 391(10131), pp.1683-1692.

D'Agostino Sr, R.B., Massaro, J.M. and Sullivan, L.M., 2003. Non‐inferiority trials: design concepts and issues–the encounters of academic consultants in statistics. Statistics in medicine, 22(2), pp.169-186.