How 'Publication Bias' May Skew Our Understanding of Xanax Efficacy

A recent Psychological Medicine paper took advantage of an underutilized method that factors in unpublished studies to analyze the efficacy of alprazolam XR. The study found that claims about the popular anti-anxiety drug’s benefits for treating panic disorder may largely exaggerate its effectiveness.

A Comprehensive Model of Predictors For Panic Disorder

COVID-19–Associated Panic Disorder

Benzodiazepine Use, Misuse, and Use Disorders

Methodology and Findings

To conduct the study, Oregon Health & Science University School of Medicine and Harvard Medical School researchers compared the results from both publicly disclosed and undisclosed research listed in the US Food and Drug Administration (FDA) database.

Regardless of the outcomes, the FDA requires registration for all drug trials seeking market approval in the US. By analyzing the full set of FDA studies, and not just those that made it to publication, the researchers were able to paint a more complete picture of the drug’s true effectiveness.  

The FDA review included five trials looking at alprazolam, sold under the brand name Xanax. Only one was positive. Of the four not-positive trials, two were published conveying a positive outcome. Two were not published. 

When looking at only published studies, alprazolam appeared to have a perfect success rate. However, when taking into account all trial results, the drug was effective in only 20 percent of cases.

The team noted that the complete FDA data set showed an effect size of .33 when comparing alprazolam groups to controls, while published studies showed an effect size of .47. The meta-analysis found that the drug was still superior to a placebo, but not nearly as much as the published data conveyed. They reported that the published data exaggerated its efficacy by over 40 percent.

“Clinicians are well aware of these safety issues, but there’s been essentially no questioning of their effectiveness,” said senior author Erick Turner, MD, professor of psychiatry at the Oregon Health & Science University School of Medicine and former FDA reviewer. “Our study throws some cold water on the efficacy of this drug. It shows it may be less effective than people have assumed.”

Silent Impact of Unpublished Trials

The study authors blamed the overinflation of the drug’s benefits on a phenomenon known as “publication bias.” 

When research studies show statistical significance they tend to find their way into a peer reviewed journal. They are often pushed to the side when they don’t. Selective publication like this can create the misleading impression that a treatment is more beneficial than actually is. For example, doctors may prescribe alprazolam more often based on this distorted view, even when better options exist.

“This study brings to light unpublished trial data and provides a more balanced and realistic view of the efficacy of alprazolam XR, compared to what has been previously reported,” the researchers wrote. They go on to say that they are unsure if similar drugs face the same publication bias. However, previous research has identified the problem in other mental health conditions, like depression and anxiety. If Xanax were approved today, the researchers pointed out, its trials would probably be more transparent due to rule changes and increased awareness about this issue.

Size of the Problem

The FDA approved Alprazolam in 1981. It belongs to the benzodiazepine class of drugs which primarily address anxiety disorders that impact an estimated 40 million adults in the U.S. Despite its widespread use, alprazolam carries a series of potential adverse effects, ranging from appetite changes and nausea to more severe outcomes like impaired motor control and cognition. Recent research also links benzodiazepines to long-term cognitive decline.

While healthcare providers generally know about these risks, few have questioned the drug’s efficacy. One report showed that about 5 percent of Americans used anti-anxiety medication. Benzodiazepine prescriptions for anxiety declined by 12 percent from 2015 to 2019 but during the early stages of the Covid-19 pandemic, this trend reversed as people’s anxiety levels soared.

Turner said the findings may be especially relevant to patients who haven’t started on benzodiazepines, as opposed to those who have used the drug in the past, or who have already become physically dependent.

“This study will reinforce being cautious about starting a prescription,” he said.