Goldwater Institute
By Jeffrey A. Singer, M.D., and Rafael Fonseca, M.D.
Much has been written about how politics and ideology influence research funding, suppress research in certain areas, and lead to the cherry-picking and misrepresentation of evidence in support of a narrative or agenda. Science journalist John Tierney explored “The Real War on Science” in an excellent essay in City Journal in 2016. Reflecting on this phenomenon in 2011, Patrick J. Michaels stated:
The process is synergistic and self-fulfilling. Periodicals like Science are what academia uses to define the current truth. But the monolithic leftward inclination of the reviewing community clearly permits one interpretation (even if not supported by the results) and not another. This type of blatant politicized science is becoming the norm in the environmental arena, and probably has infiltrated most every other discipline, too.
It certainly has infiltrated research into the emotionally charged opioid overdose problem afflicting the US and many other western nations. Policy decisions have been rooted in a narrative seemingly immune to the facts: that the problem is largely the result of greedy pharmaceutical companies manipulating careless and poorly-trained doctors into “hooking” patients on highly addictive opioids and condemning them to a nightmarish life of drug addiction.
Tierney writes of confirmation bias—the tendency of people to seek out and accept information that confirms their beliefs and prejudices. He bemoans the “groupthink” that allows confirmation bias to infiltrate the peer review process. He cites a well-known study that demonstrated reviewers were more likely to find problems with a study’s methodology if the findings were contrary to their prejudices yet overlook methodological shortcomings if the findings were confirmatory.
Sometimes investigators try to “spin” their findings to make them comport to the narrative and appear confirmatory, increasing the likelihood that their research gets published.
Both of us are practicing physicians, and each of us recently experienced reminders that research into the opioid overdose issue is not exempt from politicization and confirmation bias. We would like to present two recent examples where this confirmation bias became self-evident.
One of us, Rafael Fonseca, recently encountered a peer reviewed publication that asserted, and concluded by conjecture, that opioid manufacturers, by providing meals to physicians at educational presentations, were skewing prescription patterns and increasing the number of opioids being prescribed. A cursory review of the published data suggested that correcting for variables such as specialty was needed to understand such putative association. After undertaking a full data analysis (reported in the Healthcare Blog with Dr. John Tucker), we were able to refute the findings of that publication. In short, we found that the influence of meals provided on prescribing was negligible, and that similar effects were seen when providers attended meals provided by companies that produce other products used for the treatment of pain, but not opioids. We provided a compelling case that increased attendance to these meals, and opioid prescription, was more of a reflection of the practice pattern of such physicians (i.e. they treat pain patients) rather than a heinous quid pro quo. Readers are referred to our analysis and the original paper.
We remain disappointed by the apparent ease with which such publications appear in major medical journals as well as the scarcity of detailed rebuttals. The authors of this paper did not discuss considerations that are relevant such as multivariate analyses. Not only were these missing, but the article concluded by suggesting that policy changes are needed, and that companies should be prevented from supporting such meals. Individuals who served as peer reviewers of this article apparently missed the limitations we presented in our independent review and accepted at face value the conclusions presented. The editors of the journal did not consider confounding covariates, which as we have shown, would make the analysis questionable. Perhaps, most troubling, is that a letter to the editor was submitted with our findings and was rejected as being of “low priority.” This was problematic given that, even though our letter was an abridged version of our full analysis, it directly refuted the conclusion of the paper. We cannot claim ill intention in the process but are surprised that even when pointed out, it seems that a prevailing narrative trumped scientific rigor. The prevailing current narrative is that greedy pharmaceutical companies duped doctors to inappropriately prescribing opioids. If we want to curtail deaths associated with opioid overdoses we must find the correct factual context of the problem. Facts are established with science and can be distorted by heuristics serving ideology.
At the time of this blog post we have requested the primary data from the authors to resolve data inconsistencies. We have not received the data, because the authors claim they intend to make it public at some point in the future pending a publication they have submitted. Through social media channels an Associate Editor of the journal was recently made aware of our analysis and stated it should be submitted as a letter to the editor or as a publication. We offered to write it as a full report and noted that our letter had already been rejected, and we received no further response. We have also contacted the Editor but have yet to get a response.
An example of how researchers “spin” their findings to comport with the prevailing narrative and increase the likelihood of publication occurred on January 17, 2018 when Jeffrey Singer encountered a story in the Los Angeles Times touting a recently published study in the peer-reviewed medical journal BMJ, in which the principal finding was that refilling opioid prescriptions given to patients for acute pain dramatically increased their risk of addiction. The Times reporter wrote:
A study published Wednesday in the BMJ finds that for every additional week a patient takes drugs like oxycodone and hydrocodone, the chance that he or she will wind up abusing the drug increases by 20%. And every time a prescription for opioid painkillers is refilled, the risk of abuse rises by 44%.
The reporter was accurate. The study by researchers at Harvard and Johns Hopkins looked at 568,000 opioid “naïve” patients in the Aetna health insurance data base given prescription opioids for acute postoperative pain over the period of 2008-2016. It began the conclusion to its abstract with: “Each refill and week of opioid prescription is associated with a large increase in opioid misuse among opioid naïve patients…” But what the Times reporter neglected to mention, and what the study’s authors only mentioned in passing, was the initial finding: the “total misuse rate,” i.e., rate of all opioid misuse diagnostic codes (defined separately as dependence, abuse, and overdose—a broad category within which addiction is only one component) among the 568,000 patients prescribed the opioids, was 0.6 percent. It was only upon reading the actual study as opposed to the press coverage that this rather encouraging news—opioids prescribed for acute pain have a very low misuse rate—became apparent.
Instead of emphasizing this encouraging finding, the bulk of the study investigates the effect the duration of time a patient is on opioids—expressed principally by numbers of refills—has on the misuse rate. The authors indeed found that each refill and additional week of opioid use was “associated with an adjusted increase in the rate of misuse of 44%.”
However, looking at the actual numbers behind those percentages finds the incidence of opioid misuse rose from 145 cases per 100,000 person years, or 0.15 percent per year, in patients who had no refills, to 293 cases per 100,000 person years, or 0.29 percent per year, for persons who had one refill. Indeed, that is nearly double. But if you nearly double a very low number, you still get a low number. Also, the correlation of prescription refills with an increase in misuse does not prove causation. Possible causes are debatable, with many confounding possibilities. The study lacks any discussion or exploration into matters of cause and effect.
Rather than point out that the incidence of misuse is extremely low in patients given opioids for acute postsurgical pain, even after multiple refills, the authors chose to pass over the low overall incidence of misuse and instead focus on the “large increase in opioid misuse” seen with each refill and week of opioid use.
These are just two examples of how scientific research is susceptible to the biases of the researchers and influenced by political exigencies. They both fed into the prevailing narrative animating policy toward opioid use, abuse, and overdose. Both provide good examples of how researchers as well as peer reviewers fall easy prey to confirmation bias.
Readers should approach every new “study” reported in the peer-reviewed science literature with a modicum of skepticism.
Jeffrey A. Singer, MD, and Rafael Fonseca, MD are Visiting Fellows at the Goldwater Institute.
