78% of Doctors Inflate Estimates of Medical Procedure Success

The success of multi-step medical procedures is often overestimated by doctors.

Medical treatment is seldom simple, whether a woman is giving birth or a man is undergoing a cancer biopsy. An unanticipated problem might happen at any time, especially with treatments that involve numerous steps.

However, recent research by experts from the University of Utah Health and its partners found that doctors often have unrealistic expectations about the success of complicated medical operations. Inflated success predictions, according to researchers, might have a negative impact on treatment choices and result in unintended harm to patients.

Overall, almost 8 out of 10 doctors who responded to the study thought there was a higher possibility of getting the intended result from an operation than there was of one or more stages leading to that outcome being successful.

According to Scott Aberegg, M.D., a critical care pulmonologist at the University of Utah Health, the study, which was published in JAMA Network Open, exposes a serious logical gap among doctors who fail to recognize that each step in the process carries its own risks that can reduce the likelihood that the desired medical outcome will be achieved.

“All too often, doctors act as though the stars align more frequently than they actually do,” Aberegg says. “They tend to focus on the desired outcome rather than the actual chances of success involved in each intermediary step. We can’t continue making medical decisions that way. We need to base them on more realistic expectations.”

Conjunction Fallacy: Misjudging Probability in Medical Scenarios

In order to ascertain how often a phenomenon known as conjunction fallacy occurs in medicine, Aberegg, Hal Arkes, Ph.D., of Ohio State University, and Kevin Arpin, Ph.D., a forensic expert at Travelers Insurance in Connecticut, conducted the research.

When a person thinks that a combination of events is more probable than any one of its individual parts, it is known as the conjunction fallacy.

Suppose, for instance, that a doctor notices skin growth on a patient and has 80% suspicion that it is cancerous. There’s also an 80% chance that the pathologist sees cancer on a biopsy specimen in the lab. The false assumption—the conjunction fallacy––would be that there’s more than an 80% chance that the pathologist will see cancer on the patient’s biopsy specimen.

In reality, the probability that the pathologist will see cancer on this patient’s biopsy is 64%, because first the patient actually has to have cancer, and then the pathologist has to see it on the biopsy.

“Many physicians simply aren’t good at calculating probability,” Aberegg says. “As a result, they commonly miss opportunities to make better treatment decisions.”

In their study, Aberegg and colleagues asked 215 obstetricians and pulmonologists to evaluate scenarios that they might encounter while caring for patients.

For example, in one scenario, obstetricians were confronted with a 29-year-old pregnant woman in labor. However, the child is not positioned properly for a vaginal birth. In this case, the doctors were asked to estimate the probability that the child would move into a deliverable position and be born without the need for a C-section.

Overall, 78% of the physicians who evaluated one of three scenarios in the survey estimated that the probability of the desired outcome would be greater than the likelihood of the two individual events required for it to occur. This is a mathematical impossibility, Aberegg says.

“Our study shows that if you poorly estimate the probability of two events needing to happen to get the result you desire, then you could be putting your patients at unnecessary risk,” Aberegg says. “In the case of the childbirth scenario, you could end up waiting around for a long time for that baby and end up having to do a C-section anyway. That delay could be harmful for both mother and child.”

All of the physicians who participated in the surveys had an average of 25 years of experience. Yet this expertise did not appear to prevent them from opting for the conjunction fallacies presented in the study. However, this isn’t too surprising since previous research found that nearly 50% of medical students are prone to these types of probability errors, according to Aberegg.

Need for Improved Medical Education on Probability

“There are enormous opportunities in medical education to improve the curriculum in terms of teaching the importance of probability in medical settings,” Aberegg says. “Numbers are the most reliable source of correct decisions in medicine.”

Aberegg urges practicing physicians to not only rely on their experience but also do their best to stay up-to-date on the latest probability research published in medical journals about various conditions and procedures.

Among the study’s limitations is that the participants were asked for written responses that might have been different had they been providing care to real patients.

However, Aberegg believes the study could have broad implications.

“Our results are very strong,” Aberegg says. “We’re confident that they represent a generalized phenomenon in medicine. I’m interested in further cataloging more examples so that the full breadth of this potential problem can be exposed and hopefully resolved.”

Reference: “Analysis of Physicians’ Probability Estimates of a Medical Outcome Based on a Sequence of Events” by Hal R. Arkes, Ph.D., Scott K. Aberegg, MD, MPH, and Kevin A. Arpin, Ph.D., 27 June 2022, JAMA Network Open.
DOI: 10.1001/jamanetworkopen.2022.18804

The study was self-funded by Aberegg, Arkes, and Aprin.

Never miss a breakthrough: Join the SciTechDaily newsletter.
Follow us on Google and Google News.