18 out of 19: How Behavioral Science Nailed It in COVID Policymaking

Social Distancing Concept

A study led by Kai Ruggeri at Columbia University, with over 80 collaborators, highlights the importance of behavioral sciences in policy decisions, particularly in the context of the COVID-19 pandemic. The study revisits and validates the recommendations of a highly influential 2020 paper by Jay Van Bavel and Robb Willer, which influenced global pandemic policies. It emphasizes the need for robust evidence in policymaking, identifies gaps in the original paper, and suggests improvements for future crisis responses.

A global study validates the significant role of behavioral sciences in COVID-19 policymaking, confirming most recommendations from a seminal 2020 paper. It also underlines the importance of evidence-based policies and offers insights for future public health crises.

A new global study led by Kai Ruggeri, PhD, at Columbia Mailman School of Public Health involving over 80 collaborators from more than 30 countries underscores the crucial role of behavioral sciences in formulating policy decisions, while also asserting the need for clear standards for what evidence gets used in policy decisions. The findings were published today (December 13) in the journal Nature.

Impact of Behavioral Science on COVID-19 Policy

In April 2020, a group of researchers published a highly influential paper with 19 policy recommendations around COVID-19 based on insights from the behavioral sciences. The paper was a large collaboration of over 40 experts, led by Jay Van Bavel of New York University and Robb Willer of Stanford, and was cited thousands of times by governments, researchers, and public figures. Its recommendations covered topics such as official messaging on social distancing, how to get a vaccine once they were available, and the need to work within communities to create real impact. Now, Ruggeri et al.’s new paper in Nature evaluates evidence since the first paper’s publication supports its claims and their applicability for policymaking.

Evaluating the Evidence for Public Policy

“Governments around the world formulated pandemic policy strategies explicitly on the basis of the behavioral concepts highlighted in the 2020 paper by Jay J. Van Bavel et al.,” says Ruggeri, a professor health policy and management at Columbia University’s Mailman School of Public Health. “Given concerns over a lack of public trust in science, particularly in the context of COVID-19, we believed it was important to evaluate the evidence for public policy recommendations, in a way that promotes transparency and builds trust.”

Two independent teams of 72 experts—including both the 2020 paper’s authors, as well as an independent team of evaluators—reviewed 747 pandemic-related research articles to assess the extent to which claims in the original paper provided valid policy guidance. They treated studies conducted (and replicated) in real-world settings across large populations in multiple settings as the highest level, and flagged arguments that were not backed by empirical evidence.

Findings and Contributions of Behavioral Science to Policy

Alex Haslam, PhD, professor of psychology from the University of Queensland in Australia and study co-author, says, “In recent years, there has been a lot of discussion about the limitations of psychological and behavioral science, especially in the face of the so-called ‘replication crisis.’ As a counterpoint to this, what this research showed is that there is a core of good theory in these fields that provides a strong basis for both scientific prediction and public policy. This theory may not always be flashy, but it is the bedrock of good social science, and this study confirms that it is something we can rely on for guidance when we need it.”

Public Health Interventions and Evidence-Based Policymaking

The study finds evidence for 18 of 19 claims in the 2020 paper, including those related to sense of identity and community connectedness, leadership and trust, public health messaging, social cohesion, and misinformation. Of the 18, the 2020 paper correctly identified 16 relevant behavioral concepts during the pandemic as well as likely barriers to mitigating the spread of the disease and social challenges that would be faced by policymakers. The researchers found no effect for two proposed policies related to effective public messaging (that messages should emphasize benefits to the recipient, and that they should focus on protecting others). Notably, the team found no evidence to review for one high-profile recommendation in the 2020 paper, which suggested the phrasing “physical distancing” is preferable to “social distancing.”

The most strongly supported claims were the importance of interventions to combat misinformation and polarization, which proved to be vital for ensuring adherence to public health guidelines. Research also underlined the point that, to be effective, messaging needs to emanate from trusted leaders and to emphasize positive social norms.

Public health interventions that received the most attention were not necessarily the ones best supported by the most evidence. For example, handwashing was widely promoted as a strategy for stopping the spread of COVID, yet study effects were small to null, particularly compared to masking, isolation, distancing, and vaccines.

Regarding masking, early guidelines in some countries suggested the practice would not minimize COVID-19, but subsequent evidence pointed to the effectiveness of masking. Likewise, research also undermined guidance on the impacts of school closures and disinfecting surfaces. “While there are understandable pressures to issue guidelines quickly during a crisis, making policy decisions without adequate evidence can be costly in many ways,” says study co-author Katherine Baicker, PhD, Provost of the University of Chicago. “As new scientific evidence comes in over time, some people may view evolving policy guidance as a sign of incompetence—or even conspiracy—undermining trust in expertise. Policymakers must balance the need for expedience with the need for robust evidence and credibility.”

Identifying Gaps and Future Recommendations

The new study also identifies several domains missing from the 2020 paper. These included threat and risk perception, the role of inequality and racism, skepticism toward science, incentivizing behaviors beyond simply describing benefits (e.g., by providing financial rewards for vaccination) and the absence of clear leadership.

Finally, the research team provides recommendations to help researchers and policymakers respond to future pandemics and disasters. These include the need to study global populations, to do more field testing, and to be more specific in formulating testable questions. “The value of field testing what really works to change health behaviors can’t be overstated, and the strongest conclusions we’ve been able to draw in this article were often thanks to partnerships researchers forged with local governments and healthcare providers to carefully evaluate what actually adds value in the middle of a crisis,” says study co-author Katy Milkman, PhD, professor at the Wharton School of the University of Pennsylvania. The researchers also encourage scientists to forge more alliances with policymakers and decision-makers—in local government, hospitals, schools, the media, and beyond.

Conclusion and Perspectives From Key Authors

“This work has the potential to increase transparency and build trust in science and public health, and to directly inform the development of tools and knowledge for the next pandemic or other crisis. Researchers can be a viable source of policy advice in the context of a crisis, and our recommendations point to ways to further improve this role of social and behavioral science,” says study co-senior author Robb Willer, PhD, professor of sociology at Stanford University.

“This new paper rigorously evaluated policy recommendations from our original team to see if they were accurate, using large amounts of evidence and a new team of independent reviewers from around the globe. In addition to confirming the vast majority of our original claims, it sets a new gold standard for evaluating evidence when policy decisions, particularly urgent ones, must be made,” says Jay Van Bavel, PhD, professor of psychology, New York University, lead author of the landmark 2020 article, and co-senior author of the new paper.

Reference: “A synthesis of evidence for policy from behavioural science during COVID-19” by Kai Ruggeri, Friederike Stock, S. Alexander Haslam, Valerio Capraro, Paulo Boggio, Naomi Ellemers, Aleksandra Cichocka, Karen M. Douglas, David G. Rand, Sander van der Linden, Mina Cikara, Eli J. Finkel, James N. Druckman, Michael J. A. Wohl, Richard E. Petty, Joshua A. Tucker, Azim Shariff, Michele Gelfand, Dominic Packer, Jolanda Jetten, Paul A. M. Van Lange, Gordon Pennycook, Ellen Peters, Katherine Baicker, Alia Crum, Kim A. Weeden, Lucy Napper, Nassim Tabri, Jamil Zaki, Linda Skitka, Shinobu Kitayama, Dean Mobbs, Cass R. Sunstein, Sarah Ashcroft-Jones, Anna Louise Todsen, Ali Hajian, Sanne Verra, Vanessa Buehler, Maja Friedemann, Marlene Hecht, Rayyan S. Mobarak, Ralitsa Karakasheva, Markus R. Tünte, Siu Kit Yeung, R. Shayna Rosenbaum, Žan Lep, Yuki Yamada, Sa-kiera Tiarra Jolynn Hudson, Lucía Macchia, Irina Soboleva, Eugen Dimant, Sandra J. Geiger, Hannes Jarke, Tobias Wingen, Jana B. Berkessel, Silvana Mareva, Lucy McGill, Francesca Papa, Bojana Većkalov, Zeina Afif, Eike K. Buabang, Marna Landman, Felice Tavera, Jack L. Andrews, Aslı Bursalıoğlu, Zorana Zupan, Lisa Wagner, Joaquín Navajas, Marek Vranka, David Kasdan, Patricia Chen, Kathleen R. Hudson, Lindsay M. Novak, Paul Teas, Nikolay R. Rachev, Matteo M. Galizzi, Katherine L. Milkman, Marija Petrović, Jay J. Van Bavel and Robb Willer, 13 December 2023, Nature.
DOI: 10.1038/s41586-023-06840-9

5 Comments on "18 out of 19: How Behavioral Science Nailed It in COVID Policymaking"

  1. This is the exact opposite of anecdotal evidence, polling evidence, and pandemic outcomes, so I suggest back to the drawing board to try replicating these findings. They give a 95% for success, and well, COVID policy was an obvious disaster on all levels that has shaken the foundations of health and liberal democracy. They found an increase in trust and social cohesion and an improvement in misinformation, while social trust and cohesion is at record low levels in polls I’ve seen, and misinformation now floods from both crackpots and authorities alike. “Trust in science” was high, and I still believe in the process, but the institutions have lost the public trust. There are now growing large minorities in OECD countries believing that their governments are actively hostile to them, and at least in terms of prioritizing the community over the individual, they’re completely right. The idea that “inequity” is a main area to improve on, that it wasn’t woke enough, shows a blindness to the public reaction and the results.

    The inclusion of behavioral science into public policy is leading the public to see the policies and governments as not only scientifically wrong, but manipulative, despicable, and dystopian. Self-congratulation is unwarranted.

  2. Where do the authors of these articles get their data? The article cites the benefits of masking, isolation, distancing, and vaccines yet the more recent epidemiological work suggests that these had very limited positive, and possibly negative, effects. This article praises behavioral “science” because it created a message that governments and quasi-government agencies could push on the public at a time when that message was largely unsupported by the available data. Praising behavioral “science” for being able to manipulate peoples’ opinions without solid evidence is exactly why a large portion of society now rightfully distrusts the so-called “experts.”

    • The only possibility I can think of, is they’re congratulating themselves for people having broadly complied. We collectively wore the cloth masks that can’t stop an aerosolized virus, locked down to end the physical activity that promotes health, avoided the social contact that preserves mental health, stayed indoors where the virus spreads, and got the novel vaccine with 6-weeks of effectiveness half a dozen times with concerning side-effects, all while imposing these things on others abandoning our rights and freedoms, and calling it science.

      They need to consider fundamental behavioral science, and go back to Skinner. We’ve been shocked for a few years now and are unlikely to repeat our previously conditioned behavior; operant trust in scientific institutions and policymaking authorities is likely extinguished for generations.

  3. Wait. Something is amiss.

    Given that behavioral science is notorious for having issues with replicability, we should expect someone looking for some instances of confirmation will find them. We should also expect that someone looking for some contraindications will find them.

    Finding 18/19 confirmations isnt really a surprise. The real surprise would be someone earnestly looking for contraries doesnt find them.

  4. Charles G. Shaver | December 14, 2023 at 3:23 am | Reply

    Already investigating US chronic disease and mortality statistics in 2019 (with medical error announced to be the third leading cause of death in the US by John Hopkins researchers in May of 2016, with an average of 7,703 [CDC/NCHS/NVSS] Americans dying daily of all causes in 2017, the last year completed by then) it was immediately obvious in early 2020 that Covid-19 was not a serious threat to the healthy, just people with already epidemic preexisting conditions, comorbidities, unusual immune disorders and/or victims of medical errors; a farce, a fraud and a scam; a “scamdemic.” Perhaps, if Covid-19 had been a real pandemic the investigated policy decisions would have made some sense?

Leave a comment

Email address is optional. If provided, your email will not be published or shared.