Study finds X’s (formerly Twitter’s) community notes provide accurate, credible answers to vaccine misinformation

As the proliferation of misinformation continues to pose a significant challenge on social media platforms, a beacon of hope emerges in research from the University of California San Diego.

A new study published in JAMA led by John W. Ayers, Ph.D., from the Qualcomm Institute within UC San Diego, finds that X's Community Notes, a crowdsourced approach to addressing misinformation, helped counter false health information in popular posts about COVID-19 vaccines with accurate, credible responses.

“Since the World Health Organization declared an ‘infodemic’ of misinformation, there have been surprisingly few achievements to celebrate,” said Ayers, who is vice chief of innovation in the Division of Infectious Disease and Global Public Health at UC San Diego School of Medicine and Deputy Director of Informatics at the Altman Clinical and Translational Research Institute in addition to Qualcomm Institute scientist. “X's Community Notes have emerged as an innovative solution, pushing back with accurate and credible health information.” 

Understanding Social Media Misinformation Countermeasures

Before the inception of Community Notes, social media companies employed various tactics to tackle misinformation, including censoring, shadowbanning (muting a user or their content on a platform without informing them), and adding generic warning labels to problematic content. “However, social media companies have been unwilling to disclose details of the inner workings of their efforts against misinformation or to share the necessary data to study their utility,” said Mathew Allen, study co-author and UC San Diego medical student. 

In late 2022, X introduced Community Notes—a paradigm shift in the fight against misinformation. This novel approach empowers volunteer, independent, anonymous, and ideologically diverse contributors to identify posts containing misinformation and to rectify misinformation by appending informative "notes" to suspect posts. The process is controlled by the public, instead of decision-makers at the company.

“Because Community Notes is a uniquely open-sourced misinformation countermeasure, it can be studied and improved using the scientific method,” added Allen. 

Evaluating X’s Community Notes

The research team obtained all notes that mentioned terms related to vaccines or COVID along with their corresponding posts made during the first year of the Community Notes program, from December 12, 2022, through December 12, 2023. Of 45,783 notes, 657 address COVID-19 vaccination, with the monthly rate of notes increasing from 22 to 186 during the study. 

A team of evaluators working with an infectious disease physician and virologist evaluated the subject, accuracy, and source credibility of randomly sampled notes. Of the notes examined, 51% addressed adverse events attributed to COVID-19 vaccination; 37%, conspiracy theories; 7%, vaccine recommendations; and 5% vaccine effectiveness. In terms of accuracy, 97.5% of notes were entirely accurate; 2%, partially accurate, meaning they addressed scientifically debated conclusions; and 0.5%, inaccurate. In terms of sources, 49% of notes cited highly credible sources (such as primary data sources, like peer-reviewed studies); 44%, moderately credible sources (such as major news outlets or fact checkers); and 7%, low credibility sources (such as blogs or tabloids).

“Notes typically addressed obvious misinformation, offering corrections from credible sources,” said Nimit Desai, a study co-author and UC San Diego medical student. "It's remarkable to witness the online community's adeptness in steering conversations towards accurate and high-quality evidence when provided with the right tools."

The sample of notes studied was attached to posts that averaged 1,064,981 views, extrapolating to between 500 million and 1 billion views for all COVID-19 vaccination-related posts noted. 

"Our study shifts the focus from talking about misinformation to taking action, offering practical insights into social media strategies that protect public health," explained Mark Dredze, Ph.D., the John C Malone Professor of Computer Science at Johns Hopkins University and study co-author. "Although we couldn’t examine how these notes directly influenced people's beliefs or actions, the characteristics we analyzed have consistently been shown to predict a message's effectiveness."

Learning from and Enhancing X’s Community Notes

U.S. Food and Drug Administration Commissioner Robert Califf recently argued vaccination is approaching a dangerous tipping point because of social media misinformation. "One viable avenue for the public health community to combat this threat is to actively engage in social media-based interventions, such as Community Notes," said physician-scientist and study co-author Davey Smith, M.D., chief of the Division of Infectious Disease and Global Public Health and professor in the UC San Diego School of Medicine, co-director of the Altman Clinical and Translational Research Institute at UC San Diego, and immunologist at UC San Diego Health. "While only a fraction of vaccine misinformation posts are currently addressed, the ample room for expansion suggests significant opportunities to amplify the impact of Community Notes."

Eric Leas, Ph.D., co-author of the study, assistant professor at the UC San Diego Herbert Wertheim School of Public Health and Human Longevity Science and Qualcomm Institute affiliate, highlighted the transformative role of Community Notes in combating misinformation. “Rather than censoring misleading content, Community Notes fosters a learning environment where users can glean insights from corrections to misinformation to prevent similar misunderstandings in the future. By providing context and credible sources alongside contentious posts, the platform empowers users to discern fact from fiction, a skill they will find useful as they navigate all claims.”

Ayers concluded, "Other social media platforms should embrace transparency by open-sourcing their misinformation countermeasures. This step is crucial for enabling independent scientific scrutiny, which will enhance public trust in and amplify adoption of the most impactful strategies."

In addition to Ayers, Allen, Desai, Leas, Dredze, and Smith, authors of the JAMA paper, “Characteristics of X’s (Twitter) Community Notes Addressing COVID-19 Vaccine Misinformation” (JAMA doi:10.1001/jama.2024.4800), include Aiden Namazi, a student research intern at UC San Diego’s Qualcomm Institute.