Facebook's design makes it unable to control misinformation

WASHINGTON (September 15, 2023) – As misinformation flourished online during the Covid-19 pandemic, a number of platforms announced policies and practices aimed at combating the spread of misinformation. Did those efforts work?

New research published today in Science Advances suggests that the Covid-19 vaccine misinformation policies of Facebook, the world’s largest social media platform, were not effective in combating misinformation. The study, led by researchers at the George Washington University, found that Facebook’s efforts were undermined by the core design features of the platform itself.

“There is significant attention given to social media platforms and artificial intelligence governance today. However, this discussion largely focuses on either content or algorithms. To effectively tackle misinformation and other online harms, we need to move beyond content and algorithms to also focus on design and architecture,” says David Broniatowski, lead study author and an associate professor of engineering management and systems engineering at GW. “Our results show that removing content or changing algorithms can be ineffective if it doesn't change what the platform is designed to do – enabling community members to connect over common interests – in this case, vaccine hesitancy – and find information that they are motivated to seek out.” 

Facebook is designed to build communities around the things people care about. To do so, it uses several different architectural features, including fan pages that promote brands and community celebrities, enabling a relatively small group of influencers to reach large audiences. These influencers can then form groups – that are explicitly designed to build communities – where community members can exchange information, including how to access misinformation or other compelling content off of the platform. These group members, and especially group administrators (who are often page content creators) can then make use of Facebook's newsfeed algorithms to make sure that this information is available to those who care to see it. 

The researchers found that, while Facebook expended significant effort to remove a lot of anti-vaccine content during the Covid-19 pandemic, overall engagement with anti-vaccine content did not decrease beyond prior trends—and, in some cases, even increased. 

“This finding –that people were equally likely to engage with vaccine misinformation before and after Facebook’s extensive removal efforts–is incredibly concerning. It shows the difficulty that we face as a society in removing health misinformation from public spaces,” says Lorien Abroms, study author and a professor of public health at GW’s Milken Institute School of Public Health. 

In the content that was not removed, there was an increase in links to off-platform, low credibility sites and links to misinformation on “alternative” social media platforms like Gab and Rumble, especially in anti-vaccine groups. In addition, remaining anti-vaccine content on Facebook became more—not less—misinformative, containing sensationalist false claims about vaccine side effects that were often too new to be fact-checked in real time. There was also “collateral damage,” say the researchers, as pro-vaccine content may have also been removed as a result of the platform’s policies and, overall, vaccine-related content became more politically polarized. Furthermore, anti-vaccine content producers used the platform more effectively than pro-vaccine content producers. Although both had large page networks, anti-vaccine content producers more effectively coordinated content delivery across pages, groups, and users' news feeds. 

Even when Facebook tweaked its algorithms and removed content and accounts to combat vaccine misinformation, the researchers say the architecture of the platform pushed back. 

“Think of Facebook’s architecture like a building. An airport is designed to direct people to easily and securely get people to and from gates, and a stadium is designed to safely gather a group of people for a show. If these buildings were not designed to balance travel and entertainment against safety and security, people might routinely get hurt. Now think of Facebook’s structure in the same way: it is designed to allow motivated people to build communities and easily exchange information around any topic.” says Broniatowski, noting that Facebook’s architecture sets the policy up for failure. “Individuals that are highly motivated to find and share anti-vaccine content are just using the system the way it's designed to be used, which makes it hard to balance those behaviors against public health or other public safety concerns. You have to change the architecture if you want to create that balance.” 

Broniatowski’s work suggests that social media platform designers could promote public health and safety by working collaboratively to develop a set of “building codes” for their platforms that are informed by scientific evidence to reduce online harms. 

"Building architects have to balance a building's design goals against compliance with rules to protect people who use them,” he explains. “They have to be designed to promote public safety by complying with fire, flood, pollution, and earthquake codes. They need to promote security by incorporating design features such as well-marked, accessible entrances, exits, and turnstiles to prevent vandalism, panic, terrorism, and riots. They have to promote public health by complying with codes for plumbing, sanitation, and ventilation, and they have to be designed to respect municipal codes such as noise and other nuisance ordinances. These rules are usually developed by a partnership between industry, government, and community organizations, and they are informed by solid science and practice. Governance strategies could facilitate these partnerships and support the science."

According to the researchers, this is the first and only scientific evaluation of the efficacy of the world’s largest social media platform’s attempt to systematically remove misinformation and misinformative accounts.

The study, “The Efficacy of Facebook’s Vaccine Misinformation Policies and Architecture During The COVID-19 Pandemic” was published in Science Advances. Researchers at Johns Hopkins University contributed to this report. The research was funded by The John S. and James L. Knight Foundation through a grant to GW’s Institute for Data, Democracy & Politics and by the National Science Foundation.

-GW-