Which COVID-19 vaccine works better?

In the first head-to-head comparison of the effectiveness of the Pfizer-BioNTech and Moderna COVID-19 vaccines, researchers examined the electronic health records of veterans who had received each vaccine. Both vaccines were highly effective in preventing COVID-19 outcomes such as documented infection, hospitalization, and death.

However, the Moderna vaccine was found to offer an increased level of protection, including a 21% lower risk of documented infection and 41% lower risk of hospitalization, according to the research team, whose findings were published on December 1, 2021, in the New England Journal of Medicine.

“Both vaccines are incredibly effective, with only rare breakthrough cases,” said Dr. J.P. Casas,, a member of the research team made up of experts from the U.S. Department of Veterans Affairs (VA), the Harvard T.H. Chan School of Public Health, and Brigham and Women’s Hospital. “But regardless of the predominant strain—Alpha earlier and then Delta later—Moderna was shown to be slightly more effective,” said Casas, an epidemiologist and associate professor with Brigham and Women’s Hospital and Harvard Medical School and executive director of the VA’s Million Veteran Program for genetics and health research.

Researchers designed their comparative effectiveness study to address the previously unanswered question of which of the two mRNA vaccines is more effective. Effectiveness was measured in terms of five COVID-related outcomes: documented COVID-19, symptomatic disease, hospitalization, ICU admission, and death. The investigators relied on the electronic health records of U.S. veterans who received one of the two COVID-19 vaccines between early January 2021 and mid-May 2021.

As initially designed, the research focused on the Alpha variant that predominated at the time. The study matched 219,842 recipients of the Pfizer vaccine to the same number of recipients of the Moderna vaccine. The two groups were matched based on a variety of clinical and demographic factors that could affect outcomes.

Over the study’s 24-week follow-up period, the estimated risk of documented infection was 4.52 events per 1,000 people in the Moderna vaccine group and 5.75 per 1,000 in the Pfizer group. This represents an excess of 1.23 cases of documented infection per 1,000 people in the Pfizer group. The investigators also observed an excess of symptomatic COVID-19 (0.44 events), hospitalization (0.55 events), ICU admission (0.10 events), and death (0.02 events) per 1,000 people in the Pfizer group relative to the Moderna group, but these differences were smaller. 

This pattern of a lower risk for Moderna held up in an additional phase of research covering a time frame with Delta as the main strain. In this comparison, excess risk of documented infection over 12 weeks was 6.54 events per 1,000 people for the Pfizer vaccine, compared to Moderna. Given the shorter time frame available for this supplementary research, infection was the only outcome researchers analyzed. Also, the estimates were considered less precise because a smaller number of individuals were eligible for this analysis.

Randomized trials comparing the mRNA vaccines against placebos had previously shown both vaccines to be very effective against symptomatic COVID-19 infection (95% effectiveness for Pfizer-BioNTech, 94% for Moderna), and similar benefits were observed in real-world vaccine use.

“Given the high effectiveness of both the Moderna and Pfizer vaccines, confirmed by our study, either one is recommended to any individual offered a choice between the two,” said the study’s first author, Dr. Barbra A. Dickerman, an epidemiology instructor with the Harvard T.H. Chan School of Public Health. “However, while the estimated differences in effectiveness were small on an absolute scale, they may be meaningful when considering the large population scale at which these vaccines are deployed. This information may be helpful for larger decision-making bodies.”

The expansive VA records system, covering millions of patients nationwide, supported a very large sample size. This, in turn, allowed the study to identify even small differences in effectiveness between the Pfizer and Moderna vaccines. The researchers used a methodology known as causal inference to mirror a randomized trial—the gold standard in health research—as closely as possible. Causal inference is type of data analysis that helps researchers draw firm conclusions about cause and effect.

Causal inference experts on the research team included Dickerman and Dr. Miguel A. Hernán, a Harvard School of Public Health professor of biostatistics and epidemiology and director of the school’s CAUSALab. Dickerman, Hernán, and Casas co-direct the Methods Core of VA-CAUSAL, a VA-Harvard partnership focusing on the development of new methods for causal inference in research. 

A primary challenge for this research was ensuring that the vaccine groups under study were comparable with respect to attributes, other than the vaccine received, that may predict infection or disease severity. The VA databases allowed the researchers to precisely characterize recipients of each vaccine type and closely match them on age, sex, race, geographic location, and other attributes that could affect COVID-19-related outcomes.

“After this careful matching, we found that the two vaccine groups were extremely similar in terms of variables with respect to an extensive set of demographic, geographic, and health-related attributes,” Dickerman said. “This allowed our observational analysis to produce exceptionally credible results during a global emergency, when answers are needed fast and randomized trials can be impractical.”

As the global pandemic continues to unfold, the research team is working on answers relating to the comparative safety, versus effectiveness, of the Pfizer and Moderna vaccines. Dickerman characterizes comparative safety as an “additional piece of the puzzle to support vaccine decision-making.”

Even beyond this analysis, further evaluation of the vaccines’ comparative effectiveness and safety is needed, the authors concluded in their New England of Journal of Medicine article. Meanwhile, given the evidence already gathered, the authors concluded about the Pfizer and Moderna vaccines considered in their study, “Given the high effectiveness and safety profile of both mRNA vaccines, either one is strongly recommended.”