In January 2003, President George W. Bush asked Congress to back the President's Emergency Plan for AIDS Relief (PEPFAR) with a $15 billion commitment "to turn the tide against AIDS." Congress agreed, and the program--which was continued by the Obama and Trump administrations--has, as noted in The New England Journal of Medicine, "had an unprecedented impact on the pandemic of HIV and AIDS."
But how do we really know that? Yes, HIV rates have--with the exception of a few countries--declined, but can we say the programs PEPFAR funded in countries with the highest prevalence of HIV and AIDS were responsible for those declines? After all, childhood mortality rates from AIDS dropped in sub-Saharan countries that did not receive PEPFAR funding.
Unfortunately, PEPFAR didn't create a statistical plan at the outset that could easily answer these questions. A statistical plan is a bit like asking your accountant what data you should record before you start your business--so you'll know whether you're making or losing money. The question facing PEPFAR was whether they had collected enough data over the years to evaluate their program interventions.
These kinds of after-the-fact analyses are fraught with difficulty. "Planned evaluations are preferable, of course, but we can still learn a lot from post-hoc evaluations," says Donna Spiegelman, the newly appointed Susan Dwight Bliss Professor of Biostatistics at the Yale School of Public Health. "I would never let the perfect be the enemy of the good."
Few were as qualified as Spiegelman to take on the challenge of figuring out whether PEPFAR had worked. With a joint doctorate from Harvard in biostatistics and epidemiology and rigorous training in mathematical statistics, Spiegelman had become an expert in developing methods to evaluate public health prevention science. In 2014, she was the first (and as yet only) biostatistician to receive the National Institutes of Health Director's Pioneer Award, a sort of MacArthur genius grant for biomedical researchers that frees and funds them--$500,000 per year for five years--to pursue bold, risky, innovative research.
At around the same time as Spiegelman received the Pioneer Award, she met the newly appointed U.S. Global AIDS Coordinator and Ambassador-at-Large Deborah Birx. Birx, a physician and former colonel in the U.S. Army, had been director of the Centers for Disease Control and Prevention's Division of Global HIV/AIDS and worked as a physician in Kenya. As Kenya was one of the original recipients of PEPFAR funding, Birx thought it would be a good candidate for Spiegelman to look at; it would be the most data-rich environment among PEPFAR's recipients.
With doctoral student Dale Barnhart, Spiegelman focused on 10,000 clinics running what they call PMTCT--prevention of mother-to-child transmission--of AIDS. PEPFAR had spent $248 million supporting PMTCT in Kenya between 2004 and 2014, and child mortality for those aged under five years had halved during this period. "The basic idea," says Spiegelman, "was to see if child mortality went down as funding went up."
The rationale behind the intervention is that, in order to prevent the transmission of HIV to children, pregnant mothers have to be tested. "If they are tested," says Spiegelman, "they can be treated--and that treatment will drastically lower the transmission rate from 30 or 40 percent to below 1 percent."
That was the basic idea, but teasing out causal inferences was anything but simple. Thankfully, as PEPFAR is a U.S. government program, it has to report to Congress how much it spends each quarter. From this data, Barnhart extracted a decade's worth of spending on PMTCT. They then obtained neonatal and infant mortality data from the Kenyan government and other agencies.
By doing a "difference in difference" analysis, they were able to look at the effects of varying levels of funding for PMTCT among the Kenyan provinces over time, along with trends in local infant mortality, and then compare them to other provinces with similar demographics that did not have PMTCT programs--or had them but with less financial support. The team did a wide range of sensitivity analyses to model missing data, but these had little impact on the outcome.
"The results were quite dramatic," says Spiegelman. Provinces in the top quartile of PEPFAR spending in any given year over the study period experienced more than 30 percent lower infant mortality than those in the bottom quartile. "Cumulative PMTCT spending significantly reduced infant mortality, saving the lives of many Kenyan children," she says. "We've shown, and rigorously, that the U.S. investment in PEPFAR delivered on its promise."
All of this research was carried out while Spiegelman was professor of epidemiologic methods in the departments of epidemiology, biostatistics, nutrition and global health at the Harvard T.H. Chan School of Public Health. Since July 1, she has been director of the new Center for Methods in Implementation and Prevention Science (CMIPS) at Yale University's School of Public Health. "We're hoping to be the go-to place for implementation and prevention science methods in the world. It is my dream job."
###
JSM Talk: http://ww2.amstat.org/meetings/jsm/2018/onlineprogram/AbstractDetails.cfm?abstractid=329908
For details, contact: Donna Spiegelman.
Email: (preferred) donna.spiegelman@yale.edu
Tel: (617) 835-5119
About JSM 2018
JSM 2018 is the largest gathering of statisticians and data scientists in the world, taking place July 28-August 2, 2018, in Vancouver. Occurring annually since 1974, JSM is a joint effort of the American Statistical Association, International Biometric Society (ENAR and WNAR), Institute of Mathematical Statistics, Statistical Society of Canada, International Chinese Statistical Association, International Indian Statistical Association, Korean International Statistical Society, International Society for Bayesian Analysis, Royal Statistical Society and International Statistical Institute. JSM activities include oral presentations, panel sessions, poster presentations, professional development courses, an exhibit hall, a career service, society and section business meetings, committee meetings, social activities and networking opportunities. http://ww2.amstat.org/meetings/jsm/2018/index.cfm
About the American Statistical Association
The ASA is the world's largest community of statisticians and the oldest continuously operating professional science society in the United States. Its members serve in industry, government and academia in more than 90 countries, advancing research and promoting sound statistical practice to inform public policy and improve human welfare. For additional information, please visit the ASA website at http://www.amstat.org.