With the cost of drugs a critical issue in health care, health insurance companies and government payers need to understand how new and existing drugs compare in terms of benefits and risks.
But there's a problem. When drugs are first approved, they have typically been compared in clinical trials to either a placebo or to one standard of care, which is an established treatment that has previously been widely accepted. However, there may be multiple drugs on the market that have already been shown to be better than the standard. And in diseases with high unmet needs, drugs may even be approved without any comparisons.
"This," says David Cheng, a postdoctoral researcher at Harvard's T.H. Chan School of Public Health, "limits our ability to compare the effectiveness of new drugs to all the other available treatment options that are out there."
To get around this problem, people often engage in "a kind of naïve comparison," says Cheng. "They'd look, say, at the rates of survival for a cancer drug by a given time in one study and then compare them to another, even though the two studies would not be directly comparable. The patients might have more late-stage disease in one study and more early-stage disease in the other, or some other significant difference in patient characteristics, and this wouldn't be taken into account in the analysis. You'd end up with massive confounding."
Dealing with such confounding bias is especially challenging as analysts and researchers often only have access to the full individual patient-level data for the new drug and must rely on data summaries from academic publications for existing drugs on the market.
To overcome the problem, analysts and researchers have turned to a method called matching-adjusted indirect comparison (MAIC). "If you have access to the individual-level data from one drug trial," says Cheng, "then you could reweight the observations or adjust the final analysis so that the patient characteristics match the summaries of another trial." Results provided by MAIC have been used in more than 20 successful reimbursement submissions and included in guidance on indirect comparisons issued by the National Institute of Clinical Excellence in the UK.
Despite the increasing use of MAIC to inform drug reimbursement decisions, the statistical performance of MAIC has not been extensively studied or reported. Research conducted by Cheng and managing principal James Signorovitch and colleagues from Analysis Group--a global consulting firm with expertise in health economics and outcomes--is the first to identify conditions under which MAIC is valid. If applied correctly, MAIC can provide unbiased estimates of a treatment effect when patient populations between trials are sufficiently similar, and the probability an individual is selected into one trial versus another can be adequately modeled. It also compares the potential for bias through simulations to some other common approaches to such comparisons across studies.
"This work can help decision-makers understand when MAIC results are reliable and when there are challenges in the data that would produce unreliable results," says Cheng. "This could, in turn, enable better decision-making and ultimately inform smarter allocation of resources to drugs that work best."
###
JSM Talk: The Statistical Performance of Matching-Adjusted Indirect Comparisons ww2.amstat.org/meetings/jsm/2018/onlineprogram/ActivityDetails.cfm?SessionID=215594
For details, contact: David Cheng
Email: dcheng01@fas.harvard.edu
Tel: (312) 218-1058
About Analysis Group's HEOR Practice
Founded in 1981, Analysis Group is one of the largest economics consulting firms, with more than 850 professionals across 14 offices. Analysis Group's health care experts apply analytical expertise to health economics and outcomes research, clinical research, market access and commercial strategy and health care policy engagements, as well as drug-safety-related engagements in epidemiology. Analysis Group's internal experts, together with its network of affiliated experts from academia, industry and government, provide our clients with exceptional breadth and depth of expertise and end-to-end consulting services globally.
About JSM 2018
JSM 2018 is the largest gathering of statisticians and data scientists in the world, taking place July 28-August 2, 2018, in Vancouver. Occurring annually since 1974, JSM is a joint effort of the American Statistical Association, International Biometric Society (ENAR and WNAR), Institute of Mathematical Statistics, Statistical Society of Canada, International Chinese Statistical Association, International Indian Statistical Association, Korean International Statistical Society, International Society for Bayesian Analysis, Royal Statistical Society and International Statistical Institute. JSM activities include oral presentations, panel sessions, poster presentations, professional development courses, an exhibit hall, a career service, society and section business meetings, committee meetings, social activities and networking opportunities. ww2.amstat.org/meetings/jsm/2018/index.cfm
About the American Statistical Association
The ASA is the world's largest community of statisticians and the oldest continuously operating professional science society in the United States. Its members serve in industry, government and academia in more than 90 countries, advancing research and promoting sound statistical practice to inform public policy and improve human welfare. For additional information, please visit the ASA website at http://www.amstat.org.