Especially in the case of accelerated approvals and drugs for rare diseases (orphan drugs), informative data for the early benefit assessment of drugs are often lacking at the time of market access. The underlying studies are then too short and do not contain any comparisons with the previous health care standards, which makes the quantification of an added benefit impossible. This is where routine practice data - for example, registry data - come into play: If these are of high quality, it is possible to use them for studies and thus make them usable for the benefit assessment of drugs.
In 2020, the Institute for Quality and Efficiency in Health Care (IQWiG) published criteria for the usability of routine practice data in the benefit assessment of drugs. In a rapid report, the Institute gave manufacturers and registry operators concrete recommendations for the collection and analysis of routine practice data.
"A good two years after the publication of our report, however, we have to state that the manufacturers do not interpret the requirements for the use of routine data in the way we would like them to," says Thomas Kaiser, Head of IQWiG's Drug Assessment Department, citing the manufacturers' dossiers on the drugs amivantamab, nivolumab, dostarlimab and lanadelumab as examples.
Amivantamab: Lack of data on patient selection and risk adjustment
In an early benefit assessment, IQWiG had to assess whether monotherapy with amivantamab provides an added benefit over the appropriate comparator therapy in adult patients with advanced non-small cell lung cancer (NSCLC).
In order to prove an added benefit compared to the comparator therapy, the manufacturer used data from two registries. The problem is that these registries lacked data on very essential patient characteristics (health status, severity of the disease, type of previous treatment). Furthermore, the registries did not contain data on important outcomes. Although the manufacturer was aware of the lack of this information and had partly identified the missing patient characteristics itself as relevant confounders, he did not draw any corresponding consequences from this fact, but wanted to derive an added benefit for amivantamab without taking these aspects into account.
Nivolumab: relevant risk difference in the populations
In an early benefit assessment, IQWiG had to assess whether treatment with nivolumab in combination with ipilimumab provides an added benefit over the appropriate comparator therapy for adult patients with colorectal cancer.
In order to prove an added benefit versus the comparator therapy, the manufacturer used routine data from the US Flatiron Health Database. One problem: The region is an important influencing factor, because in the nivolumab study, the mortality of affected patients was ten percentage points higher in the USA than in Europe. The manufacturer identified this influencing factor, but did not take it into account in its analysis. Moreover, the approval study on nivolumab only included patients who had no side effects from previous therapies and no abnormal laboratory values. This selection was not possible for the comparator because this information was not recorded in the US routine data. There was therefore a relevant risk difference between the two populations considered, but this was not taken into account by the manufacturer.
Dostarlimab: Biomarker not included in the data set on the comparator therapy
In another early benefit assessment, IQWiG had to assess whether treatment with dostarlimab provides an added benefit over the appropriate comparator therapy for patients with endometrial cancer with mismatch repair-deficient (dMMR) or microsatellite instability-high (MSI-H) tumours.
While the pharmaceutical company had only submitted data from patients with dMMR/MSI-H tumours for the approval of dostarlimab, it did not take the MMR-MSI status into account when selecting studies for the comparator. However, the MMR/MSI status would have been a relevant criterion for assessing the similarity of the study populations. Moreover, the biomarker is an important prognostic factor in endometrial cancer.
Lanadelumab: Populations to be compared were too different
Finally, in another early benefit assessment, IQWiG had to assess whether treatment with lanadelumab for the routine prophylaxis of recurrent attacks of hereditary angioedema offers patients an added benefit over the appropriate comparator therapy.
A placebo-controlled study was sufficient for the approval of lanadelumab. For the early benefit assessment, the manufacturer then tried to conduct a retrospective comparison of individual patient data from this study on lanadelumab with data from studies on the comparator therapy. The manufacturer had also conducted the studies on the comparator therapy itself.
However, the summary of the studies was not suitable for the benefit assessment. This is because a look at the studies showed that two completely different populations were to be compared. These differences were so great that they could not be meaningfully compensated even with statistical methods. This means: The comparison presented by the manufacturer basically compared apples and oranges. And yet the manufacturer used these analyses to postulate an added benefit for lanadelumab.
Routine practice data: gap between theory and practice
The four examples listed show a very similar pattern: Crucial information is often missing in the data sets used, and the patient groups to be compared differ, sometimes seriously. An appropriate benefit assessment is hardly possible with such data sets. Although the manufacturers identified some of these points with reference to IQWiG's rapid report, they did not draw the necessary consequences.
“There is still a big gap between the internationally accepted methods we described in our rapid report and the practice we see in dossiers. We urgently need to take the analysis of routine practice data more seriously, otherwise we will miss the important goal of being able to use this valuable data," says IQWiG’s Director Jürgen Windeler, describing the Institute's experience in handling routine practice data data.