The level of agreement among different nutrition labels worldwide is very low

In response to growing concern about people's eating habits, governments around the world have implemented different nutrition labelling schemes to help consumers make healthy choices when shopping. The UPV/EHU's Nursing and Health Promotion research group decided to go one step further. In view of the fact that we are eating out more and more, it raised the possibility that restaurants should also include information on their menus as to whether or not their dishes are healthy: “In the same way that they mark whether the recipes contain allergens or are suitable for vegans, we think it would be good to indicate whether they are healthy. In this context, we wanted to find out whether any of the nutrition labelling systems that are already used globally to assess processed products could be valid for assessing cooked dishes. So we compared seven different methods and found so little consistency between them that we considered that none of them were suitable for our purpose,” explained Leyre Gravina, the lead researcher of the study.

The research is novel given that the reproducibility and concordance of the labels globally in relation to cooked food has not been compared until now. In order to make a comparison, 178 Mediterranean dishes served at the Leioa School of Catering were selected and assessed using the following labelling systems: Nutri-Score, the traffic lights in the UK and Ecuador, the Mazocco method, the HSR system in Australia and the warning labels in Uruguay and Chile. The results revealed disagreement among all the tools. In some cases the differences are more pronounced, but in general, the level of agreement is low. For example, when it comes to the number of unhealthy dishes detected, the systems that disagree the most are Nutri-Score and that of Ecuador. The former is the method that rates most recipes as unhealthy (38%), while the latter is the most liberal, as it does not rate any as unhealthy.

The UPV/EHU team acknowledges that they expected differences because the systems do not use the same methodologies and take different variables into consideration, but they were surprised by the level of inconsistency found: “The Uruguayan or Ecuadorian tools only take into account the content of unhealthy components such excessive sugar or saturated fat. Others also take healthy nutrients such as pulses and fibre into consideration. So, we sensed that the analyses might give different results, but it turns out that the disagreement is significant even between systems that assess similar components,” said Gravina.

For example, Nutri-Score analyses 13 nutrients and 9 of them are all covered by Australia's HSR system: calories, sugars, saturated fatty acids, fibre, fruit, vegetables, nuts and protein. However, even though they agree on many of the components to be analysed, the research revealed that the agreement between the two assessment tools is only moderate. This is illustrated by the results of the dessert analysis: Nutri-Score rates 6% as healthy, the Australian system 25%.

The need to design a new tool

According to the authors of the study, the low level of agreement among the different nutrition labels indicates that more complex strategies for assessing the food quality of ready meals need to be developed. In the absence of consensus when specifying what is healthy, Gravina proposes that a method that is adapted to the geographical context should be created: “Right now, it does not seem that a universal system can be effective. It would be a good idea to start with labelling tools that take local foods and the culinary and socio-cultural realities of each country or region into account.”

Additional information

Leyre Gravina is a member of the Nursing Department I of Leioa and its main lines of research focus on the effects of the environment on health, analysing areas such as nutrition and physical activity. She directs the research group in Nursing and Health Promotion and teaches on the Degree Course in Nursing and on the UPV/EHU’s in-house degree course in Comprehensive Care of People with Complex, Chronic and Difficult-to-Heal Wounds. She is also supervising six Doctoral theses in the PhD in Public Health and in the PhD in Medicine and Surgery.

In response to growing concern about people's eating habits, governments around the world have implemented different nutrition labelling schemes to help consumers make healthy choices when shopping. The UPV/EHU's Nursing and Health Promotion research group decided to go one step further. In view of the fact that we are eating out more and more, it raised the possibility that restaurants should also include information on their menus as to whether or not their dishes are healthy: “In the same way that they mark whether the recipes contain allergens or are suitable for vegans, we think it would be good to indicate whether they are healthy. In this context, we wanted to find out whether any of the nutrition labelling systems that are already used globally to assess processed products could be valid for assessing cooked dishes. So we compared seven different methods and found so little consistency between them that we considered that none of them were suitable for our purpose,” explained Leyre Gravina, the lead researcher of the study.

The research is novel given that the reproducibility and concordance of the labels globally in relation to cooked food has not been compared until now. In order to make a comparison, 178 Mediterranean dishes served at the Leioa School of Catering were selected and assessed using the following labelling systems: Nutri-Score, the traffic lights in the UK and Ecuador, the Mazocco method, the HSR system in Australia and the warning labels in Uruguay and Chile. The results revealed disagreement among all the tools. In some cases the differences are more pronounced, but in general, the level of agreement is low. For example, when it comes to the number of unhealthy dishes detected, the systems that disagree the most are Nutri-Score and that of Ecuador. The former is the method that rates most recipes as unhealthy (38%), while the latter is the most liberal, as it does not rate any as unhealthy.

The UPV/EHU team acknowledges that they expected differences because the systems do not use the same methodologies and take different variables into consideration, but they were surprised by the level of inconsistency found: “The Uruguayan or Ecuadorian tools only take into account the content of unhealthy components such excessive sugar or saturated fat. Others also take healthy nutrients such as pulses and fibre into consideration. So, we sensed that the analyses might give different results, but it turns out that the disagreement is significant even between systems that assess similar components,” said Gravina.

For example, Nutri-Score analyses 13 nutrients and 9 of them are all covered by Australia's HSR system: calories, sugars, saturated fatty acids, fibre, fruit, vegetables, nuts and protein. However, even though they agree on many of the components to be analysed, the research revealed that the agreement between the two assessment tools is only moderate. This is illustrated by the results of the dessert analysis: Nutri-Score rates 6% as healthy, the Australian system 25%.

The need to design a new tool

According to the authors of the study, the low level of agreement among the different nutrition labels indicates that more complex strategies for assessing the food quality of ready meals need to be developed. In the absence of consensus when specifying what is healthy, Gravina proposes that a method that is adapted to the geographical context should be created: “Right now, it does not seem that a universal system can be effective. It would be a good idea to start with labelling tools that take local foods and the culinary and socio-cultural realities of each country or region into account.”

Additional information

Leyre Gravina is a member of the Nursing Department I of Leioa and its main lines of research focus on the effects of the environment on health, analysing areas such as nutrition and physical activity. She directs the research group in Nursing and Health Promotion and teaches on the Degree Course in Nursing and on the UPV/EHU’s in-house degree course in Comprehensive Care of People with Complex, Chronic and Difficult-to-Heal Wounds. She is also supervising six Doctoral theses in the PhD in Public Health and in the PhD in Medicine and Surgery.