News Release

Why are insecurities and risks often erroneously calculated?

Peer-Reviewed Publication

Max-Planck-Gesellschaft

Research at the Max Planck Institute for Human Development shows that, to avoid leading to false conclusions, statistical information needs to be communicated better and in a more comprehensible fashion.

Experts as well as laypeople often have difficulties interpreting statistical information correctly. However, they have much less trouble reaching the correct conclusions when the same information is presented to them in the form of "natural frequencies." That is the result of recent studies involving medical students and prospective lawyers conducted by researchers at the Max Planck Institute for Human Development in Berlin as reported in the American journal "Science" in its December 22, 2000 edition.

A research group from the Center for Adaptive Behavior and Cognition at the Max Planck Institute for Human Development conducted a series of studies examining how experts and laypeople make decisions on the basis of statistical information. The two most recent studies, done by the Berlin researchers in the group including Dr. Ulrich Hoffrage, will be published on December 22, 2000 in the Policy Forum section of the journal "Science."

Underlining the relevance of this topic, Dr. Hoffrage pointed out that H. G. Wells had, in one of his political writings, said if a society wants to produce responsible citizens, they have to be taught to read, to write, and to think statistically. Hoffrage added, "In our society, we have succeeded in teaching almost everyone to read and to write, but have made virtually no progress in educating people in how to deal with statistics, that is, in dealing effectively with risk".

The Max Planck researchers were not out to demonstrate that medical tests are ridden with error - that is already well known. The point was also not to prove that many experts are unable draw reasonable conclusions from statistical information - this, too, is commonly accepted as a result of relevant research. The goal, rather, was to help experts, as well as laypeople, to correctly interpret results of diagnostic tests.

This kind of statistical information is usually presented in the form of probabilities and percentages in most medical and statistics textbooks. A man receiving a positive Haemoccult®-Test , for example, would find this sort of information there: The probability that a 50-year-old man without symptoms has colon cancer is 0.3%. If he has cancer, the probability of him having a positive test is 50%. If he does not have cancer, the probability of him receiving a positive test result anyway is 3%. Hardly any of the doctors asked by the Berlin researchers could correctly estimate the probability that the patient has colon cancer based on a positive Haemoccult®-Test result. To enhance this ability, Dr. Hoffrage and his colleagues suggest communicating the same information in natural frequencies: 30 out of 10,000 men have colon cancer. Of these 30, 15 will receive a positive test result. Of the remaining 9,970 men, 300 will receive a positive test result. This way, it is easy to see that only 15 of 315 men with a positive test result actually have colon cancer (ca. 5%).

The first study showed that a positive test result can be better evaluated by advanced medical students and prospective doctors when the relevant statistical information was presented in the form of natural frequencies. Without any further help, an increase in the number of correct answers from 18% to 57% was achieved. This result confirms one the researchers had already seen in an earlier study involving 48 doctors in which the number of correct answers rose from 16% to 46% (Hoffrage & Gigerenzer, 1998, Academic Medicine, 73, 538-540). Representing statistical information in natural frequencies is not only easier, it is also very effective and well suited to the communication and understanding of risk.



Figure 1: Percent of correct answers on four realistic diagnostic problems using medical tests

In the second study, 27 jurists (junior professors and doctoral candidates) as well as 127 advanced law students were asked what conclusion they would draw from a match in the DNA profile of a suspect and evidence left at the scene of a crime. In this test, as well, the statistical information was presented, on the one hand, as probabilities and, on the other, in natural frequencies. When the information was communicated in probabilities, only 13% of the junior professors and doctoral candidates found the correct answer, and fewer than 1% of the students did. When the information was translated into natural frequencies, 68% and 44% percent of the participants, respectively, came up with the correct answer. The representation of the information also had an impact on the verdicts delivered: With the information in probabilities, 45% of the lawyers and 55% of the students found the suspect guilty. If the test participants received the same information in natural frequencies, only 32% and 33%, respectively, of the test group were convinced that the suspect was guilty.



Figure 2: Percent of correct answers (left) and verdicts (right) in two criminal cases using DNA information as evidence

The research results have implications for clients from low-risk groups when they are counseled about AIDS. In a study published in 1998 (Gigerenzer, Hoffrage & Ebert, 1998, AIDS Care, 10, 197-211), researchers reported the difficulties AIDS counselors in German health service centers have in dealing correctly with statistical information. Also here, the difficulties were overcome with the use of natural frequencies.

Furthermore, in their Science article, the Berlin researchers discuss the relevance of their methods to risk communication in the early diagnosis of breast cancer. Usually, mammography screening is depicted as an exercise in relative risk reduction (about 25% of breast cancer fatalities could be prevented by screening). However, from the perspective of any one woman, the relevant figure is the absolute risk reduction - about 0.1% of all women participating in screening profit from the reduction of breast cancer fatalities .

The new research results can be important to education at universities and in schools. The scientists have developed an interactive, computer supported tutorial (Sedlmeier & Gigerenzer, Journal of Experimental Psychology:General, in press) as well as one for traditional classroom situations (Kurzenhäuser & Hoffrage, submitted for publication). The evaluation of these tutorials showed that it is more effective to teach students to translate statistical information into natural frequencies than it is to just plug that information into a mathematical formula.

In a final word about the relevance of the work described above, Dr. Hoffrage added, "In a highly developed technological society, in which new risks continually arise leading to the development of new diagnostic techniques (for example, DNA fingerprinting), it is important to offer people psychological methods that ease the understanding of risks and test results."

###

Original work:
Ulrich Hoffrage, Samuel Lindsey, Ralph Hertwig, Gerd Gigerenzer: Communicating Statistical Information, Science, Vol. 290, 2261-2262, December 22,2000.

For further information, please contact:
Dr. Ulrich Hoffrage
Max Planck Institute for Human Development
Lentzeallee 94
D-14195 Berlin
Phone: +49 / 0 30 / 8 24 06 - 2 73
Fax: +49 / 0 30 / 8 24 06 - 3 94
e-mail: hoffrage@mpib-berlin.mpg.de


Disclaimer: AAAS and EurekAlert! are not responsible for the accuracy of news releases posted to EurekAlert! by contributing institutions or for the use of any information through the EurekAlert system.