Dairy sector boasts 100 years of successful herd data collection

URBANA, Ill. – The U.S. dairy industry operates a comprehensive data collection program that records herd production information from farmers nationwide. The program provides crucial input for cattle breeding and genetics, and its cooperative structure ensures benefits for producers and scientists alike. A new study from the University of Illinois explores the program’s century-old history, highlighting its relevance for modern agriculture and digital data collection.

“The National Cooperative Dairy Herd Improvement Program (NCDHIP) is an interesting case study because it illustrates how to translate the benefits of data collection for all dairy producers. Its model can serve as inspiration for other agricultural sectors,” says Jared Hutchins, assistant professor in the Department of Agricultural and Consumer Economics, part of the College of Agricultural, Consumer and Environmental Sciences at the U of I. Hutchins is lead author of the study, published in Applied Economic Perspectives and Policy.

The data collection program originally started in response to a new innovation in the dairy market. The Babcock test, introduced in 1890, provided a way to measure the butterfat composition of milk.

“Previously it was commonplace for dairy farmers to water down their milk to get paid more. But farmers were now paid on butterfat instead of milk weight. The Babcock test created a paradigm shift in dairy, giving producers an incentive to learn and fund research about this new metric,” Hutchins states.

With collection of herd production data, it became possible to evaluate which bulls sired the highest-producing cows – information that could only be obtained by aggregate results across farms. USDA scientists began using the data to publish bull evaluation lists, which dairy farmers could consult for breeding decisions. 

The introduction of artificial insemination in the 1930s, and later the ability to freeze semen and ship it over large distances, dramatically increased the number of offspring each bull could produce. These new technologies greatly expanded the quantity of data available for breeding and genetics research, making it even more beneficial to producers.

The NCDHIP facilitates nationwide data collection through local Dairy Herd Improvement Associations (DHIAs). The first DHIA was established in 1905 by Helmer Rabild, a Danish immigrant who worked for the Michigan Department of Agriculture. He based the cooperative structure on milk testing cooperatives that existed in his native country of Denmark. Rabild was soon hired by the USDA to start DHIAs across the country, and the number of participating farmers rapidly increased.    

Even with the proliferation of large farms in recent years, the program continues to be popular with producers. Hutchins says there are three key aspects which make the NCDHIP so successful.

“First, there are private benefits to producers, which makes them want to be a part of this system and contribute their data to the platform. There is power in scaling up, getting data from many different farmers, and producing value for the whole sector. But farmers also gain immediate benefits, such as a benchmarking report,” he notes.

Another crucial aspect is data interoperability, which means the program employs universal data standards that allow data across platforms to work with each other.

Finally, the cooperative data governance model gives producers control over how their data are used and processed. The NCDHIP is a collaborative agreement between the cooperatives, farmers, and the USDA.

“We often see a misalignment of the interests of data producers and the people that hold and use the data. The NCDHIP has solved this problem in a very clever way with the cooperative structure,” Hutchins states.

Other agricultural sectors may have one or two of these aspects, but Hutchins says dairy is unique in having all three, which is crucial for the system to work so well.

“We are currently in the midst of a digital revolution in agriculture, with so many novel ways to measure data. The question is what we do with the data, how it is controlled and managed, and how the benefits will be distributed. We wanted to show the dairy sector was able to leverage their data revolution in a way that benefited all dairy farmers,” Hutchins concludes.

The paper, “100 years of data sovereignty: Cooperative data governance and innovation in US dairy,” is published in Applied Economic Perspectives and Policy. Authors are Jared Hutchins and Brent Hueth. [https://doi.org/10.1002/aepp.13339]. This research was supported in part by the U.S. Department of Agriculture, Economic Research Service.