Study examines accuracy of arrest data in FBI's NIBRS crime database

CHAMPAIGN, Ill. — As more police agencies transition to the FBI’s National Incident-Based Reporting System for crime reporting, a study of one state’s data found that a design flaw in NIBRS, the timings of arrests and human factors can lead to discrepancies.

When the authors of the current study compared the statuses of a statewide sample of 480 cases in NIBRS with data they collected directly from the Massachusetts jurisdictions where the crimes occurred, they found that about 16% of the cases incorrectly indicated in NIBRS whether arrests were made or summonses were issued.

“Overall, in 84% of the cases, the NIBRS data matched that of the law enforcement agencies,” said co-author Theodore P. Cross, a professor of social work and senior research specialist at the University of Illinois Urbana-Champaign. “But there were some factors that led to errors – one being delays in arrests and another being the use of a single data field to enter both arrests and summonses.”

Cross conducted the research with Alex Wagner, the director of research analytics at Emory University, and Daniel Bibel, of Criminal Justice Consultation and Research.

An NIBRS expert who managed the Massachusetts State Police Crime Reporting Unit for 27 years, Bibel helped the team obtain a random sample of cases for the study, which was funded by the National Institute of Justice and published in the journal Crime and Delinquency.

Bibel died in 2020.

The data sample included two of Massachusetts’ most frequent violent crimes – aggravated assault and sexual assault – along with simple assault and intimidation cases.

According to the study, suspects were arrested in more than 33% of the cases, and a summons was issued in slightly more than 18% of them.

However, slightly more than 13% of the cases in the sample were false negatives that incorrectly recorded in NIBRS that they had not been resolved by an arrest or summons. The researchers’ analysis showed that the majority of these false negatives occurred in cases where a summons was issued rather than an arrest made.

Nearly 41% of the sexual assault cases in the sample were false negatives, as were 32% of the intimidation cases, more than 19% of the aggravated assault cases and nearly 16% of the simple assaults, the researchers found.

False negatives were significantly associated with delayed arrests, Cross said. “If an arrest was made a day or more after the crime occurred, the accuracy of the NIBRS data was significantly lower.”

For example, in the sexual assault cases, 40% of the arrests were made a day or more after the crimes happened, according to the study.

Although police departments can update cases when arrests or summonses occur after the initial data entry, staff members may fail to make these updates because they have competing responsibilities or they are unaware that arrests have been made, Cross said. Thus, the department’s arrests may be underrepresented in NIBRS.

Some of the other false negatives the team found resulted from a design flaw in NIBRS. While the user manual instructs staff to enter both summonses and arrests in the “arrests” data field, some users were unaware of that. Thus, more than half of the summonses issued for crimes in the sample were not recorded in NIBRS.

Cross said NIBRS’ designers created a single data field for reporting arrests and summonses to streamline an otherwise rigorous data entry task. However, this can lead to underreporting of summonses because users typically differentiate arrests – i.e., taking a suspect into physical custody – from summoning someone to appear in court.

The team also found a small proportion of cases that were false positives – probable data entry errors, they hypothesized – that incorrectly indicated in NIBRS that suspects were arrested or summoned to court.

“This is a human process in which a sophisticated data collection task is put on the shoulders of law enforcement professionals who have a million different responsibilities,” Cross said. “There were factors that detracted from the data’s accuracy that can be identified and addressed through support, training and resources.”

NIBRS is considered one of the most important resources for crime statistics in the U.S., and data from it have been used in a number of groundbreaking studies. However, the team cautioned that crime trends based on NIBRS data may not be representative of the U.S. as a whole since the police agencies currently utilizing it cover just 29% of the U.S. population.

Some of the most populous U.S. cities, including Boston and Chicago, currently do not use NIBRS because they have their own data systems. In Illinois, just 1% of the state’s law enforcement agencies report statistics using NIBRS, Wagner said.

About 85% of the Massachusetts law enforcement agencies that voluntarily report crime statistics utilize NIBRS, while the remaining agencies use a different system called Uniform Crime Reports, according to the study.

Although the FBI supported both systems for many years, the bureau discontinued accepting data through Uniform Crime Reports as of Jan. 1, 2021 and began encouraging law enforcement agencies to switch to NIBRS.

NIBRS collects greater detail about crimes, and more widespread use by police agencies could be a boon for crime researchers, Cross said. 

“We think that NIBRS really could improve the quality of crime data in the U.S., especially now that the FBI is committing wholeheartedly to it and recruiting more and more police agencies to participate,” Cross said. “Attention to the human processes behind the data is going to pay off in understanding the information we’re collecting and improving the collection process.”