Tablet-based AI app measures multiple behavioral indicators to screen for autism

DURHAM, N.C. – Researchers at Duke University have demonstrated an app driven by AI that can run on a tablet to accurately screen for autism in children by measuring and weighing a variety of distinct behavioral indicators.

Called SenseToKnow, the app delivers scores that evaluate the quality of the data analyzed, the confidence of its results and the probability that the child tested is on the autism spectrum. The results are fully interpretable, meaning that they spell out exactly which of the behavioral indicators led to its conclusions and why.

This ability gives health care providers detailed information on what to look for and consider in children referred for full assessments and intervention. SenseToKnow’s ease of use and lack of hardware limitations, combined with its demonstrated accuracy across sex, ethnicity and race, could help eliminate known disparities in early autism diagnosis and intervention by allowing autism screening to take place in any setting, even in the child’s own home.

The results appear online October 2 in the journal Nature Medicine.

“Autism is characterized by many different behaviors, and not all children on the spectrum display all of them equally, or at all,” said Geraldine Dawson, director of the Duke Center for Autism and Brain Development, who is a co-senior author on the study. “This screening tool captures a wide range of behaviors that more accurately reflect the complexity and variability found in autism.”

Recent research has shown promising results from tracking children’s eye movements in response to specially designed movies that can help diagnose autism in a clinical setting. SenseToKnow, the researchers say, detects a wider range of behaviors such as facial expression, gaze patterns, head movements and blink rate. It also incorporates an on-screen bubble-popping game to assess motor movement and skills, as delays in motor skills are one of the earliest signs of autism.

The app uses almost every sensor in the tablet’s arsenal to measure and characterize the child’s response without the need for any sort of calibration or special equipment. It then uses AI to analyze the child’s responses to predict how likely it is that the child will be diagnosed with autism.

“The AI we’ve built compares each child’s biomarkers to how indicative they are of autism at a population level,” said Sam Perochon, a PhD student working in the laboratory of Guillermo Sapiro, the James B. Duke Distinguished Professor of Electrical and Computer Engineering and co-senior author of the study. “This allows the tool to capture behaviors other screening tests might miss and also report on which biomarkers were of the most interest and most predictive for that particular child.”

The AI tool is able to provide scores for both the quality of data that the app was able to capture as well as its level of confidence in its own analysis—both of which, the researchers believe, are a novel feature.

“This is an important aspect for a health care provider to know, just like they would need to know if a blood test did not have a big enough sample to produce reliable results,” said Matias Di Martino, assistant research professor of electrical and computer engineering at Duke, who co-led the analysis of the study with Sapiro and Perochon.

In the study, SenseToKnow was administered to 475 children during a pediatric well-child visit, 49 of whom were subsequently diagnosed with autism and 98 with developmental delay without autism. The app showed 87.8% sensitivity for detecting autism, meaning it correctly identified most children with the condition. Its specificity—the percentage of children without autism who screened negative—was 80.8%.

Overall, participants who screened positive for autism using the app had a 40.6% probability of subsequently being diagnosed with the condition. In comparison, only about 15% of children who screen positive using the standard parent questionnaire are later diagnosed with autism. Combining the app with the standard questionnaire boosted the probability of a positive screen resulting in later diagnosis to 63.4% -- meaning fewer children are falling through the cracks.

The American Academy of Pediatrics recommends that every toddler be screened for autism at 18 and 24 months. However, concerns have been raised that current screening methods that rely solely on parent reporting are missing children. Girls and children of color, in particular, are often missed.

SenseToKnow’s ability to detect autism was similar across children of different sexes, races and ethnicities.  While the researchers do not envision the digital screening tool replacing parent reporting, they believe it is important to augment the subjective questionnaire with objective tools to help close the gap.

“Just like when any patient goes to their doctor, the doctor listens to them describe what they are experiencing, but they also use thermometers and other objective tests to provide additional information to guide next steps and referrals for further evaluation,” Dawson said. “Such objective tests have been missing for autism.”

The researchers are currently conducting a study in which parents deliver the app at home. They hope that the app will also be useful for measuring a child’s progress within an early intervention program as well as to studying the effectiveness of such programs.

“There is a wide range of expertise amongst health care providers in knowing and being able to recognize all the potential signs of a child being on the autism spectrum,” Dawson said. “This app could help clinicians focus on the areas in which the child needs help, as well as identify areas of strength.”

This research was supported by the National Institutes of Health (NICHD Autism Centers of Excellence Research Program P50HD093074, R01MH121329, R01MH120093) and the Simons Foundation. The team is also partially supported by the National Science Foundation and the Department of Defense.

“Early detection of autism using digital behavioral phenotyping.” Sam Perochon, J. Matias Di Martino, Kimberly L. H. Carpenter, Scott Compton, Naomi Davis, Brian Eichner, Steven Espinosa, Lauren Franz, Pradeep Raj Krishnappa Babu, Guillermo Sapiro & Geraldine Dawson. Nature Medicine, Oct. 2, 2023. DOI: 10.1038/s41591-023-02574-3

# # #