Researchers foresee their techniques helping immigration officers screen people more quickly and confidently at border crossings and ports of entry. The techniques could also bolster security for buildings such as embassies and help law enforcement personnel do more thorough interrogations.
"Looking for what we call 'microexpressions' and 'microgestures' associated with deception would be a major leap over today's polygraph technology, which is time-consuming and requires professional administration," said Dimitris Metaxas, computer science professor and director of the university's Center for Computational Biomedicine Imaging and Modeling (CBIM). "Even under the most controlled conditions, lie-detector tests based on body physiology are roughly 50 percent reliable.
"On the other hand, we believe gestures and expressions are a lot harder for someone to mask, and don't vary significantly among races and cultures," he said. "Microexpressions may easily escape notice by human observers, but they can be reliably picked up on camera and quickly detected by computer, giving interrogators new tools to do their jobs confidently."
Metaxas is building on his earlier research into computer modeling of facial expressions and the dynamics of body organs – a beating heart or blood flowing through vessels. This work employs image-based "deformable models" that enable computers to realistically simulate subtle yet detailed muscular movements, such as those that make up facial expression.
Several challenges remain, however. One is to accurately correlate facial expressions to deceptive intent and verify how consistently these occur across cultures. Another is to incorporate new cues, such as gestures and body postures, into the computer models. To accomplish this, the Rutgers team will draw upon the expertise of communications experts and psychologists.
The researchers will also study how accurately they can capture movements under "real-world" conditions of varied lighting, background motion, and potential distractions such as people wearing glasses and men with facial hair. While Metaxas will do early studies with multiple cameras, his goal is to make the recognition and analysis procedures robust enough to work with a single camera.
In addition to using cameras to capture images, the team will employ three-dimensional sensor technology to capture the range of body movements. Rutgers is collaborating with Lockheed Martin on 3D sensor development and integration.
The grant runs for three years, with opportunity for subsequent renewals. Collaborating with Metaxas are computer science professors Ahmed Elgammal and Vladimir Pavlovic, Lockheed Martin computer scientist Peter Rosenfeld, and postdoctoral research fellows Shan Lu and Gabriel Tsechpenakis.