image: A research team led by Carnegie Mellon University scientists has demonstrated techniques for using scattered light to see around corners, enabling them to reconstruct images in detail. On the left is a reconstructed image of a US quarter that was scanned using non-line-of-sight methods, compared with the quarter on the right that was scanned using standard methods. view more
Credit: Carnegie Mellon University
PITTSBURGH--Computer vision researchers have demonstrated they can use special light sources and sensors to see around corners or through gauzy filters, enabling them to reconstruct the shapes of unseen objects.
The researchers from Carnegie Mellon University, the University of Toronto and University College London said this technique enables them to reconstruct images in great detail, including the relief of George Washington's profile on a U.S. quarter.
Ioannis Gkioulekas, an assistant professor in Carnegie Mellon's Robotics Institute, said this is the first time researchers have been able to compute millimeter- and micrometer-scale shapes of curved objects, providing an important new component to a larger suite of non-line-of-sight (NLOS) imaging techniques now being developed by computer vision researchers.
"It is exciting to see the quality of reconstructions of hidden objects get closer to the scans we're used to seeing for objects that are in the line of sight," said Srinivasa Narasimhan, a professor in the Robotics Institute. "Thus far, we can achieve this level of detail for only relatively small areas, but this capability will complement other NLOS techniques."
This work was supported by the Defense Advanced Research Project Agency's REVEAL program, which is developing NLOS capabilities. The research will be presented today at the 2019 Conference on Computer Vision and Pattern Recognition (CVPR2019) in Long Beach, California, where it has received a Best Paper award.
"This paper makes significant advances in non-line-of-sight reconstruction -- in essence, the ability to see around corners," the award citation says. "It is both a beautiful paper theoretically as well as inspiring. It continues to push the boundaries of what is possible in computer vision."
Most of what people see -- and what cameras detect -- comes from light that reflects off an object and bounces directly to the eye or the lens. But light also reflects off the objects in other directions, bouncing off walls and objects. A faint bit of this scattered light ultimately might reach the eye or the lens, but is washed out by more direct, powerful light sources. NLOS techniques try to extract information from scattered light -- naturally occurring or otherwise -- and produce images of scenes, objects or parts of objects not otherwise visible.
"Other NLOS researchers have already demonstrated NLOS imaging systems that can understand room-size scenes, or even extract information using only naturally occurring light," Gkioulekas said. "We're doing something that's complementary to those approaches -- enabling NLOS systems to capture fine detail over a small area."
In this case, the researchers used an ultrafast laser to bounce light off a wall to illuminate a hidden object. By knowing when the laser fired pulses of light, the researchers could calculate the time the light took to reflect off the object, bounce off the wall on its return trip and reach a sensor.
"This time-of-flight technique is similar to that of the lidars often used by self-driving cars to build a 3D map of the car's surroundings," said Shumian Xin, a Ph.D. student in robotics.
Previous attempts to use these time-of-flight calculations to reconstruct an image of the object have depended on the brightness of the reflections off it. But in this study, Gkioulekas said the researchers developed a new method based purely on the geometry of the object, which in turn enabled them to create an algorithm for measuring its curvature.
The researchers used an imaging system that is effectively a lidar capable of sensing single particles of light to test the technique on objects such as a plastic jug, a glass bowl, a plastic bowl and a ball bearing. They also combined this technique with an imaging method called optical coherence tomography to reconstruct the images of U.S. quarters.
In addition to seeing around corners, the technique proved effective in seeing through diffusing filters, such as thick paper.
The technique thus far has been demonstrated only at short distances -- a meter at most. But the researchers speculate that their technique, based on geometric measurements of objects, might be combined with other, complementary approaches to improve NLOS imaging. It might also be employed in other applications, such as seismic imaging and acoustic and ultrasound imaging.
In addition to Narasimhan, Gkioulekas and Xin, the research team included Aswin Sankaranarayanan, assistant professor in CMU's Department of Electrical and Computer Engineering; Sotiris Nousias, a Ph.D student in medical physics and bioengineering at University College London; and Kiriakos N. Kutulakos, a professor of computer science at the University of Toronto.
###
The researchers are part of a larger collaborative team, which includes researchers from Stanford University, the University of Wisconsin Madison, the University of Zaragosa, Politecnico di Milano and the French-German Research Institute of Saint-Louis, that is developing a suite of complementary techniques for NLOS imaging.
In addition to DARPA, the National Science Foundation, the Office of Naval Research and the Natural Sciences and Engineering Research Council of Canada supported this research.