Science fiction meets reality: USF researchers develop technique to overcome obstructed views

TAMPA, Fla. (Feb. 20, 2024) – After a recent car crash, John Murray-Bruce wished he could have seen the other car coming. The crash reaffirmed the University of South Florida assistant professor of computer science and engineering’s mission to create a technology that could do just that: See around obstacles and ultimately expand one’s line of vision.

Using a single photograph, Murray-Bruce and his doctoral student, Robinson Czajkowski, created an algorithm that computes highly accurate, full-color three-dimensional reconstructions of areas behind obstacles – a concept that can not only help prevent car crashes, but help law enforcement experts in hostage situations, search-and-rescue and strategic military efforts.

“We’re turning ordinary surfaces into mirrors to reveal regions, objects and rooms that are outside our line of vision,” Murray-Bruce said. “We live in a 3D world, so obtaining a more complete 3D picture of a scenario can be critical in a number of situations and applications.”

As published in Nature Communications, Czajkowski and Murray-Bruce’s research is the first-of-its-kind to successfully reconstruct a hidden scene in 3D using an ordinary digital camera. The algorithm works by using information from the photo of faint shadows cast on nearby surfaces to create a high-quality reconstruction of the scene. While it is more technical for the average person, it could have broad applications.

“These shadows are all around us,” Czajkowski said. “The fact we can’t see them with our naked eye doesn’t mean they’re not there.”

The idea of seeing around obstacles has been a topic of science-fiction movies and books for decades. Murray-Bruce says this research takes significant strides in bringing that concept to life.

Prior to this work, researchers have only used ordinary cameras to create rough 2D reconstructions of small spaces. The most successful demonstrations of 3D imaging of hidden scene all required specialized, expensive equipment.

“Our work achieves a similar result using far less,” Czajkowski said. “You don’t need to spend a million dollars on equipment for this anymore.”

Czajkowski and Murray-Bruce expect it will be 10 to 20 years before the technology is robust enough to be adopted by law enforcement and car manufacturers. Right now, they plan to continue their research to further improve the technology’s speed and accuracy to expand its applications in the future, including self-driving cars to improve their safety and situational awareness.

“In just over a decade since the idea of seeing around corners emerged, there has been remarkable progress and there is accelerating interest and research activity in the area,” Murray-Bruce said. “This increased activity, along with access to better, more sensitive cameras and faster computing power form the basis for my optimism on how soon this technology will become practical for a wide range of scenarios.”

While the algorithm is still in the development phase, it is available for other researchers to test and reproduce in their own space.

###

About the University of South Florida

The University of South Florida, a high-impact research university dedicated to student success and committed to community engagement, generates an annual economic impact of more than $6 billion. With campuses in Tampa, St. Petersburg and Sarasota-Manatee, USF serves approximately 50,000 students who represent nearly 150 different countries.  For four consecutive years, U.S. News & World Report has ranked USF as one of the nation’s top 50 public universities, including USF’s highest ranking ever in 2023 (No. 42). In 2023, USF became the first public university in Florida in nearly 40 years to be invited to join the Association of American Universities, a prestigious group of the leading universities in the United States and Canada. Through hundreds of millions of dollars in research activity each year and as one of top universities in the world for securing new patents, USF is a leader in solving global problems and improving lives. USF is a member of the American Athletic Conference. Learn more at www.usf.edu.