A Columbia University computer scientist has developed a videocamera that can see in all directions at once.
Placed atop a concert stage or at midfield of a sports event, such a camera could provide a 360-degree view -- an entire sphere -- to television viewers. They could use a set-top box and joystick to bring any frame of that view to their screens, and see not hemispheric perspective, but normal, undistorted, linear perspective. They can even create onscreen windows to see several views at once.
The technology is already in prototype at Columbia's School of Engineering and Applied Science, where Shree K. Nayar, professor of computer science, has spent six years researching his physics-based approach to artificial vision. His laboratory has developed four prototypes, with different configurations for surveillance, teleconferencing, entertainment and robotic vision.
Professor Nayar calls his innovation the Omnicam, a videocamera that uses a small parabolic mirror to obtain hemispheric views. A miniature videocamera mounted in a frame is aimed directly at the apex of the parabolic mirror, a small inverted cup of polished metal enclosed within a transparent hemisphere. Columbia graduate student Venkat Peri has developed software that allows multiple Omnicam images to be displayed on a computer screen in linear perspective at any magnification. Columbia has filed a U.S. patent application covering the Omnicam videocamera and the Omnivideo software.
"Computer vision research is attempting to construct systems that can perceive our environment using man-made sensors, such as cameras, in a way that is analogous to how we use our eyes," Professor Nayar said. "The Omnicam can view more of a scene than the human eye can, which makes it extremely valuable for a number of applications."
Two Omnicams mounted back to back can produce views of 360 degrees, a complete sphere, for surveillance or security operations. In teleconferences, an Omnicam can show simultaneously every participant seated around a table, in either hemispheric or linear perspective. It will allow a mobile robot to determine its location and direction of travel from local features. Because the camera views itself at the parabola's apex, there is one small blind spot in each hemisphere.
The Omnicam's parabolic optics ensure that it has a single effective center of projection, a single point through which all rays from a scene must pass on their way to the camera's lens. That design mimics a camera that takes in only linear perspective, and allows the Omnicam's computer software to generate linear perspective images that are free of distortion. "Columbia's Omnicam videocamera has an optical configuration that ensures a single center of projection and produces the desired full field of view," Professor Nayar said.
Other vision researchers have tried to create omnidirectional vision systems, using fisheye lenses or planar, spherical, conical or pyramidal mirrors. Most of these do not yield the single viewpoint necessary to construct linear perspective images, or, if they do, use moving parts and other complex elements to produce them, Professor Nayar said.
Professor Nayar's lab has developed several Omnicam prototypes and is working to improve the system's resolution. For further information about the new cameras, or to see an online demonstration, visit the laboratory's web site: http://www.cs.columbia.edu/CAVE.