Nothing invokes a headache like watching a poorly shot video. As the images bob up and down, in and out, and side to side, the nausea of motion sickness strikes. Until now, the only cure was to close your eyes. Dr. David Hathaway and Paul Meyer, two scientists at NASA's Marshall Space Flight Center, have a better prescription.
Hathaway and Meyer have developed the Video Image Stabilization and Registration (VISAR) technique. VISAR is a computer algorithm that corrects for zoom, tilt, and jitter. Computer and video images are made up of tiny squares of color called pixels. By registering on an object in the image, the pixels from several video frames can be lined up together. The result is a steadier video.
"With VISAR," says Hathaway, "A sequence of video images won't move around, zoom in and out, or rotate."
A Shaky Proposition
The process to develop VISAR began over two years ago, following a request from the Southeast Bomb Task Force of the FBI. The video images - showing the bombing of the 1996 Olympic Games in Atlanta - were of particularly poor quality. The FBI task force asked if anyone at NASA's Marshall Space Flight Center could help improve the video's clarity.
Both Hathaway and Meyer had reason to believe they might be able to improve the crime scene video images. Meyer, a meteorologist and computer scientist working in Global Hydrology, processes weather satellite images. As a solar physicist, Hathaway uses video stabilization techniques to enhance pictures of the Sun.
"Telescopes are always shaky," states Hathaway. This jitter affects measurements of the Sun's position or features on the solar surface.
A Bright Idea
The two scientists began working on the FBI bombing video, a dark nighttime shot filmed with a shaky handheld camcorder. Luckily, the scientists had over 400 frames (more than 13 seconds) to work with. After a long process of trial and error, they were eventually able to stabilize, sharpen, and brighten the image.
Stabilizing the image is necessary to create a clear picture from a video sequence. Most image sharpening techniques, says Hathaway, are "crude and dirty." Current techniques usually just sharpen the edges of an image, without taking into account how the blurring occurred. Zooming in, for instance, makes an image larger and more spread out. And these techniques only work on one frame at a time, so the effects of video noise, or snow, cannot be accounted for. Such sharpening techniques can bring out the noise, distorting the image even further. Because VISAR allows you to combine several video images together, noise can be averaged out among the frames. The more images you can combine, the greater the corrective power of VISAR.
Hathaway and Meyer's efforts in correcting the FBI video surpassed existing image-correction technology. The video correction methods currently in use could not compensate for the effects of zoom or tilt, as Hathaway and Meyer did. By steadying and reducing the noise in the images, they brought out a wealth of information, revealing new, previously obscured details.
Hathaway and Meyer decided to take everything they'd learned about image processing and create the new video stabilization product VISAR. They already knew their process was better than anything currently in use. They were further encouraged by scientific colleagues at the Los Alamos National Laboratory in New Mexico, who claimed that VISAR was better than anything they'd ever seen. VISAR is currently covered under a provisional patent, and will soon be available for licensing
A Vision for the Future
Hathaway believes VISAR could be a boon to law enforcement. Police often use video to identify suspects, to investigate a crime scene, or to spot identifying characteristics. VISAR could be used to steady images of car chases shot from inside a moving police car, enabling police to focus on a license plate number, or even on an image of the driver's face reflected in the rear-view mirror. VISAR could also aid in identifying faces in a crowd. This would be especially useful at arson sites, for instance, because arsonists typically love to return to the scene to watch their handiwork. VISAR could be used to scrutinize onlookers at several different arson sites to identify repeat visitors.
VISAR could have applications in medical imaging as well. Ultrasounds, for instance, are infamous for their grainy, blurred quality. Puzzled moms and dads would no longer have to peer anxiously at obscure ultrasound images. More importantly, doctors could make better medical diagnoses based on steadier, clearer images. And as laproscopic surgery becomes more common, VISAR could steady images that would help not only during the surgery, but also in the training videos shown to students afterward.
Paul Meyer would like to see VISAR applied to meteorology, to track cloud formations and storms. VISAR could be used to determine if the images of a hurricane's eye have moved, rotated, or changed dimensions. VISAR would be especially useful for tornado videos, which are often shot on home video cameras by people unlucky enough to be caught in the path of the storm. If you could steady the image of the tornado, you could track objects whirling on the outside of the tornado. Tracking a whirling object, be it a tractor, truck or cow, would help determine the tornado's wind speed.
The VISAR of Aaahs...
Hathaway believes the biggest potential market could be the home consumer, who would use VISAR to improve the quality of homemade movies. Homemade movies are notorious for their poor quality, since they are shot by amateurs who are often just learning how to use their cameras as they're shooting. Although many camcorder devices currently have built-in anti-jitter devices, there are currently no devices to fix problems having to do with zoom and tilt.
"If you've ever used a video camera, you've probably hit the wrong end of the zoom button," Hathaway states wryly. "VISAR can be used to correct these mistakes afterward."
VISAR would also allow people to give their homemade videos movie-style special effects.
|
A Thousand (or more) Points of Light
A "pixel", derived from the phrase "picture element," is a tiny square of color that makes up an image. This technique is similar to Pointillism, the painting style that uses tiny dots of color to create a picture. On computer screens, a blend of red, green and blue determines the specific color of a pixel (in print, it's a blend of cyan, yellow, magneta, and black). The sharpness of an image depends on the density of pixels in an image, or pixels per inch (ppi). The standard ppi for computer monitors is 72 pixels per inch, although they can vary from 50 to 100 ppi. Most magazines print at 300 dpi (dots or pixels per inch); art books go to 1200 dpi. |
VISAR was created using expensive and specialized image processors, but Hathaway believes such technology could soon be affordable and accessible to the general public. For instance, Hathaway and Meyer used "QuBit," a $30,000 device, to capture video images to put on a computer. However, Hathaway uses an image-capturing device that only cost $2,000 on his own computer. And the prices for such devices are dropping fast - already there are image-capturing devices on the market that cost as little as $200. With the growing market of digital camcorders, the process will be even easier and more affordable.
The language of VISAR has become more accessible as well. Originally formatted on a difficult and obscure computer language, Interactive Data Language (IDL), VISAR was translated to the Windows version of C++. Changing the software increased the processing speed from 5 minutes to 15 seconds per video frame. Hathaway and Meyer think they can eventually reduce the processing time even further. Ultimately, they would like to develop "real-time" correction - to actually correct video as it's being filmed.
The development of VISAR relied on the teamwork between Hathaway and Meyer. Whenever one person was stuck on the problem, the other usually came up with a solution. Their very different educational backgrounds and fields of work gave them a broader base of knowledge from which to work.
Created through the efforts of a solar physicist and a meteorologist, VISAR is an example of how science that often seems distant or obscure ends up being used to solve familiar, everyday problems. Soon anyone will be able to use VISAR to turn out clearer, steadier videos. Thanks to VISAR, people subjected to watching someone's home movies may no longer have to wish they'd brought the Dramamine.
VISAR's corrective power depends on how many video frames are available to be blended together, but generally VISAR can correct image jitter to about 1/10th of a pixel. It can correct magnification and zoom to 0.1 percent and angles to within 0.03 degrees.