PITTSBURGH—Smartphone and tablet computer owners have become adept at using finger taps, flicks and drags to control their touchscreens. But Carnegie Mellon University researchers have found that this interaction can be enhanced by taking greater advantage of the finger's anatomy and dexterity.
By attaching a microphone to a touchscreen, the CMU scientists showed they can tell the difference between the tap of a fingertip, the pad of the finger, a fingernail and a knuckle. This technology, called TapSense, enables richer touchscreen interactions. While typing on a virtual keyboard, for instance, users might capitalize letters simply by tapping with a fingernail instead of a finger tip, or might switch to numerals by using the pad of a finger, rather toggling to a different set of keys.
Another possible use would be a painting app that uses a variety of tapping modes and finger motions to control a pallet of colors, or switch between drawing and erasing without having to press buttons.
"TapSense basically doubles the input bandwidth for a touchscreen," said Chris Harrison, a Ph.D. student in Carnegie Mellon's Human-Computer Interaction Institute (HCII). "This is particularly important for smaller touchscreens, where screen real estate is limited. If we can remove mode buttons from the screen, we can make room for more content or can make the remaining buttons larger."
TapSense was developed by Harrison, fellow Ph.D. student Julia Schwarz, and Scott Hudson, a professor in the HCII. Harrison will discuss the technology today (Oct. 19) at the Association for Computing Machinery's Symposium on User Interface Software and Technology in Santa Barbara, Calif. A video demonstrating the technology's capabilities and possible applications can be viewed at: http://chrisharrison.net/index.php/Research/TapSense.
"TapSense can tell the difference between different parts of the finger by classifying the sounds they make when they strike the touchscreen," Schwarz said. An inexpensive microphone could be readily attached to a touchscreen for this purpose. The microphones already in devices for phone conversations would not work well for the application, however, because they are designed to capture voices, not the sort of noise that TapSense needs to operate.
The technology also can use sound to discriminate between passive tools (i.e., no batteries) made from such materials as wood, acrylic and polystyrene foam. This would enable people using styluses made from different materials to collaboratively sketch or take notes on the same surface, with each person's contributions appearing in a different color or otherwise noted.
The researchers found that their proof-of-concept system was able to distinguish between the four types of finger inputs with 95 percent accuracy, and could distinguish between a pen and a finger with 99 percent accuracy.
The Human-Computer Interaction Institute is part of Carnegie Mellon's School of Computer Science. Follow the school on Twitter @SCSatCMU.
About Carnegie Mellon University: Carnegie Mellon (www.cmu.edu) is a private, internationally ranked research university with programs in areas ranging from science, technology and business, to public policy, the humanities and the arts. More than 11,000 students in the university's seven schools and colleges benefit from a small student-to-faculty ratio and an education characterized by its focus on creating and implementing solutions for real problems, interdisciplinary collaboration and innovation. A global university, Carnegie Mellon's main campus in the United States is in Pittsburgh, Pa. It has campuses in California's Silicon Valley and Qatar, and programs in Asia, Australia, Europe and Mexico. The university is in the midst of a $1 billion fundraising campaign, titled "Inspire Innovation: The Campaign for Carnegie Mellon University," which aims to build its endowment, support faculty, students and innovative research, and enhance the physical campus with equipment and facility improvements.