NC State researchers use machine learning to create a fabric-based touch sensor

A new study from NC State University combines three-dimensional embroidery techniques with machine learning to create a fabric-based sensor that can control electronic devices through touch.

As the field of wearable electronics gains more interest and new functions are added to clothing, an embroidery-based sensor or “button” capable of controlling those functions becomes increasingly important. Integrated into the fabric of a piece of clothing, the sensor can activate and control electronic devices like mobile apps entirely by touch. 

The device is made up of two parts; the embroidered pressure sensor itself and a microchip which processes and distributes the data collected by that sensor. The sensor is triboelectric, which means that it powers itself using the electric charge generated from the friction between its multiple layers. It is made from yarns consisting of two triboelectric materials, one with a positive electric charge and the other with a negative charge, which were integrated into conventional textile fabrics using embroidery machines.

Rong Yin, corresponding author of the study, said that the three-dimensional structure of the sensor was important to get right.

“Because the pressure sensor is triboelectric, it needed to have two layers with a gap in between them. That gap was one of the difficult parts in the process, because we are using embroidery which is usually two-dimensional. It’s a technique for decorating fabric,” he said. “It’s challenging to make a three-dimensional structure that way. By using a spacer, we were able to control the gap between the two layers which lets us control the sensor’s output.”

Data from the pressure sensor is then sent to the microchip, which is responsible for turning that raw input into specific instructions for any connected devices. Machine learning algorithms are key to making sure this runs smoothly, Yin said. The device needs to be able to tell the difference between gestures assigned to different functions, as well to disregard any unintentional inputs that might come from the cloth’s normal movement.

“Sometimes the data that the sensor acquires is not very accurate, and this can happen for all kinds of reasons,” Yin said. “Sometimes the data will be affected by environmental factors like temperature or humidity, or the sensor touches something by mistake. By using machine learning, we can train the device to recognize those kinds of things.

“Machine learning also allows this very small device to achieve many different tasks, because it can recognize different kinds of inputs.”

The researchers demonstrated this input recognition by developing a simple music playing mobile app which connected to the sensor via Bluetooth. They designed six functions for the app: play/pause, next song, last song, volume up, volume down and mute, each controlled by a different gesture on the sensor. Researchers were able to use the device for several other functions, including setting and inputting passwords and controlling video games.