A sensory pen which can read Braille could improve literacy amongst the visually impaired

A pen which can transform Braille into English text has been developed by experts at the University of Bristol.

Braille literacy is frequently reported as being in decline, this is despite visually impaired people often expressing a desire to learn it, and Braille literacy being a highly valued skill by those who are capable. This is often attributed to the lack of available learning resources, particularly away from large urban centres.

The handheld device, which includes a one-centimetre sensor with 19 channels programmed to read Braille, has demonstrated high accuracy in early trials.

Lead author Dr George Jenkinson explained: “This device, Braille-tip, was designed to aid people’s ability to learn independently, and will hopefully form part of the solution to increasing Braille literacy and allow people to reap the benefits of reading and writing.”

Braille-tip is a compact soft tactile sensor which can be mounted on a standard pen and is designed to dynamically assist with reading and learning Braille.

Dr Jenkinson  continued: “I used the handheld device to read multiple passages of Braille, and analysed how accurately it could process the tactile cues (Braille bumps) into English text.”

Fluid channels are used to transmit tactile information from 19 sensitive areas under a silicone membrane to a single camera. Its sensory response is calibrated to detect raised Braille dots on embossed cardboard designed for Braille-readers, and its functionality is demonstrated by reading Braille aloud, with an 84.5% success rate when operated by hand.

The algorithm works in real time which means it does not rely on training or deep learning. It is predictable, explainable, and portable to other tactile sensors. This results in code that is robust, and straightforward to edit.

Braille is a standardised and highly structured communication medium. A variety of techniques are used by people to read Braille, where a common advanced technique involves using both index fingers from each hand. In this, the trailing finger may be used to check and re-read certain letters and to find the beginning of the next line while the leading finger continues to read. Braille-tip may be used like the leading finger, to assist learning readers.

The team plan to extend the functionality of the device so that it is easier to operate accurately.

George added: As soon as possible, the device should be tested with participants, and prototypes should be made available to the intended end-users so that their desires and the potential use for such a device can be assessed in earnest.

“A co-design approach that involves users is much more likely to have a positive real-world impact than an approach siloed in the laboratory.

“The pattern of the errors suggest that they came from the way the device was held and operated, suggesting that the algorithm and sensor are likely to be able to reach much higher accuracy close to 100% if the design is improved.”

Braille is frequently included in public spaces and devices such as transport hubs, elevators, and cash point machines to increase accessibility for users. A device capable of reading Braille letters encountered in these public spaces would enable those who are visually impaired and Braille-illiterate to engage with the space or device, and give them the ability to practice reading Braille independently.

Paper:

‘Braille-tip: Structured Small-Footprint Tactile Sensor for High Acuity Dynamic Tactile Tasks’ by George P. Jenkinson, Andrew T. Conn and Antonia Tzemanaki presented at IEEE EMB's 10th International Conference on Biomedical Robotics and Biomechatronics.