A methodology to read QR codes on uneven surfaces

Sometimes, we try to capture a QR code with a good digital camera on a smartphone, but the reading eventually fails. This usually happens when the QR code itself is of poor image quality, or if it has been printed on surfaces that are not flat — deformed or with irregularities of unknown pattern — such as the wrapping of a courier package or a tray of prepared food. Now, a team from the University of Barcelona and the Universitat Oberta de Catalunya has designed a methodology that facilitates the recognition of QR codes in these physical environments where reading is more complicated.

The new system does not depend absolutely on the underlying topography, and is applicable to QR codes that can be found on tubular surfaces (bottles), food trays, etc. It is the first technological proposal capable of combining a generalist methodology and two-dimensional barcodes to facilitate the recognition of digital information.

The study, published in the journal Pattern Recognition Letters, is first authored by Professor Ismael Benito, from the UB’s Faculty of Physics and Department of Electronic and Biomedical Engineering, and the UOC’s Department of Computer Science, Multimedia and Telecommunications Studies. Cristian Fàbrega and Joan Daniel Prades, professors at the Faculty of Physics and the UB’s Institute of Nanoscience and Nanotechnology (IN2UB), are co-authors of the research, as are experts Hanna Lizarzaburu-Aguilar and David Martínez Carpena, from the UB’s Faculty of Mathematics and Computer Science. All the authors have participated in different positions in the creation of ColorSensing, SL, a UB spin-off company in the field of smart labelling.

Why are some QR codes difficult to read?

QR codes are a variation of the typical barcode, capable of collecting information in computer language — in a two-dimensional matrix of black and white pixels — when scanned with a scanning device. They facilitate access to data of interest, save time and resources such as paper, and have revolutionized the way users access information in the digital realm.

However, it is sometimes difficult to scan a barcode correctly. According to Benito, from the UB’s Department of Electronic and Biomedical Engineering and former technology director of ColorSensing, this happens, “first of all, because of the quality of the image. Although many people today have access to good digital cameras, they cannot always capture the QR image well. Secondly, the print quality of the QR code and the colours used — with good contrast — is sometimes not satisfactory. Finally, if the printing surface is not flat enough and not parallel to the capture plane, it is also difficult to capture the information in the code”.

“For example, all these factors come into play when we try to capture a Bicing QR with the mobile app: the surface is not flat — it is a cylinder — and if we try to capture the QR too close, the deformation of the surface becomes evident and the reading fails — 5-10 centimetres; if we move too far away, the QR becomes too small and the capture is not good — 1 metre; if we are in an intermediate range, the apparent distortion of the surface is reduced and the quality is suitable for capturing it — 30-50 centimetres”, explains Benito.

An algorithm that exploits properties of QR codes

The study, which is part of Ismael Benito’s doctoral thesis at the UB, presents a new algorithm that takes advantage of the QR's own characteristics — i.e. the code’s internal patterns — to extract the underlying surface on which the code is positioned.

The texture of this surface is recovered by a generalist adjustment based on mathematical functions known as splines, which allow the topography of the surface to be adjusted locally. Benito points out that “they are functions that adapt locally to the ups and downs of the surface, and form a technique that was originally widely used in fields such as geology or photographic editing to adjust or generate deformations in surfaces”.

There are still many technological challenges to improve the whole process of QR code recognition. In the case of commercial applications activated by the user’s code reader, the expert explains that “the main challenge is to be able to provide correct and reliable readings. We are also working hard to ensure that the codes cannot be attacked by modification techniques, for example, with a fake URL that can capture data with small modifications to the code. In the case of industry, where captures are done in controlled environments, the main challenge is to reduce the speed of capture”, says the expert.

ColorSensing was created at the UB in 2020 by Professor Joan Daniel Prades, from the Faculty of Physics and the INN, and María Eugenia Martín, now CEO of this start-up company, and it won the Metropolitan Business Innovation Award 2023 for having developed smart labelling to reduce food waste. Likewise, in 2022 it received the UB’s Senén Vilaró Award for the best innovative company, being distinguished at the Sustainability Awards 2022 in the smart and active packaging category. Another of the firm’s scientific achievements is the patent granted in the United States and Europe, whose promoters are Ismael Benito (UB and UOC), Olga Casals (UB), Cristian Fàbrega (UB), Joan Daniel Prades (UB and Technical University of Braunschweig, TUB) and Andreas Waag (TUB).