Physics and emote design: Quantifying clarity in digital images

When analyzing artworks, understanding the visual clarity of compositions is crucial. Inspired by digital artists, Okinawa Institute of Science and Technology (OIST) researchers from the Mechanics and Materials Unit have created a metric to quantify clarity in digital images. As a result, scientists can accurately capture changes in structure during artistic processes and physical transformations.  

This new metric can improve analysis and decision-making across the scientific and creative domains, potentially transforming how we understand and evaluate the structure of images. It has been tested on digital artworks and physical systems. The research is published in the journal PNAS.  

Defining clarity in art with math 

At the heart of the methodology lies a straightforward approach, inspired by the practice of digital artists zooming out to evaluate their work. The researchers developed a high-school-level mathematical method to quantify "clarity" in digital images by measuring how clear visual elements stay when blurred. This metric bridges the gap between physics and art, providing a flexible analytical tool for the scientific analysis and artistic creation of digital images.  

The method involves blurring images by randomly swapping neighboring pixels and then comparing original images with their blurred versions. The researchers measure how much of the original structure remains intact, assigning higher scores to images with structural elements that remain recognizable even after blurring and zero to images with chaotically distributed pixels or solid colors.  

“In our study, we defined ‘clarity’ as the resistance to blurring or structural degradation. Artistically, this definition aligns with a common artistic practice of zooming out of a canvas to assess how clear an artwork appears. Mathematically, clarity can be expressed in terms of the contrast of colors and their spatial distribution,” Prof. Eliot Fried, head of the Mechanics and Material Unit, explained. 

The metric preserves the color diversity of an image and is effective even after image compression. It is versatile, useful for analyzing different images, detecting structural changes in physical systems, and aligns with color theory, which studies how colors interact and affect our perceptions. Essentially, it shows how much of the pattern within an image survives random changes. 

“Artists are always experimenting with their techniques and compositions. The idea came to me while drawing an emote. As an experimental physicist, I was inspired to create metrics that can measure artistic qualities like clarity, balance, and harmony. We hope these metrics will allow art researchers to experiment with different compositions and report their findings in an objective, quantitative way,” Dr. San To Chan, researcher and first author, said.  

In digital art, an "emote" typically describes small, expressive images or animations that convey emotions, especially within online communities and on streaming platforms.  

Video games, Vtubers, and emotes 

Guided by their metric, the researchers designed an emote for the video game Holo X Break. This practical use proved that their research can be useful even in commercial art projects. 

“I chose to portray the virtual YouTuber (VTuber) Sakura Miko because I am a ‘35P’; a member of her fanbase. Together with several other 35Ps, we have previously animated her for a billboard display in New York Times Square to celebrate her 5th anniversary. Following this experience, I felt that featuring her in this research would be an exciting way to connect my enthusiasm as a fan with my professional experience as a scientist," Dr. Chan added. 

Outside of his research, Dr. Chan is a freelance artist who has contributed to renowned indie game development projects such as Holocure. This study draws strength from significant professional expertise by incorporating artworks from peer artists, providing a strong foundation for the analysis. 

Dr. Chan and Prof. Fried are exploring potential applications of their clarity metric beyond its initial scope. They believe their metric could revolutionize art analysis. Just as engines convert thermal energy to mechanical work, artists convert clarity in exchange for artistic qualities such as harmony, balance, and rhythm.  

Such a thermodynamic perspective on art could help researchers understand the rationales behind certain creative decisions made by artists, providing knowledge that will be useful to human artists to perfect their crafts further.