image: If facial-recognition software can recognize three-dimensional works of art, testing will expand to include two-dimensional images such as this posthumous portrait of Lorenzo de' Medici. view more
Credit: Giorgio Vasari
RIVERSIDE, Calif. — Anyone who has admired centuries-old sculptures and portraits displayed in museums and galleries around the world at some point has asked one question: Who is that?
Three University of California, Riverside scholars have launched a research project to test — for the first time — the use of facial recognition software to help identify these unknown subjects of portrait art, a project that ultimately may enrich the understanding of European political, social and religious history.
Funded by an initial grant of $25,000 from the National Endowment for the Humanities, the research project — "FACES: Faces, Art, and Computerized Evaluation Systems" — will apply state-of-the-art facial recognition technology used in the fight against terrorism to solve old and vexing art historical problems, said Conrad Rudolph, professor of art history and project director.
"Almost every portrait painted before the 19th century was of a person of some importance," Rudolph explained. "As families fell on hard times, many of these portraits were sold and the identities of these subjects were lost. The question we hope to answer is, can we restore these identities?"
Participating in the research are Amit Roy-Chowdhury, associate professor of electrical engineering and an expert in computer-based analysis of images and videos, and Jeanette Kohl, associate professor of art history whose research focuses on images and representations of the face in the Italian Renaissance.
Technology that "reads" human faces already must contend with variations in facial expressions, age, facial hair, angle of pose, and lighting, Rudolph said. Refining that technology to recognize human faces in two- or three-dimensional art introduces further challenges, as does portrait art generally in that the image is not a photographic likeness, but rather one that is a visual interpretation on the part of the artist.
Initial subjects for FACES will be selected with as much control over variables as possible. For example, testing will begin by comparing the death or life mask of a known individual to an identified sculptural portrait of the same individual, such as 15th-century figures Lorenzo de' Medici and Battista Sforza.
"If this 3D-to-3D test is encouraging, the project would systematically expand to 3D-to-2D, and eventually test portraits of known subjects against unidentified portraits," Rudolph explained.
"These portraits are social documents that are as significant as historical documents. Today's campaign ads are political documents that are also visual. Portraits operate in the same manner. Identifying the subjects of these historical portraits can help us better understand the social history of the work of art," for example, a previously unknown connection between a nobleman and religious or political leaders of the time.
Rudolph said if the refinement of facial recognition software to identify works of art is as successful as they believe it will be, the technology could be used museums and art conservation laboratories as a standard part of curatorial and preservation practice. It could also be used to recognize variations in architectural details that would reveal new information about building processes, building history, and architectural details, and might also have potential with paleography (ancient writing), possibly allowing the determination of the origin and date of thousands of ancient manuscripts for which identification remains a very subjective matter.
The researchers plan to develop a website and a museum exhibition to demonstrate the use of facial recognition technology to identify portrait subjects.