video: Graduate student Kimberly Hsiao describes how the new artificial intelligence system Fashion++ works. view more
Credit: Kimberly Hsiao/University of Texas at Austin
People turn to many different sources for clothing style advice, from magazines to best friends to Instagram. Soon, though, you may be able to ask your smartphone.
A University of Texas at Austin computer science team, in partnership with researchers from Cornell Tech, Georgia Tech and Facebook AI Research, has developed an artificial intelligence system that can look at a photo of an outfit and suggest helpful tips to make it more fashionable. Suggestions may include tweaks such as selecting a sleeveless top or a longer jacket.
"We thought of it like a friend giving you feedback," said Kristen Grauman, a professor of computer science whose previous research has largely focused on visual recognition for artificial intelligence. "It's also motivated by a practical idea: that we can work with a given outfit to make small changes so it's just a bit better."
The tool, named Fashion++, uses visual recognition systems to analyze the color, pattern, texture and shape of garments in an image. It considers where edits will have the most impact. It then offers several alternative outfits to the user.
Fashion++ was trained using more than 10,000 images of outfits shared publicly on online sites for fashion enthusiasts. Finding images of fashionable outfits was easy, said graduate student Kimberly Hsiao. Finding unfashionable images proved challenging. So, she came up with a workaround. She mixed images of fashionable outfits to create less-fashionable examples and trained the system on what not to wear.
"As fashion styles evolve, the AI can continue to learn by giving it new images, which are abundant on the internet," Hsiao said.
Grauman and Hsiao will present their approach at next week's International Conference on Computer Vision in Seoul, South Korea.
Like all AI systems, bias can creep in through the data sets for Fashion++. The researchers pointed out that vintage looks are harder to recognize as stylish because training images came from the internet, which has been in wide use only since the 1990s. Additionally, because the users submitting images were mostly from North America, styles from other parts of the world don't show up as much. Another challenge is that many images of fashionable clothes appear on models, but bodies come in many sizes and shapes, affecting fashion choices. Next up, Grauman and Hsiao are working toward letting the AI learn what flatters different body shapes so its recommendations can be more tailored.
"We are examining the interaction between how a person's body is shaped and how the clothing would suit them. We're excited to broaden the applicability to people of all body sizes and shapes by doing this research," Grauman said.
###