Paint the town

Osaka, Japan – Scientists from the Division of Sustainable Energy and Environmental Engineering at Osaka University developed an automatic method for determining the façade color and type of building using deep learning models applied to street-level images. This work may assist in fostering neighborhood cohesion and support urban renewal.

Using the magic of modern navigation apps, such as Google Street View, you can take a virtual trip to nearly anywhere in the world, and instantly feel like you are standing in the middle of the bustling urban area of your choice. However, these street-level images remain an underutilized data source, because it is often difficult for a computer to distinguish between different buildings or extract their colors.

Now, scientists from Osaka University have developed a machine learning method for classifying both the type of building and its primary façade color. Using an open-source analog of Baidu Street View, the team created color and building classification maps for three metropolitan areas in China. “Color coordination is part of what gives each district within a city its unique identity,” first author Jiaxin Zhang explains. Building types included residence, public service, commercial, or utility. The accuracy of building façade segmentation, dominant color, and building classification were evaluated by surveying the locations to establish the ground truth.

For this project, a semantic segmentation algorithm was used to extract the outlines of the building façades from the street view images. Color calibration of the photos was done during pre-processing, then a multi-label deep learning algorithm was used to classify the façade color and building type. After that, color distribution maps were generated for each city. “Our work helps computers interact with the aesthetic part of architecture and city planning, which has been very difficult to do previously,” senior author Tomohiro Fukuda says. This approach, which produced tailored street-view datasets built for training multi-label classifiers, can be applied more broadly to help civic planners promote both tourism and social cohesion.

###

The article, “Development of a city-scale approach for façade color measurement with building functional classification using deep learning and street view images” was published in ISPRS International Journal of Geo-Information at DOI: https://doi.org/10.3390/ijgi10080551

About Osaka University

Osaka University was founded in 1931 as one of the seven imperial universities of Japan and is now one of Japan's leading comprehensive universities with a broad disciplinary spectrum. This strength is coupled with a singular drive for innovation that extends throughout the scientific process, from fundamental research to the creation of applied technology with positive economic impacts. Its commitment to innovation has been recognized in Japan and around the world, being named Japan's most innovative university in 2015 (Reuters 2015 Top 100) and one of the most innovative institutions in the world in 2017 (Innovative Universities and the Nature Index Innovation 2017). Now, Osaka University is leveraging its role as a Designated National University Corporation selected by the Ministry of Education, Culture, Sports, Science and Technology to contribute to innovation for human welfare, sustainable development of society, and social transformation.

Website: https://resou.osaka-u.ac.jp/en