Towards automatic detection of road features with deep learning

In Japan, a substantial amount of point cloud data–a set of data points in space–has been measured and accumulated for public works using mobile mapping systems and terrestrial laser scanners. However, this vast amount of data is of limited use in an unprocessed and unstructured state. Fortunately, it can be structured by automatically extracting a feature using a “plan of completion drawing” that shows the completed geometry of a construction object. 

Earlier this year, researchers from Japan, led by Professor Ryuichi Imai of Hosei University, Japan proposed another method for extracting road features using high-precision 3D (HD) map data. However, the applicability of their approach is limited to the developed sections of road maps. While the issue can be solved with deep learning-based identification, they require a large amount of manually-prepared, high-quality training data. 

Recently, Prof. Imai and his collaborators, Kenji Nakamura of Osaka University of Economics, Yoshinori Tsukada of Setsunan University, Noriko Aso of Dynamic Map Platform, and Jin Yamamoto of Hosei University developed an algorithm to automate the process of training data generation and constructed a road feature identification model from point cloud data extracted automatically from HD maps. 

“Currently, people need to visually check the point cloud data to identify road features as computers cannot recognize them. But with our proposed method, the feature extraction can be done automatically, including the features at undeveloped road map sections,” explains Prof. Imai. Their work was presented at the Joint 12th International Conference on Soft Computing and Intelligent Systems and 23rd International Symposium on Advanced Intelligent Systems on 29th November 2022. 

In their study, the researchers first separated the ground surface from the point cloud data using the CloudCompare software. Next, they generated area data from the HD map and extracted component points of features. While these points were assigned as either road signs or traffic lights, other labels were provided for the remaining data. Then, the area data corresponding to the component points was extended to generate the training data. Using this, the researchers further generated the point cloud projection images. Lastly, they used the training data to construct the identification model using a YOLOv3 object-detection algorithm. The model could detect road features based on clustering points other than those identified for the ground surface using CloudCompare. 

Having established the computational framework, the researchers performed demonstration experiments in the Shizuoka Prefecture on a road with 65 road signs, 46 traffic lights, and noise features over a distance of 1.5 kilometers. They used 258 road signs and 168 traffic lights to train their identification model and used 36 and 24 images, respectively, to calculate the algorithm determination accuracy. 

The researchers found that the precision, recall, and F-measure were 0.84, 0.75, and 0.79, respectively, for the road signs and 1.00, 0.75, and 0.86, respectively for the traffic lights, indicating zero false determinations. The precision of the proposed model was shown to be higher than the existing models. 

Prof. Imai concludes by highlighting the future implications of the work. “A product model constructed from point cloud data will enable the realization of a digital twin environment for urban space with regularly updated road maps. It will be indispensable for managing and reducing traffic restrictions and road closures during road inspections. The technology is expected to reduce time costs for people using roads, cities, and other infrastructures in their daily lives.”

And his vision may not be far from being realized!

***

Reference

Authors: Ryuichi Imai¹, Kenji Nakamura², Yoshinori Tsukada³, Noriko Aso⁴, Jin Yamamoto⁵

Title of original paper: Automatic Generation of Road Feature Identification Models from Point Cloud Data Using HD Maps

Affiliations: 

¹Faculty of Engineering and Design Hosei University

²Faculty of Information Technology and Social Sciences Osaka University of Economics

³Faculty of Business Administration Setsunan University

⁴Dynamic Map Platform

⁵Faculty of Engineering and Design Hosei University Graduate School

About Professor Ryuichi Imai

Ryuichi Imai received his Doctor of Engineering degree from the University of Tokyo, Japan. He has been actively working in the field of Civil Engineering for almost 25 years. His laboratory studies technologies for measuring, analyzing, and visualizing social spaces for learning their current status, discovering potential events, and taking measures to deal with problems. They generate and utilize product models as well as three-dimensional models of national spaces using photographs and point cloud data measured by a wide variety of devices, such as total station, laser, radar, moving object measuring vehicle, and UAV. They are also studying urban activities.