3D tree reconstruction algorithm contributes to a new era of urban planning

DIGITAL FORESTRY TOOL, GROUNDED IN REAL-WORLD IMAGERY, PROVIDES A PORTAL TO FUTURE

WEST LAFAYETTE, Ind. — Trees compete for space as they grow. A tree with branches close to a wall will develop differently from one growing on open ground.

Now everyone from urban planners and environmental scientists to homeowners may access a new algorithm for tree reconstruction developed at Purdue University to see how the trees will shade an area or learn what a tree will look like in 20 years. Purdue computer scientists and digital foresters used artificial intelligence to generate this first-ever database, which contains three-dimensional models of more than 600,000 real trees.

“These tree models are what we call ‘simulation-ready,’” said Purdue’s Bedrich Benes, professor and associate head of the Department of Computer Science in the College of Science and a member of the Institute for Digital Forestry. The database and related code are publicly available.

Benes, colleagues at Purdue’s Institute for Digital Forestry, Google and the Massachusetts Institute of Technology described the details of their Tree-D Fusion algorithm in the conference proceedings of the European Conference on Computer Vision (ECCV), 2024.

“Trees provide immense and essential value to human society and underpin diverse ecosystems worldwide. They cool the environment, improve air quality, capture carbon dioxide, produce oxygen, and have a positive effect on human physical and mental health,” the co-authors wrote. “The complex effect of trees on the environment has been studied for centuries. Currently, computational models that seek to understand these relationships are hindered by a lack of data.”

The team used the data from the Auto Arborist Dataset introduced by a Google Research team in 2022. The dataset consisted of about 2.6 million trees belonging to 32 genus-level categories from 23 North American cities.

“The particular challenge of this project was getting the three-dimensional model from a single image,” Benes said. “There is not enough input data to extract high-detail information. The generated tree models are approximations. We don’t claim that these are perfect digital twins, but they are useful, for example, for estimating the shading in urban areas.”

“We leverage recent advances in diffusion models to provide prior information for 3D tree reconstruction,” said Raymond Yeh, assistant professor of computer science, who leads the computer vision and AI efforts of the project.

Tree-D Fusion will offer more in the future, said the study’s lead author, Purdue’s Jae Joong Lee, a PhD student in Benes’ Computational Vegetation Group and a member of the Institute for Digital Forestry. “Together with my collaborators, I envision expanding the platform’s capabilities to a planetary scale. Our goal is to use AI-driven insights on social and environmental benefits on a large scale,” Lee said.

Additional co-authors include Purdue’s Bosheng Li, Raymond Yeh and Songlin Fei, all members of the Institute for Digital Forestry; Sara Beery of Massachusetts Institute of Technology; and Jonathan Huang, formerly of Google, now head of AI at Scaled Foundations.

“One goal of Digital Forestry is to improve societal human well-being. We have different projects working on tree localization and inventory from smartphone to satellite,” said Fei, the institute’s director and Dean’s Chair in Remote Sensing.

“This project provides contextualized information on urban tree structure that can be done at scale, providing managers critical information to better manage urban trees. With continued progress on this and other projects, we aim to help make our cities greener, smarter and healthier, tree by tree.”

The initial data for Tree-D Fusion came from public tree census records that many cities maintain online. The Google Research team then merged the tree census data with Street View and overhead color imagery. This made available a large-scale, computer-vision tree monitoring tool for the first time. Researchers at MIT’s Senseable City Lab have already used the new 3D tree models to plot shaded walking routes through select cities.

“Every time a tree-mapping vehicle passes through a city now, we’re not just taking snapshots — we’re watching these urban forests evolve in real time,” said Beery, assistant professor in the MIT Electrical Engineering and Computer Science Department.

“This continuous monitoring creates a living digital forest that mirrors its physical counterpart, offering cities a powerful lens to observe how environmental stresses shape tree health and growth patterns across the urban landscape.”

Comparisons between other 3D reconstruction methods and Tree-D Fusion showed that the latter performs better in many aspects, including projected shading, that are important in planning for green cities.

“The AI model we used was quite computationally demanding,” Benes said. Calculating the entire dataset with a single graphics processing unit (GPU) would have taken about 23 years. Even using all nine of the supercomputing clusters that Purdue’s Rosen Center for Advanced Computing had at the time — now it has 10 — the calculations took nearly six months to complete.

The Tree-D Fusion algorithm development was funded by the U.S. Department of Agriculture’s Natural Resources Conservation Service, National Institute of Food and Agriculture, and Agriculture and Food Research Initiative.

About Purdue Institute for Digital Forestry
Purdue University’s Institute for Digital Forestry merges digital technology and cross-disciplinary knowledge to revolutionize forest management and conservation. The institute is a collaboration of investigators from the colleges of Agriculture, Engineering, Liberal Arts and Science with Purdue Libraries and Polytechnic Institute, integrating expertise in artificial intelligence, machine learning, computer applications and modeling, geosciences, satellite and drone sensing platforms, social sciences, and education for sustainable and equitable rural and urban forest management.

About Purdue University
Purdue University is a public research university leading with excellence at scale. Ranked among top 10 public universities in the United States, Purdue discovers, disseminates and deploys knowledge with a quality and at a scale second to none. More than 107,000 students study at Purdue across multiple campuses, locations and modalities, including more than 58,000 at our main campus in West Lafayette and Indianapolis. Committed to affordability and accessibility, Purdue’s main campus has frozen tuition 13 years in a row. See how Purdue never stops in the persistent pursuit of the next giant leap — including its comprehensive urban expansion, the Mitch Daniels School of Business, Purdue Computes and the One Health initiative — at https://www.purdue.edu/president/strategic-initiatives.

Writer: Steve Koppes