New radiative transfer modeling framework enhances deep learning for plant phenotyping

A research team has developed a radiative transfer modeling framework using Helios 3D plant modeling software to simulate RGB, multi-/hyperspectral, thermal, and depth camera images with fully resolved reference labels. This innovative method markedly diminishes the necessity for labor-intensive, manually annotated datasets. The framework's capacity to generate high-quality synthetic images enables efficient training of deep learning models for high-throughput plant phenotyping, thereby enhancing crop trait analysis and providing a instrument tool for advancing agricultural research and remote sensing applications.

The integration of remote and proximal sensing methodologies facilitates the high-throughput monitoring of plant systems, providing comprehensive insights into plant function. Advances in these technologies have led to abundant high-resolution images, but challenges remain in linking this data to actionable plant traits. The current methods are inadequate for the labor-intensive data annotation and multimodal data alignment that are required.

A study (DOI: 10.34133/plantphenomics.0189) published in Plant Phenomics on 30 May 2024, aims to address these challenges by developing a novel 3D radiative transfer modeling framework.

This research verified a radiative transfer model using a variety of SKILL scores to evaluate its accuracy in simulating the radiation absorbed by objects and reflected radiation fluxes. The SKILL scores for different tests (brfpp_uc_sgl, brfpp_co_sgl, brfop, and fabs) were 98.00, 92.65, 97.52, and 99.98, respectively, demonstrating the model's high precision. Moreover , the R² values for camera calibration ranged from 0.864 to 0.930, indicating effective distortion recovery and color calibration. Synthetic images generated using the model, including RGB, NIR, and thermal images, showed high visual similarity to real images, thereby confirming the model's ability to produce high-quality, annotated plant images. These findings validate the model's efficacy in simulating intricate scenes and establish it as a robust instrument for high-throughput plant phenotyping and machine learning model training.

The study's lead researcher, Tong Lei, asserts that Helios offers a simulated environment that enables the generation of 3D geometric models of plants and soil with random variation, as well as the specification or simulation of their properties and functions. This approach diverges from traditional computer graphics rendering, as it explicitly models the physics of radiation transfer, thereby establishing a crucial link to the underlying biophysical processes of the plant.

In summary, this study introduces a radiative transfer modeling framework using Helios 3D software to simulate plant images, including RGB, multispectral, thermal, and depth images, with detailed annotations. The framework reduces the need for manual data collection and improves deep learning model training for plant phenotyping. Future developments will enhance model flexibility and incorporate more complex processes, advancing high-throughput phenotyping and agricultural research by providing efficient analysis of plant traits and physiological states.

###

References

DOI

10.34133/plantphenomics.0189

Original Source URL

https://doi.org/10.34133/plantphenomics.0189

Authors

Tong Lei ^1*, Jan Graefe ², Ismael K. Mayanja³, Mason Earles ^3,4,and Brian N. Bailey ¹

Affications

1 Department of Plant Sciences, University of California, Davis, CA, USA.

2 Leibniz Institute of Vegetableand Ornamental Crops e.V. (IGZ), Großbeeren, Germany.

3 Department of Biological and AgriculturalEngineering, University of California, Davis, CA, USA.

4 Department of Viticulture and Enology, Universityof California, Davis, CA, USA.

Funding information

This work was supported, in whole or in part, by the Bill & Melinda Gates Foundation INV-0028630 and USDA NIFA Hatch project 7003146. Under the grant conditions of the Foundation, a Creative Commons Attribution 4.0 Generic License has already been assigned to the Author Accepted Manuscript version that might arise from this submission.

About Plant Phenomics

Plant Phenomics is an Open Access journal published in affiliation with the State Key Laboratory of Crop Genetics & Germplasm Enhancement, Nanjing Agricultural University (NAU) and published by the American Association for the Advancement of Science (AAAS). Like all partners participating in the Science Partner Journal program, Plant Phenomics is editorially independent from the Science family of journals. Editorial decisions and scientific activities pursued by the journal's Editorial Board are made independently, based on scientific merit and adhering to the highest standards for accurate and ethical promotion of science. These decisions and activities are in no way influenced by the financial support of NAU, NAU administration, or any other institutions and sponsors. The Editorial Board is solely responsible for all content published in the journal. To learn more about the Science Partner Journal program, visit the SPJ program homepage.