FTI-SLAM: federated learning-enhanced thermal-inertial SLAM

A research team has introduced FTI-SLAM, a federated learning adaptation of thermal-inertial SLAM systems. This approach addresses privacy, communication, and generalisation challenges while preserving performance comparable to that of the centralised version. The work demonstrates a practical solution for deploying SLAM in environments with strict data sensitivity and diverse conditions.

Simultaneous localisation and mapping (SLAM) for environmental navigation and mapping is essential for robots and autonomous systems to determine one's position and movement trajectory within that environment without prior knowledge. However, these systems face significant challenges in visually degraded environments, such as smoke-filled or poorly lit areas. The integration of thermal imaging has proven effective, but for SLAM that utilises visual sensing as input and machine learning models, especially deep neural networks, as the frond-end, data privacy and high communication costs have remained critical barriers due to the transmission of sensitive visual data for centralised training.

In their latest work, researchers Haochen Liu and Hantao Zhong from the Department of Computer Science, University of Cambridge, alongside Weiyong Si from the School of Computer Science and Electronic Engineering, University of Essex, propose FTI-SLAM, a federated learning adaptation of existing work on TI-SLAM [1]. Their study demonstrates the feasibility of federated learning in maintaining system performance compared to the original work while addressing critical privacy and communication concerns.

The team adapted the TI-SLAM framework to incorporate federated learning using Flower, a unified and comprehensive federated learning framework. By enabling local model training and different federated aggregation strategies, FTI-SLAM maintains comparable performance levels to centralised systems and shows improvements in settings with more edge devices and diverse data configurations.

Compared to centralised SLAM, FTI-SLAM trains the SLAM front-end, sharing only aggregated parameters with a central server, offering several distinct advantages that make it particularly suitable for real-world applications.

Firstly, it ensures privacy protection by leveraging federated learning, eliminating the need to transmit sensitive data, such as thermal images, to a central server, effectively safeguarding user privacy.

Secondly, it significantly reduces communication costs by transmitting only model parameters rather than raw data, thereby alleviating bandwidth pressures.

Lastly, FTI-SLAM enhances generalisation capabilities by allowing data from diverse sources to participate in training, resulting in more robust models. This approach also enables local fine-tuning to meet specific user needs, ensuring flexibility and adaptability across various deployment scenarios.

The researchers evaluated FTI-SLAM using datasets collected by the authors of TI-SLAM. Experimental results demonstrated that, with limited computational resources and the same amount of training data, FTI-SLAM can maintain performance levels comparable to the original TI-SLAM while improving overall system performance when appropriate aggregation algorithms are employed.

The current implementation of FTI-SLAM is limited to experiments involving a maximum of six clients due to computational constraints and extra data for evaluations on larger scales. Expanding the framework to include more clients in future experiments could enable large-scale simulations, providing insights into the system's scalability and performance under diverse configurations. This would also allow researchers to further investigate the effects of data heterogeneity on the overall system.

Another challenge lies in the need for robust aggregation algorithms to address the threat of malicious attacks. In adversarial scenarios, FTI-SLAM requires stronger defences to prevent attacks and reduce their negative impact on the system. Future studies could explore applying and adapting advanced aggregation algorithms to enhance the system’s resilience and maintain performance in hostile environments.

This paper, “FTI-SLAM: federated learning-enhanced thermal-inertial SLAM”, was published in Robot Learning.

Liu H, Zhong H, Si W. FTI-SLAM: federated learning-enhanced thermal-inertial SLAM. Robot Learn. 2024(1):0003, https://doi.org/10.55092/rl20240003.