Gwangju Institute of Science and Technology and MIT researchers develop a natural and comfortable “seamless-walk” virtual reality locomotion system

Urban real-world environments have limited physical space for foot-based locomotion and present challenges to natural VR locomotion (since virtual environments are much larger than the corresponding real-world environment), a fact that has been noted in past studies (Mandal 2013; Pai and Kunze 2017). To compensate for this challenge, efficient virtual reality (VR)-based locomotion techniques have been proposed to enable natural and immersive locomotion experiences akin to walking in large, virtual environments. However, the VR locomotion systems often require attaching an equipment to the body or video-recording the user’s body pose. This leads to discomfort caused by equipment size and discontinuous adjustment as well as privacy concerns related to capturing the entire body without blind spots. Against this backdrop, researchers from Gwangju Institute of Science and Technology (GIST), Korea in collaboration with researchers from Massachusetts Institute of Technology Computer Science and Artificial Intelligence Laboratory (MIT CSAIL), USA developed a novel foot-based VR locomotion system, called “Seamless-walk,” that offers a more natural and comfortable locomotion experience without requiring the use of any walking equipment or a video of the user’s body pose during walking or interaction with objects using their hands.

In their recent article published online on 17 January 2023 in the journal Virtual Reality, the researchers, led by Dr. Kyung-Joong Kim, Associate Professor at GIST, have detailed the development of the VR locomotion system. “When we started collaborating with MIT, they introduced an interesting new sensor called the ‘intelligent carpet.’ In our view, it was a great opportunity as well as a challenge for us since it had not been developed for any specific application. Therefore, we wanted to make something practical and interesting with this sensor and our AI technology,” explains Dr. Kim. “Accordingly, we decided to develop a VR game controller with the ‘intelligent carpet’ sensor that would be useful in VR gaming.”

“Seamless-walk” has both immediate and long-term potential applications. “In the long run, we believe that our technology could be used in healthcare. ‘Seamless-walk’ is not only a VR gamepad but also a gait recognition and analysis method,” says Dr. Kim.

Seamless-walk works in the following manner: the intelligent carpet captures high-resolution foot pressure imprints in real time as the user moves around by measuring the applied pressure through resistance changes. The footprint information is then fed into a machine learning model that extracts the strong pressure points using a technique called “K-means clustering.” In this method, the pressure points are divided into two clusters, corresponding to the user’s left and right feet. From these clusters, the user’s body direction and foot intervals are then extracted to estimate the angle and movement speed.

Moreover, Seamless-walk has a modular structure that enables a scalable and inexpensive installation of a touch sensing platform. The team conducted tests on 80 individuals using Seamless-walk in a 3D virtual world exploration game, demonstrating that the novel technology of the system guarantees an immersive, natural, and comfortable experience. At the same time, it does not compromise the overall VR experience, outperforming existing VR locomotion methods.

“In the future, we plan to add more detailed gait analysis functions to the current system. This would enhance our sensor and gait analysis system to provide fall detection and health monitoring in a comfortable manner without any privacy issues,” highlights Dr. Kim. “This method could also be used at the gym for monitoring the gait of users on the treadmill or checking their balance during weight training.”

Taken together, this novel development has the potential to advance gait analysis in VR gaming and healthcare.

***

Reference

DOI: https://doi.org/10.1007/s10055-023-00750-x

Authors: Yunho Choi¹, Dong‑Hyeok Park¹ , Sungha Lee³ , Isaac Han¹ , Ecehan Akan¹ , Hyeon‑Chang Jeon³ , Yiyue Luo² , SeungJun Kim¹ , Wojciech Matusik² , Daniela Rus² , Kyung‑Joong Kim^1,3

Affiliations:     

¹School of Integrated Technology, GIST

²CSAIL, MIT

³Artifcial Intelligence Graduate School, GIST

About the Gwangju Institute of Science and Technology (GIST)
The Gwangju Institute of Science and Technology (GIST) is a research-oriented university situated in Gwangju, South Korea. Founded in 1993, GIST has become one of the most prestigious schools in South Korea. The university aims to create a strong research environment to spur advancements in science and technology and to promote collaboration between international and domestic research programs. With its motto of “A Proud Creator of Future Science and Technology,” GIST has consistently received one of the highest university rankings in Korea.

Website: http://www.gist.ac.kr/

About the author
Kyung-Joong Kim is an Associate Professor at the Institute of Integrated Technology and the head of the Game AI Center in Gwangju Institute of Science and Technology (GIST) in Korea. His group is developing human-centered game AI to enhance the game experience of human players. Before coming to GIST, he served as an Associate Professor in Computer Engineering at Sejong University, Korea for 10 years. He completed his postdoctoral training at Lipson lab at Cornell University, USA. His lab members include Yunho Choi, Dong-Hyeok Park, Sungha Lee, Issac Han, Ecehan Akan, and Hyeon-Chang Jeon, all of whom are students at GIST. His collaborators in this project include Prof. SeungJun Kim from GIST, Prof. Wojciech Matusik and Prof. Daniela Rus from MIT, and Yiyue Luo, a PhD student at the Matusik lab in MIT, USA.