How virtual cows could help us improve human-robot interactions

A video game in which participants herded virtual cattle has furthered our understanding of how humans make decisions on movement and navigation, and it could help us not only interact more effectively with artificial intelligence, but even improve the way robots move in the future.

Researchers from Macquarie University in Australia, Scuola Superiore Meridionale, the University of Naples Federico II, and the University of Bologna in Italy, and University College London in the UK used the video game as part of a study to understand more about how dynamical perceptual-motor primitives (DPMPs) can be used in mimicking human decision making.

A DPMP is a mathematical model that can help us understand how we coordinate our movements in response to what is happening around us. DPMPs have been used to help us understand how we make navigational decisions and how we move when carrying out different tasks.

This becomes particularly important in complex environments containing other people and a combination of fixed and moving objects, such as you might find on a busy footpath or on a sports field.

Previously, it was assumed that our brains were rapidly making detailed maps of our surroundings, then planning how to move through them.

But an increasing body of research now supports the idea that rather than making a detailed plan, we move naturally, taking into account our goal and making allowances for any obstacles we encounter along the way.

In the new study, published in the latest edition of Royal Society Open Science, participants were asked to work on two herding tasks, moving either a single cow or a group of cows into a pen.

The researchers tracked the order in which the players corralled the cows, and fed the information into their DPMP to see whether the model could simulate the behaviour of the human players.

Lead author, PhD candidate Ayman bin Kamruddin says the team’s DPMP model was able to accurately mimic how the players moved and also predict their choices.

“In the multi-target task, three patterns emerged when people were selecting their targets: the first cow they chose was closest to them in angular distance, all successive cows were closest in angular distance to the previous one they had selected, and when choosing between two cows, they were most likely to choose the one that was furthest from the centre of the containment zone,” Professor Richardson says.

“Once we provided the DPMP with these three rules for making decisions, it could predict nearly 80 per cent of choices on which cows to herd next, and also predict how participants would behave in new situations with multiple cows.”

Herding games are frequently used in studies like this because they mimic real-life situations where people need to control other agent.

In the past they have been based on an aerial view of the target animals, raising the question of whether this unnatural view of the field of play was skewing the findings, by causing participants to make different decisions than they would in a real situation simply because they had a full overview.

To solve this, the team developed a new type of herding game that would limit the participants’ field of vision to what a human could normally see with a first-person perspective of the task, much like that of many roleplay video games.

Senior author Professor Michael Richardson from the Macquarie University Performance and Expertise Research Centre says the change of perspective has important implications.

“While previous research has shown DPMPs can be used to predict crowd behaviour or follow a moving target, ours is the first study to look at whether the model can be extended to explain how a human guides a virtual character or robot,” he says.

“This is another step in informing the design of more responsive and intelligent systems.

“Our findings have highlighted the importance of including smart decision-making strategies in DPMP models if robots and AIs are to better mimic how people move, behave and interact.

“They also suggest that DPMPs could be useful in real-life situations, such as managing crowds and planning evacuations, training firefighters in virtual reality, and even in search and rescue missions, because they can help us predict how people will react and move.”

“Modelling human navigation and decision dynamics in a first-person herding task” by Ayman bin Kamruddin, Hannah Sandison, Gaurav Patil, Mirco Musolesi, Mario di Bernardo, and Michael J. Richardson is published in the Royal Society Open Science DOI: 10.1098/rsos.231919