Learning group interaction for sports video understanding from a perspective of athlete

In team sports, all players in a scene are usually regarded as a whole by general sports video understanding methods. It is not suitable because to mine two adversarial teams' activities and salient relationship information conveyed by them should be paid more attention.

To solve the problems, a research team led by Wang Yun-Hong (Beihang University, China) published their new research on 15 August 2024 in Frontiers of Computer Science co-published by Higher Education Press and Springer Nature.

The team proposed explores a novel Group Scene Graph Generation (Group-SGG) method to understand team sports videos from the perspective of an athlete and constructs a novel Hierarchical Relation Network to establish intra-team and inter-team relationship features in a video clip by Graph Convolutional Networks.

In the research, after all players in a video are finely divided into two teams, two kinds graphs are constructed. They build Team Relation Graph to represent intra-team relations for recognizing teams' activities, and Group Relation Graph to represent inter-team relations for recognizing interactions. Then the graph features will be enhanced by hierarchical Graph Convolutional Networks (GCNs), Team GCN and Group GCN, for finally generating the Group Scene Graph.

They test their model with different backbones of Inception-v3, VGG16, VGG19. Incorporating GCN, their models perform much better than baselines. Especially, they can improve Group-SGG significantly. The relative gap in performance with respect to the baseline increases significantly: from 1.5 percent to 49.8 percent noticeably with the VGG19 backbone. Finally, from the athlete's view, they elaborately present an interpretation that shows how to utilize Group Scene Graph to analyze teams' activities and provide professional gaming suggestions.

Future work will utilize multiple view to broaden the tested Volleyball+ dataset and make Group-SGG consummate.

DOI: 10.1007/s11704-023-2525-y