From Sports Analytics to Drone Navigation: How Competitions Lead to Excellence in Research

15 April 2024

The Institute of Distributed Intelligent Systems at the University of the Bundeswehr in Munich has participated very successfully in four international technology competitions, taking three first places and one second place. The team of Fabian Deuser, Konrad Habel and Prof Norbert Oswald - supported by the computing power of the institute's own GPU cluster "Monacum One'' - succeeded in developing innovative approaches using current models and methods from the field of deep learning and demonstrating the potential of using machine learning methods. The successes were achieved specifically in competitions for sports analytics in team sports and drone navigation.

Innovative Approaches in Sports Analytics

At ACM Multimedia 2022 in Lisbon, the team took first place in the Re-Identification Challenge as part of the "Multimedia Content Analysis in Sports'' workshop. In general, sports analytics in team sports is about evaluating the performance of individual players. This is usually done by analysing data material in the form of video sequences. Even today, the evaluation of data material is only partially automated, as players are often not easily identifiable due to effects such as motion blur, low-resolution image material, occlusions or partial fields of vision and require manual evaluation by a person.

In Lisbon, the specific task was to find all instances of a randomly selected basketball player in various short video sequences recorded at different times from the same camera perspective. Inspired by OpenAI's research in multimodal zero-shot learning, the team modified the Contrastive Language-to-Image Pre-training (CLIP) approach. Instead of determining the similarity of image-text pairs, the CLIP model was supplemented with a Contrastive Image-to-Image approach, which can be used to determine the similarity of image pairs of players. The aim of the training was to minimise the distances between the image features of a specific player. This approach achieved the best overall result with an mAP value of 98.44 and was published in the conference proceedings under the title "CLIP-ReIdent: Contrastive Training for Player Re-Identification'' [1].

At the SoccerNet Re-Identification Challenge as part of the CVPR 2023 in Vancouver [2], the successful approach from Lisbon was transferred to football. In this task, players had to be identified and assigned across multiple camera angles (sideline, back goal camera, replay zoom). This means that the bounding boxes from the object detection must be correctly assigned to a player across several viewing angles. Once again, the approach of Konrad Habel, Fabian Deuser and Prof Norbert Oswald from the VIS Institute took first place, this time with an mAP value of 93.2. In addition, second place was achieved in another competition, Number Recognition. Back numbers are often an important feature in team sports such as football, enabling different players to be identified beyond doubt. Low resolution, motion blur and back numbers that are often not fully visible pose a challenge here. The team decided not to train on the given data for the challenge, but instead to use the larger data set from the Re-Identification Challenge. With the help of an innovative approach, the back numbers were labelled almost completely automatically in order to train their own model on this self-annotated data. It is worth noting that although the team did not train on the given data for the Number Recognition Challenge, this approach achieved the second-best result. The team received a further award for the developed methodology in the form of the jury prize "for the most innovative runner-up method".

New Paths in Drone Navigation

For the Unmanned Aerial Vehicles Challenge of ACM Multimedia 2023 in Ottawa, Fabian Deuser, Konrad Habel, Prof. Martin Werner (TUM) and Prof. Norbert Oswald presented a new approach for determining location without GPS, which is particularly relevant for the use of drones in urban environments. The data set provided includes drone images taken around university buildings and satellite images showing the building in question. The aim of the training with this data set is to assign the drone images to the correct satellite image, as in a kind of memory game, in order to find the location of the drone.

The task was made more difficult by including satellite images of similar but irrelevant buildings and neighbourhoods in order to prove the accuracy and robustness of the developed approach. Using a special training procedure, the researchers were able to estimate the recording angle of drone images in relation to satellite images. These estimated orientations were suitable as additional training data to train the model to better understand different side views of the buildings. It was shown that the additional information generated helped the model to develop a better understanding of positions from different angles. With the presented solution approach for this challenge, which was published under the title ''Orientation-Guided Contrastive Learning for UAV-View Geo-Localisation'' [3], the team prevailed against several competitors and also won first place here.

Summary and Outlook

The successes of the Institute for Distributed Intelligent Systems are proof of the importance of interdisciplinary research and its application to concrete practical problems. They show how scientific competitions can lead to new findings that are important both in the academic world and in practice.

References

[1] K. Habel, F. Deuser, N. Oswald, CLIP-ReIdent: Contrastive Training for Player Re-Identification, ACM Multimedia, Lisbon, 2022

[2] A. Cioppa et al, SoccerNet 2023 Challenges Results, CVPR, Vancouver, 2023

[3] F. Deuser, K. Habel, M. Werner, N. Oswald, Orientation-Guided Contrastive Learning for UAV-View Geo-Localisation, ACM Multimedia, Ottawa, 2023