With the new "Centre for Visual Computing of Collectives" (VCC), the University of Konstanz is planning an interdisciplinary research centre for the study of collective animal behaviour and its underlying mechanisms and consequences in nature. The research carried out at the VCC will be based on the analysis and visualization of mass data. Understanding the principles of animal behaviour is crucial in a variety of practical contexts ranging from predicting pests (e.g. locust plagues) to controlling autonomous robots (e.g. self-driving cars), while also shedding new light on group behaviour among humans (e.g. during mass panics).

Collective behaviour is a fundamental property of biological systems. Animal collectives such as swarming insects, fish and birds often display highly coordinated behaviours, providing unique opportunities for viewing dynamic collective properties, their efficiency and operations through the lens of individual behaviour. One of the most remarkable and poorly understood properties of such groups is their capacity for collective perception and decision-making. Research into animal collectives provides us with insights into how collective cognition arises from individual decisions and social interactions between individuals. In addition, collective action can be harnessed to the development of new technological applications, including new control architectures for autonomous robots or collective algorithms for interpreting large and complex sets of data (particle swarm optimization).

One central challenge associated with the study of collective behaviour is that coordinated group behaviours evolve out of dynamic processes that take place on various levels, including social interactions between individual organisms, individual decision-making processes and the dynamic processing of sensory input. To date, most studies have drawn an analogy between groups of individuals and physical particles that interact with each other by means of "social forces". It is of course possible to draw certain analogies between organismal collectives and physical systems such as liquids or gasses. But such an approach to the study of collective animal behaviour fails to take the capacity of animal swarms for processing information - such as selective perception and amplified behavioural responses - into account. Individual organisms are not physical particles. They make decisions that are based on the sensory input available to them. Until quite recently, investigating the collective behaviour of migrating groups of individuals constituted a major technical issue. To be able to connect individual perceptions and decision-making processes to processes that occur at the group level, the movements of all individuals within a given group must be tracked and their body position, behavioural state and the sensory input available to them calculated. All of these observations must be made at high speed and under natural environmental conditions. It is further important to be able to influence individual and group behaviour systematically to test research hypotheses.

In that regard, the VCC will provide unique infrastructure to advance the study of animal collectives. The "Imaging Hangar" and "Fish Labs", both of which are specialist laboratories, will enable us to immerse animal groups in reactive virtual environments.

Thanks to current advances in the areas of computer, sensor and projection technologies, we will - for the very first time - be able to perform quantitative behavioural analyses: High-resolution cameras and 3D light field recording systems will enable us to capture all individuals in real time and across temporal as well as spatial scales; highly specialised and efficient signal processing algorithms will use this data to calculate the individual animals’ range of vision in real time; and state-of-the-art synthetic landscape simulations based on computer graphics, in combination with high-resolution 3D projection systems, will allow us to generate realistic virtual environments in real time that respond to the decisions made by individuals within the collective and/or the collective as a whole. All of this enables us to conduct experiments in comprehensively controlled environments, allowing us to collect all the information we need to study collective behaviour. Using the "Data Theatre Lab", we will be able to perform deep analyses of the large data sets collected in the process and, in doing so, decode the fundamental principles of collective animal behaviour. Due to these new opportunities for quantitative behavioural analysis, we are in a unique position to reconstruct and analyse the perceptions of organisms, their individual and collective decision-making processes and, thus, the complex and dynamic behaviours of groups.

The proposed experiments will focus on these three groups of animals: swarming locusts, fish and birds. Observing these three groups will enable us to study collective behaviour across several scales of complexity. Desert locusts, for instance, are a major agricultural pest that can impact the livelihoods of up to one tenth of the global population. Despite their large impact on people and economies, little is known about how flightless juveniles coordinate their behaviour, and we know next to nothing about the coordinated behaviour of flying swarms of mature locusts. Schooling fish constitute a suitable model for investigating how vertebrate brains process collective signals required to coordinate group behaviour. Until recently, they were often regarded as little more than automata. However, research carried out over the past few years suggests that fish possess highly developed cognitive abilities, including a long-term memory. Since their motor behaviour is relatively simple and controlled by a small number of neurons in the brain stem, there is a chance for us to understand the relationship between sensory input and motor response. As compared to fish, many species of birds show a much larger variety of behavioural complexities. Their social behaviour is often based on relationships between individuals and, at least in part, on sophisticated social knowledge. By examining collective behaviour across organisational scales, we can achieve a better and more comprehensive understanding of how collective animal behaviour works and which factors influence it.

The VCC encompasses three synergetic and closely interlinked research priorities that build on each other: In research priority 1, new technologies for monitoring collective behaviour in reactive 3D environments are developed that are then put to use in experiments investigating swarming behaviour carried out in research priority 2. Using visual analysis, the data collected in research priority 2 is processed in research priority 3 and compared with data obtained from animals in the wild. This information is in turn fed back into research priority 2, where it is used to adjust the experiments carried out with laboratory animals. The VCC’s central institutions, the "Imaging Hangar" and "Fish Labs", cannot operate unless all three research priorities work together: input from research priority 1 is required to generate the reactive virtual environments for the organisms studied in research priority 2. The experiments conducted in research priority 2 involve desert locusts, zebrafish and red-billed quelea. Interactive analyses of the data collected in research priority 2 are carried out in research priority 3, which in turn influence the experiments conducted in research priority 2 as well as the immersive virtual environments generated in research priority 1.

Overview of research priorities