PriMR (DFG)

Mixed Reality (MR) headsets offer great opportunities for a wide range of applications for leisure, work, education, and marketing, among others. With MR, users can be immersed in a virtual world (Virtual Reality) or augment their view of the real world with virtual content (Augmented Reality). MR enables one to, e.g., have meetings or collaborations across physical locations in which some users are on-site while others join remotely. While there is an ongoing discussion as to whether the vision of the metaverse will become a reality, it seems undisputed that MR headsets are likely to witness a similar level of proliferation as smartphones and smart watches; that is, they will become a pervasive technology.

MR headsets utilize an array of sensors (including many cameras) and bring them closer to the human body, enabling sensitive data to be captured, processed, and shared with third parties. State-of-the-art headsets already provide access to behavioral data (e.g., hand and body motion, eye gaze), physiological data (e.g., electroencephalography, heart rate), contextual data (e.g., size of tracking space, bystanders), and device specifications. Such data allow one to infer information about user demographics (e.g., age, gender, handedness), health and well-being (e.g., reaction times, fitness level), and impairments (e.g., visual or motor impairment). It is apparent that such data is sensitive. While it provides value for MR users (e.g., novel functionality) and stakeholders (e.g., to target advertisements), it has severe privacy implications for users.

The tension between utilizing and protecting sensitive data is not new: with the advent of social networks in the early 2000s, user data became a primary source of revenue. Identifying and analyzing user behavior through third-party (tracking) cookies not only allowed for targeted advertising, but also enabled pushing of political agendas (cf. the Cambridge Analytica scandal). As smartphones have become the primary means to access the internet, audio, video, and geolocation are widely available to services and apps. Wearable devices provide access to health-related information and smart home assistants allow users to be observed during activities in their most private spaces. MR headsets enable a natural continuation of this trend, adding a new factor to the endeavor of supporting users in protecting their privacy, in particular due to the use of cameras. While this is required to enable tracking and interaction, the captured camera data can also be misused. This creates an inherent challenge, since providing access to data is necessary to create a powerful Mixed Reality experience. At the same time, it is important to empower users and their bystanders to protect their data from unintended use by third parties.

The objective of this project is to examine how user interfaces for privacy control can be built for Mixed Reality. The main challenges this project addresses are (1) how users and bystanders can be made aware of and understand the privacy implications of using MR technology, and (2) how they can be empowered to make reasonable decisions regarding data collection, processing, and sharing by MR devices and apps.