The Eurecat technology centre has coordinated the European GuestXR project, which has developed an innovative artificial intelligence agent, called The Guest, that integrates into extended reality (XR) environments to moderate and encourage participation in virtual meetings, with the aim of helping participants communicate more effectively.
The solution, which has been developed with the scientific coordination of the University of Barcelona, helps to mitigate potential conflicts and to balance speaking time in a constructive way. In addition, it is able to observe behaviour, detect exclusion among participants and intervene to facilitate interaction, reduce conflict and foster empathy, supporting the group in achieving its objectives.
The agent has been tested in four use cases focused on climate change awareness, conflict resolution, social applications for participants with hearing impairments, and the recreation of historical protests to study group dynamics.
GuestXR’s machine-learning system has been developed to rehearse social situations that may trigger conflict or debate, using virtual reality and artificial intelligence to build a metaverse for good. The project has developed two applications that allow participants, on one hand, to explore different perspectives in conflict situations in order to put themselves in others’ shoes, and on the other, to examine the dynamics of social protests by exploring the factors that can turn such gatherings into violent events.
In addition, GuestXR has created an innovative extended reality game that immerses participants in scenarios where players face the consequences of poor resource management, with the aim of increasing environmental awareness as part of a study on individual and group dynamics in situations of resource scarcity.
The project’s solution has also been evaluated through interactions with participants with varying degrees of hearing impairment, incorporating new tools and methodologies to optimize acoustics in extended reality environments. The objective is to facilitate participation of people with hearing difficulties and improve speech comprehension, thereby promoting social inclusion in virtual and augmented reality environments.
Likewise, The Guest has been tested in simulations of historical protests to investigate how and when users became involved through the AI agent, and which elements contributed to violence. This specific use case was selected through an open call to make the project’s extended reality technologies accessible to other academics and professionals.
The selected team, a group of researchers from the Norwegian University of Science and Technology (NTNU) and UiT The Arctic University of Norway, used The Guest to investigate individual and group responses during protests and to analyse events from different perspectives, including pedagogical and artistic approaches. This was done through virtual recreations of women’s reproductive rights demonstrations in Warsaw, the pro-democracy “Be Water” protests in Hong Kong, and climate demonstrations.
Final presentation of the GuestXR project in Brussels
During the project’s final event, held in Brussels, the results achieved were presented on a round table conducted in a virtual environment and moderated by The Guest itself, with the participation of experts from the organizations that form the GuestXR consortium.
“Welcome to our XR round table. I am The Guest, an AI agent designed to help groups maintain constructive and inclusive discussions in virtual spaces. Today we will talk about the challenges, lessons learned and future directions of the GuestXR project,” the artificial intelligence agent introduced itself.
Its main objectives were to balance participation, maintain a constructive, calm and curious tone, focus the discussion on the project’s outcomes and, in the event of conflict, de-escalate it and focus the conversation towards understanding and shared goals, while respecting the allocated time.
“This presentation has been an example of how The Guest could be applied in real life. Whether for working groups or debates, every gathering of people held in extended and augmented reality has a purpose, and the agent’s goal is to help the group achieve that. This project has involved exciting research with a strong interdisciplinary component, breaking new ground and hopefully contributing to the success of social Virtual and Augmented Reality,” highlighted Mel Slater, scientific coordinator of the project from the University of Barcelona and member of the Institute of Neurosciences of the UB (UBneuro).
The GuestXR consortium, funded by Horizon 2020, is made up of the Eurecat technology center, the Event Lab at the University of Barcelona, the Inria Center at the Rennes University, the Advanced Reality Lab and The Baruch Ivcher Institute for Brain, Cognition and Technology at Reichman University, the Brain and Emotion Lab and the Brightlands Institute for Smart Society (BISS) at University of Maastricht, the University of Warsaw and the companies Virtual Bodyworks and g.tec medical engineering GMBH.
