Social XR: Virtual social events on future networks

Can you imagine being at a concert, talking with friends and watching your favorite band, but each one of you from your living room? This is possible thanks to the virtual recreation of the event using extended reality (XR) technologies and 6G communication networks.

People’s mobility and communication networks have created a society in which, despite distance, we can maintain a constant connection with our families and friends. This connection is now mediated by messages, social networks, calls or video calls. Although very useful, they are very different from physical connections and face-to-face communication. XR technologies and 6G networks will allow us to create connections over distances that are much more natural and similar to face-to-face. In addition, they will allow us to enjoy high quality immersive content anywhere.

In order to achieve this scenario, there are a series of main technological challenges.

Immersive content production: a major challenge with an artistic side

First of all, the creation of immersive content, however novel, is a challenge with an artistic side. On the one hand, immersive content differs from traditional audiovisual content and requires other creative processes. In the case of a concert, this content could be created by recording a real live concert, or it could be more similar to the recording of an album and be made in parts and then assembled. Both options, and other existing possibilities, have implications for technical aspects.

New tools need to be created so that content producers can work without the need for extensive technical training. In addition, recording systems for this type of content are complex infrastructures, with many connected cameras, and with high computing and communications requirements. The proliferation of immersive content capture systems is conditioned by the democratization of this type of technology.

It is necessary to create more flexible, affordable and accessible systems for different types of audiovisual production. And this is where next-generation networks will play a key role. The ability to interconnect devices with low latency and high capacity, for example through millimeter wave technologies, can be crucial for connecting systems with multiple cameras. Also relevant is the network’s ability to provide flexible computing, with paradigms such as the edge-cloud continuum. These characteristics of the network, together with the development of content capture and creation technologies, will generate the necessary adaptability to bring the production of this type of audiovisual content to different areas.

Content Distribution: new challenges and possibilities in user interaction

Production is only part of the process. Once the content exists, it needs to be delivered to users. In this regard, the experience of content delivery networks (CDNs) created for streaming video on the Internet is relevant, but insufficient. The distribution of immersive content, in general, and volumetric video, in particular, creates new challenges and possibilities. The user is not a mere spectator of the content, but can interact with it, change his or her point of view and move around with a certain freedom. Immersive content is also inherently multimedia, it can combine video and 3D objects, sound of different types, and also other sensory modalities, such as different forms of touch, and smell. These will therefore be complex multimodal experiences, which will require new methods of delivery, especially on a large scale.

Social interaction in XR: seeing, hearing, and touching other people?

Last, but a fundamental part of the experience, is the social interaction with other people. In the experiences of the future we will be able to see, hear, and even touch other people. We will also be able to represent ourselves in a variety of ways, building new expanded identities and having new augmented capabilities, taking advantage of the technological potential of socializing in virtual environments. The promise is a more natural interaction with other people in the same experience. Similar to what we would have face-to-face.

Communication networks in the future will provide a large part of the ingredients needed to enable these experiences. As with content production and distribution, interaction with others requires the sharing of complex data and media with low latency. In crowded events, it does not seem scalable that everyone can interact with everybody, networks will have to be able to adapt and distinguish which social interactions should be prioritized for a user. Moreover, social relationships can change over time, and the dynamicity of the network is essential. If the user experience is to be ubiquitous, and from different types of devices, the network will have to complement the capabilities of the users’ devices.

Social XR, Use Case in the framework of 6GDIFERENTE

6GDIFERENTE investigates the core technologies to address the above challenges. In terms of content production, the aim is to enable network-supported volumetric video recording. Using a mobile volumetric video capture system, with a number of cameras and hardware accessible to the majority of potential users, a set of tools for recording and publishing content on the network will be created. These developments use the capabilities of the network, such as dynamic resource allocation, vertical scaling of containers or millimeter wave interfaces, to provide the necessary communication and computational capabilities. Once recorded, the network will process the content to optimally match its distribution, and distribute it within the network in an appropriate manner.

With this approach, the use case is intended to be an ideal testbed for different technologies, including high-capacity and low-latency interfaces, monitoring and AI techniques, or edge-cloud continuum mechanisms.