VoluAI - Volumetric Video on the Web
VOLUAI
Context
VOLUAI is a project of the Multi-Area Internal Projects - Call 2022. It is a project led by the Digital Media team in collaboration with the Industry and Advanced Manufacturing team.
The aim of the project has been to develop the technology to capture, transmit, enhance, analyze and render volumetric representations of people. More specifically, the first objective has been to develop an end-to-end platform for capturing, storing, distributing and rendering volumetric video; and the second, to develop a library for analyzing and enhancing volumetric video.
Challenges:
The challenges faced at VoluAI have been the following:
- Produce quality content with affordable resources
- Capture, transmit and render in real time.
- Achieve a flexible and upgradeable system
- Microservices architecture, ready to be deployed in 5G/6G networks.
- Be oriented to web technologies
- Based on pointcloud
- Content analysis
- Content enhancement
Technical Results:
Validation of this technology has been/will be performed using two demonstrators:
- A concert / holoconference
- The monitoring of a worker's ergonomics.
Holoconference:
Features:
- Flexible system
- Simplicity of deployment
- Stable
- Cross-device support in both capture and rendering
- Compression
- Real-time system
- Web-ready system
- 5G-ready system
Pipeline phases:
- Capture
- Calibration
- Pointcloud generation
- Encoding with Draco
- Distribution with socket.io
- Rendering: web, unity, android, opengl
- Monitoring of a worker's ergonomics
The monitoring of a worker's ergonomics
Advanced machine learning algorithms focused on point cloud data enhancement and human posture detection and classification have been developed and tested.
In one of the use cases, the postures of a warehouse worker are captured as point clouds, enhanced and classified as ergonomic or non-ergonomic.
The entire process of machine learning algorithms, including the training of the classification model, was carried out using the machine learning process builder we developed, which can be applied to a wide range of use cases.
Scientific Results
Paper Web 3D: Volumetric Video on the Web: a platform prototype and empirical study. Sergio Cabrero, Ander Elosegi, Iñigo Tamayo, Ana Domínguez, Mikel Zorrilla
Looking for support for your next project? Contact us, we are looking forward to helping you.
