About

The vision of SHARESPACE is the creation of future Social Hybrid Spaces (SHS) shared by humans and avatars engaged in embodied collaborative tasks, where social sensorimotor primitives are transparently captured through mobile connected innovative sensors, and then reconstructed using novel extended reality (XR) technology. Our ambition is to create a hybrid, multimodal-multisensory integrated platform which adapts to individual users and enables them to interact in an embodied shared space by learning, identifying, and reconstructing the core sensorimotor primitives of social interactions. Through our cognitive architecture, virtual avatars will facilitate joint-action, distance-based learning, and social cohesion in the hybrid societies of the future, by reconstructing core social sensorimotor primitives into virtual ones. Our long-term vision is to expand the natural repertoire of sensorimotor interactions as we currently know it, with XR technology enriching key social features of sensorimotor communication tailored to the participants’ personal characteristics, with future deployment in sectors such as health, learning, entertainment, sport and at the workplace. In short, the project aims to bring XR to a radically, ethically sound, new level of presence and sociality by furthering the possibilities of interaction in a virtual space using avatars.

SHARESPACE will enable people in remote locations to interact in a shared sensorimotor space to prepare future inclusive hybrid societies. 

The vision presented above invites us to rethink how natural embodied social interactions can be transferred to the digital world, thus offering high-gain opportunities to develop new, inclusive and ethical multimodal communication media. Our overarching goal will be achieved through seven concrete and tightly intertwined objectives: 

Objective 1: Definition of embodied social interactions in XR. Conceptual, modelling, and experimental approaches rooted in (i) characterising a successful social interaction in SHS, (ii) identifying neuroscientific principles underlying the reconstruction of social sensorimotor primitives in SHS, and (iii) formalising core primitives. 

Objective 2: A novel fully mobile, un-obstructive, fine-grained, body kinematic capture system with on-body visual-inertial sensors and processing. Developing a mobile, reliable, and intuitive system, incorporating (i) self-calibration of ego-centric visual-inertial tracking, (ii) online motion prediction, segmentation, and primitive encoding, and (iii) hand skeleton tracking (from the XR-glasses camera sensors). 

Objective 3: Cognitive architectures for virtual avatars. Populating SHS with humans and avatars by exploiting a library of primitives and metrics for interaction, through (i) AI-based architectures to drive avatars with high efficiency, reliability, and different levels of autonomy, (ii) data-driven models to generate synthetic datasets to train avatars, and (iii) the multi-users, multimodal SHARESPACE communication platform. 

Objective 4: Virtual avatar animation. Rendering of remote participants onto virtual avatars in a highly personalised way by (i) respecting animation constraints, (ii) synchronising animations, (iii) mapping avatars in their environment, (iv) incorporating culture, gender, age, body types, (iv) real-time rendering through learned physics and (v) generating high-quality facial expressions out of audio (speech) and eye-tracking data. 

Objective 5: Spatial mobile eXtended Reality display. Visualising virtual avatars in (i) optical see-through multifocal XR-glasses with (ii) integrated eye-tracking and (iii) forward-oriented wide-angle camera for scene 6DoF localisation and hand detection. 

Objective 6: Socio-cultural constraints and ethics on social XR interactions. Evaluating the influence of (i) demographics, (ii) psychological and (iii) sociocultural factors in SHS experiences, to optimise interaction, reduce exclusion, preserve individualities and increase social intelligence, as well as mitigate possible negative effects of SHS. In line with commission objectives, we will also uncover ethical and legal impacts of our innovation. 

Objective 7: Real-world validation of our embodied social hybrid space. Implementing and testing our integrated solutions amplifying sensorimotor communication in SHS in three different but complementary real-world scenarios: Health (Social Low Back Pain Exergame), Sport (Family Peloton Cycling), and Art (Shared Creativity). 

In the pursuit of the completion of these objectives the Sharespace project upholds a strict Ethics by Design approach. This approach intends to prevent ethical issues from arising in the first place by addressing them during the development stage, rather than trying to fix them later in the process.