Decoupling Physical and Virtual Spaces for New Collaboration Strategies in Co-Located Mixed Reality Instruments

Pierrick Uro, Florent Berthaut, Thomas Pietrzak, and Marcelo Wanderley

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

Collaborative co-located Mixed Reality musical instruments combine some of the expressive opportunities of 3D interaction and communication and cooperation of physical multi-user instruments. However in existing instruments, the fixed coupling between the virtual and physical environments constrains the affordances brought by Mixed Reality, such as per-musician free navigation in or multi-scale control of virtual structures.We designed gRAinyCloud, as a way to reintegrate these lost affordances to a co-located instrument.It allows for the expressive exploration of a set of sounds represented by a virtual structure of shapes placed in the physical space and shared between musicians. Above all, gRAinyCloud enables each musician to freely manipulate their own viewpoint, changing its scale, position and rotation, effectively decoupling the physical and virtual spaces, and to switch between self, other's and absolute viewpoint while playing. We describe the implementation of this decoupling of spaces and analyse its uses and implications for collective musical expression, by relying on a first-person approach.

Citation

Pierrick Uro, Florent Berthaut, Thomas Pietrzak, and Marcelo Wanderley. 2025. Decoupling Physical and Virtual Spaces for New Collaboration Strategies in Co-Located Mixed Reality Instruments. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.15698823 [PDF]

BibTeX Entry

@article{nime2025_23,
 abstract = {Collaborative co-located Mixed Reality musical instruments combine some of the expressive opportunities of 3D interaction and communication and cooperation of physical multi-user instruments. However in existing instruments, the fixed coupling between the virtual and physical environments constrains the affordances brought by Mixed Reality, such as per-musician free navigation in or multi-scale control of virtual structures.We designed gRAinyCloud, as a way to reintegrate these lost affordances to a co-located instrument.It allows for the expressive exploration of a set of sounds represented by a virtual structure of shapes placed in the physical space and shared between musicians. Above all, gRAinyCloud enables each musician to freely manipulate their own viewpoint, changing its scale, position and rotation, effectively decoupling the physical and virtual spaces, and to switch between self, other's and absolute viewpoint while playing. We describe the implementation of this decoupling of spaces and analyse its uses and implications for collective musical expression, by relying on a first-person approach.},
 address = {Canberra, Australia},
 articleno = {23},
 author = {Pierrick Uro and Florent Berthaut and Thomas Pietrzak and Marcelo Wanderley},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.15698823},
 editor = {Doga Cavdir and Florent Berthaut},
 issn = {2220-4806},
 month = {June},
 numpages = {6},
 pages = {161--166},
 title = {Decoupling Physical and Virtual Spaces for New Collaboration Strategies in Co-Located Mixed Reality Instruments},
 track = {Paper},
 url = {http://nime.org/proceedings/2025/nime2025_23.pdf},
 year = {2025}
}