KinoGroove: Composing with Muscle and Motion in Extended Reality

Nathan Salin, Ronan Gaugne, Diane Haering, Valérie Gouranton, and Florent Berthaut

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

Extended reality and motion capture technologies have been explored as tools for embodied artistic performance, allowing dancers and musicians to interact with sound within immersive environments. However, motion capture systems primarily track gross motor gestures and solid body segments, which restricts their ability to represent dance styles that rely on muscular activation and subtle movement rather than large gestures. Electromyography (EMG), by contrast, captures muscular activity and can reflect finer body movements and soft tissue deformation that may not produce large-scale motion, yet are widely used in dance to convey expressive sensibility. In this work, we present KinoGroove, an extended reality interface that integrates EMG with motion capture to support expressive mapping of dance to sound. KinoGroove was developed through discussions with dancers and artists, involving close collaboration for testing, feedback, and data acquisition. Our system allows musicians to generate and control musical structures through both visible motion and internal effort, covering a wider range of dance expression. We introduce the KinoGroove design space, which formalizes how EMG signals can function as primary or complementary control parameters for musical output. Through two usage scenarios, one focusing on large-amplitude movements and the other on low-amplitude muscular activation, we illustrate how the system enables artists to shape sound through both spatial displacement and muscular intensity. We also describe methods for synchronized acquisition, processing, and visualization of motion and EMG data, highlighting its potential for creative, embodied interaction in immersive performance.

Citation

Nathan Salin, Ronan Gaugne, Diane Haering, Valérie Gouranton, and Florent Berthaut. 2026. KinoGroove: Composing with Muscle and Motion in Extended Reality. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.20784082 [PDF]

BibTeX Entry

@inproceedings{nime2026_20,
 abstract = {Extended reality and motion capture technologies have been explored as tools for embodied artistic performance, allowing dancers and musicians to interact with sound within immersive environments. However, motion capture systems primarily track gross motor gestures and solid body segments, which restricts their ability to represent dance styles that rely on muscular activation and subtle movement rather than large gestures. Electromyography (EMG), by contrast, captures muscular activity and can reflect finer body movements and soft tissue deformation that may not produce large-scale motion, yet are widely used in dance to convey expressive sensibility. In this work, we present KinoGroove, an extended reality interface that integrates EMG with motion capture to support expressive mapping of dance to sound. KinoGroove was developed through discussions with dancers and artists, involving close collaboration for testing, feedback, and data acquisition. Our system allows musicians to generate and control musical structures through both visible motion and internal effort, covering a wider range of dance expression. We introduce the KinoGroove design space, which formalizes how EMG signals can function as primary or complementary control parameters for musical output. Through two usage scenarios, one focusing on large-amplitude movements and the other on low-amplitude muscular activation, we illustrate how the system enables artists to shape sound through both spatial displacement and muscular intensity. We also describe methods for synchronized acquisition, processing, and visualization of motion and EMG data, highlighting its potential for creative, embodied interaction in immersive performance.},
 address = {London, United Kingdom},
 articleno = {20},
 author = {Nathan Salin and Ronan Gaugne and Diane Haering and Valérie Gouranton and Florent Berthaut},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.20784082},
 editor = {Benedict Gaster and João Tragtenberg and Anna Xambó and Tom Mitchell},
 issn = {2220-4806},
 month = {June},
 note = {},
 numpages = {11},
 pages = {175--185},
 presentation-video = {https://youtu.be/J-ko-8DIpCw},
 title = {KinoGroove: Composing with Muscle and Motion in Extended Reality},
 track = {Paper},
 url = {http://nime.org/proceedings/2026/nime2026_20.pdf},
 year = {2026}
}