A Framework for the Design and Analysis of Mixed Reality Musical Instruments

Karitta Christina Zellerbach, and Charlie Roberts

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

In the context of immersive sonic interaction, Virtual Reality Musical Instruments have had the relative majority of attention thus far, fueled by the increasing availability of affordable technology. Recent advances in Mixed Reality (MR) experiences have provided the means for a new wave of research that goes beyond Virtual Reality. In this paper, we explore the taxonomy of Extended Reality systems, establishing our own notion of MR. From this, we propose a new classification of Virtual Musical Instrument, known as a Mixed Reality Musical Instrument (MRMI). We define this system as an embodied interface for expressive musical performance, characterized by the relationships between the performer, the virtual, and the physical environment. After a review of existing literature concerning the evaluation of immersive musical instruments and the affordances of MR systems, we offer a new framework based on three dimensions to support the design and analysis of MRMIs. We illustrate its use with application to existing works.

Citation:

Karitta Christina Zellerbach, and Charlie Roberts. 2022. A Framework for the Design and Analysis of Mixed Reality Musical Instruments. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.21428/92fbeb44.b2a44bc9

BibTeX Entry:

  @inproceedings{NIME22_29,
 abstract = {In the context of immersive sonic interaction, Virtual Reality Musical Instruments have had the relative majority of attention thus far, fueled by the increasing availability of affordable technology. Recent advances in Mixed Reality (MR) experiences have provided the means for a new wave of research that goes beyond Virtual Reality. In this paper, we explore the taxonomy of Extended Reality systems, establishing our own notion of MR. From this, we propose a new classification of Virtual Musical Instrument, known as a Mixed Reality Musical Instrument (MRMI). We define this system as an embodied interface for expressive musical performance, characterized by the relationships between the performer, the virtual, and the physical environment. After a review of existing literature concerning the evaluation of immersive musical instruments and the affordances of MR systems, we offer a new framework based on three dimensions to support the design and analysis of MRMIs. We illustrate its use with application to existing works.},
 address = {The University of Auckland, New Zealand},
 articleno = {29},
 author = {Zellerbach, Karitta Christina and Roberts, Charlie},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.21428/92fbeb44.b2a44bc9},
 issn = {2220-4806},
 month = {jun},
 pdf = {193.pdf},
 presentation-video = {https://youtu.be/Pb4pAr2v4yU},
 title = {A Framework for the Design and Analysis of Mixed Reality Musical Instruments},
 url = {https://doi.org/10.21428%2F92fbeb44.b2a44bc9},
 year = {2022}
}