Luna: An AR Musical Instrument on the Meta Quest 2

Samuel Dietz, and Charles Martin

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

Head-mounted augmented reality (AR) computers present the opportunity to develop new musical interfaces that would be impossible to build physically or with conventional computing devices. Unfortunately, typical computer music tools have not been easy to apply within AR development tool chains. Integrating standard computer music tools in AR development would allow more rapid prototyping of new instrument ideas and transfer of knowledge from experienced computer musicians. The goal of this paper is to demonstrate that AR digital musical instruments can be developed using libpd, the library version of the standard computer music environment Pure Data. We present a case study of an AR instrument developed for the Meta Quest 2 integrating libpd in the AR development tool-chain for the interactive audio components. The iterative development process was tracked through autoethnographic reflections and analysed with thematic analysis. We found that Pure Data was an effective way to develop audio interactions on the Quest 2 and that the hand tracking on this platform was capable of complex gestural interactions. This work could enable a broader community of computer musicians to explore AR NIME development, taking advantage of the unique affordances of this medium.

Citation

Samuel Dietz, and Charles Martin. 2025. Luna: An AR Musical Instrument on the Meta Quest 2. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.15698970 [PDF]

BibTeX Entry

@article{nime2025_86,
 abstract = {Head-mounted augmented reality (AR) computers present the opportunity to develop new musical interfaces that would be impossible to build physically or with conventional computing devices. Unfortunately, typical computer music tools have not been easy to apply within AR development tool chains. Integrating standard computer music tools in AR development would allow more rapid prototyping of new instrument ideas and transfer of knowledge from experienced computer musicians. The goal of this paper is to demonstrate that AR digital musical instruments can be developed using libpd, the library version of the standard computer music environment Pure Data. We present a case study of an AR instrument developed for the Meta Quest 2 integrating libpd in the AR development tool-chain for the interactive audio components. The iterative development process was tracked through autoethnographic reflections and analysed with thematic analysis. We found that Pure Data was an effective way to develop audio interactions on the Quest 2 and that the hand tracking on this platform was capable of complex gestural interactions. This work could enable a broader community of computer musicians to explore AR NIME development, taking advantage of the unique affordances of this medium.},
 address = {Canberra, Australia},
 articleno = {86},
 author = {Samuel Dietz and Charles Martin},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.15698970},
 editor = {Doga Cavdir and Florent Berthaut},
 issn = {2220-4806},
 month = {June},
 numpages = {4},
 pages = {590--593},
 title = {Luna: An AR Musical Instrument on the Meta Quest 2},
 track = {Paper},
 url = {http://nime.org/proceedings/2025/nime2025_86.pdf},
 year = {2025}
}