VRMin: Using Mixed Reality to Augment the Theremin for Musical Tutoring

David Johnson, and George Tzanetakis

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

The recent resurgence of Virtual Reality (VR) technologies provide new platforms for augmenting traditional music instruments. Instrument augmentation is a common approach for designing new interfaces for musical expression, as shown through hyperinstrument research. New visual affordances present in VR give designers new methods for augmenting instruments to extend not only their expressivity, but also their capabilities for computer assisted tutoring. In this work, we present VRMin, a mobile Mixed Reality (MR) application for augmenting a physical theremin, with an immersive virtual environment (VE), for real time computer assisted tutoring. We augment a physical theremin with 3D visual cues to indicate correct hand positioning for performing given notes and volumes. The physical theremin acts as a domain specific controller for the resulting MR environment. The initial effectiveness of this approach is measured by analyzing a performer's hand position while training with and without the VRMin. We also evaluate the usability of the interface using heuristic evaluation based on a newly proposed set of guidelines designed for VR musical environments.

Citation:

David Johnson, and George Tzanetakis. 2017. VRMin: Using Mixed Reality to Augment the Theremin for Musical Tutoring. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1176205

BibTeX Entry:

  @inproceedings{djohnson2017,
 abstract = {The recent resurgence of Virtual Reality (VR) technologies provide new platforms for augmenting traditional music instruments. Instrument augmentation is a common approach for designing new interfaces for musical expression, as shown through hyperinstrument research.  New visual affordances present in VR give designers new methods for augmenting instruments to extend not only their expressivity, but also their capabilities for computer assisted tutoring. In this work, we present VRMin, a mobile Mixed Reality (MR) application for augmenting a physical theremin, with an immersive virtual environment (VE), for real time computer assisted tutoring. We augment a physical theremin with 3D visual cues to indicate correct hand positioning for performing given notes and volumes.  The physical theremin acts as a domain specific controller for the resulting MR environment.  The initial effectiveness of this approach is measured by analyzing a performer's hand position while training with and without the VRMin. We also evaluate the usability of the interface using heuristic evaluation based on a newly proposed set of guidelines designed for VR musical environments.},
 address = {Copenhagen, Denmark},
 author = {David Johnson and George Tzanetakis},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1176205},
 issn = {2220-4806},
 pages = {151--156},
 publisher = {Aalborg University Copenhagen},
 title = {VRMin: Using Mixed Reality to Augment the Theremin for Musical Tutoring},
 url = {http://www.nime.org/proceedings/2017/nime2017_paper0029.pdf},
 year = {2017}
}