Screenless Optical Theremin with Tremolo (ScOTT)

Michael Gancz, Justin Berry, Shu Wei, Jake Shaker, Kimberly Hieftje, and Asher Marks

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

Head-mounted extended-reality (XR) interfaces provide a flexible platform for immersive and embodied musical instrument design. By combining spatial audio, ergonomic first-person gestural control, and networked interactivity, these interfaces can facilitate expressive, interesting, and emotionally resonant performances. Unlike data gloves and hyperinstruments, XR headsets can be calibrated to the physical properties of their individual users. However, these headsets tend to obscure the eyes and other parts of the face, limiting the user’s capacity to establish eye contact and transmit facial expressions. These subtle communicative elements play a crucial role in real-world collaborative musical settings where performers utilize facial cues to negotiate surface parameters such as timing, dynamics, and breath, as well as more complex qualities like atmosphere and interpretive mimesis. In this remote performance, we present the Screenless Optical Theremin with Tremolo (ScOTT), a novel gestural MIDI controller powered by a modified screenless XR headset. ScOTT focuses on the hands and arms, mapping broad gestures to coarse-grained musical parameters (e.g. pitch and velocity) and small movements to more complex musical ornaments (e.g. tremolo width and frequency). In a structured improvisation that highlights the role of ornament in musical texture, we explore the ScOTT’s capacity to balance social presence, embodied interaction, and expressivity.

Citation

Michael Gancz, Justin Berry, Shu Wei, Jake Shaker, Kimberly Hieftje, and Asher Marks. 2025. Screenless Optical Theremin with Tremolo (ScOTT). Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.17801140 [PDF]

BibTeX Entry

@inproceedings{nime2025_music_30,
 abstract = {Head-mounted extended-reality (XR) interfaces provide a flexible platform for immersive and embodied musical instrument design. By combining spatial audio, ergonomic first-person gestural control, and networked interactivity, these interfaces can facilitate expressive, interesting, and emotionally resonant performances. Unlike data gloves and hyperinstruments, XR headsets can be calibrated to the physical properties of their individual users. However, these headsets tend to obscure the eyes and other parts of the face, limiting the user’s capacity to establish eye contact and transmit facial expressions. These subtle communicative elements play a crucial role in real-world collaborative musical settings where performers utilize facial cues to negotiate surface parameters such as timing, dynamics, and breath, as well as more complex qualities like atmosphere and interpretive mimesis. In this remote performance, we present the Screenless Optical Theremin with Tremolo (ScOTT), a novel gestural MIDI controller powered by a modified screenless XR headset. ScOTT focuses on the hands and arms, mapping broad gestures to coarse-grained musical parameters (e.g. pitch and velocity) and small movements to more complex musical ornaments (e.g. tremolo width and frequency). In a structured improvisation that highlights the role of ornament in musical texture, we explore the ScOTT’s capacity to balance social presence, embodied interaction, and expressivity.},
 address = {Canberra, Australia},
 articleno = {30},
 author = {Michael Gancz and Justin Berry and Shu Wei and Jake Shaker and Kimberly Hieftje and Asher Marks},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.17801140},
 editor = {Sophie Rose and Jos Mulder and Nicole Carroll},
 issn = {2220-4806},
 month = {June},
 note = {Remote Performance},
 numpages = {4},
 pages = {114--117},
 title = {Screenless Optical Theremin with Tremolo (ScOTT)},
 track = {Music},
 url = {http://nime.org/proceedings/2025/nime2025_music_30.pdf},
 urlsuppl1 = {http://nime.org/proceedings/2025/nime2025_music_30_file01.mp4},
 year = {2025}
}