Skip to content

A Self-Organizing Gesture Map for a Voice-Controlled Instrument Interface

Stefano Fasciani, and Lonce Wyse

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

Mapping gestures to digital musical instrument parameters is not trivial when the dimensionality of the sensor-captured data is high and the model relating the gesture to sensor data is unknown. In these cases, a front-end processing system for extracting gestural information embedded in the sensor data is essential. In this paper we propose an unsupervised offline method that learns how to reduce and map the gestural data to a generic instrument parameter control space. We make an unconventional use of the Self-Organizing Maps to obtain only a geometrical transformation of the gestural data, while dimensionality reduction is handled separately. We introduce a novel training procedure to overcome two main Self-Organizing Maps limitations which otherwise corrupt the interface usability. As evaluation, we apply this method to our existing Voice-Controlled Interface for musical instruments, obtaining sensible usability improvements.

Citation

Stefano Fasciani, and Lonce Wyse. 2013. A Self-Organizing Gesture Map for a Voice-Controlled Instrument Interface. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.4582292 [PDF]

BibTeX Entry


@inproceedings{nime2013_Fasciani,
author = {Stefano Fasciani and Lonce Wyse},
title = {A Self-Organizing Gesture Map for a Voice-Controlled Instrument Interface},
pages = {507--512},
booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
year = {2013},
month = {May},
publisher = {Graduate School of Culture Technology, KAIST},
address = {Daejeon, Republic of Korea},
issn = {2220-4806},
doi = {10.5281/zenodo.4582292},
url = {https://nime.org/proceedings/2013/nime2013_50.pdf},
keywords = {Self-Organizing Maps, Gestural Controller, Multi Dimensional Control, Unsupervised Gesture Mapping, Voice Control},
abstract = {Mapping gestures to digital musical instrument parameters is not trivial when the dimensionality of the sensor-captured data is high and the model relating the gesture to sensor data is unknown. In these cases, a front-end processing system for extracting gestural information embedded in the sensor data is essential. In this paper we propose an unsupervised offline method that learns how to reduce and map the gestural data to a generic instrument parameter control space. We make an unconventional use of the Self-Organizing Maps to obtain only a geometrical transformation of the gestural data, while dimensionality reduction is handled separately. We introduce a novel training procedure to overcome two main Self-Organizing Maps limitations which otherwise corrupt the interface usability. As evaluation, we apply this method to our existing Voice-Controlled Interface for musical instruments, obtaining sensible usability improvements.}
}