A Self-Organizing Gesture Map for a Voice-Controlled Instrument Interface

Stefano Fasciani, and Lonce Wyse

Proceedings of the International Conference on New Interfaces for Musical Expression

• Year: 2013
• Location: Daejeon, Republic of Korea
• Pages: 507–512
• Keywords: Self-Organizing Maps, Gestural Controller, Multi Dimensional Control, Unsupervised Gesture Mapping, Voice Control
• DOI: 10.5281/zenodo.4582292 (Link to paper)
• PDF link

Abstract:

Mapping gestures to digital musical instrument parameters is not trivial when the dimensionality of the sensor-captured data is high and the model relating the gesture to sensor data is unknown. In these cases, a front-end processing system for extracting gestural information embedded in the sensor data is essential. In this paper we propose an unsupervised offline method that learns how to reduce and map the gestural data to a generic instrument parameter control space. We make an unconventional use of the Self-Organizing Maps to obtain only a geometrical transformation of the gestural data, while dimensionality reduction is handled separately. We introduce a novel training procedure to overcome two main Self-Organizing Maps limitations which otherwise corrupt the interface usability. As evaluation, we apply this method to our existing Voice-Controlled Interface for musical instruments, obtaining sensible usability improvements.

Citation:

BibTeX Entry:

  @inproceedings{Fasciani2013,
 abstract = {Mapping gestures to digital musical instrument parameters is not trivial when the dimensionality of the sensor-captured data is high and the model relating the gesture to sensor data is unknown. In these cases, a front-end processing system for extracting gestural information embedded in the sensor data is essential. In this paper we propose an unsupervised offline method that learns how to reduce and map the gestural data to a generic instrument parameter control space. We make an unconventional use of the Self-Organizing Maps to obtain only a geometrical transformation of the gestural data, while dimensionality reduction is handled separately. We introduce a novel training procedure to overcome two main Self-Organizing Maps limitations which otherwise corrupt the interface usability. As evaluation, we apply this method to our existing Voice-Controlled Interface for musical instruments, obtaining sensible usability improvements.},
 address = {Daejeon, Republic of Korea},
 author = {Stefano Fasciani and Lonce Wyse},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.4582292},
 issn = {2220-4806},
 keywords = {Self-Organizing Maps, Gestural Controller, Multi Dimensional Control, Unsupervised Gesture Mapping, Voice Control},
 month = {May},
 pages = {507--512},
 publisher = {Graduate School of Culture Technology, KAIST},
 title = {A Self-Organizing Gesture Map for a Voice-Controlled Instrument Interface},
 url = {http://www.nime.org/proceedings/2013/nime2013_50.pdf},
 year = {2013}
}