A Voice Interface for Sound Generators: adaptive and automatic mapping of gestures to sound

STEFANO FASCIANI, and LONCE WYSE

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

Sound generators and synthesis engines expose a large set of parameters, allowing run-time timbre morphing and exploration of sonic space. However, control over these high-dimensional interfaces is constrained by the physical limitations of performers. In this paper we propose the exploitation of vocal gesture as an extension or alternative to traditional physical controllers. The approach uses dynamic aspects of vocal sound to control variations in the timbre of the synthesized sound. The mapping from vocal to synthesis parameters is automatically adapted to information extracted from vocal examples as well as to the relationship between parameters and timbre within the synthesizer. The mapping strategy aims to maximize the breadth of the explorable perceptual sonic space over a set of the synthesizer's real-valued parameters, indirectly driven by the voice-controlled interface.

Citation:

STEFANO FASCIANI, and LONCE WYSE. 2012. A Voice Interface for Sound Generators: adaptive and automatic mapping of gestures to sound. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1178251

BibTeX Entry:

  @inproceedings{FASCIANI2012,
 abstract = {Sound generators and synthesis engines expose a large set of parameters, allowing run-time timbre morphing and exploration of sonic space. However, control over these high-dimensional interfaces is constrained by the physical limitations of performers. In this paper we propose the exploitation of vocal gesture as an extension or alternative to traditional physical controllers. The approach uses dynamic aspects of vocal sound to control variations in the timbre of the synthesized sound. The mapping from vocal to synthesis parameters is automatically adapted to information extracted from vocal examples as well as to the relationship between parameters and timbre within the synthesizer. The mapping strategy aims to maximize the breadth of the explorable perceptual sonic space over a set of the synthesizer's real-valued parameters, indirectly driven by the voice-controlled interface.},
 address = {Ann Arbor, Michigan},
 author = {STEFANO FASCIANI and LONCE WYSE},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1178251},
 issn = {2220-4806},
 keywords = {Voice Control, Adaptive Interface, Automatic Mapping, Timbre Morphing, Sonic Space Exploration},
 publisher = {University of Michigan},
 title = {A Voice Interface for Sound Generators: adaptive and automatic mapping of gestures to sound},
 url = {http://www.nime.org/proceedings/2012/nime2012_57.pdf},
 year = {2012}
}