Designing, Playing, and Performing with a Vision-based Mouth Interface

Michael J. Lyons, Michael Haehnel, and Nobuji Tetsutani

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

The role of the face and mouth in speech production as well asnon-verbal communication suggests the use of facial action tocontrol musical sound. Here we document work on theMouthesizer, a system which uses a headworn miniaturecamera and computer vision algorithm to extract shapeparameters from the mouth opening and output these as MIDIcontrol changes. We report our experience with variousgesture-to-sound mappings and musical applications, anddescribe a live performance which used the Mouthesizerinterface.

Citation:

Michael J. Lyons, Michael Haehnel, and Nobuji Tetsutani. 2003. Designing, Playing, and Performing with a Vision-based Mouth Interface. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1176529

BibTeX Entry:

  @inproceedings{Lyons2003,
 abstract = {The role of the face and mouth in speech production as well asnon-verbal communication suggests the use of facial action tocontrol musical sound. Here we document work on theMouthesizer, a system which uses a headworn miniaturecamera and computer vision algorithm to extract shapeparameters from the mouth opening and output these as MIDIcontrol changes. We report our experience with variousgesture-to-sound mappings and musical applications, anddescribe a live performance which used the Mouthesizerinterface.},
 address = {Montreal, Canada},
 author = {Lyons, Michael J. and Haehnel, Michael and Tetsutani, Nobuji},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 date = {22-24 May, 2003},
 doi = {10.5281/zenodo.1176529},
 issn = {2220-4806},
 keywords = {Video-based interface; mouth controller; alternative input devices. },
 pages = {116--121},
 title = {Designing, Playing, and Performing with a Vision-based Mouth Interface},
 url = {http://www.nime.org/proceedings/2003/nime2003_116.pdf},
 year = {2003}
}