Digito: A Fine-Grain Gesturally Controlled Virtual Musical Instrument

Nicholas Gillian, and Joseph A. Paradiso

Proceedings of the International Conference on New Interfaces for Musical Expression

• Year: 2012
• Location: Ann Arbor, Michigan
• Keywords: Gesture Recognition, Virtual Musical Instrument
• DOI: 10.5281/zenodo.1178263 (Link to paper)
• PDF link

Abstract:

This paper presents Digito, a gesturally controlled virtual musical instrument. Digito is controlled through a number of intricate hand gestures, providing both discrete and continuous control of Digito's sound engine; with the fine-grain hand gestures captured by a 3D depth sensor and recognized using computer vision and machine learning algorithms. We describe the design and initial iterative development of Digito, the hand and finger tracking algorithms and gesture recognition algorithms that drive the system, and report the insights gained during the initial development cycles and user testing of this gesturally controlled virtual musical instrument.

Citation:

BibTeX Entry:

  @inproceedings{Gillian2012,
 abstract = {This paper presents Digito, a gesturally controlled virtual musical instrument. Digito is controlled through a number of intricate hand gestures, providing both discrete and continuous control of Digito's sound engine; with the fine-grain hand gestures captured by a 3D depth sensor and recognized using computer vision and machine learning algorithms. We describe the design and initial iterative development of Digito, the hand and finger tracking algorithms and gesture recognition algorithms that drive the system, and report the insights gained during the initial development cycles and user testing of this gesturally controlled virtual musical instrument.},
 address = {Ann Arbor, Michigan},
 author = {Nicholas Gillian and Joseph A. Paradiso},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1178263},
 issn = {2220-4806},
 keywords = {Gesture Recognition, Virtual Musical Instrument},
 publisher = {University of Michigan},
 title = {Digito: A Fine-Grain Gesturally Controlled Virtual Musical Instrument},
 url = {http://www.nime.org/proceedings/2012/nime2012_248.pdf},
 year = {2012}
}