Augmented Piano Performance using a Depth Camera

Qi Yang, and Georg Essl

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

We augment the piano keyboard with a 3D gesture space using Microsoft Kinect for sensing and top-down projection for visual feedback. This interface provides multi-axial gesture controls to enable continuous adjustments to multiple acoustic parameters such as those on the typical digital synthesizers. We believe that using gesture control is more visceral and aesthetically pleasing, especially during concert performance where the visibility of the performer's action is important. Our system can also be used for other types of gesture interaction as well as for pedagogical applications.

Citation:

Qi Yang, and Georg Essl. 2012. Augmented Piano Performance using a Depth Camera. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1178455

BibTeX Entry:

  @inproceedings{Yang2012,
 abstract = {We augment the piano keyboard with a 3D gesture space using Microsoft Kinect for sensing and top-down projection for visual feedback. This interface provides multi-axial gesture controls to enable continuous adjustments to multiple acoustic parameters such as those on the typical digital synthesizers. We believe that using gesture control is more visceral and aesthetically pleasing, especially during concert performance where the visibility of the performer's action is important. Our system can also be used for other types of gesture interaction as well as for pedagogical applications.},
 address = {Ann Arbor, Michigan},
 author = {Qi Yang and Georg Essl},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1178455},
 issn = {2220-4806},
 keywords = {NIME, piano, depth camera, musical instrument, gesture, tabletop projection},
 publisher = {University of Michigan},
 title = {Augmented Piano Performance using a Depth Camera},
 url = {http://www.nime.org/proceedings/2012/nime2012_203.pdf},
 year = {2012}
}