Phalanger : Controlling Music Software With Hand Movement Using A Computer Vision and Machine Learning Approach

Chris Kiefer, Nick Collins, and Geraldine Fitzpatrick

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

Phalanger is a system which facilitates the control of music software with hand and finger motion, with the aim of creating a fluid style of interaction that promotes musicality. The system is purely video based, requires no wearables or accessories and uses affordable and accessible technology. It employs a neural network for background segmentation, a combination of imaging techniques for frame analysis, and a support vector machine (SVM) for recognition of hand positions. System evaluation showed the SVM to reliably differentiate between eight different classes. An initial formative user evaluation with ten musicians was carried out to help build a picture of how users responded to the system; this highlighted areas that need improvement and lent some insight into useful features for the next version.

Citation:

Chris Kiefer, Nick Collins, and Geraldine Fitzpatrick. 2009. Phalanger : Controlling Music Software With Hand Movement Using A Computer Vision and Machine Learning Approach. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1177603

BibTeX Entry:

  @inproceedings{Kiefer2009,
 abstract = {Phalanger is a system which facilitates the control of music software with hand and finger motion, with the aim of creating a fluid style of interaction that promotes musicality. The system is purely video based, requires no wearables or accessories and uses affordable and accessible technology. It employs a neural network for background segmentation, a combination of imaging techniques for frame analysis, and a support vector machine (SVM) for recognition of hand positions. System evaluation showed the SVM to reliably differentiate between eight different classes. An initial formative user evaluation with ten musicians was carried out to help build a picture of how users responded to the system; this highlighted areas that need improvement and lent some insight into useful features for the next version.},
 address = {Pittsburgh, PA, United States},
 author = {Kiefer, Chris and Collins, Nick and Fitzpatrick, Geraldine},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1177603},
 issn = {2220-4806},
 keywords = {nime09},
 pages = {246--249},
 title = {Phalanger : Controlling Music Software With Hand Movement Using A Computer Vision and Machine Learning Approach},
 url = {http://www.nime.org/proceedings/2009/nime2009_246.pdf},
 year = {2009}
}