Tracking Ensemble Performance on Touch-Screens with Gesture Classification and Transition Matrices

Charles Martin, Henry Gardner, and Ben Swift

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

We present and evaluate a novel interface for tracking ensemble performances on touch-screens. The system uses a Random Forest classifier to extract touch-screen gestures and transition matrix statistics. It analyses the resulting gesture-state sequences across an ensemble of performers. A series of specially designed iPad apps respond to this real-time analysis of free-form gestural performances with calculated modifications to their musical interfaces. We describe our system and evaluate it through cross-validation and profiling as well as concert experience.

Citation

Charles Martin, Henry Gardner, and Ben Swift. 2015. Tracking Ensemble Performance on Touch-Screens with Gesture Classification and Transition Matrices. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1179130

BibTeX Entry

@inproceedings{cmartin2015,
 abstract = {We present and evaluate a novel interface for tracking ensemble performances on touch-screens. The system uses a Random Forest classifier to extract touch-screen gestures and transition matrix statistics. It analyses the resulting gesture-state sequences across an ensemble of performers. A series of specially designed iPad apps respond to this real-time analysis of free-form gestural performances with calculated modifications to their musical interfaces. We describe our system and evaluate it through cross-validation and profiling as well as concert experience.},
 address = {Baton Rouge, Louisiana, USA},
 author = {Charles Martin and Henry Gardner and Ben Swift},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1179130},
 editor = {Edgar Berdahl and Jesse Allison},
 issn = {2220-4806},
 month = {May},
 pages = {359--364},
 publisher = {Louisiana State University},
 title = {Tracking Ensemble Performance on Touch-Screens with Gesture Classification and Transition Matrices},
 url = {http://www.nime.org/proceedings/2015/nime2015_242.pdf},
 year = {2015}
}