e-baton: Recognising Conducting Gestures with Machine Learning

Marko Jeremic, and Akito Van Troyer

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

We present e-baton, a wireless conducting controller that uses an inertial measurement unit (IMU) and a user-retrainable kNN classifier to simultaneously control human performers and electronic musical parameters. During iterative development, we found that accelerometer data alone was insufficient for recognising slower, fluid conducting gestures, and that extending the feature set to include jerk and rotational data was necessary to support the gesture vocabulary used in performance. We evaluate the system through a five-participant tempo-tracking study and a debut performance with an improvising keyboardist.

Citation

Marko Jeremic, and Akito Van Troyer. 2026. e-baton: Recognising Conducting Gestures with Machine Learning. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.20784495 [PDF]

BibTeX Entry

@inproceedings{nime2026_163,
 abstract = {We present e-baton, a wireless conducting controller that uses an inertial measurement unit (IMU) and a user-retrainable kNN classifier to simultaneously control human performers and electronic musical parameters. During iterative development, we found that accelerometer data alone was insufficient for recognising slower, fluid conducting gestures, and that extending the feature set to include jerk and rotational data was necessary to support the gesture vocabulary used in performance. We evaluate the system through a five-participant tempo-tracking study and a debut performance with an improvising keyboardist.},
 address = {London, United Kingdom},
 articleno = {163},
 author = {Marko Jeremic and Akito Van Troyer},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.20784495},
 editor = {Benedict Gaster and João Tragtenberg and Anna Xambó and Tom Mitchell},
 issn = {2220-4806},
 month = {June},
 note = {},
 numpages = {4},
 pages = {1313--1316},
 title = {e-baton: Recognising Conducting Gestures with Machine Learning},
 track = {paper},
 url = {http://nime.org/proceedings/2026/nime2026_163.pdf},
 year = {2026}
}