Small Dynamic Neural Networks for Gesture Classification with The Rulers (a Digital Musical Instrument)

Vanessa Yaremchuk, Carolina Brum Medeiros, and Marcelo Wanderley

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

The Rulers is a Digital Musical Instrument with 7 metal beams, each of which is fixed at one end. It uses infrared sensors, Hall sensors, and strain gauges to estimate deflection. These sensors each perform better or worse depending on the class of gesture the user is making, motivating sensor fusion practices. Residuals between Kalman filters and sensor output are calculated and used as input to a recurrent neural network which outputs a classification that determines which processing parameters and sensor measurements are employed. Multiple instances (30) of layer recurrent neural networks with a single hidden layer varying in size from 1 to 10 processing units were trained, and tested on previously unseen data. The best performing neural network has only 3 hidden units and has a sufficiently low error rate to be good candidate for gesture classification. This paper demonstrates that: dynamic networks out-perform feedforward networks for this type of gesture classification, a small network can handle a problem of this level of complexity, recurrent networks of this size are fast enough for real-time applications of this type, and the importance of training multiple instances of each network architecture and selecting the best performing one from within that set.

Citation:

Vanessa Yaremchuk, Carolina Brum Medeiros, and Marcelo Wanderley. 2019. Small Dynamic Neural Networks for Gesture Classification with The Rulers (a Digital Musical Instrument). Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.3672904

BibTeX Entry:

  @inproceedings{Yaremchuk2019,
 abstract = {The Rulers is a Digital Musical Instrument with 7 metal beams, each of which is fixed at one end. It uses infrared sensors, Hall sensors, and strain gauges to estimate deflection. These sensors each perform better or worse depending on the class of gesture the user is making, motivating sensor fusion practices. Residuals between Kalman filters and sensor output are calculated and used as input to a recurrent neural network which outputs a classification that determines which processing parameters and sensor measurements are employed. Multiple instances (30) of layer recurrent neural networks with a single hidden layer varying in size from 1 to 10 processing units were trained, and tested on previously unseen data. The best performing neural network has only 3 hidden units and has a sufficiently low error rate to be good candidate for gesture classification. This paper demonstrates that: dynamic networks out-perform feedforward networks for this type of gesture classification, a small network can handle a problem of this level of complexity, recurrent networks of this size are fast enough for real-time applications of this type, and the importance of training multiple instances of each network architecture and selecting the best performing one from within that set.},
 address = {Porto Alegre, Brazil},
 author = {Vanessa Yaremchuk and Carolina Brum Medeiros and Marcelo Wanderley},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.3672904},
 editor = {Marcelo Queiroz and Anna Xambó Sedó},
 issn = {2220-4806},
 month = {June},
 pages = {150--155},
 publisher = {UFRGS},
 title = {Small Dynamic Neural Networks for Gesture Classification with The Rulers (a Digital Musical Instrument)},
 url = {http://www.nime.org/proceedings/2019/nime2019_paper030.pdf},
 year = {2019}
}