Tuneable Machine Learning in Musical Instruments: A Duoethnography

Chris Kiefer, Adam Staff, and Andrea Martelloni

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

Results are presented from a duoethnographic study of musical practice with instruments using embedded, tuneable, machine learning (ML); tuneable because the musicians can optimise machine learning models within the instrument, and embedded because these processes are built into the interface and processes of the instrument. Two researchers investigated their systems through their individual musical practices, and juxtaposed their experiences in a duoethnographic report. Both instruments are based on the MEMLNaut hardware and NISPS ML system, which offer embedded reinforcement learning for the design of mappings. The first system is an instrument for creative studio production, and the second is a signal processor embedded within a string feedback instrument. The design of these systems is explained, followed by interviews between the two researchers. Analysis of these interviews highlights key issues in musical practice with tunable ML, concerning the creative opportunities presented by new workflows, challenges in engaging with reinforcement learning processes, the design of biases in ML systems, and strategies that embrace or mitigate the approximate nature of creative ML.

Citation

Chris Kiefer, Adam Staff, and Andrea Martelloni. 2026. Tuneable Machine Learning in Musical Instruments: A Duoethnography. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.20784190 [PDF]

BibTeX Entry

@inproceedings{nime2026_59,
 abstract = {Results are presented from a duoethnographic study of musical practice with instruments using embedded, tuneable, machine learning (ML); tuneable because the musicians can optimise machine learning models within the instrument, and embedded because these processes are built into the interface and processes of the instrument. Two researchers investigated their systems through their individual musical practices, and juxtaposed their experiences in a duoethnographic report.  Both instruments are based on the MEMLNaut hardware and NISPS ML system, which offer embedded reinforcement learning for the design of mappings. The first system is an instrument for creative studio production, and the second is a signal processor embedded within a string feedback instrument.  The design of these systems is explained, followed by interviews between the two researchers.  Analysis of these interviews highlights key issues in musical practice with tunable ML, concerning the creative opportunities presented by new workflows, challenges in engaging with reinforcement learning processes, the design of biases in ML systems, and strategies that embrace or mitigate the approximate nature of creative ML.},
 address = {London, United Kingdom},
 articleno = {59},
 author = {Chris Kiefer and Adam Staff and Andrea Martelloni},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.20784190},
 editor = {Benedict Gaster and João Tragtenberg and Anna Xambó and Tom Mitchell},
 issn = {2220-4806},
 month = {June},
 note = {},
 numpages = {9},
 pages = {500--508},
 title = {Tuneable Machine Learning in Musical Instruments: A Duoethnography},
 track = {Paper},
 url = {http://nime.org/proceedings/2026/nime2026_59.pdf},
 year = {2026}
}