Distributed Agency in Collaborative Improvisation with Intelligent Instruments: A Phenomenological Inquiry

Halla Stefánsdóttir, Robert Ek, and Thor Magnusson

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

Recent developments in artificial intelligence (AI) have been increasingly driven by technoscientific and corporate approaches that emphasise large-scale datasets and autonomous generation systems. In response, Human-Centred AI proposes an alternative framework foregrounding human agency, values, and creative control. This paper contributes to this discourse by examining collaborative performance with AI-augmented instruments through a practice-led experiment involving two musicians. In this way, we investigate how musical agency is distributed across humans and multiple AI systems. Conducted through a laboratory process leading up to a live performance, this project went through five phases: 1) introduction of interfaces, 2) curation of datasets, 3) training of neural audio synthesis models and applying corpus-based synthesis techniques, 4) working with the intelligent instruments in rehearsals and performance, and 5) analysing the outcomes. Through combining qualitative, phenomenologically grounded methods and practice-led artistic exploration, we identify emergent creative relationships between performers and AI-augmented instruments. By analysing the agency at play, we unpack how creative control is distributed between human and machine. Situating our work within the framework of professional collaborative performance, we address the lack of phenomenological research into intelligent instruments whilst contributing methodologies for accountable, artist-centred AI development in musical contexts.

Citation

Halla Stefánsdóttir, Robert Ek, and Thor Magnusson. 2026. Distributed Agency in Collaborative Improvisation with Intelligent Instruments: A Phenomenological Inquiry. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.20784207 [PDF]

BibTeX Entry

@inproceedings{nime2026_66,
 abstract = {Recent developments in artificial intelligence (AI) have been increasingly driven by technoscientific and corporate approaches that emphasise large-scale datasets and autonomous generation systems. In response, Human-Centred AI proposes an alternative framework foregrounding human agency, values, and creative control. This paper contributes to this discourse by examining collaborative performance with AI-augmented instruments through a practice-led experiment involving two musicians. In this way, we investigate how musical agency is distributed across humans and multiple AI systems. Conducted through a laboratory process leading up to a live performance, this project went through five phases: 1) introduction of interfaces, 2) curation of datasets, 3) training of neural audio synthesis models and applying corpus-based synthesis techniques, 4) working with the intelligent instruments in rehearsals and performance, and 5) analysing the outcomes. Through combining qualitative, phenomenologically grounded methods and practice-led artistic exploration, we identify emergent creative relationships between performers and AI-augmented instruments. By analysing the agency at play, we unpack how creative control is distributed between human and machine. Situating our work within the framework of professional collaborative performance, we address the lack of phenomenological research into intelligent instruments whilst contributing methodologies for accountable, artist-centred AI development in musical contexts.},
 address = {London, United Kingdom},
 articleno = {66},
 author = {Halla  Stefánsdóttir and Robert Ek and Thor Magnusson},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.20784207},
 editor = {Benedict Gaster and João Tragtenberg and Anna Xambó and Tom Mitchell},
 issn = {2220-4806},
 month = {June},
 note = {},
 numpages = {10},
 pages = {558--567},
 title = {Distributed Agency in Collaborative Improvisation with Intelligent Instruments: A Phenomenological Inquiry},
 track = {Paper},
 url = {http://nime.org/proceedings/2026/nime2026_66.pdf},
 year = {2026}
}