Accessible Musical ALife Through LLM Co-Creation

Michael Clemens, Piotr Walas, Victor Shepardson, and Jack Armitage

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

GPU-accelerated artificial life (ALife) simulations present possibilities for musical expression, yet building them demands programming fluency that excludes many musicians and sound artists. We present a natural language interface that translates behavioral descriptions into executable particle simulations with integrated Open Sound Control (OSC) output and auto-generated SuperCollider companion scripts. Through a preliminary evaluation across three frontier models, we observe that structured context engineering appears necessary for reliable code synthesis, while providing models with extensive documentation alone yielded no improvement over unassisted generation. The pipeline's value lies in overcoming the cold start of generating domain specific languages, producing a first sketch for the artist to refine.

Citation

Michael Clemens, Piotr Walas, Victor Shepardson, and Jack Armitage. 2026. Accessible Musical ALife Through LLM Co-Creation. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.20784517 [PDF]

BibTeX Entry

@inproceedings{nime2026_170,
 abstract = {GPU-accelerated artificial life (ALife) simulations present possibilities for musical expression, yet building them demands programming fluency that excludes many musicians and sound artists. We present a natural language interface that translates behavioral descriptions into executable particle simulations with integrated Open Sound Control (OSC) output and auto-generated SuperCollider companion scripts. Through a preliminary evaluation across three frontier models, we observe that structured context engineering appears necessary for reliable code synthesis, while providing models with extensive documentation alone yielded no improvement over unassisted generation. The pipeline's value lies in overcoming the cold start of generating domain specific languages, producing a first sketch for the artist to refine.},
 address = {London, United Kingdom},
 articleno = {170},
 author = {Michael Clemens and Piotr Walas and Victor Shepardson and Jack Armitage},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.20784517},
 editor = {Benedict Gaster and João Tragtenberg and Anna Xambó and Tom Mitchell},
 issn = {2220-4806},
 month = {June},
 note = {},
 numpages = {4},
 pages = {1351--1354},
 presentation-video = {https://youtu.be/8hdLqyGU0uA},
 title = {Accessible Musical ALife Through LLM Co-Creation},
 track = {paper},
 url = {http://nime.org/proceedings/2026/nime2026_170.pdf},
 year = {2026}
}