Deep Drawing: Performance Surface Sound Source Localization

Lennon Seiders, Julie Zhu, John Granzow, Alex Zhang, and Anusha Chinthamaduka

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

Deep Drawing is an ongoing exploration of the sound of drawing through AI co-performance. As the sounds of a performer’s drawing gestures resonate through a wooden board, they are re-created, spatially localized, and visualized in real-time using a novel deep learning approach. The system foregrounds the often-overlooked, timbrally complex sounds of drawing and frames them as a shared interpretive space between human and machine. This paper presents an accessible hardware setup and a sound source localization (SSL) model that can be trained with minimal data, enabling expressive interaction without extensive calibration or large datasets. Because SSL for performance-sized surfaces is a relatively unexplored research topic, we introduce practical techniques for this setting, including high-pass filtering, data augmentation, and high fidelity data capture. Deep Drawing contributes a replicable system that emphasizes performance, embodiment, and co-creative agency.

Citation

Lennon Seiders, Julie Zhu, John Granzow, Alex Zhang, and Anusha Chinthamaduka. 2026. Deep Drawing: Performance Surface Sound Source Localization. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.20784503 [PDF]

BibTeX Entry

@inproceedings{nime2026_166,
 abstract = {Deep Drawing is an ongoing exploration of the sound of drawing through AI co-performance. As the sounds of a performer’s drawing gestures resonate through a wooden board, they are re-created, spatially localized, and visualized in real-time using a novel deep learning approach. The system foregrounds the often-overlooked, timbrally complex sounds of drawing and frames them as a shared interpretive space between human and machine. This paper presents an accessible hardware setup and a sound source localization (SSL) model that can be trained with minimal data, enabling expressive interaction without extensive calibration or large datasets. Because SSL for performance-sized surfaces is a relatively unexplored research topic, we introduce practical techniques for this setting, including high-pass filtering, data augmentation, and high fidelity data capture. Deep Drawing contributes a replicable system that emphasizes performance, embodiment, and co-creative agency.},
 address = {London, United Kingdom},
 articleno = {166},
 author = {Lennon Seiders and Julie Zhu and John Granzow and Alex Zhang and Anusha Chinthamaduka},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.20784503},
 editor = {Benedict Gaster and João Tragtenberg and Anna Xambó and Tom Mitchell},
 issn = {2220-4806},
 month = {June},
 note = {},
 numpages = {5},
 pages = {1330--1334},
 title = {Deep Drawing: Performance Surface Sound Source Localization},
 track = {paper},
 url = {http://nime.org/proceedings/2026/nime2026_166.pdf},
 year = {2026}
}