Stringesthesia: Dynamically Shifting Musical Agency Between Audience and Performer Based on Trust in an Interactive and Improvised Performance

Torin Hopkins, Emily Doherty, Netta Ofer, Suibi Che-Chuan Weng, Peter Gyory, Chad Tobin, Leanne Hirshfield, and Ellen Yi-Luen Do

Proceedings of the International Conference on New Interfaces for Musical Expression

• Year: 2023
• Location: Mexico City, Mexico
• Track: Papers
• Pages: 116–122
• Article Number: 15
• DOI: 10.5281/zenodo.11189125 (Link to paper)
• PDF link

Abstract:

This paper introduces Stringesthesia, an interactive and improvised performance paradigm. Stringesthesia was designed to explore the connection between performer and audience by using real-time neuroimaging technology that gave the audience direct access to the performer's internal mental state and determined the extent of how the audience could participate with the performer throughout the performance. Functional near-infrared spectroscopy (fNIRS) technology was used to assess metabolic activity in a network of brain areas collectively associated with a metric we call “trust”. The real-time measurement of the performer’s level of trust was visualized behind the performer and used to dynamically restrict or promote audience participation: e.g., as the performer’s trust in the audience grew, more participatory stations for playing drums and selecting the performer’s chords were activated. Throughout the paper we discuss prior work that heavily influenced our design, conceptual and methodological issues with using fNIRS technology, and our system architecture. We then describe an employment of this paradigm with a solo guitar player.

Citation:

BibTeX Entry:

  @inproceedings{nime2023_15,
 abstract = {This paper introduces Stringesthesia, an interactive and improvised performance paradigm. Stringesthesia was designed to explore the connection between performer and audience by using real-time neuroimaging technology that gave the audience direct access to the performer's internal mental state and determined the extent of how the audience could participate with the performer throughout the performance. Functional near-infrared spectroscopy (fNIRS) technology was used to assess metabolic activity in a network of brain areas collectively associated with a metric we call “trust”. The real-time measurement of the performer’s level of trust was visualized behind the performer and used to dynamically restrict or promote audience participation: e.g., as the performer’s trust in the audience grew, more participatory stations for playing drums and selecting the performer’s chords were activated. Throughout the paper we discuss prior work that heavily influenced our design, conceptual and methodological issues with using fNIRS technology, and our system architecture. We then describe an employment of this paradigm with a solo guitar player.},
 address = {Mexico City, Mexico},
 articleno = {15},
 author = {Torin Hopkins and Emily Doherty and Netta Ofer and Suibi Che-Chuan Weng and Peter Gyory and Chad Tobin and Leanne Hirshfield and Ellen Yi-Luen Do},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.11189125},
 editor = {Miguel Ortiz and Adnan Marquez-Borbon},
 issn = {2220-4806},
 month = {May},
 numpages = {7},
 pages = {116--122},
 title = {Stringesthesia: Dynamically Shifting Musical Agency Between Audience and Performer Based on Trust in an Interactive and Improvised Performance},
 track = {Papers},
 url = {http://nime.org/proceedings/2023/nime2023_15.pdf},
 year = {2023}
}