NIME or Mime: A Sound-First Approach to Developing an Audio-Visual Gestural Instrument

Alon A Ilsar, Matthew Hughes, and Andrew Johnston

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

This paper outlines the development process of an audio-visual gestural instrument—the AirSticks—and elaborates on the role ‘miming’ has played in the formation of new mappings for the instrument. The AirSticks, although fully-functioning, were used as props in live performances in order to evaluate potential mapping strategies that were later implemented for real. This use of mime when designing Digital Musical Instruments (DMIs) can help overcome choice paralysis, break from established habits, and liberate creators to realise more meaningful parameter mappings. Bringing this process into an interactive performance environment acknowledges the audience as stakeholders in the design of these instruments, and also leads us to reflect upon the beliefs and assumptions made by an audience when engaging with the performance of such ‘magical’ devices. This paper establishes two opposing strategies to parameter mapping, ‘movement-first’ mapping, and the less conventional ‘sound-first’ mapping that incorporates mime. We discuss the performance ‘One Five Nine’, its transformation from a partial mime into a fully interactive presentation, and the influence this process has had on the outcome of the performance and the AirSticks as a whole.

Citation:

Alon A Ilsar, Matthew Hughes, and Andrew Johnston. 2020. NIME or Mime: A Sound-First Approach to Developing an Audio-Visual Gestural Instrument. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.4813383

BibTeX Entry:

  @inproceedings{NIME20_60,
 abstract = {This paper outlines the development process of an audio-visual gestural instrument—the AirSticks—and elaborates on the role ‘miming’ has played in the formation of new mappings for the instrument. The AirSticks, although fully-functioning, were used as props in live performances in order to evaluate potential mapping strategies that were later implemented for real. This use of mime when designing Digital Musical Instruments (DMIs) can help overcome choice paralysis, break from established habits, and liberate creators to realise more meaningful parameter mappings. Bringing this process into an interactive performance environment acknowledges the audience as stakeholders in the design of these instruments, and also leads us to reflect upon the beliefs and assumptions made by an audience when engaging with the performance of such ‘magical’ devices. This paper establishes two opposing strategies to parameter mapping, ‘movement-first’ mapping, and the less conventional ‘sound-first’ mapping that incorporates mime. We discuss the performance ‘One Five Nine’, its transformation from a partial mime into a fully interactive presentation, and the influence this process has had on the outcome of the performance and the AirSticks as a whole.},
 address = {Birmingham, UK},
 author = {Ilsar, Alon A and Hughes, Matthew and Johnston, Andrew},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.4813383},
 editor = {Romain Michon and Franziska Schroeder},
 issn = {2220-4806},
 month = {July},
 pages = {315--320},
 presentation-video = {https://youtu.be/ZFQKKI3dFhE},
 publisher = {Birmingham City University},
 title = {NIME or Mime: A Sound-First Approach to Developing an Audio-Visual Gestural Instrument},
 url = {https://www.nime.org/proceedings/2020/nime2020_paper60.pdf},
 year = {2020}
}