Musical Interface to Audiovisual Corpora of Arbitrary Instruments

Joachim Goßmann, and Max Neupert

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

We present an instrument for audio-visual performance that allows to recombine sounds from a collection of sampled media through concatenative synthesis. A three-dimensional distribution derived from feature-analysis becomes accessible through a theremin-inspired interface, allowing the player to shift from exploration and intuitive navigation toward embodied performance on a granular level. In our example we illustrate this concept by using the audiovisual recording of an instrumental performance as a source. Our system provides an alternative interface to the musical instrument's audiovisual corpus: as the instrument's sound and behavior is accessed in ways that are not possible on the instrument itself, the resulting non-linear playback of the grains generates an instant remix in a cut-up aesthetic. The presented instrument is a human-computer interface that employs the structural outcome of machine analysis accessing audiovisual corpora in the context of a musical performance.

Citation

Joachim Goßmann, and Max Neupert. 2014. Musical Interface to Audiovisual Corpora of Arbitrary Instruments. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1178772

BibTeX Entry

@inproceedings{mneupert2014,
 abstract = {We present an instrument for audio-visual performance that allows to recombine sounds from a collection of sampled media through concatenative synthesis. A three-dimensional distribution derived from feature-analysis becomes accessible through a theremin-inspired interface, allowing the player to shift from exploration and intuitive navigation toward embodied performance on a granular level. In our example we illustrate this concept by using the audiovisual recording of an instrumental performance as a source. Our system provides an alternative interface to the musical instrument's audiovisual corpus: as the instrument's sound and behavior is accessed in ways that are not possible on the instrument itself, the resulting non-linear playback of the grains generates an instant remix in a cut-up aesthetic. The presented instrument is a human-computer interface that employs the structural outcome of machine analysis accessing audiovisual corpora in the context of a musical performance.},
 address = {London, United Kingdom},
 author = {Joachim Go{\ss}mann and Max Neupert},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1178772},
 issn = {2220-4806},
 month = {June},
 pages = {151--154},
 publisher = {Goldsmiths, University of London},
 title = {Musical Interface to Audiovisual Corpora of Arbitrary Instruments},
 url = {http://www.nime.org/proceedings/2014/nime2014_296.pdf},
 year = {2014}
}