Sonification of Fish Movement Using Pitch Mesh Pairs

Andrew Mercer-Taylor, and Jaan Altosaar

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

On a traditional keyboard, the actions required to play a consonant chord progression must be quite precise; accidentally strike a neighboring key, and a pleasant sonority is likely to become a jarring one. Inspired by the Tonnetz (a tonal diagram), we present a new layout of pitches defined using low-level harmonic notions. We demonstrate the potential of our system by mapping the random movements of fish in an aquarium to this layout. Qualitatively, we find that this captures the intuition behind mapping motion to sound. Similarly moving fish produce consonant chords, while fish moving in non-unison produce dissonant chords. We introduce an open source MATLAB library implementing these techniques, which can be used for sonifying multimodal streaming data.

Citation:

Andrew Mercer-Taylor, and Jaan Altosaar. 2015. Sonification of Fish Movement Using Pitch Mesh Pairs. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1179138

BibTeX Entry:

  @inproceedings{amercertaylor2015,
 abstract = {On a traditional keyboard, the actions required to play a consonant chord progression must be quite precise; accidentally strike a neighboring key, and a pleasant sonority is likely to become a jarring one. Inspired by the Tonnetz (a tonal diagram), we present a new layout of pitches defined using low-level harmonic notions. We demonstrate the potential of our system by mapping the random movements of fish in an aquarium to this layout. Qualitatively, we find that this captures the intuition behind mapping motion to sound. Similarly moving fish produce consonant chords, while fish moving in non-unison produce dissonant chords. We introduce an open source MATLAB library implementing these techniques, which can be used for sonifying multimodal streaming data. },
 address = {Baton Rouge, Louisiana, USA},
 author = {Andrew Mercer-Taylor and Jaan Altosaar},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1179138},
 editor = {Edgar Berdahl and Jesse Allison},
 issn = {2220-4806},
 month = {May},
 pages = {28--29},
 publisher = {Louisiana State University},
 title = {Sonification of Fish Movement Using Pitch Mesh Pairs},
 url = {http://www.nime.org/proceedings/2015/nime2015_155.pdf},
 urlsuppl1 = {http://www.nime.org/proceedings/2015/155/0155-file1.mp4},
 year = {2015}
}