AudioQuilt: 2D Arrangements of Audio Samples using Metric Learning and Kernelized Sorting

Ohad Fried, Zeyu Jin, Reid Oda, and Adam Finkelstein

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

The modern musician enjoys access to a staggering number of audio samples. Composition software can ship with many gigabytes of data, and there are many more to be found online. However, conventional methods for navigating these libraries are still quite rudimentary, and often involve scrolling through alphabetical lists. We present a system for sample exploration that allows audio clips to be sorted according to user taste, and arranged in any desired 2D formation such that similar samples are located near each other. Our method relies on two advances in machine learning. First, metric learning allows the user to shape the audio feature space to match their own preferences. Second, kernelized sorting finds an optimal arrangement for the samples in 2D. We demonstrate our system with two new interfaces for exploring audio samples, and evaluate the technology qualitatively and quantitatively via a pair of user studies.

Citation:

Ohad Fried, Zeyu Jin, Reid Oda, and Adam Finkelstein. 2014. AudioQuilt: 2D Arrangements of Audio Samples using Metric Learning and Kernelized Sorting. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1178766

BibTeX Entry:

  @inproceedings{ofried2014,
 abstract = {The modern musician enjoys access to a staggering number of audio samples. Composition software can ship with many gigabytes of data, and there are many more to be found online. However, conventional methods for navigating these libraries are still quite rudimentary, and often involve scrolling through alphabetical lists. We present a system for sample exploration that allows audio clips to be sorted according to user taste, and arranged in any desired 2D formation such that similar samples are located near each other. Our method relies on two advances in machine learning. First, metric learning allows the user to shape the audio feature space to match their own preferences. Second, kernelized sorting finds an optimal arrangement for the samples in 2D. We demonstrate our system with two new interfaces for exploring audio samples, and evaluate the technology qualitatively and quantitatively via a pair of user studies.},
 address = {London, United Kingdom},
 author = {Ohad Fried and Zeyu Jin and Reid Oda and Adam Finkelstein},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1178766},
 issn = {2220-4806},
 month = {June},
 pages = {281--286},
 publisher = {Goldsmiths, University of London},
 title = {AudioQuilt: {2D} Arrangements of Audio Samples using Metric Learning and Kernelized Sorting},
 url = {http://www.nime.org/proceedings/2014/nime2014_315.pdf},
 year = {2014}
}