MusicMapper: Interactive 2D representations of music samples for in-browser remixing and exploration

Ethan Benjamin, and Jaan Altosaar

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

Much of the challenge and appeal in remixing music comes from manipulating samples. Typically, identifying distinct samples of a song requires expertise in music production software. Additionally, it is difficult to visualize similarities and differences between all samples of a song simultaneously and use this to select samples. MusicMapper is a web application that allows nonexpert users to find and visualize distinctive samples from a song without any manual intervention, and enables remixing by having users play back clusterings of such samples. This is accomplished by splitting audio from the Soundcloud API into appropriately-sized spectrograms, and applying the t-SNE algorithm to visualize these spectrograms in two dimensions. Next, we apply k-means to guide the user's eye toward related clusters and set $k=26$ to enable playback of the clusters by pressing keys on an ordinary keyboard. We present the source code (https://github.com/fatsmcgee/MusicMappr) and a demo video (http://youtu.be/mvD6e1uiO8k) of the MusicMapper web application that can be run in most modern browsers.

Citation

Ethan Benjamin, and Jaan Altosaar. 2015. MusicMapper: Interactive 2D representations of music samples for in-browser remixing and exploration. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1179018

BibTeX Entry

@inproceedings{jaltosaar2015,
 abstract = {Much of the challenge and appeal in remixing music comes from manipulating samples. Typically, identifying distinct samples of a song requires expertise in music production software. Additionally, it is difficult to visualize similarities and differences between all samples of a song simultaneously and use this to select samples. MusicMapper is a web application that allows nonexpert users to find and visualize distinctive samples from a song without any manual intervention, and enables remixing by having users play back clusterings of such samples. This is accomplished by splitting audio from the Soundcloud API into appropriately-sized spectrograms, and applying the t-SNE algorithm to visualize these spectrograms in two dimensions. Next, we apply k-means to guide the user's eye toward related clusters and set $k=26$ to enable playback of the clusters by pressing keys on an ordinary keyboard. We present the source code (https://github.com/fatsmcgee/MusicMappr) and a demo video (http://youtu.be/mvD6e1uiO8k) of the MusicMapper web application that can be run in most modern browsers.},
 address = {Baton Rouge, Louisiana, USA},
 author = {Ethan Benjamin and Jaan Altosaar},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1179018},
 editor = {Edgar Berdahl and Jesse Allison},
 issn = {2220-4806},
 month = {May},
 pages = {325--326},
 publisher = {Louisiana State University},
 title = {MusicMapper: Interactive {2D} representations of music samples for in-browser remixing and exploration},
 url = {http://www.nime.org/proceedings/2015/nime2015_161.pdf},
 urlsuppl1 = {http://www.nime.org/proceedings/2015/161/0161-file1.mp4},
 year = {2015}
}