MusicMapper: Interactive 2D representations of music samples for in-browser remixing and exploration

Ethan Benjamin, and Jaan Altosaar

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

Much of the challenge and appeal in remixing music comes from manipulating samples. Typically, identifying distinct samples of a song requires expertise in music production software. Additionally, it is difficult to visualize similarities and differences between all samples of a song simultaneously and use this to select samples. MusicMapper is a web application that allows nonexpert users to find and visualize distinctive samples from a song without any manual intervention, and enables remixing by having users play back clusterings of such samples. This is accomplished by splitting audio from the Soundcloud API into appropriately-sized spectrograms, and applying the t-SNE algorithm to visualize these spectrograms in two dimensions. Next, we apply k-means to guide the user's eye toward related clusters and set $k=26$ to enable playback of the clusters by pressing keys on an ordinary keyboard. We present the source code (https://github.com/fatsmcgee/MusicMappr) and a demo video (http://youtu.be/mvD6e1uiO8k) of the MusicMapper web application that can be run in most modern browsers.

Citation:

Ethan Benjamin, and Jaan Altosaar. 2015. MusicMapper: Interactive 2D representations of music samples for in-browser remixing and exploration. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1179018

BibTeX Entry:

  @inproceedings{jaltosaar2015,
 abstract = {Much of the challenge and appeal in remixing music comes from manipulating samples. Typically, identifying distinct samples of a song requires expertise in music production software. Additionally, it is difficult to visualize similarities and differences between all samples of a song simultaneously and use this to select samples. MusicMapper is a web application that allows nonexpert users to find and visualize distinctive samples from a song without any manual intervention, and enables remixing by having users play back clusterings of such samples. This is accomplished by splitting audio from the Soundcloud API into appropriately-sized spectrograms, and applying the t-SNE algorithm to visualize these spectrograms in two dimensions. Next, we apply k-means to guide the user's eye toward related clusters and set $k=26$ to enable playback of the clusters by pressing keys on an ordinary keyboard. We present the source code (https://github.com/fatsmcgee/MusicMappr) and a demo video (http://youtu.be/mvD6e1uiO8k) of the MusicMapper web application that can be run in most modern browsers.},
 address = {Baton Rouge, Louisiana, USA},
 author = {Ethan Benjamin and Jaan Altosaar},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1179018},
 editor = {Edgar Berdahl and Jesse Allison},
 issn = {2220-4806},
 month = {May},
 pages = {325--326},
 publisher = {Louisiana State University},
 title = {MusicMapper: Interactive {2D} representations of music samples for in-browser remixing and exploration},
 url = {http://www.nime.org/proceedings/2015/nime2015_161.pdf},
 urlsuppl1 = {http://www.nime.org/proceedings/2015/161/0161-file1.mp4},
 year = {2015}
}