The Wandering Mind: Planetary Scale Dreaming in Latent Spaces

Gershon Dublon, Xin Liu, Nicholas Gillian, and Nan Zhao

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

Does the Earth dream? Can our dreams mesh? In the Wandering Mind, we merge audience dreams and a massive data set of audio field recordings. In dreams, we set sail on open ocean. Freed from the sense-making constraints of the waking mind, the currents pull us through a process of wandering transformations, influenced by both our subconscious minds and our surroundings. This dream-work reminds us of the operations of AI algorithms called representation learning neural networks, which uncover relationships between data samples that can facilitate travel outside of space and time. Applied to global recordings, the latent space connections are like sensory wormholes. To create the Wandering Mind platform, we fed ~70,000 field recordings scraped from the internet to a neural network called YAMNet , producing ~30 million 521-dimensional points corresponding to 1-second audio clips. We further reduce the dimensionality to 2-d, creating a traversable sound map. In the 30 minute performance, we navigate over the map to create a continuous, morphing soundscape that mixes as many as 30 distinct recordings together at any given time, while the audience drifts between waking and sleep. For the best experience, we recommend finding a darkened space and a comfortable spot to lie down. There is no requirement to sleep. We only encourage you to allow the mind to wander.

Citation

Gershon Dublon, Xin Liu, Nicholas Gillian, and Nan Zhao. 2021. The Wandering Mind: Planetary Scale Dreaming in Latent Spaces. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.21428/92fbeb44.56a514bb [PDF]

BibTeX Entry

@inproceedings{nime2021_music_35,
 abstract = {Does the Earth dream? Can our dreams mesh? In the Wandering Mind, we merge audience dreams and a massive data set of audio field recordings. In dreams, we set sail on open ocean. Freed from the sense-making constraints of the waking mind, the currents pull us through a process of wandering transformations, influenced by both our subconscious minds and our surroundings. This dream-work reminds us of the operations of AI algorithms called representation learning neural networks, which uncover relationships between data samples that can facilitate travel outside of space and time. Applied to global recordings, the latent space connections are like sensory wormholes. To create the Wandering Mind platform, we fed ~70,000 field recordings scraped from the internet to a neural network called YAMNet , producing ~30 million 521-dimensional points corresponding to 1-second audio clips. We further reduce the dimensionality to 2-d, creating a traversable sound map. In the 30 minute performance, we navigate over the map to create a continuous, morphing soundscape that mixes as many as 30 distinct recordings together at any given time, while the audience drifts between waking and sleep. For the best experience, we recommend finding a darkened space and a comfortable spot to lie down. There is no requirement to sleep. We only encourage you to allow the mind to wander.},
 address = {Shanghai, China},
 articleno = {35},
 author = {Gershon Dublon and Xin Liu and Nicholas Gillian and Nan Zhao},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.21428/92fbeb44.56a514bb},
 editor = {Eric Parren and Wei Chen},
 issn = {2220-4806},
 month = {June},
 title = {The Wandering Mind: Planetary Scale Dreaming in Latent Spaces},
 track = {Music},
 url = {https://doi.org/10.21428/92fbeb44.56a514bb},
 year = {2021}
}