Sonic Rhopalia

Kenshiro Taira, Sogen Handa, Risako Shibata, Nimisha Anand, Len Matsuda, Victoria Maki, Ryotaro Hoshino, Kenta Tanaka, Ryoho Kobayashi, Yuta Uozumi, and Shinya Fujii

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

The Jellyfish is a species with one of the most simplistic and complete Umwelts. Its pulsating movements are caused by a refined network, composing only of a few neurons. These neural receptors, named Rhopalia, receive external simulation which determines their pulses. In the ocean, these pulses then ripple into waves in the water, that is sensed by the Rhopalia, triggering the pulses of the next jellyfish. Such interactions (that cannot be seen by the naked eye) in repetition over time, have sustained the existence of the jellyfish species across 6 billion years. This mechanical system causes organic pulses, radiating an inexplicable feeling of vitality. ‘Sonic Rhopalia’ is a sound installation that is produced by the pulsation of four Aurelia aurita (a.k.a. moon jellyfish). By using Touch Designer and a web camera to capture the movement of each jellyfish, we detect its pulse through image analysis, using OpenCV and a combination of pixel color channel classification and area calculation. Once the pulse of a jellyfish is detected, it sends a bang to M4L (Max for Live) using OSC (OpenSound Control), where it is used as a trigger for music composition. Sounds are played in response to each pulse, and when the jellyfish pulse at the same time, in synchronization, different chords and sounds are produced. The work develops across one loop comprising of three phases with varying soundscapes, all determined by the number pulse synchronizations. By reframing jellyfish as an interface for music generation, the rhythms and movements of the pulsating jellyfish come together create a unique soundscape, that allows the audience to experience the vitality of jellyfish in a completely new dimension from conventional methods of bio-marine appreciation.

Citation

Kenshiro Taira, Sogen Handa, Risako Shibata, Nimisha Anand, Len Matsuda, Victoria Maki, Ryotaro Hoshino, Kenta Tanaka, Ryoho Kobayashi, Yuta Uozumi, and Shinya Fujii. 2024. Sonic Rhopalia. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.15027173

BibTeX Entry

@article{nime2024_installations_10,
 abstract = {The Jellyfish is a species with one of the most simplistic and complete Umwelts. Its pulsating movements are caused by a refined network, composing only of a few neurons. These neural receptors, named Rhopalia, receive external simulation which determines their pulses. In the ocean, these pulses then ripple into waves in the water, that is sensed by the Rhopalia, triggering the pulses of the next jellyfish. Such interactions (that cannot be seen by the naked eye) in repetition over time, have sustained the existence of the jellyfish species across 6 billion years. This mechanical system causes organic pulses, radiating an inexplicable feeling of vitality. ‘Sonic Rhopalia’ is a sound installation that is produced by the pulsation of four Aurelia aurita (a.k.a. moon jellyfish). By using Touch Designer and a web camera to capture the movement of each jellyfish, we detect its pulse through image analysis, using OpenCV and a combination of pixel color channel classification and area calculation. Once the pulse of a jellyfish is detected, it sends a bang to M4L (Max for Live) using OSC (OpenSound Control), where it is used as a trigger for music composition. Sounds are played in response to each pulse, and when the jellyfish pulse at the same time, in synchronization, different chords and sounds are produced. The work develops across one loop comprising of three phases with varying soundscapes, all determined by the number pulse synchronizations. By reframing jellyfish as an interface for music generation, the rhythms and movements of the pulsating jellyfish come together create a unique soundscape, that allows the audience to experience the vitality of jellyfish in a completely new dimension from conventional methods of bio-marine appreciation.},
 address = {Utrecht, Netherlands},
 articleno = {10},
 author = {Kenshiro Taira and Sogen Handa and Risako Shibata and Nimisha Anand and Len Matsuda and Victoria Maki and Ryotaro Hoshino and Kenta Tanaka and Ryoho Kobayashi and Yuta Uozumi and Shinya Fujii},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.15027173},
 editor = {Laurel Smith Pardue and Palle Dahlstedt},
 issn = {2220-4806},
 month = {September},
 numpages = {6},
 pages = {37--42},
 presentation-video = {https://youtu.be/HBFbDtB_Nsw},
 title = {Sonic Rhopalia},
 track = {Installations},
 url = {http://nime.org/proceedings/2024/nime2024_installations_10.pdf},
 year = {2024}
}