Tungnaá: a Hyper-realistic Voice Synthesis Instrument for Real-Time Exploration of Extended Vocal Expressions

Victor Shepardson, Jonathan Reus, and Thor Magnusson

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

This demo showcases Tungnaá, a new voice synthesis system and software instrument for real-time musical exploration of "Deep Voice Synthesis". The design of Tungnaá emphasizes real-time interaction and customization, enabling artists to manipulate various aspects of the synthesis process and to explore aesthetic artefacts unique to autoregressive neural synthesis. The synthesis engine achieves real-time streaming generation of paralinguistic and extended forms of vocal expression, while controlling them using symbolic text notations drawn from the entire unicode character set, allowing for the creation of new notation systems. The interface provides visual display and mouse- or OSC-controllable interventions into the machine vocalisations. The demo showcases Tungnaá on a laptop with headphones and a MIDI controller, allowing participants to explore the instrument via both a textual and physical interface.

Citation:

Victor Shepardson, Jonathan Reus, and Thor Magnusson. 2024. Tungnaá: a Hyper-realistic Voice Synthesis Instrument for Real-Time Exploration of Extended Vocal Expressions. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.13904943

BibTeX Entry:

  @article{nime2024_78,
 abstract = {This demo showcases Tungnaá, a new voice synthesis system and software instrument for real-time musical exploration of "Deep Voice Synthesis". The design of Tungnaá emphasizes real-time interaction and customization, enabling artists to manipulate various aspects of the synthesis process and to explore aesthetic artefacts unique to autoregressive neural synthesis. The synthesis engine achieves real-time streaming generation of paralinguistic and extended forms of vocal expression, while controlling them using symbolic text notations drawn from the entire unicode character set, allowing for the creation of new notation systems. The interface provides visual display and mouse- or OSC-controllable interventions into the machine vocalisations. The demo showcases Tungnaá on a laptop with headphones and a MIDI controller, allowing participants to explore the instrument via both a textual and physical interface.},
 address = {Utrecht, Netherlands},
 articleno = {78},
 author = {Victor Shepardson and Jonathan Reus and Thor Magnusson},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.13904943},
 editor = {S M Astrid Bin and Courtney N. Reed},
 issn = {2220-4806},
 month = {September},
 numpages = {5},
 pages = {536--540},
 presentation-video = {https://youtu.be/e6KFcsyiWa4?si=4Rnf_mtpgx0NpM9R},
 title = {Tungnaá: a Hyper-realistic Voice Synthesis Instrument for Real-Time Exploration of Extended Vocal Expressions},
 track = {Papers},
 url = {http://nime.org/proceedings/2024/nime2024_78.pdf},
 year = {2024}
}