Border: A Live Performance Based on Web AR and a Gesture-Controlled Virtual Instrument

Kiyu Nishida, Akishige Yuguchi, kazuhiro jo, Paul Modler, and Markus Noisternig

Proceedings of the International Conference on New Interfaces for Musical Expression

• Year: 2019
• Location: Porto Alegre, Brazil
• Pages: 43–46
• DOI: 10.5281/zenodo.3672860 (Link to paper)
• PDF link

Abstract:

Recent technological advances, such as increased CPU/GPU processing speed, along with the miniaturization of devices and sensors, have created new possibilities for integrating immersive technologies in music and performance art. Virtual and Augmented Reality (VR/AR) have become increasingly interesting as mobile device platforms, such as up-to-date smartphones, with necessary CPU resources entered the consumer market. In combination with recent web technologies, any mobile device can simply connect with a browser to a local server to access the latest technology. The web platform also eases the integration of collaborative situated media in participatory artwork. In this paper, we present the interactive music improvisation piece ‘Border,' premiered in 2018 at the Beyond Festival at the Center for Art and Media Karlsruhe (ZKM). This piece explores the interaction between a performer and the audience using web-based applications – including AR, real-time 3D audio/video streaming, advanced web audio, and gesture-controlled virtual instruments – on smart mobile devices.

Citation:

BibTeX Entry:

  @inproceedings{Nishida2019,
 abstract = {Recent technological advances, such as increased CPU/GPU processing speed, along with the miniaturization of devices and sensors, have created new possibilities for integrating immersive technologies in music and performance art. Virtual and Augmented Reality (VR/AR) have become increasingly interesting as mobile device platforms, such as up-to-date smartphones, with necessary CPU resources entered the consumer market. In combination with recent web technologies, any mobile device can simply connect with a browser to a local server to access the latest technology. The web platform also eases the integration of collaborative situated media in participatory artwork. In this paper, we present the interactive music improvisation piece ‘Border,' premiered in 2018 at the Beyond Festival at the Center for Art and Media Karlsruhe (ZKM). This piece explores the interaction between a performer and the audience using web-based applications – including AR, real-time 3D audio/video streaming, advanced web audio, and gesture-controlled virtual instruments – on smart mobile devices.},
 address = {Porto Alegre, Brazil},
 author = {Kiyu Nishida and Akishige Yuguchi and kazuhiro jo and Paul Modler and Markus Noisternig},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.3672860},
 editor = {Marcelo Queiroz and Anna Xambó Sedó},
 issn = {2220-4806},
 month = {June},
 pages = {43--46},
 publisher = {UFRGS},
 title = {Border: A Live Performance Based on Web {AR} and a Gesture-Controlled Virtual Instrument},
 url = {http://www.nime.org/proceedings/2019/nime2019_paper009.pdf},
 year = {2019}
}