Conjuring the Recorded Pianist: A New Medium to Experience Musical Performance

Xiao Xiao, Anna Pereira, and Hiroshi Ishii

Proceedings of the International Conference on New Interfaces for Musical Expression

• Year: 2013
• Location: Daejeon, Republic of Korea
• Pages: 7–12
• Keywords: piano performance, musical expressivity, body language, recorded music, player piano, augmented reality, embodiment
• DOI: 10.5281/zenodo.1178692 (Link to paper)
• PDF link

Abstract:

The body channels rich layers of information when playing music, from intricatemanipulations of the instrument to vivid personifications of expression. Butwhen music is captured and replayed across distance and time, the performer'sbody is too often trapped behind a small screen or absent entirely.This paper introduces an interface to conjure the recorded performer bycombining the moving keys of a player piano with life-sized projection of thepianist's hands and upper body. Inspired by reflections on a lacquered grandpiano, our interface evokes the sense that the virtual pianist is playing thephysically moving keys.Through our interface, we explore the question of how to viscerally simulate aperformer's presence to create immersive experiences. We discuss designchoices, outline a space of usage scenarios and report reactions from users.

Citation:

BibTeX Entry:

  @inproceedings{Xiao2013,
 abstract = {The body channels rich layers of information when playing music, from intricatemanipulations of the instrument to vivid personifications of expression. Butwhen music is captured and replayed across distance and time, the performer'sbody is too often trapped behind a small screen or absent entirely.This paper introduces an interface to conjure the recorded performer bycombining the moving keys of a player piano with life-sized projection of thepianist's hands and upper body. Inspired by reflections on a lacquered grandpiano, our interface evokes the sense that the virtual pianist is playing thephysically moving keys.Through our interface, we explore the question of how to viscerally simulate aperformer's presence to create immersive experiences. We discuss designchoices, outline a space of usage scenarios and report reactions from users.},
 address = {Daejeon, Republic of Korea},
 author = {Xiao Xiao and Anna Pereira and Hiroshi Ishii},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1178692},
 issn = {2220-4806},
 keywords = {piano performance, musical expressivity, body language, recorded music, player piano, augmented reality, embodiment},
 month = {May},
 pages = {7--12},
 publisher = {Graduate School of Culture Technology, KAIST},
 title = {Conjuring the Recorded Pianist: A New Medium to Experience Musical Performance},
 url = {http://www.nime.org/proceedings/2013/nime2013_28.pdf},
 year = {2013}
}