Music Everywhere --- Augmented Reality Piano Improvisation Learning System

Seth Glickman, Byunghwan Lee, Fu Yen Hsiao, and Shantanu Das

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

This paper describes the design and implementation of an augmented reality (AR) piano learning tool that uses a Microsoft HoloLens and a MIDI-over-Bluetooth-enabled electric piano. The tool presents a unique visual interface—a “mirror key overlay” approach—fitted for the AR environment, and opens up the possibility of on-instrument learning experiences. The curriculum focuses on teaching improvisation in blues, rock, jazz and classical genres. Users at the piano engage with interactive lessons, watch virtual hand demonstrations, see and hear example improvisations, and play their own solos and accompaniment along with AR-projected virtual musicians. The tool aims to be entertaining yet also effective in teaching core musical concepts.

Citation:

Seth Glickman, Byunghwan Lee, Fu Yen Hsiao, and Shantanu Das. 2017. Music Everywhere --- Augmented Reality Piano Improvisation Learning System. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1176350

BibTeX Entry:

  @inproceedings{sglickman2017,
 abstract = {This paper describes the design and implementation of an augmented reality (AR) piano learning tool that uses a Microsoft HoloLens and a MIDI-over-Bluetooth-enabled electric piano. The tool presents a unique visual interface—a “mirror key overlay” approach—fitted for the AR environment, and opens up the possibility of on-instrument learning experiences. The curriculum focuses on teaching improvisation in blues, rock, jazz and classical genres. Users at the piano engage with interactive lessons, watch virtual hand demonstrations, see and hear example improvisations, and play their own solos and accompaniment along with AR-projected virtual musicians. The tool aims to be entertaining yet also effective in teaching core musical concepts.},
 address = {Copenhagen, Denmark},
 author = {Seth Glickman and Byunghwan Lee and Fu Yen Hsiao and Shantanu Das},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1176350},
 issn = {2220-4806},
 pages = {511--512},
 publisher = {Aalborg University Copenhagen},
 title = {Music Everywhere --- Augmented Reality Piano Improvisation Learning System},
 url = {http://www.nime.org/proceedings/2017/nime2017_paper0104.pdf},
 year = {2017}
}