XR Musical Keyboard: An Extended Reality Keyboard with an Arbitrary Number of Keys and Pitches

Tatsunori Hirai, Jack Topliss, and Thammathip Piumsomboon

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

We introduce the Extended Reality (XR) Musical Keyboard, a system allowing users to overlay a virtual keyboard onto a tabletop surface, such as a standard PC keyboard. This virtual keyboard is highly customizable: users can freely program the number of keys and their respective pitches. Modern software instruments offer advanced capabilities, including microtonal scales (pitches outside the standard 12-tone equal temperament). However, playing these instruments often remains challenging due to the lack of corresponding physical hardware. Our proposed solution addresses this gap by projecting a programmable virtual keyboard onto a tangible object within the XR space. This approach combines the software's flexibility with the tactile feedback of a physical surface, enhancing playability. Users can simplify the keyboard layout (e.g., fewer keys than a piano) or expand it beyond conventional limits to explore new expressive possibilities, particularly for microtonal music. We conducted a small pilot study (N=4) involving participants mostly inexperienced with keyboards to gather preliminary feedback on the interface's ease of use for performance.

Citation

Tatsunori Hirai, Jack Topliss, and Thammathip Piumsomboon. 2025. XR Musical Keyboard: An Extended Reality Keyboard with an Arbitrary Number of Keys and Pitches. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.15698784 [PDF]

BibTeX Entry

@article{nime2025_6,
 abstract = {We introduce the Extended Reality (XR) Musical Keyboard, a system allowing users to overlay a virtual keyboard onto a tabletop surface, such as a standard PC keyboard. This virtual keyboard is highly customizable: users can freely program the number of keys and their respective pitches. Modern software instruments offer advanced capabilities, including microtonal scales (pitches outside the standard 12-tone equal temperament). However, playing these instruments often remains challenging due to the lack of corresponding physical hardware. Our proposed solution addresses this gap by projecting a programmable virtual keyboard onto a tangible object within the XR space. This approach combines the software's flexibility with the tactile feedback of a physical surface, enhancing playability. Users can simplify the keyboard layout (e.g., fewer keys than a piano) or expand it beyond conventional limits to explore new expressive possibilities, particularly for microtonal music. We conducted a small pilot study (N=4) involving participants mostly inexperienced with keyboards to gather preliminary feedback on the interface's ease of use for performance.},
 address = {Canberra, Australia},
 articleno = {6},
 author = {Tatsunori Hirai and Jack Topliss and Thammathip Piumsomboon},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.15698784},
 editor = {Doga Cavdir and Florent Berthaut},
 issn = {2220-4806},
 month = {June},
 numpages = {6},
 pages = {40--45},
 title = {XR Musical Keyboard: An Extended Reality Keyboard with an Arbitrary Number of Keys and Pitches},
 track = {Paper},
 url = {http://nime.org/proceedings/2025/nime2025_6.pdf},
 year = {2025}
}