Haptic Music Player---Synthetic audio-tactile stimuli generation based on the notes' pitch and instruments' envelope mapping

Alfonso Balandra, Hironori Mitake, and Shoichi Hasegawa

Proceedings of the International Conference on New Interfaces for Musical Expression

• Year: 2016
• Location: Brisbane, Australia
• Track: Papers
• Pages: 90–95
• DOI: 10.5281/zenodo.1175968 (Link to paper)
• PDF link

Abstract:

An entertainment environment to enrich music listening experience is purposed. This environment is composed of 3 modules: a MIDI player, a music animation and a haptic module that translates the notes played by one instrument into a resemblant vibration. To create the haptic vibration, the notes' relative pitch in the song are calculated, then these positions are mapped into the haptic signals' amplitude and frequency. Also, the envelope of the haptic signal is modified, by using an ADSR filter, to have the same envelope as the audio signal. To evaluate the perceived cross-modal similarity between users, two experiments were performed. In both, the users used the complete entertainment environment to rank the similarity between 3 different haptic signals, with triangular, square and analogue envelopes and 4 different instruments in a classical song. The first experiment was performed with the purposed amplitude and frequency technique, while the second experiment was performed with constant frequency and amplitude. Results, show different envelope user preferences. The square and triangular envelopes were preferred in the first experiment, while only analogue envelopes were preferred in the second. This suggests that the users' envelope perception was masked by the changes in amplitude and frequency.

Citation:

BibTeX Entry:

  @inproceedings{Balandra2016,
 abstract = {An entertainment environment to enrich music listening experience is purposed. This environment is composed of 3 modules: a MIDI player, a music animation and a haptic module that translates the notes played by one instrument into a resemblant vibration. To create the haptic vibration, the notes' relative pitch in the song are calculated, then these positions are mapped into the haptic signals' amplitude and frequency. Also, the envelope of the haptic signal is modified, by using an ADSR filter, to have the same envelope as the audio signal. To evaluate the perceived cross-modal similarity between users, two experiments were performed. In both, the users used the complete entertainment environment to rank the similarity between 3 different haptic signals, with triangular, square and analogue envelopes and 4 different instruments in a classical song. The first experiment was performed with the purposed amplitude and frequency technique, while the second experiment was performed with constant frequency and amplitude. Results, show different envelope user preferences. The square and triangular envelopes were preferred in the first experiment, while only analogue envelopes were preferred in the second. This suggests that the users' envelope perception was masked by the changes in amplitude and frequency.},
 address = {Brisbane, Australia},
 author = {Alfonso Balandra and Hironori Mitake and Shoichi Hasegawa},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1175968},
 isbn = {978-1-925455-13-7},
 issn = {2220-4806},
 pages = {90--95},
 publisher = {Queensland Conservatorium Griffith University},
 title = {Haptic Music Player---Synthetic audio-tactile stimuli generation based on the notes' pitch and instruments' envelope mapping},
 track = {Papers},
 url = {http://www.nime.org/proceedings/2016/nime2016_paper0019.pdf},
 year = {2016}
}