The Phone with the Flow: Combining Touch + Optical Flow in Mobile Instruments

Cagan Arslan, Florent Berthaut, Jean Martinet, Ioan Marius Bilasco, and Laurent Grisoni

Proceedings of the International Conference on New Interfaces for Musical Expression

• Year: 2018
• Location: Blacksburg, Virginia, USA
• Pages: 148–151
• DOI: 10.5281/zenodo.1302709 (Link to paper)
• PDF link

Abstract:

Mobile devices have been a promising platform for musical performance thanks to the various sensors readily available on board. In particular, mobile cameras can provide rich input as they can capture a wide variety of user gestures or environment dynamics. However, this raw camera input only provides continuous parameters and requires expensive computation. In this paper, we propose to combine motion/gesture input with the touch input, in order to filter movement information both temporally and spatially, thus increasing expressiveness while reducing computation time. We present a design space which demonstrates the diversity of interactions that our technique enables. We also report the results of a user study in which we observe how musicians appropriate the interaction space with an example instrument.

Citation:

BibTeX Entry:

  @inproceedings{Arslan2018,
 abstract = {Mobile devices have been a promising platform for musical performance thanks to the various sensors readily available on board. In particular, mobile cameras can provide rich input as they can capture a wide variety of user gestures or environment dynamics. However, this raw camera input only provides continuous parameters and requires expensive computation. In this paper, we propose to combine motion/gesture input with the touch input, in order to filter movement information both temporally and spatially, thus increasing expressiveness while reducing computation time. We present a design space which demonstrates the diversity of interactions that our technique enables. We also report the results of a user study in which we observe how musicians appropriate the interaction space with an example instrument.},
 address = {Blacksburg, Virginia, USA},
 author = {Cagan Arslan and Florent Berthaut and Jean Martinet and Ioan Marius Bilasco and Laurent Grisoni},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1302709},
 editor = {Luke Dahl, Douglas Bowman, Thomas Martin},
 isbn = {978-1-949373-99-8},
 issn = {2220-4806},
 month = {June},
 pages = {148--151},
 publisher = {Virginia Tech},
 title = {The Phone with the Flow: Combining Touch + Optical Flow in Mobile Instruments},
 url = {http://www.nime.org/proceedings/2018/nime2018_paper0032.pdf},
 year = {2018}
}