Pointing Fingers: Using Multiple Direct Interactions with Visual Objects to Perform Music

Jean-Michel Couturier, and Daniel Arfib

Proceedings of the International Conference on New Interfaces for Musical Expression

  • Year: 2003
  • Location: Montreal, Canada
  • Pages: 184–187
  • Keywords: HCI, touch screen, multimodality, mapping, direct interaction, gesture devices, bimanual interaction, two-handed, Max/MSP.
  • DOI: 10.5281/zenodo.1176501 (Link to paper)
  • PDF link

Abstract:

In this paper, we describe a new interface for musicalperformance, using the interaction with a graphical userinterface in a powerful manner: the user directly touches ascreen where graphical objects are displayed and can useseveral fingers simultaneously to interact with the objects. Theconcept of this interface is based on the superposition of thegesture spatial place and the visual feedback spatial place; i tgives the impression that the graphical objects are real. Thisconcept enables a huge freedom in designing interfaces. Thegesture device we have created gives the position of fourfingertips using 3D sensors and the data is performed in theMax/MSP environment. We have realized two practicalexamples of musical use of such a device, using PhotosonicSynthesis and Scanned Synthesis.

Citation:

Jean-Michel Couturier, and Daniel Arfib. 2003. Pointing Fingers: Using Multiple Direct Interactions with Visual Objects to Perform Music. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1176501

BibTeX Entry:

  @inproceedings{Couturier2003,
 abstract = {In this paper, we describe a new interface for musicalperformance, using the interaction with a graphical userinterface in a powerful manner: the user directly touches ascreen where graphical objects are displayed and can useseveral fingers simultaneously to interact with the objects. Theconcept of this interface is based on the superposition of thegesture spatial place and the visual feedback spatial place; i tgives the impression that the graphical objects are real. Thisconcept enables a huge freedom in designing interfaces. Thegesture device we have created gives the position of fourfingertips using 3D sensors and the data is performed in theMax/MSP environment. We have realized two practicalexamples of musical use of such a device, using PhotosonicSynthesis and Scanned Synthesis.},
 address = {Montreal, Canada},
 author = {Couturier, Jean-Michel and Arfib, Daniel},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 date = {22-24 May, 2003},
 doi = {10.5281/zenodo.1176501},
 issn = {2220-4806},
 keywords = {HCI, touch screen, multimodality, mapping, direct interaction, gesture devices, bimanual interaction, two-handed, Max/MSP. },
 pages = {184--187},
 title = {Pointing Fingers: Using Multiple Direct Interactions with Visual Objects to Perform Music},
 url = {http://www.nime.org/proceedings/2003/nime2003_184.pdf},
 year = {2003}
}