Skip to content

Towards an Affective Gesture Interface for Expressive Music Performance

Vassilios-Fivos A. Maniatakos, and Christian Jacquemin

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

This paper discusses the use of 'Pogany', an affective anthropomorphic interface, for expressive music performance. For this purpose the interface is equipped with a module for gesture analysis: a) in a direct level, in order to conceptualize measures capable of driving continuous musical parameters, b) in an indirect level, in order to capture high-level information arising from 'meaningful' gestures. The real-time recognition module for hand gestures and postures is based on Hidden Markov Models (HMMs). After an overview of the interface, we analyze the techniques used for gesture recognition and the decisions taken for mapping gestures with sound synthesis parameters. For the evaluation of the system as an interface for musical expression we made an experiment with real subjects. The results of this experiment are presented and analyzed.

Citation

Vassilios-Fivos A. Maniatakos, and Christian Jacquemin. 2008. Towards an Affective Gesture Interface for Expressive Music Performance. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1179595 [PDF]

BibTeX Entry


@inproceedings{nime2008_Maniatakos,
author = {Maniatakos, Vassilios-Fivos A. and Jacquemin, Christian},
title = {Towards an Affective Gesture Interface for Expressive Music Performance},
pages = {122--127},
booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
year = {2008},
address = {Genoa, Italy},
issn = {2220-4806},
doi = {10.5281/zenodo.1179595},
url = {https://nime.org/proceedings/2008/nime2008_122.pdf},
keywords = {affective computing, interactive performance, HMM, gesture recognition, intelligent mapping, affective interface },
abstract = {This paper discusses the use of 'Pogany', an affective anthropomorphic interface, for expressive music performance. For this purpose the interface is equipped with a module for gesture analysis: a) in a direct level, in order to conceptualize measures capable of driving continuous musical parameters, b) in an indirect level, in order to capture high-level information arising from 'meaningful' gestures. The real-time recognition module for hand gestures and postures is based on Hidden Markov Models (HMMs). After an overview of the interface, we analyze the techniques used for gesture recognition and the decisions taken for mapping gestures with sound synthesis parameters. For the evaluation of the system as an interface for musical expression we made an experiment with real subjects. The results of this experiment are presented and analyzed.}
}