Synchronization of Multimodal Recordings for Musical Performance Research

Javier Jaimovich, and Benjamin Knapp

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

The past decade has seen an increase of low-cost technology for sensor data acquisition, which has been utilized for the expanding field of research in gesture measurement for music performance. Unfortunately, these devices are still far from being compatible with the audiovisual recording platforms which have been used to record synchronized streams of data. In this paper, we describe a practical solution for simultaneous recording of heterogeneous multimodal signals. The recording system presented uses MIDI Time Code to time-stamp sensor data and to synchronize with standard video and audio recording systems. We also present a set of tools for recording sensor data, as well as a set of analysis tools to evaluate in realtime the sample rate of different signals, and the overall synchronization status of the recording system.

Citation:

Javier Jaimovich, and Benjamin Knapp. 2010. Synchronization of Multimodal Recordings for Musical Performance Research. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1177815

BibTeX Entry:

  @inproceedings{Jaimovich2010a,
 abstract = {The past decade has seen an increase of low-cost technology for sensor data acquisition, which has been utilized for the expanding field of research in gesture measurement for music performance. Unfortunately, these devices are still far from being compatible with the audiovisual recording platforms which have been used to record synchronized streams of data. In this paper, we describe a practical solution for simultaneous recording of heterogeneous multimodal signals. The recording system presented uses MIDI Time Code to time-stamp sensor data and to synchronize with standard video and audio recording systems. We also present a set of tools for recording sensor data, as well as a set of analysis tools to evaluate in realtime the sample rate of different signals, and the overall synchronization status of the recording system. },
 address = {Sydney, Australia},
 author = {Jaimovich, Javier and Knapp, Benjamin},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1177815},
 issn = {2220-4806},
 keywords = {Synchronization, Multimodal Signals, Sensor Data Acquisition, Signal Recording.},
 pages = {372--374},
 title = {Synchronization of Multimodal Recordings for Musical Performance Research},
 url = {http://www.nime.org/proceedings/2010/nime2010_372.pdf},
 year = {2010}
}