Evaluating the Perceived Similarity Between Audio-Visual Features Using Corpus-Based Concatenative Synthesis

Augoustinos Tsiros

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

This paper presents the findings of two exploratory studies. In these studies participants performed a series of image-sound association tasks. The aim of the studies was to investigate the perceived similarity and the efficacy of two multidimensional mappings each consisting of three audio-visual associations. The purpose of the mappings is to enable visual control of corpus-based concatenative synthesis. More specifically the stimuli in the first study was designed to test the perceived similarity of six audio-visual associations, between the two mappings using three corpora resulting in 18 audio-visual stimuli. The corpora differ in terms of two sound characteristics: harmonic contain and continuity. Data analysis revealed no significant differences in the participant's responses between the three corpora, or between the two mappings. However highly significant differences were revealed between the individual audio-visual association pairs. The second study investigates the affects of the mapping and the corpus in the ability of the participants to detect which image out of three similar images was used to generate six audio stimuli. The data analysis revealed significant differences in the ability of the participants' to detect the correct image depending on which corpus was used. Less significant was the effect of the mapping in the success rate of the participant responses.

Citation:

Augoustinos Tsiros. 2014. Evaluating the Perceived Similarity Between Audio-Visual Features Using Corpus-Based Concatenative Synthesis. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1178965

BibTeX Entry:

  @inproceedings{atsiros12014,
 abstract = {This paper presents the findings of two exploratory studies. In these studies participants performed a series of image-sound association tasks. The aim of the studies was to investigate the perceived similarity and the efficacy of two multidimensional mappings each consisting of three audio-visual associations. The purpose of the mappings is to enable visual control of corpus-based concatenative synthesis. More specifically the stimuli in the first study was designed to test the perceived similarity of six audio-visual associations, between the two mappings using three corpora resulting in 18 audio-visual stimuli. The corpora differ in terms of two sound characteristics: harmonic contain and continuity. Data analysis revealed no significant differences in the participant's responses between the three corpora, or between the two mappings. However highly significant differences were revealed between the individual audio-visual association pairs. The second study investigates the affects of the mapping and the corpus in the ability of the participants to detect which image out of three similar images was used to generate six audio stimuli. The data analysis revealed significant differences in the ability of the participants' to detect the correct image depending on which corpus was used. Less significant was the effect of the mapping in the success rate of the participant responses.},
 address = {London, United Kingdom},
 author = {Augoustinos Tsiros},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1178965},
 issn = {2220-4806},
 month = {June},
 pages = {421--426},
 publisher = {Goldsmiths, University of London},
 title = {Evaluating the Perceived Similarity Between Audio-Visual Features Using Corpus-Based Concatenative Synthesis},
 url = {http://www.nime.org/proceedings/2014/nime2014_484.pdf},
 year = {2014}
}