Mimi: Multi-modal Interaction for Musical Improvisation

Isaac Schankler, Alexandre François, and Elaine Chew

Music Proceedings of the International Conference on New Interfaces for Musical Expression

• Year: 2012
• Location: Ann Arbor, Michigan, U.S.A.

Abstract:

Program notes: Mimi, designed by Alexandre François with input from Elaine Chew and Isaac Schankler, is a multi-modal interactive musical improvisation system that explores the impact of visual feedback in performer-machine interaction. The Mimi system enables the performer to experiment with a unique blend of improvisation-like on-the-fly invention, composition-like planning and choreography, and expressive performance. Mimi's improvisations are created through a factor oracle. The visual interface gives the performer and the audience instantaneous and continuous information on the state of the oracle, its recombination strategy, the music to come, and that recently played. The performer controls when the system starts, stops, and learns, the playback volume, and the recombination rate. Mimi is not only an effective improvisation partner, it also provides a platform through which to interrogate the mental models necessary for successful improvisation. This performance also features custom synths and mechanisms for inter-oracle interaction created for Mimi by Isaac Schankler. Composer(s) Credits: Isaac Schankler, Alexandre François, Elaine Chew Instrumentalist(s) Credits: Isaac Schankler (keyboard & electronics), Mimi (keyboard & electronics) Artist(s) Biography: Isaac Schankler is a Los Angeles-based composer-improviser. His recent honors include a grant from Meet the Composer for his opera Light and Power, selection as finalist in the ASCAP/SEAMUS Composition Competition, and the Damien Top Prize in the ASCAP/Lotte Lehmann Foundation Art Song Competition. He is the Artist in Residence of the Music Computation and Cognition Laboratory (MuCoaCo) at the USC Viterbi School of Engineering, and an Artistic Director of the concert series People Inside Electronics. Isaac holds degrees in composition from the USC Thornton School of Music (DMA) and the University of Michigan (MM, BM). Elaine Chew is Professor of Digital Media at Queen Mary, University of London, and Director of Music Initiatives at the Centre for Digital Music. An operations researcher and pianist by training, her research goal is to de-mystify music and its performance through the use of formal scientific methods; as a performer, she collaborates with composers to present eclectic post-tonal music. She received PhD and SM degrees in Operations Research from MIT and a BAS in Music and Mathematical & Computational Sciences from Stanford. She is the recipient of NSF Career and PECASE awards, and a Radcliffe Institute for Advanced Studies fellowship. Alexandre R.J. François's research focuses on the modeling and design of interactive (software) systems, as an enabling step towards the understanding of perception and cognition. He was a 2007-2008 Fellow of the Radcliffe Institute for Advanced Study at Harvard University, where he co-led a music research cluster on Analytical Listening Through Interactive Visualization. François received the Dipl\^ome d'Ingénieur from the Institut National Agronomique Paris-Grignon in 1993, the Dipl\^ome d'Etudes Approfondies (M.S.) from the University Paris IX - Dauphine in 1994, and the M.S. and Ph.D. degrees in Computer Science from the University of Southern California in 1997 and 2000 respectively. Concert Venue and Time: Lydia Mendelssohn Theatre, Wednesday May 23, 7:00pm

Citation:

BibTeX Entry:

  @incollection{nime2012-music-SchanklerFrancoisChew2012,
 abstract = {Program notes:

Mimi, designed by Alexandre Fran\c{c}ois with input from Elaine Chew and Isaac Schankler, is a multi-modal interactive musical improvisation system that explores the impact of visual feedback in performer-machine interaction. The Mimi system enables the performer to experiment with a unique blend of improvisation-like on-the-fly invention, composition-like planning and choreography, and expressive performance. Mimi's improvisations are created through a factor oracle. The visual interface gives the performer and the audience instantaneous and continuous information on the state of the oracle, its recombination strategy, the music to come, and that recently played. The performer controls when the system starts, stops, and learns, the playback volume, and the recombination rate. Mimi is not only an effective improvisation partner, it also provides a platform through which to interrogate the mental models necessary for successful improvisation. This performance also features custom synths and mechanisms for inter-oracle interaction created for Mimi by Isaac Schankler.

Composer(s) Credits:

Isaac Schankler, Alexandre Fran\c{c}ois, Elaine Chew

Instrumentalist(s) Credits:

Isaac Schankler (keyboard \& electronics), Mimi (keyboard \& electronics)

Artist(s) Biography:

Isaac Schankler is a Los Angeles-based composer-improviser. His recent honors include a grant from Meet the Composer for his opera Light and Power, selection as finalist in the ASCAP/SEAMUS Composition Competition, and the Damien Top Prize in the ASCAP/Lotte Lehmann Foundation Art Song Competition. He is the Artist in Residence of the Music Computation and Cognition Laboratory (MuCoaCo) at the USC Viterbi School of Engineering, and an Artistic Director of the concert series People Inside Electronics. Isaac holds degrees in composition from the USC Thornton School of Music (DMA) and the University of Michigan (MM, BM).

Elaine Chew is Professor of Digital Media at Queen Mary, University of London, and Director of Music Initiatives at the Centre for Digital Music. An operations researcher and pianist by training, her research goal is to de-mystify music and its performance through the use of formal scientific methods; as a performer, she collaborates with composers to present eclectic post-tonal music. She received PhD and SM degrees in Operations Research from MIT and a BAS in Music and Mathematical \& Computational Sciences from Stanford. She is the recipient of NSF Career and PECASE awards, and a Radcliffe Institute for Advanced Studies fellowship.

Alexandre R.J. Fran\c{c}ois's research focuses on the modeling and design of interactive (software) systems, as an enabling step towards the understanding of perception and cognition. He was a 2007-2008 Fellow of the Radcliffe Institute for Advanced Study at Harvard University, where he co-led a music research cluster on Analytical Listening Through Interactive Visualization. Fran\c{c}ois received the Dipl\^{o}me d'Ing\'{e}nieur from the Institut National Agronomique Paris-Grignon in 1993, the Dipl\^{o}me d'Etudes Approfondies (M.S.) from the University Paris IX - Dauphine in 1994, and the M.S. and Ph.D. degrees in Computer Science from the University of Southern California in 1997 and 2000 respectively.

Concert Venue and Time: Lydia Mendelssohn Theatre, Wednesday May 23, 7:00pm},
 address = {Ann Arbor, Michigan, U.S.A.},
 author = {Isaac Schankler and Alexandre Fran\c{c}ois and Elaine Chew},
 booktitle = {Music Proceedings of the International Conference on New Interfaces for Musical Expression},
 day = {21-23},
 editor = {Georg Essl and Brent Gillespie and Michael Gurevich and Sile O'Modhrain},
 month = {May},
 publisher = {Electrical Engineering \& Computer Science and Performing Arts Technology, University of Michigan},
 title = {Mimi: Multi-modal Interaction for Musical Improvisation},
 year = {2012}
}