BachDuet: A Deep Learning System for Human-Machine Counterpoint Improvisation

Christodoulos Benetatos, Joseph VanderStel, and Zhiyao Duan

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

During theBaroque period, improvisation was a key element of music performance and education. Great musicians, such as J.S. Bach, were better known as improvisers than composers. Today, however, there is a lack of improvisation culture in classical music performance and education; classical musicians either are not trained to improvise, or cannot find other people to improvise with. Motivated by this observation, we develop BachDuet, a system that enables real-time counterpoint improvisation between a human anda machine. This system uses a recurrent neural network toprocess the human musician’s monophonic performance ona MIDI keyboard and generates the machine’s monophonic performance in real time. We develop a GUI to visualize the generated music content and to facilitate this interaction. We conduct user studies with 13 musically trained users and show the feasibility of two-party duet counterpoint improvisation and the effectiveness of BachDuet for this purpose. We also conduct listening tests with 48 participants and show that they cannot tell the difference between duets generated by human-machine improvisation using BachDuet and those generated by human-human improvisation. Objective evaluation is also conducted to assess the degree to which these improvisations adhere to common rules of counterpoint, showing promising results.

Citation:

Christodoulos Benetatos, Joseph VanderStel, and Zhiyao Duan. 2020. BachDuet: A Deep Learning System for Human-Machine Counterpoint Improvisation. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.4813234

BibTeX Entry:

  @inproceedings{NIME20_125,
 abstract = {During theBaroque period, improvisation was a key element of music performance and education. Great musicians, such as J.S. Bach, were better known as improvisers than composers. Today,  however,  there  is  a  lack  of  improvisation culture in classical music performance and education; classical musicians either are not trained to improvise, or cannot find other people to improvise with.  Motivated by this observation,  we  develop BachDuet,  a  system  that  enables real-time counterpoint improvisation between a human anda machine.  This system uses a recurrent neural network toprocess the human musician’s monophonic performance ona MIDI keyboard and generates the machine’s monophonic performance in real time. We  develop a GUI to visualize the generated music content and to facilitate this interaction. We  conduct  user  studies  with  13  musically  trained users  and  show  the  feasibility  of  two-party  duet  counterpoint improvisation and the effectiveness of BachDuet for this purpose.  We also conduct listening tests with 48 participants and show that they cannot tell the difference between duets generated by human-machine improvisation using BachDuet and those generated by human-human improvisation.  Objective evaluation is also conducted to assess the degree to which these improvisations adhere to common rules of counterpoint, showing promising results.},
 address = {Birmingham, UK},
 author = {Benetatos, Christodoulos and VanderStel, Joseph and Duan, Zhiyao},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.4813234},
 editor = {Romain Michon and Franziska Schroeder},
 issn = {2220-4806},
 month = {July},
 pages = {635--640},
 presentation-video = {https://youtu.be/wFGW0QzuPPk},
 publisher = {Birmingham City University},
 title = {BachDuet: A Deep Learning System for Human-Machine Counterpoint Improvisation},
 url = {https://www.nime.org/proceedings/2020/nime2020_paper125.pdf},
 year = {2020}
}