Score-Transformer: A Deep Learning Aid for Music Composition

Jeffrey A. T. Lupker

Proceedings of the International Conference on New Interfaces for Musical Expression

• Year: 2021
• Location: Shanghai, China
• Article Number: 59
• DOI: 10.21428/92fbeb44.21d4fd1f (Link to paper)
• PDF link
• Presentation Video

Abstract:

Creating an artificially intelligent (AI) aid for music composers requires a practical and modular approach, one that allows the composer to manipulate the technology when needed in the search for new sounds. Many existing approaches fail to capture the interest of composers as they are limited beyond their demonstrative purposes, allow for only minimal interaction from the composer or require GPU access to generate samples quickly. This paper introduces Score-Transformer (ST), a practical integration of deep learning technology to aid in the creation of new music which works seamlessly alongside any popular software notation (Finale, Sibelius, etc.). Score-Transformer is built upon a variant of the powerful transformer model, currently used in state-of-the-art natural language models. Owing to hierarchical and sequential similarities between music and language, the transformer model can learn to write polyphonic MIDI music based on any styles, genres, or composers it is trained upon. This paper briefly outlines how the model learns and later notates music based upon any prompt given to it from the user. Furthermore, ST can be updated at any time on additional MIDI recordings minimizing the risk of the software becoming outdated or impractical for continued use.

Citation:

BibTeX Entry:

  @inproceedings{NIME21_59,
 abstract = {Creating an artificially intelligent (AI) aid for music composers requires a practical and modular approach, one that allows the composer to manipulate the technology when needed in the search for new sounds. Many existing approaches fail to capture the interest of composers as they are limited beyond their demonstrative purposes, allow for only minimal interaction from the composer or require GPU access to generate samples quickly. This paper introduces Score-Transformer (ST), a practical integration of deep learning technology to aid in the creation of new music which works seamlessly alongside any popular software notation (Finale, Sibelius, etc.). Score-Transformer is built upon a variant of the powerful transformer model, currently used in state-of-the-art natural language models. Owing to hierarchical and sequential similarities between music and language, the transformer model can learn to write polyphonic MIDI music based on any styles, genres, or composers it is trained upon. This paper briefly outlines how the model learns and later notates music based upon any prompt given to it from the user. Furthermore, ST can be updated at any time on additional MIDI recordings minimizing the risk of the software becoming outdated or impractical for continued use.},
 address = {Shanghai, China},
 articleno = {59},
 author = {Lupker, Jeffrey A. T.},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.21428/92fbeb44.21d4fd1f},
 issn = {2220-4806},
 month = {June},
 presentation-video = {https://youtu.be/CZO8nj6YzVI},
 title = {Score-Transformer: A Deep Learning Aid for Music Composition},
 url = {https://nime.pubpub.org/pub/7a6ij1ak},
 year = {2021}
}