Sketch-Based Musical Composition and Performance

Haojing Diao, Yanchao Zhou, Christopher Andrew Harte, and Nick Bryan-Kinns

Proceedings of the International Conference on New Interfaces for Musical Expression

• Year: 2014
• Location: London, United Kingdom
• Pages: 569–572
• DOI: 10.5281/zenodo.1178748 (Link to paper)
• PDF link

Abstract:

Sketching is a natural way for one person to convey their thoughts and intentions to another. With the recent rise of tablet-based computing, the use of sketching as a control and interaction paradigm is one that deserves exploration. In this paper we present an interactive sketch-based music composition and performance system called Drawchestra. The aim of the system is to give users an intuitive way to convey their musical ideas to a computer system with the minimum of technical training thus enabling them to focus on the creative tasks of composition and performance. The system provides the user with a canvas upon which they may create their own instruments by sketching shapes on the tablet screen. The system recognises a certain set of shapes which it treats as virtual instruments or effects. Once recognised, these virtual instruments can then be played by the user in real time. The size of a sketched instrument shape is used to control certain parameters of the sound so the user can build complex orchestras containing many different shapes of different sizes. The sketched shapes may also be moved and resized as desired making it possible to customise and edit the virtual orchestra as the user goes along. The system has been implemented in Python and user tests conducted using an iPad as the control surface. We report the results of the user study at the end of the paper before briefly discussing the outcome and outlining the next steps for the system design.

Citation:

BibTeX Entry:

  @inproceedings{hdiao2014,
 abstract = {Sketching is a natural way for one person to convey their thoughts and intentions to another. With the recent rise of tablet-based computing, the use of sketching as a control and interaction paradigm is one that deserves exploration. In this paper we present an interactive sketch-based music composition and performance system called Drawchestra. The aim of the system is to give users an intuitive way to convey their musical ideas to a computer system with the minimum of technical training thus enabling them to focus on the creative tasks of composition and performance. The system provides the user with a canvas upon which they may create their own instruments by sketching shapes on the tablet screen. The system recognises a certain set of shapes which it treats as virtual instruments or effects. Once recognised, these virtual instruments can then be played by the user in real time. The size of a sketched instrument shape is used to control certain parameters of the sound so the user can build complex orchestras containing many different shapes of different sizes. The sketched shapes may also be moved and resized as desired making it possible to customise and edit the virtual orchestra as the user goes along. The system has been implemented in Python and user tests conducted using an iPad as the control surface. We report the results of the user study at the end of the paper before briefly discussing the outcome and outlining the next steps for the system design.},
 address = {London, United Kingdom},
 author = {Haojing Diao and Yanchao Zhou and Christopher Andrew Harte and Nick Bryan-Kinns},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1178748},
 issn = {2220-4806},
 month = {June},
 pages = {569--572},
 publisher = {Goldsmiths, University of London},
 title = {Sketch-Based Musical Composition and Performance},
 url = {http://www.nime.org/proceedings/2014/nime2014_517.pdf},
 year = {2014}
}