Auraglyph: Handwritten Computer Music Composition and Design

Spencer Salazar, and Ge Wang

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

Effective software interaction design must consider all of the capabilities and limitations of the platform for which it is developed. To this end, we propose a new model for computer music system design on touchscreen devices, combining both pen/stylus input and multitouch gestures. Such a model surpasses the barrier of touchscreen-based keyboard input, preserving the primary interaction of touch and direct manipulation throughout the development of a complex musical program. We have implemented an iPad software application utilizing these principles, called ``Auraglyph.'' Auraglyph offers a number of fundamental audio processing and control operators, as well as facilities for structured input and output. All of these software objects are created, parameterized, and interconnected via stylus and touch input. Underlying this application is an advanced handwriting recognition framework, LipiTk, which can be trained to recognize both alphanumeric characters and arbitrary figures, shapes, and patterns.

Citation:

Spencer Salazar, and Ge Wang. 2014. Auraglyph: Handwritten Computer Music Composition and Design. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1178927

BibTeX Entry:

  @inproceedings{ssalazar2014,
 abstract = {Effective software interaction design must consider all of the capabilities and limitations of the platform for which it is developed. To this end, we propose a new model for computer music system design on touchscreen devices, combining both pen/stylus input and multitouch gestures. Such a model surpasses the barrier of touchscreen-based keyboard input, preserving the primary interaction of touch and direct manipulation throughout the development of a complex musical program. We have implemented an iPad software application utilizing these principles, called ``Auraglyph.'' Auraglyph offers a number of fundamental audio processing and control operators, as well as facilities for structured input and output. All of these software objects are created, parameterized, and interconnected via stylus and touch input. Underlying this application is an advanced handwriting recognition framework, LipiTk, which can be trained to recognize both alphanumeric characters and arbitrary figures, shapes, and patterns.},
 address = {London, United Kingdom},
 author = {Spencer Salazar and Ge Wang},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1178927},
 issn = {2220-4806},
 month = {June},
 pages = {106--109},
 publisher = {Goldsmiths, University of London},
 title = {Auraglyph: Handwritten Computer Music Composition and Design},
 url = {http://www.nime.org/proceedings/2014/nime2014_560.pdf},
 year = {2014}
}