Stitch: a Knitting-powered Musical Interface using Computer Vision

Kate Bosen, and Dan Overholt

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

This paper describes a new instrument for musical expres- sion that makes music from knitting. This interface uses only knitting needles, yarn, and a computer as hardware. The webcam input on a laptop captures the player knit- ting in real-time, and a bespoke MaxMSP patch processes the incoming data stream. Movements are detected using computer vision principles to identify shapes, lines, and the motions of the performer’s stitches. Gestures the performer uses are then mapped to a synthesizer that produces mu- sic according to how the player moves, while they knit and purl. Each performance varies due to the speed at which the performer knits, the technical knitting style of the per- former, the kinds of stitches cast on the needles, the color and texture of yarn used during performance, and the size of the knitting project.

Citation:

Kate Bosen, and Dan Overholt. 2024. Stitch: a Knitting-powered Musical Interface using Computer Vision. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.13904888

BibTeX Entry:

  @article{nime2024_57,
 abstract = {This paper describes a new instrument for musical expres- sion that makes music from knitting. This interface uses only knitting needles, yarn, and a computer as hardware. The webcam input on a laptop captures the player knit- ting in real-time, and a bespoke MaxMSP patch processes the incoming data stream. Movements are detected using computer vision principles to identify shapes, lines, and the motions of the performer’s stitches. Gestures the performer uses are then mapped to a synthesizer that produces mu- sic according to how the player moves, while they knit and purl. Each performance varies due to the speed at which the performer knits, the technical knitting style of the per- former, the kinds of stitches cast on the needles, the color and texture of yarn used during performance, and the size of the knitting project.},
 address = {Utrecht, Netherlands},
 articleno = {57},
 author = {Kate Bosen and Dan Overholt},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.13904888},
 editor = {S M Astrid Bin and Courtney N. Reed},
 issn = {2220-4806},
 month = {September},
 numpages = {5},
 pages = {390--394},
 presentation-video = {https://youtu.be/bs9guoMieEI?si=aPphSzJqzvUHDlip},
 title = {Stitch: a Knitting-powered Musical Interface using Computer Vision},
 track = {Papers},
 url = {http://nime.org/proceedings/2024/nime2024_57.pdf},
 year = {2024}
}