FabricKeyboard: Multimodal Textile Sensate Media as an Expressive and Deformable Musical Interface

Irmandy Wicaksono, and Joseph Paradiso

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract

This paper presents FabricKeyboard: a novel deformable keyboard interface based on a multi-modal fabric sensate surface. Multi-layer fabric sensors that detect touch, proximity, electric field, pressure, and stretch are machine-sewn in a keyboard pattern on a stretchable substrate. The result is a fabric-based musical controller that combines both the discrete controls of a keyboard and various continuous controls from the embedded fabric sensors. This enables unique tactile experiences and new interactions both with physical and non-contact gestures: physical by pressing, pulling, stretching, and twisting the keys or the fabric and non-contact by hovering and waving towards/against the keyboard and an electromagnetic source. We have also developed additional fabric-based modular interfaces such as a ribbon-controller and trackpad, allowing performers to add more expressive and continuous controls. This paper will discuss implementation strategies for our system-on-textile, fabric-based sensor developments, as well as sensor-computer interfacing and musical mapping examples of this multi-modal and expressive fabric keyboard.

Citation

Irmandy Wicaksono, and Joseph Paradiso. 2017. FabricKeyboard: Multimodal Textile Sensate Media as an Expressive and Deformable Musical Interface. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1176278

BibTeX Entry

@inproceedings{iwicaksono2017,
 abstract = {This paper presents FabricKeyboard: a novel deformable keyboard interface based on a multi-modal fabric sensate surface. Multi-layer fabric sensors that detect touch, proximity, electric field, pressure, and stretch are machine-sewn in a keyboard pattern on a stretchable substrate. The result is a fabric-based musical controller that combines both the discrete controls of a keyboard and various continuous controls from the embedded fabric sensors. This enables unique tactile experiences and new interactions both with physical and non-contact gestures: physical by pressing, pulling, stretching, and twisting the keys or the fabric and non-contact by hovering and waving towards/against the keyboard and an electromagnetic source. We have also developed additional fabric-based modular interfaces such as a ribbon-controller and trackpad, allowing performers to add more expressive and continuous controls. This paper will discuss implementation strategies for our system-on-textile, fabric-based sensor developments, as well as sensor-computer interfacing and musical mapping examples of this multi-modal and expressive fabric keyboard.  },
 address = {Copenhagen, Denmark},
 author = {Irmandy Wicaksono and Joseph Paradiso},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1176278},
 issn = {2220-4806},
 pages = {348--353},
 publisher = {Aalborg University Copenhagen},
 title = {FabricKeyboard: Multimodal Textile Sensate Media as an Expressive and Deformable Musical Interface},
 url = {http://www.nime.org/proceedings/2017/nime2017_paper0066.pdf},
 year = {2017}
}