FabricKeyboard: Multimodal Textile Sensate Media as an Expressive and Deformable Musical Interface

Irmandy Wicaksono, and Joseph Paradiso

Proceedings of the International Conference on New Interfaces for Musical Expression

Abstract:

This paper presents FabricKeyboard: a novel deformable keyboard interface based on a multi-modal fabric sensate surface. Multi-layer fabric sensors that detect touch, proximity, electric field, pressure, and stretch are machine-sewn in a keyboard pattern on a stretchable substrate. The result is a fabric-based musical controller that combines both the discrete controls of a keyboard and various continuous controls from the embedded fabric sensors. This enables unique tactile experiences and new interactions both with physical and non-contact gestures: physical by pressing, pulling, stretching, and twisting the keys or the fabric and non-contact by hovering and waving towards/against the keyboard and an electromagnetic source. We have also developed additional fabric-based modular interfaces such as a ribbon-controller and trackpad, allowing performers to add more expressive and continuous controls. This paper will discuss implementation strategies for our system-on-textile, fabric-based sensor developments, as well as sensor-computer interfacing and musical mapping examples of this multi-modal and expressive fabric keyboard.

Citation:

Irmandy Wicaksono, and Joseph Paradiso. 2017. FabricKeyboard: Multimodal Textile Sensate Media as an Expressive and Deformable Musical Interface. Proceedings of the International Conference on New Interfaces for Musical Expression. DOI: 10.5281/zenodo.1176278

BibTeX Entry:

  @inproceedings{iwicaksono2017,
 abstract = {This paper presents FabricKeyboard: a novel deformable keyboard interface based on a multi-modal fabric sensate surface. Multi-layer fabric sensors that detect touch, proximity, electric field, pressure, and stretch are machine-sewn in a keyboard pattern on a stretchable substrate. The result is a fabric-based musical controller that combines both the discrete controls of a keyboard and various continuous controls from the embedded fabric sensors. This enables unique tactile experiences and new interactions both with physical and non-contact gestures: physical by pressing, pulling, stretching, and twisting the keys or the fabric and non-contact by hovering and waving towards/against the keyboard and an electromagnetic source. We have also developed additional fabric-based modular interfaces such as a ribbon-controller and trackpad, allowing performers to add more expressive and continuous controls. This paper will discuss implementation strategies for our system-on-textile, fabric-based sensor developments, as well as sensor-computer interfacing and musical mapping examples of this multi-modal and expressive fabric keyboard.  },
 address = {Copenhagen, Denmark},
 author = {Irmandy Wicaksono and Joseph Paradiso},
 booktitle = {Proceedings of the International Conference on New Interfaces for Musical Expression},
 doi = {10.5281/zenodo.1176278},
 issn = {2220-4806},
 pages = {348--353},
 publisher = {Aalborg University Copenhagen},
 title = {FabricKeyboard: Multimodal Textile Sensate Media as an Expressive and Deformable Musical Interface},
 url = {http://www.nime.org/proceedings/2017/nime2017_paper0066.pdf},
 year = {2017}
}