The world’s first contactless gesture recognition illuminative textile that changes colour via hand and body gestures
Nov 2022
Developed by Dr Jeanne Tan and her team at AiDLab (Laboratory for Artificial Intelligence in Design), the patented intelligent textile system is the world’s first contactless gesture recognition illuminative textile that changes colour via hand and body gestures.
It possesses 2 unique features: An offline system based on a self-built algorithm and a patented illuminative Polymeric Optical Fibre (POF) knitted textile. The technology had transformed conventionally passive textiles into an intuitive and interactive platform that enables immediate colour illumination customisation. It seamlessly integrates technology with the soft tactility of knitted materials. The textile can be knitted in a wide variety of surface patterns, textures, and weights.
The design rationale was to create an interactive textile that is intuitive to use, enabling users of all ages, backgrounds, and abilities to interact, customise and express themselves with the comfort and tactile familiarity of an omnipresent material.
The intelligent textile is comprised of a contactless gesture recognition system and an illuminative textile knitted from polymeric optical fibres and textile-based yarns. The former enables contactless engagement, and the latter serves as a fabric platform for different colour illuminations. It looks and feels like a typical textile which will not look obtrusive in everyday environments and can be applied for wearables and products. It can be produced via industrial flatbed knitting machines thus enabling the textiles to be produced on a mass scale.
The system is based on computer vision and deep learning that is facilitated by an integrated camera; a self-built Euclidean distance-based gesture recognition algorithm processed by minicomputer. The camera captures imagery of the hands to compute and decipher it via 21 key landmarks on the hands and provide response in the form of illumination according to the predefined gestures. The system is trained to detect and calculate distances between landmarks on the hands, it recognises gestures with immediate feedback via coloured illuminations, providing effortless and intuitive interactions. The off-line system also enables the intelligent textile to work anywhere without the need for large computational power hence reducing the bulk of components and enhancing portability.
The intelligent textile is unique. It is the first contactless gesture recognition illuminating textile system, typical gesture recognition materials on the market are often reliant on physical contact or the donning of the material. Typical illuminative textiles are woven materials that have fixed structures while the knitted textile possess stretch. The system is customisable in terms of the programmed gestures and different surface design, textures, and weights can be created to accommodate to the application purpose.
The immediate adaptability of the intelligent textile enables it to effortlessly meet the fast-evolving needs of contemporary consumers lending to product longevity and sustainability. The ability to customise interaction and colours within multiuser contexts will reduce the need to purchase copious products to meet different needs. The knitting process enables the creation of shaped pieces to fit the product construction thus only using the yarn and POFs required, resulting in very little material wastage as compared to the conventional cut and sew process of woven textiles that lends to material wastage.
The intelligent textile system had been licensed to the Hong Kong Sheng Kung Hui Welfare Council for installation at the Wong Tai Sin District Health Centre that serves over 400,000 residents in the area.
