Multimodal Data-based Deep Learning Model for Human Behavior Recognition towards Smart Health-oriented Office-working

Human factors is the key component of human-centric systems for both human well-being and overall system performance. With such a concern, strategic development plans have been proposed in different countries, such as Healthy China Initiative. The advancement of Internet-of-Things (IoT), intelligent data analytics, and service innovation contribute to the prevailing development of modern office work. A new paradigm named smart health-oriented office-working (SHOW) is defined to prevention and control measures of health issues associated with office work. It is capable of understanding the context and adapting to their demands. Human behavior recognition system is essential to incorporate intelligence into SHOW, however, several issues still exist, including ineffective use of multimodal data, and lack of a privacy-preserving and unobtrusive method. Prevailing studies have utilized cameras, wearables, smartphones, pressure sensors and infrared array sensors to recognize human behavior, by considering the pros and cons of these approaches, we have explored different multimodal data combinations. Moreover, a deep learning model-based recognition algorithm was developed with the adoption of a feature-level fusion strategy. Extensive experiments are conducted to examine and validate the performance of the proposed model using a self-collected dataset. We hope this study would bring an interdisciplinary and collaborative perspective for smart health-oriented office-working and encourage more research in this emerging and promising field.