Collaborative Edge Computing transforms AIoT by enabling real-time processing and enhanced privacy, paving the way for smarter applications in healthcare, robotics, and beyond.

Study conducted by Prof. Jiannong CAO and his research team

AIoT (Artificial intelligence of things) is the fusion of AI and IoT. By 2025, there are projected to be 42 billion IoT-connected devices globally, generating a wealth of data every day. Meanwhile, AI has shown its great power in solving complex problems by learning from big data. With AIoT, almost everything will become smarter, including smart cities, smart healthcare, industry 4.0, and smart grids.

Cloud computing is a major enabling technology for AIoT, running computation tasks and applications on cloud servers in data centres, with all the data collected from end devices. This consumes network bandwidth, creates processing delays, and raises privacy concerns. To alleviate these concerns, edge computing technology emerged. Such technology pushes computation out to base stations, edge servers, and gateways at the network’s periphery. Offloading computation tasks from resource-limited IoT devices to more powerful, resource-richer edge nodes can lower costs, deliver real time results, and preserve privacy. However, current edge computing technologies are insufficient to support advanced AIoT applications, such as autonomous driving, intelligent manufacturing, and the metaverse, which demand more computing power at the edge, which in turn impose higher requirements for real-time responses and intelligence.

As a corollary of the above, the research team led by Jiannong CAO, Dean of Graduate School, Otto Poon Charitable Foundation Professor in Data Science and Chair Professor of Distributed and Mobile Computing in the Department of Computing at the Hong Kong Polytechnic University, envisions that the future is Collaborative Edge Computing (CEC), where cloud, edge and terminals join forces to support AIoT applications. This bringing together allows for real-time processing, hyper-connectivity, and dynamic access, overcoming the limitations of traditional edge computing technologies. Their research focuses on two aspects. One is collaborative edge intelligence platform, which integrates heterogeneous resources across geo-distributed cloud servers and edge nodes, facilitating high-performance distributed task processing to meet application requirements, supporting fast and adaptive AI model development and deployment. Another is innovative and impactful AIoT applications built upon the platform, including but not limited to food safety, video analytics, multi-robot system, and large language models.

Figure 1. Collaborative edge resource management system supporting joint computing and networking resource scheduling.

Collaborative edge AI platform (CEAI): To manage the large-scale geo-distributed edge nodes and provide a unified execution environment for AIoT applications, the research team designed and developed a collaborative edge resource management system¹. The system adopts a master-client architecture. While existing works mainly consider computation resources, their system leverages two controllers in the master node to manage the computation and networking resources of edge nodes jointly. The controllers collaborate under the coordination of a novel collaborative online scheduler, that runs intelligent scheduling algorithms to generate the task scheduling policies to optimise application performance (Figure 1). To better support AIoT applications, the research team proposed E-tree², a configurable distributed model training paradigm, to support large-scale model training (Figure 2).

Figure 2. Structure overview of E-Tree training, supporting large-scale edge model training with fast convergence.

CEAI-enabled food safety management: Food freshness detection using smartphones is a promising approach to ensure food quality and safety. However, challenges arise due to the need for hyperspectral imaging, specialised equipment, and real-time processing. The research team addresses these challenges by leveraging transfer learning and edge computing. They train a neural network to translate natural images to hyperspectral images, enabling food freshness assessment. Real-time processing on the smartphone is achieved by deploying the model on edge devices. The classification performance and the ability to detect subtle changes in food texture demonstrate the effectiveness of their approach³.

CEAI-enabled real-time video analytics: The research team applied CEAI to improve public healthcare in a public hospital in the region. Specifically, they designed a set of AI models to recognise sequences of human behaviours from video, achieving high accuracy in hand hygiene monitoring and patient fall detection in real clinical environment. Therefore, their platform can dramatically reduce the risk of virus transmission via hands and the risk of patient falls. The system deploys AI models on local edge devices in a real-time responsive manner, being self-contained without sending privacy-sensitive videos to external cloud servers. The edge-AI based healthcare monitoring system latency is under 500ms, achieving over 80% accuracy⁴.

CEAI-enabled Autonomous Multi-robot System: Traditional robots often rely on a centralised architecture, which suffers from issues of latency, security risks, and scalability. To overcome these issues, the research team focuses on designing and developing an edge-AI robot to process data and make decisions at the network’s edge. For instance, they have designed a deformable robot prototype and a soft modular robot for pipeline inspection. They have trained a defect classification model using advanced edge learning techniques and embedded the model into robot systems so that the robot can perform real-time inspection without an external connection to the server. Furthermore, the research team has proposed distributed and hierarchical reinforcement learning algorithms for multi-robot cooperation and developed a real-world testbed for training and evaluation⁵.

CEAI-enabled Large Language Models: Recently, large language models (LLMs) have attracted significant attention and shown great potential in AIoT applications. However, current LLMs heavily rely on the cloud, and deploying them on heterogenous edge devices is challenging. The research team designed EdgeShard, a novel approach to partition a computation-intensive LLM into affordable shards and deploy them on distributed collaborative devices (Figure 3). The partition and distribution are non-trivial, considering device heterogeneity, bandwidth limitations, and model complexity. To this end, they formulated an adaptive joint device selection and model partition problem and designed an efficient dynamic programming algorithm to optimise the inference latency and throughput⁶.

Figure 3. Framework of EdgeLLM. It consists of three stages: offline profiling, task scheduling optimisation, and online collaborative LLM inference.

AIoT is the future of IoT. The collaborative edge computing approach aims to uniformly manage the geo-distributed edge nodes and build services to enable impactful AIoT applications. It has significant impacts on both academia and industry. Specifically, for academia, CAO’s research involves cutting-edge technologies and diverse areas, including cloud and edge computing, AI, smart building, and smart health. Thus, it will attract researchers in related areas to conduct cross-domain studies and inspire more research. For industry, CAO’s research helps service providers to deploy their applications in large-scale areas with reduced costs, inspiring more innovative AIoT applications.

1. Zhang, M., Cao, J., Yang, L., Zhang, L., Sahni, Y., & Jiang, S. (2022). ENTS: An Edge-native Task Scheduling System for Collaborative Edge Computing. Proceedings of the ACM/IEEE Seventh Symposium on Edge Computing (SEC), Seattle, WA, United States, 149–161.

https://doi.org/10.48550/arxiv.2210.07842
2. Yang, L., Lu, Y., Cao, J., Huang, J., & Zhang, M. (2021). E-Tree Learning: A Novel Decentralized Model Learning Framework for Edge AI. IEEE Internet of Things Journal, 8(14), 11290–11304.

https://doi.org/10.1109/JIOT.2021.3052195
3. Saxena, D., Kulshrestha, T., Cao, J., & Cheung, S.-C. (2022). Multi-Constraint Adversarial Networks for Unsupervised Image-to-Image Translation. IEEE Transactions on Image Processing, 31, 1601–1612. https://doi.org/10.1109/TIP.2022.3144886
4. Zhang, M., Cao, J., Sahni, Y., Chen, Q., Jiang, S., & Yang, L. (2023). Blockchain-based Collaborative Edge Intelligence for Trustworthy and Real-Time Video Surveillance. IEEE Transactions on Industrial Informatics, 19(2), 1623–1633.

https://doi.org/10.1109/TII.2022.3203397
5. Liang, Z., Cao, J., Jiang, S., & Xu, H. (2024). Hierarchical Reinforcement Learning with Partner Modeling for Distributed Multiagent Cooperation. IEEE Transactions on Parallel and Distributed Systems, 1–13. https://doi.org/10.1109/TPDS.2024.3457153
6. Zhang, M., Cao, J., Shen, X., & Cui, Z. (2024). EdgeShard: Efficient LLM Inference via Collaborative Edge Computing. arXiv.Org. https://doi.org/10.48550/arxiv.2405.14371

Professor Jiannong CAO Chair Professor of Distributed and Mobile Computing in the Department of Computing