PolyU project develops a Vision-Language Models for driving assistant, supported by Smart Traffic Fund

In complex driving scenarios, information overload can lead to distractions and delayed reactions. The introduction of Vision-Language models (VLMs) presents new possibilities for advanced driver assistance systems (ADAS).

Led by Prof. Pai ZHENG, Associate Professor of the Department of Industrial and Systems Engineering of The Hong Kong Polytechnic University (PolyU), the project titled “Develop a Vision-Language Model-based Smart Driving Assistant for Enhancing Safety and Convenience of Motorists” has successfully secured over HK$5 million for a duration of 24 months from the Smart Traffic Fund.           

This project aims to develop a smart driving assistant for vehicle cockpits leveraging vision-language models. The objective is to improve driver safety and convenience by analyzing the environment in real time and drivers’ needs to offer appropriate interactive strategy.

Prof. ZHENG said, “Besides existing ADAS, our approach offers personalised driving tips to enhance the user experience. The system will collect and analyse personalised interaction data from users, including interactive language descriptions and visual environment information acquired during user participation experiments. This system will dynamically retrieve and generate customised driving strategies based on historical and real-time data, catering to the habits and preferences of individual drivers.”

PolyU has long been committed to the research and application of vehicle-related innovation and technology, with a total of 23 projects receiving grants from the Smart Traffic Fund to date.

The Smart Traffic Fund provides funding support to local organisations and enterprises for conducting research and applying innovation and technology to enhance commuting convenience, enhance the efficiency of the road network or road space, and improve driving safety.

***END***