PolyU unveils large language model-based tourist satisfaction index, providing comprehensive analysis to enhance Hong Kong tourism service quality

As tourism, one of the major pillars of the Hong Kong economy, gradually recovers from the pandemic, the government has planned a series of measures to promote the all-round development of the tourism industry. The Research Centre for Digital Transformation of Tourism (RCdTT) of The Hong Kong Polytechnic University (PolyU) School of Hotel and Tourism Management (SHTM) has developed an innovative, large language model (LLM)-based assessment framework – the Hong Kong Tourist Satisfaction Index (HKTSI) – to evaluate the satisfaction level of inbound tourists to Hong Kong from 2012 to 2024 across different tourism-related sectors, temporal scales and regions. Findings reveal that, while performance variations across different regions were clearly indicated, the TSI rebounded and reached its highest-ever recorded score after the temporary decline during the pandemic.

Under the leadership of Prof. Haiyan SONG, Principal Investigator and SHTM Associate Dean, RCdTT Director and Mr and Mrs Chan Chak Fu Professor in International Tourism, the TSI was initially introduced in 2009, using a questionnaire-based methodology to evaluate tourists’ satisfaction with various tourism services. Seeking to provide a more comprehensive and accurate analysis, the team has transformed the framework by adopting an interdisciplinary approach that integrates theories from management science, economics and computer science, and leveraging the advanced LLM technology.

The enhanced TSI framework offers multi-dimensional analysis, enabling TSI measurement across different dimensions (such as service topics, service sectors and trip types), temporal scales (monthly and annually) and regions. This adaptable approach provides stakeholders with timely, data-driven insights to guide service improvement, policy development and strategic decision-making.

Prof. Kaye CHON, SHTM Dean, Chair Professor and Walter and Wendy Kwok Family Foundation Professor in International Hospitality Management, said, “Hong Kong’s competitiveness as a global tourism hub hinges on it delivering exceptional service quality that meets travellers’ expectations. The RCdTT-SHTM’s ongoing development of the TSI underscores our commitment to advancing Hong Kong’s reputation as a world-class destination for international visitors.”

The team gathered over 1.25 million reviews from TripAdvisor, a global Online Travel Agency (OTA) platform, covering 13,694 Hong Kong service providers across five tourism service sectors, namely attractions, hotels, restaurants, retail shops and transportation. The LLM-based assessment framework then utilises Alibaba’s open-source General Text Embedding model to extract semantic insights from the visitor reviews in different languages and formats.

Key findings include:

1. Topic-based analysis reveals that the largest proportion of reviews are about “service reliability and safety”. This reflects that reliability in tourism services emerged as the most crucial factor influencing tourist satisfaction, forming the foundation of trust in the travel industry.

2. Since 2012, the Hong Kong TSI has shown an overall upward trend, while consistently scoring above 75 points on a 0-100 scale for the past decade. Despite a temporary decline during the pandemic, Hong Kong TSI levels had fully recovered to pre-pandemic levels in 2024.

3. Notably, the transportation sector has consistently achieved high TSI scores, largely due to Hong Kong’s well-developed public transport infrastructure, which continues to enhance visitor experiences. Retail shops and restaurants, on the other hand, have lagged behind, primarily owing to price-related factors. The traditional price advantage of retail shops is gradually diminishing, significantly impacted by the rise of e-commerce. Similarly, restaurants in Hong Kong tend to be relatively expensive, prompting some tourists to seek more affordable dining options on the mainland.

Figure 1: TSI at destination and sector levels

4. From a spatial perspective, all districts recorded TSI scores above 73 points, yet a north-south divide persists, with higher satisfaction levels in the southern regions and lower levels in the north. The highest TSI values were concentrated along both sides of Victoria Harbour, such as in the Central and Western, and Yau Tsim Mong Districts, while Kowloon City District recorded considerably lower TSI levels, creating a distinct localised satisfaction gap compared to its surrounding districts.

Figure 2: TSI Spatial distribution

5. Tourists with different trip types exhibit significant variations in their satisfaction levels towards Hong Kong. Business tourists generally report the highest satisfaction across most sectors, whereas solo travellers exhibit lower satisfaction overall. This may be due to their different needs: business tourists typically prioritise standardised functional requirements, while the solo travellers place greater emphasis on unique and personalised experiences. 

6. Language-based analysis further highlights differences in the TSI, with reviews in English and European languages yielding higher satisfaction levels than the Japanese- and Korean- language reviews. This highlights the impact of cultural differences on tourist satisfaction.

Objectives of the TSI

The LLM-based TSI framework is crafted to deliver regular, data-driven insights that benefit tourism service providers, policymakers and destination management organisations. Through topic-level TSI analysis, for instance, businesses can pinpoint specific areas for service enhancement, while destination- and sector-level assessments provide a broader perspective for strategic planning and policy formulation. Additionally, monthly TSI updates ensure that assessment results remain timely and relevant, enabling swift and informed decision-making.

Although the Hong Kong economy has largely recovered from disruptions caused by the pandemic, challenges such as labour shortages, supply constraints, changing economic conditions, evolving consumer behaviour and competition persist. Highlighting the significance of the TSI in tackling these challenges and facilitating relevant decision-making and planning, Prof. Song remarked, “To sustain ongoing recovery, accurate and timely assessment of the satisfaction level of inbound tourists is essential for policymakers and industry practitioners to develop sustainable tourism strategies that will further fortify Hong Kong’s tourism advantages, propel its tourism industry and ultimately promote long-term economic growth in the City.”

As technology advances, both destinations and visitors increasingly rely on information and communication technologies to make their decisions. In support of the long-term sustainability of the tourism sector, by harnessing artificial intelligence, LLM and big data analytics, the enhanced TSI assessment framework aims to offer more precise and targeted advice for strengthening tourism service quality and bolstering Hong Kong’s global competitiveness.

***END***