Prof. Lik-Hang Lee, Assistant Professor of the PolyU Department of Industrial and Systems Engineering, has developed an AI-driven assessment system, the Language Model Linguistic Personality Assessment (LMLPA)
Large language models (LLMs) are at the forefront of artificial intelligence (AI) and have been widely used for conversational interactions. However, assessing the personality of a given LLM remains a significant challenge. A research team at The Hong Kong Polytechnic University (PolyU) has developed an AI-driven assessment system, the Language Model Linguistic Personality Assessment (LMLPA), with capabilities to quantitatively measure the personality traits of LLMs through linguistic analysis.
This innovative interdisciplinary research in AI and computational linguistics has led to the development of robust, data-driven AI tools for evaluating nuanced LLM personality traits and behaviours. The LMLPA system represents a critical step forward in understanding LLMs and developing them to be more aligned with human values and needs. Led by Prof. Lik-Hang LEE, Assistant Professor of the PolyU Department of Industrial and Systems Engineering, the research has been published in Computational Linguistics.
LMLPA is designed to evaluate and characterise the personalities of LLMs by examining the linguistic patterns, style and other language-related features in their outputs. The system comprises two main components: the Adapted Big Five Inventory (Adapted BFI) and the AI rater. LMLPA first administers the Adapted-BFI, which is derived from previous language-based personality assessment theories, to LLMs. The AI rater then evaluates the responses, converting the textual answers into quantifiable numerical values representing personality traits.
| Department of Industrial and Systems Engineering |
