The gradual development of technology and artificial intelligence (AI) has placed at our disposal robust and adaptive decision-making (DM) methods that can reduce the occurrence of human error and risk index in engineering applications. Human factors (HFs) need to be fully considered in the DM process, especially in the field of air traffic control (ATC). Air traffic controllers (ATCOs) play efficiently an important role in aviation safety and reliability. The research on DM has but rarely considered the internal state of ATCOs and HFs in the ATC. It is possible to design the ATC system to be more intelligent and to be more adaptive to the behaviour of the ATCOs. This research proposal aims to design human-centred adaptive DM method that combines subjective and objective measurement made by various human factors and ergonomics equipment via intelligent automation (IA). In this research proposal, the expected prototype and the outcomes of the research proposal can capture subjective and objective data, take into the consideration the HFs information to provide real-time online feedback and adjust the decision support system to HFs

Keywords : Air traffic control; adaptive decision-making; human factors and ergonomics; cognitive modelling and situation awareness; intelligent automation

The main goal of this research project is to develop AI-based and IA-based modelling for human factors and ergonomics assessment in air traffic control (ATC). The budget plan is mainly for equipment purchase, software license and experimental expenses for the use of human factors and ergonomics research. The research outcome and deliverables can largely contribute the development of PolyU AAE in research, academic, academic achievement and industrial collaboration. The objective will be as follows:

• 1Establishment of automated human factors and ergonomics assessment in aviation research

  This framework will bring further understand the performance of human and machine for adjusting intelligently the system DM behaviour in relation to the external and internal working conditions and the skills, tasks, and cognitive abilities of specific personnel, through intuitive data and good synchronisation. After acquiring data results, it can support the combination with ATC system applications, the system behaviour can be adjusted accordingly via the data-driven approach. The possible applications mainly include: real-time online feedback that eases the capture the state of air traffic control officers (ATCOs) to realise human-centred system adjustment, safety warning reminder according to the critical point of information implied by the data and level of automation in human machine interface. Also, it serves as a data-driven model that provides objective data to empirically prove the advantages and disadvantages of applications of HFs, and further supports the subsequent design optimisation. Data-driven adaptive DM supports and regulates system parameters, including adaptive automation level, workload scheduling, ergonomics interface and interaction. For instance, the system automation level (from low to high) is adjusted by detecting the cognitive fatigue level of ATCOs. When the fatigue level is high, the automation level is adjusted to a higher level or even takes over control to assist ATCOs in decision making (DM), and vice versa. The real-time online feedback system can also be incorporated in ATC operation to distinguish state changes when ATCOs participate in different human factor tasks to extract new and identify opportunities for its application in future HFs design applications. The automated human factors and ergonomics assessment in aviation could provide great impact on the humanised design of future ATC and human factors research.

• 2Establishment of human factors and ergonomics education in PolyU AAE

  PolyU AAE can introduce the fundamental scientific principles of human factors and ergonomics, IA and AI in aviation to PolyU AAE students teaching and learning (T&L) by hands-on experiments and provide research training to research students (RS). PolyU AAE students have the unique opportunity to mirror what scientists, engineers and researchers being done in actual environment, and carry out the experiments during laboratory teaching and capstone project. We hope that the T&L will provide students an authentic learning experiences and use modern human factors and ergonomics equipment. The knowledge and practices can benefit students in their future careers, such as systems & human factors engineer in aviation, aviation safety inspector, human-centred IA/AI engineer in aviation industry and human-systems integration designer for ATC tower system. The set-up of the human factors research in PolyU AAE can provide facilities and encourage university students in project competitions. PolyU AAE students can design and develop their own human factors applications in aviation and perform rigorous empirical evaluations using the human factors equipment. This enables students’ vision to better understand in human factors research via domestic or international student project competitions.

• 3Support of research activities and industrial collaboration

  Scholar and faculty members need to think about the beneficial consequences of academic research and society, or namely research impact. According to Australian Research Council, research impact is “the demonstrable contribution that research makes to the economy, society, environment and culture beyond the contribution to academic research”. The dissemination of our human factors research outcome is one of the avenues for industrial collaboration. Such attraction can show our research capabilities and benefit in initiating collaborations with the industries.

ATCOs should be at the centre of the entire process of IA and adaptive DM of the ATC system. The following example is an illustrative case of fNIRS application in ATC. The proposed method considers the application and impact of HFs on the system and the subjective behaviour and neurophysiological changes that the ATCOs show. Figure 1 illustrates the overall framework of the proposed research method, which is divided into three parts: data input layer, data analysis layer and data application layer.

The data analysed data can then be used in the implementation of some data-driven application services, such as the adjustment of human-machine interface, the optimisation of radar maps, and the fine-tuning of voice-interactive and automation levels to the changes in ATCOs internal information to meet the needs of the system in its current state. Understanding the role of the HFs in the ATC system is necessary for achieving the ideal state of human-machine integration and decision processing in the ATC system. The evaluation system of the ATC design scheme based on the subjective and objective score of the design scheme can be used to further support and improve the ATC system design, as shown in Figure 2.

In the past few decades, the aviation industry has specialised in experiencing rapid growth and development. The ongoing globalisation of world markets, technological progress and the establishment of new business models are significant examples for illustrating the dynamic development of air transportation within the last decades. However, this also marks the emergence of aviation accidents with increasing flying services. Aircraft safety is one of the significant values within the whole aviation industry. Behind the prosperity of the whole aviation industry, all potential threats that could influence aircraft safety are controlled simultaneously. Due to the advanced and mature development of technology and equipment, aviation accidents are no longer mainly induced by technical shortcomings but also human errors. The significance of the human errors in aviation safety has been recognised for many years as one of the most significant hazards that threaten the high-level standard of aviation safety. One of the leading causes of aircraft accidents is mainly due to “human error”, while the other two causes are “technical failure” and “other”. Approximately, 88% of aviation accidents with severe human error components, can be traced back to the problems with situation awareness, poor decision-making, fatigue, distraction, etc.

With the development of neuro-ergonomics technologies, such as EEG, fNIRS, and ECG, etc., the performance and characteristics of pilots whiling performing different tasks could be revealed and investigated. Therefore, the area of research direction would be on the human factor of pilots, with the goals of

• 1Monitoring pilots’ performance envelope like mental efforts, workload, situational awareness, cognitive decision-making, etc.

• 2Investigating the effects of system parameters and individual traits on pilots’ response from multimodal data perspectives

• 3Assisting the skill-backward, novice, and student pilots in training with an efficient way via revealing their feedback from behavioural and physio-psychological aspects

• 4Assessing the influence of the current cockpit instrument on pilots’ operation and providing recommendations for designing cockpit panel

• 5Realising a pilot-oriented adaptive decision-making system.

Teams formulated by aviation professionals are essential in maintaining a safe and efficient aerodrome environment. Nonetheless, the shared situational awareness between the flight crews under adverse weather conditions might be impaired. This research aims to evaluate the impact of a proposed enhancement in communication protocol on cognitive workload and develop a human-centred classification model to identify hazardous meteorological conditions. Thirty groups of subjects completed four post-landing taxiing tasks under two visibility conditions (CAVOK/CAT IIIA) while two different communication protocols (presence/absence of turning direction information) were adopted by the air traffic control officer (ATCOs). Electroencephalography (EEG) and the NASA Task Load Index were respectively used to reflect the pilot’s mental state and to evaluate the pilot’s mental workload subjectively. Results indicated that impaired visibility increases the subjective workload significantly, while the inclusion of turning direction information in the ATCO’s instruction would not significantly intensify their cognitive workload. Mutual information was used to quantitatively assess the shared situational awareness between the pilot flying and the pilot monitoring. Finally, this research proposes a human-centred approach to identify potentially hazardous weather conditions from EEG power spectral densities with Bayesian neural networks (BNN). The classification model has outperformed other baseline algorithms with an accuracy of 66.5%, an F1 score of 61.4%, and an area under the ROC of 0.749. Using the concept of explainable AI with Shapley Additive Explanations (SHAP) values, the exploration of latent mental patterns formulates novel knowledge to gain insights into the vital physiological indicators of the pilots in response to different scenarios from the BNN model. In the long term, the model facilitates the decision regarding the necessity of providing automation and decision-making aids to pilots.

Keywords : Bayesian neural networks; aerodrome operations; adverse weather conditions; mutual information; shared situational awareness

The study was conducted in the Human Factors and Ergonomics Laboratory, Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University. The simulation platform utilised is developed in-house with the support of existing flight simulation and ATC simulation software. The platform adopted for the experiment is illustrated in Yiu, Ng, Lee, Chow, Chan, Li and Wong. It consists of a CockpitSonic A320 FTD flight simulator and a desktop computer for EEG signals recording and videotaping. Figure 1 shows the experimental setting.

Figure 2 shows the comparison between the situation of ceiling and visibility OK (CAVOK) and the precision approach operations of CAT IIIA, which refers to decision height not available or lower than 100 ft/30 m, with the runway visual range not less than 700 ft/200 m in the flight simulator setting.

Figure 3 shows the graphical timeline of the experimental procedure. The procedure of analysis is shown in Figure 4.