Research on Comprehensive Evaluation Method of Airport Runway Operational Effectiveness

In order to assess whether the runway can meet safe, reliable and efficient objectives, a comprehensive evaluation indicator system needs to be established for evaluation. Firstly, the runway operation system is established, and the evaluation indicators of runway performance are divided into five aspects: personnel factors, support facility, mission demand, operating environment and management factors and then establish a comprehensive evaluation index system. Secondly, the TOPSIS evaluation model of game theory combination weighting is established, game theory is used to optimize the weights of the weights obtained by AHP and CRITIC, and the ideal solution (TOPSIS) is used to comprehensively evaluate the runway performance. Finally, the model was used to evaluate the runway performance levels of the three airports, and the three evaluation methods were compared. The results show that the results of the runway performance evaluation using the model are in line with the actual operation of the airport. The method is simple and the results are scientific and objective, and it have good versatility.


Introduction
The runway is the main component of the airport, and the operating efficiency of the runway is directly related to the operating efficiency of the entire airport. Therefore, accurate assessment of runway operation efficiency is of great significance to the improvement of airport operation efficiency. Foreign studies on runway operation evaluation are relatively early. From the perspective of runway capacity, Eckhard Urbatzka et al. [1] evaluated the runway operation capability of German airports from the perspective of runway capacity, taking off, landing, and mixed modes. With the development of computers, simulation is gradually applied to run evaluation. EEC and ASI developed simulation platform models, simulated the operation of runways, taxiways and aprons, and evaluated the capacity of the flight area [2][3]. FAA has developed an airport capacity assessment system to assess the capacity of airport runways and taxiways [4]. F.A. Cetek [5], Manuel Villegas Dí az, etc. established a discrete event simulation model to evaluate the capacity of the airport runway. Domestic research on the evaluation of runway operations mainly focuses on runway capacity. Lü Zongping [7], Xu Xiaohao [8] and others simulated runway operations and proposed methods for evaluating the theoretical capacity of parallel runways under different runway operation modes. . Li Xiong [9] Feng Xiaolei [10] used simulation software to simulate different operation modes of parallel runways and evaluated the runway capacity. Li Xiong [11] took the Chengdu New Airport as an example and established an airport simulation model to evaluate the operating efficiency of the runway. The above-mentioned scholars have done in-depth research on the evaluation of airport runway operations. This article evaluates the airport runway operations from the perspective of effectiveness.
Airports have different operating modes and resource allocations from civil airports. The indicators that need to be considered when evaluating airport runway operations are quite different. In response to this situation, this article is based on the aircraft take-off and landing guarantee process, combined with the characteristics of airport runway operations, and establishes a comprehensive evaluation index system for runway operation effectiveness from five aspects: personnel factors, support facilities, mission requirements, operating environment, and management factors. . The game theory method is used to comprehensively determine the weights, and the TOPSIS method is used to evaluate the operation efficiency of the airport runway, which can provide a theoretical basis for the evaluation of the airport runway operation efficiency.

Runway operation system
The runway operation system refers to the fact that when the maintenance personnel and field personnel enter the field, the maintenance personnel start to check the aircraft, refuel, oxygenate, and inflate the aircraft, and the field personnel perform bird repellent and runway cleaning to prepare before takeoff. After the preparation is completed, the command The crew directs the aircraft to enter the runway, takes off and executes the mission to land. In this process, the organic whole that can be directly generated is regarded as a runway operation system.
The basic interpretation of effectiveness is: the degree to which the final goal of the system is achieved, or the degree to which the system expects to achieve a set of specific task requirements. It is proposed for the needs of weapon system evaluation. Some scholars introduce the definition of effectiveness into the system. System effectiveness refers to the ability of the system to complete the specified use target under specified conditions [12][13].
According to the definition of system effectiveness, the operational effectiveness of the runway system is defined as the degree to which the operational capability of the runway system can meet the mission requirements under specified conditions. The runway system, as a key node in the execution of the aircraft, depends on its value ability.

Comprehensive evaluation index system
The airport's aircraft takeoff and landing tasks are completed by the airport runway operation system. Therefore, when establishing an evaluation index system, it is necessary to analyze it from the perspective of the system. This paper introduces the theory of system engineering, and establishes an evaluation index system from the man-machine-environment-management-responsibil ity.
The selected indicators follow the principles of systemicity, scientificity, authenticity, and quantification. By consulting relevant experts in this field and referring to [14] and other specifications, this article builds a comprehensive evaluation index system for airport runway operation efficiency from five aspects: personnel factors, support facilities, operating environment, mission requirements, and management factors. The system is divided into three levels from top to bottom: target layer, criterion layer, and object layer. The target layer is the runway operation efficiency, which evaluates the airport's runway operation efficiency; the criterion layer is divided into personnel factors, support facilities, operating environment, and task requirements. , Management factors 5 indicators, there are 21 indicators in the object layer, the comprehensive evaluation indicator system is shown in Figure 2: Figure2. Road operation efficiency evaluation index system (1) Human factors: Human factors are directly related to the safety of the runway and take-off efficiency, etc., and are important factors that affect the efficiency of the runway. Specific evaluation objects include air traffic control personnel, ground support personnel, and pilots. The three categories of personnel are evaluated from three indicators: professional knowledge, equipment operation proficiency, and social ability.
(2) Guarantee facilities: including equipment such as navigation aid lights, communications, navigation, surveillance, blocking nets, and bird repellents. The safety and reliability of runway operations depend on the quality of facilities and equipment, and support facilities have a direct impact on the efficiency of runway operations. Navigation aids are evaluated from four indicators: installation location, light intensity, number of installations, and failure rate; communication equipment, monitoring equipment, and navigation equipment are evaluated from five indicators: accuracy, clarity, available distance, scope of applicable equipment, and stability Evaluation; bird repellent equipment includes bird repellent vehicles and bird repellent shotguns, which are evaluated in terms of number, type, and scope of use; arresting nets are evaluated in terms of lifting speed and installation position.
(3) Operating environment: including airport clearance, runway configuration, runway operation mode, weather conditions, runway surface conditions and other soft and hard environments, these operating environments have a direct impact on runway operating efficiency. The airport headroom can meet the headroom requirements from both sides of the airport, one side of the airport meets the headroom requirements, and the two sides do not meet the headroom requirements for evaluation; the runway configuration is from single runway, cross runway, parallel runway, and open V-shaped runway. The weather conditions are selected from the representative months of each quarter, and the five indicators of temperature, air pressure, visibility, wind speed, and cloud height are evaluated; the runway operation mode is selected from the independent parallel instrument approach mode and related Parallel instrument approach mode, independent parallel departure mode, and isolated parallel operation mode are evaluated in four aspects; road surface conditions are evaluated from five indicators including appearance quality, service life, skid resistance, bearing strength, and flatness.
(4) Task requirements: From the perspective of system tasks, analyze the task requirements in runway operations. The main task of the runway operation system is to provide services for aircraft taking off and landing to ensure their safe and efficient operation on the runway. There are three types of tasks: training tasks, exercise tasks, and combat tasks. These three tasks evaluate the completion degree of takeoff and landing sorties from the runway and the time used for a wave.
(5) Management factors: The management level also has an important impact on the efficiency of runway safety operations. The evaluation objects include flight support procedures, material support levels, and coordinated management levels. The convenience of the flight support process is evaluated from three indicators: the time spent in each process, the number of people required, and the amount of materials required; the level of coordination management is evaluated from two indicators: the commander's communication and coordination ability and emergency response ability; material support The level is evaluated from whether the delivery is fast and whether the supply meets two indicators.

Comprehensive Evaluation Method of Runway Operation Effectiveness
There are many influencing factors in the evaluation of airport runway operation efficiency. A single evaluation method often cannot comprehensively, objectively and scientifically evaluate the airport runway operation efficiency. Therefore, it is necessary to use a comprehensive evaluation method to evaluate various indicators of airport runway operation efficiency. There are many comprehensive evaluation methods, which are generally divided into subjective empowerment method and objective empowerment method. Commonly used subjective empowerment methods include: Delphi method, AHP method, G1 method, etc.; commonly used objective empowerment method (Objective empowerment) includes: mean square error method, CRITIC method, entropy method, etc. When it is difficult to carry out quantitative statistics, it is necessary to adopt subjective weighting method, and better results can be obtained according to expert score; in order to reduce the distortion of data, it is necessary to choose objective weighting method, which determines the weight based on the real data of the indicator. Its advantages It is highly objective, without human subjective opinions. Currently, the most commonly used method for determining weights is Analytic Hierarchy Process, which is an optimization method based on expert scores. Combining the characteristics of multiple indicators of the airport, the objective weighting CRITIC method is selected, which can objectively reflect the weight of each indicator.
In this paper, AHP weighting method and coefficient of variation weighting method are used comprehensively, using game theory to find the optimal weight, so that the evaluation results can accurately and reasonably reflect the subjective opinions of experts and the objectiveness of various indicators. Information, and finally approach the ideal solution to compare the results of the three methods and evaluate the comprehensive weights obtained.

Subjective weighting method to determine weight
In this paper, AHP weighting method and coefficient of variation weighting method are used comprehensively, using game theory to find the optimal weight, so that the evaluation results can accurately and reasonably reflect the subjective opinions of experts and the objectiveness of various indicators. Information, and finally approach the ideal solution to compare the results of the three methods and evaluate the comprehensive weights obtained.

Objective weighting method to determine weight
The CRITIC method is an objective weighting method for processing the data of various indicators. The advantage of using this method to determine the weight is that on the one hand, it calculates the amount of information contained in each indicator, and on the other hand, it can reflect the differences between different schemes. Conflicts with various indicators, so the results are more objective and true [14]. Combining the characteristics of multiple indexes of the airport, this paper uses the CRITIC method to determine the weights of various indexes of airport runway operation efficiency.
(1)Calculate the coefficients between the various indicators of each runway operating efficiency

Game theory combination to determine weight
The combination of game theory uses Nash equilibrium as a coordination method to assign weight to subjective and objective combinations. It seeks consistency and compromise within the conflict of subjective and objective weights. It is a process of comparison and coordination, so as to find both subjective and objective methods. The optimal value. The importance of each index exists objectively. When using the subjective weighting method, it is subject to the influence of people. Therefore, it is necessary to use subjective and objective weights to comprehensively assign weights in order to scientifically and comprehensively reflect the importance of evaluation indicators. . The game theory method comprehensively determines the weight, and combines the advantages of the analytic hierarchy process (AHP) and the CRITIC method to reduce the error caused by human subjectivity, so as to effectively determine the comprehensive weight of the runway operation efficiency. Use the above two methods to combine weights for runway operation efficiency indicators to construct a weight vector 12 [ , ,... ] arbitrary linear combination of these 2 weight vectors is: In the equation: k  is the linear combination coefficient Using game theory, combine the weights required by the two methods to minimize the standard deviation. The combined coefficients of the two weight vectors are combined and solved to obtain the best weight. Therefore, the following countermeasure models can be obtained: It can be known from the differential characteristics of the matrix that when the first derivative condition is 4 Case analysis

Data Sources
In order to make the data more comprehensive, objective and representative, three representative airports were selected and followed by experts for investigation. According to the comprehensive evaluation index system established above, combined with the characteristics of specific airports, the above indexes are classified and investigated. By issuing questionnaires, referring to relevant standards, experts scoring and comprehensively considering and calculating the scores of various indicators are shown in Table 1.

Determine the weight
(1) Analytic hierarchy process to calculate weight According to the hierarchical structure of airport runway operation efficiency established in section 2.1, and the judgment matrix constructed according to the established indicators, the weight of each indicator is finally solved. The weight value of the indicator is shown in Table 2. (2)CRITIC method and game theory method to calculate weight Calculate the coefficients between the runway operation efficiency indexes of the yard, and calculate the quantitative indexes, standard deviations, information amount and other values, and the weights of each index can be obtained as shown in Table 3: Use the game theory method for combination weighting to obtain the optimal first-order linear

TOPSIS method evaluation
TOPSIS is an evaluation method to solve complex problems with multiple targets and multiple features [17]. By calculating the Euclidean distance between the evaluation object and the positive and negative ideal solutions, the closeness of each evaluation object to the positive and negative ideal solutions can be determined. According to the divided evaluation index interval, combined with the required closeness to classify. The higher the degree of closeness between the evaluation object and the positive ideal solution, the better; otherwise, the worse.
(2)Construct a normalized matrix Since different evaluation indicators have different dimensions, it is difficult to carry out effective evaluation. In order to eliminate the dimensionality, the matrix A needs to be normalized: Analyze the closeness of each assessment object to the positive and negative solutions. When the relative closeness is greater, it indicates that the runway efficiency of the assessment object is higher, and vice versa.
Get 0< j F <1，When the maximum is the maximum, the higher the level is near the excellent target and the far away from the poor target, and vice versa. The closeness of specific standard grades is shown in Table 4: (0 ~ 0.3］ The Euclidean distance between the airport runway operating efficiency and the positive and negative ideal solution is calculated. According to equation (17), the relative closeness of the indicators of the three airport runways can be obtained. According to the divided indicator interval, the operating efficiency levels of the three airport runways are determined. The evaluation results are shown in Table 5  From the above data, we can get the radar chart of the airport runway efficiency of C, D, F, and we can intuitively see the distribution of personnel factors, management factors, support facilities, operating environment, and mission requirements of each airport as shown in Figure 3.  Table 5, it can be seen that in C, D, and F, it can be seen that the evaluation result of airport D is the best, which is consistent with airport D's ability to efficiently complete tasks in daily life. It can be derived from the relative closeness. The sorting situation of runway operation efficiency of each airport: D>C>F.
It can be seen from Figure 3: Airport C: The management factor is good, and other indicators are good. On the whole, the runway operation efficiency of Airport C is good, and there is room for improvement. It can be from the aspects of guarantee facilities, personnel training, and management Factors and other aspects to improve. D Airport: All five aspects are good overall, management factors, mission requirements are excellent, other indicators are good, runway operation efficiency evaluation is excellent, and it can be used as a demonstration unit. F Airport: The guarantee facilities and operating environment are evaluated as good, other indicators are good, and the comprehensive evaluation is good. F Airport was constructed relatively early, and the overall facilities were outdated. Since F Airport was relatively close to the urban area when it was planned at that time, with the development of urbanization, the clearance area of F Airport was limited. Therefore, the evaluation of the operating environment is relatively low. From the perspective of runway operating efficiency, it is recommended that Airport F be upgraded.
In order to be able to find out the reasons that restrict the efficiency of runway operations, the target level indicators are further analyzed. According to the weighting matrix C, the weights of different indicators of the three airports can be obtained. In order to be more intuitive, the weight of ownership can be enlarged by 100 times, and the following Fig.4~Fig.8 can be obtained.     Figure 3 shows that the three indicators of A1, A2, and A3 of Airport F in terms of human factors need to be improved. It can be seen from Figure 4 that in terms of support facilities, Airport C can be improved from the three indicators of B5, B8, and B10; Airport F can be improved from the four indicators of B4, B6, B7, and B9. It can be seen from Figure 5 that in terms of operating environment, Airport F can be improved from the three indicators B11, B14, and B15. It can be seen from Figure   6 that in terms of mission requirements, B16 indicators of C airport need to be improved; B16, B17, and B18 of F airport need to be improved. It can be seen from Figure  7: Management factors: C Airport B19, B21 indicators need to be improved.
In order to evaluate the results obtained by AHP, CRITIC, and Game Theory, the three airports surveyed were evaluated using TOPSIS, CRITIC, and Game Theory respectively. The results are as follows:   Table 6 that there are certain differences between the evaluation results of AHP and CRITIC method and the method of game theory combination weighting. There are two reasons for this: When conducting a questionnaire survey, it will be due to human factors. Some indicators of the airport are too high, which will cause certain errors. The CRITIC method has good applicability to indicators that can be objectively evaluated. For some indicators that are not easy to quantify, experts need to score, and these indicators cause differences.

Conclusion
1. From a systematic point of view, this paper establishes a comprehensive evaluation index system for runway operation efficiency, including five aspects: personnel factors, support facilities, mission requirements, operating environment, and management factors. From these five aspects, the performance of airport runway operations is scientifically and efficiently evaluated. Comprehensive and objective assessment.
2. Three airports were selected, using game theory to combine subjective weights and objective weights, and using the TOPSIS method to evaluate the runway operation efficiency of the three airports, and analyze the constraints on the runway operation efficiency of the three airports Specific reasons, and proposed three airport runway operational efficiency improvement recommendations.
3. The comprehensive evaluation index system for the operation of the airport flight area, and the evaluation of the operation efficiency of the flight area from the perspective of simulation will be the direction of future research.