Research on subjective evaluation method of intelligent parking assist system based on typical parking scenario

With the rapid development of intelligent driving technology, Intelligent Parking Assist systems have been widely used. Through analyzing the technical characteristics of the Intelligent Parking Assist system, this article brings up subjective evaluation indicators of Intelligent Parking Assist system from the perspective of consumers' daily use; Practical verification is carried out for three typical parking scenarios including parallel parking spaces, vertical parking spaces and inclined train spaces, thus a set of subjective evaluation methods suitable for Intelligent Parking Assist systems for passenger cars is summarized.


Foreword
With the rapid development of intelligent driving technology, Intelligent Parking Assist system (IPA) technology becomes more mature, playing a significant role in reducing accidents during parking and loosing driver's driving intensity. However, different models have different levels of automated driving, accordingly there are also differences in user experience. As an ordinary consumer, it is most unlikely to go through a professional objective performance testing to understand the performance differences between different Intelligent Parking Assist systems, but could only judge whether the actual use needs are met through physical senses during daily use.
At present, the research scope of Intelligent Parking Assist system technology mostly focuses on the objective testing area, for example, test method of assisted parking system [1] , research on advanced parking aid (APA) [2] , etc. There is relatively few research on Intelligent Parking Assist system based on subjective evaluation of user experience. Therefore, Through analyzing the technical characteristics of the Intelligent Parking Assist system, this article brings up subjective evaluation indicators of Intelligent Parking Assist system from the perspective of consumers' daily use; Practical verification is carried out for three typical parking scenarios including parallel parking spaces, vertical parking spaces and inclined train spaces, thus a set of subjective evaluation methods suitable for Intelligent Parking Assist systems for passenger cars is summarized, which provides a new evaluation method for vehicle companies in the evaluation of Intelligent Parking Assist system.

Analysis on technical features of Intelligent Parking Assist system 1.Definition of Intelligent Parking Assist system
Intelligent Parking Assist system (IPA): when the vehicle is being parked, it automatically detects the parking space and provides the driver with auxiliary functions such as parking instructions and / or direction control.
The Intelligent Parking Assist system architecture is mainly composed of three layers: perception layer, control layer and execution layer. As shown in figure 1: The perception layer consists of camera(s) and ultrasonic radar(s), identifying surrounding environmental information and detecting nearby vehicles or obstacles, thus deliver the collected environmental data to the control layer. The control layer is the core of the entire Intelligent Parking Assist system. It analyze and process the data uploaded by the perception layer, consequently obtains the surrounding environment information and the relative relationship with the current position of the vehicle. Based on these technical parameters, it maps parking path through internal algorithms and control strategies, and then sends control instructions to the execution layer. After receiving the instruction, the execution layer will accurately control the movement of the steering, throttle and braking system, so that the vehicle can complete the parking process according to the planned path [3] .

Classification and work flow of Intelligent Parking Assist system
At present, common parking systems are mainly divided into semi-automatic parking systems and fully automatic parking systems. The semi-automatic parking system belongs to the L1 level intelligent driving function, which will not completely take over the vehicle but only response for lateral control of the vehicle. During the automatic parking process, the driver needs to cooperate to complete acceleration, braking, shifting and other tasks to achieve longitudinal control of the vehicle. Fully automatic parking system belongs to L2 level intelligent driving function. With this level, the system will completely take over the vehicle, responsible for both of the lateral and vertical control of the vehicle. Most of these systems are capable of identifying parking spaces. With the rapid development of intelligent driving technology, companies are working on fully autonomous remote parking technology with higher levels of intelligent driving, so that the last mile of "valet parking" can be completed.
The working processes of L1 and L2 smart parking assist systems are shown in table 1 [4] : Table 1. Work flow chart of Intelligent Parking Assist system.
Step 1 System activation Active the parking system manually or active automatically according to the vehicle speed Step 2 Search for parking spaces Customized human-computer interaction mode by manufacturer Step 3 Find parking spaces Customized human-computer interaction mode by manufacturer Step 4 Choose a parking space Customized human-computer interaction mode by manufacturer. Including prompt type, location selection Step 5 Automatic parking Real-time feedback of surrounding environment information. L1 level requires driver to perform longitudinal control according to prompts Step 6 Parking accomplishment Information reminds that parking is complete

Construction of subjective evaluation system for Intelligent Parking Assist System
Based on the technical features of the Intelligent Parking Assist system and the actual demands of consumers,this article constructs a relevant subjective evaluation system.

Definition of subjective evaluation
Subjective evaluation refers to that the trained assessors use the human vision, hearing, touch, body feel and other sensory organs to evaluate the performance of the vehicle in a typical driving road or evaluation environment according to subjective evaluation standards, then weigh and quantify the evaluated results, which can help assess the overall performance level of the vehicle quickly.

Criteria of grade scoring
The grade scoring method is an evaluation method of scoring various performance indicators for a vehicle according to a stipulated scoring criteria. Ten-point scoring method is recommended, in which the assessor scores for each indicator based on gap between vehicle actual performance and perfection. In order to make the subjective evaluation data reflecting the slight difference between vehicle performance, 0.25 point is used as the minimum indexing value, that is, 1 point is divided into four score levels of 0, 0.25, 0.5, and 0.75 [5] .

Subjective evaluation system
To evaluate the Intelligent Parking Assist system from five aspects: Human-computer Interaction, Operation Logic, Parking Space Searching Ability, Parking Accomplishment, and Parking Comfortableness, and introduce the subdivision items and subjective evaluation method:

Items and methods for subjective evaluation of Human-computer Interaction
1) Icon visuality: Evaluate if the IPA icon indicator or signal device is easy to recognize, including the indicator position and size, the display location in the dash board, the color of the signal device, etc.
2) System setting: Evaluate IPA interface operation logic, user-friendliness of interface, sense of technology, etc. Whether the operation logic is clear and easy to understand, and the interface switching speed, etc.
3) Process displaying: Evaluate whether the display method of the IPA system during parking is advanced, with sense of technology, and whether it is a 360 ° panoramic image display.

Items and methods for subjective evaluation on operation logic
1)Function Quit: Evaluate whether the IPA system quits in time when the driver is steering the vehicle manually; 2) Prompt of function activating: Evaluate whether the IPA system's prompt method to the driver is reasonable after finding a parking space, including video prompts, sound prompts, etc.
3)Operation convenience: Evaluate whether it is convenient and comfortable for the driver to operate the parking system (gear position, brake pedal) according to the instructions of the system in the parking process.

Items and methods for subjective evaluation on the ability of searching parking space
1)Parking space recognition: Evaluate the IPA system's ability of identifying parking spaces and determine whether they can accurately identify parking spaces without a target vehicle; 2)Vehicle speed at the time of searching Parking space: Evaluate the maximum allowed vehicle speed by the IPA system when searching for parking spaces; 3)Parking space lateral distance: Evaluate the maximum lateral distance to the parking space when IPA is searching for parking spaces; 4)Parking space recognition time duration: Evaluate the recognition speed of the IPA system when searching for parking spaces.

Items and methods for subjective evaluation on parking accomplishment
1)Parking completion: Evaluate whether the IPA system can successfully complete automatic parking after finding parking space; 2)Pull-out ability: Evaluate whether the IPA system can successfully pull out the vehicle; 3)Parking location: Evaluate whether the vehicle's position is appropriate within the parking lot boundaries (front, back, left, and right)after parking by IPA; 4)Steering wheel position: Evaluate whether the steering wheel position returns to normal after parking by the IPA system.

Items and methods for subjective evaluation on parking comfortableness
1)Parking smoothness: Evaluate the overall smoothness of the IPA system during parking. Whether the process is smooth, and whether the consistency of steering and body posture is natural; 2)Steering control: Evaluate whether the steering control of the IPA system is smooth and natural, whether the steering wheel rotates at a uniform speed, and whether there is no sudden change in steering; 3)Brake control: Evaluate the braking comfortableness driven by the IPA system during parking and whether the brake control is abrupt; 4)System prompt: Evaluate whether the IPA system's gear operation prompts and speed control prompts are consistent with steering control during parking; 5)Sense of security: Evaluate whether the IPA system has high sensitivity to avoid obstacles during parking.

Information of the test ground
Clean, dry and flat ground with vertical parking spaces, parallel parking spaces and inclined parking spaces.

Evaluation results of vertical parking spaces
The schematic of vertical parking space is shown in Figure 2: The subjective evaluation results of the IPA system for five vehicles at the scene of vertical parking spaces are shown in Table 5: In the vertical parking scene, five cars can basically complete the parking smoothly, while the IPA systems got high score for most of the indicators of human-computer interaction, operation logic, parking space search ability, parking effect and parking comfortableness index, but low score for a few indicators.
1) #1 vehicle is equipped with L2 level IPA system which automatically controlled steering and braking throughout the process. The only problem is that the vehicle 's brake control is relatively sudden that brought discomfort during parking. The braking intensity is a little too strong with loud braking sound, which negatively impacted the sense of quality; 2) #2 and #5 vehicle got slightly lower scores for a few indicators in terms of parking space search ability, that can't quickly identify parking spaces; 3) In the vertical parking scene, the presence or absence of the parking line didn't affect the parking completion, and only #1 vehicle could recognize the parking space line.

Evaluation results of parallel parking spaces
The schematic of parallel parking space is shown in Figure 3: The subjective evaluation results of the IPA system for five vehicles at the scene of parallel parking spaces are shown in Table 6: Parking Effect Parking completion 7.0 6.5 7.0 6.5 5.5 1) Compared with vertical parking spaces, parallel parking is more difficult. #2 and #5 vehicles got low scores in the three indicators of parking space search ability, parking effect and parking comfortableness, especially in three subdivision indicators of pull-out ability, parking smoothness and sense of security, for which the scores are all less than 6. During the field verification process these two vehicles had collision with other vehicle so couldn't complete the parking successfully.
2) In the parallel parking scene, the presence or absence of the parking line didn't affect the parking completion, and only #1 vehicle could recognize the parking space line.

Evaluation results of inclined parking spaces
The schematic of inclined parking space is shown in Figure 4: The subjective evaluation results of the IPA system for five vehicles at the scene of inclined parking spaces are shown in Table 7: 2) If there are no border lines in the inclined parking spaces, only the #3 vehicle can successfully complete the parking, but not the #1 vehicle.

Overall evaluation results
Through analyzing the scoring trends of the five indicators in the IPA system, such as human-computer interaction, operation logic, parking space search ability, parking effect and parking comfortableness, in three parking scenarios of vertical parking spaces, parallel parking spaces, and inclined parking spaces (average scores from Table 4-6), it can verify the technical maturity of the current IPA system in different parking scenarios, as shown in Figure 5: As shown in Figure 5, in three parking scenarios, on the two indicators of parking effect and parking comfortableness, the parking difficulty degree increases in sequence of vertical space parking, parallel space parking, and inclined space parking, and the inclined space parking is the most difficult one. As shown in Table 7, in the field verification, several vehicles have been not able to complete the parking.

Conclusion
This article analyzed the technical features of the Intelligent Parking Assist system and the actual demands of consumers for daily parking. Through field verification for three typical parking scenes of vertical space parking, parallel space parking, and inclined space parking, a subjective evaluation system for IPA system was constructed and verified, which is executable and can evaluate the vehicle's IPA system comprehensively and effectively.
Through the analysis on the evaluation results, the vertical space parking is the relatively simplest, and whether L1 or L2 intelligent driving vehicle can complete the parking successfully. The difficulty factor of parallel space parking is higher than vertical space parking. Some vehicles equipped with L1 intelligent driving collided or bumped during parking, and can't complete the parking. Inclined space parking is the most difficult one, and only two vehicles completed the parking in the field verification. If there are no parking space border lines for the inclined parking spaces, even the vehicles equipped with L2 intelligent driving couldn't complete the parking.
In general, the experience of L1 level intelligent parking function is generally lower than L2 level, and the intelligent parking system equipped with camera sensor which can help recognize the parking line satisfies the demands of the consumers more. IPA system can improve the convenience of parking, and avoid the trouble to those who are not able to do parking, but there are some limitations. e.g. in the parking process, there are many steps needing to follow the default logical sequence of the system, which can't cope with the flexible and various real situations. With the development of automobile technology, higher-level automatic driving functions such as automatic valet parking and automatic remote control parking will be gradually applied. From the perspective of subjective evaluation, the development of parking technology should meet the actual requirements of consumers, bringing more convenience to people's daily lives.