Research on the Application of Computer Vision Technology in Power System UAV Line Inspection

. The stability of transmission lines is one of the important factors to ensure stable power supply. In this paper, a set of UAV intelligent power line inspection flight control system is designed according to the special flight requirements of UAV in power line inspection operation based on the principle of photogrammetry. The key technologies of the system include UAV flight attitude control, high-precision time synchronization between multiple sensors, defect detection and intelligent diagnosis, airborne sensor calibration, wireless communication, and ground data processing. In this paper, a camera and an on-board computer are innovatively used for real-time image processing and pattern recognition for wire feature acquisition. At the same time, this paper proposes an improved Canny edge detection algorithm combined with image segmentation technology to achieve wire feature extraction and identification. The field test results show that the system can complete the remote control of the UAV and realize the intelligent navigation and inspection of the transmission line UAV.


Introduction
The stable supply of electricity is a key factor to ensure the stable development of people's life and production.Large-scale power outages caused by line failures will cause serious economic losses.Due to the extremely harsh working environment of the equipment in the transmission line, it has been affected by exposure to sunlight, acid rain, freezing and typhoons over the years, and failures may occur within the service period.In order to prevent large-scale power outages due to failure of electrical equipment, it is necessary to regularly inspect the transmission lines and replace damaged equipment in a timely manner.The detection methods of transmission lines mainly include manual line inspection, humanhelicopter inspection, robot inspection and unmanned aerial vehicle inspection.Manual line inspection is not only cost-effective and inefficient, but also threatens the personal safety of maintenance personnel.Although manned helicopter patrol line has high work efficiency, it requires high technical level of operators.Although the robot patrol line has a long battery life and no patrol blind spots, it will cause certain damage to the power line due to its own weight [1].UAV has the advantages of low working cost, easy operation, and low operation risk.Therefore, the use of UAV line inspection can effectively reduce the operating cost of the power grid and improve the efficiency of inspection work.UAV line inspection has now become one of the important means of transmission line inspection.In this paper, the work process of UAV line inspection based on machine vision is that the UAV is equipped with camera equipment (optical, remote sensing, infrared), and machine vision technology is used to identify, locate and shoot the detection target and determine the fault.This paper adopts the existing technical products and modules, and comprehensively uses the millimeter-wave radar, ultrasonic radar, RTK high-precision fusion positioning and improved image recognition technologies to achieve a breakthrough in the key technologies of UAV intelligent tracking, navigation and inspection.

UAV line inspection system design 2.1 System process design
The electric line patrol flight control system is a flight control software customized for the line patrol requirements of the UAV.Through simple and reasonable route design, the purpose of making the supporting UAV realize automatic or semi-automatic operation is achieved.The technical process is as follows as shown in Figure 1 (the picture is quoted in FAILURE ANALYSIS OF A UAV FLIGHT CONTROL SYSTEM USING MARKOV ANALYSIS).1.The flow chart of the UAV flight control system in the detection of tree obstacles in power line inspection Figure 1 describes the working principle and specific process of the power line patrol flight control system.First enter the coordinates of the tower, then set parameters such as flight height and photo overlap, select the head and tail towers that need to perform the line patrol task, then upload the task and wait for the UAV to load, click Start task to make the UAV start fully automatic flight, the flight status can be monitored throughout the flight, and the task is completed when the drone returns and lands.

System Architecture
The structure of the power line safety inspection system based on remote sensing technology is divided into four parts: UAV platform and data acquisition system, ground measurement and control station, data communication link system, and ground data processing system [2].The main block diagram of the drone inspection system is shown in Figure 2 (the picture is quoted from Present state and future prospect of autonomous control technology for industrial drones).

UAV platform and multi-sensor data acquisition system
This subsystem is the main platform for power line inspection data collection and acquisition, including UAV flight platform, multi-sensor data acquisition system, stable platform, positioning and attitude system (POS), autonomous early warning and obstacle avoidance system, data encoding and decoding system and other equipment.
The composition and functions of the main equipment are as follows: The UAV flight platform can be divided into two categories: fixed wing and rotary wing.The UAV flight platform is responsible for carrying sensor systems, stable platforms, POS, autonomous early warning and obstacle avoidance systems and other necessary airborne equipment [3].According to the specific requirements of inspection tasks, flight distance, flight speed, etc., different flight platforms should be selected.For example, to carry out rapid inspection of visible light shooting, a small fixed-wing UAV platform can be selected; For fine inspections including infrared, ultraviolet, visible light, etc., and at the same time need to have sufficient flight distance, a large rotary-wing UAV platform should be selected.POS is mainly used for the real-time determination of the position and attitude information of the UAV.In the power line safety inspection system based on remote sensing technology, it is also used to locate and mark the data obtained by the UAV, so that various sensor data can be unified.It is convenient to carry out multisource data collaborative identification of defects and hidden dangers in the same spatial coordinate system.

Ground TT&C Station
The ground measurement and control station subsystem are mainly used for the monitoring and control of the flight status of the UAV during the flight process, the piloting of the flight, and the control of the sensor data acquisition method.The subsystem includes UAV platform ground control system, multi-sensor ground control system, data encoding and decoding system, and real-time data analysis system and other equipment.

Data communication link system
This subsystem is the key equipment for the communication connection of the whole system, including the signal receiving/transmitting equipment of the airborne and ground measurement and control stations and the UAV communication relay equipment.This subsystem is mainly responsible for ensuring the data communication between the ground measurement and control station and the UAV flight platform, especially in the environment with harsh communication conditions such as mountainous terrain with complex terrain to ensure reliable communication between the UAV and the ground [4].The data link should first ensure the real-time download of UAV positioning and attitude data and the upload of control commands, so as to complete the tracking and control of the working state of the UAV.The transmission rate of the data link is customized according to the actual needs.Generally, at the rate of 4Mbps to 8Mbps, the real-time download of video data can be realized, so that the ground measurement and control personnel can intuitively understand the flight scene.

Ground data processing system
The subsystem is a post data processing, storage and application system, including a multi-sensor data preprocessing and geometric processing system, an intelligent expert system for power line safety inspection based on laser, optical, infrared, ultraviolet and other sensors, as well as a three-dimensional system of line corridors.visualization system, etc.The ground data processing system uses photogrammetry and remote sensing data processing methods and technical processes to perform high-precision geometric processing on various images, point clouds, videos, coordinates, and attitude data.On this basis, according to the characteristics of power lines, towers, corridor features, etc. and their appendages, through the artificial intelligence, pattern recognition and various visualization technologies of the expert system, the identification and confirmation of transmission line and channel defects and hidden dangers are realized.Quickly locate the accident point of the transmission line, and realize the timely diagnosis and troubleshooting of the line safety status.The quadrotor drone must be unlocked before use.The take-off process is that after the drone is unlocked, the flight controller controls the altitude and speed of the drone by controlling the governor according to the control signal [5].After entering the detection area, you can choose to enter the automatic inspection mode.

Line-following flight
After the drone reaches the desired height that needs to be detected, it will hover at the desired height, and then enter the automatic line inspection mode.The drone will automatically conduct automatic inspection according to the direction of the power line.And we will know the location of the fault.At this time, we only need to send a special person for targeted maintenance, which can greatly save manpower and time [6].The specific process is that when the drone enters the route flight mode, the flight controller will read the current position information, and then use the barometer to determine the current altitude information, and also obtain the current position information through GPS, and then use the Karl Mann filter fusion, the PID is controlled by the fused signal to change the flying height of the UAV, and control the change of the horizontal position of the UAV, and finally realize the inspection of the UAV.

Automatic landing of UAV
After the drone completes the line patrol action, it needs to return.In this design, a one-key return is designed.In the seventh channel on the remote control, a one-key return channel operation is designed.When the remote control is dialed to automatically land the gear is, the drone will obtain the position of the drone, and the barometer will obtain the air pressure state at the altitude and compare it with the initial air pressure state.If it is different from the initial state, the flight control will perform PID tuning.To control the altitude state of the drone, and then obtain the position information through GPS, if the desired horizontal position is not reached, the flight control will perform PID tuning to control the horizontal position of the drone, and finally perform PID altitude tuning to finally achieve landing.

Design and implementation of line patrol algorithm
When the drone is patrolling the line, no single sensor can get accurate control feedback information.The multisensor data fusion of UAV is actually a process of learning from each other's strengths and complementing each other's weaknesses.At the same time of data fusion, image recognition is used to further improve the accuracy of fusion data [7].Therefore, the system adopts data fusion algorithm and image recognition algorithm to realize the intelligent line patrol process of UAV.

Data fusion
The attitude calculation adopts the complementary filtering algorithm, which integrates the data of the gyroscope, accelerometer and / GPS RTK to obtain the angle between the UAV and the horizontal plane.Among them, the gyroscope can obtain the rotation angle within the integration time t  , that is, the rotation quaternion t Q  , and if the current attitude is Q , the attitude at the next moment can be predicted as , / ccbody a GPS RTK of the machine system can be obtained from the accelerometer, the velocity data enu v  of the geographic system can be obtained, and the acceleration data ccenu a of the geographic system can be obtained after the difference.Therefore, the gravity vector under the machine system can be expressed as: The angle between the vectors Build the observation and prediction equations: Where v and w are observation noise and process noise, respectively.The revised state estimate is then updated using the Kalman state update equation:

Image recognition algorithm
For the currently acquired video data of the power line, the system image recognition algorithm flow is shown in Figure 4 (the picture is quoted from A UAV-based machine vision method for bridge crack recognition and width quantification through hybrid feature learning).Among them, the steps of Gauss filtering, Canny edge detection and morphological filtering are image preprocessing, and the last two steps are feature detection and feature screening respectively, and finally realize wire detection.The image recognition algorithm uses grayscale image preprocessing, and according to the environmental characteristics of the detected object, uses Gauss filter to convolve the image to smooth the image, reduce the obvious noise effect on the edge detector, and effectively improve the next step.Edge detection effect.
The system uses a depth camera and an ultra-wide-angle lens for image recognition, and uses the improved Canny algorithm to detect the edge of the image.After processing, not only can the positioning accuracy of edge detection be improved, but also the image edge will not be blurred.The core of Canny is to transform the edge detection problem into a local maxima problem of image gradient.The Gaussian function can be used for the best approximation.

System Check
In April 2021, the author's unit used a multi-rotor drone to inspect the overhead transmission line channel of a 220kV line No. 093-134, with a total length of 20 kilometers and a line corridor width of 400 meters.The landforms in this area are dominated by mountains and hills, making it difficult to fly.In this experiment, the multi-rotor UAV was used for aerial flight operations, and the PTZ camera with an effective pixel of 20 million was used for vertical aerial photography, with a relative altitude of 115 meters.The 3-axis stabilization system mounted on the gimbal can greatly reduce the jitter during shooting, improve flight stability and ensure the accuracy of subsequent aerial triangulation of images.In the case of using the newly developed power line patrol UAV flight control system throughout the process, the UAV always flies according to the preset trajectory, image overlap, and relative altitude, and at the same time flies stably and has a good attitude.The clarity of the captured images fully meets the requirements of subsequent use of various postprocessing software.The precise position and attitude of the images in the air are determined through aerial triangulation, and then the ground 3D point cloud and wire point cloud are obtained through intensive matching.The distance error between the measuring wire and the three-dimensional point cloud on the ground meets the obstacle distance detection standard of power inspection.The subsequent process of aerial triangulation through software is shown in Figure 5 below:

Conclusion
This paper proposes a set of power inspection system based on UAV.The UAV will complete the line inspection task according to the pre-established flight route, and the staff can also perform power inspection by manual operation.During the line patrol mission, the UAV and the ground server will transmit images and videos with high bandwidth and high stability through the operator's wireless network card, and support up to 1080p60i full HD video live broadcast.For the received logs, images, videos and other files, the system will intelligently archive and save them for easy retrospection.At the same time, the system will automatically perform temperature measurement, calibration and analysis on the images, and alarm for abnormally high temperature components and towers, ensuring that the power patrol It improves the real-time and accuracy of inspection tasks, and also improves the quality and efficiency of inspection.

Fig. 2 .
Fig. 2. Block diagram of the main components of the UAV inspection system Design a cascade PID controller to control the position, speed and acceleration of the UAV at the horizontal and vertical heights.The structure of the controller is shown in Figure3(picture quoted from Recent Advances in Unmanned Aerial Vehicles: A Review).Compared with the ordinary position controller, the cascade PID controller simultaneously performs closed-loop control on the position, speed and acceleration of the UAV.The position information obtained according to the accelerometer data is used as the feedback value of the position, velocity and acceleration PID control.This constitutes a three-level closed-loop control, which can greatly enhance the anti-interference ability of the UAV.

Fig. 3 .
Fig. 3.The realization of the line-following function between the drone and the horizontal plane.The correct attitude angle can be estimated by taking the complementary correction of the included angle predicted by the gyroscope.The position data adopts the Kalman filter algorithm, fuses the data of the position sensor ( / GPS RTK , millimeter wave, ultrasonic), and estimates the real-time position and velocity information.Set the state variable s represents position and v represents velocity.