Substation Indoor Wheeled Robot Inspection Visual Blind Pressure Plate Recognition

In order to solve the problem of wheeled robots in the main control room of the substation due to the height of the robot and the shooting distance, the main camera cannot take a complete picture of the bottom pressure plate of the relay protection pressure plate cabinet during the inspection process and perform open/close state and label image recognition. Propose install a wide-angle secondary camera on the side of wheeled robot, and targeted proposal for the correction of distorted images taken with a wide-angle lens using a division model, improving the accuracy of decompression plate checks. Experiments have shown that, the proposed approach is practical and feasible, effective solution for indoor wheeled robot relay panel cabinet inspection visual blind spot shooting problem, enhanced the effectiveness of wheeled robotic inspections in the main control room of the substation, improved platen verification accuracy.


Introduction
Due to the large number of substation device pressure plates, the manual checking task is heavy and prone to errors. Based on research in the literature [1][2][3], an extremely valuable and feasible option at present is to use an indoor wheeled robot to replace the manual indoor inspection of substations and perform pressure plate verification. However, during the inspection process of the wheeled robot, due to its own height, the main camera forms a blind area in the lower area of the pressboard screen cabinet, and cannot take a complete picture of the presser plate in this area. As a result, it is impossible to take a complete picture of the presser plate of the cabinet. The purpose of recognizing the switching state of the pressure holding board.
In order to correct the nonlinear aberrations caused by wide-angle lenses, a lot of experimental research work has been carried out at home and abroad.Considering the modeling direction, the correction algorithms for nonlinear aberrations can be divided into 2 major categories: one is to first assume the aberration correction model of the aberration map and the ideal map, and then interpolate and fit the parameters in the model according to the selected control points. C. Hughes used the polynomial model method to achieve the correction of aberrated images by fitting the aberration law of the wide-angle lens with a polynomial.The literature [3] uses a 3D mapping geometric modeling method to obtain the correspondence between the pixel points on the aberration map and the ideal map through an arc mapping transformation to achieve aberration correction of the image.The literature [4] proposes a method for three-time fitting equation distortion correction, the concentric circle field calibration correction algorithm of the literature [5], and the BP neural network-based method for image geometric distortion correction proposed by the literature [6].However, since the model itself is an approximation, the correction of distorted images by modeling will lead to low correction accuracy and different models for different wide-angle lenses, thus having certain limitations in use.
An easy way to think of is to install a wide-angle sub-camera, though it will cause the stabbing distortion of the captured image, which cannot achieve the purpose of accurately identifying the throw-out status of the relay pressure plate. To address the problem, an solution is proposed, which is to add a wide-angle secondary camera to the wheeled robot to solve the problem of image acquisition in the blind area of the main camera, the distorted images taken with the wide-angle lens are then corrected using a division model and a convolutional neural network image recognition method is used to solve the problem of accurate recognition of the platen throwback state based on the corrected images.
This paper applies the identification method to a wheeled robot in a 220kV substation relay room (45 relay cabinets, 20 of which have a blind shot problem) and performs platen status identification with a success rate of 100%. Field tests have demonstrated the effectiveness and practicality of the method.

Blind spot problem
As the relay protection and automatic device screen cabinet is over 220cm high, the lowest point of the receding pressure plate from the ground is 22cm, the height of the wheeled robot body is 43cm and the lowest distance from the ground for the retractable lifting camera is 70cm, as shown in Figure 1.
The distance between the aisles of each row of screen cabinets in the substation relay room is about 80~120cm. In order to ensure a safe distance, the robot can only inspect the screen cabinets at a distance of 30~50cm. We tested the visible area of the main camera of the wheeled robot at distances of 48cm, 58cm and 68cm from the screen cabinet, as shown in Table 1. A diagram of the visible field of view of the main camera at different shooting distances is shown in Figure 1.  In the inspection specified safety distance 30~50cm range to relay protection pressure plate screen cabinet for inspection and photography, the analysis of the experimental results are as follows: (1) The minimum height at which the main camera can capture the screen cabinet is 28cm above the ground.(2) The actual relay protection pressure plate is at a minimum distance of 24cm from the ground and the pressure plate label is at a minimum distance of 22cm from the ground.(3) If the camera looks down and shoots, the platen label cannot be captured in its entirety because it is blocked by the platen, which hinders the identification and verification of the platen image and the platen label.
In summary, the main camera of the wheeled robot is constrained by the robot's own height size and shooting distance when inspecting and taking pictures of the platen screen cabinet, the main camera is mainly unable to shoot and check the platen in the area below 28cm from the ground, while other areas do not have visual blind spots.

Image acquisition for wheeled robots with blind visual fields
In view of the aforementioned visual blindness problem of robots and the study of literature [3][4], the proposed method of mounting a secondary camera on the side of the robot body to solve the visual blindness image acquisition problem below 28cm from the ground. A wide-angle camera is mounted on the side of the wheeled robot at 34 cm from the ground, with a schematic diagram of the camera mounting position, as shown in Figure 2.
The secondary camera is a Hikvision model DS-2ZMN0407(C) HD integrated network movement camera. The specific technical parameters are shown in Table 2.

Algorithm for distortion correction of platen images taken by wide-angle lens
Due to the distortion of the barrel shape of the image taken by the wide-angle lens, the shape of the platen arrangement in the image has changed and the platen arrangement has changed from a straight line to an arc. The platen status on both sides of the arc cannot be accurately identified.

Principle of the Image distortion correction algorithm
From the literature [5], as the wide-angle lens uses a convex lens, the further away from the center of the lens, the stronger the refraction effect, so that the picture is taken with the center of the lens as the center of the circle, a circular outward distortion. This distortion is shown in figure 2.
a-the barrel distortion principle b-Barrel distortion diagram  . (x 0 , y 0 ) is the coordinate of the center of the distorted image. λ is the distortion parameter, estimated from the measurement image.

Example of distorted image correction
According to the principles described in 3,the distortion parameters are determined by studying whether the linear image is distorted or not. Let y=kx+b be the linear equation, where k is the slope and b is the intercept, to obtain the linear undeformed equation y u =kx u +b, which can be obtained from equation (1) − −

(2)
Simplified as flows: Sets: From the relationship between A, B, C, we get In summary, the workflow of the algorithm for estimating image distortion parameters is as follows: (1) Select n (n≧3) "straight lines" li (i=1,2,3,...,n) from the image. (2) Using equation (3) to fit each "straight line" l i to a circle, determine the (A i , B i , C i ). (3) Calculate λ according to equation (4).
Based on the estimated x 0 , y 0 and λ distortion parameters, the corrected image coordinates can be calculated according to equation (1) to obtain the corrected image. In this example, an image with a resolution of 886*720 pixels is used as the object of study.

Mobile-net theory
CNN have now been used extensively in computer vision applications, and with good results, with the following problems:(1)In order to improve the accuracy of the network, the complexity becomes higher with the depth of the model. (2)In some application scenarios, it is difficult to be used due to hardware resource constraints and low latency requirements. The Mobile Net is a small CNN which is characterized by: performance is maintained and model complexity is reduced. The core of the model is depth-level separable convolution, the central idea being that convolution is divided into two operations: deep convolution and point-by-point convolution. Deep convolution corresponds to a convolution kernel for each output. Point-by-point convolution uses a 1x1 convolution kernel to integrate the outputs of the deep convolution. The depth-level separable convolution principle is shown in figure 4.

Verification and analysis of blind area problem solving methods
The secondary camera was photographed and tested for visual visibility at a distance of 30cm, 40cm and 50cm from the screen cabinet, the results are shown in Table 3. A diagram of the visible field of view of the secondary camera at different shooting distances is shown in figure 5. c-Taken at a distance of 50cm From experimental tests it follows that:(1) When the camera is 30cm away from the object, the camera can shoot things above 16cm from the ground. (2) At a distance of 40cm, it is possible to photograph things more than 11cm above the ground.(3) At a distance of 50cm, things can be photographed more than 9cm above the ground.(4) Areas where the shooting area overlaps alternately with the main camera field of view.In summary, the secondary camera effectively solves the problem of the robot's main camera blind spot.

Graphical distortion correction experiments and discussion of results
In a 220kV substation relay room during the pressure plate verification experiment, 45 relay protection and automatic device cabinets were inspected and verified.
In this study, MobileNet was used to implement platen image recognition checking. The platen verification experiments were carried out on the distorted and corrected images taken by the robot, with the image resolution of 704*576 pixels. The number of recognizable regional platens was 27, and the number of recognizable platens in the distorted image was 21, with a recognition rate of 77.7%; the number of recognizable platens in the corrected image was 27, with a recognition rate of 100%. The comparison between the results of the distorted image and the corrected image is shown in figure 6. a-Recognition results before image correction b-Recognition results after image correction The experimental results show that the recognition results are seriously affected by the distortion of the image taken by the robot's secondary camera at close range. After correcting, the image recognition accuracy of the platen throw-back state is effectively improved.
By using the inspection visual blindness solution and distortion image correction method proposed in this paper, the pressure plate verification problem of 20 of the cabinets with blind areas was solved and the accuracy rate of pressure plate verification reached 100%.

Conclusion
This study uses wheeled robots to carry out pressure plate timing inspections and special inspection checks on 45 relay protection and automatic device panel cabinets in 220kV substation relay rooms, and the following conclusions were reached: (1) Affected by the height of the robot and the shooting distance, the visual blind zone is mainly the inability to shoot and check the pressure plate in the area below 28cm from the ground, and there are no other blind zone problems. (2) This paper proposes for the first time the method of installing a secondary camera on the side of the robot, which enables the robot to capture information about the platen at a minimum distance of 9cm above the ground, effectively solving the problem of image acquisition in the robot's blind zone. (3) Using a division model, the distorted image captured by the wide-angle lens is effectively corrected, making the accuracy of the corrected image target recognition reach 100%.