Non-contact measurement method of bridge deflection based on machine vision

. Bridge deflection is a very important parameter for bridge structure. It directly reflects the vertical overall stiffness of bridge structure, and is an important basis for reflecting the linear change of bridge. It is closely related to the bearing capacity of the bridge and the ability to resist earthquake and other dynamic loads. Computer vision is to obtain image data through image acquisition device, and use special image processing software to get the shape and displacement information of the object. This method has high precision, low cost, and simple operation. In this paper, based on the industrial camera, the optical flow method and sub-pixel corner detection algorithm are used for target tracking and sub-pixel detection. Through the comparison between the laboratory data and the micro pan tilt data, the usability of this method in bridge deflection monitoring is verified and evaluated.


Introduction
With the rapid development of China's economy and society, China has become the world's largest bridge country, and many bridges have entered the stage of maintenance and repair. Many bridges are faced with detection and evaluation problems. Bridge deflection monitoring is of great significance to understand the health status and working state of existing bridges, and can provide important reference for the bearing capacity and operation status of bridges.
Bridge automatic measurement methods include: accelerometer, microwave interferometry, GPS, measuring robot, tension line method, inclination measurement method, connecting pipe method, etc. The high frequency component is very important in the data collected by accelerometer. When the displacement is calculated according to the integral of acceleration data, the low frequency component will be buried. The deflection of the bridge just belongs to the low frequency component. At the same time, the error will be introduced into the two integral operations. Therefore, the accuracy of this method is not high. Microwave interferometry is used to calculate the bridge deflection according to the phase difference of reflected wave before and after the bridge deformation. This method has high measurement accuracy, but it is not suitable for the situation that the bridge has obvious lateral and longitudinal deformation at the same time. The GPS method is very practical, but the cost is high, and the measurement accuracy is centimetre level. Measurement robot is an automatic and intelligent processing based on total station. The tension line method needs to install contact sensors, so it is difficult to select materials and has low engineering application. The measurement method of inclination angle is expensive, and it is not suitable for large-scale intensive measurement. However, the connecting pipe method needs to lay the connecting pipeline on the main beam of the bridge, and the operation is complex.
In this paper, a non-contact measurement method of bridge deflection based on machine vision is proposed. Based on the analysis of existing technology and equipment, this paper chooses image technology to realize the deflection monitoring of concrete beam bridge. The bridge deflection monitoring system based on image processing is a kind of intelligent visual perception, which uses optical flow method and sub-pixel corner detection algorithm to track the target position and do sub-pixel displacement analysis. The technology has the advantages of high speed, non-contact, high precision, and simple operation. The applicability of this method in bridge deflection monitoring is verified and evaluated by comparing the displacement data of micrometre micro pan tilt in laboratory environment.

Overall design of the system
The whole monitoring system consists of industrial camera, light supplement equipment, target, computer, and image analysis software. The schematic diagram of bridge deflection non-contact measurement system based on machine sensing is shown in Figure 1. Firstly， the monitoring target is fixed at the mid span position of the bridge, or the location point with obvious characteristic value on the monitored bridge is selected as the monitoring point. Then, the video image of the bridge feature point or target is collected by the imaging equipment installed at the static position, and the collected video image is analysed and processed intelligently. Finally, the deflection parameters of the bridge can be obtained by calculating the displacement of the target.

Optical flow matching
Optical flow was first proposed by Gibson [1] in 1950, which refers to the surface motion of grey mode of an image. Optical flow is the 2D instantaneous motion of each point in the image. Once the brightness mode of the image is given, the time-varying feature point matching can be used to measure the time difference. The corner points in the image are taken as the feature points, and the corner points are detected and tracked in the image sequence, Then, the position of the corner in the image sequence is recorded, so that the optical flow field can be calculated by the displacement field of the corner points between adjacent image frames. The methods of optical flow estimation are based on the following assumptions: the change of image grey distribution is completely caused by the movement of the target or background. In other words, the grey level of the target and background does not change with time [2].

Sub pixel corner detection
Corner [3][4] is an important local feature of an image. It contains a lot of important shape information on the image, and has rotation invariance. It is almost unaffected by the illumination conditions. The corner only contains about 0.05% of the pixels in the image. Without losing the image data information, the corner is the minimum amount of data to be processed and accelerates the speed of target detection, Corner detection has a certain practical value, especially in the field of real-time target tracking. At present, the commonly used corner detection algorithms [5] are Harris, Shi Tomashi corner detection and sub-pixel corner detection methods. In fact, the corner detection results in opencv are not accurate enough, because in the real calculation, some positions may have the maximum value in the floatingpoint space, so we need to fit in the pixel neighbourhood space through the given response value to achieve subpixel corner detection. The related APIs and explanations in opencv are as follows: Image single channel input image, 8 bits or floatingpoint number. Corners is the coordinate set of key points in input and output. Winsize indicates the window size of interpolation calculation. Zerozone represents half of the side length of the dead region in the middle of the search region, which is sometimes used to avoid the singularity of the autocorrelation matrix. If the value is set to (-1, -1), there is no such area. Criteria corner is used to refine the termination condition of the iteration process.
The results of corner extraction by sub-pixel corner detection algorithm are shown in Figure 2.and text. In addition, some colour figures will degrade or suffer loss of information when converted to black and white, and this should be considered when preparing them.

Target tracking algorithm
The main idea of using optical flow algorithm to track feature points in this paper is to realize target tracking by optical flow matching with sub-pixel points. The flow chart of the algorithm is shown in Figure 3. As can be seen from Figure 3, the basic processing steps are as follows: (1) After selecting the monitoring target area, image preprocessing is carried out, and sub-pixel corner detection algorithm is used to extract the corner features of the target, and then a certain target area is selected to exclude the corner of the background; (2) For each subsequent frame to be matched, the optical flow method is used for point matching to find the corresponding position of each corner in the new image. The displacement field is obtained by accumulating the optical flow field Select the ROI 3 Experimental results

Experimental overview
The schematic diagram of the experimental structure is shown in Figure 4. The industrial camera and the macro guide are both installed on a tripod, and the target is pasted on the hand-operated macro guide (accuracy 0.01mm). The industrial camera is connected to the computer through the GigE interface. The computer is equipped with special image acquisition and image processing analysis software.
In the experiment, the industrial camera is mv-ch120-10gm of Hikvision. The camera is equipped with Sony imx304 chip, with a frame rate of 9.4 FPS and a resolution of 12 million pixels. The performance parameters of industrial camera are shown in Table 1.  First, run the image acquisition software as shown in the figure, preview the image captured by the camera, adjust the aperture, focal length, angle, exposure time, gain, frame rate and other parameters of the lens according to the picture, and then click the image save button. Finally, move the pan tilt by hand for a fixed distance, and then click to save the image. Through the above operation, a series of pictures are collected. Then run the image analysis and processing software as shown in the figure, import the series of pictures taken before. First, set the reference picture sequence, start picture sequence, end picture sequence, sequence interval and other information. Then select the monitoring area and draw ROI area by mouse click. Finally, click the run button for analysis and processing.

Data comparison and analysis
In order to verify the accuracy and stability of the system, after the installation and debugging of the system is completed, five photos are taken as the initial value, and then five photos are taken after each 1cm movement of the micro cloud platform. In the end, 45 photos were taken. Through the image analysis and processing software, the test data curve is obtained as shown in the figure below. The abscissa represents the number of photo frames taken, and the ordinate represents the target displacement (unit cm). The deviation compared with the micrometre data is within 0.1mm.
The actual displacement refers to the displacement caused by micrometre movement. The software processing data refers to the cumulative displacement of the corner from the initial position to the end position, which is accumulated by the displacement of the corresponding target in two or more adjacent frames.

Conclusion
This paper introduces a non-contact measurement method of bridge deflection based on machine vision. Traditional bridge deflection monitoring methods have some problems, such as inconvenient operation, unsuitable for long-term monitoring, low measurement accuracy, high cost, complex operation, and poor engineering practicability. In this scheme, industrial camera, target, fill light and computer are used in hardware, and optical flow method and sub-pixel corner detection algorithm are used in software for target tracking and sub-pixel detection. Finally, through the comparison between the laboratory data and the micro motion pan tilt data, the error is less than 0.05mm. Through the data verification, the practicability of this method in bridge deflection monitoring is evaluated.