Vehicle automatic driving control system based on image recognition

. With the development of technologies such as intelligent, automatic control and automatic detection, the research of smart cars has been gradually applied to all levels of people's social life. The design aims to use K60 as the control chip of smart car and combined with image recognition sensor. Identify road condition information, feedback steering gear, motor control, to complete the intelligent driving of the smart car under complicated road conditions. The experimental results show that the control system has good response characteristics, good stability and meets technical requirements.


Preface
The smart car is a control system with environment sensing function, route planning function and automatic driving function. The environment awareness can collect real-time environmental information through the sensing device and transmit it to the control chip through various communication methods; the route planning is determined by the control chip according to the real-time environmental road information to plan a suitable driving path(eg: whether to bend Whether it is a cross bend, whether it is a ring bend, Whether to enter the bend, whether to out of the bend, whether there is an obstacle, etc.); for the automatic driving, the steering path planned by the control chip controls the steering gear and the motor through the PID algorithm to obtain meet the angle and speed of the driving conditions and complete the automatic driving according to the planned route.

System hardware design
First, the design process of the entire smart car system involves computer, sensing, information, communication, navigation and automatic control technologies, and is a typical high-tech complex. Therefore, the design of the smart car will start from the following three aspects: chip control circuit, sensor signal acquisition circuit, steering gear and motor control circuit. The block diagram of the smart car system is shown in Figure 1. The smart car uses the K60 chip as the main control chip. The chip has a 32-bit ARM Cortex-M4 core, 512KB of Flash; 128KB of SRAM; 4 PIT timers; 16 DMA channels; 4 FTM modules; 2 I2C modules and 2 LPTMR modules. The FTM module is a multi-function timer module. The main functions are PWM output, input capture, output comparison, timing interrupt, pulse updown counting, and pulse period pulse width measurement.
The K60 control chip can accept the image of the road ahead from the image sensor and decompress the collected information. The control chip decodes and calculates the information collected from each field, extracts the corresponding black line information, and then fits the midline to find the deviation in the driving process of the intelligent vehicle. Then it adjusts the steering gear angle of the intelligent vehicle through the PD algorithm, so that the driving direction of the intelligent vehicle can be adjusted accordingly and the intelligent vehicle can be guaranteed. Drive as far as possible in the midline. Then the motor is controlled by the PID algorithm, so that the intelligent vehicle can adjust the speed independently.

2.1FormattingSensor circuit and speed acquisition circuit design
The design uses the wildfire eagle eye OV7725 camera as the environmental image data acquisition and processing module. It is used to collect real-time road information during the running of the smart car. At the same time, the collected front road information is transmitted to the control chip for processing. [1] The camera has hardware binarization, ideal binarization effect, excellent image processing effect, image data acquisition speed is extremely fast, the rate can reach 150 frames per second, the output image is output from left to right, and there is no parity. Points, strong denoising ability, high stability and low light sensitivity, can be used for image acquisition during high-speed driving, to ensure that the captured image data has a certain clarity. At the same time, the image signal collected by the camera is observed by using the 1.44inch LCD screen outside the mountain, and the processing of the image signal of the smart car during driving is monitored, and the relevant information and working parameters of the smart car in the driving process are displayed in real time. In order to complete the planning of different driving lines at any time by changing parameters, find and plan the optimal route of the smart car, and complete the modification of the realtime parameters of the smart car.
In order to achieve speed monitoring during the driving of the smart car, we completed design with the Omron's E6A2-CW3C dual-phase 500-line encoder.The encoder is a device that compiles signals or data into communications, transmissions, and storage. Divided into incremental and absolute. The incremental encoder converts the speed signal into a periodic electrical signal, converts it into a pulse signal, calculates the number of pulses per unit time to calculate the speed; the absolute encoder has a corresponding definition at each position. The numerical code, whose value is only related to the initial and end positions of the measurement, independent of the intermediate process of the measurement. Therefore incremental encoders are generally used for speed measurement, and absolute encoders are generally used for displacement measurement. In a speed closed-loop control system, the motor speed is measured by an encoder to measure the speed of the smart car at a certain moment, and then the speed difference is adjusted by the software to make the smart car reach the expected running speed.

FormattingDesign of motor control circuit
In the operation of the smart car, the control chip can use the PWM technology to control the speed and forward and reverse of the motor. Considering that the output signal current of the chip is too small, it is not enough for the motor to work normally. It is necessary to use the motor drive module to amplify the output signal of the control chip and drive the motor to work. The motor used in this smart car has a large power, so the N-MOS H-bridge is used to complete the work of the drive motor. The design will use the IR2104 chip as a half-bridge driver. The driving chip has a bootstrap floating power supply, and the driving circuit is relatively simple, and the upper and lower bridge arms can be driven simultaneously by one circuit, and has dead time control, stable driving, and can suppress the bias voltage, etc., can avoid circuit misoperation, improve circuit Stability performance. N-channel power MOSFETs driven by floating channel technology operate from 10 to 600V. The structure diagram of the H-bridge drive circuit is shown in Figure 2. The circuit shown in Figure 2 is an H-bridge circuit consisting of two half-bridge circuits. During operation, the upper and lower arms of the half bridge will alternately open. When the lower arm is opened and the upper arm is closed, the potential of the VS pin is the saturation conduction voltage drop of the lower arm power tube, usually maintaining the ground potential. The bootstrap capacitor C8 (C11) is charged by the bootstrap diode D3 (D10) until its voltage rises above 12V. When Q2 (Q4) is turned off, the voltage at the VS terminal will rise. At this time, the voltage across the capacitor is still close to 12V. When Q2 (Q4) is turned on, use C8 (C11) as the floating voltage source to drive Q2 (Q4);In the next cycle, the charge lost by C8 (C11) during the opening of Q2 (Q4) will be replenished, thereby achieving the bootstrap power mode by constantly swinging the level of the VS terminal between high and low levels. The bootstrap circuit charges the capacitor and the voltage across the capacitor floats up and down based on the source voltage of the high side output transistor. D3, D10, C8, C11, etc. are all auxiliary components of IR2104. These components are strictly selected and designed in pulse width modulation applications, and are adjusted in circuit experiments to make the circuit work optimally. Among them, D3 and D10 are important bootstrap devices that block the high voltage on the DC main line and block the product of the gate charge and the switching frequency when receiving current. In order to reduce the charge loss, we selected the fast recovery diode (1N5819) with a small reverse leakage current. The power supply in the high voltage section comes from the charge on the bootstrap capacitors C8 and C11 in the figure, and the size of the two capacitors is reasonably chosen. C9, C12 uses a 20V10UF tantalum capacitor to filter the 7.2V supply and 12V output. The D5, D6, D11, and D13 diodes function as a freewheeling effect for the IN5819 diode. In order to avoid overvoltage in the motor drive circuit, the diode is reversed at both ends of the motor. When the IR7843 is turned on, the diode is reversed. When the IR7843 is closed, the diode conducts a closed loop with the motor and the current continues. [2]

Figures Control strategy and software implementation 3.1Steering gear PD control strategy
The software program written by the control chip controls the steering gear of the smart car to ensure that the smart car can stably travel according to the planned route during the running process, and if there is a deviation, the servo work is adjusted through the software program.
The control chip controls the steering of the smart car through a software program combined with the PD algorithm to control the steering angle of the steering gear to realize the cornering control of the smart car. By measuring the deviation between the current position of the smart car and the centerline of the road, and using multiple measurements, combined with the PD algorithm to achieve rapid movement of the car on the centerline.The PD algorithm controls the source code as follows: if(SUM_offset<0)//Deviation value is less than 0 steer1=(uint16)(steer0+KP*SUM_offset); else steer1=(uint16)(steer0+KP*SUM_offset); Steer=(uint16)(steer1+KD*(steer1-steer2)); if(Steer<400) Steer=400;// Leftmost 400 if(Steer>560) Steer=560;// Most right 560 ftm_pwm_duty(S3010_FTM,S3010_CH,Steer); steer2=steer1; } SUM_offset is the deviation between the current position of the cart and the centerline of the road. This variable is the deviation obtained when the road is detected. KP is the proportional coefficient. When the value is small, the static difference will occur, and the response speed will be significantly reduced. When the coefficient is increased, the static difference can be eliminated to some extent, and the response speed can be improved. The KD value can reduce the static difference.

Figures Experimental testing and data analysis 4.1SteeringIdentify the left and right sideline driving experiment
The original image captured by the camera takes into account various road conditions, light intensity problems, image noise, and sharpness problems, resulting in reduced image effects. In the actual design, the above interference information will be eliminated through the software program, to effectively identify the planning path, and provide as much information as possible of the road condition identification program, so that the chip can effectively control the steering gear. [3] The search for the expansion of the left and right sides of the four columns, you can also choose to expand the six columns, 22 columns, 10 columns, etc. according to the actual situation, to avoid the image acquisition problem of the found edge, affecting the planning of the overall driving route of the smart car . The image during the experiment of identifying the left and right sidelines is shown in Figure 3. Through comparison between the two figures, it is found that when the smart car is driving on the road, it will travel in the middle of the road based on the left and right side lines in the road. If the running track is deviated, the smart car will continuously pass the steering gear according to the deviation angle from the left and right side lines and the center line. Make a callback to ensure that the smart car is driving normally on the track. On the display screen, we can find that the middle dotted line is the added center line, which is used to judge whether the driving track of the smart car deviates.

PositioningRecognition of Crossroads and Bends Driving Experiments
When the smart car needs to pass through the crossroads and bends, the image information collected by the camera will be combined to complete the driving path planning. Specifically, when the camera collects and detects the current road condition until there are multiple lines of white, there is no left and right black line, and then the front is determined to be a crossroads and bends. [4]The image acquired during the cross bend is shown in Figure4. According to the comparison between the two figures, when the display screen does not recognize the left and right side lines, the smart car will travel along the center line track added in the image, set the center line to the middle column of the whole image, and pass straight through;When the left and right sidelines are detected again, they continue to follow the sideline identification mode and finally pass through the crossroads and bends area.

PositioningIdentify obstacles driving experiment
In the process of driving a smart car, how to avoid obstacles is an inevitable problem. If the corresponding route planning is not done, the smart car will directly hit the obstacle, causing the vehicle to deviate from the route, roll over and even damage.Therefore, when the smart car encounters an obstacle,(black obstacle) it will re-plan the route, bypass the obstacle and continue driving. [5]When the obstacle is identified, the image captured during the driving of the smart car is shown in Figure 5. According to the comparison chart, when the obstacle is encountered while driving, the left and right side lines of the road will change. If the obstacle appears on the right side of the road, the right side line will start from the left edge of the obstacle. Detection, the obstacles are excluded from the edge of the right side of the road, so that the smart car can drive on the remaining roads except the obstacles based on the midline track, ensuring that obstacles are avoided in time while driving, keeping the smart cars stable.

Summary
According to the actual road conditions, the program judgment design under different complicated road conditions can be increased accordingly to ensure the stability of the smart car during the automatic driving process. At the same time, various types of sensor detection devices can be integrated on the smart car to complete the unknown road. Exploration. Identifying the left and right sideline driving experiments,Recognition of Crossroads and Bends Driving Experiments, and identifying the obstacle driving experiment shows that the hardware design and algorithm can achieve the expected experimental parameters.