The control system of universal platform for agriculture based on machine vision technology

. The development of the agricultural industry is impossible without automation of the processes of field preparation and harvesting. One of the ways to solve this problem is the implementation of machine vision technologies supplemented with neural networks in the implementation of automated control systems for agricultural equipment. The implementation of machine vision algorithms will allow the recognition of objects in the workspace, the adjustment of the route of movement of technology, the realization of its various operating scenarios. Neural networks will allow you to analyze the surrounding objects and choose the best route to move. In this article, we consider an algorithm for determining objects based on machine vision technologies and the selection of a working area on a frame. The analysis of the intersection of the working area with recognized objects allows you to create controls that regulate the trajectory of traffic. The obtained results are experimentally verified on a laboratory prototype of a universal platform for agricultural machinery. Various approaches to the selection of object boundaries are considered and tested.

Abstract. The development of the agricultural industry is impossible without automation of the processes of field preparation and harvesting. One of the ways to solve this problem is the implementation of machine vision technologies supplemented with neural networks in the implementation of automated control systems for agricultural equipment. The implementation of machine vision algorithms will allow the recognition of objects in the workspace, the adjustment of the route of movement of technology, the realization of its various operating scenarios. Neural networks will allow you to analyze the surrounding objects and choose the best route to move. In this article, we consider an algorithm for determining objects based on machine vision technologies and the selection of a working area on a frame. The analysis of the intersection of the working area with recognized objects allows you to create controls that regulate the trajectory of traffic. The obtained results are experimentally verified on a laboratory prototype of a universal platform for agricultural machinery. Various approaches to the selection of object boundaries are considered and tested.
The area of agriculture is important for the development of the economy of each state; the development of this area has a huge impact on the living conditions of society. Cultivation of crops requires the participation of a large number of people, considerable time spent on the preparation of fields and gardens. Subsequent harvesting and quality control take considerable time intervals and do not always correspond to the desired result due to carelessness or negligence of the working staff. The disadvantages of manual work are obvious: -limited working time of staff; -different modes of operation of each person-operator, which is affected by the internal parameters of each person, such as psycho-physical condition, moral fatigue from monotonous work; -inability to work at night; -inability to maintain the quality of work during the entire working time.
Based on this, the task of minimizing the number of workers and service personnel in agriculture is relevant for each country. Automating the process of harvesting and preparing the fields will reduce the time spent on crop care. The use of automation will allow you to carry out agricultural work day and night, over long periods of time with maintaining high quality workflow.
The analysis of recent achievements and publications in this area showed that the general trend of development of agricultural equipment is its automation, the introduction of unmanned vehicles. For example, Russian company Cognitive Technologies, based in Innopolis (Tatarstan), presented an unmanned tractor that was successfully tested in the field. The tractor uses machine vision technology and GPS technology.
GPS technology is applied to position agricultural machinery during stream harvesting from fields. Machine vision technologies allow you to automatically set the trajectory of movement of equipment. Rows of mown crops are distinguished for harvest by means of the camera. The work of the systems proposed by this company cannot be fully called automatic due to the need for the machinery to be constantly monitored by the operator of the machine, whose presence in the machine is mandatory.
A model of a system that determines foreign bodies in the fields, such as power lines or windbreaks is also presented. The use of the system is fully justified with proper software recognition of surrounding objects, but the lack of operation of the system will be an algorithm for the implementation of equipment's turn when passing through the work area. Without human intervention, the trajectory of the turnaround of machine is dangerous and requires a large number of objects to be determined. For this case, the implementation of specially trained neural networks is necessary [1].
With all the advantages of the systems offered by Cognitive Technologies, they still require the direct presence of service personnel in each agricultural machine. Therefore, these systems cannot be fully called automatic, since they are not able to carry out a complete workflow without the direct presence of the human operator.
Company FarmView has developed a prototype software and hardware platform for detecting diseases that occur on sorghum plants. The analysis of the fields with plants is carried out on platforms like quadrocopters. Algorithms of machine vision and neural networks are used to search for diseases. A similar approach to the analysis of large volumes of agricultural land is relevant in the climatic conditions of South Carolina. The disadvantages of this method are the use of drones unable to fly in certain climatic and weather conditions, short flight time, limited by the amount of batteries used, and the obligatory presence of specialized cameras on each drone [2,3].
Thus, the automation of agricultural machinery management processes, the reduction of a person's work in the process are promising current research areas [4].
In order to automate the processing of agricultural areas, it is necessary to develop a laboratory prototype of a software and hardware platform for universal use, which is designed to apply additional agricultural equipment. The platform prototype must provide continuous operation according to the installed equipment. The control over the platform prototype is carried out by software that is based on algorithms using machine vision. Additional positioning should be carried out by GPS and GLONASS. The prototype is designed for continuous operation in normal weather conditions, without direct human intervention. The prototype is capable of transmitting the operating parameters to the operator's controls. An operator, located at a considerable distance, should control the workflow on all agricultural machines in the area. .
An algorithm was developed to solve the problem within this study that allows you to determine the surrounding objects, such as trees, bushes and similar objects [5]. The algorithm for analyzing the surrounding objects allows us to define the trajectory of the prototype, implement the search algorithm and the possibility of its movement along rows of trees and bushes [6].
The analysis of images obtained from cameras installed on the prototype is carried out to select the trajectory of the software and hardware platform. The incline of the camera is set when mounting the camera on the body of agricultural machine. The camera should not be perpendicular to the level of the horizon or shoot vertically down. This position of the camera lens allows you to record objects directly in front of the attachments used on the equipment, and also contributes to a more accurate calculation of the distance to the working object. Figure 1 shows a general view of image decomposition using machine vision algorithms. x y x y x y (1) which corresponds to the received image of the camera. Equilateral trapezoid ABCD is a work area (WA). When a restrictive object (RO) enters it, the software calculates the change in the trajectory of the prototype of the software-hardware complex. The shape of the trapezoid arises from the optical effect of the straight linear perspective appearing in the camera lens when tilted from the horizon, physically the distance between the top points of the trapezium and the bottom points is the same, and represents a rectangular area of earth, but the distance to the far points is smaller when the rails converge to a point on the horizon. The angle at the base of the trapezoid is calculated automatically by the system, based on the angle of inclination of the camera relative to the horizontal plane. When calculating it is necessary to take into account that WA for the prototype is determined by the height position of the chamber and the overall width. Extreme points A and D are located beyond the overall dimensions of the prototype at a distance of frame processing error, which is usually about 10% of the width of the agricultural machine, which allows the prototype to pass along a trajectory calculated by WA points. GH section is the midpoint of the picture frame; it is required for the visual separation of work areas.
Moving on the landfill before the prototype, there were obstacles that mimic real objects, such as trees, bushes, people, animals, and other agricultural equipment. These objects are combined into a class of bounding objects (BO). They are described by a red rectangular area, according to the maximum coordinates.
Trapezoid of scope area (SA) is designated EBCF. When the extreme point of the RO gets into this area, the software starts the calculation of the trajectory of the reversal or deviation of the prototype, and also rebuilds the trajectory of movement bypassing RO.
The trapezoid AEFD limits the area of movement (AM). When any limiting object enters this area, the prototype stops waiting for the possibility of continued movement. Also, the possibility of reversing with a long-term limitation of forward movement is realized. Using some equipment, it must be turned off to avoid uncontrolled movement of trailed elements. Figure 2 shows a frame from the working program of the laboratory prototype. RO are square and numbered. The work object group is displayed next to RO number. The use of neural networks [7,8] for object recognition allows you to divide RO into a large number of groups or subgroups; this in turn allows you to create algorithms for the reaction of the prototype to external objects, based on the optimal choice of the solution. So, for example, when a group of live animals on a field of RO occurs, subgroups of animals optimally slow down or stop moving, give an audible sound and light signal, then wait until the RO leaves the prototype WA. The warning signals, if necessary, are repeated after a certain period of time. With the appearance of RO group plant, the algorithm changes the trajectory of the prototype in detour of RO [1].
Improved object selection algorithms were proposed according to the proposed algorithm. In Figure 2, the selection of an object is carried out by a base rectangle, which is parallel to the base of the frame; this type of selection allows you to know the coordinates of the extreme points of each RO. Figure 3 shows a simpler RO recognition algorithm. Trapezoidal algorithm allows recognizing the presence of an object in any area, and finding the extreme points of each RO.   It is necessary and sufficient to apply a rectangular filter with bases parallel to the frame to recognize RO group of a plant. This algorithm allows you to find the extreme points of each RO and, based on their position, to calculate the trajectory of the prototype of agricultural equipment. This allocation algorithm is suitable for solving the problem.
The developed algorithms were tested on a laboratory prototype of a universal platform for agriculture presented in Figure 5. Testing took place at the test site with objects simulating the agricultural fruit trees. The prototype successfully passed a complicated wave-like maze using the selected algorithm ( Figure 5). The developed prototype of a universal platform for agriculture using algorithms based on machine vision and neural networks allows identifying a large number of various objects, and their classification allows achieving an optimal algorithm for the movement of machines. The application of wide-angle cameras or cameras with a viewing angle of 360 degrees allows you to cover the entire area around agricultural equipment to determine the route. It is necessary to use thermal imagers for the most accurate work of algorithm. These imagers allow you to find not only visual objects, but also thermal traces, which will permit to find the majority of living objects on the route, as well as possible sources of fire in the fields. Taking into account that the prototype is designed to work in the environment of people, animals and various means of transportation, the use of cameras of different spectra gives a significant result when creating a trajectory of movement, and also allows to increase the level of safety of using such automatic equipment in real conditions. Addition of GPS and GLONASS system solves the problem of delivery of the prototype to the working area, as well as its positioning relative to the surrounding buildings and similar objects.
The works were performed as part of the grant 13127GU / 2018 dated 05.23.2018 "Development of a leveling system for construction equipment".