Algorithm for recognizing and measuring parameters of biological objects in agriculture based on deep learning convolutional neural networks

. As of today existing techniques and tools for measuring leaf area involve the detachment of leaves for further scanning and calculations to determine leaf area. The disadvantages of existing solutions for determining the area of the sheet surface are labor intensity, the duration of these studies, the relatively low accuracy of measurements. Due to these facts this study is an important work aimed to simplifying the process of analyzing biological parameters and other important characteristics of plants, as well as increasing the efficiency of this agrotechnical task. This work aims developing a set of software tools with trained neural networks to determine whether a photographed leaf belongs to the leaves of a soybean crop, assess the health of soybean plants and determine the surface area of a soybean leaf with geotagging.


Introduction
When conducting scientific research related to the methodology for determining the biological potential of plants, a prerequisite for substantiating the prospects of their use in breeding and agricultural practice is the registration of the vegetative state of plants, the determination of the parameters of the assimilation apparatus of the studied plants and the study of the leaf area [1,2].
For breeders and agronomists, the determination of leaf area is a task that must be solved in the course of many quantitative physiological studies of plants. Estimation of leaf area is needed as a morphometric indicator of plant growth. An important characteristic of plants is also the ratio between the area of their leaves and the total mass of the aboveground part, and the height [3].
Determination of the area of the leaf apparatus of plants is necessary for the work of agronomists, breeders, physiologists, biologists, ecologists. In particular, knowledge of the area of the leaf apparatus is needed in agronomy to determine. To date, existing techniques and tools for measuring leaf area imply leaf detachment for further scanning and calculations to determine leaf area [4].
Currently, the measurement of leaf area is carried out by weighing paper layouts, by the method of calculating the product of height by width, corrected by the leaf shape factor, or using mechanical planimeters. The applied methods for determining the area of the sheet surface are laborious, cumbersome, associated with the duration of these studies and often cannot provide sufficient measurement accuracy [5,6,7].
In connection with these facts, this study is an important work aimed at simplifying the process of analyzing the biological parameters of plants, as well as increasing the efficiency of this agrotechnical task.

Collecting data for a neural network for plant identification
The collection of data for training the convolutional neural network was carried out on the selection fields in the ISA -a branch of the Federal State Budgetary Scientific Institution FNATS VIM. The amount of training data collected was more than 12,000 images for the main phases of the growing season of soybean plants. This data set is needed to achieve the required recognition accuracy of the soybean leaf apparatus. Data collection was carried out with a Canon EOS 550D camera. Exposure settings, white balance and other settings were selected depending on the shooting conditions. In the future, the obtained materials are subject to processing in specialized software for use in training the algorithm for evaluating the parameters of biological objects and other important characteristics of plants.

Development of a software tool for preparing and marking data
ITS.ImageCroper software was developed to automate the data preparation process. The program allows you to simplify the work of selecting the target segment of the original image ( Figure 1). In comparison with existing software solutions, ITS.ImageCroper has a higher speed of selection of the desired segment. The software ITS.ImagePrepare was developed, which allows you to automatically create a training image mask from an image cleared from the background.
The program allows you to fully automate the processing of the collected training data for a segmented neural network, automatically create a training image mask from a background-cleared image and a topology of directories and training files for training a segmented neural network.

Development of a software tool for data markup
The software ITS.ImagePrepare was developed, which allows you to automatically create a training image mask from an image cleared from the background.
The program allows you to: -fully automate the processing of the collected training data for the neural network segment; automatically create a training image mask from the background cleared image.
-automatically create a topology of directories and training files to train a neural network segment The appearance of the working software, in the rendering mode of the generated masks, is presented below.

Preparing data for training a classification neural network of deep learning
A test dataset was prepared for training a classification neural network. The entire array of collected data (6000 images) was processed using the ITS.ImageCroper software to select the target segment of the original image. The result is data suitable for training a classification neural network.

Data markup for training a deep learning segmented neural network
A test dataset was prepared for training a segmented neural network. Using the web resource remove.bg, the background data of the array of target image segments was removed ( Figure  2). With the help of the ITS.ImagePrepare software, training masks were created from images cleared from the background, and a topology of directories and training files for training a segmented neural network was created.

Choice of neural network architecture
A deep learning convolutional neural network must provide functionality and meet the following specifications: The neural network must evaluate the object presented in the form of a photograph, classify it and give the result in the form of a probability as a percentage of its belonging to a certain biological object with an accuracy of at least 91% with the volume of training data of at least 6000 images for each class.
Pattern recognition speed should not exceed 0.5 sec. one image.
In case of classification of the presented image as a known biological object, it is necessary to segment the object with an accuracy of at least 85%, the speed of image segmentation should not exceed 0.8 sec. one image.
The VGG19 architecture is taken as a basis as the fastest of the satisfying in terms of accuracy. Also, the advantages of the chosen architecture can be attributed to the ability to use the already obtained weights on the classification part for training the segmentation part [8,9,10,11].

Classification (coding) neural network training
The architecture of a reconciliation neural network based on the VGG19 architecture for a classification neural network has been developed. The prepared data were divided into training and test data in the ratio of ¾ and ¼. The network showed the best results in 5 eras. The number of passes in an epoch is 100, the size of the training data packet is 32 images.
The following layers have been introduced into the standard CNN VGG19 architecture: [13,14] flatten -an alignment layer of the output data parameters in the vgg19 network for their processing in subsequent layers. fc1 -fully connected layer with relu activation function, experience layer. Introducing nonlinearity in network decision making (to reduce the likelihood of memorizing training data). There is a possibility that the network can remember all the training images and show acceptable results based on them, but be very wrong on the new data.
fc2 -Fully connected layer with sigmoid activation function. This layer is used to return the result about the belonging of the input image to a certain class.
During training, the CNN characteristics changed in accordance with the following graphs ( Figure 3) on the training data, and after the end of each epoch, the CNN characteristics were checked against the test data ( Figure 4).
As a result, we can conclude that the trained CNN shows the required results on new data that the network was not trained with.

Training a segmentation (decoding) neural network
The architecture of a reconciliation neural network based on the VGG19 architecture for a segmentation neural network has been developed. To train this network, data from the previously trained classification neural network are also used.
The principle of semantic (per-pixel) segmentation was used, with pixel-by-pixel image restoration at the deconvolution stage (restoration of the original image). The prepared data were divided into training and test data in the ratio of ¾ and ¼.
The network showed the best results in the 400 era. The number of passes in an epoch is 100, the size of the training data packet is 30 images, 30 masks. Each image is linked to a corresponding mask.
Into the standard VGG19 neural network were introduced the following layers: Pool1_11 -fully connected layer with relu activation function, with experience function. Introducing nonlinearity in network decision making (to reduce the likelihood of memorizing training data). There is a possibility that the network can remember all the training images and show valid results based on them, but be very wrong on the new data.
During the training process, the CNN characteristics changed in accordance with the following graphs ( Figure 5) on the training data, and after the end of each epoch, the CNN characteristics were checked against the test data ( Figure 6).  As a result, we can conclude that the trained CNN shows the required results on new data that the network was not trained with.

Conclusion
Within the framework of this work -collected photos of soybean leaves for CNN [15] training; The volume of training data was 12000 images for the main phases of the growing season of soybean plants. Of these, 6,000 are photographs of healthy leaves and 6,000 of sick leaves. Data collection was carried out in different weather conditions and at different times of the day to achieve higher recognition rates; ITS.ImageCroper software tool was developed to automate the data preparation process. It made it possible to speed up the processing of the collected training data and simplified the work on the selection of the target segment of the original image; he software tool ITS.ImagePrepare was developed, which allows you to automatically create a training image mask for an image cleared from the background. It made it possible to fully automate the processing of the collected training data for a segmentation neural network, automatically create a training image mask from an image cleared from the background, and automatically create a topology of directories and training files; performed the preparation of a test dataset for training a classification neural network; -performed the markup of the test dataset for training the segmented neural network; the VGG19 architecture is taken as a basis as the fastest of those satisfying in accuracy. Also, the advantages of the chosen architecture include the ability to use the already obtained weights on the classification part for training the segmentation part; -developed the architecture of a reconciliation neural network based on the VGG19 architecture for a classification neural network; -developed the architecture of a reconciliation neural network based on the VGG19 architecture for a segmentation neural network.
As a result of the work carried out, a complex of software tools with trained neural networks was developed to determine whether a photographed leaf belongs to the leaves of a soybean crop, to assess the health of soybean plants and to determine the surface area of a soybean leaf with georeferenced.