Algorithms and programs based on neural networks and local binary patterns approaches for monitoring plankton populations in sea systems

. The work is devoted to the method of multispectral space images analyzing of aquatic coastal systems for identifying phytoplankton populations of complicated structures: determining their boundaries, distributing color gradations and, based on this, determining the distribution of phytoplankton concentrations within patches and the location of the <mass= center. A combination of local binary patterns (LBP) and neural networks methods is considered. Due to these characteristics it is possible, basing on a series of processed images of the same water area for different time points (dates), to determine the changing rate in the spots boundaries and their concentrations, the shift of the mass center which are influenced by the aquatic environment movement and the processes of phytoplankton growth and death. The results of the work allow us to determine the Azov Sea state. Experimental data of the program are given in confirmation part.


Introduction
Nowadays the important role of the ocean as a huge mineral and biological raw materials repository is constantly increasing.Problems of controlling various pollution forms appear.It becomes vital to predict the consequences of scientific and industrial human activities.And this is exactly what is meant to be a purposewhy to make an operational monitoring of the planet state.Mathematical modeling methods have long been considered one of the most effective tools.They are applied to analyze and predict natural processes.However, to make these models fully operating, more and more new input parameters are needed, and today it will not be based on just field data, but also include the use of Earth remote sensing (ERS) datawhich makes the further progress.The creation of mathematical models based on ERS will make it possible to obtain forecasts of the aquatic ecosystems development in real time, and in some cases -to increase the forecasts reliability and accuracy.
Satellite monitoring methods have a wide range of possibilities for studying marine areas, which include a wide coverage; efficiency; the ability to work in any hard-to-reach sea and ocean areas; obtaining data of various spatial and temporal resolutions in various spectral regions of electromagnetic waves; a wide range of recorded parameters of the aquatic environment; and high reliability of the obtained data [1].
Scientific works similar to the topic of this article show that the problem of using satellite data to monitor and study the dynamics and forecast of algal bloom development can have different formulations.On one side, there are created information products which are aimed directly at the algae detection and identification [1][2][3][4][5][6].On the other side, there are being carried out complex studies which are aimed at understanding natural processes [7][8][9][10][11].As a result, all of them pursue one goal -to study the conditions and features of the phytoplankton population development, and that is an issue today.

Methods
The geographical position and relatively small size of shallow water bodies, like the Azov Sea, determine the high spatial and temporal variability of the main abiotic factors within the marine ecosystem, especially salinity, which is defined by a significant, compared with the total sea, river flow, water exchange with the Black Sea and fluctuations in the total moisture content of the basin.The currently available data on the plankton production characteristics require specification due to changes in the species composition of zooplankton as a result of fluctuations in climatic and oceanographic factors in the study area [7].
Due to trophic interactions in the aquatic ecosystem of various individuals, a kind of ecological pyramid is formed, the basis of which is phytoplankton.
In moderate concentrations, many types of plankton are necessary for the sustainable development of all living things (the entire trophic pyramid).Therefore wavy ecosystem degradation is caused by phytoplankton deficiency, while its sharp growth (owing to significant increase of the incoming biogenic substances -nitrogen, phosphorus, silicon compounds) leads to the reservoir "blooms", with the formation of oxygen deficiency zones -hypoxia zones.For various water areas, including the Azov Sea, the problem of "blooming" is very acute [3].
Today a huge number of both Russian and foreign satellites have been launched and functioned.This makes it possible to cover most of the planet and transfer remote sensing data almost in real time.Figure 1 shows color-synthesized multispectral images from the WorldView [12], Kanopus-V [13], and Sentinel-2 L2A [14] satellites obtained in July-August 2020 over the eastern part of the Azov Sea.The image illustrates how the blue-green plankton population appear and distribute under the influence of circular currents, forming stable S-structures of currents.As image processing algorithms it is considered a combination of methods: Local Binary Patterns (LBP) and a neural network approach.These characteristics allow us, having a series of processed images of the same water area for different time points (dates), to determine the rate of change in the spots boundaries and their concentrations, the shift of the mass center due to the influence of the aquatic environment movement and processes of phytoplankton growth and death.It continues the study described in [3].

Results
Let us define the contour as a closed boundary of plankton spots.Then, the neural network will be taught to determine the boundaries of the contours and the concentration of plankton spots based on remote sensing data.To improve the reliability of determining the boundaries, there should be used the image pre-processing applying the LBP method.Figure 2 displays the complex principle of "LBP-neural_network" operation.
The "LBP-neural_network" complex includes control and implementation modules.
The control modules comprise: -a control block; a block for input of initial coordinates and studied image; a block for input to compare the studied images; -a block for outputting to receive information and assess the dynamics of plankton spots; -a block for outputting to receive information and estimating the dynamics of plankton spots based on a series of satellite images for different time points (dates).Implementation modules include hidden blocks (available only to the administrator): a filtration unit; a block for determining the contours by the LBP method of the phytoplankton distribution on the water surface; -a block for approbation by a neural network of the phytoplankton distribution presence on the water surface; a block for calculating the area of phytoplankton concentrations; a block for calculating the normalized masses of plankton spots and their mass centers; -a block for outputting area values of plankton spots and their mass centers; -a block for outputting the contour of phytoplankton concentrations.

Implementation module, without calculation block
As already stated above [3], edge recognition plays an important role in image analysis.A spatially extended gap and a drop or an abrupt change in brightness values are important parameters that allow us to determine the contour.The LBP-neural_network complex is implemented using a neural network with Local Binary Patterns (LBP) support.The software implementation is based on deep learning and three-layer convolutional networks (teaching with a teacher).At first, it is estimated the difference between the brightness of the current and surrounding pixels as an integer value (from 0 to 255), which corresponds to a rectangular area of the image.It corresponds to formula (1): where: (X,Y)coordinates of the upper left pixel within the area under consideration; (W,H)=( 3w,3h)area size for calculation; s_(i,j)sum of pixels over the rectangle; p(x,y)image pixel value; 1, iĀ ÿ ≥ Ā 0, e ÿ.In this case s_(i,j) is calculated by formula (2): After segmentation, the neural network unites the typically different instances of the object into a whole one.The Panoptic Quality (PQ) metric calculates, basing on the matches of the matched segments, according to the formula (3): where: ppredicted segments; gground truth segment (object environment); SPis a threshold value equal to 0.5; MPmatched pairs; UPunmatched predicted; UTunmatched ground truth segments.Figures 3, 4 and 5 show images: manual contour selection, using LBP and using both a neural network and LBP.At that point it should be understood if it is neccesasary to make an additional customization with LBP.To answer this question it needs to give a comparison with the neural network selection when the LBP module is turned off. Figure 6 illustrates this.
After the received results in practice, it can be concluded that the «LBP-neural_network method» implements the quality of work.Such combination use is justified, as it allows us to weed out most of the information that is unnecessary for work.
The next stage is the possibility of full-fledged work with the revealed information.The further goal is to determine the concentrations of the phytoplankton distribution inside the spots or the correspondence of the identified objects to the color scale.

Implementation module, calculation block
One of the methods to control the change dynamics in plankton spots is to determine the area of phytoplankton concentrations, in accordance with the color and brightness of individual image areas (pixels) and to define their concentration points, i.e. mass centers.To calculate the area it is applied the well-known formula of rectangles, while to calculate the concentration point it is used the formula taken from the course of classical mechanics.More details are described in [3].The result is displayed in Figure 7.The basic objective of determining the plankton population spots from remote sensing images is to obtain a source of input data for the possibility of predictive modeling and improve the accuracy and reliability of predictive modeling.In that connection, the next step is to prepare the obtained information for assessing the dynamics of plankton spots based on a series of satellite images for different time points (dates).That requires determining the rate of change in the spots boundaries and their concentrations, i.e. identifying the area of each recognized object and their mass centers.The result will show the shift due to the influence of the aquatic environment movement and the processes of phytoplankton growth and death.

Block for outputting the received information to estimate the dynamics of plankton spots based on a series of satellite images for different time points (dates)
Succeeded the previous steps, it becomes possible to analyze the data obtained.To do this, the program should provide the possibility of combining already obtained information.For the convenience displaying, the combined contours are placed on the images of the studied area, then it is manually marked which colors correspond to which days. Figure 8 shows the results obtained.In this way, having such a functional, the analysis of the information obtained allows us to interpret the results of processing multispectral satellite images obtained during monitoring of the aquatic environment.

First section
The results obtained allow us to draw conclusions about the state of the Azov Sea.Various processes taking place today can be observed with the help of these methods.The article describes the relevance of the use and the new method of filtering and assimilation of remote sensing data.The article considers the processing method for satellite images of aquatic coastal systems within multispectral imaging to identify phytoplankton populations of a patchy structure: determine their boundaries, distribute color gradations and, based on this, define the distribution of phytoplankton concentrations within patches and the location of the mass center.This work can be introduced to the use of a multi-core processor, as well as on supercomputer systems, including hybrid architecture, which will make it possible to perform predictive calculations in a more efficient mode.
The study was supported by a grant from the Russian Science Foundation (project № 21-71-20050).

Fig. 7 .
Fig. 7. Determination of the phytoplankton concentration area and their mass centers using the program.