Development of the “Ecological Diptychs” application for detecting the boundaries of forest areas on satellite images and aerial photographs

. Environmental monitoring using satellite imagery and aerial photography is an important tool for assessing the state of the environment at the global and regional level. The article discusses the process of developing an application for environmental monitoring based on the analysis of high-altitude photographs. The authors describe in detail the choice of data processing methods and technologies underlying the developed application. An analysis of algorithms for detecting and analyzing environmental changes using high-precision satellite data was carried out. The application provides users with access to real-time visualization of environmental parameters, facilitating effective monitoring and management of natural resources. The results demonstrate the potential of using modern technologies in the field of ecology and highlight the importance of developing relevant tools for sustainable environmental management.


Introduction
The ubiquity of civilian drones equipped with high-definition cameras and the ability to fly along a pre-programmed trajectory, as well as widespread access to up-to-date satellite imagery of the Earth's surface, provide a unique opportunity for more effective and comprehensive environmental monitoring.Many studies have been devoted to these problems.[1][2] As part of the growing need for environmental surveillance systems, developing an application for analyzing satellite images becomes an important step in providing accurate and up-to-date information.Developing said application can be useful for several reasons:  Forest Management: Accurate delineation of forest areas helps forest departments and authorities to effectively manage forest resources, control logging and prevent illegal logging.
 Environmental protection: Monitoring the state of forests using such an application allows you to quickly identify forest fires, diseases and pests, which contributes to more effective control and protection of forest ecosystems. Agriculture: Knowing the boundaries of forest areas helps agriculture and rural communities avoid illegal logging and take forest areas into account in land use planning. Advanced technologies: Modern technologies of computer vision and machine learning make it possible to automate the process of detecting forest boundaries, which significantly reduces labor costs and increases accuracy. Research and Monitoring: Scientists and environmentalists can use such data to conduct research on changes in forest area over time and their impact on climate and the environment. Support solving environmental problems: Effective tracking of forest boundaries can help solve a variety of environmental problems, such as ensuring biodiversity, conserving water resources and combating climate change.All these aspects make the development of an application for detecting forest boundaries on satellite and aerial photographs a relevant and important task for various fields, including ecology, resource management, agriculture, scientific research and much more.

Materials and methods
When working on the application, we had to take into account a number of factors, which was reflected in the methods we used.In particular, since the main source of spatial data used in the application is satellite imagery and aerial photography, there are some features that need to be taken into account.These include very limited image resolution and the possibility of distortion due to weather conditions or other factors.
To obtain information about changes on maps, data in raster form is limited, so it needs to be converted to vector format.This approach makes it possible to easily and efficiently store and process information about geographic objects.And also use it for a variety of purposes: to analyze or predict environmental changes.
As a result, the following methods were used in our work:  To detect the boundaries of forest areas and their detection, an area of IT is used, which is called computer vision -this is a technology for creating software capable of detecting, tracking and classifying visual objects [8].Over the past few years, most problems in this area have been solved by using neural networks [7].Linking data on satellite images to a map of the area. Machine learning is used to analyze satellite map images and find the boundaries of forest areas. Image analysis is used to determine forest contours and satellite images. Image segmentation is used to analyze satellite images and to highlight the contours of forest areas.

Detection of forest boundaries
All existing edge detection algorithms can be divided into two groups: methods based on searching for maxima, and methods based on searching for zeros.[6] Peak-based methods typically find the gradient of the image and then look for local maxima of the "edge strength", assuming that the direction of the edge is usually perpendicular to the gradient vector.In turn, "methods based on finding zeros look for intersections of the x-axis of the second derivative expression, which are usually the zeros of the Laplacian or the zeros of a nonlinear differential equation."[9] In technical vision, the Roberts cross operator is one of the early edge detection algorithms [10], which calculates the sum of squared differences between diagonally adjacent pixels in an image.The Pruitt operator [5,7,11], like the Roberts operator, operates on the image area, but its scale is determined by the value and is described by slightly different masks.The Sobel operator works similarly, implementing the following kernels: In which * denotes the two-dimensional convolution operation.At each point in the image, the approximate value of the gradient can be calculated by using the obtained approximate values of the derivatives:  , and therefore the direction of the gradient For the software execution of the method for determining boundaries and contours, the OpenCV library of open-source computer vision algorithms was chosen -this is a port of the OpenCV library for JavaScript.It provides a wide range of image processing functions, including edge detection.
A fragment of application code that converts an image to grayscale and uses OpenCV.js to process images using the Sobel operator in question: In this code, the line cv.cvtColor(inputMat,inputMat, cv.COLOR_RGBA2GRAY);: converts the input image, represented by the inputMat matrix, from RGBA color format to grayscale.cv.COLOR_RGBA2GRAY is a constant indicating the conversion type.
Then the line cv.Sobel(inputMat, outputMat, -1, 1, 1, 3, 1, 0, cv.BORDER_DEFAULT); applies a Sobel filter to an image.The parameters of the cv.Sobel function specify the following: inputMat -an input matrix containing a grayscale image; outputMat -output matrix in which the processing result will be saved; -1 -image depth (-1 means the depth will be taken from inputMat); 1 and 1 are the order of derivatives with respect to x and y, respectively.These parameters determine how the filter will calculate gradients in the image; 3 -the size of the Sobel filter kernel (the size of the neighborhood for calculating the gradient); 1 and 0 -coefficients that determine the scaling of gradients (usually equal to 1 and 0); cv.BORDER_DEFAULT -type of processing of border pixels.
When using this method in software implementation, we get a contour with a large number of boundary points.Contours represent the boundaries of a shape with equal intensity.It doesn't store all the coordinates of the shape's border, but it does store them.This is specified by the contour approximation method.
To simplify the creation of a vector object, most of the points of the selected contour can be eliminated.For example, to find the contour of a straight line, we only need two endpoints of a given line.In the selected open source library OpenCV, this is done by the cv.CHAIN_APPROX_SIMPLE method.It removes all unnecessary points and compresses the outline, thereby saving memory.
An example of using this method: This code uses the OpenCV.jslibrary (a JavaScript port of the OpenCV library) to process the image.The first line of the above code finds the outlines in the image.In particular, cv.RETR_EXTERNAL specifies the contour extraction mode, which means extracting only external contours (that is, contours that bound objects), and cv.CHAIN_APPROX_SIMPLE, as mentioned above, specifies the contour approximation method.This method approximates contours using the simplest approximation, removing all points that do not contribute to the shape of the contour.
Line const simplifiedContours = contours.map(contour=> contour.approxPolyDP(0.01,true)); is designed to approximate a contour using the Douglas-Pecker approximation (implemented by the approxPolyDP method).0.01 is an approximation parameter that determines the accuracy of the approximation.The lower the value, the closer the approximated contour will be to the original.true indicates that the paths are closed (or the first and last points of the path are connected).
Then a new object is created which is a copy of the original image and for each simplified path in the image green paths with a thickness of 2 pixels are drawn.This way you will have an original image with red outlines drawn around the objects, and these outlines are approximated to simplify their shape.
The considered methods were used in the software implementation of automatic detection of vegetation on satellite images using JavaScript.The OpenCV plug-in library supports a wide range of programming languages, including the one we selected.

Linking data on satellite images to a map of the area
The contours obtained at the first stage must be correlated with the contours obtained earlier and stored in the database.This is necessary to analyze the dynamics of changes in the contours of biological zones for their subsequent analysis.[12] This process corresponds to determining the geographic coordinates of points in the image.
The images are initially collected to collect Ground Control Points (GCPs) -points on the Earth's surface that are clearly identified in images obtained using satellite imagery or other survey methods.Because control points are points with known geographic coordinates, then a comparison of the specified points on the images allows us to correlate the resulting contours (obviously related to the existing images) with known geographic coordinates, and therefore, with the contours previously saved in the database.GCPs can be buildings, road intersections, recognizable natural objects, etc.The process that involves locating GCPs in images and associating them with known geographic coordinates is called geocoding.
After receiving GCPs, the control points in the new and vector images saved in the database are compared.For these purposes, the most effective is the use of the SURF (Speeded Up Robust Features) algorithm -this is a computer vision algorithm designed to detect and describe key points in images.It was developed in 2006 by Herbert Bay and Tin Tuite.SURF is an improved version of the SIFT (Scale-Invariant Feature Transform) algorithm and aims to solve some of its limitations while providing high performance.
The SURF method solves two problems: searching for special (scale-invariant) image points and creating their descriptors.
The search for scale-invariant points is carried out by using a Gaussian-based filter and searching for local extrema in scale space.For this purpose, unlike the Hessian-Laplace detector of Mikolajczyk and Schmid, SURF uses a Hessian matrix.Its determinant (Hessian) reaches an extremum at the points of maximum change in the brightness gradient.For a two-dimensional function, its determinant is defined as follows: x y H f x y ff x y y Where H is the Hessian matrix; f(x,y) is a function of changing the brightness gradient.The Hessian remains unchanged under rotation, but is not invariant under changes in scale.In this regard, the SURF method uses filters of various scales to extract Hessians.
For each key point, the direction of the greatest change in brightness (gradient) and the scale are determined, which is taken from the scale factor of the Hessian matrix.The gradient at a point is calculated using Haar filters.Once keypoints are identified, SURF creates their descriptors, represented by a set of 64 (or 128) numbers for each keypoint.These numbers reflect the direction of the gradient around the key point.
Since the key point is a local maximum of the Hessian, this guarantees the presence of regions with different gradients in the vicinity of the point.This provides a variety of descriptors for different key points.
SURF has high performance due to the use of integral images and other optimizations.
To use the SURF algorithm you will also need OpenCV: Example code snippet using the SURF algorithm: Here, the first line creates a SURF object, and 400 is a threshold value for highlighting key points that can be configured.The second and third lines search for key points.The last two lines are intended to calculate their descriptors.

Image Analysis
The next stage is the assignment of objects limited by contours and tied to the area to certain categories (forest, field, etc.).[13] Machine learning methods are used for this.One of the most popular methods chosen for implementation and used to solve this problem is an image processing method based on deep neural networks, called semantic segmentation.
Semantic segmentation allows you to classify each pixel in an image into a specific class.In our environmental monitoring problem, classes may include land, water, or other objects.To train a neural network, a training set is compiled that contains images that are manually labeled so that each pixel is labeled with the correct class.
Once trained, the neural network can be used to classify pixels in new images and recognize the contours of forested areas.This method can be used to detect areas where deforestation is occurring.In this case, new areas will be different from the rest of the forest and can be more easily recognized using this method.
To recognize the contours of forest areas on satellite maps, it is possible to use convolutional neural networks (CNN), specially designed for image processing.
Each of the CNN layers processes the input data using convolutional operations, pooling, and nonlinear activation functions.The goal of a CNN is to learn features in images to effectively identify objects in images.
To implement CNN in JavaScript, you can use deep learning frameworks and libraries such as TensorFlow.js,Keras.js or Brain.js.For example, to create and train a CNN on TensorFlow.js, you first need to define a model.The model consists of layers, each of which processes input data and passes the results to the next layer.The most commonly used layers for image processing are convolutional layers, pooling layers, and fully connected layers.
It should be noted that special attention should be paid to areas of the images in which changes in contours are noted.This may indicate either a malicious change in the biosystem of a given area (deforestation, desertification, etc.), or absolutely opposite processes.In addition, another problem is the seasonal change in the appearance of the terrain in the images.The neural network is designed to help determine the factors that led to changes in boundaries.
It is important to remember that the process of building and training a neural network is an iterative process that requires repeated attempts and adjustments to achieve the best quality recognition of forest contours on satellite maps.At the moment, this stage is under implementation and has not yet been completed.

Conclusion
To conclude this article on developing an application for environmental monitoring of forest boundaries using satellite and aerial imagery, several key conclusions can be drawn. The development of the application has shown that the use of advanced image processing and machine learning methods can significantly improve the accuracy of determining the boundaries of forest areas (and other biological systems) in satellite images and aerial photography data.This opens up new opportunities for more effective monitoring of forest conditions. The developed application demonstrates the benefits of automation in the field of environmental monitoring.The ability to analyze the boundaries of biological massifs over large areas in a short period of time, including in dynamics, can significantly simplify the task and increase the efficiency of decision-making. The developed application opens up new prospects for scientific research in the field of ecology and forestry.The ability to monitor changes in forest areas based on satellite data can be useful in identifying trends and developing effective strategies for sustainable forest management. The need for further improvement and upgrade.Despite the progress achieved, it is worth noting that the intended functionality has not yet been fully implemented.And the development itself in the field of environmental monitoring requires constant improvement and upgrade.Technologies and methods are rapidly evolving, and it is important to keep the application up-to-date to ensure highly accurate and reliable results.
Overall, the developed application represents an important step towards more efficient and accurate monitoring of forest and other biological resources, which is of great importance for the conservation of ecosystems and sustainable management of natural resources.