Cataloging system for Russian satellites remote sensing data: main characteristics and operating experience

. A local archive and a system for cataloging satellite data from Russian spacecraft that are received by the FRC KSC SB RAS through the services of the Roscosmos Geoportal have been developed. An automated system for remote sensing data preprocessing has been created, which significantly simplifies access to information, implements search services and web visualization of images registered in the catalog. Practical experience in operating the system has revealed a number of shortcomings of the existing Roscosmos data distribution system. Presented data on the execution of the Roscosmos satellite survey order in 2019 they show that the capabilities of the Russian remote sensing system are very limited, and a significant part of the planned research work has not been implemented due to the lack of ordered satellite data.


Introduction
The main source of Earth remote sensing (ERS) data from Russian spacecraft is the Roscosmos Geoportal, which is operated by the Scientific Center for Earth Operational Monitoring (NTs OMZ) of Russian Space Systems JSC. It is a web resource that combines the means of searching and viewing, ordering of new and archived remote sensing data from Russian spacecrafts. Data receiving is possible through the web interface of the geoportal or via the ftp protocol.
Several institutes and divisions of the Federal Research Center of the Krasnoyarsk Science Center Siberian Branch of Russian Academy of Sciences (FRC KSC SB RAS) are interested in using data from Russian spacecrafts. Large volumes of remote sensing data that need to be obtained through the Roscosmos Geoportal, as well as an analysis of its capabilities, which revealed a number of fundamental limitations, led to the need to create a local archive of satellite information, access to which is provided to all interested divisions of the organization. A technology was developed for automated processing of data coming from the ftp server of Roscosmos in the form of a specialized service running on the server. The proposed approach made it possible to simultaneously solve a number of discovered technical problems associated with the received ERS data.
The development of Earth remote sensing systems is currently due to a combination of factors such as an increase in the number of spacecraft and an improvement in their technical characteristics, an increase in the availability of satellite information associated with the development of the Internet: an increase in speed and a decrease in the cost of access, the development of web services and geospatial transmission standards. data [1]. New technologies for automating the processes of receiving and processing Earth remote sensing data, creating super-large archives in specialized data centers have made it possible to organize fundamentally new schemes for working with satellite data, to create a new generation of systems for operational monitoring of the natural environment. Geographic information web technologies, interactive online cartographic systems with direct access to satellite information, provided with the ability to customize various parameters of visualization of geospatial data and build complex analytical queries, have become their key elements.
Technologies of geoinformation web systems and services are dynamically developing at present [2][3][4]. More than a hundred Russian geoportals, dozens of geoservices, web GIS, cartographic services and data visualizers can be found on the Internet. Including specialized solutions for working with satellite data. Along with the already traditional tasks of simple visualization of terrain maps and mosaics of satellite images, modern systems provide the user with the possibility of flexible adjustment of various parameters of cartographic visualization, services for processing spatial data [5].

Subject of research
The developed system for cataloging satellite data coming from Russian spacecraft is currently focused on data from the Resurs-P and Canopus-V satellites. The data are centrally loaded for the needs of the departments included in the Federal Research Center of the KSC SB RAS, according to a common request for all departments. For the convenience of the end users of the catalog, the received data is processed and brought to a single format. Now these data are automatically downloaded from the Roscosmos geoportal (https://www.gptl.ru) and are pre-processed in a semi-automatic mode.
Access to images from the Roscosmos geoportal is organized through the ftp server. Data on the geoportal server is stored for a limited time (usually no more than a month), therefore it is necessary to periodically upload them to your storage. Each image contains shapefiles with its borders or borders of each channel separately, xml with the image metadata, jpeg with an overview image and one or more tiff files with raster data with one or more bands. XML metadata format and file name format for each type of spacecraft is different. XML formats also differ for different levels of image processing. The data is prepared and placed on the portal by the operators of the supplier manually, and as practical experience has shown, this leads to a lot of errors, different variants of the data presentation structure.
The disadvantages include the fact that the file names contain the processing time of the operator, although it would be more convenient to store the shooting time in them. Folder names can contain Russian characters, which introduces a number of problems when working with them in Unix systems when converting the encoding.

Software technologies
To search and navigate the satellite data catalog, ftp access was organized within the network of the Federal Research Center of the KSC SB RAS and a web application for the satellite images catalog was developed using the server programming language PHP, PostgreSQL DBMS, SQLite3 and the client web interface in the TypeScript programming language using the Angular framework 5 [2,3].
The developed web application uses a set of automatically generated multiscale "natural color" overview images, as well as overview images with the NDVI index. Specially prepared multiscale overviews provide a high degree of interactivity, very responsive web user interface. An example of web user interface is presented on Fig. 1.   Fig. 1. Web interface of the Cataloging system for Russian satellites remote sensing data.
For multispectral satellite images Canopus-V, when creating overview images, a combination of 3-2-1 bands is used, for multichannel images, the following sets of channels are used: 3-2-1 natural colors; 4 and 3 for calculating the NDVI index overview images.
For multispectral satellite images of the Geoton device of the Resurs-P satellite, when creating overview images, a combination of 4-3-1 channels for multichannel images and one channel for a panchromatic spectrum is used.
When forming color images, LUT (Look Up Table) is used, this is a kind of "correction table" for making changes to each of the three channels. Previously, to improve the quality of the image, spectral transformation was used, which is based on working with a spectral diagram showing the relationship between the number of image pixels and the values of spectral brightness. Spectral transformations change such a parameter as contrast. To increase the contrast, a linear stretching of the histogram was used, which means that all brightness values are assigned new values in order to cover the entire possible range from 0 to 255. Applying a common LUT to all images allows you to change the brightness value of image pixels using linear interpolation between, allows you to make images brighter and comparable in contrast to each other. At the same time, the tables allow converting the pixel value of images to 8 bits from 16 bits.
As a result of the work for a quick and convenient search in the satellite data catalog, to minimize the load on the server software and hardware, a set of server applications was prepared for preliminary processing of satellite data, including the following processing steps: 1. Converting the original raster data to GeoTIFF format with converting the original projection to the azimuthal Lambert equal-area projection which is the standard on the regional geoportal.
2. Creation of a basic color image for different display scales, consisting of several spectral channels. Such images will be used for detailed viewing of a satellite image while maintaining the original resolution.
3. Creation of a raster image in PNG format for displaying quicklook in a web application for different display scales, but with a lower resolution (up to 1 km per point). Unlike the basic color image, quicklook in a web application can be displayed in groups depending on the user's choice, while the basic color image will only be active for one selected snapshot.
4. Form of raster multichannel images in GeoTIFF format with NDVI data. 5. Conversion from 16-bit format to 8-bit, which is more suitable for displaying satellite images in a web application and requires significantly less resources for the data storage.
It should be noted that two data formats are used: TIF and PNG. The fast data view is implemented using client-side Javascript using PNG image files. In turn, collections of multichannel overview TIF files of different sizes are used by the web application on the server side. The raster data required for the client part of the software is dynamically created by the server components of the developed system.
An example of the developed web interface of the Cataloging system for Russian satellites remote sensing data is shown in Fig. 1. It also includes the set of links for selected data ftp download. Additional information such as a standardized metadata file and a satellite image outline in the popular GIS file format (SHP-file) can also be downloaded.

Results and discussion
In 2019, the Krasnoyarsk Scientific Center placed an order with Roskosmos for images from April to October 2019 inclusively from the Canopus spacecraft of the PSS (panchromatic imaging system) and MSS (multispectral imaging system) equipment with a resolution of 2.7 m and 12 m, respectively [6]. And also, an order from the Resurs-P apparatus of the Sangur-1U equipment with panchromatic and spectral imaging with a resolution of 1 and 3 m, respectively. Data were requested with processing levels 1A and 2A with cloud cover no more than 40%.
Areas of interest are shown in Figure 2. Areas of interest include areas in the Krasnoyarsk Territory, the Republic of Sakha, Novosibirsk, Kemerovo, Irkutsk Regions, the Republic of Buryatia and Tyva. In total, within the framework of the submitted order, 31 sites with a total area of about 690 thousand square kilometers were formed.
Data for 2019 has been uneven and the latest data for 2019 was obtained in March 2020. A total of 1103 images were obtained from the Canopus apparatus and 110 images from the Resurs-P apparatus. The total number did not include images with damaged or missing raster files in the amount of 7 pieces.
The disadvantages include the small size of some images, which are located in the corners of the order sites, mainly for images from Canopus vehicles. The size of such images may not exceed 10% of the total area of the site and there is little practical purpose in them. For small areas during the year from the Resurs-P apparatus, on average, one or two multispectral images were collected and usually these images do not completely cover the area. For large areas, the maximum number of images reached 6 and they also do not completely cover the territory. It is impossible to observe the dynamics during the year according to the data from the Resurs-P apparatus. With the data of the Canopus apparatus, the situation is slightly better, but not in all areas. To provide all interested users within the FRC KSC SB RAS, a centralized archive of all received images from the Roscosmos geoportal was created. A complex of programs was developed for semi-automatic processing of the obtained data. Additional semi-automatic processing is associated with bringing the catalog of images to a unified format with the solution of all the problems described above. As a result, the image file names contain information about the time the image was taken, the type (panchromatic or multispectral survey) and the identifier of the satellite equipment. The hierarchy of image catalogs on the first level contains the shooting month, on the second level the images themselves are located, where the files of each image are in a separate folder. Additionally, for the convenience of users, there is a hierarchy of order sites with symbolic links to folders with images. The developed system processes the data automatically, but in situations where it is impossible to do it, it sends a message to the system administrator for making a decision and manual correction.
To search and navigate the satellite data catalog, ftp access was organized within the network of the Federal Research Center of the KSC SB RAS and a web application for the satellite images catalog was developed using the server programming language PHP, PostgreSQL DBMS, SQLite3 and the client web interface in the TypeScript programming language using the Angular framework 5 [7,8].

Conclusions
The creation of an effective software and technological toolkit for the tasks of regional satellite monitoring based on technologies of geoinformation web systems has significant prospects. The developed approach can become the basis for solving urgent applied