Complex for reception and real time processing of remote sensing data

. The basic problems and trends in the development of modern systems for the reception, storage and real-time processing of satellite data are considered. Abrupt increase in the capability of satellite systems, significant increase in the amount of satellite information and its availability, the development of data processing and presentation technologies, and the use of web technologies are discussed. Data sources of modern remote sensing systems of the Earth and the features of their practical use are considered. It is concluded that the most effective way to obtain real-time information from meteorological satellites are satellite stations that receive data in the X-band at a frequency of 8 GHz. The performance characteristics and capabilities of the equipment of the new satellite receiving complex at Krasnoyarsk Science Center are given. Use of up-to-date computer equipment (high-performance servers and storage systems, local area network with a bandwidth of 10 Gbit/s) and logical separation into the stages of data conversion (data reception, primary and thematic processing) provide the construction of a modern scalable data-processing system for remote sensing data. The paper presents the results of the work on creation of specialized software for information and analytical systems for real-time satellite monitoring.


Introduction
The geographical features of the Krasnoyarsk Region (extensive territory with a diverse relief and climate, high proportion of poorly developed and inaccessible zones, large reserves of natural resources and other factors) objectively lead to the need for the use of Land remote sensing systems (RSS) and various thematic information products based on them. Integration of the RSS data with real processes of providing vital functions to public authorities and the population assumed major importance of a strategic factor for further accelerating the social and economic development of the Krasnoyarsk Region.
One of the stages in providing the effective use of space activities in the Krasnoyarsk Region was purchase and put into operation of a new satellite receiving complex FRC KSC SB RAS on the basis of UniScan-36 station by ScanEx Co. in 2017. This complex provides the receiving RSS data from all the relevant low-orbiting meteorological satellites of medium spatial resolution (TERRA / AQUA, Suomi NPP / NOAA-20, FengYun-3).
A large number of information systems based on earth remote sensing data have been developed in the world [1][2][3][4][5]. We can assume that the general arrangement of the RSS data processing has already been sufficiently established; basic units include receiving and archiving, as well as primary and thematic processing [6]. At the first stages of monitoring systems development all the stages of processing remote sensing data had to be implemented in the construction of a specific system and a special infrastructure had to be completely created. But today the use of specialized centers for data storage, processing and analysis is more efficient and appropriate from different points of view [7]. It should be noted that significant changes are currently taking place in the area of applying RSS satellite data in solving various scientific and applied problems as follows [8][9][10]:  The capabilities of RSS satellite systems have abruptly increased. It became possible to organize monitoring of rapidly occurring processes due to the increase in the number of RSS satellites. Over the past 4 years, the number of satellites has increased by an order of magnitude, the availability of high-resolution data has increased (up to 3 m/pixel).  The number of RSS satellite systems with "measuring" properties has increased. Such systems provide not only qualitative but well-calibrated quantitative information about various objects, processes and phenomena.  The level of information availability has increased. The data of many satellite systems began to be freely expanded both through the Internet and directly from satellites in an unencrypted form.  A significant increase in the amount of satellite data and the development of the requirements and cost of the systems for their reception and primary processing led to the intensification of transition from the use of personal RSS processing systems to service-oriented solutions based on regional specialized remote sensing centers. Along with the extensive development of RSS satellite systems in the last decade, there has been a fairly abrupt increase in the amount of satellite data received from individual spacecraft, as well as the increase in the nomenclature and total amount of information products generated on the basis of this data processing [11][12][13]. Thus, the growth rate of information amount exceeds the growth rate of the number of remote sensing systems. For example, the volume of RSS data archives in NASA and NOAA, accumulated over the year, increased by more than an order of magnitude: from almost 1 Petabyte in 2000 to more than 10 Petabytes in 2011, although the number of RSS satellites increased during this time not more than in 6 times. Entering new RSS data into the archiving system in 2018 according to NOAA plans will be 20 Petabytes per year, and their total volume will be more than 100 Petabytes. In this case, the amount of data per satellite will increase by 100 times. Similar plans to build up RSS grouping and a rapid increase in the amount of RSS data were accepted for implementation in Russia (the Federal Space Program of Russia for 2016-2025).
Thus, we should mention here the abrupt increase in the capabilities of satellite systems, a significant increase in the amount of satellite information and its availability, the development of data processing and presentation technologies, and the use of web technologies [14][15][16][17].

Data sources of modern remote sensing systems
The equipment of modern remote sensing systems is traditionally built around the main sources of remote sensing data. RSS data can be obtained 1) on request from the official RSS data providers (usually high spatial resolution data), 2) via the Internet from free sources and, finally, 3) direct reception of data from satellites operating in broadcast mode. Free information of low and medium resolution, and recently also high resolution, mostly is available for download from official resources, such as the USGS Earth Explorer service of the US Geological Survey [18]. This way for obtaining data in our country is gradually assuming increasing importance as Internet access becomes faster and cheap. However, not all the satellite data is available through such services, especially for new and developing systems. Another problem is delays in data processing. They are not always processed quickly enough. This factor can become important for tasks such as real-time monitoring of floods or fires. An unpleasant theoretically possible prospect for such services is the introduction of a fee for use, which can completely change the principles of financial support for the processing of remote sensing data.
The analysis showed that the most efficient and reliable option for obtaining medium and low resolution data is one's own (or partner) station for receiving space information. Among the existing standards of data transmission in the ranges of 137 MHz, 1.7 and 8 GHz, the latter one allows to transfer the maximum amount of information per session. Such a station is capable of receiving up to 320 Mbit/s depending on the diameter of the antenna, the number of reception frequencies (receivers) and the type of phase modulation [19].Taking into account the orbital altitude of the polar-orbiting satellites of about 800 km and the capture band of 2-3 thousand km, the viewing area of such a station represents approximately an ellipse measuring 5 by 8 thousand km long in longitude. For the station located near the geographical center of Russia, in Krasnoyarsk, about 80% of the territory of Russia is viewed twice a day from each receiving device. Such a station of X-band frequencies (8 GHz) forms the basis of the satellite receiving complex of FSC KSC SB RAS.

Organization of satellite data processing
The hardware-software arrangement of the RSS data processing in case of having its own satellite data reception station consists of 3 logical groups of servers -receiving servers, preliminary processing servers and thematic processing servers. It includes also workstations for interactive thematic processing (Fig. 1). Data storage systems for remote sensing data, characterized by large daily and session volumes, can be integrated into the servers of these logical groups or be located separately, perhaps as part of a larger infrastructure. The division of servers and processes into these three groups logically follows from the characteristics of each group of receiving and processing. The group of data receiving servers is allocated for the reason of working in real time mode at the moments of data reception by the radio channel and therefore is not loaded with any additional processing loads. The task for the servers of this group is to accept and to record on the disk memory the entire data stream from all sessions without loss.
The preliminary preprocessing servers group includes the processes of gridding and radiometric calibration of the received data. It is logically isolated because preliminary processes are required for all the space RSS and are performed firstly. Moreover, they are implemented based on the software specific for each RSS that is provided by the developers of this system.
The third group of thematic processing servers, in fact, fulfills the target data processing with obtaining the final useful information for consumers. This group includes thematic software of FSC KSC SB RAS own design. Processing here is more homogeneous and consistent, and also most often change due to the development of new versions or applications. This group includes a web interface built on the basis of geo-portal technologies, as the basic access point for consumers to remote sensing information [20][21][22].
The satellite reception complex includes the following equipment:  The receiving station UniScan-36 manufactured by SCANEX Company [19] includes a parabolic antenna with a diameter of 3.1 m, a three-axis supporting-rotating unit, two channels of data reception and recording based on the rack-mountable PC, and stationary GPS receiver. The antenna is installed on the roof of the building, on a specially constructed metal structure site. The GPS receiver is mounted on the guard. A rack with receiving equipment is located in the room on the top floor, directly below the antenna, since the maximum length of the high-frequency cable of the antenna is 50 m.  Data processing servers (2 pcs) by HP Company, one of which is responsible for the operational real-time primary data processing (Level 1), the second is designed for indepth thematic processing and solution of applied problems.  Clustered storage systems (2 pcs) based on hybrid disk arrays Network Appliance FAS2554 [23] (each of them includes 20 HDDs per 4 TB + 4 solid state drive SDD for data caching. The available capacity of the fault-tolerant storage is 36 TB).  High-performance workstations (4 pcs).  Local area network of 10G standard with data transmission at rates up to 10 Gbit/s through the copper twisted pair, built on the basis of the 16-port switch Extreme X620-16T [24],which provides full compatibility with less rapid standards. Cable lines in 10G standard are limited by a length of 50 m and require a cable and "twisted pair" connectors of 6 category or higher. Practically, the line with length of 48 m showed an exchange rate according to the protocol CIFS2 between the workstation and the server during 10 GB file copying in the beginning -1100 Mb/s, and at the end -190 Mb/s, which corresponds to the real recording rate on the disk array. On the line with a length of 53 m the adapters already automatically switched to the 1G standard. Thus, a modern compact local computer network is many times surpassed any disk array according to its throughput, even on the basis of solid-state devices, and is no longer a "narrow throat" of the system. As already noted above, the software used and developed for the satellite reception complex at FSC KSC SB RAS consists of three parts: 1) reception, 2) preliminary and 3) thematic processing. Let's consider its structure and characteristics:  The data receiving software was developed by SCANEX Company, a provider of a satellite receiving station. It provides full automation of all processes: downloading of orbital elements, receiving GPS time signals, scheduling based on given priorities for receiving data from various spacecrafts, managing multiple receiving servers, and so on. If necessary, the operator is provided with all the current information on the real-time process of the station, including the spacecraft flight schedule and the actual video stream from it (if possible, decoding it during the fly).  The preprocessing software includes three standard packages for data of different satellite systems: TERRA/AQUA for the MODIS radiometer, Suomi NPP/NOAA-20 for the VIIRS, CrIS, ATMS radiometers, Fengyun-3A/B/C for the MERSI radiometer. All packages operate in the Linux/CentOS environment on a single server. Processing scripts are started after a specified time interval (15 minutes).  Thematic data processing is performed in accordance with the tasks to be performed in parallel on two servers. In particular, for each received satellite image TERRA/AQUA, Suomi NPP/NOAA-20 a set of multi-scale multichannel overview images / "Quicklooks" (popular spectral channel combination, surface temperature, NDVI) generates for substantially instant display of information on the Web page. The cartographic web interface provides the user with a large set of visualization capabilities for remote sensing data.
The range of tasks to be solved on the basis of the data of RSS is constantly expanding. Investigations in the development of real-time satellite monitoring systems, carried out at FSC KSC SB RAS, involves the creation of new and, at the first stage, the use of existing available programs for RSS processing [25], including algorithms for radiometers MODIS and VIIRS: search of active fires, evaluation of aerosols content, evaluation of fume area, masking cloudiness, determination of the optical properties of cloudiness (temperature, height, fraction and aggregate state of the upper limit), atmospheric correction of visible and near IR channels, recovery of vertical atmosphere profiles, calculating NDVI and EVI.

Conclusion
The significant increase in the amount of satellite data, observed at the present time, leads to the need of developing methods and technologies for the systems of receiving, storing and processing satellite information. The analysis of the developed solutions and our R&D experience shows that one of the most promising approaches in creating systems of this type is the implementation of a set of interconnected autonomous functionally-limited components that interact with each other in a service-oriented architecture with a user interface built on the basis of geo-information web-system technologies.