Intelligent model for recovering data damaged by high-energy particles

Northern (Arctic) Federal University, Northern Dvina emb., 17, Arkhangelsk, 163000, Russia

^* Corresponding author: This email address is being protected from spambots. You need JavaScript enabled to view it.

Abstract

Stratospheric and space exploration provide valuable scientific data collected by probes and transmitted to Earth via telemetry messages. However, data transmission in stratospheric conditions is subject to interference and distortion, which leads to message corruption and loss of information unsuitable for automatic processing by standard parsers. This paper addresses the pressing problem of developing a system for intelligent recovery of corrupted telemetry data. The aim of the study was to create and evaluate the effectiveness of a neural network model capable of automatically correcting corrupted JSON messages. To achieve this goal, a synthetic data generator based on real telemetry parameter profiles and an algorithm for modeling various types of damage (breaks, character distortions, encoding errors, JSON structure violations) were developed to create a training dataset. The main architecture chosen was a Seq2Seq transformer model implemented using the PyTorch library, which takes into account the context of previous messages and uses character-level tokenization. The model was trained on the generated dataset for 20 epochs. Evaluation of the recovery quality on examples and analysis of inference performance on the CPU showed that the model demonstrates high efficiency in structural recovery of damaged JSON messages, successfully generating syntactically correct strings even with significant damage. However, recovery of semantically correct numerical values turned out to be limited. The resulting solution has acceptable performance for integration into the operator’s software, but requires further research to improve the accuracy of recovery of measured parameters.