Identification of Non-Stationary Objects Based on Training a Neural Network and Tuning the Parameters of a Generalized Model

. Methods and algorithms have been developed for identifying non-stationary objects of various types using statistical, dynamic, neural network models, which are taken into account when solving problems of conditions of a priori insufficiency, uncertainty, low reliability of data. Mechanisms are proposed that provide effective identification based on combining the features of dynamic models with the properties of random time series. The possibilities of algorithms based on mechanisms that use statistical, dynamic, specific data characteristics, as well as the properties of self-adaptation, approximation, organization, self-learning of neural networks have been expanded. A generalized function identification algorithm has been developed and its functions have been expanded by adaptive segmentation of time series, setting the informative interval of element values, the size of the training set, training multilayer neural networks, database, and knowledge base. The training algorithms for a three-layer neural network are modified based on the mechanisms for regulating interneuronal connections in layers, weight coefficients of neurons, variable activation functions, network architecture, and superposition of continuous input-output dependencies. A software package for identifying random time series in the C++ language in the CUDA parallel computing environment has been developed to predict the annual power consumption of the industrial zone of the Samarkand region using software tools for data preprocessing, filtering, smoothing; determining the boundaries of the informative interval of time series elements.


Introduction
In automated production and technological complexes for various purposes (industrial, agricultural, water resources management, power consumption), information on technical and economic indicators when solving problems for identification is considered as non-stationary objects.The proposed approach allows, in turn, the widespread use of methods and algorithms for data mining (DM) based on neural networks, optimization tools for dynamic identification of random time series (RTS) in substitutions, differential, recurrent equations, and other complex dependencies [1,2,3,4].Existing technologies for solving problems of identification, search for extrema, optimization, are determined by labor-intensive calculations and the use of highly iterative algorithms.In addition, they are characterized by insufficient generalization, the use of properties of non-stationary objects, models, statistical parameters, dynamic, specific characteristics [5,6,7,8].It is necessary to research and develop a wide range of identification algorithms that contribute to the achievement of a higher quality of information processing in conditions of insufficient a priori information, parametric uncertainty, non-stationary and low data reliability [9,10,11,12].Of great importance are methods and algorithms for data processing based on the generalization of the properties and features of neural networks (NN) with the properties of dynamic models for describing random time series (RTS), as well as aimed at improving the accuracy of calculations, reducing time and cost [13,14,15,16].This work is devoted to the study, and development of methods and models, as well as the implementation of mechanisms for using the statistical, dynamic, specific characteristics of the information of non-stationary objects, the properties of self-adaptation, approximation, organization, and self-learning of the NN, designed to significantly expand the capabilities of the traditional technology of identification of the RTS [17,18,19,20]..

Basic approaches, principles and mechanisms for optimizing the identification of non-stationary objects
An approach is proposed aimed at solving the following problems of identification of non-stationary objects: determining and setting the size of a set of training data; informative interval of parameter values; reduction of redundant features and parameters in the structure of the object description model; search for global and local extrema of the optimization function; extraction and use of statistical, specific characteristics, dynamic properties of data, features of the National Assembly; adaptive learning of NN; non-linear identification; formation of a database (DB) and a knowledge base (KB), setting model parameters [21 -25].The issue of optimizing the identification of non-stationary objects with the mechanism of selecting informative elements of the RTS and adaptive learning of the NN has been studied.Experimental training sets are defined and will be presented for data analysis and processing.Selected from the attribute space of the object for each identification parameter, the rules for the selection of informative elements of the RTS are defined.A training set with a RTS element is denoted by x , and S is a separate set.The attributes of the set of elements of the RTS x at the model output corresponds to the value of the variable s y .The total interval of possible values of the RTS element is divided into a sequence of sections (segments).The initial task is to determine the number of separating planes perpendicular to the axis of each feature, the number of segments in the total interval, the serial numbers of the segments, the left and right boundaries of the segments, and the rational size of the training set [26][27][28][29].
The task algorithm is presented in the following steps.
Step 1. Initialization of RTS identification parameters.The vector of the input element x is specified as a matrix, the rows of which are linearized, and the columns form an array } { * s y y  with binary elements.
Step 1.1.Formation of an array } { j D , which has a size equal to the number of RTS elements N .For each parameter, the number of segments in the total range of values of the RTS element is determined.
Step 1.2.Set: , where j is the ordinal number of the current element., then set the following variables: A generalized RTS identification algorithm has been developed, which is synthesized with the mechanisms of segment selection, determination of segment boundaries in the general range of RTS element values for each parameter, selection of informative elements, formation of a rational learning set, and adaptive learning of the NN [30][31][32][33][34].

A generalized algorithm for optimizing the identification of the RTS.
A mechanism is proposed aimed at reducing the number of segments in the total range of values of the RTS element.It is believed that the smaller the number of segments, the more rational the set of training data will be.The segmentation-based identification mechanism uses the following parameters: } , { y x -sets of pairs that are presented as arrays of information; } { j D is the number of segments in the total range of values of the RTS element; are serial numbers of each segment.
The mechanism includes algorithms designed to compare the calculation results for each set of k x and q x , check the segments, and form an informative set of training data.The informativeness of the RTS element within the boundaries of each segment and in the general range of values is estimated by the coefficient of equivalence of elements k x and q x .The equivalence coefficient between the k -th segment of the i -th parameter, the s -th set and the q -th segment of the j -th parameter of the g -th set in the general range of values of the RTS elements is determined from the following conditions: , , , ( The equivalence ratio between sets is defined as The coefficient of mutual equivalence of sets is given as The coefficient of mutual equivalence between the k -segment of the i -th feature and the q -th segment of the j -th feature in the total range of values of the RTS elements for each parameter of the object is determined.An algorithm for reducing the number of segments by equivalence coefficients with the following steps has been developed. Step Step 2.2.Determine coefficients: Step 3 Install: , then follow steps 4.1 and 4.2. Step , then delete the k -th segment of the i -th feature in the total range of values of the RTS element.Step 6.2.3.Install: . Go to step 6.2.
Step 7. Stop.An algorithm for optimizing the size of the training set based on setting the boundaries of the RTS segments has been developed.To adjust the boundaries of segments, a rule is specified, which is executed in the form if , where s i y is the sequence number of the segment in the s -th set by is the membership function (MF) of the i -th RTS carrier.
The issues of choosing the type of membership function, determining the boundaries of the membership function, and the common interval of supports of the corresponding sets are investigated.The trapezoidal model of the MF, the rules for estimating the parameters in the form are considered: The values of the parameter for Z -shaped membership function are evaluated according to the following conditions: For each training set, the compliance of the membership function parameters is checked.The correspondence of the parameters to the common interval of values of the RTS element requires the following condition to be met: Output s i y is represented by the rule If there is a maximum correspondence of the parameter ) ( . A mechanism for adaptive learning of a three-layer NN is proposed.Mechanism of variables adaptation in teaching a three-layer NN.The activation functions of a three-layer NN are defined as is the activation function of the i -th neuron of the  -th layer of the NN; function of the postsynaptic potential of the j -th output of the i -th neuron in the  -th layer of the network; ) , ( i w  - weight coefficients of the input ) , ( i x  , the i -th neuron in the  -th layer of the network.

The weight coefficients
) , ( i w  , i -th neuron, in layer  are determined by the following conditions: The mechanism for setting the parameters of the NN structural components was developed on the basis of the conjugate gradient optimization method.Various NN models, methods for calculating the weights of neurons, interneuronal connections, choosing an appropriate activation function, NN architecture, and searching for global and local extrema have been studied [35][36][37][38][39][40].
The NN learning optimization mechanism based on the conjugate gradient method represents modifications of gradient descent with a computational scheme for setting the weights of interneuron connections in the form , where is the value of the change in the weights of connections between the i -th neuron of the ( 1  q )-th layer and the j -th neuron of the q -th layer;  is the coefficient that determines the learning rate of the NN, The change in the weights of connections between neurons are defined as , where j  is the error value of the j -th neuron in layer q ; i x is the value of the i -th input signal for the j -th neuron of layer q .To adjust neuron offsets, i x is replaced by "1".The error value of the residual type in the output layer of the NN is estimated as , where k i y , is the required output of the NТ; k i f , is the actual output value of the k -th neuron for the i -th set of training data; ))' (  ( , S f k i is the value of the derivative of the activation function of the k -th neuron for the i -th set.
The output errors of the hidden layers are estimated as , where is the error of the ith neuron in the q -th layer of the NN; , a is a constant, then its derivative is given as . The value of errors in the activation function of the i -th neuron is determined as which for the q -th NN layer is calculated as Setting the values of the weights of interneuronal connections is performed according to the computational scheme A mechanism for retranslating the properties of the RTS, dynamic identification models on the NN, instead of complex nonlinear recurrent dependencies, is proposed.
The learning algorithm of the NN includes the following computational schemes: setting the parameters of the structural components of the NN; providing a dialogue with the formed databases and knowledge bases; extraction and use of statistical, specific characteristics, dynamic properties of information; computational schemes of NN structural components.Thanks to the implementation of these computational schemes, the algorithm acquires additional advantages, as well as the possibility of significant expansion due to the synthesis of smoothing and filtering algorithms for sharp jumps, random bursts in the RTS dynamics.
The generalized identification algorithm is based on the use of a matrix data representation model, and the knowledge base involves the synthesis of integer matrices of the following types: matrix Q , which specifies the description of an object parameter with k -attribute characteristics in the form of a sequence m z z ,..., 1 ; a difference matrix R that specifies the results of splitting (segmentation) of the RTS for clustering; matrix I , which defines the description of the object parameter with informative characteristics in the form of For what, from the set of rows of the matrix R , one column matrix ' R is formed, in which non-informative elements of the RTS are marked, as well as placed as an element of the matrix Q .A binary diagnostic test matrix T is formed, the columns of which are compared with the column of the matrix Q .In each row of the matrix T , informative features are marked with a "+" sign.The matrix model is synthesized with the implication matrix ) (U , in which various criteria, rules, conditions, and patterns are specified to identify features in an informative data set composed of different parameters.The learning algorithm with network parameter tuning includes the following steps.
Step 1.The weight coefficients of neurons w are set, where k is the weight index of the j-th neuron in the i-th layer of the network.
Step 1.1.The output n O of the network is calculated and the value max E of the network error function is determined.
Step 2. Sequence numbers of the presented training data sets are registered.
Step 3. The total value of the network error function is calculated for the input sets of training pairs It is proved that the generalized learning algorithm of the NN with parameter tuning with a strong variation of the elements of the RTS becomes more stable and converges 4-5 times faster than the algorithms built on the basis of typical computational schemes for learning the NN..

Conclusions
Scientific and methodological foundations for optimizing the identification and processing of data of non-stationary objects with mechanisms for setting the informative range of values of the RTS elements, the size of the training data set based on the segmentation of the RTS, adaptive learning of the NN, the formation of a database and a knowledge base have been developed.A generalized algorithm has been developed that synthesizes mechanisms for setting the boundaries of RTS segments, selecting an informative data set, calculating feature equivalence coefficients, and adaptive learning of the neural network.High-precision tools for identification of RTS and data processing, adaptive learning of the NN in real time have been obtained.The algorithms have transparent structures, are error tolerant, and address the shortcomings of highly iterative algorithms.The properties of the NN and non-stationary objects are effectively emulated, as a result of which the accuracy of the RTS forecasting increases to the required level and becomes less expensive.
A generalized RTS identification algorithm has been developed and implemented in the MATLAB package environment.The mechanisms for setting the informative interval of RTS values, adaptive segmentation, adjusting the size of the NN training set, neurons in layers, coefficients of interneuronal connections, network architecture, functional dependence "input-output" are implemented.The efficiency of algorithms for identification, RTS segmentation, adaptive learning of the NN has been studied and the results obtained have been compared with the efficiency of the traditional NN learning algorithm.A software package for identifying RTS has been developed to predict the technical and economic indicators of annual power consumption by enterprises in the industrial zone of the Samarkand region, which is represented by the following functional modules for data pre-processing, filtering, smoothing, RTS segmentation; definition of segment boundaries, the informative interval of parameters.
range of values of the MF element, then the segment 1 layer of the NN; ij wweight of interneuron connections.The value of the error of the first layer of the NN proportionally affects the value of the errors of neurons in the next layer.The change in the error value of the output NN layer for the k-th set of training data determines the network learning rate.When the activation function of the NN is given as a sigmoid

Fig. 1 .Fig. 2 .
Fig. 1.Graphs of the RTS approximator a) according to the statistical model; b) based on NN.In figure2illustrated the results of the approximation of the dynamic and neural network models of a non-stationary object with the tuning algorithm for the output process.

1 ;
matrix E that defines the estimates of the dynamic characteristics of the RTS.The matrix model is given in interval form.Informative features * identification model in the form of one row of the matrix Q .The results are compared with the row elements of matrix Q .

4 .
The neuron weight coefficients ijk w are adjusted.The weights are corrected in the direction from the last layer to the first.Step 5.The value of the network error function E is corrected.Step 6