Determining the Resource of Safe Operation for Objects by Images

In this paper, a systematic study of the microstructure damage process of metals and alloys was carried out. The main elements of the microstructure surface image, as well as the rules for the formation and interaction of rough slip traces and cracks to determine the model of damage accumulation on the image of the microstructure surface under cyclic loading are determined. A classifier that allows to determine the number of loading cycles before a sample goes out of service is proposed. A modernized structure of the convolutional neural network was developed to classify images of the damaged microstructure of the metals and alloys surface. The proposed classifier for determining the number of loading cycles made it possible to achieve a classification accuracy of 78.43%.


Introduction
The complication of technical systems, the transition to higher parameters of technological environments in order to increase the efficiency of production processes leads to an increase in the risk of severe accidents in various industries, transport, energy, oil and gas production, during transportation and processing of energy carriers. The increasingly severe and potentially possible environmental and economic consequences of such accidents lead to an increase in requirements for ensuring high reliability and safety in the operation of such systems. The solution to this complex problem includes, among other things, determining the resource characteristics of the machines and equipment of the systems, including the residual resource, assessing the operating conditions of production facilities, metal structures and equipment, predicting their condition during operation. The growing relevance of these tasks is evident both in the case of combating aging and destruction of structures during their operation, and when creating new materials with desired properties.
One of the perspective areas for solving the problems of operational control and diagnostics of the metal structures state is to use surface images of the microstructures of metals and alloys, the dynamics of which during operation allows an assessment of the resource characteristics of objects. Conducting research in the field of assessing the damage to the microstructures of materials exposed to cyclic influences during their life cycle is an urgent task to determine the resource of safe operation of objects. The problem of safe operation of industrial facilities is devoted to a lot of work and research. A variety of factors make it impossible to solve this problem in a general way. However, a number of specific tasks within the framework of this problem are completely solved.
The results of the study of fatigue processes and damage accumulation are relevant in solving problems of improving the mechanical properties of materials, and can also be used in developing programs for the inspection of metal structures.
Visual control of microstructure images, which allows predicting the fatigue resistance of metals and alloys without damaging effects on the object, is a promising area in the development of automated systems for ensuring the reliability of metal structures, state monitoring and diagnostics in comparison with other approaches.
Accumulation of damage on the surface microstructure during operation leads to the formation of rough slip traces (RST), the development of which leads to the formation of cracks and the destruction of metal structures. An important task is not only monitoring the accumulation of microdamages, but also developing criteria for determining the pre-defective state of the material. In theoretical terms, the task is to develop a model of the process of fatigue failure, taking into account the rotation and deformation of grains, the accumulation of microdamage, the formation of microdefects, their fusion and the formation of a main crack.

Matherials and methods
A significant contribution to the study of images of the surface microstructure of metals and alloys was made by V.S. Ivanova, V.F. Terentyev, J. Carroll, A. Clayton, A. Cherrone, A.L. Zhiznyakov.
Theoretical and experimental studies in the field of loading of materials, leading to the appearance of plastic deformations, the development of which ends with the formation of RST and the subsequent destruction of structures, are reflected in the works of A.V.Guryev, A.A.Weinstein, Yu.V.Sovetova, V.P.Baranova, V.E.Stepanova, D.I.Shetulova, A.V. Gonchar and others.
Assessing the technical condition and quality of the material of structural elements and machine parts in order to ensure their safe operation is an urgent task, for the solution of which non-destructive testing methods are widely used. The stage of accumulation of microdamage during power loading, which begins from the moment the facility is commissioned, requires additional research. Currently, in all industrialized countries, methods and techniques of non-destructive testing of the actual state of the material are one of the most popular developments. At the Institute of Strength Physics and Materials Science of Siberian Branch of Russian Academy of Sciences it is proposed to diagnose the stages of damage accumulation in a material long before the appearance of visible cracks on the basis of the optical-television and acoustic-emission method [1]. The system records strain and acoustic emission signals during material deformation.
At the Baikov Institute of Metallurgy and Materials Science, RAS, studies of the processes of localization of deformation and fracture by replica methods, acoustic emission, magnetic memory of metals, ultrasonic attenuation, micro hardness and electrical resistance are carried out. In particular, the relationship of the estimated physical parameters with the size of the plastic zone and the concentration of micro cracks in it was considered [2]. The mechanical properties of structural steels are studied by acoustic and magnetic methods in the process of stretching flat specimens with a concentrator and the physical parameters characterizing the achievement of the ultimate state corresponding to the yield strength and strength of structural materials are determined [3].
Part of the research devoted to the problem of assessing the actual state of the material relates to the problem of predicting and detecting the moment of crack initiation on the surface of the material using information about the state of the microstructure of the material. As a rule, studies use X-ray diffraction analysis and scanning electron microscopy, which do not allow such "thin" studies at industrial facilities.
In [4], localization of crystallographic slip in slices was studied under uniaxial compression of a single crystal of copper using long-range high-energy diffraction microscopy. During the study, a unique mobile detector step was used, which provided access to many diffraction peaks with high angular resolution. The authors obtained pole figures of different orientations of the single crystal, which made it possible to track the evolution of the distribution of the lattice orientation, which develops as the slip localizes.
In the works of D.I.Shetulov [5,6] on the basis of images, the complex adverse character of geyser instability, causing heat removal disturbances and additional adverse effects on the structural material of pipelines and the main technological equipment of nuclear power plants, in the form of difficultly predicted cyclic loads, was considered.

Estimation of material durability based on microstructure analysis
Obviously, the material used in the structures is subjected to periodic load of the body and gradually destroyed.
It should be noted that the result of fatigue tests also depends on a set of factors -the conditions under which studies were conducted, among which structural, technological, metallurgical, and operational factors are shared ( fig. 1).
There are works [7][8][9][10], in which a relationship was revealed between the damage to the surface of the microstructure F (between the concentration of rough slip traces) and the number of loading cycles before the appearance of a macrocrack, i.e. the current state of structural elements.
To perform a quantitative assessment, damage F must be represented as a function of many factors, such as n31 -the number of grains on the microstructure, n32 -the number of damaged grains, n33 -the difference in the number of intact and damaged grains over their entire area, n34 -the number of grains damaged by wide slip bands , np1 is the total number of strips in the damaged grains, np2 is the number of wide, tortuous and intermittent slip bands, Fm is the actual area of the microstructure, Fobr is the area of the working surface of the sample: Theoretical and experimental studies of the relationship between the parameters of a damaged microstructure and the number of loading cycles have been carried out by many researchers.
For example, in [7] it was established that the damage to the surface of microstructures is directly proportional to the following relative values: n32⁄n31, n34⁄n33, np2⁄np1, Fm⁄Fobr, and the revealed dependence has the form: It is obvious that with an increase in the number of cycles N, the values n31, Fm and Fobr do not change.
Thus, using the data obtained by images processing of the microstructure of the surface of metals and alloys, it is possible to determine the number of loading cycles based on experimentally obtained quantitative information.

Fig. 1. Fatigue test model
Obviously, the type of dependence of the degree of damage to the surface microstructure on the number of loading cycles is determined by a specific set of acting factors during the operation of metal structures or during their production.
The form of this dependence can be extremely complicated, and its experimental preparation is a long and expensive process.
In this regard, at present, the degree of damage to surface microstructures is assessed by operators based on visual analysis of the microstructure of the surface of the sample.
Obviously, due to the presence of the human factor, such estimates are specific: • a high degree of subjectivity of the results; • low accuracy of the results; • significant time costs for performing image analysis operations on the surface microstructure of metals and alloys. Due to the need to improve the accuracy of such estimates, it becomes urgent to solve the problems of automating the process of analyzing the image of the surface microstructure and developing a procedure for determining the number of loading cycles based on surface damage.
Experimentally obtained images of microstructures, data on the number of loading cycles and descriptions of the conditions under which cyclic loading was performed, allow us to associate surface damage with a specific number of loading cycles in a certain ndimensional space of conditions for fatigue tests.
Comparing the degree of damage with the number of loading cycles in a particular test, one can see the relationship connecting F and N for specific combinations of acting factors ( fig. 2).
Obviously, knowing the type of such a dependence, we can predict both the state of the structural material under cyclic loading (assess the degree of damage to the microstructure of its surface) and predict the durability characteristics, for example, evaluate the residual life of a part or structure. Thus, the urgent task of determining the analytical form of this dependence.
Damage to the surface can be considered as the difference between the reference image (the image of the initial surface microstructure) and the image of the surface microstructure after cyclic loading.
The training and test samples for the current task consisted of images that were obtained during fatigue tests described in [13]. An example of such images before and after 2,2·105 cyclic loading is shown in the fig. 3.

Model
Formally the problem of assessing the degree of damage to the surface microstructure depending on the number of loading cycles can be represented as follows.
A reference image of the surface microstructure si ϵ S is presented, representing its initial state before the test, and for it there is a set of image samples { }. Each element in the sample set is supplied with a numerical value corresponding to the number of loading cycles N. The set also contains images before and after the formation of a crack.
The task is to build an algorithm that, for each reference image and an image from the set , would determine the number of loading cycles N that led to this result.
. If we consider images after each loading cycle in the form of elements of one class, then the problem to be solved can be attributed to the image classification problem, where the number of loading cycles characterizes the class number.
In other words, it is necessary to formulate a classifier that allows us to determine whether the image of the damaged surface microstructure belongs to a class that characterizes the number of loading cycles.
Neural networks are used to classify images [11,12]. In this case, the main task that has to be solved when using artificial neural networks is the process of selecting network characteristics, such as the number of layers, the number of neurons in each layer, as well as the formation of training and test samples for each specific case.
There are deep learning neural networks, which include convolutional neural networks. Work on convolutional neural networks showed quite good results in image processing tasks.
The use of classical neural networks for image processing problems is usually difficult with a large array of input values, since in order to reduce the likelihood of errors of the second type, it is advisable to choose a large number of inputs of the neural network.
The increase in the number of neurons and the connections between them entails a significant increase in the complexity of the learning process, and also requires significant costs for the computational processes used in training the network.
For example, an experimentally obtained image of a metal microstructure has a size of 325 by 255 pixels, that is, an array of values of 82875 in the input layer. Let's say our neural network contains two hidden layers. The number of neurons in the first hidden layer is 4 times the number of input values of the neural network.
Thus, the first hidden layer contains 4 * 82875 = 331500 neurons and accordingly has 82875*331500 connections to the input layer, and the second 165750 neurons and 165750*331500 connections to the first hidden layer.
Since the study uses three groups of images, in the output layer we have 3 neurons and 497,250 connections with the previous layer.
Obviously, in this setting, the task of computing and training the network for classifying images is resourceintensive.
Convolutional neural networks can solve this problem. In addition, they showed quite good results in image processing tasks, since they allow you to take into account information about the relative position of the image pixels.
The main concept in convolutional neural networks is convolutional layers consisting of feature maps that are formed by applying the same weighting factors to a group of neurons.
Thus, each neuron of the feature map associated with a part of the neurons of the previous layer performs a convolution operation.
Also, these types of networks may contain layers of subsampling, which make it possible to achieve partial scale invariance. Such types of layers reduce the spatial dimension of the image, but at the same time retain the features highlighted on previous layers.
In addition to convolutional layers and layers of subsampling, these types of networks contain fully connected layers. The output layer, as a rule, is always fully connected. All three types of layers can alternate, which allows you to create hierarchies of feature maps, and therefore recognize them.
The architecture of the proposed convolutional neural network for classifying images of the microstructure of the surface of metals and alloys to determine the number of loading cycles is shown in fig. 4. The vector obtained at the output of the "image transition into the vector of signs" unit is used when comparing the reference sample image s and the sample image d after loading. According to the vector space model [14], the similarity between two vectors can be definitely defined as the cosine of the angle between them: Expression (1) can be reduced to the form: where M is the similarity matrix of the reference image s and the sample after loading d. The similarity matrix is a network parameter and is configured during the learning process. The size of the matching layer is 20 + 20 + 1 = 41 neurons and 20 * 20 + 20 * 20 = 800 connections with the previous layer.
Training the model with 250 epochs resulted in the highest test accuracy of more than 86,37%, and the test accuracy was 78.43%.

Conclusion
Using a neural network classifier to determine the number of loading cycles, a classification accuracy of 78.43% was achieved. The results of the neural network classifier indicate the possibility of using the developed architecture as an element of an expert decision support system. In the case of assessing the degree of damage to structural elements, increased attention indicates the need to use expert opinion to decide on the possibility of further operation of the facility. The proposed approach will significantly reduce the accident rate at critical facilities.