Localized growth distribution on the abdominal aortic aneurysm surface using deep learning approaches

. An abdominal aortic aneurysm (AAA) is a dangerous pathology that needs regular monitoring based on medical images. Currently, only visual estimates of the growth rate and methods based on the assessment of changes in the maximum diameter of the aneurysm in clinical practice are used. However, the quantitative assessment of vessel wall growth rate based on deformable image registration is gaining popularity in research. This paper presents a study of the applicability of the neural network approach of image registration for the quantitative growth assessment problem. In this study, we analyzed classical and neural network methods of image registration and used VoxelMorph and HyperMorph neural network architectures to evaluate local AAA growth based on the available dataset. Also, we compared the results of the obtained maximum local deformations of the AAA with the method of estimating the change in the maximum diameter.


Introduction
Abdominal aortic aneurysm (AAA) is a local or diffuse expansion of the abdominal aortic diameter compared to the normal diameter by 50% or more.The progression of aneurysmal enlargement is accompanied by increase in the non-physiological load on the wall that may lead to aortic rupture.This vascular disease has a significant mortality rate in the developed world [1].The main problem of diagnosing this disease is that it passes almost asymptomatically [2].Therefore, early diagnosis of AAA and subsequent monitoring of its development are necessary to reduce mortality and postoperative complications.However, in modern clinical practice there are no accurate and fully automated tools for tracking the growth and development of aneurysms.
Recent papers [3][4][5][6] have proposed a Vascular deformation mapping (VDM) method for quantifying soft tissue growth using deformable image registration.Studies [4,5] have shown the success of using this method in monitoring the growth of AAA.The main task of VDM is to find a diffeomorphic mapping that transforms the initial geometry of the aneurysm into the final one between the series of images.Then a field of vectors of three-dimensional deformations is constructed and their projections are mapped on the vessel wall.VDM has a number of advantages over standard tools for monitoring the growth of AAA and assessing the risk of its rupture based on measurements of an increase in the maximum diameter of the vessel.The error of manual measurements of the maximum aneurysm diameter can reach ± 0.5 cm [5] which critically affects the results of monitoring the development of small and medium aneurysms.
All the approaches used for deformable image registration can be divided into classical and neural network methods.Traditional approaches require high time costs to solve the problem on large dimensions and cannot support a fully automated registration process without using manual settings of launch parameters.However, neural network approaches solve problems that classical methods cannot cope with due to the possibility of training on large datasets and the variability of architecture modifications.Therefore, this paper presents a research of the applicability of neural networks to the problem of deformable registration of segmented CT images of AAA for an accurate quantitative assessment of its growth.

Definition
The registration of medical images consists in establishing spatial anatomical correspondences between the initial and follow-up images.
1 × 2 × 3 denotes the set of all diffeomorphic transformations that translate the image space of dimension  1 ×  2 ×  3 into itself.The term ℒ  (. ) is similarity metric between the transformed moving and the fixed images, ℒ ℎ (. ) is responsible for the smoothness of the resulting transformation, and ℒ  (. )   is similarity metric between the transformed moving segmentation masks and the fixed segmentation masks.

Classical approaches
Classical methods solve the problem of minimizing the error function separately for each pair of images.Such methods are based on the task of continuous or discrete optimization; however, they differ in approaches to both modeling a deformable body and searching for the necessary transformation.There are approaches derived from approximation theory, for example, algorithms using radial basis functions [7] and free form deformation (FFD) using B-splines [8].There are also examples of modeling an image with deformations applied to it as an elastic body using the Navier-Cauchy equation [9,10] or as a viscous liquid using the Navier-Stokes equation [11].The Large Deformation Diffeomorphic Metric Mapping (LDDMM) [12,13], which optimizes using an algorithm based on gradient descent, became a direct continuation of the approach with modeling the behavior of a viscous fluid.Its modification was the symmetric image normalization method (SyN) [14] which is used in the ANTs and greedy software packages [12,15].

Deep learning approaches
Approaches based on deep learning algorithms are the most common methods in the recent studies on deformable image registration, since they have a great advantage due to the absence of the need to optimize the transformation for each pair of incoming images.After the process of assignment parameters at the training stage, the neural network can provide a quick response output at the testing stage.Methods based on reinforcement learning were among the first to appear [16].However, the registration process with the help of such neural networks took a lot of time due to the iterative nature of their work, so later architectures supporting teacher training began to be developed [17,18].Currently, all architectures described in the medical image registration literature rely on variations of unsupervised learning approaches.These include methods based on image similarity [19,20] and GANs [21] for solving the problem of deformable registration between multimodal images.Weekly supervised methods [22][23][24] are applicable for data sets consisting not only of pairs of medical images, but also of segmented masks of some internal organs (as in equation (1.2)).The VoxelMorph and HyperMorph networks [24,25] which are used in this study can also be attributed to this class of architectures.

Materials and Methods
To solve the problem of deformable registration of AAA CT images, two neural network architectures were considered: VoxelMorph and its modification HyperMorph.The initial dataset consisted of 60 pairs of CT images with contrast of the abdominal aorta corresponding to different patients.In each pair there were two pictures of the same patient taken with a time interval from six months to five years.The images were segmented into classes of lumen, thrombotic masses and calcinates using a neural network model based on the 3D Unet architecture with the resnext 10132x8d encoder [28].Rigidly registered pairs of CT images and their segmentation masks obtained from the developed data preprocessing algorithm were used to train each of the registration neural networks.

VoxelMorph and HyperMorph Learning Process
To solve the problem of deformable AAA registration, we chose the VoxelMorph neural network, since it is one of the most popular and studied architectures, for training which does not need markup in the form of correct fields of deformations between images, but at the same time it allows you to input segmented images as sources of additional information.The VoxelMorph architecture is based on a convolutional neural network (CNN), similar to UNet [29].The learning process consists in selecting the optimal values of the neural network weights that minimize the value of the registration error function from formula (1).However, one of the main difficulties in VoxelMorph training was the problem of selecting hyperparameters.The loss function contains the parameters  and  that regulate the properties of the resulting transformation.As a rule, many neural network training runs with different values of  and  are used to solve this problem, but this is a long, time-consuming and inefficient process.To solve this problem, a modification of VoxelMorph called HyperMorph appeared [25].HyperMorph trains not only to find deformations between two images, but also to find the dependence of the resulting deformations on the values of the specified parameters.A feature of HyperMorph is the addition of a fully-connected hypernetwork ℎ  to the original CNN   where  is the set of network weights.It takes as input the randomly generated hyperparameter values in the form of a set  = { ′ ,  ′ } and calculates the values of weights  of the main network   .The network scheme is shown in Figure 1.At each iteration of HyperMorph training procedure the input data   ,   ,   ,   are generated as well as the values of the hyperparameters  = { ′ ,  ′ } obtained as random numbers from a uniform distribution [0,1].The error function ( 1) is modified as follows: ℒ ℎ (  ,   ,   ,   , , ) = (1 −  ′ )(1 −  ′ )ℒ  (  ∘ ,   ) + ′ ℒ ℎ () +  ′ ℒ  (  ∘ ,   ). (2) Multiplier (1 −  ′ )(1 −  ′ ) before the first term provides coverage of all possible weighting ratios between each error term using the values of the parameters  ′ ,  ′ from 0 to 1.
Before the training procedure we divided the total dataset into training and test samples with a ratio of 50:10.The mean square error (MSE) was chosen as the value of the ℒ  (. ) function.A set of channels of sequential convolution stages of the form [16,32,32,32] for the encoder and [32,32,32,32,32,16,16] for the decoder.Adam with a learning rate of about 10 −6 was chosen as the optimizer.The size of the butch was one pair of pictures.Each training epoch consisted of 50 iterations.The training was provided on an NVIDIA A100-PCIE-40GB graphics card.

Deformable Registration
In this work, the results obtained using VoxelMorph and HyperMorph neural networks were evaluated by the accuracy metrics of transforming AAA from a moving to fixed image and compared with the results of deformable registration obtained using greedy software package [15].Deformable registration for all methods was carried out on pairs of images preprocessed using an intensity-based rigid registration algorithm.
We found that it is difficult for neural network to get good quality for deformable registration of thrombotic masses due to their low contrast in relation to the external environment while analyzing the HyperMorph results.Therefore, we decided to develop an algorithm that increases the contrast by AAA segmentation masks.The algorithm based on finding the minimum of intensity image gradients.This approach helped to visually distinguish AAA thrombotic masses from the general environment.
Comparative results of deformable registration are presented in Table 1.To obtain the HyperMorph prediction, parameters were selected that give the largest averaged Dice metric for all segmentation classes in the test sample.HyperMorph showed an increase in the quality of registration of the AAA wall in comparison with other methods presented.At the same time, HyperMorph with local contrastive algorithm is most accurately registered thrombotic masses.

Validation of the results by estimating the change in the maximum diameter
To assess the applicability of the HyperMorph neural network approach to the task of quantifying the growth of AAA, we compared the results obtained with the method of estimating growth by changing the maximum diameter.We used an algorithm based on measuring the maximum diameter of the vessel using a centerline built with the vmtk library [30].The maximum diameter was measured sequentially on each image from the pairs of the test sample, and then the difference between the measurements for these time series was calculated.The obtained value of the change in the maximum diameter was compared with the maximum value of the displacement vector of the aneurysm transformation calculated using HyperMorph.Tables 2 and 3 present a comparison of the two calculated estimates.It shows that the estimation of the maximum deformation of HyperMorph and HyperMorph with local contrastive algorithm gives a deviation of about a millimeter for the AAA wall compared with the methods of maximum diameter estimation.Thus, the results of the two methods can be considered comparable.The deformation field obtained by HyperMorph registration of the baseline image and follow-up scan was applied to this baseline outer aortic wall mesh, producing an aortic deformation map with local growth at every node on the mesh.This surface mesh can be displayed as a "heatmap" of the outer aortic wall, with the magnitude of normal component of deformation displayed at every mesh vertex to describe the local change in that region of the aorta over the period between the baseline and follow-up CT images.Figure 2 shows heat maps of deformations for the outer wall of the abdominal aorta for four patients.

Conclusions
In this study, a computational technique is proposed to solve the problem of automatic registration of CT images of an abdominal aortic aneurysm in order to accurately quantify the growth of AAA.Due to the optimal choice of hyperparameters of deformable registration using HyperMorph, it was possible to obtain metric values for the aortic lumen that are better in quality than classical methods.The HyperMorph with local contrastive algorithm approach showed better results on thrombotic masses due to the preprocessing algorithm that distinguishes this class on the atlas.The evaluation of the maximum deformations of the outer

Fig. 1 .
Fig. 1.The scheme of HyperMorph training procedure for deformable registration of segmented AAA CT images.

Fig. 2 .
Fig. 2. Heatmaps of local deformations (mm) of the abdominal aortic wall constructed using HyperMorph with local contrastive algorithm.The time interval between the images A: 1 year; B:1 year; C :2 years; D:1 year.
Let   ,   denote the fixed and moving images of dimension  1 ×  2 ×  3 and   ,   denote their segmentation masks. ∘  denotes the image  which was affected by the transformation .

Table 1 .
Comparison of results of deformable registration approaches.

Table 2 .
Estimation of the difference between measurements of the change in the maximum diameter of the AAA wall and the maximum local deformation of the aneurysm constructed using HyperMorph and HyperMorph + contrast.

Table 3 .
Estimation of the difference between measurements of the change in the maximum diameter of the AAA outer wall and the maximum local deformation of the aneurysm constructed using HyperMorph and HyperMorph + contrast.