Effective thermal conductivity of unsaturated soils based on deep learning algorithm

Soil thermal conductivity plays a critical role in the design of geo-structures and energy transportation systems. Effective thermal conductivity (ETC) of soil depends primarily on the degree of saturation, porosity and mineralogical composition. These controlling parameters have nonlinear dependencies, thus making prediction a nontrivial task. In this study, an artificial neural network (ANN) model is developed based on the deep learning (DL) algorithm to predict the effective thermal conductivity of unsaturated soil. A large dataset is constructed including porosity, degree of saturation and quartz content from literature to train and validate the developed model. The model is constructed with a different number of hidden layers and neurons in each hidden layer. The standard errors for training and testing are calculated for each variation of hidden layers and neurons. The network with the least error is adopted for prediction. Two sand types independent of training and validation data reported in the literature are considered for prediction of the ETC. Five simulation runs are performed for each sand, and the computed results are plotted against the reported experimental results. The results conclude that the developed ANN model provides an efficient, easy and straightforward way to predict soil thermal conductivity with reasonable accuracy.


Introduction
The non-conventional energy generation facilities and methods have generated a massive demand for construction of energy infrastructure for generation, transportation and distribution along with the updating the existing one [1]. The energy storage [2] and distribution systems [3] build in the ground require precise knowledge of the heat transport capacity of the soil mass which is heterogeneous and multiphase, i.e. solid soil skeleton, water and air in voids. The ability of the soil mass to dissipate heat is termed as the apparent or effective thermal conductivity is of engineering and scientific importance.
The effective thermal conductivity (ETC) of soils is measured or estimated with many different mathematical [4,5], empirical [6], semi-empirical [7], continuum [8] and dis-continuum numerical methods [9,10]. Although the empirical and semi-empirical methods are fast and easy to implement, they often produce significant errors as these models are developed for a specific material for a prescribed boundary condition. The numerical methods are accurate but require a good understanding of the method and significant computational power. The recent development in computer science has allowed the computer to learn from the data and forecast. There exist many variants of machine learning and are used for many different purposes [11]. The approach mainly used in civil engineering is the Artificial Neural Network (ANN) which is derived from the deep learning algorithm [12]. In the past researchers have used ANN to predict the thermal conductivity of soil from moisture content and dry density [13][14][15]. However, many different studies have pointed out the importance of the mineral composition of the soil and its effect on thermal conductivity [16].
In this paper, we computed the ETC of sandy soil form three input parameters, namely, the porosity, degree of saturation and quartz content. The training, validation and testing data are collected from literature, and the model is trained against it. The trained model is then used to predict the thermal conductivity of a different set of sands of different mineralogy and origin. The model predicted values showed a good agreement with the experimental results.

Material and Method
For training and validation of the ANN, a total of 384 data points are collected from literature to establish a generic dataset [17,6,18,19]. Fig. 1 shows the variation in thermal conductivity along with porosity and degree of saturation. The colour bar is used to plot the quartz content of each sample. A general trend is observed as the quartz content of the soil increases the ETC value follows an upward trend too. The data is classified into two subsets for training and testing with a ratio of 4:1. However, the approach usually runs into a problem known as overfitting and to avoid it, the training data set is further subdivided into two parts with a ratio of 70:30. The technique of subdividing the data set into three parts is known as cross-validation. Therefore, the data is subdivided into three parts for training, validation and testing sets.

Artificial Neural Network
Artificial intelligence mimics the natural human brain intelligence into machine intelligence. A.I. is the most growing field nowadays which have applications in every scientific and engineering branch. It's the most discussing and researched topic in computer science. ANN presents the human brain structure. It consists of multiple artificial neurons which are connected and make a fully connected neural network (Fig.2). To define a good ANN, more emphasis is given to the topology, architecture, activation function, learning algorithm, learning rate, epochs, parameters and potential. Typical ANN has three main layers, the input layer, the hidden layers and the output layer, which is also called collectively as a feed-forward Multi-Layer Perceptron (MLP). Each layer consists of neurons which are connected to the neurons of the next layer through weighted connections. Fig.2 shows the ANN architecture which we have used for solving our problem. It has one input, three hidden layers and an output layer. The input layer receives three parameters, namely -Porosity (n), Saturation (Sr) and Quartz content (mq). The hidden layers in an ANN is a black box, which sets the weight values of the connections randomly. These values then gradually increase or decrease using Back-Propagation (B.P.) algorithm. During the training of the neural network, the output we get from the ANN is compared to the actual experimented values, and the errors are computed. Then using B.P. algorithm weights are readjusted. The rate of change to the weight values depend on activation function and learning rate.

Data collection
To conduct this analysis, laboratory-measured data is collected. The dataset contains three inputs porosity (n), Saturation (Sr) and Quartz content (mq) and one output parameter thermal conductivity (k). We divide our data into two subsets -Training and Testing dataset.

Model construction
To predict the effective thermal conductivity of unsaturated soil, the input and the output nodes are fixed, but the number of neurons in the hidden layer is varied. The procedure is applied because the neurons in the hidden layer differ from data to data. As the computation in the hidden layers of an ANN is a black box, the number of hidden layers and the number of neurons is selected through hit and trial method. Starting with only two hidden layers with four neurons each, we computed till three hidden layers with ten neurons each. The author has used a combination of only even neurons in each hidden layer based on his previous experiences [15]. The activation functions help us determine the learning rate. There are various activation functions: tanh, Sigmoid and ReLu. Fig. 3 shows the graph of the three functions. Sigmoid output ranges in between 1 to 0. The tanh output ranges in between -1 to 1 and ReLu produces exact output if the value is positive otherwise 0. We used the tanh activation function for the network. Above mentioned equations are the activation function for testing the model. To implement the model high level deep learning library of python named Keras is used.
Keras provides an excellent suite of all deep learning functions which are easy to use. To optimise the model weight parameters, Adam optimiser is used. Adam optimiser calculates the adaptive learning rate for each parameter. Adam optimiser helps not just by defining the learning rate explicitly, but instead, it chooses itself based on its training. Epoch size for training was varied from 400 to 800, and the batch size is fixed to 10 ( Table 1).

2.1.3Error calculation and selection of model
Above mentioned error formulas are used to check the model parameters. is the measured thermal conductivity value and Ŷ is the predicted value. n is the number of measurements.

Results and Discussion
The neural network model is trained with the data set shown in Fig 1. The standard errors corresponding to each model are tabulated in Table 1. The first column of Table  1 shows the number of hidden layers adopted in each deep neural network model. The second column of the table shows the number of neurons in each hidden layer. The number of neurons in each layer is chosen on an ad hoc basis. Here, the authors based on experience have adopted even number of neurons in each hidden layer. A few reported studies related to the computation of effective thermal conductivities of fine-grain soil [20] and textured soils [14] have used the ANN models. Although the models use the ANN methodology, the deep learning (DL) implementation is not employed. Both the networks, as mentioned earlier, are only limited to one or two hidden layers which limit the ANN's ability for forecasting the scenarios other than for which it is trained [15]. A prerequisite for the implementation of DL is the availability of large and varied data set, as shown in Fig. 1.   Table 1 shows the errors corresponding to various combinations of network design. A maximum of 3 hidden layers and ten neurons in each layer is used to build the network. The three standard errors are calculated, namely the MSE, MAE and R 2 . The MSE is chosen as the basis for the selection of the network. The network corresponding the minimum training testing difference has opted for scenario forecasting. The last column of the table shows the difference in each combination of the network. The network with three hidden layers of 8 neurons each offers the least difference and therefore is chosen as the network for the scenario forecasting. However, an interesting observation is made during training and testing. For every single run of the network training, a slightly different result is observed shown in Table 1(see 8-8-8 network sets). The discrepancy arises due to each trained network has a slightly different weight assignment to the connections. Thus, each trained network is different in connection strength but ultimately tries to converge to training data set with each iteration of training called epochs. For the networks in the study, the networks are trained for epochs between 700 to 900. The issue related to overfitting is avoided by selecting the epoch manually where the first minimum is reached.

ETC computation with DL
To test the developed neural network based on deep learning algorithm with two environmental input parameters (Sr, n) and one compositional (mq) is applied for two different sand types with different mineral composition. The solid skeleton of three-phase unsaturated soil is primarily made up of minerals, such as Quartz, Feldspar, Dolomite, Calcium polymorphs and Mica. The thermal conductivity of Quartz varies between (7.2-8.8) W/m.K at room temperature and is the highest among the other minerals with lies between (1.7-2.7) W/m.K. Therefore, the amount of Quartz in the sand is a controlling parameter and is included in many empirical and semi-empirical models [7,16].
The mineral composition of the sand was determined semi-quantitatively by the X-ray diffraction (XRD) analysis. The first sand is pure Quartz> 99%, and the second sand is composed of 70% quartz with other minerals such as 10% Calcite and 20% Albite. Thermal conductivity measurements were performed at room temperature and with varying degree of saturation. A detail description of the experimental program and measurement technique is given elsewhere [21].

For pure Quartz sand
To predict the thermal conductivity of the pure quartz sand, the trained neural network with three hidden layers and eight neurons in each layer is selected. The model corresponds to the minimum MSE error, as shown in Table 1. A total of five runs are performed with the network, and each network is then applied to predict the thermal conductivity of sand with varying saturation degree. Fig. 4 shows the ANN predicted values plotted against the degree of saturation for the pure quartz sand. All five predictions from the neural network model show a minor deviation but are close by the experimental value. As the model is trained with three inputs which include the quartz content, the ANN model produces a very close correlation with the experimental result. At lower saturation, as shown in Fig. 4, the model predicts the ETC with better accuracy, but as the nonlinear behaviour set in motion, the ANN predicted results depreciate in terms of accuracy (ANN1, ANN2 and ANN4). However, the other two trained models ANN3 and ANN5 produces an accurate prediction. It could be deduced from the results that the average of all the trained ANN networks is closer to the experimental results.
The inability of the model to produce a better prediction at higher saturation value is the availability of dataset. The model is trained with all types of soil, ranging from silty clay to coarse sand. As the model is trained for a limited number for high saturated sand, the forecasting of these scenarios is not accurate. A better data set with more sand type and input parameters such as the shape factors of the grain could significantly improve the model prediction and will be reported elsewhere.

For mixed mineral sand
Here, the trained ANN model is applied to predict the change in the ETC of sand containing other minerals such as calcite and albite. The amount of Quartz in the sand in about 70%. The three input with three hidden layers of eight neurons with one output model is employed here too. Five different runs are performed for, and the model predicted values are plotted in Fig. 5. The ANN predicted results show a large scatter at all saturation levels, but the accuracy depreciates with increasing degree of saturation. Although the models predicted a nonlinear trend, but are far from the experimental results. The model with three input parameters is not sufficient to capture the behaviour of mixed mineral sand.
From the above-performed computations with different ANN models, it is self-evident that more input parameters and a more extensive training dataset are required for accurate and precise prediction of ETC of soil. The input parameters could include the clay, silt and sand contents chosen for machine learning method [20] or based on grain size distribution and size and shape factors with more mineralogical information as implemented in a

Conclusion
In this paper, an artificial neural network model based on deep learning algorithm with Adam optimiser has been implemented. Following conclusions could be drawn from the study. 1. Soft computation methods such as the ANN are capable of predicting the nonlinear dependencies from small training dataset with significant accuracy. 2. The deep neural network outperforms the single or double hidden layer networks in terms of accuracy. 3. The ANN model with two environmental parameters and one mineralogical composition can predict the ETC of quartz sand with reasonable accuracy. 4. The ANN model fails to provide an acceptable prediction for ploy mineral sand. A sizeable dataset with variation in controlling parameter is required for better training of the ANN. As many studies have shown in the past, the ANN models are superior for interpolation rather than extrapolation, a few data with extensive scatter covering the lower and upper bounds are required.