A SURVEY ON DIFFERENT ALGORITHMS USED IN DEEP LEARNING PROCESS

.


Introduction
Deep learning is one of the machine learning procedures that imparts computers to organize the natural behavior of a human being like driverless cars [1] which should sense the stop sign, a pedestrian from a lamppost, intruders and all hurdles by its own. It forms as a mainsource to control voices in devices likes handsfree speakers, cell phones, tablets, TVs, etc. [2,3], In addition to the above application it is also used in fraud detection [4], email/spam filtering [5], credit scores [6], web-searches [7]as well as well as it gives solutions for abundance of complex data from diverse sources like visual, audio, medical, social, and sensor.Deep learning is also called as a subset of Artificial Intelligence that replicates the working of the human brain and copies the ability of processing data in objects detection, speech recognition, languages translation, and decisions making.
In deep learning, modelling is done by a computer model that itself is designed to classify the tasks directly from sound, text or images. Deep learning models has attained its state-of-the-art accuracy, in such a way that it has the possibility to even overrule human-level performance. Computer Models are accomplished by using a large set of labeled data and combining the principle of neural network architectures which basically contains multiple processing layers. The capacity to develop large numbers of features in simple procedures makes deep learning very prevailing when it is dealt with unstructured data. Eventually, deep learning algorithms areoverindulgence for less complex problems as they need access to a huge amount of data to be 1Correspondingauthor: kchelladurai05@gmail.com E3S Web of Conferences 387, 05008 (2023) https://doi.org/10.1051/e3sconf/202338705008 ICSERET-2023 operative. For example, ImageNet, a standard deep learning models is trained particularly for wide-ranging image recognition, and it has the feasibility to access more than 14 million images.
The success of deep learning in the recent years resides in the image recognition and natural language processing over other machine learning methods [8]. Still the neural networks that was processed with multiple layers along with hidden layers leads to chaos for the beginners as well as researchers. This distorts the main purpose of deep-learning algorithm and its simplicity in solving complex problems [9]. The achievement of deep learning is entrenched in the capability of deep neural networks to acquire descriptors of data with different levels of ideas without human intervention [10]. One of the chief assessments of machine learning algorithms is that it is deficient of innovative laws, understanding, and information that has been ascending from their usage. This is of the fact that the algorithms utilized in the machine learning are essentially machine-built models appears to be complex for human understanding. Hence a handsome of knowledge of deep learning algorithms and its approaches are required for everyone to explore many new facts in real life scenario. This review gives a wide insight of the concept, different algorithmic methodused in the deep learning process.

Fig. 1. Concept of Deep Learning
Deep learning being a part of machine learning algorithms which utilizesmultiple layers to gradually extract higher-level characteristics from the raw input. It mainly focuses image processing, in which the perception is divided into several layers. The higher layers classify the notions similar to human such as digits or letters or faces and lower layers mightcategorize the edges.Deep learning process is to investigate data with a given logical structure. This is attained by the usage of multi-layered assembly of algorithms called neural networks. The purpose of the design of neural network is in such a way that it replicated the function of a human brain.
Similar to our brain, neural networks are modelled to recognize patterns and categorize different types of information received from the real life and trained to complete the similar tasks on data. Each and every time our brain obtains new information, it has the capability to relate it with a known object, which is the same concept in deep neural networks also. Such networks perform different tasks like clustering, classification, regression on large sets of datas to sort the unlabeled data according to its matching capacity among the wellknown data set. To make it still clear, here are few methods which have inimitable abilities that thrive the deep learning models to solve tasks that machine learning models struggles to solve.

2.1Activation Functions
Activation functions are similar to a light switch, representing two terms ON, OFF, which suggests the nodes in the neural network to activate or not. Different types of activation functions are known but the most commonly used one is Rectified Linear Unit function (ReLU). Figure 2. represents the Model resembling a Perceptron's in neural network using activation function.For example,in this model, X 1 , X 2 , …, X n are binary inputs given as a vector) with precisely the same number of weights W 1 , W 2 ,…, W n . Here the inputs and weights are multiplied and summed up. This was denoted as sum  called as the pre-activation.After taking the weighted sum, activation function, sigma is applied and it is also called as step function.

Cost Function
A cost function is a measure of in what waythe neural network is beneficial with respect to the values that it predicts compared to the actual values. This function is inversely proportional to the eminence of a model i.e. if the model is good the cost function is low and vice versa. Its main persistence of a cost function is to optimize the value. By minimalizing the cost function of a neural network, one can accomplish the optimal weights and parameters of the model which in turn exploits its performance.

Backpropagation
Neural Networks study the chosen function by means of large amounts of data by making use of iterative algorithm called backpropagation. This closely bonds with the cost function. This algorithm is trained to calculate the forward phase for each input-output pair, then the backward phase for each pair, again combine the distinct gradients and apprise the weights based on the learning rate and the total gradient.

Convolutional Neural Networks
Convolutional Neural Networks is a type of neural network which uses a mathematical function known as convolution. Its purpose is first to take the input images then allocates specific position with various features of the image and predicts the output.This makes them in finding the spatial relationships between the dataand captures the pixel dependencies for the entire image [11].

Recurrent Neural Networks
Recurrent Neural Network (RNNs) is one among the neural network,which isextraordinarilyused for consecutive data for its capacity to consume inputs of variable sizes. It contemplates both the present input as well as preceding inputs, so that it can precisely yield a dissimilar output based on the earlier inputs provided to it. This type of neural network is chiefly handled for time dependent datas or sequential datas.

Long Short-Term Memory Networks
Long Short-Term Memory (LSTM) networks is a type of Recurrent Neural Networks which discourses the shortfalls of regular RNNs and it has a short-term memory.It is designed for studying and remembering long Activation function -term information. It is a chain-like process with stacked layers. It is applicable for speech synthesis, time-series estimates, modelling languages and conversion, composition of music's, and development in pharmaceutical.

Weight Initialization
Weight initialization is a method in neural network, that prevent the convergence of trivial solution in its process. When the weight at every point is initialized to similar value like zero then all the units in the network will receive the same signal and each layer act as a single cell. Thus, the weight function randomly initializes all the values nearly to zero not exactly zero. This model is mainly trained for supporting the stochastic optimization algorithm.

Batch and Stochastic Gradient Descent
Batch gradient descent and stochastic gradient descent are two dissimilar methods mainly utilized for computing the gradient. The Batch gradient descent merely figures the gradient for the whole dataset. Its process is much slower for huge number of datasets. It goes well with curved or smooth error manifolds. The stochastic gradient descent is administered for training single sample at a time. This method is less expensive and its computation is quite faster.

Hyper-parameters
Hyper-parameters consist of certain variables which performs two tasks namely controlling the network structure and governing the training of the network. Its parameters include the number of layers, number of hidden units, learning rate, network weight initialization, batch size etc.,

Learning Rate
The Learning rate is a type of hyper-parameter commonly used in neural network. This process regulates the adjustment of the model in retort to the predicted error whenever the weight of the model is updated. The updates are trained in the model depending upon the learning rate, if it goes too low, then there is only minimal update which are completed to the weights through each iteration, thus numerous updates have undergone before attaining the minimum point. When learning rate is too high, unwanted conflicting performance will take place leading to loss function owing to extreme updates in weights, and possibly will flop to converge.

Unsupervised Deep Learning
Deep Learning is simply a group of algorithms resembling the model of a human brain in handling different datas and by the workings of the human brain in processing data and generating patterns which is useful for making decision, for use in decision making, intensifying and enlightening the idea of a single model architecture which is commonly known as Artificial Neural Network. Deep Learning generally called as deep structured learning with differential programming concept belongs to the broader class of machine learning algorithm which is in turn formed by artificial neural network. The method is based on supervised and unsupervised deep learning using multiple layers of neural networks particularly designed for computing and processing data large number of datas. Similarly, Unsupervised Deep Learning is categorized as Self Organizing map, Boltzmann machine and Autoencoders. These are the common algorithms mainly used in deep learning for analyzing and solving problems in image processing, image recognition, segmentations, etc., A deep learning process undergoes two important phases namely training and inferring. First, the training phase, focuses on labeling huge amounts of data and recognizing their identical characteristics. In this step, the system relates thefeatures and learnsto make proper decision on encountering similar data in the subsequent time. Second, in the inferring phase, the process comes to a conclusionand tags the unexposed data with the help of the previous analysis it made during the first step. So, the above algorithms pave way for the solutions to the real-life problems in a better way similar to a human brain. Table 1. displays the different deep learning algorithms, its contributor and year along with its features.

Benefit of Deep learning over machine learning
Machine learning is simply the progression of extraction of information from a large dataset and loading them into the machine for further process. But deep learning was technologically advanced as an alternative method of machine learning. The methodology used in machine learning is to resolve a substantial number of tasks under the control of a human. But deep learning algorithm is competent of generating novel features by itself. In Machine learning, features are to be recognized precisely by users. Again, in deep learning, problems are resolved on an end-to-end basis, but machine learning solves it by dividing the tasks into several small pieces and then the obtained results are bought together to complete it. In all together, deep learning is a process of artificial intelligence, which enables the processing of data similar to a human brain.

Conclusion
The importance of deep learning has a boundless extent owing to the steps it has achieved in the field of science, engineering, technology and medicine. Deep learning is nothing but a subdivision of machine learning algorithms that basically utilizes multiple layers to increasingly extract higher-level features from the input. Deep learning algorithms can examine and study from transactional data to recognize different patterns that specify possible fraudulent as well as certain criminal activity. It provides varies applications such as in computer vision, image processing, image segmentation, driverless car, recognition of speech, medicine and in various other applications that can progress the efficacy and usefulness of examining analysis by removing patterns and indications from video and sound recordings, images, and documents, which benefits implementation of quite large amounts of data more rapidly and precisely. This review provides us a brief knowledge on the concept of deep learning, different approaches for deep learning algorithm, its advantage over conventional machine language and its practical application.