Artificial Intelligence (AI) Framework for Multi-Modal Learning and Decision Making towards Autonomous and Electric Vehicles

An electric vehicle with autonomous driving is a possibility provided technology innovations in multi-disciplinary approach. Electric vehicles leverage environmental conditions and are much desired in the contemporary world. Another great possibility is to strive for making the vehicle to drive itself (autonomous driving) provided instructions. When the two are combined, it leads to a different dimension of environmental safety and technology driven driving that has many pros and cons as well. It is still in its infancy and there is much research to be carried out. In this context, this paper is aimed at building an Artificial Intelligence (AI) framework that has dual goal of “monitoring and regulating power usage” and facilitating autonomous driving with technology-driven and real time knowledge required. A methodology is proposed with multiple deep learning methods. For instance, deep learning is used for localization of vehicle, path planning at high level and path planning for low level. Apart from this, there is reinforcement learning and transfer learning to speed up the process of gaining real time business intelligence. To facilitate real time knowledge discovery from given scenarios, both edge and cloud resources are appropriately exploited to benefit the vehicle as driving safety is given paramount importance. There is power management module where modular Recurrent Neural Network is used. Another module known as speed control is used to have real time control over the speed of the vehicle. The usage of AI framework makes the electronic and autonomous vehicles realize unprecedented possibilities in power management and safe autonomous driving.


Introduction
Among the new technological trends have been arisen in the last decade, Autonomous Driving gained a lot of attention, with significant effort and resources have invested by both academia and enterprises. The breakthroughs in self-driving cars made possible by the emergence of novel algorithms and real-time computing systems in the field of Artificial Intelligence, Deep Learning and Cloud computing. In this paper we build an AI based API that governs autonomous driving by exploiting diversified set of sensors, multi-modal learning, edge and cloud computing in real time to ascertain traffic, making well informed decisions on movements and steering with highly accurate judgements. It is a software product working at application level dealing with data of different sensors associated with autonomous vehicle. It facilitates multimodal learning as the it includes different deep learning phenomena. In fact, it has many deep learning models for various co-ordinated functions. It is a data science approach with AI playing central role in making highly accurate decisions on autonomous driving. It makes use of local resources, edge resources and cloud resources based on the requirements without compromising the speed or real time decision making. The AI framework with its rich set of API facilitates big data analytics to have comprehensive business intelligence (BI) that drives home required decision making process. Latency is an important measure used by the proposed system. It needs to compute a threshold dynamically based on the runtime traffic conditions. However, it needs to have certain acceptable lower bound and upper bound. However, it is dynamic in nature and determined on the fly to assist real time decision making. The offloading of storage and processing to edge computing resources or cloud computing resources is sensitive to the latency needs of the situation in question. The AI based framework with number of deep learning models for plethora of purposes has integrated and holistic E3S Web of Conferences 309, 01167 (2021) ICMED 2021 https://doi.org/10.1051/e3sconf/202130901167 approach towards real time and accurate decision making. It should be so as the vehicle is expected to drive itself without a human expert driver. In addition to the driving, the proposed system also takes care of data dissemination and data storage for leveraging training data for continuously increasing heuristics or knowledge. Yet another important consideration in the framework is monitoring the battery of electric vehicle and trigger energy saving mode when the energy drops below certain threshold. Therefore, the system is aimed at self-learning as humans do and acquire BI incrementally leading to more intelligent vehicle in realizing autonomous driving.

Related Work
In the contemporary era, it is the time for electric and autonomous vehicles to have strong foundation to technical feasibilities. In terms of AI technologies that cater to the autonomous driving and smart energy management, there are immense possibilities as found in the state of the art. Erick and Folly [1] focused on power flow management in charging stations for electric vehicles to leverage performance through reinforcement learning that employs dynamic programming paradigm. Q-learning and Markov Decision Process (MDP) are used for power flow management. They intended to use deep reinforcement learning in future for further improvement. Jinil and SofanaReka [2] on the other hand used deep learning techniques in order to ascertain the power requirements of electric vehicles. Based on the driving range, the energy of the battery gets depleted. It is the major issue that needs to be considered in electric vehicles. Therefore, it is essential to predict power demands of the vehicle and monitor and control distribution of power supply to different parts of the vehicle. They built an algorithm based on Modular Recurrent Neural Network (MRNN). They intended to improve it in future with more intelligent approach with different parameters such as distance from vehicle to nearby recharge station.
In [3][4][5] the authors investigated on speed control of electric vehicles based on AI methods. They intended to leverage performance of Switched Reluctance Motor (SRM) which is widely used in such vehicles. They used a hybrid approach that contains techniques such as Direct Torque Controller (DTC), Adaptive-Neuro Fuzzy Inference Strategy (ANFIS) and fuzzy logic. In future, they intend to improve their hybrid model by including 5 phase SRM. Grigorescu et al. [6] reviewed different deep learning techniques that are widely used in the research of autonomous driving. They used different techniques such as deep CNN, deep RNN and deep RL to understanding driving sense with localization, semantic and instance segmentation, bounding box like object detection and so on. It also focused on safety deep learning to realize autonomous driving. It establishes feasibility of using deep learning for AI based self-driving cars.
Asaii and Sathiakumar [7] introduced a new controller based on Artificial Neural Network (ANN). They investigated on different neural networks and their efficiency in controlling electric vehicles. Talamini et al. [8] investigated on the techniques used in autonomous vehicles and the impact of rules of using such vehicles. They opined that there are many ethical and moral issues involved in the usage of autonomous vehicles in the real world. As the autonomous vehicles use novel approaches in communicating with other road users, it is essential to think about changing existing traffic rules. They further investigated on the usage of RL for learning optimal behaviour for autonomous vehicles. They came to a conclusion that deep learning is relevant for autonomous vehicles and AI-driven approaches are feasible. Litman [9] investigated on the impact of autonomous vehicles in the future in terms of planning and transportation. They discussed the potential benefits of such vehicles. The benefits include increased safety, cost savings, increased road capacity, parking cost reduction, reduced pollution and energy consumption and support for vehicle sharing.
Chen [10] studied the need for deep learning for selfdriving cars. They suggested image processing where mapping of image sequences to driving actions with deep learning based computer vision method. They identified some actions like left, right, brake and accelerate. Identification of stop sign, identification of pedestrians, feature selection and communication need to be part of computer vision application. They suggested AI based deep learning algorithms to realize autonomous cars. Kellehar [11] investigated on the ruse and recycling of batteries of electric vehicles. They focused on both individual batteries and also energy storage systems in terms of reuse. They analysed cost dynamics associated with reuse of batteries. They also studied on the environmental impacts of using such vehicles. Kogan et al. [12] proposed an architecture that is useful for testing learning based autonomous vehicles in terms of control design and advantages. It has both online and offline learning techniques vehicle controller testing.
ILi Haiying et al. [13] proposed a methodology to monitor and manage battery of electric vehicles intelligently. It includes identification of battery faults, performance analysis and intelligent management. It also has features for security control with respect to safety. In [14][15][16] authors investigated on deep learning based AI for self-driving cars in terms of chances and challenges. They found challenges such as datasets creation and usage with completeness, implementation of neural networks and usage of transfer learning. These challenges are related to functional safety of the autonomous vehicle. They intended to devise a common solution to solve these problems in future. Sanguesa et al. [17] studied the growth of electric vehicles, technologies associated with them and challenges. They found many advantages with such vehicles such as zero emissions, simplicity, reliability, cost, comfort, efficiency, accessibility, driving range and charging time. They found that AI based communications leverage functionality of electric vehicles. In future, they intend to use AI based solution in the smart cities use cases. In [18][19][20][21][22] authors have been proposed image transformation and classification techniques used to classify features of the real time autonomous and electric vehicles images and also provided details about real time datasets.
Beaudet et al. [23] focused on "cost effective and energy efficient solutions for recycling end-of-life electric vehicle batteries". They investigated on the economic and environmental drivers to the usage of electric vehicles in future. They studied the battery life cycle, its reusability and intelligent management to realize more benefits in such vehicles. They envisaged that electric vehicles will become an important trigger towards environment safety in future. Yamamoto and Suganuma [24] opined that localization is very essential technique for autonomous vehicles. In this regard, they developed an algorithm for location to leverage performance of such vehicles. Their method for localization is found to be useful in improving vehicle performance in terms of self-driving. From the literature, it is understood that there are three drivers for further research on autonomous vehicles. First, electric vehicles are more likely to be used as autonomous vehicles. Second, deep learning methods such as reinforcement learning are found feasible to deal with monitoring and intelligence management of battery power and energy needs of vehicles. Third, deep learning techniques are found feasible to form AI based solution to have effective autonomous cars with selfdriving.
The main objective of the research is to build an AI based API that governs autonomous driving by exploiting diversified set of sensors, multi-modal learning, edge and cloud computing in real time to ascertain traffic, making well informed decisions on movements and steering with highly accurate judgements.

Methodology
There are many considerations while dealing with Electric Autonomous Vehicle (EAV). They include speed control, driving path learning, power management and a host of other activities that are essential to learning path and movement in the current state of observations. The data collected from sensors and cameras are used for real time processing. Edge resources are used in order to reduce latency in processing. Cloud computing resources are also used in order to have the actions to be offloaded that are not latency-sensitive. There are many kinds of deep learning and machine learning techniques required to realize autonomous driving. However, the broader outline of the methodology is presented in this section. The usage of sensor data, image processing modules, machine learning and deep learning modules, local resources, edge resources and cloud resources work in coordination in order to have efficient control of the vehicle. The vehicle motor named Switched Reluctance Motor (SRM) is controlled using different techniques such as fuzzy logic, Adaptive-Neuro Fuzzy Inference Strategy (ANFIS) and Direct Torque Control (DTC). Power monitoring and monitoring of the vehicle is made using a modular Recurrent Neural Network (RNN) as it has memory to learn from continuous and time-series data. This module helps in controlling power usage as the EAV functions based on battery power. There are many deep learning based methods that are used to make autonomous driving decisions of EAV.
As presented in Figure 1, there is deep learning based localization and perception module that is essential. Unless, EAV knows where it is, it cannot take moving decisions. The perception conceived from the live camera image of the current scenario is very important. Then the EAV needs to know the further details pertaining to traffic condition and possibility to have high level planning. The high level path learning is based on deep learning module that helps in making a broad decision on the path. Then on the high level learned path, there is further need for low level path learning that is based on deep learning which ensures that the EAV can make some decision to move on. There is reinforcement learning to update learning outcomes from time to time. The reinforcement module is indispensable as the traffic scenarios are very dynamic and the knowledge gained from the other modules need to be updated constantly to make adjustments in driving decisions. The learning patterns saved to cloud are reused with transfer learning. Unless transfer learning is used, it becomes very difficult to reduce latency. High performance in terms of real time learning and moving is possible or in other words is supported by transfer learning which is associated with machine learning and deep learning modules.  As presented in Figure 2, Long Short Term Memory (LSTM) module which is enhanced and used in different modules such as perception and localization, high level and low level path learning modules. Since LSTM has memory facility, it is best used to have time-series data processed efficiency. The reuse of data and patterns that are accumulated from time to time helps in learning in real time and make decisions in real time. (1)

Runtime
(3) The SRM of EAV needs to be monitored and controlled in terms of speed. The module presented in Figure 3 takes care of this. There are different techniques used in order to characterize and control the speed of the vehicle in accordance with traffic conditions. Broadly, fuzzy logic, Direct Torque Control (DTC) and radial basis function network-based adaptive fuzzy system (RBFN-AFS) work together in order to achieve better control of the speed of the EAV. There is a technique to quantify flux linkage of SRM and there is co-energy strategy that extracts stage torque attributes. There is a phenomenon known as self-arranging learning calculation that is used to help RBFN-AFS to learn SRM faster including its electromagnetic attributes. The main architecture and the speed controlling modules are seamlessly integrated to have a combined system that works efficiently.
As presented in Figure 4, the modular RNN is one of the deep learning methods that is employed to obtain power requirements of the vehicle. Here G (i) is block representing recurrent layer, G (k) is input to the layer Z(k) is output of the layer and U (i) is bias weight vector. Based on the information about power supply possibilities, it makes important observations and coordinates to function certain power-consuming modules. The aim of the modular RNN is to consider electricity-driven context of the EAV and ensure that power is managed efficiently. Here the usage of RNN is very important because it has memory structure that is reused and thus time-series data is processed more efficiently when compared with other deep learning techniques.

Results and Discussions
The proposed system (AI based framework for API to realize autonomous driving) which is based on multimodal deep learning has its results. The proposed system is to have a reusable and out of the box solution (framework) that can be integrated with autonomous vehicle (car) for automatic driving. In other words, software product that is made up of AI based API for autonomous driving is the ultimate outcome of this proposed methodology. It is a deep learning based solution that has multiple models and algorithms. It supports data analytics in real time with appropriate usage of edge and cloud resources. It has mechanisms to deal with security issues resulting in end to end secure communications. The system becomes rich with

Conclusions
Self-learning as humans do and acquire BI incrementally leading to more intelligent vehicle in realizing autonomous driving. There are many considerations while dealing with Electric Autonomous Vehicle (EAV). They include speed control, driving path learning, power management and a host of other activities that are essential to learning path and movement in the current state of observations. The data collected from sensors and cameras are used for real time processing. Edge resources are used in order to reduce latency in processing. Cloud computing resources are also used in order to have the actions to be offloaded that are not latency-sensitive. There are many kinds of deep learning and machine learning techniques required to realize autonomous driving. However, the broader outline of the methodology is presented in this paper. The usage of sensor data, image processing modules, machine learning and deep learning modules, local resources, edge resources and cloud resources work in coordination in order to have efficient control of the vehicle. The vehicle motor named Switched Reluctance Motor (SRM) is controlled using different techniques such as fuzzy logic, Adaptive-Neuro Fuzzy Inference Strategy (ANFIS) and Direct Torque Control (DTC). Power monitoring and monitoring of the vehicle is made using a modular Recurrent Neural Network (RNN) as it has memory to learn from continuous and time-series data.