Anomaly Detection for Predictive Maintenance in Industry 4.0- A survey

Maintenance and reliability professionals in the manufacturing industry have the primary goal of improving asset availability. Poor and fewer maintenance strategies can result in lower productivity of machinery. At the same time unplanned downtimes due to frequent maintenance activities can lead to financial loss. This has put organizations’ thought process into a trade-off situation to choose between extending the remaining functional life of the equipment at the risk of taking machine down (run-to-failure) or attempting to improve uptime by carrying out early and periodic replacement of potentially good parts which could have run successfully for a few more cycles. Predictive maintenance (PdM) aims to break these tradeoffs by empowering manufacturers to improve the remaining useful life of their machines and at the same time avoiding unplanned downtime and decreasing planned downtime. Anomaly detection lies at the core of PdM with the primary focus on finding anomalies in the working equipment at early stages and alerting the manufacturing supervisor to carry out maintenance activity. This paper describes the challenges in traditional anomaly detection strategies and propose a novel deep learning technique to predict abnormalities ahead of actual failure of the machinery.


IIoT-Industrial Internet of Things:
IoT or Internet of Things has boomed in recent years. IoT allows machines to communicate with each other over the Internet in real-time. One of the application areas of IoT in the industrial domain is called as Industry 4.0. Currently, Industry 4.0 is undergoing a revolution with innovation at its core for those who wish to revamp the organizational processes in the industry [1]. This has led to the upsurge of the Industrial Internet of Things (IIoT) technology wherein industry equipment is largely driven by sensors and sensor data. For any industry, whether it is an automobile or thermal or any manufacturing industry, it runs on the oil of data that the manufacturer can use to gain meaningful insights into his businesses (e.g. to identify a fault via predictive analytics). Also, data can be considered as a critical asset for structural health checks of the equipment used in the industry.
However, in most of the industrial checks, it is difficult to identify alert rules, since the processes are too complex, and dynamic. Today, inadequate maintenance techniques can reduce an equipment's overall productive capacity by five to twenty percent [2]. Studies carried out recently also show that machine downtime which is unplanned or has occurs at last minute due to some faults costs industrial manufacturing organizations an estimated $50 billion each year [3]. Predictive maintenance is one of the major goals of Industry 4.0 and it depends majorly on Anomaly detection. Anomaly detection or outlier detection is a set of major techniques with an aim to identify anomaly patterns that deviate from normal behavior. At the same time, manufacturing processes generate a large volume of data that needs to be monitored in real-time for generating timely anomaly alerts. This paper outlines the need for anomaly detection, summarizes the current research work in this domain and propose a deep-learning based technique for identifying anomalies in an efficient manner.
The rest of the paper is structured in the following way: The next section describes the evolution and need for Predictive Maintenance followed by Section 2 describing the current state-of-art methodology for Anomaly detection in Predictive Maintenance. Section 3 describes the challenges faced in detecting anomalies for predictive maintenance and Section 4 describes some of the current research work done in the area of anomaly detection. In Section 5 the authors propose a novel anomaly detection framework using the power of Artificial Intelligence, its benefits and also present some of the available datasets as case studies. At the end the authors present a conclusive summary about the need of using AI for predictive maintenance. It's a common practice for manufacturers who have been carrying out maintenance techniques for years according to their companies' maturity in dealing with machine failures. Figure 1 describes how maintenance activity has evolved with changing technology:

Figure 1: Evolution of Predictive Maintenance
Predictive maintenance has evolved in a big way that can be characterized as follows:  Reactive Maintenance -This basically allows you to repair after the failure has occured.  Preventative maintenance -This involves carrying out regular maintenance activities to avoid failures. However, many of the times it is done without the actual need of doing it. For example replacing an equipment part after 6 months when it could have run successfully for at least 10 more cycles.  Rule-based predictive maintenance -Maintenance is carried out on basis of hardcoded threshold rules and alert is sent in case a measurement crosses the thresholds.  Machine learning-based predictive maintenance -Makes the use of advanced analytics and machine learning techniques to predict when the next failure would take place and accordingly carry out maintenance ahead of time.
Some of the benefits of Predictive Maintenance (PdM) are as follows:  Avoid unplanned downtime  Reduce planned downtime  Increase productivity  Optimize the use of maintenance resources  Improve asset health and performance  Increase customer satisfaction 2.Anomaly Detection for Predictive Maintenance:

Need for Anomaly Detection
One of the biggest issues faced by the manufacturing industries is their processes can identify only 20 % anomalies beforehand [4]. This means that most of the time the anomalies go undetected and the industry are unable to handle them due to ineffective anomaly detection techniques.
Anomaly detection is the process to identify variables or items that do not belong to an expected pattern in the same dataset and is usually unobservable to human eye. Such anomalies which we can term as early signs of failure usually result into equipment breakdown or lead to faults in the working of the equipment. Anomaly detection is tough because:  The current knowledge is not enough to define a class.
 Often, no examples are available in the data to describe the anomaly accurately.
Example: A worn-out roll bearing that leads to the breakdown of the machine.

Anomaly Detection for Condition Monitoring in Machines:
A rotating machine such as a pump or a steam turbine or a non-rotating equipment such as a distillation column will eventually start deteriorating due to regular use and starts generating abnormalities which we can term as 'anomalies' here. That state should not be considered as a case of total shutdown but should be considered as a warning that the machine is not in its optimal state and requires to undergo a maintenance activity.
Condition Monitoring or Statistical Process Control which was introduced in 1924, is an effective statistical technique for quality control and assurance in manufacturing domain. Data in the form of machine sensor measurements are recorded in real-time during the manufacturing process and a graph is then plotted with predetermined threshold limits that reflect the capability of the process. The data points that falls within the thresholds indicates that everything is normal and working operating as expected with some natural variation that is expected as part of the process. If data points fall outside of the threshold limits, this signifies an assignable issue that should be addressed at the earliest before faults occur. In this way, SPC is an efficient method to detect anomalies at early stages. Figure 2 describes the typical lifecycle of condition monitoring in the manufacturing industry. The author Thomas Brand (2017) in his article points out the use of some of the current analytical techniques such as time-based and frequency-based techniques for condition monitoring using vibration sensors [5]. The time domain based statistical analysis gives more idea about the vibration degree of the system over a period and checks whether it is within the permissible limits. The statistical analysis based on frequency allows monitoring of vibration in the form of the fundamental frequency along with harmonic components for fault diagnosis. The accurate tracking of the basic frequency is especially important because the effective values and other statistical parameters can deviate with speed over time. If the parameters deviate significantly from the last measurement, the basic frequency must be verified to avoid false alarms.

Challenges faced by Anomaly Detection:
Researcher Pramit Choudhary (2019) in his article Introduction to Anomaly Detection points out some of the issues in Anomaly detection using traditional statistical methods [6]:  The statistical data contains noise which could be alerted as an anomaly, because the difference between normal and abnormal behavior is mostly not specific.  The characteristics of anomalous or normal may frequently change hence the average thresholds will change frequently.  The anomaly pattern is mostly based on seasonality. Hence this requires more dynamic techniques to understand the change in anomaly trends.
Predictive Maintenance (PdM) due to its varied benefits such as significant savior in maintenance costs, reduction in downtimes, etc. has become a sought-after goal for many manufacturers.
Traditionally manufacturers have been carrying out PdM using SCADA systems manually hard-coding thresholds limits, setting static alert rules and other configurations. This kind of semi-manual technique fails to consider the dynamic and complex behavioral patterns of the machinery or is unable to relate to the contextual data relating to the manufacturing process at large. For example, a sensor on an automobile machinery may suddenly record a rise in temperature. A rigid rule-based system would not take into account that this is due to the fact that the machine is undergoing sterilization and would further trigger a false-positive alarm.
Flovik, V. (2018) in his article "How to use machine learning for anomaly detection and condition monitoring" mention challenges faced by the traditional methods of anomaly detection [7]:  Generation of false positives: The process of having hardcoded anomaly alarm limits tends to send a huge number of false alerts for conditions which in actual are normal situations for a specific sensor. This unnecessarily wastes the time and efforts of the supervisor overlooking the process and also impacts the availability of the sensor.

 Missing anomaly alarms:
There might also be cases of missing alarms for situations that are actually problematic but are not alarmed. This is a more serious situation as the damage is already done and would lead to repair costs and lost production.
The above two challenges have the same root cause as follows as Flovik V.(2018) states: "The health of a complex piece of sensor equipment cannot be reliably judged based on the analysis of each measurement on its own" [7] In Figure 3, author R.Silipo (2019) clearly states that anomaly detection problems do not offer a supervised dataset with labeled classes such as a signal from a normally functioning system and a signal from a system with an analogy. In this case, we can only train a machine learning model on a training set with "normal" examples and use a distance measure between the original signal and the predicted signal to trigger an anomaly alarm [8].

4.Current Research Work in Anomaly Detection
Tang, Z. et al. proposed a unique anomaly detection method for a long-width cable-stayed bridge [9]. Here the descriptions of anomaly patterns were studied using the convolutional neural network (CNN). The authors have implemented a semi-supervised deep learning approach for anomaly classification and further research is needed for unsupervised classification.
Li, X.,et al. in their paper had chief emphasis on data augmentation techniques while carrying out anomaly detection. This was done to get high accuracy even for a small original training dataset as it is difficult to obtain accurately labeled data in real industries [10]. It was observed that individual augmentation techniques achieved better accuracy than when they were used in combination.

Graß, A. et al. proposed a novel Generic Anomaly
Detection for Production Lines (GADPL), for detecting anomalies for a real reflow oven using old sensor data which was recorded over a period of more than seven years [11]. However the work did not study the association of anomalies within multivariate data. The work needed further improvement on how one can correlate varied dimension information within a particular dissimilar measure to find anomalies.
Żabiński, T. et al.in their paper propose a platform that utilizes a toolset for intelligent condition monitoring systems for novelty detection and classification depicting an example of CNC milling tool [12]. However the diagnosis of CNC tool head imbalance was not explored in real time.
Liang, Y. C. et al. proposed to use fruit fly optimization (FFO) algorithm to optimize the threshold values based on historical data to achieve more accurate anomaly detection [13]. However different data sources such as vibration, force data, etc. were not fused for better prediction. Deep learning algorithms were not explored.

Deep Learning Framework for Anomaly Detection:
There is a strong research needed in case of detecting a true anomaly by considering a blend of several sensor measurements to get an accurate and real warning of the situation as depicted in Figure 4.

Proposed Deep Learning technique:
The chief goal of predictive maintenance is to detect anomalies and failure patterns and provide early alerts to the manufacturing personnel. The authors of this paper propose a novel deep learning technique which is proactive in terms that will "learn" and study patterns leading to anomalies and faults in machinery beforehand and proactively warn the personnel about possible issues well ahead of time. Figure 5 describes the steps of the proposed technique. This will benefit the personnel in two ways:  The warnings and alerts can enable efficient maintenance of the faulty components E3S Web of Conferences 170, 0 (2020) EVF'2019  The resources required for maintenance can be optimally utilized.

Figure 5:
Steps in proposed deep learning framework

Benefits of proposed Deep Learning technique for Anomaly Detection
The authors propose to use the combination of AutoEncoders and Long Short Term Memory (LSTM) deep learning model variants for effective anomaly detection in manufacturing domain. An autoencoder is an unsupervised artificial neural network and is used to "learn" from input data. Autoencoders are mostly useful for noise removal in data and for dimensionality reduction which is the case in most of the manufacturing sensor data. Autoencoders are helpful for data representation and visualization in situations where the input data has high dimensionality and cannot be plotted easily. Using the dimensionality reduction feature of Autoencoders we can explore the data in 2D or 3D space. Figure 6 depicts a typical Autoencoder neural network.
However most of the manufacturing sensor data are sequential in nature and are collected over a longer period. Prediction over a longer sequence is difficult due to varying lengths of the input sequence. The timebased ordering of the observations can also make it challenging to consider only required features suitable for use as input to supervised deep learning models. The LSTM (Long short-term memory) architecture consists of an Encoder-Decoder system and can support input sequences of variable length and generate output sequences accordingly. LSTM has the capability to understand the dynamics and complex interactions within the time-series based ordering of input sequences and at the same time store this information in memory.
The style for the names is First Name, typed in italic 10point Times, then Last Name, typed in 10-point Times, with a comma after all except the last author, which is separated by comma + "and". Do not use academic titles.

Case Studies/Available Datasets:
Here the authors describe some of the publicly available sample datasets in the domain of manufacturing where the proposed algorithm can be utilized effectively.

Mill Data Set
The data in this dataset describes experiments conducted from various runs on a milling machine under different operating conditions. The primary motive of this experimentation was to investigate tool wear. Data was captured using three types of sensors mainly the current sensor, acoustic emission sensor and vibration sensor. Readings were recorded at different instances of time [17]. Figure 7 depicts the experimental setup of the milling machine.
Need for anomaly detection in investigating tool wear: In case of milling machine, a sharp tool is always desired to maintain a high-quality product with accurate dimensions and smooth surface finish. A dull one may deform the surface to a great extent and the cost of repair would be high. A dilapidated tool would in turn cause more friction resulting in higher cutting temperatures and other unwanted effects. Therefore, in this case, tool wear can be considered as a true anomaly condition which has to be controlled at the right time before the damage is done. Some of the implications of tool wear are:  flank wear which can be caused due to friction between the tool and the workpiece.  The speed of cutting leading faster damage  intuitive rounding of the cutting edge The above Figure 8 describes the sensor parameters used in the milling experiment. By analyzing the threshold values of these multiple sensors using deep learning models, an alert regarding possible tool wear can be predicted.

Turbofan Engine Degradation Simulation Data Set:
The turbofan or fanjet is a category of airbreathing jet engine that is widely used in aircraft propulsion as shown in Figure 9. In this case study, engine degradation simulation was carried out using C-MAPSS (Commercial Modular Aero-Propulsion System Simulation). Four different conditions were simulated for different groupings of functioning conditions and fault modes. The simulator captured recordings of various sensor channels to diagnose fault evolution [18].   Need for Anomaly Detection for investigating Equipment wear: The above scenario describes the simulation data for aircraft engines throughout their usage. An engine unit may be deployed under different usage conditions from one flight to another. The volume and speed of damage accretion will be different for each engine. It is assumed that the amount of damage accumulated will vary depending on various factors during a particular flight and hence, one must depend on sensor information recorded during each flight. This scenario describes deterioration due to equipment wear based on the usage trend of the engines and may not necessarily be due to any particular fault cause. This justifies the need to use deep learning to study anomalies in the various sensor measurements to monitor the flight engine health during or after the flight journey.

Conclusion and Summary:
Maintenance and reliability professionals in the manufacturing industry are posed with huge challenges, but the primary goal of any manufacturing organization remains constant throughout i.e to improve asset availability.
Anomaly detection in particular, and predictive analytics as a whole, cannot be an afterthought -it must be implemented in your business goals and planning from the beginning. Extensive research is needed in the development of effective anomaly detection techniques which not only alert manufacturers well ahead of time but also abstain them from carrying out unnecessary maintenance activities resulting in wastage of resources. The current statistical approaches fail to consider combination of various parameters that would lead to anomalies in the working of the equipment. The proposed deep learning framework overcomes this by detecting abnormalities and checking correlations while searching for anomaly patterns across thousands of data feeds which is not possible by manual approach. Also, most of the current research work consider semi-supervised approach for classifying anomalies. The proposed framework makes use of autoencoders variant of deep learning which is an unsupervised learning technique. This will help uncover new classes of anomalies. Artificial Intelligence empowers the Industry 4.0 due to its capability to analyse huge volume of data in real time and generate timely alerts for issues that may arise. The framework will assist in the evaluation of the asset health and behaviour and the system as a whole and help identify component degradation prior to its malfunctioning.

Future Scope:
The authors propose to apply this technique in real world manufacturing scenarios in order to check its accuracy in predicting anomalies on real streaming time series data. The algorithm can also be compared against other deep learning model variants for better accuracy.