Prediction of Indoor Air Quality using Long Short-Term Memory with Adaptive Gated Recurrent Unit

. There is significant evidence that the COVID-19 virus may be spread by inhaling aerosols leading to risk of infections across indoor environments. Having said that, it is clear that the formulation of indoor air quality (IAQ) needs to be carefully examined. In general, IAQ can be controlled by proper ventilation system across buildings. Nevertheless, different buildings require different mechanistic approaches and it may not be an effective solution for the buildings. Thus, statistical approaches have great potential to evaluate the IAQ in real occupied buildings. Numerous machine learning (ML) techniques were introduced to forecast the indoor environmental risk across buildings. However, there is inadequate data available on how well these ML techniques perform in indoor environments. Recurrent neural network (RNN) is a ML technique that deals with sequential data or time series data. However, the RNN model gradient tends to explode and vanish, leading to inaccurate prediction outcomes. Therefore, this study presents the development of a time based prediction model, Long Short-Term Memory (LSTM) with adaptive gated recurrent units for the prediction of IAQ. Using an advanced LSTM model, the study focuses on the performance of the prediction accuracy and the loss during training and validation. Also, the developed model will be assessed with other RNN models for data validation and comparisons. A set of particulate matter (PM2.5) dataset from commercial buildings is assessed, preprocessed and clean to ensure quality prediction outcomes. This study demonstrates the performance of the hybrid LSTM model to remember past information, minimize gradient error and predict the future data precisely, ensuring a healthier indoor building environment.


Introduction
The sick building syndrome describes a situation in which building occupants experience acute health and/or comfort effects that appear to be linked to time spent in a particular building, but where no specific illness or cause can be identified. The complaints may be localized in a particular room or zone, or may be spread throughout the building. Also, indoor air quality (IAQ) describes how inside air can affect a person's health, comfort, and ability to work. It can include but not limited to temperature, humidity, mould, bacteria, poor ventilation, or exposure to other chemicals. Indoor air pollution has received little attention in the past compared with air pollution in the outdoor environment. Nowadays, indoor air quality has becoming world-wide concern especially after the pandemic of COVID-19 where people mostly spend 90% of their time indoor. Therefore, IAQ is ranked as one of the top five environmental risks to global health and well-being [1].
Research in the field IAQ has a long tradition in the environmental engineering field. However, the determination to design such robust forecasting model is technically challenging, especially when dealing with non-linear data. Recently due to the pandemic of COVID-19, research have shown significant interest in modelling prediction model for IAQ monitoring or statistical model which has been referred as data driven model. The type of model used usually require the concept of input and output of the dataset without needing the mechanistic model of the buildings. The model is basically constructed based on the sequential data.
In this modern day, more robust recurrent neural network (RNN) has been introduced in order to solve the vanishing and exploding gradient of the conventional RNN. Before that, RNN processes to stop those events from occur are by having minimum clip at the gradient of RNN during the process of backpropagation. But, still this idea of solution did not able to memorize long term of the time series sequential data. This is where the Long Short-Term Memory (LSTM) comes in. The model is one of the variants of the RNN model. LSTM has solid abilities to memories short and long-term series of historical data. To date, there are still few of published research that deploy LSTM approach in the environmental engineering problem, especially indoor air quality. Therefore, it is necessary to use LSTM in order to predict short and long term of IAQ historical data. This study examines the ability of LSTM to enhance the data driven method in order to predict better accurate forecast and reduce loss during the deep learning training.
This paper organized as follows, in Section 2, discussed the LSTM network architecture, Gated Recurrent Unit (GRU) model and datasets of the IAQ. Further, the section includes the deep learning evaluation metrics to calculate the model performances. Results and discussions of the application of the LSTM and GRU model studies are presented in Section 3. Followed by Section 4, deduces the summary of the research. Last but not least, the challenges of the research and future direction of the research are addresses in this manuscript.

LSTM Network Architecture
A vanilla and efficient model for IAQ prediction is introduced in this research. The IAQ parameters from a commercial building in Kuala Lumpur, United State (US) Embassy is used as an input dataset. The total available dataset was used for training and testing using LSTM and GRU prediction model. LSTM is a variant of RNN which deal with vanishing and gradient problem. The highlight feature of LSTM model is it has layer which is called memory cells. The model also consists of input layer and output layers. Each memory cell has three input gates which control and maintain the memory state (St). The gates are the forget gate (ft), input gate (it) and output gate (ot). The overview of the LSTM model architecture shown in Fig.1.
LSTM layer also consists of hidden state, which is known as the output state. At every timestep t, it contains the output of the LSTM. Other than hidden state, the cell state also part of the LSTM layer. The purpose of the cell state is to store the present information of time step t that the layer learned in the previous time steps. Every time step, cell state will decide either to keep or remove the information that state learned. The cell state controls and updates the information by using the three gates mentioned earlier. Also, at every time step, the information xt is comes from the output of the previous time step, output ht. Based on the references [2][3][4][5] the gates in the cell state have their own responsible. The purpose of the three gates is mentioned in the Table 1. Table 1. Purpose of gates in LSTM cell state.

Gate Role
Forget Decides which information need to be removed from the current cell state Input Decides which information need to feed to the current cell state Output Decides which information will be used in the current cell state Overall architecture of the LSTM model is based on the hyperparameters defined in the Table 2, to forecast the time steps by one step and update the new information at next memory cell. The network architecture is tuned according to the Tensorflow packages available. In order to train the proposed network with the IAQ parameters, the overall mechanism of LSTM is shown in the Fig.1 according the [3]. The fully connected layers are then connected to the proposed model.

Gated Recurrent Unit Model
After 17 years of the presence of vanilla and improved version of LSTM [3,5,6,18], another variant of RNN was developed. Gated Recurrent Unit (GRU) was introduced to simplify the computational power of LSTM while improving the output from the RNN model. At the same time, GRU preserves the LSTM performance while optimising the network layout. The GRU network structure, which can address the prediction issue of long interval long delay time series, only has two gate structures compared to the LSTM network structure's four. The update gate is used to regulate how much information from one moment is incorporated into the current moment. The reset gate is used to regulate how much of the information from the previous moment is ignored. The output of the reset gate rt at time t. Meanwhile the output of the update gate at time t and ht is zt. ht-1 is the output at time t respectively. The input at time t is xt and is the activation function at each gate. The computation for GRU is referred in equation (1).

Hybrid LSTM with GRU model
The LSTM model has been proven in terms of their capacity to remember long term and short-term predictions. GRU on the other hand, has significant computational power due to the simplified gates architecture while preserving reliable performances. Combining these two models into one powerful model, will realize a powerful sequential forecasting performance, deep learning approach using the LSTM-GRU. This is where a hybrid LSTM with GRU comes in, to improvise the performance of the IAQ prediction model. Training data will be fed into the LSTM model first, then continue to be fed into the GRU model. Finally, the data will go through the output layer which contains the dense layer. The overall flow of the LSTM with GRU model is presented in the figure below.
In the LSTM with GRU model, the LSTM has two layers, each layer has 50 hidden units with tanh activation function. Then the training data will be fed into dropout layer before it is pass to the second layer of LSTM. Dropout layer is set at 20 percent so that it can regularize the output of data from the previous layer. The overall parameter tuning can be referred to Table 2. This process intents to prevent complex computation during the training process [8,9].
The GRU section also has two layers for training. Similar to the previous LSTM layer, it also contains 50 hidden units and a dropout layer for each GRU layer. Finally, the data will be passed to the dense layer for the deep learning neural network learning process. This is where the process of the learning and tuning of the overall model dimension occurs. The output of the training model will be validated and test data in order to evaluate the performance of the LSTM with GRU model.

Prediction model evaluation metrics
Evaluation metric is one of the crucial stages of building a good prediction model. The purpose of an evaluation metric is to benchmark the desired prediction model efficiency. Based on the evaluation metric, one can deduce the performance of any prediction model [10,13]. Therefore, to demonstrate the performance of the IAQ prediction model in this research, root mean square error formulation in equation (2) is assessed. (2) In the equation (2) the data point of the dataset is defined as n, the predicted PM2.5 is represented by y'. Meanwhile y is the observed PM2.5 at the time i, y is the mean of the actual of PM2.5 at the US embassy. In this research, Python Keras library and Tensorflow were used as the platform for the prediction model computation and development.

Indoor air quality data description and analysis
The dataset used in this research comprises of the database from the United State Embassy and Consulate building in Kuala Lumpur. The set of data collection consists of the particulate matter 2.5 micron (PM2.5). In general, PM2.5 which is referred as fine particles are one of the high risk to the deterioration of respiratory system [16]. Plus, PM 2.5 are harmful because of their size which allow them to pass to the lung and even blood circulation directly. Back then in 2008 in Beijing, US Embassy began air quality monitoring in their premises in order to make sure their citizen receive accurate information related to the air surrounding air quality [12].
In 2015 the United State Environment Protection Agencies (EPA) dealt and agreement to step up the scale of the PM2.5 monitoring at US embassies where the local PM2.5 data were not available [8]. Therefore, US Department of State is working together to monitor and control the indoor air qualities in U.S embassies around the world. In Kuala Lumpur, the consulates effectively links the information of the IAQ to the EPA AirNow International website [8].
All the daily and hourly averaged PM2.5 from the website mentioned previously is scrapped and downloaded through June until November 2022. The website, AirNowTech checks the lowest, highest and the performance of the PM2.5 reading (Analysis of fine particle pollution data measured at 29 US diplomatic posts worldwide). The dataset also provides a column for the Air Quality Indices (AQI), indicating the air quality catogeries ("Good" and "Moderate"). The daily PM2.5 concentration in the dataset were calculated using formula provided [8]. It is stated that the between 18 to 24 hours should have valid data in the measurement. On the other hand, for hourly data measurement (1-h average), it uses the same measurement validation. And the formula calculated for 1-h average, it used the EPA NowCast theory [8]. The data consists of 3,236 of hourly sampling for the past five months. 2,589 data were examined as training and 647 data used as validation respectively.

Results and Discussions
The simulation of the IAQ prediction model was done using Keras, Tensorflow and Numpy packages in the Python library. In order to have a good prediction training process, data standardization is implemented to leverage the overall data into a simpler feature range. By having this process, one can transform the scale of the real-world data into a scale of zero to one. By having this method, the learning algorithm performs better when the actual numerical data is scale to range of value which have the most precision.
In this study, the input of the architecture was initialized using 2588 data sampling and it was simulated using 647 validation data using a set of time step (25 percent of the final time series), and it was continued by repeating the remaining network learning of the LSTM model and LSTM with GRU model. The simulation of the validation results are illustrated in Fig.  4 for LSTM model and in Fig. 5 for LSTM with GRU model with orange line colours.
Quantitatively, the results of the validations predictions are presented in the Fig. 6 and Fig. 7 with orange line colours. From Fig. 7, the LSTM with GRU model shows considerable precision in term of the average of the pattern flows.
The RMSE was computed in order to evaluate the performance of the actual PM2.5 concentration at the embassy building. These two assessed performances were computed from the actual PM2.5 value. The results presented in Table 3 show the LSTM model RMSE is 0.31862, while the LSTM with GRU model shows RMSE 0.20345. Also, the computational process time during both models training did not give any differences during the experimental simulation. In deduction, the LSTM with GRU model outperformed the LSTM model with lower RMSE value.

Conclusion
This paper was arranged to develop a hybrid and improvised version of LSTM for IAQ prediction using vanilla LSTM and hybrid LSTM with GRU model. The research has investigated the idea of RNN and its variances. The hybrid of LSTM with GRU not only outperforms the conventional LSTM, but also combines both RNN variants to improvise loss function of the time series prediction algorithm. With the evolution of ML architectures, the hybrid version of LSTM with GRU is potentially to be deployed in indoor space environmental problems.
For upcoming study, engaging multivariate input features can be included in order to find the relationship impact of poor IAQ based on the prediction performance. Furthermore, the escalation of the RNN variants in order to have promising performance with significant lower computation power will be more challenging in the process of development of building and environmental forecasting cases.