Sentiment Analysis for Video on Demand Application User Satisfaction with Long Short Term Memory Model

Customer reviews are important information that can be used by providers of goods or services to maintain customer loyalty, understand customer feelings and analyze the business competition. Users use customer feedback on social media or e-commerce as material for consideration before using or buying products. This study aims to conduct a sentiment analysis for user satisfaction of the Video on Demand application in Indonesia with the Long Short-Term Memory (LSTM) model. LSTM, which is a deep learning method that is widely implemented in natural language processing research. Sentiment analysis is applied to find out how customers feel about products on the market. The study results indicate that the LSTM model's implementation for sentiment analysis using two positive and negative labels obtained the value of precision 73.81%, recall 73.81%, f1 score 73.81%. In addition, the accuracy value obtained is 73.90% can be used as a consideration for the company in knowing user sentiment to meet the expectations and desires of customers and keep customers using the service.


Introduction
Deep Learning outperforms machine learning approaches to text classification, such as sentiment analysis [1]. Recurrent Neural Networks is a method of deep neural networks applied for text classification that can connect previous information to the current activity [2]. RNN has a problem in the learning process, namely vanishing gradient, which makes the learning process difficult [3]. Long short-term memory (LSTM) is one of the RNN architectures that can overcome the vanishing gradient problem and has good performance in text classification because it can represent sentences sequentially [4].
Many studies on sentiment analysis on the LSTM model have been carried out, one of which [5] researched sentiment classification at the document level using datasets from IMDB and Yelp. Companies use sentiment Analysis (SA) to understand customer opinions on a product [6]. In addition, the customer opinion can be used to make competitor analysis which is essential in the design of a product to be accepted by the public [7]. Sentiment analysis is applied to identify positive, negative and neutral sentiments related to comments, reviews, tweets, feedback and even harsh words according to the sentiments conveyed by users or customers [8].
This study focuses on aspect-based sentiment analysis for users' satisfaction of the Netflix application in Indonesia with the Long short-term memory (LSTM) model. Implementation of Long short-term memory (LSTM) model to perform classification. Furthermore, the review data will be classified with the polarity of positive or negative sentiment on each customer review.

Word Embedding
Word2vec developed [9] is a word embedding method to process words from large datasets using a neural network model. This method will produce a vector representation of each word. Word2vec has two types of architectural models that can be used to represent word vectors, namely skip-gram and continuous bag-of-words (CBOW) [10]. CBOW is used to predict words (output) around the word (input) and has the advantages of higher word representation quality and increased training speed.

Long Short-Term Memory
Long Short-Term Memory (LSTM) is one type of recurrent neural network (RNN) that has the advantage of predicting and studying sequential data. The LSTM has a memory structure that can remember information for long periods of time and three gates, which are input gates, forget gates and output gates to regulate the flow of information into and out of the cell. The implementation of LSTM in several applications shows good results, especially in speech recognition applications, speech synthesis, language modelling and translation, and handwriting recognition [11]. The formula used for the aspect-level model of sentiment classification is denoted in equations (1) to (6). The formula used to calculate the forget gate is denoted in formula (1).
The next step is to determine the information that must be stored cell state ܿ ௧ by using formula (2).
Next, the tanh layer that creates the value on the new candidate ‫ܥ‬ ሚ ௧ (formula 2.3). The value produced by the soil layer is between -1 and 1. ‫ܥ‬ After determining the new information that should be stored on the cell state, the next step is to update the previous cell state ܿ ௧ିଵ become a new cell state ܿ ௧ . Previous step ܿ ௧ିଵ multiplied by݂ ௧ , multiplication result is added by ݅ ௧ multiplied ‫ܥ‬ ሚ ௧ . this is done to update the value in each state.
To determine the output gate can use formula (5). The result obtained from the output gate is entered into the tanh function (the value obtained in the tanh function is between -1 and 1), then multiplied by the output from the sigmoid gate to produce a predetermined output value. ℎ ௧ = ‫‬ ௧ ∘‫݊ܽݐ‬ℎ ‫݊ܽݐ‬ℎ (ܿ ௧ ) (6) In the previous study, the output of each sentence was used to represent the sentence. Sentence representation is then used as a feature for sentiment classification. Softmax is used to classify words as positive, negative or neutral [16]. The formula used to calculate softmax can be seen in formula (7), where C is the number of sentiment categories: The architecture of the LSTM model proposed in this study is shown in Figure 2. The LSTM model uses an array of inputs from the preprocessing results. In the next step, the array goes into the Word2vec embedding layer. The embedding layer matches the preprocessed vocabulary and vectors. Vectors that have been processed in the word embedding layer using word2vec will be used as input to the LSTM layer. The LSTM cell processes each word and the resulting output will be fed to the fully connected layer using the ReLU activation function. The output of the fully connected layer will be processed at the output layer using the softmax activation function.

Implementation
To analyze customer satisfaction for vod in Indonesia using the Indonesian language, we used the LSTM method for text classification, as shown in Figure 1. The process consisted of literature study, data set collection, text preprocessing, word embedding, sentiment analysis training using LSTM, model testing and model evaluation. In the research procedure, we propose, the collection of review data contained in the google play store is carried out using the scraping method using the python programming language. Next, pre-processing is required, then word embedding, splitting of training and testing data, model training, and model testing. The customer satisfaction classification model will then be evaluated, and the final result is the polarity of sentiment from customer reviews in the form of positive or negative.

Data Description
The dataset used in this study is review data given by users of the video-on-demand application. There are three stages to prepare the dataset: data collection, preprocessing, and word embedding representation using word2vec. The dataset used is taken by utilizing a library in the python programming language to retrieve review data, namely google-playscraper. Data is saved in files .xlxs format. The data labelling of the sentiment polarity was done manually. In this study, two sentiment polarity labels are used, namely positive and negative.

Experimental Results and Discussion
The implementation of LSTM model for sentiment analysis was using Keras library. The LSTM model was built using SGD for optimization methods, 1 LSTM layers, 2 dense layers and softmax activation in output layer. The experiment using 100 epoch and 0.001 for learning rate. The 2000 datasets are divided into training data and test data with a value of 80% for training and 20% for testing. The example of customer reviews can be seen in Table  1. The experiment results with epoch 100 can be seen in Table 2.

Conclusion
Based on the test results from the implementation of sentiment analysis using the LSTM model, it can be concluded that customer reviews of vod applications in Indonesia can be used by companies to evaluate the services provided to customers. The results of the evaluation of vod user review data in Indonesia get an average score of precision 73.81%, recall 73.81%, f1 score 73.81%. In addition, the accuracy value obtained is 73.90% can be used as a consideration for the company in knowing user sentiment. So that the company can meet the expectations and desires of customers and keep customers using the service.