Feature Optimization on Dual Leap Motion Controller for Indonesian Sign Language

. Sign language is a language formed by a combination of finger, hand, body movements and facial expressions used by persons with disabilities such as deaf and speech impaired. One of these sign language recognitions is recognition using Leap Motion Controller (LMC) sensor technology. In addition to the sign language that is formed has diversity such as folded fingers, hidden fingers, indonesian sign forms also have characteristics and shapes that are almost similar to one another. The LMC sensor is not always able to recognize all forms of signs properly. In this study, optimization is proposed at the feature level where optimization aims to provide more detailed features and characteristics of each sign language formed. The stages of the process are designing the layout of the sensors, adding features and combining feature data from each sensor. The test of the feature optimization on this dual LMC sensor can provide an increase in the recognition accuracy of the given Indonesian sign language. The Indonesian sign language can be recognized well with an average accuracy of 87.24% and the optimization carried out is able to produce an increase in accuracy of up to 2.88%.


Introduction
Sign language is a language formed by a combination of finger movements, orientation of hand, arm, body movements as well as lip movements and facial expressions.Sign language communication in Indonesia is one of the sign language communications built by adopting the American Sign Language (ASL) sign language which is owned by America.Communication between persons with disabilities will certainly be very difficult for normal people to understand in general and will find it difficult to know and understand the sign language conveyed by the deaf and speech impaired.This is due to differences in methods and ways of communication, and vice versa. So we need a system that can translate the difference in communication methods between sign language communication and normal communication..
In the field of technology, of course, during the last decade many studies have been carried out and some of these sign language recognition studies have focused on the algorithms used or methods consisting of static and dynamic cues [1]. Many research works have been directed and carried out to develop the recognition of different sign languages [2]. Some of these studies are American Sign Language(ASL) [3], Malaysian Sign Language [4], Indian Sign Language (ISL) [5] and Arabic Sign Language (ArSL) [6]. Other sign language research uses an image-based and sensor-based * Corresponding author : syaiful_ft@unmus.ac.id approach, where the sign language algorithm is divided into three parts, namely recognition based on letters, words and sentences. Where the word recognition technique is to capture and analyze each continuous image that represents the whole word in sign language [7].
The development of sign language research today, of course, does not only depend on algorithms and sign recognition techniques, but researchers have also presented many of the existing sensor technologies. The combination of combined static features for static cues and dynamic features for gestures, obtained from LMC [8]. However, in practice, it is said that motion capture sensors such as Kinect and LMC cannot always recognize all fingers. Because not only fingers touching each other, but folded, hidden or blocked fingers by other fingers can cause the camera sensor to not recognize and capture properly. [9]. So it takes several other sensor sources or more than one sensor (multi sensor) to be able to minimize and overcome this. In previous studies, the results obtained from the use of dual sensors stated that the accuracy of some of these scenarios could increase sensor performance by 5.35% or with an average accuracy of 84.36%. [10].
Therefore, in this study, an optimization scenario is proposed to improve the performance of sensor accuracy in recognizing signs in Indonesian Sign Language, namely by adding the characteristics or characteristics of each sign obtained. So that the use of E3S Web of Conferences 328, 0 (2021) ICST 2021 this scenario can optimize the detection and capture sensor process for the given hand signs and can improve the recognition accuracy of Indonesian Sign Language.

Methodology
In this section, several supporting research methodologies, research designs and the stages of research carried out are described, as follows:

Leap Motion Controller
The LMC system is a system that can recognize hand movements, using optical sensors and infrared light. The sensor is pointed along the y-axis direction when the controller is in standard operation and has a 150-degree field of view. The effective distance of the LMC extends over the device from 25 -600 millimeters (1 inch -2 ft). The LMC can detect hands, fingers and tools that are in the LMC's field of view. Each frame object represents a frame containing a list of detection data such as hands, fingers and finger-shaped devices, as well as motion detection [11].

Dual Sensor Design
The sensor layout design is used to position each sensor to be able to capture the form of a given sign language that is captured from various angles, namely 0 degrees, 90 degrees, 120 and 120 degrees. [10].

Data Collection
A total of 26 letters of the Indonesian Sign Language alphabet as shown in Figure 3, will be tested in this study. Indonesian cues are categorized into two parts, namely static cues and dynamic cues [12]. Dynamic cues are signs given by finger or hand movements. Therefore, it is necessary to add several other features that have been selected to give the characteristics and characteristics of each class or letter. For one data recording is done using 10 frames, and each letter will be recorded ten times from the existing letters of the alphabet plus one position or normal hand condition. So the total of the data obtained is 270 datasets per scenario.

Feature Optimization
The process carried out is the collection of samples or datasets obtained from the detection results on each LMC sensor for hand signs based on Indonesian Sign Language. The coordinate data of the sign is used as input material into the system which will be processed at a later stage. Feature extraction is the process of selecting or taking a feature and characteristic from the previously inputted sign form. There is table 1 above are the features used in previous studies [10]. Therefore, in this study, several other features were added which aim to provide more detailed characteristics or in other words optimize the formation of features from the given signal. Table 3 below is a combination of the features added and used in this study [13].

Results and Discussion
The testing were carried out on the use of this dual sensor with the addition of the features already provided. The first scenario is to test dual LMC sensors based on the sensor layouts in Figures 2a, 2b and 2c. The results obtained will be compared between scenarios so that the best scenario can be obtained in optimizing recognize for Indonesian sign language. In this test, the dataset is divided into training data as much as 70% of the dataset and 30% of the dataset as test data. So that the test data becomes 81 data and each class has 3 test data.

Scenario 1
Scenario 1 is a combination test at the feature level with reference to the layout of the two LMC sensors forming 90 degrees as shown in Figure 2a.

Scenario 2
Scenario 2 is a combination at the feature level with reference to the layout of the two LMC sensors forming 120 degrees as shown in Figure 2b.

Scenario 3
Scenario 3 is a combination at the feature level with reference to the layout of the two LMC sensors forming 180 degrees or facing each other as shown in Figure 2c. From the three scenarios above, the results of the recognition accuracy of the test data have been obtained from each scenario. The results obtained for each target class in each scenario are very varied. Recognition errors still occur for letters with a high degree of similarity such as E, M, N and R. However, with the addition of features made previously, it can optimize the recognition of sensors for Indonesian Sign Language. The following is a comparison chart of the accuracy of the tests carried out based on scenarios. The results of the tests that have been carried out, by optimizing the features of this Indonesian sign, the LMC sensor overall has a good recognition accuracy. In previous studies, it was stated that the use of dual sensors obtained an average accuracy of 84.36% from the combination of feature data and sensor layout [10]. By referring to the addition of features or characteristics of each signal form, the resulting accuracy can be increased compared to using three static features and one dynamic feature, so that in this study each class has six static features and one dynamic feature. The scenario with the highest accuracy results was obtained from the results of scenario 2, which was 90.12%. Based on the analysis carried out, this can occur because the maximum sensor capture power for hand signals is sourced from two LMC sensors. Letters that have a high degree of similarity can be minimized by adding features, optimizing features for the formed signs. Therefore, the weakness of LMC which cannot recognize Indonesian language cues such as hidden, folded or blocked fingers by other fingers, can be captured optimally based on the sensor layout scenario, feature combination and feature optimization of these features.

Conclusion
The addition and selection of features in this study is very necessary to be able to increase the diversity of characteristics and characteristics of each letter, because Indonesian Sign Language has several signs that have a high similarity. After various tests were carried out and the results obtained, it can be concluded that with the design of the sensor location, combination of features and optimization through additions made to the feature level of the two LMC sensors, the Indonesian sign system can be well recognized with an average accuracy of 87 .24% and the optimization carried out was able to produce an increase in accuracy results reaching 2.88%.