Research on Hand Action Pattern Recognition of Bionic Limb Based on Surface Electromyography

. Hands are important parts of a human body. It is not only the main tool for people to engage in productive labor, but also an important communication tool. When the hand moves, the human body produces a kind of signal named surface electromyography (sEMG), which is a kind of electrophysiological signal that accompanies muscle activity. It contains a lot of information about human movement consciousness. The bionic limb is driven by multi-degree-freedom control, which is got by converting the recognition result and this can meet the urgent need of people with disabilities for autonomous operation. A profound study of hand action pattern technology based on sEMG signals can achieve the ability of the bionic limb to distinguish the hand action fast and accurately. From the perspective of the pattern recognition of the bionic limb, this paper discussed the human hand action pattern recognition technology of sEMG. By analyzing and summarizing the current development of human hand movement recognition, the author proposed a bionic limb schema based on artificial neural network and the improved DT-SVM hand action recognition system. According to the research results, it is necessary to expand the type and total amount of hand movements and gesture recognition, in order to adapt to the objective requirements of the diversity of hand action patterns in the application of the bionic limb.


Introduction
Hands are not only tools for people to work, but also tools for people to communicate, to convey emotions and intentions. Therefore, the recognition of hand actions is important in the daily life and has also become a research hotspot in HMI application field. For people who lose their hands or arms, the prosthetics can be a substitute for real hands. However, due to the defects in action recognition and the movements, the current bionic prosthesis is far from the real limb in many aspects, such as the functions and physiological feedbacks. Hence, how to expand the type and number of hand movements and gesture recognition to make the prosthesis adapt to the diversity of hand movement patterns that can be recognized by bionic rehabilitation equipment is crucial for new HMI applications. Because EMG signals are generated when the hand moves, which contains abundant information of human actions, this paper will make a profound study of hand action pattern technology based on sEMG signals in order to make the bionic limb distinguish the hand action fast and accurately. Machine learning specializes in the study of how computers simulate or implement human learning behaviors to acquire new knowledge or skills, and reorganize the existing knowledge structure to continuously improve its own performance. Based on the analysis and comparison of some current classifiers, this paper designs a classification and recognition system for a variety of hand movements and gestures based on machine learning to meet the needs of users for multi-modal classification applications.

Research status of action recognition based on sEMG
The core task of sEMG-based bionic arm control and HMI applications is to decode human movement intentions through EMG signals [1]. Human actions can be regarded as a combination of discrete and continuous movements. Therefore, there two kinds of studies in the field of human action pattern recognition based on sEMG. One of them is to analyze sEMG signals to identify discrete movement states of limbs [2], such as the flexion and extension of fingers, fisting, wrist turning and other independent movements. The second is to analyze sEMG signals to estimate some changes in continuous variables of joints when the limb moves [3], such as the joint torque and joint angle. The following will be described separately from these two aspects.

Research status of discrete action pattern classification
The study of discrete action patter classification is currently the most researched, most mature method, and most fruitful in the field of human action recognition based on sEMG [4,5].
The general process of this research is: Firstly, collecting sEMG signals during body movements; Secondly, preprocessing the sEMG signals and extract their features. Thirdly, using sEMG feature samples to train the classifier with high recognition rate; Fourthly, testing and evaluating the actual classification performance of the trained classifiers online; Fifthly, using classification results as decision output to drive bionic prostheses or actuators. The two core technologies in this process are feature extraction and pattern classification. Through the comprehensive analysis of Table 1, in order to improve the effect of action recognition, a large amount of research work continues to focus on the selection and combination of features and the design and training of classifiers.
Although the research results of discrete action classification are significant, there are still many problems: first the action types are all predefined, and unknown action types are not considered. sEMG signal acquisition and preprocessing have high requirements for the environment, and may not be guaranteed in actual applications, and that may affect the true classification success rate; Second the description of the action is relatively simple, lacks a unified standard and scale. The classification does not distinguish the individual tester habits. Hence, the classification has the more arbitrariness; Third most of the experimental conclusions have not been tested for a long time, and stability cannot be guaranteed.

Research status of continuous motion mode estimation
Ensuring the continuous matching of man-machine actions is a prerequisite for realizing the safety control of multiple service robots [6] such as power-assisted exoskeleton robots, rehabilitation robots that assist paralyzed patients in rehabilitation training, etc. Discrete actions classification results can only identify a few discrete motion states of the limbs, and only depend on the recognition results of discrete motion states cannot control the machine to complete continuous smooth actions. Therefore, using sEMG signals to estimate continuous human motion has become a new hot spot in surface EMG research. At present, the general process of using sEMG to estimate joint continuous motion is: first calculate muscle activity or signal envelope information by extracting sEMG, then estimate joint torque, angular velocity, angle and other motion based on joint dynamics/motion model, and finally according to the estimated result Auxiliary control of the robot terminal. Table 2 lists some of the results of recent research on joint continuous motion estimation. The continuous action model researches cannot fully describe the intrinsic electro-biochemical-physical relationship between sEMG and muscle activity now. Due to the non-linearity, non-stationarity, and weak antiinterference ability of sEMG itself, as well as the difficulty in accurately measuring human physiological parameters, although researchers continue to improve joint dynamics/motion models, the current overall effect of the models still cannot match the complex and changeable state of motion. The constantly changing state of motion is only approximate. Therefore, the research progress of continuous action mode is more difficult than discrete action classification, and more updated analysis theories and detection techniques are needed to promote its development.

Current problems and shortcomings
The current research on the physiological mechanism of human movement/change is still in the exploratory stage. Therefore, when performing action pattern recognition, there is still a lack of unified standards for the selection of sampled muscle groups, the selection and extraction of sEMG features, and the continuous action/motion modelling method. Since the human-machine interface based on the EMG signal does not adopt a personalized design, the researchers mainly determine the plan based on the previous experience or the analysis and summary of the previous experimental data. However, different people have different EMG signals, and the change of the measurement position can also cause the change of the EMG signal. These factors cause the disabled to adjust the control parameters of the prosthetic hand according to their EMG signals when using the bionic arm. So, the user usually spends much time on training the bionic arm to flexibly control the prosthetic hand. When the bionic limb is tested, these subjective factors make the conclusion more uncertain. In addition, the bionic arm has poor operability and weak adaptive ability.
At present, the number and types of hand movements that can be stably recognized by bionic arm are very limited, and there are less than 10 kinds of hand movements that can be recognized 10 in most researches. The insufficient action type is able to inevitably restrict the use of bionic rehabilitation equipment and the application prospects of new HMI equipment. Hence, how to effectively obtain the abundant body movement information contained in the sEMG signals and expand the types of fine movement recognition is one of the problems that need to be solved for practical applications.

Design of hand action Multi-pattern classifier system
When extracting the sEMG signals of a human arm, multiple channels are set in the human forearm area to collect the signals. These collected sEMG signals are used to recognize hand movements. Therefore, an ideal hand action recognition system not only can extract the different EMG signal eigenvectors, but also equip with a pattern classifier which can generate action type decision output based on input feature vector data. Hence, the design of the classifier plays a vital role in the application system for the purpose of pattern recognition. It directly determines the performance of the entire recognition system. The design process of the classifier mainly includes the type selection of the classifier, the revision of the structural parameters of the classifier, offline training and verification, and online application performance verification.
Based on the comparative analysis of commonly used classifiers, this chapter proposes a BP neural network classifier design method and an improved DT-SVM (Decision Tree SVM) classifier design method for the recognition of multiple movements of the bionic limb.

Typical pattern recognition method
Pattern Recognition (Pattern Recognition) refers to the technology and theory that uses computer technology to simulate human perception to automatically distinguish and classify objective events or physical objects. Based on the pattern recognition method, the use of human sEMG signals to control bionic prostheses, rehabilitation equipment and interactive equipment is the current research hotspot in the application of bionic technology [7]. A typical pattern recognition system is shown in Figure 2, which mainly includes five parts: sEMG data acquisition preprocessing, feature selection and extraction, classifier decision-making, and classifier design. As shown in Figure 1, a complete pattern recognition system design process is divided into two parts. The first part belongs to the classifier design stage, which mainly completes the collection of training samples, data processing and feature extraction according to the known action state, as well as the train and modify of the discriminant function based on the training sample data. The second part belongs to the stage of classifier performance evaluation and online application. It mainly completes testing the trained classifier and evaluating the classification effect of unknown action types [8].

Application and design of classifier
With the continuous development of artificial intelligence technology and pattern recognition theory, the research on pattern recognition of limbs based on surface EMG signals has made great progress, and many pattern classifiers with excellent performance have been applied. The pattern classification methods commonly used by these classifiers include cluster analysis, Bayesian decision-making, artificial neural networks, decision trees, support vector machines, hidden Markov models, etc. The mechanism and classification effects of classifiers designed by these methods have their own advantages. So, understanding the characteristics of different classifiers is helpful to learn from each other and improve the generalization ability of the designed classifier.

Typical classifier types  Cluster Analysis
The thinking of cluster analysis is to calculate the closeness between subjects according to certain standards and classify similar subjects into the same category. For example, the most widely used C-means algorithm in cluster analysis [9] takes the mean of various samples as its core, and calculates the Euclidean distance between an unknown samples and various mean points to determine its attributable cluster. Using this classification algorithm can avoid the tedious training process of many non-linear classification methods and achieve better timeliness in classification training. The biggest disadvantage of cluster analysis is that it has higher requirements for EMG signal characteristics, when applied to pattern recognition. Therefore, it is difficult to deal with multi-modal classification problems with it. As the complexity of action patterns increases, the recognition rate of the classifier drops sharply.
 Bayesian Decision Bayesian decision is a pattern recognition method based on probability and statistics theory. For example, people often hope to obtain the result with the smallest error in the actual pattern classification problem, and the minimum error rate Bayesian decision is to use the Bayesian formula in probability theory to obtain the classification rule that minimizes the error rate. Since the use of Bayesian decision-making usually requires prior knowledge of the distribution of various samples in the feature space and empirical statistical data, this hinders the further improvement of classification performance to a certain extent, and this method is not suitable for complex pattern classification.
 Artificial Neural Network Artificial neural network, also called neural network or connection model, is the abstraction and simulation of the working mechanism and function of human brain neural network using computer technology. Artificial neural network, also called neural network, is a technology that using computer technology to abstractly simulate the mechanism and functions of human brain neural networks. Because of its self-learning, self-organization, better fault tolerance and excellent nonlinear approximation ability, it has been deeply researched and applied in many fields. When the neural network is used in the design of supervised learning classifier, the classifier can learn under the condition of known input pattern and expected output. Neural network achieves the purpose of information processing and classification by adjusting the connection between a large number of nodes in the network [10]. Today, there have been many kinds of neural networks, which have different algorithms and structures, including feedforward neural network, feedback neural network, stochastic neural network and competitive neural network. And the back propagation (BP) neural network is one of the most widely used models.
 Decision Tree Decision Tree is a very commonly used classification method. It is a kind of supervised learning, that is, given a bunch of samples, each sample has a set of attributes and a category. These categories are determined in advance. Then a classifier can be obtained through learning, which can give a new object. Correct classification. Such machine learning is called supervised learning. In the structure of Decision Tree, each non-leaf node represents a test on a characteristic attribute, each branch represents the output of this characteristic attribute in a certain range, and each leaf node represents a pattern category. The root node is the starting point of the decision tree classification and judgment process. In this process, the item to be classified is continuously tested for feature attributes, and then the node to be passed and the branch to be entered are determined according to the results of the test output. Finally, the pattern category corresponding to the leaf node of the item to be classified is the result of this classification. Therefore, the entire decision tree is a table of classification and analysis rules [10]. The advantage is that the rules are easy to understand, the classification speed is fast, and the accuracy is high, but the disadvantage is that the scalability is weak and it is difficult to deal with large training sets.
 Supper Vector Machine The design principle of support vector machine is to minimize structural risk. It is a self-learning classification algorithm developed based on statistical learning theory. And the advantages of it are short training time, strong adaptability, global optimization, good generalization performance, etc. In particular, it shows unique advantages when solving small sample, nonlinear and high-dimensional pattern recognition problems. However, its main disadvantage is that multiple two-class support vector machines are required for joint processing when solving multi-classification problems [11].
 Hidden Markov Model Hidden Markov model is method, which identifies features of a time-series signal, based on probability estimation [12]. The time-series signal, which is generally a sequence of eigenvectors of the sample, is the input signal. And The output is the likelihood probability of the feature sequence generated by the model. Hidden Markov model has a strong ability to model based on time series. It can analyze the changes of sEMG signals when people do different actions, and classify the action patterns according to the difference of the signal change rules. The characteristics of this model are conciseness, clear physical meaning, and mature algorithm. But the disadvantage is that it is not suitable for medium and longterm forecasting.

Design of pattern classifier based on Neural Network
The essence of Neural Network Identification (NNI) is to select a suitable neural network model to approximate the actual dynamic system. Then it learns the sample data and adjusts the connection weight of the network to minimize the required error criterion, thus inducing the non-linear mapping relationship implicit in the sample data. At present, neural networks have been used to identify and classify the surface EMG signals of the lower limbs to realize the recognition of different road conditions by the prosthesis and make corresponding changes. In other words, the artificial prosthesis can adjust the joint bending angle and torque according to the user's intention.
The prosthesis designed based on neural network classification can be applied to different environments to better meet the needs of users. The BP neural network in the neural network has powerful nonlinear mapping and classification recognition capabilities. It can self-learn, self-organize, and has good fault tolerance and nonlinear approximation capabilities. Therefore, it can be applied to the design of bionic arms. The BP neural network can be used to construct the classifier, and then carry out the pattern recognition training and verification of the sEMG feature data of various hand movements, and finally explore the key technology of the classifier design to realize the multi-action pattern recognition of the hand. Based on the structure and basic theory of BP neural network, the characteristics of sEMG signal are extracted to obtain time domain feature matrix, AR model coefficient feature matrix, wavelet packet transform relative energy coefficient matrix, and finally the corresponding BP neural network classifier is constructed, as shown in Figure 2.
In this formula, n represents the number of input layer units.
The value of n depends on the number of features selected during classification training. The input vector X of neural network is in the form of column vector, that is, each row input is a characteristic variable, and each column input is a sample of characteristic matrix.
After the experiment has verified the effectiveness of the algorithm of the activity segment of the extracted EMG signal, the corresponding action label can be added to the feature vector participating in the training of the pattern classifier. Next, the original sEMG signals of 20 kinds of hand movements and gestures including "Relaxation" are collected. Then the feature vectors of 19 different action types are extracted. Each piece of raw data contains a relaxation action and a hand action. Then add action label code to each piece of data. For example, the label of "flexing index finger" is "01000000000000000000", that is, the third bit from left to right is 1, the remaining bits are 0, and the action code is 2.
 Output vector Y and expected output vector E In the formula, q is the number of cells in the output layer. The actual value of q mainly depends on the number of action types. Take the training test of 20 kinds of action pattern recognition experiments as an example, q=20, and the value of the corresponding expected output vector T is based on the binary action label.

Multi-classification algorithm based on DT-SVM
Support Vector Machine (SVM) is a general machine learning method proposed by Vapnik et al. based on statistical theory. This method improves the generalization ability of the learning machine by seeking the structure with the least risk. Since its inception, because SVM has shown excellent characteristics in solving problems such as nonlinear, high-dimensional space, small samples, and local poles. It is widely used in function approximation, pattern recognition, signal prediction and fault diagnosis [16]. SVM has become a research hotspot in machine learning today, and can be applied to pattern recognition and classification of sEMG by bionic limbs. The research goal of SVM is to solve the classification problem of two linearly separable tasks in the field of pattern recognition. The design idea of SVM is to find a hyperplane. This hyperplane cannot only correctly classify the linearly separable two types of data, but also ensure that the distance between the hyperplane and the separated two types of data is maximized (the structural risk is minimal) so that the data in the two categories can get the largest interval space. Because the probability of the data being misclassified can be minimized, the generalization ability of the classifier is reflected [15].
Decision tree (DT) is a clustering method that uses a divide and conquer strategy. The algorithm divides all categories to be divided into two sub-categories at the root node according to a certain separability. Then each subcategory is further divided into two sub-categories. This continues to decompose layer by layer, and after each division, the number of categories contained in the secondary sub-categories continues to shrink, until each specific category forms a "leaf" node. The above process is like an inverted decision tree, each node (except leaf node) in the tree structure is a decision point. If SVM is used to realize the classification function of decision points, it constitutes a DT-SVM classifier. Decision trees can be subdivided into two types: balanced decision trees and unbalanced decision trees.
In the process of dividing each decision point, the balanced decision tree divides the categories into the same or extremely similar number of subcategories as much as possible. Balanced decision tree training speed is relatively fast due to balanced training samples. But the unbalanced decision tree is slightly better than it in classification accuracy. Although the DT-SVM method has the advantages of short training period, fast recognition speed, and can effectively avoid the "rejection" phenomenon of other multi-class classifiers based on SVM, it has the inherent shortcoming of "error accumulation", that is, the classification errors of the subclassifiers will be conducted down step by step. At the same time, the classification order also has an impact on error transmission. At the same time, the order of classification also affects error transmission because the node with a high error rate is closer to the root node, the greater the impact on the overall classification accuracy. Therefore, the construction process of DT-SVM needs to follow the classification principle of "easy first and difficult later". In the construction process, a certain measurement method is used to evaluate the separability of different categories, and the categories with good separability are arranged in the previous classification nodes of the decision tree for priority separation.
The research of constructing the most suitable order of classification is the key to the performance of DT-SVM.
Due to the diversity of category classification, especially when the total number of categories to be divided is large, all the divided categories are trained and then the optimal classification group is screened. The design of traditional DT-SVM mostly uses greedy algorithms to find a better grouping scheme. This method has shortcomings. For example, in the feature vector space, the sample features of different categories are clustered to form hypercubes with different shapes, and the size of the class separation spacing is not strictly consistent with the degree of class separability. As shown in Figure 3, although the class 1 and class 2 in the figure have a larger class separation distance, it is obvious that the separation of class 3 and class 4 in the figure is stronger. In addition, when constructing a balanced decision tree, each node deliberately separates the sample into two sub-categories of equal numbers, so that the characteristics of the sample data itself are often ignored, and this may cause the loss of classification recognition rate. In order to more accurately reflect the separability of feature vectors of different categories, the Fisher ratio concept [13] is introduced to define the category separability measure. This measure can be described as the ratio of the degree of dispersion between classes to the degree of cohesion within classes. The class centre μ � of class C � is defined as the mean of all sample vectors in C � , where C � is the total number of samples in C � : The class radius R � of class C � is defined as the average Euclidean distance between all sample vectors in μ � and C � : D �� is defined as the center distance between class C � and class C � : G �� is defined as the boundary interval between class C � and class C � : S �� is defined as a measure of separability between classes: In the above formula, σ � � is the sample variance of class C � . It can be seen from the formula that the class separation distance G �� can reflect the degree of dispersion between classes, while the sample variance σ � � reflects the degree of cohesion of the data within the class. The larger the ratio, the better the separability between classes. During the reorganization of the category, the improved DT-SVM method can group each node strictly in the order of the size of the separability. It makes it easy for the categories with large separability to be separated first. This satisfies the "easy first, then difficult" classification principle. At the same time, it avoids the defect that the balanced decision tree deliberately divides the category samples and causes the recognition rate to be damaged. In the process of dividing the sub-category clusters of each node of the decision tree, the balanced or unbalanced classification can be flexibly selected according to the distribution characteristics of the sample itself, so that the classification effect of each node can be guaranteed to be close to optimal, and thereby the effect of " cumulative error" is reduced and the overall performance of the classifier is improved.

Conclusion
In conclusion, in this paper an improved DT-SVM (Decision Tree SVM) classifier for the inaccurate SVM classification is proposed. It has the advantages of simple structure, low resource occupancy rate, fast training and classification speed of traditional DT-SVM classifier. In addition, it can obtain relatively ideal classification recognition rate, balanced performance, and has particular potential and certain value of promotion.
Although classifiers based on BP neural network or DT-SVM have increased the number of actions that the bionic arm can recognize. However, most of the classifiers trained in the experimental phase have overtrained the training samples, thus resulting in a decline in generalization promotion ability. In the future, a truly application-oriented EMG control system should have the ability of self-learning. It has the ability to continuously accept new motion sample data to complete iterative upgrade. Realizing online learning is bound to be the research hotspot of the bionic arm in the future. From the aspect of portability, the current EMG signal collectors are insufficient in size and function, and are not suitable for portable use at any time. This is not conducive to future application and promotion. Therefore, the development of a more compact, integrated and intelligent acquisition module of surface EMG signals is an important step for the bionic limb to truly enter people's lives.
At last, the improvement in human hand motion pattern recognition technology based on surface EMG signals can solve the difficulties of people without hands or arms and provide them with more reliable and humanized auxiliary treatment equipment, as well as bring new HMI experiences. Therefore, this technology has great potential for development and application.