Research on Interactive Design of Social Interaction Training APP for Children with Autistic Spectrum Disorder (ASD) Based on Multi-modal Interaction

. A number of studies have found that the difficulty in social interactive behaviors and recognize items are two core syndromes of Autistic Spectrum Disorder (ASD) children. Previous research has found that ASD children paid more attention to robots comparing to people.Intervention programs with robots as instructors or partners were also proved to be effective in improving ASD children’s social skills.Yet, few studies find robot intervention works in ASD children training.So research explore the possibility of apply such intervention programs into product design.The present study aimed at involving an animated character of robot in an application (APP) of intervention program to facilitate ASD children’s learning of social interactive skills.In order to achieve the goal, a training system based on sensory integration training was established by the design software of Sketch.Augmented Reality (AR) technology was used to create a multi-modal interactive environment of the APP.The present study is among the first attempt to apply sensory integration training in APP design, aiming at promoting social and cognitive performance in ASD children.


Introduction
Autism spectrum disorder (ASD) is a complex and heterogeneous neuro-developmental disorder [1] . Autism spectrum disorder can occur in early childhood and is accompanied by a lifetime.ASD children are mainly characterized by deficits in social interaction,verbal and non-verbal communication, and repetitive or stereotypical behaviors, and sensory abnormalities [2] .The prevalence of ASD is estimated at 6.2 per 1000 people worldwide [3] .This is a lifelong neuro-developmental disease. The ASD children with mild and moderate can live an independent life after early intervention. However, for children who do not intervene in time, they cannot live by themselves in adulthood, interfering significantly with quality [4] .It also brings a huge burden to the family.While the global burden of autism is currently unknown, in the United States and in the UK, the annual societal cost of the condition exceeds several billions [5] .Therefore,it is important that finding a convenient intervention for ASD children. Cognitive deficit is one of the core syndromes of ASD children [6] .The cognitive performance differed significantly between ASD children and typical developing (TD) children in various aspects [7] .For example, ASD children have the phenomenon of sensitivity and dullness in sight, hearing and touch [8] .Previous research shows it significant to improve cognition capability through Multi-directional sense.So the multi-sensory design applying in APP is most suitable for ASD children.On the other hand,ASD children also have strengths in perception, and mechanical memory [9] .For example, study found ASD children have good mechanical memory for vision materials [10] .Therefore, it will be beneficial for learning effects if vision materials is used in the training APP.

Deficit of Social Skills in ASD children.ASD
children feel difficult to understand other people's language and behaviors.This severely interferes their normal social communication with others [9] .These problems are evident from an early age,despite cognitive potential [11] .They cause daily difficulties, become more pronounced with age,and can lead to negative long-term outcomes [12] .The impairments are expressed in behaviors such as inappropriate affect, social isolation, and failure to initiate interactions with peers, cooperate, share, make friends, express empathy, or provide emotional support [13] .

Application programs developed for ASD children
With the rapid development of scientific information technology,a few training APP developed for ASD children.For example,The application of Look At Me is a software developed by South Korea 's Samsung company to helps users to recognize different facial expressions with assistance of photos and animations.There is also a software called Xiao Yu Di with many users in China to train language organization ability of ASD children.Virtual reality technology has developed rapidly,the significant advantage of rapid transportation of big data in the 5G era.So virtual reality applied in the training of ASD children shows obvious effect.But the training APP designed for ASD children still have much limitation,such as no combination with intervention.So our research focus on applying effective intervention in the training APP.

Multi-modal interactive design strategy
Multi-modal interaction is a multi-sensory interaction that combines visual,auditory,tactile and olfactory interactions [14] .Human-computer interaction is carried out through text,voice,vision,action, environment and other methods,fully simulating the interaction between people.The multi-modal interaction also benefits from the rapid development of computer technology,allowing modern software applications to integrate multiple technologies.The training APP mobilizes their multi-sensory experience to enhance the memory of learning content and master the learning content.In the process of the training software design,the use of pictures with voice,short audio and video to complete the interaction of text,voice and vision.AR technology applied in the APP to achieve interaction with the environment.

ABA (Applied Behavior Analysis)
ABA applied behavior analysis method is a commonly intervention method in autism training. Applied behavior analysis emphasizes the process learning of perception, understanding, expression and generalization.ABA applied behavior analysis will decompose the target task into a series of smaller steps in a certain way and order.Meanwhile it give ASD children appropriate reinforcement to complete the goal task in steps.Based on the ABA applied behavior analysis method mostly adopts the Discrete Trial Teaching (DTT) method into teaching [15] .

Robot guidance
Some studies have found that children with autism prefer robots to real human beings.Dautenhahn and Werry regard 4 ASD children aged 5-10 years with verbal impairment as experiment participant, comparing their responses to robot movement and non-robot mobile toys,result shows two quarters ASD children shows strong interest in robot [16] .Related research has also been done in China, and the research found that children with autism have a higher interest in social robots.Compared with human therapists,social robot intervention can improve the imitation, common attention,eye gaze and active social skills of ASD children [17] .

Method
According to the characteristics of ASD children and the needs of their parents, autism training software is mainly designed for social and cognitive core symptoms( Figure  1).So the main functional module including the cognitive training function module for recognize items( Figure  2),social function module for communication training and user data center function module for data storage.

Cognition training
The function module of object recognition is mainly to recognize objects through AR scanning and then teach objects.The main user goal is to hope ASD children can master the item names, usage and recognition.This module is designed based on the theoretical basis of ABA applied behavior analysis method.It mainly uses DTT round teaching,and each object is used as the unit of teaching.Each object has 4 rounds, including display, usage, expression and found.Each round contains different training. different rounds correspond to different stages of ABA applied behavior analysis.Display round belongs to the perception stage.It mobilize the child's vision and hearing through voice and pictures, and the tactile sensation of feedback and touch.Multi-dimensional sensory design helps information processing.The user needs to recognize objects by reading and selecting.Each object needs to be read three times,each object has 6 items.Because the software designed for preschool children,focused on voice interaction and text equipped for parents assistance.Usage belongs to the understanding stage.The children are required to understand the specific use method of the object after knowing object.The three training include demonstration and display of the object, judging whether the application of the object is correct and dragging the object to the appropriate position.Expression belongs to the expression stage.It designed for user can express the learned object.Found belongs to the generalization stage.The user goal is promptly promoting to the same type of item after complete an item learning.This stage includes training to find 1, 2, and 3 similar items among several unrelated items.And the same kind of search is to find 3 corresponding items in the real environment after turning on the camera through the environment interaction.

Communication training
The social function module is designed based on the robot's efficiency research in autism training.This functional block including training content such as robot teaching, robot communication, virtual human teaching and virtual human communication.All the teaching content is based on the scene-based teaching mode.The virtual human image is a real person,but the body movements and reactions controlled by computer.The teaching mode from robots to virtual human regard as social transition for real person to eliminate the communication barriers of ASD children.They communicate with robot and virtual people in the real environment implemented by VR technology.

User data center
This functional module is equivalent to user's data center, and will contain learning achievements, learning video storage, teaching tests, personal information, general settings, etc. It mainly facilitate parents to master some of the child's learning situation, save individual data and help the therapist track the child's intervention.

User process design
In the use of the software,users mainly have two use processes (Figure 3),one is the process of using the object recognition function,and the other is the process of using the social function for social training.Both processes are shown in the figure.Considering that the user's age is relatively small,the process is simplified as much as possible in the design.Many logical judgments are determined by the system,which reduces the user's burden of use.

Interactive feedback design
3.2.1.Cognition feedback design.The software as a educational product positive equipped with full feedback and negative feedback.Feedback design has an intensifying effect on the response of ASD children in autism training software,which are very important and critical design content.In the object recognition training,each step operation will receive feedback.For example,the follow-up training in the Display, each object needs to be read three times.Each time if the follow-up is correct, a blue circle  (Figure 4(a)).But the follow-up is wrong or unrecognizable,the system will give a voice prompt to encourage the user try again.In all the selection and judgment training of the software (Figure 4(b)),if the selection is correct,a cheerful voice will be emitted, and the page will jump to the next page.If the selection is wrong, you will receive tactile vibration and prompt sound.In the object-dragging object usage training,if the object is dragged to a suitable position,a flashing animation effect with a pleasant voice will appear.If not appreciate, the object will return to its original position with tactile vibration.In the search for similar items, there will be cheerful music and the object will jump into the item box by itself after scanning the corresponding correct items.For users' better experience,each round will display the number of rounds and can jump to other functions.

3.2.2.Communication feedback design.The feedback
of social functions is more complicated than the feedback of object recognition functions.It includes two parts: voice prompts and text prompts.Robot teaching is a follow-up training.If there is no response for 5 seconds after the follow-up voice prompts are completed, the voice prompt will be issued again.The prompt will be given again when the robot is clicked.After the prompt is completed,the robot will continue the conversation until it is necessary to follow up again (Figure 4(c)).The text prompt will always be displayed on the interface.Yellow text background means robot speaking and red text background means user speaking.After completing the teaching,you will enter the mode of robot free communication.In this mode,the text prompt will not always appear in the interface.When the text prompt does not appear,it means user speaking and the system will give text and voice prompt judge from what users say.Only user completed robot teaching and robot communication can enter teaching and training of the virtual human (Figure 4(d)).The feedback mechanism same as the robot.The music feedback is removed from communication function for increasing the immersion in the communication process and making social teaching more effective.

Interface design
3.3.1. Interface style. Because training software designed for preschool children, the design in a cute and simple style.Lovely style elements can attract users' attention and increase their interest in learning. The concise page is relatively clean. Because the information is clear and the focus is prominent, which can improve the user's learning efficiency.

Interface color.
The software determined blue and yellow as the main colors.Blue meaning for blue sky,yellow meaning for stars in the night sky.The ASD children also called children like stars. White and red as auxiliary colors are applied in some informative and general information.

3.3.3.Interactive feedback design.
Because the integration AR technology into the training software, the interface often needs to open the camera to capture the real environment.The page is occupied by the environmental image.The main interface for scanning the recognition object and social training designed in entire full-image interface which can provide better immersion (Figure 4(e)).However, in the training of similar items scanning, the interface layout in half-screen image window facilitate user to perform other operations while recognizing the scan (Figure 4(f)). ASD children as the direct users of the training software is so young that use APP maybe need parents help.The parents are also indirect users, and the displayed information is displayed in the form of voice plus text.The difficulty in designing is not only to ensure the integrity of the information, but also to keep the page simple.Therefore, the shape, size, and color of the main elements are adjusted in the design in order to display the information in layered manner.

Conclusion
The interactive design of the training software for ASD children is very special.It required design consider the characteristics of ASD children's own perception,feeling,and language system.Multi-modal interaction design requires both technical realization and ingenious design.The training software integrates training content in the way of multiple sensory interactions for better experience.It is a new attempt that design training software for the ASD children.With the continuous development of computing information technology in the future,there will more intelligent solutions for ASD children's problem.Interaction design can also play a more important role in the interaction of education products for ASD children.Perhaps designer will focus on the multi-modal interaction training content of ASD children in the next years.