Bachelor Thesis Spring 2011

Rana Mansour

ID: 13-633

Email: rana.mansour@student.guc.edu.eg

How could media help educate autistic Children

First Supervisor: Prof.Daniel Fetzner

Second Supervisor: Bernd Dudzik

  1. Abstract
  2. Outline
  3. Research Methodology
  4. Realization
  5. Testing and Results
  6. Conclusion
  7. Future Recommendation


Autistic children are part of the Egyptian society just like any other country but they are unfortunately neglected. This thesis observes the use of interactive media as a tool that helps impaired children in learning. An interactive visual is designed and tested by impaired children and special need teachers to evaluate its effectiveness. In this thesis the goal is to experiment how an interactive educational game could be a tool of education to autistic children.

Keywords: Autism - Interactive - Media - Education - Children




  • 1.1.Background
  • 1.3. Purpose of the research

2.Research Methodology

  • 2.1. Autism from a medical perspective
  • 2.2. Interviews & Documentaries
    • 2.1.1 Parent
    • 2.2.2 Psychiatrist
    • 2.2.3 Education for autistic a necessity: Temple Grandin
    • 2.2.4 The impact of technology on autistic children
  • 2.3.Children observations
    • 2.3.1 Advance Society: special needs school
    • 2.3.2. The Class
  • 2.4 Summary & Analysis of Research


  • 3.1. Concept
  • 3.2. Content
    • 3.3 Interaction Design
    • 3.3.1. Setting
    • 3.3 2. Navigational Structure
    • 3.3.3. Technical Aspect
  • 3.4.Testing the interface
  • 3.5.Evaluating Results


5.Future recommendation





 Autistic children have delays in the development of many skills and they lack the ability to use imagination [3].

They are also unable to have a smooth conversation as they struggle with non-verbal communication.

 They also agreed that an early intervention and treatment allows the child to maximize his ability to speak, 

learn and function [3,6,11,19,25]. They should be structured to meet the child’s educational needs [6]. Autistic children are not curious at all by nature; they have to be taught how to use objects and certain behaviours. [6,19,23]

Another common point was that autistic children feel more comfortable with repetitive behaviours [23], they do not like change. They can stay for hours repeating the same movement and do not get bothered. [3,18,19,25] Children with autism show impairments to joint attention, which is their ability to coordinate the focus of attention on another person and an object of mutual interest [7,18]. To have an autistic child’s full attention, at least for a short while, one must call his name and try to show him something interesting. [23]

    Autistic children are hypersensitive to sounds with a high pitch, they prefer humming ones. Only one 

research proved that sound helps autistic children understand the environment they’re in. [3,5,18,25] Computer games have recently been proven to be very useful tools in treating autism, as these children are interested in technological gadgets. [5,6,7,19]. Therefore observing them is a necessity. Encouraging the child to colour, play with puzzles is also important in the treatment. [6]

The researches done agreed mainly that autistic children are very good visual learners [5,8,9,11,12,17,18,19,23] and they have good visual memory. They relate everything they see with pictures they have in their memory. [11,23] Software platform use entertaining content to make educational settings more attractive.

 By using photos of real objects or sketches, autistic children learn to distinguish objects based on the
 colour,the size, etc. This kind of interactive learning games motivates the child to correlate the object to its 

characteristics. [5]

     Children with autism show greater improvements in emotion recognition when programs include cartoons 

rather than photographs of real face. Clinical and parental reports show that they tend to spend more time looking at cartoons. [5,23]

The use of emotional avatar advances the educational process. Educators suggest that most of the time autistic children are able to recognize avatar’s emotional state provided by facial expressions. [5]

    Blocks and Lego are a usual toys of autistic children moreover, everyday practice in autistic schools shows
 that children like playing with cubes. [7,18]

Rewards are necessary as they encourage the child to fulfill the task that he was asked to do. [23]

Temple Grandin Movie , Youtube

Temple Grandin Movie, Cast Opinion , HBO , Youtube

Autistic Child expresses profound intelligence, , Youtube

The conversation: Ipad gives a voice to autistic children, Youtube



    The final concept of the educational game is to have colored cubes with 

letters and a screen with a letter, a picture and a word depending on the level as the game is divided in three levels. The visual items are combined with the spoken words. The child would have to match the letter on the screen with one of the cubes and place it in a specific area where the camera would then be able to detect the color. To be able to guide the child throughout the game, a colored emoticon appears each time the child places a different cube and either has a happy face or a sad face depending on the answer.

    The installation has for aim to make education in some way enjoyable and 

playful, so that the child would be interested. It was specially designed to educate autistic children, as they are very visual and require special games. The installation does not require the child to have any knowledge in using a computer in any way as colored cubes generate the interface; therefore the use of a mouse or keyboard is neglected because of their difficulties to accomplish difficult tasks. All that is required is that their teacher or parent shows them how to start the installation and how to use it and the rest is automatically generated.


    The idea of the installation is to have a set of cubes in front of the child 

with different colors and each one with a letter except for the green and red one as they are used to start and stop the game. The task is that the child recognizes different letters visible on the screen, grabs the right cube and place it in a specific area.(see figure)

    The cubes are orange, purple, blue yellow, green and red. The use of 

vibrant colors was to attract children to play with the game. The cubes are 16.5-centimeter square to make it easier for the child to grab with two hands and to prevent holding more than one cube at the time. [7]. The cubes are made out of pressed foam so that it doesn’t weight much, they are as well wrapped with adhesive colored paper, and therefore it could be cleaned if needed for hygiene reasons, as there are children involved. The use of coloring sprays was considered but then neglected as it odor could affect the child. Each cube hold a different letters, they are placed individually on the six sides of the cube in a large font.

The green and red cubes are used to start the installation and to stop it, green to start and red to stop. These colors may be familiar to us as we use them on a daily basis because they are signs that are common in the world. For example on phones the green icon is to call and the red one is to end the call. Therefore these two colors were chosen to make it more familiar to autistic children and to integrate them in the society.

   The letters and words were picked based on the observation [23] of the 

academic session which taught the child familiar words that they use on a daily basis like “ akol “ (eat), “ nadi” (play), “ bye bye “ and “madrasa” (school). The target while picking the words was to have different letters at the beginning of the word just like children are taught the ABCs around the world; a for apple, b for boat, c for cat, etc…The letters and words chosen are in Arabic as the aim of this educational game is to teach the children the native language. The letters were written on Adobe Illustrator, for the quality. They were all written in black with a white font so that the child does not relate any color to the cubes.

    The images used were all drawn on Adobe Illustrator. The images used 

where a reproduction of the ones the Advance center uses, therefore it is easier for the autistic child to recognize them as they are used on a daily basis during Academics session and in their schedule (see 2.3.2). One of the images, which correspond to the word “ madrasa “ (school), was designed from scratch representing the front view of the school. The images were changed after the first observation as the child had difficulties linking them to the words as they were not familiar. Therefore a redesign of the images was done.

    The sound is pre-recorded. It consists of four different words that are “ 

akol “ (eat), “ nadi” (play), “ bye bye “ and “madrasa” (school). The sound is then combined with the images, the letters and the words. Each movie has either and image, a letter or a word and a sound that is repeated four times so that it becomes more familiar to the child. The movies were done on Final Cut Pro.

The camera is used to detect the color of the cube, as it would be difficult to recognize the letter because it would require a different type of programming that would need to recognize the letter no matter how it is positioned in front of the camera. . The camera used is not the one from the laptop as the screen’s reflection on the cube disturbs the color detection and as well gives little space for the child to play and autistic children could not connect the built-in camera to its function. Therefore a usb-camera is connected to the computer and is placed facing the space specified for the placement of the cubes nearby so that the child views the screen and the cubes without having to struggle. The use of a soft light is required for the color detection as it is monotone.

    To guide the child throughout the interface, an animated colored emoticon 

was designed using Adobe Flash. The choice of animating the emoticon was to add a playful object and to replace an avatar, as it was mentioned in one of the researches [5], that the use of avatar enhances the performance. The use of emotion was as well to show the child if the answer he chose is correct or not .The emoticon are in four different colors yellow, orange, purple and blue. There are two types of animations, the happy face and the sad one. If the child chooses the right cube, for example the blue one, a blue emoticon shows a happy face. If he chooses any of the three other colors, a blue emoticon shows a sad face.

    The setting of the game is as follow. The game takes place in a room 

where no other machines or other objects, that could capture the child’s attention, are present. The screen is placed on a table close to the wall. On the right, there is a stand with the camera placed on top of it. On the left are the six cubes not necessarily aligned, but not on top of each other so that it is easier to grab when needed. The delimited area is placed in front of the screen so that the child could look at the screen while placing the cube. The setting is mainly black with colored motifs as black is the most suitable color; any other color would affect the color detection.


Letters & Words





The game is placed in a corner of the room where only the child is present accompanied by his teacher, it consists of a one-to-one session where the role of the teacher could be at first teaching the child how to use the interface and later on just as observer of the child’s progress. The screen is placed on a table and in front of it is the specified area to place the cube .It is mainly black and a white line limits the area. The connected Camera is placed on the right of the screen on a stand so that it captures the area where the cube will be placed on completely. The set of 6 cubes are placed on the right side of the screen. They are not placed in order.

Before starting the game, the camera must open so that the color detection could work but this is the only task of the adult. The child only deals with the cube.

 To start the game, the child must hold the green cube and place it in the 

area so that the camera detects the color and starts the installation. After it starts, a letter appears on the screen and the word is repeated four times. During this time the child has to put the cube corresponding to the letter on the screen in the designated area. If it is the right cube, a happy emoticon with the same color as the cube is triggered on the left side of the screen, if not, a sad emoticon with the color of the right cube is triggered at the place of the happy one. If the child got the right answer or the wrong one, the next letter appears. He has to repeat the same steps as previously for the rest three letters.

     After the fourth letters appear, an image appears, this is part of the 

second level. The steps are repetitive so that the child does not have to perform a difficult task but rather concentrate on choosing the appropriate letter. After the fourth image appears and the child picks a cube and the emoticon appears, the first word appears on the screen, this is part of the third level. As the previous levels, the child has to grab and show the right cube to the camera. After the fourth word and the steps that follow are fulfilled, the child has to stop the installation and therefore he has to grab the red cube and place it correctly in the designated space.

    Throughout the game, there is no time limit and the teacher’s presence is 

only to teach the child how to use the interface and to encourage him hence the child is not under pressure and the environment is rather relaxed.

    Even if the child does not place the right cube that matches the letter on 

the screen, he continues with the next letter so that he is not affected and so that he does not repeat the same mistake twice, the emoticon shows the right color he should have used. The cubes do the total control of the interface; hence the child does not have to learn how to use a computer the only task required is to move cubes around at the correct time.

The idea of the game is to help with the education process of the child, learning in a playful way.

Game setting

Navigational Structure

Navigational Structure

The reason behind choosing this specific tri-level structure was to have a repetitive sound for a different visual; therefore the child would link the images to the same sound, and not to have confusion with more than one image at the same time. It takes the child from letter recognition to relating the image and the sound to the letter and at last relating the word to the letter. This method is used to avoid confusion and to have more than one visual relating to the same letter therefore the child learns to recognize different ways of using a letter.



The test of the interactive game had for aim to observe if the child responds and if it was suitable for her.The game was tested on a nine-year-old autistic girl who started learning the Arabic alphabets 10 weeks earlier.

The testing was done on two separate days .The first day was to introduce the cubes, explain the game manually and show her how it works. And the second one was to test the game as it was designed.

I worked with her during a one to one session. First, I showed her teacher the different components of the game and how it functions. Her feedback was positive and told me that the child will be able to play with the game without problems. She asked me as well to explain to the child the game to see her reaction and response to my explanation.

Afterwards I prepared my installation, I put the laptop in the back center of the table, in front of the laptop was the black square where she was suppose to place the cubes in. Then the teacher asked her to sit facing the laptop.

At the beginning I asked her to recognize the cube’s colors and the letters on the cubes, both were successful, then I showed her how the game works. I triggered the game manually to show her the four letters on the screen, appearing one by one, explaining to her where to put the cube, what happens if she has the right cube and if she has the wrong one. She was very cooperative and understood right away the system.

After showing her the three levels, I started the game manually again as if she was actually playing it. I only had to ask her “ where is the similar letter? What is the girl doing? And what is the letter that corresponds with the word?” She was taught as well where to place the cube and to put it back in its place once another letter appears on the screen.

The only problem she faced was to relate the pictures with the appropriate letter. She would recognize the picture and say corresponding word but she would find it hard to link it to a letter.

The teacher advised me to have a cardboard with the same colors but the letters would be replaced with words matching the picture therefore she could link them together.

The game was repeated several times and each time she showed some improvements, seeing that she would place the right cube when the letter appears. She was concentrated due to the fact that colors attract her but it was not constant. She had several attempts to use the track pad on the laptop in front of her since I was told she likes playing on the computer and knows how to use it.

During the game, she would only babble words that were not understood and only said one time, “ I don’t want to “ but afterwards she continued playing the game normally. When the game was over, she grabbed cube-by-cube piling them on top of each other, separating the ones with letters from the ones without.

The teacher was most of the time standing in the back where the girl would not see her to observe her reactions, she would sometimes sit behind her to encourage her. I was told that the girl liked the game seeing that she was responding and that during the next performance she would improve since she was taught how to play with it.

The game took approximately 20 minutes to be completed. In the next session, the game will be working automatically using the color detection and less guidance will be provided to observe how she interacts using only the interactive game.

On the second day of testing, while installing the game, the girl came straight away to the laptop, sat in front of it and started using the track pad. The setting was different than the previous testing, because the camera detection would not work unless the frame was empty. If it were put in front of her, her hands would have disturbed the detection. Therefore the laptop was placed close to the girl and the camera as well as the black square were placed at the back area of the table (See figure). This time, the teacher was only observing and was sitting behind us in a way that the child does not see her.

The only explanation needed was to inform the girl that to be able to use the game, she needs to put the green cube first, and if she does not want to play anymore, she should place the red one. This process did not take time and the game was started shortly. The first trial was without sound

During the first trial, I had to assist her providing her with words and asking her “where is the similar letter?” “ What does the picture refer to? “ and “ what word is that?” .I also had to remind her of the place where she is suppose to put the cube in as the setting was different than last time. She only hesitated with words and images. She responded well to this first trial, she answered 7 out of 12 correctly. Most of the time, she was concentrated and would point at the screen but did not pay much attention to the emoticons. The first trial took 35 minutes as color detection took time.

The second trial was more successful than the first one because she already knew the process of the game. Before starting the game, I showed her that the emoticons change depending on her answer. She placed at the beginning the green cube, and then played the game this time with the sound. It was easier for the child to link the letters and images with sound than without it. During the second trial, I only interfered if there were technical problems due to the camera tracking, otherwise I was only observing her.

Like the first trial, she was concentrating and paying attention to the sound, she would repeat each word. She recognized the emoticons and would say out loud “ happy” or “ sad”. More than once, she expressed amusement by giggling; even the teacher noted that she was enjoying the game. After the game was over, she started to pile the cubes on top of each other. She did not use the red cube to end the game.

The second trial took approximately 30 minutes to be completed. She got 9 over 12 answers correct.



The interaction with sounds, visuals and colors provided a more interesting experience to the child. Sound offered her guidance throughout the game and helped her understand more the system of the interactive game. [] Visuals helped her recognize the sound and connect them to the illustrations already seen before. [4,5,6,14,16,19,20,21,22] The colors provided a playful aspect to the game and a kind of reward, as the child is attracted to colors. Teaching the child how to use the interface was mandatory which supports the suggestion of teaching them in advance how to use objects. [6,16,18] Repetition was one of the main successful tools used in the installation as it was proved previously that autistic children feel comfortable with repetitive actions. Since the game was a tailor-made design, it gave the child motivation to play. [18] Computer games have been proven to be a tool that could be used intensively with autistic children as they respond positively to them. [3,6,14,18]

Autistic children have capacities to learn faster if appropriate tools were applied. The child should be offered a variety of learning tools that attracts him according to his preferences. To be able to design an interactive game for autistic children, there must be a thorough research that revolves around the child’s needs and preferences. The needs could be psychological and educational. The preferences are based on the child’s passion, as it is necessary to combine it with education for a better result.

Nowadays, teachers in special needs schools are open to new technologies in their methods therefore more games should be implemented to enhance the education not only improving literacy but as well developing social skills, artistic skills, etc. By improving autistic children’s abilities, they become more active in society and could be a source of inspiration to many.


The future works of this game would include more letters therefore more colors and a more sophisticated tracking system that does not require a specific lighting. But the letters won’t be all at the same time; they would be added to the game gradually depending on the child’s progress. The use of programmed cubes would also be considered so that fewer cubes would be used for more letters; one cube would display more than one letter.

 It would include as well the consideration of diversity in behavioral patterns. 

Before the child starts the game, the teacher will select a specific pattern depending on the preference of the child, as each autistic child have a passion about certain objects. [23]. It would be a tailor-made game that fits each and every autistic child that is ready to start learning the alphabets.

The further research would test the effectiveness of the game and if it enhances the learning process which requires a constant evaluation and more than just one autistic child.