Around

Players navigate their space and search for shapes with an iPad and augmented reality (AR) technology identifies shapes and displays them on the screen. For typical learners, this technology can be used as an assessment tool to check for understanding. However, for autistic learners, it is a tool that bridges abstract concepts to reality. It fills in the current gap in traditional learning methods, and increase students’ motivations to learn and stay motivated.

Why Shapes?

Shape recognition is a fundamental skill that leads to recognizing letters and words. It is an essential skill that transfers to literacy skills. 

Primary and secondary research

• Orchestrating wholistic experience

• Ideating to prototyping

• Conduct user test

Research show that early color and shape recognition connects to the letter recognition. Also, visual processing of graphic shapes is a very fine spatial skill. Graphic shapes have precisely defined configurations and orientations that are crucial features for quick recognition for efficient reading (Coltheart, Rastle, Perry, Langdon, & Ziegler, 2001). This perception also involves seeing, recognition, preparation of actions and emotional responses (Treisman & Kanwisher, 1998). Also, for children with Low Functioning ASD, color and shape recognition is one of the early cognitive skills to be acquired. Those skills are considered as a foundation for other occupational therapy skills. (Kuhaneck, 2010).

• Children with ASD are more passive and show less active engagement with materials than do children without disabilities .

• Most children with ASD frequently lose motivation and attention from paper-based visual supports in the classroom even though active engagement is an important component to promote positive learning outcomes for the children with ASD.

• Children with ASD have a lower understanding of physical and social worlds and a low level of responsivity in the classroom.

We wanted to test how the children with ASD will interact and how much they would engage with AR technology. With Unity3D and Vuforia, we developed AR shape scavenger hunt game with the famous TV characters. Teachers chose the characters and shapes based on the children’s character preferences.

The participant was required to find the character which corresponds to the shape which the conductor asked the participant to find. The scavenger hunt activity was ten minutes long, and the participant was asked to find one shape at a time. The order of the scavenger hunt activity was Spongebob, Angrybird and Cookie Monster. If the participant finds the right character image for the shape, the participant was praised. For the non-verbal child, the teacher conducted the playtest with Proloquo2go application to communicate.

The children could learn how to interact with the AR technology with excitement and surprise.

• The children were enthusiastic to find the right character for the shape. They successfully paired the shape to the right character images. After they found the shape, they indicated the excitement and wanted to find another shape.

• All children wanted to replay the game and learn about the shapes.

Based on the playtest result, we decided to expand the prototype using Open CV technology to create markerless AR application which detects circles, squares, and triangles from any objects the user can see in daily life.

• Instead of using pre-chosen character images as a marker image for AR, the children will be able to point the camera to any real objects to learn the shape of the objects.