We lack effective tools for assessing behavior and cognition in the most severely impaired individuals with autism. Jeffrey Munson and his colleagues at the University of Washington are addressing this need through the application of motion-controlled gaming technologies.

Munson’s goal is to build experiences that offer individuals with extremely limited communication abilities an opportunity to ask questions of their own and discover the principles that govern the games. He and his team aim to design tasks that are worthy of participants’ attention and active involvement. This way, the learning process can be documented as it takes place.

By recording movement in high fidelity as well as the game context within which it takes place, researchers can make inferences regarding underlying cognitive processes. This approach is suitable for use with young children as well as adults, and it’s easy to deploy in the laboratory, home or classroom. Motor movement is one of the few avenues available for understanding the underlying cognitive competencies and impairments of nonverbal individuals with autism. Most studies of motor behavior focus on older or higher-functioning children with autism spectrum disorders.

In their pilot work, Munson and his team introduced a virtual balloon-popping game to several nonverbal children and adolescents with autism. Using a Microsoft Kinect platform, players see a camera image of themselves with a colored ball attached to each hand. Colored balloons floating around the player pop if touched by their hand with a matching color. Although repetitive movements were common among the participants, they were often incorporated into successful play.

Early observations suggest that players reduce their speed of movement in proportion to the precision required to successfully complete the task. This demonstrates the self-awareness necessary to modulate motor output to maximize success.

The researchers are continuing to develop dependent measurements of these movement data in order to quantify repetitive errors that individuals made during the game. They are also looking at bilateral symmetry of movement — whereby left and right side movements are symmetrical — and changes in movement patterns as a function of the game context.