Developments in Artificial Intelligence (AI) technology are allowing more and more sophisticated interaction between humans and computers. Now systems can recognise a smile or a shrug and be programmed to react accordingly.
The Echoes project are using this technology to develop a learning environment that they hope will help them learn more about how children’s social skills develop.
The project are working with children aged 5-7 in the Asperger’s Spectrum as well as typically developing children. Initial findings show that the Echoes environment could help enhance childrens’ communication skills.
About the project
- The project has funding of £1.2m and runs from December 2008 – November 2011
- The project leader is Dr Kaska Porayska-Pomsta of the London Knowledge Lab, Institute of Education
- Other team members are based at Heriot-Watt, Edinburgh, Birmingham, Dundee and Strathclyde Universities and at the University of Wales Institute, Cardiff.
- The 16 members of the interdisciplinary team include experts in AI, special needs education and multimodal environments.
The ECHOES environment is made up of a range of hardware and software and designed in conjunction with children, researchers, parents and teachers.
- large multi-touch-screen
- three webcams for detecting gaze and facial expressions
- virtual characters that use AI to react to children’s actions in real-time
- gaze and touch tracking software that informs the environment’s reactions to the child
- social signal processing and user modeling software
Emerging themes and findings
Initial observations from a recent study of attention and gaze following in children with ASD showed that they were able to learn to follow gaze cues while communicating with the ECHOES virtual character and that they treated the virtual character as an agent and as an equal partner in the interaction.
One very exciting observation is that some children were also able to self-regulate during the interactions, which surprised and delighted their daily carers.
In particular, some children displayed behaviours that, their carers say, they normally find difficult. These behaviours included sustained focus of attention, anticipation as well as knowingly searching for interaction cues from the virtual character.
These initial trials highlight the potential of technology such as the Echoes environment, whereby social contexts can be paced and enriched according to each child’s individual needs.
A BECTA Emerging Technologies video on ECHOES is available here: http://www.youtube.com/watch?v=Ku2dIif7hWA.
Echoes in a tweet: an interactive multimodal environment where children can experiment with touch & gaze in learning social skills
Whats next for Echoes and TEL?
- Project Team
- Project Video
- Project Website
- ECHOES TEL 1 Project (2006/2007)
- Project Poster
- Download a recent report from the Echoes project (PDF)