AR experience design for training purposes using Hololens
1 designers, 1 developer, 2 researchers
Mixed Reality Toolkit
Within recent years Augmented Reality (AR) has been developed as a way of adding information to the real-world vision. One of the use cases of AR can be its application for educating people in using such tools and devices.
In this research, we aim at investigating the effect of employing Microsoft Hololens and Mixed Reality on learning performance and compare its usability in users' experience with common tools like tablets and paper documents.
In this experiment, we decided to evaluate the learning process of operating a poster plotter. By preparing an instructional guide for tablets and the same content for the Hololens, we evaluated the effect of using the Hololense on the learning curve.
One of the use cases of AR can be its application for educating people in using such tools and devices.
In our experiment, we developed an instructional augmented reality application that will teach users in the operation of a poster plotter at the iSchool IT Lab.
Working with new tools and devices might need some training for many people. Existing methods of learning require practitioners to first learn and then do. It requires a considerable cognitive capability for many people and makes the whole process of learning cumbersome.
Designing a proof-of-concept that helps users learn a skill while doing it at the same time by using the benefits of Augmented Reality headsets
AR in Education
There are many benefits that can arise from the use of AR in education. state of mind and increased motivation enhanced understanding of content with the development of spatial ability and increased memorization of content, and etc.
Benefits of AR
Research results show that the most commonly cited benefits of AR included better learning performance, motivation, and engagement.
AR for training
Researchers tested AR efficiency and effectiveness compared to traditional training methods and found that AR platforms do have a promising result. This study suggested that using AR platforms for training purposes should be encouraged.
Based on the test plan, we prepared an instruction document to help users operate the printer step by step. We used this instruction in two formats, one for tablet or paper and the other for Microsoft Hololens.
Instruction pages on the tablet
Instruction pages on the Hololens (1st iteration)
System Usability Scale (SUS)
Behavioral Analysis & Observations
How might we improve users' experience in learning a new tool with the use of Mixed Reality HMDs?
For user testing, we designed a test plan and divided the work into two sections. The first section was testing users with a physical guidebook and the second section was two iterations of the same content in AR format with the use of Microsoft Hololens.
Total number of users tested
Users tested with guidebook
Users tested with Hololens
Users, who have had no experience of working with the machine before, have been asked to fill a pre-task questionnaire, consisting of four questions about their perception of the task and the level of their confidence to do the task. During the test, one researcher took the time of the process and another one observed the users' behavior and responded to users' questions.
First, 8 users were asked to follow the instructions on the tablet to operate the machine.
Observing the user operating without the Hololens
Observing the user operating with the Hololens
The Hololens part was conducted in two iterations. The first iteration relied mostly on Gaze and Airtap interaction with 8 users, which we found inconvenient for users and the results convinced us to develop the app with a voice command in dominance and we recruited another 5 more users to test.
First iteration on Hololens (Hand interaction)
Second iteration on Hololens (Voice interaction)
After doing the test, a post-task questionnaire with five questions similar to the pre-task questionnaire and a System Usability Scale (SUS) questionnaire was used as the post-test evaluation to measure the subjective usability of the application and the whole testing session experience.
The results of questionnaires were used to gather quantitative analysis of the process and the observation results were used mostly for qualitative evaluation of the user's experience.
Users’ lack of confidence to be able to complete the task without guidance or training was the common issue in the pre-test questionnaire. The average time for completing the task using the instruction on the tablet was 16 minutes. Post task questionnaire results show that even after doing the print job, users still do not feel confident doing this job alone.
in time of doing
in learning efficiency
For the first iteration with the Hololens, the average time of completion was a little more than 8 minutes. For the second iteration, the average completion time was about 6.5 minutes.
Rough Learning performance curve inspired by research results
The post-test interview and observation revealed some usability issues related to using Hololens. Some of these issues are reflected below.
Hand gestures slow down the process
Prefer to use voice command
background brightness affects the experience
The weight of the device hurts the nose
Field of view is a barrier and slows downs the process
Voice commands should work without gaze
In this research, we tested the effect of Augmented Reality, deployed on Microsoft Hololens, on learning performance. We found that Mixed-Reality can increase the performance to a considerable extent.
Though we had a convincing reason to use the same instructional content for tablet experiment and Hololens, we already didn't employ the power of Hololens at spatial recognition and Computer VIsion. In future works, If we can make more context-dependent content to annotate the exact parts of the tool or machine, the learning performance will be probably different.
Through doing this research, which was designed and implemented in 1 month, we had too many obstacles related to technology, time, and users. We had no previous experience of building for Hololens. So two members of the team focused on solving technical barriers and learning about the Hololens, the other two members focused on recruiting the users and implementing and documenting tests. It was a sample of great teamwork for us and we learned that with collaboration and hard work, complex problems can be tackled in a short period of time.