June 20, 2016
Study focuses on use of instructional videos to aid problem solving
WEST LAFAYETTE, Ind. – New research aims to help educators quantify how the best students perform problem solving with the aid of instructional videos, a step toward learning how to better coach students in difficult engineering curricula.
The research features a new experimental format that uses data from a system that tracks students' "eye gaze" with cameras, also documenting individual problem-solving methods with "smartpens" that record handwriting and audio. Unlike previous research, the new approach defines the order in which a student performs specific actions, which could be crucial to the success of problem-solving methods, said Edward Berger, a Purdue University associate professor of engineering education and mechanical engineering.
Twenty-four students participated in the study.
"We give them a problem to solve, and we record their actions as they engage with the video," he said. "Everybody in the world makes instructional videos now. But what we don't currently understand is exactly how students use videos in support of their problem solving. So we watch them use a video while they solve a problem, which has not been done before in any significant depth."
The results suggest that students' problem-solving methods can be broadly described by several "archetypes."
"If we can identify which archetypes are more successful, then we can apply that knowledge to better coach students," Berger said. "We can say, here is the pattern of usage we see in students who are really good at solving this type of problem."
The research is detailed in a paper being presented during the American Society for Engineering Education's Annual Conference & Exposition, June 26-29 in New Orleans. The paper was authored by Berger and doctoral student Michael Wilson. A video about the research is available at https://youtu.be/y8jbsf2Abio.
The students were graded on the problem, and their grades also were recorded during a sophomore-level mechanical engineering course that allows students to access hundreds of instructional videos and animations.
The experiment takes about 30 minutes to complete.
"We have a code structure to characterize various actions and quantify exactly what the student is doing," Berger said. "These actions could be something they write, something they do with the video and even something they say because we instruct the students to speak out loud during the exercise. We are identifying and trying to understand these archetypes."
Among the 24 students, the researchers have identified about four such archetypes thus far.
"Say they took 25 minutes to do this problem, they spent eight minutes looking at the video, and here is exactly what they were looking at in the video while they solved the problem. Did they watch the same part of the video more than once? Did they focus on a diagram or the equation, or something else? Then we look at all of the data and say, 'what are the key things they did when they were solving this problem and how did they engage with the video? ' "
The actions are then charted using a system of "code stripes" to visualize the specific approaches and tools students used to solve the problem, and in what order.
The experiments are related to an area of research called cognitive load theory, which attempts to break down how hard the brain is working at a particular task.
"The challenge is that it's really hard to accurately measure cognitive load, and we are using one of several available approaches," Berger said. "We are trying to understand how access to a video helps a student solve problems. Does it help them manage their cognitive load while they are doing the problem?"
Future work will aim to conclusively define the various archetypes.
Purdue researchers also are conducting a new project funded by the National Science Foundation to understand how students use the videos in the Purdue Mechanics Freeform Classroom, pioneered by Charles Krousgrill, a professor of mechanical engineering. The Freeform Classroom approach has been used since 2011 in two mechanical engineering core courses with hundreds of students enrolled annually. The new study is led by Jennifer DeBoer, an assistant professor of engineering education.
Writer: Emil Venere, 765-494-4709, firstname.lastname@example.org
Source: Edward Berger, 765-496-0193, email@example.com
A laboratory study of student usage of worked-example videos to support problem solving
Edward Berger and Michael Wilson
School of Engineering Education, Purdue University
Despite the commonplace usage of video resources for engineering instruction, an understanding of precisely how students use such videos to support their problem solving and learning is incomplete. Researchers generally find that both students and faculty like using instructional videos (if "well constructed"), especially in the format of so-called "worked examples" in which an expert records a problem solution for learner consumption. Cognitive load theory (CLT) has successfully affirmed instructional worked-example interventions as more effective and efficient than problem solving in novice-phase skill acquisition. However, most worked-example studies look at pre/post performance on problem solving in which the worked-example is the intervention, rather than studying student use of the worked-example itself in great detail. This study begins to address the gap in usage understanding. In this laboratory-based research, we studied problem-solving processes of a group of 24 students enrolled in a required mechanical engineering sophomore-level course. In the experiment, students were presented a dynamics problem to be solved, provisioned with an equation sheet, an online calculator, and a video described as "potentially useful." Real-time data about student problem solving process and use of the video was captured via a Livescribe smartpen and a Mirametrix eye-gaze capture system (which captured their interactions with the video). Pre- and post-surveys about student attitudes about technology, perceptions of task difficulty and academic transcript information are also included in the data set. Experimental videos and transcripts were coded for themes, and data about both task efficiency and task performance were extracted from the experimental evidence. Taken together, the results suggest that student usage of video resources can be broadly described by several archetypes, although in this study successful problem solution was possible regardless of archetype. These results will continue to inform academic coaching of students in our classes about optimal use of video resources.