In education and learning we cannot isolate our research objects from outside influences, unlike in positive sciences. In a physics experiment we would carefully select variables we want to measure (dependent variables) and variables that we believe could influence those (independent variables). In education this is not possible. Even in Randomized Controlled Trials (RCT), put forward by researchers as Duflo and Banerjee (see my post that discusses their wonderful book ‘Poor Economics’) as a superior way to investigate policy effects, we cannot, in my opinion, fully exclude context.
This is why, according to Diana Laurillard, many studies talk about the ‘potential’ of technology in learning, as it conveniently avoids dealing with the messiness of the context. Other studies present positive results, that take place in favourable external circumstances. Laurillard argues that the question if technology improves education is senseless, because it depends on so many factors:
There is no way past this impasse. The only sensible answer to the question is ‘it depends’, just as it would be for any X in the general form ‘do X’s improve learning?’. Try substituting teachers, books, schools, universities, examinations, ministers of education – any aspect of education whatever, in order to demonstrate the absurdity of the question. (Laurillard, 1997)
In H810 we discussed theories of institutional change and authors such as Douglas North and Ozcan Konur, who highlighted the importance of formal rules, informal constraints and enforcement characteristics to explain policy effects in education. Laurillard talks about ‘external layers of influence’. A first layer surrounding student and teacher (student motivation, assessment characteristics, perceptions, available hard- en software, student prior knowledge, teacher motivation to use technology etc.) lies within the sphere of influence of student and teacher. Wider layers (organisational and institutional policies, culture of education in society, perceived social mobility…) are much harder to influence directly.
That doesn’t mean she believes educational research is impossible. She dismisses the ‘cottage industry’ model of education (See this article from Sir John Daniel on the topic), in which education is seen as an ‘art’, best left to the skills of the teacher as artist. Rather, she argues for a change in direction of educational research.
Laurillard dismisses much educational research as ‘replications’ rather than ‘findings’, a statement that echoes the plea from Clayton Christensen to focus more on deductive, predictive rather research than descriptive, correlational studies. He argues to focus less on detecting correlations and more on theory formation and categorisation of the circumstances in which individual learners can benefit from certain educational interventions. A body of knowledge advances by testing hypotheses derived from theories. To end with a quote from the great Richard Feynman (courtesy the fantastic ‘Starts with a Bang‘ blog):
“We’ve learned from experience that the truth will come out. Other experimenters will repeat your experiment and find out whether you were wrong or right. Nature’s phenomena will agree or they’ll disagree with your theory. And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven’t tried to be very careful in this kind of work.” -Richard Feynman