Don’t believe the Hype…
19 January 2018
How do we make sense of the evidence?
As Teachers, we are extremely busy people determined to do the very best we can for students in our charge. We seek and in some cases are bombarded with many types of information, all claiming to be proven by the latest evidence. On the surface, all of these sources contains potential sliver bullets that could make the difference to our pedagogy or practice and ultimately to the progress and achievement of our pupils.
The information we receive can be classified into 4 main groups¹:
- Primary Literature-mainly new material consisting of journal articles, conference papers etc.
- Secondary Literature-material in which information that has appeared somewhere else has been repackaged and spread more widely, this can be through review journals or academic textbooks.
- Grey literature-defined as “articles and information published on the internet without a commercial purpose or the mediation of a commercial publisher” ,for teachers this could be a Government report or a select group report”²
- Personal communication-rapidly becoming the most accessible and the most prevalent through the inexorable rise of social media.
As a teacher interested in research or a potential research lead , how do we make sense of the evidence? I have made use of the PROMPT criteria³ . A score of 1 indicating good research with a strong evidence base, 5 indicating poorly researched or lacking evidence.
The information in this article is not clearly communicated. The Muscle Food website gained immense traction and media interest (the hype) but was later forced to admit the claim was based on customer search trends not as a result of any scientific testing³
This information was presented on social media and could quite easily be picked up and used ‘off the shelf’ for staff CPD. The information given here is based on a sample size of 3 pupils from 2007⁴. The information clearly doesn’t match the needs of the searcher who presumably would be looking for evidence based behaviour interventions.
The results in this fascinating and incredibly important paper shows clear correlational data,however the language veers over into making causal claims throughout claim other scientists⁵. The author has published a recent popular science book which makes the same claims as the article and cements her negative stance on digital activities. she also provides paid consultancy work on the topic and is the go to voice on the subject for news outlets.
Does this affect an authors objectivity? Is there any clear bias in articles you read? Is the authors position of interest made clear and are there any hidden, vested interests?
VERDICT – More research needed to establish causation 2
More applicable to research reports
Here we are looking at whether the article we are reading is respected and reliable . Are the authors acknowledged experts in their field? It is also worth checking how many times their work has been cited by other authors in the same field.
This is a personal favourite of mine and despite the concept of learning styles lacking any evidence it is still very widely used. Some of the information on learning styles was produced almost 30 years ago and became widespread in schools from 2004. when we are reviewing articles ask yourself is it clear when the information was produced? Is the information obsolete? ⁶
Hopefully you will continue to engage with research and evidence and you will be able to exercise wisdom and judgement and in the words of Flavor Flav ⁷
Posted in: Evidence
Tags: Education, evidence, Method, Objectivity, Presentation, Provenance, References, Relevance, Research, Teachers, Timeliness, Verdict