Investigating concepts associated with psychology requires an indefinite amount of reading. Hence, good literature reviews are an inevitably needed part of providing the modern scientists with a broad spectrum of knowledge. In order to help, this blog post will introduce you to the basics of literature reviews and explain a specific methodological approach towards writing one, known as the systematic literature review.
Literature review is a term associated with the process of collecting, checking and (re)analysing data from the existing literature with a particular search question in mind. The latter could be for example:
- What are the effects of yoga associated with individual’s subjective well-being?
- Does brief psychotherapy produce beneficial outcomes for individuals diagnosed with agoraphobia?
- What personality traits are most commonly associated with homelessness in the modern literature?
A literature review (a) defines a specific issue, concept, theory, phenomena; (b) compiles published literature on a topic; (c) summarises critical points of current knowledge about the problem and (d) suggests next steps in addressing it.
Literature reviews can be based on all sorts of information found in scientific journals, books, academic dissertations, electronic bibliographic databases and the rest of the Internet. Electronic databases such as PsycINFO, PubMed, Web of Science could be a good starting point. Some of them, like EBSCOhost, ScienceDirect, SciELO, and ProQuest, provide full-text information, while others provide the users mostly with the abstracts of the material. Besides scientific literature, literature reviews often include the so called gray literature. This refers to the material that is either unpublished or published in non-commercial form (e.g., theses, dissertations, government reports, fact sheets, pre-prints of articles). Excluding it completely from a literature review is inappropriate because the search should be always as complete as possible in order to reduce the risk of publication bias. However, when reviewing the material on for example Google Scholar, Science.gov, Social Science Research Network, or PsycEXTRA it should be kept in mind that such search engines also display the material without peer-review and have therefore less credibility regarding the information they are disclosing.
When performing literature reviews, the use of appropriately selected terminology is essential, since it allows the researchers much clearer communication. In psychology, without some commonly agreed lists of terms, we would all get lost in the variety of concepts and vocabularies that could be applied. A typical recommendation for where to look for such index terms would be ‘Thesaurus of Psychological Index Terms (2007)’, which includes nearly 9,000 most commonly cross-referenced terms in psychology. In addition, electronic databases mentioned before sometimes prompt the use of the so-called Boolean operators, simple words such as AND, OR, NOT, or AND NOT. These are used for combining and/or excluding specific terms in your search and sometimes allow to obtain more focused and productive results in the search. Other tools to make search strategy more comprehensive and focused are also truncations – a tool for searching terminologies that have same initial roots (e.g., anxiety and anxious) and wildcards for words with spelling deviations (e.g., man and men). It is worth noting that the databases slightly differ in how they label the index terms and utilize specific search tools in their systems.
Among authors, there is not much coherence about different types of literature reviews but in general, most recognize at least two: traditional and systematic. The main difference between them is situated in the process of collecting and selecting data and the material for the review. Systematic literature review, as the name implies, is the more structured of the two and is thought to be more credible. On the other hand, traditional is thought to heavily depend on the researcher’s decisions regarding the data selection and, consequently, evaluation and results. Systematic protocol of the systematic literature review can be therefore understood as an optional solution for controlling the incomplete and possibly biased reports of traditional reviews.
THE SYSTEMATIC LITERATURE REVIEW
The systematic literature review is a method/process/protocol in which a body of literature is aggregated, reviewed and assessed while utilizing pre-specified and standardized techniques. In other words, to reduce bias, the rationale, the hypothesis, and the methods of data collection are prepared before the review and are used as a guide for performing the process. Just like it is for the traditional literature reviews, the goal is to identify, critically appraise, and summarize the existing evidence concerning a clearly defined problem.
Systematic literature reviews allow us to examine conflicting and/or coincident findings, as well as to identify themes that require further investigation. Furthermore, they include the possibility of evaluating consistency and generalization of the evidence regarding specific scientific questions and are, therefore, also of great practical value within the psychological field. The method is particularly useful to integrate the information of a group of studies investigating the same phenomena and it typically focuses on a very specific empirical question, such as ‘Does the Rational Emotive Therapy intervention benefit the well-being of the patients diagnosed with depression?’.
Systematic literature reviews include all (or most) of the following characteristics:
- Objectives clearly defined a priori;
- Explicit pre-defined criteria for inclusion/exclusion of the literature;
- Predetermined search strategy in the collection of the information and systematic following of the process;
- Predefined characteristic criteria applied to all the sources utilized and clearly presented in the review;
- Systematic evaluation of the quality of the studies included in the review;
- Identification of the excluded sources of literature and justification for excluding them;
- Analysis/synthesis of the information (i.e., comparison of the results, qualitative synthesis of the results, meta-analysis);
- References to the incoherences and the errors found in the selected material.
The process of performing a systematic literature review consists of several stages and can be reported in a form of an original research article with the same name (i.e., systematic literature review):
1: Start by clearly defining the objective of the review or form a structured research question.
Place in the research article: Title, Abstract, Introduction.
Example of the objective: The objective of this literature revision is to systematically review and analyse the current research on the effects of music on the anxiety levels of children in hospital settings.
Example of a structured research question: What are the most important factors associated with the development of PTSD in soldiers?
Tip: In the title, identify that the report is a systematic literature review.
2: Clearly specify the methodology of the review and define eligibility criteria (i.e., study selection criteria that the published material must meet in order to be included or excluded from the study). The search should be extensive.
Place in the research article: Methods.
Examples of inclusion criteria: Publication was an academic and peer-reviewed study. Publication was a study that examined the effects of regular physical exercise intervention on depression and included a control group.
Examples of exclusion criteria: Publication was involving male adults. Studies that also examined non-physical activities as interventions. Studies that were only published in a language other than English.
Tips: The eligibility criteria sometimes fit to be presented in tables.
3: Retrieve eligible literature and thoroughly report your search strategy throughout the process. (Ideally, the selection process is performed by at least two independent investigators.)
Place in the research article: Methods.
Example: The EBSCOhost and PsychInfo electronic databases from 2010 to 2017 were searched. These were chosen because of the psychological focus that encompasses psychosocial effects of emotional abuse in childhood. Search terms were ‘emotional abuse’, ‘childhood’, ‘psychosocial effects’, and ‘psychosocial consequences’. The EBSCOhost produced 200 results from the search criteria, while PsychInfo produced 467, for a total of 667 articles. […] Articles were rejected if it was determined from the title and the abstract that the study failed to meet the inclusion criteria. Any ambiguities regarding the application of the selection criteria were resolved through discussions between all the researchers involved.
Tip: Sometimes it is nice to represent the selection process in a graphical representation; in the form of a decision tree or a flow diagram (check PRISMA).
4: Assess the methodological quality of the selected literature whenever possible and exclude the articles with low methodological quality. Keep in mind that the quality of the systematic review depends on the validity and the quality of the studies included in the review.
Place in the research article: Methods.
Examples of the instruments available for evaluating the quality of the studies: PEDro, Jadad scale, the lists of Delphi, OTseeker, Maastricht criteria.
Tip: Present the excluded articles as a part of the selection process mentioned in step 3.
5: Proceed with the so-called characterization of the studies. Decide which data to look for in all the selected studies and present it in a summarized way. If the information is missing in some specific paper, always register it in your reports. (Ideally, the characterization of the studies is performed by at least two independent investigators.)
Place in the research article: Results.
Examples of the information that should and/or could be collected for characterization of the literature: authors, year, sample size, study design, aims and objectives, findings/results, limitations.
Tip: Sometimes results can be presented nicely in a form of a table depicting the main characteristics.
6: Write a synthesis of the results – integrate the results of different studies and interpret them in a narrative form.
Place in the research article: Interpretation, Conclusions.
Patterns discovered as results should be summarized in a qualitative, narrative form. Modulate one (or more) general arguments for organizing the review. Some trick to help you do this is to choose two or three main information sources (e.g., articles, books, other literature reviews) to explain the results of other studies through a similar way of organization. Connect the information reported by different sources and do not just summarize the results. Find patterns in the results of different studies, identify them, address the theoretical and/or methodological conflicts and try to interpret them. Summarize the principal conclusions and evaluate the current state on the subject by pointing out possible further directions.
The results emerging from the data that were included in such retrospective studies can lead to a certain level of credibility regarding their conclusions. Actually, systematic literature reviews are thought to be one of our best methods to summarize and synthesize evidence about some specific research question and are often used as the main ‘practice making guidelines’ in many health care disciplines. Therefore, it is no wonder why systematic reviews are gaining popularity among researchers and why journals are moving in this direction as well. This also shows in the development of more and more specific guidelines and checklists for writing systematic literature reviews (see for example PRISMA or Cochrane Handbook for Systematic Reviews of Interventions). To find examples of systematic literature review articles you can check Cochrane Database of Systematic Reviews, BioMed Central’s Systematic Reviews Journal, and PROSPERO. If you are aware of the concept of ‘registered reports’, it is worth mentioning that submitting with PROSPERO provides you with the option of publishing the latter as well. I suggest that you go through the list of useful resources provided below and hopefully, you can get enough information about anything related that remained unanswered. Now, I encourage you to try to be a little more to be systematic whenever researching some topic, to try to write a systematic literature review yourself and to maybe even consider submitting it to JEPS.
- Cochrane Database of Systematic Reviews: http://www.cochranelibrary.com/cochrane-database-of-systematic-reviews/
- Sampaio, R. F., & Mancini, M. C. (2007). Systematic review studies: A guide for a careful synthesis of scientific evidence. Brasilian Journal of Physical Therapy, 11(1), 77-82. doi:http://dx.doi.org/10.1590/S1413-35552
- Tuleya, L. G. (2007). Thesaurus of psychological index terms. Washington, DC: American Psychological Association.