Copyright © Baylor® University. All rights reserved.
Report It | Title IX | Mental Health Resources | Anonymous Reporting | Legal Disclosures
Systematic Review seeks to systematically search for, appraise and synthesize research evidence, often adhering to guidelines on the conduct of a review.
Systematic reviews are research projects which are more than just an organized collection of articles; they are primary studies which provide synthesized information, built on other studies but providing new insight into a phenomenon, topic, or situation. The most straightforward and useful definition of systematic reviews, applied across all questions and disciplines is that they are reviews of the research literature using systematic and transparent methods (EPPI Centre 2015; Gough, Oliver, and Thomas 2012). These processes are the following: plan, identify, evaluate, collect and combine, explain, and summarize (PIECES) what can be ascertained from existing literature in order to answer a well-defined research question.
Excerpt from:
Foster, M. J., & Jewell, S. T. (Eds.). (2017). Assembling the pieces of a systematic review : A guide for librarians. ProQuest Ebook Central https://ebookcentral.proquest.com
Planning Step |
Explanation |
|---|---|
| Determine the purpose of the review |
To help you decide what type of review you want to conduct. |
| Pose a research question |
Which should use an established framework, if possible. |
| Conduct exploratory searches |
To identify:
|
| Determine whether the project is feasible |
Based on time, resources, originality, relevance, etc. |
| Pick a team |
Of at least 3-6 members, including:
|
| Select preliminary synthesis and analysis methods |
Will depend on types of studies (can be qualitative, quantitative, or integrative--a review that integrates both quantitative and qualitative studies) |
| Design a protocol |
Includes eligibility criteria, a timeline, data management, and project management. May be tweaked as the research progresses but gives direction for the project. |
Identifying Step |
Explanation |
|---|---|
| Identify databases you will search |
Including both general and subject specific databases. |
| Create a search based on your research question |
Using advanced search strategies such as combining synonyms with OR and using wildcards and proximity operators to find variants. |
| Translate the search | For each database using appropriate controlled vocabulary and correct syntax. |
| Store the results of each search |
Separately, using a citation manager. |
| Document each search |
Including:
|
| Search grey literature |
Such as:
to address publication bias |
| Search other sources | Such as reference lists, relevant journals that are not entirely indexed in databases, and researcher bibliographies. |
Evaluating Step |
Explanation |
|---|---|
| The citations from the searches are de-duplicated |
These are combined into a master list with the resulting number of citations documented. |
| Each article is evaluated for relevancy |
At least two reviewers determine whether a citation meets the eligibility criteria for inclusion as set up in the protocol. Often inclusion/exclusion can be based on title and abstract but sometimes full-text is required for the determination. Tools are available that can help track the status of each citation. Some have AI which can speed up the process. |
| Full text is obtained |
For those citations which inclusion cannot be determined by title and abstract alone. For all citations to be included in the synthesis. |
| Master list of studies is complied |
Some studies may be written up in more than one article. Some articles may include more than one study. |
| Each study is evaluated for quality and bias (Critical Appraisal) |
At least two reviewers determine the quality of each study. A valid quality assessment tool or checklist appropriate for the type of study should be used. |
| A list of studies to be included in the synthesis is compiled | If any studies are deemed to be of lower quality or biased, documentation of their inclusion or exclusion is necessary. |
Collecting & Combining Data Step |
Explanation |
|---|---|
| Confirm synthesis and analysis methods |
Will depend on the types of studies. Can be qualitative, quantitative, or integrative. |
| Determine the data elements |
That are needed to answer the research question. Drafting potential evidence tables and figures that can help confirm what data should be eventually shown. |
| Develop forms |
Determine how each data element will be extracted/coded. Categorical coding allows for faster and more consistent extraction. Open coding allows for more information and nuance but takes longer to extract and analyze. |
Explaining the Synthesis & Analysis |
Explanation |
|---|---|
| Narrative explanations |
Can describe trends, themes, frameworks, perspectives, characteristics, quality, etc. Especially useful for empirical research. Can use structured narratives. Often accompanied by tabular explanations. |
| Tabular explanations |
Uses tables to explain the synthesis. Can be used to describe study characteristics, study measures, study quality, study results, etc. Accompanies narrative explanations. |
| Graphical explanations |
Uses graphical methods to explore and present data. Can include concept maps, forest plots, harvest plots, idea webs, logic models, mind maps, and network analysis. |
Identifying Step |
Explanation |
|---|---|
| Identify your audience(s) |
|
| Choose your summary type |
|
| Review Standards |
Most standards will include these sections:
|
| Write Your Summary |
Make sure your work is clear, auditable, replicable, and transparent. |
| Submit Your Summary |
|
| Search grey literature |
Such as:
to address publication bias |
| Search other sources | Such as reference lists, relevant journals that are not entirely indexed in databases, and researcher bibliographies. |
Framework - An established structure that helps define the research question. Can address different disciplines and question types.
For example: PICO is the most common and well-known standard framework for medical and health systematic reviews.
Framework |
Discipline/Question Type |
|---|---|
| PICO (Richardson et al. 1995) Patient, Intervention, Comparison, Outcome |
Clinical medicine |
| BeHEMoTh (Booth and Carroll 2015) Behavior of interest, Health context (service/policy/ intervention), Exclusions, Models or Theories |
Questions about theories |
| CHIP (Shaw 2010) Context How Issues Population |
Psychology, qualitative |
| CIMO (Denyer and Tranfield 2009) Context Intervention Mechanisms Outcomes |
Management, business, administration |
| CLIP (Wildridge and Bell 2002) Client group, Location of provided service, Improvement/Information/ Innovation Professionals |
Librarianship, management, policy |
| COPES (Gibbs 2003) Client-Oriented, Practical, Evidence, Search |
Social work, health care, nursing |
| ECLIPSE (Wildridge and Bell 2002) Expectation, Client, Location, Impact, Professionals, SErvice, |
Management, services, policy, social care |
| PEO (Kahn et al. 2003) Population, Exposure, Outcome |
Qualitative |
| PECODR (Dawes et al. 2007) Patient/population/problem, Exposure, Comparison, Outcome, Duration, Results |
Medicine |
| PESICO (Schlosser and O’Neil-Pirozzi 2007) Person Environments Stakeholders Intervention Comparison Outcome |
Augmentative and alternative communication |
| PICO specific to diagnostic tests (Kim et al. 2015) Patients/participants/population, Index tests, Comparator/reference tests, Outcome |
Clinical medicine |
| PICO+ (Bennett and Bennett 2000) Patient, Intervention, Comparison, Outcome, +context, patient values, and preferences |
Occupational therapy |
| PICOC (Petticrew & Roberts, 2006) Patient, Intervention, Comparison, Outcome, Context |
Social sciences |
| PICOS (Moher et al. 2009) Patient, Intervention, Comparison, Outcome, Study type |
Medicine |
| PICOT (Richardson et al. 1995) Patient, Intervention, Comparison, Outcome, Time |
Education, health care |
| PIPOH (ADAPTE Collaboration 2009) Population, Intervention, Professionals, Outcomes, Health care setting/context |
Diagnostic questions |
| ProPheT (Booth et al. 2016) Problem, Phenomenon of interest, Time |
Screening |
| SPICE (Booth 2004) Setting, Perspective, Interest, Comparison, Evaluation |
Social sciences, qualitative, library science |
| SPIDER (Cooke et al. 2012) Sample, Phenomenon of Interest, Design, Evaluation, Research type |
Library and information sciences |
| WWH Who What How |
Health, qualitative research |
Adapted from Foster, M. J., & Jewell, S. T. (Eds.). (2017). Assembling the pieces of a systematic review : A guide for librarians.
These websites provide checklists for critical appraisal of different types of studies.
Adapted from Foster, M. J., & Jewell, S. T. (Eds.). (2017). Assembling the pieces of a systematic review : A guide for librarians.
You will need to extract data from the selected studies to synthesize and analyze. In addition to collecting general information about each study and those extracting the data, you may want to consider if they are relevant to your research question.
Category |
Data Elements |
|---|---|
| Participants | Total number of participants, Setting, Diagnostic criteria, Age, Sex, Race/Ethnicity, Location (country, state, county, etc.), Co-morbidities, Socio-demographics, Spectrum of presenting symptoms and current treatments, Date of study, Date of recruitment and follow-up, Participant sampling |
| Intervention | Total number of intervention groups, Specific intervention, Intervention details, Integrity of intervention |
| Outcomes | Outcomes and time points (i) collected & (ii) reported, Outcome definition (with diagnostic criteria if relevant), Unit of measurement |
| Comparisons | Comparison |
| Results | Sample size, Missing participants, Summary data for each intervention group, Estimate of effect with confidence interval ( P value), Subgroup analyses, Adverse events and side effects for each study group |
| Interpretation | Overall evidence, Generalizability: external validity of trial findings |
| Objectives | Research questions and hypotheses |
| Method | Reference standard and its rationale, Technical specifications, Study design, Total study duration, Sequence generation, Allocation sequence concealment, Blinding, Methods used to generate random allocation sequence, implementation, Other concerns about bias, Methods used to compare groups for primary outcomes and for additional analyses, Methods for calculating test reproducibility, Definition and rationale for the units, cutoffs and/or categories of the results of the index tests and reference standard, Number, training, and expertise of the persons executing and reading the index tests and the reference standard, Participant flow |
| Qualitative | Noting patterns and themes, Seeing plausibility (ensuring conclusions make good sense), Clustering, Making metaphors, Counting, Making contrasts/comparisons, Partitioning variables, Subsuming particulars into the general, Noting relations between variables, Building a logical chain of evidence, Making conceptual/theoretical coherence |
| Miscellaneous | Funding source, Key conclusions of the study authors, Clinical applicability, Miscellaneous comments from the study authors, References to other relevant studies, Correspondence required, Miscellaneous comments by the review authors |
Adapted from Jonnalagadda, S.R., Goyal, P. & Huffman, M.D. Automating data extraction in systematic reviews: a systematic review. Syst Rev 4, 78 (2015). https://doi.org/10.1186/s13643-015-0066-7 and
Booth, A. (2016). Systematic approaches to a successful literature review (Second edition.). Sage.
Copyright © Baylor® University. All rights reserved.
Report It | Title IX | Mental Health Resources | Anonymous Reporting | Legal Disclosures