Guides: Evidence Synthesis: Types of Reviews

Review Types

Review Type	Description	Search	Appraisal	Synthesis	Analysis
Systematic review	Seeks to systematically search for, appraise and synthesis research evidence, often adhering to guidelines on the conduct of a review.	Aims for exhaustive, comprehensive searching.	Quality assessment may determine inclusion/exclusion.	Typically narrative with tabular accompaniment.	What is known; recommendations for practice. What remains unknown; uncertainty around findings, recommendations for future research.

Adapted from Grant, M.J. and Booth, A. (2009), A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal, 26: 91-108. https://doi.org/10.1111/j.1471-1842.2009.00848.x

Also useful
Sutton, Clowes, M., Preston, L., & Booth, A. (2019). Meeting the review family: exploring review types and associated information retrieval requirements. Health Information and Libraries Journal, 36(3), 202–222. https://doi.org/10.1111/hir.12276

**Systematic Review:** Seeks to systematically search for, appraise and synthesis research evidence, often adhering to guidelines on the conduct of a review.
Step	Difference from Standard Evidence Synthesis Process
Planning	Follow PIECES
Identifying	Follow PIECES
Evaluating	Follow PIECES
Collection	Follow PIECES
Explaining	Synthesis is typically narrative with tabular and perhaps graphical accompaniment. Analysis focuses on what is known leading to recommendations for practice. Analysis also focuses on what remains unknown and the uncertainty around findings leading to recommendations for future research.
Summarizing	Audience tends to be the research community and practitioners. Protocols are often registered Summaries are usually submitted to peer-reviewed journals.

Review Type	Description	Search	Appraisal	Synthesis	Analysis
Scoping Review	Preliminary assessment of potential size and scope of available research literature. Aims to identify nature and extent of research evidence (usually including ongoing research).	Completeness of searching determined by time/scope constraints. May include research in progress.	No formal quality assessment.	Typically tabular with some narrative commentary.	Characterizes quantity and quality of literature, perhaps by study design and other key features. Attempts to specify a viable review.

**Scoping Review:** Preliminary assessment of potential size and scope of available research literature. Aims to identify nature and extent of research evidence (usually including ongoing research).
Step	Difference from Standard Evidence Synthesis Process
Planning	The research question can be broader and more topical.
Identifying	Same as for systematic review
Evaluating	Review all citations for inclusion due to relevancy Do not need critical appraisal
Collection	Not needed
Explaining	Synthesis is typically tabular with some narrative commentary. Analysis characterizes quantity and quality of literature, perhaps by study design and other key features.
Summarizing	Audience tends to be the research community and practitioners but also policy makers and agencies Protocols are sometimes registered, and summaries submitted to peer-reviewed journals or as reports.

Review Type	Description	Search	Appraisal	Synthesis	Analysis
Rapid review	Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research.	Completeness of searching determined by time/scope constraints.	Time-limited formal quality assessment.	Typically narrative and tabular.	Quantities of literature and overall quality/direction of effect of literature.

**Rapid Review:** Assessment of what is already known about a policy or practice issue, by using systematic review methods to search and critically appraise existing research.
Step	Difference from Standard Evidence Synthesis Process
Planning	Does not need a team.
Identifying	Completeness of searching determined by time constraints. May search fewer databases and grey literature sources.
Evaluating	Review all citations for inclusion due to relevancy. Critical appraisal maybe truncated due to time limitations.
Collection	Data collection more focused with fewer data elements. Sometimes data is directly put into tables rather than using forms.
Explaining	Synthesis is typically narrative and tabular. Analysis focuses on the auantities of literature and overall quality/direction of effect of literature.
Summarizing	Audience tends to be policy makers and agencies. Summaries are often submitted as reports.

Review Type	Description	Search	Appraisal	Synthesis	Analysis
Narrative Review	Generic term: published materials that provide examination of recent or current literature. Can cover wide range of subjects at various levels of completeness and comprehensiveness. May include research findings.	May or may not include comprehensive searching.	May or may not include quality assessment.	Typically narrative.	Analysis may be chronological, conceptual, thematic, etc.

**Literature Review:** Generic term: published materials that provide examination of recent or current literature. Can cover wide range of subjects at various levels of completeness and comprehensiveness. May include research findings.
Step	Difference from Standard Evidence Synthesis Process
Planning	Can be conducted by a single researcher. Do not need to pose a research question using an established framework or establish a protocol.
Identifying	May use comprehensive searching as for systematic review.
Evaluating	May use formal criteria for inclusion due to relevancy. May include critical appraisal.
Collection	Not needed
Explaining	Synthesis is typically narrative. Analysis may be chronological, conceptual, thematic, etc.
Summarizing	Audience, summary type, and dissemination depend on the purpose of the review. Protocols are not registered.

Review Type	Description	Search	Appraisal	Synthesis	Analysis
Meta-analysis	Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results.	Aims for exhaustive searching. May use funnel plot to assess completeness.	Quality assessment may determine inclusion/exclusion and/or sensitivity analyses.	Graphical and tabular with narrative commentary.	Numerical analysis of measures of effect assuming absence of heterogeneity.

**Scoping Review:** Technique that statistically combines the results of quantitative studies to provide a more precise effect of the results.
Step	Difference from Standard Evidence Synthesis Process
Planning	Same as for systematic review
Identifying	Same as for systematic review
Evaluating	Same as for systematic review
Collection	Same as for systematic review May include sensitivity analysis
Explaining	Synthesis is graphical and tabular with narrative commentary. Meta analysis incorporates a numerical analysis of measures of effect assuming absence of heterogeneity.
Summarizing	Audience tends to be the research community and practitioners. Protocols are often registered Summaries are usually submitted to peer-reviewed journals.

Meta analysis is a statistical technique that can be used to aggregate the results of individual quantitative studies.

Deciding whether to base the analysis on a fixed- or random-effects model
- Fixed effect if the factor is assumed constant across studies and variation among studies is random
Computing a summary estimate of effect (if appropriate)
- Weighted based on precision of estimates
- Inverse variance weighting (sample size/measurement error)
Presenting the data (usually graphically)
- Usually a “forest plot” – point estimate and confidence interval of effect in each study
Evaluating possible reasons for heterogeneity of study results (i.e. why different studies produce different estimates)
- Cochran’s Q, Higgins I2
- Subgroup analysis, stratified analysis
- graphical (Galbraith plot, scatterplot of effect size vs factor)
- meta regression (I2, adjusted R2, Tau2)
Searching for evidence of publication bias and evaluating the influence that individual studies have on the outcome
- Funnel plot of each study’s SE against estimated effect
- Tests based on funnel plot (Begg, Egger)

Glossary of Terms

Analysis - assesses the strength of the evidence for drawing conclusions based on the synthesis. The trends and patterns can be used in comparisons, to discover explanatory or confounding variables, to develop themes or frameworks, to inform best practices, etc.

Bias - systematic error in research studies that can lead to erroneous conclusions. Can occur in clinical trials, systematic reviews, and all types of research.

Forrest Plot - also known as a blobogram. A graphical display designed to illustrate the relative strength of effects in multiple studies

Funnel plot - a graphical device for for exploring possible publication bias by plotting study size vs. effect size

Synthesis - summarizes what the literature says and informs the conclusions that are drawn in the systematic review and focuses on both the methodology and results of the studies

Review Type	Description	Search	Appraisal	Synthesis	Analysis
Critical review	Aims to demonstrate writer has extensively researched literature and critically evaluated its quality. Goes beyond mere description to include degree of analysis and conceptual innovation. Typically results in hypothesis or model.	Seeks to identify significant items in the field.	No formal quality assessment. Attempts to evaluate according to contribution.	Typically narrative, perhaps conceptual or chronological.	Significant component: seeks to identify conceptual contribution to embody existing or derive new theory.
Mapping review/systematic map	Map out and categorize existing literature from which to commission further reviews and/or primary research by identifying gaps in research literature.	Completeness of searching determined by time/scope constraints.	No formal quality assessment.	May be graphical and tabular.	Characterizes quantity and quality of literature, perhaps by study design and other key features. May identify need for primary or secondary research.
Mixed studies review/mixed methods review	Refers to any combination of methods where one significant component is a literature review (usually systematic). Within a review context it refers to a combination of review approaches for example combining quantitative with qualitative research or outcome with process studies.	Requires either very sensitive search to retrieve all studies or separately conceived quantitative and qualitative strategies.	Requires either a generic appraisal instrument or separate appraisal processes with corresponding checklists.	Typically both components will be presented as narrative and in tables. May also employ graphical means of integrating quantitative and qualitative studies.	Analysis may characterize both literatures and look for correlations between characteristics or use gap analysis to identify aspects absent in one literature but missing in the other.
Overview	Generic term: summary of the [medical] literature that attempts to survey the literature and describe its characteristics.	May or may not include comprehensive searching (depends whether systematic overview or not).	May or may not include quality assessment (depends whether systematic overview or not).	Synthesis depends on whether systematic or not. Typically narrative but may include tabular features.	Analysis may be chronological, conceptual, thematic, etc.
Qualitative systematic review/qualitative evidence synthesis	Method for integrating or comparing the findings from qualitative studies. It looks for ‘themes’ or ‘constructs’ that lie in or across individual qualitative studies.	May employ selective or purposive sampling.	Quality assessment typically used to mediate messages not for inclusion/exclusion.	Qualitative, narrative synthesis.	Thematic analysis, may include conceptual models.
State-of-the-art review	Tend to address more current matters in contrast to other combined retrospective and current approaches. May offer new perspectives on issue or point out area for further research.	Aims for comprehensive searching of current literature.	No formal quality assessment.	Typically narrative, may have tabular accompaniment.	Current state of knowledge and priorities for future investigation and research.
Systematic search and review	Combines strengths of critical review with a comprehensive search process. Typically addresses broad questions to produce ‘best evidence synthesis’.	Aims for exhaustive, comprehensive searching.	May or may not include quality assessment.	Minimal narrative, tabular summary of studies.	What is known; recommendations for practice. Limitations.
Systematized review	Attempt to include elements of systematic review process while stopping short of systematic review. Typically conducted as postgraduate student assignment.	May or may not include comprehensive searching.	May or may not include quality assessment.	Typically narrative with tabular accompaniment.	What is known; uncertainty around findings; limitations of methodology.
Umbrella review	Specifically refers to review compiling evidence from multiple reviews into one accessible and usable document. Focuses on broad condition or problem for which there are competing interventions and highlights reviews that address these interventions and their results.	Identification of component reviews, but no search for primary studies.	Quality assessment of studies within component reviews and/or of reviews themselves.	Graphical and tabular with narrative commentary.	What is known; recommendations for practice. What remains unknown; recommendations for future research.