About Systematic Reviews
Quality Assessment Tools for Systematic Reviews
Automate every stage of your literature review to produce evidence-based research faster and more accurately.
Systematic reviews, with or without meta-analysis are considered the highest level of evidence in medical literature [1]. There are different types of systematic reviews designed to help answer various research questions. For example, to answer broad questions, and identify key concepts in a research area, systematic scoping reviews are employed. For guidance on conducting systematic scoping reviews, you can learn more at the link provided. Since the intended purpose of a scoping systematic review is to gather information as opposed to recommending a clinical practice, quality assessment is either not undertaken or is not as stringent as in the case of a typical systematic review.
As systematic reviews are considered the highest form of evidence, conducting a thorough quality assessment of the study is required. In this article, we will look at the quality assessment tools which can be used to assess the internal validity of a systematic review.
What Is Quality Assessment and Why Is It Important?
Quality assessment is also known as quality appraisal, critical appraisal, and risk of bias assessment, with the terms sometimes being used interchangeably. They refer to the assessment of the methodological quality, and rigor of the trials or studies included in a systematic review. Although systematic reviews are designed in a way to produce robust, reliable, and reproducible results, they are still open to biases and errors. Errors in the study design and implementation have the potential to bias the results in favor of one intervention over the others. Given the importance of systematic reviews in evidence-based medicine, we need to be aware of these biases. Quality assessment, therefore, helps in minimizing the risk of bias and increases confidence in review findings.
Assessing the quality of evidence of findings from a systematic review is as critical as the data analysis process. Studies that are conducted poorly are usually skewed; the critical appraisal procedure allows researchers to interpret them with caution.
Learn More About DistillerSR
(Article continues below)
Quality Assessment Tools For Systematic Reviews
There are various tools offered to assist quality assessment and critical appraisal of a systematic review. Some of them are listed below,
AMSTAR (Assessing the methodological quality of systematic reviews)
AMSTAR is a popular instrument for critically appraising systematic reviews of randomized controlled clinical trials. It was further developed to enable appraisal of systematic reviews of randomized, and non-randomized studies of healthcare interventions. The revised instrument AMSTAR 2 has 16 items, simpler response categories than the original AMSTAR, includes a more comprehensive user guide, and has an overall rating based on weaknesses in critical domains.
Cochrane Risk-Of-Bias (RoB 2) Tool
This is the recommended tool for assessing the quality and risk of bias within the randomized clinical trials included in the systematic review. Review authors assess risk of bias in six domains of potential bias as being either high, low, or unclear. The six domains of potential bias include selection bias, performance bias, detection bias, attrition bias, reporting bias, and other bias.
Joanna Briggs Institute (JBI) Checklist
The JBI Critical Appraisal Checklist for Systematic Reviews and Research Syntheses is an 11-item checklist for evaluating systematic reviews.
AHRQ Methods Guide For Effectiveness and Comparative Effectiveness Reviews
Provides detailed information on selecting eligibility criteria, searching the literature, data extraction, quality assessment, and other steps involved in the creation of a systematic review.
CASP-Systematic Review Checklist
The critical appraisal skills program has over 25 years of experience in developing detailed checklists for the quality assessment of different types of studies including systematic reviews.
Newcastle-Ottawa Scale (NOS)
This tool facilitates the appraisal of non-randomized studies included in the systematic review. Non Randomized studies, including case-control and cohort studies, can be challenging to implement and conduct. Assessment of the quality of such studies is essential for a proper understanding of non-randomized studies. The Newcastle-Ottawa Scale (NOS) is an ongoing collaboration between the Universities of Newcastle, Australia, and Ottawa, Canada. It was developed to assess the quality of nonrandomized studies with its design, content, and ease of use directed to the task of incorporating the quality assessments in the interpretation of meta-analytic results.
Other Tools And Resources
Along with the standardized checklists available for critically appraising the quality of the systematic reviews, researchers can also use systematic review management software tools that are specifically tailored to the review team’s needs. One such tool is DistillerSR. The software automates every stage of the process involved in a systematic review, reduces error or duplication, and by incorporating one or more of the aforementioned quality assessment checklists, it can assist in the critical appraisal of the review.
Conclusion
Assessing the quality of evidence contained in a systematic review is just as important as synthesizing the findings of the review. During quality assessment, one considers the relevance of the methods utilized in addressing review questions, the relevance and quality of methods used within individual studies, and the extent of evidence from reported findings [2]. Due to the importance of systematic reviews in evidence-based practice, it is crucial to conduct a stringent and thorough quality assessment of the review. There are a number of tools available to critically appraise a systematic review. By gaining an understanding of the tools and their implementation, researchers can publish robust, high-quality systematic reviews.
References
- Guyatt, G. H.; Sackett, D. L.; Sinclair, J. C.; Hayward, R.; Cook, D. J.; Cook, R. J. “Users’ guides to the medical literature IX. A method for grading health care recommendations.” JAMA, 274 (22) (1995): 1800-1804.
- Pussegoda, K., Turner, L., Garritty, C., Mayhew, A., Skidmore, B., Stevens, A., Boutron, I., Sarkis-Onofre, R., Bjerre, L.M., Hróbjartsson, A., Altman, D.G. and Moher, D. (2017). Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study. Systematic Reviews, 6(1).
3 Reasons to Connect
Follow Us
