Funnel Plot in a Systematic Review

Systematic reviews are considered the gold standard in evidence-based research. These intensive “studies of studies” involve taking a systematic approach to collect, assess, and synthesize relevant literature on a specific subject, including peer-reviewed journal articles, gray literature, and other sources, to answer a well-defined research question. Systematic reviews, like other reviews, can also be prone to bias. The bias types that are most evaluated in systematic reviews are selection bias, reporting bias, detection bias, and attrition bias. But this can be prevented through several checks before you start the review. For example, ensuring that the eligibility criteria in a systematic review is well defined when designing the systematic review protocol of a systematic review article, will reduce the risk of selection bias during study selection. An effective tool called a funnel plot is designed to examine the existence of a publication bias, among studies included in a systematic review, or the tendency of authors to only publish studies with significant results, other reporting biases, and small study effects.

What Is A Funnel Plot In Systematic Reviews?

A funnel plot is a simple scatter plot of the treatment effects estimated from individual studies against a measure of the study size. The x-axis (horizontal axis) shows the results of the study, expressed as an odds or risk ratio or a percent difference, while the y-axis (vertical axis) displays the sample size or an index of precision. Other measures could also be plotted, such as reciprocals, or variances.

The scale of the y-axis is reversed; studies with higher precision are placed at the top while studies with lower precision are placed at the bottom. At the bottom where the low precision studies are placed, the points that represent the mean value of effect in each study are widely spread. The spread of these points begins to reduce, as you move upwards in the y-axis. This effect creates a plot that resembles a pyramid or an inverted funnel.

What Is The Purpose Of Funnel Plots?

A funnel plot is designed to check for the existence of publication bias, other reporting biases, and systematic heterogeneity in a systematic review. These are biases caused by the absence of information from unpublished sources (missing studies), or selective outcome reporting of a study’s result (missing outcomes). For example, the study authors may omit information that they may feel does not agree with their findings.
In the absence of publication bias, the funnel plot, as its name suggests, should create a symmetrical funnel-shaped distribution. Deviations from this, like an asymmetrical plot, may indicate that there is a bias.

That said, it’s important to note that publication bias is only one of the issues examined by funnel plots. These plots can also assess small study effects, or the tendency for smaller studies to show larger treatment effects.

How to Interpret a Funnel Plot?

Funnel plots, typically, are symmetrical or asymmetrical. Here’s how to interpret them:

Symmetrical Funnel Plot

A “well-behaved” data set, one where the precision of the estimated intervention effect increases as the size of the study increases will yield a symmetric inverted funnel shape. This signifies the unlikeliness of bias.

Asymmetrical Funnel Plot

An asymmetrical funnel plot can indicate the presence of a bias, suggesting a relationship between treatment effect estimate and study precision. With these deviant shapes, you can assume the possibility of publication bias, small-study effects, or study heterogeneity. Asymmetry can also be caused by the use of an inappropriate effect measure. Whatever the cause, an asymmetric funnel plot leads to doubts about the systematic review. In this case, an investigation must be done to get to the bottom of the possible cause and correct the mistake.

Conclusion

Systematic reviews are one of the most rigorous research processes. But they’re not without risks as they can be prone to biases. Researchers can use tools to prevent these challenges—a funnel plot is one way to do it as it’s designed to check for publication bias, other reporting biases, and small study effects. Another great way to ensure that your systematic review is accurate, objective, and comprehensive is to use a literature review software like DistillerSR, which helps you automate each stage of your review to securely produce evidence-based research faster, and more precisely. This gives you the time and energy to focus on evaluating other information to ensure that your protocol is free from any biases.