Publication Bias & Small-Study Effects

Publication bias is the tendency for “positive” / statistically significant studies to be published while null ones stay in the file drawer. Small-study effects describe the related observation that smaller trials report larger effects than large ones. Both inflate a meta-analysis’s pooled estimate. They are suggested by funnel-plot asymmetry and tests such as Egger’s, and mitigated by prospective trial registration and grey-literature searching.

Every study in a meta-analysis plotted by its effect estimate (x) against its precision (y). Large, precise studies cluster near the pooled line at the top; small studies scatter wider toward the bottom, forming an inverted funnel. Here the bottom-left corner is empty — the visual signature of missing small null trials.

A roughly symmetric funnel is reassuring. An asymmetric one — small studies bunched to the “positive” side with a hole opposite — suggests small null trials went unpublished, dragging the pooled effect away from the truth. Egger’s test puts a p-value on that asymmetry. But asymmetry is a clue, not a verdict.

A meta-analysis is only as honest as the literature it pools. If positive trials are over-represented, the pooled effect — and any guideline built on it — over-states benefit. This is the case for trial registries (ClinicalTrials.gov, ISRCTN) and for searching unpublished and grey literature before trusting a synthesis.

• Symmetric funnel → reassuring
• Asymmetry / gap → possible publication bias or small-study effects
• Asymmetry is a clue, not proof — heterogeneity can mimic it

Turner et al. (NEJM 2008) — comparing 74 FDA-registered antidepressant trials with what reached the journals: 31% were never published, and almost all of these were non-positive. Selective publication inflated the apparent effect size by 32% overall (range 11–69% across drugs) — a documented, quantified case of publication bias distorting the evidence base. The unpublished trials were not flawed — they were simply negative. This is why funnel plots and trial registration matter: the missing data are systematically the “disappointing” ones.

• Reading funnel asymmetry as proof of publication bias — heterogeneity, poor methods in small trials, or chance can all cause it.
• Forgetting small-study effects are a distinct mechanism — small trials can genuinely over-estimate (e.g. lower methodological quality), not just be selectively published.
• Overlooking outcome-reporting bias — bias also operates within published trials when only the favourable endpoints are reported.