Type I & Type II Errors

A Type I error (α) is rejecting a true null hypothesis — a false positive, conventionally accepted at α = 0.05. A Type II error (β) is failing to reject a false null — a false negative. Power = 1 − β: the probability of detecting a true effect of a given size, conventionally set at ≥ 80%.

Two overlapping distributions — the null hypothesis (H0) and the alternative (H1) — split by the critical value, with the α tail and β region shaded, plus a truth × decision 2×2. Each shaded area is one of the two ways a study can reach the wrong conclusion.

The red tail (α) is the false-positive rate; the amber region (β) the false-negative rate. Lowering α without increasing the sample size pushes the cut-off to the right and increases β — the two errors trade off against each other unless you add more patients.

Many “negative” EM trials are simply underpowered (Type II) — the effect was real but the study was too small to catch it. Conversely, testing many outcomes inflates Type I. Recognising both stops you over-believing a lone significant p-value and under-believing a non-significant one.

• Type I (α) = false positive (usually 0.05)
• Type II (β) = false negative
• Power = 1 − β (aim ≥ 80%)

ATACH-2 (Qureshi et al., NEJM 2016) — 1,000 patients with acute intracerebral haemorrhage, intensive vs standard BP lowering. Death or severe disability (mRS 4–6) at 3 months was 38.7% vs 37.7% → adjusted RR 1.04 (95% CI 0.85–1.27). The power calculation assumed a 60% control event rate, but only ~38% of patients reached the outcome — far fewer events than planned, so the trial was underpowered. A “no significant difference” with a CI spanning 0.85–1.27 is inconclusive, not proof of no effect — a worthwhile benefit or harm can’t be excluded.

• Non-significance ≠ no effect — a wide CI usually means the trial was underpowered (Type II).
• Multiple comparisons inflate Type I — test enough outcomes and one will “reach” p<0.05 by chance.
• Confusing α (the pre-set acceptable false-positive rate) with the p-value of one specific result.