Validity & Generalisability

Validity asks whether a finding is true. Internal validity = free of bias, confounding and chance; construct validity = the study actually measured what it intended. Generalisability (external validity) is a different question: does the finding apply beyond the study — to other patients and settings? A study must be internally valid first, but high internal validity does not guarantee generalisability.

Two gates in series. The first asks whether the result is true within the study; only if it passes do you reach the second, which asks whether it transfers to your patients and setting. A result can sail through the first gate and stall at the second.

Always judge internal validity first — a biased result is not worth generalising. If it holds, switch questions entirely: compare the trial’s population, setting, intervention and outcomes with yours. Closeness on those is what licenses applying the finding; rigour alone does not.

The two are often in tension. A strict, narrowly-selected explanatory RCT maximises internal validity but enrols an atypical population, so it may not reflect your real ED case-mix. Construct validity is the quiet third axis: if the outcome measured isn’t the one that matters, even a “valid” trial answers the wrong question.

• Internal = is it true? · External = does it apply?
• Internally valid first — but it does not imply generalisable
• Don’t forget construct validity: measuring what was intended

Rothwell, Lancet 2005 — in “External validity of randomised controlled trials: to whom do the results of this trial apply?”, Rothwell argued external validity is the most frequently neglected aspect of RCTs: strict eligibility criteria, run-in periods and specialist centres yield internally valid results in patients who may look little like those in routine practice. A tightly-controlled explanatory RCT can be impeccable internally yet a poor guide to your unselected ED case-mix. Internal validity is necessary but not sufficient — “to whom do the results apply?” is a separate question the trial’s rigour cannot answer for you.

• The internal–external validity trade-off — assuming the most rigorous trial is also the most generalisable.
• Overlooking construct validity — a valid measurement of the wrong thing.
• Over-claiming applicability — extrapolating to populations and settings the trial never sampled.