Appraising a Cohort or Case-Control Study

A structured judgement of whether an observational study’s reported association is trustworthy. Four questions drive it: were the groups recruited and comparable appropriately (case and control selection; exposed vs unexposed)? Were exposure and outcome measured objectively, ideally blind to status? Was follow-up complete and long enough (for cohorts)? And, decisively, was confounding identified and adjusted for? Anchor the read with the Newcastle–Ottawa Scale or the relevant CASP cohort / case-control checklist.

A top-to-bottom appraisal path. The first three boxes test how the study was assembled and run; the final box — confounding — is where most observational studies live or die, because the groups were never randomised and so may differ systematically in ways that drive the outcome. The Newcastle–Ottawa Scale mirrors this, scoring selection, comparability and outcome/exposure (8 items, max 9 stars).

Start at the top and only trust the result if each gate is passed. Selection bias appears in box 1 — in case-control work, controls drawn from the wrong population; in cohorts, an unrepresentative exposed group. Information bias (recall, measurement) sits in box 2. Attrition sits in box 3. Then ask the decisive question in box 4: which confounders were anticipated, and were they adjusted for (matching, stratification, multivariable regression)?

Observational designs are often all you have for harms, prognosis or rare exposures — but they can only show association. An apparently strong link can collapse once a confounder is controlled for (or appear once selection is skewed). Knowing that confounding, not sample size, is the headline threat is exactly the discrimination the exam rewards — and it is why GRADE starts observational evidence at “low” certainty.

• Groups → Measurement → Follow-up → Confounding
• NOS = selection · comparability · outcome (8 items, max 9)
• Tools: Newcastle–Ottawa · CASP cohort / case-control

Appraising a registry cohort in an SAQ — you are handed an observational study reporting that ED patients given an early antibiotic had lower mortality than those who did not. Work the flow: were the two groups comparable (box 1)? Sicker, more recognisable sepsis patients may have been treated faster — confounding by indication (box 4). Was outcome ascertainment blind (box 2), and follow-up complete (box 3)? The mark is not “antibiotics work” but “this is an association, vulnerable to confounding by indication; adjustment helps but cannot remove unmeasured confounders.” ⚠ The single biggest threat to any observational causal claim is confounding — name it explicitly, then selection bias.

• Forgetting residual confounding — adjustment never fully removes unmeasured confounders.
• Missing selection / recall bias — especially inappropriate controls in case-control studies.
• Ignoring attrition (loss to follow-up) in a cohort, which can reintroduce bias.