Risk-of-Bias Appraisal Tools
Match the tool to the design — each study type has its own domain-based instrument for judging risk of bias.
Risk-of-bias tools assess, domain by domain, how far a study’s result might be distorted — and the right tool depends on the study design. The core map: RoB 2 for randomised trials, ROBINS-I for non-randomised/observational studies of interventions, QUADAS-2 for diagnostic-accuracy studies, PROBAST for prediction models, AMSTAR-2 / ROBIS for systematic reviews, and the Newcastle–Ottawa scale for cohort/case-control studies.
One question first — what design is this? — then the tool follows.
A look-up from study design to the appropriate appraisal instrument. Each tool was built for a specific design and assesses bias by domain (e.g. QUADAS-2’s patient selection, index test, reference standard and flow/timing). Using the matching tool means the domains it asks about are the ones that actually threaten that design.
Identify the design first, then read across to its tool — never start from a favourite tool and force the study into it. If it is randomised, RoB 2; if it is an observational intervention study, ROBINS-I; a diagnostic study, QUADAS-2; and so on. A systematic review needs a review-level tool (AMSTAR-2/ROBIS), not a trial tool.
A mismatched tool asks the wrong questions: applying RoB 2 to an observational study ignores confounding (the dominant bias there), while ROBINS-I has a whole domain for it. Choosing correctly is exactly the discrimination the exam tests — and it underpins how GRADE downgrades a body of evidence for risk of bias.
RoB 2= RCTs ·ROBINS-I= non-randomised interventionsQUADAS-2= diagnostic ·PROBAST= prediction modelsAMSTAR-2 / ROBIS= reviews ·Newcastle–Ottawa= cohort/case-control
Choosing the tool in an appraisal station — you are handed a registry study comparing pre-hospital tranexamic acid against no TXA, with patients not randomised. The instinct to reach for RoB 2 is wrong: this is a non-randomised study of an intervention, so the correct instrument is ROBINS-I, whose first and dominant domain is confounding — precisely the threat that allocation-by-circumstance introduces. Swap the scenario to a D-dimer accuracy study and the answer becomes QUADAS-2; a derived sepsis prediction score, PROBAST. Matching the tool is not optional polish — the wrong tool simply will not ask about the bias that matters most for that design.
- Choosing the wrong tool for the design — classically RoB 2 on a non-randomised study (use ROBINS-I).
- Confusing reporting-quality checklists (CONSORT, PRISMA) with risk-of-bias tools.
- Relying on a single summary quality score (e.g. Jadad) instead of a domain-based assessment.
Quick check
Which tool appraises a non-randomised study of an intervention?
Answer: ROBINS-I — the Risk Of Bias In Non-randomised Studies of Interventions tool, whose lead domain is confounding (the dominant bias when participants were not randomised). RoB 2 is for randomised trials only.
Ready to build your plan? EMF Premium gives you all 40,000+ questions, 20 mocks and 1,215 OSCE stations from £29/month — or a one-off 3- or 6-month pass.