Notebook AtlasSURG0002 · Research methods: go to the course map

AtlasSession 4L12: Introduction to measurement errors and diagnostic test accuracy studiesp.23, 33–35, 61–62

Agreement: kappa

  1. 2 earlier topics
  2. Accuracy vs precision
  3. Random vs systematic error
  4. Assessing measurement error
  5. Diagnostic test accuracy (2×2)
  6. Agreement: kappa

Fig. 1

Two blinded histologists call 200 explanted subcutaneous scaffolds neovascularised or not (scaffolds)

Fig. 1. Two observers rate 200 scaffolds; matching calls are lit. They match on 160 of 200 (80%). Taking away the luck, kappa is 0.52: moderate.

Matching calls
160 of 200, 80%
Beyond luck (kappa)
0.52, moderate

2×2 Workbench · First look

Layout layout
Agreement by luck luck

Level 1 · First look · Lecture 12 · p.23, 33–35, 61–62

Count how often two observers agree and see that some agreement happens by luck.

Compute percentage agreement between two observers, place a kappa value on the Landis–Koch scale, and explain why high agreement can still be only moderate kappa.

  1. Two referees (the step you are on)

    They agree on most calls, partly because most moments have no foul at all.

    Two referees agree on most calls, partly because most moments have no foul at all: agreeing that nothing happened is easy.

step 1/4