Reliability metrics · an interactive primer

The four-quadrant screen

A dummy example on 100 chest X-rays. An AI answers a yes/no question for each one, and every case gets two behavioral tests: reword the question, then change the image. Consistency crossed with image reliance sorts the 100 answers into four cells, and only one cell is safe. Run the screen step by step, and toggle the models to see why a low flip rate proves nothing by itself.

The two tests. Test 1, reword the question ("Is there an effusion?" becomes "Can an effusion be identified?"): does the answer hold? That splits the cases into consistent and inconsistent. Test 2, change the image (remove it, gray it out, or swap in a scan showing the opposite finding): does the answer move? That splits each half into image-reliant and text-driven. A model can pass the first test by ignoring the image entirely, which is exactly what the second test exists to catch.
Flip rate (the naive read)
…
flipped / 100
Consistent answers
…
pass test 1
Consistent but text-driven
…
Dangerous share of consistent
Ideal cell
…
consistent and image-reliant

Toggle the models and watch the reversal. Model B posts the better flip rate, 4% against 12%, and the flip-rate card even turns green for it. Then the second test exposes the purchase: 81% of B's consistent answers never looked at the image, so its reliability is a costume. Model A flips more, yet most of its consistency is genuine. A low flip rate is where the audit starts, never where it ends, because consistency can be bought from the text prior, and the Dangerous cell is where that purchase hides.

Is the screen grading itself?

The fair objection. Any behavioral taxonomy risks circularity. The screen labels a prediction Dangerous using consistency and image reliance, then reports that Dangerous predictions are text-driven. Stated that way, it could be a relabeling rather than a finding. The answer is convergent validation: check the labels against signals the labels never touch.

The cleanest check is information-theoretic. For every prediction, measure the Kullback-Leibler divergence between the output distribution with the image and without it: how much does removing the image move the whole distribution, not just the final answer? The quadrant labels are discrete and never see this number. If the labels are real, Dangerous cases should sit near zero and Ideal cases should sit high. Drag the threshold and watch how well one continuous signal recovers the behavioral labels on the 100 dummy cases.

0.12
AUROC, KL vs labels
…
threshold-free, this toy
Agreement at threshold
…
Dangerous if KL below line
Signals the labels used
0
of the three below
Check 2 · swap invariance. Show an opposite-finding image from a different patient. A grounded answer should move; a text-driven answer survives.
Check 3 · null-image control. Replace the scan with a uniform gray placeholder. Same logic: surviving the swap and the blank is evidence the image never mattered.

Three independent signals, all agreeing with the behavioral labels, so the screen is a finding rather than a relabeling. In the study that introduced this screen, the divergence check separates Dangerous from Ideal at AUROC 0.76 on real predictions; this toy is cleaner than reality on purpose, so the mechanism is easy to see.