Thrust 5 asks whether anything can flag the predictions a clinician should look at. These two demos run that question on the real data: 861 PadChest questions, the same rows the dissertation reports. Move a threshold or switch a rule off and the numbers recompute in your browser.
What these demos are
They replay saved offline evaluation on a research dataset. They are not a clinical tool, they do not run a model, and admitting a prediction here means only that a retrospective rule would not have escalated it. The gate demo reproduces the dissertation's published admission and accuracy figures exactly; the build fails if it ever stops doing so.
1. Predictive entropy as a filter
The model produces a probability for yes and no on every question. Predictive entropy measures how close that distribution is to a coin flip. The claim in Chapter 7 is that this single number, from one forward pass, ranks both errors and paraphrase flips. Drag the threshold: everything at or below it is answered automatically, everything above is escalated to a person.
Stable and flipped questions by predictive entropy. The line is your threshold.
Answer stayed the same under paraphrase
Answer flipped under paraphrase
The same numbers as a table
Measure
At this threshold
If you answer everything
What this shows
One forward pass ranks flip risk: entropy separates flipped from stable questions well enough to score an AUROC of 0.823 for flips and 0.862 for errors on this model. Tightening the threshold buys accuracy on what remains.
What this does not show
The two distributions overlap, so entropy ranks flip risk rather than separating it: no threshold admits only stable questions. Softmax entropy, temperature-scaled entropy and absolute margin are rank-equivalent, so this is one signal reported three ways, not three independent checks. The threshold is specific to this model and dataset. Most importantly, a confident answer can still be one the model reached without the image: entropy says nothing about grounding, which is what the next demo is about.
2. The deployment-readiness audit
Entropy cannot tell you whether the image was used. So Chapter 7 builds a rule out of several signals at once, and asks it to admit only predictions that are both stable under rephrasing and dependent on the image. Switch the conditions off and on to see what each one is doing.
This is an audit, not a gate you could deploy
The region condition needs a radiologist's bounding box around the finding, which by definition does not exist for a patient who has not been read yet (it is available for 637 of these 861 questions and for none of MIMIC). The swap condition needs a second image known to show the opposite label. Both are things you can do when evaluating a model offline, and neither is something a clinical workflow could run. The rule below is the one the dissertation reports; the "text-only" condition is shown because it is not in that rule, and turning it on shows why.
The rule
Admit when the paraphrases agreeand (swapping the image changes the answerorthe region and the background disagree)
Six real questions, and what the rule did with them
These are actual rows from the evaluation, not illustrations. Each shows the paraphrases the rule compared, what happened when the image was swapped or masked to the annotated region, and whether the question was answerable from the text alone.
What this shows
The rule works on its own terms: on Targeted LoRA it admits 33.0% of questions at 96.8% accuracy, against 91.5% if you answer everything. Every condition you remove admits more and is right less often.
What this does not show
It does not show that the admitted predictions are grounded. Watch the fourth number: most of what the rule admits is exactly what the question text alone already answers. The dissertation's answer to "can a gate admit image-grounded cases?" is no. On the slice where the model's text-only answer disagrees with its full answer, the same Targeted LoRA scores 2.9% accuracy (n=241), and Qwen2-VL gets 0 of 279 right under every rule tried. High admitted accuracy here comes from high finding base rates making the text prior usually correct, not from the model reading the radiograph.