The defense story
Ask the same question twice.
One small habit of distrust runs through this dissertation: take a question the model answers, say the same thing in different words, and watch whether the answer holds. The defense tells that story in five thrusts. Measure the failure, diagnose it, explain and fix it, build the safety screen, then decide what deployment can trust. Every number below links into the evidence it comes from.
Thrust 1 · Measure
Flips are common, and the obvious suspects do not explain them.
How often do equivalent questions change model answers?
Start with what an LLM calls a paraphrase: judged against a clinical equivalence rubric, only half of the candidate rewordings preserve the question. After the audit, the PadChest regeneration, and filtering, 92,856 pairs over three countries remain. On that set the six models flip between 6.4% and 54.7% of the time, and the bigger models are not the steadier ones.
Thrust 2 · Diagnose
The steadier the answers, the less the model looks at the scan.
Does consistency mean the model is using the image?
Consistency can mean the model reads the image the same way every time, or that it never reads it at all. On a 107-pair diagnostic set the answer is uncomfortable: the most consistent backend keeps answering when the image is removed, and across the three backends consistency and text-reliance rise together. The set predates the equivalence audit, so the figures are illustrative rather than estimates.
Thrust 3 · Explain & fix
A candidate circuit, a 59% fix, and a cause in the training data.
Can paraphrase flips be reduced without degrading observed accuracy?
The mechanism work points to a narrow band where the answer commits, then partly undoes itself: the layers the diagnosis named are not the best place to intervene, and single-feature ablation restores only a handful of flips. The parametric fix that works is blunt by comparison, a targeted LoRA on layers 15 to 19 that cuts flips by about 59% on a clean patient-disjoint split. A controlled probe then shows why data-side coverage is the larger lever, and a re-audit shows the adapter buys part of its consistency from the question text, so the fix is real but bounded.
Figures use an illustrative, licence-cleared chest X-ray (Wikimedia Commons); the underlying pairs are credentialed data and are not shown.
Open this thrust Filter the evidence The causality experiments
Thrust 4 · Safety-evaluate
Consistency without grounding is the safety paradox.
Do consistency improvements produce safer, more image-dependent behavior?
A model can pass every consistency check and still not be reading the image. Crossing consistency with image reliance in a four-quadrant screen shows most consistent predictions are image-invariant, 81% on average across ten settings, and consistency-optimized adapters concentrate in exactly the quadrant you would not want. The screen flags behavior, not per-prediction error: on PadChest the text-reliant cell is often the more accurate one.
Thrust 5 · Deploy
Abstaining buys safety.
Can uncertainty and audit signals identify predictions that should be escalated?
The deployment question is what can be caught before a prediction leaves the building. Predictive entropy from a single forward pass ranks flip risk and error risk well, and it transfers across architectures. The gate built on these signals is honest about its limits: it is an offline readiness audit whose region term needs a radiologist's box, and what it admits skews toward questions the text already answers.
The story in full
The dissertation carries the full argument; the papers carry the peer-reviewed pieces; the evidence explorer carries every number with its sample and source.