Binesh Sadanandan PhD Dissertation Companion
Read thesis

Thrust 4 · Chapter 6

Safety

Do consistency improvements produce safer, more image-dependent behavior?

A four-quadrant audit crossing consistency with image reliance shows that, averaged across ten model-dataset settings, 81% of each model's consistent predictions are image-invariant (range 45-100%). Consistency-optimized adapters concentrate predictions in the Dangerous quadrant (consistent but text-reliant), reaching 84.1% for Targeted LoRA on PadChest. Correctness is a separate axis: on PadChest the text-reliant cell is often more accurate than the grounded one, so the screen flags behavior, not per-prediction error.

How I ran it

Classify each prediction on curated flip banks by paraphrase consistency and text-only agreement into four quadrants, with attention-grounding (true vs shifted box), occlusion faithfulness, and demographic fairness stratification as companion analyses.

Datasets
MIMIC-CXR, PadChest
Models
MedGemma-4B, Targeted LoRA, Full LoRA, LLaVA-Rad, LLaVA-Rad LoRA
Thesis tables
tab:quadrant_counts, tab:grounding, tab:fairness

The limitation that matters most

The quadrant audit is a behavioral screen, not a per-prediction safety verdict: on PadChest the Dangerous cell can be more accurate than the grounded cell (Base 100% vs 33%), and the widely-quoted r=-0.86 consistency-grounding correlation is largely definitional, falling to r=-0.15 once conditioned on consistency.

Confirmed Reducing flips can create a false sense of safety: averaged across ten model-dataset settings, 81% of each model's consistent predictions are image-invariant (range 45-100%), so consistency and grounding routinely come apart.

Negative result No: attention does not reliably indicate grounding. True-box coverage only marginally exceeds a displaced box (0.296 vs 0.261 for Base), and patch-rank correlation between attention and causal occlusion importance is near zero across all three MedGemma variants.

Mixed Partially: the worst-calibrated model also shows the largest demographic disparities. Full LoRA has the worst calibration (ECE of about 0.25 or higher for every group) and the largest sex gaps, while Targeted LoRA has the smallest sex ECE gap (0.012) but the steepest age accuracy gradient (13 pp).

Primary results

81% (range 45-100%)

model-dataset

81% of consistent predictions are image-invariant

Averaged across ten model-dataset settings, 81% of each model's consistent predictions are unchanged when the image is removed (range 45-100%), so a low paraphrase-flip rate is not evidence of grounded visual reasoning.

Multiple models · Multiple datasets · n = 10 · unweighted mean over 10 settings; range 45-100%

Source, denominator, and limits
How far this goesStated deliberately as a level, not a correlation: the r = -0.86 between flip rate and the image-invariant fraction is largely definitional (the fraction is by construction a subset of consistent predictions, and (1-flip) alone correlates r = +0.86) and falls to r = -0.15 once conditioned on consistency. The level, not the slope, is the finding.
Metric
mean share of consistent predictions that are image-invariant (text-only agreement)
Denominator
model-dataset (per model-dataset setting)
Sample
n = 10
Model
Multiple models
Dataset
Multiple datasets
Split
eval
Comparison
range 45-100% across the ten model-dataset settings
Uncertainty
unweighted mean over 10 settings; range 45-100%
Thesis
Chapter 6, tab:quadrant_counts
Source artifact
dissertation/tables/thrust4/table_quadrant_counts.tex
Last verified
2026-07-15
Open in Evidence Explorer

84.1%

question-level

Targeted LoRA on PadChest: 84.1% of questions in the Dangerous quadrant

After targeted LoRA adaptation, 84.1% of PadChest questions land in the Dangerous quadrant (consistent under paraphrase but not image-reliant): the adapter buys consistency largely by leaning on the text prior.

Targeted LoRA · PadChest · n = 861

Source, denominator, and limits
How far this goesThese are the ORIGINAL-PIPELINE adapters on the curated 861-question PadChest flip bank, not the clean patient-disjoint mitigation fleet. Correctness is a separate axis: on PadChest the Dangerous cell is often MORE accurate than the grounded (Ideal) cell because high finding base rates make the text prior usually right, so this behavioral screen is not a per-prediction safety verdict.
Metric
share of questions classified Dangerous (consistent + text-reliant)
Denominator
question-level (per question)
Sample
n = 861
Model
Targeted LoRA
Dataset
PadChest
Split
ood
Comparison
Base MedGemma-4B 22.5%, Full LoRA 60.6% on the same 861 PadChest questions
Uncertainty
not reported for this value
Thesis
Chapter 6, tab:quadrant_counts
Source artifact
dissertation/tables/thrust4/table_quadrant_counts.tex
Last verified
2026-07-15
Open in Evidence Explorer

r = -0.15 (conditioned)

model-dataset

The flip-rate vs Dangerous-fraction correlation is largely definitional

The headline r = -0.86 correlation between flip rate and the Consistent-Text-driven fraction is largely a definitional consequence of that fraction being a subset of consistent predictions; conditioning on consistency reduces it to r = -0.15.

Multiple models · Multiple datasets · n = 10 · Spearman -0.13 conditioned; 10 data points

Source, denominator, and limits
How far this goesReport the level (81% of consistent predictions image-invariant, range 45-100%), not the slope. The unconditioned r = -0.86 (and the older public r = -0.89, now retracted) should never be presented as an empirical finding: the (1-flip) factor alone correlates r = +0.86 by construction.
Metric
Pearson r between flip rate and image-invariant share of consistent predictions, conditioned on consistency
Denominator
model-dataset (per model-dataset setting)
Sample
n = 10
Model
Multiple models
Dataset
Multiple datasets
Split
eval
Comparison
-0.86 unconditioned
Uncertainty
Spearman -0.13 conditioned; 10 data points
Thesis
Chapter 6, tab:quadrant_counts
Source artifact
dissertation/chapters/05_thrust4.tex
Last verified
2026-07-15
Open in Evidence Explorer

0.296 true box vs 0.261 shifted

case-level

Attention coverage barely beats a shifted box (0.296 vs 0.261)

Base MedGemma-4B places 29.6% of attention mass inside the radiologist-annotated pathology box, only marginally more than the 26.1% it places in a same-sized shifted box: attention is a coarse localizer, not a faithful one.

MedGemma-4B · PadChest · n = 637

Source, denominator, and limits
How far this goesThe grounding-637 subset is all-positive PadChest cases with radiologist boxes and must not be pooled with other evaluation sets. Coverage far exceeds the random-far baseline, so attention is not random; the failure is precision, not localization altogether. The pattern holds for Targeted (0.353 vs 0.310) and Full LoRA (0.385 vs 0.338).
Metric
true-box attention coverage vs shifted-box baseline
Denominator
case-level (per case)
Sample
n = 637
Model
MedGemma-4B
Dataset
PadChest
Split
ood
Comparison
shifted-box 0.261; random-far 0.056 (roughly 5-6x above random, marginally above shifted)
Uncertainty
not reported for this value
Thesis
Chapter 6, tab:grounding
Paper
Attention Without Grounding: Causal Evaluation of Visual Explanations in Medical VLMs
Source artifact
dissertation/tables/thrust4/table_grounding.tex
Last verified
2026-07-15
Open in Evidence Explorer

13 pp (83.3% under-40 to 96.3% over-80)

question-level

Targeted LoRA accuracy climbs 13 pp from youngest to oldest patients

Targeted LoRA accuracy on PadChest rises from 83.3% for patients under 40 to 96.3% for patients over 80, a 13 percentage-point age gradient with younger patients least accurate - the steepest of the three MedGemma variants.

Targeted LoRA · PadChest · n = 861

Source, denominator, and limits
How far this goesThe youngest bin is small (n = 48), so its 83.3% estimate is noisy. The gradient may also reflect age-correlated case mix (finding prevalence differs by age) rather than a pure model disparity. PadChest only; no demographic data is available for the MIMIC set.
Metric
accuracy range across age bins (<40, 40-60, 60-80, 80+)
Denominator
question-level (per question)
Sample
n = 861
Model
Targeted LoRA
Dataset
PadChest
Split
ood
Comparison
Base is nearly flat across age (76.8-81.1%); Full LoRA spans 63.1-71.3% at much lower overall accuracy
Uncertainty
not reported for this value
Thesis
Chapter 6, tab:fairness
Paper
Predictive Entropy as a Joint Screen for Error and Paraphrase Instability in Medical Vision-Language Models
Source artifact
results/uai/fairness_analysis.json
Last verified
2026-07-15
Open in Evidence Explorer

Charts from this thrust

Open any of these in the evidence explorer to filter by model, dataset, or metric.

Four-quadrant safety screen: consistency against image reliance

Ten model-dataset settings on the curated behavioural flip banks: MedGemma variants on MIMIC-CXR n=98 and PadChest n=861; LLaVA-Rad variants on MIMIC-CXR n=88 and PadChest n=732 (its text-only-baseline subset).

The Dangerous cell (consistent but text-reliant) is the largest in eight of the ten settings, and it grows when consistency is optimised: the base model on MIMIC-CXR is 25.5% Dangerous while Targeted LoRA is 59.2% and Full LoRA 78.6%. Averaged across the ten settings, 81% of each model's consistent predictions are image-invariant (range 45-100%).

Image-swap sensitivity: how often a contradicting image changes the answer

One population only: the 861-question PadChest flip bank, four models (base MedGemma-4B, Targeted LoRA, Full LoRA, LLaVA-Rad). MIMIC-CXR cells are deliberately excluded so that no cross-population comparison is possible.

Replacing the radiograph with one showing the opposite finding changes the answer for at most 39.5% of questions, and for LLaVA-Rad only 14.5%. Even the most image-sensitive model here leaves roughly six answers in ten unchanged when the visual evidence is contradicted.

Accuracy by patient age band

PadChest flip bank only, n=861 questions per model split into four age bands (under 40 n=48, 40-60 n=233, 60-80 n=416, over 80 n=164), for base MedGemma-4B, Targeted LoRA and Full LoRA.

Targeted LoRA is the most accurate arm in every band but has the steepest age gradient: 83.3% for patients under 40 against 96.3% for those over 80, a 13-point spread. The arm with the smallest between-sex calibration gap is therefore also the one with the widest age disparity, so no model is uniformly the most equitable.

Cite


  

  

     Permanent link
    
  




  

    
    
    
    
  

  

    

    
Type to search. Press Escape to close.