Binesh Sadanandan Dissertation
Read dissertation

Demos

Demonstrations that support this dissertation

Begin with the end point as a demonstration: the gated read this research evaluates, applied retrospectively to saved cases. Then rewind to how the medical VLM emerged, see the paraphrase failure itself, read one case layer by layer, check where the answer commits across 1,396 pairs, and try the signal the gate reads. The evidence-driven exhibits recompute from saved evaluation rows in your browser.

These demos replay saved offline evaluation on a research dataset. They are not a clinical tool and nothing here is medical advice.

1. The gate this research suggests

Before the failure, the end point. This exhibit applies a retrospective admission rule to ten selected PadChest cases from the 861-question bank: a prediction is admitted only when every phrasing agrees, the answer margin clears the bank median, and predictive entropy stays low; anything else would go to a radiologist. Six cases are admitted, four are held back. The cases illustrate the rule's behavior and do not establish clinical readiness. The radiographs shown are openly licensed illustrative images matched to each finding, not the PadChest case films: the research-use agreement prohibits publishing those without written permission. The clinician view keeps the arithmetic off the screen; the case files keep all of it.

Open the clinician view full screen Open the case files

2. How the medical VLM emerged

A five-era visual primer connects task prompting, contrastive vision-language learning, medical question answering, instruction-tuned multimodality, and the medical VLM studied in this dissertation. Use the timeline, arrow keys, or space bar to move through it.

Method animations

Open animation full screen

Open the journey full screen MP4 GIF

3. What a paraphrase flip looks like

Eight curated cases show the failure the later signals are meant to catch: questions that mean the same thing get contradictory answers. Within a case the image never changes; only the wording does.

GPT-5-mini and Claude Haiku 4.5 and claude-opus-5 and muse-glimmer-30b and gpt-5.6 appear as general-purpose reference models for contrast; the dissertation's benchmark covers six medical vision-language models. Each verdict chip compares a model's answers with the reference read: whether the original answer was right, and whether any rewording changed it. In practice, a flipped presence answer is the difference between a finding flagged for review and one missed.

Case 01 · Curated demonstration set (MIMIC-CXR)

Pneumothorax

3 flipped answers 3 of 7 models flip
Illustrative pictogram, standing in for the study image

The radiograph is not shown

This case uses a MIMIC-CXR study. The PhysioNet licence for MIMIC-CXR prohibits redistributing images, so the radiograph is not published here. Anyone with credentialed access can obtain it from PhysioNet.

The question and the answers are below. The image never changes within a case; only the wording does.

Original question

Is there evidence of pneumothorax in this image?

Reference answer

No

Why this case mattersThe image shows no pneumothorax, yet a single more specific rewording about pleural air pushes all three original models into the same false positive. The frontier arms added in August 2026 do not flip here: GPT-5.6 Sol, Muse-Glimmer-30B, and reasoning-off Claude Opus 5 stay correct on every wording, while reasoning-on Claude Opus 5 is stable but wrong throughout. Kimi K3 is excluded from this case because its responses overran the token budget mid-reasoning and were unreadable.
Why these questions count as equivalentEach paraphrase keeps the same clinical operator (presence of a finding) and the same finding (pneumothorax), varying only syntax, scope wording, specificity, or synonyms, which is the class of rewrite the equivalence audit rubric treats as clinically equivalent.

MedGemma-4B

Correct until rephrased

Answer to the original question: No

  1. Does this image demonstrate a pneumothorax?

    Syntactic restructuring No Same as the original Matches the reference answer

  2. Is there evidence of pneumothorax, unilateral or bilateral, on this image?

    Scope quantification No Same as the original Matches the reference answer

  3. Is there radiographic evidence of pleural air consistent with pneumothorax in this chest radiograph?

    Specificity modulation Yes Flipped from the original Conflicts with the reference answer

  4. Is there air in the pleural space suggesting pneumothorax on this image?

    Lexical substitution No Same as the original Matches the reference answer

GPT-5-mini

Correct until rephrased

Answer to the original question: No

  1. Does this image demonstrate a pneumothorax?

    Syntactic restructuring No Same as the original Matches the reference answer

  2. Is there evidence of pneumothorax, unilateral or bilateral, on this image?

    Scope quantification No Same as the original Matches the reference answer

  3. Is there radiographic evidence of pleural air consistent with pneumothorax in this chest radiograph?

    Specificity modulation Yes Flipped from the original Conflicts with the reference answer

  4. Is there air in the pleural space suggesting pneumothorax on this image?

    Lexical substitution No Same as the original Matches the reference answer

Claude Haiku 4.5

Correct until rephrased

Answer to the original question: No

  1. Does this image demonstrate a pneumothorax?

    Syntactic restructuring No Same as the original Matches the reference answer

  2. Is there evidence of pneumothorax, unilateral or bilateral, on this image?

    Scope quantification No Same as the original Matches the reference answer

  3. Is there radiographic evidence of pleural air consistent with pneumothorax in this chest radiograph?

    Specificity modulation Yes Flipped from the original Conflicts with the reference answer

  4. Is there air in the pleural space suggesting pneumothorax on this image?

    Lexical substitution No Same as the original Matches the reference answer

Claude Opus 5 (reasoning on)

Consistent but wrong

Answer to the original question: Yes

  1. Does this image demonstrate a pneumothorax?

    Syntactic restructuring Yes Same as the original Conflicts with the reference answer

  2. Is there evidence of pneumothorax, unilateral or bilateral, on this image?

    Scope quantification Yes Same as the original Conflicts with the reference answer

  3. Is there radiographic evidence of pleural air consistent with pneumothorax in this chest radiograph?

    Specificity modulation Yes Same as the original Conflicts with the reference answer

  4. Is there air in the pleural space suggesting pneumothorax on this image?

    Lexical substitution Yes Same as the original Conflicts with the reference answer

Claude Opus 5 (reasoning off)

Consistent and correct

Answer to the original question: No

  1. Does this image demonstrate a pneumothorax?

    Syntactic restructuring No Same as the original Matches the reference answer

  2. Is there evidence of pneumothorax, unilateral or bilateral, on this image?

    Scope quantification No Same as the original Matches the reference answer

  3. Is there radiographic evidence of pleural air consistent with pneumothorax in this chest radiograph?

    Specificity modulation No Same as the original Matches the reference answer

  4. Is there air in the pleural space suggesting pneumothorax on this image?

    Lexical substitution No Same as the original Matches the reference answer

GPT-5.6 Sol

Consistent and correct

Answer to the original question: No

  1. Does this image demonstrate a pneumothorax?

    Syntactic restructuring No Same as the original Matches the reference answer

  2. Is there evidence of pneumothorax, unilateral or bilateral, on this image?

    Scope quantification No Same as the original Matches the reference answer

  3. Is there radiographic evidence of pleural air consistent with pneumothorax in this chest radiograph?

    Specificity modulation No Same as the original Matches the reference answer

  4. Is there air in the pleural space suggesting pneumothorax on this image?

    Lexical substitution No Same as the original Matches the reference answer

Muse-Glimmer-30B

Consistent and correct

Answer to the original question: No

  1. Does this image demonstrate a pneumothorax?

    Syntactic restructuring No Same as the original Matches the reference answer

  2. Is there evidence of pneumothorax, unilateral or bilateral, on this image?

    Scope quantification No Same as the original Matches the reference answer

  3. Is there radiographic evidence of pleural air consistent with pneumothorax in this chest radiograph?

    Specificity modulation No Same as the original Matches the reference answer

  4. Is there air in the pleural space suggesting pneumothorax on this image?

    Lexical substitution No Same as the original Matches the reference answer

The claim this illustrates · Benchmark flip rates · The PSF-Med paper

4. What each layer is thinking

You have just seen answers flip. This opens one pair and follows the answer position through all 33 layers. The colored paths show when two equivalent wordings begin leaning toward different answers; the layer control gives you an exact readout at any point. The amber marker is the causal transplant locus, which is not necessarily the first visible split. Open the expert view only when you want the complete prompt-token matrix.

Case

Source: jlens_paraphrase/jspace_data.json, MedGemma-4B, 33 layers. Chapter 5.

Loading the layer trajectories (about 400 kilobytes).

Read Thrust 3 Where the answer commits

5. Where the answer commits

The section above reads one pair. This one asks whether that split is a property of the model in general or of that pair alone. Take two versions of the same question, one the model answers correctly and one it does not, and copy the model's internal state from the first into the second at a single layer. If the answer changes, that layer is carrying the decision. Drag through the 34 layers of MedGemma-4B and watch where it happens, across 1,396 pairs at once.

Controls

Layer 16
Jump to

Source: jlens_paraphrase/patch_full.json, 1,396 pairs across 91 findings, PadChest, MedGemma-4B. Chapter 5.

—

Of 1,396 pairs flip when the state is swapped here

—

—

Flip when an image token is swapped instead, at the same layer

The control: flips here would mean patching any position changes the answer.

—

What this layer is doing

—

The answer commits in a narrow band

Flip rate by transplant layer. The line is the layer you picked.

Median commit layer 16, 95% bootstrap interval [16, 16]. 71% of pairs commit within layers 15 to 17.

The same curve as a table
LayerAnswer-position transplantImage-token control

The answer is not assembled gradually. Through layer 13 the transplant does almost nothing, under 0.2%. Then it goes 8.2% at layer 14, 35.2% at 15, 73.2% at 16, and 100% by 20. The image-token control never exceeds 0.07%, one pair of 1,396: the flips are specific to the answer position. That is what makes layer 16 the point where this model has decided.

Read Thrust 3 Sample and source The layer-17 features

6. Predictive entropy as a filter

Knowing where the answer commits does not help at inference, because you cannot open the model up on a live case. So: is there a signal you could actually read? The model produces a probability for yes and no on every question, and predictive entropy measures how close that distribution is to a coin flip. The claim in Chapter 7 is that this single number, from one forward pass, ranks both errors and paraphrase flips. Drag the threshold: everything at or below it is answered automatically, everything above is escalated to a person.

Controls

0.350 nats
Jump to

Source: results/uai/{model}/padchest/softmax_entropy.jsonl, 861 questions, Chapter 7.

—

Answered automatically

—

—

Wrong, among those answered

—

—

Would flip under paraphrase, among those answered

—

—

Escalated but actually right

The cost of caution

Where the threshold falls

Stable and flipped questions by predictive entropy. The line is your threshold.

  • Answer stayed the same under paraphrase
  • Answer flipped under paraphrase
The same numbers as a table
MeasureAt this thresholdIf you answer everything

One forward pass ranks flip risk. Entropy separates flipped from stable questions well enough to score an area under the curve of 0.823 for flips and 0.862 for errors on this model, on a scale where 0.5 is a coin flip and 1.0 is perfect. Tightening the threshold buys accuracy on what remains.

Cite


  

  

     Permanent link
    
  




  

    
    
    
    
  

  

    

    
Type to search. Press Escape to close.