Thrust 5 · Chapter 7
Deployment
Can uncertainty and audit signals identify predictions that should be escalated?
One forward pass is enough to rank flip risk. Predictive entropy predicts paraphrase flips at an AUROC of 0.823 and errors at 0.862 for Targeted LoRA on the PadChest flip bank, and it carries across architectures: LLaVA-Rad LoRA reaches 0.830 on PadChest and 0.905 on MIMIC. The gate is an offline readiness audit: its region term needs a radiologist's bounding box that no undiagnosed patient has, so that term cannot run online, though the MIMIC configuration, where the rule already reduces to paraphrase agreement plus an image-swap check, is deployable as written. And what it admits is mostly what the question text already answers: on the slice where text and image disagree, Targeted LoRA scores 2.9%.
How this thrust was run
Compare uncertainty methods (softmax entropy, Monte Carlo dropout, deep ensemble, temperature scaling) for flip and error prediction, calibration, corruption stress, and descriptive conformal coverage, then evaluate a multi-signal admit rule combining paraphrase agreement, swap sensitivity, and region evidence.
- Datasets
- MIMIC-CXR, PadChest
- Models
- MedGemma-4B, Targeted LoRA, Full LoRA, LLaVA-Rad, LLaVA-Rad LoRA, Qwen2-VL
- Thesis tables
- tab:bridge_auroc, tab:gate_results, tab:conformal_shift, tab:clean_calibration
Confirmed Yes: single-pass predictive entropy predicts paraphrase flips (AUROC 0.823) as well as errors (AUROC 0.862) on PadChest, and the flip bridge replicates across architectures (LLaVA-Rad LoRA 0.830 PadChest, 0.905 MIMIC).
Negative result No. Gates admit the cases the text already answers over the ones grounded in the image. Admitted-case accuracy does rise, to 96.8% against 91.5% for Targeted LoRA on PadChest. Yet that same model scores 2.9% on the slice where text and image disagree, and Qwen2-VL gets 0 of 279 such questions right under every rule tried.
Negative result No: no single internal monitor transfers across model families. Gemma, LLaVA-Rad, and Qwen2-VL each require different monitors, and the multi-pass alternatives fail in family-specific ways: adapter-only Monte Carlo dropout is an uninformative epistemic probe and the MedGemma deep ensemble collapses out of distribution.