Beyond snapshots: the imperative for temporal and contextual integration in multimodal LLMs for retinal diagnostics

Introduction

The recent evaluation of multimodal large language models (LLMs) for retinal disease diagnosis via optical coherence tomography (OCT) by Agbareia et al. ¹ marks a notable step toward realizing Generalist Medical Artificial Intelligence in ophthalmology. By demonstrating that few-shot prompting strategies can meaningfully improve diagnostic accuracy over single-shot approaches across models such as GPT-4o and Claude Sonnet 3.5, the study provides a practical framework for leveraging frontier AI in clinical imaging workflows. This work compellingly illustrates how contextual prompting can bridge the gap between complex retinal imaging data and structured clinical reasoning outputs.

However, as the field progresses from proof-of-concept benchmarking toward clinical deployment, several architectural and methodological considerations merit further discussion. We highlight two critical imperatives temporal integration and diagnostic explainability that we believe are essential for translating these models from experimental tools into trusted clinical decision-support systems.

The temporal imperative: Moving from static snapshots to longitudinal intelligence

A fundamental limitation of the current LLM-based diagnostic paradigm, as tested by Agbareia et al., is its reliance on single cross-sectional OCT images. In routine clinical practice, the management of chronic retinal conditions such as neovascular age-related macular degeneration (nAMD) or diabetic macular edema (DME) rarely hinges on a solitary scan. Treatment decisions, particularly regarding the initiation, continuation, or cessation of anti-vascular endothelial growth factor (anti-VEGF) therapy are fundamentally driven by the temporal trajectory of disease biomarkers: the rate of subretinal fluid accumulation, the progressive thinning of outer retinal layers, or the evolution of hyperreflective foci across serial visits.

Recent advances in longitudinal deep learning demonstrate the feasibility of this temporal approach. Self-supervised methods have shown promise in modeling disease trajectories from sequential OCT volumes, capturing subtle morphological changes that precede clinical conversion events.^2,3 For multimodal LLMs to transition from acting as sophisticated image describers to functioning as genuine clinical advisers, they must evolve to ingest and reason over four-dimensional (4D) datasets incorporating the temporal axis alongside the three spatial dimensions to predict disease trajectory rather than merely identifying pathology present at a single time point. This capability gap represents perhaps the most significant barrier between current LLM performance and the kind of reasoning that clinicians perform daily in retinal practice.

Explainability and robustness: Safeguarding against hallucinated clinical reasoning

The few-shot prompting approach, while demonstrably effective at improving classification accuracy, introduces a nontrivial risk of what may be termed hallucinated clinical reasoning. LLMs may construct linguistically fluent and superficially plausible diagnostic justifications that are anchored not on genuine pathological features but on incidental imaging artifacts, noise patterns, or nonpathological structural variations present in the reference examples. This opacity is particularly concerning in ophthalmic diagnostics, where the clinical consequence of a false positive (unnecessary intravitreal injection) or false negative (missed choroidal neovascularization) carries substantial patient morbidity.

To mitigate this, multimodal diagnostic models must incorporate pixel-aligned Explainable AI (XAI) layers that compel the system to visually substantiate its textual output.^4,5 A model recommending a diagnosis of DME, for instance, should be required to highlight the specific intraretinal cystic spaces or areas of retinal thickening on the OCT B-scan that informed its conclusion. Such saliency-driven architectures would not only enhance clinical trust but also create an auditable diagnostic trail a feature increasingly demanded by regulatory frameworks governing medical AI devices.

Key implications for future research

Building upon the foundational work of Agbareia et al., we identify several priority areas for advancing multimodal LLMs toward clinical readiness in retinal diagnostics:

First, longitudinal modeling capability: Future architectures should prioritize the development of models capable of ingesting and comparing serial OCT scans against historical baseline data, enabling disease progression analysis rather than isolated classification.

Second, cross-platform hardware neutrality: Validation across diverse OCT platforms (Zeiss Cirrus, Heidelberg Spectralis, Topcon Maestro) is essential to prevent vendor-specific diagnostic bias and ensure generalizability across clinical settings. ⁶

Third, contextual clinical guardrails: AI models must incorporate systemic clinical variables—patient age, glycated hemoglobin (HbA1c) levels, treatment history—to ground imaging interpretations within the broader biological context and reduce isolated image-based diagnostic errors. ⁷

Fourth, standardized evaluation benchmarks: The community urgently needs consensus on evaluation frameworks that go beyond classification accuracy to include metrics for temporal reasoning fidelity, explanation quality, and calibration under distribution shift. ⁸

Conclusion

Agbareia and colleagues have established a valuable benchmark for the intersection of multimodal LLMs and ophthalmic imaging. Their demonstration that prompting strategy materially influences diagnostic performance is an insight with immediate practical relevance. However, to bridge the distance from experimental benchmarking to bedside adoption, the academic and clinical AI communities must now direct sustained effort toward building robust medical AI systems that are temporally aware, inherently explainable, and rigorously validated across diverse clinical environments. Only through this disciplined approach can we transform these promising models from research curiosities into the trusted clinical partners that modern retinal care demands.