Rethinking the Evaluation of Generative AI in Clinical Diagnosis and Prognosis

Source abstract

Generative AI increasingly supports clinical diagnosis and prognosis, but safe deployment into clinical practice requires standardized evaluation. Evaluation practices from 2022 to 2025 are surveyed, and a five-dimensional framework is introduced: (I) Factual and Contextual Correctness, (II) Safety, Alignment, and Robustness, (III) Reasoning Quality, (IV) Calibration and Uncertainty, and (V) Clinical Anthropomorphism. Current automated and human evaluation methods rely heavily on accuracy, lack multimodal assessment, and are not aligned with human judgment. Emerging benchmarks are assessed and linked to our framework. A hybrid evaluation approach combining automated scalability with expert clinical judgment is recommended for safe and effective deployment.