Narrative-based Explainable AI for Clinical Decision Making

Abstract

The study presents a hybrid framework that integrates survival modeling, explain-able AI, and large language models (LLMs) to generate interpretable narrative explanations for individual patients with multiple myeloma. A Random Survival Forest (RSF) is trained to predict median survival time using clinical features, with survival functions estimated accordingly. SHAP (SHapley Additive exPlanations) values are computed using SurvSHAP(t) at the predicted median survival time to quantify feature contributions. These SHAP values are then used to construct structured prompts that guide locally deployed LLMs in generating patient-specific explanations. To ensure privacy and offline capability, all LLMs are deployed locally using tools such as Ollama and vLLM. Experimental results show that the RSF provides superior predictive performance, and qualitative analysis of the LLM-generated outputs reveals variation across models in terms of factual alignment, reasoning quality, and linguistic fluency. Among all tested models, DeepSeek-R1 with 70B parameters produced the most coherent and clinically plausible explanations. This work demonstrates the potential of combining explainable survival models and LLMs for trustworthy, personalized AI interpretation in clinical settings.