Explainable Remote Sensing Classification using XAI-Guided Large Language Models

Abstract

Deep learning (DL) models have proven very effective in land cover classifications in remote sensing (RS). However, the black-box nature of these models limits their interpretability and leads to lower trust in their results; this is especially true for parameters related to decision-making and the need for transparency in large-scale operations. Although there are many explainable artificial intelligence (XAI) techniques that visualize how an example from a model works, these methods often create output that is either too technical or too difficult for anyone other than a model expert to interpret. To overcome this shortcoming, this thesis develops a unified framework to integrate XAI techniques with large language models (LLMs) in order to create automatically generated natural-language explanations of DL model classifications for satellite images. The framework consists of two primary components: the first is the development of a methodology for translating visual outputs from XAI techniques into a structured text representation of these outputs; the second is to input this text into an LLM so that the LLM can generate a concise natural-language explanation that will link the visual features identified by each XAI and how those features affect the result of a classification. In addition, a combined XAI experiment is conducted, where the outputs of all three methods are merged into a single prompt to capture complementary spatial and feature-level information. To evaluate explanation quality, an LLM-as-a-judge framework was adopted with multi-criteria scoring based on specificity and usefulness, combined with pairwise and per-class comparison analysis. Experimental results on the EuroSAT dataset demonstrate that the proposed framework significantly improves explanation quality, achieving a win rate of 75.83\% and a +18.50\% improvement over the baseline when the three XAI methods are combined. These findings highlight the importance of integrating complementary XAI signals to enhance the clarity, reliability, and usability of LLM-generated explanations for non-expert users in RS applications.