An Ontology-Guided Verification Pipeline for Accurate Medical Specialist Classification Using LLMs: Benchmarking Batch and Quantization Effects

Abstract

<p>Patients usually lack the medical knowledge to choose an appropriate specialist to consult, resulting in delayed diagnoses, wasted clinical resources, and unnecessary consultations. While scalable automated specialist selection is possible with large language models (LLMs), instability, hallucination risk, and reliability degradation because of quantization are deployment challenges. Safe usage also requires sensitive data handling that complies with GDPR and HIPAA; locally hosted LLMs help meet the requirement than commercial cloud-based systems. This study proposes an ontology-guided verification pipeline, paired with batch processing and a self-evaluation approach to improve the accuracy of seven LLM families, including both general-purpose and medical-tuned open weight models. The pipeline integrates weighted clinical ontologies, automated ambiguity detection, ensemble voting with contrast validation, confidence calibration, and multi-prompt batch verification to enhance decision robustness. Also, prioritization of lighter-weight quantized local models facilitates real-time operational and efficient resource utilization within e-health ambient environments. Furthermore, these models demonstrate that high performance can be achieved without compromising privacy. The models were evaluated across 4,999 expert-labeled clinical cases. Batch processing improved accuracy by approximately 1–3% for full-precision models and produced substantially larger gains for lighter-weight quantized models, where stabilization effects played a significantly stronger role. Importantly, medical-tune models in the lower-billion parameter range, particularly MedGemma-4B and MediPhi-4B, consistently outperformed larger general-purpose LLMs under both full-precision and quantized inference settings. These findings indicate that domain-specialized models and structured verification can outweigh base model scaling for reliable clinical recommendation systems.<br></p>