A Fine-Tuned Fuzzy-Optimized UNet++ for Retinal Vessel Segmentation

Abstract

The human eye is a vital organ responsible for vision, and the health of the retina is crucial for preserving sight. Retinal vessel segmentation plays a key role in the early detection of ophthalmic diseases such as diabetic retinopathy, glaucoma, and hypertension, where vascular abnormalities indicate disease progression. This study introduces a novel hybrid framework that enhances vessel segmentation performance using Fuzzy-Optimized UNet++ architecture, demonstrating a 2% improvement over the baseline UNet++ (95.3% accuracy). The proposed method is trained and validated on five benchmark datasets DRIVE, HRF, IOSTAR, ARIA, and CHASE_DB1offering diversity in image resolution, pathology, and vessel morphology to evaluate cross-dataset generalization potential. To address limitations such as class imbalance, noise sensitivity, and poor micro-vessel continuity, we incorporate fuzzy logic for enhanced boundary refinement and Harris Hawks Optimization (HHO) for robust parameter tuning and convergence acceleration. Additionally, a synthetic vessel generation module, VesselGAN, is used to expand dataset diversity, achieving an SSIM score of 0.89 while preserving anatomical accuracy. Comprehensive evaluation is performed using 10-fold cross-validation and external testing on five independent datasets RETA, IDRiD, IOSTAR (external), Kaggle, and clinical-grade images. The integrated approach achieves superior performance across all key metrics, including Dice Coefficient, IoU, SSIM, and F1-score, especially under noisy, low-contrast, and ultra-thin vessel conditions. This research presents a unified, end-to-end system that advances the state-of-the-art in retinal vessel segmentation. Its superior accuracy, resilience to data variability indicates robustness across unseen clinical domains and suggest suitability for real-world deployment in ophthalmic diagnostic systems.