Leveraging Large Language Models for AI-Assisted Cybersecurity Requirement Assessment for Air Traffic Control Systems

Abstract

Air Traffic Control (ATC) systems operate within a strict cybersecurity compliance landscape, where Air Navigation Service Providers and product vendors must rec oncile overlapping obligations from international standards such as ICAO Annex 17 and the NIST SP800series, regional instruments such as the EU NIS2 Directive, and project-specific customer requirements expressed in heterogeneous formats. Align ing these sources with internal product documentation is a manual, time-consuming, and error-prone activity that no prior AI-assisted approach has addressed for the multi-layer evidence structure of safety-critical aviation. This thesis presents the first application of Retrieval-Augmented Generation (RAG) to ATC cybersecurity compliance assessment. The system organises evidence into three coupled layers covering regulatory obligations, customer requirements, and product configuration documentation, and processes each query through a staged pipeline of routing, hybrid retrieval, relevance filtering, and structured generation. Each assessment returns a verdict with inline citations to the supporting passages, runtime diagnostics, and an append only audit trail, shifting the engineer’s task from search to verification. The system runs entirely on-premises and is positioned as an advisory tool that preserves cybersecurity experts as final decision-makers. Following a design science approach, the system was evaluated against a manually validated Gold Standard of 40 queries developed with industrial experts, exceed ing all five pre-registered targets covering retrieval completeness, evidence ground ing, citation correctness, hallucination rate, and justification quality. A controlled comparison between MiniMax-M2.5 and Qwen2.5-VL-72B-Instruct showed that lo cally deployable model choice materially affects hallucination rate and verdict pre cision, with MiniMax-M2.5 having a 0.05 hallucination rate compared to 0.15 from Qwen2.5-VL-72B-Instruct, supporting MiniMax-M2.5 as the default model to be used. A code-level vulnerability assessment identified some weaknesses of the system that were promptly patched. The thesis contributes a domain-specific architecture, a curated evaluation framework, and empirical evidence that evidence-grounded compliance assessment is feasible in safety-critical, confidentiality-constrained envi ronments without removing expert oversight.