loading page

Integrating Large Language Model for Common Criteria Security Document Generation Based on PySide6
  • Bagus Tri Atmaja,
  • Jiann-Liang Chen
Bagus Tri Atmaja
National Taiwan University of Science and Technology College of Engineering

Corresponding Author:[email protected]

Author Profile
Jiann-Liang Chen
National Taiwan University of Science and Technology College of Engineering
Author Profile

Abstract

As cyber threats proliferate with increasing sophistication, the timely and accurate generation of security documents adherent to Common Criteria (CC) standards becomes imperative. This study continues the trajectory set by previous research on CCGenTool, which simplifies the creation of Security Target documents, and elevates it through the integration of Llama2, a cutting-edge Large Language Model (LLM), aimed at enhancing justifications for security objectives within CC documentation. Bringing this advancement to fruition, This research employed Parameter-Efficient Fine-Tuning (PEFT) strategies such as Low-Rank Adaptation (LoRA), optimizing Llama2 without compromising its pre-trained complexity. Adopting an extensive dataset of 692 items derived from various Protection Profiles efficiently annotated for relevance and contextuality—the split designated 70% to train the model while reserving 30% for evaluation purposes. By facilitating a sophisticated fine-tuning protocol that allowed tailor AI outputs closely aligned with industry-specific requirements. Benchmark results exhibit commendable performance: in tests contrasting this research fine-tuned LLM against GPT-3.5 utilizing GPT-4 as an analytical frame, it shows that the result is more towards Llama2 in 25 instances against only seven for GPT-3.5; both models’ outcomes converged in quality 18 times. These findings accentuate how AI can assist not only in safeguarding against emerging cyber threats but also caters to the dynamic landscape where up-to-date CC documentation is crucial for effective defense strategies. By intelligently abbreviating complex document production processes entailed in cybersecurity measures, this innovation confers upon developers a significant advantage—reducing barriers and expediting compliance timelines amidst evolving digital risks.
01 Feb 2024Submitted to Software: Practice and Experience
26 Aug 2024Submission Checks Completed
26 Aug 2024Assigned to Editor
27 Aug 2024Review(s) Completed, Editorial Evaluation Pending
07 Sep 2024Reviewer(s) Assigned