SekiAto: Leveraging Language Models for Software Security Vulnerability

Software code forms the backbone of modern applications and websites, yet ensuring its security remains a complex challenge. This research presents SekiAto, a novel approach to identifying and mitigating vulnerabilities in source code, particularly within PHP, JavaScript, and C/C++ environments. The proposed system builds upon the strengths of transformer-based language models by fine-tuning DistilRoBERTa and CodeBERT to enhance their effectiveness in vulnerability detection tasks. Unlike earlier models that often suffer from elevated false positive rates and struggle with identifying sophisticated vulnerabilities, SekiAto aims to bridge this gap. The model is trained on a carefully curated dataset that draws from publicly available repositories and verified sources of security flaws, ensuring both relevance and reliability in training data. DistilRoBERTa is specifically optimized for analyzing C/C++ code, while CodeBERT is tailored to handle vulnerabilities in PHP and JavaScript. This domain-specific fine-tuning contributes to notable improvements in detection performance, especially in terms of precision and recall. The approach highlights the effectiveness of fine-tuning transformerbased models using a domain-specific dataset, contributing to more accurate and efficient vulnerability detection. This research has strong implications for improving the security assessment process and enabling early detection of potential threats in software systems.