AI-Driven Insecurity: Assessing Security Gaps in AI Generated IT Guidance
The increasing reliance on AI-generated technical guidance for IT system configuration introduces significant security risks. This study assesses these risks through a case study: setting up an Apache web server on a Rocky Linux system using instructions from seven AI models. This inquiry also addresses the potential for over-reliance on AI and the possible erosion of cybersecurity skills among IT professionals.
The research demonstrates the variability and potential security gaps in AI-generated instructions by analyzing responses to two carefully designed prompts. The findings highlight that AI models, in their native state, often do not adequately account for cybersecurity best practices, and that security-focused prompts are essential to elicit more secure configuration guidance. These results emphasize the critical need for human oversight, validation, and security expertise in AI-driven IT operations.
SANS_AI_Driven_Insecurity_Assessing_Security_Gaps_in_ AI_Generated_IT_Guidance_Ed_Abbott (PDF, 0.51MB)
13 May 2025Related Content
Secure By Design: An Exploration of the Application of Generative AI in Threat Modeling Technical Design Documents
Research PaperThis paper explores the efficacy of large language models (LLMs) for creating comprehensive threat models by analyzing technical design documents, particularly when provided with additional contextual information about the product's underlying infrastructure and deployment environment.
- 27 May 2026
Leveraging Large Language Models for Cross-Vendor Firewall Configuration Migration: A Comparative Case Study of Claude and ChatGPT
Research PaperThis paper investigates how two current-generation large language models (LLMs) perform on a single, representative firewall migration task.
- 12 May 2026
Infrastructure as Code-Driven Group Policy Infrastructure: A Comprehensive Engine for Group Policy Architecture and Enforcement
Research PaperThis study introduces a PowerShell-based Infrastructure as Code (IaC) engine developed to automate the setup and enforcement of a STIG-compliant Group Policy framework.
- 5 Dec 2025
No-Cost Detection of Endpoint Hard Drive Removal
Research PaperThis paper analyzes low-cost detection methods, using existing hard drive counters from Self-Monitoring, Analysis, and Reporting Technology (S.M.A.R.T.) and the Windows Registry, for their fidelity in detecting hard drive removal.
- 19 Nov 2025
Defending Vulnerable Populations Against Scams: Effectiveness of Browser Extensions in Mitigating Scammer Attack Chains
Research PaperThis research evaluates the effectiveness of a browser extension as a security control—Grandma’s Guardian—designed for simplicity and accessibility so that even non-technical home users can benefit from enterprise-grade protection.
- 19 Nov 2025
Automating Generative AI Guidelines: Reducing Prompt Injection Risk with 'Shift-Left' MITRE ATLAS Mitigation Testing
Research PaperAutomated testing during the build stage of the AI engineering life cycle can evaluate the effectiveness of generative AI guidelines against prompt injection attacks.
- 7 Nov 2025
Can Your Security Stack Handle AI? An Empirical Assessment of Enterprise Controls Versus Generative AI Risks
Research PaperEnterprise security teams face a critical dilemma. Executives want AI productivity gains, but it remains uncertain if existing security controls can handle the risks.
- 6 Nov 2025
Building Scalable Detection-as-Code Pipelines with Agentic Validation and Refinement
Research PaperThe proposed DaC pipeline uses large language models (LLMs) for logic conversion, variant analysis, and simulation testing via Atomic Red Team, with queries executed against Splunk to measure true positives and false negatives.
- 6 Nov 2025
Isolated Trust: Zero Trust in Standalone Systems
Research PaperThe use of air-gapped, isolated systems remains an essential tool for organizations that require high confidentiality or integrity, including those in the government, industrial control systems, and the banking industry.
- 6 Nov 2025
"You Again": Fingerprinting and Tracking Mechanisms of Malicious Sites
Research PaperBrowsers provide many APIs for any visited site to perform stateful and stateless tracking, and legitimate websites utilize these capabilities. Yet little is widely known about what tracking, if any, malicious sites perform.
- 26 Sep 2025
Fixing What You Broke: Can AI Be Used to Thwart AI-Generated Malware?
Research PaperSecurity professionals are starting to rethink their approach to access control and monitoring for...
- 3 Sep 2025
Trust But Verify: Evaluating the Accuracy of LLMs in Normalizing Threat Data Feeds
Research PaperThis paper examines whether Large Language Models (LLMs) can be reliably applied to the normalization of Indicators of Compromise (IOCs) into Structured Threat Information Expression (STIX) format.
- 16 Jul 2025
Evaluating Zero Trust Network Access: A Framework for Comparative Security Testing
Research PaperEvaluating Zero Trust Network Access: A Framework for Comparative Security Testing
- 11 Jul 2025
Do AI Coding Assistants Make Bad Coders Worse? A Security Evaluation of GitHub Copilot
Research PaperAs AI coding assistants become increasingly integral to software development, the security of their generated outputs is under greater scrutiny.
- 11 Jul 2025
SIEM Detection Logic Conversion with LLMs
Research PaperThis research explores how Large Language Models (LLMs) and automation scripts can expedite the translation of detection logic between SIEMs, converting detections in minutes instead of hours.
- 2 May 2025
Validating the Effectiveness of MITRE Engage and Active Defense
Research PaperThis research examines the impact of Active Defense compared to a traditional security posture when an adversary employs common tactics and techniques to identify high-value targets or exfiltrate sensitive data.
- 29 Mar 2025
Shift Left the Awareness and Detection of Developers Using Vulnerable Open-Source Software Components
Research PaperThe number of open-source software components, as well as the number of existing security...
- 26 Mar 2025
Leveraging Large Language Models for Security-Focused Code Reviews
Research PaperThis study investigates the potential application of Large Language Models (LLMs) in enhancing...
- 26 Mar 2025
Strolling Through the STIG
Research PaperThe CKL file has become the unofficial common language amongst the Department of Defense activities...
- 7 Mar 2025
Building Resilient IoT Devices: Binary Hardening with Yocto and Clang
Research PaperThis paper addresses the critical need for enhanced security in Internet of Things (IoT) devices by evaluating the implementation of binary hardening techniques using Clang security features within the Yocto build environment.
- 3 Mar 2025