Large Language Models (LLMs), such as GPT-4, Claude, and Gemini, are reshaping the cyber threat landscape, particularly in the domain of social engineering. These models empower adversaries to automate, personalize, and scale phishing, impersonation, and business email compromise (BEC) attacks with unprecedented realism. Unlike traditional social engineering techniques, LLM-driven threats can adapt to contextual cues, simulate executive communication patterns, and generate deepfake audio or video to enhance credibility. As such, conventional security awareness programs and static detection mechanisms are proving insufficient against the sophistication and speed of these AI-enabled attacks.

This paper investigates the role of generative AI in enabling next-generation social engineering threats and introduces a multi-layered defense strategy. The proposed framework spans technical solutions such as behavioral anomaly detection, AI-driven phishing simulation, and real-time synthetic media analysis as well as human-centric and policy-based countermeasures. Additionally, the study explores adversarial AI, data poisoning, and red teaming as both offensive and defensive mechanisms. Grounded in emerging trends, case studies, and explainable AI (XAI) techniques, this research emphasizes the urgency of adopting adaptive, intelligence-driven cybersecurity practices. The findings aim to inform practitioners and policymakers on building resilient systems capable of detecting, mitigating, and responding to AI-powered social engineering attacks in real time..

Large Language Models (LLMs), Generative AI, Business Email Compromise (BEC), Deepfakes, Synthetic Media

Verizon. 2023 Data Breach Investigations Report. https://www.verizon.com/business/resources/reports/dbir/

Abnormal Security. The New Frontier of Email Threats: AI-Powered Impersonation Attacks, 2023. https://abnormalsecurity.com

Europol. Facing the Impact of LLMs on Cybercrime, 2023. https://www.europol.europa.eu

Gartner. Emerging Tech: AI and the Evolution of Social Engineering Threats, 2024. https://www.gartner.com

IBM X-Force Threat Intelligence Index 2024.

https://www.ibm.com/reports/threat-intelligence

FBI IC3 Report. Internet Crime Report 2022. https://www.ic3.gov/Media/PDF/AnnualReport/IC3Report2022.pdf

Abnormal Security. Generative AI’s Role in Enterprise Email Attacks, 2024. https://abnormalsecurity.com

KnowBe4 Research. Phishing with LLMs: User Behavior under Realistic Conditions, 2024. https://www.knowbe4.com

CISA. AI-Enabled Red Teaming Playbook for Critical Infrastructure, 2024. https://www.cisa.gov

Gartner. Emerging Tech Predictions for Enterprise AI Security, 2024. https://www.gartner.com

Trend Micro. Underground Economy and the Rise of AI-Driven Cybercrime, 2024. https://www.trendmicro.com

Gartner. Predicts 2026: AI Identity Verification and Deception Detection, 2024. https://www.gartner.com

MIT Sloan Management Review. Human-AI Decision Making in the Age of Deepfakes, 2024. https://sloanreview.mit.edu

U.S. DHS. AI System Security and Integrity Directive 2025, Cybersecurity & Infrastructure Security Agency. https://www.cisa.gov

Deloitte. AI in Cybersecurity: Identity, Integrity, and Deception Detection, 2024. https://www.deloitte.com

KnowBe4 Labs. AI-Based Security Awareness Effectiveness Metrics, 2024. https://www.knowbe4.com