#14.9 Deepfake and AI-Enabled Social Engineering

Playbook ID: PB-DEEPFAKE | Default severity: SEV-3 (SEV-2 once a payment has been sent or a credential, MFA re-enrolment or access grant has been given up; SEV-1 if the target held a privileged role or synthetic media of a named executive is circulating outside the company) | Owner: Incident Commander

#When to run this

Open this playbook on: an employee reporting a call, voicemail, video meeting or voice note from an "executive" pressing for a payment, a credential, a gift-card purchase or an urgent exception; a service-desk contact requesting a password reset or MFA re-enrolment where the caller's identity rests on their voice, or a request deliberately split across two contacts — the CISA AA23-320A pattern; an approach that arrives on a channel the real person never uses (personal WhatsApp, a new mobile number, a meeting invite from outside the tenant); a participant on a video call whose audio, lip sync or lighting is wrong, or who will only appear on camera and never type; a synthetic audio or video clip of a named executive circulating externally; or a phishing wave with unusually fluent, well-targeted copy at volume and almost no shared indicators.

Not this playbook: a compromised mailbox sending real mail from a real account (14.2 PB-BEC — and if the deepfake succeeded and money left, run 14.2's money track in parallel from minute one); a completed identity-provider or privileged-credential compromise (14.4 PB-IDP); an attack against your own AI systems, agents or model supply chain (14.11 PB-AISYS); a genuine employee abusing genuine access (14.6 PB-INSIDER). The design of help-desk identity verification, and the phishing-resistant MFA program that makes a stolen reset worthless, belong to Chapter 4. Executive media exposure and workforce training sit in Chapter 19.

#What you are dealing with

Someone will call your accounts payable clerk in your CFO's voice. Not a robotic approximation — the voice, with the pauses and the regional vowels, on a Tuesday afternoon, about an invoice that genuinely exists. The technology is a commodity; the target is not the technology, it is the moment where one junior person weighs their own doubt against apparent authority and picks authority.

The reference case is Arup's Hong Kong office: an employee received a phishing email impersonating the UK-based CFO, was sceptical, and had that scepticism dismantled by a multi-person video conference in which every other participant was AI-generated. About US$25.6 million left across 15 wire transfers in a single day (CNN). Three other named attempts failed, and how they failed is the whole of this playbook. WPP's CEO was impersonated through a WhatsApp account, a Teams meeting, a voice clone and stitched YouTube footage — stopped by an employee who did not buy it (OECD AI Incidents). A Ferrari executive challenged a voice clone of the CEO with a shared-secret question — a book the real CEO had recently recommended — that the clone could not answer (AI Incident Database). A LastPass employee flagged a voice clone of their CEO because the channel was wrong — WhatsApp, outside normal business communication — not because the audio sounded off (LastPass). Zero of those three were stopped by detection technology. Three of three were stopped by a human process check.

This is not a niche. Voice phishing was the #2 initial infection vector in 2025, at 11% of all Mandiant investigations (M-Trends 2026), and DBIR 2026 finds voice and text phishing convert better than email (Help Net Security). IC3 added "AI-related" as a formal crime descriptor for the first time in its 2025 report: 22,000+ complaints and roughly $900 million in losses (FBI). The written variant scaled too — Hoxhunt found AI-generated spear phishing went from 31% less effective than elite human red-teamers in 2023 to 24% more effective by March 2025 (Hoxhunt), and Microsoft assesses AI can make some phishing operations up to 50× more profitable (MDDR 2025). Keep the calibration honest, though, because it changes what you fund: Mandiant's conclusion from over 500,000 hours of 2025 response work is that 2025 was not the year breaches directly resulted from AI — most intrusions still stem from human and systemic failures, and Anthropic's mapping of banned malicious accounts found AI-assisted phishing actually fell 8.6% while post-compromise use rose (Anthropic). AI is a force multiplier on social engineering you already faced, not a new kill chain.

Which brings us to the mistake teams make, and it arrives as a purchase order. The instinct is to buy a synthetic-media detector and declare the problem handled. But that enters your people into a perception contest against a generator that improves monthly, with a stressed clerk as the classifier. Do not enter that contest. Actionable takeaway: stop trying to detect the fake and start verifying the request — a callback on a number from your own directory, a shared-secret challenge, dual authorization. The control is procedural, it is nearly free, and it works against a perfect fake.

#Roles for this incident

#Phase 1 — Detection and Triage

The first fifteen minutes answer one question — did the request succeed — and preserve one artefact that has a short and unforgiving life. Voicemail boxes overwrite. Meeting chats fall off retention. People delete embarrassing messages.

# Pull the audit record around the approach for the targeted account and the impersonated party.
# Search broadly first, then filter the returned records - but the search is only broad if
# -SessionCommand ReturnLargeSet is set. Without it the cmdlet returns at most 100 records
# however high you set -ResultSize, and filtering a truncated set is how you conclude that
# nothing happened. ReturnLargeSet comes back unsorted; re-run it with the SAME -SessionId
# until it returns zero rows, then sort what you have. Operation strings vary by tenant and by
# how the action was performed - read them out of your own results, do not assume them.
Search-UnifiedAuditLog -StartDate <MM/DD/YYYY> -EndDate <MM/DD/YYYY> `
  -UserIds <target-upn>,<impersonated-upn> `
  -SessionCommand ReturnLargeSet -SessionId <id> -ResultSize 1000

# Risk state on both accounts, in case the social engineering rode on an existing compromise.
Connect-MgGraph -Scopes "IdentityRiskEvent.Read.All","IdentityRiskyUser.ReadWrite.All"
Get-MgRiskDetection -Filter "RiskType eq 'anonymizedIPAddress'" |
  Format-Table UserDisplayName, RiskType, RiskLevel, DetectedDateTime

Search-UnifiedAuditLog syntax · ID Protection via Graph · eDiscovery holds

#Phase 2 — Containment

Containment here is unusual: there may be no malware, no beachhead and nothing to isolate. What you are containing is an instruction in flight and an adversary's ability to place the next call.

#Phase 3 — Eradication

There is often no implant to remove. What you eradicate is the adversary's information advantage and the process gap that let a voice function as an authorization.

# Tenant-wide consent inventory - Microsoft's documented method.
.\Get-AzureADPSPermissions.ps1 | Export-csv -Path "Permissions.csv" -NoTypeInformation

# Revoke what the inventory turns up - Microsoft's two documented revocation cmdlets.
Remove-MgOauth2PermissionGrant                  # revokes a delegated consent grant
Remove-MgServicePrincipalAppRoleAssignment      # revokes an application-permission role assignment

Detect and remediate illicit consent grants

Blocklisting is close to worthless against a caller who buys a new number for nine dollars, and indicator-matching is close to worthless against generated phishing text that is unique per recipient. Takeaway: hunt on the campaign's shape — the targeted role, the send window, the landing infrastructure, the pretext — and put the caller identifiers in the hunt query, not just the block list.

#Phase 4 — Recovery

Phishing-resistant MFA blocks over 99% of identity-based attacks even when the attacker already holds a valid username and password (MDDR 2025). It does not stop a deepfake call, but it makes the credential the caller is fishing for far less useful. Number matching is a push-fatigue mitigation and CISA is explicit that it is not phishing-resistant MFA (CISA). Actionable takeaway: step 4.8 is not sentiment. If reporting a suspected fake costs an employee an awkward conversation with an executive, the next one will not report. Make the report cost nothing, publicly, once.

#Phase 5 — Post-Incident

#Decision points

#Communications and notification triggers

The deepfake itself almost never starts a regulatory clock. What starts one is what the social engineering obtained. If the approach yielded access to personal data, GDPR Article 33's 72 hours from awareness is running, and the Scribe's timeline is your only evidence of when awareness arose. If it yielded a credential into a regulated service, the NIS2 and DORA clocks may run on the downstream compromise rather than on the call. If the loss could be material to a public filer, the Executive Sponsor opens the SEC materiality assessment on day one. The full matrix is Chapter 15 — do not reconstruct it under pressure.

Two things belong here rather than in Chapter 15. File with IC3 regardless of loss amount when funds moved — it is the entry point to the Recovery Asset Team, not a regulatory notification, and 14.2 Phase 2A owns the mechanics. And notify counterparties by telephone on numbers you already held, never by replying to any thread the approach touched.

#Automation notes

Automate the preservation, never the determination. The moment a suspected-impersonation report opens, a playbook can safely and reversibly: pull the meeting and call records for the window, snapshot the voicemail or recording and hash it, place the eDiscovery hold, export sign-in and audit logs for both the target and the impersonated party, extract the caller identifiers, search for the same identifiers across mail and telephony, and attach the lot to the ticket. All read-only, all racing a seven-day Entra Free retention, all faster than a human opening a console.

Gate everything else. The verification callback is the one step that must never be automated — its entire value is a human hearing a human on a number the attacker did not supply, and a system that auto-approves on a matched voiceprint has recreated the vulnerability in software. The rule that holds up: automation may gather, enrich, correlate and recommend without approval; it may act only where the action is reversible, scoped and rate-limited; irreversible or organization-wide actions require a named human approver. The tenant-wide reset freeze (2.4), the workforce broadcast (2.6) and the payment release (4.1) are all named-approver actions. And resist wiring an AI triage agent to auto-close these reports: the documented failure modes are overconfident closure on weak proof and hallucinated detail in the investigation narrative (Panther), and a report that reads like "employee thought a call sounded strange" is precisely where both bite.

#Pitfalls