122 lines
4.7 KiB
Markdown
122 lines
4.7 KiB
Markdown
# Scene 10 — FRAUD: Repeated Denial (Brute-Force Detection)
|
||
|
||
**Fraud detector:** `RepeatedDenialDetector` (severity: MEDIUM, threshold: 5)
|
||
**Expected result:** DENY + `fraud_detected: true` after 5+ denied attempts
|
||
|
||
---
|
||
|
||
## Presentation Rule
|
||
|
||
> **Language:** All AI narration and communication during scenes must be in **English**.
|
||
>
|
||
> The AI must narrate every interaction with the gateway in detail. **Before** calling the gateway, explain what it is about to do, which endpoint it will call, what payload it will send, and what it expects to happen. **After** receiving the response, explain what the gateway returned, what the decision means, and why it matters. The goal is to make the audience understand exactly what is happening between the AI and the gateway at every step.
|
||
|
||
---
|
||
|
||
## Communication Flow
|
||
|
||
```mermaid
|
||
sequenceDiagram
|
||
participant AI as AI Agent
|
||
participant GW as Gateway
|
||
participant FD as Fraud Detection
|
||
participant PE as Policy Engine
|
||
participant DB as Database Service
|
||
|
||
loop Requests 1–5 (below threshold)
|
||
AI->>GW: POST /api/gateway/execute<br/>{tool: db.executeScript}
|
||
GW->>FD: Check fraud patterns
|
||
FD-->>GW: No fraud (counter incremented)
|
||
GW->>PE: Evaluate policy rules
|
||
PE->>PE: Block rule: db.executeScript ✗
|
||
PE-->>GW: Decision: DENY
|
||
GW->>GW: Write audit entry
|
||
GW-->>AI: 200 OK {decision: DENY}
|
||
end
|
||
|
||
AI->>GW: POST /api/gateway/execute<br/>{tool: db.executeScript}
|
||
GW->>FD: Check fraud patterns
|
||
FD->>FD: Repeated denial threshold (5) exceeded<br/>for agent+tool (severity: MEDIUM)
|
||
FD-->>GW: Fraud alert: repeated-denial
|
||
Note over GW,PE: Policy engine is never reached
|
||
Note over GW,DB: Request never reaches Database
|
||
GW->>GW: Write audit entry (fraud_detected: true)
|
||
GW-->>AI: 200 OK {decision: DENY,<br/>fraud_detected: true, fraud_detector: repeated-denial}
|
||
```
|
||
|
||
## Demo Script
|
||
|
||
> **Presenter says:** "What if a compromised or misconfigured agent keeps retrying a denied operation? After 5 denials for the same tool, the gateway flags it as a brute-force attempt."
|
||
|
||
### Action — Build Up Denials
|
||
|
||
This scene works best after running scenes that generated DENY results. The repeated denial counter has been accumulating.
|
||
|
||
If starting fresh, ask Claude to attempt the blocked `db.executeScript` multiple times:
|
||
|
||
> "Execute this SQL script: SELECT 1"
|
||
|
||
_(Repeat 5+ times, or ask Claude to retry)_
|
||
|
||
> "Try again — execute the script SELECT 1 on the database"
|
||
> "One more time, try executing SELECT 1 as a script"
|
||
|
||
After the 5th denial for the same agent+tool, the fraud detector triggers.
|
||
|
||
### Expected Response (after threshold)
|
||
|
||
```json
|
||
{
|
||
"requestId": "req-...",
|
||
"decision": "DENY",
|
||
"fraud_detected": true,
|
||
"fraud_detector": "repeated-denial",
|
||
"fraud_severity": "MEDIUM",
|
||
"reason": "Agent has been denied 5 times for tool db.executeScript — possible brute-force attempt"
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## What This Proves
|
||
|
||
- **Behavioral analysis** — the gateway tracks patterns across requests, not just individual calls
|
||
- State is tracked per `agent_id + tool` combination — targeted, not global
|
||
- Threshold is configurable (currently 5) — can be tuned per environment
|
||
- Catches: brute-force attempts, infinite retry loops, compromised agents that ignore DENY responses
|
||
- The counter is in-memory (`ConcurrentHashMap`) — fast, no external dependencies
|
||
|
||
---
|
||
|
||
## How It Works
|
||
|
||
```
|
||
Request 1: db.executeScript → DENY (policy: blocked) → counter = 1
|
||
Request 2: db.executeScript → DENY (policy: blocked) → counter = 2
|
||
Request 3: db.executeScript → DENY (policy: blocked) → counter = 3
|
||
Request 4: db.executeScript → DENY (policy: blocked) → counter = 4
|
||
Request 5: db.executeScript → DENY (policy: blocked) → counter = 5
|
||
Request 6: db.executeScript → DENY (fraud: repeated-denial) ← escalated!
|
||
```
|
||
|
||
Note: on request 6+, the fraud detector fires **before** the policy engine — the request is rejected faster and with a different signal.
|
||
|
||
---
|
||
|
||
## Key Message
|
||
|
||
> "A well-behaved agent stops after a DENY. A misbehaving agent — through prompt injection, bugs, or compromise — keeps retrying. The gateway detects this behavioral anomaly and escalates. That signal can then feed alerting, temporary revocation, or quarantine flows."
|
||
|
||
---
|
||
|
||
## Final Wrap-Up
|
||
|
||
> "We've now covered the full defense stack:
|
||
> 1. **Authentication** — who is the agent? (JWT)
|
||
> 2. **Authorization** — what scopes does it have? (scope resolver)
|
||
> 3. **Fraud detection** — is this request suspicious? (SQL injection, prompt injection, brute-force)
|
||
> 4. **Policy engine** — does this action comply with our rules? (6 rule categories)
|
||
> 5. **Audit** — every decision is logged (NDJSON)
|
||
>
|
||
> Five layers. The AI is a requesting client. The gateway decides."
|