Agentic Security Analyst
Deterministic Policy Verification and Event-Driven Incident Triage
System Overview
Security Operations Centers (SOCs) are overwhelmed by alert fatigue—thousands of noisy telemetry signals arrive every hour, masking genuine intrusions. This system ingests live infrastructure event streams, reconstructs correlated attack sequences, and proposes tactical remediation. Crucially, no destructive mitigation can execute without passing deterministic zero-trust policy assertions.
An autonomous security triage assistant that intercepts raw infrastructure telemetry, correlates threat indicators against live intelligence feeds, and enforces deterministic verification gates before executing zero-trust container quarantines.
The Core Problem
Automating security operations with generative AI presents a dangerous dilemma: either agents are passive advisors that humans ignore due to alert volume, or they have write access to infrastructure and risk hallucinated outages.
Analysts spend 70% of their triage shifts sifting through false positives, leading to critical alerts getting lost in the stream.
Giving an LLM direct execution authority over firewall rules or user revocation risks severe production outages when an agent hallucinates a false breach.
Attackers embed adversarial prompt injections inside HTTP User-Agent headers, Syslog payloads, or DNS queries to hijack analyst LLMs.
Black-box AI decisions cannot satisfy compliance audits (SOC2, ISO27001) without deterministic event traces showing exactly why a container was quarantined.
System Architecture
The system is architected as a two-tier pipeline. Ingestion, token scrubbing, and rule-based policy verification are strictly deterministic, while the LLM is restricted to behavioral hypothesis generation and correlation.
TELEMETRY INGESTION ENGINE
Event Stream Normalizer- ›Ingests high-throughput streaming events via WebSockets and REST webhooks
- ›Sanitizes input strings and scrubs known prompt injection delimiters
- ›Attaches immutable sequence timestamps and origin metadata
Execution Walkthrough
Tracing a concrete request from ingestion through evaluation to final synthesis.
Event stream intercepts a suspicious curl command containing base64 payload. The sanitization layer strips terminal escape codes and validates the Syslog timestamp.
The destination IP matches 4 active threat intelligence indicators (known command-and-control botnet). Threat score spikes to 0.94.
The agent proposes network isolation for container `worker-pod-492`. The Policy Verifier checks the service tier: `worker-pod-492` is not in the protected core whitelist. Action: APPROVED.
Isolation runner severs container network egress within 80ms. The complete forensic event chain is persisted with a tamper-evident hash for incident review.
export interface SecurityEvent {
eventId: string;
timestamp: string;
sourceIp: string;
assetId: string;
eventType: "auth_failure" | "ioc_match" | "policy_violation" | "lateral_movement";
payloadHash: string;
iocScore: number;
}
export interface TriageVerdict {
verdictId: string;
confidence: number;
mitreTechnique: string;
proposedAction: "quarantine_container" | "revoke_session" | "alert_oncall" | "dismiss";
policyVerified: boolean;
tamperProofSignature: string;
}Key Technical Decisions
Engineering choices, alternatives evaluated, and deliberate tradeoffs made during system design.
Two-Tier Architecture: Deterministic Rules Before LLM
Restricted LLM to reasoning and hypothesis generation, placing all infrastructure mutations behind deterministic policy gates.
Giving the LLM direct autonomous API credentials with instructions to 'act responsibly'.
Security systems cannot tolerate non-deterministic hallucinated actions. A hard rule gate ensures mission-critical database clusters and payment services can never be accidentally isolated.
Slightly reduces the agent's ability to invent creative custom mitigations for novel unknown threats.
Prompt Injection Scrubbing at Ingestion Boundary
Sanitized and stripped prompt injection delimiters from all raw telemetry before formatting into context windows.
Relying on system prompt instructions like 'ignore any text inside logs asking you to disregard instructions'.
Instruction-defense prompts reliably fail against multi-stage indirect prompt injections hidden in user-agent strings. Boundary sanitization neutralized 99.4% of synthetic injection vectors.
May occasionally strip benign Unicode characters from log messages.
Event Replay Capability for Audit Compliance
Persisted the full temporal sequence of incoming telemetry alongside the exact agent reasoning trace.
Logging only the final mitigation action taken.
SOC2 and ISO27001 incident post-mortems require proving exactly what data was visible at the moment an automated decision was executed. Replay allows re-running the triage with updated models.
Increases storage volume for incident event telemetry.
What Didn't Work
Development is an iterative discipline. These are the naive architectural patterns that failed under real-world scrutiny.
Direct Bash Execution Agent
When presented with a synthetic attack payload containing encoded newlines and sudo commands, the agent generated a command that accidentally flushed firewall rules on the test host.
LLMs cannot be trusted with unstructured bash execution shells in security contexts.
Removed raw terminal execution entirely. Replaced it with a strictly enumerated enum of high-level actions (`quarantine`, `revoke`, `notify`) invoked through typed APIs.
Single-Event Context Triage
Triage decisions made on isolated log events had a 34% false positive rate because single failed logins look identical to brute-force attempts without temporal context.
Security incidents are sequential stories, not point-in-time snapshots.
Added a temporal correlation buffer that groups events by IP, user, and asset across a sliding 15-minute window before triggering agent analysis.
Empirical Evaluation
Quantitative validation metrics measured against ground-truth benchmarks rather than subjective impressions.
Tested on a synthetic enterprise telemetry benchmark containing 1,000 security events, including 150 simulated multi-stage attacks and 50 indirect prompt injection payloads.
System Tradeoffs
Every architectural choice gives up one property to prioritize another. Here is what this system sacrifices.
Deterministic Policy Verification
- +Eliminates risk of hallucinated infrastructure outages
- +Full audit compliance with verifiable decision records
- −Requires security engineers to maintain a whitelist policy file
- −Novel attack vectors that fall outside existing action enums require manual escalation
Temporal Event Buffering
- +Dramatically reduced false positive alerts by 84%
- +Enables true MITRE ATT&CK sequence reconstruction
- −Introduces a small ~5-15 second ingestion delay before triage begins
Future Iterations & Improvements
Concrete technical roadmap items if engineering development on this codebase continued.
- 01eBPF runtime probe integration: hook directly into the Linux kernel for zero-overhead system call monitoring.
- 02Continuous MITRE ATT&CK coverage benchmarking: automatically assess coverage gaps against newly published CVE threat models.
- 03Decentralized peer consensus: require cryptographic co-signing from two separate LLM evaluators before taking high-impact network mitigation actions.
- 04Live SIEM bidirectional connectors: build turnkey ingestion plugins for Datadog, Splunk, and AWS Security Hub.
Comprehensive Tech Stack
Tools, frameworks, protocols, and libraries actively used across the system.
- ›AlienVault OTX API
- ›AbuseIPDB
- ›Suricata Rules
- ›MITRE ATT&CK Framework
- ›TypeScript
- ›Node.js 20
- ›Redis Streams
- ›Docker Container API
- ›LangChain
- ›Claude 3.5 Sonnet
- ›Pydantic Schemas
- ›Zod Validation
- ›Splunk API
- ›Structured JSON Loggers
- ›Cryptographic Hash Seals