JOUNES // REPORTS
Home  ·  Projects  ·  Essays  ·  GitHub  ·  LinkedIn  ·  Email
// // RESEARCH REPORT

Analyzing the structural convergence of memory-corruption and prompt-injection as a

·10 citations

Overview

This research focuses on treating Prompt Injection not as a linguistic failure to be solved via alignment, but as a structural failure of capability-binding analogous to memory-corruption vulnerabilities [C001, C002]. In these attacks, the LLM fails to distinguish between system instructions and untrusted data—the "Confused Deputy" problem—allowing external data to hijack the agent's execution flow [C001]. To resolve this, the research implements a "Semantic-Physical" security boundary, shifting the defense from probabilistic alignment (relying on the model to refuse a malicious request) to architectural guarantees where the execution environment physically lacks the capability to perform unauthorized actions [C001, C002].

This shift is critical because traditional heuristic defenses—such as guard models, input sanitization, and prompt engineering—are fundamentally bypassable and cannot provide the deterministic guarantees required for production deployment in high-risk environments like healthcare [C002, C005, C009].

Defense Approach Mechanism Primary Weakness Security Guarantee
Behavioral RLHF, Guardrails, Sanitization Heuristic; susceptible to evasion [C002, C009] Probabilistic/None
Structural Capability-Based Security, Sandboxing Potential utility trade-offs [C001] Deterministic/Architectural [C001, C002]

Recent developments demonstrate that implementing these boundaries at the operating system level eliminates ambient authority across the agentic loop:

Landscape

Current efforts to secure agentic loops have shifted from behavioral alignment toward structural constraints that treat the LLM as a compromised orchestrator [C009]. These approaches generally fall into three categories:

1. OS-Level Structural Isolation
This approach implements the "Controlled Body" by moving security boundaries from the prompt to the kernel. Namespace-Bounded Agents (developed by NERV Systems) replaces API schemas with Plan 9-style 9P filesystem semantics, where capabilities are defined as filesystem paths [C001]. Because resources outside the mounted namespace are inaccessible in the execution context, this architecture provides high resistance to attack [C000]. Similarly, production implementations in healthcare and other contained environments utilize gVisor to provide user-space kernel isolation, intercepting syscalls to deliver VM-level security at container speed [C005, C006].

2. Dynamic Capability & Credential Sidecars
Rather than granting agents long-lived API keys, these systems use sidecars to inject task-scoped credentials. The VAC Protocol utilizes datalog policies and signed tokens to ensure the agent never sees the raw API key, requiring signed receipts for subsequent calls [C007]. This is complemented by "Caging" architectures that combine credential proxy sidecars with strict network egress policies, restricting agents to allowlisted destinations to prevent data exfiltration [C005].

3. Automated Red-Teaming and Benchmarking
To quantify the failure of behavioral defenses, new frameworks formalize the agentic attack surface. Agent Security Bench (ASB) evaluates attacks across 10 scenarios (e.g., finance, autonomous driving), reporting average attack success rates as high as 84.30% and noting the limited effectiveness of current defenses [C004]. To address the resource intensity of testing long-context models, FlashRT provides optimization-based red-teaming that reduces GPU memory consumption by 2x-4x and runtime by 2x-7x compared to baseline nanoGCG [C003].

Defense Strategy Comparison

Approach Mechanism Trust Assumption Primary Failure Mode
Behavioral Guard models, RLHF, Prompting [C002, C009] Model will follow safety instructions Adversarial bypass/jailbreak [C004]
Dynamic Sidecars, Task-scoped tokens (VAC Protocol) [C007] Policy engine and sidecar are secure Classifier bypass/Logic error [C005]
Structural 9P Namespaces, gVisor sandboxing [C001, C006] Kernel/Hypervisor isolation Over-isolation reducing utility [C006]

Key Findings

Evidence indicates that behavioral alignment and heuristic filters are insufficient for securing agentic loops, with Agent Security Bench (ASB) reporting an average attack success rate (ASR) of 84.30% across 13 LLM backbones [C004]. Because prompt injection is fundamentally a failure of the LLM to distinguish between system instructions and untrusted data, defenses based on input sanitization or guard models remain brittle and bypassable [C002, C009].

Research has shifted toward structural guarantees that treat the LLM as a compromised binary, implementing security at the runtime and OS level:

The following table compares the effectiveness and operational impact of these defense strategies:

Strategy Mechanism Security Guarantee Impact on utility/Performance
Behavioral RLHF, Prompt Filtering Heuristic (Bypassable) [C002] Low latency; high flexibility [C004]
Structural 9P Filesystem Namespaces OS-level (0% ASR) [C001] 83.5% reduction in tool-calling tokens [C001]
Isolation gVisor / Sandboxing Syscall-level isolation [C005, C006] Higher overhead; requires image rebuilds [C006]
Dynamic VAC Protocol Cryptographic (Task-scoped) [C007] Heartbeat-based revocation (seconds) [C007]

These findings demonstrate that shifting the security boundary from the "Semantic" (the LLM's understanding of a prompt) to the "Physical" (the OS's enforcement of a namespace or syscall) eliminates the reliance on model alignment [C001, C009]. This convergence treats prompt injection as a memory-corruption equivalent, where the "buffer overflow" is the injection of untrusted data into the instruction stream, necessitating a strict capability-based security model [C001, C008].

Tensions and Tradeoffs

Practitioners implementing capability-binding boundaries face a fundamental conflict between agentic autonomy and structural security. The shift from behavioral alignment (RLHF) to architectural constraints forces a tradeoff between the flexibility of the "untrusted brain" and the rigidity of the "controlled body" [C001, C009].

The primary tension lies in the mechanism of capability acquisition. Some high-assurance models implement a system where agents start with minimal permissions and must request new capabilities through a human-in-the-loop (HITL) process, with each approval triggering a full container image rebuild [C006]. While this ensures immutable infrastructure and a complete audit trail, it eliminates the agent's ability to perform autonomous tool discovery, transforming the agent into a high-latency, human-gated executor [C006, C008].

Another critical tradeoff exists in the interface between the LLM and its tools. Traditional JSON-RPC tool calling relies on semantic schemas that are susceptible to poisoning [C001]. Transitioning to 9P filesystem semantics—where capabilities are filesystem paths—converts semantic poisoning attacks into syntactic errors that can be verified at the implementation level [C001]. However, this requires abandoning standard API patterns in favor of a Plan 9-style namespace architecture [C001].

Finally, the move toward "Dynamic Capability" introduces a shift in the attack surface. The VAC Protocol removes raw API keys from the agent's reach by using a sidecar to inject task-scoped credentials based on datalog policies [C007]. While this prevents credential exfiltration, it concentrates risk into the sidecar's policy engine; if the policy logic is bypassed or misconfigured, the agent gains high-value access without the friction of traditional identity management [C007].

Strategy Security Gain utility/Performance Cost
9P Namespacing 0% ASR on benchmark; eliminates semantic tool poisoning [C001] Requires non-standard OS-level filesystem integration [C001]
HITL Gating Immutable infrastructure; prevents unauthorized capability escalation [C006] High latency; removes autonomous tool discovery [C006]
Sidecar Proxies Prevents raw secret exposure; heartbeat-based revocation [C007] Increased architectural complexity; policy engine becomes a single point of failure [C007]
Kernel Isolation VM-level isolation via gVisor prevents syscall abuse [C005, C006] Increased compute overhead compared to native containers [C005]

Opportunities

Structural Architectures to Build

To move beyond heuristic defenses [C002], development should shift toward OS-level enforcement that treats the LLM as a compromised binary [C001].

Critical Research Questions

References

Provenance: Published 2026-05-11 · 10 inline citations · 10 references
// GENERATED FROM A LIVE OBSIDIAN VAULT · CLOUDFLARE PAGES · DRAFTED WITH AGENTS
← back to Reports