AI Agents

AgentChaos brings chaos engineering to agent systems by intercepting and mutating LLM API responses at the shared HTTP layer — no source changes required. It defines crash, omission, and value faults over content and tool-call fields, then verifies each fault actually triggered to avoid understating impact. Across multiple systems, benchmarks, and backbones under 65 fault configurations, pass@1 drops by up to 50 points, and rankings hold across models — suggesting robustness lives in the harness, not the model. Existing fault diagnosis stays below 53% accuracy.

HarnessSafe targets a risk most benchmarks ignore: attacker-influenced content that persists through memory, skills, tools, and shared artifacts, then detonates on a later benign request. It offers 328 executable cases across seven persistent-carrier families, each modeled as a Persistent-Risk Lifecycle traced from entry through cross-boundary persistence to trigger and violation. The trace-based evaluation measures how far each attack chain progresses rather than just end-to-end success. Finding: containment is carrier-specific and depends heavily on the harness-model pairing, so aggregate attack-success rates hide where risks actually stop.

LivePlan monitors programming agents in real time to catch the failure modes of long trajectories: plan drift, repeated failed actions, terminating without a patch. The design separates judging from advising — a deterministic, rule-based monitor watches trajectory signals without an LLM, and only consults an advisor LLM when something trips, avoiding the cost and misfires of blanket re-planning. Built on SWE-agent and tested with five models across SWE-bench Verified and Pro, it lifts resolution rates by up to 15.2% (avg 9.9%) for eight cents per instance, gains concentrated on medium and hard cases.

PDFuzzer uses an LLM to build context-free grammars and infer API-call relationships from JavaScript manuals and execution traces, then a constraint solver generates meaningful call sequences — moving past single-call fuzzing that misses sequence-dependent bugs. Against Acrobat Reader, Foxit, and PDF-XChange, it beats prior PDF fuzzers and LLM-based fuzzers by up to 48% coverage and found 31 zero-days, from info leakage to arbitrary code execution, all disclosed and bounty-paid. The ablation reports 93-98% LLM accuracy per pipeline stage. Real vulnerabilities, not just coverage claims.

ReASearch asks how much of an optimizer's outer loop — evolutionary search, bandits, textual gradients — can be absorbed into a single tool-using agent's reasoning. The agent decides what to evaluate, diagnoses failures, chooses edits, and allocates budget over long horizons via persistent memory, using one scaffold across prompt, program, and ML-workflow optimization. Across 14 tasks it matches or beats specialized systems by 2-40%, sometimes topping prior human best-known results, and complex search behaviors emerge from reasoning rather than hand-coded controllers. The generality claim is the interesting part.

Claude Code now ships with auto mode as the default, folding permission prompts into autonomous execution for most actions. This is the rollout the earlier writeups previewed, now landing. The blog is the vendor's framing, so the eval numbers behind the confidence — the mid-session dangerous-action swap, the prompt-injection claims — aren't reproduced here. If you run Claude Code in CI or shared environments, the change to what the agent can touch without asking is the part that matters.

Docker is shipping Sandboxes, a GA product offering disposable, isolated environments for agents to execute code and tool calls. The pitch is straightforward: give agents a throwaway container instead of your host. That's a real need as auto-mode agents get more execution latitude. The catch is that this is a marketing page, not an engineering doc — no detail on isolation boundaries, escape surface, filesystem or network egress controls, or how ephemeral the teardown actually is. The primitive is welcome; the guarantees are unstated.