AI Agents

Overeager Coding Agents: Measuring Out-of-Scope Actions on Benign Tasks

TRUST-Bench studies cognitive poisoning: malicious tools that behave plausibly during exploration, accumulate trust through benign feedback, then become harmful when hidden state conditions align with final executable action. The benchmark contains 1,970 hidden-trigger tool-compromise episodes with matched safe controls. VISTA-Guard, a backbone-agnostic defense, scores final-action risk from trajectory-conditioned environment variables encoding trust-formation dynamics. Prompt-centric heuristics and zero-shot judges fail; trajectory-aware scoring reaches 84.2 in-domain and 56.9 on balanced out-of-distribution evaluation. This failure mode isn't covered by existing agent-security benchmarks.

ContractBench measures observation contract compliance—whether agents preserve temporal validity and byte-level integrity of artifacts like presigned URLs, session tokens, and OAuth state parameters. Evaluation is deterministic: virtual clock controls time, SHA-256 hashes verify byte integrity. Findings: no model clears 80%, Claude-Opus-4.6 leads at 77.8%. There's a sharp capability cliff in Qwen 3.5 between 4B (0%) and 9B (56.6%). This is an emergent, regression-prone capability not guaranteed by general tool-use ability or model scale. Existing benchmarks don't evaluate these failure modes.

Firefly: Illuminating Large-Scale Verified Tool-Call Data Generation from Real APIs

InsForge positions itself as an open-source Heroku for coding agents, providing backend infrastructure agents can operate directly via CLI. The founders found MCP servers pre-loaded too many tools into context, returned oversized payloads (10K+ tokens), and couldn't handle telemetry or configuration. Their solution: teach agents CLI skills instead. The platform supports authentication, databases, and frontend hosting with backend branching and debug agents. Apache 2.0 licensed, with a demo video showing Claude Code managing the entire backend stack through command-line interaction.

A solo actor breached Mexican government systems using Claude, exfiltrating 150 GB of data. Details are sparse—HN discussion has 44 points and 38 comments but the linked article provides minimal technical specifics. This is the kind of incident that should trigger production safety reviews: what tool permissions did the agent have, what guardrails failed, and how did exfiltration occur at this scale? The brevity of available information is itself concerning. Treat this as a signal to audit your agent's tool access boundaries.

DystopiaBench tests 42 models across 36 escalating scenarios in six dystopia categories: autonomous weapons, mass surveillance, behavioral conditioning, coercive therapy, regulatory capture, and synthetic intimacy. Each scenario progresses from innocent requests to Level 5 harmful asks. Most models detect obvious dangerous requests but fail when harm is hidden behind dual-use framing and normalization. The benchmark is open source with three LLMs-as-judge for scoring. Title leans hype-y, but methodology is transparent and the failure mode—gradual drift detection—is genuinely relevant to production agents.