2026-07-22

AI Agents

A controlled study of multi-step tool agents finds that the obvious fix for entity errors — locking the first binding and reusing it — actually amplifies wrong actions 3x to 8.5x, while a single cheap re-verifier call cuts them 79%. A measurement study shows LLM browser agents defeating the bot defenses meant to stop them. Around these: papers on failure attribution, data-leakage hardening, cost-aware recovery routing, plus notes from the Claude Code team.

5 papers 2 news 1 blogs 32 appendix 498 considered

Papers

5

AgentDebugX: An Open-Source Toolkit for Failure Observability, Attribution, and Recovery in LLM Agents

arxiv:arxiv-agents — Kunlun Zhu, Xuyan Ye, Zhiguang Han, Yuchen Zhao, Bingxuan Li, Weijia Zhang, Muxin Tian, Xiangru Tang, Pan Lu, James Zou, Jiaxuan You, Heng Ji 8/10 observabilitycode-agentstool-use
An open-source debugging framework built around the observation that the step where an agent error surfaces is rarely the step that caused it. It organizes debugging as a Detect–Attribute–Recover–Rerun loop, with a multi-turn root-cause component doing global trajectory analysis and cross-examination. On the Who-and-When benchmark it hits 28.8% exact agent-and-step attribution on qwen3.5-9b versus 21.7% for the best single-pass baseline; on GAIA it repairs 13 of 73 failed tasks in one rerun, lifting accuracy from 55.8% to 63.6%. Ships as library, CLI, and web console.

Binding Drift in Multi-Step Tool-Augmented Agents

arxiv:arxiv-llm-systems — Rahul Suresh Babu, Shashank Indukuri 8/10 tool-useevalsobservability
This formalizes binding drift — an entity correct at step 1 that silently drifts wrong later — as distinct from error propagation, scoring them on disjoint workflow sets so they can't be conflated. The counterintuitive result: an entity lock that persists the first binding amplifies wrong actions 3.0x (up to 8.5x on Opus 4.5), because it faithfully carries seeded errors forward. A single cheap re-verifier call that re-reads the original instruction cuts wrong actions 79%, within a point of oracle. 200 workflows, eight backends, four enterprise domains.

Broken Gates: Re-evaluating Web Bot Defenses in the Age of LLM Agents

arxiv:arxiv-agents — Behzad Ousat, Nikita Turkmen, Lalchandra Rampersaud, Dillan Bailey, Amin Kharraz 8/10 safetyevals
A systematic measurement study of whether web bot defenses hold up against LLM browser agents, spanning seven CAPTCHA solvers and six agents against hCaptcha, reCaptcha v2/v3, and Cloudflare Turnstile. Challenge-based defenses fall to commercial solvers at near-zero cost and to agents with a solver module. The more interesting finding is on non-interactive defenses like reCaptcha v3: its apparent resistance isn't a security property, since two agents with nearly indistinguishable behavioral traces get scored differently. The signal is in the environment, not the agent.

Data Leakage Prevention in Agentic Applications via Preemptive Hardening

arxiv:arxiv-agents — Akansha Shukla, Emily Bellov, Parth Atulbhai Gandhi, Yuval Elovici, Asaf Shabtai 8/10 safetymulti-agentobservability
A pre-deployment pipeline that scans agentic apps for data-leakage patterns, then hardens and validates them. It analyzes prompt templates, tool interfaces, and invocation code to flag leakage-enabling patterns, prioritizes high-risk tools, and applies minimally invasive mitigations — schema tightening, boundary sanitization, allowlist gating, least-privilege checks. Validation auto-generates jailbreak and instruction-override attacks alongside benign variants to confirm functionality survives. Evaluated on five real apps plus AgentDojo with 91-100% leak reduction. The two-sided validation — attacks and benign inputs together — is the part worth stealing.

CodeRescue: Budget-Calibrated Recovery Routing for Coding Agents

arxiv:arxiv-agents — Qijia He, Jiayi Cheng, Chenqian Le, Rui Wang, Xunmei Liu, Yixian Chen, Jie Mei, Zhihao Wang, Xupeng Chen, Yuhuan Chen, Tao Wang 8/10 code-agentscost-latencyevals
Most cost-aware systems treat a failed cheap-model attempt as a cascade trigger: escalate to the expensive model. This paper points out that in coding, execution feedback can make further cheap-model recovery worthwhile, so it frames the post-failure choice as recovery routing over heterogeneous actions and trains a supervised router from rollouts. A Conformal Risk Control layer retunes the cost penalty at deploy time without retraining. In the GPT-5.4-nano/GPT-5.4 setting, one calibrated point beats always-escalate solve rate at 35% of its mean recovery cost. Code released.

News

2

Show HN: CodeAlmanac – Karpathy-style codebase wiki from your conversations

hn:hn-agent — divitsheth 6/10 memorycode-agents
A local, open-source tool that maintains an almanac/ folder of linked Markdown pages inside your repo, capturing decisions and rationale that never make it into code comments. Every five hours it spins up an agent via the Codex/Claude Code SDK to read new conversations and update the wiki, indexed in SQLite and queryable by CLI; instructions in AGENTS.md/CLAUDE.md point future sessions at it first. The mechanism is concrete and the failure mode it targets — stale hand-written MANUAL.md files — is real.

Claude Is Not a Compiler

hn:hn-claude — bryanmikaelian 6/10 code-agents
A practitioner argument that treating a coding agent as a deterministic build step is a category error: same prompt, different output, no reproducibility guarantee. The framing is useful for anyone tempted to wire an agent into CI as if it were a compiler pass. The raw text here is just the HN header (151 points, 159 comments), so the specific arguments and any proposed mitigations are behind the link — read it for the mental model, not a checklist.

Blogs

1

A Fireside Chat with Cat and Thariq from the Claude Code team

rss:simonw 8/10 code-agentsevalstool-use
An edited transcript of a fireside chat with two Claude Code engineers, dense with dogfooding detail. Their Slack integration now lands 65% of the team's product-engineering PRs; features ship internally first and survive only if they show retention in that cohort; critical changes stay human-reviewed while outer layers lean on automated review. The prompt-engineering notes are the sharpest bit: they cut the system prompt 80% and report that examples and "don't do X" negative lists now hurt on their latest models.
32 more items the ranker flagged but didn't feature

Papers

News

Blogs