AI Agents

A two-stage empirical study of the generate-run-revise loop in LLM program repair, built on 7,745 SWE-bench leaderboard traces plus 3,000 controlled repair attempts across Claude Code, Codex, and OpenCode under four execution paradigms. Agents average 8.8 test runs per task (range 2-19), with late-stage executions succeeding more often. The headline: prohibiting execution entirely costs only 1.25pp of resolve-rate on SOTA models—not statistically significant—while saving substantial token and wall-clock cost.

placeholder

A clean theoretical result with empirical teeth: any multi-model policy that returns one member's answer is capped at 1-beta accuracy, where beta is the rate every model fails the same query. Crucially, the usual pairwise error-correlation rho cannot identify beta. Across 67 models from 21 providers, a calibrated single-factor model underprices the all-wrong tail ~2.5x (observed beta 0.052 vs 0.023 on open math). On checkable code tasks, combining models rarely beat the single best model.

This asks what CoT training actually improves in agents, given that verbalized CoT is often post-hoc. By comparing prompt actions (no CoT) against CoT actions across checkpoints, the authors find prompt-action quality improves substantially while CoT's relative advantage stays flat—and later checkpoints revise their action in response to CoT less often. So training shifts capability into the prompt-conditioned policy, not the reasoning. Selectively masking action-token supervision on a fraction of examples then improves out-of-domain generalization.

The first empirical audit of ERC-8004, a permissionless trust layer for AI agent economies built on on-chain Identity, Reputation, and Validation registries. Crawling Ethereum, BSC, and Base through May 2026, the authors find most registrations are placeholders—only 3%, 4%, and 15% expose a valid registration file with a live endpoint. The reputation registry fails as a trust signal: values aren't commensurable, feedback is rarely grounded in verifiable interactions, and reputation is manipulable at minimal cost, with 73.6-90.6% of reviewers suspect.

Raschka walks through running open-weight models inside local coding harnesses as a substitute for Claude Code and Codex subscriptions. The value here is the hands-on framing: which harnesses accept which models, what the setup actually involves, and where local inference holds up against hosted frontier coding agents. For anyone weighing the cost and data-control tradeoffs of taking coding agents in-house, this is a concrete starting point rather than a benchmark table.