AI Agents

This paper introduces ClawArena-Team, a benchmark designed to isolate an LLM's ability to manage a team of subagents. The main agent is deliberately constrained, perceiving only text and accessing a limited part of the workspace, and must delegate tasks to a fixed pool of subagents. The surprising result across twelve models is that the key bottleneck is not perception or task decomposition, but the manager's inability to grant the correct workspace permissions to subagents, with precision never exceeding 50%.

The authors argue that standard GRPO's uniform credit assignment is flawed, as it can punish useful exploration in failed runs and reward redundant actions in successful ones. They propose TRIAGE, a framework that adds role-typed credit. A structured judge classifies action segments as 'decisive progress', 'useful exploration', etc., and applies fixed, segment-level process rewards based on these roles. This corrects GRPO's blind spots while keeping verifier outcomes as the main optimization signal, showing consistent gains on ALFWorld, Search-QA, and WebShop.

Evaluating dense supervision signals for long-horizon agents is expensive, usually requiring a full training run. This paper introduces QVal, a training-free testbed to directly evaluate these signals by measuring how well they align with the Q-values of a strong reference policy (Q-aligned). Benchmarking 21 methods across four environments, the authors find a surprising negative result: simple prompting baselines for generating supervision often outperform more recent, complex methods. This suggests many recent techniques may not be generating qualitatively better signals.

This paper reframes world-model collapse in long-horizon agents as a measurable phase transition, like water boiling. A large grid search reveals a phase diagram with a narrow critical boundary where a small change in task complexity causes sudden failure. Tracing the failure mechanism, they find that the agent's internal world-state fidelity collapses before it starts choosing invalid actions. The agent isn't just picking a bad action; it's acting from a corrupted world model. Stronger models shift the boundary but don't eliminate the transition.

Existing context management techniques like truncation are irreversible—once information is dropped, it's gone. ACE (Adaptive Context Elasticizer) is a plug-and-play module that addresses this by maintaining a lossless history of both raw messages and compressed summaries. At each step, its orchestration layer decides whether to include each historical step as raw, as a summary, or not at all. This reversible design provides a compact context without permanent information loss. The authors show it boosts performance across four agent frameworks without retraining.

Simon Willison releases shot-scraper video, a new tool to have agents record video demos of their work. You define a routine in a storyboard.yml file, and the tool uses Playwright to execute and record it. Willison argues this is a key step in "proving code actually works" for coding agents. The post includes a sample video of an agent using a new Datasette feature and the simple command-line invocation to generate it. It's a concrete, actionable pattern for improving agent observability and verification.

Simon Willison unpacks the developer-relevant changes for Claude Sonnet 5, which Anthropic positions as near Opus 4.8 performance at a lower price. The actionable details are in the API changes: temperature, top_p, and top_k are no longer supported. The context window is 1M tokens with 128k max output, and adaptive thinking is on by default. Willison notes that while the token price is the same as 4.6, a new tokenizer means more tokens for the same text, effectively raising the price for some users.