2026-10-04

AI Agents

A new paper tests whether an agent can improve just by writing explanations of its own attempts — no reward signal, no verifier — and matches GRPO on SWE-bench in half the updates. A security study finds that routing a harmful request through a tool call raises the model's effective refusal threshold, making tool-mediated agents easier to push into bad behavior. Also today: serving-infra work on live attention-layout switching and KV cache reuse, a memory-versus-documentation debate, and the case for hard budget caps.

5 papers 2 news 1 blogs 17 appendix 497 considered

Papers

5

SPLASH: Switching Parallel Layouts of Attention with Seamless Handoff for LLM Serving

arxiv:arxiv-agents — Chuan Liu, Shuoming Zhang, Zhicheng Li, Qianqi Sun, Ruiyuan Xu, Qiuchu Yu, Xiyu Shi, Huimin Cui, Jiacheng Zhao 8/10 infracost-latency
SPLASH switches the attention parallelism layout — tensor, data, or context — while requests are still running, instead of fixing it at launch. The insight is that modern attention with few KV heads decouples where the KV cache lives from how weights are sharded, so most state already sits where the next layout needs it; the handoff happens at a batch boundary with under 0.51% median overhead. It also introduces Decoupled Ownership Parallelism, claiming 27-60% more KV capacity and 1.3-1.73x throughput.

Tool Mediation Alters Refusal Mechanisms in Large Language Models

arxiv:arxiv-agents — Abel Rodríguez, Giuseppe Garofalo, Lieven Desmet, Vera Rimmer 8/10 safetytool-use
A representation-level study finds that routing a harmful request through a tool call doesn't make models perceive it as less harmful — the harm signal stays encoded and transfers across modes — but it raises the effective threshold at which that perception converts into a refusal. Tool-mediated refusal is also more brittle: weakening the refusal computation breaks it at lower intervention strength than conversational refusal, with benign capability intact. Across several open-weight models, the mechanism analysis is the contribution, not just the behavioral gap.

Counterfactual Rollout Replay: Forkable Environments as Free Process Rewards for Software Engineering Agents

arxiv:arxiv-software-eng — Yuanhao Li, Hongbo Wang, Xuhong Chen, Yiming Cao, Xunzhu Tang 8/10 code-agentsresearch
CRR extracts step-level process rewards from outcome-only RL by exploiting forkable executable environments: at selected decision points it restores state, samples an alternative action, rolls the branch forward under the policy, and replaces the step's advantage with the terminal-return difference. No human process labels, no learned reward model. With a 14B policy it lifts pass@1 on SWE-bench Verified, Live, and rebench, and at equal wall-clock beats extended outcome-only GRPO 41.7% versus 36.7% — fork overhead included. Honest about the limit: it needs cheap, reliable state restoration.

ATTUNER: Recomputation-Free KV Cache Reuse via Query-Side Adaptation

arxiv:arxiv-llm-systems — Xinghao Chen, Junnan Dong, Cai Ke, Chak Tou Leong, Haocheng Sun, Keyu Chen, Siyu An, Ruizhi Qiao, Xing Sun, Wenjie Li, Xiaoyu Shen 8/10 memoryinfracost-latency
ATTUNER isolates why position-independent caching degrades quality: not the positional mismatch, and not the KV states themselves, which stay faithful — the failure is in attention when the model must select among multiple cached artifacts. Swapping in full-prefill attention scores recovers performance with the cache unchanged. The fix is low-rank adapters on the query projections, trained by distilling full-prefill distributions, touching under 0.05% of parameters and needing no recomputation at inference. On Qwen3-4B/8B across seven skill, document, and memory benchmarks, 3.73x speedup.

Shockingly Simple Self-retrospection Improves Agentic Models Without RL

arxiv:arxiv-agents — Jonathan Light, Christopher Zhang Cui, Jeonghye Kim, Roger Creus Castanyer, Emiliano Penaloza, Zhengyan Shi, Alessandro Sordoni, Marc-Alexandre Côté, Xingdi Yuan, Minseon Kim 8/10 code-agentsresearch
ROFT asks whether an agent can improve by training only on explanations of its own experience. The loop: attempt a task, observe feedback, write a retrospective, fine-tune with next-token loss on the explanation tokens alone — no teacher, no reward update, no verifier. With Qwen3.5-4B it reaches 49.2% and 26.8% on held-out SWE-bench Verified and Pro after 20 updates, versus GRPO's 48.0% and 25.3% after 40. Strikingly, it solves tasks where all 64 base attempts failed, and behavioral analysis shows it indirectly assigns credit to actions.

News

2

Agents don't need memory, they need documentation

hn:hn-agent — kmeh 7/10 memoryframeworks
The claim: most of what gets built as agent "memory" is better served by documentation. Instead of a vector store accumulating opaque state, write durable, retrievable docs the agent reads like any other reader — versioned, inspectable, correctable. The pitch resonates with anyone who has debugged memory-induced drift and found no way to audit what the agent thinks it knows. It's an opinion piece, not an eval, and the line between "documentation" and "a well-designed memory store" blurs under pressure.

Getting the most out of Opus 5.5 in Claude and Claude Code

hn:hn-claude — saikatsg 6/10 code-agentsframeworks
Vendor guidance accompanying the Opus 5.5 release, aimed at Claude Code and Claude users: how to prompt, when to lean on extended reasoning, and workflow patterns for the new model. Useful if you operate Claude Code in anger, but it's first-party advice with the usual incentives — treat the usage recommendations as a starting hypothesis, not measured results. No independent benchmarks here, and the HN thread will likely surface more honest tradeoffs than the post itself.

Blogs

1

We're going to need default hard budget caps on pretty much everything

rss:simonw 6/10 cost-latencysafety
Willison argues that pay-per-use APIs and hosted platforms need hard budget caps as the default — spend $X, return errors, full stop — not soft warning emails. The motivating case is coding and personal agents that cheaply spin up services calling paid APIs or billable compute, where a runaway loop overnight becomes a four-figure surprise. He grants businesses may prefer uptime, but makes opt-out dangerous mode an explicit checkbox. It's a product-policy argument, light on how providers would actually enforce it.
17 more items the ranker flagged but didn't feature

Papers

News