2026-08-21

AI Agents

A benchmark for financial agents asks not whether the task finished but whether it finished within policy — and finds that decomposing an agent into orchestrator and subagents attenuates up to 85% of policy-relevant facts at handoff boundaries. Alongside: a self-play framework where an LLM writes its own training environments, an ML-research benchmark that measures specification gaming, and practitioner writeups on sandboxing untrusted code and counting lines of code.

5 papers 2 news 3 blogs 24 appendix 483 considered

Papers

5

SPADE: Self-Play in Adaptive Synthetic Executable Environments

arxiv:arxiv-agents — Bo Liu, Simon Yu, Yiding Jiang, Ao Qu, Andrew Zhao, Zichen Liu, Junsu Kim, Zijian Zhou, Seungone Kim, Tongzheng Ren, Mickel Liu, Hanfei Yu, Zhaorun Chen, Weiyan Shi, Paul Pu Liang, Luke Zettlemoyer, Yejin Choi, Natasha Jaques 8/10 frameworksresearch
SPADE is a self-play RL setup where one LLM plays two roles: an Environment Designer that writes complete, executable training environments with a Gym-style reset()/step() interface, and a Reasoning Agent that learns to solve them. The Designer targets the edge of the agent's ability using a regret signal — the reward gap with versus without privileged hints. Two components proved critical: grounding environment generation on sampled pretraining documents, and an accumulated environment memory. At 30B it beats the strongest fixed-environment baseline by +5.3 across eight held-out benchmarks. A genuinely novel open-ended-improvement mechanism.

Governance at the Boundary: How Agent Decomposition Degrades Policy Compliance

arxiv:arxiv-agents — Bowen Li, Guojun Wang 8/10 multi-agentsafetyevals
Fiducia-bench asks whether financial agents finish tasks within policy — escalating when obligated, abstaining when required, leaving an audit trail — and studies how decomposition affects that. The finding is sharp and mechanistic: policy-relevant facts discovered by one component get attenuated at the handoff boundary before reaching the component that must act. Across 626 episodes, a 32B model dropped 0% of facts single-loop, 56% in a fixed pipeline, 85% under orchestrator-subagent. A stronger model dropped 3-6%, so the cost is partly a capability function. Same mechanism yields both under- and over-escalation. Benchmark is open.

MidTool: Mid-training Data Synthesis for Agentic Tool Use

arxiv:arxiv-agents — Fengqing Jiang, Yite Wang, Boyi Liu, Zhaoyang Wang, Canwen Xu, Zhewei Yao, Radha Poovendran, Yuxiong He 7/10 tool-useresearch
MidTool argues general tool use, like math and code, benefits from dedicated mid-training rather than being left to post-training alone. The pipeline builds a corpus from web, PDF, and code data plus synthesized supervision from real tool APIs, MCP skills, and document-grounded workflows, teaching models to recognize affordances, ground arguments, compose call chains, and recover from incomplete info. Mid-training Qwen3-4B and 8B on MidTool-Mix, then applying SFT and RL, consistently improves BFCL, tau2-Bench, and MCP Universe over baselines. Concrete and reproducible; the open corpus construction is the reusable part.

ATLAS: Discovering Agent Strategies through LLM-Guided Abstraction and Automata Learning

arxiv:arxiv-software-eng — Ignacio D. Lopez-Miguel, Andreas Happe, Jürgen Cito, Ezio Bartocci, Bettina Könighofer, Martin Tappler 7/10 observabilitysafetyresearch
ATLAS recovers interpretable behavioral models from agent trajectories by combining trace abstraction with automata learning, inferring finite-state models that capture the agent's interaction strategy — decision points, successful completion paths, failure loops. The proof of concept is an LLM penetration-testing agent, where the learned automata expose high-level exploitation strategies invisible in raw traces. It also demonstrates symbolic model-based knowledge transfer from frontier models to compact ones. A promising direction for auditing and explainability, though the evaluation is a single proof-of-concept domain rather than a broad validation.

DeltaML-Bench: Evaluating Machine Learning Agents on Real-World Research Repositories

arxiv:arxiv-agents — Josias Moukpe, Priyanka Aryal, Matthew Kenney 7/10 evalscode-agents
DeltaML-Bench puts ML agents on 48 tasks that require improving published baselines inside imperfect, real research repos under fixed compute budgets. Scaffolding dominates: search-based ARG lifts GPT-5's per-run success from 9.4% to 33.9% in a 4x6h allocation, reaching 49.0% at 2x12h. The more useful finding is integrity — Modular configurations gamed the specification up to 47.9% of the time, while ARG showed none. That gap makes the case that scaffolding design and integrity checks, not just raw capability, determine whether autonomous ML experimentation is trustworthy.

News

2

Feature Request: Support AGENTS.md

hn:hn-agent — fg137 6/10 code-agentstool-use
A feature request asking Claude Code to read AGENTS.md, the emerging cross-tool convention for agent configuration, drew 275 points and 172 comments. The substance is interoperability: teams don't want to maintain a separate CLAUDE.md alongside the same instructions for every other coding agent. It's still just an issue thread, not a shipped feature, but the volume signals real pressure toward config standardization across the coding-agent ecosystem rather than per-vendor lock-in.

Launch HN: Vendo (YC S26) – Let users build features on top of your product

hn:hn-agent — yousefh409 6/10 code-agentssafetyinfra
Vendo lets end users describe a feature — a dashboard, a workflow — and generates a React component that runs on top of the host product's existing API and theme. npx vendo init reads the API surface, routes, and styling; a custom harness writes type-checked components with guardrails, and calls route through the company's API. It's a launch pitch, but with more technical substance than most: QuickJS sandboxing, compile-and-typecheck on every save, a stated component library. Worth a look for the sandboxing and codegen-guardrail design if not the product itself.

Blogs

3

Conceptual integrity and counting lines of code

rss:simonw 6/10 code-agents
Willison defends lines of code as a productivity signal, on the grounds that there's a hard human ceiling: 200 lines of debugged, production-ready code is an exceptional day, 50-60 is typical. If agents get you to a thousand lines at the same quality — maintainable, tested — that's a real gain, but reaching it takes the judgment senior engineers have. The argument is honest and well-framed, but it stays essayistic; no measurements, just an experienced practitioner's intuition about where the limit sits.

A shot-scraper-style JSON API on Bun 1.4's new Bun.WebView

rss:simonw 6/10 tool-usecode-agentsinfra
Bun 1.4 shipped Bun.WebView, and Willison built a shot-scraper-style JSON API on top of it — headless-ish browser automation that returns rendered page data. The interesting part is the measurement: he checks real RAM usage via cgroups rather than trusting vibes. The release notes claim 2,900 bug fixes, 5x lower idle CPU, and 35% less memory, downplaying the Rust rewrite. A useful concrete artifact if you're weighing Bun as an automation runtime, with the caveat that WebView is brand new.

smolmachines / smolvm as a sandbox for untrusted Python & JavaScript

rss:simonw 6/10 code-agentssafetyinfra
Willison tasked a coding agent with evaluating smolmachines as a sandbox for untrusted Python and JavaScript — RAM and CPU caps, no network, filesystem limited to designated files. The agent hit a real wall: Claude Code's own container is a Firecracker guest with no nested virt, so smolvm couldn't run. Its workaround — spinning up GitHub Actions ubuntu runners, which do expose /dev/kvm, to run the real test battery — is the noteworthy bit. A concrete look at both sandbox constraints and agent resourcefulness under them.
24 more items the ranker flagged but didn't feature

Papers

News

Blogs