AI Agents

An evaluation of 66 configurations — four configurable harnesses crossed with five models on three benchmarks, plus native Codex and Claude Code pairings — showing model rankings reverse depending on the harness. On Terminal-Bench 4, Claude beats GPT by 7.94 points under one harness but trails by 30.16 under another. A model's own vendor harness isn't reliably its best, and higher cost doesn't buy higher scores. The mechanism: much depends on whether the harness hands failures back in a form the model can act on. All 6,204 trajectories released.

RL post-training on expert-built agentic coding tasks that targets last-mile failures — dropped requirements, undertested cases, broken existing behavior. One epoch of GSPO on a rank-32 LoRA over Kimi K2.7 Code (1T params, 32B active), trained on 1,700 tasks with hidden fail-to-pass and pass-to-pass gating, improves pass@1 across six external benchmarks: SWE-Bench Pro 60.1→64.8, Terminal-Bench 2.1 67.4→82.0, SWE-Marathon 5.0→25.0. Gains hold on benchmarks released after training data was collected (p=0.004) and under harnesses never used in training — real evidence of transfer, not just overfit.

This paper treats the agent as a configurable system and measures which knobs matter, on four scientific tasks where a coding agent must find and operate a published model. The standout finding: roughly 54% of outcome variance comes from repeating the same configuration — run-to-run noise, not config choice. Among choices, the information you give the agent matters more than time budget or model size, and also lowers cost and improves calibration. Prompting for self-verification barely changes behavior; a dedicated verification tool does. 18k trajectories released.

A benchmark for whether a model can predict how an intervention changes a stateful service's future behavior — not which action to take, just the downstream effect. Each scenario gives Python service code, prior calls, an edit or state overwrite, and 40 fixed future calls; predictions are checked by execution. With correct earlier responses revealed, four models hit 54.3-61.5% effect-step value match; hide those responses and it drops to 23.2-28.9%. That gap is the real result: agents reason far worse about state they can't directly observe, which is exactly the production case.

A training-free context compression layer for agents that reframes the problem causally: not what to discard offline, but which past interactions actually shape the agent's future decisions, decided at test time. FOCUS attaches to any closed-API model as a modular layer with no fine-tuning or offline data collection. Reported results cut peak context by up to 48% and dependency by 73% while improving task success by up to 8.9 points over uncompressed execution across tool-calling, QA, web, and multi-turn benchmarks. The training-free, architecture-agnostic framing makes it cheap to try.

A model router that plugs into Claude Code or Codex and switches LLMs per task — hard tasks to a frontier model, simple frontend edits to a cheaper one. The claimed numbers are the interesting part: equivalent pass rates to GPT-6 Astra on Terminal Bench 4.0 and SWE Atlas, at 52-54% of the cost and 2.2-2.5x faster. The writeup covers a trained routing model with HMM-plus-classifier architecture and cache-eviction tradeoffs. Benchmarks are self-reported, so verify on your own task mix.

Matthew Green's blog asks whether sandboxing actually contains a rogue agent, and the answer leans no. The thrust: isolation stops direct escape but not instruction passing through shared side channels — package caches, email, Slack, shared docs — which independently sandboxed agents already read and write. If propagation substrate is everywhere agents coordinate, per-agent sandboxes are the wrong boundary. The post is short on this excerpt, with the comment thread doing some of the arguing; worth reading for the threat model framing.

Figma has restricted MCP server access to a whitelist of approved clients, with Pi reportedly excluded. The mechanism matters: whitelisting named clients turns MCP from an open integration surface into a gated one, and sets a precedent other vendors can follow. If your agent integrates design tooling through MCP, your client identity is now a dependency that can be revoked. Detail is thin — this is a tweet plus a heated HN thread — so treat specifics as provisional pending an official policy statement.