AI Agents

This paper introduces probe-and-refine tuning, a procedure for automatically improving repository guidance files (AGENTS.md) for coding agents. Instead of manual tuning, the system uses synthetic bug-fix probes to diagnose weaknesses in the guidance and then uses single-shot LLM calls to patch it. On SWE-bench with Qwen3.5-35B-A3B, this method lifted resolve rates from 25.5% (unguided) to 33.0%. The key insight is that better guidance primarily helps the agent find the right files, rather than writing better code once there.

MemTrace is a new benchmark for long-term agent memory that argues aggregate accuracy is a misleading metric. Instead of scoring questions, it tracks “knowledge points”—individual facts about the user—across sessions. It probes each fact's recall based on age and context. The paper's most critical finding: in failure cases, the necessary evidence was successfully retrieved 10 times more often than it was missing. This suggests the bottleneck isn't retrieval, but the agent's ability to properly use the evidence it finds.

CEO-Bench moves beyond short-horizon tasks to evaluate if agents can “play the long game.” The benchmark simulates running a startup for 500 days, requiring the agent to manage pricing, marketing, and budgeting through a Python API. Success demands analyzing noisy data, forming a coherent strategy, and adapting to a changing environment. The results are humbling: even state-of-the-art models like Claude Opus 4.8 and GPT-5.5 struggle to turn a profit, showing a deep gap in long-horizon strategic reasoning.

This paper proposes the Sovereign Execution Broker (SEB), a critical architectural pattern for safely giving agents mutation authority over production infrastructure. It addresses the danger of housing authority inside a non-deterministic agent. The SEB acts as a runtime enforcement boundary, consuming a time-bound, action-specific certificate from an assurance layer. It verifies the request against the certificate and live system state before minting a short-lived, scoped identity to perform the action. This turns authority into a revocable, auditable, and verifiable capability.

LegalHalluLens argues that a single hallucination rate (e.g., ~52%) is useless for compliance. The framework introduces typed hallucination profiles (numeric, temporal, etc.) and a “Risk Direction Index” to distinguish between errors of omission and invention. The key contribution is closing the loop: these fine-grained diagnostics are then used to calibrate a multi-agent debate pipeline. By targeting the specifically diagnosed failure modes, the debate system reduced fabricated detections by 45%, showing the power of moving from generic to targeted mitigation.

Phoenix presents a concrete, safety-focused multi-agent system for resolving GitHub issues automatically. The architecture uses six specialized agents (Planner, Coder, Tester, etc.) coordinated via a GitHub webhook-driven state machine. The system is notable for its seven layered safety controls and a test strategy that verifies changes against a baseline run before opening a pull request. A pilot on 42 real-world issues demonstrated 100% correctness preservation, though the authors note localization limitations they are actively addressing.

This paper argues that to understand why agent-generated pull requests fail, we must look before the PR. It introduces grite, an open-source coordination substrate that logs agent actions (like claiming a file) into a shared, signed, append-only log stored directly in the git repository. This provides an observable record of coordination that is otherwise lost. On a test workload, grite eliminated redundant work and surfaced failure modes like conflicting edits and lock starvation that are invisible from PR history alone.

OpenRath tackles the fragmented runtime state in agent systems where transcripts, tool calls, and memory events are logged separately. It proposes a programming model centered on Session, a first-class, PyTorch-like runtime abstraction. The Session object carries all runtime state—transcripts, sandbox placement, lineage, token usage—making operations like branching and replaying explicit programming constructs. This provides a unified handle for inspecting and manipulating an agent's execution state, improving reproducibility and debuggability.

Instead of agents making chatty, sequential API calls, this paper proposes they should generate an entire tool program and send it for execution. The ToolPro system allows an agent to express a complex workflow with loops and conditionals as a single, executable program. This program is then run server-side, drastically reducing round-trips. On real-world workflows, this approach cut end-to-end latency by up to 53.4% and client-side traffic by 96.1%, turning a high-latency conversation into a single, efficient request.

This work addresses the practical problem of code generation for “no-resource” languages, such as proprietary DSLs, which are absent from large training corpora. The authors create new benchmarks and find a common pitfall: while continued pre-training on limited available code improves domain knowledge, it can catastrophically degrade the instruction-following capabilities of the base model. The paper then proposes and validates a technique to gain the benefits of pre-training without this harmful side effect, a critical lesson for in-house model specialization.

Simon Willison spotlights a key insight from Sean Lynch on the Model-Context Protocol (MCP). The argument is that MCP's most valuable feature isn't just standardizing tool use, but isolating authentication flows entirely outside the agent's context window and harness. This reframes MCP as, potentially, an auth gateway for agent APIs, providing a critical security boundary by design. The agent never sees the credentials, only the capability to call the tool. It's a simple but powerful architectural take.

This guest post by Vishal Singh tackles the well-worn topic of why traditional data stacks fall short for AI applications. Without a detailed abstract, the piece likely covers the usual suspects: the need to handle unstructured data, vector embeddings, real-time feature serving, and the different query patterns required by LLMs versus traditional BI. It serves as a high-level architectural discussion on the shift from batch-oriented data warehousing to AI-native data infrastructure.