AI Agents

A post-training recipe for autonomous ML engineering, built into MLExpert, a 35B MoE agent. The core idea is Windowed Interaction RL: treat the multi-turn decisions between successive experiment results as an execution window and assign it a shared outcome-based reward, sidestepping the credit-assignment mess of long MLE trajectories. Hierarchical agentic SFT transfers tool-use and coding first; asynchronous collection with policy correction lets updates happen before trajectories end. The claim is MLE-Bench gains with generalization — worth checking whether the window reward actually isolates useful supervision.

Instead of reactive retrieval at inference time, this trains a builder agent to proactively reorganize a raw corpus into a hierarchical filesystem of synthetic files and reference links — a workspace tailored for a downstream answering agent. Workspace construction is framed as an MDP and the builder is optimized via RL to maximize answering accuracy. Reported gains are large: 33 points over the strongest retrieval and precomputation baselines on BrowseComp-Plus, 31-point zero-shot transfer on BRIGHT, with upfront construction cost amortized across repeated queries.

A benchmark that drops the usual well-specified-ticket premise: 95 end-to-end delivery tasks across 18 sectors, with scripted stakeholders and modifiable customer systems, scored by how much the agent improves the customer's own success metric and penalized for messages, trial runs, and waiting. Across 13 configurations from seven providers, the best scores 0.56 mean and clears 14 of 95 tasks cleanly. Discovering unstated constraints is the primary bottleneck — but the paper notes information discovery alone doesn't close the gap, which is the more interesting finding.

A clean negative result on scaling automated research: iterating longer, running agents in parallel, and letting them communicate all plateau as agents accumulate history and anchor to their own past attempts, making smaller edits. The fix is to forget — periodically discard search state and every session's candidate history, restart from the best candidate found so far. On four tasks spanning heuristic programming, kernel optimization, and model training, forgetting yields up to 15% over parallel agents and 39% over a single agent on the same budget.

A runtime-verification architecture, AgenticRV, that decouples monitoring formalism from execution boundaries. Adapters emit canonical events; each monitor gets its own projection and abstraction; a decision policy turns normalized verdicts into reactions, including blocking an operation before its effect lands. It distinguishes rejected attempts from completed effects, so a blocked request doesn't leave monitors stuck in permanent violation. The Python implementation supports LangGraph, MCP, and A2A, with a LangGraph experiment combining a temporal monitor and a programmatic report monitor plus an incident-response example — concrete and directly deployable.