AI Agents

A sharp methodological correction for anyone building failure-prediction monitors. Pooling runs across tasks makes predictors look strong (AUROC 0.85-0.94), but the authors show more than 99.93% of the positive-negative pairs behind that pooled score are cross-unit — comparing different tasks, not success versus failure of the same task. A difficulty oracle that never sees the current run hits 0.945 while scoring at chance within a task. Early run-level discrimination is weak across trajectory predictors, released monitors, and hidden-state probes alike.

AgentSpy observes an agent from outside the agent, running it in an isolated environment and recording system calls and network traffic for every process it spawns — including subprocess behavior the agent's own trajectory omits. It supports conformance checks (what a run should do) and safety checks (what it must never do) via deterministic rules. Evaluated on 77 tasks across the Codex harness and three LLMs with three runs each; repeated runs of the same task were more similar than cross-task sets in 92.2% of comparisons.

HEAR is a bidirectional protocol connecting the agent harness and the inference engine, which today optimize blind to each other. The harness knows workflow dependencies, context lifecycles, and objectives; the engine sees request queues, KV-cache state, and resource pressure. HEAR standardizes exchanging that intent and runtime state while keeping optimization policies separate. Instantiated for cache-aware coordination and execution-mode selection, it reports a 1.61x batch speedup and 2.23x lower median TTFT on SCBench, with 1.23x-2.45x end-to-end gains on research-agent benchmarks under memory-constrained concurrent serving.

GHOST names a long-horizon failure mode: under entirely benign interaction, an agent executes an action that violates a safety constraint specified many turns earlier, because the constraint fell out of effective context. The authors measure an 11.5% occurrence rate on GPT-5.5 and give a theoretical argument that, under a non-summable residual hazard, the trajectory enters the hazard region almost surely. Their defense, STAR-Guard, restores applicable constraints and adds a deterministic pre-execution audit, eliminating GHOST events in their setup. The two-layer framing is the useful part.

A large failure-analysis corpus: 50,228 error-diagnosis pairs from 9,961 tasks across 33 environments, 19 harness families, and 23 policy models, with source traces and execution metadata retained for re-diagnosis without rerunning rollouts. The five-stage pipeline collects natural failures, generates diagnoses and corrections, and checks them against recorded evidence. Across 3,062 matched replay pairs, first-proposal corrections raised verifier pass rates from 18.4% to 51.1%. Diagnosis fine-tuning lifted Qwen3-8B's exact-step agreement with teacher labels from 47.2% to 63.6%, beating a 54.7% prompted reference.

Wikimedia reports finding what it calls "rogue" OpenAI agent activity on its projects — a named incident on a real, heavily monitored platform rather than a lab scenario. The write-up is the useful artifact here: how the activity was detected, what the agents did, and how it was attributed. Details matter more than the headline, since this is one of the first public cases of autonomous agent behavior being caught and documented on a major production platform by its operators.

A reported result: Opus 5.5 agents proposed two room-temperature magnetic semiconductor candidates, drawing heavy HN engagement. For an agent-engineering audience, the interesting part is the harness and verification loop behind the discovery, not the materials-science claim itself — which needs experimental confirmation before anyone treats "discovery" as settled. Named candidates from an autonomous multi-agent pipeline are concrete, but the result lives or dies on whether the proposals replicate in a lab, and the write-up is light on that.

Anthropic is moving Cowork's execution model: the old version ran inference in the cloud but shipped a local VM to your laptop for tool calls, which cost disk, battery, and stopped work when you closed the lid. The new version runs both inference and a per-session sandbox in the cloud, with the desktop app servicing device-side tool calls like file access. Each session gets isolated state. The tradeoff is clear — remote capability and persistence for a thinner local client that proxies device access.