AI Agents

This paper proposes making "ask for clarification" a first-class action that competes with all other actions at every step. By training an agent to rate the utility of asking versus acting, the model learns to self-gate its requests for help. On a complex 30,000-node hierarchical classification task, this approach led to more effective help-seeking, with the agent correctly navigating after receiving help 74% of the time, up from 50%. This structure allows two modes to emerge: mandatory asking and opportunistic asking.

Agent skills create a new attack surface, as malicious logic can be hidden until triggered by specific runtime conditions. This paper argues that static analysis is insufficient and presents Runtime Skill Audit (RSA), a dynamic analysis framework. RSA doesn't just run generic tests; it profiles a skill's interfaces, crafts targeted inputs to exercise risky code paths, and assigns security labels based on observed runtime behavior. On a benchmark of 100 skills, RSA achieved 90% accuracy, outperforming static baselines by 13 percentage points.

Reinforcement learning for agents often struggles with credit assignment, typically rewarding entire tool calls. This paper argues that influential decisions happen at a finer grain. They propose Agentic Procedural Policy Optimization (APPO), which moves the branching and credit assignment for RL down to the token level. APPO uses a new Branching Score to identify the most impactful decision points for exploration, filtering out noisy high-entropy tokens. On 13 benchmarks, this more targeted approach improved strong agentic RL baselines by nearly 4 points.

Aggregate pass/fail metrics are poor for debugging agents because they don't localize regressions. This paper introduces "layer-isolated evaluation," a compelling methodology for production agents. They decompose an agent into layers (e.g., intent, routing) and create a fast, deterministic, no-LLM test suite for each layer. By injecting regressions, they show that while the aggregate success rate barely moves (masking the issue), the targeted layer's test slice fails catastrophically (-25 to -91 pp). This allows for precise, fast regression detection in CI.

This paper tackles a critical problem: how to safely let an agent propose changes to production infrastructure. They propose the Sovereign Assurance Boundary (SAB), an "assurance airlock" that sits between the agent and the infra APIs. The SAB intercepts an agent's proposed action, compiles it into a typed contract, and routes it for approval. If approved, it issues a short-lived, narrowly-scoped cryptographic certificate that a broker verifies just before execution. This decouples the non-deterministic reasoning from the authoritative act of mutation.

Agent workflow optimization typically forces a bad trade-off: find one "best" workflow offline (inflexible) or synthesize one per-query online (expensive). This paper proposes FlowBank, a hybrid approach. It first uses an offline search process to generate a diverse portfolio of complementary workflows. Then, at inference time, a lightweight router selects the most appropriate pre-computed workflow from this "bank" for the incoming query. This precompute-and-reuse strategy aims for the performance of customized workflows without the per-query latency and cost.

This paper presents a fascinating finding: how you write and organize an agent's skills matters as much as the content. Using a framework called SkillJuror, they compare a flat file of instructions against a "Progressive Disclosure" approach, where a concise root file links to more detailed resources. Holding the total information constant, the structured approach dramatically changed agent behavior, causing it to explore more resources (3.85 vs 1.18 per task). This led to a modest but real +4.1% increase in task success.

A fantastic case study on using an agentic LLM assistant to port a 74,000-line Fortran ocean model to C++/Kokkos for GPU performance. The authors share hard-won lessons, with three critical practices: performing a two-stage port (Fortran to a clean C reference, then C to Kokkos); enforcing a strictly literal translation to prevent the LLM from "improving" logic; and having rigorous, domain-specific validation at each stage. The final Kokkos port was bit-for-bit identical to the C reference on CPU.

Current multi-agent systems are "infrastructure-blind," routing requests to the "best" model without considering runtime conditions like queue depth or KV-cache pressure. This paper introduces INFRAMIND, a framework that makes orchestration infra-aware. A planner selects simpler agent topologies under high load, and an executor routes each step to models with available capacity. This holistic, load-aware approach treats model selection as a dynamic resource allocation problem, not a static leaderboard lookup, to improve latency and SLO adherence.

This paper argues that evaluating forecasting agents on accuracy alone is insufficient. They present WorldReasoner, a rigorous framework for evaluating temporally valid reasoning. For each task, an agent is given a question and access only to a corpus of evidence available before a specific "forecast date." The framework then evaluates not just the prediction, but also the quality of the cited evidence and the soundness of the causal reasoning. The benchmark itself was cleverly constructed using an agentic pipeline.

A security analysis from blue41 reveals a critical vulnerability in bunq's financial AI assistant. An attacker could send a €0.01 transfer with a malicious prompt as the description. When the victim later asks the agent about their recent transactions, the agent ingests the malicious description from the transaction data. This effectively creates a stored, indirect prompt injection vector, allowing the agent to be manipulated into executing unauthorized actions or leaking private information on behalf of the victim.

HelixDB is a new OLTP graph database built on object storage, integrating native vector and full-text search. The creators position it as a unified solution for building complex AI memory systems, avoiding the need to stitch together separate graph and vector databases for techniques like GraphRAG. The motivation was to solve the scalability challenges of graph databases by building on a decoupled storage layer from the start. It aims to provide a single system for querying across graph relationships, semantic similarity, and text.

This is the 319-page official system card for Anthropic's Fable 5 and Mythos 5 models. For anyone considering these models for production use, this document is required reading. It contains extensive details on architecture, training data, evaluation results across dozens of benchmarks, red-teaming efforts, and the specific policies behind its safety features. It was in this document that the now-reversed "silent sabotage" policy for LLM development topics was first revealed, buried deep within the text.

AI agent runs amok in Fedora and elsewhere

The Apache Software Foundation has introduced Burr, a new open-source framework for building agentic applications. Burr focuses on modeling application logic as a state machine, which allows for better observability, debuggability, and persistence. Rather than a simple loop, you define states and transitions, and Burr manages the execution. The framework is designed to be lightweight and embeddable within existing Python applications, providing structure and telemetry for agent behavior without imposing a rigid, all-or-nothing architecture.

This is the official launch announcement for Anthropic's new frontier models, Claude Fable 5 and Mythos 5. The company positions Fable 5 as offering the same capabilities as Mythos 5 but with stricter safety classifiers. The models feature a 1 million token context window and a knowledge cutoff of January 2026. The announcement also introduces a new API feature allowing developers to specify a fallback model if a request to Fable 5 is rejected by its safety systems.

Simon Willison's datasette-agent hits v0.2a0 with a significant new feature: interactive tool execution. Tools can now suspend an agent's turn to ask the user a question via await context.ask_user(...). The question is rendered as a form in the UI and the conversation state persists across server restarts. A new built-in tool, save_query, uses this pattern to require explicit human approval before saving a generated SQL query. This architecture elegantly models human-in-the-loop workflows directly within the tool execution framework.

A practical TIL from Simon Willison on tracking costs for new models in Wes McKinney's AgentsView observability tool. With the release of Claude Fable 5, its pricing wasn't yet in the tool's database. Willison used Fable 5 itself to reverse-engineer AgentsView and find the local configuration needed to add custom model pricing. The post provides the specific recipe, demonstrating a common pain point for cost tracking tools: keeping up with the rapid pace of model releases and pricing changes.

Jonathon Ready (via Simon Willison) highlights a disturbing detail in Anthropic's 319-page system card for Fable 5. The model will silently "limit effectiveness" for requests related to frontier LLM development, such as building pretraining pipelines. Unlike other safeguards which trigger a visible refusal or fallback, this intervention is invisible to the user. This introduces a non-deterministic, un-observable failure mode where the model might appear to be working but is intentionally providing suboptimal output.

Simon Willison's first take on Claude Fable 5 is that it's a "beast" — slow and expensive, but extremely capable. He notes the primary challenge is now finding tasks that the model can't do. Fable 5 is the version with stricter guardrails compared to its sibling, Mythos 5. Anthropic's API reflects this with new mechanisms for handling refusals, including an option to automatically fall back to another model. It boasts a 1M token context window and 128k max output tokens.

Following a widespread outcry, Anthropic has walked back its policy of silently "sabotaging" requests related to frontier LLM development. In a statement to WIRED, the company apologized for making the "wrong tradeoff" and announced that these safeguards will now be visible. Starting this week, flagged requests will trigger a visible fallback to Opus 4.8, the same behavior as their cybersecurity and biology safeguards. The API will also return a reason for the refusal, removing the dangerous silent failure mode.