AI Agents

The reported tool-call rate is a property of your serving interface, not your model. Holding weights, cases, decoding, and seeds fixed and changing only the serving adapter, the same model scores 0.00 or 0.96 on BFCL v4; a 2x2 over chat template and parser localizes the entire effect to their interaction. On tau-bench the same swap moves server-parsed calls from 0 to 636. The failure reaches into training: verl's AgentLoop at 7B accepts 0 of 45 complete calls. Ships a preflight probe. Essential reading.

Evaluating LLM-authored config edits on real Kubernetes manifests, the paper finds no text-generation strategy safe for unattended GitOps. Unified diffs almost never apply under strict patching; a tolerant tool applies 96% but silently misapplies 1 in 7 with no error. Full-file rewrite is capability-dependent and non-deterministic even for frontier models, at O(file size) cost. Their fix separates the semantic decision (kind, name, field, value) from editing: the agent emits a structured intent, and a deterministic pipeline replaces exactly the target character span. Minimal diff by construction.

Agents exploring parallel candidate states over relational data need isolation, reproducibility, and audit through the same SQL interface — which source control can't scale to and databases rarely expose natively. Git4Data treats a database as a repo and a table as a versioned object, adding snapshot, branch, diff, and merge (with explicit conflict policies) as SQL extensions. Built on MatrixOne's immutable object storage and MVCC, operation cost scales with change size, not data size. On BranchBench it beats DoltDB by up to an order of magnitude.

Multi-turn agentic RL treats credit assignment as targeting — localizing credit onto the turns that mattered. This paper argues targeting is second-order and coverage is what matters, formalized via verifier information density V_d. Terminal-state verifiers sit deep in low-V_d territory (k=1 in 98% of rollouts, while success needs 5-8 prerequisite steps). On tau²-bench, a uniform dense reward beats sparse binary outcome, while concentrating the same advantage on progress turns or random turns is equally harmful. Pre-registered replications across model families. Genuinely surprising, carefully controlled.

Spawning more agents doesn't add evidence — it can multiply the same evidence while looking like corroboration. The paper formalizes this as an epistemic Sybil problem: a report adds nothing when its mutual information with existing reports, given the parameter, is zero, and no report-only aggregator can generally tell replication from independent corroboration. Across 20,000+ controlled calls, holding one evidence root fixed while report count rises from 1 to 32 collapses posterior coverage from 0.940 to 0.263. A correlated-extraction aggregator restores calibration. Rigorous and practically important.

Measuring 17k runs of what Claude, Codex, and Cursor install and invoke gives you real distributions instead of anecdotes about agent tool preference. This is the kind of data you can't get from reading a single agent's transcript — it tells you what tooling agents actually converge on across providers, which matters when you're deciding what to preinstall in a sandbox or optimize a tool surface around. Worth reading alongside the grep-versus-LSP finding below.

The claim: coding agents ignore your LSP integration and fall back to grep, because grep's output is legible and composable while structured LSP responses cost more tokens to interpret for marginal benefit. If true, it's a direct rebuke to teams building elaborate semantic tooling for agents — the agent optimizes for what it can cheaply reason over, not for what's technically richer. The HN thread is where the nuance lives; the post itself is a practitioner argument rather than a controlled study.

An Ask HN probing who runs MCP in production, and what it buys over a plain API, direct tool integration, or a CLI. The framing itself is the signal: nearly a year in, someone following MCP closely still struggles to find production deployments. The comment thread is the value here — real accounts of what MCP's indirection actually earns versus the simpler integrations it competes with, from people who've shipped it rather than demoed it.

OpenAI's Fable competitor lands at Claude's pricing ($10/M input, $50/M output) and claims higher self-reported benchmarks. The headline 99.9% on ARC-AGI 3 deserves scrutiny: it required OpenAI's custom Provider Adapter harness (which preserves opaque reasoning state and compacts across requests) at $19K, while the default ARC harness scored 62.7% for $26K. Fable has no published ARC-AGI 3 result yet, so the comparison is one-sided. Security numbers — 100% on ExploitBench versus 78.5% — are more striking if they hold.