AI Agents

The first large-scale measurement of "skill name hallucination": across 15,000 prompts and 12 configurations, agents recommend skills that exist in no registry 36-43% of the time, generating 5,669 distinct fake names. The supply-chain angle is what makes this exploitable—the fake names repeat across prompts and models, so an attacker can harvest them and pre-register malicious packages. Retrieval grounding cut hallucination from 40.8% to 3.2% but hurt usability. A rigorous, directly actionable threat model.

A sharp methodological critique of automatic harness evolution for agents. The authors point out that evolving a harness is itself a search procedure, so it should be compared against plain test-time scaling under matched feedback and inference budgets—and that searching and evaluating on the same benchmark invites overfitting. On Terminal-Bench 2.1 with GPT-5.4 and Claude Opus 4.6, harness evolution doesn't consistently beat simple test-time scaling and generalizes poorly to held-out tasks. A useful reset for how these gains get reported.

A carefully controlled null result: across 18 runs varying learning rate, KL weight, seeds, and initialization, GRPO does not improve a strong supervised 4-8B web agent on tasks it has already largely mastered, and moderate-to-high learning rates make the text track credibly worse. Crucially, the same harness shows a 22-point gain on tasks with reachable reward—so GRPO helps only where the sampled policy already beats greedy. The mechanistic double-dissociation analysis distinguishes a genuine null from a broken pipeline.

An empirical study of what makes a bug report "agent-ready." Using 441 bug reports from SWE-bench Verified, annotated sentence-by-sentence by information type and run through mini-swe-agent on GPT-5-mini, MiniMax M2.5, and Gemini 3 Flash, a binomial regression estimates each information type's contribution to repair success. The finding: localization cues that narrow the search space help most, while reproduction steps matter less than intuition suggests. Directly useful if you're feeding issue trackers into repair agents.

A benchmark for whether agents honor the logical structure inside skill files—preconditions that gate actions, constraints on how they run, fallbacks for recovery. Scanning 5,000+ public skills, 70% contain at least one such relation; the 86-case SLBench shows Codex and Claude Code violating them at rates up to 70%, causing privacy leaks and unsafe config changes. A lightweight inference-time scaffold, SLGuard, cuts violations by 63%. It names logical-relation following as a distinct reliability problem from raw capability.

An RL-trained agent that itself runs RL training loops on models, built for roughly $1.3k with published code. The framing is recursive self-improvement, but the concrete artifact is more interesting than the slogan: an agent orchestrating the search over training configurations rather than a human. HN interest is moderate (104 points). Treat the cost figure as a headline, not a benchmark—the value is a reproducible experiment you can inspect, not a validated claim about compounding gains.

A writeup of an attack that coaxes Claude's memory feature into surfacing personal details it stored across sessions—a concrete demonstration that persistent memory is an exfiltration surface, not just a convenience. Strong HN attention (349 points), though this is a practitioner blog, so read the exact mechanism from the source rather than the framing. The general lesson stands: anything an agent remembers about a user is something an adversary can try to elicit through the model itself.

Codex has started encrypting the prompts it sends to sub-agents, which drew 419 points and a long argument on HN. The security rationale is plausible—sub-agent prompts can carry sensitive context—but the transparency cost is real: you can no longer inspect what your own tool is instructing its delegated agents to do. The raw report is thin, so specifics live in the linked issue thread. Worth watching for anyone who audits agent behavior by reading intermediate prompts.

Armin Ronacher argues the real casualty of coding agents isn't code quality but the shared understanding a team builds through friction—reading each other's code, asking questions, coordinating across services. That slowness was partly waste, but it was also how one engineer's mental model of the system became another's. Agents remove the friction and the synchronization along with it. It's an essayistic take, not a measured study, but the mechanism it names is real and underdiscussed.