AI Agents

An instruction-following benchmark for coding agents that separates real compliance from coincidence. Its key move is Against-Prior Accuracy: score only rules that oppose a model's unprompted defaults, verified by re-running tasks with the rule withheld across nine probe builds. Across 12 frontier models, accuracy spans 72.1–85.9% but AP-Acc drops to 66.1–78.6%, and every model is worse on against-prior rules (mean 5.81 points). A conflict pilot finds precedence doesn't follow prompt depth — system prompts and project files outrank tool and skill descriptions.

An empirical study attributing agent failures to specific loaded skills via differential analysis — comparing a skill-guided run against a no-skill or matched-skill reference solving the same task. On SkillsBench and SWE-Skills-Bench it isolates 307 skill-induced failures: 125 functional, 182 efficiency regressions. The sharp finding: functional failures rarely come from obviously irrelevant skills; seemingly relevant skills push agents to misimplement or omit required elements. Efficiency regressions aren't explained by prompt length. Ships SkillTriage, a taxonomy-guided attribution tool.

A study of how safety rules survive context compaction in long-running agents. The central finding: a presence check is not a safety check. When compaction doesn't drop a rule outright, it often leaves a residue that reads like the rule but no longer acts like one — and behavioral replay shows the model performs the prohibited action far more often (+34 and +57 points across two replay models). Rule-form items are retained more than matched facts, which is exactly why presence-based audits feel adequate while offering no protection.

Controlled experiments on repository-level issue fixing across six tool architectures that hold information and actions constant while varying how they're organized and exposed — three actors, 11,700 trajectories. Results: structured low-level interfaces improve consistency across repeated attempts by up to 4.7x; natural-language search increases access to relevant files by over 11%; Python CodeAct-style interfaces hit similar task performance with 41.6% fewer steps and 56.3% lower tokens. Text-based cognitive-scaffolding tools (letting the agent record reasoning) had limited effect. A rare properly-controlled study of interface design.

A diagnostic study of gist-based context compression using a salience-weighted framework as a probe, evaluated on all ten LoCoMo conversations (1,501 questions after excluding adversarial). The task-type interaction is clean: gist compression beats truncation on multi-hop and single-hop factual questions but hurts badly on temporal questions. The mechanism is traced precisely — the abstraction prompt preserves relational and event structure while discarding dates and times. A one-sentence prompt change lifts temporal-expression preservation roughly 20-fold (3.05% to 62.39%).

A YC launch pitching multi-agent systems to discover new materials for semiconductor thermal management — dielectrics for 3D-stacked HBM, thermal interface materials, substrates — motivated by GPU TDPs climbing from 700W (H100) to 2.3kW (Rubin). The framing of the physics problem is concrete and the beat-adjacent multi-model angle is interesting, but the post is largely a launch narrative: no benchmark details, no evaluation methodology, and no specifics on how the agents actually propose or validate candidate materials.