2026-09-16
AI Agents
A measurement audit of the SWE-bench leaderboard finds the top entries statistically indistinguishable — the leading two both resolve 396 of 500, and paired tests separate none of the 29 adjacent top-thirty pairs. Elsewhere, a formal model asks how much of what multi-agent leaves discover reaches the root, plus a plan-injection attack that beats chain-of-thought monitors, a repository-scale vulnerability benchmark, and native skill routing read from a frozen model.
Papers
5The Router Within: Eliciting Native Skill Routing from a Frozen LLM
The claim: a frozen agent LLM already encodes which skill to route to in its own forward passes, and two trained linear maps can read it out with no skill text in context. Gavel does a cheap glance across compact per-skill banks, then resumes shortlisted skills for a verdict, fused as a product of experts. On Qwen3-32B it beats progressive disclosure and retrieve-and-rerank pipelines that add 1.2B–16B external parameters, by up to 21.9 points when the skill need arises mid-rollout. Routing improves as the backbone does.
Coding Agents Have Converged: Why the SWE-bench Leaderboard Can No Longer Order Its Top Entries, and What to Measure Instead
A careful audit of 254 SWE-bench submissions, done without rerunning models, argues the leaderboard cannot order its top entries. On Verified the leading two both resolve 396 of 500; the top ten share 285 successes and 51 failures, and exact paired McNemar tests separate none of the 29 adjacent top-thirty pairs at alpha=0.05. Scores swing up to 29.8 points on scaffold choice alone. The fix they propose: report comparison-set-specific resolution and model-scaffold provenance instead of treating small aggregate gaps as rankings.
Corrupt Plans, Clean Traces: Evading Chain-of-Thought Monitoring with Plan Injection
Chain-of-thought monitoring assumes the actor's reasoning trace surfaces unsafe intent for a monitor to catch. This shows you can plant harmful-but-benign-sounding reasoning in the actor's context, steering it to adversarial actions while the monitor stays clean — 25–33% evasion across benchmarks, scaling to DeepSeek-R1. Actors paraphrase injected plans as their own reasoning without attribution. The sharp finding: giving the monitor access to the injected plan drops detection by up to 50% on Bio-Math, and extra thinking budget gets spent rationalizing rather than flagging.
Decomposition Buys Integrity, Not Yield
A formal model of decomposition asks how much of what leaf agents discover reaches the root. Model each agent as keeping any handed item with probability r(b); if r(b)=1/b every tree delivers exactly one finding, verified to 1e-15 on 20,000 random trees. Under r(b)=Cb^(-δ), a depth-k tree yields C^k·N^(1-δ): task size and architecture separate, architecture contributes only C≤1 per level, so flat is optimal for yield. On 600 production deep-research traces δ=0.34. Depth buys context integrity, not recall.
Vulnerability Localization Benchmark: Measuring Agentic Security Analysis at Repository Scale
A benchmark for a capability existing security evals skip: not detecting or repairing a vulnerability, but locating the code. VLoc Bench pairs 500 real vulnerabilities from 290 repos across 147 CWE categories with before/after snapshots — the agent gets only a CWE description and read-only terminal access. Results are grim: 27 models and four static analyzers, and the strongest system hits 0.229 File F1, with 38.4% of tasks localized by nobody. Good localizers also hallucinate locations on patched repos, so accuracy doesn't imply reliable post-fix behavior.
News
1Show HN: Pizza Bot – An inbox for AI agents that work in the background
A self-hosted desktop app that runs background agents behind an email-like inbox: finished work lands in Unread, approval-gated actions in Action. Apache 2.0, no signup or telemetry, bring-your-own provider (Anthropic, Bedrock, Gemini, OpenAI, OpenRouter, or local Ollama). It grew out of an internal Amazon tool that drove a CRM via CDP and Playwright, and now leans on DeepAgents, LangGraph, MCP addons, and checkpointing. The inbox-as-interface framing for long-running agents is the interesting part, and it's actually shippable.
36 more items the ranker flagged but didn't feature
Papers
- Verifiable Social Reasoning for LLM Assistantsarxiv:arxiv-agents
- Universal Defenses for Tool-Integrated LLM Agents Against Adversarial Attacksarxiv:arxiv-llm-systems
- Agentic Societies Need a Social Harnessarxiv:arxiv-agents
- AURA: Agentic Diagnosis and Refinement for Production Recommender Systems at Scalearxiv:arxiv-agents
- EchoPath: Execution-Level Replayable Memory for GUI Agentsarxiv:arxiv-agents
- Coaching Qwen3 Coder 30B to Think Like a CodeClash Arena Agentarxiv:arxiv-software-eng
- little m: An AI Agent for Industrial Process Optimizationarxiv:arxiv-agents
- Evaluating Verified Autonomy in Quantum Engineeringarxiv:arxiv-agents
- HypoEvolve: Genetic Algorithms Enable Multi-Agent LLMs to Discover Scientific Hypothesesarxiv:arxiv-llm-systems
- Large Language Models in the Loop: A Stability- and Network-Aware Survey in Networked Control, Cyber-Physical, and Multi-Agent Systemsarxiv:arxiv-agents
- Assurance Envelopes for Autonomous Coding Agents: Minimum-Cost Evidence for Software Changearxiv:arxiv-agents
- ENCP: Episode-Normalized Conformal Prediction for Vision-and-Language Navigationarxiv:arxiv-agents
- Interactive Memory Learning for Long-Term Conversationsarxiv:arxiv-agents
- RepoAtlas: Guiding Coding Agents via Evolving Multimodal Repository Viewsarxiv:arxiv-agents
- After the Party: Governing What a Viral Agent-Skill Ecosystem Left Behindarxiv:arxiv-agents
- ThinkFlow: Self-Evolving Probabilistic Latent Memory for Lifelong Conversational Agentsarxiv:arxiv-agents
- ScienceBuddy: Recursive-in-Recursive Self-Improvement for Interactive Scientific Agentsarxiv:arxiv-agents
- Grounding SWE-Agent Decisions in Architecture-0 Design: Navigating Unknown Unknowns through Physical Mappingarxiv:arxiv-agents
- FlashVector: Agent for Hierarchical Model Serving Stack Optimizationarxiv:arxiv-agents
- Turn-level Multiscale Density Ratio Estimation for LLM Agentsarxiv:arxiv-agents
- EvoOntology: A Self-Evolving Ontology Layer for Data Agentsarxiv:arxiv-llm-systems
- The MAL Simulator: Cyber Operations Simulation based on Attack & Defense Graphsarxiv:arxiv-agents
- Using Agentic AI for contextualized and multifaceted code review at Ericssonarxiv:arxiv-software-eng
- IWC-Bench: Evaluating Web Application Generation from a Software Testing Perspectivearxiv:arxiv-software-eng
- End-to-End Latency-Minimizing and Load-Balanced Request Scheduling for Edge LLM Inference in Agentic AI Servicesarxiv:arxiv-agents
- Protocol-Preserving Context Trimming for Agentic Workflows: Benefits, Failure Regimes, and Budget Guardrailsarxiv:arxiv-agents
- World Models for Embodied Intelligence: From Plausible to Controllable to Actionablearxiv:arxiv-agents
- Interpreting and Steering LLM Agents for Social Simulationsarxiv:arxiv-agents
- Toward Governance-Aware Autonomous GIS: A Narrative Review of Ethical and Privacy Risks in LLM-Enabled GeoAIarxiv:arxiv-agents
- A Cyber Range Evaluation of Autonomous Network Incident Response Agentsarxiv:arxiv-agents
News
Blogs
- Gemini Live audiorss:simonw
- Can Skills Learned in Games Transfer to Real-World Work?rss:latent-space