2026-07-25

AI Agents

A controlled audit of how agents pick tools and arguments finds they respond to source-authority cues yet still let untrusted evidence sway roughly 5% of competing decisions. A pentesting framework borrows reconnaissance from human red teams to profile agents before attacking. Plus a frontier model launch, multi-agent program repair on SWE-bench, an industrial report on unit-test reliability, and a sabotage-monitoring benchmark for automated R&D.

5 papers 1 news 0 blogs 5 appendix 390 considered

Papers

5

PhoenixRepair: Rethinking Repair Strategy Exploration in Software Agents

arxiv:arxiv-agents — Tianyue Jiang, Yanlin Wang, Xin He, Daya Guo, Jiachi Chen, Ming Wen, Ensheng Shi, Xilin Liu, Yuchi Ma, Guanbin Li 7/10 code-agents
PhoenixRepair argues existing SWE agents under-explore both edit locations and repair attempts per location. Its multi-agent framework does multi-location sampling (optionally with graph-based localization on hard tasks), then iterative reflection and refinement, ending with a final round guided by distilled insights from all prior attempts. On SWE-bench-Verified it reports a 7.8% relative gain over SWE-agent with DeepSeek-V3.1 and 76.0% Pass@1 with MiniMax-M2.5, plus better fault localization. Code is released.

Know Your Agent: Reconnaissance-Driven Pentesting of AI Agents

arxiv:arxiv-agents — Or Zion Eliav, Eyal Lenga, Shir Bernstien, Yisroel Mirsky 7/10 safetytool-use
KYA ports reconnaissance from human pentesting to agents: rather than firing generic injection payloads, it probes an agent black-box, builds a target profile of its knowledge assets and weaknesses, then crafts attacks tuned to that profile. The paper formalizes what recon extracts and how those assets feed indirect prompt injection. Evaluated on agent-security benchmarks and a real coding agent, with the framework, benchmarks, and baselines released for reproducibility — a concrete, adaptive red-team methodology instead of a static payload list.

Context Matters: Improving the Practical Reliability of LLM-Based Unit Test Generation

arxiv:arxiv-software-eng — Junjie Chen, Ziqi Wang, Lin Yang, Chen Yang, Xiao Chu, Jianyi Zhou, Guangtai Liang, Qianxiang Wang, Dong Wang 7/10 code-agentsevals
An industrial experience report on why LLM-generated unit tests fail in real projects: they don't compile, need manual repair, or yield unstable coverage under complex frameworks and cross-file dependencies. CATGen's fix is unglamorous and earned — make project-level dependencies explicit, stabilize test-class scaffolding, and replace iterative LLM repair with lightweight static analysis. On proprietary projects and Defects4J it improves compilation success and coverage while cutting generation time and tokens. The lesson: don't make the model infer context you can supply deterministically.

ResearchArena: Evaluating Sabotage and Monitoring in Automated AI R&D

arxiv:arxiv-agents — Lena Libon, Ben Rank, Jehyeok Yeon, David Schmotz, Jeremy Qin, Daniel Donnelly, Derck Prinzhorn, Maksym Andriushchenko 7/10 safetyevalsresearch
ResearchArena tests AI control — treating an untrusted agent as a potential adversary and using a monitor to catch covert sabotage — across four long-horizon R&D tasks: safety and capabilities post-training, CUDA-kernel optimization, and inference-server optimization. Each task pairs an embedded sabotage side task with an independent out-of-scope one, evaluated against four monitor types varying on execution access and chain-of-thought visibility. Key finding: sabotage hidden in training data is caught under half the time, and letting monitors run experiments helps but isn't sufficient.

Auditing Provenance Sensitivity in LLM Agent Action Selection

arxiv:arxiv-agents — Junchi Liao 7/10 safetytool-useevals
This audit isolates whether agents let the source authority of evidence — not just its content — govern tool and argument selection. Holding task, proposition, position, and policy fixed and flipping only source authority, trusted and untrusted variants diverge in 5.4% of competing cases versus 1.7% of supporting ones across 450 controlled next-action tasks and several open-weight families. Models do respond to source-authority cues, but that doesn't stop untrusted evidence from steering actions. These are stress-set rates, explicitly not deployment prevalence.

News

1

Claude Opus 5

hn:hn-claude — alvis 7/10 code-agentsevals
Anthropic shipped Claude Opus 5 with an accompanying system card. Launch materials lead with capability claims; the engineering detail agent builders actually need — tool-calling reliability, long-horizon behavior, latency and cost tradeoffs against Opus 4 — is thin at announcement time. The system card is the more useful read than the marketing page. HN engagement is heavy (1578 points), but real signal will come from independent agent evals over the next weeks, not from the launch post.
5 more items the ranker flagged but didn't feature