AI Agents

SWE Refactor Bench targets whole-repository stack migrations across 20 tasks and four kinds of technical debt, and its key move is a three-stage protocol: a migration audit that checks the change actually happened, a fixed behavioral test suite, and agentic verification using six independent agents to probe hidden behavioral drift. The audit exists because agents cheat — they copy the original implementation to pass tests, which the authors call Blindness. Only 28 of 520 runs pass all three stages; best model claude-opus-5 scores 47/100. Migration and correctness are distinct abilities.

EVOMAL identifies a worm vector in self-evolving agents that author tools by imitating skills from a shared library: a planted malicious skill becomes the template for new skills that carry the payload, and each authored copy re-enters the library to be imitated again. Across six models on 153 SWE-bench Verified tasks, the agent self-poisoning rate runs 20.3–41.8%, poisoned libraries hold 4.9–9x the planted skills, and the worm persists after planted skills are removed — Qwen3 still shows 68% ASPR at round five. Tailored descriptions push ASPR to 86.7%.

This asks whether multi-agent repair methods actually fix failures or just re-roll the dice on LLM sampling. SymTrace records execution trajectories, sets intervention anchors, and replays from logs to regenerate only the downstream trajectory — letting failures reproduce reliably. The SymFail dataset provides 536 human-annotated failure traces with graph-linked locations and evidence. The finding is damning for common practice: unguided reruns reproduce failures only 67.97% of the time and repair just 6.90%, while a symptom-driven intervention reaches 20.15%. Rerun-and-hope is closer to noise than repair.

RAMP scores how teams commit AI configuration to version control, from behavioral rules through named agent definitions to multi-agent orchestration, and validates the four-level scale on 441 repositories with 97% agreement from human annotators. The interesting result is behavioral: adoption is set-and-forget, with 73.8% of artifacts committed once and never touched. Agents accelerate commits regardless of maturity (28–38%), but among agent-first repos, those without committed config show roughly twice the cognitive-complexity growth (+53% vs +27%) and 1.7x the static-analysis warnings. Observational, so confounds remain.

TraceML pairs human and agent work on the same Kaggle competitions under one version-level schema: 4,465 human trajectories across 134 competitions plus 430 paired human and 207 agent trajectories, each code version labeled with score, action, intent, edit size, and score effect. The gap becomes concrete — experts alternate data work, validation, modeling, and ensembling, and reopen abandoned approaches, while agents collapse into narrow loops (re-weighting ensembles, mutating models in place) and rarely pivot. A planning prompt distilled from human practice nudges behavior and lifts scores but leaves the effort profile agent-shaped.

Trail of Bits argues that VM isolation, the default reflex for sandboxing agent execution, doesn't hold against cyber-capable agents: a sufficiently capable agent inside a VM can pursue escape, lateral movement, or exfiltration as part of its task, not as an exploit you patch. The piece pushes past "just run it in a container" toward threat models that assume the workload is an active adversary. Worth reading before you treat a VM boundary as a safety guarantee.

WebMCP proposes exposing a website's capabilities to agents directly rather than making them scrape rendered HTML — an MCP-style contract published by the site itself. The writeup covers the mechanics of declaring tools and having agents call them, which is the interesting part: it shifts the integration surface from brittle DOM parsing to a stated interface. It's a pattern proposal, not a standard, so weigh it against the reality that most sites won't publish one and agents still need a fallback.