AI Agents

ECLoop interposes between a coding agent and the repository, compiling per-task conditions specifying what the agent must observe before each type of edit or patch submission, then postponing any action whose evidence conditions remain unmet. On all 500 SWE-bench Verified instances across two models and two scaffolds, it raises Pass@1 by 4.8-11.8 points with no retraining and no scaffold changes, while cutting token consumption by up to 12.1%. Structured conditions beat an equivalent natural-language summary, and the ablation shows each of the three operations pulls its weight.

This paper pins agent safety degradation on a specific cause: schema-formatted tool specifications weaken the model's internal refusal signals, shown via white-box representation analysis. SafeKeep decouples the safety judgment from execution — it evaluates requests against flattened textual specs while keeping the original schema for the actual tool call. Across two benchmarks and four models (white- and black-box), refusal on harmful requests rises from 23.8% to 70.6%, and attack success under observation-level prompt injection drops from 25.6% to 2.5%, with task capability preserved.

Most agent evaluations reduce failures to system-level pass/fail, which hides where the fault originated and which fix would help. This taxonomy assigns each of 41 failure modes to an edge between two components (model, harness, user, tools, memory, environment) plus a fault side marking where the repair belongs — model-side means post-training, harness-side means scaffolding, environment or grader means eval redesign. It applies across coding assistants, long-horizon assistants, and multi-agent systems, grounded in worked examples from benchmarks, system cards, and logged trajectories, with a reproducibility check.

When a repair agent sees a test pass, that outcome gets treated as evidence about the bug. BSG-VA replays each validation command against the buggy code, the candidate patch, and the gold fix to assign every event an evidence role. Across 3,730 events in 643 rollouts, 46% of positive comparable events carry no bug-discriminating information, and 23.8% of baseline rollouts close with a patch whose entire positive evidence is of that kind. Feeding back the buggy-state replay outcome cuts evidence-inadequate closure by 7.8 points — but below the prespecified effect size, so practical magnitude stays uncertain.

ARCTIC reframes AI code review around three capabilities instead of the usual style nitpicks: intent prediction infers why a change was made from conversation logs and metadata (0.86 F1); drift detection measures divergence between intent and the agent's output via backtranslation (QWK 0.907 against human annotators); and spotlight ranks the diff regions most needing human scrutiny, beating a baseline reviewer 2.4x on quality estimation at 5x fewer tokens. In rollout, drift scoring cut misalignment by 5.76 points and no defects were attributed to self-reviewed diffs since launch.