AI Agents

This paper introduces a clever method for self-improving tool-using agents. First, they use a graph-based planner, ToolGraph, to improve baseline performance. The key innovation is how they generate preference data for DPO tuning: they automatically identify "divergence points" where different agent rollouts made different choices, using the successful path to create a (chosen, rejected) pair. This approach boosts performance on tau2-bench by 16.8% over the baseline, providing a scalable way to refine agents from their own execution history.

A sobering security analysis argues that self-evolving agent systems introduce a new class of persistent, amplified threats. The authors introduce the Module-Lifecycle Attack Surface (MLAS) matrix to systematically map vulnerabilities. Their core finding is that self-evolution transforms session-bounded attacks into permanent, lineage-wide infections. Adversarial changes get baked into the agent's parameters or memory, propagating to future generations without sustained attacker access. Case studies on open-source frameworks show these systems are structurally vulnerable and static defenses are nearly useless.

This paper presents Litmus, a "zero-label" system for designing evaluation metrics by analyzing the source code of an AI pipeline. Instead of starting with a predefined metric, Litmus elicits the developer's intent by inspecting the code and asking targeted questions. It then generates a custom portfolio of metrics that map directly to the system's intended behavior and potential failure modes. On three real-world pipelines, Litmus outperformed baselines in coverage and validity, showing the power of deriving evaluations from the system's own implementation.

A brilliant statistical deep-dive reveals that simple comparisons of code agent performance are misleading. The authors start with a textbook Simpson's Paradox: in aggregate, AI-co-authored PRs have lower merge rates, but per-agent, they have higher rates. But they don't stop there. By progressively adding controls for repository choice and PR structure (like commit count), they show that even the per-agent effect disappears. The conclusion is stark: observed differences are likely selection artifacts, not evidence of causal impact.

This paper identifies and diagnoses "premature commitment," a subtle failure mode where an agent latches onto an early hypothesis and stops reasoning, even as it continues to generate tokens. The authors define a diagnostic based on "representational commitment"—the convergence of hidden states across different runs. On HotpotQA, hidden-state similarity at step 4 predicts final answer consistency, allowing a runtime monitor to detect when an agent has stopped exploring the solution space. Crucially, this signal indicates if an agent has settled, not if it's correct.

A longitudinal study of 400 developers reviewing AI-generated code reveals a troubling trend: reviewers habituate over time. The study finds that as reviewers see more AI PRs, their approval rate increases significantly (+14.5 pp from first to tenth decile), while their review effort (measured by comment volume) decreases (-22%). This isn't because the code is getting better; human PR approval rates declined over the same period. The authors conclude this is likely reflexive habituation under increased workload, not rational trust calibration.

This work identifies a novel failure mode in agent memory systems called "Memory Contagion." When an agent learns from a biased evaluator (e.g., one with a preference for verbose answers), that bias is stored in the agent's memory. The paper shows that this bias then "infects" future agents that retrieve from the same memory store, even if the memory consolidation process is perfect. The bias propagates across time and agent instances, demonstrating that a contaminated memory store can systemically poison an entire agent population.

GroundEval is a direct challenge to the "LLM-as-judge" paradigm. It's a deterministic framework for evaluating agents not on the plausibility of their final answer, but on the validity of the evidence path they took to get there. It scores trajectories based on whether the agent correctly searched, fetched, and cited evidence while respecting access controls and time constraints. A case study shows a frontier LLM judge scoring a response at 0.85, while GroundEval gives it 0.000 because the agent hallucinated its evidence.

As agent skill marketplaces grow, how do you know if a third-party tool is effective or safe? SkillAudit is a framework that aims to solve this by automatically auditing agent skills. Instead of testing on a fixed benchmark, it takes a skill package as input, analyzes it, and generates a custom suite of evaluation tasks to measure its utility, efficiency, and safety in a sandbox. Scanning real-world skills, the framework found that over 7% were risky, highlighting the need for this kind of automated, skill-centric assessment.

Existing privacy benchmarks for LLMs focus on one-to-one conversations, but agents are increasingly used in group chats. This paper introduces MuPPET (Multi-Party Privacy Exposure Testing), the first benchmark designed to measure privacy leaks in multi-party settings. The core risk is that information private to one user might be inappropriately shared with others in the group. Experiments show that models leak significantly more private data in these settings than single-user tests would suggest, and existing defenses are only partially effective.

Simon Willison documents his process of using a code agent to port the Moebius 0.2B image inpainting model from its PyTorch/CUDA origins to run entirely in the browser with WebGPU. It's a detailed, step-by-step account of a real-world software engineering task, showcasing the agent as a capable but not-quite-autonomous pair programmer. He got it working, and the post walks through the code, the prompts, and the dead ends. The value here is the practical, hands-on application.

Simon Willison provides a clear summary of a new paper on "Prompt Injection as Role Confusion." The research finds that models pay more attention to the style of text than to explicit role tags like <system> and <user>. An attacker can append text that mimics the model's internal monologue or system instructions, causing it to override its safety policies. This "role confusion" is a fundamental vulnerability, suggesting that simple XML tagging is not a robust defense against sophisticated injection attacks.

OpenAI board member Zico Kolter and Gray Swan CEO Matt Fredrikson join the Latent Space podcast to discuss AI security. The central argument is that securing AI systems is not simply an extension of traditional cybersecurity; it's a new discipline with its own unique failure modes and threat models. The episode promises a discussion on advanced red-teaming techniques and why a different mindset is required for building and defending agentic systems, moving beyond standard pentesting approaches.