AI Agents

Tool-using agents can fail in ways that text-only safety evals miss. SafeClawBench introduces a staged security benchmark that separates semantic agreement from actual system harm, evaluating attack acceptance, audit evidence, and sandbox-observed effects independently. Crucially, the authors found that hundreds of observed sandbox harms occurred on tasks that passed the semantic check. This demonstrates a dangerous gap between what an agent says it will do and the damage it can cause, making a strong case for measuring executable outcomes.

This paper presents Data Intelligence Agents (DIA), a production-deployed system of three autonomous coding agents that collaborate on enterprise data integration tasks. The key architectural idea is treating agents as producers of concrete, executable artifacts (code, schemas) rather than just text. The system's Query Generator agent matches or beats state-of-the-art results on seven SQL benchmarks across four dialects, demonstrating that an execution-grounded, multi-agent architecture can generalize well with only natural-language instruction changes.

Current memory benchmarks focus on single users, but what about shared assistants in homes or offices? GateMem is a new benchmark for this multi-principal setting, evaluating an agent's ability to simultaneously provide utility, enforce access control, and handle deletion requests. Testing various architectures reveals a stark trade-off: no current method can do all three well. Retrieval and external-memory systems, while cheaper, were particularly prone to leaking unauthorized or deleted information, showing these systems are not ready for trusted shared deployment.

This work argues that PII redaction isn't a simple entity recognition task but a problem of contextual integrity—a phone number in a public directory is different from one in a medical record. The paper introduces RedactionBench, a benchmark built on this principle, and a new metric, R-Score, that handles semantic and formatting variations. Human evaluations on the benchmark show high agreement on clearly private or public information but only 47.7% agreement on contextual cases, highlighting the subjective nature of privacy.

Agents often need to use private information (like a passport number) to complete a task. The TRAP benchmark evaluates the fundamental tension between using this data for tool calls and preventing it from leaking in chat responses. The authors find that instruction-following ability correlates with leakage rates across 22 models. Most importantly, they provide a formal argument that for any softmax-based model, no prompt-based defense can achieve high task success with zero leakage, suggesting this trade-off is inescapable without architectural changes.

Training RL agents on static datasets is inefficient because the most informative samples—those at the boundary of the agent's current ability—are quickly exhausted. RODS proposes a clever solution: use the reward variance from training rollouts as a zero-cost detector for this capability boundary. It then synthesizes new, structurally similar tasks to target this boundary, continuously generating the most valuable data. This online synthesis approach achieves comparable performance to a massive offline dataset with 20x fewer trajectories.

The field gets a major new resource for training code agents with Open-SWE-Traces, a dataset of over 207,000 agentic trajectories. Sourced from 20,000 real-world pull requests across nine programming languages, it provides a massive, diverse, and permissively licensed foundation for distillation. The authors validate the data by fine-tuning Qwen models, achieving a 61.7% resolve rate on SWE-bench Verified. The dataset's hybrid approach, capturing both explicit 'thinking' and direct 'non-thinking' traces, provides rich material for training more capable models.

GUI agents trained via imitation learning often fail when they encounter a screen state not present in the expert data. This paper's SGCD framework addresses this by intentionally letting the agent go 'off-trajectory', then using a more capable, skill-guided policy to generate a successful continuation from that novel state. These newly generated successful paths are then added to the training data, providing supervision exactly where it was missing. This iterative self-improvement loop boosted success rates on OSWorld from the low 30s to over 50%.

Automated program repair often fails because it lacks the nuanced understanding of a human debugger. PracRepair is a framework that aims to close this gap by mimicking a human-like workflow. It constructs a context from static and dynamic traces, uses question-driven diagnosis to form explicit repair hypotheses, and iteratively refines patches using detailed feedback from validation runs, not just a pass/fail signal. This more dynamic, trace-aware approach significantly outperforms prior methods on Defects4J, fixing 162 bugs with GPT-4o.

This paper proposes a paradigm shift for agent memory: represent the user not as retrieved text but as executable code. In the User as Code (UaC) model, user interactions are compiled into typed Python objects for state and functions for rules. This allows the agent's memory to be 'run' to answer questions that are impossible for retrieval, like aggregating data over time or enforcing complex rules. While matching retrieval on simple recall, it achieves near-perfect scores on aggregation tasks where retrieval-based systems completely fail.

Z.ai has open-sourced GLM-5.2, a massive 753B parameter Mixture of Experts model with a 1M token context window. Independent benchmarks from Artificial Analysis place it as the new top open-weights text model, outperforming models like MiniMax-M3 and DeepSeek V4 Pro. It’s an MIT-licensed, text-only model, and while it's a powerful new baseline for open source capabilities, initial analysis suggests it's also quite token-hungry, which has significant cost and latency implications for any production use case.