2026-06-05
AI Agents
Today's work highlights the shift from agent capabilities to production readiness. We see new architectures for managing skills and data safety, a systems-level characterization of agent memory, and rigorous studies identifying critical failure modes in tool use and self-correction—often with simple, structural fixes.
Papers
5The Self-Correction Illusion: LLMs Correct Others but Not Themselves
This paper presents strong evidence that an LLM agent's inability to self-correct its own reasoning is not a capability deficit but a chat-template artifact. Researchers kept an erroneous claim byte-identical but varied its role label (
<thought>, user, tool, etc.). Relabeling the agent's own flawed reasoning as if it came from an external source (user or <memory>) boosted the correction rate by up to 93 percentage points. This is a crucial, prompt-only intervention with immediate practical use.Microskill Architecture: A Modular Skill-Driven Framework for AI-Native Code Generation
The paper proposes "MicroSkill Architecture," a new paradigm for code agents that treats knowledge like microservices. Instead of injecting the entire codebase into the context window, this approach partitions knowledge into atomic "skill capsules." A router then dynamically selects only the relevant capsules for a given task, framed as a token-budget-constrained optimization problem. An enterprise case study showed a 90% reduction in token consumption and a near doubling of first-try compilation success rates.
Description-Code Inconsistency in Real-world MCP Servers: Measurement, Detection, and Security Implications
A large-scale study on "Description-Code Inconsistency" (DCI), where a tool's natural language description misrepresents what its code does. The authors'
DCIChecker tool found that DCI is widespread (9.93% of 19,200 pairs from 2,214 real-world MCP servers). This creates a major security blind spot, as agents rely on these descriptions to make decisions, potentially leading to anything from operational failures to malicious actions. The paper provides a formal taxonomy and an automated detection framework.Agent Memory: Characterization and System Implications of Stateful Long-Horizon Workloads
This paper provides the first systems-level characterization of agent memory. The authors introduce a taxonomy for memory systems, build a profiler to attribute costs to different memory phases (construction, retrieval, generation), and benchmark ten representative systems. Their analysis reveals how different design choices shift costs between the write and read paths, culminating in ten concrete system recommendations for practitioners building and managing stateful, long-horizon agents at scale.
Data Flow Control: Data Safety Policies for AI Agents
This paper argues that data safety for agents is a data infrastructure problem, not a prompting problem. It introduces Data Flow Control (DFC), a framework to enforce declarative safety policies (e.g., privacy, business rules) directly within a DBMS. Their query rewriter, Passant, enforces DFC policies by reasoning over provenance without materializing it. It runs on five major DBMS engines with near-zero overhead, moving safety from brittle post-hoc checks into the database layer itself.
News
3Show HN: Boxes.dev: ditch localhost; run Claude Code and Codex in the cloud
boxes.dev is a new cloud-based development environment built specifically for working with code agents like Codex and Claude Code. The pitch is to replace clunky localhost setups by giving each agent its own sandboxed cloud computer. This aims to solve concrete problems like managing git worktrees for parallel work, avoiding resource constraints from multiple local agents, and decoupling development from a physical laptop. It uses coding agents to scan and port your local dev setup to the cloud.Show HN: Formally verified polygon intersection – Opus 4.8 oneshots, prev failed
A developer demonstrates that Opus 4.8 can now one-shot generate a formally verified polygon intersection algorithm, complete with the formal proof in the Lean theorem prover. Previous models required a multi-step process with human guidance on proof strategies. While the domain is computational geometry, this showcases a significant leap in model capability for generating high-assurance code. The correctness guarantee comes from the Lean checker, not from trusting the LLM itself.
Fine-tuning an LLM to write docs like it's 1995
A practitioner write-up on fine-tuning an LLM to generate documentation in the style of 1995-era tech manuals. While the goal is stylistic, the post offers a transparent look at the process: curating a small, high-quality dataset, the costs involved, and the effectiveness of fine-tuning for a specific voice and format. It's a concrete example of using fine-tuning for stylistic control rather than just knowledge injection, with transferable lessons for anyone doing similar work.
Blogs
2Reality: The Final Eval — Lukas Petersson and Axel Backlund of Andon Labs
This podcast talks to the builders of VendingBench, an evaluation suite for frontier models. They discuss their methodology for creating novel, lasting benchmarks from scratch to test models from Haiku to the hypothetical Mythos. This offers a look inside the process of designing evaluations that push the boundaries of what models can do, moving beyond standard leaderboards to probe for specific, complex capabilities, which is essential for understanding what today's systems can actually do.
AI enthusiasts are in a race against time, AI skeptics are in a race against entropy
Charity Majors frames a common engineering team dynamic: AI enthusiasts racing to ship capabilities versus AI skeptics trying to preserve system reliability. Enthusiasts see an existential threat in moving too slowly, while skeptics see one in eroding trust and institutional knowledge by shipping code faster than it can be understood. The core problem identified is the lack of a natural feedback loop between these two groups, requiring an intentional leadership and engineering solution to bridge the gap.
89 more items the ranker flagged but didn't feature
Papers
- Towards the Readability of LLM-Generated Codes through Multitask Representation Engineeringarxiv:arxiv-software-eng
- DAST: A VLM-LLM Framework for Cross-Interface Anomaly Detection in O-RANarxiv:arxiv-agents
- HarnessForge: Joint Harness and Policy Evolution for Adaptive Agent Systemsarxiv:arxiv-llm-systems
- TeleSWEBench: A Commit-Driven Benchmark for Evaluating LLM-Powered Software Engineering in Telecommunicationsarxiv:arxiv-software-eng
- SafeMCP: Proactive Power Regulation for LLM Agent Defense via Environment-Grounded Look-Ahead Reasoningarxiv:arxiv-llm-systems
- Evaluating Agentic Configuration Repair for Computer Networksarxiv:arxiv-agents
- Enhancing Software Engineering Through Closed-Loop Memory Optimizationarxiv:arxiv-software-eng
- REStack: A Large-Scale Dataset of Reverse Engineering Discussions from Stack Exchangearxiv:arxiv-software-eng
- A Finite Certificate for the Positive $n=9$ Vasc Inequalityarxiv:arxiv-agents
- Exploring the Topology and Memory of Consensus: How LLM Agents Agree, Fragment, or Settle When Forming Conventionsarxiv:arxiv-llm-systems
- WorldFly: A World-Model-Based Vision-Language-Action Model for UAV Navigationarxiv:arxiv-agents
- Ghost Tool Calls: Issue-Time Privacy for Speculative Agent Toolsarxiv:arxiv-llm-systems
- Benchmarking Counterfactual Prediction in Epidemic Time Series with Time-Varying Interventionsarxiv:arxiv-agents
- More than a Judge: An Empirical Study of Agent-Human Interaction in Crowdsourced Testing Assessmentarxiv:arxiv-software-eng
- Statistical Priors for Implicit Preferences: Decoupling Skill Selection as a Local Harness in Personal Agentsarxiv:arxiv-agents
- PersonaTree: Structured Lifecycle Memory for Person Understanding in LLM Agentsarxiv:arxiv-llm-systems
- Tangram: Unlocking Non-Uniform KV Cache for Efficient Multi-turn LLM Servingarxiv:arxiv-software-eng
- Agentic Molecular Recovery via Molecule-Aware Explorationarxiv:arxiv-agents
- FederatedSkill: Federated Learning for Agentic Skill Evolutionarxiv:arxiv-llm-systems
- Unsupervised Skill Discovery for Agentic Data Analysisarxiv:arxiv-agents
- From Prompt to Process: a Process Taxonomy and Comparative Assessment of Frameworks Supporting AI Software Development Agentsarxiv:arxiv-software-eng
- AgentCL: Toward Rigorous Evaluation of Continual Learning in Language Agentsarxiv:arxiv-llm-systems
- Can LLMs Write Correct TLA+ Specifications? Evaluating Natural-Language-to-TLA+ Generationarxiv:arxiv-software-eng
- LadderMan: Learning Humanoid Perceptive Ladder Climbingarxiv:arxiv-agents
- An Infectious Disease Spread Simulation Based on Large Language Model Decision Makingarxiv:arxiv-agents
- Beyond Similarity: Trustworthy Memory Search for Personal AI Agentsarxiv:arxiv-agents
- Ekka: Automated Diagnosis of Silent Errors in LLM Inferencearxiv:arxiv-software-eng
- Harnessing Generalist Agents for Contextualized Time Seriesarxiv:arxiv-llm-systems
- Retrospective Harness Optimization: Improving LLM Agents via Self-Preference over Trajectory Rolloutsarxiv:arxiv-agents
- Do More Agents Help? Controlled and Protocol-Aligned Evaluation of LLM Agent Workflowsarxiv:arxiv-agents
- ArcANE: Do Role-Playing Language Agents Stay in Character at the Right Time?arxiv:arxiv-llm-systems
- UModel: An Agent-Ready Observability Data Modeling Method at Scalearxiv:arxiv-software-eng
- Revisiting Vul-RAG: Reproducibility and Replicability of RAG-based Vulnerability Detection with Open-Weight Modelsarxiv:arxiv-software-eng
- LatentSkill: From In-Context Textual Skills to In-Weight Latent Skills for LLM Agentsarxiv:arxiv-agents
- AdaMEM: Test-Time Adaptive Memory for Language Agentsarxiv:arxiv-agents
- Memory is Reconstructed, Not Retrieved: Graph Memory for LLM Agentsarxiv:arxiv-agents
- Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teamingarxiv:arxiv-llm-systems
- Willing but Unable: Separating Refusal from Capability in Code LLMs via Abliterationarxiv:arxiv-software-eng
- CRAB-Bench: Evaluating LLM Agents under Complex Task Dependencies and Human-aligned User Simulationarxiv:arxiv-llm-systems
- AgentRedBench: Dynamic Redteaming and Integration-Aware Defense for LLM Agents over SaaS Integrationsarxiv:arxiv-llm-systems
- Vortex: Efficient and Programmable Sparse Attention Serving for AI Agentsarxiv:arxiv-agents
- A Taxonomy of Runtime Faults in Model Context Protocol Serversarxiv:arxiv-software-eng
- TOKI: A Bitemporal Operator Algebra for Contradiction Resolution in LLM-Agent Persistent Memoryarxiv:arxiv-agents
- CollabBench: Benchmarking and Unleashing Collaborative Ability of LLMs with Diverse Players via Proactive Engagementarxiv:arxiv-agents
- DragOn: A Benchmark and Dataset for Drag-Based GUI Interactionsarxiv:arxiv-agents
- Critic-Guided Heterogeneous Multi-Agent Reasoning for Reliable Mathematical Problem Solvingarxiv:arxiv-agents
- From Failed Trajectories to Reliable LLM Agents: Diagnosing and Repairing Harness Flawsarxiv:arxiv-software-eng
- TAPO: Tool-Aware Policy Optimization via Credit Transfer for Multimodal Search Agentsarxiv:arxiv-agents
- RealClawBench: Live OpenClaw Benchmarks from Real Developer-Agent Sessionsarxiv:arxiv-llm-systems
- EGTR-Review: Efficient Evidence-Grounded Scientific Peer Review Generation via Multi-Agent Teacher Distillationarxiv:arxiv-agents
- MARDoc: A Memory-Aware Refinement Agent Framework for Multimodal Long Document QAarxiv:arxiv-agents
- Human oversight of agentic systems in practice: Examining the oversight work, challenges, and heuristics of developers using software agentsarxiv:arxiv-software-eng
- ODTQA-FoRe: An Open-Domain Tabular Question Answering Dataset for Future Data Forecasting and Reasoningarxiv:arxiv-llm-systems
- When Should Memory Stay Silent: Measuring Memory-Use Boundaries in Memory-Augmented Conversational Agentsarxiv:arxiv-agents
- DeployBench: Benchmarking LLM Agents for Research Artifact Deploymentarxiv:arxiv-software-eng
- When Tools Fail: Benchmarking Dynamic Replanning and Anomaly Recovery in LLM Agentsarxiv:arxiv-agents
- Agent Planning Benchmark: A Diagnostic Framework for Planning Capabilities in LLM Agentsarxiv:arxiv-llm-systems
- Adaptive Latent Agentic Reasoningarxiv:arxiv-llm-systems
- From Reward-Hack Activations to Agentic Risk States: Context-Calibrated Mechanistic Monitoring in LLM Agentsarxiv:arxiv-agents
- Asuka-Bench: Benchmarking Code Agents on Underspecified User Intent and Multi-Round Refinementarxiv:arxiv-software-eng
- Reducing Hallucinations in Complex Question Answering using Simple Graph-based Retrieval-Augmented Generation (long version)arxiv:arxiv-agents
- Scaffold, Not Vocabulary? A Controlled, Two-Tier, Pre-Registered Study of a Popperian Code-Generation Skillarxiv:arxiv-software-eng
- VistaHop: Benchmarking Multi-hop Visual Reasoning for Visual DeepSearcharxiv:arxiv-llm-systems
- HANDOFF: Humanoid Agentic Task-Space Whole-Body Control via Distilled Complementary Teachersarxiv:arxiv-agents
- Will the Agent Recuse Itself? Measuring LLM-Agent Compliance with In-Band Access-Deny Signalsarxiv:arxiv-agents
- Inducing Reasoning Primitives from Agent Tracesarxiv:arxiv-llm-systems
- MemTrain: Self-Supervised Context Memory Trainingarxiv:arxiv-llm-systems
- COMAP: Co-Evolving World Models and Agent Policies for LLM Agentsarxiv:arxiv-llm-systems
- Beyond Semantic Organization: Memory as Execution State Management for Long-Horizon Agentsarxiv:arxiv-agents
- Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environmentsarxiv:arxiv-agents
- ADK Arena: Evaluating Agent Development Kits via LLM-as-a-Developerarxiv:arxiv-software-eng
- SAGE: A Quantitative Evaluation of Socialized Evolution in Agent Ecosystemsarxiv:arxiv-llm-systems
- Self-Reflective APIs: Structure Beats Verbosity for AI Agent Recoveryarxiv:arxiv-software-eng
- Towards Persistent Case-Based Memory for Autonomous Data Science: A CBR-Augmented R&D-Agent with a Locally Deployable Small Language Modelarxiv:arxiv-software-eng
- SWE-InfraBench: Evaluating Language Models on Cloud Infrastructure Codearxiv:arxiv-software-eng
- From Risk Classification to Action Plan Remediation: A Guardrail Feedback Driven Framework for LLM Agentsarxiv:arxiv-agents
- AdaPlanBench: Evaluating Adaptive Planning in Large Language Model Agents under World and User Constraintsarxiv:arxiv-llm-systems
- Towards Healthy Evolution: Exploring the Role and Mechanisms of Human-Agent Interaction in Self-Evolving Systemsarxiv:arxiv-agents
- SubtleMemory: A Benchmark for Fine-Grained Relational Memory Discrimination in Long-Horizon AI Agentsarxiv:arxiv-agents
- Goedel-Architect: Streamlining Formal Theorem Proving with Blueprint Generation and Refinementarxiv:arxiv-agents
- ToolChoiceConfusion: Causal Minimal Tool Filtering for Reliable LLM Agentsarxiv:arxiv-agents
- Entropy-Based Evaluation of AI Agents: A Lightweight Framework for Measuring Behavioral Patternsarxiv:arxiv-agents
- RiskFlow: Fast and Faithful Safety-Critical Traffic Scenario Generationarxiv:arxiv-agents
- SmellBench: Towards Fine-Grained Evaluation of Code Agents on Refactoring Tasksarxiv:arxiv-software-eng
- Beyond tokens: a unified framework for latent communication in LLM-based multi-agent systemsarxiv:arxiv-llm-systems
- MLEvolve: A Self-Evolving Framework for Automated Machine Learning Algorithm Discoveryarxiv:arxiv-agents
- Benchmark Everything Everywhere All at Oncearxiv:arxiv-agents
- Humans' ALMANAC: A Human Collaboration Dataset of Action-Level Mental Model Annotations for Agent Collaborationarxiv:arxiv-agents
- MobEvolve: An Agentic Self-Evolving Heuristic System for Interpretable Human Mobility Generationarxiv:arxiv-llm-systems