Hamel Husain argues "it's hard to eval" is a product smell—if you can't verify an agent's output, neither can your users, so redesign for verifiability before building evals. Two memory papers collide with reality: "Manufactured Confidence" shows mem0/LangMem-style consolidation upgrades a hedged remark into a confident dated fact agents then obey, with "reportedly" treated like a flat assertion. TraceLab releases 4,300 real Claude Code and Codex sessions (350K LLM steps, 430K tool calls) for serving analysis, and Trellis treats the agent experience graph as queryable database state behind Meta's KernelEvolve.
-
TUE · JUN 30 -
MON · JUN 29 Daniel Russo's study of 930,000 agent-authored PRs finds repository-level integration friction concentrates 2x more from agents than humans (ICC 0.30 vs 0.16), arguing governance belongs to the repo not the agent. OpenThoughts-Agent ships an open data pipeline with 100+ ablations, lifting Qwen3-32B to 44.8% across seven agentic benchmarks. Heuresis runs 3,222 scored research runs and finds zero "Original" ideas plus 40 reward-hacking fabrications. Fernando Irarrázaval's hackmyclaw challenge survived 6,000 prompt-injection attempts against Opus 4.6.
-
SUN · JUN 28 Two papers challenge execution-heavy program repair: "To Run or Not to Run" finds that prohibiting code execution costs only 1.25pp of resolve-rate on SOTA agents while saving substantial tokens and wall-clock, across 7,745 SWE-bench traces. icat-agent replaces shared context with event-based message passing and rubric-gated workflow pivots, beating Claude Code by 6.3-18.5% on SWE-bench Pro at $1.18 less per instance. Josef Chen's co-failure ceiling shows ensemble accuracy is capped at 1-beta, with beta underpriced ~2.5x across 67 models.
- To Run or Not to Run: Analyzing the Cost-Effectiveness of Code Execution in LLM-Based Program Repair
- Unlocking Model Potentials Through Adaptive Multi-Agent Scaffolding for Efficient Issue Resolution
- When Does Combining Language Models Help? A Co-Failure Ceiling on Routing, Voting, and Mixture-of-Agents Across 67 Frontier Models
-
FRI · JUN 26 Today's research details several production-deployed agent systems and critical architectural patterns. `AgentX` and `NOVA` describe autonomous systems managing the full experimentation and architecture-evolution loop for industrial recommender systems. For architecture, `MemStrata` uses a temporal ledger to solve stale-fact errors in RAG, `ESAA-Conversational` uses event-sourcing for shared memory across coding assistants, and a study on quantization reveals it can unexpectedly increase latency and energy costs.
-
THU · JUN 25 Today's research focuses on the components of agent architecture. For safety, the 'Unfireable Safety Kernel' proposes a formally verified, out-of-process authorizer. For training, 'Progress Advantage' derives a step-level reward signal directly from RL post-training, no annotations needed. For reasoning, 'Representation Matters' shows that AST+PDG graphs beat raw source for vulnerability detection (83.2% vs 53.5%). And for software tasks, the SWE-Pro benchmark reveals a huge gap: LLMs achieve negligible gains where experts see a 15.5x speedup.
-
WED · JUN 24 A focus on reliable agent construction today. The EDV paper coins the "Self-Confirmation Trap" in agent memory, proposing a multi-agent fix to prevent agents from reinforcing their own errors. CodeTeam tackles repository-level code-gen with a team of Architect, CTO, and Developer agents. For safety, AutoSpec uses inductive logic programming to automatically evolve rules from annotated traces, while Holmes is an agentic system that diagnosed real-world mobile crashes at WeChat, reducing investigation time by 98%.
-
TUE · JUN 23 A massive day for evaluation and safety research, with a dozen new papers and posts tackling the subtle, difficult, and often-hidden failure modes of agent systems. The focus is shifting from "can it work?" to "how can it fail, and how would we know?"
-
MON · JUN 22 Today's focus is on the hard engineering of robust agents. We cover new patterns for evaluation and security, infrastructure for dynamic tool deployment, and deep dives into novel failure modes and defense mechanisms that emerge when agents operate in complex, real-world environments.
-
SUN · JUN 21 This week's papers move past capabilities to focus on the hard engineering of production agents: new architectures for efficiency, robust frameworks for safety and compliance, and a wave of new benchmarks and evaluation methods that look beyond simple pass/fail metrics.
-
SAT · JUN 20 Today's papers and posts focus on the engineering of production-ready agentic systems, moving beyond proofs-of-concept to introduce concrete architectural patterns, programming models, and evaluation frameworks for building and operating them reliably.
-
FRI · JUN 19 Today's papers move beyond basic capabilities to focus on the engineering of reliable, measurable, and safe agent systems. The throughline is a shift from 'can it work?' to 'how does it break, and how do we build systems that don't?'—with a heavy emphasis on new evaluation methods and architectural patterns for robustness.
-
THU · JUN 18 A wave of new benchmarks and datasets signals a maturing field, moving from 'can it work?' to the much harder questions of 'how does it fail, and how can we measure it?'. The focus is on robust evaluation for complex, real-world failure modes like multi-user memory safety, contextual privacy, and the security gaps inherent in tool use.
-
WED · JUN 17 Today's research moves past 'can it work?' to 'how does it fail, and how do we fix it?'. We're seeing a wave of work that taxonomizes agent failures—from routing degradation at scale and assertion-free tests to subtle provenance errors—and proposes concrete architectural patterns and validation frameworks to build more reliable systems.
-
TUE · JUN 16 Today's work is all about moving beyond pass/fail metrics to audit *how* agents operate. We're seeing techniques for fingerprinting agent behavior, creating verifiable evidence graphs for data analysis, and securing the API infrastructure that connects them. This push for observability and verifiability signals a maturation from academic prototypes toward production-ready systems.
-
MON · JUN 15 Today's work zeroes in on how agents fail. We're seeing a wave of new benchmarks designed to find subtle but critical vulnerabilities: decomposition attacks, social engineering in PRs, latent planning errors, and even denial-of-service against the guardrails themselves. This shift from capability to reliability is driving more robust architectural patterns, from privacy-preserving UI brokers to specialized sub-agents for efficiency.
-
SUN · JUN 14 This week's research converges on managing complexity in long-horizon agentic tasks. Instead of relying solely on raw model scale, the most compelling work introduces explicit structures—hierarchical memory, decision trees for rules, and dynamic compute routing—to make agent planning more robust, efficient, and auditable.
-
SAT · JUN 13 This week's research marks a clear shift from scaling models to scaling systems. The most compelling work isn't just about agent capability, but about the architectures, protocols, and economic models needed to run them reliably and efficiently in production. From tradable KV caches to formal governance frameworks and standardized evaluation, the focus is moving to the infrastructure that makes agents viable.
-
FRI · JUN 12 Today's highlights reveal a growing tension in agent development: while sophisticated new architectures promise greater capability, critical issues like cost control, model transparency, and robust evaluation are becoming unavoidable hurdles to production deployment.
-
THU · JUN 11 The big story this week is the release of Anthropic's Claude Fable 5, the immediate controversy over its "silent sabotage" safety feature, and the company's rapid course correction in the face of developer backlash. It's a case study in model transparency and production reliability.
-
WED · JUN 10 Today's papers all point to a field grappling with the lifecycle of agentic systems. Several papers introduce novel self-evolution frameworks where agents improve themselves or their own training data, while others tackle the corresponding complexities of security and evaluation, from hardening benchmarks against reward hacking to identifying new multi-step attack surfaces.
-
TUE · JUN 9 Today's items highlight the growing focus on agent infrastructure and reliability. From hardware-aware serving simulators and novel security architectures to better debugging tools and uncertainty estimation for code generation, the field is clearly tackling the hard problems of running agents in production.
-
MON · JUN 8 A clear theme today is the shift toward architectural patterns for more robust and scalable agent systems. Instead of monolithic designs, we're seeing proposals for memory standards, retrieval-based tool use, and multi-agent frameworks that separate planning from governed execution.
-
SUN · JUN 7 A clear theme emerges today: the shift from building proof-of-concept agents to engineering reliable, secure, and measurable systems. We're seeing a surge in new benchmarks, formal methods for production concerns like memory and cost, and frameworks for diagnosing the entire agentic system, not just the model.
- How to design a multi-agent enterprise with OpenClaw and Hermes
- Tokenomics: Quantifying Where Tokens Are Used in Agentic Software Engineering
- Domain-Conditioned Safety in Frontier Computer-Using Agents: A 793-Episode Browser Benchmark, a Coding-Domain Cross-Reference, and a Reproducibility Audit of Recent Red-Teaming
-
SAT · JUN 6 Today's focus is on hardening the infrastructure around agents. We're seeing a clear shift from simple retrieve-and-prompt patterns to more sophisticated, secure, and efficient architectures for memory, skill management, and code execution. This reflects a growing maturity in the field, moving from what agents can do to how they can do it reliably and safely in production.
-
FRI · JUN 5 Today's work highlights the shift from agent capabilities to production readiness. We see new architectures for managing skills and data safety, a systems-level characterization of agent memory, and rigorous studies identifying critical failure modes in tool use and self-correction—often with simple, structural fixes.
-
THU · JUN 4 The conversation around agents is maturing from core planning loops to the practicalities of building and running them in production. Today we see new tools for memory and UI, real-world examples of cost management, and major players tackling safety for critical infrastructure.
-
WED · JUN 3 Today's focus is on the maturation of code agents. We're seeing major players like Microsoft release purpose-built models, while new open-source tooling tackles the critical challenge of safe code execution through sophisticated sandboxing.
-
TUE · JUN 2 Safety theory collided with production reality today. The Meta AI account takeover demonstrates that prompt injection remains a critical vector in support flows, bypassing containment strategies discussed yesterday. Combined with new benchmarks showing >50% violation rates in stateful workspaces and significant compositional skill risks, the data confirms our current guardrails are insufficientfor
-
MON · JUN 1 Production safety is the bottleneck. Today's items converge on a hard truth: as agents gain autonomy, the attack surface expands faster than our defenses. Multi-step trojans, retrieval-induced safety degradation, and prompt fragility all point to the same gap—single-turn evals miss real-world failure modes.
- From Prompt Injection to Persistent Control: Defending Agentic Harness Against Trojan Backdoors
- How Coding Agents Fail Their Users: A Large-Scale Analysis of Developer-Agent Misalignment in 20,574 Real-World Sessions
- Automating Low-Risk Code Review at Meta: RADAR, Risk Calibration, and Review Efficiency