A new paper asks whether an automated gate — a reward model or a reviewer bot — can ever reliably decide if an agent's proposed self-modification is a genuine improvement, and proves it can't in the general case. A browser hook quizzes you on your AI's code before it ships. The rest: two runtimes for carrying project state across long-horizon tasks, a failure taxonomy for agentic workflows, and a decoding trick that cuts chain-of-thought length.
-
SUN · OCT 18 -
SAT · OCT 17 Two pieces converge on a workflow shift: Anthropic's Claude Code team describes moving code review from an end-of-road PR gate to continuous checking that runs like spellcheck, catching routine mistakes before a human looks. A separate paper argues that at equal token budget, showing coding agents whole files verbatim beats both retrievers and compressors. Also today: an audit that collapses one benchmark's reported success from 94% to 65%, plus context compaction and midtraining work.
- When Agent Evaluation Becomes Unreliable: An Empirical Study of Oracle Failures and Configuration-Dependent Measurement in an LLM Agent Benchmark
- Skip the Compressor: Coding Agents Fix More Issues When They Pick Whole Files
- Prune the Summary, Not the History: Source-Judged Compaction for Long-Horizon Agents
-
FRI · OCT 16 A new benchmark drops agents into forward-deployed-engineer work — clarifying vague requirements, probing live systems, dealing with scripted stakeholders — and the best model configuration resolves only 14 of 95 delivery tasks, with discovering unstated constraints the main choke point. A separate study of automated research finds the counterintuitive fix for stalled discovery is forgetting. Also today: RL-trained long-horizon ML engineering, anticipatory context curation, and runtime verification for agentic systems.
-
THU · OCT 15 A measurement study asks what coding agents don't tell their human supervisors, and finds report-only readers recover just 52% of task states versus 93% for anyone with full execution context — a 40-point blind spot. Elsewhere, a Swedish TV production loses an entire edited season to an agent, plus papers on reusable agent-options, tree-navigating search, exploit-guided patching, and a localized Japanese tool-use benchmark.
-
WED · OCT 14 Anthropic explains why it abandoned Git: its monorepo went from 2,500 merges a day in May to over 30,000 in September as coding agents took over, and the old tooling buckled. Elsewhere, a formal runtime shield that compiles procedural manuals into context-aware guardrails, a paper arguing the 'first mistake' labels we debug agents by point at symptoms not causes, and benchmarks for dependency fixes and code bloat.
-
TUE · OCT 13 A new paper traces the exact internal mechanism that decides whether an agent calls a tool or just answers, finding a single causal vector that analysis verbs suppress and execution verbs leave intact. Another warns that LLM-generated regression tests encode faults as expected behavior 8-17% of the time, and those bad tests persist. The rest of the issue covers coding-agent evaluation, repository construction, and cheaper agent sandboxes.
-
MON · OCT 12 A new paper investigates whether the files, configs, and caches that agent skills leave behind can be weaponized across sessions — and finds attack success rates of 46.7-77.4% against Codex CLI, Gemini CLI, and Claude Code. Another shows reward hacking spreading agent-to-agent through shared code and plans. The rest of today's papers section covers runtime guardrails, white-box deception probes, and credit assignment for long-horizon agent training.
-
SUN · OCT 11 A benchmark puts frontier models in charge of a running online store — pricing, restocking, surviving supplier failures over a simulated year — and every one of them loses to a scripted heuristic (49% vs 97%). A separate attack shows LLM guardrails flipping to allow prohibited actions after six lines of injected log text. Elsewhere: production program repair at Google scale, a surprising null result on in-context RL, and field reports from coding agents.
-
SAT · OCT 10 A benchmark checks whether agents can estimate and control their own runtime, and finds the gaps are wide: one setup overshoots a requested duration by 2.9x, and 14 of 158 runs quietly slept after appearing to finish. A separate monitoring technique catches failing agent runs mid-stream in about a millisecond per step. The rest of the day runs through safety guards, eval-task-flow mismatch, and hands-on coding-agent accounts.
-
FRI · OCT 9 A fault-injection study wraps a function-calling benchmark and shows agents flag a problem 91% of the time when a tool returns an explicit error, but only 59% when it returns a plausible wrong value — and reasoning models notice less than their instruct siblings. Another tool catches reward-hacking before an agent acts by formally certifying when a task's tests contradict its intent. The rest of the day digs into eval reliability, sub-agent concurrency, trajectory steering, and durable actor infrastructure.
-
WED · OCT 7 A security benchmark runs 2,500 trials against six real coding agent harnesses and finds that flipping on auto-approve raises attack success from 29% to 96% — the convenience toggle is the attack surface. Alongside: a stateless architecture that owns research state in the harness to cut tokens by 84%, agent memory bootstrapped without oracles, a parallelization criterion priced in tokens, and more rogue-agent incidents on real platforms.
-
TUE · OCT 6 A new paper challenges the recent wins in agent failure prediction: it shows that AUROC scores of 0.85-0.94 mostly come from recognizing that some tasks are harder than others, not from spotting a run heading toward failure — a difficulty oracle hits 0.945 while staying at chance within a task. Alongside it, work on observing agents from outside via syscalls and network traffic, a harness-engine serving protocol, and more on long-horizon safety failures.
-
MON · OCT 5 A new paper on computer-use agents shows that the Dual-LLM pattern's security guarantees collapse in graphical environments: an attacker can steer an agent down a pre-approved branch without injecting any instruction, hitting 89.5% success — and a proposed defense drops that to zero. Another study finds prompt-injection detector rankings don't transfer between agent benchmarks at all. Elsewhere: action-speculation latency, self-managing context, and self-evolving harness optimizers.
-
SUN · OCT 4 A new paper tests whether an agent can improve just by writing explanations of its own attempts — no reward signal, no verifier — and matches GRPO on SWE-bench in half the updates. A security study finds that routing a harmful request through a tool call raises the model's effective refusal threshold, making tool-mediated agents easier to push into bad behavior. Also today: serving-infra work on live attention-layout switching and KV cache reuse, a memory-versus-documentation debate, and the case for hard budget caps.
-
SAT · OCT 3 A 584-run study takes one coding agent on one real ML task and finds that rerunning the same config varies more than switching agents entirely — making few-run comparisons meaningless and a handful of rule-breaking runs the top scorers. Elsewhere: a per-turn context compaction scheme with measured tradeoffs, an audit of whether tool-agent benchmarks actually test what they claim, a privacy-boundary memory architecture, and a local MCP memory recorder.
-
FRI · OCT 2 Three new papers argue the same uncomfortable thing: your agent's performance depends less on the model than on everything around it. One tests 66 model-harness pairings and finds rankings flip entirely depending on the scaffold; another shows about 54% of outcome variance comes from rerunning the same config, with the information you hand the agent mattering more than model size. Plus a model router with real cost numbers, sandbox-containment skepticism, and work on RL transfer, context compression, and stateful prediction.
-
THU · OCT 1 A new paper probes whether the action a human approves in Claude Code or Cursor is actually the one the harness runs, and finds six reproducible ways enforcement silently swaps A for A' after approval. Another measures how much agent capability survives once real-world hazards show up, with pass rates falling from 69.5% to 31.3%. Four more papers on agent security, transferable skills, and whether elaborate harnesses earn their keep.