2026-09-10
AI Agents
A study of 3,171 public coding-agent setups goes looking for supply-chain defects in the config layer — instruction files, MCP declarations, skills, hooks — and finds 16% carry a validated security exposure, from unpinned MCP servers to skills that pre-approve the shell for whoever installs them. Alongside: a multimodal prompt-injection benchmark, two takes on knowing when agents succeed or fail, and tooling for auditing agent permissions.
Papers
5Scanning the Harness: An Empirical Study of Supply-Chain Defects in AI Coding-Agent Configurations
This studies the coding-agent config layer as a real dependency surface: 3,171 repos of instruction files, skills, hooks, MCP declarations, and subagents installed with developer privileges, no lockfile, no install-time check. Findings are validated three ways before counting. 9.8% of setups install an MCP server with no version pinned, 3.1% pre-approve arbitrary execution behind scoped-looking grants like
Bash(python:*), and 3.8% ship a skill that pre-approves the shell for anyone who installs it — 16% carry a confirmed security defect. Rigorous methodology, uncomfortable numbers.An Experimental Evaluation of Multimodal Prompt Injection Attacks on Agentic AI Frameworks
MMPIBench delivers prompt-injection attacks through six visual carriers — OCR text, overlays, EXIF, QR codes, fake interfaces, hybrids — and traces each injected instruction from perception through planning to tool call. Across 720 runs, attacks are attempted in 12.8% of cases but complete in only ~1%, with the gap closed almost entirely at planning. The model, not the framework, decides whether injection lands. Audio is the weak channel: where it's delivered, completion hits 49%, and 75% for one model.
Do Agents Know When They Succeed? Calibrating Agent Confidence from Internal Representations
This asks whether an agent's internal representations predict eventual task success better than surface signals. Two methods: Latent Trajectory Dynamics summarizes residual-stream changes across a trajectory, and the Action Representation Probe predicts success from representations at action-decision points. Across Bash, SQL, and Python benchmarks and three model families (Qwen14B/7B, DeepSeek6.7B), both beat generation- and sequence-based calibration baselines with no prompt changes or multi-sample rollouts — a zero-overhead reliability monitor. Open-weight models only, since it needs residual-stream access.
JarvisGUI: Towards Cross-Device GUI Agents with Dynamic Task Composition
JarvisGUI benchmarks GUI agents on workflows that span Android, Windows, and Ubuntu, formulating tasks as typed input-output transformations so multi-step cross-device workflows compose automatically. The type system is the clever bit — it lets the benchmark generate and score coordinated tasks dynamically rather than scripting each one. State-of-the-art open-source agents struggle with state-transfer awareness, cross-platform reasoning, and long-horizon dependencies, a gap single-device benchmarks hide entirely. A good corrective to over-optimistic GUI-agent readiness claims.
TRACE: Training Reasoning Agents for Causal Exploration with Synthesized Rewards
TRACE tackles diagnostic reasoning where ground truth is expensive by engineering verifiability: sample an intervention, inject it into a simulator, generate the observations, and use the hidden intervention as an oracle reward while the agent investigates noisy, confounded evidence. In a digital-advertising environment with 12 root causes, RL with synthesized rewards lifts Qwen3.5-35B from 0.159 to 0.757 FullAttr@1 — past Claude Opus 5's 0.686 and a larger prompted Qwen — while using fewer tool calls. The simulator-as-oracle trick generalizes to any domain you can simulate.
News
2Show HN: Geiger – See every AI agent on your machine and what it can touch
Geiger inventories the AI agents running on a machine and shows what each one can touch — files, network, credentials. The premise is the right one: most teams have no idea how many agent processes are live or what scope they hold. Early Show HN (45 points), so treat the maturity accordingly, but the category — a permissions map for local agents — is exactly what's missing as coding agents and background workers multiply.
Show HN: Self-hosted company OS, Claude Code and Codex agents in departments
OtoDock is a self-hosted multi-tenant 'company OS' where each agent runs Claude Code or Codex CLI as a persistent process in a bubblewrap kernel sandbox with always-on network isolation via pasta, backed by its own workspace, memory, and schedules. Agents can also run identically on remote machines over a single outbound WebSocket — no inbound ports. The isolation model is the interesting part; the built-in Twilio/office-file features read more as demo surface than the reason to look.
Blogs
1GPT-6 Astra, Looped Transformers, and Hidden Reasoning
Raschka walks through recurrent-depth architectures and looped transformer blocks — models that iterate the same layers to reason before emitting tokens, keeping the chain of thought in latent space rather than text. It's a clear survey of where hidden-reasoning research sits, tied to the GPT-6 Astra rumor. Useful if you care about where inference compute is heading, though it's architecture pedagogy, not anything you'll wire into an agent loop this quarter.
35 more items the ranker flagged but didn't feature
Papers
- Strangers to Themselves: What Language Models Say About Themselves Is Genericarxiv:arxiv-agents
- Safe to Stop? Risk-Constrained Stopping for Sequential Clinical Diagnosis Agentsarxiv:arxiv-agents
- AgentAudit: An Open, Extensible Framework for Full-Lifecycle Trust Evaluation of AI Agentsarxiv:arxiv-agents
- LatentMD: Benchmarking Markdown Boundary Failures in LLM-Generated Textarxiv:arxiv-software-eng
- Quality Metrics for LLM-Generated Asset Administration Shells: A Perturbation-Based Evaluation Approacharxiv:arxiv-software-eng
- Multi-Agent Agentic Graph Learning via Structural Signaturesarxiv:arxiv-agents
- TrajMark: Ownership Attribution and Segment-Level Tamper Localization for Coding-Agent Trajectoriesarxiv:arxiv-software-eng
- GANDR: Claim Auditing for Verifiable Legal Answer Generationarxiv:arxiv-agents
- Beyond Training: A Feasibility Taxonomy for Inference-Time AI Governancearxiv:arxiv-agents
- CityPlanner: A Sandbox Agent for Executable Urban Planningarxiv:arxiv-agents
- Direct Diversity Optimization for Diverse Successful Trajectories in Preference Post-Trainingarxiv:arxiv-agents
- Keep Evaluation Fair: Detecting Data Leakage in Code Generation Benchmarks via Membership Inference Attacksarxiv:arxiv-software-eng
- Procedural Memory Under Change: Reuse and Interference in Controlled Web Tasksarxiv:arxiv-agents
- IBIB: A Protocol for Measuring Enterprise AI Systems by Serving Route, Not Model Identifierarxiv:arxiv-agents
- Show-Harness: Just a VLM Agent Can Play Robotsarxiv:arxiv-agents
- RobustSGPO: Search-Space Control for Agent Harness Evolutionarxiv:arxiv-agents
- BRACE: Anchored Bellman-Residual Correction for Stale Critics in Asynchronous RLarxiv:arxiv-agents
- Introducing Consort: A Spec-First Agent Framework for Enforced, Test-Driven Development on Live Database Branchesarxiv:arxiv-agents
- Pairit: A Platform for Live Experiments on Human-AI Collaborationarxiv:arxiv-agents
- RAP: Research Attention Prediction Reveals Target-Conditioned Evidence Acquisition Biasesarxiv:arxiv-agents
- A-JIT: Agentic Just-In-Time Software Constructionarxiv:arxiv-agents
- HLSFactory-Agent: Large-Scale Agentic HLS Dataset Construction from Academic and Open-Source Projectsarxiv:arxiv-software-eng
- Can AI Agents Detect and Repair Artifact Drift in Network Experiments?arxiv:arxiv-agents
- Smart Adaptive Computing Across the Continuum: LLMs in IoT-Edge-Cloud Resource Managementarxiv:arxiv-agents
- LexAgentHallu: A Hierarchical Benchmark for Profiling Hallucinations in Legal Agentsarxiv:arxiv-agents
- The Era by Eon Benchmark: A Generated Enterprise Estate with Exact Ground Truth for Benchmarking LLM Agentsarxiv:arxiv-agents
- Retrofitting Code Using LLMs to Support Exceptional Behaviorarxiv:arxiv-software-eng
- Towards Scalable and Cost-Efficient Vulnerability Detection: A Study on Automatic Query Generationarxiv:arxiv-software-eng
- An Autonomous GeoAI Agent for Arctic Eco-Navigationarxiv:arxiv-agents
News
Blogs
- .blend URL Viewerrss:simonw
- Quoting Calif Researchrss:simonw