AI Agents

A production update on a twelve-card V100 SXM2 cluster reveals critical serving tradeoffs for local agents. The author abandoned vLLM for llama.cpp to support MoE GGUFs, noting dense models underperform on Volta architecture. NVLink topology dictates throughput; splitting models across PCIe-bound boards causes severe penalties. This isn't just hobbyist tinkering but a documented workflow for legal drafting via Claude Code, proving local clusters can handle specialized workloads if architecture aligns with hardware constraints.

An autonomous agent identified a valid macOS kernel vulnerability, assigned CVE-2026-28952, marking a shift from theoretical security agents to operational auditors. This isn't a sandbox demo; it's a high-credibility kernel flaw found in production-grade software. The discovery validates agent-based security scanning as a viable layer in CI/CD pipelines, provided guardrails prevent accidental exploitation during testing. It suggests security teams should prepare for agent-generated patch reports alongside traditional static analysis.

Community benchmarks identify Qwen3.6 35B A3B as the current leader for local agentic workflows, outperforming Gemma4 and GLM 4.7 on tool call stability. Testing focused on IQ4_NL quants via Unsloth, highlighting failure modes like infinite loops in competitors. While not perfect, the model sustains multi-step reasoning without collapsing, a common failure point for smaller local variants. This data helps architects select base models for offline agent deployments where API latency is unacceptable.

A 2016 Mac Pro with D700 GPUs demonstrates viable local inference using Vulkan drivers and llama.cpp, achieving usable tokens-per-second on Qwen 3.5 9B. The local model outperformed Claude Sonnet 4.6 on specific .NET planning tasks, leveraging external documentation retrieval effectively. This validates repurposing legacy enterprise hardware for low-latency planning agents where cloud round-trips introduce unacceptable overhead. It challenges the assumption that only modern silicon can support production-grade local inference.