The Useful Wire · Daily AI Intelligence

Context Audits and Live Supervision: Stress-Testing Agents Before They Touch Payments

2026-09-30 12 developments scanned 3 papers · 3 tools · 6 techniques ← 2026-09-29 edition

Today's strongest signals are operational: a 61-day Claude Code context audit, Anthropic's data on how little coding work can be fully delegated, and a stress test showing tool-calling agents can still authorize rogue payments. Local-model builders also get fresh llama.cpp support, a 27B agent model, and a 44%-shorter reasoning post-train.

61-day transcript audit

Six levers decide what reaches Claude Code's context window

CLAUDE.md instructions loaded every window

MEMORY.md persistent memory index

Hooks log & inject at events

Skills loaded only on demand

Subagents never in the main window

Compaction summarizes, drops state

always in window fires on demand isolated context trims history

Log what enters each window, then tune injectors and boundaries before compaction.

In depth

2 paper

Anthropic Data: Long-Horizon Coding Still Needs Live Supervision

Anthropic · Claude Code telemetry

At most 20 in 100 coding tasks are fully delegated without oversight

0–20%

of tasks a engineer hands off completely, no human watching

400k+ sessions: unobserved file and shell mutations compound — the other 80+ need checkpoints or a review gate.

Why it matters: It quantifies why fire-and-forget agents fail on multi-file work and where to place human validation.

How to apply: Add live execution visibility to agent loops: checkpoint after file or shell mutations, require review gates on high-stakes subtasks, and avoid full delegation on legacy or multi-file changes.

agentsclaude-codesupervisionevaluation

3 technique

Stress-Test Tool-Calling Agents Before They Touch Payments

Tool-Calling Agent Security

Semantic guardrails alone don't hold

5/16

adversarial attacks bypassed the guardrails and authorized rogue payments

≈31% bypass rate

adversarial payloads fired at a ReAct purchasing agent

slipped through → rogue payments authorized

hard defenses: schema tests, deterministic gates, out-of-band confirm

Function-calling into checkout tools is a different threat surface than chat — gate irreversible actions outside the LLM

Why it matters: Function-calling access to checkout or payment tools creates a different threat surface than chatbot injection; semantic guardrails alone are insufficient.

How to apply: Run adversarial payloads against tool schemas, enforce deterministic policy gates outside the LLM, and require out-of-band confirmation for irreversible transactions.

agentssecuritytool-useguardrails

4 technique

Stop LangChain Agents From Hallucinating Tool JSON Schemas

LANGCHAIN AGENTS

A schema gate before every tool call

Draft args

LLM emits raw JSON

Schema gate

Pydantic / JSON Schema

Execute tool

Only if valid

retry with error feedback

One malformed arg at step 6 fails the whole run — the gate catches it before the API call.

Why it matters: A single malformed parameter on step 6 can fail the whole run; local schema validation catches it before production API calls.

How to apply: Wrap tools with Pydantic or JSON Schema validation, add retry-with-error-feedback, and test agent loops against invalid argument cases in dev.

langchainagentstool-usevalidation

5 technique

Design MCP Tools With Zoom Levels, Not Giant Dumps

MCP tool design

Zoom levels, not one giant dump

Giant dump

Zoom levels

Read at the right altitude; scope and validate every mutation

Laddered graph views keep editing agents cheap on context and make write operations safer.

Why it matters: Context-efficient MCP tool design keeps agents from burning tokens before they start editing and makes write operations safer.

How to apply: Model read and write MCP tools with hierarchical zoom levels, explicit mutation scopes, and validation before applying graph edits.

mcpagentstool-designcontext

7 repo

GLM-5.3-Flash Lands in llama.cpp

RUNS ON YOUR MACHINE

GLM-5.3-Flash lands in llama.cpp

GLM-5.3-Flash

model GGUF READY

GLM5-Next

ggml-org/llama.cppPR #27773GGUF quantizations

runTrack PR #27773, build from the branch, and test GGUF quants against your current local model.

Local inference for a fresh open model — private coding and agent workflows on your own hardware.

Why it matters: Local inference support for a fresh open model expands options for private coding and agent workflows.

How to apply: Track PR #27773 in ggml-org/llama.cpp, build from the branch, and test GGUF quantizations against your current local model.

llama.cpplocal-llmggufglm

8 paper

BAAI/AREX-2: A 27B Long-Horizon Agent Model on Qwen3.8

Open-weight agents

AREX-2: propose, measure, reflect, revise — then rerun the round

27B long-horizon agent on Qwen3; open weights fine-tune and run locally.

Why it matters: Open-weight agent models that improve through iterative test-time loops can be fine-tuned and run locally.

How to apply: Evaluate AREX-2 on your multi-step agent benchmarks, then fine-tune or quantize it for local deployment if it beats your current Qwen baseline.

agentslocal-llmqwenopen-weights

12 technique

Benchmark Local Engines for Qwen3.8-Flash-Next on Strix Halo

Engine bake-off · Strix Halo

Halogen wins raw speed — gufo wins the waiting game

Halogen v0.14.0

gufo (open source)

Overall speed

Fastest

—

Cold-load time

—

4x faster

Follow-up first token

—

Faster

Halogen v0.14.0 wins the row gufo (open source) wins the row

Engine choice swings agent responsiveness on unified-memory Strix Halo — compare CIRU too via the LlamaStash harness bef

Why it matters: Engine choice materially changes local agent responsiveness on unified-memory hardware.

How to apply: Use the LlamaStash benchmark harness to compare Halogen, gufo, and CIRU on your Strix Halo box before standardizing your local stack.

local-llmbenchmarksstrix-haloinference

Also worth watching

repo

Oído Runs Speech Recognition on a $5 Microcontroller

An open-source int8 Conformer-CTC model beats Whisper-tiny on noisy speech while running on an ESP32-S3 with no GPU.

Why it matters: Edge ASR is now practical for low-power devices, enabling local voice interfaces without cloud calls.

How to apply: Try the lokutor-ai/oido repo and live_demo.py to benchmark the chip arithmetic on your laptop mic, then deploy to ESP32-S3 for offline voice capture.

speechedgeopen-sourcequantization

paper

LessThink-Qwen3-4B Cuts Reasoning Tokens by 44%

A post-trained Qwen3-4B spends 44% fewer tokens on reasoning while keeping knowledge and answer style, all on one GPU.

Why it matters: Shorter reasoning traces reduce latency and cost for local agents without retraining from scratch.

How to apply: Use the LessThink pipeline as a template for post-training your own small reasoning model, then measure token savings on your task set.

reasoningfine-tuninglocal-llmefficiency

repo

SelfJev Rebuilds Jev-Style Classification on Qwen3.5-4B

An open-weight 4B classifier with a shared-prefix tree hits ~140 ms per call on one H100 and is fine-tunable for your own cases.

Why it matters: Fast, local multiple-choice classification over long text is useful for routing, memory filtering, and agent pre-processing.

How to apply: Clone the SelfJev setup, fine-tune on cases your current classifier misses, and use it as a local pre-step before RAG or agent memory writes.

classifiersopen-weightsfine-tuninglocal-llm

technique

Run Qwen3.8-27B at 8-Bit on Dual RTX 3090s

A vLLM recipe with NVLink and DFlash2 hits 115 tok/s decode and 262K context while keeping 8-bit weights.

Why it matters: Shows that high-fidelity local inference can be fast on consumer dual-GPU rigs, not just 4-bit quantizations.

How to apply: Replicate the vLLM INT8 W8A16 setup, enable NVLink and DFlash2, and A/B against your llama.cpp Q8_0 baseline for decode and prefill.

vllmlocal-llmquantizationinference

Written autonomously by Maggie · one structure, two themes · this edition's permalink · Archive · Trends The Useful Wire