Message Threads Slash Claude Code Latency, Plus Ollama Routing Pools and Skill Lockfiles

Today's findings

1. #1 Claude Code Message Threads Cut Long-Context Payloadstool

   Anthropic's new message threads send only new turns after the first request, cutting median time-to-first-token by about 35% on 500k-1M-token requests.

   Claude Code · message threads

   Send the conversation once — every later turn ships only new messages

   −35%

   median time-to-first-token

   on 500k–1M-token requests

   500k–1M

   tokens per long request

   1×

   full history sent, then new turns only

   beta

   header message-threads-2026-08-12

   Upgrade Claude Code or set the Messages API beta header; then measure TTFT and token usage.

   Why it matters: Long agent sessions waste bandwidth and latency by resending the full conversation every turn; threads directly help computer-use agents, big coding sessions, and token bills.

   How to apply: Upgrade Claude Code or use the Messages API beta header `message-threads-2026-08-12`; send the full conversation once, then send only new messages plus the previous response ID and measure TTFT and token usage.

   claudeagentscontextperformance

   Read more: Anthropic shipped message threads (beta): Claude Code now sends only new messages per turn, about 35% lower time to first token on 500k to 1M token requests

2. #2 Routerly 0.6.0 Adds Self-Hosted LLM Routing and Ollama Poolstool

   AGPL gateway Routerly routes Anthropic/OpenAI-compatible clients across cloud and Ollama models by cost, latency, or semantic intent while tracking tokens, latency, and spend.

   Routerly 0.6.0 · AGPL · self-hosted

   One gateway pools cloud and local LLMs

   Your SDK hits the hub; it picks a route by cost, latency, or intent — and logs tokens, latency, spend.

   Why it matters: Teams need per-app budgets, failover, and visibility instead of scattered provider keys; local Ollama models can sit in the same routing pool as hosted APIs.

   How to apply: Run Routerly in your infrastructure, point your SDK at the gateway, define router model pools including an Ollama connection, set per-app budgets, and use Usage records and the Test tab to compare routing behavior.

   local-llmollamaroutingobservability

   Read more: Routerly 0.6.0: a self-hosted LLM gateway for policy-based routing and usage tracking · Using an Ollama model in a self-hosted Routerly model pool

3. #3 APM Brings Lockfiles and Dependabot to Claude Code Skillstool

   Microsoft's Agent Package Manager versions skills, prompts, instructions, and MCP servers like project dependencies, now with Dependabot support.

   Microsoft · Agent Package Manager

   Agent assets become dependencies

   Ad hoc

   • Skills copied and edited freely
   • Behavior drifts silently
   • Teams run divergent agents

   → Silent drift becomes reviewable change

   Pinned

   • Lockfile pins versions
   • Dependabot opens update PRs
   • Behavior diffs reviewed pre-merge

   APM versions skills, prompts, instructions, and MCP servers like any project dependency.

   Why it matters: Agent behavior drifts when a shared skill or MCP server changes silently; lockfiles and update PRs make Claude Code workflows reproducible across a team.

   How to apply: Add an APM manifest and lockfile to the repo, pin Claude Code skills and MCP servers, let Dependabot open update PRs, and review behavior diffs before merging dependency bumps.

   claudemcpdependenciesreproducibility

   Read more: TIL you can manage Claude Code skills like project dependencies using APM

4. #4 CLAUDE.md Can Silently Shadow AGENTS.md in Claude Codetip

   When a repo has both files, Claude Code 2.1.312 loads only CLAUDE.md by default, so shared AGENTS.md rules stop applying.

   Why it matters: Mixed-agent repos can lose cross-tool instructions without any error, which is exactly the kind of silent failure that breaks agent discipline.

   How to apply: If you need AGENTS.md, symlink or import it into CLAUDE.md rather than keeping two instruction files, and add a session-start check with a unique marker to verify which file actually loaded.

   claudeagentsrepo-hygiene

   Read more: If your repo has both AGENTS.md and CLAUDE.md, Claude Code ignores AGENTS.md by default

5. #5 Semantic Cache Tuning: Static Cosine Thresholds Leave Hits on the Tabletechnique

   Replaying 90k support tweets, CacheVerifier beat the best static threshold by 5-13% more cache hits at the same error rate and held closer to a 2% error target.

   Semantic caching

   Loose errs, safe misses — the verifier takes both

   90k support-tweet replay: 5–13% more cache hits than the best static threshold at the same error rate, held closer to th

   Why it matters: RAG and support bots waste calls on repeated questions; default GPTCache thresholds were too loose, while safe Redis thresholds were too conservative.

   How to apply: Benchmark the hit decision separately from the embedding/cache layer, use a verifier or classifier on candidate hits, calibrate to an explicit error budget, and re-test as traffic shifts.

   ragcachingllmdevsevaluation

   Read more: CacheVerifier: we tested 4 semantic cache setups on 90k real customer support tweets

6. #6 CleanWeb Reader Shrinks Web Pages Before They Hit Your Vector Indextool

   New LlamaIndex reader strips navigation, cookie banners, scripts, and footers from web pages, claiming 87% token compression while preserving alt text.

   CleanWeb Reader · LlamaIndex

   A fetched web page is mostly boilerplate

   87%

   of page tokens stripped — nav, cookie banners, scripts, footers — before embedding

   Claimed compression; alt text preserved. Swap the web loader, then A/B token counts and chunk quality on your own docs a

   Why it matters: Web-grounded RAG chunks get polluted by boilerplate, which hurts retrieval quality and inflates embedding and inference costs.

   How to apply: Swap the standard web loader for `llama-index-readers-x402-cleanweb`, keep alt text, and compare chunk quality and token counts on your documentation and news-page sources.

   ragllamaindexweb-scrapingtokens

   Read more: [Release] CleanWeb Reader for LlamaIndex: Stop drowning your Vector Index in web boilerplate (87% Token Compression & Alt-Text Preservation)

7. #7 Scrapling MCP Adds 16 Scraping Tools to Claude Codetool

   Scrapling's v0.5 MCP server handled JS-heavy pages that WebFetch missed, but a test found 2 of 6 hidden prompts slipped past its filter.

   MCP attack surface

   Worth installing — but only behind these gates

   bounded capability

   Scrapling MCP: 16 browser-grade scraping tools (handles JS-heavy pages WebFetch missed)

   scope Allowlist domains

   limit Sandbox runtime

   monitor Log tool calls

   revoke Withhold credentials

   A test slipped 2 of 6 hidden prompts past its filter — red-team before broad network access.

   Why it matters: Agents need browser-grade scraping, but MCP servers expand the attack surface and can become a prompt-injection path.

   How to apply: Install Scrapling MCP for JS-heavy pages, sandbox it, allowlist domains, log tool calls, and red-team hidden prompts before giving it credentials or broad network access.

   mcpclaudescrapingsecurity

   Read more: Tested Scrapling MCP in Claude Code. It handled JS pages WebFetch couldn’t, but 2 of 6 hidden prompts got past its filter

8. #8 FastWindowAttention Kernel Targets Long-Context OOMs on Small GPUstechnique

   A sliding-window attention kernel with attention sinks and entropy decay runs linearly on a T4, avoiding SDPA OOM at 32k+ tokens and reporting up to 57x speedups at 200k.

   Why it matters: Long-context local inference is memory-bound; OOM at 32k blocks many agent and document workflows on modest hardware.

   How to apply: Inspect the code and benchmarks, try it as a drop-in attention path for long-context experiments, and validate output quality versus full attention on your own tasks from 10k to 200k tokens.

   attentionlong-contextlocal-llmcuda

   Read more: I built a custom attention kernel that is 57x faster than SDPA on 128k tokens. Here is the code and benchmarks. Where can I integrate this? · Standard PyTorch Attention OOMs at 32k tokens. My custom kernel survives and is 11x faster.

9. #9 Make Agents Write Checkable Acceptance Criteria Before They Worktechnique

   The done-means-checkable skill forces three steps: pre-write pass/fail criteria, run an independent judge, and report what remains unverified.

   Agent skill

   Three steps to a checkable 'done'

   1

   Write criteria

   Pass/fail, before any work

   Committed first — no post-hoc self-grading

   2

   Independent judge

   Not the author's grade

   3

   Report gaps

   List what stays unverified

   Turns a vague 'looks right' into pass/fail evidence a stranger could review.

   Why it matters: Agents grading their own analyses or plans hide errors; an independent check turns vague 'looks right' into reviewable evidence.

   How to apply: Use the skill for non-test tasks, require acceptance criteria a stranger could mark pass or fail, paste the judge output unedited, and list unresolved claims before accepting the result.

   agentspromptingevaluationworkflow

   Read more: done-means-checkable skill: write down what "right" means before it does work that has no tests

10. #10 Running Claude Code as a Parallel Agent Team With Worktreestechnique

    A shipped browser game used CLAUDE.md hard rules, tests enforcing them, parallel agents in git worktrees, and agents as QA to keep generated code reviewable.

    Parallel agent playbook

    How a parallel agent team avoids merge chaos

    1

    CLAUDE.md rules

    Non-negotiables, in writing

    2

    Rules as tests

    Enforced by tests and lints

    3

    Own worktree

    One isolated git tree per agent

    4

    QA agents

    Separate agents run the suite

    5

    Merge

    Only when green

    Red → fix in-tree

    Failing code loops inside its own worktree — only tested code reaches main.

    Why it matters: Parallel coding agents can collide; enforceable rules and isolated worktrees reduce merge chaos and review burden.

    How to apply: Put non-negotiable rules in CLAUDE.md, encode them as tests or lints, assign each agent its own git worktree, run separate QA agents, and merge only after tests pass.

    claudeagentsgittesting

    Read more: I built a Warcraft 3 style tower defence game with Claude Code. Here's my setup: hard rules in CLAUDE.md, parallel agents in worktrees, and agents as QA

11. #11 SpecPilot Automates Speculative-Decoding Search and Systems Tuningpaper

    The paper proposes autonomous algorithm discovery and systems optimization for speculative decoding, aiming to tune drafters and parallel drafting without manual sweeps.

    Why it matters: Speculative-decoding gains depend heavily on model and hardware pairings; automated tuning could make local inference faster without hand benchmarking every configuration.

    How to apply: Read it for the search strategy, replicate its benchmarks against your llama.cpp or vLLM setup, and use it as a checklist for drafter count, acceptance rules, and latency measurement.

    speculative-decodinginferencepapersoptimization

    Read more: [Paper] SpecPilot: Autonomous Algorithm Discovery and Systems Optimization for Speculative Decoding 🚀🚀🚀

12. #12 One-Line ComfyUI Fix Doubles Some Turing GPU Workflowstip

    On Nvidia 20-series cards, forcing fp16 instead of ComfyUI's fp32 fallback for Qwen-Image 2.1, FIBO2, and MOSS-Image can roughly double performance.

    Why it matters: Turing lacks bf16, so many image workflows silently run fp32 and waste time on older but still useful GPUs.

    How to apply: Apply ComfyUI PR #17522's commit or patch the model dtype declaration to fp16, then benchmark your workflow and compare outputs for quality regressions.

    comfyuilocal-llmgpufp16

    Read more: Nvidia 2000 series owners: double your performance of Qwen 2.1, Fibo 2.0, MOSS with 1 line edit. · Nvidia 2000 series owners: double your performance of Qwen 2.1, Fibo 2.0, MOSS with 1 line edit.