Glossaries, Ground-Truth-Free Training, and Quantization You Can Actually Trust

Today's findings

1. #1 Give Claude Code a Glossary to Cut Verbositytip

   A shared glossary of project jargon turns bloated Claude Code explanations into one-word references.

   Claude Code

   Give Claude Code a Glossary to Cut Verbosity

   Without glossary

   • Re-explains jargon every session
   • Tokens burned on derivations
   • Context reset per session

   → Add glossary to CLAUDE.md

   With glossary

   • One-word jargon references
   • Tokens preserved for reasoning
   • Single glossary, many sessions

   A shared glossary of project jargon lets Claude reference recurring terms instead of re-deriving explanations.

   Why it matters: Verbose agent output isn't a model quirk — it's missing context, and re-explaining domain terms every session burns tokens and time across a team.

   How to apply: Add a glossary section to CLAUDE.md or a skills file mapping recurring domain terms to short canonical names (per mattpocock/skills' "Agent Is Way Too Verbose" pattern) so Claude references them instead of re-deriving explanations.

   claude-codepromptingagents

   Read more: The verbosity was not the model. I had never told it what my words mean.

2. #2 Session-to-Session Memory for Claude Code via Hookstechnique

   A SessionStart hook plus a /brain-update command gives Claude Code persistent memory of decisions and failures across sessions.

   Why it matters: Eliminates the "re-explain everything" tax at the start of every Claude Code session, a common daily friction point on long-lived projects.

   How to apply: Build a slash command that extracts progress, decisions, and failures from the session transcript into local Markdown plus JSON at session end, and a SessionStart hook that turns those records into a pre-session briefing.

   claude-codeagentsmemory

   Read more: Gave Claude Code memory between sessions.

3. #3 Serving Engine Choice Can Swing Benchmarks More Than Hardwaretip

   On identical GB300 hardware, switching from TensorRT-LLM to SGLang changed the measured tokens-per-megawatt result by roughly 11x at a fixed interactivity target.

   Serving Stack Impact

   Engine Choice Swings Throughput More Than Hardware

   11x

   tokens-per-megawatt variance

   TensorRT-LLM → SGLang on GB300

   tokens/MW

   metric

   same hardware, same latency

   controlled

   When benchmarking, test your target engine at your real latency target—engine choice can matter more than GPU generation

   Why it matters: Hardware comparisons that don't control for serving engine are misleading — engine choice can matter more than GPU generation for real throughput and cost.

   How to apply: When benchmarking or provisioning self-hosted inference, test your actual target engine (SGLang, vLLM, TensorRT-LLM) at your real interactivity/latency target instead of trusting vendor numbers measured on a different stack.

   inferencebenchmarkingsglang

   Read more: Same GB300, same workload: the serving engine moved the benchmark result by 11x

4. #4 Perplexity Overstates How Well Q4_K_M Quantization Recovers Qualitytechnique

   Measuring the full output probability distribution instead of just perplexity shows Q4_K_M recovers about 67% of lost quality, not the roughly 89% perplexity implies.

   Model Quantization

   Perplexity Overstates Q4_K_M Quality Recovery

   vs

   Perplexity

   Full Distribution

   Recovery Rate

   ~89%

   ~67%

   Measures

   1 token prob

   Vocab shift

   Verdict

   Optimistic

   Realistic

   Perplexity wins the row Full Distribution wins the row

   Perplexity ignores full distribution shift; Q4_K_M recovery appears 22 points better by this metric than by KL divergenc

   Why it matters: Perplexity only scores the probability assigned to the actual next token and ignores how the rest of the distribution shifts, making quantized models look closer to full precision than they behave in practice.

   How to apply: When validating a quantization scheme, compare full next-token distributions (e.g., KL divergence across the whole vocab) against a float reference rather than relying on perplexity alone.

   quantizationevaluationllama.cpp

   Read more: Perplexity says Q4_K_M repairs 89% of the damage. Measured on the full output distribution, it repairs 67%.

5. #5 Voodoo Dynamic Quant Method Goes MIT-Licensedtool

   A previously private dynamic-quantization method claiming SOTA results at aggressive quant levels on small Qwen models is now open and MIT-licensed.

   Quantization Trade-off

   Dynamic Quantization Preserves Quality at Aggressive Levels

   Standard Quantization

   • Severe quality loss at Q2/Q3
   • Limits aggressive VRAM optimization

   vs

   Voodoo Dynamic Quant

   • Maintains quality at Q2/Q3 levels
   • SOTA results on small models
   • MIT-licensed, now open-source

   Dynamic quantization enables aggressive memory optimization without the usual quality collapse.

   Voodoo's dynamic-quant method now available for GGUF production workflows.

   Why it matters: Gives teams producing their own GGUF quants a new open technique for squeezing models into tight VRAM without the usual quality collapse at aggressive quant levels.

   How to apply: Pull the released methodology and apply it when producing dynamic quants for small-to-mid size open models, especially at Q2/Q3 where standard quantization degrades badly.

   quantizationggufopen-source

   Read more: Voodoo Dynamic Quant - Now MIT Licensed

6. #6 TTPO: Test-Time Training Without Labels via Vote Disagreementpaper

   A new method (arXiv 2608.27448) trains on disagreement-with-majority-vote rather than the vote itself, avoiding the standard failure where a wrong pseudo-label poisons the whole update.

   Why it matters: Majority-vote self-distillation is a common way to adapt models at test time without ground truth, but it silently reinforces wrong answers when the vote is wrong — this asymmetry-based fix is a cheap correction.

   How to apply: If you're doing test-time or on-policy self-distillation with majority-vote pseudo-labels, weight or select rollouts by disagreement-with-vote rather than treating agreement as automatically correct.

   fine-tuningtest-time-trainingpaper

   Read more: TTPO uses disagreement with the majority vote as the training signal, so test-time training works without labels

7. #7 Open-Source Runtime Beats Vendor rkllm on Rockchip RK3576tool

   An open runtime outperforms Rockchip's proprietary rkllm on RK3576 hardware: 1.4x decode speed, better time-to-first-token up to 602 tokens, and half the quantization loss.

   Why it matters: Edge/embedded LLM deployment on Rockchip NPUs has been stuck with closed vendor tooling; a faster, more accurate open alternative removes a real blocker for on-device projects.

   How to apply: If deploying LLMs on RK3576/RK3588 boards, benchmark this open runtime against rkllm for your workload before committing to the vendor SDK.

   edge-inferencequantizationnpu

   Read more: Open runtime on mainline rocket beats rkllm on the RK3576: 1.4x decode, ahead on TTFT to 602 tokens, half the quantisation loss

8. #8 Stress-Testing Your Own LLM-Judge Eval Harnesstechnique

   Applying Dan Luu's "what's wrong with this benchmark?" exercise to a homegrown LLM-judge eval pipeline surfaced four real defects, including judge non-determinism swinging disagreement rates 2/21 to 6/21 on identical inputs.

   Why it matters: Teams building LLM-as-judge evals often trust their own harness by default; this is a concrete, repeatable way to find where your eval's noise floor is wrong.

   How to apply: Run 16+ repeated judge scores on the same fixed cases to measure judge noise before treating any single run's disagreement rate as ground truth, and re-check thresholds whenever the eval suite grows.

   evaluationllm-judgesagents

   Read more: I applied Dan Luu's "what's wrong with this benchmark?" reading to my own two LLM judges. Five exercises.

9. #9 Verifying a GraphRAG Subgraph Actually Supports Its Answertool

   Open Ontologies is an open-source system that checks GraphRAG retrieval at the level of individual claims, not just subgraph coverage, to catch a retrieved slice that drops the one premise an answer depends on.

   Why it matters: Standard GraphRAG eval (coverage, accuracy, citation quality) can pass even when the retrieved subgraph doesn't logically support the answer — a real gap for anyone shipping GraphRAG-backed answers.

   How to apply: Add claim-level support-checking to your GraphRAG eval pipeline instead of relying on coverage metrics, or try Open Ontologies directly if you already build on a knowledge graph.

   raggraphragopen-source

   Read more: GraphRAG retrieves a plausible subgraph—but can it prove that the subgraph supports the answer?

10. #10 Detect MCP Server Capability Drift on Reconnecttip

    Snapshot an MCP server's tool names, descriptions, and schemas at approval time and diff them on every reconnect to catch new or widened write access before it's silently granted.

    MCP Security

    Detect MCP Server Capability Drift on Reconnect

    bounded capability

    Reconnected MCP server

    scope approved tool manifest

    monitor diff on reconnect

    limit re-approve new or widened tools

    Trusted servers that gain write access or widen schemas need re-authorization, not silent inheritance.

    Why it matters: MCP servers you already trust can add write tools or widen input schemas in an update; without diffing, that expanded capability inherits the same blanket trust as the original read-only approval.

    How to apply: Store an approved tool manifest per MCP server/profile, diff it on reconnect, auto-surface additive read-only changes, and require re-approval for new or widened state-changing tools.

    mcpsecurityagents

    Read more: How do you detect capability drift when an MCP server updates its tools?

11. #11 vramble: A Priority-Queue VRAM Scheduler for Local Model Swappingtool

    An open-source dispatch layer automatically loads and unloads local models by priority so multiple workflows can share one GPU without manual juggling.

    Why it matters: Running several local models or tools on one workstation usually means each one grabs all available VRAM; a scheduler removes the manual babysitting.

    How to apply: Put vramble in front of local model servers if you run multiple Ollama/llama.cpp workloads on shared hardware and want automatic load/unload by priority instead of manual process management.

    local-llmtoolingollama

    Read more: Built a VRAM scheduler (vramble) to automate model swapping on my local AI workstation

12. #12 ar-tokenwise Cuts the Arabic "Tokenizer Tax" on Claude API Callstool

    An open-source Python library preprocesses Arabic text to reduce token counts (and cost) before sending it to the Claude API.

    Why it matters: Non-Latin-script languages routinely tokenize far less efficiently than English, so this directly cuts API spend for any product handling Arabic input or output.

    How to apply: If your product sends Arabic text through the Claude API, pipe it through ar-tokenwise's preprocessing first, or use the included SKILL.md directly inside the model.

    tokenizationclaude-apiopen-source

    Read more: Built ar-tokenwise: An open-source Python library to beat the Arabic "tokenizer tax" on Claude, GPT, and Gemini. Would love your feedback!