Lean Compute Day: 1-Bit LLMs Hit 100 tok/s on CPUs, GGUF LoRAs, Test-Gated Agents

Today's findings

1. #1 BitRun: Standalone 1-Bit LLM Runner Hits 100+ tok/s on CPUstool

   A zero-compile runner for 1-bit ternary LLMs delivers 98-127 tok/s on an M4 Pro CPU and 40-45 tok/s on an Intel i5, with 0% GPU usage.

   BitRun · 1-bit inference

   1-bit LLMs at 100+ tok/s on a plain laptop CPU

   98–127

   tokens/sec on an M4 Pro — CPU only

   0% GPU

   40–45

   tok/s on an Intel i5

   0

   compile steps — runs BitNet ternary GGUFs directly

   1-bit

   ternary weights (−1 / 0 / +1)

   Standalone runner; also patches bitnet.cpp stride bugs that silently corrupted batched inference.

   Why it matters: 1-bit models are the cheapest path to running useful LLMs on laptops and edge boxes; BitRun removes the build pain and fixes kernel bugs in Microsoft's bitnet.cpp that silently corrupted batched inference.

   How to apply: Clone BitRun, point it at a BitNet-style ternary GGUF, and benchmark against your current llama.cpp setup on the same hardware; the stride-clobbering fix is worth porting upstream if you maintain your own kernels.

   local-llmquantizationinferenceapple-silicon

   Read more: I built BitRun: A standalone runner for 1-bit LLMs (100–127 t/s on Apple Silicon & 40–45 t/s on PC CPUs with zero compile) · I built BitRun: A standalone runner for 1-bit LLMs (100–127 t/s on Apple Silicon & 40–45 t/s on PC CPUs with zero compile)

2. #2 Train LoRA Directly on Quantized GGUFs, No PyTorch or FP16 Parenttechnique

   You can fine-tune a LoRA straight on a quantized GGUF file without the original FP16 weights or a PyTorch stack.

   Fine-tuning

   LoRA without the full-precision parent

   Classic LoRA path

   • FP16 parent in VRAM
   • PyTorch training stack
   • VRAM for model + adapter

   → Train on what you already ship

   GGUF-native LoRA

   • Quantized GGUF file
   • No PyTorch required
   • Adapter kept separate

   Works on a small Qwen or Llama GGUF; compare evals against a full-precision LoRA baseline.

   Why it matters: Most teams can't afford to hold a full-precision parent model in VRAM just to train an adapter; training on the quantized artifact you already ship collapses the memory and tooling requirements.

   How to apply: Try the workflow on a small Qwen or Llama GGUF you already run, keep the adapter separate, and compare eval scores against a full-precision LoRA baseline before rolling it into production.

   fine-tuningloragguflocal-llm

   Read more: Train LoRA directly on Quantized GGUFs, no PyTorch, no Parent(FP16) needed, just your GGUF and the data

3. #3 A Stop Hook That Forces Claude Code to Run Tests Before Saying Donetechnique

   Wire a Stop hook that runs your test command when Claude Code finishes a turn; exit code 2 feeds failures back and keeps it working.

   Claude Code · Stop Hook

   Stop hook turns 'done' into 'done only if the tests pass'

   1

   Turn ends

   Claude Code tries to stop on 'done'

   2

   Hook fires

   gate runs your test command

   3

   Tests fail

   exit 2 — failures fed back to the agent

   4

   Tests pass

   exit 0 — the stop is finally allowed

   keeps working

   Check stop_hook_active first to avoid an infinite loop, and point the hook at the same command your CI runs.

   Why it matters: The most common Claude Code failure mode is a confident 'done' on code that doesn't pass tests; moving the check into the harness makes correctness a gate instead of a hope.

   How to apply: Add the hook to your Claude Code settings, check stop_hook_active first to avoid infinite loops, and point it at the same test command your CI uses.

   claude-codetestinghooksworkflow

   Read more: A Stop hook that makes Claude Code run the tests before it says "done"

4. #4 llama.cpp Adds --spec-draft-n-ubatch to Tune Speculative Decoding VRAMtip

   A new llama.cpp flag lets you cap the micro-batch size for the draft model in speculative decoding, trading VRAM for throughput.

   LLAMA.CPP · SPECULATIVE DECODING

   One flag dials draft VRAM against draft speed

   --spec-draft-n-ubatch caps the draft model's micro-batch buffers; sweep values on your hardware.

   Why it matters: Speculative decoding is one of the biggest free speedups for local inference, but the draft model's batch buffers can blow your VRAM budget; this knob gives you direct control.

   How to apply: Pull the latest llama.cpp, add --spec-draft-n-ubatch to your launch script, and sweep values on your target hardware to find the point where draft throughput stops improving.

   llama-cppspeculative-decodingvramlocal-llm

   Read more: common : add --spec-draft-n-ubatch by ggerganov · Pull Request #30459 · ggml-org/llama.cpp

5. #5 Bind Human Approval to the Payload, Not the Prompttechnique

   Recompute a digest over the material fields at execution time, compare it to what the human approved, and consume the grant in one conditional update.

   Agent approvals

   Bind approval to the payload, not the prompt

   Prompt-bound

   • Approves a prose description
   • Executing action drifts from it
   • Approval UI is theater

   vs

   Payload-bound

   • Grant stores the payload hash
   • Digest recomputed at execution
   • Mismatch → reject, no re-prompt

   Compare and consume the grant in the same conditional update that runs the action.

   The approved digest is the contract — prose can drift.

   Why it matters: Agents routinely drift between the action a human approved and the action that actually executes; without payload binding, your approval UI is theater.

   How to apply: Store the approved payload hash alongside the grant, re-derive it inside the same transaction that performs the action, and reject on mismatch instead of re-prompting.

   agentssecurityapprovalsguardrails

   Read more: Human approval on an AI agent is not authorization: bind it to the payload

6. #6 Eval Suite Auditor: Catch Judges That Never Actually Rantechnique

   A skill.md that audits your existing eval suite after a run passed at 0.9 while its LLM judge had silently 404'd and returned a default 0.5.

   Why it matters: Dead judges and swallowed errors make eval scores look healthy while hiding regressions; auditing the suite is cheaper than rewriting it.

   How to apply: Run the auditor against your eval config, assert that every judge model is reachable and returns a non-default verdict, and fail the suite when a check is skipped rather than passed.

   evalstestingllm-judgeprompt-engineering

   Read more: Eval Suite Auditor skill.md: my eval passed at 0.9 while its judge had never executed

7. #7 ComfyUI v0.40.0 Ships Qwen-Image 2.1 Turbo and LoRA Stack Loaderstool

   The new release adds the 8-step Qwen-Image 2.1 Turbo checkpoint with text-to-image and edit templates, plus ordered multi-LoRA stack loaders.

   Why it matters: Qwen-Image 2.1 Turbo cuts generation to 8 steps, and stacked LoRA loaders remove a common source of workflow spaghetti for teams chaining style and subject adapters.

   How to apply: Update ComfyUI, pull the Qwen-Image 2.1 Turbo templates from the workflow repo, and replace chained single-LoRA nodes with the new stack loader in your existing graphs.

   comfyuiimage-genloraqwen

   Read more: ComfyUI v0.40.0 · ComfyUI v0.40.0 released · ComfyUI v0.40.0 released · ComfyUI v0.40.0 released

8. #8 Qwen-Image 2.1 Commercial Use Is Free Up to 300k Images a Monthtip

   Alibaba's licensing reply confirms a free commercial tier up to 300,000 images per calendar month, then 20% revenue share up to 3M, then $500k/year plus 15%.

   Why it matters: The Qwen Research License scared teams off commercial use; the actual terms cover almost every internal or small-shop workload without a paid agreement.

   How to apply: If you generate under 300k images/month, proceed under the Free-Tier Commercial License Addendum and keep a monthly count; above that, budget for the revenue-share tier before shipping.

   licensingqwenimage-gencommercial

   Read more: Asked Alibaba about commercial use of Qwen-Image 2.1 — up to 300k images/month is free. Here's what the agreement says. · Asked Alibaba about commercial use of Qwen-Image 2.1 — up to 300k images/month is free. Here's what the agreement says.

9. #9 Official MCP Servers List Passes 300 Maintained Projectsrepo

   A curated repo tracks 300+ official MCP servers from the companies behind the products, filtered to well-known and actively maintained projects.

   Why it matters: MCP is becoming the default tool interface for Claude and local agents, but the ecosystem is noisy; a vetted list saves hours of hunting for a server that actually works.

   How to apply: Bookmark the repo, wire the servers you need into your Claude Desktop or agent config, and send PRs when you find an official server that's missing.

   mcpagentstoolsopen-source

   Read more: Updated list of official MCP servers (300+), well-known and widely used projects only

10. #10 Liquid AI's LFM2.5-24B-A2B Targets Local Agent Workloadstool

    An open-weight 24B MoE reasoning model with 2B active parameters, positioned for local AI agents with published benchmarks and hardware requirements.

    architecture

    LFM2.5: 24B of knowledge, 2B awake per token

    router

    top-1 gate

    24B total — 12 × 2B shares2B active per token

    weighted

    merge

    24B

    total parameters

    2B

    active per token

    1 GPU

    consumer card — or a Mac

    Why it matters: A 2B-active MoE at 24B total is a sweet spot for local agent loops: enough reasoning capacity to plan, small enough to run on a single consumer GPU or a Mac.

    How to apply: Download the weights, run it through your existing llama.cpp or Ollama setup, and benchmark it against your current agent model on tool-calling and multi-step tasks.

    local-llmmoeagentsopen-weights

    Read more: Liquid AI Releases LFM2.5, a 24B Reasoning Model Built for Local AI Agents

11. #11 Your QLoRA Run Is Probably Data-Bound, Not GPU-Boundtip

    An H100 was only 15% faster than a 4070 for a 7B QLoRA run because the 25GB dataset was streaming from S3 on every epoch.

    Why it matters: Teams rent expensive GPUs to speed up fine-tuning and get a fraction of the expected speedup; the bottleneck is usually data loading, not FLOPs.

    How to apply: Before renting a bigger GPU, profile dataloader wait time, cache the tokenized dataset on local NVMe, and only then decide whether the compute upgrade is worth it.

    fine-tuningqloraperformancedata-loading

    Read more: h100 was 15% faster than my 4070 for qlora, not the 4-5x i expected

12. #12 Event2Vec: Additive Event Embeddings with Exact Per-Event Attributionpaper

    A small library for learning embeddings of discrete event sequences where a linear score over a history decomposes exactly into per-event terms.

    Event2Vec

    The score is exactly the sum of its events

    Each arrow is an exact per-event term — attribution built in, not post-hoc.

    Why it matters: Most sequence models give you a score but not a clean attribution; additive embeddings make it trivial to explain which past event drove a prediction, which matters for logs, purchases, and clinical histories.

    How to apply: Drop Event2Vec into a pipeline where you already have timestamped events, train on your history, and use the per-event decomposition for debugging and stakeholder explanations.

    embeddingsinterpretabilitysequencespaper

    Read more: Event2Vec: additive event embeddings where a linear score on a history decomposes exactly into per-event terms [P]