Qwen Speculative Decoding Lands in llama.cpp, Plus SQLite Memory and Agent SQL Guardrails

Today's findings

1. #1 Qwen MTP Support Merged into llama.cpptool

   llama.cpp merged MTP support for Qwen Flash Next, enabling faster speculative decoding for local Qwen models.

   LOCAL LLM TOOLING

   llama.cpp merges MTP support for Qwen Flash Next

   MTP for Qwen3.8-Flash-Next

   feature merged

   Multi-token prediction — faster decode, no new weightslatest llama.cpp revision

   runUpdate llama.cpp, pull the quants, enable MTP in your config

   Same weights, more tokens per second — bigger local Qwen models become practical for interactive coding and agents

   Why it matters: MTP can raise tokens per second without changing model weights, making larger local Qwen models more practical for interactive coding and agents.

   How to apply: Update llama.cpp to the merged revision, pull the Qwen3.8-Flash-Next GGUF quants, and enable MTP in your local inference config to benchmark decode speed.

   llama-cppqwenmtplocal-llm

   Read more: Qwen4Exp: add MTP by am17an · Pull Request #29761 · ggml-org/llama.cpp

2. #2 Hillock Gives Ollama SQLite-Backed Memorytool

   Hillock replaces vector DBs with SQLite and hypervector fact extraction, giving Ollama persistent memory in under 1.2GB VRAM.

   LOCAL LLM MEMORY

   Persistent memory without the vector DB

   Vector-DB RAG

   • Embeddings eat VRAM
   • Slows the main model
   • Extra store to run and query

   vs

   Hillock memory

   • Under 1.2GB VRAM in total
   • Sub-300MB bi-encoder extracts facts
   • One SQLite file holds everything
   • Queried before the LLM runs

   A light memory layer keeps Ollama responsive while document facts stick.

   Hillock swaps the vector DB for SQLite + hypervector fact extraction.

   Why it matters: Local RAG often consumes VRAM and slows the main model; a lightweight SQLite memory layer keeps the model responsive while retaining document facts.

   How to apply: Run Hillock alongside Ollama, feed it documents for sub-300MB bi-encoder fact extraction, and query the SQLite store before invoking the LLM.

   ragollamasqlitelocal-llm

   Read more: I built a local memory engine that replaces vector DBs with SQLite and runs in <1.2GB VRAM · Built an SQLite-based memory system for Ollama that doesn't eat all your VRAM

3. #3 Aegis Proxy Blocks Destructive Agent SQLtool

   Aegis is an open-source Go proxy that parses SQL in flight to stop AI agents from running destructive Postgres queries.

   AGENT GUARDRAILS

   Writes allowed, destruction denied

   bounded capability

   Agent SQL writes to Postgres

   scope Proxy on 5433

   monitor Parse in flight

   limit Allowed patterns

   revoke Deny destructive

   Agents point their connections at Aegis, not Postgres — benign statements pass, destructive SQL never lands.

   Why it matters: Agents often need write access, but a single hallucinated DROP TABLE can destroy production data; read-only users are too blunt an instrument.

   How to apply: Deploy Aegis on port 5433, configure allowed statement patterns, and point agent database connections at the proxy instead of Postgres directly.

   agentssecuritypostgresguardrails

   Read more: I built a beta Go proxy for my own projects to stop AI agents from running DROP TABLE

4. #4 M-Anchor Deterministic Gate for LLM Record Updatestool

   M-Anchor is a deterministic Python gate that checks LLM-proposed record updates against admitted evidence before saving.

   Why it matters: Prompt-injected or unsupported changes can corrupt stored records; a non-LLM gate adds a verifiable checkpoint between model output and database writes.

   How to apply: Run M-Anchor between your model's proposal and the record update, inspect its attack scenarios, and export logs for reproducible counterexamples.

   guardrailsprompt-injectionpythonsecurity

   Read more: A deterministic Python gate for LLM record updates · Can a prompt attack change the stored record? A demo with a deterministic gate

5. #5 Per-Ticket Sandboxes for Claude Codetechnique

   Drive Claude Code from ticket threads inside per-ticket sandboxes, so each approved ticket gets an isolated workspace and preview URL.

   Why it matters: Isolated branches, services, and databases reduce context bleed and make agent work reproducible without developers manually opening laptops.

   How to apply: Have an orchestrator watch approved tickets, provision a fresh branch plus backend/frontend/DB copy per ticket, and let the agent work only inside that sandbox.

   claude-codeagentssandboxworkflow

   Read more: We run Claude Code inside a per-ticket sandbox and drive it only from the ticket thread. Developers never open a laptop.

6. #6 Qwen3.8-Flash-Next 125B on Strix Halotechnique

   Qwen3.8-Flash-Next 125B runs at 43 tok/s on a Strix Halo mini-PC with 2x faster tool calls and low KL divergence.

   LOCAL LLM · STRIX HALO MINI-PC

   A 125B MoE model, running locally

   43 tok/s

   decode speed on a consumer mini-PC

   2× faster tool calls

   125B

   MoE parameters

   0 GPUs

   datacenter hardware needed

   Low KL

   divergence vs full precision

   Local agent and coding workloads are now feasible on desk-sized hardware.

   Why it matters: It shows a large MoE model can run on a consumer mini-PC, making local agent and coding workloads more feasible without a datacenter GPU.

   How to apply: Replicate the Strix Halo setup with similar quantization and runtime flags, then benchmark tool-calling and KL divergence against full precision.

   local-llmqwenstrix-haloquantization

   Read more: Qwen3.8-Flash-Next 125B running locally on a Strix Halo mini-PC: 43 tok/s, tool calls 2× faster, KL divergence 0.116 vs full precision. The model diagnosed and wrote one of the runtime fixes itself.

7. #7 InkDoc Offline Document-to-Markdowntool

   InkDoc converts messy PDFs and documents to clean Markdown fully offline, ready for local embeddings or context windows.

   Why it matters: Local RAG pipelines often leak private documents to cloud parsers or mangle tables; offline preprocessing keeps data local and improves retrieval quality.

   How to apply: Run InkDoc locally, use its drag-and-drop UI or localhost REST API, and feed the resulting Markdown into your embedding or context pipeline.

   raglocal-llmdocumentsoffline

   Read more: Stop sending private docs to cloud APIs just to get clean text for your local model - InkDoc runs fully offline

8. #8 Claude Code Handoff to macOS On-Device Modeltechnique

   Use macOS 27's on-device `fm` model to summarize large text before Claude Code reads it, cutting token use.

   Local-first context

   Summarize on the Mac, hand Claude only the gist

   1

   Raw text

   logs · transcripts · long docs

   2

   fm summarize

   macOS 27 on-device model

   runs on-device — raw content never leaves the Mac

   3

   Summary

   only this goes upstream

   4

   Claude Code

   reads the distilled result

   Setup: `sudo fm license`, then a Claude Code skill pipes large inputs through the local model.

   Why it matters: Summarizing transcripts, logs, and long documents locally reduces token spend and keeps sensitive raw text on the Mac.

   How to apply: Enable `fm` with `sudo fm license`, build a Claude Code skill that pipes large inputs through the local model, and pass only the summary to Claude.

   claude-codelocal-llmmacostokens

   Read more: I had Claude Code hand off big summarizing jobs to the on-device model in macOS 27

9. #9 Prompt Injection as Information Flowtechnique

   Stop asking only whether content is malicious; track whether untrusted text can influence privileged parameters or tool calls.

   Why it matters: Allowlists miss sequences where safe content steers a privileged action, such as choosing an email recipient or supplying a database argument.

   How to apply: Add taint tracking from untrusted sources to privileged parameters, separate read and write tools, and require provenance before tool arguments are used.

   agentssecurityprompt-injectionguardrails

   Read more: Prompt injection stopped being a content problem and most agent stacks havent caught

10. #10 Roll Back Agent Changes as One Bundletechnique

    Treat prompt, tool schema, and model version as one release bundle so rollbacks restore a known-good agent state.

    Why it matters: Agent regressions are hard to attribute when multiple components change together; partial rollbacks can break compatibility or leave tuned paths behind.

    How to apply: Version all three artifacts together, keep the previous bundle in shadow for live comparison, and roll back the full bundle when metrics drop.

    agentsmlopsrollbackversioning

    Read more: How do you roll back an agent change when the prompt, tool schema and model version all shipped together?

11. #11 NVIDIA Kumo Tabular Open Foundation Modelstool

    NVIDIA released Kumo Tabular, an open tabular foundation model that predicts new rows in a single forward pass and tops TabArena.

    Tabular foundation models

    From per-dataset training to one forward pass

    Classic GBDT pipeline

    • Engineer features by hand
    • Fit a model per table
    • Retune for each task

    → New rows predicted in a single forward pass

    Kumo Tabular

    • Open weights
    • No per-dataset training
    • Predict rows in one pass

    Open and self-hostable — tops TabArena

    Why it matters: Tabular data is common in production, and an open foundation model can be self-hosted and evaluated without proprietary APIs.

    How to apply: Benchmark Kumo Tabular on your own tabular datasets for row prediction, imputation, or feature generation, and compare against your current gradient-boosted baseline.

    tabularopen-weightsfoundation-modelsnvidia

    Read more: NVIDIA Releases Kumo Tabular: Open Tabular Foundation Models That Predict New Rows in a Single Forward Pass

12. #12 Multi-Agent Swarms Can Converge on False Beliefspaper

    A new paper examines how multi-agent swarms can converge on false beliefs, with implications for agent-team design.

    Why it matters: Multi-agent systems can amplify errors through consensus; teams need diversity, independent verification, and monitoring for false agreement.

    How to apply: Add independent verifier agents, avoid homogeneous model/prompt configurations, and track consensus confidence separately from ground-truth checks.

    multi-agentswarmsfalse-beliefsresearch

    Read more: "Extraordinary Multi-Agent Delusions and the Madness of Crowds" (how do swarms converge on false beliefs?)