AWS Open-Sources a Pointer-Head Decision Model, Plus Frugal Hallucination Checks and Qwen3.8 Flash Quants

Today's findings

1. #1 Strands Decider 2B: AWS Open-Sources a 1.9B Pointer-Head Decision Modelpaper

   AWS released a 1.9B decision model that swaps the LM head for a pointer head, hitting 115 ms median on a 3090 under Apache-2.0 with a full training recipe.

   Strands Decider · AWS

   A tiny model built to decide, not to write

   115 ms

   median decision latency, single RTX 3090

   replaces an expensive LLM call

   1.9B

   params, pointer head

   3 tasks

   routing · tool choice · guardrails

   Apache-2.0

   open weights + training recipe

   Fine-tune on your own routing labels and slot it in front of the agent loop.

   Why it matters: Cheap, fast decision-making is the bottleneck in agent routing, tool selection, and guardrails; a dedicated small model can replace an expensive LLM call for these steps.

   How to apply: Pull the weights and training recipe, fine-tune on your own routing or classification labels, and slot it in front of your agent loop as a fast decider.

   agentsdecision-modelsopen-weightsinference

   Read more: Strands Decider 2B: AWS open-sourced a 1.9B "decision model" that drops the LM head for a pointer head. 115 ms median on a 3090, Apache-2.0, full training recipe included

2. #2 Detecting Local-Model Hallucinations Without Burning VRAMtechnique

   A practical alternative to Semantic Entropy flags hallucinations by sampling fewer responses and clustering meanings with a small NLI cross-encoder, avoiding 45 pairwise comparisons.

   Hallucination detection

   Catch hallucinations without doubling VRAM

   vs

   Pairwise semantic entropy

   Clustered NLI detector

   NLI calls

   45 pairwise

   cluster meanings

   Judge size

   heavy judge model

   small DeBERTa

   VRAM + latency

   2× cost

   kept light

   Local deploy

   strained

   viable

   Pairwise semantic entropy wins the row Clustered NLI detector wins the row

   Recipe: sample K answers at temp 0.7, cluster, threshold on entropy — tested across 1.5B–120B.

   Why it matters: Running a heavy judge model to catch hallucinations doubles your VRAM and latency; a lightweight detector keeps local Ollama deployments viable in production.

   How to apply: Sample K responses at temperature 0.7, cluster with a small DeBERTa NLI model, and threshold on entropy; test across your 1.5B-120B model range.

   hallucinationlocal-llmollamaevaluation

   Read more: Detecting hallucinations in local models without eating VRAM: What we learned testing 1.5B to 120B models · Detecting hallucinations in local models without eating VRAM: What we learned testing 1.5B to 120B models

3. #3 Qwen3.8 Flash Runs on One GPU: 5.05 bpw exl3 and a Low-Bit Quanttechnique

   Two community quants show Qwen3.8 Flash running at 863 t/s prefill and 35 t/s decode at 230k context on a single R9700, plus a low-bit quant retaining 95% of bf16 accuracy.

   Why it matters: Long-context local inference is now feasible on a single consumer GPU, which changes what you can self-host for document QA and agent memory.

   How to apply: Grab the exl3 5.05 bpw build (head 6-bit, vision 6-bit, MTP 5-bit) with the exllamav3 ROCm fork, or the DJLougen low-bit quant, and benchmark against your own context lengths.

   quantizationlocal-llmqwenlong-context

   Read more: Qwen 3.8 flash for a single spark · Qwen3.8-Flash-Next (5.05bpw + ngram at bf16) exl3 on one r9700: 863 t/s prefill and 35 t/s decode at 230k context (256k max), is that ok or am i missing something?

4. #4 Médula: A Cheap Decision Kernel for Parallel Coding Agentsrepo

   An MIT-licensed lab coordinates multiple Claude Code agents on one repo with a pluggable decision kernel, publishing every session, diff, and SQLite decision log.

   Médula · MIT-licensed lab

   Parallel agents, one repo, one referee

   Swap in your own decider and race it against one-branch-per-task on the 6-task / 37-test booking scenario.

   Why it matters: Parallel agents on one codebase silently break each other's work; a coordination layer with published raw runs lets you evaluate the fix instead of trusting a demo.

   How to apply: Clone the lab, run the 6-task/37-test booking API scenario, and swap in your own decider to see whether it beats one-branch-per-task.

   agentsmulti-agentclaude-codeorchestration

   Read more: Open lab: does a cheap decision model keep parallel coding agents from breaking each other's code? All runs published raw, decider is pluggable (MIT, author here) · I gave several AI coding agents the same repo. They broke each other's work in every isolated run, and started messaging each other when I let them

5. #5 MockAgent Catches Tool-Call Schema Drift Before It Burns Your Creditstool

   A lightweight virtual gateway validates agent tool calls with AJV in real time, catching string-vs-int drift and hallucinated parameters before they trigger infinite retry loops.

   Tool-call validation

   How schema drift becomes a credit-burning loop

   1

   Tool call

   drifted by turn 4

   2

   Schema drift

   string, not int

   3

   Opaque 400

   nothing to correct

   4

   Blind retry

   same bad payload

   until credits burn

   MockAgent answers with a structured AJV error the model can actually correct — spiral ends at one turn.

   Why it matters: Schema drift on turn 4 of a trace is one of the most expensive failure modes in multi-tool agents, and generic 400/500 errors give the model nothing to correct.

   How to apply: Drop MockAgent between your agent and tools during local dev, define JSON schemas for each tool, and let it return structured validation errors instead of opaque API failures.

   agentstool-callingvalidationlangchain

   Read more: How are you handling parameter drift and retry loops in multi-tool agents? · Catching schema drift & infinite loops in LangChain tool calls

6. #6 Manifesto: One Typed Action Runtime for UI and Agentrepo

   An MIT-licensed project defines domain transitions in MEL so the UI, backend routes, and agent all submit typed actions through the same SDK runtime and observe snapshots.

   Why it matters: When agents and UIs mutate state through different paths, validation, approvals, and audit trails drift apart; a shared action runtime keeps them honest.

   How to apply: Model your domain transitions in MEL, route UI and agent writes through the same SDK, and enable the optional Lineage and Governance extensions for history and approvals.

   agentsarchitecturetyped-actionsopen-source

   Read more: Manifesto: letting a UI and an agent use the same app-owned actions

7. #7 Row-Bot 5.0 Rebuilds the Local-First Assistant Around Persistent Goalstool

   Row-Bot 5.0 swaps NiceGUI for a React interface and adds persistent agent goals plus a per-conversation workspace, staying local-first.

   Open-source tool update

   Three changes, one constant: data stays local

   Before 5.0

   • NiceGUI front end
   • Goals not persistent
   • No per-chat workspace

   → Still local-first — runs on Ollama / llama.cpp

   Row-Bot 5.0

   • React front end
   • Persistent agent goals
   • Per-conversation workspace

   A concrete alternative to cloud chat UIs for teams wanting agent memory without shipping data out.

   Why it matters: A local-first assistant with persistent goals is a concrete alternative to cloud chat UIs for teams that want agent memory without shipping data out.

   How to apply: Run it against your local Ollama or llama.cpp endpoint, use the workspace-per-conversation model to keep project context isolated, and extend the React front end.

   local-llmagentsollamaopen-source

   Read more: Row-Bot 5.0 is available: a new React interface, persistent agent goals, and a workspace for every conversation · Row-Bot 5.0 is available: a new React interface, persistent agent goals, and a workspace for every conversation · Row-Bot 5.0 is available: a new React interface, persistent agent goals, and a workspace for every conversation · Row-Bot 5.0 is available: a new React interface, persistent agent goals, and a workspace for every conversation

8. #8 AutoSynthData: Generating Training Data for Enterprise Agentstool

   A Hugging Face blog from ServiceNow walks through synthesizing training data for enterprise agents, aimed at teams fine-tuning on their own workflows.

   Agent fine-tuning

   Turn your own workflows into agent training data

   1

   Seed

   tool schemas + ticket history

   2

   Synthesize

   generate agent tasks

   3

   Fine-tune

   your domain agent

   4

   Validate

   held-out real tasks

   the reality check

   Synthetic data earns trust only when held-out real tasks confirm it.

   Why it matters: Fine-tuning an agent on your own domain beats prompt-stuffing, but the data bottleneck is real; a repeatable synthesis pipeline is the missing piece.

   How to apply: Read the recipe, adapt the generation loop to your tool schemas and ticket history, and validate the synthetic set against a held-out slice of real tasks.

   fine-tuningagentstraining-dataenterprise

   Read more: AutoSynthData: Generating Training Data for Enterprise Agents

9. #9 CacheVerifier: 23% of 'Wrong' Semantic Cache Hits Were the Same Prompttool

   An audit of the SemCacheLMArena and SemCacheSearchQueries benchmarks found 23.3% of near-duplicate hits labeled wrong were literally identical after lowercasing and punctuation stripping.

   Semantic-cache benchmark audit

   23 of every 100 'wrong' cache hits were literally the same prompt

   23.3%

   of near-duplicate hits labeled wrong were identical after lowercasing and punctuation stripping

   CacheVerifier re-audited the SemCacheLMArena and SemCacheSearchQueries benchmark labels.

   Why it matters: If your semantic cache is tuned against noisy labels, you are leaving real savings on the table and possibly rejecting safe reuse.

   How to apply: Re-audit your cache benchmark labels, normalize text before scoring, and compare a small verifier against a plain similarity threshold at the same error rate.

   cachingevaluationragbenchmarks

   Read more: CacheVerifier: we finally audited the semantic caching benchmark we'd been tuning on, 23% of the "wrong" cache hits were literally the same prompt

10. #10 Sparse Attention on RK3588: 1.58× Faster Decode at 4K, 18% Slower at 1Ktechnique

    A month-long benchmark shows decode-side top-k KV blocks give 1.58× faster decode at 4K context, but prefill-side block skipping costs 18% at 1K.

    Sparse attention on RK3588

    Two Sparse Switches, Opposite Fates

    Decode-side sparse

    vs

    Prefill-side skip

    Pick by context length — flip decode-side on, leave prefill-side off below 4K.

    Month-long benchmark of both sparse modes on RK3588-class edge hardware.

    Why it matters: Edge inference engines expose two different 'sparse attention' switches; turning on the wrong one silently taxes short-context workloads.

    How to apply: On RK3588-class hardware, enable decode-side sparse attention for long-context workloads and leave prefill-side skipping off unless you are consistently above 4K.

    edge-inferenceattentionbenchmarkslocal-llm

    Read more: Sparse attention on RK3588: 1.58× faster decode at 4K, 18% slower at 1K

11. #11 llama.cpp Adds a /v1/systemone API for Jev-Style Classification Modelsrepo

    PR #29818 adds a /v1/systemone endpoint to llama-server, letting laya, julia-1, lev, openjev, and kev run through the standard server without a separate Jev stack.

    Why it matters: If you already run llama.cpp, you can now serve classification-style models through the same server you use for chat, simplifying local deployments.

    How to apply: Track the PR, pull the branch, and point your existing OpenAI-compatible client at /v1/systemone to test the new model family.

    llama-cpplocal-llmserverclassification

    Read more: llama, server: add /v1/systemone API (models: laya, julia-1, lev, openjev, kev) by ngxson · Pull Request #29818 · ggml-org/llama.cpp

12. #12 Claude Code Browsing in Your Signed-In Chrome Without Stealing the Keyboardtool

    A free MIT-licensed macOS tool lets Claude Code drive your signed-in Chrome session without hijacking your keyboard or requiring a separate browser profile.

    Why it matters: Authenticated browsing is the missing piece for agents that need to check dashboards, fill forms, or read internal tools behind a login.

    How to apply: Install the tool on macOS, point Claude Code at your existing Chrome profile, and keep your own input focus while the agent works in a background tab.

    claude-codebrowseragentsopen-source

    Read more: I made Claude Code browse in my signed-in Chrome without stealing my keyboard (free, open source, macOS)