Open-Weight 27B Decision Scoring Lands, 53-Cent Distills, and Cache-Breaking Prefix Edits

Today's findings

1. #1 Drex Max ships open weights for local 27B decision scoringtool

   A 27B multimodal decision model now runs locally via GGUF, llama.cpp, Ollama, and MLX, returning answer-option probabilities in one forward pass.

   Decision models

   Open weights for local 27B decision scoring

   Drex Max

   model open weights

   27B multimodal · option probabilities in one forward passToday

   GGUFllama.cppOllamaMLX

   runQ4 GGUF in Ollama or llama.cpp

   Runs on a 24 GB Mac or modest GPU — decision heads, no brittle text parsing.

   Why it matters: Decision-style heads can replace brittle text parsing for routing, classification, and tool-choice checks while running on a 24 GB Mac or modest GPU.

   How to apply: Try the Q4 GGUF in Ollama or llama.cpp for tool-choice or policy checks; benchmark it against your current classifier before swapping it into a workflow.

   local-llmdecision-modelsggufollama

   Read more: Drex Max is now open weights: a 27B multimodal decision model that runs on a 24 GB Mac · Nace AI open-sources Drex Max v1, a 27B decision model that runs locally on Apple Silicon (67.52 on Decision Index 0.4)

2. #2 Distill Qwen3-1.7B with $0.53 of API callstechnique

   $0.53 of DeepSeek API-generated data lifted Qwen3-1.7B-Base to official post-trained GSM8K level, and the pipeline is open-source.

   Cheap distillation

   The entire bill for matching official post-training

   $0.53

   of DeepSeek API calls, spent generating the SFT data

   Qwen3-1.7B-Base

   small base model as the student

   GSM8K

   reaches the official post-trained level

   distillops

   open-source pipeline, yours to run

   For narrow tasks that must run locally, distilling your own teacher data beats waiting on a giant release.

   Why it matters: Cheap task-specific distillation can beat waiting for giant models, especially for narrow internal tasks where a small model must run locally.

   How to apply: Use the distillops pipeline to generate teacher outputs for your own task, filter them, then SFT the small base model and validate against a held-out eval.

   distillationfine-tuningqwenlocal-llm

   Read more: $0.53 of DeepSeek API calls took Qwen3-1.7B-Base to the level of the official post-trained Qwen3-1.7B on GSM8K. Open-source pipeline to do the same for your own task · $0.53的DeepSeek API调用将Qwen3-1.7B-Base提升到了官方后训练的Qwen3-1.7B在GSM8K上的水平。开源管道使您能够为自己的任务做到这一点

3. #3 Avoid prefix changes that silently break Claude Code prompt cachetip

   Claude Code resends the whole conversation each turn; edits near the top of the prompt break exact-prefix caching and inflate cost.

   Prompt caching

   Pin the stable front; append the volatile tail

   Churn near the top

   • Per-turn edits sit early
   • Exact prefix breaks
   • Whole history re-sent

   → Same tokens, different order — cache holds

   Churn appended last

   • System & tools pinned first
   • Volatile files & notes last
   • Only the tail re-sent

   One edit near the top invalidates everything after it.

   Why it matters: Prompt caching is often the difference between usable and expensive long agent sessions.

   How to apply: Pin system and tool instructions first, append volatile files or per-turn notes at the end, and review cache hit rate after changing rules or startup context.

   claudeprompt-cachingcost-optimizationagents

   Read more: Things that quietly break your prompt cache in Claude Code (and a few that don’t)

4. #4 Llama.cpp expert cache keeps Qwen-Flash-Next fast at 262K contexttip

   A Vulkan Q4 setup with an expert cache holds 19-21 tok/s at full 262K context, but explicitly setting --ctx-size tanked throughput.

   Why it matters: MoE offload tuning can make enormous-context local inference practical on mixed AMD GPUs.

   How to apply: Start with the posted llama-server flags, cap the MoE cache to free VRAM, and A/B test context flags instead of assuming a larger -c helps.

   llama.cppmoevulkanlocal-llm

   Read more: Llama.cpp setup Qwen-flash-next Vulkan Q4 with Expert cache. · Llama.cpp setup Qwen-flash-next Vulkan Q4 with Expert cache.

5. #5 TinyWebUI gives local models a one-command web UItool

   TinyWebUI is a lightweight Open WebUI alternative installable with 'npx tinywebui', bundling tinysearch and tinycontext for quick local chats.

   Why it matters: A zero-config web UI lowers friction for trying local models or shipping a small internal chat surface.

   How to apply: Run 'npx tinywebui' against an Ollama or LM Studio local endpoint, then evaluate whether its search and context features cover your team's needs before deploying a heavier UI.

   local-llmweb-uiollamatools

   Read more: TinyWebUI - Lightweight Open WebUI alternative, just npx and go

6. #6 Put budgets and loop detection before agent API callstechnique

   Runaway agents need pre-call guardrails: per-run dollar budgets, identical-tool-call loop limits, and velocity caps before the provider bills you.

   Runaway-agent guardrails

   Agent autonomy, fenced before the first call

   bounded capability

   Agent tool & API calls

   limit Per-run dollar budget

   limit Identical-call loop cap

   limit Velocity cap

   revoke Stop on repeated errors

   monitor Alert on $/min spikes

   Gates trip in-flight, pre-billing; dashboards only see the burn after.

   Why it matters: Cost dashboards are too late; a looping agent can burn a budget in minutes.

   How to apply: Tag every call with a run ID, enforce a shared max spend, stop after N identical calls or errors, and alert on dollar-per-minute spikes.

   agentsguardrailscost-optimizationllmops

   Read more: What actually stops a runaway agent before the bill? Notes from building pre-call guardrails

7. #7 HERCULES is a Rust harness for accountable agent executionrepo

   HERCULES is an open-source Rust agent harness focused on verifiable execution and accountability rather than another prompt wrapper.

   Why it matters: Rust harnesses can add typed tool boundaries, logging, and execution proof to agent workflows.

   How to apply: Review the repo for its audit and trace model, then test it on a constrained internal automation before adopting it broadly.

   agentsrustopen-sourceobservability

   Read more: My first open-source release — HERCULES, a Rust agent harness focused on accountability and verifiable execution

8. #8 Fix inflated Recall@K when RAG questions are unanswerabletool

   Treating unanswerable questions as perfect retrieval inflates Recall@K; an open-source evaluator separates retrieval metrics from answerability.

   RAG EVAL PITFALL

   How you score unanswerables decides your Recall@K

   Naive scoring

   • Unanswerable questions scored as perfect retrieval
   • Recall@K reads inflated
   • Dashboards look better than production

   vs

   Answerability-aware

   • Unanswerables excluded or scored separately
   • Retrieval judged only where answers exist
   • Unanswerable slice added to the eval

   Separate answerability from retrieval — then trust Recall@K

   Open-source evaluator: local baselines, no API key needed

   Why it matters: Many RAG dashboards look better than production because unanswerable cases are scored incorrectly.

   How to apply: Add an unanswerable slice to your eval, exclude or separately score those cases, and run the open-source starter without an API key for local baselines.

   ragevaluationretrievalopen-source

   Read more: I built a small open-source RAG evaluator after finding a misleading Recall@K score

9. #9 Use a rubric prompt to replace one-to-ten vibestip

   A criterion-by-criterion rubric with evidence quotes and no premature total score makes LLM grading far more consistent than single-number ratings.

   Why it matters: Structured rubrics reduce variance in eval, content review, and agent self-checks.

   How to apply: Paste the rubric template, define concrete low, medium, and high anchors, require evidence per criterion, and compute the total only after all criteria.

   promptingevaluationrubrics

   Read more: My rubric prompt for getting consistent scores instead of vibes

10. #10 Tiny 109M model beats larger image-text aligners with 42x less datapaper

    A 109M-parameter model reportedly outperforms models up to 72B on image-text alignment while using 42x less training data.

    Why it matters: Compact alignment models could cut retrieval and multimodal indexing costs dramatically.

    How to apply: Read the method for training-data and architecture tricks, then test a small image-text encoder as a lightweight retriever or pre-filter before larger multimodal models.

    multimodalimage-textsmall-modelspapers

    Read more: new research: A tiny 109M-parameter model outperforms models up to 72B parameters at image-text alignment, using 42× less training data