Starkweather Digital

No. 01

Local AI: A Review of the Receipts

Local models now write code that real projects merge. In every convincing case, though, the local model is the worker in a loop, not the whole loop: someone else writes the brief, and someone checks the result before it lands.

12 minute read

Each tick is a post, placed by the embedding of its text. This one is inked.

Local models can now write code that real projects merge. The best evidence, a few dozen public PRs and commits, comes from dense ~27B models such as Qwen3.6 and Qwen3.8 running on a single DGX Spark, Mac or consumer GPU. In every convincing case, though, the local model is the worker in a loop, not the whole loop: a person or a frontier model writes the brief, and someone checks the result before it lands.

That matches what I’ve seen in my own tests. Local AI is useful and getting better quickly. It is not yet a drop-in replacement for a frontier coding agent, and most of the gap I hit turned out to be setup and workflow rather than the model.

In this article, I review the most complete and documented examples of local AI use I could find online, as well as compare to my own setup for reference.

What counted as evidence

There is no shortage of posts saying local models have replaced cloud. Most show a setup guide and a tokens-per-second number. This review only counts sources that leave something checkable behind:

  • Receipts: committed or merged code whose PR body, commit message or trailer names the local model, ideally with a published transcript.
  • Proof it was local: the author names their own hardware or server, not a cloud API serving the same open model.
  • Controlled tests: runs scored against tests or independent checks, preferably with the harness published.

Every PR and commit cited here was checked through the GitHub API for merge date, diff size and attribution text, as of November 2026.

The literature at a glance

The sources fall into four groups, from strongest to weakest: merged code in real projects, smaller repos, controlled tests, and first-hand reports.

Real projects with merged code

The review leans hardest on these: public PRs in maintained projects that name the local model that wrote them.

  • Clawrium and Devashish’s write-up. Qwen3.6-27B FP8 under vLLM on one DGX Spark, driven by pi, OpenCode and Hermes Agent. Seven merged PRs credit the local model, produced inside a plan, work and judge loop. Strong evidence.
  • Simon Willison. Qwen3.8-27B Q4_K_M in LM Studio on a DGX Spark, driven by pi. One prompt produced a working 139-line tool at no cost, and the full transcript is published. Strong evidence, but a small task.
  • Cogent Core #393. “Qwen3.8 FN” on a Mac. Full LSP support for the project’s editor, about 14,500 added lines, later heavily revised in a Claude review. Medium-strong evidence.

Smaller repos

Hobby and small-project code. These show the range of what people are trying more than the quality of the result.

Controlled tests

Scored runs against tests or independent checks. They say less about real projects and more about where the limits are.

  • Rudy Bohrer’s lab. Qwen 3.6, Gemma 4 26B and gpt-oss-120B on a DGX Spark, each run through OpenCode, Codex CLI and Claude Code. Qwen fully passed 19 of 27 runs, Gemma 10 and gpt-oss 7. No artifacts are published, so medium evidence.
  • zjshen14. Qwen3.8-27B Q4_K_M on an RTX 3090 with OpenCode. 31 of 40 independent checks passed on first attempts, and one task was lost to runaway reasoning. The reproduction kit is published. Strong evidence.
  • Simon Couch, helperbench. Laptop-sized models from 2.6B to 24B on one refactor task. In late 2025 none passed. By September 2026 an 8B model passed 8 of 10, while every model of 4B or smaller still scored zero. The harness is published. Strong evidence.

First-hand reports

  • Kevin Welter and the Hacker News discussion. Mostly 27B models on Sparks, Macs and RTX 3090-5090 cards. “So are they good? not really. Do they work? yes” sits beside “rock solid.” Anecdotal evidence.

The strongest case: Clawrium

Clawrium is an Apache-2.0 tool for running fleets of AI agents across machines, with 578 merged PRs and about 4,800 tests. Its maintainer, Devashish, runs Qwen3.6-27B (FP8, reasoning on) under vLLM on one DGX Spark and documents the stack in Owning Inference. He measured about 14-16 tokens per second with two requests in flight, a 95% prefix-cache hit rate, and roughly 50 million tokens in one weekend.

What makes it convincing is that the PRs credit the model themselves:

MergedPRChangeSize
2026-09-03#969Restores a missing line in a docs mirror (“Authored by a local model”)+2
2026-07-08#867Replaces regex error-sanitizing with constant messages (Authored-By: Pi<Qwen3.6-27B FP8 (256K, inx GB10)>)+236 / -103
2026-07-06#859GUI lifecycle endpoints return 502 on failure, and the UI shows it+254 / -43
2026-07-06#855Validates agent names before they reach file paths+323 / -1
2026-07-06/07#861, #862, #864Release notes, a docs page, a blog post (Hermes Agent)+508 total

These are ordinary maintenance PRs: input validation, error handling, release notes and docs. A larger hardening pass, #954, came out of the same workflow according to the project’s run ledger, but the PR itself credits Claude Sonnet, so it isn’t counted here. The repo is equally candid about failures. #969 notes that its review step never finished. Two earlier docs PRs, #879 and #883, were closed; both bodies credit Claude Code, but in #955 the maintainer blames the closures on the small model, whose “documented failure mode is claiming completion when the work is unfinished.”

His answer was a workflow, not a bigger model. The clawctl-lmwork skill (#955) separates three roles. An orchestrator picks issues small enough to hand off. The local model, via pi, writes the code. Claude Code judges the result against the brief, for up to three rounds, and anything still failing opens as [ITX-STUCK] rather than pretending to be done. His summary: “The frontier is called for planning, not for typing.”

Corroborating builds

Simon Willison has the cleanest single receipt. In August he ran Qwen3.8-27B (a 17 GB Q4_K_M build in LM Studio) on a DGX Spark, drove it with pi, and asked for a script to convert pi’s session logs to Markdown. The result is commit 29f7ba0 (+139 lines), and the full transcript shows every step at zero cost. In his write-up he calls the model excellent, with one catch: it defaults to maximum reasoning and quickly overruns LM Studio’s 8,192-token default context.

Cogent Core is the largest. Randall O’Reilly’s PR #393 adds full LSP support to the project’s code editor: 66 files, +14,571 lines, merged in October. The PR says it was “written by Qwen3.8 FN locally on my mac :) of course directed, reviewed, and tested extensively by me.” One honest footnote from the commit history: a later “Claude opus 5.5 code review” commit changed about 6,600 lines, so the merged code is a local-cloud collaboration.

Smaller repos fill out the picture. octopus-invaders is a 3,263-line game built by Qwen 3.5 9B on a 12 GB RTX 3060, with every prompt committed. ulams #509 merged 2,096 lines of generated tests from qwen2.5-coder:14b via Ollama, after dropping nine files that failed. linux-hello ships small fixes tagged Assisted-by: Aider with a local model. Beyond these, Aider’s ollama_chat/ commit trailer appears on more than 20,000 commits, nearly all pushed straight to personal repos with no review to speak of.

Controlled tests and the skeptics

The controlled tests agree with the builds on what works, and are clearer about where it stops.

  • Model size is a cliff, not a slope. Simon Couch’s helperbench gave agents a refactor any frontier model gets right almost every time. In late 2025, no laptop-sized model passed once. By September 2026, 8B models passed 8 of 10, while every 3-4B model still scored zero.
  • Model choice matters more than precision. Rudy Bohrer’s Spark experiments ran the same tasks across three agent harnesses. Qwen 3.6 fully passed 19 of 27 runs, Gemma 4 26B 10, and gpt-oss-120B 7. Moving Qwen from 4-bit to 8-bit made it about 63% slower and no more accurate.
  • Reasoning can eat the whole budget. zjshen14’s RTX 3090 test of Qwen3.8-27B passed 31 of 40 independent checks on first attempts, but one task spent all 8,192 output tokens thinking and never produced a patch.

The first-hand reports, collected well in a Hacker News discussion of Vicki Boykis’s “Running local models is good now,” split along the same line. People giving 27B models well-specified tasks call them “rock solid” and Sonnet-4.5-class. People hoping to hand over a large task and walk away find they “cannot really be given a large-ish task and left to just drive it.” Kevin Welter’s Spark write-up puts it plainly: for “complex multi-file changes, long agent runs, or unclear requirements, Claude Code stays ahead.”

Analysis: what the successes have in common

Read side by side, the working examples share five traits. None of them is exotic hardware.

  1. A dense model around 27B, with reasoning on. Qwen3.6-27B and Qwen3.8-27B appear in nearly every strong receipt. Devashish’s earlier attempts with an 80B mixture-of-experts model, a 4B model and a non-reasoning 35B mixture-of-experts model with 3B active “didn’t do much beyond basic summarization.”
  2. Somebody else holds the plan. The maintainer, or a frontier model, decides what to build and breaks it into tickets small enough to finish. The local model executes.
  3. Somebody checks the work. Every merged change passed a human review, a test suite, or a frontier-model judge. Small models’ signature failure is declaring victory early, and an outside check turns that from a trust problem into a retry.
  4. The plumbing is right. The failures people describe are often configuration, not intelligence: a tool-call parser that doesn’t match the model’s output format, a proxy rejecting a parameter the agent sends, a context window or output budget too small for a reasoning model.
  5. Patience is budgeted in. These setups run at about 15-50 tokens per second per request. Work gets done by running several tasks at once and waiting, not by watching one turn stream.

The pattern also explains the split in opinion. Local models are judged harshly when compared to a frontier agent doing everything end to end. They look much better when judged on the narrower job they’re actually given in the cases that work.

Corroboration from my own homelab

I run a fairly elaborate setup: two DGX Sparks behind my own inference gateway, which routes requests, queues them and manages token budgets. One Spark currently serves a Qwen3.8 27B variant with a 262K context. For a while I couldn’t get anything I’d call useful out of off-the-shelf agents on it, which is what sent me looking for receipts. My notes from the past six months line up with the literature almost point for point.

  • The parser bug, independently. In June, every tool-using agent on one Spark silently got zero tool calls. Requests returned HTTP 200, but the model’s calls came back as plain text. The cause was vLLM’s hermes tool parser reading a Qwen model that emits XML; switching to qwen3_xml fixed it. Devashish hit the identical bug and wrote the identical fix.
  • Model shape matters. In July I ran the neutral mini-SWE-agent harness on a 23-task set. gpt-oss-120B resolved about 5%, and its structured tool calls broke down over long loops, around 14 format errors per 80 turns. Gemma 4 26B drove itself to a real bug and fixed it exactly; its real limit was a 32K context window. That ordering matches Rudy Bohrer’s Spark results.
  • The “capability wall” was usually the harness. Gemma’s apparent fabrication turned out to be a cluttered prompt, and gpt-oss’s poor score traced to a lossy tool interface rather than the model. That became a standing rule in my notes: check what the model actually received before concluding it can’t do something.
  • Context and budgets bite. My gateway’s compaction only kicks in for tracked conversations, so self-looping agents that keep their own history skipped it entirely and hit the context wall; agents have to manage their own context. Reasoning models also used to burn the whole token budget and return empty answers, until the gateway learned to reserve room for the answer. Willison and zjshen14 describe the same trap.
  • Speed is what it is. A dense 27B model on one Spark decodes at about 15 tokens per second for a single request, and about 180 in aggregate with 16 running at once, close to Devashish’s numbers. Every timeout had to be sized in minutes, not seconds.
  • Today’s check. Pointing pi at that 27B model through the gateway, plain chat and a real shell-tool call worked on the first try, with reasoning displayed correctly. The friction was elsewhere: with a dozen MCP servers configured, pi sometimes hung at startup.

What I haven’t done yet is the part every success story has: a stronger model writing the plan, small tickets, and a judge checking the output before it lands. My experience doesn’t contradict the literature. It’s the literature’s prediction for someone who skipped that step.

Conclusion: how it actually is

The optimists are right that local coding agents are real. A single desk-sized box running a free 27B model is writing validation logic, error handling, tests and docs that maintainers merge, at no per-token cost and with nothing leaving the house. A year ago the controlled tests showed zero passes; now they show most.

The skeptics are right that it isn’t a swap. Nobody with receipts has replaced a frontier agent end to end. The working pattern is a division of labor: a stronger model or a person plans and reviews, and the local model does the high-volume typing in between. That division is also where most of the savings live, since the typing is where most of the tokens go.

If you’re setting this up, the literature and my own notes point to the same short list: pick a dense ~27B model with reasoning on, confirm the tool parser matches the model’s output format, give it room to think, feed it small well-specified tickets, and put a check between its “done” and your main branch. Do that, and local AI is a genuinely useful teammate. Skip it, and it looks a lot worse than it is.

Sources

GitHub items verified through the GitHub API on 2026-11-15.

local inferencecoding agents