How I Built Local LLM Updates: A Source-Locked Workflow

Direct Answer: What I Built and Why the Workflow Matters

I built Local LLM Updates as a source-backed publication and index for people who need to understand local-model releases without relying on generated summaries. Its public structure separates release updates, pinned model records, and benchmarks that record the exact runtime and hardware used instead of presenting every number as a universal ranking.

The useful part is not another AI news feed. It is the chain of evidence behind each claim: a source, a verification state, an applicable version, and a caveat when the evidence does not support a broader conclusion. That chain matters because a model name alone does not tell a reader which artifact, quantization, runtime, parameters, or hardware produced a result.

This case study explains the editorial and data-product structure behind the publication. It covers the decisions visible in the public methodology, model records, update ledger, benchmark pages, and correction policy. It does not claim traffic, revenue, speed advantages, or a creation timeline that the public record cannot verify.

The Information Problem Is Version Drift, Not a Lack of Announcements

Local-model information becomes ambiguous as soon as a release moves through multiple repositories, runtimes, and distribution formats. A vendor announcement may identify a model family while leaving the exact downloadable artifact unspecified. A community build may add a quantization label. A runtime release may change compatibility or performance. A benchmark may then describe only the family name and hardware brand.

Those fragments can all be individually accurate and still produce a misleading summary. Consider the questions a reader has to answer before acting on a local-model claim:

• Which model repository and immutable revision does the claim cover?
• Is the artifact a base model, a conversion, or a quantized distribution?
• Does the published quantization label match the runtime’s aggregate file-type label?
• Which runtime version or commit loaded the file?
• Which backend, arguments, context, and batch settings were used?
• What hardware, memory, operating system, and power state applied?
• Is the source a vendor statement, repository metadata, or an independent run?

Broad market data can answer a different question. My LLM market share analysis is useful for provider and workload context, but market-level shares do not verify a local artifact. The source-locked workflow starts where those aggregate views stop.

The Methodology Separates Vendor Claims From Demonstrated Evidence

The first editorial rule is to label who made a claim and what was independently checked. A release note can establish that a developer announced a feature or fix. It cannot, by itself, establish that every artifact works on every runtime or that a performance claim reproduces on different hardware.

The public methodology uses four claim states: verified, partially verified, unverified, and disputed. “Partially verified” is especially important for model metadata. A repository can be pinned to an immutable revision and its published task, artifact label, and license metadata can be recorded, while hardware fit, runtime compatibility, and performance remain untested.

This distinction also prevents source laundering. If an update says that a vendor reports a change, the publication should not rewrite that statement as an independently measured fact. If a benchmark tests one file, the result should not migrate into a general claim about the model family. The same principle applies to live-web research: a clean page retrieval, such as the workflow described in my RAG Web Browser guide, provides evidence to inspect, not permission to invent missing fields.

The approach follows the documentation logic behind model cards, which describe intended uses, performance characteristics, and evaluation conditions. The NIST AI Risk Management Framework provides broader context: it is a voluntary framework for incorporating trustworthiness considerations into the design, development, use, and evaluation of AI. Neither reference certifies this publication or its benchmarks.

The Evidence Ledger Comes Before the Draft

A publishable update begins as a ledger entry, not as prose. The workflow records the primary source URL, the affected version, the exact material claim, its verification state, and any caveat that changes interpretation. Only then does the editorial layer turn that record into a concise update.

The public source and verification methodology defines that contract. It separates what a developer announced from what the available evidence demonstrates and requires a visible correction or update note when a material fact changes.

The workflow is deliberately linear:

1. Capture the official release note, repository, model card, paper, or vendor documentation.
2. Identify the exact claim and the version or artifact it applies to.
3. Record which fields were checked and which remain missing.
4. Assign a verification state without upgrading uncertainty into certainty.
5. Draft only from the ledger, with the source and caveat still attached.
6. Review the finished copy against the ledger before publication.

This is different from asking a model to summarize a collection of links and then checking a few details afterward. In a source-locked process, the draft cannot introduce a specification, date, quote, license, compatibility statement, or benchmark result that does not exist in the approved evidence package.

An Exact Record Needs More Than a Model Name

The minimum useful record changes with the type of claim, but the identity fields must remain explicit. Release reporting needs a version and source. Model profiles need a repository, immutable revision, and honest statement about unverified compatibility. Benchmarks need the artifact, environment, workload, and raw result boundary.

Record type	Required identity	Evidence boundary	What it must not imply
Release update	Developer, version, date, primary source	What the developer announced and what was checked	Compatibility beyond the named version
Model profile	Repository, immutable revision, task, artifact label, published metadata	Fields visible at the pinned revision	Hardware fit, runtime support, quality, or speed without a test
Benchmark runtime fields	Runtime version or commit, backend, applicable platform	Behavior documented or reproduced for that benchmark build	Equivalent behavior in another build or backend
Benchmark	Exact artifact and checksum, runtime, arguments, OS, hardware, workload, repetitions	Results from the recorded setup	A general model or hardware ranking
Correction	Original claim, changed fact, date, reason	What changed and why	That a silent rewrite never occurred

Quantization deserves its own caution. A distribution label can describe a mixed-tensor artifact, while a runtime displays an aggregate file-type label from embedded metadata. Those two labels may be accurate at different levels. Replacing one with the other can make a reproducible file look inconsistent or make a mixed artifact look uniform.

The same problem appears in parameter counts. A model-family label, total parameter count, active parameter count, and file size answer different questions. The editorial rule is to retain the source’s terminology and avoid translating one measurement into another unless the source provides the mapping.

Updates, Profiles, Benchmarks, Methodology, and Corrections Have Different Jobs

The publication is structured so that no single page has to carry every kind of evidence. Updates are short and version-scoped. Model profiles act as maintained repository records. Benchmarks preserve a controlled run. Methodology explains the rules, and corrections keep material changes visible.

This separation helps readers enter at the right level:

• Updates answer what changed in a named release and where the statement came from.
• Model profiles answer which repository revision and published metadata were checked.
• Benchmarks answer what happened for one artifact, runtime, workload, and machine.
• Methodology answers how claims are classified and which fields a controlled run requires.
• Corrections answer how factual errors, software updates, disputes, and retired results are handled.

It also reduces pressure to turn an update into a verdict. A runtime fix can be important without proving that one local stack is “best.” A model profile can be useful while explicitly stating that hardware and runtime compatibility remain unverified. A benchmark can be reproducible without measuring answer quality.

For comparison, broad lists such as my ChatGPT alternatives guide help readers survey product choices. A maintained local-model record serves a narrower purpose: it preserves the evidence needed to verify a technical claim after versions move on.

The Benchmark Page Shows the Reproduction Boundary

The clearest example is a reproducible benchmark with the exact model artifact, runtime and hardware recorded. The page reports one checksum-pinned Qwen3.8-27B GGUF from the Unsloth UD-Q2_K_XL distribution on one MacBook Air with Apple M5, 10 logical cores, 24 GB of memory, and macOS 26.6.2.

The run used llama.cpp llama-bench build 10470 at commit 34af94cd9, with BLAS and Metal. Its arguments included 512 prompt tokens, 128 generated tokens, 5 repetitions, 10 threads, 99 GPU layers, batch size 2048, micro-batch size 512, and flash attention set to auto.

For that setup, the five-run means were 39.439795 tokens per second for prompt processing and 1.865259 tokens per second for generation. Those values do not rank Qwen3.8-27B in general. They do not establish performance for another quantization, another Mac, another runtime build, or another prompt set.

The missing fields remain part of the result. Power mode, thermal state, room conditions, and energy use were not recorded. The arguments did not contain an explicit context-length flag. The workload measured synthetic token processing, not response quality or application-level performance.

That negative space is useful. It tells a reader exactly what can be reproduced and what cannot. It also prevents a precise number from acquiring an imprecise meaning when quoted elsewhere.

Source Locking Changes How AI Can Assist the Workflow

AI can help monitor, deduplicate, extract, and review, but it should not decide that missing evidence exists. In this workflow, automation can surface a release, copy a source field into a ledger, flag a stale profile, or compare a draft against approved evidence. Human review still controls publication, source sufficiency, benchmark validity, rankings, corrections, and legal judgment.

The boundary is operational, not rhetorical. In a source-locked workflow, a drafting prompt receives an approved evidence set and must fail closed on absent specifications. Review then checks every material sentence against the ledger. If the draft adds a license, date, runtime, parameter count, or conclusion that the ledger does not contain, the sentence should be removed or the evidence package expanded before publication.

This is one practical response to the same measurement problem I cover in AI code assistant statistics: aggregate adoption or productivity claims cannot substitute for the exact context in which a tool or model was evaluated.

Source locking does not make a language model truthful by default. It reduces the space in which unsupported claims can enter, makes the review target explicit, and leaves a correction path when evidence changes.

What the Workflow Still Cannot Prove

A well-documented record narrows uncertainty; it does not remove it. Public repository metadata can be incomplete or wrong. Vendor release notes can omit regressions. A pinned model card can change at a later revision. Hardware behavior can vary with power mode, thermal conditions, background processes, and runtime compilation.

The publication also has coverage limits. It cannot test every artifact, quantization, runtime, backend, device, or workload. A partially verified profile is therefore not a recommendation. An absent benchmark is not evidence that a model is slow. A disputed result is not resolved merely because a newer post repeats it.

Independent verification has limits too. Reproduction requires access to the artifact and environment, and some historical evidence may lack raw files or sensitive system details. The correct response is to disclose the missing fields and narrow the claim, not to reconstruct them from a filename or assume defaults.

The Datasheets for Datasets proposal makes a related point: documentation is valuable because it records motivation, composition, collection process, and recommended uses. The same discipline applies to model and benchmark records, even though the fields differ.

How a Reader Can Verify a Published Claim

A direct verification path is to move backward from the conclusion to the exact evidence object. Do not begin with the model-family name. Begin with the claim, then inspect the version, artifact, and environment that bound it.

Use this checklist:

1. Open the cited primary source and confirm that it supports the wording, not merely the topic.
2. Check the release version, repository revision, or runtime commit.
3. For a model record, distinguish published metadata from independently tested compatibility.
4. For a benchmark, match the artifact checksum, runtime build, backend, command arguments, operating system, hardware, and memory.
5. Check the workload, repetition count, aggregation method, and raw samples.
6. Read the limitations before comparing the number with another result.
7. Look for a correction, update note, disputed status, or retired result.

If a comparison changes hardware, version, quantization, prompt set, or context, treat it as a different experiment unless the page explicitly controls that variable. If the source cannot answer a material question, keep the question open.

What I Would Reuse in Another Fast-Moving Technical Publication

The reusable product is the contract between evidence, structured records, prose, and correction history. A technical marketing team, developer-relations group, or independent builder can apply the same sequence to APIs, runtimes, security advisories, pricing changes, or developer tools.

Start with a small schema that matches the claim type. Make uncertainty a first-class field. Separate announcement coverage from independent testing. Keep controlled results attached to exact artifacts and environments. Give corrections a visible surface. Let automation accelerate collection and review, but do not let it manufacture missing evidence.

Sentence-level speed is not the only system constraint. A publication that follows fast-moving releases also needs traceability, bounded conclusions, and a way to correct the record without hiding the original error.

FAQ

What does “source-locked” mean in this workflow?

It means the draft is restricted to an approved evidence ledger. A material claim needs a source, verification state, applicable version, and caveat before it can enter the published copy. Missing fields remain missing.

Is a vendor release note enough to mark a claim verified?

It can verify what the vendor announced for a named release. It does not independently prove compatibility, performance, or behavior outside that release. The wording should preserve that distinction.

Why is a model checksum important for a local LLM benchmark?

A checksum identifies the exact file tested. Model-family and quantization labels can cover multiple artifacts, so the checksum prevents a result from being detached from the file that produced it.

Can two tokens-per-second results be compared directly?

Only when the artifact, runtime, backend, arguments, hardware, workload, and relevant environment fields are controlled or clearly reconciled. Otherwise the numbers describe different setups.

Does source locking eliminate AI hallucinations?

No. It constrains drafting and makes unsupported additions easier to detect. Human review, evidence checks, and visible corrections are still required.

Why keep corrections visible instead of silently updating a page?

Visible corrections preserve the reliability of the record. Readers can distinguish a factual error from a later software update, a disputed result, or a benchmark retired because it can no longer be reproduced.

Last verified: September 2, 2026.