Local inference and speculative feature branches: who pays to choose?

Question, October 20, 2026. Can cheap local inference change an agent-built feature from one proposal to several pre-built alternatives, and lower the cost of a maintained accepted change rather than only the marginal token charge? Evidence would connect a particular local/hybrid harness and hardware to variant count, human comparison and integration effort, independent acceptance, and repair after release.

Before the search

Unit economics counts failed and discarded attempts plus human time and downstream work; the request ledger demands accepted behavior, not PR count. Parallel work can shift cost into supervision. Subquestions: (1) what Thompson observed versus proposed; (2) model, hardware and license constraints from Meta/Apple; (3) whether comparable candidate-and-human-choice work exists with measured delivered outcomes.

October 20 Update: observation versus proposed workflow

Ben Thompson, “Muse Spark Open, Meta + Apple, Thin and Thick” (October 20, subscription item received in full) begins with the model launch and restricted open-weight license, its potential enterprise distribution and Meta’s position versus Chinese open weights. At an Apple-hosted demonstration Thompson saw two 512GB Mac Studios linked through Thunderbolt, serving an on-device coding alteration requested on a MacBook Pro, plus local editing of large video files. He describes the coding demonstration as a single prompted change. He then proposes a different, not-demonstrated harness: build multiple permutations of a feature and possible follow-on work overnight, let a human select a path in the morning, discard the rest. His CPU branch-prediction analogy makes the option value vivid; it does not show that the speculative paths were run, compared or maintained. He argues that prepaid local inference makes discarded tokens nearly free at the margin but himself acknowledges slower Macs, hardware capital, electricity and often more economical cloud service. His more durable motive may be autonomy over model use; it does not settle the economics.

Primary checks and one counterexample to novelty

• Meta’s October 15 launch describes 816B total/42B active weights, two M5 Ultra Mac Studios and MLX. It says the team used local Muse Code for iOS changes to Ice Cubes, but provides no identified PR, customer acceptance, complete model-bill or human minutes for the speculative branching proposal. Meta’s benchmark methods mix external and model-provider/self-reported coding numbers across different harnesses; those do not validate a many-branch feature workflow.
• Apple’s October 15 MLX research benchmarks 2-Mac Muse Spark Open (NVFP4) decoding at 24.99 tokens/s in its displayed 8,192-token context configuration and 4-Mac at 36.73. It says quantized weights occupy about 493GB, but intermediate computation prevents a single 512GB Mac from serving them. Its approximately 300W peak-power illustration is for other, smaller models (Qwen 3.8 and Mistral Medium 3.5), not measured Muse Spark Open system energy per overnight branch. Its 4.6× startup-speed comparison likewise concerns smaller models on one M5 versus M3 Ultra, not the full Muse workflow. Apple measures inference, not acceptable feature variants per dollar.
• Meta’s model license imposes an acceptable-use policy, attribution in product/docs/settings and a separate agreement for model-as-a-service operators with annual revenue over $20 million; internal and non-model-as-a-service product uses are explicitly excepted from the latter clause. Downloadable weights are not an unrestricted open-source license; avoid giving deployment/legal advice from a digest.
• Cursor’s April 2 announcement and its worktree documentation already allow /best-of-n: multiple models attempt the same task in isolated worktrees and an agent compares results; a human picks or combines before application. Thompson’s novel extension is the precomputed follow-on tree after each choice under local marginal-cost assumptions, not the mere idea of generating alternative patches. Cursor supplies a product mechanism and company testimony, not an independent comparison of reviewer touch time or requester outcome.

What remains unknown. No inspected source here tests Thompson’s proposed overnight branching tree against a single-path agent on identical feature requests with human decision and review time, integration failures, production acceptance and 30-day repair. A useful comparison records for each request: model and hardware utilization/capital depreciation, electricity and tests for all branches, human deliberation and merged-path verification, and the same requester’s later outcome. Branching may spend otherwise idle capacity, but discarded code still consumes compute and verification attention. Locality can matter for control and privacy even when cloud inference wins on full cost; hardware and model quality determine the crossover. Do not confuse CPU branch prediction’s microscopic validation and rollback with human evaluation of feature intent.

Search disposition. Checked the subscriber item against its canonical public title/date, Meta launch/evaluation methods and license, Apple MLX measurement, and Cursor’s original feature/docs. Targeted searches for a production speculative-feature tree with paired human and requester outcomes did not surface a qualifying primary study; no assertion that none exists globally.