Swift and Metal inference for Gemma 4 26B-A4B on Apple Silicon.
This checkout is for running and reporting existing behavior. Do not edit source, change runtime defaults, or start optimization work unless the user asks.
Sources/TurboFieldfare/ is the runtime; Sources/TurboFieldfareRepack/,
Sources/TurboFieldfareCLI/, Sources/TurboFieldfareServer/, and
Sources/TurboFieldfareApp/ contain the installer, CLI, loopback server, and
Mac app.
Tests/ contains focused public tests; docs/ contains design, benchmark, and experiment notes.
swift run -c release TurboFieldfareRepack --output scratch/gemma4.gturbo
swift run -c release TurboFieldfareRepack --output scratch/gemma4.gturbo --resume
swift build -c release
.build/release/TurboFieldfareMac
swift run -c release TurboFieldfareCLI \
--model scratch/gemma4.gturbo \
--prompt "The capital of France is" \
--max-new 64The installer streams the pinned model without staging the full source checkpoint. Set HF_TOKEN only if requested. The download is about 15 GB. Cancellation preserves verified completed ranges; continue them with --resume or remove them with --discard-partial --output scratch/gemma4.gturbo.
Follow the server guide for launch commands, health checks, client setup, prompt reuse, tool loops, and supported API behavior. Apply the model-process checks below first; never start a second model process or terminate an existing one.
Keep the server on its default 127.0.0.1 binding unless the user explicitly
requests --bind tailnet and intends the Tailnet ACL to be the access boundary.
It has no application-level authentication or TLS, so never bind it to a
wildcard interface or expose it through a proxy or tunnel. A tool call from the
local model never bypasses the client's normal permission policy. Keep the
execution session alive while the server is needed, and stop only a server you
launched.
Before a model run, require macOS 26+, Swift 6.2+, enough disk, acceptable memory_pressure -Q, a completed scratch/gemma4.gturbo, and no process from pgrep -fl 'TurboFieldfareServer|TurboFieldfareMac|TurboFieldfareDecodeService|TurboFieldfareCLI|TurboFieldfarePackageTests|swiftpm-testing-helper|mlx_lm|mlx-lm'. If a check fails, inform the user and stop; do not terminate apps or delete or reinstall the model.
Run package tests through Scripts/test.sh. Run only one app, CLI, or model-using test at a time.
For performance results, build release once and follow the community benchmark guide exactly. Do not enable experimental controls or profiling.
Do not download a full checkpoint, duplicate the .gturbo model, create a worktree, or purge caches just to run tests.
Report the commit, hardware and RAM, macOS, Swift version, exact command, exit code, complete timing footer or error, and every protocol deviation. Treat results as measurements, not performance ceilings.
The Mac app sends prompts through the pinned Gemma 4 IT chat format. It
exposes context length, temperature, Top-K, Top-P, expert-cache slots, prefill,
and RDADVISE. The defaults are temperature 0.2, Top-K 64, and Top-P 0.95.
Responses can use the context space left after formatting the prompt, and FP16
is the runtime KV format. The HUD shows generation rate, token count, and
decode-service memory; Last run also shows time to first token and I/O. Build
the app with its sibling TurboFieldfareDecodeService; it never loads a second
in-process model. See README and Runtime controls.