LLM Launcher (llml) is a TUI for people who already have models on disk and are
tired of reconstructing launch commands from shell history.
It scans your local filesystem for GGUF, Hugging Face-style safetensors, and
NInfer (.ninfer) models, plus oMLX and Splash models on Apple Silicon,
detects installed runtimes
(llama.cpp,
vLLM,
Ollama,
KoboldCpp,
NInfer,
mlx-lm,
mlx-vlm,
oMLX, and
Splash),
plus the Strata engine packs it finds in a Strata install,
and lets you save named parameter profiles per model β so the command that worked
last time is always one keystroke away.
Browse local models. Detect the right runtime. Launch with one key.
Works alongside llama.cpp, vLLM, Ollama, KoboldCpp, NInfer, mlx-lm, mlx-vlm, oMLX, Splash, and Strata β not a replacement for them. llml is the profile manager for whatever backend you already run.
Note
Importable profiles are available at llml.dev. The catalog is actively curated β new profiles are added regularly. Watch the catalog repo (Watch β Custom β Releases) to get notified when new ones land.
- Model discovery β auto-scans common paths for GGUF files, safetensors model
directories, and NInfer artifacts (including the
models/directory of a configured NInfer checkout), models in oMLX's model directories, Splash bundles in the Hugging Face cache, and Strata engine configs (strata-<model>.json) in a Strata install; add extra roots viaLLML_MODEL_PATHSand/orconfig.toml. Results are cached under{UserConfigDir}/llml/cache/models.tomlso the next launch can skip the filesystem walk when the cache is still valid. - Runtime detection β finds installed
llama-server,vllm,koboldcpp,ninfer-serve,mlx_lm.server,mlx_vlm.server,omlx, andsplashbinaries, a Strata install'sserve/server.py, and installedollamaplus the configured Ollama host, then maps each model to its compatible runtime. GGUF models can use llama.cpp or KoboldCpp, safetensors models vLLM, mlx-lm, or mlx-vlm, via profile selection, and Strata engine packs run on Strata. - Named parameter profiles β save multiple profiles per model (e.g.
fast-laptop,quality,api-8080), each storing runtime args, env vars, port, and context settings. The active profile is always one key away. - Profile export β share your parameter profiles with others via the TUI
(
E) or CLI (llml export). Profiles are written to a portable TOML file matching the same schema thellml-importskill reads. Filter by model or profile name, toggle individual profiles or entire model groups, and handle file collisions (overwrite or auto-suffixed save). - Portable profile format β the shared import/export contract is documented in
docs/profile-format.mdand used by the repo-managedllml-importagent skill. - One-keystroke launch β s
10BC8
elect a model, select a profile, press
R. The generated command is shown before execution and server output streams directly in the TUI. - Persistent status and alert history β long-running work such as Ollama preloads stays visible in a persistent status line, while warnings and errors remain inspectable in a dedicated alert-history pane.
- Ollama preload flow β Ollama models are discovered via the Ollama API. Pressing
Rstartsollama serveif needed, then preloads the selected model into the shared Ollama service withkeep_alive: -1. - Zero required setup β common model directories and binary locations are checked automatically; configure only what differs from the defaults.
- Runtime engine (at least one): llama.cpp (
llama-server) or KoboldCpp (koboldcpp) for GGUF models, vLLM (vllm), mlx-lm (mlx_lm.server), or mlx-vlm (mlx_vlm.server) for safetensors models (the MLX runtimes on Apple Silicon and Linux), NInfer (ninfer-serve) for.ninferartifacts on Linux, oMLX (omlx) or Splash (splash) on Apple Silicon, Strata for its installed engine packs on Windows and Linux (NVIDIA / AMD), and/or Ollama (ollama) for Ollama models are installed (see Runtime Engines). - Models in default scan locations, or configure custom roots with
LLML_MODEL_PATHS(see Model Discovery).
Pick one path; you only need a single install method.
Install the latest version of llml directly to $HOME/.local/bin/ (or /usr/local/bin/ if run with sudo):
curl -fsSL https://llml.dev/install.sh | shRequires Go 1.26+. Ensure $(go env GOPATH)/bin is on your PATH.
go install github.com/flyingnobita/llml/cmd/llml@latestbrew tap flyingnobita/tap
brew install --cask llmlUpgrade later with brew upgrade --cask llml.
The macOS binary is not signed or notarized by Apple. The cask clears the quarantine flag Homebrew sets on downloads, so macOS opens it without asking. If you installed 0.8.0 or earlier and macOS says "llml" Not Opened, run brew upgrade --cask llml, or clear the flag yourself:
xattr -d com.apple.quarantine "$(brew --prefix)/Caskroom/llml/$(brew list --cask --versions llml | cut -d' ' -f2)/llml"llml is published to the maintainer bucket, not the default Scoop main bucket.
scoop bucket add flyingnobita https://github.com/flyingnobita/scoop-bucket
scoop install flyingnobita/llmlUpgrade later with scoop update llml.
After the package manifest merges into the public Windows Package Manager catalog:
winget install --id FlyingNobita.llmlUpgrade later with:
winget upgrade --id FlyingNobita.llmlFor each GitHub release, archives are published for Linux and macOS (tar.gz) plus Windows (zip). Names follow GoReleaserβs pattern, for example llml_1.2.3_Linux_x86_64.tar.gz, llml_1.2.3_Darwin_arm64.tar.gz, or llml_1.2.3_Windows_x86_64.zip (adjust version and OS/arch to match your download). Extract the llml binary or llml.exe.
# Example: Linux x86_64 β use the archive name from the release you downloaded
tar -xzf llml_1.2.3_Linux_x86_64.tar.gz
chmod +x llmlOn macOS, an archive downloaded with a browser is quarantined, and macOS refuses to open the unsigned binary ("llml" Not Opened). Clear the flag after extracting with xattr -d com.apple.quarantine llml. Archives fetched with curl, including by the install script, are not quarantined.
Install on your PATH if you like (Linux/macOS/WSL):
sudo mv llml /usr/local/bin/llmlNo-sudo option:
mkdir -p "$HOME/.local/bin"
mv llml "$HOME/.local/bin/llml"(Ensure ~/.local/bin is on your PATH.)
Verify the download against llml_<version>_checksums.txt on the release page if you rely on checksums (for example llml_1.2.3_checksums.txt).
- Go 1.26+
git clone https://github.com/flyingnobita/llml.git
cd llml
go build -o llml ./cmd/llmlTo install a development build from your clone, run mise run install-dev. It builds to ~/.local/bin/llml-dev, so it never shadows a released llml installed with Homebrew or the install script.
llml --version (and the footer of the ? shortcuts panel) names the running build. A release prints its version, such as 0.7.1. Any other build prints its commit, the commit's date, and modified if the tree had uncommitted changes, such as dev (16728e6, 2026-09-28, modified), so a stale build is easy to spot.
llmlllml will automatically scan common locations for models and binaries. If your setup is non-standard (e.g., binaries not on PATH or models in custom folders), see the Configuration section to point the app to the right directories. Select a model in the UI and press R to launch.
| Key | Action |
|---|---|
hjkl/ββββ |
Move selection; horizontal scroll when the path column is wider than the terminal |
E |
Open profile export modal β select profiles by model group, filter by name or backend, and save to a portable TOML file you can share across machines |
r |
Reload [runtime] from config.toml and re-detect binaries (no model rescan) |
S |
Full model filesystem rescan; refresh the discovery cache (cache/models.toml) |
R |
Run server (split view: table + log pane) |
ctrl+R |
Run server full-screen |
c |
Edit runtime environment (paths, ports) and turn runtimes on or off |
p |
Edit parameter profiles for the selected model |
m |
Edit extra model search paths (saved in config.toml) |
, / . |
Change sort column / reverse sort direction |
enter |
Copy the launch command for the selected row to the clipboard |
a |
Toggle alert history pane |
t |
Cycle theme (dark β light β auto β β¦) |
? |
Toggle the full shortcut help overlay |
q |
Quit |
R runs the server in a split layout: the model table stays in the upper half and server logs stream into a scrollable pane below. tab switches focus between the two panes; s stops the server. You can still quit with q or ctrl+c while a model is running, but llml will warn first and require explicit y confirmation before closing.
ctrl+R runs full-screen: the TUI is suspended and the server process is attached directly to your terminal. On Linux/macOS, after the server exits you are prompted to press Enter before the TUI redraws. On Windows there is no Enter prompt; you return when the server process exits.
For Ollama rows, R and ctrl+R do not start a dedicated per-model port.
Instead, llml ensures the shared Ollama daemon is running on the configured host
and preloads the selected model into memory with keep_alive: -1. The selected row
still matters, but the service endpoint remains the shared Ollama host.
llml separates active work from alert history:
- a persistent status line shows in-flight operations such as starting Ollama or loading a model
aopens a bottom alert-history pane with timestampedINFO,WARN, andERRORentries- when the pane is closed, the footer shows an unread alert count
Minor confirmations such as copy-to-clipboard remain transient. Important failures and lifecycle events remain inspectable in alert history until you dismiss or replace them by later work.
Each model path can have multiple named profiles. Each profile stores:
- Environment variables (
KEY=valueper line). - Extra arguments (
--flag valueper line).
R / ctrl+R use the active profile (the highlighted row in the p profile list is prefixed with (active) in the name column). Changes persist automatically. tab cycles: profile list β env β extra args. On the profile list: a add profile, c clone (duplicate) the highlighted profile, d delete (not the last), r rename. esc closes the panel (and n cancels a delete confirmation).
A profile's Backend chooses the runtime a model launches on. GGUF models choose llama.cpp or
KoboldCpp. Safetensors models choose vLLM, mlx-lm, or mlx-vlm, and also oMLX for models inside one
of oMLX's model directories.
(none) keeps the default: llama.cpp for GGUF, oMLX inside oMLX's model directories, vLLM
elsewhere. The table's Runtime column shows the choice. llml does not check that the weights suit
the runtime: MLX weights launched on vLLM fail when the server starts. A runtime that is turned off
in c is still offered, labelled (off), so you can set a profile up before turning it on.
Profiles are stored in model-params.json (see Storage & Locations).
This repo keeps the canonical llml-import skill in:
.agents/skills/llml-import/SKILL.md
That workspace path is already consumed directly by many agentic tools. The repo also tracks a Claude workspace compatibility copy at:
.claude/skills/llml-import/SKILL.md
Keep the canonical .agents file as the source of truth. When you update it, refresh
the Claude compatibility copy with:
./scripts/sync-skill --workspace --tool claudeTo install user-level copies for local agent tools, use:
./scripts/sync-skill --user --all-detectedThe llml-import skill imports launch settings from a URL or local file into llml
parameter profiles.
Typical flow:
- Make sure your coding agent can see the skill from
.agents/skills/llml-import/or from a user-level install created by./scripts/sync-skill. - Invoke the skill with a source URL or file path, for example:
/llml-import https://huggingface.co/...
- Review the extracted profiles, choose which ones to keep, and attach them to one of your local models.
- Open
llml, pressp, and inspect or activate the imported profiles.
What it imports:
- portable runtime args and env vars for
llama.cpp,vLLM, Ollama, KoboldCpp, NInfer, mlx-lm, mlx-vlm, oMLX, Splash, or Strata - profile names, model hints, and optional one-line descriptions
What it does not keep:
- model-location parameters such as
--model,-hf,HF_HOME, or similar path/cache settings, becausellmlsupplies the model path itself at launch time
Imported profiles are written into model-params.json. The shared portable contract
used by the skill is documented in docs/profile-format.md.
llml follows a strict precedence for settings:
Environment Variables (highest) β config.toml β Default/Auto-detected paths (lowest).
User data and settings are stored in a dedicated folder. Routine app upgrades do not delete these files.
| Platform | Path |
|---|---|
| Linux (XDG) | $XDG_CONFIG_HOME/llml/ (usually ~/.config/llml/) |
| macOS | ~/Library/Application Support/llml/ |
| Windows | %AppData%\llml\ |
Key Files:
config.toml: Stores your global settings (ports, binary paths, extra model roots). llml rewrites it only when you save a changed setting from thecormpanels, so comments and formatting survive model scans and runtime on/off toggles.cache/models.toml: The model discovery cache. Machine-owned and safe to delete; llml rebuilds it on the next scan.model-params.json: Stores your named parameter profiles (args/env) for each model.runtimes.toml: Which runtimes are on, as set with the checkboxes in thecpanel. Owned by llml, but not disposable like the cache: it is backed up before every overwrite. No environment variable overrides it.backups/: Automatic timestamped snapshots created before the app overwrites configuration.
Configure how llml finds and launches servers. You can edit these interactively in the UI (c).
The panel lists the runtimes this machine can run, grouped by model format, each marked running,
found, or not found. β/β pick a runtime, tab or β move into its fields, enter
saves, and esc discards. Under each field, an in use line shows the value llml runs with and
where it came from: the environment variable's name, config, default, or detected (a program
found with no path configured). An environment variable wins over the saved value until it is unset.
Each runtime has a checkbox. space (in the list only, never in a field) turns the highlighted
runtime on or off, and enter saves it with your field edits. A runtime you turn off shows off
in the list and "Off" in its detail pane: llml stops probing it and leaves it out of the
missing-runtime warning in the footer, and its fields stay editable. A runtime you tick back on shows no status mark until you
save, since llml has not probed it. Saving re-detects runtimes without rescanning models.
The on/off choices are stored in runtimes.toml, not config.toml, and no environment variable
overrides them. The first time llml sees a runtime, it turns it on only if it finds the program or
its server answers, and turns it off otherwise; a single alert lists any runtimes turned off this way.
After that only your choice changes it, so a runtime you install later stays off until you tick it.
Models whose runtime is off stay in the table, dimmed and marked (off) after the runtime name, so
you can still see what is taking up disk space. You can select them, copy their command, and open
their p panel, but R / ctrl+R will not launch them; an alert names the runtime to turn on
in c, and a note under the launch preview says the runtime is off. A GGUF model follows its
active profile: it is dimmed only when that profile's runtime (llama.cpp or KoboldCpp) is off, so
switching profiles can make it launchable again.
When Ollama is off, llml does not contact the Ollama daemon at all: startup and S neither start
ollama serve nor call its API. The Ollama models from the last scan stay in the table, dimmed, and
may be out of date until you turn Ollama back on and scan again.
| Feature | Environment Variable | config.toml key (under [runtime]) |
Default |
|---|---|---|---|
| llama.cpp path | LLAMA_CPP_PATH |
default_llama_cpp_path |
(auto) |
| llama.cpp port | LLAMA_SERVER_PORT |
default_llama_server_port |
8080 |
| vLLM path | VLLM_PATH |
default_vllm_path |
(auto) |
| vLLM venv | VLLM_VENV |
default_vllm_venv |
(auto) |
| vLLM port | VLLM_SERVER_PORT |
default_vllm_server_port |
8000 |
| Ollama path | OLLAMA_PATH |
default_ollama_path |
(auto) |
| Ollama host | OLLAMA_HOST |
default_ollama_host |
127.0.0.1:11434 |
| KoboldCpp path | KOBOLDCPP_PATH |
default_koboldcpp_path |
(auto) |
| KoboldCpp port | KOBOLDCPP_PORT |
default_koboldcpp_port |
5001 |
| NInfer path | NINFER_PATH |
default_ninfer_path |
(auto) |
| NInfer port | NINFER_SERVER_PORT |
default_ninfer_server_port |
8080 |
| NInfer host | NINFER_SERVER_HOST |
default_ninfer_server_host |
127.0.0.1 |
| oMLX path | OMLX_PATH |
default_omlx_path |
(auto) |
| oMLX port | OMLX_PORT |
default_omlx_port |
8000 |
| oMLX host | OMLX_HOST |
default_omlx_host |
127.0.0.1 |
| oMLX model dirs | OMLX_MODEL_DIRS |
- | (from oMLX app) |
| Splash path | SPLASH_PATH |
default_splash_path |
(auto) |
| Splash port | SPLASH_PORT |
default_splash_port |
8000 |
| Splash host | SPLASH_HOST |
default_splash_host |
127.0.0.1 |
| mlx-lm path | MLX_LM_PATH |
default_mlx_lm_path |
(auto) |
| mlx-lm port | MLX_LM_PORT |
default_mlx_lm_port |
8080 |
| mlx-lm host | MLX_LM_HOST |
default_mlx_lm_host |
127.0.0.1 |
| mlx-vlm path | MLX_VLM_PATH |
default_mlx_vlm_path |
(auto) |
| mlx-vlm port | MLX_VLM_PORT |
default_mlx_vlm_port |
8080 |
| mlx-vlm host | MLX_VLM_HOST |
default_mlx_vlm_host |
127.0.0.1 |
| Strata path | STRATA_PATH |
default_strata_path |
(auto) |
| Strata port | STRATA_PORT |
default_strata_port |
8080 |
| Strata host | STRATA_HOST |
default_strata_host |
127.0.0.1 |
| TUI Theme | LLML_THEME |
- | auto |
Detection Logic:
- Explicitly configured paths (Env/TOML).
- Common system directories (e.g.,
/usr/local/bin,/opt/homebrew/bin,~/.local/bin). - Binary names available on your system
PATH. - Probing each runtime's port for an already-running server. Several runtimes share a
default port (llama.cpp, NInfer, mlx-lm, and mlx-vlm on
8080; vLLM, oMLX, and Splash on8000), so llml checks which server answers: NInfer, oMLX, and Splash by the owner they report in/v1/models, mlx-lm by a/healthstatus ofokfrom Python's HTTP server (aServerheader startingBaseHTTP/), mlx-vlm by a/healthstatus ofhealthy, llama.cpp by a/healthanswer from a server that is not NInfer, mlx-lm, or mlx-vlm, and vLLM by an answer from a server that is not oMLX, Splash, mlx-lm, or mlx-vlm. Strata shares 8080 too and names itself in its/healthbody ("service": "strata"). - (vLLM only) Common venv locations (e.g.,
~/.venv-vllm-metal/binon macOS). - (KoboldCpp only) Platform-specific name variants with CUDA preference on Linux.
- (NInfer only) NInfer has no install target, so
NINFER_PATHmay point at the checkout root (llml looks inbuild/apps/), the build directory, orninfer-serveitself. The checkout'smodels/directory is added to the scan roots. The default port is8080, ninfer-serve's own default, which llama-server shares; only one of them can listen on it at a time. - (oMLX only) llml uses the CLI shim the oMLX app installs (
~/.omlx/bin/omlx) and reads the app's model directories from itssettings.json, so models placed there appear as oMLX rows.Rrunsomlx serve --model-dir <dir>; oMLX serves every model in that directory and applies the per-model settings (draft model, sampling) saved in the app. Stop the app's own background server first if it is using the same port. A profile can launch such a model on vLLM instead (see Parameter profiles). - (Splash only) Splash bundles are found in the Hugging Face cache and launched with
splash serve --model <org/repo>. - (mlx-lm only) llml looks for the
mlx_lm.serverscript atMLX_LM_PATH(the script, or the folder holding it) and then onPATH. It does not search for venvs, and there is no common-directory step. See Setting up mlx-lm. - (mlx-vlm only) The same for the
mlx_vlm.serverscript andMLX_VLM_PATH. See Setting up mlx-vlm. - (Strata only)
STRATA_PATHmay be the Strata install directory, itsserve/server.py, itsengine/strata, or one of its engine config files. When unset, llml reads the installs from Strata's ownsettings.jsonand also checks~/Applications/strataand~/Strata. Eachstrata-<model>.jsonengine config becomes a row; R runs the install's venv Python withserve/server.py --engine strata --config <config>.
NInfer is shown only on Linux, oMLX and Splash only on Apple Silicon macOS, mlx-lm and mlx-vlm
only on Apple Silicon macOS and Linux (x86_64 and aarch64), and Strata only on Linux and Windows
(the platforms its NVIDIA / AMD engines ship for); the c panel hides runtimes the platform
cannot run. A runtime turned off in c gets no server probe, but
llml still looks for its program on disk so the panel can show whether it is installed.
mlx-lm runs a safetensors model directory with Apple's MLX. To use it, choose mlx-lm as the
Backend of a safetensors model's profile in p. R then runs:
mlx_lm.server --model <absolute model path> --host <host> --port <port> <profile args>llml adds nothing else, and passes the profile's env vars as for any runtime.
- macOS (Apple Silicon): install
mlx-lmwith pip, uv, or mise into any Python environment, for examplepip install mlx-lmoruv tool install mlx-lm. Ifmlx_lm.serveris not on yourPATH, setMLX_LM_PATH(or the Path field inc) to the script or the folder that holds it, such as a venv'sbin/. The venv does not need to be activated: the script starts its own Python. - Linux (x86_64 and aarch64): a plain
pip install mlx-lminstalls no MLX backend on Linux, so the server cannot start. Install a backend next to it:pip install mlx-lm "mlx[cuda12]"for CUDA 12,"mlx[cuda13]"for CUDA 13, or"mlx[cpu]"without a GPU. The CUDA build needs an NVIDIA GPU of compute capability 7.5 or newer and a recent driver. The CPU build works but is far too slow for real models. MLX compiles its CUDA kernels at run time against the toolkit thatCUDA_PATH(orCUDA_HOME) points to, so that toolkit must match the extra: withmlx[cuda12]and a CUDA 13 toolkit onCUDA_PATH, the server starts and answers/health, but the first request hangs while kernels fail to compile. Install the matching extra, or setCUDA_PATHto a CUDA 12 toolkit in the profile's env (for exampleCUDA_PATH=/opt/cuda-12.9.1).
Things to know:
- Answer length: mlx-lm stops each answer at 512 tokens unless the profile passes
--max-tokens(for example--max-tokens 8192) or the client sendsmax_tokens. - Model id: the launch preview shows the model id clients must send, which is the model's
absolute path. mlx-lm loads whatever model a request names, and downloads it from Hugging
Face if it is not a local folder, so a client that sends another id makes it switch models.
Leaving
modelout of the request also uses the launched model. - Missing folders: llml will not launch mlx-lm on a model folder that no longer exists, since the server would take the path for a Hugging Face repo id. Press S to rescan.
- Port: mlx-lm's default port is
8080, the same as llama.cpp's and NInfer's. Only one server can listen on it at a time; setMLX_LM_PORTto run them side by side.
mlx-vlm runs a safetensors model directory with Apple's MLX, including vision and audio
models. To use it, choose mlx-vlm as the Backend of a safetensors model's profile in
p. R then runs:
mlx_vlm.server --model <absolute model path> --host <host> --port <port> <profile args>llml adds nothing else, and passes the profile's env vars as for any runtime. It always
passes --host (127.0.0.1 unless you set MLX_VLM_HOST), because mlx-vlm otherwise
listens on all interfaces.
- Version: mlx-vlm 0.6.5 or later is needed to serve text-only models. Older releases serve only the vision model families they know.
- macOS (Apple Silicon): install
mlx-vlmwith pip, uv, or mise into any Python environment, for examplepip install mlx-vlmoruv tool install mlx-vlm. Ifmlx_vlm.serveris not on yourPATH, setMLX_VLM_PATH(or the Path field inc) to the script or the folder that holds it, such as a venv'sbin/. The venv does not need to be activated. - Linux (x86_64 and aarch64): a plain
pip install mlx-vlminstalls MLX without a backend, soimport mlx.corefails and the server cannot start. Install a backend next to it:pip install mlx-vlm "mlx[cuda12]"for CUDA 12,"mlx[cuda13]"for CUDA 13, or"mlx[cpu]"without a GPU. Do not usepip install "mlx-vlm[cuda]": it pulls an outdatedmlx-cuda(0.30.0) beside a newermlx. The CUDA and CPU notes for mlx-lm apply here too.
Things to know:
- Model id: the launch preview shows the model id clients must send, which is the model's
absolute path. Clients must send exactly that id: mlx-vlm loads whatever model a request
names, and downloads it from Hugging Face if it is not a local folder, so any other id makes
it swap or download a model. Unlike mlx-lm, leaving
modelout of the request does not use the launched model. - Missing folders: llml will not launch mlx-vlm on a model folder that no longer exists, since the server would take the path for a Hugging Face repo id. Press S to rescan.
- Port: mlx-vlm's default port is
8080, the same as llama.cpp's, NInfer's, and mlx-lm's. SetMLX_VLM_PORTto run them side by side.
Strata runs its own large Qwen3.8-Flash-Next model on an
NVIDIA or AMD card, on Windows or Linux. Install it with its START-HERE.bat / ./setup.sh;
setup writes one engine config per model (strata-<model>.json) in the Strata folder. llml
discovers each config as a row and R runs:
<install>/.venv/bin/python <install>/serve/server.py \
--engine strata --config <install>/strata-<model>.json \
--host <host> --port <port> <profile args>- Path: when
STRATA_PATH(or thecpanel's Path field) is unset, llml reads the installs from Strata's ownsettings.jsonand also checks~/Applications/strataand~/Strata. Set it to the install directory (or itsserve/server.py, itsengine/strata, or an engine config) when your install lives elsewhere. - Port: Strata's setup default is
8080, which its own/healthbody distinguishes (it names"service": "strata"). SetSTRATA_PORTto move it. - Platform: shown on Windows and Linux only, the platforms Strata ships NVIDIA / AMD engines for.
- Profiles: the engine config already carries the model, engine args, tokenizer, and data
paths, so a profile for a Strata row usually just adds server flags. Model-location flags
(
--config,--engine,--pack,--native,--mtp, β¦) are stripped on import.
llml scans several common directories (e.g., ~/models, ~/.cache/llama.cpp, ~/.cache/lm-studio/models) by default.
Add extra directories to scan via the UI (m), config.toml (discovery.extra_model_paths), or environment variables:
| Environment Variable | Role |
|---|---|
LLML_MODEL_PATHS |
Comma-separated list of extra roots to scan. Example: export LLML_MODEL_PATHS="/data/models,/opt/weights" |
HUGGINGFACE_HUB_CACHE |
Specific root for Hugging Face hub cache (overrides HF_HOME/hub). |
HF_HOME |
Root for HF home (cache resolves to $HF_HOME/hub). |
Example config.toml entry:
[discovery]
extra_model_paths = ["/data/models", "/opt/weights"]The app recognizes the following model types:
- GGUF: Files ending in
.gguf. - Safetensors: Directories containing
config.jsonand*.safetensorsfiles (Hugging Face style). - NInfer: Files ending in
.ninfer. - Splash: Bundles in the Hugging Face cache.
- Strata:
strata-<model>.jsonengine configs in a Strata install (read directly, not walked).
To protect your settings and cache, llml maintains a history of your configuration:
- Atomic Writes: Files are written to a temporary location before being moved, preventing corruption.
- Automatic Backups: The newest 10 versions of each file are kept in the
backups/directory. - Upgrade Snapshots: When the
llmlversion changes, a snapshot ofconfig.toml,model-params.json, andruntimes.tomlis created automatically so you can roll back if needed. The file.last-run-versionin the same directory records the last run version for that behavior.
- mise β manages Go, Node.js, and tasks. Install mise first.
- Go 1.26+ and Node.js (LTS) β both installed automatically by
mise install.
Clone the repository and install tooling:
mise install # installs Go, Node.js, pre-commit (pipx), and other tools from mise.toml
npm ci # installs Prettier + markdownlint
pre-commit install # optional: enable git pre-commit / pre-push hooks (see .pre-commit-config.yaml)mise run run # go run ./cmd/llml
mise run build # build to bin/llml
mise run install-dev # build to ~/.local/bin/llml-dev
mise run format # auto-fix: gofmt + prettier + markdownlint
mise run lint # check only: gofmt + vet + prettier + markdownlint
mise run test # go test -race ./...
mise run check # lint + test (run before opening a PR)cmd/llmlβ entrypoint.internal/configβconfig.tomlread/write and cache helpers.internal/tuiβ Bubble Tea UI.internal/modelsβ discovery, metadata, runtime detection.scripts/βgofmt-check.sh,precommit-docs-fix.sh.
Contributions are welcome. Please run mise run check before opening a pull request.
You can support this project in a few simple ways:
- β Star the repo
- π Report bugs
- π‘ Suggest features
- π Contribute code
- π¦ Contribute a profile to the catalog β share a working parameter set with others who run the same model
MIT Β© Flying Nobita