Repository navigation
feat(scaling): Linux hosts over SSH as a provider - #589
Merged
Merged
Conversation
Each unit is the first-boot script started in its own process group on one of the provider's hosts, up to per_host each; letting it go stops the group. Host keys are trusted on first use when an admin saves or checks the provider, kept when a later check fails, and held to on every call. Paramiko is the ssh extra, fetched when the first SSH provider is saved. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ALjNbLobvPe7VwvDL8j7V3
…-azm2mb # Conflicts: # README.md # docs/requirements.md # docs/scaling.md # fastapi_backend/uv.lock
The mesh's install line landed (#591), so a group on a cloud, Proxmox or SSH provider could launch, but a node runs no `lens worker` yet: its units would never count as joined and only cost until given up on. can_join now also needs the mesh to say its nodes run workers (WORKER_NODES), and waiting_for says which of the three it waits for. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ALjNbLobvPe7VwvDL8j7V3
…-azm2mb # Conflicts: # README.md # docs/scaling.md # fastapi_backend/app/domain/scaling.py # fastapi_backend/tests/api/test_scaling_clouds.py
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Before: auto scaling could add cloud machines (AWS, DigitalOcean, Hetzner) and Proxmox clones, but not Linux machines you already have.
After: a provider of kind "Linux hosts over SSH" takes a list of
user@host[:port]and a private key. Each unit is the first-boot script (the mesh's install line, which makes the host a worker node) started on one of those hosts, up to "Workers on each host" per host; letting the unit go stops its process group. Nothing is created or deleted, so it costs nothing unless you give the provider a price.How:
scaling_ssh.pyuses Paramiko (an opt-insshextra, fetched the first time an SSH provider is saved). Host keys are trusted on first use when the provider is saved or checked, recorded on the provider (check.host_keys, kept across a failed check via the newDriver.kept), and every other call refuses a host whose key changed. The first-boot script is written withumask 077to~/.lens-scaling/<name>.shand deleted by the unit as it starts, since it holds the join token. Like the clouds and Proxmox, the group waits for the mesh (needs_join,can_joinfrom #595).Motivation and Context
Part of the auto scaling ask (remote Linux instances as worker nodes); follows #556, #573, #579 and #559.
Types of changes
Checklist:
test_scaling_ssh.py: host specs, trust on first use and refusal of a changed key, start/count/stop against a stand-in that runs the real shell commands).docs/scaling.mdupdated (driver table, "Machines elsewhere", status).Done questions (AGENTS.md)
per_hostdefaults to 1.user_dataand sealed secrets.Local CI (
ci/local-ci.sh) on 00c753d: every job passed except FastAPI (SurrealDB server), where 1 of 1730 tests failed with a websocket HTTP 502 intest_mesh.py(this container's server). After merging main: scaling tests pass locally (44), ruff clean, requirements and threat model checks pass.🤖 Generated with Claude Code
https://claude.ai/code/session_01ALjNbLobvPe7VwvDL8j7V3