Repository navigation
Conversation
🔬 Flycrocosm E2E: planThis PR adds What I'll check All logins use the email/password path (
What I won't check
Testing on Flycrocosm |
❌ Flycrocosm E2E: the PR does not match its plan
|
| Expected | Observed | |
|---|---|---|
fly machine stop <id> (rwcC) |
exits 0, machine stops | exits 0, machine stops, but prints failed to release lease for machine <id>: unauthorized |
fly machine start <id> (rwcC), once stopped |
exits 0, machine starts | exits 1: failed to obtain lease: ... lease currently held by <uuid>@tokens.fly.io, expires at ... |
- The API treats releasing a machine lease as a delete, so a token without
dcannot
release the 120s lease flyctl takes forstop,start,destroyand similar commands.
The next lease-taking command on that machine fails until the lease expires. - This answers the open question in the description ("Whether releasing a machine lease
counts asd"): it does, and it affects plainmachine stop/start, not only deploys. - Control: the harness's unrestricted token runs the same stop → start sequence without
error in the same script. startitself is allowed under rwcC: after the script clears the stuck lease with the
harness token, the restrictedfly machine startsucceeds (and again fails to release
its own lease).
What passed
- Flag guards.
--org,--maskand--expirywithout--restrictedeach fail with
--<flag> requires --restricted. With--restricted,--mask rwx,--org nopeand
--expiry 0sfail withinvalid mask,organization nope not foundand
--expiry must be positive, and nothing is saved. - Restricted login.
--restricted --org personal --expiry 1hwith no--maskand no
TTY printsrestricted to personal with permissions rwcC, expiring in 1h0m0sand
successfully logged in as dev@fly.local, with no "Failed revoking" warning. The saved
access_tokenis twofm2_tokens.fly tokens listshows exactly one new
flyctl restricted logintoken, expiring in 1h. - "No deletes" is enforced. With the saved token,
fly machine listand
fly machine stopwork;fly machine destroy --forceandfly apps destroyare
rejected asunauthorized, and the machine and app still exist. --mask ris read-only. The login printspermissions r, expiring in 720h0m0s.
fly apps listandfly machine listwork;fly machine stopand
fly apps create e2e-pr-1-roare rejected asunauthorized.
Not verified, and caveats
- User token not saved. Flycrocosm's email/password login returns an
fm2_token, not
fo1_, so the prefix check cannot tell the user token from the org macaroons. The
evidence that the saved bundle is the attenuated one is that deletes are rejected with it. - User token revoked. The script never sees the user token, so it only checks that no
"Failed revoking" warning is printed. - Not exercised, as stated in the plan: browser login, the interactive org and preset
prompts,fly auth logout, and deploys under a restricted session. - Test image. The API returned a 500 resolving
nginx:alpine(about 14s per attempt),
so the test machine runsregistry.fly.io/dashboard-demo:e2efrom the local registry,
withnginx:alpineas the fallback. - From reading the diff, not tested. When a restricted login fails after
authentication (bad--mask,--orgor--expiry),restrictTokenreturns before
revokeUserToken, so that session's user token is neither saved nor revoked. - Cleanup. The harness token is not allowed to revoke tokens, so the script does a
plain login in its temporary config dir and revokes the two org tokens it created
(cleanup: 2 tokens revoked).
Provenance
| Flycrocosm | 127ec8d (dirty), topology standard |
| flyctl-hackathon | head 37b5a9805, base 33ed7a3a3 |
| Built | binary=bin/flyctl.real version=flyctl.real v2026.10.3-dev.1791010324 linux/amd64 Commit: <none> BuildDate: 2026-10-03T19:52:04+13:00 BranchName: dev |
| Test | demo.sh sha256 d3f130da3c84 |
| Independent run | asciinema recording, exit 1 |
Generated test (demo.sh)
#!/usr/bin/env bash
# Demo/verification for `fly auth login --restricted` (PR 1) against Flycrocosm.
set -euo pipefail
EMAIL="dev@fly.local"
PASSWORD="1q2w3e4r"
ORG="personal"
APP="e2e-pr-1-restricted"
RO_APP="e2e-pr-1-ro"
TOKEN_NAME="flyctl restricted login"
IMAGES="registry.fly.io/dashboard-demo:e2e nginx:alpine"
DENIED='unauthorized|not authorized|forbidden'
WORK="$(mktemp -d)"
# Logins save into an isolated config dir, never the real one.
export FLY_CONFIG_DIR="$WORK/fly"
mkdir -p "$FLY_CONFIG_DIR"
CONFIG="$FLY_CONFIG_DIR/config.yml"
TOKENS_BEFORE="$WORK/tokens.before"
FAILURES=0
fail() { echo "FAIL: $*"; exit 1; }
# Record a failed check but keep running; the script still exits non-zero.
soft_fail() { echo "FAIL: $*"; FAILURES=$((FAILURES + 1)); }
pass() { echo "PASS: $*"; }
OUT=""
RC=0
# run CMD...: no TTY on stdin/stdout, capture combined output in OUT and exit code in RC.
capture() {
set +e
OUT="$("$@" 2>&1 </dev/null)"
RC=$?
set -e
}
run() {
capture "$@"
[ -z "$OUT" ] || printf '%s\n' "$OUT" | sed 's/^/ /'
}
# list_machines TOKEN: `fly machine list --json` with TOKEN, printed as "id state region".
list_machines() {
capture fly machine list --app "$APP" --json --access-token "$1"
if [ "$RC" -eq 0 ]; then
OUT="$(jq -r '.[] | "\(.id) \(.state) \(.region)"' <<<"$OUT")"
fi
[ -z "$OUT" ] || printf '%s\n' "$OUT" | sed 's/^/ /'
}
quiet() { "$@" >/dev/null 2>&1 </dev/null; }
expect_ok() { [ "$RC" -eq 0 ] || fail "$1 (exit $RC)"; }
expect_out() { grep -qF -- "$1" <<<"$OUT" || fail "expected output to contain: $1"; }
expect_denied() {
[ "$RC" -ne 0 ] || fail "$1 succeeded, expected the API to reject it"
grep -qiE -- "$DENIED" <<<"$OUT" || fail "$1 failed, but not with an authorization error"
}
login() { run fly auth login --email "$EMAIL" --password "$PASSWORD" "$@"; }
saved_token() { [ -f "$CONFIG" ] && sed -n 's/^access_token: *//p' "$CONFIG" | tr -d "\"'" || true; }
# IDs of the org's "flyctl restricted login" tokens (harness token).
restricted_token_ids() {
local list
list="$(fly tokens list --scope org --org "$ORG" </dev/null 2>&1)" || { echo "$list" >&2; return 1; }
{ grep -F "$TOKEN_NAME" <<<"$list" || true; } | awk '{print $1}' | sort
}
machine_state() { fly machine list --app "$APP" --json </dev/null 2>/dev/null | jq -r --arg id "$MID" '.[] | select(.id == $id) | .state'; }
wait_state() {
local want="$1" state=""
for _ in $(seq 1 30); do
state="$(machine_state || true)"
[ "$state" = "$want" ] && return 0
sleep 2
done
fail "machine $MID is '$state', expected '$want'"
}
# Drop any lease left on the machine (harness token), so a later rejection is
# about permissions and not about a held lease.
clear_lease() { quiet fly machine leases clear "$MID" --app "$APP" || true; }
app_exists() { fly apps list --json </dev/null 2>/dev/null | jq -e --arg n "$1" 'any(.[]; (.Name // .name) == $n)' >/dev/null; }
cleanup() {
set +e
quiet fly apps destroy "$APP" --yes
quiet fly apps destroy "$RO_APP" --yes
# Revoke the org tokens this run created. The harness token may not revoke
# tokens, so use a plain login saved in the isolated config dir.
local ids=""
[ -f "$TOKENS_BEFORE" ] && ids="$(restricted_token_ids 2>/dev/null | comm -13 "$TOKENS_BEFORE" -)"
if [ -n "$ids" ] && quiet fly auth login --email "$EMAIL" --password "$PASSWORD"; then
# shellcheck disable=SC2086
fly tokens revoke $ids --access-token "$(saved_token)" </dev/null 2>&1 | grep "tokens revoked" | sed 's/^/cleanup: /'
fi
rm -rf "$WORK"
}
trap cleanup EXIT
echo "==> PR binary under test"
run fly version
expect_ok "fly version"
# ---------------------------------------------------------------------------
echo
echo "==> 1. Flag guards: --org/--mask/--expiry need --restricted"
for args in "--org $ORG" "--mask r" "--expiry 1h"; do
flagname="${args%% *}"
echo "--> fly auth login --email $EMAIL --password *** $args"
# shellcheck disable=SC2086
login $args
[ "$RC" -ne 0 ] || fail "login with $flagname but without --restricted exited 0"
expect_out "$flagname requires --restricted"
pass "$flagname without --restricted is rejected"
done
echo
echo "==> 1b. Invalid values with --restricted fail and save nothing"
echo "--> fly auth login --restricted --org $ORG --mask rwx"
login --restricted --org "$ORG" --mask rwx
[ "$RC" -ne 0 ] || fail "--mask rwx exited 0"
expect_out "invalid mask"
pass "--mask rwx is rejected as an invalid mask"
echo "--> fly auth login --restricted --org nope"
login --restricted --org nope
[ "$RC" -ne 0 ] || fail "--org nope exited 0"
expect_out "organization nope not found"
pass "--org nope is rejected as not found"
echo "--> fly auth login --restricted --org $ORG --expiry 0s"
login --restricted --org "$ORG" --expiry 0s
[ "$RC" -ne 0 ] || fail "--expiry 0s exited 0"
expect_out "--expiry must be positive"
pass "--expiry 0s is rejected"
[ -z "$(saved_token)" ] || fail "a token was saved to $CONFIG by a failed login"
pass "no access token was saved by the failed logins"
# ---------------------------------------------------------------------------
echo
echo "==> 2. Restricted login (default preset) saves macaroons only"
restricted_token_ids > "$TOKENS_BEFORE" || fail "fly tokens list failed with the harness token"
echo "--> fly auth login --restricted --org $ORG --expiry 1h (no --mask, no TTY)"
login --restricted --org "$ORG" --expiry 1h
expect_ok "restricted login"
expect_out "restricted to $ORG with permissions rwcC, expiring in 1h0m0s"
expect_out "successfully logged in as $EMAIL"
! grep -qF "Failed revoking" <<<"$OUT" || fail "the user token could not be revoked"
pass "login succeeded with the \"No deletes\" (rwcC) default and no revocation warning"
ND_TOKEN="$(saved_token)"
[ -n "$ND_TOKEN" ] || fail "no access_token saved in $CONFIG"
! grep -q "fo1_" "$CONFIG" || fail "an fo1_ user token was saved in $CONFIG"
n_total=0
IFS=',' read -ra parts <<<"${ND_TOKEN#FlyV1 }"
for p in "${parts[@]}"; do
n_total=$((n_total + 1))
[[ "$p" == fm2_* ]] || fail "saved access_token has a non-macaroon part (prefix ${p:0:4})"
done
echo " saved access_token: $n_total token(s), all fm2_ macaroons"
pass "saved config holds only fm2_ macaroons, no fo1_ user token"
echo "--> fly tokens list --scope org --org $ORG (harness token)"
restricted_token_ids > "$WORK/tokens.after" || fail "fly tokens list failed with the harness token"
new_ids="$(comm -13 "$TOKENS_BEFORE" "$WORK/tokens.after")"
n_new="$(grep -c . <<<"$new_ids" || true)"
echo " new \"$TOKEN_NAME\" tokens: ${new_ids:-none}"
[ "$n_new" -eq 1 ] || fail "expected 1 new \"$TOKEN_NAME\" org token, found $n_new"
pass "one new \"$TOKEN_NAME\" token exists for $ORG"
row="$(fly tokens list --scope org --org "$ORG" </dev/null 2>/dev/null | grep -F "$new_ids")"
expires="$(awk -F'│' '{print $4}' <<<"$row" | sed 's/ UTC//' | xargs)"
revoked="$(awk -F'│' '{print $5}' <<<"$row" | xargs)"
echo " expires at: $expires revoked at: ${revoked:-never}"
ttl=$(( $(date -u -d "$expires" +%s) - $(date -u +%s) ))
[ "$ttl" -gt 3300 ] && [ "$ttl" -le 3600 ] || fail "token expires in ${ttl}s, expected about 3600s (--expiry 1h)"
[ -z "$revoked" ] || fail "the new org token is already revoked"
pass "the org token expires in 1h and is not revoked"
# ---------------------------------------------------------------------------
echo
echo "==> 3. \"No deletes\" is enforced by the API"
echo "--> setup (harness token): app $APP with one machine in dc1"
quiet fly apps destroy "$APP" --yes || true
quiet fly apps destroy "$RO_APP" --yes || true
run fly apps create "$APP" --org "$ORG"
expect_ok "creating $APP"
# Prefer an image already in the local registry; resolving Docker Hub images
# depends on the stack's outbound network.
for image in $IMAGES; do
run fly machine run "$image" --app "$APP" --region dc1 \
--vm-size shared-cpu-1x --vm-memory 256 --detach
[ "$RC" -ne 0 ] || break
done
expect_ok "creating the machine"
MID="$(fly machine list --app "$APP" --json </dev/null | jq -r '.[0].id')"
[ -n "$MID" ] && [ "$MID" != null ] || fail "no machine found in $APP"
wait_state started
echo " machine $MID is started"
echo "--> control (harness token): stop, then start as soon as it is stopped"
run fly machine stop "$MID" --app "$APP"
expect_ok "machine stop with the harness token"
wait_state stopped
run fly machine start "$MID" --app "$APP"
expect_ok "machine start with the harness token"
wait_state started
pass "an unrestricted token can stop and then start the machine"
echo "--> fly machine list (restricted token)"
list_machines "$ND_TOKEN"
expect_ok "machine list with the restricted token"
expect_out "$MID"
pass "restricted token can list machines"
echo "--> fly machine stop $MID (restricted token)"
run fly machine stop "$MID" --app "$APP" --access-token "$ND_TOKEN"
expect_ok "machine stop with the restricted token"
wait_state stopped
pass "restricted token can stop the machine"
STOP_OUT="$OUT"
echo "--> fly machine start $MID (restricted token, once it is stopped)"
run fly machine start "$MID" --app "$APP" --access-token "$ND_TOKEN"
if [ "$RC" -eq 0 ]; then
wait_state started
pass "restricted token can start the machine"
else
soft_fail "restricted token cannot start the machine it just stopped (exit $RC)"
if grep -q "failed to release lease" <<<"$STOP_OUT" && grep -q "lease currently held" <<<"$OUT"; then
echo " cause: the stop could not release its machine lease (unauthorized without 'd'),"
echo " so the machine stays locked until that 120s lease expires."
fi
# Keep going so the remaining checks are still shown; the run fails at the end.
echo "--> clear the stuck lease (harness token), then retry the start (restricted token)"
run fly machine leases clear "$MID" --app "$APP"
expect_ok "clearing the lease with the harness token"
run fly machine start "$MID" --app "$APP" --access-token "$ND_TOKEN"
expect_ok "machine start with the restricted token after clearing the lease"
wait_state started
echo " start itself is allowed once the lease is cleared"
fi
clear_lease
echo "--> fly machine destroy $MID --force (restricted token)"
run fly machine destroy "$MID" --force --app "$APP" --access-token "$ND_TOKEN"
expect_denied "machine destroy"
[ -n "$(machine_state)" ] || fail "machine $MID is gone after the rejected destroy"
pass "machine destroy is rejected and the machine still exists"
clear_lease
echo "--> fly apps destroy $APP --yes (restricted token)"
run fly apps destroy "$APP" --yes --access-token "$ND_TOKEN"
expect_denied "apps destroy"
app_exists "$APP" || fail "app $APP is gone after the rejected destroy"
pass "apps destroy is rejected and the app still exists"
# ---------------------------------------------------------------------------
echo
echo "==> 4. --mask r is read-only"
echo "--> fly auth login --restricted --org $ORG --mask r"
login --restricted --org "$ORG" --mask r
expect_ok "read-only restricted login"
expect_out "restricted to $ORG with permissions r, expiring in 720h0m0s"
expect_out "successfully logged in as $EMAIL"
RO_TOKEN="$(saved_token)"
[ -n "$RO_TOKEN" ] || fail "no access_token saved in $CONFIG"
! grep -q "fo1_" "$CONFIG" || fail "an fo1_ user token was saved in $CONFIG"
[ "$RO_TOKEN" != "$ND_TOKEN" ] || fail "the second login did not replace the saved token"
pass "read-only login succeeded with the 30 day default expiry"
echo "--> fly apps list (read-only token)"
run fly apps list --access-token "$RO_TOKEN"
expect_ok "apps list with the read-only token"
expect_out "$APP"
pass "read-only token can list apps"
clear_lease
echo "--> fly machine list (read-only token)"
list_machines "$RO_TOKEN"
expect_ok "machine list with the read-only token"
expect_out "$MID"
pass "read-only token can list machines"
echo "--> fly machine stop $MID (read-only token)"
run fly machine stop "$MID" --app "$APP" --access-token "$RO_TOKEN"
expect_denied "machine stop"
[ "$(machine_state)" = started ] || fail "machine $MID is no longer started after the rejected stop"
pass "machine stop is rejected and the machine is still started"
echo "--> fly apps create $RO_APP (read-only token)"
run fly apps create "$RO_APP" --org "$ORG" --access-token "$RO_TOKEN"
expect_denied "apps create"
! app_exists "$RO_APP" || fail "app $RO_APP was created with the read-only token"
pass "apps create is rejected and $RO_APP does not exist"
echo
echo "(ran in ${SECONDS}s)"
if [ "$FAILURES" -gt 0 ]; then
echo "==> $FAILURES check(s) failed"
exit 1
fi
echo "==> All checks passed"
Test output
--> control (harness token): stop, then start as soon as it is stopped
Sending kill signal to machine 024d899f4e8793...
024d899f4e8793 has been successfully stopped
024d899f4e8793 has been started
PASS: an unrestricted token can stop and then start the machine
--> fly machine list (restricted token)
024d899f4e8793 started dc1
PASS: restricted token can list machines
--> fly machine stop 024d899f4e8793 (restricted token)
Sending kill signal to machine 024d899f4e8793...
024d899f4e8793 has been successfully stopped
failed to release lease for machine 024d899f4e8793: unauthorized
PASS: restricted token can stop the machine
--> fly machine start 024d899f4e8793 (restricted token, once it is stopped)
Waiting on lease for machine 024d899f4e8793...
Error: failed to obtain lease: failed to get lease on VM 024d899f4e8793: machine ID 024d899f4e8793 lease currently held by d696615f-9397-52ed-b355-5a9047f0d6f4@tokens.fly.io, expires at 2026-10-03T07:09:43Z (Request ID: 01M409G24J4W1SRF0K71P1RZDP-dc0)
FAIL: restricted token cannot start the machine it just stopped (exit 1)
cause: the stop could not release its machine lease (unauthorized without 'd'),
so the machine stays locked until that 120s lease expires.
--> clear the stuck lease (harness token), then retry the start (restricted token)
clearing lease for machine 024d899f4e8793
Lease(s) cleared
024d899f4e8793 has been started
failed to release lease for machine 024d899f4e8793: unauthorized
start itself is allowed once the lease is cleared
--> fly machine destroy 024d899f4e8793 --force (restricted token)
machine 024d899f4e8793 was found and is currently in started state, attempting to destroy...
failed to release lease for machine 024d899f4e8793: unauthorized
Error: could not destroy machine 024d899f4e8793: failed to destroy VM 024d899f4e8793: unauthorized (Request ID: 01M409G3XY7VP42VW5A80EVJWB-dc0)
PASS: machine destroy is rejected and the machine still exists
--> fly apps destroy e2e-pr-1-restricted --yes (restricted token)
Error: unauthorized (Request ID: 01M409G45FRXVFSPDGDAFFXZ8D-dc0)
PASS: apps destroy is rejected and the app still exists
==> 4. --mask r is read-only
--> fly auth login --restricted --org personal --mask r
restricted to personal with permissions r, expiring in 720h0m0s
successfully logged in as dev@fly.local
! Environment variable FLY_ACCESS_TOKEN is set, so flyctl will continue using it instead of the access token just saved. Unset FLY_ACCESS_TOKEN to use the new access token.
PASS: read-only login succeeded with the 30 day default expiry
--> fly apps list (read-only token)
NAME │ OWNER │ STATUS │ LATEST DEPLOY
dashboard-demo │ personal │ pending │
e2e-pr-1-restricted │ personal │ suspended │
PASS: read-only token can list apps
--> fly machine list (read-only token)
024d899f4e8793 started dc1
PASS: read-only token can list machines
--> fly machine stop 024d899f4e8793 (read-only token)
Waiting on lease for machine 024d899f4e8793...
Error: failed to obtain lease: failed to get lease on VM 024d899f4e8793: unauthorized (Request ID: 01M409HZHPQ7626Q1B4E0RYTQZ-dc0)
PASS: machine stop is rejected and the machine is still started
--> fly apps create e2e-pr-1-ro (read-only token)
Error: unauthorized (Request ID: 01M409HZQAQM3XREJDN9KDMNM3-dc0)
PASS: apps create is rejected and e2e-pr-1-ro does not exist
(ran in 145s)
==> 1 check(s) failed
cleanup: 2 tokens revoked
Claude Code: 78 turns, 1100s, $5.75
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Add
fly auth login --restrictedAdds a way to log in with org-scoped tokens that can do less than a full login, for example a session that can't delete machines or apps.
This automates the community forum recipe of creating an org token and attenuating it with
fly tokens attenuate.Why a flag on
login, not just attenuationA normal login saves an OAuth user token (
fo1_) plus short-lived per-org macaroons that flyctl creates from it every 10 minutes. Attenuating those macaroons alone doesn't restrict anything, for two reasons:fo1_token, which can't be attenuated.fetchOrgTokensreplaces the attenuated macaroons with fresh, unrestricted ones once they expire.So a restricted login has to save macaroons only.
What
--restricteddoes-i), but never saves the resulting user token.--org a,b).rwcC) or Read-only (r). Any mask can be passed with--mask. Without a TTY it uses "No deletes".deploy_organizationtoken (the same kindfly tokens create orgmakes) with an expiry. The default is 30 days;--expirychanges it.Organization{ID, Mask}caveat to each token and saves only the attenuated bundle.With no user token saved,
fetchOrgTokensdoes nothing, so the restrictions stay in place.--org,--maskand--expiryreturn an error unless--restrictedis also set.Changes
internal/command/auth/restricted.go: picks the orgs and mask, then creates and attenuates the tokens.internal/command/auth/login.go: new flags and the restricted save path.internal/command/auth/webauth/webauth.go: addsSaveTokenFor(ctx, token, userToken), which saves one token but looks up the user with another.SaveTokennow calls it, so existing callers behave the same.internal/command/auth/restricted_test.go: tests for checking the mask, looking up orgs by slug, and creating a token against a fake API. The last one confirms the narrowed token still allows writes and rejects deletes.Known limitations / needs manual verification
fo1_session doesn't also revoke the org tokens we just created.fly auth logoutrevokes the restricted bundle (it uses the sameLogOutmutation asfly tokens revoke supplied).dis decided server-side.whoami,orgs list, ...) may fail, the same as withFLY_API_TOKENset to an org token.fly tokens listuntil they expire or are revoked.Future work
RunWebLoginalready sendslock_organization/lock_appmetadata, driven by theFLY_TOKEN_LOCK_*env vars. A server-side version could have the browser consent page issue a macaroon-only session directly. Then no full-access token would ever reach the client, and no extra tokens would pile up.🤖 Generated with Claude Code
The forum link is a placeholder, so swap in the real thread URL. Tick the manual checklist items once you've tried a real login.