Repository navigation
Conversation
Some long-distance pairs (e.g. Japan <-> Hong Kong) get a poor direct route on certain ISPs, going through Singapore and back. Measured from real players, the direct RTT between them swings between ~75 ms and ~175 ms depending on the ISP pair and the day, while a relay on GCP Premium Tier reaches both sides through Google's backbone and stays around 70 ms. Flycast already relays through peers, but that cannot avoid the slow international leg when all remote players are on the other side. `gdxsv relay` forwards GGPO packets between the players of a match: - It reuses GGPO's existing peer-relay header (type 99 with relay_to_endpoint), restores it like a relaying peer does, and sends the packet to the destination peer of the sender's session. - Players join a session with a small UDP ping carrying the session id, peer id and a per-match token; only bound members of the same session can relay to each other. - It registers with the lobby like mcs, reports its sessions every 3 s, and exits when the lobby tells it to or when the lobby has been unreachable for 15 minutes. The VM's startup script then shuts the VM down. - -relay_test_session=<id>:<hex token> runs it without a lobby, for local tests. The lobby manages relays when GDXSV_RELAY_REGIONS (comma separated) is set: - It starts a relay VM through mcsfunc (role=relay, e2-micro) when some online pair is far apart (>= 50 ms through every GCP region, from the latencies clients report at login), an allowed region is within 10 ms of their best meeting point, and the relayed estimate is <= 120 ms. - It stops a relay after its region has been unneeded for 15 minutes and no match is using it. - For each match whose players all report relay_server=1, it offers one relay (the one with the lowest worst-pair estimate) with a fresh token in P2PMatching.relays. A single relay keeps both sides of a pair on the same server. The relay listens on UDP 9879, inside the existing gdxsv firewall rule. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2 of 7 tasks
The lobby now runs a relay itself on GDXSV_RELAY_ADDR (UDP 9879 by default, empty to disable). It is free and always there, so a match can use a relay before a VM is up, and pairs no VM is started for still get one. A match is offered every connected relay (up to 4, what clients accept), best first: relay VMs by their worst estimated relayed RTT between participants, then the lobby's own. Each is given the match's session. Clients pick a relay per peer and, when two peers disagree, settle on the earlier one, so the order is the same for every player. Relays take IPv6 too, so a player on IPv6 and one on IPv4 can reach each other through one: - `gdxsv relay` and the lobby's relay listen on both families when the address has no host. - A relay reports its IPv6 address (GDXSV_RELAY_PUBLIC_ADDR6), and the lobby passes it as RelayServer.ip6. - The addresses come from the machine's own interface, not an external lookup, skipping temporary and deprecated ones. - mcsfunc creates relay VMs in the custom-mode network `gdxsv-relay`, whose subnet `gdxsv-relay-<region>` is dual-stack, with external IPv4 and IPv6, tagged `gdxsv-relay`. The `http` option is left out for them: the client library would replace their access configs and add a firewall rule to the default network. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deciding which regions need a relay VM compared every pair of online players in the lobby's event loop, parsing 22 latencies per pair. With 400 players that stalled the lobby for ~58 ms every 10 s. - The event loop now only copies each eligible player's latencies into a fixed array (0.14 ms for 400 players). Pairs are checked in a goroutine (1.1 ms for 400), and the result comes back through the event loop. - Only players whose client supports relays count, and only pairs on the same game disk, since only they can play each other. - Stop as soon as every allowed region is known to be needed. - A relay address must be a numeric IP. Resolving a name could stall the event loop while a match is being made. - Start thresholds are loosened so fewer pairs that need a relay go without one: far apart from 40 ms (was 50), relayed estimate up to 150 ms (was 120). - A relay VM is stopped after 30 minutes unneeded (was 15). The relay keys sessions and bindings by netip.AddrPort and reads and writes with ReadFromUDPAddrPort / WriteToUDPAddrPort. Pongs reuse the ping's buffer. Forwarding a packet takes ~22 ns and allocates nothing (checked by a test), so a busy relay does not make the GC run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The per-pair latency heuristic that decided when to start relay VMs was hard to follow. Replace it with a simple rule: while at least 4 players whose clients support relays are online, from at least two region groups (the existing gcpRegionGroup of each player's best region, as used by the region-friendly team shuffle), start a relay VM in every region of GDXSV_RELAY_REGIONS. The check only walks the online players, so it needs no goroutine. GDXSV_RELAY_REGIONS stays a list, so another region can be added when one is not enough. Clients now ping a relay over both IPv4 and IPv6 and use the faster family that answers. A relay therefore keeps up to one address per family for each player and accepts packets from either. Packets for a player go to the address it last sent a game packet from, falling back to its last ping before it plays. The family it plays over keeps its NAT mapping open. Forwarding still allocates nothing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The nodejs12 runtime mcsfunc was deployed on can no longer be deployed to, so it moves to nodejs22, the newest GA runtime for gen1 functions. functions-framework 1.9 does not start there (missing ./middleware/cloudevent_to_background_event), so it moves to 3.5.1. Deployed with the same URL, service account and limits. A test copy listed the mcs VMs and created a relay VM in gdxsv-relay with IPv4 and IPv6, which stopped itself when the relay exited. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Anyone can connect to the lobby, and the lobby hands registered relays
to players, so a stranger could register as a relay and sit in the
middle of matches. A relay now sends GDXSV_RELAY_SECRET in its status,
and the lobby accepts only relays that send the same secret. The
comparison is constant time, and the secret is not kept in the stored
status, so it never reaches the logs. A lobby without a secret accepts
no relay VM. Its own relay needs none.
mcsfunc passes its GDXSV_RELAY_SECRET to relay VMs in their startup
script. It refuses to start a relay VM unless the secret is set and
plain ([A-Za-z0-9_-]{16,}), so nothing else can end up in the script.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add relay servers that rollback players can route their GGPO packets through when their direct route is poor.
Some long-distance pairs (e.g. Japan ↔ Hong Kong) get a detour through Singapore on certain ISP pairs. From real match logs, the direct RTT between them swings between ~75 ms and ~175 ms depending on the ISP pair and the day, which sometimes pushes the input delay to 4–5 frames. A relay on GCP Premium Tier reaches both sides through Google's backbone. A relay VM in asia-east2 reached the Hong Kong ISPs in 2–3 ms and Japanese players in ~65 ms, and a real UDP relay test matched the sum of the legs (the relay added ~0 ms). The existing peer relay can't help when every remote player is on the far side.
gdxsv relay(relay.go): forwards GGPO packets. Forwarding takes ~22 ns per packet and allocates nothing, so a busy relay does not make the GC run (sessions and bindings are keyed bynetip.AddrPort; a test checks for 0 allocations). It forwards them using GGPO's existing peer-relay header (type 99 +relay_to_endpoint). Players join a session with a small UDP ping carrying session id, peer id and a per-match token, so only bound members of the same session can relay to each other. It registers with the lobby like mcs (newlbsExtRelayStatus/lbsExtRelayControl). The status carriesGDXSV_RELAY_SECRET, and the lobby accepts only relays with the same secret (compared in constant time, and not kept in the stored status). Otherwise anyone who connects to the lobby could register as a relay and sit in the middle of matches. It exits when the lobby says so, or after losing the lobby for 15 minutes.-relay_test_sessi 8000 on=<id>:<hex token>runs it without a lobby for local tests.GDXSV_RELAY_ADDR(UDP 9879 by default, empty to disable). It is free and always there, so a match can use a relay before any VM is up.lbs_relay.go), enabled byGDXSV_RELAY_REGIONS(comma separated; VMs only ever start in these regions):gcpRegionGroupof their best GCP region, the same one the region-friendly team shuffle uses. So Japan + Hong Kong starts relays; Japan only, Japan + Korea, or Hong Kong + Taiwan does not.relay_server=1, the match gets every connected relay (up to 4), best first. Relay VMs come first, ordered by their worst estimated relayed RTT between participants, then the lobby's own relay. Each relay gets the match's session and a fresh token (P2PMatching.relays). Clients pick a relay per peer and settle disagreements on the earlier one.RelayServer.ip6, so IPv6 and IPv4 players can reach each other through one. Clients ping both families. A relay keeps one address per family for each player and accepts both. It sends packets for a player to the address that player last sent a game packet from, so the family the player actually plays over keeps its NAT mapping. The addresses come from the machine's own interface (not an external lookup), skipping temporary and deprecated ones: in the relay VM startup script, and inlaunch-lbs.shfor the lobby's relay.role=relay: starts an e2-micro VM runninggdxsv relayin the custom-mode networkgdxsv-relay. It passes its ownGDXSV_RELAY_SECRETin the startup script, and refuses to start a relay unless that secret is set and plain ([A-Za-z0-9_-]{16,}). The VM gets external IPv4 and IPv6 and shuts itself down when the relay exits. Thehttpoption is left out for these VMs: the client library would replace their access configs and add a firewall rule to the default network.Client side: inada-s/flycast#411
GCP setup (done)
gdxsv-relay(custom mode), subnetgdxsv-relay-asia-east2(10.200.0.0/24, IPv4/IPv6 with external IPv6).gdxsv-relay-udp4(0.0.0.0/0) andgdxsv-relay-udp6(::/0): udp:9879 to taggdxsv-relay.Deploy notes
gdxsv-relaywith IPv4 and IPv6, which stopped itself when the relay exited. Until this is released, a relay VM stops right away, because the released binary has norelaycommand.GDXSV_RELAY_SECRETon the lobby and on mcsfunc (done on both). It is not in the repository.launch-lbs.shnow sets the relay addresses. UDP 9879 is open on the VPS for IPv4 and IPv6. Enable relay VMs withGDXSV_RELAY_REGIONS=asia-east2.Test
go test -race ./...and golangci-lint: pass.tools/rbk_testcases D4–D6 run 4 local clients through local relays: one relay, two relays settling on one, and a dual-stack relay where players with an unreachable relay IPv4 address pick IPv6 on their own. All pass.Rollout checklist
Before releasing:
gdxsv-relay, subnetgdxsv-relay-asia-east2(IPv4/IPv6), firewallgdxsv-relay-udp4/-udp6(udp:9879).GDXSV_RELAY_SECRET.launch-lbs.shsetsGDXSV_RELAY_ADDR,GDXSV_RELAY_PUBLIC_ADDR,GDXSV_RELAY_PUBLIC_ADDR6,GDXSV_RELAY_REGIONS=asia-east2andGDXSV_RELAY_SECRET.After the restart, in Cloud Logging (
logs/gdxsv-lbs):local relay listeningshowspublic_addr=153.121.44.150:9879and the VPS's IPv6 inpublic_addr6. There is nolocal relay not started, and noignoring relay IPv6 address.mcsfunc allocfor rolerelayappears, and VMgdxsv-relay-asia-east2-latestis RUNNING with IPv4 and IPv6.relay registeredwithregion=asia-east2, and norelay status with a wrong secret.P2PMatchingReportbefore_log hasRELAY0 rtt=[...]lines, andRelay:2when a peer went through a relay (see Route rollback packets through relay servers when faster flycast#411).relay peer bound(4 per match) andrelay session removed ... forwarded=N.relay no longer needed, shutting it down, and the VM is TERMINATED.Checked on 2026-10-01 by starting a relay VM through the production mcsfunc (
/alloc?region=asia-east2&role=relay) with no players around. It came up with IPv4 and IPv6 and registered at 10:11:45 UTC with the secret. At 10:41:53 the lobby shut it down, and the VM was TERMINATED at 10:42:29.The relay VM's own log (
/var/log/gdxsv-relay.log) is not shipped to Cloud Logging. Its sessions and forwarded counts are only visible on the VM, and thegdxsv-relaynetwork allows no SSH.To roll back:
GDXSV_RELAY_REGIONSso no relay VM starts.GDXSV_RELAY_ADDR=""to turn off the lobby's own relay.🤖 Generated with Claude Code