With lua-parallel-scripts on, KEYS/SCAN still take the command gate alone and stall every connection (after #757) · Issue #772 · kividbio/kividb · GitHub
You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
With lua-parallel-scripts on, KEYS/SCAN still take the command gate alone and stall every connection (after #757) #772
Follow-up to #754 / #757. With lua-parallel-scripts yes (Lua builds), keyspace walks (KEYS, SCAN, DBSIZE, RANDOMKEY, the flushes) take the command gate alone (tag_locks::sees_the_whole_keyspace → takes_the_gate_alone), so a walk can never see a parallel script half done. As a result, #757's fix (other connections keep being served during a long KEYS/SCAN) does not apply there: a KEYS over 2M keys still stalls every connection for its whole duration (~280 ms in CI).
Options to evaluate:
walks take each hash-tag's lock (or every tag stripe) shard by shard instead of the gate, so a script holding tag X only blocks the walk while it's in X's shard;
walks read through a consistent per-shard snapshot, and only wait for scripts touching that shard;
accept it and document that with parallel scripts on, KEYS/SCAN are exclusive (Redis's KEYS is exclusive too, but it's one thread for everything anyway).
Measure the probe-PING worst case and script throughput (BullMQ bench) for each.
Activity
DBSIZE and RANDOMKEY keep taking the gate alone (#772)
Every command that reads keys it does not name keeps scripts out (#772) 7C7E p>
Merge pull request #925 from kividbio/perf/scan-keys-shared-gate
KEYS, SCAN, DBSIZE and RANDOMKEY no longer stop every connection with…