Repository navigation
Lua: a script's false is sent as $-1 to RESP3 clients (Redis: _); redis-py 8 fails with "Protocol Error", Celery 5.7 chords never complete #431
Description
On a RESP3 connection (HELLO 3), a Lua script's false (which is also what redis.call gives back for a nil reply, such as GET of a missing key) is sent to the client as the RESP2 null bulk string $-1. Redis sends the RESP3 null _. redis-py's RESP3 parser rejects $-1 with InvalidResponse: Protocol Error: b'1', and redis-py 8 uses RESP3 by default, so any script that returns false or a missing value, at the top level or inside a table, fails from Python.
Found while testing Celery 5.7.0 on kividb/kividb:v1.2.0-lua (part of #33): Celery 5.7 counts chord header results with a Lua script (CHORD_PART_RETURN_LUA in celery/backends/redis.py) that returns {readycount, totaldiff, size, claimed}, where totaldiff is redis.call('GET', tkey) of a key that usually does not exist. Every chord on KiviDB fails (ChordError: Join error: TypeError("int() argument must be a string, a bytes-like object or a real number, not 'ResponseError'") in the worker, the chord never completes); the same chord completes on Redis 8.10.2.
Sequence (raw RESP; missing does not exist)
HELLO 3
EVAL "return {1, redis.call('GET', KEYS[1]), '3', 0}" 1 missing
EVAL "return false" 0
EVAL "return redis.call('GET', KEYS[1])" 1 missing
{1, GET missing, '3', 0} |
return false |
return GET missing |
|
|---|---|---|---|
| Redis 8.10.2 | *4 :1 _ $1 3 :0 |
_ |
_ |
| KiviDB 1.2.0 (lua) | *4 :1 $-1 $1 3 :0 |
$-1 |
$-1 |
Under RESP2 both send $-1, as they should. A Lua nil at the end of a returned value is already sent as _ under RESP3; only false is not.
Through redis-py 8.1.0 (protocol=3, the default):
r = redis.Redis(port=P, protocol=3)
r.eval("return {1, redis.call('GET', KEYS[1]), '3', 0}", 1, "missing")
# Redis 8.10.2: [1, None, b'3', 0]
# KiviDB 1.2.0: redis.exceptions.InvalidResponse: Protocol Error: b'1'Expected: as Redis: a Lua false (in a script that has not called redis.setresp(3)) is a null reply, sent as _ to a RESP3 client and $-1 to a RESP2 one.
Same on master at b8ca3bf: in src/commands/lua.rs the Lua-to-RESP conversion sends LuaValue::Nil as resp_null_resp3() when the client speaks RESP3, but LuaValue::Boolean(false) always as resp_null_bulk() (around line 3041), and the fallback arm _ => Ok(resp_null_bulk()) (around line 3165) does the same.
Reproduction scripts and their output (repro_resp3_lua.py, repro_chord_script.py) are in benchmarks/celery-compat/ on the compat PR.
Activity
Lua: take the client's protocol from the connection loop (#431)
Merge pull request #436 from kividbio/fix/431-lua-false-resp3