Repository navigation
Description
Summary
KiviDB embeds Lua 5.4 (mlua with lua54, Cargo.toml). Redis embeds Lua 5.1. Two 5.4 number rules make scripts that work on Redis fail or behave differently on KiviDB:
string.formatinteger conversions (%d,%x, …) reject non-integral numbers. Lua 5.1 truncates1.5to1. Lua 5.4 raisesbad argument #N to 'format' (number has no integer representation), so the script fails./always returns a float, and floats print with.0.tostring(10 / 2)is"5"on Redis and"5.0"on KiviDB. The same applies to..concatenation, so a key built as"k:" .. n / 2is a different key. This one is silent.
return 7 / 2 is unaffected: both reply 3.
None of the docs (SCRIPTS.md, COMPATIBILITY.md, MIGRATING_FROM_REDIS.md) say the engine runs Lua 5.4.
Minimal repro
KiviDB 1.2.2 (kividb/kividb:v1.2.2-lua, prebuilt image) and redis:7 (7.4.12), side by side, using redis-cli EVAL "<script>" 0:
| Script | Redis 7.4.12 | KiviDB 1.2.2 |
|---|---|---|
return string.format('%d', 1.5) |
1 |
ERR user_script:1: bad argument #2 to 'format' (number has no integer representation) |
return string.format('%d', 3.0) |
3 |
3 |
return string.format('%d%03d', 1731571200, 123456 / 1000) |
1731571200123 |
ERR user_script:1: bad argument #3 to 'format' (number has no integer representation) |
return string.format('%x', 255.9) |
ff |
ERR user_script:1: bad argument #2 to 'format' (number has no integer representation) |
return tostring(10 / 2) |
5 |
5.0 |
return 7 / 2 |
3 |
3 |
return _VERSION |
Lua 5.1 |
Lua 5.4 |
Real-world impact: affinidi-messaging-mediator
affinidi/affinidi-messaging-mediator (commit 8dce7b5) uses Redis as its storage backend through a Lua function library (FUNCTION LOAD, conf/atm-functions.lua). We ran the four Redis test suites from its CI against both servers, with the same build and the same commands:
cargo test -p affinidi-messaging-mediator --no-default-features --features didcomm,redis-backend \
--test store_conformance --test redis_reconnect --test redis_storage_full --test redis_streaming_heartbeat
| Suite | Redis 7.4.12 | KiviDB 1.2.2 | KiviDB 1.2.2, one script line patched |
|---|---|---|---|
redis_reconnect |
1/1 | 1/1 | 1/1 |
redis_storage_full |
1/1 | 0/1 | 1/1 |
redis_streaming_heartbeat |
1/1 | 1/1 | 1/1 |
store_conformance |
17/17 | 15/17 | 17/17 |
| Total | 20/20 | 17/20 | 20/20 |
All three failures are the same line, conf/atm-functions.lua:91, inside store_message. Every message store goes through it:
local time = string.format("%d%03d", time[1], time[2] / 1000)time[2] is the microseconds from TIME, so time[2] / 1000 has a fraction. Redis truncates it; KiviDB fails:
store_message: DatabaseError(14, "session-id", "Couldn't store message in database: ResponseError:
user_script:1: bad argument #3 to 'format' (number has no integer representation) script: store_message, on @user_script:1.")
Failing tests: redis::message_store_and_counters, redis_db0_checks and a_full_database_is_reported_as_storage_full. The last one fails at its first store, before it gets to the memory limit.
Wrapping the argument in math.floor(...) makes all 20 pass on KiviDB. Everything else in their suite already matched Redis:
- the function library loads;
- streams and consumer groups (claim, ack, autoclaim) work;
- pub/sub failover between instances works;
- reconnect works;
- the
INFOversion check (redis_version:8.10.0against their>=7.1, <9.0) passes; - out-of-memory refusal with
maxmemory 1andnoevictionreturns the sameOOM command not allowed …error as Redis.
Proposed fix
For Redis script compatibility, make the embedded Lua follow 5.1 number semantics where they show:
string.format: for%d %i %o %u %x %X %c, truncate a non-integral float toward zero, as 5.1 does, instead of raising. One way is to wrapstring.formatin the script sandbox: walk the format string and coerce the matching arguments withmath.tointegeror truncation.tostringand concatenation of integral floats: decide whether5.0should print as5. Changing it is more invasive, since it touchestostring,..andstring.format('%s'). At minimum, document it.- Docs: say in
SCRIPTS.mdandCOMPATIBILITY.mdthat scripts run on Lua 5.4. List the differences from Redis's Lua 5.1: the two above, plus any others the sandbox lets through. Candidates to check, not yet tested:unpackvstable.unpack,setfenv/getfenv,loadstring,//and the bit operators. - Tests: add the repro table above as compat tests.
Environment
- KiviDB 1.2.2,
kividb/kividb:v1.2.2-lua, run with--port 6379 --protected-mode no.INFO:redis_version:8.10.0,kividb_version:1.2.2. - Redis
redis:7(7.4.12). - linux x86_64, run on AWS CodeBuild with privileged Docker. The mediator was built with Rust 1.95 (
rust:1.95-bookworm), as in their CI.
Activity
fix(lua): no memory leak on an error path in string.format (#747 review)
Merge pull request #768 from kividbio/fix/lua-string-format-747
fix(lua): string.format takes fractions for integer conversions, as R…
Lua: string.format is the patched str_format, at a C call's cost (kiv…