8000 Lua scripts: Lua 5.4 number rules break Redis (Lua 5.1) scripts — string.format %d rejects fractions, 10/2 prints "5.0" · Issue #747 · kividbio/kividb · GitHub
[go: up one dir, main page]

Skip to content

Lua scripts: Lua 5.4 number rules break Redis (Lua 5.1) scripts — string.format %d rejects fractions, 10/2 prints "5.0" #747

Closed

Description

@murtazayusuf

Summary

KiviDB embeds Lua 5.4 (mlua with lua54, Cargo.toml). Redis embeds Lua 5.1. Two 5.4 number rules make scripts that work on Redis fail or behave differently on KiviDB:

  1. string.format integer conversions (%d, %x, …) reject non-integral numbers. Lua 5.1 truncates 1.5 to 1. Lua 5.4 raises bad argument #N to 'format' (number has no integer representation), so the script fails.
  2. / always returns a float, and floats print with .0. tostring(10 / 2) is "5" on Redis and "5.0" on KiviDB. The same applies to .. concatenation, so a key built as "k:" .. n / 2 is a different key. This one is silent.

return 7 / 2 is unaffected: both reply 3.

None of the docs (SCRIPTS.md, COMPATIBILITY.md, MIGRATING_FROM_REDIS.md) say the engine runs Lua 5.4.

Minimal repro

KiviDB 1.2.2 (kividb/kividb:v1.2.2-lua, prebuilt image) and redis:7 (7.4.12), side by side, using redis-cli EVAL "<script>" 0:

Script Redis 7.4.12 KiviDB 1.2.2
return string.format('%d', 1.5) 1 ERR user_script:1: bad argument #2 to 'format' (number has no integer representation)
return string.format('%d', 3.0) 3 3
return string.format('%d%03d', 1731571200, 123456 / 1000) 1731571200123 ERR user_script:1: bad argument #3 to 'format' (number has no integer representation)
return string.format('%x', 255.9) ff ERR user_script:1: bad argument #2 to 'format' (number has no integer representation)
return tostring(10 / 2) 5 5.0
return 7 / 2 3 3
return _VERSION Lua 5.1 Lua 5.4

Real-world impact: affinidi-messaging-mediator

affinidi/affinidi-messaging-mediator (commit 8dce7b5) uses Redis as its storage backend through a Lua function library (FUNCTION LOAD, conf/atm-functions.lua). We ran the four Redis test suites from its CI against both servers, with the same build and the same commands:

cargo test -p affinidi-messaging-mediator --no-default-features --features didcomm,redis-backend \
  --test store_conformance --test redis_reconnect --test redis_storage_full --test redis_streaming_heartbeat
Suite Redis 7.4.12 KiviDB 1.2.2 KiviDB 1.2.2, one script line patched
redis_reconnect 1/1 1/1 1/1
redis_storage_full 1/1 0/1 1/1
redis_streaming_heartbeat 1/1 1/1 1/1
store_conformance 17/17 15/17 17/17
Total 20/20 17/20 20/20

All three failures are the same line, conf/atm-functions.lua:91, inside store_message. Every message store goes through it:

local time = string.format("%d%03d", time[1], time[2] / 1000)

time[2] is the microseconds from TIME, so time[2] / 1000 has a fraction. Redis truncates it; KiviDB fails:

store_message: DatabaseError(14, "session-id", "Couldn't store message in database: ResponseError:
user_script:1: bad argument #3 to 'format' (number has no integer representation) script: store_message, on @user_script:1.")

Failing tests: redis::message_store_and_counters, redis_db0_checks and a_full_database_is_reported_as_storage_full. The last one fails at its first store, before it gets to the memory limit.

Wrapping the argument in math.floor(...) makes all 20 pass on KiviDB. Everything else in their suite already matched Redis:

  • the function library loads;
  • streams and consumer groups (claim, ack, autoclaim) work;
  • pub/sub failover between instances works;
  • reconnect works;
  • the INFO version check (redis_version:8.10.0 against their >=7.1, <9.0) passes;
  • out-of-memory refusal with maxmemory 1 and noeviction returns the same OOM command not allowed … error as Redis.

Proposed fix

For Redis script compatibility, make the embedded Lua follow 5.1 number semantics where they show:

  • string.format: for %d %i %o %u %x %X %c, truncate a non-integral float toward zero, as 5.1 does, instead of raising. One way is to wrap string.format in the script sandbox: walk the format string and coerce the matching arguments with math.tointeger or truncation.
  • tostring and concatenation of integral floats: decide whether 5.0 should print as 5. Changing it is more invasive, since it touches tostring, .. and string.format('%s'). At minimum, document it.
  • Docs: say in SCRIPTS.md and COMPATIBILITY.md that scripts run on Lua 5.4. List the differences from Redis's Lua 5.1: the two above, plus any others the sandbox lets through. Candidates to check, not yet tested: unpack vs table.unpack, setfenv/getfenv, loadstring, // and the bit operators.
  • Tests: add the repro table above as compat tests.

Environment

  • KiviDB 1.2.2, kividb/kividb:v1.2.2-lua, run with --port 6379 --protected-mode no. INFO: redis_version:8.10.0, kividb_version:1.2.2.
  • Redis redis:7 (7.4.12).
  • linux x86_64, run on AWS CodeBuild with privileged Docker. The mediator was built with Rust 1.95 (rust:1.95-bookworm), as in their CI.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingredis-compatA test in Redis's own test suite fails against KiviDB

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0