Repository navigation
stringify integers outside JavaScript's safe range in JSON results - #210
Conversation
Large ClickHouse integ 8000 ers (UInt64 and similar) exceed JavaScript's MAX_SAFE_INTEGER, so a JSON consumer parsing them as doubles silently corrupts the value (1875924584784080993 becomes 1875924584784081000). Recursively emit integers whose magnitude exceeds 2^53-1 as strings in the serialized tool result; smaller integers and booleans stay numeric. Closes ClickHouse#111
There was a problem hiding this comment.
Hey @knQzx nice work here. Thanks for doing this.
I ran into performance regression with the original approach and a few other things so I made some changes on top. Specifically I:
- Replaced the original recursive prewalk, which rebuilt every nested container, with
simplejson’s nativebigint_as_stringencoder. This keeps the result correct without the original CPU and allocation regression. - Added a cycle-aware, copy-on-change fallback for environments without simplejson’s C encoder. Safe containers are returned unchanged and only branches containing unsafe integers are copied.
- Kept safe integers and booleans as JSON values. Only integers outside
[-9007199254740991, 9007199254740991]become decimal strings. - Moved async chDB result serialization to a separate worker thread. The original implementation serialized on the event loop whic caused noticeable stalls for large results. Serialization also remains outside the query timeout and query executor.
- Lowered the
simplejsondependency floor to3.17.2for compatibility with existing 3.x pins. The required bigint behavior works just the same in that version. - Added some more extensive tests.
- Added documentation and CHANGELOG now describe the wire behavior.
- Performance improved from ~270ms for the original implementation to ~150ms for a 100k-row FastMCP result which is back to roughly equal with current main timing.
Great work!
There was a problem hiding this comment.
Pull request overview
This PR hardens the MCP server’s JSON output contract so that ClickHouse integer values beyond JavaScript’s Number.MAX_SAFE_INTEGER are emitted as decimal strings, avoiding silent precision loss for JS clients while keeping safe-range integers and booleans as native JSON types.
Changes:
- Introduces a new tool-result serializer that uses
simplejson’sbigint_as_stringwhen its C encoder is available, with a stdlibjsonfallback that recursively stringifies unsafe integers. - Moves chDB result serialization off the event loop in the async tool path to avoid blocking during JSON encoding.
- Adds contract-level serialization tests and documents the behavior in the README and changelog.
Reviewed changes
Copilot reviewed 6 out of 7 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
mcp_clickhouse/mcp_server.py |
Adds safe-range integer stringification logic and integrates it into all tool result serialization; adjusts async chDB path to serialize off-loop; updates tool descriptions. |
tests/test_serialization.py |
New contract tests covering boundaries, nesting, circular refs, stdlib/simplejson parity, and async off-loop behavior. |
tests/test_chdb_tool.py |
Adds an integration-style check ensuring wide/nested Int128/UInt128 values serialize as strings when unsafe. |
README.md |
Documents the JSON contract: unsafe integers returned as decimal strings for JS safety. |
CHANGELOG.md |
Records the user-visible fix and the new simplejson dependency. |
pyproject.toml |
Adds simplejson as a runtime dependency. |
uv.lock |
Locks the new simplejson dependency. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
large ClickHouse integers (UInt64 and similar) exceed JS
MAX_SAFE_INTEGER, so a JSON consumer parsing them as doubles silently corrupts the value (1875924584784080993 becomes 1875924584784081000)recursively emit integers whose magnitude exceeds 2^53-1 as strings; smaller integers and booleans stay numeric
closes #111