8000 stringify integers outside JavaScript's safe range in JSON results by knQzx · Pull Request #210 · ClickHouse/mcp-clickhouse · GitHub
[go: up one dir, main page]

Skip to content

stringify integers outside JavaScript's safe range in JSON results - #210

Merged
joe-clickhouse merged 3 commits into
ClickHouse:mainfrom
knQzx:fix/stringify-large-integers
Sep 1, 2026
Merged

joe-clickhouse merged 3 commits into
ClickHouse:mainfrom
knQzx:fix/stringify-large-integers

Conversation

@knQzx
@knQzx knQzx commented Jul 18, 2026
Copy link
Copy Markdown
Contributor

large ClickHouse integers (UInt64 and similar) exceed JS MAX_SAFE_INTEGER, so a JSON consumer parsing them as doubles silently corrupts the value (1875924584784080993 becomes 1875924584784081000)

recursively emit integers whose magnitude exceeds 2^53-1 as strings; smaller integers and booleans stay numeric

closes #111

Large ClickHouse integ
8000
ers (UInt64 and similar) exceed JavaScript's
MAX_SAFE_INTEGER, so a JSON consumer parsing them as doubles silently
corrupts the value (1875924584784080993 becomes 1875924584784081000).
Recursively emit integers whose magnitude exceeds 2^53-1 as strings in
the serialized tool result; smaller integers and booleans stay numeric.

Closes ClickHouse#111
@joe-clickhouse joe-clickhouse left a comment
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @knQzx nice work here. Thanks for doing this.

I ran into performance regression with the original approach and a few other things so I made some changes on top. Specifically I:

  • Replaced the original recursive prewalk, which rebuilt every nested container, with simplejson’s native bigint_as_string encoder. This keeps the result correct without the original CPU and allocation regression.
  • Added a cycle-aware, copy-on-change fallback for environments without simplejson’s C encoder. Safe containers are returned unchanged and only branches containing unsafe integers are copied.
  • Kept safe integers and booleans as JSON values. Only integers outside [-9007199254740991, 9007199254740991] become decimal strings.
  • Moved async chDB result serialization to a separate worker thread. The original implementation serialized on the event loop whic caused noticeable stalls for large results. Serialization also remains outside the query timeout and query executor.
  • Lowered the simplejson dependency floor to 3.17.2 for compatibility with existing 3.x pins. The required bigint behavior works just the same in that version.
  • Added some more extensive tests.
  • Added documentation and CHANGELOG now describe the wire behavior.
  • Performance improved from ~270ms for the original implementation to ~150ms for a 100k-row FastMCP result which is back to roughly equal with current main timing.

Great work!

Copilot AI left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens the MCP server’s JSON output contract so that ClickHouse integer values beyond JavaScript’s Number.MAX_SAFE_INTEGER are emitted as decimal strings, avoiding silent precision loss for JS clients while keeping safe-range integers and booleans as native JSON types.

Changes:

  • Introduces a new tool-result serializer that uses simplejson’s bigint_as_string when its C encoder is available, with a stdlib json fallback that recursively stringifies unsafe integers.
  • Moves chDB result serialization off the event loop in the async tool path to avoid blocking during JSON encoding.
  • Adds contract-level serialization tests and documents the behavior in the README and changelog.

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated no comments.

Show a summary per file
File Description
mcp_clickhouse/mcp_server.py Adds safe-range integer stringification logic and integrates it into all tool result serialization; adjusts async chDB path to serialize off-loop; updates tool descriptions.
tests/test_serialization.py New contract tests covering boundaries, nesting, circular refs, stdlib/simplejson parity, and async off-loop behavior.
tests/test_chdb_tool.py Adds an integration-style check ensuring wide/nested Int128/UInt128 values serialize as strings when unsafe.
README.md Documents the JSON contract: unsafe integers returned as decimal strings for JS safety.
CHANGELOG.md Records the user-visible fix and the new simplejson dependency.
pyproject.toml Adds simplejson as a runtime dependency.
uv.lock Locks the new simplejson dependency.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@joe-clickhouse
joe-clickhouse merged commit 82f0d12 into ClickHouse:main Sep 1, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

UInt64 values get corrupted in JSON responses

3 participants

0