Repository navigation
fix(query): bound log queries and release database resources - #1271
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Log queries could wait indefinitely before reaching ClickHouse, and scan/transaction failures could retain connections in the shared ten-connection pool. This change gives logs/charts, metadata reads, permission-lock waits, connection initialization, timeline optimization, and optional counts one request deadline (configurable
app.queryTimeout, default30s) and returns identifiable timeout/cancellation errors.It also closes ClickHouse/Databend/trace rows on every exit, checks iteration errors, rolls back failed trace batches, and emits in-flight/completed slow-query diagnostics with correlated request/table/instance IDs, the actual stage and pool, safe error classifications, and
sql.DB.Stats()(app.querySlowThreshold, default2s). The pinned ClickHouse v2.3.0 driver requires a contextual connector and native socket cancellation during handshake/network reads. The pinned Databend v0.2.1 driver also ignores HTTP contexts; a local reader adapter now applies the shared deadline to response headers, bodies and pages, while retaining its parsers and write/batch implementation. Reader bind arguments and namedtls_configfail explicitly instead of falling back to the unbounded path; ordinary ClickVisual reader SQL uses neither.Validation: fault-injection tests cover pool exhaustion/recovery, cancellation without retries/fallback, shared retry/timeline/log/count budgets, metadata timeouts, stream/scan errors, and transaction failures. Real TCP tests cover native handshake deadlines and cancellation after handshake; real HTTP tests cover Databend headers, bodies, pagination, client cancellation, early close, schema/empty pages, and connection reuse. Backend build and scoped
go vetpass. Existing unrelated full-package failures were reproduced before edits: SQL-format expectations inTest_adaSelectPart/Test_adaSelectPart1, andTestSettingUpdaterequiring an external default configuration.Incident analysis found independent prom2click pools, a shared page/trace pool, pre-query MySQL reads, and a permission mutex whose holders can access external stores; CMDB event enqueue is nonblocking. Without incident goroutine stacks, none can be claimed as the unique production cause. This PR targets GitHub
master; the divergent Shimo upstream needs its own adaptation and verification.Agent smoke passed against the built backend with temporary SQLite and a synthetic ClickHouse HTTP peer: normal logs/count/charts; ten concurrent blocked queries returned an explicit timeout at ~1s; the eleventh recorded InUse=10, WaitCount=1 and ~0.991s of pool wait; a disconnected client canceled its downstream query and left the query pool at InUse=0; the next query succeeded. No production or UI changes were exercised.