8000 Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile · Issue #33 · Altinity/altinity-oauth-helper · GitHub
[go: up one dir, main page]

Skip to content

Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33

Closed

Description

@BorisTyshkevich

Purpose

Implement ADR #32 by replacing the current general-purpose LDAP server/parser dependency stack with a small first-party compatibility implementation for the documented ClickHouse LDAP profiles.

This is the execution tracker for ADR #32. It is intentionally implementation-oriented and may be delivered as one or several PRs.

Architectural decisions belong in #32. If implementation discovers evidence that invalidates an architectural assumption, stop and bring that evidence back to #32 rather than silently expanding scope here.

Preconditions

Architectural constraints inherited from #32

The implementation MUST preserve these decisions:

  • ClickHouse is the supported LDAP client; this is not a general LDAP server.
  • Support is defined by documented ClickHouse compatibility profiles.
  • Required operations are LDAPv3 simple Bind, the documented bounded Search shapes, Abandon as a no-response compatibility operation, and Unbind.
  • Connection handling remains bounded and simple; do not recreate generic per-request scheduling, Cancel/Abandon control-plane capacity, a generic route mux, or a generic recursive filter engine.
  • Authentication state is connection-local. Failed re-Bind clears prior auth state; successful re-Bind replaces it.
  • JWT/password bytes are not retained after verification and must never reach logs.
  • Role mapping remains a Bind-time snapshot, and Search size-limit overflow remains fail-closed rather than silently truncating roles.
  • Production ch-oauth-ldap must no longer depend on the current general LDAP server/client/BER stack.
  • cryptobyte is preferred for low-level ASN.1 primitives unless supported ClickHouse wire evidence demonstrates incompatibility.
  • If the implementation starts becoming a general LDAP parser/server, stop and revisit ADR: Own the ClickHouse LDAP compatibility profile and remove general LDAP dependencies #32.

Phase 1 — freeze evidence and select the primitive layer

1.1 Reconstruct the supported wire profile

Review the relevant supported ClickHouse source and the exact product configuration. Include OpenLDAP/libldap source where ClickHouse delegates behavior to it.

The profile evidence should cover, for every tracked ClickHouse line:

  • Bind operation and LDAP version;
  • Search request fields actually emitted by the supported configuration;
  • scope, dereference mode, size/time limits, typesOnly, requested attributes, controls;
  • filter structure and placeholder behavior;
  • MessageID behavior, including 127/128 boundaries;
  • Unbind;
  • timeout-triggered Abandon;
  • any version-specific differences that matter to the server.

Create a committed human-readable wire-profile document and non-secret raw fixtures. A suggested layout is:

docs/clickhouse-ldap-wire-profile.md
internal/ldap/testdata/clickhouse-wire/<clickhouse-line>/...

The exact layout may change if another structure is cleaner, but the evidence must be committed and testable rather than existing only in notes or packet captures outside the repository.

Add a contract test so the documented supported ClickHouse lines cannot silently drift away from the committed fixture set, following the same anti-drift principle already used by internal/securitytest/docs_contract_test.go.

1.2 Resolve cryptobyte vs local BER primitives

OpenLDAP currently initializes request BER with:

ld->ld_lberoptions = LBER_USE_DER

This is a strong prior that ClickHouse/libldap request encodings will be accepted by DER-oriented golang.org/x/crypto/cryptobyte helpers.

Prototype every supported raw request fixture through cryptobyte.

Decision rule:

all supported fixtures accepted safely
    -> use cryptobyte + first-party LDAP profile

any valid supported fixture requires a BER form cryptobyte rejects
    -> use a tiny bounded first-party BER cursor

Do not introduce another general-purpose ASN.1/BER packet-tree library as a substitute for the dependencies being removed.

1.3 Record baselines

Before replacement work changes the tree, record:

  • go list -deps ./cmd/ch-oauth-ldap;
  • current production LDAP LOC and the exact counting method;
  • current redaction/security inventory relevant to internal/ldap and the vendored forks;
  • current supported ClickHouse matrix results.

Phase 2 — implement the compatibility profile

Keep the implementation under the existing internal/ldap ownership boundary unless a package split has a concrete security/readability benefit.

2.1 Connection/framing model

Target model:

accept loop
  -> one goroutine per admitted connection
       -> read one bounded request
       -> process synchronously
       -> write bounded response
       -> repeat

Preserve or improve the current resource bounds:

  • maximum admitted connections: 256;
  • maximum LDAP message body: 64 KiB before body allocation;
  • read deadline: 30 s;
  • write deadline: 30 s.

The named pre-authentication message-body budget remains:

256 * 64 KiB = 16 MiB

This is a body-buffer bound, not a total RSS claim. Avoid proportional duplicate copies of attacker-controlled message bodies.

2.2 Supported operations

Implement only the supported profile:

  • LDAPv3 simple Bind;
  • documented Search shapes;
  • AbandonRequest recognition/drop with no response;
  • Unbind/close;
  • re-Bind semantics already described above.

Malformed framing should close the connection.

For a recognizable but unsupported LDAP operation, a single small protocol-correct error response is acceptable only if it can be emitted using common envelope/result primitives without adding operation-specific decoding. If diagnostics start expanding the decoder, close the connection instead.

Unsupported operations must never invoke OAuth verification or role logic.

2.3 Search/filter handling

Do not build a generic LDAP filter AST/evaluator.

For the current role-mapping profile, decode and validate the structural equivalent of:

AND
 ├── objectClass == groupOfNames
 └── member == currentBoundDN

Either child order is acceptable. Reject missing, duplicate, additional, or different predicates.

Honor the supported Search fields exactly as reconstructed from ClickHouse/libldap evidence.

Preserve size-limit behavior: emit at most N eligible entries, and if another valid role exists return sizeLimitExceeded so ClickHouse does not accept a silently truncated role set.

2.4 DN ownership and the {bind_dn} pipeline

Remove production dependence on github.com/go-ldap/ldap/v3.

Implement only the DN grammar required by documented ClickHouse profiles. Do not claim arbitrary RFC 4514 normalization compatibility.

Preserve the important ClickHouse/libldap escaping pipeline:

username
  -> ClickHouse escapeForDN
  -> bind DN string
  -> ClickHouse escapeForFilter
  -> libldap parses RFC 4515 filter escapes
  -> BER equalityMatch assertionValue contains DN text
  -> server decodes restricted DN form
  -> structural comparison with authenticated BoundDN

Example:

username:                  a,b
Bind DN text:              uid=a\,b,ou=users,...
ClickHouse filter text:    member=uid=a\5C,b,ou=users,...
BER assertion value:       uid=a\,b,ou=users,...

The server therefore must not add a general RFC 4515 filter-text unescaper; that transformation has already happened inside libldap before transmission.

Required DN behavior includes:

  • validate configured bases at startup;
  • require the supported leading user RDN shape;
  • reject unsupported multi-valued RDNs unless later requirements explicitly add them;
  • reject malformed DN escapes;
  • avoid loose suffix/string-split authorization checks;
  • preserve exact successful BoundDN for connection state;
  • compare Search membership DN structurally under the supported grammar;
  • safely escape synthetic group-entry CN values.

2.5 Result encoding

Implement only the response forms needed by the supported profile:

  • BindResponse;
  • SearchResultEntry;
  • SearchResultDone;
  • the minimal common error result needed by the chosen unsupported-operation policy.

MessageID must round-trip exactly, including values >= 128.

Phase 2 testing

Adapt the existing black-box TCP tests rather than recreating tests for generic machinery that is being deliberately removed.

Delete tests coupled to concepts that no longer exist, such as generic route handling, broad critical-control dispatch, request maps, Cancel scheduling, or generic recursive filter parsing.

Add native Go fuzzing for the new parser/security boundary. Suggested targets include:

FuzzLDAPFrame
FuzzBindRequest
FuzzSearchRequest
FuzzRestrictedDN
FuzzMemberAssertionDN

Exact target names may change; the required coverage does not.

Seeds should include:

  • committed supported ClickHouse wire fixtures;
  • MessageID 127/128 boundaries;
  • previous fork vulnerability shapes;
  • truncated and oversized lengths;
  • escaped/special-character DN cases;
  • equivalent and non-equivalent member DNs.

Required property:

arbitrary input results in a supported bounded response, bounded rejection, or connection close; never panic, unbounded allocation/concurrency, credential leakage, or stuck shutdown.

Temporary differential oracle

A small Phase-2-only differential harness is allowed and encouraged for profile-valid generated inputs:

same supported input
    -> current decoder
    -> new decoder
    -> compare normalized semantic result

Use it for supported valid messages and boundary encodings only.

Do NOT require equivalence for arbitrary malformed BER, unsupported generic LDAP operations, generic filters, or behavior that exists only because of bugs/overbreadth in the old parser.

Delete the differential oracle when the old implementation/dependencies are removed in Phase 4.

Phase 3 — security, compatibility, and release gates

Before cutover, all of the following must be green on the replacement implementation:

Redaction/security inventory re-baseline

When the vendored forks are removed:

  • remove vendored logger scanning only because those source files no longer exist;
  • delete stale third_party inventory rows;
  • keep replacement first-party parser/server sinks AST-enumerated;
  • add marker proofs for every credential-reachable local sink;
  • do not weaken the phase5release gate merely to accommodate the rewrite.

Passing the gate must mean the new implementation is audited, not that the old inventory was simply deleted.

Dependency closure contract

Add a committed security test, suggested name:

internal/securitytest/dependency_contract_test.go
TestDependencyContract_ProductionClosureHasNoGeneralLDAP

It must inspect the actual production closure (go list -deps ./cmd/ch-oauth-ldap) and fail if it contains:

github.com/vjeantet/ldapserver
github.com/vjeantet/goldap
github.com/go-ldap/ldap/v3
github.com/go-asn1-ber/asn1-ber
github.com/Azure/go-ntlmssp

or if the contract check becomes vacuous.

If another general-purpose LDAP server/client or generic BER tree library is introduced under a different module name, extend the contract/review accordingly rather than treating the literal denylist as exhaustive policy.

LOC/scope guardrail

Count all non-test production Go code under the LDAP implementation reachable from ./cmd/ch-oauth-ldap, excluding independent TLS transport plumbing introduced specifically for #31.

Expected post-rewrite landing zone is approximately 2,000–2,500 LOC.

If the implementation grows beyond approximately 3,500 LOC before independent TLS plumbing, pause for architecture review. This is a scope-creep indicator, not a claim that LOC itself measures security.

Phase 4 — cut over and delete the old stack

Only after Phase 3 is green:

  1. switch the production command to the new implementation;
  2. remove the vendored third_party/ldapserver and third_party/goldap code;
  3. remove their replace directives;
  4. remove production go-ldap usage;
  5. verify go-asn1-ber and go-ntlmssp disappear from the production command closure unless another unrelated production dependency genuinely requires them;
  6. keep any independent LDAP client used for black-box testing test-only and outside the released command closure;
  7. delete the temporary differential oracle and tests for machinery that no longer exists;
  8. rerun all PR/security/ClickHouse/release gates.

The cutover and dependency deletion should be one coherent final migration step so the repository does not permanently carry two production LDAP parsers.

Rollback

Do not add a permanent runtime switch between old and new parsers; that would retain both attack surfaces and dependency trees.

Rollback is operational/source-level:

revert the cutover change
or
redeploy the previous known-good release

Keep the immediately previous release/artifact available according to normal release practice until the replacement has proven stable.

Suggested PR slicing

Agents may implement this as one PR or several. A reasonable split is:

  1. evidence + committed fixtures + primitive-layer decision;
  2. new parser/profile implementation + tests/fuzz/differential harness while old production path remains active;
  3. security/dependency/redaction gates and full compatibility validation;
  4. production cutover + deletion of old forks/dependencies.

This is guidance, not an architectural requirement. Smaller or combined PRs are acceptable if reviewability and rollback remain clear.

Definition of done

  • ADR: Own the ClickHouse LDAP compatibility profile and remove general LDAP dependencies #32 is accepted.
  • ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23/equivalent required PR-time gate is active before production parser replacement merges.
  • Supported ClickHouse/libldap behavior is documented from source and committed non-secret wire evidence.
  • cryptobyte is selected or rejected based on those fixtures, with the result recorded.
  • First-party bounded Bind/Search/Abandon/Unbind profile is implemented.
  • Restricted DN handling replaces production go-ldap usage and preserves the {bind_dn} security semantics.
  • Existing auth/session/role semantics remain unchanged.
  • Size-limit overflow remains fail-closed.
  • Resource invariants remain at least as strong as 256 connections / 64 KiB body / 30 s read-write deadlines.
  • Native fuzzing and malformed-input/resource tests cover the new parser boundary.
  • Redaction inventory and phase5release gate are re-baselined without weakening coverage.
  • Production dependency-closure contract is committed and enforced in CI.
  • Supported ClickHouse matrix and relevant HA/TLS gates pass on the replacement.
  • Production LDAP implementation remains comfortably below the ~3,500 LOC architecture-review trigger, or a review explicitly accepts the exception.
  • Old LDAP forks/replaces and production go-ldap usage are deleted.
  • Temporary differential machinery is deleted at cutover.
  • Rollback procedure is documented as revert/redeploy, not a permanent dual-parser mode.

Related: #32 (ADR), #23 (required PR CI), #30 (ClickHouse version compatibility), #31 (LDAPS/TLS deployment mode), #19 (original implementation).

Activity

  1. BorisTyshkevich commented on Aug 30, 2026

    @BorisTyshkevich
    CollaboratorAuthor

    Prerequisite satisfied — the PR-time gate from #23 is active and enforced

    #23 shipped in PR #34 (merge commit e9c7c3c). This issue's gating precondition required
    three things, and all three now hold:

    1. workflow merged — .github/workflows/pr-gate.yml, workflow PR gate, one job
      Required PR gate, running go build ./..., go vet ./..., go test -race ./...,
      go test -tags phase5release ./internal/securitytest -count=1, and
      bash integration/clickhouse/tests/lib-tests.sh on every pull request and every push
      to main, with no path filters;
    2. a successful push-event run observed on main — run 33283876015
      at e9c7c3c, all five steps green;
    3. protection actively requires it — main is protected, required context is exactly
      Required PR gate bound to the github-actions app (id 15368, observed rather than
      guessed), strict: true, enforce_admins: true, no review requirement. Read back
      fresh from the API; the full table is in ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23's body.

    Phase 2 parser-replacement work here is no longer blocked on CI enforcement.

    Three things from #23 worth folding into this issue's plan

    1. go test -race ./... does not reach the vendored LDAP forks — and that is where the
    parser you are replacing lives.
    third_party/goldap and third_party/ldapserver are
    separate Go modules consumed by the root go.mod only through replace directives, so the
    root module pattern never compiles their test suites. #23's issue text was corrected to say
    this explicitly. Consequence: parser tests intended to be gated must live inside the root
    module, or this issue must add a command for those modules to pr-gate.yml.
    A test that
    lives only under third_party/** is not gated by anything today.

    Related and pre-existing: third_party/ldapserver's own suite cannot currently run
    standalone at all — go test ./... there fails with missing go.sum entry for module providing package github.com/vjeantet/goldap/message. Its six test files
    (cancel_test.go, client_close_test.go, client_test.go, e2e_test.go,
    packet_test.go, server_test.go) are effectively unrunnable. Worth fixing if this issue
    plans to keep any tests there.

    2. The planned TestDependencyContract_ProductionClosureHasNoGeneralLDAP will be picked
    up automatically.
    The gate names the whole ./internal/securitytest package with
    -count=1 and no -run filter or allowlist, so a new tagged or untagged contract test
    there enters the gate with no workflow edit. This was proven during #23: a newly added
    fatal test turned the required command red with zero workflow change.

    3. The gate is not tamper-resistant, by design — do not lean on it as one. A pull
    request runs its own copy of the workflow, and a job skipped by a job-level conditional
    reports Success to branch protection. So it enforces process for trusted maintainers and
    catches ordinary regressions, but an authorized author can neutralize both the workflow and
    its self-test in the same PR. internal/securitytest/pr_gate_contract_test.go is an
    accidental-drift detector and says so in its own doc comment. If this issue's threat model
    needs more than that, the mechanism is an organization-level required-workflow ruleset
    sourced outside this repository, not more in-repo tests.

    Also note two operational constraints recorded in the workflow itself: repository
    automation must not retarget a pull request's base using GITHUB_TOKEN (that produces
    no gate run, leaves the head SHA unchanged, and can let a prior green check stand — a real
    fail-open), and if a merge queue is ever enabled, the workflow needs a merge_group:
    trigger or queue entries stall waiting for a check that is never reported.

    Full detail, including every review finding and the complete run ledger, is in #23's
    ship log.

  2. BorisTyshkevich commented on Aug 30, 2026

    @BorisTyshkevich
    CollaboratorAuthor

    Ship log — #33

    Phase Status PR
    1 — freeze evidence and select the primitive layer shipped #37
    2 — implement the compatibility profile shipped #38
    3 — security, compatibility, and release gates shipped #39
    4 — cut over and delete the old stack shipped #41

    Phase 1 — handoff

    • Preconditions verified 2026-08-30: ADR ADR: Own the ClickHouse LDAP compatibility profile and remove general LDAP dependencies #32 reads Status: Accepted; Required PR gate is the required status check on main (strict:true).
    • Landed (PR feat(#33): freeze ClickHouse LDAP wire evidence and select the primitive layer (phase 1) #37, branch feat/33-phase1-wire-profile off origin/main @ e9c7c3c): docs/clickhouse-ldap-wire-profile.md (source-reconstructed profile for 24.8.11.51285 / 25.8.28.10001 altinitystable incl. per-line commit + blob SHAs and contrib/openldap pins 2.5.16 / 2.6.10; decision marker <!-- ldap-primitive-decision: … -->); committed non-secret fixtures internal/ldap/testdata/clickhouse-wire/{24.8,25.8}/{success,timeout-abandon} + constructed/message-id-boundary (127 → 02 01 7f, 128 → 02 02 00 80); internal/wirefixture (schema, Stable* projections, constructed builder, config-element hash); integration/clickhouse/wire 8 capture< 10BC8 /code> (ldap-wire-recorder: serve/sanitize/construct/compare) + compose-wirecapture.yml + capture-ldap-wire.sh --mode generate|verify + reciprocal ch-wirecap* preflights in run.sh/run-ha.sh + 3 daemon-free shell cases; internal/securitytest/{wire_profile_contract_test.go,dependency_contract_test.go} + redaction scopeDirs extended to both new packages; baselines in internal/ldap/testdata/phase1-baseline/ (production deps, LOC 8614 via committed script, security inventory, ClickHouse matrix, source head, toolchain). Production cmd/ch-oauth-ldap, non-test internal/ldap, third_party/** are byte-identical to main.
    • Primitive-layer decision (1.2): recorded by TestClickHouseWireCryptobyteDecision (internal/ldap/clickhouse_wire_cryptobyte_test.go) as the single computational owner of the doc marker — every committed fixture parses through golang.org/x/crypto/cryptobyte (DER-form checks: definite minimal lengths, minimal INTEGER incl. 00 80 for 128, context tags). golang.org/x/crypto moved indirect → direct in go.mod for that test only; TestDependencyContract_NoNonStandardCryptobyte proves it is outside the production closure. Phase 2 should read the marker value from the doc rather than this log.
    • Decisions taken (not in the issue):
      • Plan gate: the ChatGPT-authored plan reached the 5-pass cap with Fable REVISE, blocker: none; human ruled "approve with the 7 pass-5 findings as binding amendments" (recorded in the plan's ## Coordinator-approved amendments (binding); precedent Implement LDAP OAuth helper for ephemeral ClickHouse users and dynamic roles #19 phase 4). Conversation: https://chatgpt.com/c/6a93ea5b-fafc-83eb-ac9c-58a499894698
      • Fixture non-secrecy: the recorder writes raw PDUs only to a recorder-private 0700 tmpfs; the credential reaches sanitize via stdin only; only sanitized/ is exported; host-side lib/leakscan.sh runs over transcript/diagnostics/service logs/export while the token is live; Bind password bytes are replaced by a length-preserving x run. Committed corpus contains no JWT-shaped bytes (scanner test).
      • MessageID ≥128: ClickHouse opens one LDAP connection per authentication, so captured MessageIDs are 1..4; the 127/128 pair is constructed via wirefixture.BuildConstructedSimpleBind, labelled as such, and must regenerate byte-identically (TestWireProfileContract_ConstructedMessageIDFixturesReproduce).
      • Real Abandon is produced by libldap (LDAP_TIMEOUT in synchronous search); the recorder's stall-after-bind mode only withholds the Search result — it never fabricates an Abandon.
      • session.json carries token_claim_recipe (wirefixture.FixedTokenClaimRecipe) and per-PDU expected_semantics; both are stable-compared fields, so verify-mode also pins the minted claim set.
      • docs/clickhouse-ldap-wire-profile.md is engineering evidence owned by wire_profile_contract_test.go, deliberately not a third operator document under docs_contract_test.go (no config fences allowed in it).
    • Deviations from the spec: none in substance. Implementation ran through the decompose loop in 16 sub-tasks over two runs; the first run errored (worktree-base + red-gate-merge defects in the /ship script, fixed in e4da1ea chore(ship), which rides in this PR as tooling).
    • Known scope limitation (plan §50): integration/clickhouse/ha/session-probe (pre-existing credential-handling integration tool) stays outside redaction scopeDirs; recorded, not claimed.
    • Deferred acceptance criteria: every DoD bullet except "behavior documented from source + committed non-secret wire evidence" and "cryptobyte selected/rejected with result recorded" → Phases 2–4. Phase 3's dependency-closure contract now has a live predecessor (dependency_contract_test.go + internal/securitytest/testdata/production-nonstdlib-deps.txt) that Phase 4's cutover must update, not delete.
    • Manual proofs run by the coordinator: capture-ldap-wire.sh --mode verify → all four captured sessions compare OK, leak scans clean, no ch-wirecap leftovers; run-all-builds.sh → both tracked images PASS = clickhouse-matrix.tsv.
    • Internal review: one High-risk read-only pass (no blocking findings; 5 should-fix + nits — wrong upstream line citations, an untrue README statement, stale compose comments, unpopulated schema fields, missing zero-PDU test — all fixed in 731bbb6/92b21b7); security review: no findings.
    • Plan review: not approved after 5 passes → human ruling above (conversation URL above).
    • Review outcome: PR feat(#33): freeze ClickHouse LDAP wire evidence and select the primitive layer (phase 1) #37 merged 2026-08-30 as edebb6d (merged head 3240069), Required PR gate green at every pushed head (92b21b7, ceb674e, 3e8d4ea, 3240069). Code review: 3 formal pr-mode passes, none certifying on its own — pass 1 (head 92b21b7, feat(#33): freeze ClickHouse LDAP wire evidence and select the primitive layer (phase 1) #37 (comment)): 6 findings, 4 accepted+fixed (f70d340: derefAliases/Controls documented; FixedTokenClaimRecipe groups=→roles= + independent derivation from synthetic-IdP source; TLS setter table corrected; cryptobyte decision test dual-outcome-reachable), 2 rejected (40-char scanner print — taken anyway as hardening in ceb674e; bundled chore(ship) commit — disclosed, kept); pass 2 (head ceb674e, feat(#33): freeze ClickHouse LDAP wire evidence and select the primitive layer (phase 1) #37 (comment)): 3 findings, 2 accepted+fixed (3e8d4ea: independent goldap well-formedness decode gates the local-BER-cursor fallback + discriminating sabotage test; derefAliases/Controls machine-checked against raw fixture bytes), 1 carried rebuttal; pass 3 (head 3e8d4ea, feat(#33): freeze ClickHouse LDAP wire evidence and select the primitive layer (phase 1) #37 (review)): 3 findings, 1 accepted+fixed (3240069: Search filter of every captured session tied structurally to AND(objectClass=groupOfNames, member=<that session's decoded Bind DN>) + AND→OR sabotage test), 2 rejected (sandbox disclosure; scope note). Certified via the mandatory post-cap architecture consultation (VERDICT: SHIP, reported_reviewed_sha = 3240069, same conversation https://chatgpt.com/c/6a93ea5b-fafc-83eb-ac9c-58a499894698): "the architecture is sound; the three passes exposed insufficiently discriminating evidence checks, not evidence that the compatibility-profile premise is wrong"; no plan change needed. Unresolved findings: none.
    • Phase 4 must-do recorded here: internal/ldap/clickhouse_wire_cryptobyte_test.go and internal/securitytest/wire_profile_contract_test.go now import the vendored third_party/goldap/message as the independent second decoder (test-only). Deleting the fork in Phase 4 must replace that oracle (e.g. with the Phase 2 first-party decoder or a bounded test-only cursor) — do not silently drop the well-formedness/filter-structure checks.

    Phase 2 — handoff

    • Branch: feat/33-phase2-compat-profile off origin/main @ edebb6d (Phase 1 merge).
    • Plan review: ChatGPT-authored plan reached the 5-pass cap with Fable verdict REVISE, blocker: none (7 specificity findings). Conversation: https://chatgpt.com/c/6a945528-4f08-83eb-87e8-0f23e2bd086e
    • Decision taken (human ruling at the plan gate): approve the latest plan with the 7 pass-5 findings appended as binding amendments (## Coordinator-approved amendments (binding)): add the 64 KiB response cap (adminLimitExceeded) and the startup UserRDNAttribute descriptor regex to the Phase 3 acceptance list of deliberate narrowings; Bind auth choice [3] sasl → result 7, any other tag → close, out-of-range ENUMERATED scope/deref → malformed/close vs !=0 → 50 disambiguated; closed reason= log vocabulary enumerated, legacy literals preserved byte-identically, AST-enforced like diagnostic; go vet -tags phase5release ./internal/securitytest in the validation list; Phase 4 disposition names inverting TestDependencyContract_ProfileImplementationIsNotProduction, flipping NoNonStandardCryptobyte, regenerating production-nonstdlib-deps.txt; Abandon [APPLICATION 16] IMPLICIT integer read under the application tag then validated by the shared minimal-positive-INTEGER rule; test-fake ownership resolved without a non-test testsupport package.
    • Plan shape: new sub-package internal/ldap/profile (cryptobyte primitives; outside the cmd/ch-oauth-ldap closure until Phase 4 cutover; no runtime switch), per-file LOC guardrail table, differential oracle vs vendored goldap (Phase-2-only), fuzz targets seeded from internal/wirefixture. Risk High, Large → decompose-and-implement loop.
    • Landed (PR feat(#33): first-party ClickHouse LDAP compatibility profile as dormant internal/ldap/profile (phase 2) #38, branch feat/33-phase2-compat-profile off origin/main @ edebb6d, head d273414): dormant package internal/ldap/profile (framing/envelope/Controls on cryptobyte, restricted DN grammar, Bind + re-Bind state machine, fixed-shape Search with fail-closed size limit, Abandon/Unbind, encoders with exact ≥128 MessageID round-trip, closed reason/diagnostic log vocabularies, 2,659 physical production LOC measured per wc -l over the 11 non-test files); both-backend real-TCP compat tests; differential oracle vs vendored goldap (Phase-2-only); wire replay of the whole committed corpus; 5 fuzz targets seeded from it; dependency contracts (profile+cryptobyte absent from production closure; profile closure requires cryptobyte, forbids general-LDAP modules); AST architecture contracts; redaction inventory extended. Production cmd/, legacy non-test internal/ldap, third_party/**, go.mod untouched.
    • Decisions taken (coordinator, under the standing "don't stop on human gates" authorization): LOC consolidation trigger (plan's 2,500) tripped at 2,659 → accepted: overshoot is the plan table's omitted config.go/doc.go; well below ADR ADR: Own the ClickHouse LDAP compatibility profile and remove general LDAP dependencies #32's ~3,500 architecture-review trigger; consolidation review folds into Phase 4's deletion. Recorded in docs/clickhouse-ldap-wire-profile.md §11.1.
    • Ten Phase-3-visible narrowings recorded in the wire-profile doc §11.3 (incl. 64 KiB response cap → adminLimitExceeded, UserRDNAttribute startup regex, version-0-closes decode note).
    • Internal review: High-risk read-only pass — no blocking findings; fixed in d273414: write-stall vs timeLimitExceeded classification, DN-error input-byte stripping (redaction rows updated), ASCII-only wire descriptor folding, narrowing/LOC records. Security review: no findings ≥0.7 (deliberate parity noted: token expiry not re-evaluated mid-connection, matches legacy).
    • Phase 4 must-dos accumulated: invert TestDependencyContract_ProfileImplementationIsNotProduction; flip TestDependencyContract_NoNonStandardCryptobyte; regenerate internal/securitytest/testdata/production-nonstdlib-deps.txt; delete differential_test.go; transform compat tests; replace the test-only goldap oracles (also the Phase 1 ones in clickhouse_wire_cryptobyte_test.go/wire_profile_contract_test.go).
    • Review outcome (in progress): PR feat(#33): first-party ClickHouse LDAP compatibility profile as dormant internal/ldap/profile (phase 2) #38 code review session 1: pass 1 (head d273414, feat(#33): first-party ClickHouse LDAP compatibility profile as dormant internal/ldap/profile (phase 2) #38 (comment) — comment landed despite a null return field): 3 findings, 2 accepted+fixed (ba8862d: AST guards extended to inferred make()/method-value shapes; CLAUDE.md closure claim scoped, §11.4 decoder roles un-swapped, Bind version ≥128 delta recorded), 1 rejected (SASL-before-version ordering — legacy also never reads version on non-simple; coverage cell added later). Pass 2 (head ba8862d, #issuecomment-5471642821 sibling): 5 findings, 4 accepted+fixed (1dbaa59: streamed pre-auth AttributeSelection + allocation regression test; size/time-limit precedence reordered to legacy loop order with simultaneous-limit tests; remaining inferred map/chan forms; v2×SASL coverage), 1 rejected (modifydn casing — closed lowercase op vocabulary). Pass 3 (head 1dbaa59, #issuecomment-5471745093): 2 accepted+fixed (685fdf0: var-composite-literal/result-type detection; transitive-alias/interface-parameter closure), 1 rejected (simultaneous-limit two-backend seam — poor complexity-to-assurance return, later endorsed by the consultation). Post-cap architecture consultation (same conversation, reported_reviewed_sha = 685fdf0): VERDICT: REVISE with a single concern — the AST-only checker cannot exhaustively establish the four type-semantic invariants it advertised as mechanical; no remaining runtime/parser concern; plan needs no architectural change.
    • Decision taken (coordinator, standing authorization): adopted the consultation's preferred resolution — rewrote invariants 4/6/12/13 on go/types (stdlib export-data importer via go list -deps -export -json, go.mod untouched), keep syntactic invariants AST-based, reconcile every "exhaustive/mechanical" claim; then re-certify in a fresh pr-mode session continuing the same conversation (sanctioned continuation per review-loops.md). Fallback if type-checking proves infeasible: downgrade the checker claim to heuristic drift detection + mandatory Phase 3 typed-check gate item.
    • Review outcome (final): PR feat(#33): first-party ClickHouse LDAP compatibility profile as dormant internal/ldap/profile (phase 2) #38 merged 2026-08-31 as e26e30f (merged head b62c1c2), Required PR gate green at every pushed head. Session 1 (3 passes, heads d273414→ba8862d→1dbaa59→685fdf0): 11 findings, 8 accepted+fixed, 3 rejected with recorded rebuttals. First post-cap consultation: REVISE — sole concern the AST checker's exhaustiveness claim; no runtime/parser concern. Coordinator ruling: type-aware rewrite (landed f3bf13b: go/types over stdlib export-data importer, go.mod untouched; AST file keeps only syntactic invariants). Session 2 (fresh 3-pass budget, same conversation, heads f3bf13b→aef938b→78570aa→b62c1c2): 6 findings, all accepted+fixed — every one a soundness refinement of the new checker (TypeParam/Union terms; generic Named TypeArgs(); direct-call kind for the Verify/Roles singleton) or doc/PR-description staleness; zero runtime findings across 6 formal passes + 2 consultations. Second post-cap consultation: recommendation "proceed with the current implementation architecture at b62c1c2; do not turn selection enumeration into whole-program reachability analysis" — verdict line initially missing (fail-closed), explicit VERDICT: SHIP obtained in the same session (reported_reviewed_sha = b62c1c2). Certified via post-cap consultation, conversation https://chatgpt.com/c/6a945528-4f08-83eb-87e8-0f23e2bd086e. Unresolved findings: none.
    • Phase 3 note from the consultation: tighten the evidence record, not the parser — exact supported-matrix results + head SHA, explicit fuzz-smoke command/duration, HA/TLS applicability, disposition of every documented client-visible narrowing (wire-profile doc §11.3), final LOC measurement, dependency-contract inversion prep.

    Phase 3 — handoff (in progress)

    • Branch: feat/33-phase3-gates off origin/main @ e26e30f (Phase 2 merge).
    • Plan review: ChatGPT-authored plan approved on pass 3 (no cap exhaustion). Conversation: https://chatgpt.com/c/6a94c8b3-a2c4-83eb-a64b-a8fa06cec23c
    • Landed (PR feat(#33): certify the replacement against all release gates via a temporary phase3profile integration tag (phase 3) #39, head 6040290): temporary phase3profile compile-time seam in cmd/ch-oauth-ldap (ldap_backend_legacy.go sole default importer of internal/ldap; ldap_backend_phase3profile.go sole tagged importer of internal/ldap/profile; backend-neutral config.go; lifecycle preserved verbatim); integration image's helper build line is the tag's ONLY consumer, so the whole Docker suite exercises the replacement; untagged securitytest contracts (tagged composition compiles/tests in CI; staged general-LDAP denylist over the tagged closure with Phase 4 production flip recorded; selector contract pinning the tag to exactly one Dockerfile line and to zero occurrences in the publish path; §11.5 evidence-shape contract incl. independently recomputed certified-surface digest); evidence record (§11.1 LOC correction: merged Phase 2 baseline 2,682 @ e26e30f, Phase 3 final 2,702; §11.3 eleven-row narrowing ACCEPT disposition; §11.4a generation freeze; §11.5 Stage C evidence, tested_behavior_head e4deecd); docs sync (operator guide §8.1/§10, README, CLAUDE.md); Bind version 127/128 boundary subtest (row 1a citation now exact).
    • Coordinator attestation (independent of the Stage B sub-task's identical run): at head 6040290 — run-all-builds.sh both tracked images PASS (expectations unedited); run-ha.sh PASS (all Docker HA claims, leak scans clean); capture-ldap-wire.sh --mode verify PASS ×4 sessions, Search-precedes-Abandon confirmed; fuzz smoke 5×20 s PASS, no crashers; no Docker leftovers; tree unchanged. Raw logs retained at /Users/altinity/tmp/p3-attest/.
    • Internal review: High-risk pass — no blocking findings (should-fix: version-128 subtest, landed 6040290; nit: tagged _test.go files sit outside every CI compile path — documented residual, carried below). Security review: no findings ≥0.7 (sub-threshold notes: no untagged type-assertion on newLDAPServer; deterministicGoListEnv doesn't pin GOFLAGS — both fail closed).
    • Phase 4 must-dos accumulated (added this phase): delete the phase3profile seam (both adapter files, tagged tests, the Dockerfile tag) at cutover; flip the staged denylist to the production closure and regenerate internal/securitytest/testdata/production-nonstdlib-deps.txt; note the tagged test files' CI-compile residual disappears with the seam.
    • Review outcome (in progress): PR feat(#33): certify the replacement against all release gates via a temporary phase3profile integration tag (phase 3) #39 session 1: pass 1 (head 6040290, feat(#33): certify the replacement against all release gates via a temporary phase3profile integration tag (phase 3) #39 (comment)): 3 findings accepted+fixed (0bd5564: untagged backstop RUNS the tagged command tests; selector bounds total helper-build lines and /out/ch-oauth-ldap writers; §11.5 table extractor anchored to its subsection). Pass 2 (head 0bd5564): 2 accepted+fixed (73452c9: compound-single-RUN artifact-writer decomposition incl. go install/cp/mv/redirects; exact runtime-COPY tuple; Phase 4 handoff lists gain …CommandTests). Pass 3 (head 73452c9, #issuecomment-5473427481): 2 accepted+fixed (9b5396c: GOBIN=… go install writer form; CLAUDE.md tagged-gate prose sync). Post-cap consultation (reported_reviewed_sha = 9b5396c): VERDICT: REVISE — single required change: selector must also reject COPY/ADD instructions writing /out/ch-oauth-ldap (first-class Dockerfile writers, not shell tricks); explicitly: no architectural change needed, staged-closure interpretation correct, retaining the 6040290 Docker attestation defensible (later commits are contracts/docs only), and simplify rather than generalize the selector. Coordinator ruling (standing authorization): apply the surgical COPY/ADD fix, then re-certify in a fresh pr-mode session continuing the same conversation.
    • Review outcome (final): PR feat(#33): certify the replacement against all release gates via a temporary phase3profile integration tag (phase 3) #39 merged 2026-08-31 as 88c96bb (merged head c9547db), Required PR gate green at every pushed head. Session 1 (3 passes, heads 6040290→0bd5564→73452c9→9b5396c): 8 findings, all accepted+fixed. Post-cap consultation: REVISE with one required change (COPY/ADD writer hole); explicitly no architectural change, staged-closure interpretation endorsed, 6040290 Docker attestation retention endorsed. Coordinator ruling: surgical fix (6f98cff) + fresh session. Session 2 (heads 6f98cff→a87795a→c9547db): pass 1 — 3 accepted+fixed (JSON-array COPY/ADD; denylist cardinality guard; PR-body attestation SHA named); pass 2 — 1 accepted+fixed (directory-destination cp/mv + trailing-slash GOBIN normalization); pass 3 — certified-pending-proofs at c9547db (the run's first formal certification; no findings), feat(#33): certify the replacement against all release gates via a temporary phase3profile integration tag (phase 3) #39 (comment). Unresolved findings: none. Conversation: https://chatgpt.com/c/6a94c8b3-a2c4-83eb-a64b-a8fa06cec23c
    • Phase 4 must-dos consolidated (authoritative list for the cutover plan): switch cmd/ch-oauth-ldap's default build to internal/ldap/profile and delete the phase3profile seam (both adapter files, tagged test files become untagged, the integration Dockerfile tag, TestDependencyContract_Phase3ReplacementCommandBuilds + …CommandTests, the whole phase3_selector_contract_test.go); remove third_party/goldap + third_party/ldapserver + their replace directives and production go-ldap/go-asn1-ber/go-ntlmssp; flip TestDependencyContract_ProductionClosureHasNoGeneralLDAP to postPhase4 and regenerate internal/securitytest/testdata/production-nonstdlib-deps.txt; invert TestDependencyContract_ProfileImplementationIsNotProduction, flip TestDependencyContract_NoNonStandardCryptobyte; replace the test-only goldap oracles (clickhouse_wire_cryptobyte_test.go's independentlyWellFormedBER, wire_profile_contract_test.go's decode legs, the differential harness — delete differential_test.go); transform profile_compat_test.go (legacy backend gone); delete legacy internal/ldap and its tests; redaction-inventory re-baseline (drop third_party rows because the files no longer exist, keep profile rows); docs/CLAUDE.md sync; rerun all gates incl. Docker matrix/HA/wire-verify on the now-default replacement; rollback = revert/redeploy, no dual-parser mode.

    Phase 4 — handoff (in progress)

    • Branch: feat/33-phase4-cutover off origin/main @ 88c96bb (Phase 3 merge).
    • Plan review: ChatGPT-authored plan approved on pass 3. Conversation: https://chatgpt.com/c/6a9504f8-4c90-83eb-9e5c-ee1e9a5e6416
    • Landed (PR feat(#33): cut production over to internal/ldap/profile and delete the general LDAP stack (phase 4) #41, head 0ccbd43): default cmd/ch-oauth-ldap now constructs internal/ldap/profile.Server (ldap_backend.go = the Phase-3-certified adapter minus the tag); third_party/goldap + third_party/ldapserver + replace directives, legacy internal/ldap package, production go-ldap/go-asn1-ber/go-ntlmssp all deleted (−28,994 lines; go mod tidy no-op); oracles replaced not dropped (Oracle A bounded cursor in the relocated primitive-decision test; Oracle B independent semantic cursor in wire_oracle_b_test.go); differential + compat harnesses deleted after transplant audit; raw-PDU TCP client + raw net.Conn HA probe replace all go-ldap usage; contracts inverted (ProductionClosureHasNoGeneralLDAP unconditional — the stage enum deleted outright, stronger than the planned postPhase4 flip; ProfileIsProductionImplementation; cryptobyte-required; production-nonstdlib-deps.txt regenerated, 40 entries, reviewer-recomputed exact) plus new root test-graph + go.mod-metadata denylists and ch_oauth_ldap_build_contract_test.go (retired selector may never reappear in the build/publish path); §11.5 historicalized, §11.6 Phase 4 evidence added (final LOC 2757 = profile 2693 + adapter 64, reviewer-recomputed exact); redaction inventory re-baselined (26 legacy + 22 fork + 3 adapter rows removed with their files, 1 adapter row added, profile rows untouched); docs synced; fuzz targets relocated with the package.
    • Coordinator attestation at 76e8bcc (every later commit is comment/doc/test-only; §11.6 records the digest refresh honestly): run-all-builds.sh both tracked images PASS; run-ha.sh PASS (leak scans clean); capture-ldap-wire.sh --mode verify PASS ×4; fuzz 5×20 s PASS. Logs at /Users/altinity/tmp/p4-attest/.
    • Internal review: no blocking findings (2 stale-comment items fixed in 0ccbd43; stale third_party/** publish-workflow path-filter pair deferred to new issue ci: drop stale third_party/** path filter from build-ch-oauth-ldap.yml (+ helm assertion pair) #40). Security review: no findings ≥0.7 (cutover parity logic-identical; every deleted legacy security suite has a profile owner; oracles strictly stricter; contracts non-vacuous).
    • Review outcome (final): PR feat(#33): cut production over to internal/ldap/profile and delete the general LDAP stack (phase 4) #41 merged 2026-08-31 as c2902fc (merged head 154b8ab) — issue Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33 closed by this merge. Code review (one session, conversation https://chatgpt.com/c/6a9504f8-4c90-83eb-9e5c-ee1e9a5e6416): pass 1 (head 0ccbd43, #issuecomment-5475047580) — 1 finding rejected with evidence (the §11.6 digest refresh was a disclosed comment-only update); pass 2 (#issuecomment-5475116061) — 1 accepted+fixed (25089e0: digest mechanically bound to tested_behavior_head via git-object reads + reachability check; head advanced to the digest's true commit); pass 3 (#issuecomment-5475273487) — 2 accepted+fixed (154b8ab: fetch-depth: 0 on the PR gate's checkout + gate-contract pin, fixing the one red CI run of the entire issue — run 33368931796, shallow clone couldn't resolve the recorded head; §11.6 gains a distinct manual_verification_head (3c12ab7) and the attestation sentence names it). Certified via the post-cap architecture consultation ("I would merge feat(#33): cut production over to internal/ldap/profile and delete the general LDAP stack (phase 4) #41 once the normal merge conditions are satisfied, update the Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33 ship log to the actual merged SHA, and close Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33"; explicit VERDICT: SHIP obtained in the same session after the verdict line was initially omitted; evaluated head 154b8ab named in the response). Unresolved findings: none. Follow-up filed during review: ci: drop stale third_party/** path filter from build-ch-oauth-ldap.yml (+ helm assertion pair) #40 (stale third_party/** publish path filter + helm assertion pair). Consultation's open suggestion for the maintainer: decide whether §11.6 remains a permanent live recertification gate or becomes frozen historical evidence like §11.5.
  3. added 13 commits that reference this issue on Aug 30, 2026
  4. 80 remaining items

  5. added 13 commits that reference this issue on Aug 31, 2026
  6. added a commit that references this issue on Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0