8000 ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate · Issue #23 · Altinity/altinity-oauth-helper · GitHub
[go: up one dir, main page]

Skip to content

ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23

Closed

Description

@BorisTyshkevich

Summary

This repository currently has no PR-time CI. .github/workflows/build-ch-jwt-verify.yml builds/pushes an image on push to main, but pull requests are not automatically gated.

That was already a reliability gap for the #19 work. It becomes a security prerequisite for ADR #32 and its implementation tracker #33, which replace the unauthenticated LDAP parser/dependency boundary. Parser/security changes must not rely on a coordinator running local commands by hand.

This issue adds a cheap, mandatory, non-Docker PR gate and keeps the real ClickHouse Docker matrix separate.

Relationship to ADR #32 and implementation #33

ADR #32 requires an active PR-time gate before production parser replacement work can merge. The concrete multi-phase implementation is tracked in #33.

This issue, or an equivalent workflow providing the same guarantees, MUST be completed before production LDAP parser replacement changes from #33 are merged.

The gate must automatically pick up future first-party security contract tests added under internal/securitytest, including the planned production dependency-closure contract from #33.

Required workflow

Add a workflow under .github/workflows/ triggered on:

pull_request:
  types: [opened, synchronize, reopened, edited]
push:
  branches: [main]

Do not use pull_request_target for this gate.

types must be listed explicitly, and edited is load-bearing — this is a normative
requirement, not a stylistic detail.
A bare pull_request: runs only for opened,
synchronize and reopened. A base-branch retarget is delivered as an edited activity
and keeps the same head SHA, and the check-run is recorded against that head SHA — so
without edited, a pull request proven green against some other base can be retargeted
onto protected main and carry that same green required check across, with the new merge
result never tested. Base-retarget revalidation is therefore an acceptance invariant of
this issue.

Known limit, stated precisely: a pull_request.edited event produced with the
repository's own GITHUB_TOKEN does not create a workflow run. The recursion guard
has documented exceptions — workflow_dispatch and repository_dispatch always create
runs, and pull_request with opened/synchronize/reopened does create runs but in an
approval-required state — and edited is not one of them. So edited covers base
changes whose events actually reach Actions: a human, or an actor other than
GITHUB_TOKEN. The two GITHUB_TOKEN cases are not equivalent:

  • base retarget (edited) can fail OPEN — no run is created and the head SHA does not
    change, so a previous successful Required PR gate on that head remains the status
    satisfying branch protection: exactly the stale-green path this types list exists to
    close. strict: true does not rescue it, because an unchanged head can already contain
    the newly selected base's tip and so already satisfies the up-to-date requirement
    without any fresh verification. Repository automation must not retarget a pull
    request's base with GITHUB_TOKEN — this is a prohibition, not a preference.
  • synchronize fails closed — it creates a run in an approval-required state, so the
    check does not go green until someone with write access approves it and it succeeds.
    Safe, but not silent revalidation; do not expect it to behave like a human-generated
    synchronize.

(Both the explicit types list and this rationale were added during implementation, after
review established the stale-green retarget path; the original snippet here showed a bare
pull_request:. See the ship log.)

Run the following on every pull request and push to main:

go build ./...
go vet ./...
go test -race ./...
go test -tags phase5release ./internal/securitytest -count=1
bash integration/clickhouse/tests/lib-tests.sh

Why each command is required

go build ./...

Proves all normal production/test packages compile under the repository's actual module graph.

go vet ./...

Keeps the existing cheap static correctness pass in the default gate.

go test -race ./...

Runs the complete normal Go test suite of the root module under the race detector.
This is especially important for connection lifecycle/session state changes in
internal/ldap.

Note the boundary, because it matters for #33: ./... does not traverse
third_party/goldap or third_party/ldapserver. Those are separate Go modules that the
root go.mod only pulls in via replace directives, so this command never compiles
their own test suites — and that is where the general LDAP/BER parser #33 replaces
currently lives. Parser tests intended to be gated must live inside the root module, or
a separate command must be added for those modules. (Corrected during implementation;
see the ship log.)

Do not replace this with a small package allowlist; new packages/tests should enter the PR gate automatically.

go test -tags phase5release ./internal/securitytest -count=1

The phase5release security gate must run on pull requests, not only at release time.

It currently protects audited security/redaction contracts and must remain green through the #33 LDAP rewrite. Future dependency-closure and wire-profile security contracts should be included in internal/securitytest so this workflow picks them up automatically.

-count=1 is intentional so this gate is not satisfied from the Go test cache.

integration/clickhouse/tests/lib-tests.sh

Runs the daemon-free tests for the integration harness's shell libraries, including credential-safe error paths, leak-scan plumbing, teardown/trap behavior, and expectation-table logic.

These are Docker-daemon-free and need no ClickHouse image, but they are not
literally bash/coreutils-only: lib-tests.sh sources the real
integration/clickhouse/lib/common.sh, which runs detect_compose_cmd at source time
and dies if neither docker compose nor docker-compose is on PATH. Verified:
with PATH reduced to /usr/bin:/bin:/usr/sbin:/sbin the script aborts with
neither 'docker compose' nor 'docker-compose' is available on PATH. ubuntu-latest
ships the Docker CLI, so the gate is unaffected — recorded here so a future runner-image
change is not a mystery failure. (Corrected during implementation; see the ship log.)

Security requirements for the workflow

This is a verification workflow for untrusted pull-request code. Keep the workflow itself narrow:

  • repository permissions should be read-only (contents: read) unless GitHub requires an additional read permission for checkout/status reporting;
  • do not expose repository, cloud, IdP, Docker registry, or release secrets to the PR job;
  • do not use pull_request_target to execute PR-controlled code;
  • do not publish images or release artifacts from this workflow;
  • third-party GitHub Actions should be pinned to immutable commit SHAs rather than floating tags where practical;
  • no test in this gate may require production credentials or network access to an IdP.

The LDAP wire fixtures introduced by #33 must use generated/non-secret credentials, so they are safe for this PR job.

Docker ClickHouse matrix stays separate

Keep the real supported-ClickHouse matrix outside this cheap PR gate for now:

./integration/clickhouse/run-all-builds.sh

It requires Docker Compose, pulls multiple ClickHouse/Altinity images, and is currently single-instance per Docker daemon. It can remain manual, scheduled/nightly, or release-gated.

This issue does not weaken that matrix as a product/release gate; it only avoids making every PR depend on Docker infrastructure.

If the Docker matrix is later moved into concurrent CI, make COMPOSE_PROJECT_NAME and fallback network names run-specific first.

Branch protection / merge enforcement

A workflow that merely runs but can be ignored is insufficient for #32/#33.

After the workflow is merged and has produced its stable check name, configure the repository's protected main branch/ruleset so that the PR-time gate is a required status check before merge.

If branch-protection configuration is managed outside this repository, record the exact required check name and verify enforcement as part of this issue's completion.

Failure behavior

Any failure in build, vet, race tests, phase5release, or shell library tests must fail the single required PR gate.

Do not mark security-test steps continue-on-error, optional, or advisory.

Avoid path filters initially. Changes to dependency files, shared packages, workflows, generated fixtures, or docs/security contracts can affect the LDAP/security boundary in non-obvious ways; the gate is cheap enough to run for every PR.

Optional follow-ups, not required here

  • golangci-lint may be added later once the repository has an agreed configuration.
  • The Docker ClickHouse matrix may move to scheduled or PR CI separately.
  • Native fuzzing added by Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33 should have a bounded CI smoke strategy and a longer scheduled corpus run, but designing that policy belongs to the implementation tracker rather than this base-gate prerequisite.

Definition of done

  • A PR-triggered workflow exists and also runs on pushes to main.
  • It runs go build ./....
  • It runs go vet ./....
  • It runs go test -race ./....
  • It runs go test -tags phase5release ./internal/securitytest -count=1.
  • It runs bash integration/clickhouse/tests/lib-tests.sh.
  • The PR job executes with read-only/minimal permissions and no production/release/IdP secrets.
  • It does not use pull_request_target.
  • Its required third-party Actions are pinned to immutable revisions where practical.
  • The gate is configured as a required status check for merges to main (or the external ruleset owner has confirmed equivalent enforcement).
  • A deliberately failing Go test/security test causes the workflow to fail, proving it is not advisory.
  • Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33 production parser replacement does not merge before this gate is active and enforced.
  • The Docker ClickHouse compatibility matrix remains available as the separate integration/release gate.

Context

CLAUDE.md already requires lint/CI additions to be wired into a real workflow rather than existing only as local commands. This issue is that workflow-level enforcement.

Originally raised by the post-cap review of the #19 series / PR #22. Updated as a prerequisite of ADR #32 and implementation tracker #33.


✅ Delivered — .github/workflows/pr-gate.yml is active and enforced

Shipped by PR #34 (merge commit e9c7c3c). All 13 Definition-of-Done items above are
satisfied; each is evidenced in the ship log,
which also carries the complete run ledger and a round-by-round record of what review found.

Enforcement, read back fresh from the API after configuration:

Field Value
main protected true
required context Required PR gate (exact)
check bound to app github-actions, app id 15368 — the app observed emitting the check on main, not a guessed id
strict (require branches up to date) true
enforce_admins true
required reviews / restrictions none (deliberate — see below)

No other policy was introduced: no required signatures, no linear-history requirement, no
force-push or deletion allowances, no conversation-resolution requirement, no lock.

strict: true is load-bearing. With loose required checks, a PR green against
main = B0 can merge unchanged after another PR moves main to B1 — it never receives a
synchronize event, so a green gate would authorize an untested B1+H merge result. That is
the same fail-open class as the base-retarget path the edited activity type closes, for the
far more common trigger.

No review requirement, deliberately. The repository has no always-available human
approver; requiring one would convert a possible weakness into a guaranteed merge deadlock.
Classic branch protection is repository-admin-managed policy in any case, so a review
requirement would not make it an independent authority against a determined maintainer.

Accepted residual risk, stated plainly: a pull request runs its own copy of
.github/workflows/pr-gate.yml, and a job skipped by a job-level conditional reports
Success to branch protection. So this gate enforces process for trusted maintainers and
catches ordinary regressions; it is not tamper-resistant against an authorized author who
deliberately neutralizes both the workflow and its self-test. Closing that would require an
organization-level required-workflow ruleset sourced from outside this repository — org-owner
authority, a different enforcement architecture, and out of scope here.

#33's precondition is now satisfied. All three conditions hold: the workflow is merged; a
push-event Required PR gate succeeded on main at e9c7c3c
(run 33283876015,
all five steps); and protection actively requires that exact context, read back above.

One boundary #33 must plan around, corrected in this issue's own text above: go test -race ./... covers the root module only. third_party/goldap and third_party/ldapserver are
separate modules consumed via replace, so the gate never compiles their test suites — and
that is where the parser #33 replaces currently lives. Parser tests intended to be gated
must live inside the root module, or the workflow needs an additional command for those
modules.

Activity

  1. changed the title [-]ci: add PR-time gate (go build/vet/test -race + integration/clickhouse/tests/lib-tests.sh), keep the Docker ClickHouse matrix manual[/-] [+]ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate[/+] on Aug 29, 2026
  2. added 2 commits that reference this issue on Aug 29, 2026
  3. BorisTyshkevich commented on Aug 29, 2026

    @BorisTyshkevich
    CollaboratorAuthor

    Ship log — #23

    Phase Status PR
    (unphased — one whole-issue unit) shipped #34

    Merged: e9c7c3c (merge commit, 2 parents, verified on origin/main).
    Certified by: the post-cap architecture consultation's VERDICT: SHIP at head
    f743963, not a formal pr-mode pass — reported_reviewed_sha matched the PR head
    exactly. Conversation: https://chatgpt.com/c/6a933fd2-2388-83eb-8c55-6b22979c4e99
    Enforcement live: main protected; required context Required PR gate bound to app
    github-actions (id 15368, observed); strict: true; enforce_admins: true; no review
    requirement. Read back fresh from the API — see the enforcement section in the issue body.

    Unit — handoff

    Scope. Issue #23 has no ## Phases checklist, so this is one unit: one branch
    (ci/pr-gate), one PR, one review loop, one merge. All 13 Definition-of-Done items are
    claimed by this unit; none deferred.

    Landed on the branch (2 commits):

    • ca79a9e ci(#23): add required PR verification gate — .github/workflows/pr-gate.yml
      (workflow PR gate, single job Required PR gate, pull_request unfiltered +
      push.branches: [main], permissions: contents: read, persist-credentials: false,
      cache: false, timeout-minutes: 15, both actions pinned to full commit SHAs), plus
      the docs reconciliation this makes mandatory (CLAUDE.md, README.md,
      skills/ship/references/repo-footguns.md, skills/ship/SKILL.md step 2.5).
    • c8538af fix(#23): address internal review — docs accuracy and a machine-checked gate contract — two factually inverted doc claims, four accuracy fixes, and the new
      internal/securitytest/pr_gate_contract_test.go.

    Plan review: approved on pass 3 of 5 (ChatGPT authored, Fable/high approved).
    Conversation: https://chatgpt.com/c/6a933fd2-2388-83eb-8c55-6b22979c4e99

    Decisions taken (not in the issue)

    1. One job with five sequential steps, not multi-job + aggregator. Gives exactly one
    merge-relevant check name and no needs graph. Decisive reason: a job skipped by a
    job-level condition reports success, so an aggregator is a silent-waiver shape.

    2. Required context is the job name Required PR gate (workflow PR gate, job id
    required-pr-gate). Treated as an externally consumed interface: renaming the job means
    migrating branch protection in the same change. Now machine-enforced (see 5).

    3. Classic branch protection, not the repo's first ruleset. Narrowest mechanism for
    one required status on one branch. Configuration — this is the authoritative rollout
    procedure; apply exactly this:

    Setting Value
    required status check context Required PR gate (exact string, app github-actions)
    strict (require branches up to date) true
    enforce_admins true
    review-count requirement none

    strict: true is a correction, recorded here in full because the original value was
    wrong and this table is what a future operator will apply.
    This entry previously said
    strict/up-to-date false. The post-cap architecture consultation identified that as the
    same class of fail-open the pull_request types: [... edited] fix closed, but for the
    more common case — base advancement rather than retarget. Concretely: PR H is
    green against main = B0; another PR lands and moves main to B1; H itself does not
    change, so there is no synchronize event and no new gate run. With loose required
    checks GitHub permits H to merge without being brought up to date, so a green
    Required PR gate authorizes an untested B1+H merge result. Rerunning on retarget
    while accepting untested base advancement applies the opposite rule to routine
    repository activity — the two settings were logically inconsistent. The cost of
    strict: true is extra branch updates and CI churn when another PR lands; at this
    repo's traffic, with a ~2-minute gate, that is clearly the right trade. If it ever
    becomes painful, a merge queue plus a merge_group: trigger is the structurally
    cleaner scaling answer (the workflow already carries a comment saying so).

    A later review pass caught that an earlier commit message claimed this correction was
    "recorded in the ship log" when it had not yet been — the log still specified
    false. That is why it is spelled out as a table here rather than as prose.

    On read-back, verify all four fields, not just the context string: strict == true,
    enforce_admins == true, the exact context Required PR gate, and — where GitHub
    exposes it — that the required check is bound to the observed GitHub Actions app rather
    than a guessed app id.

    4. Residual risk, accepted deliberately and recorded here verbatim — the gate is not
    tamper-resistant against an authorized PR author.
    Because the trigger is
    pull_request (correctly, not pull_request_target), a pull request runs its own
    copy
    of .github/workflows/pr-gate.yml. A PR that keeps the job name Required PR gate but either replaces the five steps with something trivial, or adds a job-level
    if: false, produces a green required check that verified nothing — and once merged
    that neutered file governs every later PR. The non-obvious part: a skipped job
    satisfies a required status context
    , so reading the steps during review is not
    sufficient to catch this.

    Consulted ChatGPT on the mitigation (same conversation as above; ad hoc, not a formal
    review pass). Its ruling, adopted:

    • Do not add a required approving review or a CODEOWNERS review requirement. With no
      human approver available it "would convert a possible security weakness into a
      guaranteed operational deadlock", and more fundamentally classic branch protection is
      repository-admin-managed policy — against a malicious admin it was never an
      independent boundary, so a review requirement buys much less than it costs.
    • Keep enforce_admins: true: it prevents an accidental admin bypass and costs nothing
      when only a status check is required.
    • Do not fold repo-wide Actions permission defaults into ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23 — a PR author with write
      access can override permissions: in the workflow anyway, and this workflow already
      declares contents: read explicitly. Separate hardening follow-up.
    • What would actually close this hole is an organization/enterprise-level
      required-workflow ruleset, which sources the enforced workflow from outside the
      repository being modified. That needs organization-owner authority (not merely repo
      admin) and changes the agreed enforcement architecture, so it is out of scope for ci: add required PR-time Go/race/security gates; keep Docker ClickHouse matrix separate #23
      and worth evaluating separately.

    Therefore this gate's honest claim is: it enforces process for trusted maintainers and
    catches ordinary regressions; it is not tamper-resistant against an authorized actor
    deliberately modifying the gate itself.
    #33 should be unblocked on that understanding,
    not on a stronger one.

    5. Added internal/securitytest/pr_gate_contract_test.go — an explicit extension of
    the approved plan, authorized by the plan's own author.
    The plan's invariant map left
    three invariants ("all five commands control one merge result", "no skipped aggregator
    can report misleading success", "required-check name cannot silently drift") enforced
    only by human static review. ChatGPT extended the plan to add an executable contract in
    this unit, untagged — matching convention, since release_gate_test.go is the only
    phase5release-tagged file in that package and every other contract test there is
    untagged. Untagged means it runs in plain go test ./... and in the gate's own
    go test -race ./... step. Uses gopkg.in/yaml.v3, already a direct dependency.

    Two design constraints that are the point of the test: every expectation is a literal
    in the test source
    (a test that reads the job name out of the YAML under test and
    compares it to itself detects nothing), and action pins are asserted for
    immutability (known identity + ^[0-9a-f]{40}$), not today's exact SHA values —
    hardcoding those would turn a legitimate actions/checkout upgrade into unrelated test
    churn for no security gain.

    Its stated scope, in its own doc comment: an accidental-drift detector, not tamper
    resistance
    — it lives in the same PR-controlled tree as the workflow and runs only
    because that workflow runs it, so decision 4's deliberate case defeats it too.

    6. -race on the whole tree, no allowlist, and the phase5release step names the
    whole package with -count=1, so future internal/securitytest contracts (including
    the planned TestDependencyContract_ProductionClosureHasNoGeneralLDAP) enroll with no
    workflow edit. Proven locally: a new tagged fatal test turned the exact required command
    red with zero workflow change.

    Deviations from the spec

    • The issue body's claim that integration/clickhouse/tests/lib-tests.sh "need[s]
      only bash/coreutils and do[es] not require Docker" is imprecise.
      It is Docker-
      daemon-free but not Docker-CLI-free: integration/clickhouse/lib/common.sh
      calls detect_compose_cmd at source time and dies, so with PATH reduced to
      /usr/bin:/bin:/usr/sbin:/sbin the script aborts with neither 'docker compose' nor 'docker-compose' is available on PATH (reproduced). ubuntu-latest ships the Docker
      CLI, so the gate is unaffected — but this is now documented in CLAUDE.md so a future
      runner-image change is not a mystery red.
    • The gate does not reach third_party/goldap / third_party/ldapserver — separate
      modules the root go.mod only replaces in, so go test -race ./... never compiles
      them. That is where the LDAP parser Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33 replaces currently lives. Not a DoD miss (the
      issue fixes the command list at five), but relevant to Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33: parser tests may need to
      live inside the main module to be gated at all.
      Now noted in CLAUDE.md.
    • Part of #23, not Closes #23, on the PR: DoD items 10–12 (branch protection, the
      hosted red proof, Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33 blocking) are coordinator-owned and not satisfied by the diff
      alone.

    #33 prerequisite — the unblock condition

    DoD item 12 ("ADR #32 Phase 2 production parser replacement does not merge before this
    gate is active and enforced") binds #33 (Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile); #32 is the ADR itself.

    Merging this repository change is NOT sufficient. #33 stays blocked until all three
    hold, each verified and recorded below:

    1. the workflow is merged to main;
    2. a successful Required PR gate run has been observed on a main SHA (push event);
    3. classic protection on main actively requires the exact context Required PR gate,
      read back from the API.

    Docker ClickHouse matrix

    Unchanged and deliberately still separate: integration/clickhouse/run-all-builds.sh
    (and run.sh / run-ha.sh) are neither invoked by nor changed by this unit, and remain
    the manual/local product gate. tests/lib-tests.sh — daemon-free — is the only part of
    that directory now in CI.

    Verification at branch head c8538af

    Coordinator-run (not worker self-report): go build ./..., go vet ./...,
    go test -race -count=1 ./..., go test -tags phase5release ./internal/securitytest -count=1, bash integration/clickhouse/tests/lib-tests.sh,
    go vet -tags phase5release ./internal/securitytest — all exit 0. Full
    bash helm/ch-oauth-ldap/test.sh → GATE RESULT: PASS (0 failures) (it makes literal
    substring assertions against CLAUDE.md and repo-footguns.md, including
    assert_not_match "$footguns" 'only workflow'). Pinned actionlint v1.7.7 clean on
    pr-gate.yml. Both pinned action SHAs independently confirmed against the GitHub API:
    actions/checkout@11d5960a… = v4.4.0, actions/setup-go@40f1582b… = v5.6.0.
    Workflow structure independently re-derived by parsing the committed YAML.

    Sabotage cases for the new contract test — 5 run by the fix agent (job rename, path
    filter, continue-on-error, contents: write, unpinned @v4) plus 1 run independently
    by the coordinator (if: false, i.e. decision 4's exact attack shape). All six failed
    with a message naming the security consequence; pr-gate.yml restored byte-identical
    each time.

    Follow-ups filed

    Still outstanding at the time of writing

    Hosted proofs: healthy PR run, deliberate red run + normal revert, restored green,
    certification, merge, main push run, branch protection + read-back.


    Update — hosted CI evidence (all four runs)

    PR: #34
    Emitted check context, confirmed empirically (not assumed): Required PR gate,
    app github-actions. This is the exact string branch protection must require.

    The gate found two real defects on its very first run

    Run 33276319268
    at head c8538af — pull_request — failure at step 6 Race tests; steps 7-8 skipped
    (sequential job). Neither defect reproduces on darwin/arm64 (-race -count=3: ok).

    Defect A — a test that could not prove its property on Linux.
    TestAdversarial_WriteDeadlineClosesStalledConnectionAndUnblocksGracefulShutdown
    created backpressure by sending a 20 MiB role with WriteTimeout lowered to 300ms and
    the client reading nothing for 1500ms. On the runner 41,943,235 bytes — the entire
    ~40 MiB response — were delivered
    : no server write ever blocked >300ms, WriteTimeout
    never fired, the connection stayed open, and the client's own 3s read deadline produced
    i/o timeout. The mechanism under test was never engaged. Cause: darwin's
    net.inet.tcp.sendspace/recvspace are 131072 (128 KiB), so a 20 MiB write blocks
    instantly here; the hosted runner buffered the whole response. The test's premise that
    20 MiB exceeds "any realistic default buffer size" is false there.

    Worth recording because it nearly cost a wasted cycle: an internal reviewer predicted
    this failure but proposed raising truncatedCeiling from 15 MiB toward 18 MiB. That
    fix would have done nothing
    — the run failed at adversarial_test.go:432 on
    copyErr != nil, before the ceiling check at line 436 is ever reached.

    Defect B — a DATA RACE that was a consequence of Defect A, not a production bug.
    ldapserver.Logger (a package-level global in the vendored fork) written by
    internal/ldap.New() at server.go:184, read by client.close(). The tempting fix was
    to move that assignment into init()/sync.Once — a production change. The actual root
    cause, per the goroutine attribution in the trace: the reader was created by Defect
    A's test (adversarial_test.go:392) and the writer was the next test's New(),
    because Defect A's test t.Fatalfs at line 432 before reaching its own teardown at
    lines 442-462. A failing test leaked a live server into the next test's New(). That is
    a test-lifecycle violation of the constraint server.go's own type doc already declares
    ("at most one Server per process at a time"); production calls New() once per process
    before Serve() and cannot hit it.

    Consulted ChatGPT (same conversation, ad hoc — not a formal pass). Its ruling, adopted:
    fix the cause in the tests, leave production server.go untouched, and if the race still
    reproduced afterwards, report rather than work around. It also rejected candidate fixes
    1-3 for Defect A (bigger payload, socket-buffer-derived sizing, client SetReadBuffer)
    as more buffer-size roulette — the test's own doc comment already recorded SetReadBuffer
    and many-small-responses as tried and unreliable.

    Fixes — entire diff in *_test.go; git diff origin/main..HEAD -- internal/ldap/server.go
    is empty:

    • fb2f6e5 — Defect A: the stall now runs over an unbuffered net.Pipe behind a
      one-connection test listener, so backpressure is deterministic on every host, with no
      payload-size heuristic (the 20 MiB role, fullResponseFloor and truncatedCeiling are
      gone). Bind is now genuinely validated (ber.ReadPacket + result code) rather than
      "some bytes arrived". The graceful-shutdown half is preserved intact. Defect B: a
      shared startServing helper gives 7 server-start sites across
      adversarial_test.go, conncap_test.go, midsearch_test.go and protocol_test.go
      fail-safe sync.Once + t.Cleanup teardown registered before any assertion can
      abort, preserving startTestServer's load-bearing ordering (own listener Close →
      receive serveErr → srv.Stop() → cancel()).
    • ff25f23 — third_party/ldapserver/PATCHES.md still described this test as using "a
      shrunk receive buffer plus a volume of ordinary, fast-completing Binds". Stale before
      the change, doubly so after.

    Proof the rewritten test still detects the real defect (it must not pass for the
    wrong reason). Coordinator-run sabotage, independent of the implementer's: set
    WriteTimeout = 60s →
    adversarial_test.go:538: server delivered 163 bytes of the stalled response after 2.5s of the client reading nothing — its write was never bounded by WriteTimeout (500ms), so the connection was never closed. The only difference between pass and fail is whether
    writeMessage's SetWriteDeadline fired. File restored byte-identical.

    Proof Defect B is fixed at the root: -race -count=5 ./internal/ldap → exit 0, zero
    DATA RACE (143.7s). Stronger: with the write-deadline test deliberately failing, the
    following test still saw no race — i.e. a failing test no longer leaks a server.

    Run 2 — fully green

    Run 33277452419
    at ff25f23 — pull_request — success, 1m42s (vs. timeout-minutes: 15). All five
    commands executed: Build, Vet, Race tests, Phase 5 security tests, ClickHouse shell
    library tests.

    Run 3 — the deliberate red proof (DoD item 11)

    Run 33277581488
    at e37e24d — pull_request — failure:

    Step Conclusion
    4. Build success
    5. Vet success
    6. Race tests success
    7. Phase 5 security tests failure
    8. ClickHouse shell library tests skipped
    job Required PR gate failure

    --- FAIL: TestPRGateDeliberateFailureProof /
    FAIL github.com/altinity/altinity-oauth-helper/internal/securitytest 0.033s.
    Check-run Required PR gate conclusion=failure.

    Two deliberate design choices make this a proof rather than a coincidence:

    1. Steps 4-6 all passed, so the red result is attributable to the security step alone.
    2. The test carried //go:build phase5release, making it invisible to
      go test -race ./... and fatal only to
      go test -tags phase5release ./internal/securitytest -count=1.

    This mattered: run 1's accidental failure was in the -race step, which skipped steps
    7-8, so the security step's wiring to the required status had never actually been
    exercised. Asked ChatGPT whether run 1 already satisfied DoD item 11; it said no, for
    exactly that reason, and that keeping the purpose-built proof "distinguishes incidental
    failure detection from a controlled proof of the security gate's wiring".

    • e37e24d test(#23): prove phase5release PR gate fails closed
    • a11eac0 revert(#23): remove deliberate phase5release gate failure — a normal
      inverse commit, not a force-push
      , so the red run and its cause remain in this PR's
      history as the audit trail. Tree at a11eac0 is byte-identical to the already-green
      ff25f23 (git diff --stat ff25f23 HEAD empty).

    Out-of-scope discoveries (follow-ups, deliberately not in this PR)

    • third_party/ldapserver's own test suite cannot run standalone:
      missing go.sum entry for module providing package github.com/vjeantet/goldap/message.
      Pre-existing — the entry is absent on origin/main too, and this branch's only
      third_party/ edit is a markdown file. CLAUDE.md documents running
      third_party/goldap's suite explicitly (it passes: ok …/goldap/message) but not
      ldapserver's, so cancel_test.go, client_close_test.go, client_test.go,
      e2e_test.go, packet_test.go and server_test.go exist but are unrunnable.
    • midsearch_test.go's two tests still reach their stall over real TCP, so on a
      runner with large buffers they tolerate rather than deterministically create the
      stall. Their assertions do not depend on the stall (verified), so they pass either way
      — but converting them to the same net.Pipe substrate would make them deterministic.
    • Repo-wide Actions hardening (default_workflow_permissions write → read,
      can_approve_pull_request_reviews → false); no github-actions dependabot ecosystem,
      so the two new SHA pins will silently go stale; .DS_Store untracked and not
      gitignored.
    • gofmt -l internal/ldap/ flags entry_test.go and filter_test.go — pre-existing
      drift, neither in this branch's diff.

    Definition of Done — no amendment

    Asked ChatGPT whether fixing internal/ldap tests inside a CI unit warranted amending
    #23's DoD. Ruling adopted: no — these are remedial changes required to make one of
    #23's existing mandatory commands executable and trustworthy, not a new product
    requirement. It also ruled against splitting them into a predecessor PR, which "would
    create the perverse sequence where code needed to make the mandatory race gate truthful
    is merged before that gate exists."


    Update — three formal review passes, a post-cap consultation, and a re-certification round

    Conversation (one, throughout): https://chatgpt.com/c/6a933fd2-2388-83eb-8c55-6b22979c4e99

    Seven accepted findings across passes 1-3, three more from the post-cap consultation, and
    three more from the re-certification round. Plus three defects I found myself while
    verifying those, and one finding the loop's own verification pass rejected that I fixed
    anyway. Recorded because several are the kind of thing that would otherwise be
    re-discovered the hard way.

    The findings that mattered most

    Node 20 removal would have broken this gate about three weeks after it landed
    (pass 1).
    The original pins actions/checkout@v4.4.0 and actions/setup-go@v5.6.0
    both declare using: node20 in their action.yml. Node 20 leaves the runner images on
    2026-09-23; those actions run today only because the runners re-execute node20
    actions on Node 24 as a compatibility shim. A brand-new mandatory gate was depending on
    a shim about to be withdrawn. Now checkout@v7.0.1 / setup-go@v7.0.0, both node24,
    SHAs verified against the API.

    A base-branch retarget could carry a stale green check onto main (pass 3). A bare
    pull_request: trigger defaults to opened/synchronize/reopened. A retarget arrives
    as edited, keeps the same head SHA, and the check-run is recorded against that head SHA
    — so a PR proven green against another base could be retargeted onto protected main
    carrying its green Required PR gate, with the new merge result never tested. Fixed with
    types: [opened, synchronize, reopened, edited]; the contract test now requires that
    exact list, inverting its original "no types configuration at all" invariant. Note the
    pre-PR internal reviewer had examined this scenario and called it fail-closed — it
    reasoned about retargeting away from main and missed that the dangerous direction is
    onto it.

    Loose required checks would have allowed an untested merge result on every base
    advancement (post-cap consultation).
    See decision 3 above; this is why strict: true.

    Job-level permissions: was unchecked (pass 2). A job-level block replaces the
    workflow-level grant, so jobs.required-pr-gate.permissions: contents: write would have
    passed every existing test while handing PR-authored code write authority.

    Defects introduced by this PR's own review cycle, then removed

    Worth recording as a pattern: two of the fixes were themselves defective, and both were
    regressions against code that was already correct on origin/main.

    • t.Errorf does not halt. Asked to close a fail-open in startServing's teardown, I
      added t.Errorf to the two empty timeout branches and described that as the fix. It was
      not: Errorf is Logf+Fail, so control still fell through to the unsynchronized
      srv.Stop(). The hazard was labelled, not removed.
    • The failSafeShutdownTimeout machinery should never have existed. It was introduced
      during this PR. On timeout it reported and returned, releasing the next test while the
      previous server was still live — trading a bounded hang for silent cross-test
      contamination, which is precisely how a wedged Serve becomes an inexplicable
      ldapserver.Logger race in an unrelated test later in the run. Teardown is now
      ln.Close(); <-serveErr; srv.Stop(); cancel(), unbounded — byte-for-byte what
      startTestServer used unchanged throughout the Implement LDAP OAuth helper for ephemeral ClickHouse users and dynamic roles #19 series and what origin/main still
      has. Net −180 lines, including three tests whose only subject was the removed
      branches. A teardown bug must fail-stop the process; go test's own timeout panics with
      a full goroutine dump, better diagnostics than an Errorf, well inside the gate's
      15-minute job bound.

    Lesson for the next unit touching internal/ldap test scaffolding: the existing
    unbounded ln.Close(); <-serveErr; srv.Stop() ordering is load-bearing and already
    correct. Do not "harden" it with timeouts.

    The node24 floor check: what it does and does not prove

    TestPRGateContract_ActionPinVersionCommentsAreAtOrAboveTheNode24Floor (renamed from
    ...ActionPinsAreAtOrAbove... after review pointed out the original name promised more
    than it delivers) verifies the trailing # <identity>@vX.Y.Z comment: that it names
    the step's own action, and a version at or above that action's node24 floor. Its
    diagnostic says "version comment claims v…", not "pinned to v…", so a bare CI log reads
    honestly.

    It cannot bind a SHA to a version or a declared runtime — that needs network access a
    unit test must not have. So a pin whose SHA was regressed to a node20 release but whose
    comment was updated to a plausible above-floor version for the same action still passes.
    That gap is accepted deliberately: an approved identity→SHA→runtime table would make the
    binding real but converts every legitimate action upgrade into unrelated churn and
    re-couples to exact SHAs, which is what pinning-for-immutability exists to avoid.
    Immutability itself is proven completely and independently by
    TestPRGateContract_ExternalActionsArePinnedToFullCommitSHA. Treat the floor check as
    version-comment hygiene, not a runtime oracle.

    Floors, verified by fetching each action.yml at the tag boundary (both were wrong
    initially, in the same way, and were corrected one pass apart):

    Action node20 first node24 floor
    actions/checkout v4.3.1 v5.0.0 v5.0.0
    actions/setup-go v6.1.0 v6.2.0 v6.2.0

    setup-go's node24 line starts mid-v6, not at the major bump — the initial comment
    claimed "setup-go v6.x is still node20", which was false.

    Issue body corrected (two factual overclaims)

    Both fixed directly in #23's body, each marked as corrected during implementation, since
    #23 is the prerequisite contract #33 will rely on:

    • lib-tests.sh "need only bash/coreutils and do not require Docker" → it is
      Docker-daemon-free but not CLI-free: lib/common.sh runs detect_compose_cmd at
      source time and dies. Reproduced with a reduced PATH.
    • go test -race ./... described as "the complete normal Go test suite" → it is the
      complete suite of the root module. third_party/goldap and
      third_party/ldapserver are separate modules pulled in only via replace, so this
      command never compiles their test suites — and that is where the parser Implement ADR #32: replace general LDAP stack with ClickHouse compatibility profile #33 replaces
      currently lives. Parser tests intended to be gated must live inside the root module, or
      a sixth command must be added.

    Complete sabotage matrix (all coordinator-run; workflow restored byte-identical each time)

    Mutation Caught by
    job renamed Required PR Gate ExactlyOneJobNamedForBranchProtection
    paths: filter added under pull_request TriggersAreUnfilteredPRAndMainPush + NoPathFiltersAnywhere
    continue-on-error: true on the security step JobIsUnconditionalAndFailsClosed
    if: false on the job JobIsUnconditionalAndFailsClosed
    top-level permissions: contents: write TopLevelPermissionsAreReadOnly
    job-level permissions: contents: write TopLevelPermissionsAreReadOnly
    actions/checkout@v4 (unpinned) ExternalActionsArePinnedToFullCommitSHA
    version comment identity swapped to the other action ActionPinVersionComments…Node24Floor
    setup-go comment set to v6.1.0 same, naming v6.2.0
    edited dropped from types TriggersAreUnfilteredPRAndMainPush
    write-deadline test with WriteTimeout = 60s the test itself fails as required
    singleConnListener Close()→Accept() 5007/10000 before the fix, 0/10000 after

    Hosted CI — every run keyed to its exact head

    Run Head Result
    33276319268 c8538af ❌ step 6 — found two real defects on the gate's first run
    33277452419 ff25f23 ✅ all five steps, 1m42s
    33277581488 e37e24d ❌ step 7 only — deliberate phase5release red proof (steps 4-6 passed)
    33277682239 a11eac0 ✅ restored green
    33278654950 30c4030 ✅ first run on the node24 pins
    33279680085 e82c749 ✅
    33280538211 22b9aad ✅
    33281026583 67e1fb7 ✅

    Still outstanding

    Merge commit → push-event run on the resulting main SHA → classic protection per the
    table in decision 3 → four-field read-back → only then is #33's three-part unblock
    condition satisfied.


    Complete hosted-run ledger

    Every Required PR gate run on this PR, each keyed to its exact head SHA. This lives here
    rather than in the PR description because editing an issue comment does not retrigger the
    gate, so unlike the PR body this table can actually have a fixed point.

    # Head Run Result
    1 c8538af 33276319268 ❌ step 6 Race tests — found two real defects on the gate's first ever run
    2 ff25f23 33277452419 ✅ all five steps, 1m42s
    3 e37e24d 33277581488 ❌ step 7 only (4-6 passed) — deliberate phase5release red proof, DoD item 11
    4 a11eac0 33277682239 ✅ restored green after the normal revert
    5 30c4030 33278654950 ✅ first run on the node24 pins
    6 e82c749 33279680085 ✅
    7 22b9aad 33280538211 ✅ first run with types: [... edited]
    8 67e1fb7 33281026583 ✅
    9 56b2532 33281621345 ✅
    10 8a0dc6a 33282032417 ✅ (push)
    11 8a0dc6a 33282035937 ✅ (PR-body edit — live proof the edited trigger fires)
    12 7006c60 33282478996 ✅
    13 7006c60 33282562218 ✅ (body edit)
    14 84af026 33282886993 ✅
    15 84af026 33282968379 ✅ (body edit)
    16 443449f 33283175836 ✅ all five steps
    17 443449f 33283309545 ✅ (body edit)

    Rows 13 and 15 were wrong in the first version of this table: run 33282968379 was
    attributed to head 7006c60 when its real head_sha is 84af026, and the actual second
    run at 7006c60 (33282562218) was missing. Corrected after review caught it by
    checking each run's head_sha against the API rather than trusting the table — the same
    discipline every CI wait in this unit used, applied belatedly to the ledger itself.

    Two independent runs at both 8a0dc6a and 7006c60 are not duplicates: the second of each
    pair was started by a PR-description edit. That is the types: [opened, synchronize, reopened, edited] addition working in production — the same mechanism that closes the
    stale-green base-retarget path — and the accepted cost of it is that editing a PR
    description now costs a gate run.

    Review rounds and what each found

    Round Findings Severity of the worst
    Code review pass 1 2 accepted, 1 rejected Major — both pinned actions declared node20, due for removal 2026-09-23
    Pass 2 3 accepted Major — t.Errorf does not halt, so the "closed" fail-open was still open
    Pass 3 2 accepted, 1 rejected Major — bare pull_request: omits edited; stale-green base retarget
    Post-cap consultation 1 4 Major — strict: false allows an untested merge on every base advancement
    Re-cert pass 1 3 accepted Major — the strict: true correction was never actually written to the ship log
    Re-cert pass 2 2 accepted, 0 rejected Minor, non-blocking (both cosmetic)
    Re-cert pass 3 1 accepted, 1 rejected Minor (a stale table row); rejected nit fixed anyway
    Post-cap consultation 2 4, no code bypass Major — issue's normative snippet still showed the unsafe bare trigger
    Post-cap consultation 3 3, no code bypass Medium — GITHUB_TOKEN claim too broad; two sibling diagnostics still fail-open-wrong

    The recurring defect class in this unit was never a broken gate — it was records asserting
    things that were not true.
    A version floor justified by the wrong first-node24 release; a
    scope limit the test's own name contradicted; a rollout table specifying a setting nobody
    had recorded; a diagnostic claiming path filters let PRs merge green when they actually
    deadlock them; a GITHUB_TOKEN rule stated without its exceptions. Every one was caught by
    review rather than by a test, because none of them is machine-checkable — which is the
    honest limit of pr_gate_contract_test.go and the reason its own doc comment says so.

  4. added 17 commits that reference this issue on Aug 29, 2026
  5. BorisTyshkevich commented on Aug 30, 2026

    @BorisTyshkevich
    CollaboratorAuthor

    All 13 Definition-of-Done items are satisfied and main is protected with Required PR gate required (strict: true, enforce_admins: true), read back fresh from the API. Delivered by #34 (e9c7c3c); a push-event run succeeded on that main SHA. Follow-ups: #35, #36. #33's precondition is satisfied and recorded there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions

      0