Repository navigation
build(license): Apache-2.0 compliance - no GPL/LGPL code in the connector JARs, THIRD_PARTY_NOTICES.md and license texts - #1560
Merged
Conversation
…ctor JARs, THIRD_PARTY_NOTICES.md, license texts The 2.11.0 lightweight JAR shaded in code under licenses that may not be bundled into an Apache-2.0 work (ASF Category X) or that do not belong in a production JAR: - MySQL Connector/J 9.1.0 (GPL-2.0 + Universal FOSS Exception): 1,127 entries - MariaDB Connector/J 2.7.12 (LGPL-2.1): 283 entries - JUnit 4.10 (CPL-1.0, via json-simple's mis-scoped dependency): 242 entries The org.honton.chas license-maven-plugin 0.0.6 "compliance" goal ran in every build and never failed it. Java - mysql-connector-j is "provided": on the compile/test class path, not shaded. Debezium's MySQL connector links directly against com.mysql.cj.jdbc.Driver, com.mysql.cj.CharsetMapping and com.mysql.cj.protocol.a.NativeConstants, so no other driver can replace it; it is supplied at run time instead. The JAR manifest carries Class-Path: mysql-connector-j.jar lib/mysql-connector-j.jar, and mvn package copies the unmodified driver and its sources JAR to target/mysql-driver/. -Pbundle-mysql-driver restores the old shading for builds that are never redistributed. - MySqlJdbcDriver.check refuses a MySQL start without the driver and names the three ways to supply it; PostgreSQL is never affected. - ConnectionTimeZonePreflight calls Connector/J's own zone resolver by reflection: no project source links against GPL classes. - mariadb-java-client -> test scope (unused by src/main); junit excluded from json-simple; unused org.glassfish:javax.json removed. - maven-enforcer bannedDependencies replaces the no-op plugin: the build fails if Connector/J, the MariaDB driver or JUnit 4 reaches compile/runtime scope. Python (ch-sink-tools) - psycopg2-binary (LGPL-3.0) -> pg8000 (BSD-3-Clause, pinned >=1.30,<2). Dict rows from cursor.descri 8000 ption; explicit cursor close; the 20 s timeout bounds only the connect (pg8000 would keep it on every socket read, so it is cleared after connecting); statement_timeout=0 still sent; the checksum snapshot uses SET SESSION CHARACTERISTICS ... REPEATABLE READ, READ ONLY before the implicit BEGIN and now verifies READ ONLY too. - mysql-connector-python (GPL-2.0) -> PyMySQL (MIT) in sink-connector/tests and generate_15k_tables.py. Attribution and packaging (per the 2.10.3 third-party license review) - THIRD_PARTY_NOTICES.md for both JARs, the Go client, the Docker bundles and ch-sink-tools, keeping the review's license elections. The Maven table is generated from mvn dependency:tree -Dscope=runtime. - licenses/: GPL-2.0 + UFE, MPL-2.0, EPL-2.0, CDDL-1.0, CDDL-1.1 texts. - Both JARs carry THIRD_PARTY_NOTICES.md and licenses/ under META-INF/. - Lightweight images ship the driver as a separate file next to /app.jar, plus /licenses/ with the notices, the license texts and the driver's complete corresponding source. - The release JAR zip includes the notices and licenses/; the notices are also attached as a release asset. - Kafka images no longer download io.confluent:kafka-schema-registry (the registry server, Confluent Community License): the Avro converter needs only kafka-avro-serializer and kafka-schema-registry-client. - sink-connector/deploy/libs/mysql-connector-java-8.0.27.jar (GPL) removed: the Kafka sink never connects to MySQL. - e2e podman scripts mount the driver next to /app.jar; doc/licensing.md documents the policy, the gate and how to supply the driver. Checks: publish.yml parses with a duplicate-key-detecting YAML loader; the edited shell scripts pass bash -n; postgres_dumper.py passes the Python unit suite (810 passed); README text only. Jinja-Render-Check: rendered=6; formats=yaml,shell,text; result=pass
…pin pg8000 to the verified 1.31 line Spec-first gate: the license change touched spec-governed sources (MySqlJdbcDriver, DebeziumChangeEventCapture, ConnectionTimeZonePreflight) without a spec update. - Spec 01.01: new section 3.2.1 (the driver is provided-scope, supplied at run time via the manifest Class-Path; MySqlJdbcDriver.check runs first and refuses a MySQL start without it; non-MySQL connectors untouched), Codebase Mapping entry, start-sequence step, FM-01.01-2 trigger and test, and the MySqlJdbcDriverTest verification citations. - Spec 07.03: the session-zone preflight resolves with the driver's own TimeUtil.getCanonicalTimeZone, now invoked by reflection through MySqlJdbcDriver.canonicalTimeZone. - pg8000 pinned to >=1.31,<1.32: the connect-timeout fix reads pg8000's private _usock, verified on 1.31.5; it still fails loudly (with a test) if the attribute is ever missing.
…TESTYLE, PGGEQO, PGSSLMODE) python-toolset-e2e-postgres failed on test_zone_less_timestamp_keeps_its_wall_clock_in_a_non_utc_zone (D-13.05-8): the dumper detected the server zone as UTC instead of America/Chicago and created DateTime64(6, 'UTC'). Root cause: psycopg2 is libpq-based and libpq sends PGTZ (and PGDATESTYLE, PGGEQO) from the environment as session parameters; pg8000 is pure Python and reads none of them, so the session kept the server default. get_postgres_connection now passes those variables as pg8000 startup parameters (TimeZone, DateStyle, geqo) and maps PGSSLMODE to pg8000's ssl_context (prefer/allow -> None, which is pg8000's TLS-if-offered default and equals libpq's default; disable -> False; require -> True; verify-ca / verify-full -> a verifying context using PGSSLROOTCERT). PGCLIENTENCODING is not mapped: pg8000 decodes UTF-8 only. Verified: - the PostgreSQL e2e suite against a real PostgreSQL 16 + ClickHouse 24.8 (external-stack mode): before the fix 1 failed, 30 passed, 2 xfailed (the failure above); after it 31 passed, 2 xfailed. - python unit suite (db_compare, db_dump, db_load, tests): 818 passed. - new unit tests for the environment and PGSSLMODE mapping.
Collaborator
Author
|
CI is fully green on 5487049 (15 of 15 checks). This covers the parts the PR description listed as not runnable locally:
Whole-Change-Verification: ran=full CI pipeline on 5487049 (15 checks); result=pass |
This was referenced Oct 10, 2026
minguyen9988
added a commit
that referenced
this pull request
Oct 10, 2026
com.google.protobuf:protobuf-java:3.19.6 (BSD-3-Clause) is a compile-scope dependency of io.debezium:debezium-connector-postgres 3.1.3.Final. sink-connector-lightweight/pom.xml neither excludes it nor filters it out of the shaded JAR, so it ships in the lightweight connector JAR. The 2.10.3 notice in #1561 lists it; the notice regenerated for 2.11.0 in #1560 does not. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
minguyen9988
added a commit
that referenced
this pull request
Oct 10, 2026
* Add THIRD_PARTY_NOTICES.md * THIRD_PARTY_NOTICES.md: align with the notice regenerated for 2.11.0 (#1560) This branch and #1560 both add THIRD_PARTY_NOTICES.md, so the PR cannot merge into 2.11.0 (add/add conflict). Take the 2.11.0 version as the base of this PR; the next commit adds the one runtime component this branch listed that the 2.11.0 notice lacks. * THIRD_PARTY_NOTICES.md: align with 2.11.0 again after #1564 #1564 added the protobuf-java row to the 2.11.0 notice, which re-created the add/add conflict with this branch. The file here is now identical to the 2.11.0 version, so the PR merges cleanly. * ci: re-run checks after #1568 The Review gates job failed on this branch because the workflow ran the check script from the PR head, which predates it; #1568 makes it use the base branch copy. No content change. * ci: re-run checks after #1569 Lean proofs and spec validator failed here because the job checked out this branch's head, which predates formal_specs/; #1569 makes it validate the merge commit with 2.11.0. No content change. --------- Co-authored-by: minguyen9988 <136291932+minguyen9988@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Brings the 2.11.0 artifacts in line with the Apache License 2.0, applying the release-packaging requirements of the third-party license review of the 2.10.3 artifacts.
The 2.11.0 lightweight JAR shaded in code that may not be bundled into an Apache-2.0 work (ASF Category X) or that does not belong in a production JAR: MySQL Connector/J 9.1.0 (GPL-2.0 + Universal FOSS Exception, 1,127 entries), MariaDB Connector/J 2.7.12 (LGPL-2.1, 283 entries) and JUnit 4.10 (CPL-1.0, 242 entries, via json-simple). The org.honton.chas license-maven-plugin 0.0.6 compliance goal ran in every build and never failed it.
Java
Python (ch-sink-tools)
Attribution and packaging
Operational impact
A MySQL pipeline started from a JAR built from this branch needs mysql-connector-j.jar next to the JAR, or the JAR must be built with the bundle-mysql-driver profile. Without either, it refuses to start with that instruction. Docker images include the driver, so their entrypoints are unchanged. This PR does not touch versioning, de-duplication, watermarking or offset commit timing.
Verification
Whole-Change-Verification: ran=mvn offline clean package of both modules with the enforcer + lightweight unit tests (720 run, 0 failures, 2 environment-only errors) + JAR class-path and refusal checks + pytest db_compare/tests db_dump/tests db_load/tests tests (810 passed); IT, e2e and live-PostgreSQL runs not runnable in the sandbox; result=not-runnable
Opened on behalf of the team that reviewed the 2.10.3 licensing. Maintainers own the review and the merge.