Repository navigation
Migrate CLI package to new test framework - #52007
Merged
Merged
Conversation
Member
- closes: Migrate cli package to the new test framework #50756
michalvavrik
force-pushed
the
in-review/50756
branch
from
August 24, 2026 21:12
6c0c797 to
5d53882
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Migrates CLI integration tests from Arquillian into the new tests/base framework.
Changes:
- Ports kcadm/kcreg tests, fixtures, and process utilities.
- Registers CLI tests in base, FIPS, and session suites.
- Removes obsolete Arquillian CLI coverage.
Reviewed changes
Copilot reviewed 38 out of 46 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
testsuite/integration-arquillian/tests/base/testsuites/volatile-sessions-suite |
Removes migrated CLI test. |
testsuite/integration-arquillian/tests/base/testsuites/fips-suite |
Removes migrated FIPS tests. |
testsuite/integration-arquillian/tests/base/testsuites/clusterless-suite |
Removes migrated CLI test. |
testsuite/integration-arquillian/tests/base/testsuites/base-suite |
Removes legacy CLI group. |
testsuite/integration-arquillian/tests/base/src/test/java/org/keycloak/testsuite/cli/AbstractCliTest.java |
Deletes legacy base class. |
testsuite/integration-arquillian/tests/base/src/main/java/org/keycloak/testsuite/cli/OsUtils.java |
Removes redundant OS utility. |
testsuite/integration-arquillian/tests/base/src/main/java/org/keycloak/testsuite/cli/OsArch.java |
Removes redundant OS model. |
testsuite/integration-arquillian/tests/base/src/main/java/org/keycloak/testsuite/cli/exec/NullInputStream.java |
Removes obsolete stream helper. |
testsuite/integration-arquillian/tests/base/src/main/java/org/keycloak/testsuite/cli/exec/ExecutionException.java |
Removes obsolete exception. |
tests/base/src/test/resources/org/keycloak/tests/cli/testrealm.json |
Adds CLI test realm fixture. |
tests/base/src/test/resources/org/keycloak/tests/cli/kcreg/saml-sp-metadata.xml |
Adds SAML registration fixture. |
tests/base/src/test/resources/org/keycloak/tests/cli/kcreg/reg-cli-keystore.p12 |
Adds PKCS12 registration key. |
tests/base/src/test/resources/org/keycloak/tests/cli/kcreg/reg-cli-keystore.jks |
Adds JKS registration key. |
tests/base/src/test/resources/org/keycloak/tests/cli/kcadm/admin-cli-keystore.p12 |
Adds PKCS12 admin key. |
tests/base/src/test/resources/org/keycloak/tests/cli/kcadm/admin-cli-keystore.jks |
Adds JKS admin key. |
tests/base/src/test/resources/org/keycloak/tests/cli/idp-keycloak-without-sync-mode.json |
Adds IdP creation fixture. |
tests/base/src/test/resources/org/keycloak/tests/cli/idp-keycloak-9167.json |
Adds IdP update fixture. |
tests/base/src/test/java/org/keycloak/tests/suites/VolatileSessionsTestSuite.java |
Registers CLI session coverage. |
tests/base/src/test/java/org/keycloak/tests/suites/FipsStrictTestSuite.java |
Registers strict-FIPS CLI tests. |
tests/base/src/test/java/org/keycloak/tests/suites/FipsNonStrictTestSuite.java |
Registers non-strict FIPS tests. |
tests/base/src/test/java/org/keycloak/tests/suites/ClusterlessTestSuite.java |
Registers clusterless CLI coverage. |
tests/base/src/test/java/org/keycloak/tests/suites/Base2TestSuite.java |
Includes the CLI package. |
tests/base/src/test/java/org/keycloak/tests/cli/TempFileResource.java |
Adds temporary-file cleanup helper. |
tests/base/src/test/java/org/keycloak/tests/cli/registration/KcRegUpdateTokenTest.java |
Migrates token-update tests. |
tests/base/src/test/java/org/keycloak/tests/cli/registration/KcRegUpdateTest.java |
Migrates registration updates. |
tests/base/src/test/java/org/keycloak/tests/cli/registration/KcRegTruststoreTest.java |
Migrates truststore tests. |
tests/base/src/test/java/org/keycloak/tests/cli/registration/KcRegTest.java |
Migrates core kcreg tests. |
tests/base/src/test/java/org/keycloak/tests/cli/registration/KcRegCreateTest.java |
Migrates registration creation. |
tests/base/src/test/java/org/keycloak/tests/cli/registration/KcRegConfigTest.java |
Migrates configuration tests. |
tests/base/src/test/java/org/keycloak/tests/cli/registration/AbstractRegCliTest.java |
Adapts registration fixtures. |
tests/base/src/test/java/org/keycloak/tests/cli/KcRegExec.java |
Adapts kcreg process execution. |
tests/base/src/test/java/org/keycloak/tests/cli/KcAdmExec.java |
Adapts kcadm process execution. |
tests/base/src/test/java/org/keycloak/tests/cli/exec/StreamReaderThread.java |
Moves process stream reader. |
tests/base/src/test/java/org/keycloak/tests/cli/exec/LoggingOutputStream.java |
Moves logging stream helper. |
tests/base/src/test/java/org/keycloak/tests/cli/exec/InteractiveInputStream.java |
Moves interactive input helper. |
tests/base/src/test/java/org/keycloak/tests/cli/exec/AbstractExecBuilder.java |
Moves process builder. |
tests/base/src/test/java/org/keycloak/tests/cli/exec/AbstractExec.java |
Adds framework-aware execution support. |
tests/base/src/test/java/org/keycloak/tests/cli/admin/KcAdmUpdateTest.java |
Migrates admin update tests. |
tests/base/src/test/java/org/keycloak/tests/cli/admin/KcAdmTruststoreTest.java |
Migrates admin truststore tests. |
tests/base/src/test/java/org/keycloak/tests/cli/admin/KcAdmTest.java |
Migrates core kcadm tests. |
tests/base/src/test/java/org/keycloak/tests/cli/admin/KcAdmSessionTest.java |
Migrates session tests. |
tests/base/src/test/java/org/keycloak/tests/cli/admin/KcAdmCreateTest.java |
Migrates admin creation tests. |
tests/base/src/test/java/org/keycloak/tests/cli/admin/AbstractAdmCliTest.java |
Adapts admin fixtures. |
tests/base/src/test/java/org/keycloak/tests/cli/AbstractCliTest.java |
Adds new-framework CLI base. |
tests/base/pom.xml |
Adds CLI dependencies and distributions. |
Suppressed comments (2)
tests/base/src/test/java/org/keycloak/tests/cli/admin/KcAdmTruststoreTest.java:51
- This HTTPS branch uses the migrated Arquillian truststore, but
TlsEnabledConfigleavesManagedCertificatesto generate a fresh random server certificate, so that store cannot trust the server and the-Dtest.intermittent=truepath fails with PKIX errors. Configure the test server with the matching migrated key material, or make the generated client truststore available to the CLI test.
tests/base/src/test/java/org/keycloak/tests/cli/registration/KcRegTruststoreTest.java:51 - This HTTPS branch uses the migrated Arquillian truststore, but
TlsEnabledConfigleavesManagedCertificatesto generate a fresh random server certificate, so that store cannot trust the server and the-Dtest.intermittent=truepath fails with PKIX errors. Configure the test server with the matching migrated key material, or make the generated client truststore available to the CLI test.
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+160
to
+164
| <dependency> | ||
| <groupId>org.keycloak</groupId> | ||
| <artifactId>keycloak-admin-cli</artifactId> | ||
| <scope>test</scope> | ||
| </dependency> |
Member
Author
There was a problem hiding this comment.
mvn clean install -pl tests/base -am is passing even when I delete keycloak-client-cli-dist-999.0.0-SNAPSHOT.zip from my local .m2. keycloak-client-cli-dist gets in via fw
edewit
previously approved these changes
Aug 25, 2026
shawkins
previously approved these changes
Aug 26, 2026
Contributor
There was a problem hiding this comment.
LGTM, thanks @michalvavrik
Contributor
|
Let's see if anyone else wants to weigh in from a core team given that they were tagged as reviewers, and if not we'll go ahead and merge by end of week. |
michalvavrik
force-pushed
the
in-review/50756
branch
from
August 29, 2026 15:31
5d53882 to
a07947c
Compare
Member
Author
|
I resolved merge conflict with |
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 38 out of 46 changed files in this pull request and generated 1 comment.
Suppressed comments (2)
tests/base/src/test/java/org/keycloak/tests/cli/admin/KcAdmTruststoreTest.java:52
TlsEnabledConfiggenerates a fresh random server certificate, but this HTTPS call configures the copied legacykeycloak.truststore. The truststore cannot contain that runtime-generated certificate, so the-Dtest.intermittent=truebranch will fail TLS validation; configure the server with a certificate trusted by this store or make the generated client truststore available to the CLI.
tests/base/src/test/java/org/keycloak/tests/cli/registration/KcRegTruststoreTest.java:52TlsEnabledConfiggenerates a fresh random server certificate, but this HTTPS call configures the copied legacykeycloak.truststore. The truststore cannot contain that runtime-generated certificate, so the-Dtest.intermittent=truebranch will fail TLS validation; configure the server with a certificate trusted by this store or make the generated client truststore available to the CLI.
michalvavrik
force-pushed
the
in-review/50756
branch
from
August 29, 2026 15:41
a07947c to
710d3d0
Compare
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 39 out of 47 changed files in this pull request and generated no new comments.
Suppressed comments (1)
tests/base/src/test/java/org/keycloak/tests/cli/registration/KcRegCreateTest.java:75
- This stores the initial token under the HTTPS
keycloakUrls.getBase()endpoint, but the delete and verification later in this method use the HTTPserverUrl. Consequently the HTTPS token is never deleted and the assertion only checks a different, already-empty endpoint; use the same server URL for all three operations.
closes: keycloak#50756 Signed-off-by: Michal Vavřík <dev@michalvavrik.net>
michalvavrik
force-pushed
the
in-review/50756
branch
from
August 29, 2026 16:04
710d3d0 to
bf08b19
Compare
shawkins
enabled auto-merge (squash)
September 4, 2026 11:44
shawkins
self-requested a review
September 4, 2026 11:44
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.