Repository navigation
Test update-compatibility without building or starting Quarkus to save little time - #48662
Conversation
This should save roughly 100 seconds as we don't need Quarkus build to verify this command. * Closes: keycloak#48661 Signed-off-by: Michal Vavřík <michal.vavrik@aol.com>
731dd13 to
9ae9930
Compare
|
I am in favor of these types of conversions, but wanted to add some caveats more thoughts / context. For this particular test class, while the command is capable of auto-building it's not expected by the tests, and the current logic does not actually use the augmentation / server start. Should that ever change new tests will need to be added to elsewhere. The half measure had been to add the DryRun annotation - that would save the bulk of test time here. As an picocli test it's still significantly faster as we're avoiding setting up the distribution and using process calls. Part of the rationale for DryRun if I recall correctly is that there could be an integration test run that disables the affect of DryRun in case we ever wanted the "full coverage" back. However that has not come up since, so I doubt it will ever be needed especially in light of us needing to manage the test run times yet again. The Quarkus UT jobs do run against both linux and windows, so we 8000 do have basic multi-platform coverage there. |
On current main branch running on my workstation the diff --git a/quarkus/tests/integration/src/test/java/org/keycloak/it/cli/dist/UpdateCommandDistTest.java b/quarkus/tests/integration/src/test/java/org/keycloak/it/cli/dist/UpdateCommandDistTest.java
index 37cca7707a..f1155a0274 100644
--- a/quarkus/tests/integration/src/test/java/org/keycloak/it/cli/dist/UpdateCommandDistTest.java
+++ b/quarkus/tests/integration/src/test/java/org/keycloak/it/cli/dist/UpdateCommandDistTest.java
@@ -29,6 +29,7 @@ import org.keycloak.compatibility.KeycloakCompatibilityMetadataProvider;
import org.keycloak.config.DatabaseOptions;
import org.keycloak.it.junit5.extension.CLIResult;
import org.keycloak.it.junit5.extension.DistributionTest;
+import org.keycloak.it.junit5.extension.DryRun;
import org.keycloak.it.junit5.extension.RawDistOnly;
import org.keycloak.it.utils.KeycloakDistribution;
import org.keycloak.jgroups.certificates.DefaultJGroupsCertificateProviderFactory;
@@ -61,24 +62,28 @@ import static org.junit.jupiter.api.Assertions.assertTrue;
@RawDistOnly(reason = "Requires creating JSON file to be available between containers")
public class UpdateCommandDistTest {
+ @DryRun
@Test
@Launch({UpdateCompatibility.NAME})
public void testMissingSubCommand(CLIResult cliResult) {
cliResult.assertError("Missing required subcommand");
}
+ @DryRun
@Test
@Launch({UpdateCompatibility.NAME, UpdateCompatibilityMetadata.NAME})
public void testMissingOptionOnSave(CLIResult cliResult) {
cliResult.assertNoMessage("Missing required argument");
}
+ @DryRun
@Test
@Launch({UpdateCompatibility.NAME, UpdateCompatibilityCheck.NAME})
public void testMissingOptionOnCheck(CLIResult cliResult) {
cliResult.assertError("Missing required argument: " + UpdateCompatibilityCheck.INPUT_OPTION_NAME);
}
+ @DryRun
@Test
public void testCompatible(KeycloakDistribution distribution) throws IOException {
var jsonFile = createTempFile("compatible", ".json");
@@ -106,6 +111,7 @@ public class UpdateCommandDistTest {
result.assertNoError("Rolling Update is not available.");
}
+ @DryRun
@Test
public void testWrongVersions(KeycloakDistribution distribution) throws IOException {
var jsonFile = createTempFile("wrong-versions", ".json");
@@ -145,6 +151,7 @@ public class UpdateCommandDistTest {
result.assertError("[%s] Rolling Update is not available. '%s.jgroupsVersion' is incompatible: 0.0.0.Final -> %s.".formatted(CacheEmbeddedConfigProviderSpi.SPI_NAME, CacheEmbeddedConfigProviderSpi.SPI_NAME, jgroupsVersion));
}
+ @DryRun
@Test
public void testCacheLocalChange(KeycloakDistribution distribution) throws IOException {
var jsonFile = createTempFile("compatible", ".json");
@@ -160,6 +167,7 @@ public class UpdateCommandDistTest {
result.assertError("[%s] Rolling Update is not available. '%s.enabled' is incompatible: null -> true.".formatted(JGroupsCertificateProviderSpi.SPI_NAME, JGroupsCertificateProviderSpi.SPI_NAME));
}
+ @DryRun
@Test
public void testChangeCacheRemoteToEmbedded(KeycloakDistribution distribution) throws IOException {
var jsonFile = createTempFile("compatible", ".json");
@@ -183,6 +191,7 @@ public class UpdateCommandDistTest {
result.assertError("[%1$s] Rolling Update is not available. '%1$s.version' is incompatible: null".formatted(CacheEmbeddedConfigProviderSpi.SPI_NAME));
}
+ @DryRun
@Test
public void testRollingUpdatePatchCompatibility(KeycloakDistribution distribution) throws IOException {
var jsonFile = createTempFile("patch-compatible", ".json");
@@ -203,6 +212,7 @@ public class UpdateCommandDistTest {
result.assertMessage("[OK] Rolling Update is available.");
}
+ @DryRun
@Test
public void testChangeCacheEmbeddedToRemote(KeycloakDistribution distribution) throws IOException {
var jsonFile = createTempFile("compatible", ".json");
@@ -221,6 +231,7 @@ public class UpdateCommandDistTest {
result.assertError("[%1$s] Rolling Update is not available. '%1$s.version' is incompatible:".formatted(CacheEmbeddedConfigProviderSpi.SPI_NAME));
}
+ @DryRun
@Test
public void testChangeCacheStack(KeycloakDistribution distribution) throws IOException {
var jsonFile = createTempFile("compatible", ".json");
@@ -237,6 +248,7 @@ public class UpdateCommandDistTest {
result.assertError("[%1$s] Rolling Update is not available. '%1$s.stack' is incompatible: null -> jdbc-ping-udp".formatted(CacheEmbeddedConfigProviderSpi.SPI_NAME));
}
+ @DryRun
@Test
public void testDatabaseTypeChanged(KeycloakDistribution distribution) throws IOException {
var jsonFile = createTempFile("compatible", ".json");
@@ -253,6 +265,7 @@ public class UpdateCommandDistTest {
result.assertError("[%1$s] Rolling Update is not available. '%1$s.db' is incompatible: dev-file -> postgres".formatted(DatabaseCompatibilityMetadataProvider.ID));
}
+ @DryRun
@Test
public void testDatabaseUrlChanged(KeycloakDistribution distribution) throws IOException {
var jsonFile = createTempFile("compatible", ".json");
@@ -270,6 +283,7 @@ public class UpdateCommandDistTest {
result.assertExitCode(CompatibilityResult.ExitCode.ROLLING.value());
}
+ @DryRun
@Test
public void testDatabaseUrlOptions(KeycloakDistribution distribution) throws IOException {
var jsonFile = createTempFile("compatible", ".json");it turns to be 25.709 s. So it is better, but still worse than 6.508 s for Regarding |
Note that the annotation can be applied at the class level as well.
That's not what I'm saying. I'm just adding more context about some of the thinking behind DryRun - and also saying why that may no longer be valid.
Yes, we are saying the same thing.
For failures that happen pre server start, there may not be much difference. At best DryRun is preventing a reaugmentation from happening in these scenarios - but that may not even be happening as it depends on the test ordering and whether any build-time options are being used. |
|
@shawkins sure, I understood that you provide a context and I think it was helpful 👍
Thanks for the context. And I also remembered that when I tried to measure it, then these scenarios where I was adding |
There was a problem hiding this comment.
LGTM, thanks @michalvavrik
|
@pruivo Do you want to review this too since you contributed to this test? |
There was a problem hiding this comment.
LGTM. Thank you, @michalvavrik .
…oak#48662) This should save roughly 100 seconds as we don't need Quarkus build to verify this command. * Closes: keycloak#48661 Signed-off-by: Michal Vavřík <michal.vavrik@aol.com>
Uh oh!
There was an error while loading. Please reload this page.