8000 [Old Arquillian testsuite] Migrating Assert JUnit4 to Assertions JUnit5 by lhanusov · Pull Request #47893 · keycloak/keycloak · GitHub
[go: up one dir, main page]
Skip to content

[Old Arquillian testsuite] Migrating Assert JUnit4 to Assertions JUnit5 - #47893

Merged
stianst merged 1 commit into
keycloak:mainfrom
lhanusov:migration/junit4_to_junit5
Apr 21, 2026
Merged

[Old Arquillian testsuite] Migrating Assert JUnit4 to Assertions JUnit5#47893
stianst merged 1 commit into
keycloak:mainfrom
lhanusov:migration/junit4_to_junit5

Conversation

@lhanusov
@lhanusov lhanusov commented Apr 9, 2026 •
edited
Loading
Copy link
Copy Markdown
Contributor

Closes #47900

@lhanusov
lhanusov force-pushed the migration/junit4_to_junit5 branch 5 times, most recently from d88abee to 712ec8a Compare April 9, 2026 11:15
@lhanusov
lhanusov marked this pull request as ready for review April 9, 2026 11:15
@lhanusov
lhanusov requested review from a team as code owners April 9, 2026 11:15
@lhanusov lhanusov self-assigned this Apr 9, 2026
@lhanusov lhanusov added the area/test-framework Test framework — infrastructure, Realm DSL, page objects, migration utils label Apr 9, 2026
This was referenced Apr 9, 2026
Closed
Closed
keycloak-github-bot[bot]
keycloak-github-bot Bot reviewed Apr 9, 2026
View reviewed changes
@keycloak-github-bot keycloak-github-bot Bot left a comment
8000 Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
keycloak-github-bot Bot commented Apr 9, 2026
Copy link
Copy Markdown

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.model.session.AuthenticationSessionTest#testConcurrentAuthenticationSessionsRemoval

Keycloak CI - Store Model Tests

java.lang.AssertionError: 
threads didn't terminate in time: [main (RUNNABLE):
	at java.management@25.0.2/sun.management.ThreadImpl.dumpThreads0(Native Method)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:505)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:493)
...

Report flaky test

org.keycloak.testsuite.model.session.AuthenticationSessionTest#testConcurrentAuthenticationSessionsRemoval

Keycloak CI - Store Model Tests

java.lang.AssertionError: 
threads didn't terminate in time: [main (RUNNABLE):
	at java.management@25.0.2/sun.management.ThreadImpl.dumpThreads0(Native Method)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:505)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:493)
...

Report flaky test

org.keycloak.testsuite.model.session.UserSessionInitializerTest#testUserSessionPropagationBetweenSites

Keycloak CI - Store Model Tests

java.lang.AssertionError: 
threads didn't terminate in time: [main (RUNNABLE):
	at java.management@25.0.2/sun.management.ThreadImpl.dumpThreads0(Native Method)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:505)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:493)
...

Report flaky test

@lhanusov
lhanusov force-pushed the migration/junit4_to_junit5 branch from 712ec8a to 4d37712 Compare April 9, 2026 12:52
@keycloak-github-bot keycloak-github-bot Bot mentioned this pull request Apr 9, 2026
Closed
keycloak-github-bot[bot]
keycloak-github-bot Bot reviewed Apr 9, 2026
View reviewed changes
@keycloak-github-bot keycloak-github-bot Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
keycloak-github-bot Bot commented Apr 9, 2026
Copy link
Copy Markdown

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.model.singleUseObject.SingleUseObjectModelTest#testCluster

Keycloak CI - Store Model Tests

java.lang.AssertionError: 
threads didn't terminate in time: [main (RUNNABLE):
	at java.management@25.0.2/sun.management.ThreadImpl.dumpThreads0(Native Method)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:505)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:493)
...

Report flaky test

org.keycloak.testsuite.model.user.UserModelTest#testAddRemoveUserConcurrent

Keycloak CI - Store Model Tests

org.opentest4j.MultipleFailuresError: 
Multiple Failures (2 failures)
	java.lang.NullPointerException: Cannot invoke "org.keycloak.models.KeycloakSessionFactory.create()" because "factory" is null
	org.infinispan.commons.IllegalLifecycleStateException: Cache container has been stopped and cannot be reused. Recreate the cache container.
	Suppressed: java.lang.NullPointerException: Cannot invoke "org.keycloak.models.KeycloakSessionFactory.create()" because "factory" is null
...

Report flaky test

org.keycloak.testsuite.model.user.UserModelTest#testAddRemoveUser

Keycloak CI - Store Model Tests

org.opentest4j.MultipleFailuresError: 
Multiple Failures (2 failures)
	java.lang.NullPointerException: Cannot invoke "org.keycloak.models.KeycloakSessionFactory.create()" because "factory"
8000
 is null
	java.lang.NullPointerException: Null keys are not supported!
	Suppressed: java.lang.NullPointerException: Cannot invoke "org.keycloak.models.KeycloakSessionFactory.create()" because "factory" is null
...

Report flaky test

@lhanusov
lhanusov force-pushed the migration/junit4_to_junit5 branch from 4d37712 to 8aed119 Compare April 10, 2026 07:25
keycloak-github-bot[bot]
keycloak-github-bot Bot reviewed Apr 10, 2026
View reviewed changes
@keycloak-github-bot keycloak-github-bot Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
keycloak-github-bot Bot commented Apr 10, 2026
Copy link
Copy Markdown

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.model.session.UserSessionProviderOfflineModelTest#testOfflineSessionLazyLoading

Keycloak CI - Store Model Tests

java.lang.AssertionError: 
threads didn't terminate in time: [main (RUNNABLE):
	at java.management@25.0.2/sun.management.ThreadImpl.dumpThreads0(Native Method)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:505)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:493)
...

Report flaky test

org.keycloak.testsuite.model.session.UserSessionProviderOfflineModelTest#testOfflineSessionLazyLoading

Keycloak CI - Store Model Tests

java.lang.AssertionError: 
threads didn't terminate in time: [main (RUNNABLE):
	at java.management@25.0.2/sun.management.ThreadImpl.dumpThreads0(Native Method)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:505)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:493)
...

Report flaky test

@lhanusov lhanusov mentioned this pull request Apr 10, 2026
Closed
@stianst
stianst force-pushed the migration/junit4_to_junit5 branch from 8aed119 to 15c914b Compare April 10, 2026 10:05
@keycloak-github-bot
keycloak-github-bot Bot commented Apr 10, 2026
Copy link
Copy Markdown

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.model.session.UserSessionInitializerTest#testUserSessionPropagationBetweenSites

Keycloak CI - Store Model Tests

java.lang.AssertionError: 
threads didn't terminate in time: [main (RUNNABLE):
	at java.management@25.0.2/sun.management.ThreadImpl.dumpThreads0(Native Method)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:505)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:493)
...

Report flaky test

org.keycloak.testsuite.model.session.OfflineSessionPersistenceTest#testPersistenceMultipleNodesClientSessionsAtRandomNode

Keycloak CI - Store Model Tests

java.lang.AssertionError: 
threads didn't terminate in time: [main (TIMED_WAITING): locked on java.util.concurrent.FutureTask@3de90b18 waiting for 91117 ms blocked for 0 ms
	at java.base@25.0.2/jdk.internal.misc.Unsafe.park(Native Method)
	at java.base@25.0.2/java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:271)
	at java.base@25.0.2/java.util.concurrent.FutureTask.awaitDone(FutureTask.java:508)
...

Report flaky test

keycloak-github-bot[bot]
keycloak-github-bot Bot reviewed Apr 10, 2026
View reviewed changes
@keycloak-github-bot keycloak-github-bot Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@lhanusov
lhanusov force-pushed the migration/junit4_to_junit5 branch from 15c914b to 6043061 Compare April 14, 2026 09:24
This was referenced Apr 14, 2026
Closed
Closed
@keycloak-github-bot
keycloak-github-bot Bot commented Apr 14, 2026
Copy link
Copy Markdown

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.model.session.UserSessionProviderModelTest#testStreamsMarshalling

Keycloak CI - Store Model Tests

java.lang.AssertionError: 
threads didn't terminate in time: [main (RUNNABLE):
	at java.management@25.0.2/sun.management.ThreadImpl.dumpThreads0(Native Method)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:505)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:493)
...

Report flaky test

keycloak-github-bot[bot]
keycloak-github-bot Bot reviewed Apr 14, 2026
View reviewed changes
@keycloak-github-bot keycloak-github-bot Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@lhanusov
lhanusov force-pushed the migration/junit4_to_junit5 branch 2 times, most recently from b2a2d75 to 0b2c3a7 Compare April 14, 2026 15:11
@keycloak-github-bot keycloak-github-bot Bot mentioned this pull request Apr 14, 2026
Closed
keycloak-github-bot[bot]
keycloak-github-bot Bot reviewed Apr 14, 2026
View reviewed changes
@keycloak-github-bot keycloak-github-bot Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
keycloak-github-bot Bot commented Apr 14, 2026
Copy link
Copy Markdown

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.forms.RegisterTest#registerExistingUsernameForbidden

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserMissingTermsAcceptance

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserManyErrors

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserSuccessWithEmailVerificationWithResend

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#testRegisterShouldFailBeforeUserCreationWhenUserIsInContext

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserSuccessEditUsernameDisabled

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserUmlats

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserMissingOrInvalidEmail_emailAsUsername

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserNotUsernamePasswordPolicy

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserSuccess_emailAsUsername

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUpperCaseEmailAsUsername

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerWithLoginHint

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserSuccessEditUsernameEnabled

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserMissingEmail

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerPasswordPolicy

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUpperCaseEmailWithChangedEmailAsUsername

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerExistingEmailAllowed

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserMissingUsername

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserSuccessWithEmailVerification

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserSuccessTermsAcceptance

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserChangedLocaleSuccess

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserSuccessTermsAcceptanceWithRequiredActionEnabled

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerWithLoginHint_emailAsUsername

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUpperCaseEmail

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#testEmailAsUsernameWhenEditUserNameDisabled

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#customRegistrationPageFormTest

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserSuccess

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserNotContainsUsernamePasswordPolicy

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserInvalidEmail

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerExistingEmailForbidden

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerExistingUser_emailAsUsername

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserMissingPassword

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserNotEmailPasswordPolicy

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.forms.RegisterTest#registerUserInvalidPasswordConfirm

Keycloak CI - Forms IT (firefox)

java.lang.IllegalStateException: Could not start mail server smtp:localhost:3025, try to set server startup timeout > 2000 via ServerSetup.setServerStartupTimeout(timeoutInMs) or -Dgreenmail.startup.timeout
	at com.icegreen.greenmail.util.GreenMail.start(GreenMail.java:125)
	at org.keycloak.testsuite.util.GreenMailRule.before(GreenMailRule.java:56)
	at org.jboss.arquillian.junit.Arquillian$7$1.invoke(Arquillian.java:273)
	at org.jboss.arquillian.container.test.impl.execution.ClientBeforeAfterLifecycleEventExecuter.execute(ClientBeforeAfterLifecycleEventExecuter.java:88)
...

Report flaky test

org.keycloak.testsuite.model.session.UserSessionProviderModelTest#testCreateUserSessionsParallel

Keycloak CI - Store Model Tests

java.lang.AssertionError: 
threads didn't terminate in time: [main (RUNNABLE):
	at java.management@25.0.2/sun.management.ThreadImpl.dumpThreads0(Native Method)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:505)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:493)
...

java.lang.AssertionError: 
threads didn't terminate in time: [main (RUNNABLE):
	at java.management@25.0.2/sun.management.ThreadImpl.dumpThreads0(Native Method)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:505)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:493)
...

Report flaky test

@ahus1
ahus1 commented Apr 15, 2026
Copy link
Copy Markdown
Member

@lhanusov - there are now conflicts, and suspicously many flaky tests.

@lhanusov
lhanusov force-pushed the migration/junit4_to_junit5 branch from 0b2c3a7 to 80b7a8e Compare April 17, 2026 12:28
@lhanusov
lhanusov commented Apr 17, 2026
Copy link
Copy Markdown
Contributor Author

@ahus1 I've been working on it through the week, there was a mvn dependency clash of a junit platform at the end (quite complicated). Now it should be ready.

keycloak-github-bot[bot]
keycloak-github-bot Bot reviewed Apr 17, 2026
View reviewed changes
@keycloak-github-bot keycloak-github-bot Bot left a comment
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unreported flaky test detected, please review

@keycloak-github-bot
keycloak-github-bot Bot commented Apr 17, 2026
Copy link
Copy Markdown

Unreported flaky test detected

If the flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR.

org.keycloak.testsuite.model.user.UserModelTest#testAddDirtyRemoveFederationUsersInTheSameGroupConcurrent

Keycloak CI - Store Model Tests

org.opentest4j.MultipleFailuresError: 
Multiple Failures (2 failures)
	java.lang.NullPointerException: Cannot invoke "org.keycloak.models.KeycloakSessionFactory.create()" because "factory" is null
	org.infinispan.commons.IllegalLifecycleStateException: ISPN000323: realms is in 'TERMINATED' state and so it does not accept new invocations. Either restart it or recreate the cache container.
	Suppressed: java.lang.NullPointerException: Cannot invoke "org.keycloak.models.KeycloakSessionFactory.create()" because "factory" is null
...

Report flaky test

org.keycloak.testsuite.model.user.UserModelTest#testAddRemoveUserConcurrent

Keycloak CI - Store Model Tests

org.opentest4j.MultipleFailuresError: 
Multiple Failures (2 failures)
	java.lang.NullPointerException: Cannot invoke "org.keycloak.models.KeycloakSessionFactory.create()" because "factory" is null
	java.lang.NullPointerException: Null keys are not supported!
	Suppressed: java.lang.NullPointerException: Cannot invoke "org.keycloak.models.KeycloakSessionFactory.create()" because "factory" is null
...

Report flaky test

org.keycloak.testsuite.model.singleUseObject.SingleUseObjectModelTest#testCluster

Keycloak CI - Store Model Tests

java.lang.AssertionError: 
threads didn't terminate in time: [main (RUNNABLE):
	at java.management@25.0.2/sun.management.ThreadImpl.dumpThreads0(Native Method)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:505)
	at java.management@25.0.2/sun.management.ThreadImpl.dumpAllThreads(ThreadImpl.java:493)
...

Report flaky test

@lhanusov
[Old Arquillian testsuite] Migrating Assert JUnit4 to Assertions JUnit5
032dfc1 
Signed-off-by: Lukas Hanusovsky <lhanusov@redhat.com>
@lhanusov
lhanusov force-pushed the migration/junit4_to_junit5 branch from 80b7a8e to 032dfc1 Compare April 20, 2026 13:49
stianst
stianst reviewed Apr 21, 2026
View reviewed changes
Comment thread testsuite/integration-arquillian/tests/base/pom.xml
@stianst stianst Apr 21, 2026
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are all these dependencies needed? Isn't it just junit-jupiter-api that is required, and shouldn't scope be test instead of compile?

@stianst
stianst merged commit 55b70fe into keycloak:main Apr 21, 2026
92 of 96 checks passed
@stianst stianst mentioned this pull request Jun 1, 2026
Closed
sashyo pushed a commit to tide-foundation/keycloak-IGA that referenced this pull request Jul 20, 2026
@lhanusov @sashyo
[Old Arquillian testsuite] Migrating Assert JUnit4 to Assertions JUni…
e86ef5d 
…t5 (keycloak#47893)

Signed-off-by: Lukas Hanusovsky <lhanusov@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@stianst stianst stianst left review comments
+1 more reviewer
@keycloak-github-bot keycloak-github-bot[bot] keycloak-github-bot[bot] left review comments
Reviewers whose approvals may not affect merge requirements

Assignees

@lhanusov lhanusov

Labels

area/test-framework Test framework — infrastructure, Realm DSL, page objects, migration utils flaky-test team/cloud-native team/core-clients team/core-iam

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

[Old Arquillian testsuite] Migrating Assert JUnit4 to Assertions JUnit5

3 participants

@lhanusov @ahus1 @stianst
0