Repository navigation
Conversation
2 flaky tests on run #9541 ↗︎
Details:
|
|||||||||||||||||||||||||||
| Test | Artifacts | |
|---|---|---|
| Clients test > Accessibility tests for clients > Check a11y violations on load/ clients list tab |
Test Replay
Output
Screenshots
|
|
realm_settings_general_tab_test.spec.ts • 1 flaky test • chrome
| Test | Artifacts | |
|---|---|---|
| Realm settings general tab tests > Test all general tab switches |
Test Replay
Output
Screenshots
|
|
Review all test suite changes for PR #23892 ↗︎
There was a problem hiding this comment.
Unreported flaky test detected, please review
Unreported flaky test detectedIf the below flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR. org.keycloak.testsuite.ui.account2.LDAPAccountTest#createdNotVisibleTestKeycloak CI - Account Console IT (firefox) org.keycloak.testsuite.ui.account2.WelcomeScreenTest#resourcesTestKeycloak CI - Account Console IT (firefox) org.keycloak.testsuite.forms.ResetCredentialsAlternativeFlowsTest#resetCredentialsVerifyCustomOtpLabelSetProperlyKeycloak CI - Forms IT (chrome) |
d2a5920 to
057cdf6
Compare
There was a problem hiding this comment.
Unreported flaky test detected, please review
Unreported flaky test detectedIf the below flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR. org.keycloak.testsuite.x509.X509BrowserCRLTest#loginSuccessWithCRLSignedWithIntermediateCA3FromTruststoreKeycloak CI - FIPS IT (strict) |
|
The following screenshots show the new behaviour when the device-flow is disabled for the Keycloak Server: Realm Info lists device-flow as disabled feature Tokens Tab in Realm Settings doesn't show device flow settings When creating a new client Capability config doesn't show the device flow When editing a client device flow is also not shown: /realms/{realm}/device shows 404 page |
25136a7 to
5dcdf07
Compare
Unreported flaky test detectedIf the below flaky tests below are affected by the changes, please review and update the changes accordingly. Otherwise, a maintainer should report the flaky tests prior to merging the PR. org.keycloak.testsuite.ui.account2.DeviceActivityTest#currentSessionTestKeycloak CI - Account Console IT (firefox) org.keycloak.testsuite.ui.account2.DeviceActivityTest#timesTestsKeycloak CI - Account Console IT (firefox) |
There was a problem hiding this comment.
Unreported flaky test detected, please review
5dcdf07 to
d0f6908
Compare
There was a problem hiding this comment.
Changes to the front-end code look good to me. I will leave it to other team members to judge if this feature is something we want and if the Java code is adequate.
fc5ba3e to
8692de8
Compare
@thomasdarimont It uses keycloak distribution instead of the undertow distribution. Maybe you can replicate the issue when you run the test with something like: ? Just a guess (did not tried it myself). |
|
@mposolda I gave this another try, and it seems as if the When I run the tests for Could it be that the undertow based test-server implementation doesn't consider the feature flags? |
|
@mposolda the problem seems to be that the test is run with |
dc37888 to
4ceb3d6
Compare
|
I could successfully run the build via: but it failes with: I needed to configure I think we need something like: @mposolda do you have an idea how to ensure that the |
There was a problem hiding this comment.
After many hours of trying to making this test work with -Pauth-server-quarkus I gave up and just assume that we run the test with undertow. The test works fine in the IDE and with -Pauth-server-undertow.
For some reason the @DisabledFeature annotation is now applied when the test suite is executed with -Pauth-server-quarkus.
There was a problem hiding this comment.
That is fine, Thanks a lot for this investigation!
Created follow-up issue for looking at this test #24241. When thinking about this, I wonder if it can have some relation to OIDC well-known not being refreshed after feature is enabled/disabled? But not sure...
The new flag "DEVICE_FLOW" is enabled by default for now. At a later stage we might disable the feature by default. Users can now disable the device flow support for the whole server via: -Dkeycloak.profile.feature.device_flow=disabled or --features-disabled=device_flow See discussion: keycloak#23700 Fixes keycloak#23891
Signed-off-by: Thomas Darimont <thomas.darimont@googlemail.com>
Signed-off-by: Thomas Darimont <thomas.darimont@googlemail.com>
For some reason @DisableFeature is not considered when executed with -Pauth-server-quarkus. Current workaround is to ignore exceptions during -Pauth-server-quarkus.
3398c1c to
5841a1f
Compare
|
Thank you for this fix |
The new flag "DEVICE_FLOW" is enabled by default for now. At a later stage we might disable the feature by default.
Users can now disable the device flow support for the whole server via: -Dkeycloak.profile.feature.device_flow=disabled
or
--features-disabled=device_flow
See discussion: #23700
Fixes #23891