Repository navigation
Replies: 18 comments 91 replies
|
I am unsure about the timeline of this blog post: Does that mean that you plan to release Keycloak 17 in December 2021? |
|
KC 17 should be out December yes, it may slip to January though. KC 16 should have been out already, but we're having some challenges with WildFly upgrades. We hope to have it out very soon. |
|
We are eagerly awaiting for KC 16 release to try out the front channel logout feature. |
|
Taking reference from https://www.keycloak.org/2021/10/keycloak-x-update.html, As Keycloak 17 missed the timeline mentioned in Roadmap. |
|
Is it somehow possible to start Keycloak-X / the Quarkus distribution in Quarkus Dev-Mode with hot code replacement and attachable debugger? And this not only in a local distribution, but also in a Docker container? We can use such an option with the Wildfly distro when developing SPIs and providers and this is very useful and valuable. How can we achive this with Quarkus based Keycloak-X running in Docker? |
|
Yes, standard remote debugging into a KCX container also works for me, but I struggle with hot code replacement somehow. A dedicated hot deployment support for reloading newly deployed providers would be very nice, indeed! |
|
@dasniko We want/need to improve our dev story. As you know, right now we are not leveraging all capabilities from Quarkus like live reload as well as dev services. Once we have the core server bits in place, we should focus on this area. |
|
@dasniko One thing we have been discussing is to provide a BOM and potentially some tooling (e.g.: maven/gradle plugins) that allows you to easily start a project and run an embedded Keycloak. Right now, you can achieve live reload and develop custom providers if you write code within the server module. I think that is the experience we would like to achieve but without having to rely on the main code base. |
|
@pedroigor Thanks for your answer.
Great to hear 👍, I‘m really looking forward to a better Developer Experience! 🙂
Having a BOM and some tooling is nice, but most of it is convenience. 😇 For me, from a developer perspective, the killer feature is hot-/live-reload.
Running an embedded Keycloak (or even developing in the code base, which is IMHO NOT an alternative) might be not enough in some cases. I have a few customers where I do a lot customization with various custom SPI implementations. Theses also require some Keycloak configuration and some other services/infrastructure around. So, having a "real" Keycloak running in a container network with other required services for the use case, deploying the SPI extensions to the container and having hot-/live-reload is necessary for these environments. For simple use-cases it might be enough to have something like an embedded Keycloak, but I doubt that this will work for all the cases. |
|
I see. I think there are two main usages:
As each one has its own audience. I think I agree that we should initially focus on container usage as it is more aligned with the goals of Keycloak.X. That said, I think we should be able to leverage Keycloak Dev Services from Quarkus. If not fully, with some tweaks to make it aligned with our goals. If not possible to reload code in a live container we should be able to at least restart it with the updated code (hoping we do fast restarts). But yeah, I'm also expecting this a lot :) |
|
I'm sorry if this is an obvious question but is there a Docker image for Quarkus (I see on the downloads page a specific Following on from this, is there also an operator to have quarkus running in Kubernetes? Thank you for all of your work. |
|
Yes, take a look at: quay.io/keycloak/keycloak-x:15.1.0 |
|
🤦🏻 thank you! I was looking at keycloak/keycloak! |
|
Will there be something similar like the |
|
There is currently no explicit support for adding additional JVM options like |
|
Thanks, but this doesn't work for the memory settings ( |
|
@DGuhr @pedroigor WDYT? Will there be a |
|
@dasniko Sounds like a good addition to me. Do you want to create an issue for it? |
|
Sure, done: #9188 |
|
Sorry, but I can't see the awesomeness of Keycloak.X yet. I'm struggling for over 2 hours now to get it up and running with some basic config changes (bare metal, Windows). But I cannot even get to the admin console. After starting the server with the custom profile, opening https://idp.mydomain.local:8943/auth/ I get the welcome page, but the link to the admin console points to https://idp.mydomain.local/auth/admin/ |
|
I think you need to add the port number to the hostname name in the hostname parameter |
|
It shouldn't be needed to add port to the hostname, so this should be fixed if that's the case. @mduchrow Could you open a bug for this one? @pedroigor FIY |
|
The port should be either the port from the request or hardcoded to 443 (assuming TLS enabled) if using proxy. |
|
Thanks for the hints. |
|
Configuration of custom JDBC driver is broken (leads to Azure SQL cannot be used as database) In previous versions of keycloak.x it was possible to configure the JDBC driver by settings the environment variable KC_DB_DRIVER . With keycloak.x 15.1.0 this does no longer work - the database driver is now set based on the db settings to a hard coded driver class. I added a unit test to show the behaviour to ConfigurationTest.java. Has this been changed intentionally or is this a bug? |
|
Created issue #9133 and PR #9134 to fix this. |
|
@ioemat Thanks for catching! Would be nice if you could look at @pedroigor s changes in PR #9134 and see if that would work for you. Your changes seem to have some unwanted side effects :) |
|
I already commented in the PRs, the fixes from @pedroigor are fixing the issue, thanks for taking care! |
Better CLI help
vs. It is often unclear which option is a build-time option and needs a "config rebuild" and
Default context-path / relative-path settingThe Wildfly based Keycloak has been using the Perhaps Keycloak.X could ship with a Ability to restrict access to admin and management endpointsIn Wildfly based Keycloak it was possible to expose the admin console or management endpoints via dedicated ports, and Auto-build featureThe auto-build feature works quite well and should IMHO be the default for container based Keycloak deployments. Ability to add additional JVM options to docker containerAs discussed above, users might want to supply additional JVM options to the Keycloak container - as is currently possible with the Ability to run an embedded Keycloak.X within Keycloak extension moduleIt would be helpful, if we could develop and test custom extensions in an embedded quarkus runtime. The IDELauncher from the quarkus/server module already provides initial support for this, but needs some tweaking to correctly discover extension classes and themes on the classpath. This would help to speed up Keycloak theme and extension development. |
|
@thomasdarimont could you split this into multiple comments, as that would make it easier to reply and have a conversations on the separate topics? |
The By default, the
It should not really matter if you are using space or
The See
I think you have a PR for that, don't you?
Exactly ... That is the idea. For optimal runtime always prefer building the server first using a custom docker image (which btw was drastically simplified)
We should be focusing on Dev.X experience soon. |
|
Docker/Container Image - Environment variables Would be good if the Docker/Container image could support all env vars that the Wildlfy-based image supports. And if it uses the same names for them (not |
But that's the point. The container is missing an "abstraction layer" for this (as Keycloak does in general). I don't want to configure infinispan or jgroups in the first place. I want to configure clustering, like in 'I want to run a cluster with x nodes and a fault tolerance up to y nodes down.' An API for the container (in terms of env vars) should hide these details of configuring infinispan or jgroups or whatever.
@pedroigor I think it would be good to mention that in the README for the container.
I get this from technical point of view, but I don't know many users who see it this way. The wast majority of customers I work with use the container image instead of the distribution. The container is the product in their point of view. That said, they need a conveniant API for the container together with good documentation as it is availbe for the distribution. |
|
It is the embedded Infinispan server that requires clustering, not Keycloak itself. Keycloak itself is stateless. So, having a "clustering" option in Keycloak just makes no sense in regard to Keycloak.X where we are refactoring the new store, which we are also looking at making the cache layer an optional thing. For small scale deployments having just Postgres is perfectly fine, and for large multi-region deployments just CockroachDB may be just fine. Then for inbetween scenarios Postgres + an external Infinispan cache may be the best option, again even in this scenario there's no "clustering" to configure for Keycloak, rather for the state services which are Postgres and Infinispan. |
I think we are trying (perhaps not there yet) do achieve that level of abstraction by:
All that to say that to deploy in k8s, for instance, you should be able to leverage the default settings from Keycloak + ISPN and only set the I agree we can do even more to make this configuration easier. And with the new store I would also expect more improvements in this area.
+1. And that should be our focus: Container-first. This requirement is also a good justification to have the container image as simple as possible and be just a wrapper on top of the distribution. With that in mind, you share the same experience when doing standalone and containers. Anything specific to containers should be properly documented. |
As of now that is not the case. As long as there is some kind of caching involved, Keycloak has state and needs inter-node communication at least for cache invalidation. This will also be the case if the caching is optional with the new store. If I want to apply this option then I need a way to configure it. This is a design decision and not specific to infinispan. The new store is a great simplification, fully agree! But it does not remove the need for a configuration API. Actually, I did not even want to start a discussion about caching/infinispan in general. I just took that as an example for container configuration. My point was about env vars which I understand as a configuration API for the container. @pedroigor had some good points. I also think that options like
Customers in general do not have the need for the same experience with standalone and containers. This simplifies developers life, but is not very end-user/customer centric. They either use one or the other. And because there runtime environment is different in both cases, they may need a different configuration API. |
|
@sventorben Clustering/stack/etc are all requirements to be configured for the embedded Infinispan cache, there's nothing else in Keycloak that requires this configuration, so makes no sense to make that a global config thing. |
|
On "Better CLI help" Thanks for the clarification when the Regarding the On "--option=value or just --option value" It's nice that the cli parsing is that flexible but this might cause some confusion. On "--verbose" I really missed that |
|
I also think having something text based is better - then you can also "grep" for it. |
|
makes absolute sense. I'm not hard on that one. ;) |
|
I agree with the text approach as well, but please don't include runtime. Have a marker for build-time properties only, so should be: Not convinced about |
|
hum... (build) ? kinda uncreative on that one too, though. |
|
I prefer just build over build time |
|
On "Ability to restrict access to admin and management endpoints" Yes the PR for KEYCLOAK-15773 Enable admin endpoints and admin-console via Feature flags is still on my mind, but I need to follow up the discussion on that. |
|
I'm okay with just doing the profile feature flags for now. In the future we want to wrap that up as proper extensions that are enabled/disabled, but that's a future thing. |
|
On the "Default context-path / relative-path setting" As stated above, having the "old" |
|
@dasniko Yer, I have the same feeling. And anything we add now should be there for a long time and hard to remove (therefore introducing legacy to something new). |
|
There's not a chance we're putting the "/auth" thing back in there. It should never have been there, and this is just a really good opportunity to get rid of it :) Isn't this really something that is solved by a good migration guide? We want to make it as easy as possible to migrate for sure, but we are not making trade-offs for the future to make it backwards compatible, or to eliminate some minor migration issues. |
|
We have something between 200-1000 clients integrated with Keycloak instances and it is pretty safe to say adapting all these applications’ URLs is not going to happen. So having a config option to set the old path if needed is really necessary. |
|
@sschu you have it, with |
|
That’s what I wanted to hear!😉 |
|
Hi I'm aware that migrating from I tried to upgrade my It's noteworthy to mention that the DB location moved from Once this has been figured out, I tried at first to launch Finally I resolved this by explicitly setting an empty db-username and password, but I had to specify them using the environment variables Is there any default username/password for the H2 database now ? |
|
@nicph We have some defaults for database settings, including username and password. By default, username is If the database was previously created using those credentials, passing them to their corresponding option should be enough. |
|
Sorry for the confusion, but the actual default username is |
|
I also tried to export and import keycloak's data before I solved my previous issue. I had no issue exporting the data from Trying to export data from a brand new |
|
@pedroigor @DGuhr I did some testing of quarkus distribution. Maybe some points were already repeated from other people. In case you can consider the feedback from me as "another vote to improve this" :-) I focused especially on the usability and on the idea when you won't need documentation when you use
This is fine for most of java developers, but someone not familiar with Java might be a bit lost with such message and will need to google to figure what's going on. Maybe it is better to use something more user-friendly like simple message: "Your Java version is too old. Please use Java 11 or newer. See command 'java -version' to figure what is your Java version".
but I see the error related to the JDBC driver. At this point, it may not be 100% clear to me what I should do to enable JDBC driver. I personally know that it requires re-augmentation, but IMO it may not be clear enough for the average admin. Maybe some better overview of this can be mentioned in the DATABASE section of the
which does not imply to me that I need this to configure the DB. The TBH I am not sure about perfect name either, but I just consider
The output is error message (referred later as So I tried to build with dev profile and run the command: But seeing another error: This is likely only bug in the error message But seeing another error: So it seems I cannot use profile "dev" with any of
works as expected and I can run server. Then I stopped the server and I run this: but seeing error message like: which is different error message in comparison to the use of and server is successfully started. Console even shows: However this is not very great as server was in fact started with the I suppose this is because
Now I used this command to build the Then I decided to go with the other profile: and it failed to start due the expected strict hostname resolution. Hence I decided to revert to the working but seeing the error about strict hostname resolution even if I understand this is due the fact that "persisted" properties take preference over the properties from the configuration file. But not sure the order of preference is clear for typical admin (again).
This will be nice, but not sure if realistic to do and if it is too much overhead for the server to support multiple augmented profiles? Just an idea... Another idea: When I use |
Created #9215.
Created #9216.
The command above should not run becaue Perhaps this is better solved by a guide? Or perhaps we can be a bit smarter here and say: "The choosen JDBC URL does not match with the current database installed". Or something like that?
Yeah, we had that before (you added, btw :)). But we are trying to slim down bootstrapping including messages. You can still have that information by running using
As you know, previously it was But one thing that I think you are missing is the
You should some examples in CLI about how to configure auto-complete. We have one in the help for the main command. The idea behind the What do you suggest to improve that?
That should come with the guide/man pages. But profiles are not hardcoded they can be whatever you want, with any configuration you want. By default, we have two:
Perhaps a guide?
The help message states which options are available and how to use them. The format you tried is not correct hence the help message.
We should probably remove that from the description because it should be hidden and transparent to users when using
Isn't the format wrong? The
That is on purpose, you can only build for production or any profile other than There is a clear separation between what should be done for production and for dev. This should be solved by my first proposal to this particular item. Remove references to dev profile.
Yes, You have here an interesting case that I did not try before. But I'm not sure if the behavior is wrong. Although you have indirectly built the server using the dev configuration (when running But yeah, we probably solve that by having a specific marker on the persisted properties to indicate that the image was built for dev mode. @DGuhr wdyt?
I'm not sure about your proposal. The way we have it is simple enough to what we need. We can discuss it in more detail as well as discuss alternatives that could help to avoid any confusion. For instance, the Again, this "new concept" is tricky and not natural for most people. But we can find some balance for sure.
Sounds good to me. @DGuhr wdyt? |
|
Hey Marek! Thanks for this feedback :) I generally go with @pedroigor s answers above. Some picks from my side:
At the moment, the "tools" command in itself has nothing to do, and the completion command should always be used together with source cmd as stated when you type
@pedroigor I also think we shall remove the help msg. dev profile is atm exclusively "usable" with the start-dev command.
@pedroigor +1 for a "marker" solution. perhaps not only for dev profile, but always when --profile="..." is used, check if the server was previously built using a different profile, and add at least a warning that there's a mismatch, so the/some properties from the previously built profile are applied.
The caching part is hard to achieve I guess, and perhaps we should focus on simplifying the ux here another way first, but I know what you mean.
@pedroigor sounds good to me. We didn't put much thought into some more value --verbose could bring, but this might be something worth to look into. |
|
I try to configure Keycloak X according to the following requirements:
It's extremely simple to achieve all the requirements using auto-build (see appendix), but it almost doubles startup time of a container. So, I'd like to make it works using custom Docker image with a prebuilt Keycloak X. And this is where the problems begin. Overriding of persisted db.usernameInside a Dockerfile, I run the command Configuring Vault as a credentials provider for datasourceTo configure Vault as a credentials provider for datasource, I try to use external All this properties are overridable at runtime according to Quarkus documentation. Therefore, I suppose they are allowed to be specified after the Appendix: successful configuration with auto-buildRelevant part of Vault's AppRole secret id can be specified using |
|
@flipp5b What is the most common authentication method used in Hashicorp? The one that is also considered the most secure? Asking because I'm thinking about improving vault configuration a bit and for that, I would like to base it on the most common configuration people are using. The idea is to make database credentials from a vault a bit easier to set up and without exposing Quarkus properties. |
|
@DGuhr, @pedroigor Sorry for the delay.
We use AppRole for services/applications and deliver Role ID and Secret ID to applications through different channels (Secret ID is delivered at the startup time). But AFAIK the recommended way is to use platform-specific auth method (e.g. Kubernetes service accounts) if it's available (and it's not available in our case). So it's hard for me to suppose what method is more common.
I'll try to check it anytime soon! :) |
|
@DGuhr, it looks like env variables take precedence over persisted settings now (for distro built from latest main). This is great! But at the same time, it looks like
Now, keycloak tries to use default value in such a scenario and fails to start. It's possible to override default value using |
|
@flipp5b yah well, that's bad timing i think 😅 we just changed the format of the properties from See here for the discussion and here for the PR containg the changes. |
|
@DGuhr yes, There are some difficulties configuring Vault as a credentials provider still. I can hardly wait upcoming improvements in this field :) Thanks to the whole keycloak team! |
|
hello everyone. i have a problem with deployment KeycloakX 16.1.0 image to Kubernetes (Ingress NGINX - Service - KX Pods) The problem is infinity redirections (with login & password correct) on realm after press login button. In version 14 we solve this by adding flag --http.proxy-address-forwarding=true on run Keycloak in pod. Now in 16th a lot of changes come to distribution configuration and we saw error: Unknown option: '--http.proxy-address-forwarding=true' I have been tried start --auto-build --hostname-strict=false --hostname-strict-https=false --http-enabled=true --proxy=edge --db=postgres options but they don't solve problem Also tried setup ingress with option: Does not help ether. Url to keycloak: https://somesubdomaun.subdomain.domain.com But if i will do just port forwarding from pod to locahost with --hostname=localhost parameter set than everything working without issue So what can be analog of parameter --http.proxy-address-forwarding=true for KeycloakX 16.1.0 image? How make it working correctly after ingress controller in k8s? |
|
@1Const1 By setting the Also, I'm not sure what might be happening and we have tested a similar setup as yours: Ingress NGINX - Service - KXPods). Would you mind creating an issue and sharing the deployment spec? |
|
Yes, i can for reproduce bug Where is more correct to post issue about KeycloakX? OR |
|
The latter, please. |
|
thank you, now it described here: #9644 |
|
Hello, But I have this error at startup : It is trying to load a Wildfly class... is this supported ? If not, is it in the roadmap ? |
|
Awesome. Thanks, @thomasdarimont. We should be fine then with the upcoming changes so that the missing dependencies are available from the distribution. We should probably update our guides with your examples. |
|
@thomasdarimont Btw, due to the Infinispan upgrade. Using a custom JGroups stack was not possible. Something changed in latest Infinispan. The fix is here https://github.com/keycloak/keycloak/pull/10053.` |
|
Thanks a lot @thomasdarimont for your very detailed example, very useful. It works also fine with DIGEST SASL authentication mechanism : |
|
Thanks for the tip with digest authentication, just updated the example :) |
|
We should be able to improve clustering configuration including when using an external Infinispan server. Perhaps this discussions deserves its own discussion, but here are some thoughts:
@thomasdarimont Wdyt? |
|
As this thread is becoming very long we have added a new category specifically for the Quarkus distribution: This thread will now be locked, so please open new discussions in the above category. |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Update 8 Feb
As this thread is becoming very long we have added a new category specifically for the Quarkus distribution:
https://github.com/keycloak/keycloak/discussions/categories/keycloak-x-quarkus-distribution
This thread will now be locked, so please open new discussions in the above category.
Keycloak 15.1.0 came with a lot of improvements to the Quarkus distribution. We are rapidly moving towards this being the default distribution, and will most likely deprecated the WildFly distribution in Keycloak 17.0.
We would love to make the Quarkus distribution as good as it possible can be, and make the migration as simple as possible to everyone. So, please try out the Quarkus distribution and provide us with feedback.
We are particularly interested in feedback on areas like:
All reactions