Repository navigation
Replies: 7 comments 29 replies
|
Hello @ahus1 , can you please link to the actual benchmark test page from which following results were captured under https://www.keycloak.org/high-availability/concepts-memory-and-cpu-sizing
Under the benchmark test page, I am expecting a webpage which captures
|
|
You can find more information on this page: https://www.keycloak.org/keycloak-benchmark/benchmark-guide/latest/report/rosa-benchmark-key-results The Keycloak Benchmark is an incubating space and more information will move to Keycloak's main repository as it matures. About item (1) - in order to do normalization, one would need to do multiple tests on different architectures. As this is not on the immediate road map, this is unlikely to happen soon, as other activities have a higher priority. About items (2) to (4): Assume the defaults, and be aware that Keycloak provides its own official images, so why should the benchmark use something else? |
|
@ahus1 I think for the current statements in the benchmark result
can be interpreted as -
Rather, what you may have meant -
Or,
Please clarify further. If my understanding is correct, then you should please reword the benchmark summary result such a way it does not keep scope of misinterpretation. |
|
The correct answer is "For each 40 user logins per second, per keycloak-cluster (deployment/statefulset) where cluster consists of three Pods with 1 vCPU per Pod", and I would suppose the example details this. As this is an open source project, please consider creating a pull request to update the docs. |
|
Hi @ahus1, I am currently trying to do a Keycloak deployment with an external Infinispan. But I am getting the following error: My environment details:
As there are many files and the post could get quite long, I am attaching the complete files and logs in a zip, with the following contents:
keycloak-with-external-infinispan.zip Do you notice something wrong in my configuration? By the way, I did try to install Keycloak with the operator v.23.0.3 following this example, but I got the error |
|
Thank you for the update. As you found out, the Keycloak Operator needs to run in the same Namespace where you run Keycloak. This is sadly a line at the end of the document: "Currently the Operator watches only the namespace where the Operator is installed." The logs look good. There are actually two Infinispans clusters setup: One inside of Keycloak, and another external Infinispan. We haven't documented to put the JARs into Infinispan, and haven't documented how to use Infinispan's console. It is not essential for running this setup to do this, we might add it in the future. As those JARs would need to match the version of Keycloak, you would need to keep them in sync, which can be a bit of a headache (or even nightmare), so we might not support it. |
|
You're right, maintaining those jars in sync will be a headache. About the Keycloak operator namespace, I would have never thought about running Keycloak in the same namespace if I hadn't read it from the link of the documentation that you posted. As stated from the beginning, it is a feature under preview, so I guess that at some point that will be changed/improved. I also got it to work with vanilla Kubernetes (NOT RECOMMENDED as the Operator takes care of this in the correct way): deployment-with-external-infinispan.yml.txt And the sessions are now kept during restarts. Thanks again! |
|
I see you're setting I also see that you start Keycloak with "start-dev", which you shouldn't do in production, but only for development. I see a lot of problems with this approach as you're re-inventing the wheel when not using the Operator. |
|
Thanks, I am aware that it is not ideal and the |
|
Thank you for updating the previous post! |
|
Thanks for the guides @ahus1! Would it be possible / worthwhile to clarify the upgrade procedure in this deployment model? In Known limitations it is mentioned:
I assume both active and passive instance must be running same versions of software (e.g. due to DB schema differences) and the upgrade procedure for active-passive deployment would follow Upgrading the Keycloak server but now applying to deployment as a whole? The deployment model would not improve upgrades to achieve (near) zero-downtime? |
No it doesn't. There are some cases where it can improve when you only upgrade Keycloak, and the online user sessions are still available in the external Infinispan. In the past, patch upgrades of Keycloak usually didn't require downtimes, but that's not guaranteed. |
|
If you think it could be useful, I'd be glad to submit a PR with clarification, a new bullet under Known limitations / Upgrades with something like:
It could also mention about patch releases, though I wonder if there is a way for user to find out which patch releases are safe to upgrade without downtime. If there isn't, maybe it is not good to add that. |
|
From a user's point of view, this is something that should be stated in the release notes. At the moment, we don't state it there. I would need to be an extra (manual) step in our release process to determine that. I'll make a note about this, but can give a date when we'll come around for that. |
|
No worries. I was rather just thinking that IF patch releases would be supported with no downtime, then there should be some way to find that info. What I was actually looking for, was just to document that even in multi-site deployment we need to take both sites down and upgrade them at once. It is not possible to avoid downtime by keeping running e.g. Keycloak 22 in one site while upgrading the other one to Keycloak 23. Someone could now think this is possible. Therefore I was suggesting another bullet point in Known limitations / Upgrades. |
|
With Keycloak 24, the A/P deployment is now fully supported and no longer preview. Enjoy! |
|
@ahus1 is there any documents for more realms in your configs ? |
|
@jwklijnsma Were you able to create an issue for the behavior you are experiencing? Or have you been able to solve it? |
|
we solvit but the probleem is that we need restart the pods then the cache is correct so maybe keycloak needs update for that |
|
Thank you for the response @jwklijnsma. Restarting pods should not be necessary and it does not not sound like a solution. Could you please create a new bug report for this with steps to reproduce so we can check on this? |
|
#27975 bug if you needs te config |
|
Hey @ahus1, kudos on the fantastic work you've done! It's definitely a significant leap forward. I'm curious, are there any upcoming plans to implement active/active setups? Also, I'm currently delving into designing a multi-site deployment strategy with more than two sites, incorporating both on-prem and hyperscaler environments (a hybrid cloud setup). My aim is to establish a seamless cluster across all availability zones, effectively presenting it as a singular KC system. However, it's crucial that each instance remains independent and operational if the connection between on-prem and hyperscalers experiences any disruptions. Do you have any pointers, documentation, or insights on achieving this? Your guidance would be immensely appreciated. Thanks a lot! |
|
For an A/A setup you would require all session information to be present on the other site before the primary site returns the response to the caller. At the moment some of of the processes are still asynchronous, especially around the processing of invalidations via the work cache. So I'd say A/A is not supported at the moment, and I wouldn't recommend it. Only A/P is supported and tested for now. For A/A we would need to complete at least #28265, #28745 and some method to prevent a split-brain between A/A as described in #29303 Regarding preview features: We're working on reducing the number of preview features one by one, and we'll eventually share a road map. As the list is quite long, it might take a while until we get to token exchange. |
|
A different setup we might eventually have earlier is an Active-Standby setup, where the standby is in a different region and gets its data via an asynchronous database replication. In this setup, you will lose some seconds of data on a failover, and need to start you nodes on the standby site once the database has the standby site has taken over. Such a setup should be possible with an Aurora Global Database. As of today such a setup would loose all online sessions, with #28265 it might even keep the sessions (although they will be some seconds out-of-date on failover due to data lost in the replication delay). |
While theoretically true, couldn't this be addressed with some form of session stickiness in most cases?
Could you elaborate on its benefits compared to an active/passive configuration, please? I might be overlooking something here. Or are you suggesting implicitly that an active/passive setup should also not span regions?
Can I reasonably assume that it won't be deprecated but rather transitioned to a stable state eventually? |
As long as users access Keycloak via their browser, this might be possible. Still the OIDC protocol when confidential clients are involved, they directly contact Keycloak for example to exchange the code for the token. In this scenario sticky sessions don't work, so it is an overall "no" answer to that question.
Yes, and we documented it here. In cross-region setups, the latency between Infinispan will grow. As for each request by Keycloak there are multiple interactions with Infinispan on the other site, the latencies will multiply. For similar reasons there are also no synchronous database setups for cross-region when you look in the AWS portfolio. Aurora with synchronous replication is only available within a single region.
I know there are several people out there using it, and they would be very sad if we would do that. Still with the current state of the feature it allows us to withdraw or deprecate the feature. I won't make promises either way. With the capacity of people helping maintaining Keycloak, we will always need to find the balance between new development and the right amount of supported (and maintained) features. Supporting more features would eventually require more people maintaining Keycloak. |
Good catch. I almost ever forget that part ;) Thanks @ahus1 - that was really helpful. |
|
I discovered this discussion by accident. I´d be interested in A/A single-site and multi-site, and not dependent on AWS or other cloud providers. Is there any documentation or information available in that direction? |
|
This is my situation too. Low latency interconnect between two giant rke2 clusters in two datacenters. The CDN provides connectivity to both sites. But I’ve not yet figured out how to building two KC instances that share synchronously, or how to have CDN poison the route to the passive KC. I was daydreaming about the idea of using CNPG to do cross cluster replicas for the Postgres container but I am too new at this to wrap my head around how to even test it. lol |
Keycloak apparently assumes there is only one active DB instance (c.f. https://www.packtpub.com/en-us/product/keycloak-identity-and-access-management-for-modern-applications-9781800562493 p. 159). My take is, you would at least have to enforce administration (p. 153ff) plus any background imports go to the active instance. However I haven´t figured out how to configure an instance to be passive. If the data model doesn´t support active/active, then probably don´t try - e.g. you might end up with duplicate users in case both instances import them independently. This is a mess, as I do have redundant DC in my active directory, and keycloak is a single point of failure. |
Uh oh!
There was an error while loading. Please reload this page.
Deploying Keycloak to multiple independent sites is essential for some environments to provide high availability and a speedy recovery from failures. This release adds preview-support for active-passive deployments for Keycloak.
A lot of work has gone into testing and verifying a setup which can sustain load and recover from the failure scenarios. To get started, go to the "High Availability" section at the Keycloak Guides, or go directly to Multi-site deployments. This replaces the "Cross-DC" documentation which was available in previous releases of Keycloak.
While we're continuing to work on this, we'd be happy to get your feedback as comments to this post.
Thanks!
All reactions