Repository navigation
Replies: 16 comments 18 replies
|
We are looking at CockroachDB support in the new store. |
|
An alternative to Cockroach that is getting attention lately is YugaByte. |
There's more to supporting a DB vendor than SQL statements. When they say Postgres compatible I assume that means queries, but what about schema and JDBC driver? There's also testing, documentation, and I would assume that if you optimize for CockroachDB (or YugaByte for that matter) you can take more advantage of it, compared to "just using it as if it was Postgres". |
|
Absolutely correct, the partitioning of the data will be done in different ways and it affects a lot how it's going to be used! The gain is that the (Postgres)JDBC driver is compatible and (probably) most of the queries will be too(at some extend at least), overall switching or supporting another one "should" be less painful than adopting a completely different DB. |
|
Is there a list of priorities of work on the new store somewhere? Has there been a survey of Keycloak users to determine the value in pursuing support for different implementations of the new store? I ask because I have many customers with on-prem deployments that use Keycloak for a limited use case (shim between an application and their in-house IdP) that want Keycloak to be more lightweight (most requested is running without Infinispan). |
|
We did a proof of concept using CockroachDB for Keycloak a few years ago (https://github.com/cloudtrust/keycloak/tree/cockroach-db). Unfortunately we didn't have the resources to go further, so it's now a fairly old version, but maybe this fork can give you an idea of the changes/impacts. Even though we had to freeze this initiative on our side, I think it would be very interesting for Keycloak to support this kind of database in the future, as it is particularly suitable for large scale deployment in cloud. |
|
Thanks, could you elaborate a bit on why you looked into the PoC for CockroachDB? Do you have experience with CockroachDB in general? Why do you think it would bring value to Keycloak say compared to PostgreSQL. The obvious benefit is of course that CockroachDB can at least in theory be deployed in active/active to multiple regions, while PostgreSQL is mostly limited to active/active in multiple AZs, and probably also to less "datacenters" at the time. |
|
As far as I know, PostgreSQL is a master-slave database, so if the master node fails, you have to promote one of your slaves. This promotion can't be done automatically in a safe way because in some scenarios you can have multiple small connection problems that would quickly put your whole cluster in a stale state.
Initially we were very interested in CockroachDB as it was a very serious distributed
8000
SQL database project. We had some historical problems with MariaDB which is not really stable/consistent when there is a bit of load and writes occur on multiple masters in parallel. If keycloak supports CockroachDB, it would have an open-source, consistent multi-master database among the possible setup. This would allow for highly available standard deployment across multiple data centers. It would also be a good step (I imagine there are other impediments) in the direction of large cloud deployments where Keycloak could be used as an IDP for a large enterprise where a multi-datacenter and multi-region deployment is required (e.g. an international bank). |
|
@harture Thanks for the work on this. I'm just looking at your PoC from a few years ago. We've got some resources to pick this up, and are in the process of trying a similar thing to address the transaction retry differences in CockroachDB. It helped to understand both how Keycloak does these transactions, and an approach to do retries in CockroachDB. Were there any outstanding issues you weren't able to address? |
|
@xgp That was some time ago now so I don't have all the details in my head, but if I remember correctly there were no unresolved issues except the incremental update of the schema as changes of Primary Keys were not supported few years ago. |
True, but I think we should pick on, not two, that we support, and in this regard CockroachDB is more mature, and seems to be the better option, but I don't have any real insight which is actually the better for Keycloak. |
|
+1 for official CockroachDB support. There already are CockroachDB dialects in newer Hibernate releases: 5.4 to 5.6, 6.0. So it looks like most of the work would be to upgrade to that version, find or implement a way to use the appropriate dialect as the driver and jdbc are still going to be postgresql, implement tests and update documentation. For now I don't see anything else that would need to be specifically reworked or optimized for CockroachDB in Keycloak code. See also https://blog.flant.com/ha-keycloak-infinispan-kubernetes/ for a recent example of an automated deployment of Keycloak with CockroachDB as data storage. |
|
With keycloak.x 15.0.2 I hit an issue on start up of keycloak. I have manually done an IMPORT PGDUMP to prepare the crdb. The error I am seeing liquibase still trying to run and possibly migrate the db schema and the error it is getting is on creating an index It would be great if there was a way to tell keycloak to not attempt the schema change with liquibase on startup. |
|
Actually there is a way, which is to tell Keycloak to write out a SQL migration script file on startup rather than execute the queries itself. Then you can edit the script, execute it yourself and restart Keycloak. See https://www.keycloak.org/docs/latest/server_installation/index.html#database-configuration and set The issue you linked says it has been implemented in CRDB 21.2 that is already released, which version are you currently using to get this error? |
|
thanks. I am using 21.2, but seems there is still an issue. would be good to not have to manually modify the XML though. For the SQL, when I make that change in keycloak standalone it doesn't seem to generate, just fails when it gets to that db migrate step. with keycloak.x I wasn't able to find how to indicate the migration strategy (may keep me from having to manually modify the XML if I was able to do it) |
|
The multi-region capability of Cockroach DB sounds really nice. However, there don't seem to be any offers by the big cloud providers (definitely not on Azure). There is a marketplace offering in Azure but I don't know if its used that much, at least it doesn't have any rating. We would most likely not operate our own DB, so I am wondering how many people would actually be able to use Cockroach DB at this time. |
|
GCP I know has it in the marketplace, and the cockroachdb cloud can use different clouds like azure and gcp, pretty sure aws is there. |
|
I meant its not operated by the cloud operator, in contrast to things like PostgreSQL or MySQL that you can get from Microsoft on Azure. |
|
Chances are that the CRDB licensing slows down adoption by large operators. Commercial licensing from Cockroach Labs is required to offer CRDB as a service. The cost is definitely going to be higher than a PostgreSQL as a service, thus much less people would be interested, thus less incentive to provide it as a service. |
|
With keycloak 19.0 new storage you just need to use |
|
Dear Keycloak maintainers,
Currently we have Keycloak 21. Can we say that Keycloak is ready to be used with CockroachDB? I've been evaluating previous version of Keycloak and observed that one of the major problems with Cockroach in Keycloak 19 ("TransactionWriteTooOld") was fixed in Keycloak 20 (#13210). Not sure about the rest though. |
|
We were choosing a distributed database for ourselves and finally settled on YugabyteDB. It is simpler, more convenient and, most importantly, much more productive than CockroachDB. To confirm my words, here are the prufs: |
|
Rather that marketing spam, the thing that would be useful to this discussion would be a validation (one way or another) if Yugabyte actually works with Keycloak, and if so, what are the steps to get it running. That has already been performed for Cockroach, and is the reason people are here. |
|
What marketing are you talking about, lol, what's with the baseless attacks? services:
yugabytedb:
container_name: kc_yugabytedb
image: yugabytedb/yugabyte:latest
volumes:
- yugabytedb_data:/home/yugabyte/yb_data
ports:
- "5433:5433"
- "7000:7000"
- "9009:9000"
- "9042:9042"
environment:
YSQL_DB: keycloak
YSQL_USER: keycloak
YSQL_PASSWORD: password
command: bin/yugabyted start --base_dir=/home/yugabyte/yb_data --daemon=false
keycloak:
container_name: kc_keycloak
image: quay.io/keycloak/keycloak:latest
environment:
KC_DB: postgres
KC_DB_URL_HOST: yugabytedb
KC_DB_URL_PORT: 5433
KC_DB_URL_DATABASE: keycloak
KC_DB_USERNAME: keycloak
KC_DB_PASSWORD: password
KEYCLOAK_ADMIN: admin
KEYCLOAK_ADMIN_PASSWORD: password
ports:
- "8080:8080"
command: start-dev
depends_on:
yugabytedb:
condition: service_started
volumes:
yugabytedb_data:
driver: local |
|
Thank you for the example. Notes to other users trying this:
|
|
A bit late to this thread, but because there will never be CockroachDB support for the “legacy” store, we patched Keycloak in order to enable it. We maintain a docker image that is up to date with Keycloak releases that contains our patch. Keycloak on CockroachDB docker images It must be run with a few configuration options set: |
|
We use CockroachDB and Postgres today. We've looked at Yugabyte and it's not a good solution for us. Cockroach is our preferred database and am really looking forward to official Keycloak support for this database. Thank you, @xgp for creating the patch. We will test it out. |
|
I think it needs to be said, in this Issue where we are supposed to justify the use of cockroachdb, that there was already such a demand for what it provides that Keycloak began working on a new storage strategy using cockroachdb. Do we really still need to justify it? I would think, if Keycloak was being written from scratch today, that a decision might be made to go with a solution which scales such as cockroachdb rather than postgres. (The discontinuation post speaks about wanting cross-DC support from the get go, is postgres really the best way to go if you could choose any database?) In a home project I started out with postgres and then began looking into cockroachdb so that I could offer a db which scales in addition to postgres, then it occurred to me, why would I start with postgres (other than it is popular), when I could just start with a single-node cockroachdb deployment by default for small scale installations, and for large deployments simply scale up? The idea of justification is around avoiding having to support several databases, as has been done in the past with Keycloak and over time database support for several databases was dropped to simplify the code base ... I get it, but why stick with postgres when the base database could be cockroachdb? Maybe there should be an issue where folks need to post the reasons why postgres should be a supported database option instead of just assumed. In an HA scenario where the postgres master has failed and its time to switch over to the slave, how common knowledge are those next steps to recover things? Will it just be handled automatically? How much time will it take to switch over to the slave? With cockroachdb things just keep going right? Isn't that the preferred desired user experience? Why wouldn't that be the default/first choice? |
|
We also need to run Keycloak with CockroachDB. This feature is essential for our setup and would greatly enhance our system's reliability and scalability. Despite trying a simple example with CockroachDB To reproduce, use this Docker Compose setup: Can anyone provide an update on the current state of Keycloak's support for CockroachDB and any potential solutions or workarounds? |
|
See the comment above #8885 (comment) We keep a fork updated with each Keycloak release that is compatible with CockroachDB https://quay.io/repository/phasetwo/keycloak-crdb |
|
thank you very much.. can you also please provide the link to the source code which you maintain (if it is open-source), or the patch PR which you have added the support for cockroach? |
|
From the README on quay:
And you can easily do a diff from the mainline using this format: |
|
Thank you very much 🙏 |
|
@m-yosefpor Can you able to spin up cockroachdb with keycloak now? |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
CockroachDB could provide a lot of benefits for larger deployments and deployments that require additional disaster recovery such as multi-region deployments.
Some more information can be found in the Geographically Distributed Stateful Workloads blog series:
In addition the Disaster Recovery Strategies for Applications Running on OpenShift provides useful information.
All reactions