8000 Replace operation set wrong lifespan in remote infinispan database an… by martin-kanis · Pull Request #15619 · keycloak/keycloak · GitHub
[go: up one dir, main page]

Skip to content

Replace operation set wrong lifespan in remote infinispan database an… - #15619

Merged
mposolda merged 1 commit into
keycloak:mainfrom
martin-kanis:10755-replace-operation-set-wrong-lifespan-in-remote-infinispan-db
Nov 23, 2022
Merged

mposolda merged 1 commit into
keycloak:mainfrom
martin-kanis:10755-replace-operation-set-wrong-lifespan-in-remote-infinispan-db

Conversation

@martin-kanis
Copy link
Copy Markdown
Member

…d leads to session eviction

Fixes #10755

@martin-kanis martin-kanis added area/storage Storage layer — JPA entities, database schemas, Liquibase migrations, caching area/infinispan Infinispan — distributed caches, clustering, JGroups, cross-site replication team/storage-sig labels Nov 22, 2022
@martin-kanis
Copy link
Copy Markdown
Member Author

@mposolda Hi, do know why we use -1 for a lifespan when we add an entry to the remote cache for the first time instead of actual lifespan which is calculated for the entry? This seems to be causing problems with replace operation which sets timestamp in the past in the DB.

@martin-kanis
Copy link
Copy Markdown
Member Author

@daviddelannoy Would you have a chance to verify the fix please? I think we don't have a easy way to test it in our testsuite as it involves a remote Infinispan with a database.

@daviddelannoy
Copy link
Copy Markdown
Contributor

I will try to see if we still have some test environment to test it, but not quite soon (not this month)
Because of this bug we dropped the external ISPN cluster with DB solution 8 months ago in favor of direct JDBC store from a distributed cache, which also required a small fix proposed before by @thomasdarimont involving the SKIP_CACHE_STORE flag.

As other few people were interested in a fix for this issue I will try to test it, thanks for the PR 😉👍

@thomasdarimont
thomasdarimont commented Nov 22, 2022 •
Copy link
Copy Markdown
Contributor

@martin-kanis thank you very much for your efforts! Your code change seems to do the trick.

I just gave this a quick spin with my example setup: https://github.com/thomasdarimont/keycloak-project-example/tree/main/deployments/local/clusterx#run-keycloakx-cluster-with-database-backed-user-sessions
and I could verify that the timestamp calculations (discussed here #10577) in the string_table used by infinispan-cachestore-jdbc are now correct 👯

Here is an example for the string-keyed-jdbc-store configuration: https://github.com/thomasdarimont/keycloak-project-example/blob/main/deployments/local/clusterx/haproxy-database-ispn/cache-ispn-database.xml#L43

However as @daviddelannoy mentioned, I still need to apply my fix for org.keycloak.models.sessions.infinispan.CacheDecorators to ignore skipCacheStore hints.

Tables generated by string-keyed-jdbc-store configuration:

  • ispn_clientSessions
  • ispn_clientSessions_META (purpose is not clear to me)
  • ispn_sessions
  • ispn_sessions_META (purpose is not clear to me)
    image

Empty ispn_sessions:
image

ispn_sessions after login with user1 to client1
image

ispn_sessions after user1 accesses client2
image

ispn_clientSessions with 2 active client sessions
image

ispn_sessions after login with user2 to client1
image

ispn_clientSessions with 3 active client sessions for 2 users
image

ispn_sessions after user1 logout
image

ispn_clientSessions after user1 logout
image

@mposolda mposolda left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@martin-kanis @thomasdarimont Thanks for the fix and for the testing with the fix applied.

TBH I don't recall the reasoning for using -1 here. I don't see a problem with your change and hence I am approving this PR.

We can run cross-dc to doublecheck if there is a regression, but I don't know if it is possible to run cross-dc tests against main branch with the latest changes (removed Wildfly etc).

I am approving and leaving to you whether you want to run some more tests or whether you want also someone else approval for this PR.

@martin-kanis
Copy link
Copy Markdown
Member Author

@thomasdarimont Thank you for the verification!
@daviddelannoy @thomasdarimont Regarding the skipCacheStore flag. I would keep it in the separate issue #10803 which I plan to pick up soon :) This issue is about wrong lifespan in JDBC store which is present only with a remote Infinispan (RemoteCacheInvoker file).

@martin-kanis
Copy link
Copy Markdown
Member Author

@martin-kanis @thomasdarimont Thanks for the fix and for the testing with the fix applied.

TBH I don't recall the reasoning for using -1 here. I don't see a problem with your change and hence I am approving this PR.

We can run cross-dc to doublecheck if there is a regression, but I don't know if it is possible to run cross-dc tests against main branch with the latest changes (removed Wildfly etc).

I am approving and leaving to you whether you want to run some more tests or whether you want also someone else approval for this PR.

Thank you Marek for the review. Yeah, it is not possible to run cross-dc test with current main as we removed all the Wildfly stuff. I can try to apply my patch to older Keycloak and let the cross-dc tests run on it.

@martin-kanis
Copy link
Copy Markdown
Member Author

Thank you Marek for the review. Yeah, it is not possible to run cross-dc test with current main as we removed all the Wildfly stuff. I can try to apply my patch to older Keycloak and let the cross-dc tests run on it.

At the end, GHA keycloak build was failing because of a node js issue so I run cross-dc locally on Keycloak 19 + this patch and tests passed.

@thomasdarimont
Copy link
Copy Markdown
Contributor

@daviddelannoy @thomasdarimont Regarding the skipCacheStore flag. I would keep it in the separate issue #10803 which I plan to pick up soon :) This issue is about wrong lifespan in JDBC store which is present only with a remote Infinispan (RemoteCacheInvoker file).

Okay, good to know. One idea I had about this was, that instead of using a flag like I did (for testing ;-)), we could allow writes to cachestore if a cache is explicitly configured with persistence or a remote-store.

@mposolda
Copy link
Copy Markdown
Contributor

@martin-kanis Thanks for doublecheck with the tests. I am merging the PR then.

@mposolda
mposolda merged commit 08061af into keycloak:main Nov 23, 2022
@thomasdarimont
Copy link
Copy Markdown
Contributor

@martin-kanis another note: When using a database for cache storage, then we currently have to explicitly configure either a jdbc connection-pool or a datasource. It would be great if we could just reference the quarkus datasource somehow. Perhaps this could be implemented with a custom QuarkusDataSource or a QuarkusJdbcDriver ... or implement a dummy JNDI lookup.

In my spring-boot-keycloak-server-example I use the later approach for exposing the Spring Boot datasource with a "interal" JNDI lookup to the Keycloak infrastructure.
See: https://github.com/thomasdarimont/embedded-spring-boot-keycloak-server/blob/master/embedded-keycloak-server-spring-boot-support/src/main/java/com/github/thomasdarimont/keycloak/embedded/support/KeycloakInitialContext.java#L11
See: https://github.com/thomasdarimont/embedded-spring-boot-keycloak-server/blob/96915013a024320f84f450a6571eac494e2c5892/embedded-keycloak-server-spring-boot-support/src/main/java/com/github/thomasdarimont/keycloak/embedded/support/DynamicJndiContextFactoryBuilder.java#L49

I think the same would work for quarkus keycloak without any modifications to infinispan.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/infinispan Infinispan — distributed caches, clustering, JGroups, cross-site replication area/storage Storage layer — JPA entities, database schemas, Liquibase migrations, caching

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Replace operation set wrong lifespan in remote infinispan database and leads to session eviction

4 participants

0