Client session timestamp not updated in the database if running multiple nodes

Before reporting an issue

I have read and understood the above terms for submitting issues, and I understand that my issue may be closed without action if I do not follow them.

Area

core

Describe the bug

Hi everyone, I'm reaching out seeking help to solve a weird behavior that we are observing with our Keycloak instances.

On Keycloak 26.2.5 or 26.3.1, every time we deploy a new version of our Keycloak with custom providers, some user sessions are losing their client sessions, even if they are not expired yet. See the following screenshot before a deployment:

Then, after the Keycloak pods were restarted, one session lost its client:

This is in one of our lower environments, the problem is a lot worse in production, where there are many sessions active.
Then, we start to see a lot of REFRESH_TOKEN_ERROR in our logs:

type="REFRESH_TOKEN_ERROR", realmId="1234", realmName="redacted", clientId="redacted", userId="null", sessionId="4321", ipAddress="redacted", error="invalid_token", reason="Session doesn't have required client", grant_type="refresh_token", refresh_token_type="Refresh",

This is our session lifetime configs:

We currently have two Keycloak pods running on Kubernetes

Version

26.3.1

Regression

The issue is a regression

Expected behavior

The user sessions should not lose their clients after each Keycloak restart, or if this is expected, a clear documentation on why it's needed and how to tweak it.

Actual behavior

Refresh of access tokens fails because user sessions are losing their client sessions during Keycloak restart. This started to happen after we upgraded from Keycloak 26.1.4 to 26.2.5.

How to Reproduce?

Unfortunately, I couldn't find a reliable way to reproduce this other than creating the OIDC sessions and restarting the application manually.

Anything else?

We also observe some logs from ClientSessionPersistentChangelogBasedTransaction, like:

client-session not imported from persister for sessionId=null, offline=false, removing from persister.

Enabled features:

This appears to be an issue related to clustering environment. Probably something for @keycloak/cloud-native or @keycloak/sre team.

Just to add a bit more detail:

Realm config:
SSO Session Idle: 7 days
SSO Session Max: 30 days

Some clients have a configuration like (never bigger than realm's config):
Client Session Idle: 30 min
Client Session Max: 30 hours

What we observed is that after each restart of Keycloak's instances, we are losing recently active sessions, not the sessions that have been lingering around for days. We suspect that this PR might "fix" our problem, but we are not sure what the root cause is yet.

client-session not imported from persister for sessionId=null, offline=false, removing from persister.
Is this log entry redacted, or is the sessionId really null (not found)?

Is updating an option? The fix is included in KC 26.2.9.

@pruivo

The log message is not redacted, it's really null. There was a commit improving those log messages and I can grab them again if needed.

@pruivo yes, it's an option. We are on 26.3.3, but I don't see 26.2.9 in the releases page. Am I missing something?

@marcusdacoregio

@marcusdacoregio 26.2.9 is not released yet; please ignore my comment. For some reason, I thought you were using KC 26.2.

The commit d291e71 is already included in 26.3.3. If you can enable debug log level for that class, it would be great! Thank you.

[26.3] ClientSession timestamp not updated in the database

As a workaround, I suggest to update the client session timestamps in the database before a redeployment to match their respective timestamps in the user session.

We were thinking of updating the timestamp during the REFRESH_TOKEN event, do you think that's feasible? And, what would be the best way to update the column? We are currently doing this:

private void syncClientSessionTimestamp(KeycloakSession session, String userId) {
    EntityManager em = session.getProvider(JpaConnectionProvider.class).getEntityManager();
    KeycloakModelUtils.runJobInTransaction(session.getKeycloakSessionFactory(), (sess) -> {
        ClientModel client = session.getContext().getClient();
        String clientId = client.getId();
        List<PersistentClientSessionEntity> rows = em.createQuery("SELECT c FROM PersistentClientSessionEntity c "
                        + "WHERE c.userSessionId IN (SELECT us.userSessionId FROM PersistentUserSessionEntity us WHERE us.userId = :userId)"
                        + "AND c.clientId = :clientId", PersistentClientSessionEntity.class)
                .setParameter("userId", userId)
                .setParameter("clientId", clientId).getResultList();
        rows.forEach(row -> row.setTimestamp(Time.currentTime()));
    });
}

Unfortunately, session.getContext().getUserSession() returns null, that's why I need the subquery there.

ClientSession timestamp not updated in the database

@marcusdacoregio

@marcusdacoregio - Looking at the code, it looks good to me. Still only tests will show if it works :-D

I didn't think of an event listener. IMHO it is a good idea. I was originally thinking of some SQL you run just before the deployment - running it continuously and as part of Keycloak is a lot better and a lot less scripting involved.

The user session ID should be present in the event as the "sessionId". This should allow you to simplify the statement and narrow down the items you want to update.

Wrapping it in its own transaction might not be necessary, you might want to give it a try without, as it comes with the cost of another transaction and acquiring a DB connection. If load/resource usage is not a problem, you can leave it as is.

I hope we'll have the updated patch release out by the end of the week, let's see.

KC 26.3.4 is now available and includes this fix.

Thanks! We already upgraded to 26.3.4 in our dev environments. After some tests, it looks like both columns are now in sync 🎉 . We will leave it running and perform more tests before moving to production in a few days.

Hi everyone, we updated our Keycloak version to 26.3.4 in all our environments and the problem is gone!!! Thanks for the amazing work and the quick turnaround.

@marcusdacoregio

Great news @marcusdacoregio
Thank you for the feedback!

[26.2] ClientSession timestamp not updated in the database

Navigation Menu

Repository navigation

Client session timestamp not updated in the database if running multiple nodes #42012

Before reporting an issue

Area

Describe the bug

Version

Regression

Expected behavior

Actual behavior

How to Reproduce?

Anything else?

49 remaining items

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Client session timestamp not updated in the database if running multiple nodes #42012

Description

Before reporting an issue

Area

Describe the bug

Version

Regression

Expected behavior

Actual behavior

How to Reproduce?

Anything else?

Activity

mposolda commented on Sep 2, 2025

marcusdacoregio commented on Sep 2, 2025

pruivo commented on Sep 3, 2025

marcusdacoregio commented on Sep 3, 2025

pruivo commented on Sep 3, 2025

49 remaining items

marcusdacoregio commented on Sep 10, 2025

ahus1 commented on Sep 10, 2025

ahus1 commented on Sep 12, 2025

marcusdacoregio commented on Sep 12, 2025

marcusdacoregio commented on Sep 18, 2025

pruivo commented on Sep 18, 2025

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions