Infinispan cache OutOfMemory #19567

benzitohhh · 2023-04-06T11:37:44Z

benzitohhh
Apr 6, 2023

Hi there,

we have

Keycloak running in Kubernetes, as a cluster of two servers
Infinispan distributed cache
CACHE_OWNERS_COUNT=2
CACHE_OWNERS_AUTH_SESSIONS_COUNT=2
JAVA_OPTS="-Xmx1280m"

The cache uses the default settings (cache-ispn.xml)

Recently we started doing some automated tests, where a Keycloak login is simulated once per minute (fresh session).

Since then, we have noticed some OutOfMemory errors:

java.lang.OutOfMemoryError: Java heap space

The error happens on one Keycloak instance, then soon on the other. Both errors are fatal.

For the previous 3 years we have been running Keycloak with 650mb memory fine. We tried increasing it to 1200mb now (see config above), but it does not help.

Is it possible that the cache is filling up too fast with user sessions and/or data? And that these items are not being evicted fast enough?

Do we need to change something in the cache config (cache-ispn.xml see defaults below).

Or do we just need to add more memory?

<infinispan
        xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
        xsi:schemaLocation="urn:infinispan:config:11.0 http://www.infinispan.org/schemas/infinispan-config-11.0.xsd"
        xmlns="urn:infinispan:config:11.0">

    <cache-container name="keycloak">
        <transport lock-timeout="60000"/>
        <local-cache name="realms">
            <encoding>
                <key media-type="application/x-java-object"/>
                <value media-type="application/x-java-object"/>
            </encoding>
            <memory max-count="10000"/>
        </local-cache>
        <local-cache name="users">
            <encoding>
                <key media-type="application/x-java-object"/>
                <value media-type="application/x-java-object"/>
            </encoding>
            <memory max-count="10000"/>
        </local-cache>
        <distributed-cache name="sessions" owners="2">
            <expiration lifespan="-1"/>
        </distributed-cache>
        <distributed-cache name="authenticationSessions" owners="2">
            <expiration lifespan="-1"/>
        </distributed-cache>
        <distributed-cache name="offlineSessions" owners="2">
            <expiration lifespan="-1"/>
        </distributed-cache>
        <distributed-cache name="clientSessions" owners="2">
            <expiration lifespan="-1"/>
        </distributed-cache>
        <distributed-cache name="offlineClientSessions" owners="2">
            <expiration lifespan="-1"/>
        </distributed-cache>
        <distributed-cache name="loginFailures" owners="2">
            <expiration lifespan="-1"/>
        </distributed-cache>
        <local-cache name="authorization">
            <encoding>
                <key media-type="application/x-java-object"/>
                <value media-type="application/x-java-object"/>
            </encoding>
            <memory max-count="10000"/>
        </local-cache>
        <replicated-cache name="work">
            <expiration lifespan="-1"/>
        </replicated-cache>
        <local-cache name="keys">
            <encoding>
                <key media-type="application/x-java-object"/>
                <value media-type="application/x-java-object"/>
            </encoding>
            <expiration max-idle="3600000"/>
            <memory max-count="1000"/>
        </local-cache>
        <distributed-cache name="actionTokens" owners="2">
            <encoding>
                <key media-type="application/x-java-object"/>
                <value media-type="application/x-java-object"/>
            </encoding>
            <expiration max-idle="-1" lifespan="-1" interval="300000"/>
            <memory max-count="-1"/>
        </distributed-cache>
    </cache-container>
</infinispan>

Thanks!

Navigation Menu

Repository navigation

Infinispan cache OutOfMemory #19567

Uh oh!

{{title}}

Uh oh!

Replies: 0 comments

Select a reply

Uh oh!

Infinispan cache OutOfMemory #19567

Uh oh!

benzitohhh Apr 6, 2023

Replies: 0 comments

benzitohhh
Apr 6, 2023