Repository navigation
External Infinispan Configuration - Multiple Encoding Issues #42827
Replies: 19 comments 7 replies
|
Where did you find the |
|
Looking at the source code, the option to configure the site name is Side note: the configuration that you're using is no longer valid with KC 26 and will fail to start. |
|
@pruivo Thank you for the update. I have now resolved the Multiple datacenters issue by using the KC_SPI_CONNECTIONS_INFINISPAN_QUARKUS_SITE_NAME you suggested. Now I'm experiencing persistent encoding/marshalling issues when integrating Keycloak with external Infinispan, regardless of the media-type configuration used. I've tested multiple encoding combinations and consistently encounter ClassCastException or EncodingException errors. Current issue:
Configuration Approach Deployment files infinispan-overrides.yaml - Infinispan server configuration keycloak-ha-values.yaml - Keycloak Helm values create-keycloak-caches-updated.sh - Manual cache creation script Additional Context Any guidance on the correct configuration pattern for this specific scenario would be greatly appreciated. |
|
External Infinispan caches require I strongly suggest dropping Keycloak 25, as it has not had security updates for a while, and moving to Keycloak 26 (documentation here). All this work will be discarded as it is no longer supported. |
|
@pruivo As suggested, i have moved to Keycloak 26.3.2. Still i am facing some encoding related error. With application/x-protostream
With application/x-java-object
D:\Projects\Keyclaok\keycloak-ha\keycloak-vdrc\infinispan-overrides-x-protostream.yaml - Infinispan deployment file with application/x-protostream D:\Projects\Keyclaok\keycloak-ha\keycloak-vdrc\infinispan-overrides-x-java-obj.yaml - Infinispan deployment file with application/x-java-object D:\Projects\Keyclaok\keycloak-ha\keycloak-vdrc\keycloak-ha-values.yaml - Keycloak deployment file I understand application/x-protostream is the media type suggested in the official documentation, but i also tried application/x-java-object as suggested by @ahus1 in the thread - #29274 Any suggestion to resolve this would be greatly appreciated. Thanks in advance. |
|
@preetham6dtech can you share the full log file? You don't need a custom Keycloak configuration as the encoding is already properly set by default. |
|
@pruivo Please find the attached complete logs. I havnt mentioned any custom configuration in keycloak for encoding. I am specifying the encoding in the infinispan configuration in the deplyment file. The shared logs is when keycloak is deployed with external infinispan deployed with media-type as application/x-protostream |
|
@preetham6dtech can you share your keycloak configuration (features enabled/CLI options/environment variables)? Don't forget to mask the password and hostname :) |
|
@preetham6dtech - I see you are running We run out tests with the latest 15.0.x version, as Keycloak is also using that version on the client side. Please also share the complete Infinispan Cache Configuration so we can see that all caches are present. |
|
@pruivo Please find the keycloak deployment file : @ahus1 Am running my infinispan with the image with quay.io/infinispan/server:latest, i will try out with infinispan 15.0 as mentioned and update the test results, for timebeing sharing the complete Infinispan deployemt file am using: |
|
@preetham6dtech The I don't recognize the syntax in Independently of the answer, your XML is not totally right.
|
|
Thanks for reporting this issue, but there is insufficient information or lack of steps to reproduce. Please provide additional details, otherwise this issue will be automatically closed within 14 days. |
|
@pruivo I am not using infinispan operator, i am using the helm charts to deploy infinispan as mentioned as an alternative in the official documentation Command used to deploy Infinispan using helm - Also, additionally, as you have suggested have tried making all the changes mentioned in the infinispan deployment file, but still getting the same error Please find the complete keycloak logs for reference - Please find the updated infinispan deployment file for reference with the changes updated - |
|
@preetham6dtech the Another thing is that helm chart does not support creating cross-site deployment as it is only available through the operator. Any reason why you can't use the Infinispan Operator? |
|
8BD0
Hi @pruivo After multiple iterations, i was able to properly deploy infinspan using infinspan operator instead of helm charts. I was successfully able to connect the external infinispan to keycloak when trying with a single site. Infinispan deployment file - Site A(vdrc) - infinispan_deployment.yaml Infinispan deployment file - Site B(vmz) - infinispan_deployment.yaml |
url: kubernetes://kubernetes.default.svc:443
secretName: xsite-token-secret
|
|
As this is more of a question, I'm converting this to a discussion. |
|
@pruivo Yes both the namespaces are in the same cluster. As suggested i tried removing the url and secret name, but still the behaviour is same. If i comment out the service.sites config from the infinispan deployment file, the infinispan pods are properly coming up. But if i mention the service.sites configuration, the pod is not coming up, when i check the events in the namespace am getting below errors. As i understand, the service.sites configuration is required in order to establish cross site connectivity. Error creating: pods "infinispan-router-6494b5697c-kr9sr" is forbidden: failed quota: common-vodacom-cd-bss-int-vdrc-dev-quota: must specify limits.cpu for: gossiprouter; limits.memory for: gossiprouter; requests.cpu for: gossiprouter; requests.memory for: gossiprouter (combined from similar events): Error creating: pods "keycloak-cache-deployer-rff8b" is forbidden: failed quota: common-vodacom-cd-bss-int-vdrc-dev-quota: must specify limits.cpu for: cache-deployer; limits.memory for: cache-deployer; requests.cpu for: cache-deployer; requests.memory for: cache-deployer But in my deployment file i have already mentioned the cpu and memory config in the below format spec: Attaching the complete infinispan deployment file am using for reference: infinispan_deployment.yaml Looks like there is a mandatory ResourceQuota to be mentioned for all the containers to be deployed in my namespace, and without the ResourceQuota the pods wont come up. The above resource quota mentioned seems to be not considered for the gossiprouter and cache-deployer containers which infinispan is internally trying to create. Is there any specific format for defining the cpu and memory resource quota for the gossiprouter and cache-deployer. |
|
@pruivo Just an update, i have deployed a LimitRange in the namespace to overcome the mandatory resource quota defnition. Now i am no longer getting the resource quota definition missing related error. Please find the logs am getting in the Infinispan controller Deployed with spec.sites, following are the logs: DEBUG controller-runtime.webhook.webhooks wrote response {"webhook": "/validate-infinispan-org-v1-infinispan", "code": 200, "reason": "", "UID": "39bb474e-3d49-4891-8900-531a308fc96e", "allowed": true} 2025-09-24T10:35:27.087Z INFO controllers.Infinispan Done {"infinispan": "common-vodacom-mz-bss-transformation-int-dev-01/infinispan-cluster", "requeue": true, "requeueAfter": "0s", "error": "resourceVersion should not be set on objects to be created"} 2025-09-24T10:35:27.087Z ERROR Reconciler error {"controller": "infinispan", "controllerGroup": "infinispan.org", "controllerKind": "Infinispan", "infinispan": {"name":"infinispan-cluster","namespace":"common-vodacom-mz-bss-transformation-int-dev-01"}, "namespace": "common-vodacom-mz-bss-transformation-int-dev-01", "name": "infinispan-cluster", "reconcileID": "49aa248d-ebe9-4670-8570-3a7c3cc4e438", "error": "resourceVersion should not be set on objects to be created"} Deploying without spec.sites configuration: ERROR Reconciler error {"controller": "infinispan", "controllerGroup": "infinispan.org", "controllerKind": "Infinispan", "infinispan": {"name":"infinispan-cluster","namespace":"common-vodacom-mz-bss-transformation-int-dev-01"}, "namespace": "common-vodacom-mz-bss-transformation-int-dev-01", "name": "infinispan-cluster", "reconcileID": "83415b07-fae1-4d30-951b-c3b2b52d8ce2", "error": "Operation cannot be fulfilled on infinispan.infinispan.org "infinispan-cluster": stale resource version"} Could you please suggest if am missing anything, or please let me know if any further details required |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Before reporting an issue
Area
infinispan
Describe the bug
Environment
Keycloak Version: 25.0.1
Deployment Type: Kubernetes (bare metal cluster)
Cache Configuration: External Infinispan (single replica)
Database: MySQL
Java Version: Container default
###Issue Description
When deploying Keycloak with an external Infinispan cache in a Kubernetes cluster, Keycloak fails to start with the following error:
Version
25.0.1
Regression
Expected behavior
Keycloak should successfully connect to the external Infinispan instance and start without site configuration errors, especially when using a single-replica Infinispan deployment.
Actual behavior
Keycloak startup fails with the IllegalStateException related to datacenter/site configuration, even though:
How to Reproduce?
Deploy Infinispan using the provided configuration (single replica)
Deploy Keycloak with external Infinispan cache configuration
Observe startup failure with the IllegalStateException
Anything else?
Keycloak Environment Variables
Infinispan Configuration
Deployment: Single replica using Infinispan Operator
Image: quay.io/infinispan/server:latest
Storage: Ephemeral
Expose Type: LoadBalancer on port 11222
Infinispan Cache Configuration
Analysis
The error suggests that Keycloak/Infinispan is detecting multiple datacenters when none are explicitly configured. This might be related to:
Questions
Workaround Attempts
All reactions