Deprecated: File Map Storage: How To and Q&A #16768

hmlnarik · 2023-02-01T14:55:35Z

hmlnarik
Feb 1, 2023

Setup Keycloak server with files as a storage

Currently available in #16001

To run, execute the following command:

mvn -pl testsuite/utils exec:java \
 -Pkeycloak-server,map-storage \
 -Dkeycloak.mapStorage.provider.default=file \
 -Dkeycloak.singleUseObject.map.storage.provider=concurrenthashmap

You can now also use nightly Quarkus distribution:

bin/kc.sh start-dev --storage=file --storage-file-dir=data/file

Known issues and limitations

Summary:

No-downtime upgrade not supported yet
File locking not supported yet
Single-use store not applicable to file store, currently uses chm store is configured instead (other implementation can be used via --storage-area-single-use-object option)
Format of the files is in alpha stage and will change
No caching in place, thus file store is slow

Open issues

dominikschlosser · 2023-03-28T16:42:30Z

dominikschlosser
Mar 28, 2023

We tested it and it works without any problems.

We want to use this store for mostly static data (realms, clients, clientscopes) in a configuration-as-code like manner. Right now we are using a third-party tool (Config-CLI) to provision a running Keycloak instance with configuration data from YAML files.

Some things that are bothering us are:

The file and directory structure of current file storage can't be used easily in Kubernetes since ConfigMaps can only represent one directory. For realms, clients, clientscopes and roles we currently need to define 14 configmaps for 3 realms.
It would be nice if there was an option to put everything belonging to a realm in single file.
Currently there seems to be no way to substitute environment variables inside the configuration. This is necessary for secrets like private keys, which are provided as Kubernetes-secrets and exposed as environment variables inside the container.

1 reply

ahus1 Apr 13, 2023
Collaborator

Thanks for sharing your experiences. About the file structure: While I can't promise any changes today, as an intermediate solution considering zip'ing the file storage, putting it into a ConfigMap, and unzip'ing it either at Keycloak startup, or using a sidecar.

kereis · 2023-04-13T11:54:39Z

kereis
Apr 13, 2023

I was redirected to here by mailing list as this feature might be relevant for my use case.

When I look at #16001, the handling of authz related settings hasn't been implemented yet. Is there a possible ETA when this will be implemented? If there's no plan to implementing it in 8000 the near future, is it possible to contribute to this feature?

2 replies

hmlnarik Apr 14, 2023
Author

Thank you very much for your interest and offer!

Support for authz is improved in a private branch which should be gradually migrated to main. If you are interested and able to try it, I would warmly appreciate your feedback.

hmlnarik May 16, 2023
Author

The authz has been merged to the main branch today and should be ready to be tested in the next nightly. If you are still interested in trying it out, we would welcome your feedback!

kereis · 2023-04-17T07:32:15Z

kereis
Apr 17, 2023

I haven't looked enough into detail, but how does the new file map storage handle changes in static files? Does a change recreate the affected entity or update only the changed properties?

My use case here: We have configured a user federation in READ_ONLY mode, meaning that any user-specific information, e.g. role mappings, are saved in Keycloak DB only. If a realm file is updated and thus leading to the realm being recreated, user role mappings will disappear.

1 reply

hmlnarik May 16, 2023
Author

Thank you for the question. The file storage stores the data only in the file system, so any change in static files is immediately reflected by Keycloak.

Please beware that the map storage does not work with the user federation at this moment.

sjhr007 · 2023-06-29T18:16:03Z

sjhr007
Jun 29, 2023

How can stored data files be managed in a versioning system (like Git)? Does Keycloak provide ways to commit files to a versioning system when files get updated through operations from Keycloak UI?
How can changes be propagated to a versioning system like updating realm or client definition/attributes from Keycloak UI updates files in the filesystem?

0 replies

ahus1 · 2023-11-08T22:57:06Z

ahus1
Nov 8, 2023
Collaborator

As announced to the community in this blog post, we are no longer developing the Map Store, and it will be removed from Keycloak's codebase. Follow #24111 for the removal steps.

0 replies

Deprecated: File Map Storage: How To and Q&A #16768

Uh oh!

Uh oh!

hmlnarik Feb 1, 2023

Setup Keycloak server with files as a storage

Known issues and limitations

Replies: 5 comments · 4 replies

Uh oh!

Uh oh!

dominikschlosser Mar 28, 2023

Uh oh!

ahus1 Apr 13, 2023 Collaborator

Uh oh!

kereis Apr 13, 2023

Uh oh!

hmlnarik Apr 14, 2023 Author

Uh oh!

hmlnarik May 16, 2023 Author

Uh oh!

kereis Apr 17, 2023

Uh oh!

hmlnarik May 16, 2023 Author

Uh oh!

sjhr007 Jun 29, 2023

Uh oh!

ahus1 Nov 8, 2023 Collaborator

hmlnarik
Feb 1, 2023

Replies: 5 comments 4 replies

dominikschlosser
Mar 28, 2023

ahus1 Apr 13, 2023
Collaborator

kereis
Apr 13, 2023

hmlnarik Apr 14, 2023
Author

hmlnarik May 16, 2023
Author

kereis
Apr 17, 2023

hmlnarik May 16, 2023
Author

sjhr007
Jun 29, 2023

ahus1
Nov 8, 2023
Collaborator