Repository navigation
Replies: 3 comments 9 replies
|
Hi @dteleguin, this sounds really nice! My (personal) main focus would be the embedded use cases and perhaps it's worth to focus on that in first sight!? I've a few questions/ideas: From your And also in best case, In If I'm wrong or if I misunderstand something, please give me a little more insight to your thoughts/ideas. |
|
@dteleguin goal, if I'm not mistaken, is to simplify the use of a cache in a custom extension or provider. Right now, there is no easy way to achieve and you need to be aware of that checklist to make it work. It sounds like you're suggesting we replace Infinispan with an abstraction that lets you plug your own implementation. You can do that already, see for example, a Redis implementation: https://github.com/p2-inc/keycloak-redis-cache Feel free to share this idea or bring new ones here: #48573 (@dteleguin you should share this topic too) |
|
@pruivo You did reply to my message, but mentioned @dteleguin at the beginning. Who did you actually want to mention? I don't suggest to replace Infinispan and I don't want to replace Infinispan. I'm completely happy with the current state and evolvement of Infinispan in Keycloak. I just want to make use of the embedded cache solution in Keycloak to use my own custom caches in my extensions. If there would be an easy way, it would be a charme! |
|
Please, read my first sentence as "Dmitry's goal..." - a bad decision from me in mentioning. |
|
Hey @dasniko 👋
That would be the same as for any other Keycloak cache: InfinispanConnectionProvider connection = session.getProvider(InfinispanConnectionProvider.class);
BasicCache<K, V> cache;
if (InfinispanUtils.isRemoteInfinispan()) {
cache = connection.getRemoteCache(cacheName); // actual class: RemoteCache<K, V>
LOG.debugf("Remote Infinispan enabled. Using remote cache '%s'", cache.getName());
} else {
cache = connection.getCache(cacheName); // actual class: Cache<K, V>
LOG.debugf("Remote Infinispan disabled. Using embedded cache '%s'", cache.getName());
}
Something along the lines of BasicCache, but even more generic and cache agnostic? Honestly, I don't know how to do that properly. Some advanced but important features like transactional caches and queries will always remain implementation-specific. But yes, we can leave that to cache consumers to do
That would be impossible I'm afraid - the actual cache setup is multi-step and is spread between the Keycloak components and different initialization phases. So the purpose of this SPI is to let the users tell Keycloak/Infinispan what they want ("I want a cache named Foo, it should be distributed in embedded mode and should be also supported in remote mode, it will store entity Bar, here are some basic params, now do everything for me"). Hence, the SPI is currently more "Keycloak-facing" than "user-facing", in a sense that the user would provide the methods in the SPI and Keycloak would consume them (not vice versa). |
|
@dteleguin, my 2 cents, way too complex. A single method should be fine: // returns the caches to configure
Stream<CacheDefinition> getCaches(DeploymentMode mode);
record CacheDefinition(
String name, //name
Configuration configuration, // configuration to use
List<String>indexedEntities) // indexed entity for remote
enum DeploymentMode {
LOCAL, // local mode only means no JGroups or cluster is available. must use local caches only
CLUSTERED, // embedded cache with clustering available
EXTERNAL // external cache
}No mention of schemas since they are loaded at build time here and automatically registered in the embedded caches. We just need to register them in the external caches. ps. I'm shit with names, feel free to come up with a better name. Let me know what you think. |
|
Makes sense @pruivo , my only concern is that |
|
It is intentional @dteleguin. I'm not sure what teaks Keycloak can provide (if any) to a custom cache, as
E870
we don't know what the cache is used for. |
|
Good point, didn't think about |
|
I am a bit worried as this is jumping from "I want to use caching for Authenticators" to "and this is how I want to do it with Infinispan/Caching". To me. this is all about discussing the how, but now the why and the requirements. Without knowing those requirements everyone has probably something else in mind. Once we have the SPI, it would be hard to explain what it should be used for, and what not. Do you want to store state or actually cache information that is stored somewhere else? What is the cost of losing the data that is stored in memory? How many entries do you want to store and for how long? How large is each entry? How many read and write operations do you want to perform per second? How is the number determined, for example does it need to scale with the number of interactive logins? Why are you looking for a distributed store and not a local one? Did you consider putting the information into the database? What other existing options did you consider? What is your cache hit ratio? What is the cost of stale data? What operations do you want to execute on those caches - the abstraction shown above leaks a lot of Infinispan internals, and therefore Infinispan might be the only implemention to satisfy it. This is a pattern we usually want to avoid. |
|
Hi @ahus1 , For our particular case, we can identify two categories of custom caches.
Authentication Session notes could be a candidate here, but the data we need to share (and query) is more complex than simple strings; also, it is not always possible to associate an input to custom REST endpoints with an Authentication Session (as it might be even non-existent at that moment). The answer to your questions would be "approximately the numbers for the
It really does - but at the moment Infinispan is the only implementation. And my gut feeling says it will remain so for at least a couple of years (correct me if I'm wrong). The decision to use custom Infinispan caches was the most conservative - "use what Keycloak is using for the same". |
|
"Custom bruteforce implementation" -> Putting this data in caches is IMHO not a good idea: The data grows (possibly to the number of users in the system), and then it is difficult to handle. This data is better put into the database. Putting the current data model of brute force detection into memory was IMHO not a good idea with hindsight / looking at it today. "Custom login protocol support": When you write "the data we need to share (and query) is more complex than simple strings" this makes me wonder if a cache with a lookup by string is a good choice for you. Maybe the authentication sessions will move to the database eventually. In the meantime, maybe authentication session nodes plus actionTokens would be good enough. |
Uh oh!
There was an error while loading. Please reload this page.
Cache SPI
Motivation
For the Keycloak custom components like Authenticators, it is a valid requirement to have a distributed data layer. It is totally possible to create custom Infinispan caches in Keycloak; however, the process is not straightforward. The code that Keycloak uses to implement its internal caches is not easily reusable because of specific class/interface hierarchy, private fields/methods etc. Therefore, supporting a custom cache currently requires a lot of copy-pasted code, which results in poor maintainability.
This becomes even more complicated with remote Infinispan (aka multi-site / clusterless mode).
Currently, a power user wanting to implement a custom cache needs to cover the following aspects:
¹ <26.2, see #34971
² <26.3, not implemented in recent Keycloak versions
The proposed Cache SPI should allow power users to declare/describe their caches, and let Keycloak do the heavy lifting, thus encouraging code reuse and resulting in better code quality.
Design
org.keycloak.spi.infinispanorg.keycloak.spi.infinispan.CacheSpiorg.keycloak.spi.infinispan.CacheProviderFactoryorg.keycloak.spi.infinispan.CacheProviderAll reactions