8000 adding some notes about v2 default handling by shawkins · Pull Request #39547 · keycloak/keycloak · GitHub
[go: up one dir, main page]

Skip to content

adding some notes about v2 default handling - #39547

Closed
shawkins wants to merge 1 commit into
keycloak:feature/admin-api-v2from
shawkins:v2-defaults
Closed

shawkins wants to merge 1 commit into
keycloak:feature/admin-api-v2from
shawkins:v2-defaults

Conversation

@shawkins
@shawkins shawkins commented May 7, 2025 •
Copy link
Copy Markdown
Contributor

Added a rough in of ClientService method refinements. In an ideal world this would all be pretty similar across types, possibly even allow for some generic base interfaces / classes.

Jackson notes:

I'd advocate for using NON_ABSENT at a top level to allow for "empty" values such an empty string. However this has the side effect that any possibly fully empty child objects or collections will need to be specifically marked as NON_EMPTY (fabric8 does this).

The usage of required or NON_ABSENT makes using the object model for creating JSON Merge patches difficult - the lack of a value will cause an exception when deserializing, or empty lists / arrays won't be serialized, or you can't use explicit nulls - which prevents some patching behavior. If we don't think that the representation classes will be used for this purpose, then we can ignore this concern.

Defaults:

Re-reading #38551 there seems to be some intermixing of the ideas of what is a default and what is runtime state. If we are pursuing versioning, then these should be distinct concepts. That is for a given representation of a specific version, the same default values will always be the same. Changing the default values on a given version would likely be seen as a breaking change - although Kube does not prevent this. With this perspective I contend it doesn't matter if these then appear in the representations returned by Keycloak. The user is already maintaining their own representation with whatever defaults they want to be included - they don't have to overwrite that with what is returned by the server.

The other behaviors of always pruning defaults, or selectively trying to retain only those defaults that have been supplied in a PUT or POST have additional canonicalization (which we haven't addressed yet) or complexity (the v1 logic is setting defaults, so legacy exports / imports will introduce the defaults anyway) that don't seem to be needed.

With this in mind for "simple" defaults - those that can be specified in the OpenApi spec / json schema, I would propose using JsonProperty.defaultValue. We'll have to ensure:

  • that it's picked up by the schema generation (it is with fabric8 logic)
  • since it is not used by serdes logic, we store these defaults and update the field / setter with the default (shown in this first commit). Granted that is not well encapsulated - you end up needing to put the default on the JsonProperty, the field, and handle null in the setter. However this also introduces a problem if we try to use the same representation class for patching purposes - as the default will always be present.

We can further describe what is set by the server as either:

Complex defaults / implied state: these may not be expressable as a static default, but are fully determinant based upon the rest of the representatio 8000 n - with no possibility of interpolating values from a changing source, such as server variables. Is there anything under the ClientRepresentation that would fall under this category? Assuming the model logic computes these already, we may not have to do any special handling other than needing the description to elaborate on what the default will be. However I do see there could be some complexity relying on the model layer computation if there could be a mismatch between the serving version and the storage version.

or runtime state - which definitely should have separate handling. What is currently under the ClientRepresentation that would fall under this category?

The final thought would be around templating / expressions. We talked about this early on, but I'm not sure if it's still part of the conversation. Allowing for arbitrary expressions in what is sent to keycloak, similar to openshift templates, is difficult with a typed object model - that is if you have a boolean field and you want it to take the value of ${var} - you have to parse something that is a string instead. You end up needing to have special deserialization logic (we have that in fabric8) or you have to parse generically, resolve the values, then parse as the representation. To understand what is not a default, or needs to be re-evaluated when variables change, you would end up needing to track both the "template" and the instanciation (just like openshift templates).

@keycloak/cloud-native WDYT?

Signed-off-by: Steve Hawkins <shawkins@redhat.com>
@shawkins
shawkins requested a review from a team as a code owner May 7, 2025 20:46
@shawkins
shawkins marked this pull request as draft May 9, 2025 09:50
@vmuzikar vmuzikar added the team/admin-api-wg Admin API v2 Working Group label May 12, 2025
@shawkins
Copy link
Copy Markdown
Contributor Author

Based upon our discussion a better term to use may be "effective default", rather than default. We're referring to a class of fields that have a logical default in a keycloak version, but we also want some kind of auto-migration behavior. That is if left unset when initially created / later manipulated, then in some later version of keycloak the value / behavior may change automatically when the logical default changes.

Honestly this seems contrary to the point of versioning with a declarative representation / defaults. That is if I specify a particular instanciation of client vN that has a given set of defaults - it should remain in perpetuity effectively a vN client until some action is taken to migrate it to a later representation. No matter what version of keycloak I'm on (with potentially a different storage version), as long a clients of vN are supported, I should expect exactly the same behavior / defaults for that client. When using a different serving version and fetch it as a vN+1 client, it should still exhibit, as much as possible, its vN nature.

To support auto-migration from a declarative perspective could mean differentiating between whether you want the "default" or the "effective default" - which could relate somewhat to the templating discussion above if this were done through an expression. Otherwise it's not clear to me if I leave a field omitted or null [*] which behavior I should get.

Alternatively this class of "effective defaults" should not be declaratively described as defaults in the object model / openapi - and potentaily even treated as runtime state.

[*] kube tries to discourage the idea of an explicit null as a value, which using NON_ABSENT / NON_NULL for jackson serialization effectively enforces.

@slaskawi
Copy link
Copy Markdown
Contributor

What I miss from this analysis is that Kubernetes enforces the API to be isomorphic. In other words, v2 can always be converted to v1 and vice-versa. Kubernetes delivers a false promise that you can even remove old, deprecated fields from your API but you can't. You can only clean the REST payloads but all deprecated fields will always need to stay in the API as optionals for v2 -> v1 conversion.

Looking at Keycloak, we have exactly the same analogy. We need to provide a conversion path from ClientModel (our v1 API) into whatever representation we have - v1 ClientRepresentation, OIDC representation or v2 Client Representation.

In my opinion there are two very significant aspects that will greatly influence the implementation. Can we foresee the v3 of the Client Representation and how this should look like? If we assume full isomorphism, we can assume we'll always be transitioning through ClientModel, in other words v3 -> Client Model (v1) -> v2. This assumption greatly simplifies the code and makes Client Representations totally independent from each other (v3 can look drastically different than v2). The other aspect is how to tie defaults to it. If we assume isomorphism, probably the only way is to tie the defaults with a particular implementation. In other words, v3 will use its own mechanism for defaulting that can be independent of v2.

@vmuzikar
vmuzikar commented May 12, 2025 •
Copy link
Copy Markdown
Contributor

I gave it some more thought. There are basically two problems we're facing here: resource/storage versioning, and API versioning. From my perspective, only the latter is in the scope of Admin API v2. That said, we still have the defaults problem that significantly touches the resource versioning.

My suggestion would be the following as I'm feeling we might easily overengineer it.

  • IMHO resource versioning is out of scope of the Admin API. That would be a whole new concept to Keycloak, and it would introduce new problems to solve, e.g. resource migration (e.g. how do I migrate Client v1 to Client v2?).
  • Admin API (regardless of its version) is always just a view and management layer of the stored resource.
    • Admin API will be versioned. Different versions can introduce various breaking changes to the endpoints and/or representations, but it is always backed by the same storage.
    • The API versions are interchangable. I can create a Client using v1 and access it via v2, and vice versa.
  • Resource versions are tied to Keycloak. I.e. we do not have "Client v1", but "Client in Keycloak 26".
    • Storage (DB schema) migrations are handled as we do today – during server upgrade. No changes in that area.
  • There is the defaults problem.
    • From my perspective, the main motivations for having floating/ephemeral defaults are:
      • Declarative configuration. If I create a Client in Keycloak 24 and upgrade to 26 that have different defaults, I want to get the same Client as if I created the Client in 26 directly.
      • Security enhancements. If we change some default to improve security, it should be applied to all existing Clients that don't already explicitly define that field. This would be also aligned with how we change defaults for e.g. server config.
    • We now persist the defaults. So this would be the main change we'd need to do on the storage layer.
      • Eventually, we would stop persisting the defaults.
        • Default values for existing Clients would be treated as user specified.
      • The defaults would be resolved at runtime by the model layer for given Keycloak version. So it would not be tied to a "resource version", or API version but a Keycloak version.
      • Until Admin API v1 is removed, this would be controlled via a feature flag.
        • To avoid touching the v1 code as much as possible, we'd keep persisting the default. This way, Admin API v1 doesn't need to resolve the defaults at runtime but keeps it using as is now.
        • To allow for floating defaults, we'd temporarily introduce additional column to the DB schema. It would be a list of field names that have the default values. If then fetched via v2, the persisted defaults would be ignored, and instead resolved at runtime.
        • Once Admin API v1 is finally removed, we'd remove the extra column and null the persisted default values of existing resources during the one time migration.

WDYT?

@shawkins
shawkins commented May 12, 2025 •
Copy link
Copy Markdown
Contributor Author

Followed up with @slaskawi on his comment off-line. I think he better understands what I was trying to communicate about what we are calling defaults.

IMHO resource versioning is out of scope of the Admin API. That would be a whole new concept to Keycloak, and it would introduce new problems to solve, e.g. resource migration (e.g. how do I migrate Client v1 to Client v2?).

It's fine to drop the idea of versioning beyond the admin api level, but I believe we are still going to need to track the serving version.

Resource versions are tied to Keycloak. I.e. we do not have "Client v1", but "Client in Keycloak 26"

Are you saying that users can't expect stable behavior across keycloak versions unless they fill in all values (including defaults) for themselves and those choices do not get pruned?

Declarative configuration. If I create a Client in Keycloak 24 and upgrade to 26 that have different defaults, I want to get the same Client as if I created the Client in 26 directly.

From my perspective this is mixing two contructs - declarative configuration and migration. You are trying to create a situation where migration will seemingly happen automatically. But is that actually "the same client"? If I am on KC 26.2 and have "declaratively" configured a client, why should the behavior / defaults of that client change if I upgrade to KC 27? Shouldn't it remain function identically, or as closely as possible, as to the one created in KC 26.2? Without this consistency won't tooling be dependent upon interogating the keycloak version, and also make it harder for users to reason about upgrades?

Security enhancements. If we change some default to improve security, it should be applied to all existing Clients that don't already explicitly define that field. This would be also aligned with how we change defaults for e.g. server config.

The only way this makes sense is if it follows something from the previous comment - that is what we are calling defaults are not actually "simple" defaults.

@shawkins
shawkins commented May 15, 2025 •
Copy link
Copy Markdown
Contributor Author

After discussions with @vmuzikar and @slaskawi I think we may be reaching some concensus over how to categorize what has been called defaults and what exactly a user is declaring via configuration. As long as we're consistent with what could be a simple/complex default, and what could be runtime state (even if derived from effective defaults), things will be clearer to discuss.

I would categorize what @vmuzikar is describing as more akin to a declarative template, not versioned declarative configuration a la Kuberentes, Wildfly, and many other systems. The user story seems to roughly be to define a mostly unversioned [*] Client template such that any configuration not explicitly set may vary for any number of reasons - Keycloak version, feature enablement, and CVEs.

[*] the only versioning notion is at a coarse level - AdminApi v2. There are no expected revisions, such as v2.1, nor resource level versioning. It's not clear whether even the origination under AdminApi vX would be tracked via the representation.

If that is a correct charactiziation, the downsides/caveats to the template approach seem to be:

  • we should not express anything as a default in a static openapi contract that may not be stable. To make users / tooling aware of the effective defaults, we must do something like:
    • require fetching the effective openapi contract from the running server - with sufficient documentation that defaults are not stored
    • provide a metadata facility to retrieve the effective defaults
  • no clear separation between configuration and runtime state. It sounds like almost any field could be runtime state, is that correct? If so, this leads to the kind of tracking that @slaskawi is proposing - where the full representation is the template that is managed separately
  • may be hard to reason about behavior across upgrades or feature changes - do we have some examples of what may be changing in these respective scenarios and when - minor version, major versions, etc.?
  • exports of the "full representation" could either be a powerful way to pin as much behavior as possible to a specific runtime state, or may need to be discouraged because they are not intended as a way to manage Keycloak.
  • related to the previous point, it's not well-defined how resources created / managed under AdminApi v1 (which will still include the admin console for several versions) will be represented in simple v2 get / list operations because the server is only tracking the "full representation". @vmuzikar is proposing some mechanism to compensate for this, but I'm not clear how we're able to differentiate after the fact whether somehing in v1 picked up a given default explicitly or implicitly.

@vmuzikar
Copy link
Copy Markdown
Contributor

Before going further into discussing potential final solutions, I wanted to summarize the two proposed approaches. @shawkins Let me know if I understood Option 1 properly. :)

Option 1: Resource versioning

We would introduce a new concept of resource versioning. A versioned resource would define not only the resource's interface (the representation) but more importantly also its behaviour incl. the default values. It would be basically a "model version", Keycloak internals would need to be able to work with different versions of given resource.

A resource version would be also reflected by the API version. Each resource would be managed by a different group of endpoints which would have a corresponding version of the API. I.e. we would not make breaking changes to the API unless we introduced a new resource (and API) version.

However, resource version is not affecting only the user facing contract, it's the model layer too. It also defines how Keycloak works with that resource.

Example

Let's have a Client v1. A user will decide to create a Client v1 in Keycloak 26.2. The Client has a well defined behaviour and also some defaults for fields that the user didn't specify.

In Keycloak 26.4, Client v2 was introduced which changes some default values of the Client, adds some fields, removes some. It is a breaking change. However, thanks to resource versioning, any Clients that are v1 are unaffected, e.g. the defaults remain unchanged. Users can even still create Clients v1 even in 26.4.

Next, Keycloak 27.0 is released and Client v1 is finally removed. Users only then need to perform some migration to v2 (if not performed for them automatically).

Considerations

Pros
  • Fully declarative. E.g. a Client v2 will be the very same Client v2 as long as given Keycloak version supports it.
  • Stable and predicable. No unexpected changes, "v2 is v2 no matter what".
  • 1:1 mapping between API version and the underlaying resource/model version.
  • Resource versioning might be a concept familiar to users coming e.g. from K8s environment.
Cons
  • Completely new concept to Keycloak, possibly outscoping the work around Admin API v2.
    • We'd e.g. need to figure out how to migrate from one resource version to another.
  • Partially clashes with the current concept around Features. Right now, Keycloak's behaviour is driven by Features which have versions and support level (preview, deprecated, etc.). Resource version would be an additional way of how Keycloak's behaviour is affected.
  • Challenging implementation. We'd introduce a new abstraction layer above the storage layer that would potentially have not insignificant "mapping" logic since there'd still be a single "storage version" – a single DB schema for all currently supported resource versions. This resembles the efforts around New Store back in the day.
  • It would not be possible to e.g. through API v2 access Client that was created through API v1.

Option 2: API versioning

We would version only the API which would be decoupled from the Resource/model version. Resources/model layer would not be versioned beyond Keycloak version. The API would be just a view and management layer above the model, i.e. the very same resource could be accessed by all supported API versions.

Additionally, default values would not be persisted and would not be tied to an API version but rather to Keycloak version. With a Keycloak upgrade, defaults might change as well, that includes existing resources.

Similarly to Option 1, we would not version the API as a whole but rather logical endpoints "groups", e.g. Client API v2.

For more details, see #39547 (comment).

Example 1

In Keycloak 26.4, the user created a Client via Client API v2. Some fields that were not specified had default values assigned. At the same time the user can access the very same Client via Client API v1. Since the model is the same, some runtime mapping might be in place to translate between the actual Client model and API v1 and v2 which might have e.g. different fields in the representations.

Then, Keycloak 27.0 is released and it changed some of the Client defaults. However, since the defaults are resolved at runtime, they changed for the existing Client (created in Keycloak 26.4) too.

Example 2

The user has some Client representation stored in Git and manages Keycloak in GitOps style. They are first using Keycloak 26.4 but then upgrade to 27.0 which introduced some changes to defaults as in the previous example. Thanks to the defaults not being persisted, the defaults are updated for the managed Client with the upgrade to 27.0.

(If the defaults were persisted, the user would have a discrepancy in the resulting Client based on whether it existed prior to 27.0, or not. That would break idempotency principle required for declarative style configuration like GitOps.)

Considerations

Pros
  • Simpler implementation, there's only a single "model version" being tied to Keycloak version.
  • Features remain the sole concept of versioning breaking changes in Keycloak's behaviour. A Client might have a different behaviour based on a Feature Flag.
  • The API versions are interchangable
Cons
  • The API version is a weak contract since the defaults can change with a Keycloak upgrade.
    • Potentially confusing to users.
    • We probably could not state the defaults on the OpenAPI spec as they would not be contractually binding.

While I do believe Option 1 is conceptually more correct, my main concern would be the implementation. Hence Option 2 seems more viable to me. The biggest issue is the defaults. But possibly we might be able to find some solution in wording it properly.

@shawkins
shawkins commented May 15, 2025 •
Copy link
Copy Markdown
Contributor Author

Let me know if I understood Option 1 properly. :)

Yes, I think it matches. I'll add some more comments here.

Resource versioning might be a concept familiar to users coming e.g. from K8s environment.

It might not be as relevant nor directly comparable, but I believe Wildfly functions similarly with explicit configuration schema, such that the structure, defaults, etc. remain consistent - https://github.com/wildfly/wildfly-core/tree/main/server/src/main/resources/schema

Partially clashes with the current concept around Features. Right now, Keycloak's behaviour is driven by Features which have versions and support level (preview, deprecated, etc.). Resource version would be an additional way of how Keycloak's behaviour is affected.

These don't necessarily have to be seen as in conflict - unless the only way a feature can affect the model is to change what is stored.

It would not be possible to e.g. through API v2 access Client that was created through API v1.

I don't completely follow why. There is a single storage model, and each serving version already maintains a bidirectional mapping storage model.

Option 2: 8000 API versioning

Can you incorporate more of the cons from #39547 (comment) into your post?

I want to comment more on this:

no clear separation between configuration and runtime state. It sounds like almost any field could be runtime state, is that correct? If so, this leads to the kind of tracking that @slaskawi is proposing - where the full representation is the template that is managed separately

This isn't specific to Keycloak representations. Kubernetes takes a lax approach to fully separating configuration and runtime state - spec fields are changed in what is stored / returned to the user. We do this in the operator for several effective defaults (replicas and hostname on openshfit). Kube can do this for defaults (which is mostly benign) but also to track the affects of things like LimitRange - this means that what you get back from a PUT, POST, event, etc. may contain truly runtime information in the spec that you did not directly specify. Since that is persisted, if you later change the LimitRange defaults for example, your Pods won't automatically pick that up.

This isn't necessarily a bad thing. Kube users are generally NOT supposed to directly use what is returned by the api server to overwrite / replace their managed copy of the resource (especially for resources like Pods), nor use that as the basis for additional manipluation. Instead you are to think in terms of asserting your desired state and let apply / server-side-apply (runtime / default fields should be un-owned) determine if anything needs to actually be done.

The Keycloak API Versioning proposal is doing something akin to the kubectl apply last applied tracking, but to make that a formal part of the api when getting the non-full representation.

This relates to mutability in general:

Allowing multiple actors to modify the same resource (not even considering modifications from different admin api versions) is difficult to reason over. From the perspective of maintaing a desired state having just tracking the last applied is insufficient - which is why server-side-apply was developed.

Can you clarify which of these most closely matches our goal for mutability:

  1. We're enforcing a concept of desired state. Clients created by an actor over AdminApi v2 cannot be changed via the Admin Console (regardless of whether that is using v1 or v2 under the covers) because that is effectively a different actor - I realize I'm being very loose here with how we're tracking actors.
    a. Does this apply to the field level? That is fields set and possible explicitly unset by the owning actor can't then be changed.
  2. There isn't an enforced notion of desired state, rather Keycloak is maintaining both a notion of a template and a working copy. If someone is separately maintaining resources like Clients, which may have initialized this state, they must propagate any changes made from the working copy into those.
    a. A reason to track the template / last applied state is because you will always want to sync to that.
    b. How does this work across AdminApi versions? A change made via v1 to a v2 created resource will need to somehow update the template / last applied.

Or are there other paradigms that would better describe it?

@vmuzikar
Copy link
Copy Markdown
Contributor

These don't necessarily have to be seen as in conflict

Yes, that's why I say "partially". :) Obviously these two can technically co-exist, my point was just we'd basically have two "feature versioning" concepts.

I don't completely follow why. There is a single storage model, and each serving version already maintains a bidirectional mapping storage model.

At the very least, there'd be the defaults problem. If v1 has some defaults, and v2 changed those, how would that work? I can see two possible approaches here.

  1. We persist defaults. E.g. I create a Client using v1, defaults were persisted to the storage layer. Then I fetch the Client using v2 which has different defaults, but I get v1 defaults (and not v2 default) as those are persisted.
  2. We do not persist defaults, and resolve them at runtime. E.g. Client created via v1 but fetch via v2 will have v2 defaults. However, this messes up with the model layer. Which version should Keycloak internals use – v1 or v2 defaults?

To me, the first approach seem better but not ideal.

Can you incorporate more of the cons from #39547 (comment) into your post?

As far as I understood, there were the following concerns:

  • OpenAPI contract. That's already included in cons.
  • "no clear separation between configuration and runtime state" @shawkins Could you please clarify what do you mean? My assumption was we'd still have some sort of flag (or a dedicated endpoint) to get the full representation, i.e. user specified config + the runtime defaults. Normally, only the user specified config would be returned. So there'd be a clear distinction.
  • Concerns about changing behaviour during upgrades or via feature flags. That might be both seen as a pro and con, IMHO. It follow our current paradigm in KC (breaking changes in major releases, otherwise guarded by a feature flag). On the other hand, it's not following the usual API practices e.g. from K8s world. @shawkins It's currently stated as a pro, do you want me to included in cons as well?
  • I see "full representation" spec as a good way to pin the behaviour. @shawkins Do you see it really as a con?
  • Migration from Admin API v1. Since v1 persisted the defaults (and then returned them on GET etc.), it feels natural v2 would treat them as user specified. We can't really know what was a true default, and what was specified by the user but was matching the default value.

This isn't specific to Keycloak representations. Kubernetes takes a lax approach to fully separating configuration and runtime state

This isn't necessarily a bad thing.

But it doesn't necessarily mean it's ideal approach either, IMHO. :) It's actually one of the usability issues I see – from just fetching the resource it's unclear to the user what is actually user specified (the "desired state") and what fell back to defaults.

Can you clarify which of these most closely matches our goal for mutability:

  1. We're enforcing a concept of desired state. Clients created by an actor over AdminApi v2 cannot be changed via the Admin Console (regardless of whether that is using v1 or v2 under the covers) because that is effectively a different actor

Yes, that is my current thinking now. That said, I see that as an additional layer, it's more authz related than API related, and we'll leverage FGAP for it. And it will be specific to the Operator (and potentially other GitOps-like consumers). I see that as a different discussion from defaults and versioning.

a. Does this apply to the field level?

Potentially yes, but we need to discuss that further.

  1. There isn't an enforced notion of desired state, rather Keycloak is maintaining both a notion of a template and a working copy. If someone is separately maintaining resources like Clients, which may have initialized this state, they must propagate any changes made from the working copy into those.
    a. A reason to track the template / last applied state is because you will always want to sync to that.
    b. How does this work across AdminApi versions? A change made via v1 to a v2 created resource will need to somehow update the template / last applied.

I'm not sure I fully follow. @shawkins Could you please provide perhaps some example?


Thinking about it further, maybe we could find some middle ground:

  • Model would not be versioned, Keycloak would internally work with a single resource version. So actual Keycloak behaviour would not be affected by a resource version. I.e. the resource version would not be persisted.
  • All the resource versioning would be done only at the API level at runtime.
  • As a results of the points above, it'd be possible to use different API/resources versions interchangeably. E.g. a Client created through v1 would be accessible through v2.
  • Defaults would be persisted and would be part of the contract for given version.
    • It would create the discrepancy when doing GET between versions as I mentioned above, but we'd document that.
    • Keycloak would keep track of the persisted values that are actually defaults. That means that I could "migrate" e.g. from v1 to v2 just by doing PUT of the existing Client through v2. Keycloak would know which fields were default in v1 and update them to new v2 defaults.

@shawkins How does that sound to you?

@shawkins
shawkins commented May 16, 2025 •
Copy link
Copy Markdown
Contributor Author

To me, the first approach seem better but not ideal.

The first approach seems the most consistent / workable and follows the reasoning used by Kubernetes.

"no clear separation between configuration and runtime state" @shawkins Could you please clarify what do you mean?

My next comment followed up on that - it's not a fair critism in isolation about having both "template" Client represenation and the "full representation", but I try to tie that into mutablity across actors or adminapi versions.

@shawkins It's currently stated as a pro, do you want me to included in cons as well?

Yes it should be, or at least it needs more specificity. If not then there doesn't seem to be clear expectations on how to maintain consistent configuration based behavior across upgrades, or exactly how feature flag changes affect things (immediately affect all resources, affect only newly created resources, have storage level side effects, etc.).

I see "full representation" spec as a good way to pin the behaviour. @shawkins Do you see it really as a con?

Again it's something that doesn't seem clear. We're making it a goal to maintain the template representation and are touting the benefits of the automatic management that is then happening, so we'll also have to document when / why you'd want to use these full representations to pin behavior.

The usability isn't clear - do you have to first create the entity, then GET the full representation, then save that as your managed copy and PUT it back into the server to assert the defaults are set? Or will there be some kind of short-cut for this - a POST with a "fill out full representation option", or would users be responsible for interogating the metadata facilities for defaults / runtime settings and filling out the full representation on their own before they POST? Or are you thinking that this is more applicable to migration scenarios?

Yes, that is my current thinking now. That said, I see that as an additional layer, it's more authz related than API related, and we'll leverage FGAP for it. And it will be specific to the Operator (and potentially other GitOps-like consumers). I see that as a different discussion from defaults and versioning.

It's related in that if we allow mutability to the "template" and expect the external actors (the operator) to assert their desired state, then they should be able to effeciently detect when to do that. If there is no mutability, then this problem largely goes away.

I'm not sure I fully follow. @shawkins Could you please provide perhaps some example?

Using @slaskawi 's PR as a mental model, let's suppose I create a Client with adminapi v2.

  • The server is tracking the template (what was sent in the post) and the "full representation" via the model.

  • If I make a PUT or a PATCH via adminapi v2 presumably that will overwrite or be applied to the template, then the model state will be updated accordingly - the template and "full represention" are fully syncronized and anyone wanting to maintain an externally managed copy of the Client can do so by getting the template.

  • If I make a PUT or a PATCH via adminapi v1 what happens?

    • Is the model state updated? If so what happens to the v2 template? Do we need logic to determine what affect the PUT / PATCH has on it? Does it become invalid / effectively revert to the full representation - as it would as if the Client were created of v1 in the first place. Or do we need to maintain a notion of what is set / unset in the model layer itself?
    • The modification should not be allowed

@vmuzikar
Copy link
Copy Markdown
Contributor

@shawkins Thank you for your reply. Before going further, I'd be interested in your thoughts on the "middle ground" proposal – see the end of my previous comment. :) I believe it would solve most of the concerns. As I'm contemplating the idea, I feel it might be the best path to take in the end.

@shawkins
Copy link
Copy Markdown
Contributor Author

@shawkins Thank you for your reply. Before going further, I'd be interested in your thoughts on the "middle ground" proposal – see the end of my previous comment. :) I believe it would solve most of the concerns. As I'm contemplating the idea, I feel it might be the best path to take in the end.

Sorry, I had started this response as something separate, I should have mentioned that in the other comment.

Keycloak would keep track of the persisted values that are actually defaults.

Can you elaborate more - are you saying that the model logic will be expanded to track what fields have been explicitly set? Thus a model "template" concept becomes a first-class citizen along side the working state / "full representation".

That means that I could "migrate" e.g. from v1 to v2 just by doing PUT of the existing Client through v2. Keycloak would know which fields were default in v1 and update them to new v2 defaults.

It might be good to break down the migration scenarios:

  • Existing Clients created under adminapi v1. My understanding here is that defaults have already been persisted, so there shouldn't be an automatic way to understand what was explicitly vs implicitly set. Or do you see a good auto-migration story here?

  • After these changes, new clients created via v1 could be migrated to v2 in a way that you are suggesting. However is that really necessary? This is effectively assuming that users want to use the work they've done via the v1 admin api / admin console as the basis for a "minimial v2" representation. This obviously excludes anyone who would use the full representation as their starting point.

More generally doesn't this boil down to how are we facilitating migration for users who externally manage their keycloak representations vs those managed via the admin console (or other ad hoc tooling).

For those that are externally managing, for example via gitops, won't they will start with v2 and have v1 (pending the mutablity discussion) be a legacy concern. Using Kuberenetes as a reference with externally managed resources - the migration of apiVersion can be as simple as just changing that field in your resources and doing a PUT - only if there are new required missing fields, validation is enabled and you are trying send removed fields, etc. is there additional migration effort. In any case that is all done externally, there is no expectation that Kubernetes platform will provide you with what the migration automatically.

So does all the thinking around the need for default tracking mostly come down to providing a "better" migration story than just using the full representation for those who are using the admin console / ad hoc tooling to manage their Keycloak instance?

@vmuzikar
vmuzikar commented May 16, 2025 •
Copy link
Copy Markdown
Contributor

@shawkins Your last comment got me thinking and I realized I confused even myself. :D I take back what I said about tracking defaults, we do not need that. Sorry about the confusion.

My main reasoning was that v2 PUT should update any v1 defaults, but it'll do it regardless. With PUT the whole representation is replaced, which implicitly includes default – no tracking needed. Similarly, any fields removed via v2 PATCH will use v2 defaults. The only inconsistency is with GET – if you use v2 API to fetch a resource that was previously created or updated via v1, you'll get v1 defaults, and vice versa. But we can live with that.

The only other bigger caveat would be the Admin Console – it would effectively treat any defaults as user specified, as it is doing now. But that should be ok, it's not a declarative-like config.

So are we reaching a consensus here? :)

  • Model would not be versioned, Keycloak would internally work with a single resource version. So actual Keycloak behaviour would not be affected by a resource version. I.e. the resource version would not be persisted.
  • All the resource versioning would be done only at the API level at runtime.
  • As a results of the points above, it'd be possible to use different API/resources versions interchangeably. E.g. a Client created through v1 would be accessible through v2.
  • Defaults would be persisted.
    • Updates/migration of changed defaults would happen only when updating the whole resource (or the affected fields) via a new API version.

@shawkins
shawkins commented May 16, 2025 •
Copy link
Copy Markdown
Contributor Author

@shawkins Your last comment got me thinking and I realized I confused even myself. :D I take back what I said about tracking defaults, we do not need that. Sorry about the confusion.

No problem. It's a confusing topic because so much has been discussed. Thank you for working through all this with me.

But we can live with that.

👍

But that should be ok, it's not a declarative-like config.

Great, that clarifies we're not trying to somehow make the server treat it as such.

So are we reaching a consensus here? :)

I think so - and hopefully saving ourselves a fair amount of work.

the resource version would not be persisted.

I think we can live with that. This is implicitly following the pragmatic approach with CRDs, which is to avoid multiple api versions at all costs. Kubernetes does provide differring openapi specs (1.30 vs 1.31, etc.) based upon the release version for the same core apiVersion - we'd likely do something similar.

We'll basically ensure:

  • changes should not be breaking. We can add fields, but removing can only be done after sufficient deprecation time.
  • if a breaking change seems necessary (field type change or renaming) it can only be done over multiple releases where support for both forms exist simultaneously and/or a feature flag is used to validate only the new form is used. For type changes the process requires that intermediate releases widen to a type that will cover both the old and new, then in a later release narrow to the new type.
  • that we are careful on what we treat as a defau 10000 lt in the openapi contract, or sufficiently document how they are to be interpretted

Does that seem correct?

Updates/migration of changed defaults would happen only when updating the whole resource (or the affected fields) via a new API version.

Just want to highlight more explicltly how this would work with patching. What I would like is for patching to function as a straight-forward composite operation. On a patch the server logically performs a GET, applies the PATCH, then PUTs the result.

If we create a v1 Client, then submit a patch against v2, that will first get the Client as if it were a v2, apply the patch, then go through the normal pipeline of a PUT (defaulting as if v2, validation, canonicalization). Given that v1 defaults have been persisted, you can end up with some kind of hybrid of v1 and v2 state - but I'm fine with that as that seems to be the closest semantic match to what the users is doing via a PATCH. If they want the full replacement with v2 defaults, they either have to expliclty unset the affected fields in the patch (as you mention above), or use a PUT instead.

@vmuzikar
vmuzikar commented May 16, 2025 •
Copy link
Copy Markdown
Contributor

No problem. It's a confusing topic because so much has been discussed. Thank you for working through all this with me.

Thank you for challenging my confused ideas. :D

Does that seem correct?

I think so. I'd just like to add explicitly that syntactical breaking changes would be done solely through introducing a new (sub-)API version (e.g. Client API v2). We'd also put some guardrails around when a new API version becomes the default and when the old one is removed – this is effectively a breaking change, so that would have to be done in Keycloak major releases. Semantical/behavioural changes would be on the other hand driven by Feature Flags.

Given that v1 defaults have been persisted, you can end up with some kind of hybrid of v1 and v2 state

Yes, that was my thinking as well and I'm fine with that.

@shawkins
shawkins commented May 16, 2025 •
Copy link
Copy Markdown
Contributor Author

I'd just like to add explicitly that syntactical breaking changes would be done solely through introducing a new (sub-)API version (e.g. Client API v2). We'd also put some guardrails around when a new API version becomes the default and when the old one is removed – this is effectively a breaking change, so that would have to be done in Keycloak major releases.

We may be saying the same thing, but I'd differentiate between the logical api version and the spec version. Borrowing from the kubernetes example again. There is a core v1 openapi spec (for things like pods) for Kube 1.30. There is a different one for Kube 1.31, etc. They all represent the same logical api version, which is core v1.

The contract is that everything that you can do in core v1 1.30 spec which is not deprecated (and most deprecated things stay that way for several releases), you can do using the core v1 1.31 spec. Of course anything newly added in 1.31 spec cannot be used on a 1.30 or earlier platform - with validation enabled you would see this as an upfront error.

So it may not be for every major Keycloak version, but we could have something similar where we release a new openapi spec that logically represents a non-breaking revision over the last N spec releases.

Or are you saying that you don't want to try to maintain logical api version compatibility, and instead will introduce multiple / incompatible schema that will be supported simultaneously? Beyond of course the split we'll already have with admin api v1 and v2.

@vmuzikar
vmuzikar commented May 19, 2025 •
Copy link
Copy Markdown
Contributor

@shawkins I think it sounds good but I'd maybe would not do the differentiation between API version and spec version – IMHO spec/representation is part of the API.

Based on what you said, I'd propose the following workflow leveraging at most one stable and one preview API version at a time.

  • [Keycloak minor releases] I need to do a non-breaking change the API, i.e. add a new field to the representation, add a new endpoint, etc.
    • Add it to the stable API version and clearly mark it as "Available since Keycloak X.Y".
      • If there's a preview API version, make the change there simultaneously as well.
  • [Keycloak minor releases] I need to do a breaking change to the API, i.e. change a default field value, remove or rename a field, change endpoint behaviour, etc.
    • If there is no preview version of the API, add it.
      • We might consider marking the stable API version as deprecated, although this might get us in a weird state when the stable API is deprecated and the preview not yet ready to be used in production.
    • Make the change in the preview version.
  • [Keycloak major releases] Release a new stable API version if needed.
    • If there is a preview API version, promote it to stable.
    • If there's been enough non-breaking changes in the stable API and no preview API exists, re-release the stable API as a new version (to get rid of many "Available since ..." labels).
    • If I need to do breaking changes that were not previously present in minor releases, release them right away as a new stable API version.
    • Remove the previous stable API. (SInce it is a major release, my understanding is we can make breaking changes even without deprecation period.)

@vmuzikar
Copy link
Copy Markdown
Contributor

Or are you saying that you don't want to try to maintain logical api version compatibility, and instead will introduce multiple / incompatible schema that will be supported simultaneously?

So basically yes, that is my proposal. :) To have at most two incompatible API versions at a time.

@shawkins
shawkins commented May 19, 2025 •
Copy link
Copy Markdown
Contributor Author

So basically yes, that is my proposal. :) To have at most two incompatible API versions at a time.

Ok, we can refine that as the goal with the understanding that we'll avoid breaking changes in favor of schema evolution as much as possible.

Wanted to comment more on tracking the api version on the resource - this doesn't introduce wholely different behavior than what we are discussing here. In fact it could just be contruct of the api layer and not stored. In Kuberentes the api version on the resource is used to:

  • allow for client side parsing to infer how tooling should handle the resource - the client needs to construct the resource url including the version. kubectl apply, fabric8, etc. is doing this under the covers
  • to allow webhooks and other server side logic to pass around just the resource and not have to seperately track path metadata

So having the api version on ghe resource gives us a path to having "default" / smart behavior via tooling. That is if you do ./kcadm.sh create clientv2.json - the tooling would parse the client to determine the api version, or if not set use the default, so that the user does not have to specifiy it on the command line.

cc @ssilvert @slaskawi @vmuzikar

@vmuzikar
Copy link
Copy Markdown
Contributor

@shawkins Including the "spec version" in the representation itself certainly makes sense from the API clients perspective. However, from the API itself it feels to me like an additional clutter – basically both the URL path and the payload would need to contain the version, if I understood correctly.

@shawkins
Copy link
Copy Markdown
Contributor Author

However, from the API itself it feels to me like an additional clutter – basically both the URL path and the payload would need to contain the version, if I understood correctly.

I wouldn't call it clutter necessarily - it's all about what you want to support.

  • To support client tooling or version aware server side handling having the api version on the resource makes sense.
  • If you want to include "effective defaults" in the openapi spec and they are indeed frequently changing, the only way that is understandable to me is via a versioning scheme like this - you'd be explicit that you are creating a client v2.1, v2.2, etc.

You also don't have to make it a required field - if left unspecified depending on the context it will be an error, or default to latest supported version.

@vmuzikar
Copy link
Copy Markdown
Contributor

support client tooling

This use case certainly makes sense to me.

version aware server side handling

That would imply we'd have two distinct versioning schemas: the API version (defined by the URL path) and the spec version defined in a representation field. I'd rather go for a more simpler and IMHO clear single versioning scheme. We version just the API which also includes the spec. We include the version in the URL path and might include it in the representation as well.

If you want to include "effective defaults" in the openapi spec and they are indeed frequently changing, the only way that is understandable to me is via a versioning scheme like this - you'd be explicit that you are creating a client v2.1, v2.2, etc.

Based on the discussion above, I would not consider the concept of "effective defaults" any more.

@vmuzikar
vmuzikar commented May 20, 2025 •
Copy link
Copy Markdown
Contributor

@shawkins Just wanted to revisit the "defaults tracking" concept as I realized we might need to consider it in the end. The reason I a bit forgot about is the ability to fetch a resource without the defaults, i.e. do an HTTP GET just with the fields that the user explicitly provided. Another alternative IIRC was to store the user provided representation alongside of the "full representation" with the persisted defaults. Neither is ideal and implies quite a lot of additional logic, though just tracking a list of fields that have the defaults feels a bit more robust to me than having to maintain "two sources of truth" in sync.

Now, the question is whether we really need to be able to fetch a resource without the defaults. The use cases I have in mind are basically two:

  • Some automation maintaining a declarative config state. The automation might want to check and compare if Keycloak is in the desired state. Though this might be solved by simply doing PUT rather than GET and then conditionally PUT, or just PATCH for selected fields the automation is interested in.
  • User experience when manually managing resources e.g. via CLI. It's very confusing and user unfriendly that a resource user created and fetched suddenly has e.g. 10 extra fields the user didn't specify. Which I do believe this is quite an important use case.

How real is the problem also boils down to how often/how many defaults we plan to persist. If we want to have robust design for the API, I do believe we'd rather be on the "persist as much defaults as possible" side of the spectrum rather than "let's leave most of the fields set to null and leave the interpretation on various Keycloak logics".

So to summarize, I'm afraid we might need the defaults tracking in the end. That said, the additional complexity is probably not worth the benefits. I'll capture it in some final design overview of the defaults as a known caveats and we'll also see the feedback on it.

@shawkins
Copy link
Copy Markdown
Contributor Author

Another alternative IIRC was to store the user provided representation alongside of the "full representation" with the persisted defaults.

As we discussed above this does not work unless you try to move the "template" logic into the model layer itself.

though just tracking a list of fields that have the defaults feels a bit more robust to me than having to maintain "two sources of truth" in sync.

How do either of these work with existing v1 entries?

Some automation maintaining a declarative config state. The automation might want to check and compare if Keycloak is in the desired state. Though this might be solved by simply doing PUT rather than GET and then conditionally PUT, or just PATCH for selected fields the automation is interested in.

See the discussion above about mutabilty - if you are allowing for mutabilty and expect actors to effectively engage in a reconciliation loop, then yes there ideally would be a way to efficiently determine when to assert your state. However in a multi-actor scenario just tracking just a single notion of the last applied / template state will be difficult to reason over. I believe this would lead us more towards something like server side apply where field ownership is tracked as well.

The questions here are:

  1. Do we want to support this type of mutability? In previous comments it seemed like your answer was no.
  2. If we do want to support it, do you also want to support multiple actors via something like server side apply?

User experience when manually managing resources e.g. via CLI. It's very confusing and user unfriendly that a resource user created and fetched suddenly has e.g. 10 extra fields the user didn't specify. Which I do believe this is quite an important use case.

Why is this a problem? This is a common user experience with databases or Kubernetes and it is generally understood as how they function. The question is what are you using the returned resource for? In the previous comment I was trying to say that Kuberentes users should not be using the result of a GET (or mutative operation) to overwrite their managed version of a resource or even as the direct basis for additional changes - for example simply modifying a field on it and PUTting it again. For one thing that defeats the purpose of SSA tracking as you become the owner of all fields. For another there's a lot of additional state - server generated fields in the metadata, status, etc. which are simply thrown away. You may also not be appropriately reasoning over locking / resource version - do you want to apply your change only based upon this revision (if so inheritting the resource version is the correct thing to do) or do you want to generally assert your change (in which case you need to clear the resource version field).

@slaskawi
Copy link
Copy Markdown
Contributor

To support client tooling or version aware server side handling having the api version on the resource makes sense.

@shawkins has a good point here. If we wanted to introduce aliases in the API paths, so for example the /admin/realms/master/clients would route to /admin_v2/realms/master/clients, having a resource version in the payload enables both the tooling and the Server recognize what's the target version. On the other hand, having this requires adding some sort of validation and throw an error if the version from path and payload do not match.

The question is what are you using the returned resource for?

@shawkins The most common use case is a reconciliation loop that manages this resource - for example an Operator. We need to provide a way for the Operator can ensure that its internal state matches the reality. Kube employs "generation" and "observedGeneration" fields for it. There are also examples of a full payload comparison, such as the MongoDB Atlas Operator.

In general, I prefer more the "generation" approach but this needs proper documentation and proper support in Keycloak Terraform Provider as well as the Operator.

@vmuzikar
Copy link
Copy Markdown
Contributor

How do either of these work with existing v1 entries?

They wouldn't. Any v1 defaults would be considered user specified. We might consider adding support for the defaults tracking retrospectively to the v1.

Do we want to support this type of mutability? In previous comments it seemed like your answer was no.

I'd say no. If we allow for declaratively managed resources to be managed by multiple actors, we'd probably end up with something over engineered like SSA.

This is a common user experience with databases or Kubernetes and it is generally understood as how they function.

This IMHO doesn't mean it's correct and the best possible experience. :) The only question is if the complexity to solve that is worth the benefits it brings.

@vmuzikar
Copy link
Copy Markdown
Contributor

The question is what are you using the returned resource for?

Possibly some declarative config automation for checking the current actual state, if for whatever reason it doesn't want to update it via PUT.
Or when a user decides to manage Keycloak via a CLI manually.

@shawkins
Copy link
Copy Markdown
Contributor Author

@shawkins The most common use case is a reconciliation loop that manages this resource - for example an Operator. We need to provide a way for the Operator can ensure that its internal state matches the reality. Kube employs "generation" and "observedGeneration" fields for it. There are also examples of a full payload comparison, such as the MongoDB Atlas Operator.

@slaskawi two things:

  • are we designing an eventually consistent system? That is would you describe this as a system where various actors are engaged in reconciliation loops to assert their desired state, such that only eventual consistency matters?
  • generation and observedGeneration work well from the prespective of single actor and allows for easy filtering of events you may not interested in related to status changes which will increment the resource version, but not modify the generation. However if another actor causes an increment in the generation, everyone else has reconciliation work to do.

This IMHO doesn't mean it's correct and the best possible experience. :) The only question is if the complexity to solve that is worth the benefits it brings.

Correct, I'm not convinced it's worth the complexity on the server side.

Using a database / sql as an example - if you do an INSERT into ... , followed by SELECT * - you will rightly see all the defaults. If you instead do a SELECT - you will see just what you explicitly inserted. In this paradigm, it's on the caller to project only what is meaningful to them. I'd rather start by making this a client-side problem, rather than try to implement nuanced and potentially complex handling on the server side.

@vmuzikar
Copy link
Copy Markdown
Contributor

it's on the caller to project only what is meaningful to them

Yes, I'm also not really convinced it's worth the complexity. And projection/fields filtering would certainly be a solution.

@slaskawi
Copy link
Copy Markdown
Contributor

are we designing an eventually consistent system? That is would you describe this as a system where various actors are engaged in reconciliation loops to assert their desired state, such that only eventual consistency matters?

@shawkins From the v2 Admin API perspective - no, we're not. But the client application might be implemented as such a system. Think of a fundamental assumption of a controller patter - if the system doesn't match the desired state, a reconciliation action is triggered. So an eventual consistent system is the actual client of the API and there needs to be something to enable it compare the state. It have to be either some clever mechanism (like generations) or there needs to be a way to return a previously submitted object from the API and compare it against the desired state.

generation and observedGeneration work well from the prespective of single actor and allows for easy filtering of events you may not interested in related to status changes which will increment the resource version, but not modify the generation. However if another actor causes an increment in the generation, everyone else has reconciliation work to do.

@shawkins The design can't assume a single actor. Imagine a Client is being created by an Operator using the API and then a Keycloak Admin modifies it. From this perspective, the only legal action for the Operator is to try to override the changes that the administrator did. This is exactly the Controller Pattern I described the above. The Operator now needs to have something to detect what changed.

@shawkins
Copy link
Copy Markdown
Contributor Author

From the v2 Admin API perspective - no, we're not.

Given that, the goal will be to not over engineer what is implemented on the server side. If clients wish to implement something that looks like multiple actors engaged in reconciliation loops over the server side state, then that is should be viewed as a client side problem to resolve.

The design can't assume a single actor.

Please the the exchanges with @vmuzikar on this topic. We will persue this as a design goal before entertaining more elaborate server side mechanics.

@vmuzikar
Copy link
Copy Markdown
Contributor

So, just to summarize the discussion above (and also some offline ones), the current proposal is as follows.

  • Defaults will be in the end persisted. The reasoning is:
    • We want to provide a stable spec contract. I.e. a given API version (which also includes the spec/representation) also covers the default values which do not change with Keycloak versions. Defaults can change only when the API version changes.
      • That will allow us to expose defaults via OpenAPI.
    • We do not want to introduce the concept of resource versioning. I.e. a resource would also persist its version and the model layer would based on that need to resolve the defaults at runtime.
      • This would mean lots of changes would need to be made to the model layer, significantly increasing the implementation effort while adding complexity to the Keycloak logic.
    • We want to support interchangeable usage of different API versions on a single resource. E.g. a Client created through API v2 would be accessible through v1.
      • This is to allow seamless migration to new API versions while keeping the possibility to downgrade if the new (preview) version is not working well for the user.
      • Defaults would be persisted based on the API version that persisted the resource. E.g. a Client created via v1, and later updated (PUT) via v2 will have v2 defaults, and vice versa.
      • Keycloak internally doesn't differentiate between the versions, it will use whatever defaults were persisted.
  • The downside of persisting defaults are:
    • It would not be possible to GET a resource without the defaults.
      • This mostly affects the declarative/reconciling scenarios where the client wants to compare the actual and desired state.
        • We believe, this is not strictly necessary. The client can simply do PUT rightaway (instead of doing GET and the comparison before that).
      • Another affected area is user UX. It is very confusing (especially in the CLI scenarios) that when a resource is created only with a handful of fields, it suddenly has many more fields due to the defaults after it's created.
      • This might be solved, e.g. by tracking the default fields so that they are excluded when doing GET. However, this adds some complexity and considering the limited scenarios that would benefit from it, it doesn't seem really worth it.
  • Any breaking changes would be done only through introducing a new API version.
    • What is a breaking change: fields renaming or removal from the representation, default values changes, endpoints removal or renaming or changing their existing behaviour.
    • What is not a breaking change: fields addition, adding a default for previously required field, adding a new endpoint or new HTTP vers, new parameters etc.
    • The intention is to keep the changes as the non-breaking as much as possible to avoid introducing new API versionss too often.
    • We'd introduce a process for rolling out a new API version. See adding some notes about v2 default handling #39547 (comment) for a proposal.
      • Before removing a stable API version in a major release, it would need to be deprecated for at least 1 minor release (including prod releases).

@shawkins @slaskawi Let me know if that makes any sense.

@shawkins
Copy link
Copy Markdown
Contributor Author

Defaults can change only when the API version changes.

I'm not sure about this based upon your past descriptions of when / why the effective defaults change.

Are you specifically referring to "simple defaults" here?

That will allow us to expose defaults via OpenAPI.

I wouldn't support this if this applies to the effective defaults previously described as changable based upon the "Keycloak version, feature enablement, and CVEs" - otherwise it seems like we're introducing what could be seen as breaking changes in the contract far too often.

At the very least we'd have to document that you must retrieve the open api spec from a running server to have an accurate reflection of the defaults, which may violate the principle of least surprise given taht we will have a published / static openapi spec at least per keycloak release correct?

The other path to address this (at least the idea that defaults can change per version) under an otherwise structurally compatible schema is what I was getting at with #39547 (comment) - that is to sub-version the open api specs, and to specify that on your resources - that gives me an understandable contract wrt published defaults.

What is a breaking change: fields renaming or removal from the representation, default values changes, endpoints removal or renaming or changing their existing behaviour.

While some of these are immediately breaking, such as changing the semantic meaning of an existing field (which should obviously be avoided), as mentioned in #39547 (comment) some otherwise breaking changes can be approached by schema evolution. In these cases there would be a temporal motivation to treat them as breaking - that is we want to accelerate the removal of the old schema structure rather than letting it evolve out.

Before removing a stable API version in a major release, it would need to be deprecated for at least 1 minor release (including prod releases).

That is probably too short of a guarentee.

@vmuzikar
vmuzikar commented May 22, 2025 •
Copy link
Copy Markdown
Contributor

Are you specifically referring to "simple defaults" here?

Yes, I thought we do not consider anything else at the moment. :) That the "effective defaults" are off the table which makes sense to me.

I wouldn't support this if this applies to the effective defaults previously described as changable based upon the "Keycloak version, feature enablement, and CVEs" - otherwise it seems like we're introducing what could be seen as breaking changes in the contract far too often.

As mentioned, I'm convinced we should not have "effective defaults".

some otherwise breaking changes can be approached by schema evolution

You mean the proposal to support both fields at the same time while deprecating the old one, and then removing the old one after a few releases?

That is probably too short of a guarentee.

Yes it might be but that is my current understanding of the Features lifecycle. If that's not the case, we should probably align with whatever that is. I see an API versions just as another Feature basically.

@shawkins
Copy link
Copy Markdown
Contributor Author

Yes, I thought we do not consider anything else at the moment. :) That the "effective defaults" are off the table which makes sense to me.

Great, we have a common understanding then. So before we mark anything as a default in the contract we'll ask ourselves - is there any reason why this would change anytime soon. And if the answer could be yes, then it's not a simple default.

You mean the proposal to support both fields at the same time while deprecating the old one, and then removing the old one after a few releases?

Yes, or type widening, then narrowing to support a type change.

@vmuzikar
Copy link
Copy Markdown
Contributor

Great, we have a common understanding then. So before we mark anything as a default in the contract we'll ask ourselves - is there any reason why this would change anytime soon.

Exactly.

And if the answer could be yes, then it's not a simple default.

I would not say a "simple default". Just it's not a default, period. We should not consider a default something we can't truly commit to (unless it's a preview field or something).

Yes, or type widening, then narrowing to support a type change.

Well yes but it'd need to be done very carefully on case by case basis. It'd would technically break the contract for a given API version.

@vmuzikar
Copy link
Copy Markdown
Contributor

Created the final proposal based on this discussion: #38551 (comment)

@shawkins @slaskawi Does it make sense to close this PR?

@shawkins
Copy link
Copy Markdown
Contributor Author

@shawkins @slaskawi Does it make sense to close this PR?

There are jackson and other changes here not related to defaults that I'd still like to apply. We can do that in a different pr if you want.

@vmuzikar
Copy link
Copy Markdown
Contributor

We can do that in a different pr if you want.

Yes, that's make sense. Thank you.

@shawkins shawkins closed this May 30, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

team/admin-api-wg Admin API v2 Working Group

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants

0