8000 fix: initial implementation of memory-safe streaming by shawkins · Pull Request #49028 · keycloak/keycloak · GitHub
[go: up one dir, main page]
Skip to content

fix: initial implementation of memory-safe streaming - #49028

Draft
shawkins wants to merge 1 commit into
keycloak:mainfrom
shawkins:iss48651
Draft

fix: initial implementation of memory-safe streaming#49028
shawkins wants to merge 1 commit into
keycloak:mainfrom
shawkins:iss48651

Conversation

@shawkins
@shawkins shawkins commented May 15, 2026
Copy link
Copy Markdown
Contributor

closes: #48651

What we're doing here:

  • running the streaming operation in "batch" mode
  • inhibiting the creation of managedentities
  • periodically clearing the persistence context

This should allow us greater lattitude in setting reasonable maximums for the number of resources that can be retrieved via admin api endpoints.

relates to #47611 - the logic on either end is still based upon an array in the payload, so clients that lack a streaming parser will still have a problem with large results. This same logic though is applicable to NDJSON or other streaming schemes.

Assumptions:

  • that the operation is read-only and there's no need to create nested entitymanagers
  • that not much, if anything, will be shared across the resources such that periodically clearing persistence context while not having the managed entities won't result in a lot of re-fetching. I believe this is the case with what we're doing with clients currently.

Next steps:

  • something like streamInBatch could be added to existing endpoints that return a stream. Not sure if it's general enough of a concern, or if it could / should be done in a request filter.
  • if we do expand usage, then we'll need to more pervasively inhibit adding managed entities, the changes here only account for what v2 Clients is accessing

@shawkins shawkins mentioned this pull request May 15, 2026
Closed
@shawkins
fix: initial implementation of memory-safe streaming
3b34f6a 
closes: keycloak#48651

Signed-off-by: Steve Hawkins <shawkins@redhat.com>
@shawkins shawkins mentioned this pull request Aug 25, 2026
Closed
1 of 2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot
Copilot will automatically review once the pull request is marked ready for review

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

team/cloud-native
4C01

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Ensure endpoints returning streams are memory safe

2 participants

@shawkins @stianst
0