Repository navigation
Replies: 8 comments 10 replies
|
@abstractj thanks for posting this discussion! I think that old images purging is the outstanding pain point we would need to tackle first on this subject. |
|
@andreaTP the overall idea is to create an extra layer with the Keycloak base image and use it in the Keycloak Server image. In this way, for every new CVE we only have to update the base image, and the tag + SHA will remain unchanged in the Keycloak server image. |
|
I'm lost 😢, even creating a new/separate base image on a new layer you would need to update the final image for the changes to take effect, as far as I know there is no way to update layers independently. |
|
@andreaTP you are correct considering the immutability of those images :). I completely misunderstood the conversation that I had with @ASzc. I don't see any other alternative other than agree on the format and frequency of those builds, as you suggested. Another thing that would be nice, is to have a changelog for the changes between those builds. |
|
A changelog would be great! |
|
Sorry I wasn't clearer Bruno |
|
We wrote a blog post about a similar topic (creating a custom keycloak image with zero cves) a week ago: https://blog.codecentric.de/en/2022/05/keycloak-x-but-secure-without-vulnerable-libraries/ We were able to create a CVE free keycloak image (according to Aquasec and whitesource scanners) based on alpine: Regarding the current keycloak image: |
|
Re-spinning images should be done when needed, and not on a regular cadence. For RH-SSO images we have tools like FreshMaker that takes care of re-spinning images as needed. I'd suggest talking to @ASzc, @pskopek or @hmlnarik on that. If I'm not fully mistaken here RH-SSO images are re-spun when the UBI images are re-spun. When the image is re-spun the ideal would be to have an incremental build number suffix (18.0.0-1, 18.0.0-2, etc.) rather than a date. That's slightly harder as we'd need to either manually input the number, or have the workflow do it automatically which would be best. An additional thing to consider here is should we do a microdnf update ourselves, or fully rely on UBI images to be re-spun as needed. The latter would allow us to release new KC images when there are new UBI images released. I would imagine the team managing the UBI images are responsible when it comes to updating those images when needed. One last thing to mention as well it's not just about re-spinning whenever there is a CVE. There's such a magnitude of false positives reported by CVE scanners, and in addition many true positives that doesn't affect Keycloak at all. |
|
PR to enable SHA pinning as discussed: |
|
Opened a draft PR to automate the images re-spinning in GH action: |
|
Hi @stianst , from what I understand of Red Hat Container Image Updates for UBI, it would be great to have that automated approach of respinning when a new UBI base image is available. That way, critical base image fixes would find their way immediately into a Keycloak image, as new UBIs are immediately built when a new version of RHEL is released with critical security fixes. Lower severity security fixes land in new UBI images regularly every six weeks. It would be great if Keycloak would reflect this great level of security fixes automatically given with UBI base images! |
|
The seemingly corresponding #12144 is closed, but I'm not sure it was merged? I'm asking because the current 18.0.2 image has a critical security issue "RHSA-2022:5683: java-11-openjdk security, bug fix, and enhancement update (Important)" that would be fixed in the underlying UBI image: https://quay.io/repository/keycloak/keycloak/manifest/sha256:47962e880d372dc14273e271afc2869b38a512de17e32f3e2eb66ebde3dba710?tab=vulnerabilities |
|
The thread's subject suggests the discussion is about remediation of CVEs, however, it digressed to the pinning aspect for those who appreciate stable tags. I'd like to add my perspective on it, as I could only find what I want in @thomasdarimont 's reply (thanks a lot for that). Keycloak is one of many softwares we use in our infrastructure(s) and the only chance to manage such a multitude is automation (CVE scanners). Therefore we pull everything through our harbor registry (which comes with trivy by default) and get alerted about CVEs (latest when we try to deploy). From what I've read in some issues/discussions here, you also check with trivy (and other scanners) on a regular basis and you pointed out that a lot of false positives are reported by these vulnerability scanners. Since we're also working in software development, we are well aware of that and we also maintain suppression lists after we analyzed the reports. Now here's my problem: There are two possibilities (I see) to use keycloak and sleep well at night.
About the pinning aspect – I understand that some folks prefer stability over security, but I am not one of those. I'd rather shut the entire thing down with no return until there's at least a workaround. I'm not saying there shouldn't be this pinning option for those who like it, but I personally wouldn't use that for the reasons mentioned above. If there's no other option than manually selecting the latest patch version on every update I deploy, I can live with it, though. |
|
We are in a similar position as @tumbl3w33d.
Exactly. I understand that it is a lot of work. Should we unite our development efforts for this?! |
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
Summary
Today, we publish the Keycloak container images on Quay.io and DockerHub using GitHub actions. From time to time, we trigger manually the rebuild of those images based on the latest tag to update container dependencies for known vulnerabilities.
Motivation
Regularly update container images is a security best practice, and necessary to provide to our community the latest bug fixes. The current workflow of rebuilding, overriding and having SHAs purged is not ideal considering the impact on customers and our community.
Current workflow
Proposal
Proposal 2 (from @andreaTP) - Discussion in progress
The simplest solution we can put in place is to add a tag to the image containing the date, those images will not be overwritten and enable SHA pinning for the users requesting it.
Proposal 3
Something on top of what Andrea suggested. Creating images with the date format suffix makes a lot of sense, at the same time it will generate a lot of noise into the registry, so we need to agree on the frequency. So, we have a few options:
Build those images on the weekly basis, appending the date metadata suffix to those images, and link it to latest.
Introduce a new container tag like
18.0.0-nigthly. The image would be the outcome from the nightly build of Keycloak 18.0.0 for example, with CVE updates. That means being able to override18.0.0-nightlykeeping the tag 18.0.0 without any SHA changes.I'd appreciate any input about it, so we can improve the current process.
Relevant PRs
All reactions