8000 Remove testing groups in CI to limit concurrent jobs by andreaTP · Pull Request #9917 · keycloak/keycloak · GitHub
[go: up one dir, main page]

Skip to content

Remove testing groups in CI to limit concurrent jobs - #9917

Closed
andreaTP wants to merge 2 commits into
keycloak:mainfrom
andreaTP:less-concurrency
Closed

andreaTP wants to merge 2 commits into
keycloak:mainfrom
andreaTP:less-concurrency

Conversation

@andreaTP
@andreaTP andreaTP commented Feb 1, 2022
Copy link
Copy Markdown
Contributor

We are currently completely flooding the GH Action runners, this PR should reduce the number of concurrent jobs and, eventually, leave more bandwidth to effectively run the checks for all of the PRs.

cc. @hmlnarik since looks like this matrix was introduced by you.

@andreaTP
andreaTP requested review from hmlnarik and stianst February 1, 2022 13:56
@andreaTP
andreaTP commented Feb 1, 2022
Copy link
Copy Markdown
Contributor Author

The current Queue has 13 Workflows waiting (some since more than 6 hours) and only 4 in-progress.
This change removes 8 workflows from each run (from 12 down to 4).

@hmlnarik
hmlnarik commented Feb 1, 2022
Copy link
Copy Markdown
Contributor

Please introduce GH issue and provide analysis of expected savings before filing a PR.

According to the previous successful builds, preparation before running each test group takes 2-3 minutes, most of the time is spent in the tests. It seems that this PR expects saving of 8x 2-3 minutes of total running time at the cost of getting the results in longer total time, correct? For example, in the case of wildfly from this run, the results would be available in appx 2 + 43 + 46 + 41 = 132 rather than max(2+43, 2+46, 2+41) = 48 minutes - is this calculation correct?

Side note: The Build phase should terminate in not half an hour without and ~ 15 minutes with a cached maven repository. In this PR, it is already over 90 minutes and counting, and there are builds in the queue with build phase running for over 5 hours. Today does not seem to be a representative day for GitHub Actions performance.

@andreaTP
andreaTP commented Feb 1, 2022
Copy link
Copy Markdown
Contributor Author

In this PR, it is already over 90 minutes and counting

That's exactly the problem, the job got queued for a crazy amount of time(e.g. hours) and the same is happening to most PRs.
Tomorrow I'll follow up with the formalities.
If this is just a symptom of GH Actions problems I'm happy to close this one!
I'll be happy to explore alternatives!

Thanks for the review and insights @hmlnarik !

@hmlnarik
hmlnarik commented Feb 1, 2022 •
Copy link
Copy Markdown
Contributor

That's exactly the problem, the job got queued for a crazy amount of time(e.g. hours) and the same is happening to most PRs.

The job is not queued but running for over 140 minutes. Feel free to check the console to validate

@andreaTP
andreaTP commented Feb 2, 2022
Copy link
Copy Markdown
Contributor Author

The job is not queued but running for over 140 minutes. Feel free to check the console to validate

Confirm, it was a network hiccup.

Please introduce GH issue and provide analysis of expected savings before filing a PR.

This PR resolves #9927 that, I believe, is still valid.
No issues in closing both issue and PR if the current behaviour is acceptable.

@hmlnarik hmlnarik left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe this PR really helps to mitigate the problem, the gain is too low for the cost. Let's take step back and discuss the right approach on the original issue #9927.

@hmlnarik hmlnarik linked an issue Feb 2, 2022 that may be closed by this pull request
@andreaTP
andreaTP commented Feb 2, 2022
Copy link
Copy Markdown
Contributor Author

@hmlnarik let's discuss on the issue, I move back this PR to draft but I let the CI run to(hopefully) have numbers to compare.

@andreaTP
andreaTP marked this pull request as draft February 2, 2022 09:59
@stianst stianst left a comment
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see how this helps at all. Sure, it may mean we have more PRs running in parallel, but that doesn't mean the waiting time for completion will be shorter. What I think the result of merging this will be:

  • No real difference when there's loads of contention
  • Longer total running time when there isn't contention

@andreaTP
andreaTP commented Feb 2, 2022
Copy link
Copy Markdown
Contributor Author

This PR was opened aiming to try this approach out, evaluate the results and iterate, since we are not confident experimenting in this direction I close it. 👍

@andreaTP andreaTP closed this Feb 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve CI setup

3 participants

0