8000 test(browser): stop browser scenarios flaking on slow external requests and LiveView joins by mwoods79 · Pull Request #700 · huddlz-hq/huddlz · GitHub
[go: up one dir, main page]

Skip to content

test(browser): stop browser scenarios flaking on slow external requests and LiveView joins - #700

Open
mwoods79 wants to merge 5 commits into
mainfrom
micah/stabilize-browser-ci
Open

mwoods79 wants to merge 5 commits into
mainfrom
micah/stabilize-browser-ci

Conversation

@mwoods79
@mwoods79 mwoods79 commented Nov 7, 2026 •
Copy link
Copy Markdown
Contributor

The Chromium browser job failed 11 times in its last 100 runs, each time with one scenario timing out, and a different one most times. The suite passes reliably locally, so the failures came from timing on GitHub's runners. Two causes:

Google Fonts holding the load event (8 of 11 failures). Every page links the Google Fonts stylesheet, and visit waits for the load event with phoenix_test_playwright's 2 s default timeout. In the failing traces the app answered in ~250 ms, but fonts.googleapis.com occasionally took ~2 s. Each scenario's browser context now answers every https:// request locally, so the suite no longer depends on Google at all. The Google Fonts stylesheet is answered with Inter's Latin subset, served from test/fixtures/fonts (48 KB, SIL Open Font License), so text measures as it does in production. An empty stylesheet let CI's wider Linux fallback font wrap card text and break the overview card layout check. Everything else, such as Maps embeds, gets an empty response; those were getting 403s with the test key anyway.

Interacting before LiveView joins (member menu failures). visit returns on load, before the LiveView socket connects. On a slow runner the member menu step pressed Enter before the join render, which reset the menu. Steps now use a BrowserHelpers.visit/2 that also waits for the [data-phx-main] root to be phx-connected, and await_live_view/1 covers full page loads that visit didn't make. With page loads now faster, the email confirmation scenario showed the same race after its plain form post, so it calls await_live_view/1 there.

Notes for review:

  • Playwright resolves a bare https://** glob to https://**/, which only matches URLs ending in /, so the route uses https://*/**. That's commented in the hook.

  • The fast (non-browser) suite has its own intermittent failure in test/features/email_confirmation.feature ("Retrying a failed send does not claim an email was sent"). It's untouched here and passes when run alone.

  • Also updates ash_ai to 1.5.1: a new advisory against 1.5.0 (EEF-CVE-2026-119613) fails the dependency audit, on main too, so this PR couldn't go green without it.

Every page links the Google Fonts stylesheet, and visit waits for the load
event with PhoenixTest Playwright's 2 second default. On CI that
stylesheet occasionally took close to 2 seconds, so a different scenario
timed out on navigation each time: 8 of the last 11 failed browser jobs.

Route every https request in each scenario's browser context to an empty
local response. Pages render with the fallback font, Maps embeds no longer
call Google, and the suite runs about 40% faster locally.
PhoenixTest's visit returns on the load event, before the page's LiveView
joins. On a slow CI runner the member menu scenario pressed Enter on the
trigger before the join render, which then reset the menu; the trace shows
the mount diff arriving 25 ms after the key press.

Steps now use a BrowserHelpers.visit/2 that also waits for the
[data-phx-main] root to be phx-connected, and await_live_view/1 covers
full page loads made another way. The email confirmation scenario needs
it after its plain form post, which the faster page loads from stubbing
external requests exposed.
With Google Fonts answered by an empty stylesheet, CI's Linux fallback font
is wider than Inter. The RSVPs card hint wrapped to two lines, which
stretched the overview row and left the Show rate sparkline more than 24 px
above its card's bottom edge.

Answer the Google Fonts stylesheet with Inter's Latin subset and serve the
font file from test/fixtures/fonts, so scenarios measure text as production
does without contacting Google. Inter is under the SIL Open Font License,
included alongside it.
…er-ci

* origin/main:
  chore(deps): ignore assent advisory instead of overriding assent
  chore(deps): override assent to 0.3.2 for EEF-CVE-2026-117656
  test(auth): sign in again after setting a password in profile scenarios
  chore(deps): patch ash_authentication, ecto and ash_sql advisories
  chore(ci): restore dedicated test step
  style(test): apply formatter output for query assertion
  test(deps): check saved search time zone independently of other filters
  style(test): wrap query parameter assertion
  test(deps): use supported PhoenixTest query assertion option
  chore(deps): remove temporary refresh verification workflow
  chore(deps): apply resolved stable lockfile updates
  chore(deps): refresh stable runtime and resolve dependencies
The dependency audit now flags ash_ai 1.5.0: it sends unrecognized tool
errors verbatim to MCP clients and the model. 1.5.1 fixes it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

0