CI flakes under load: Chromium crash in the first e2e test, TLS handshake timeout

littlefox commented

2026-10-17 19:04:23 +00:00

Owner

Seen on forge's shared runners while several runs overlapped (2026-10-17):

e2e, first spec (aux.spec.ts): "Protocol error (Runtime.callFunctionOn): Internal server error, session closed" — Chromium died. Runs #75 and #95. Always the first test, so possibly start-up under memory pressure.
Integration test tls_front_end: NotTls("inget TLS-svar"), i.e. the 10 s TLS handshake timeout against the devserver's stunnel. Run #95.
The runner lost its Docker socket after all tests had passed ("dial unix /run/dind/docker.sock: i/o timeout"). Run #92.

All passed on re-run. Ideas:

e2e: a warm-up page load before the first spec, or Playwright retries=1 in CI only (reported as flaky, not hidden).
tls_front_end: a longer timeout in the test only (connect_with already takes parameters).
Fewer overlapping runs: D-034 (#72) already removes most packaging runs.

Also a product question raised by (2): a handshake timeout is reported as "no TLS", which then offers plain text. On lyskomd that's the only signal (it waits silently on a TLS ClientHello), but on a slow link a user could be offered plain text by mistake. Consider wording the prompt as "no TLS answer within 10 s" and offering a retry.

Seen on forge's shared runners while several runs overlapped (2026-10-17): 1. e2e, first spec (aux.spec.ts): "Protocol error (Runtime.callFunctionOn): Internal server error, session closed" — Chromium died. Runs #75 and #95. Always the first test, so possibly start-up under memory pressure. 2. Integration test tls_front_end: NotTls("inget TLS-svar"), i.e. the 10 s TLS handshake timeout against the devserver's stunnel. Run #95. 3. The runner lost its Docker socket after all tests had passed ("dial unix /run/dind/docker.sock: i/o timeout"). Run #92. All passed on re-run. Ideas: - e2e: a warm-up page load before the first spec, or Playwright retries=1 in CI only (reported as flaky, not hidden). - tls_front_end: a longer timeout in the test only (connect_with already takes parameters). - Fewer overlapping runs: D-034 (#72) already removes most packaging runs. Also a product question raised by (2): a handshake timeout is reported as "no TLS", which then offers plain text. On lyskomd that's the only signal (it waits silently on a TLS ClientHello), but on a slow link a user could be offered plain text by mistake. Consider wording the prompt as "no TLS answer within 10 s" and offering a retry.

littlefox added the

area/ci

kind/infra

labels

2026-10-17 19:04:23 +00:00

littlefox referenced this issue from a commit

2026-10-17 19:04:26 +00:00

Re-run CI (run #95: flakes under load, see #76)

littlefox referenced this issue

2026-10-17 19:14:41 +00:00

Tell "no TLS answer in time" apart from "no TLS"; offer a retry #77

littlefox referenced this issue from a commit

2026-10-17 19:14:41 +00:00

Tell "no TLS answer in time" apart from "no TLS", and offer a retry

littlefox referenced this issue

2026-10-17 21:41:20 +00:00

0.2: errors, command palette, first-run tour, reconnecting #84

littlefox commented

2026-10-17 21:41:21 +00:00

Author

Owner

Another flake, WebKit (iphone profile, local run): tls.spec 'no TLS where there was TLS before is a warning' didn't show the dialog within 15 s once; passed 3/3 on repeat. Also seen once: Chromium crash in the first spec (aux.spec) on forge runners under load.

littlefox referenced this issue from a commit

2026-10-17 21:59:06 +00:00

Re-run CI (run #124: Chromium crash in the first spec, #76)

littlefox referenced this issue

2026-10-17 21:59:07 +00:00

Relative times ("2 h sedan"), exact on tap and hover #86

littlefox referenced this issue from a commit

2026-10-17 21:59:07 +00:00

e2e: date checks via the tooltip; one retry in CI

littlefox referenced this issue from a commit

2026-10-17 22:17:07 +00:00

e2e: swipe again until a post shows; one retry in CI

littlefox referenced this issue from a commit

2026-10-17 22:18:04 +00:00

e2e: 120 s per test in CI

littlefox commented

2026-10-18 01:46:33 +00:00

Author

Owner

Closing: the causes are understood and handled.

'Chromium crash in the first test' / 'session closed': starting a browser on a loaded shared runner took 50 s or more, counted against the first test's 60 s. CI now allows 120 s per test and one retry (retried passes are reported as flaky). Run #161: 169/169 without retries.
tls_front_end handshake timeout: the test allows 30 s and logs the handshake time.
Runner losing its Docker socket after the tests passed: forge infrastructure, outside the repo; an empty commit re-runs.
Also fixed on the way: the soak test counted a post that wasn't made (#66), and screenshot references depended on spec order (#65).

Closing: the causes are understood and handled. - 'Chromium crash in the first test' / 'session closed': starting a browser on a loaded shared runner took 50 s or more, counted against the first test's 60 s. CI now allows 120 s per test and one retry (retried passes are reported as flaky). Run #161: 169/169 without retries. - tls_front_end handshake timeout: the test allows 30 s and logs the handshake time. - Runner losing its Docker socket after the tests passed: forge infrastructure, outside the repo; an empty commit re-runs. - Also fixed on the way: the soak test counted a post that wasn't made (#66), and screenshot references depended on spec order (#65).

littlefox closed this issue

2026-10-18 01:46:33 +00:00

Rows
Columns

CI flakes under load: Chromium crash in the first e2e test, TLS handshake timeout #76