Screenshot comparison in CI #65

New issue

Closed

opened 2026-10-17 17:54:17 +00:00 by littlefox · 1 comment

littlefox commented

2026-10-17 17:54:17 +00:00

Owner

Today screenshots are looked at by eye. With a phone layout (#50) and four themes (#56), a layout regression would easily slip past.

What:

Playwright's toHaveScreenshot for key screens: login, unread list, reading a text, composer, settings, at each project size from the phone-size issue.
The devserver seed and a fixed clock and font keep the images stable.
Updating baselines is a deliberate step (tools/e2e.sh --update-snapshots), reviewed in the PR.

Done when:

A changed layout fails CI with a diff image in the job's artifacts.
Ten runs in a row of an unchanged tree are green (no flakes).

From docs/ROADMAP.md, "Under the hood"; filed for 0.2.

Today screenshots are looked at by eye. With a phone layout (#50) and four themes (#56), a layout regression would easily slip past. What: - Playwright's `toHaveScreenshot` for key screens: login, unread list, reading a text, composer, settings, at each project size from the phone-size issue. - The devserver seed and a fixed clock and font keep the images stable. - Updating baselines is a deliberate step (`tools/e2e.sh --update-snapshots`), reviewed in the PR. Done when: - A changed layout fails CI with a diff image in the job's artifacts. - Ten runs in a row of an unchanged tree are green (no flakes). From docs/ROADMAP.md, "Under the hood"; filed for 0.2.

littlefox added this to the 0.2: Feels like an app milestone

2026-10-17 17:54:17 +00:00

littlefox added the

kind/infra

area/ci

area/ui

labels

2026-10-17 17:54:17 +00:00

littlefox referenced this issue from a pull request that will close it,

2026-10-18 00:44:05 +00:00

Screenshot comparison in CI; soak test fix #89

littlefox closed this issue

2026-10-18 01:05:33 +00:00

littlefox referenced this issue from a commit

2026-10-18 01:05:34 +00:00

Screenshot comparison in CI for key screens

littlefox referenced this issue from a commit

2026-10-18 01:22:13 +00:00

Screenshot comparison runs first, on the freshly seeded server

littlefox commented

2026-10-18 01:46:32 +00:00

Author

Owner

Merged in #89, made order-independent in #87: _visual.spec.ts runs first in each profile on the freshly seeded server; login, reading, writing and the palette against references per profile. CI renders identically to local runs (run #161 passed the comparisons). Update: tools/e2e.sh _visual --update-snapshots.