Stable screenshots

Find flaky snapshots#

A changed snapshot shows "Looks flaky" in the build page when its image changed at least twice over the last 10 builds on the baseline branch and went back to an image it had before. It also shows when another build of the same commit has a different image, for example after a CI re-run. Fix the cause with the steps below, then approve.

Render in the same place#

Screenshots render in your CI, so fonts and anti-aliasing depend on the machine. Take them in the same environment every time, for example the Playwright Docker image with the same Playwright version as your project. Do not mix screenshots from your laptop with screenshots from CI.

Freeze the page#

import { test } from "@playwright/test";
import { snapshot } from "stateofpixel/playwright";

test("dashboard", async ({ page }) => {
  await page.clock.setFixedTime(new Date("2026-01-01T10:00:00Z"));
  await page.goto("/dashboard");
  await page.getByRole("table").waitFor();
  await snapshot(page, "App/Dashboard");
});

• snapshot() from the Playwright integration already waits for fonts, disables CSS animations and transitions, and hides the caret.
• Fix the clock with page.clock so dates and relative times do not change.
• Wait for the content you want, not for a fixed time.
• Replace random or live data, like avatars, ads and charts of today's numbers, with fixed test data, or hide it with CSS before the screenshot.
• For Storybook, use --wait-for-selector and --delay for stories that render late.

Tune the threshold#

Three project settings decide when a snapshot counts as changed. Anyone with write access finds them under Settings, Diff:

• Threshold, from 0 to 1, default 0.1. How different a pixel's color has to be to count. Higher ignores more.
• Count anti-aliased pixels as changes, off by default. Anti-aliased edges are ignored unless you turn it on.
• Changed pixels to ignore, from 0 to 10,000, default 0. A snapshot with this many changed pixels or fewer counts as unchanged. A few pixels absorbs edge noise. Larger values can hide a small real change, like a new icon.

--threshold and --max-diff-pixels on the CLI, or threshold and maxDiffPixels on the Playwright reporter, override the settings for that upload. To try values before you change them, compare two folders on your machine:

npx stateofpixel compare screenshots baseline --threshold 0.2
# Writes stateofpixel-report/index.html

Mark snapshots unstable#

When you cannot fix a snapshot yet, mark it unstable so its changes stop blocking the check. Changes to an unstable snapshot are approved automatically. In the build page snapshot list they move to a collapsed Unstable group, so the Changed group only lists changes that need a look, and the snapshot says "Approved automatically, marked unstable". New snapshots with a matching name still wait for review.

• On the build page, open a changed snapshot and click Mark unstable. It adds the story name, without the browser and width in brackets, so every variant of the story is covered.
• Under Settings, Diff, Unstable snapshots lists one name per line. A line matches a story name or a full snapshot name, and * matches any text without /, so Charts/* covers every story under Charts. Delete a line to make the snapshot need review again in later builds.

Either way, the change also applies to builds that already wait for review: changes to matching snapshots in the latest build of each branch and open PR are approved in the background, and the GitHub checks update. Deleting a line does not undo approvals that were already made.

The approved image becomes the baseline like any other approval, so the build page still shows how the snapshot changed. Anyone with write access can change the list, up to 100 lines.