Repository navigation
fix(bridge): fail a missing image snapshot in CI mode instead of writing it - #249
YevheniiKotyrlo wants to merge 5 commits into
Conversation
|
@YevheniiKotyrlo is attempting to deploy a commit to the Callstack Team on Vercel. A member of the Team first needs to authorize it. |
dfeedaf to
41af5b7
Compare
…ing it In Jest's CI mode, toMatchImageSnapshot fails when its snapshot does not exist and saves the received image beside it as <name>-actual.png, where it wrote the snapshot and passed. The Jest session hands the bridge Jest's updateSnapshot state; a bridge context that names none writes a missing snapshot as before. The received and diff images take the whole snapshot name, and toMatchImageSnapshot refuses .not, which would turn the failure into a pass. The playground commits the baselines CI's Android emulator, iOS simulator and Chromium capture for its screenshot and out-of-bounds specs and ignores the failure images saved beside them. The UI testing guide and the CLI's UI skill say how snapshots are created in CI mode, and the CLI reference lists --ci.
659f68b to
ad57c9e
Compare
# Conflicts: # packages/bridge/src/__tests__/image-snapshot.test.ts # packages/bridge/src/image-snapshot.ts # packages/bridge/src/server.ts # packages/runtime/src/expect/matchers/toMatchImageSnapshot.ts
…ounded edges (#298) ## What is this? The playground's E2E Android suite fails `should screenshot specific element only` on every pull request since Oct 22: the element screenshot is 262×262 px, and the test expects 263×263. This fixes the test, which was wrong. The capture is correct. Fixes #297. ## How does it work? Android region captures round each edge of the region to a whole pixel (#289), so they cover exactly the pixels the element covers on screen. The assertion rounded the size instead: on the CI emulator (`PixelRatio` 2.625) a 100 dp element is 262.5 px, which rounds to 263, but its edges at 408.75 px and 671.25 px round to 409 and 671, so it covers 262 px. The test now measures the element with `measureInWindow` and expects the size its rounded edges span. Where the pixel ratio makes element edges whole pixels, as on the iOS simulator and in Chromium, this is the same size as before. ## Why is this useful? E2E Android goes green again on pull requests, so it can catch real regressions again, starting with #249 and #257, which are blocked on this failure today.
Since callstackincubator#288, toMatchImageSnapshot saves the received image of a missing snapshot in CI mode as __diff_output__/<name>-actual.png, as the UI testing guide says. The version plan and the CLI's UI skill still said it is saved beside the snapshot.
|
The playground's web suite in CI mode, headless Chromium on Linux:
The two snapshots To run it: With the baselines committed, this PR's own E2E run on |
|
Hey @V3RON! Sorry for the noise, what do you think about this changes? happy to apply your suggestions. |
What is this?
In Jest's CI mode (
--ci, on by default when Jest detects a CI environment),toMatchImageSnapshotnow fails when its snapshot does not exist, and saves the received image as__diff_output__/<name>-actual.png. Until now it wrote the snapshot and passed on CI too, so a run without committed snapshots compared nothing and a wrong capture could not fail it (#198). WithCI=trueand no committed snapshots, the playground's web suite passes onmainand writes both specs' snapshots; on this branch the two specs fail with:How does it work?
The session hands the bridge Jest's
updateSnapshotstate. When it is"none", CI mode's value, a missing image snapshot is not written and the match fails; outside CI mode, or with--ci=false, it is written as before, and so it is for acreateHarnessBridgecontext that names no state. The received and diff images take the whole snapshot name, so a name holding.pngkeeps it. Like Jest's own snapshot matchers and jest-image-snapshot,toMatchImageSnapshotnow refuses.not, which would turn that failure into a pass. The UI testing guide gets a "Creating snapshots" section and says.notis refused, the CLI reference lists--ci, and git ignores the__diff_output__folders beside the playground's committed snapshots.The playground's
screenshotandout-of-boundsspecs need committed baselines now, so this PR adds the ones CI's Android emulator, iOS simulator and headless Chromium capture. Android'sout-of-boundsbaseline shows the row's ten squares, which the app draws with one copy ofreact-nativein the bundle, as it has since #227. This PR's E2E run on3f1a29apasses both specs against these baselines on iOS and web (run); its Android job stopped at the size assertion #298 fixed, and with #298 merged in, E2E Android passes both on my fork (run). The CI-mode run againstmain, and how to repeat it, are in this comment.Why is this useful?
A CI run now fails where it used to pass while comparing nothing: a snapshot that was never committed, or a capture that went wrong, shows up as a failed test with the received image saved for review.
Fixes #198