Proof with every ticket

When a ticket ships, the reviewer should be able to see the change, not only read it. Proof takes a screenshot of your running app before the ticket (on your base branch) and after it (on the ticket's final tree), for one to three pages you choose. The pictures are stored with the ticket, shown on the ticket page and mentioned in the pull request.

It is off until you switch it on. It never changes whether a ticket ships. If there is no app, no browser or no time, the ticket page says why there is no picture.

What you get

• On the ticket page, a Proof panel: for each page, the before and the after side by side (two tabs on a narrow screen), the short commit ids, and a click to see an image large.
• In the pull request, a "Proof" section in the description: for each page, whether the before and the after were captured (or why not), the commits and, when your install has a public address set, a link to the ticket in the dashboard. A reader needs to sign in to see the images. The pictures themselves are never put in the pull request.
• A reason instead of a picture when one could not be taken, stored with the ticket.

Turn it on

Open the project's Settings → Checks page. The Proof card has every setting.

Your app is started with the settings of the runtime smoke test (see Checks and toolchains): its start command, health URL, environment values, boot timeout and teardown command. Proof needs a start command and a health URL on this machine (localhost, 127.0.0.1 or [::1]). It works whether or not the runtime smoke test itself is switched on. Without a start command or a health URL nothing is captured, and the ticket page says so.

What has to be in place

Proof drives a headless Chromium browser, the same optional browser Bug Hunt uses. See Install the browser. On Docker, the browser and your app run in the runner container, so it is the runner's image that needs the browser.

How a run takes it

1. Before. Right after the ticket's branch is entered, and only if the branch is exactly your base commit and clean, the app is started, each page is captured and the app is stopped. A branch that already has work of its own, such as a retry, is not photographed: the page says "not captured" and why.
2. After. When the ticket's change is final, after every check and the review have passed and before it is committed, the app is started again and the same pages are captured. Anything the app wrote into the working tree while it ran is removed again, so it cannot end up in the commit.
3. Stored. The images and a row per page are kept with the ticket.

Limits that always hold:

• Four minutes for one ticket's proof, both sides together.
• One minute for one capture.
• The browser may load pages and files only from your app's own address. Every other request, a redirect to another site included, is blocked.
• At most 2 MB for one image. A larger one is refused, not shrunk.

It never changes the result

Proof is not a check. A missing picture, a crashed app or an error inside proof leaves the ticket's status, its commit and its pull request exactly as they would have been, apart from the note saying what could not be captured. It runs only for a ticket on its own branch, which is the default branch strategy.

Where the pictures are, and who sees them

• On your install only. The images are stored with the ticket's other artifacts and are never sent to the model provider or to Reqursor. Only a signed in user who may read the project's tickets can open one.
• For the retention period. They are removed with the other artifacts after the retention period. The ticket page then shows that the image was removed. They are not part of a backup. See Backups and updates.
• A screenshot shows whatever your app shows. A page can contain personal data or a secret, and a screenshot cannot be masked the way text is. Point proof at pages that show test data, and run it against an app started with development settings.

Proof uses no model work.

When there is no picture