Pull request checks

How it works

1. You open a pull request. Vercel, Netlify or Cloudflare Pages builds a preview deploy of it.
2. When the preview is ready, a GitHub Action asks Flowproof to run your watch (say, signup) once against the preview’s address instead of your live site.
3. Flowproof signs up on the preview with a fresh test address, waits for the emails, follows the verification link, grades everything, and deletes the test user, exactly as a Flow Watch run does.
4. The grade, the headline and the top fixes appear as one comment on the pull request, updated on every push. If the grade is below the bar you set, the check fails.

Your watch’s nightly runs are not affected: pull request runs never alert you, never pause or reschedule the watch, and never become the normal it is compared with. They appear under Pull requests on the watch’s page.

Before you start

• A Flow Watch or Agency plan and a watch. After a passing sign-up test, ask your agent:

That signup flow passed. Use Flowproof to create a Flow Watch for it that runs every night, then check the verification run with list_watches and tell me whether it passed.

• Cleanup that reaches the preview’s data. Every check signs up a real test user on the preview, and Flowproof removes it through the project’s armed cleanup connection. If your previews share your production database, the watch’s project already covers them. If previews use their own database, create a separate project for them (for example acme-preview), connect and arm cleanup against the preview database there, and create the watch in that project.

Set it up

1. Allow your preview hosts

   In the dashboard, open Projects, find Preview deploys on the project, and add a pattern for the addresses your previews get. Flowproof will only ever open a preview that matches one. A * stands for one part of the name.

   Host	Preview addresses look like	Pattern
   Vercel	myapp-git-fix-signup-yourteam.vercel.app	*-yourteam.vercel.app
   Netlify	deploy-preview-42--yoursite.netlify.app	*--yoursite.netlify.app
   Cloudflare Pages	abc123.yourproject.pages.dev	*.yourproject.pages.dev
   Your own domain	pr-42.preview.example.com	*.preview.example.com

   Patterns that would match other people’s sites are refused: *.vercel.app is every Vercel site in the world, so on shared hosts the pattern must include your own part, like your Vercel team’s name. Up to 10 patterns per project.

2. Add your API key to GitHub

   Create an API key on the dashboard’s Setup page. In your GitHub repository, go to Settings → Secrets and variables → Actions and add it as a secret named FLOWPROOF_API_KEY.

3. Add the workflow

   Save this as .github/workflows/flowproof-preview.yml. Change watch to your watch’s name.

   .github/workflows/flowproof-preview.ymlCopy

   name: Flowproof email check
   
   on:
     deployment_status:
   
   permissions:
     contents: read
     pull-requests: write
   
   jobs:
     email-flows:
       if: github.event.deployment_status.state == 'success'
       runs-on: ubuntu-latest
       timeout-minutes: 20
       steps:
         - uses: flowproof-dev/check-action@v1
           with:
             api-key: ${{ secrets.FLOWPROOF_API_KEY }}
             watch: signup
             fail-on: B

   Vercel, Netlify and Cloudflare Pages tell GitHub about each preview deploy; when one is ready this workflow runs with the preview’s address. Production deploys are skipped. The workflow needs pull-requests: write to comment.

4. Open a pull request

   When the preview is ready, the check runs for one to three minutes, then comments:

   Pull request commentCopy

   ✅ Flowproof: signup · B+ (87/100)
   Passed. B+ meets the threshold (B or better).
   > Your verification email arrives in 4s, but DMARC is missing.
   
   Email          Arrived  Time  Authentication                 Link
   verify email   yes      4.2s  spf=pass dkim=pass dmarc=none  works
   
   Top fixes
   1. Publish a DMARC record  Add a TXT record at _dmarc.example.com ...
   
   Full report (sign-in required) · Preview: myapp-git-fix-yourteam.vercel.app · Commit: abc1234

When the check passes or fails

fail-on is the lowest grade that passes. With fail-on: B, a B passes and a B- fails. Use fail-on: none to see the grade without ever failing on it.

Vercel previews behind a login

Vercel can put preview deploys behind a login (Vercel Authentication, on by default for newer projects), which stops Flowproof’s browser too. In your Vercel project, open Settings → Deployment Protection, create a Protection Bypass for Automation secret, save it in GitHub as a secret named VERCEL_AUTOMATION_BYPASS_SECRET, and add this line under with::

          vercel-protection-bypass: ${{ secrets.VERCEL_AUTOMATION_BYPASS_SECRET }}

Flowproof encrypts it for that one run, sends it only with requests to the preview itself (never to other sites the page loads), and deletes it as soon as the run starts. It never appears in the comment, the logs or the dashboard.

Limits

• Each check counts as one run on your plan, like an agent test run.
• At most 3 checks queued or running at once per account, and 12 per watch per hour.
• At most 6 runs an hour against any one site, counting watches and checks together.
• A check that can’t start within 20 minutes is dropped, and the action reports a warning.
• If a check’s cleanup doesn’t finish, that watch’s next checks wait until it does, so test users can’t pile up on your previews. Retry the cleanup from the run’s page.

What Flowproof checks before opening a preview

• The address is https, on the normal port, with no password in it.
• It matches one of the project’s preview patterns.
• Its name resolves only to public internet addresses: never localhost, a private network or a cloud metadata address.
• The project’s cleanup is armed and your plan includes Flow Watch.

All of it is checked when the check is requested and again just before the browser starts. The comment masks the test user’s name and address and removes codes and link tokens; the full report stays behind your sign-in.

Other CI systems

The action (flowproof-dev/check-action) is a small client of two HTTPS calls, so any CI can do the same. Start a run, then poll it about every 10 seconds until done is true:

curl -s -X POST https://mcp.flowproof.dev/api/ci/runs \
  -H "Authorization: Bearer $FLOWPROOF_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"watch": "signup", "preview_url": "https://myapp-git-fix-yourteam.vercel.app"}'

curl -s https://mcp.flowproof.dev/api/ci/runs/RUN_ID \
  -H "Authorization: Bearer $FLOWPROOF_API_KEY"

The result has status, grade, score, headline, fixes, emails, cleanup and report_url. Also see the API reference.

If it doesn’t run