Context
PR #7192 adds a Vercel preview smoke test to the frontend PR workflow. The current approach parses preview URLs from the Vercel bot PR comment and curls them. This works reliably but has a few known limitations worth tracking.
Known limitations
-
Grep pattern is brittle — https://flagsmith-frontend[^ )]* depends on Vercel's comment format and project naming. If either changes, the grep silently finds nothing and the job warns but doesn't test anything.
-
First-ever PR push — if the Vercel bot hasn't commented yet (brand new PR, first deployment still building), the poll waits up to 5 minutes. Usually enough, but not guaranteed if Vercel is slow.
-
No content validation — curl checks HTTP status only. A broken deployment serving Vercel's error page still returns 200. Could add a body check (e.g., grep for a known string in the HTML response).
Possible improvements
- Pin the grep pattern to the Vercel comment's markdown structure (e.g., match the table row format) instead of raw URL matching
- Add a simple body check:
curl ... | grep -q 'Flagsmith' to verify the app actually loaded
- Consider using the Vercel API directly (
api.vercel.com/v6/deployments) with a Vercel token for more reliable URL resolution — though this adds a secret dependency
References
Context
PR #7192 adds a Vercel preview smoke test to the frontend PR workflow. The current approach parses preview URLs from the Vercel bot PR comment and curls them. This works reliably but has a few known limitations worth tracking.
Known limitations
Grep pattern is brittle —
https://flagsmith-frontend[^ )]*depends on Vercel's comment format and project naming. If either changes, the grep silently finds nothing and the job warns but doesn't test anything.First-ever PR push — if the Vercel bot hasn't commented yet (brand new PR, first deployment still building), the poll waits up to 5 minutes. Usually enough, but not guaranteed if Vercel is slow.
No content validation — curl checks HTTP status only. A broken deployment serving Vercel's error page still returns 200. Could add a body check (e.g., grep for a known string in the HTML response).
Possible improvements
curl ... | grep -q 'Flagsmith'to verify the app actually loadedapi.vercel.com/v6/deployments) with a Vercel token for more reliable URL resolution — though this adds a secret dependencyReferences
.github/workflows/frontend-pull-request.yml