Skip to content

feat(visual-regression): testes visuais para todos os fields#2792

Open
alinelariguet wants to merge 10 commits intomasterfrom
visual-regression/prototype-playwright-v4
Open

feat(visual-regression): testes visuais para todos os fields#2792
alinelariguet wants to merge 10 commits intomasterfrom
visual-regression/prototype-playwright-v4

Conversation

@alinelariguet
Copy link
Copy Markdown
Member

@alinelariguet alinelariguet commented Mar 31, 2026

feat(visual-regression): testes visuais para todos os fields

Summary

Adds Playwright-based visual regression testing for all 20 field components with multiple state combinations, plus 11 basic sample-based tests for non-field components.

What's included:

  • A dedicated Angular application (visual-app in e2e/visual/app/) that renders components in isolation — separate from the existing projects/app/
  • 20 field component test suites organized in e2e/visual/fields/po-{component}/, each with co-located HTML template, TypeScript component, and Playwright spec
  • State combinations per field: basic, label, label+helper, label+help-text, disabled, readonly, required, required+errorMessage, loading, etc. (~186 total test cases)
  • CI job (test-visual) with po-style branch detection, multi-layer caching, baseline generation, and HTML report + diff artifact uploads
  • playwright.config.ts with toHaveScreenshot() comparison (1% pixel diff tolerance, 5% for po-rich-text disabled)
  • npm scripts: test:visual, test:visual:update, test:visual:report
  • Landing page at / listing all 20 field test scenarios and 12 sample component routes as clickable cards

Components covered: po-input, po-decimal, po-email, po-login, po-number, po-password, po-url, po-combo, po-datepicker, po-datepicker-range, po-lookup, po-multiselect, po-select, po-textarea, po-rich-text, po-upload, po-checkbox, po-checkbox-group, po-switch, po-radio-group

CI Efficiency Optimizations

Phase 1:

  • Cache node_modules via actions/cache@v4 keyed on hashFiles('package.json')npm i skipped on cache hit
  • Cache Playwright browsers (~/.cache/ms-playwright) — split into system deps (install-deps, always runs) and browser binary (install, skipped on cache hit)
  • 2 Playwright workers in CI (up from 1)
  • Stale po-style protection — reinstalls @po-ui/style from npm when cache hits but no matching po-style branch exists
  • Baselines cache without restore-keys — on cache miss, uses --update-snapshots (force-regenerate all) to avoid cross-environment baseline mismatch

Phase 2:

  • Paths filter via dorny/paths-filter@v3 — visual tests only run on PRs that touch relevant files (e2e/visual/**, projects/ui/src/lib/**, package.json, playwright.config.ts, angular.json, ci.yml). Always runs on push to master/development via always() && !cancelled().
  • UI build cachedist/ng-components cached keyed on hashFiles('projects/ui/src/**', 'package.json')
  • Retries reduced from 2 to 1 — saves time on genuine failures while still handling occasional flakes

Estimated time savings: From ~5.7min to ~1.5min when all caches hit (~74% reduction). PRs not touching visual-relevant files skip the job entirely.

Review & Testing Checklist for Human

  • Verify push event behavior: The test-visual job uses always() && !cancelled() to run on push events even when the changes job is skipped. Verify that visual tests actually execute on direct pushes to master/development (cannot be tested from a PR context). If they don't run, the always() workaround may not be sufficient.
  • Cache invalidation tradeoffs: node_modules cache keyed on package.json (not package-lock.json since it's gitignored). Transitive dependency updates won't invalidate cache. UI build cache keyed on hashFiles('projects/ui/src/**', 'package.json') — changes to tsconfig.lib.json or ng-package.json won't invalidate. Monitor for stale artifacts.
  • Paths filter coverage: Filter checks e2e/visual/**, projects/ui/src/lib/**, package.json, playwright.config.ts, angular.json, ci.yml. Changes to root tsconfig.json or global styles won't trigger visual tests. Verify scope is appropriate.
  • Test with cache-miss scenario: First CI run after spec/HTML changes generates baselines then compares against themselves (trivial pass). Trigger a second CI run (e.g., empty commit or re-run) to verify actual regression detection works.
  • Download CI artifacts from test-visual job: confirm (a) caches hit as expected, (b) execution time decreased vs. baseline, (c) on a PR without visual changes, job is skipped.

Notes

  • Artifact retention: 15 days for both HTML report and test results
  • Committed baseline PNGs (~200 files) generated locally. CI regenerates separate baselines for Linux environment (e.g., 7px height difference for <textarea>). Baselines cache ensures cross-run consistency.
  • po-rich-text disabled state uses 5% tolerance vs. global 1% due to minor rendering variations
  • CI uses deprecated actions/checkout@v3 and actions/setup-node@v3 (Node.js 20) — warnings in logs; consider v4 upgrade in follow-up
  • po-style branch detection uses unauthenticated GitHub API (60/hour limit) — add token if rate limits occur
  • Landing page (VisualHomeComponent) uses inline styles and onmouseover/onmouseout instead of Angular best practices — acceptable for test utility page
  • Third-party action dorny/paths-filter@v3 added for change detection — ensure your org's security policies allow third-party actions

Devin Session: https://totvs.devinenterprise.com/sessions/5c85ab5d7935486484961f45a8077db3
Requested by: @alinelariguet


Open with Devin

@devin-ai-integration
Copy link
Copy Markdown
Contributor

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant