Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions frontend/.claude/commands/visual-regression.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Visual Regression Test Runner

Run local visual regression tests by capturing baselines from main on this machine, then comparing against the current branch. Both runs use the same OS and browser, so diffs reflect actual style changes rather than platform rendering differences.

## Prerequisites

- Docker running with Flagsmith services on localhost:8000

## Workflow

1. Capture local baselines from main (stashes changes, checks out main, runs E2E, switches back):

```bash
npm run test:visual:baselines
```

2. Run E2E tests on the current branch with visual regression screenshot capture:

```bash
VISUAL_REGRESSION=1 npm run test
```

3. Compare captured screenshots against local baselines:

```bash
npm run test:visual:compare
```

4. If there are failures, open the HTML report for visual diff inspection:

```bash
npm run test:visual:report
```

5. Report results: how many comparisons passed/failed, and whether failures are style/layout regressions or data differences.
20 changes: 14 additions & 6 deletions frontend/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,20 +149,26 @@ E2E_RETRIES=0 SKIP_BUNDLE=1 E2E_CONCURRENCY=1 npm run test -- tests/flag-tests.p

Visual regression screenshots are captured during E2E tests via `visualSnapshot()` calls. They are a no-op unless `VISUAL_REGRESSION=1` is set. Comparison runs as a separate step after all E2E retries complete, so flaky tests don't affect the report.

##### Local development

To check for visual regressions locally, capture baselines from main on your machine then compare against your branch. Both runs use the same OS and browser, so diffs reflect actual style changes.

```bash
# 1. Run E2E tests with screenshot capture (with retries)
VISUAL_REGRESSION=1 npm run test
# 1. Capture baselines from main (stashes changes, checks out main, runs E2E, switches back)
npm run test:visual:baselines

# 2a. Generate/update baselines from captured screenshots
npm run test:visual:compare -- --update-snapshots
# 2. Run E2E tests on your branch with screenshot capture
VISUAL_REGRESSION=1 npm run test

# 2b. Compare screenshots against baselines (generates Playwright report with diffs)
# 3. Compare screenshots against baselines (generates Playwright report with diffs)
npm run test:visual:compare

# 3. Open the report
# 4. Open the report
npm run test:visual:report
```

##### CI

Visual diffs never fail CI — they are reported via PR comment and the Playwright HTML report.

Screenshots are saved to `e2e/visual-regression-screenshots/`, baselines to `e2e/visual-regression-snapshots/` (both git-ignored). In CI, the main branch uploads screenshots as baseline artifacts, and PRs download them for comparison.
Expand All @@ -181,3 +187,5 @@ When using Claude Code, these commands are available for e2e testing:
- `/e2e-create [description]` - Create a new test following existing patterns

The optional `[N]` argument sets `E2E_REPEAT` to run tests N additional times after passing (defaults to 0). E.g., `/e2e 5` runs tests, then repeats 5 more times to detect flakiness.

- `/visual-regression` - Download CI baselines, run E2E with screenshot capture, compare and report
46 changes: 46 additions & 0 deletions frontend/e2e/capture-baselines.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
import { execSync } from 'child_process'
import * as fs from 'fs'
import * as path from 'path'

const SNAPSHOTS_DIR = path.resolve(__dirname, 'visual-regression-snapshots')
const SCREENSHOTS_DIR = path.resolve(__dirname, 'visual-regression-screenshots')

function run(cmd: string) {
execSync(cmd, { stdio: 'inherit' })
}

function runQuiet(cmd: string) {
execSync(cmd, { stdio: 'pipe' })
}

const branch = execSync('git rev-parse --abbrev-ref HEAD', { encoding: 'utf-8' }).trim()

console.log(`Current branch: ${branch}`)
console.log('Checking out main source code (keeping branch e2e tests)...')

// Only checkout main's source code, not e2e/ test files.
// This ensures both baseline and branch runs use the same test code.
runQuiet('git checkout main -- web/ common/')

try {
console.log('Running E2E on main source with VISUAL_REGRESSION=1...')
if (fs.existsSync(SCREENSHOTS_DIR)) {
fs.rmSync(SCREENSHOTS_DIR, { recursive: true })
}
run('cross-env VISUAL_REGRESSION=1 npm run test')

console.log('Copying screenshots to baselines...')
if (fs.existsSync(SNAPSHOTS_DIR)) {
fs.rmSync(SNAPSHOTS_DIR, { recursive: true })
}
fs.cpSync(SCREENSHOTS_DIR, SNAPSHOTS_DIR, { recursive: true })

const count = fs.readdirSync(SNAPSHOTS_DIR).filter((f) => f.endsWith('.png')).length
console.log(`Captured ${count} baseline snapshots`)
} finally {
// Restore branch's source code
console.log('Restoring branch source code...')
runQuiet('git checkout -- web/ common/')
}

console.log('Baselines ready. Now run: VISUAL_REGRESSION=1 npm run test && npm run test:visual:compare')
2 changes: 2 additions & 0 deletions frontend/e2e/tests/roles-test.pw.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,13 +19,15 @@ test.describe('Roles Tests', () => {
login,
logout,
waitForElementVisible,
waitForToastsToClear,
} = createHelpers(page);

const rolesProject = 'project-my-test-project-7-role'
log('Login')
await login(E2E_USER, PASSWORD)
await click(byId(rolesProject))
await createFeature({ name: 'test_feature', value: false })
await waitForToastsToClear()
log('Go to Roles')
await click(byId('organisation-link'))
await click(byId('users-and-permissions'))
Expand Down
1 change: 1 addition & 0 deletions frontend/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
"test:unit:watch": "jest --watch",
"test:unit:coverage": "jest --coverage",
"test:visual": "cross-env VISUAL_REGRESSION=1 npx playwright test",
"test:visual:baselines": "npx tsx e2e/capture-baselines.ts",
"test:visual:compare": "npx tsx e2e/compare-visual-regression.ts && npx playwright test -c playwright.visual.config.ts",
"test:visual:report": "npx playwright show-report e2e/visual-regression-report",
"test:report": "npx playwright show-report e2e/playwright-report",
Expand Down
2 changes: 1 addition & 1 deletion frontend/web/components/ActionButton.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ const ActionButton: FC<ActionButtonType> = ({
onClick()
}}
>
<div className='pointer-events-none'>
<div className='pe-none'>
<Icon name='more-vertical' width={16} fill='#656D7B' />
</div>
</Button>
Expand Down
2 changes: 1 addition & 1 deletion frontend/web/components/DateSelect.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ const DateSelect: FC<DateSelectProps> = ({
const [isOpen, setIsOpen] = useState(false)

return (
<Flex style={{ position: 'relative' }}>
<Flex className='position-relative'>
<DatePicker
className={`${className} ${!isValid && touched ? 'invalid' : ''}`}
dateFormat={dateFormat}
Expand Down
2 changes: 1 addition & 1 deletion frontend/web/components/ExternalResourcesTable.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ const ExternalResourceRow: FC<ExternalResourceRowType> = ({
}, [isDeleted])
return (
<Row className='list-item' key={externalResource?.id}>
<div className='table-column text-left' style={{ width: '100px' }}>
<div className='table-column text-start' style={{ width: '100px' }}>
<div className='font-weight-medium mb-1'>
{
Constants.resourceTypes[
Expand Down
2 changes: 1 addition & 1 deletion frontend/web/components/IdentitySelect.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ const IdentitySelect: FC<IdentitySelectType> = ({
.slice(0, 10)
}, [ignoreIds, data])
return (
<Flex className='text-left'>
<Flex className='text-start'>
<Select
onInputChange={(e: InputEvent) => {
searchItems(Utils.safeParseEventValue(e))
Expand Down
2 changes: 1 addition & 1 deletion frontend/web/components/InlineModal.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ const InlineModal: FC<InlineModalProps> = ({
})

return (
<div className={relativeToParent ? '' : 'relative'}>
<div className={relativeToParent ? '' : 'position-relative'}>
{isOpen && (
<div ref={modalRef} className={classNames('inline-modal', className)}>
<div className='d-flex py-2 d-lg-none justify-content-end px-4'>
Expand Down
4 changes: 2 additions & 2 deletions frontend/web/components/LicensingTabContent.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -56,15 +56,15 @@ const LicensingTabContent: React.FC<LicensingTabContentProps> = ({
<input
type='file'
ref={licenceInputRef}
style={{ display: 'none' }}
className='d-none'
onChange={() =>
setLicence(licenceInputRef.current?.files?.[0] ?? null)
}
/>
<input
type='file'
ref={licenceSignatureInputRef}
style={{ display: 'none' }}
className='d-none'
onChange={() =>
setLicenceSignature(
licenceSignatureInputRef.current?.files?.[0] ?? null,
Expand Down
2 changes: 1 addition & 1 deletion frontend/web/components/PageTitle.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ const PageTitle: FC<PageTitleType> = ({ children, className, cta, title }) => {
</Row>
)}
</div>
{!!cta && <div className='float-right ms-lg-2'>{cta}</div>}
{!!cta && <div className='float-end ms-lg-2'>{cta}</div>}
</div>
<hr className='mb-0 mt-3' />
</div>
Expand Down
2 changes: 1 addition & 1 deletion frontend/web/components/PanelSearch.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -267,7 +267,7 @@ const PanelSearch = <T,>(props: PanelSearchProps<T>): ReactElement => {
{props.filterElement && props.filterElement}

{sorting && (
<Row className='mr-3 relative'>
<Row className='mr-3 position-relative'>
<Popover
renderTitle={(toggle: () => void, isActive: boolean) => (
<a
Expand Down
12 changes: 6 additions & 6 deletions frontend/web/components/SegmentOverrides.js
Original file line number Diff line number Diff line change
Expand Up @@ -107,7 +107,7 @@ const SegmentOverrideInner = class Override extends React.Component {
<Row className='p-0 table-header px-3 py-1' space>
<div
ref={dragHandleRef}
className={classNames('flex flex-1 text-left', {
className={classNames('flex flex-1 text-start', {
'drag-handle': dragHandleRef,
})}
>
Expand Down Expand Up @@ -430,7 +430,7 @@ const SegmentOverrideListInner = ({
}}
projectFlag={projectFlag}
/>
<div className='text-left'>
<div className='text-start'>
<JSONReference
showNamesButton
title={'Segment Override'}
Expand Down Expand Up @@ -476,7 +476,7 @@ const SegmentOverrideListInner = ({
}}
projectFlag={projectFlag}
/>
<div className='text-left'>
<div className='text-start'>
<JSONReference
showNamesButton
title={'Segment Override'}
Expand Down Expand Up @@ -708,7 +708,7 @@ class TheComponent extends Component {
!this.props.disableCreate &&
!this.props.showCreateSegment &&
!this.props.readOnly && (
<Row className='text-left gap-2'>
<Row className='text-start gap-2'>
<div className='flex-1'>
<SegmentSelect
className='w-100'
Expand Down Expand Up @@ -794,7 +794,7 @@ class TheComponent extends Component {
<div className='my-4'>
<InfoMessage
collapseId={'segment-overrides'}
className='mb-4 text-left faint'
className='mb-4 text-start faint'
>
Segment overrides override the environment defaults,
prioritise them by dragging it to the top of the list.
Expand Down Expand Up @@ -842,7 +842,7 @@ class TheComponent extends Component {
hideViewSegment={this.props.hideViewSegment}
highlightSegmentId={this.props.highlightSegmentId}
/>
<div className='text-left mt-4'>
<div className='text-start mt-4'>
<JSONReference
showNamesButton
title={'Segment Overrides'}
Expand Down
7 changes: 4 additions & 3 deletions frontend/web/components/Switch.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -20,16 +20,17 @@ const Switch: FC<SwitchProps> = ({
}) => {
if (E2E) {
return (
<div style={{ display: 'inline-block', height: '28px' }}>
<div className='d-inline-block' style={{ height: '28px' }}>
<button
role='switch'
type='button'
style={{
color: 'black',
pointerEvents: 'all',
position: 'relative',
}}
className={checked ? 'switch-checked' : 'switch-unchecked'}
className={`position-relative ${
checked ? 'switch-checked' : 'switch-unchecked'
}`}
{...rest}
onClick={() => {
onChange?.(!checked)
Expand Down
2 changes: 1 addition & 1 deletion frontend/web/components/base/forms/GhostInput.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ const GhostInput = forwardRef<HTMLInputElement, GhostInputProps>(
}, [value])

return (
<div style={{ display: 'inline-block', position: 'relative' }}>
<div className='d-inline-block position-relative'>
<span
ref={spanRef}
style={{
Expand Down
2 changes: 1 addition & 1 deletion frontend/web/components/base/forms/Radio.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ const Radio: React.FC<RadioProps> = ({ checked, label, onChange }) => {
return (
<label
onClick={handleChange}
className='relative cursor-pointer flex-row align-items-center'
className='position-relative cursor-pointer flex-row align-items-center'
>
{checked && <CheckedSVG />} {!checked && <UncheckedSVG />}
{<span className={classNames('ml-2', className)}>{label}</span>}
Expand Down
2 changes: 1 addition & 1 deletion frontend/web/components/modals/CreateMetadataField.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -357,7 +357,7 @@ const CreateMetadataField: FC<CreateMetadataFieldType> = ({
<Button
disabled={!name || !typeValue || !metadataFieldSelectList}
onClick={save}
className='float-right'
className='float-end'
>
{isEdit ? 'Update Custom Field' : 'Create Custom Field'}
</Button>
Expand Down
2 changes: 1 addition & 1 deletion frontend/web/components/modals/InviteUsers.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -206,7 +206,7 @@ const InviteUsers: FC = () => {
)}
/>
</Flex>
<Flex className='mb-2' style={{ position: 'relative' }}>
<Flex className='mb-2 position-relative'>
<Select
data-test='select-group'
placeholder='Select a group'
Expand Down
2 changes: 1 addition & 1 deletion frontend/web/components/modals/Payment.js
Original file line number Diff line number Diff line change
Expand Up @@ -149,7 +149,7 @@ const Payment = class extends Component {
Pay Monthly
</h5>
</div>
<Row className='pricing-container align-start'>
<Row className='pricing-container align-items-start'>
<Flex className='pricing-panel p-2'>
<div className='panel panel-default'>
<div
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -644,7 +644,7 @@ const Index = class extends Component {
<div className='pr-3'>
{identity ? (
<div className='mb-3 mt-4'>
<p className='text-left ml-3 modal-caption fs-small lh-small'>
<p className='text-start ml-3 modal-caption fs-small lh-small'>
This will update the feature value for the
user <strong>{identityName}</strong> in
<strong>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ const IdentitySaveFooter: FC<IdentitySaveFooterProps> = ({
return (
<div className='pr-3'>
<div className='mb-3 mt-4'>
<p className='text-left ml-3 modal-caption fs-small lh-small'>
<p className='text-start ml-3 modal-caption fs-small lh-small'>
This will update the feature value for the user{' '}
<strong>{identityName}</strong> in
<strong> {environmentName}.</strong>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ const IdentityOverridesTab: FC<IdentityOverridesTabProps> = ({
searchPanel={
!isEdge && (
<div className='text-center mt-2 mb-2'>
<Flex className='text-left'>
<Flex className='text-start'>
<IdentitySelect
isEdge={false}
ignoreIds={data?.results?.map((v) => v.identity?.id)}
Expand Down
2 changes: 1 addition & 1 deletion frontend/web/components/modals/payment/Payment.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,7 @@ export const Payment: FC<PaymentProps> = ({

<PricingToggle isYearly={yearly} onChange={setYearly} />

<Row className='pricing-container align-start'>
<Row className='pricing-container align-items-start'>
<PricingPanel
title='Start-Up'
priceYearly='40'
Expand Down
2 changes: 1 addition & 1 deletion frontend/web/components/mv/VariationValueInput.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ export const VariationValueInput: React.FC<VariationValueProps> = ({
/>
</div>
{!!onRemove && !readOnly && (
<div style={{ position: 'relative', top: '27px' }} className='ml-2'>
<div style={{ top: '27px' }} className='ml-2 position-relative'>
<button
onClick={onRemove}
id='delete-multivariate'
Expand Down
Loading
Loading