Skip to content

Commit 9098d49

Browse files
BioPhotonBioPhoton
authored
feat(plugin-lighthouse): add lighthouse runner (#549)
1 parent ab2402b commit 9098d49

26 files changed

Lines changed: 983 additions & 62 deletions

‎.github/workflows/ci.yml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -105,6 +105,15 @@ jobs:
105105
uses: nrwl/nx-set-shas@v4
106106
- name: Install dependencies
107107
run: npm ci
108+
- name: Set custom Chrome path for Windows only
109+
if: matrix.os == 'windows-latest'
110+
# This path is considered in `testing/setup/src/lib/chrome-path-setup.ts` and used in different test configurations
111+
run: |
112+
echo "CUSTOM_CHROME_PATH=C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
113+
shell: pwsh
114+
#- name: Log all environment variables
115+
# run: |
116+
# printenv
108117
- name: Integration test affected projects
109118
run: npx nx affected -t integration-test --parallel=3 --coverage.enabled
110119

‎CONTRIBUTING.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,6 +12,25 @@ Make sure to install dependencies:
1212
npm install
1313
```
1414

15+
## Environment Variables
16+
17+
This table provides a quick overview of the environmental setup, with detailed explanations in the corresponding sections.
18+
19+
| Feature | Local Default | CI Default | Description |
20+
| -------------------------------- | ------------- | ------------------ | ----------------------------------------------------------------------------------------------------------------------------- |
21+
| `env.INCLUDE_SLOW_TESTS` **❗️** | `false` | `true` | Controls inclusion of long-running tests. Overridden by setting. Details in the [Testing](#Testing) section. |
22+
| `env.CUSTOM_CHROME_PATH` | N/A | Windows **❗️❗️** | Path to Chrome executable. See [plugin-lighthouse/CONTRIBUTING.md](./packages/plugin-lighthouse/CONTRIBUTING.md#chrome-path). |
23+
| Quality Pipeline | Off | On | Runs all plugins against the codebase. |
24+
25+
**❗️** Test Inclusion Logic
26+
27+
- `INCLUDE_SLOW_TESTS='false'` skips long tests.
28+
- without `INCLUDE_SLOW_TESTS`, tests run if `CI` is set.
29+
30+
**❗️❗️** Windows specific path set only in CI
31+
32+
- some setups also require this setting locally
33+
1534
## Development
1635

1736
Refer to docs on [how to run tasks in Nx](https://nx.dev/core-features/run-tasks).
@@ -41,6 +60,15 @@ npx nx affected:lint
4160
npx nx code-pushup -- collect
4261
```
4362

63+
## Testing
64+
65+
Some of the plugins have a longer runtime. In order to ensure better DX, longer tests are excluded by default when executing tests locally.
66+
67+
You can control the execution of long-running tests over the `INCLUDE_SLOW_TESTS` environment variable.
68+
69+
To change this setup, open (or create) the `.env` file in the root folder.
70+
Edit or add the environment variable there as follows: `INCLUDE_SLOW_TESTS=true`.
71+
4472
## Git
4573

4674
Commit messages must follow [conventional commits](https://conventionalcommits.org/) format.

‎packages/plugin-lighthouse/CONTRIBUTING.md‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# Contributing
2+
3+
## Setup
4+
5+
Make sure to install dependencies:
6+
7+
```sh
8+
npm install
9+
```
10+
11+
### Chrome path
12+
13+
In this plugin we provide Lighthouse functionality exposed over the `lighthousePlugin`.
14+
To test lighthouse properly we work with a predefined testing setup.
15+
16+
On some OS there could be a problem finding the path to Chrome.
17+
18+
We try to detect it automatically in the set-setup script.
19+
20+
If no chrome path is detected the error looks like this: `Runtime error encountered: No Chrome installations found.`
21+
22+
To prevent this from happening you have to provide the path manually in your `.env`:
23+
24+
```bash
25+
CUSTOM_CHROME_PATH=/Applications/Google Chrome.app/Contents/MacOS/Google Chrome
26+
```
27+
28+
In the CI you can set the env variable like this:
29+
30+
```yml
31+
# ...
32+
- name: Set custom Chrome path for Windows only
33+
if: matrix.os == 'windows-latest'
34+
run: |
35+
echo "CUSTOM_CHROME_PATH=C:\\Program Files\\Google\\Chrome\\Application\\chrome.exe" | Out-File -FilePath $env:GITHUB_ENV -Append -Encoding utf8
36+
shell: pwsh
37+
38+
# Optional debug log
39+
- name: Log all environment variables
40+
run: printenv
41+
# ...
42+
```
43+
44+
We added consider this path in our `beforeAll` hook.
45+
46+
```ts
47+
beforeEach(() => {
48+
try {
49+
vi.stubEnv('CHROME_PATH', getChromePath());
50+
} catch (e) {
51+
const customChromePath = process.env['CUSTOM_CHROME_PATH'];
52+
if (customChromePath == null || customChromePath === '') {
53+
throw new Error('Chrome path not found. Please read the in the packages CONTRIBUTING.md/#trouble-shooting section.');
54+
}
55+
vi.stubEnv('CHROME_PATH', customChromePath);
56+
}
57+
});
58+
```
59+
60+
### Testing chrome flags
61+
62+
1. run `npx chrome-debug --<chromium-flag>` to pass terminal arguments to Chrome. E.g. `npx chrome-debug --headless=shell`.
63+
`npx chrome-debug --headless=shell --@TODO-PUT-OTHER-EXAMPLE-FOR-FLAG`
64+
65+
For a full list of available flags check out [this document](https://peter.sh/experiments/chromium-command-line-switches/).
66+
67+
> [!NOTE]
68+
> To pass chrome flags to lighthouse you have to provide them under `--chrome-flags="<chrome-flags-as-array>"`.
69+
> E.g. `lighthouse https://example.com --chrome-flage="--headless=shell"`
70+
71+
2. Check if the flag got accepted. This is quite unintuitive as we would expect the passed flag to be visible under `chrome://flags/` but as you can see in the screenshot it is not visible.
72+
<img width="1202" alt="chrome-flags" src="./docs/images/chrome-flags.png">
73+
Instead open `chrome://version/` and look under the "Command Line" section.
74+
<img width="1202" alt="chrome-chrome-version" src="./docs/images/chrome-version.png">
75+
76+
### Chrome User Data
77+
78+
To bootstrap Chrome with a predefined for setting we have to provide a couple of config files that we located under `<project-root>/mock/chromium-user-data`.
79+
When executing Lighthouse we provide the path to this folder over the `Flag` object.
80+
81+
To generate initialise or edit the file structure under `chromium-user-data` do the following steps:
82+
83+
1. Spin up Chrome by running `npx chrome-debug --user-data-dir=./packages/plugin-lighthouse/mock/chromium-user-data`
84+
<img width="1202" alt="chrome-blank-screen" src="./docs/images/chrome-blank-screen.png">
85+
86+
2. If you do this the first time you should already see content under `<project-root>/mock/chromium-user-data`
87+
3. Edit the configuration over the Chrome UI. E.g. adding a profile
88+
4. Close chromium and open it again, and you should see chromium bootstraps as the configured user
89+
<img width="1202" alt="chrome-blank-screen-pre-configured" src="./docs/images/chrome-blank-screen-pre-configure.png">
90+
91+
To reset the above just delete the folder and apply the settings again.
92+
93+
_A helpful chromium setup is preconfigured with the following settings:_
94+
95+
- A user profile is set up. This enables certain debugging related options as well as help to visually distinguish between test setups as the header bar is colored.
96+
<img width="1202" alt="chrome-settings-manage-profile" src="./docs/images/chrome-settings-manage-profile.png">
97+
98+
#### Resources
99+
100+
- [chromium flags guide](https://www.chromium.org/developers/how-tos/run-chromium-with-flags/)
101+
102+
## Troubleshooting
103+
104+
1. Verify Chrome Installation
105+
Ensure Chrome is correctly installed and accessible to the Lighthouse process.
106+
Run `npx chrome-debug` to test it. Read further under [chrome-path](#chrome-path)
107+
108+
2. Increase Timeout
109+
Lighthouse has a longer runtime which can time out in different environments.
110+
**Try increasing the test timeout** in `lighthouse-plugin.integration.test.ts` for `runner creation and execution` test suite.
111+
112+
3. Turn on debug mode
113+
Show debug logs of Lighthouse. Set the following environment variable: `DEBUG='*'`
114+
115+
4. Understand error messages (⏳ could also be because of timeout problems :D )
116+
117+
- Could not find `report.json` (⏳)
118+
![lighthouse-error-2.png](./docs/images/lighthouse-error-2.png)
119+
- Lighthouse Error - `Could Not Connect to Chrome` (⏳)
120+
![lighthouse-error-1.png](./docs/images/lighthouse-error-1.png)
121+
Your Chrome path is set incorrectly. Read further under [chrome-path](#chrome-path)
122+
- Lighthouse Error - `start lh:<any>:<performancemark>" performance mark has not been set` (⏳)
123+
![lighthouse-error-3.png](./docs/images/lighthouse-error-3.png)
124+
If this error pops up you are able to launch Chrome but had problems to communicate over the ports.

‎packages/plugin-lighthouse/README.md‎

Lines changed: 119 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,121 @@
11
# @code-pushup/lighthouse-plugin
22

3-
TODO: docs
3+
[![npm](https://img.shields.io/npm/v/%40code-pushup%2Flighthouse-plugin.svg)](https://www.npmjs.com/package/@code-pushup/lighthouse-plugin)
4+
[![downloads](https://img.shields.io/npm/dm/%40code-pushup%2Flighthouse-plugin)](https://npmtrends.com/@code-pushup/lighthouse-plugin)
5+
[![dependencies](https://img.shields.io/librariesio/release/npm/%40code-pushup/lighthouse-plugin)](https://www.npmjs.com/package/@code-pushup/lighthouse-plugin?activeTab=dependencies)
6+
7+
🕵️ **Code PushUp plugin for measuring web performance and quality with Lighthouse.** 🔥
8+
9+
---
10+
11+
The plugin parses your Lighthouse configuration and lints all audits of the official [Lighthouse](https://github.com/GoogleChrome/lighthouse/blob/main/readme.md#lighthouse-------) CLI.
12+
13+
Detected Lighthouse audits are mapped to Code PushUp audits. Audit reports are calculated based on the [original implementation](https://googlechrome.github.io/lighthouse/scorecalc/).
14+
Additionally, Lighthouse categories are mapped to Code PushUp groups which can make it easier to assemble the categories.
15+
16+
For more infos visit the [official docs](https://developer.chrome.com/docs/lighthouse/overview).
17+
18+
## Getting started
19+
20+
1. If you haven't already, install [@code-pushup/cli](../cli/README.md) and create a configuration file.
21+
22+
2. Install as a dev dependency with your package manager:
23+
24+
```sh
25+
npm install --save-dev @code-pushup/lighthouse-plugin
26+
```
27+
28+
```sh
29+
yarn add --dev @code-pushup/lighthouse-plugin
30+
```
31+
32+
```sh
33+
pnpm add --save-dev @code-pushup/lighthouse-plugin
34+
```
35+
36+
3. Add this plugin to the `plugins` array in your Code PushUp CLI config file (e.g. `code-pushup.config.ts`).
37+
38+
Pass in the URL you want to measure, along with optional [flags](#flags) and [config](#config) data.
39+
40+
```ts
41+
import lighthousePlugin from '@code-pushup/lighthouse-plugin';
42+
43+
export default {
44+
// ...
45+
plugins: [
46+
// ...
47+
await lighthousePlugin('https://example.com'),
48+
],
49+
};
50+
```
51+
52+
4. Run the CLI with `npx code-pushup collect` and view or upload the report (refer to [CLI docs](../cli/README.md)).
53+
54+
### Optionally set up categories
55+
56+
@TODO
57+
58+
## Flags
59+
60+
The plugin accepts a second optional argument, `flags`.
61+
62+
`flags` is the Lighthouse [CLI flags](https://github.com/GoogleChrome/lighthouse/blob/7d80178c37a1b600ea8f092fc0b098029799a659/cli/cli-flags.js#L80) as a JS object.
63+
64+
Within the flags object a couple of other external configuration files can be referenced. E.g. `configPath` , `preset` or `budgetPath` reference external `json` or JavaScript files.
65+
66+
For a complete list the [official documentation of CLI flags](https://github.com/GoogleChrome/lighthouse/blob/main/readme.md#cli-options)
67+
68+
> [!TIP]
69+
> If you are not used to work with the Lighthouse CLI you would pass flags like this:
70+
> `lighthouse https://example.com --output=json --chromeFlags='--headless=shell'`
71+
>
72+
> Now with the plugin it would look like this:
73+
>
74+
> ```ts
75+
> // code-pushup.config.ts
76+
> ...
77+
> lighthousePlugin('https://example.com', { output: 'json', chromeFlags: ['--headless=shell']});
78+
> ```
79+
80+
## Config
81+
82+
The plugin accepts a third optional argument, `config`.
83+
84+
`config` is the Lighthouse [configuration](https://github.com/GoogleChrome/lighthouse/blob/7d80178c37a1b600ea8f092fc0b098029799a659/types/config.d.ts#L21) as a JS object.
85+
86+
For a complete guide on Lighthouse configuration read the [official documentation on configuring](https://github.com/GoogleChrome/lighthouse/blob/main/docs/configuration.md)
87+
88+
> [!TIP]
89+
> If you are not used to work with the Lighthouse CLI you would pass a config like this:
90+
> `lighthouse --config-path=path/to/custom-config.js https://example.com`
91+
>
92+
> And in a separate file you would place the following object:
93+
>
94+
> ```typescript
95+
> // custom-config.js file
96+
> export default {
97+
> extends: 'lighthouse:default',
98+
> settings: {
99+
> onlyAudits: ['first-meaningful-paint', 'speed-index', 'interactive'],
100+
> },
101+
> };
102+
> ```
103+
>
104+
> Now with the plugin it would look like this:
105+
>
106+
> ```ts
107+
> // code-pushup.config.ts
108+
> ...
109+
> lighthousePlugin('https://example.com', undefined, {
110+
> extends: 'lighthouse:default',
111+
> settings: {
112+
> onlyAudits: [
113+
> 'first-meaningful-paint',
114+
> 'speed-index',
115+
> 'interactive',
116+
> ],
117+
> }
118+
> })
119+
> ```
120+
121+
If you want to contribute, please refer to [CONTRIBUTING.md](./CONTRIBUTING.md).

‎packages/plugin-lighthouse/docs/images/chrome-blank-screen-pre-configure.png‎

56.2 KB
Loading

‎packages/plugin-lighthouse/docs/images/chrome-blank-screen.png‎

57.7 KB
Loading

‎packages/plugin-lighthouse/docs/images/chrome-flags.png‎

573 KB
Loading

‎packages/plugin-lighthouse/docs/images/chrome-settings-manage-profile.png‎

700 KB
Loading

‎packages/plugin-lighthouse/docs/images/chrome-version.png‎

584 KB
Loading

‎packages/plugin-lighthouse/docs/images/lighthouse-error-1.png‎

504 KB
Loading

0 commit comments

Comments
 (0)