Skip to content

Commit fc61608

Browse files
craigfowlercraigfowler
committed
WIP #272 - Update BrowserStack config
This attempts to resolve a few issues: Make it simpler to run local tests - no extra requirements upon disabling BS. BS is disabled by default and only activated by: * Specifying an MSBuild property * Providing a config Attempt to fix BS test duplicationg by switching off NUnit parallelism when BS is enabled. Activate all seven build configs for the CBT CI build. This should demonstrate whether I get improved CBT by using the SDK. Add README files to signpost this configuration to devs.
1 parent 6b9d791 commit fc61608

15 files changed

Lines changed: 129 additions & 54 deletions

‎.github/workflows/crossBrowserTesting.yml‎

Lines changed: 13 additions & 32 deletions
Original file line numberDiff line numberDiff line change
@@ -29,34 +29,13 @@ jobs:
2929
fail-fast: false
3030
matrix:
3131
include:
32-
- browserName: Chrome
33-
browserVersion: latest-50
34-
os: Windows
35-
osVersion: 11
36-
# - browserName: Chrome
37-
# browserVersion: latest
38-
# os: Windows
39-
# osVersion: 11
40-
# - browserName: Edge
41-
# browserVersion: latest
42-
# os: Windows
43-
# osVersion: 11
44-
# - browserName: Firefox
45-
# browserVersion: latest
46-
# os: Windows
47-
# osVersion: 11
48-
# - browserName: Firefox
49-
# browserVersion: latest-40
50-
# os: Windows
51-
# osVersion: 11
52-
# - browserName: Safari
53-
# browserVersion: 17.3
54-
# os: OS X
55-
# osVersion: Sonoma
56-
# - browserName: Safari
57-
# browserVersion: 26.2
58-
# os: OS X
59-
# osVersion: Tahoe
32+
- browserStackConfig: chromeLatest
33+
- browserStackConfig: chromeOld
34+
- browserStackConfig: edgeLatest
35+
- browserStackConfig: firefoxLatest
36+
- browserStackConfig: firefoxOld
37+
- browserStackConfig: safari17
38+
- browserStackConfig: safari26
6039

6140
name: Run tests
6241
runs-on: ubuntu-24.04
@@ -93,21 +72,23 @@ jobs:
9372
cd CSF.Screenplay.JsonToHtmlReport.Template/src
9473
npm ci
9574
cd ../..
75+
- name: Copy appropriate BrowserStack config
76+
run: cp Tests/CSF.Screenplay.Selenium.Tests/Tools/browserstack.${{ matrix.browserStackConfig }}.yml Tests/CSF.Screenplay.Selenium.Tests/browserstack.yml
9677

9778
# Build and test the solution
9879

9980
- name: Build the solution
100-
run: dotnet build -c ${{ env.Configuration }}
81+
run: dotnet build -c ${{ env.Configuration }} -p:BrowserStack=true
10182
- name: Run .NET Selenium tests only
10283
id: dotnet_tests
103-
run: dotnet test -c ${{ env.Configuration }} --no-build Tests/CSF.Screenplay.Selenium.Tests
84+
run: dotnet test -c ${{ env.Configuration }} -p:BrowserStack=true --no-build Tests/CSF.Screenplay.Selenium.Tests
10485
continue-on-error: true
10586

10687
# Post-test tasks (artifacts, overall status)
10788
- name: Upload Screenplay JSON report artifact
10889
uses: actions/upload-artifact@v4
10990
with:
110-
name: Screenplay JSON reports ${{ matrix.browserName }}_${{ matrix.browserVersion }}_${{ matrix.os }}_${{ matrix.osVersion }}
91+
name: Screenplay JSON reports ${{ matrix.browserStackConfig }}
11192
path: Tests/CSF.Screenplay.Selenium.Tests/**/ScreenplayReport_*.json
11293
- name: Convert Screenplay reports to HTML
11394
continue-on-error: true
@@ -121,7 +102,7 @@ jobs:
121102
- name: Upload Screenplay HTML report artifact
122103
uses: actions/upload-artifact@v4
123104
with:
124-
name: Screenplay HTML reports ${{ matrix.browserName }}_${{ matrix.browserVersion }}_${{ matrix.os }}_${{ matrix.osVersion }}
105+
name: Screenplay HTML reports ${{ matrix.browserStackConfig }}
125106
path: Tests/**/ScreenplayReport.html
126107
- name: Fail the build if any test failures
127108
if: steps.dotnet_tests.outcome == 'failure'

‎.github/workflows/dotnetCi.yml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,6 @@ jobs:
4242
DISPLAY: :99
4343
# Change selected factory to VerboseChrome to debug Chrome-related issues
4444
WebDriverFactory__SelectedConfiguration: DefaultChrome
45-
BROWSERSTACK_AUTOMATION: false
4645

4746
steps:
4847
- name: Checkout

‎README.md‎

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,17 @@ One such extension is [CSF.Screenplay.Selenium], which allows Screenplay to cont
2222
CI builds are configured via **GitHub Actions**, with static code analysis & reporting in **SonarCloud**.
2323
This project is tested with BrowserStack.
2424

25+
[![.NET CI](https://github.com/csf-dev/CSF.Screenplay/actions/workflows/dotnetCi.yml/badge.svg)](https://github.com/csf-dev/CSF.Screenplay/actions/workflows/dotnetCi.yml)
26+
[![Test coverage](https://sonarcloud.io/api/project_badges/measure?project=csf-dev_CSF.Screenplay&metric=coverage)](https://sonarcloud.io/summary/new_code?id=csf-dev_CSF.Screenplay)
27+
2528
The contents of the `docs/` directory of this repository is auto-generated from the `CSF.Screenplay.Docs` project.
2629
There is nothing hand-editable inside.
2730

28-
[![.NET CI](https://github.com/csf-dev/CSF.Screenplay/actions/workflows/dotnetCi.yml/badge.svg)](https://github.com/csf-dev/CSF.Screenplay/actions/workflows/dotnetCi.yml)
29-
[![Test coverage](https://sonarcloud.io/api/project_badges/measure?project=csf-dev_CSF.Screenplay&metric=coverage)](https://sonarcloud.io/summary/new_code?id=csf-dev_CSF.Screenplay)
31+
**When running tests** for this solution, there is a [BrowserStack] integration available for
32+
[the tests for the Selenium extension].
33+
It is disabled by default, but instructions for enabling it and using it are available
34+
in [the README.md file in the test directory].
35+
36+
[the tests for the Selenium extension]: https://github.com/csf-dev/CSF.Screenplay/tree/master/Tests/CSF.Screenplay.Selenium.Tests
37+
[BrowserStack]: https://browserstack.com/
38+
[the README.md file in the test directory]: https://github.com/csf-dev/CSF.Screenplay/tree/master/Tests/CSF.Screenplay.Selenium.Tests/README.md

‎Tests/CSF.Screenplay.Selenium.Tests/CSF.Screenplay.Selenium.Tests.csproj‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
<IsPackable>false</IsPackable>
77
<IsTestProject>true</IsTestProject>
88
<RootNamespace>CSF.Screenplay.Selenium</RootNamespace>
9+
<DefineConstants Condition="'$(BrowserStack)' == 'true'">$(DefineConstants);NO_NUNIT_PARALLELISM</DefineConstants>
910
</PropertyGroup>
1011

1112
<ItemGroup>
@@ -23,7 +24,7 @@
2324
<PackageReference Include="Moq" Version="4.20.70" />
2425
<PackageReference Include="AutoFixture.NUnit3" Version="4.18.1" />
2526
<PackageReference Include="AutoFixture.AutoMoq" Version="4.18.1" />
26-
<PackageReference Include="BrowserStack.TestAdapter" Version="0.23.0" />
27+
<PackageReference Include="BrowserStack.TestAdapter" Version="0.23.0" Condition="'$(BrowserStack)' == 'true'" />
2728
<PackageReference Include="System.Text.Json" Version="9.0.7" />
2829
</ItemGroup>
2930

‎Tests/CSF.Screenplay.Selenium.Tests/Properties/LevelOfParallelismAttribute.cs‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
/* If we are running with the BrowserStack SDK enabled then we must switch off parallelism in NUnit,
2+
* or else the two mechanisms 'trample over' one another and we end up receiving very confusing
3+
* test runs, with many duplicate tests.
4+
*/
5+
6+
#if NO_NUNIT_PARALLELISM
7+
[assembly:LevelOfParallelism(1)]
8+
#endif

‎Tests/CSF.Screenplay.Selenium.Tests/README.md‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# Selenium extension tests
2+
3+
This directory contains tests for the Selenium extension.
4+
Note that (for CI/Cross Browser Testing reasons) this test project includes an integration with [the BrowserStack SDK].
5+
BrowserStack is primarily used by the **Cross-browser testing GitHub action** configuration; see `.github\workflows\crossBrowserTesting.yml` in the root of this repository.
6+
7+
## Testing locally (without BrowserStack)
8+
9+
The BrowserStack integration is _disabled by default_ to make local testing as simple as possible.
10+
11+
The WebDriver for local testing is chosen via the `appsettings.json` file.
12+
This file is a configuration for [a universal WebDriver factory].
13+
You may add/adjust WebDrivers present there, to your requirements.
14+
The default configuration is to test with a locally-installed Google Chrome browser.
15+
You may select a different browser/WebDriver for tests by either:
16+
17+
* Setting an environment variable `WebDriverFactory__SelectedConfiguration` to the name of your chosen configuration, for example `VerboseChrome`
18+
* Using a command-line parameter for `dotnet test`:
19+
20+
```cmd
21+
dotnet test ---WebDriverFactory::SelectedConfiguration VerboseChrome
22+
```
23+
24+
[a universal WebDriver factory]: https://csf-dev.github.io/CSF.Extensions.WebDriver/docs/index.html
25+
26+
## Enabling BrowserStack
27+
28+
The BrowserStack integration is via [the BrowserStack SDK].
29+
There are two steps to enabling this, in this project.
30+
31+
[the BrowserStack SDK]: https://www.browserstack.com/blog/introducing-browserstack-sdk/
32+
33+
### An MSBuild property
34+
35+
When using `dotnet build` or `dotnet test`, include the property `BrowserStack` with a value of `true`.
36+
For example:
37+
38+
```cmd
39+
dotnet test -p:BrowserStack=true
40+
```
41+
42+
This will activate two changes in the `.csproj` file in this directory:
43+
44+
* It will install the BrowserStack SDK NuGet package: `BrowserStack.TestAdapter`
45+
* It will disable NUnit's parallelism feature
46+
* _When NUnit parallelism is enabled, it interferes with BrowserStack's own parallelism, creating confusing results_
47+
48+
### Add a `browserstack.yml` configuration
49+
50+
To use the BrowserStack SDK, you must provide a configuration file.
51+
The configuration file must be named `browserstack.yml` and it must be in the directory which contains the tests `.csproj` file _(IE: this directory, which contains this README file)_.
52+
53+
There are seven sample configuration files available in the `Tools/` directory, which target seven different web browser/OS combinations.
54+
These happen to be the targets for the CBT CI run.
55+
You are welcome to copy one of these YAML files into the `CSF.Screenplay.Selenium.Tests` directory and rename it to `browserstack.yml`.
56+
Alternatively [you may craft your own BrowserStack configuration file].
57+
58+
If you use one of the sample configurations, note that the `userName` and `accessKey` properties are omitted.
59+
_You will need to provide your own BrowserStack credentials_.
60+
61+
[you may craft your own BrowserStack configuration file]: https://www.browserstack.com/docs/automate/capabilities

‎Tests/CSF.Screenplay.Selenium.Tests/Tools/README.md‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# Sample BrowserStack configs
2+
3+
These configs are present for Cross Browser Testing (CBT) using [BrowserStack].
4+
They are used only by the appropriate CBT CI builds on GitHub Actions.
5+
6+
Note that each omits the `userName` and `accessKey` properties, as these are secrets.
7+
If you wish to use them, you must provide your own BrowserStack credentials.
8+
9+
[BrowserStack]: https://browserstack.com

Tests/CSF.Screenplay.Selenium.Tests/browserstack.chromeLatest.yml renamed to Tests/CSF.Screenplay.Selenium.Tests/Tools/browserstack.chromeLatest.yml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
1-
parallelsPerPlatform: 5
21
platforms:
32
- browserName: Chrome
43
os: Windows
54
osVersion: 11
65
browserVersion: latest
6+
parallelsPerPlatform: 5
77
browserstackLocal: true
8-
buildName: bs-sdk-build
8+
buildName: CBT Chrome (Latest/Windows 11)
99
projectName: CSF.Screenplay
1010
video: true
1111
testObservability: true

Tests/CSF.Screenplay.Selenium.Tests/browserstack.chromeOld.yml renamed to Tests/CSF.Screenplay.Selenium.Tests/Tools/browserstack.chromeOld.yml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,9 @@ platforms:
33
os: Windows
44
osVersion: 11
55
browserVersion: latest-50
6+
parallelsPerPlatform: 5
67
browserstackLocal: true
7-
buildName: bs-sdk-build
8+
buildName: CBT Chrome (50 versions old/Windows 11)
89
projectName: CSF.Screenplay
10+
video: true
11+
testObservability: true

Tests/CSF.Screenplay.Selenium.Tests/browserstack.yml renamed to Tests/CSF.Screenplay.Selenium.Tests/Tools/browserstack.edgeLatest.yml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,11 +1,11 @@
1-
parallelsPerPlatform: 5
21
platforms:
3-
- browserName: Chrome
2+
- browserName: Edge
43
os: Windows
54
osVersion: 11
65
browserVersion: latest
6+
parallelsPerPlatform: 5
77
browserstackLocal: true
8-
buildName: bs-sdk-build
8+
buildName: CBT Edge (Latest/Windows 11)
99
projectName: CSF.Screenplay
1010
video: true
1111
testObservability: true

0 commit comments

Comments
 (0)