Skip to content

Commit 2b3244f

Browse files
committed
Add run-fast-tests.ps1 and document test environment variables
- Add PowerShell version of run-fast-tests for Windows - Add --keep-outputs/-k flag to run-tests.ps1 (matches bash version from ed683a3) - Document run-fast-tests.sh environment variables with detailed comments - Add comprehensive test environment variables section to README.md Environment variables documented: - QUARTO_TESTS_NO_CONFIG - Skip configure-test-env - QUARTO_TESTS_FORCE_NO_VENV - Skip .venv activation - QUARTO_TEST_KEEP_OUTPUTS - Keep test artifacts (new flag support in PS1) - QUARTO_TEST_VERBOSE - Verbose output - QUARTO_TESTS_NO_CHECK - Legacy/unused variable Changes bring feature parity between bash and PowerShell test runners.
1 parent 4225c70 commit 2b3244f

4 files changed

Lines changed: 118 additions & 11 deletions

File tree

tests/README.md

Lines changed: 64 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -140,20 +140,80 @@ Tests are run using `run-tests.sh` on UNIX, and `run-tests.ps1` on Windows.
140140
./run-tests.ps1 smoke/extensions/extension-render-doc.test.ts
141141
```
142142

143-
#### Prevent configuration for dependencies (R, Julia, Python, ...)
143+
#### Test environment variables
144144

145-
Those files will run `configure-test-env` scripts to check for requirements and set up dependencies (except on Github Action as this is done in the workflow file directly).
146-
You can prevent test configuration locally by setting `QUARTO_TESTS_NO_CONFIG` environment variable to a non-empty value.
145+
The test scripts support several environment variables to control their behavior:
146+
147+
**QUARTO_TESTS_NO_CONFIG**
148+
- Skip running `configure-test-env` scripts
149+
- Useful for faster test runs when environment is already configured
150+
- Tests will still activate `.venv` if present
147151

148152
```bash
149153
QUARTO_TESTS_NO_CONFIG="true" ./run-tests.sh
150154
```
151155

152156
```powershell
153-
$env:QUARTO_TESTS_NO_CONFIG=$true
157+
$env:QUARTO_TESTS_NO_CONFIG="true"
158+
./run-tests.ps1
159+
```
160+
161+
**QUARTO_TESTS_FORCE_NO_VENV** (replaces deprecated `QUARTO_TESTS_FORCE_NO_PIPENV`)
162+
- Skip activating the `.venv` virtual environment
163+
- Tests will use system Python packages instead of UV-managed dependencies
164+
- Use with caution: Python tests may fail if dependencies aren't in system Python
165+
166+
```bash
167+
QUARTO_TESTS_FORCE_NO_VENV="true" ./run-tests.sh
168+
```
169+
170+
```powershell
171+
$env:QUARTO_TESTS_FORCE_NO_VENV="true"
154172
./run-tests.ps1
155173
```
156174

175+
**Quick test runs with run-fast-tests scripts**
176+
177+
For convenience, `run-fast-tests.sh` and `run-fast-tests.ps1` are provided to skip environment configuration:
178+
179+
```bash
180+
# Linux/macOS
181+
./run-fast-tests.sh
182+
183+
# Windows
184+
./run-fast-tests.ps1
185+
```
186+
187+
These scripts set `QUARTO_TESTS_NO_CONFIG` automatically. Use after running `configure-test-env` at least once.
188+
189+
**QUARTO_TEST_KEEP_OUTPUTS** (or use `--keep-outputs`/`-k` flag)
190+
- Keep test output artifacts instead of cleaning them up
191+
- Useful for debugging test failures or inspecting generated files
192+
- Can be set via environment variable or command-line flag
193+
194+
```bash
195+
# Using flag
196+
./run-tests.sh --keep-outputs
197+
./run-tests.sh -k
198+
199+
# Using environment variable
200+
QUARTO_TEST_KEEP_OUTPUTS="true" ./run-tests.sh
201+
```
202+
203+
```powershell
204+
# Using flag
205+
./run-tests.ps1 --keep-outputs
206+
./run-tests.ps1 -k
207+
208+
# Using environment variable
209+
$env:QUARTO_TEST_KEEP_OUTPUTS="true"
210+
./run-tests.ps1
211+
```
212+
213+
**Other environment variables**
214+
- `QUARTO_TEST_VERBOSE` - Enable verbose test output
215+
- `QUARTO_TESTS_NO_CHECK` - Not currently used (legacy variable)
216+
157217
#### About smoke-all tests
158218

159219
`docs/smoke-all/` is a specific folder to run some tests written directly within `.qmd`, `.md` or `.ipynb` files (but files starting with `_` will be ignored). They are run through the `smoke/smoke-all.tests.ts` script. To ease running smoke-all tests, `run-tests.sh` has a special behavior where it will run `./smoke/smoke-all.tests.ts` when passed a `.qmd`, `.md` or `.ipynb` file, not starting with `_`.

tests/run-fast-tests.ps1

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
# Run tests without reconfiguring the environment (faster for repeated test runs)
2+
#
3+
# Environment variables set:
4+
# QUARTO_TESTS_NO_CONFIG - Skip running configure-test-env.ps1
5+
# (don't reinstall/update R, Python, Julia dependencies)
6+
#
7+
# Note: This still activates .venv if present, so Python tests use correct dependencies.
8+
# Only use after running configure-test-env.ps1 at least once to set up environment.
9+
10+
$env:QUARTO_TESTS_NO_CONFIG="true"
11+
& .\run-tests.ps1 @args

tests/run-fast-tests.sh

Lines changed: 14 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,16 @@
11
#!/usr/bin/env bash
22

3-
QUARTO_TESTS_FORCE_NO_PIPENV=true QUARTO_TESTS_NO_CONFIG=true QUARTO_TESTS_NO_CHECK=true ./run-tests.sh $*
3+
# Run tests without reconfiguring the environment (faster for repeated test runs)
4+
#
5+
# Environment variables set:
6+
# QUARTO_TESTS_FORCE_NO_VENV - Skip activating .venv virtual environment
7+
# (tests use system Python packages instead)
8+
# QUARTO_TESTS_NO_CONFIG - Skip running configure-test-env.sh
9+
# (don't reinstall/update R, Python, Julia dependencies)
10+
# QUARTO_TESTS_NO_CHECK - Not currently used by run-tests.sh
11+
# (kept for backward compatibility in case checked elsewhere)
12+
#
13+
# Note: Only use after running configure-test-env.sh at least once to set up environment.
14+
# Consider removing QUARTO_TESTS_FORCE_NO_VENV if Python tests require .venv packages.
15+
16+
QUARTO_TESTS_FORCE_NO_VENV=true QUARTO_TESTS_NO_CONFIG=true QUARTO_TESTS_NO_CHECK=true ./run-tests.sh $*

tests/run-tests.ps1

Lines changed: 29 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -80,39 +80,62 @@ if ( $MyInvocation.Line -eq "" ) {
8080
# when script is ran from a child process using -F
8181
# e.g pwsh -F ./run-tests.ps1 smoke/smoke-all.test.ts -- docs\smoke-all\2023\02\08\4272.qmd
8282
$customArgs = $MyInvocation.UnboundArguments
83+
} elseif ($MyInvocation.InvocationName -eq '&') {
84+
# when script is called via call operator from another script
85+
# e.g & .\run-tests.ps1 @args
86+
# Use UnboundArguments directly as it contains the actual arguments passed
87+
$customArgs = $MyInvocation.UnboundArguments
8388
} elseif ($MyInvocation.Line -match "^[.] '[^']*'") {
8489
# when script is ran from a child process using -command
8590
# e.g pwsh -command ". 'run-tests.ps1' smoke/smoke-all.test.ts -- docs\smoke-all\2023\02\08\4272.qmd"
8691
# This is what happens on GHA when using 'run: |' and 'shell: pwsh'
8792
$argList = ($MyInvocation.Line -replace "^[.] '[^']*'\s*" -split '[;|]')[0].Trim()
8893
# Extract the argument list from the invocation command line.
89-
94+
9095
# Use Invoke-Expression with a Write-Output call to parse the raw argument list,
9196
# performing evaluation and splitting it into an array:
92-
$customArgs = $argList ? @(Invoke-Expression "Write-Output -- $argList") : @()
97+
$customArgs = $argList ? @(Invoke-Expression "Write-Output -- $argList") : @()
9398
} else {
9499
# When script is called from main process
95100
# e.g ./run-tests.ps1 smoke/smoke-all.test.ts -- docs\smoke-all\2023\02\08\4272.qmd
96101
$argList = ($MyInvocation.Line -replace ('^.*' + [regex]::Escape($MyInvocation.InvocationName)) -split '[;|]')[0].Trim()
97102
# Extract the argument list from the invocation command line.
98-
103+
99104
# Use Invoke-Expression with a Write-Output call to parse the raw argument list,
100105
# performing evaluation and splitting it into an array:
101-
$customArgs = $argList ? @(Invoke-Expression "Write-Output -- $argList") : @()
106+
$customArgs = $argList ? @(Invoke-Expression "Write-Output -- $argList") : @()
107+
}
108+
109+
# Check if keep-outputs mode is enabled and filter it from arguments
110+
$KEEP_OUTPUTS = $false
111+
$FILTERED_CUSTOM_ARGS = @()
112+
foreach ($arg in $customArgs) {
113+
if ($arg -eq "--keep-outputs" -or $arg -eq "-k") {
114+
$KEEP_OUTPUTS = $true
115+
} else {
116+
$FILTERED_CUSTOM_ARGS += $arg
117+
}
118+
}
119+
$customArgs = $FILTERED_CUSTOM_ARGS
120+
121+
if ($KEEP_OUTPUTS) {
122+
$env:QUARTO_TEST_KEEP_OUTPUTS = "true"
123+
Write-Host "> Keep outputs mode enabled - test artifacts will not be deleted"
102124
}
103125

104126
## Short version syntax to run smoke-all.test.ts
105127
## Only use if different than ./run-test.ps1 ./smoke/smoke-all.test.ts
106128
If ($customArgs[0] -notlike "*smoke-all.test.ts") {
107-
129+
108130
$SMOKE_ALL_TEST_FILE="./smoke/smoke-all.test.ts"
109131
# Check file argument
110132
$SMOKE_ALL_FILES=@()
111133
$TESTS_TO_RUN=@()
112134

113135
ForEach ($file in $customArgs) {
114136
$filename=$(Split-Path -Path $file -Leaf)
115-
If ($filename -match "^^[^_].*[.]qmd$" -Or $filename -match "^[^_].*[.]ipynb$" -Or $filename -match "^[^_].*[.]md$") {
137+
138+
If ($filename -match "^[^_].*[.]qmd$" -Or $filename -match "^[^_].*[.]ipynb$" -Or $filename -match "^[^_].*[.]md$") {
116139
$SMOKE_ALL_FILES+=$file
117140
} elseif ($file -Like "*.ts") {
118141
$TESTS_TO_RUN+=$file

0 commit comments

Comments
 (0)