Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions BREAKINGS.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ This document outlines all breaking changes introduced in CTBase v0.18.0-beta co

## Non-breaking note (0.18.12-beta)

- **TestRunner progress bar threshold**: Added configurable `full_bar_threshold` parameter to `CTBase.run_tests` (default: 50). Allows users to customize the maximum number of tests for full-resolution progress bar display. Propagated to internal functions `_make_default_on_test_done`, `_format_progress_line`, and `_bar_width`. Documentation and tests updated. No breaking changes; purely additive feature with backward-compatible default. No migration required.
- **TestRunner progress bar threshold**: Added configurable `full_bar_threshold` parameter to `CTBase.Extensions.run_tests` (default: 50). Allows users to customize the maximum number of tests for full-resolution progress bar display. Propagated to internal functions `_make_default_on_test_done`, `_format_progress_line`, and `_bar_width`. Documentation and tests updated. No breaking changes; purely additive feature with backward-compatible default. No migration required.

## Non-breaking note (0.18.11-beta)

Expand Down Expand Up @@ -237,11 +237,11 @@ CTBase.postprocess_coverage()

```julia
# v0.17.4 - Had basic implementation
CTBase.run_tests()
CTBase.Extensions.run_tests()

# v0.18.0-beta - Requires extension
using CTBase.Extensions.TestRunner
CTBase.run_tests()
CTBase.Extensions.run_tests()
```

#### Extension Error Handling
Expand Down Expand Up @@ -565,7 +565,7 @@ using CTBase
using CTBase.Extensions.TestRunner

# Test all functionality
CTBase.run_tests()
CTBase.Extensions.run_tests()
```

### 🧪 Testing Migration
Expand All @@ -591,7 +591,7 @@ end
function test_extension_loading()
# Test that extensions work
using CTBase.Extensions.TestRunner
@test_nowarn CTBase.run_tests(dry_run=true)
@test_nowarn CTBase.Extensions.run_tests(dry_run=true)
end
```

Expand Down
2 changes: 1 addition & 1 deletion CHANGELOGS.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

#### **TestRunner Progress Bar Customization**

- **Configurable threshold**: Added `full_bar_threshold` parameter to `CTBase.run_tests` (default: 50)
- **Configurable threshold**: Added `full_bar_threshold` parameter to `CTBase.Extensions.run_tests` (default: 50)
- **Flexible display**: Users can now customize the maximum number of tests for full-resolution progress bar
- **Terminal adaptation**: Smaller thresholds for narrow terminals, larger for wide displays
- **Internal propagation**: Parameter propagated to `_make_default_on_test_done`, `_format_progress_line`, and `_bar_width`
Expand Down
18 changes: 9 additions & 9 deletions docs/src/guide/test-runner.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ CurrentModule = CTBase
# Test Runner Guide

This guide explains how to set up a modular testing infrastructure for Julia packages using the **TestRunner** extension of `CTBase.jl`.
The entry point is [`CTBase.run_tests`](@ref), activated by loading the `Test` weak dependency. This setup enables granular test execution and is friendly both for human developers and AI agents.
The entry point is [`CTBase.Extensions.run_tests`](@ref), activated by loading the `Test` weak dependency. This setup enables granular test execution and is friendly both for human developers and AI agents.

## Architecture Overview

Expand Down Expand Up @@ -37,7 +37,7 @@ MyPackage.jl/

## Setting up `runtests.jl`

The `runtests.jl` file is the entry point for your test suite. By using [`CTBase.run_tests`](@ref), you enable a powerful mechanism to filter and execute specific tests using command-line arguments. This is crucial for fast iteration cycles.
The `runtests.jl` file is the entry point for your test suite. By using [`CTBase.Extensions.run_tests`](@ref), you enable a powerful mechanism to filter and execute specific tests using command-line arguments. This is crucial for fast iteration cycles.

### Example `test/runtests.jl`

Expand All @@ -51,7 +51,7 @@ using MyPackage # Your package
const TEST_DIR = @__DIR__

# Run tests using the CTBase test runner
CTBase.run_tests(;
CTBase.Extensions.run_tests(;
args=String.(ARGS), # Pass command line arguments
testset_name="MyPackage Tests", # Name of the main testset
available_tests=[ # List of available test groups/files
Expand Down Expand Up @@ -165,10 +165,10 @@ The threshold can be customized via the `progress_bar_threshold` parameter:

```julia
# Use a smaller threshold for narrow terminals
CTBase.run_tests(; args=String.(ARGS), progress_bar_threshold=30)
CTBase.Extensions.run_tests(; args=String.(ARGS), progress_bar_threshold=30)

# Use a larger threshold for wide displays
CTBase.run_tests(; args=String.(ARGS), progress_bar_threshold=100)
CTBase.Extensions.run_tests(; args=String.(ARGS), progress_bar_threshold=100)
```

### Cursor-style display
Expand Down Expand Up @@ -213,13 +213,13 @@ The progress bar correctly detects **both** types of failures:
Set `show_progress_line=false` to disable the entire progress line:

```julia
CTBase.run_tests(; args=String.(ARGS), show_progress_line=false)
CTBase.Extensions.run_tests(; args=String.(ARGS), show_progress_line=false)
```

To display minimal output without the graphical bar, set `show_progress_line=true` and `show_progress_bar=false`:

```julia
CTBase.run_tests(; args=String.(ARGS), show_progress_line=true, show_progress_bar=false)
CTBase.Extensions.run_tests(; args=String.(ARGS), show_progress_line=true, show_progress_bar=false)
# Output: ✓ [01/76] suite/test.jl (0.2s)
```

Expand Down Expand Up @@ -268,7 +268,7 @@ Called after eval completes (or after skip/error). The built-in progress bar is
### Example: custom callbacks

```julia
CTBase.run_tests(;
CTBase.Extensions.run_tests(;
args=String.(ARGS),
on_test_start = info -> begin
print(" [$(info.index)/$(info.total)] $(info.spec)...")
Expand All @@ -293,7 +293,7 @@ CTBase.run_tests(;
You can use glob patterns to organize tests hierarchically:

```julia
CTBase.run_tests(;
CTBase.Extensions.run_tests(;
args=String.(ARGS),
testset_name="MyPackage Tests",
available_tests=[
Expand Down
2 changes: 1 addition & 1 deletion docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ end
- **[Getting Started](getting-started.md)** — installation, mental model, 5-minute walkthrough.
- **[Descriptions](guide/descriptions.md)** — catalogue API, pattern matching, error handling.
- **[Exceptions](guide/exceptions.md)** — exception hierarchy, choosing the right type, best practices.
- **[Test Runner](guide/test-runner.md)** — modular test infrastructure with `CTBase.run_tests`.
- **[Test Runner](guide/test-runner.md)** — modular test infrastructure with `CTBase.Extensions.run_tests`.
- **[Coverage](guide/coverage.md)** — post-processing coverage artifacts with `CTBase.postprocess_coverage`.
- **[API Documentation](guide/api-documentation.md)** — auto-generating per-module API pages.

Expand Down
19 changes: 19 additions & 0 deletions ext/CoveragePostprocessing/CoveragePostprocessing.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
"""
Coverage post-processing backend for CTBase.

This extension implements `CTBase.postprocess_coverage` and provides utilities
to collect `.cov` files, generate reports, and move artifacts into a dedicated
`coverage/` directory.

Most functions in this module have filesystem side effects.
"""
module CoveragePostprocessing

using CTBase: CTBase
using Coverage: Coverage
import DocStringExtensions: TYPEDSIGNATURES

include("helpers.jl")
include("entry_point.jl")

end
108 changes: 108 additions & 0 deletions ext/CoveragePostprocessing/entry_point.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
"""
$(TYPEDSIGNATURES)

Post-process coverage artifacts produced by `Pkg.test(; coverage=true)`.

This implementation:

- Collects coverage source directories under `root_dir` (`src/`, `test/`, `ext/` when present)
- Resets the `coverage/` directory
- Removes stale `.cov` files (keeping the most complete PID suffix when multiple runs exist)
- Optionally generates reports (LCOV + markdown report)
- Moves `.cov` files into `coverage/cov/` (recursively from source dirs)

# Keyword Arguments

- `generate_report::Bool=true`: If `true`, write `coverage/lcov.info` and `coverage/cov_report.md`.
- `root_dir::String=pwd()`: Root directory of the project.
- `dest_dir::String="coverage"`: Destination directory for coverage artifacts.
- `worst_n_files::Int=20`: Maximum number of lowest-covered files to list in the report.
- `max_uncovered_lines::Int=200`: Maximum number of uncovered lines to display in the report.

# Returns

- `Nothing`

# Notes

This function **creates/removes/moves files and directories** under `root_dir`.

# Example

```julia
using CTBase

# CTBase.postprocess_coverage(; generate_report=true, root_dir=pwd())
```
"""
function CTBase.postprocess_coverage(
::CTBase.Extensions.CoveragePostprocessingTag;
generate_report::Bool=true,
root_dir::String=pwd(),
dest_dir::String="coverage",
worst_n_files::Int=20,
max_uncovered_lines::Int=200,
)
# Validate report limits
worst_n_files > 0 || throw(
CTBase.Exceptions.IncorrectArgument(
"worst_n_files must be > 0, got $(worst_n_files)"
),
)
max_uncovered_lines > 0 || throw(
CTBase.Exceptions.IncorrectArgument(
"max_uncovered_lines must be > 0, got $(max_uncovered_lines)"
),
)

println("✓ Coverage post-processing start")

# Define paths relative to root_dir
source_dirs = String[]
for d in ["src", "test", "ext"]
path = joinpath(root_dir, d)
if isdir(path)
push!(source_dirs, path)
end
end

coverage_dir = joinpath(root_dir, dest_dir)
cov_storage_dir = joinpath(coverage_dir, "cov")

_reset_coverage_dir(coverage_dir, cov_storage_dir)

n_cov = _count_cov_files(source_dirs)
if n_cov == 0
throw(
CTBase.Exceptions.PreconditionError(
"Coverage requested but no .cov files were produced";
reason="no .cov files found in src/, test/, or ext/",
suggestion="run Pkg.test(...; coverage=true) to generate coverage data",
),
)
end

_clean_stale_cov_files!(source_dirs)

n_cov = _count_cov_files(source_dirs)
n_cov == 0 && throw(
CTBase.Exceptions.PreconditionError(
"Coverage requested but no usable .cov files were found after cleanup"
),
)

generate_report && _generate_coverage_reports!(
source_dirs, coverage_dir, root_dir, worst_n_files, max_uncovered_lines
)

moved = _collect_and_move_cov_files!(source_dirs, cov_storage_dir)
isempty(moved) && throw(
CTBase.Exceptions.PreconditionError(
"Coverage requested but no .cov files were found to move"
),
)
println("✓ Moved $(length(moved)) .cov files to $cov_storage_dir")

println("✓ Coverage post-processing completed successfully")
return nothing
end
Original file line number Diff line number Diff line change
@@ -1,128 +1,3 @@
"""
Coverage post-processing backend for CTBase.

This extension implements `CTBase.postprocess_coverage` and provides utilities
to collect `.cov` files, generate reports, and move artifacts into a dedicated
`coverage/` directory.

Most functions in this module have filesystem side effects.
"""
module CoveragePostprocessing

using CTBase: CTBase
using Coverage: Coverage
import DocStringExtensions: TYPEDSIGNATURES

# Main entry point for coverage post-processing
"""
$(TYPEDSIGNATURES)

Post-process coverage artifacts produced by `Pkg.test(; coverage=true)`.

This implementation:

- Collects coverage source directories under `root_dir` (`src/`, `test/`, `ext/` when present)
- Resets the `coverage/` directory
- Removes stale `.cov` files (keeping the most complete PID suffix when multiple runs exist)
- Optionally generates reports (LCOV + markdown report)
- Moves `.cov` files into `coverage/cov/` (recursively from source dirs)

# Keyword Arguments

- `generate_report::Bool=true`: If `true`, write `coverage/lcov.info` and `coverage/cov_report.md`.
- `root_dir::String=pwd()`: Root directory of the project.
- `dest_dir::String="coverage"`: Destination directory for coverage artifacts.
- `worst_n_files::Int=20`: Maximum number of lowest-covered files to list in the report.
- `max_uncovered_lines::Int=200`: Maximum number of uncovered lines to display in the report.

# Returns

- `Nothing`

# Notes

This function **creates/removes/moves files and directories** under `root_dir`.

# Example

```julia
using CTBase

# CTBase.postprocess_coverage(; generate_report=true, root_dir=pwd())
```
"""
function CTBase.postprocess_coverage(
::CTBase.Extensions.CoveragePostprocessingTag;
generate_report::Bool=true,
root_dir::String=pwd(),
dest_dir::String="coverage",
worst_n_files::Int=20,
max_uncovered_lines::Int=200,
)
# Validate report limits
worst_n_files > 0 || throw(
CTBase.Exceptions.IncorrectArgument(
"worst_n_files must be > 0, got $(worst_n_files)"
),
)
max_uncovered_lines > 0 || throw(
CTBase.Exceptions.IncorrectArgument(
"max_uncovered_lines must be > 0, got $(max_uncovered_lines)"
),
)

println("✓ Coverage post-processing start")

# Define paths relative to root_dir
source_dirs = String[]
for d in ["src", "test", "ext"]
path = joinpath(root_dir, d)
if isdir(path)
push!(source_dirs, path)
end
end

coverage_dir = joinpath(root_dir, dest_dir)
cov_storage_dir = joinpath(coverage_dir, "cov")

_reset_coverage_dir(coverage_dir, cov_storage_dir)

n_cov = _count_cov_files(source_dirs)
if n_cov == 0
throw(
CTBase.Exceptions.PreconditionError(
"Coverage requested but no .cov files were produced";
reason="no .cov files found in src/, test/, or ext/",
suggestion="run Pkg.test(...; coverage=true) to generate coverage data",
),
)
end

_clean_stale_cov_files!(source_dirs)

n_cov = _count_cov_files(source_dirs)
n_cov == 0 && throw(
CTBase.Exceptions.PreconditionError(
"Coverage requested but no usable .cov files were found after cleanup"
),
)

generate_report && _generate_coverage_reports!(
source_dirs, coverage_dir, root_dir, worst_n_files, max_uncovered_lines
)

moved = _collect_and_move_cov_files!(source_dirs, cov_storage_dir)
isempty(moved) && throw(
CTBase.Exceptions.PreconditionError(
"Coverage requested but no .cov files were found to move"
),
)
println("✓ Moved $(length(moved)) .cov files to $cov_storage_dir")

println("✓ Coverage post-processing completed successfully")
return nothing
end

"""
_reset_coverage_dir(coverage_dir, cov_storage_dir)

Expand Down Expand Up @@ -404,5 +279,3 @@ function _generate_coverage_reports!(
println("✓ Wrote coverage report to $lcov_file")
return nothing
end

end
Loading
Loading