Skip to content

test: raise coverage to 100% (and fix output_to_json metric gating)#522

Merged
zkoppert merged 2 commits into
mainfrom
zkoppert/raise-coverage-to-100
Jun 12, 2026
Merged

test: raise coverage to 100% (and fix output_to_json metric gating)#522
zkoppert merged 2 commits into
mainfrom
zkoppert/raise-coverage-to-100

Conversation

@zkoppert

@zkoppert zkoppert commented Jun 12, 2026

Copy link
Copy Markdown
Collaborator

Pull Request

Proposed Changes

stale-repos ships with --cov-fail-under=80, and the working tree sat at ~90% line coverage with eleven branches uncovered across auth.py, env.py, and stale_repos.py. Raising coverage to 100% also matches the in-flight 100%-coverage push across the rest of the github-community-projects fleet.

While writing the masking-free tests I found a latent bug in output_to_json that all three review models independently flagged: output_to_json was gating the optional daysSinceLastRelease / daysSinceLastPR JSON fields on literal "release" / "pr" keys in the per-repo dict, but set_repo_data (the only production producer of those dicts) only ever writes "days_since_last_release" / "days_since_last_pr". The result: the JSON output, including the value written to $GITHUB_OUTPUT, silently dropped both fields for every real caller, even when ADDITIONAL_METRICS was configured. Markdown output was unaffected because it gates on additional_metrics directly. I fixed this tightly-coupled bug in a separate commit by threading additional_metrics into output_to_json and mirroring the markdown writer's gate.

What changed

  • I added unit tests covering every previously-uncovered branch (see the per-file breakdown below).
  • I added a [report] exclude_lines block to .coveragerc covering pragma: no cover (already implicit) and if __name__ == .__main__.:.
  • I bumped --cov-fail-under from 80 to 100 in the Makefile.
  • I fixed output_to_json to gate optional JSON fields on additional_metrics (separate commit), matching the markdown writer's semantics. The main() caller now passes additional_metrics to output_to_json.

New test coverage (per uncovered branch)

Module Lines New test
auth.py 47 test_auth_to_github_raises_when_connection_is_none
env.py 118-119 test_get_env_vars_loads_dotenv_when_not_test
stale_repos.py 148-149 test_exempt_repos_env_var_is_parsed_and_applied
stale_repos.py 190-193 test_returns_none_on_type_error (get_days_since_last_release)
stale_repos.py 194-195 test_returns_none_when_no_releases
stale_repos.py 210-211 test_returns_none_when_no_pull_requests
stale_repos.py 234 test_unsupported_activity_method_raises_value_error
stale_repos.py 277, 279 test_includes_release_and_pr_keys_when_additional_metrics_set + test_release_only_metric_omits_pr_field
stale_repos.py 286-287 test_writes_to_github_output_when_env_var_set
stale_repos.py 327-328 test_github_exception_on_release_is_logged
stale_repos.py 335-336 test_github_exception_on_pr_is_logged

Before / after coverage

Module Baseline After
auth.py 94% (16 stmts, 1 miss) 100%
env.py 96% (45 stmts, 2 miss) 100%
markdown.py 100% 100%
stale_repos.py 84% (119 stmts, 19 miss) 100%
Total 89.81% 100.00%

Readiness Checklist

Author/Contributor

  • If documentation is needed for this change, has that been included in this pull request
  • run make lint and fix any issues that you have introduced
  • run make test and ensure you have test coverage for the lines you are introducing

Testing

  • make lint - all 5 linters pass (flake8 syntax + warnings, isort, pylint 10.00/10, mypy, black). No suppression comments added.
  • make test - 53 tests pass (6 sub-tests), Required test coverage of 100% reached. Total coverage: 100.00%. Verified on the Python 3.11 baseline used by the Makefile and on the Python 3.12 fallback that uv selected on this machine.
  • Multi-model code review (Opus, Sonnet, GPT-5.5) run on the staged diff before opening. All three independently surfaced the output_to_json gating bug; the fix landed in this PR as a separate commit.

Rollout

  • No production behavior change beyond the output_to_json bug fix. With ADDITIONAL_METRICS unset (the default), the JSON output and $GITHUB_OUTPUT value are byte-identical to today. With ADDITIONAL_METRICS=release or pr, the JSON now correctly includes daysSinceLastRelease / daysSinceLastPR, which downstream consumers (e.g., scripts that read $GITHUB_OUTPUT) may have been silently missing.
  • The threshold bump (--cov-fail-under=80100) means any future PR that drops coverage will fail CI rather than slipping in unnoticed. This is the desired behavior and matches the fleet convention.

Tradeoffs / Alternatives considered

  • For the output_to_json fix I considered (a) gating on "days_since_last_release" in repo_data (always true post set_repo_data, would force the field into the output of callers who don't opt in - silent behavior change) and (b) threading additional_metrics through (chosen, mirrors markdown.py, no behavior change for non-opt-in callers).
  • For the if __name__ == "__main__": line I considered adding # pragma: no cover inline instead of an exclude_lines entry. The .coveragerc rule is more discoverable and matches conventions in other repos in the fleet.

zkoppert and others added 2 commits June 11, 2026 22:14
Add unit tests covering all previously-uncovered branches in auth.py,
env.py, and stale_repos.py, and bump --cov-fail-under from 80 to 100
in the Makefile so the threshold matches reality.

New tests:
- auth_to_github raises ValueError when github3.login returns None
- get_env_vars loads the .env file when test=False
- get_inactive_repos honors EXEMPT_REPOS
- get_days_since_last_release handles TypeError and empty iterators
- get_days_since_last_pr handles empty iterators
- get_active_date raises on unsupported ACTIVITY_METHOD
- output_to_json emits daysSinceLastRelease/daysSinceLastPR when present
- output_to_json appends to GITHUB_OUTPUT when set
- set_repo_data swallows and logs GitHubException on release/pr lookup

Also adds a [report] exclude_lines block to .coveragerc covering
'pragma: no cover' (already implicit) and 'if __name__ == "__main__":'.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Zack Koppert <zkoppert@github.com>
output_to_json was gating the optional release/pr JSON fields on
literal "release" and "pr" keys in repo_data, but set_repo_data
never produces those keys - it always writes "days_since_last_release"
and "days_since_last_pr". As a result, the JSON output (including
the value written to $GITHUB_OUTPUT) silently dropped both fields
for every real caller, even when ADDITIONAL_METRICS was configured.
The markdown output was already correct because it gates on
additional_metrics, not the dict keys.

Mirror the markdown writer: thread additional_metrics through to
output_to_json and gate the optional fields on it. The main() caller
already has the list, so this is a no-op for any caller that does not
opt in.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Zack Koppert <zkoppert@github.com>
@zkoppert zkoppert self-assigned this Jun 12, 2026
@github-actions github-actions Bot added the automation automation label Jun 12, 2026
@zkoppert zkoppert marked this pull request as ready for review June 12, 2026 05:43
@zkoppert zkoppert requested a review from jmeridth as a code owner June 12, 2026 05:43
Copilot AI review requested due to automatic review settings June 12, 2026 05:43

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR raises the project’s CI coverage bar to 100% by adding targeted unit tests for previously uncovered branches, and it fixes a production bug where output_to_json silently omitted optional daysSinceLastRelease / daysSinceLastPR fields even when ADDITIONAL_METRICS was configured. The JSON gating logic is aligned with the markdown writer by explicitly threading additional_metrics into output_to_json from main().

Changes:

  • Add unit tests to cover previously uncovered branches in auth.py, env.py, and stale_repos.py.
  • Fix output_to_json to include optional JSON fields based on additional_metrics (matching markdown semantics).
  • Enforce 100% coverage (--cov-fail-under=100) and exclude __main__ blocks via .coveragerc.
Show a summary per file
File Description
test_stale_repos.py Adds branch-coverage tests for exempt repos parsing, exception paths, JSON optional-metric output, $GITHUB_OUTPUT writing, and set_repo_data exception handling.
test_env.py Adds coverage for loading .env via load_dotenv when test=False.
test_auth.py Adds coverage for the github3.login(...) returns None failure path.
stale_repos.py Fixes JSON optional-metric gating by passing and honoring additional_metrics in output_to_json.
Makefile Raises coverage enforcement threshold from 80% to 100%.
.coveragerc Adds report-time exclusions for pragma: no cover and if __name__ == "__main__":.

Copilot's findings

  • Files reviewed: 6/6 changed files
  • Comments generated: 0

@zkoppert zkoppert merged commit 0cdf6a3 into main Jun 12, 2026
39 checks passed
@zkoppert zkoppert deleted the zkoppert/raise-coverage-to-100 branch June 12, 2026 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automation automation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants