Skip to content

test: expand mutmut coverage to all files in src/flameconnect#33

Merged
deviantintegral merged 3 commits into
mainfrom
copilot/update-paths-to-mutate
Feb 26, 2026
Merged

test: expand mutmut coverage to all files in src/flameconnect#33
deviantintegral merged 3 commits into
mainfrom
copilot/update-paths-to-mutate

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Feb 26, 2026

paths_to_mutate in setup.cfg only listed 4 files, silently excluding cli.py, const.py, exceptions.py, models.py, and the entire tui/ subdirectory (13 files) from mutation testing.

Changes

Expand mutation coverage

Replace the explicit file list with the package directory so mutmut discovers all Python files automatically:

# Before
[mutmut]
paths_to_mutate =
    src/flameconnect/protocol.py
    src/flameconnect/client.py
    src/flameconnect/auth.py
    src/flameconnect/b2c_login.py

# After
[mutmut]
paths_to_mutate = src/flameconnect

Fix test compatibility with mutmut

Remove the overly-aggressive builtins.__import__ patch from test_tui_import_errorsys.modules["flameconnect.tui"] = None alone is sufficient to trigger ImportError, and the blanket import patch was breaking mutmut's coverage tracking during the stats phase.

Speed up mutmut runtime

The expanded coverage increased mutmut runtime to ~35 minutes. Three optimizations were added to reduce this:

  • --max-children 4 (was 2) in CI — GitHub Actions ubuntu-24.04 runners have 4 vCPUs
  • do_not_mutate — excludes */__init__.py, const.py, and exceptions.py (re-exports, constants, and trivial class declarations that don't benefit from mutation testing), reducing from 23 to 19 mutated files
  • max_stack_depth = 8 — limits how deep mutmut traces call stacks when mapping tests to mutants, speeding up the stats/mapping phase

💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI changed the title [WIP] Expand paths_to_mutate in setup.cfg for src files Expand mutmut coverage to all files in src/flameconnect Feb 26, 2026
@deviantintegral deviantintegral changed the title Expand mutmut coverage to all files in src/flameconnect tests: expand mutmut coverage to all files in src/flameconnect Feb 26, 2026
@deviantintegral deviantintegral marked this pull request as ready for review February 26, 2026 16:19
@deviantintegral deviantintegral changed the title tests: expand mutmut coverage to all files in src/flameconnect test: expand mutmut coverage to all files in src/flameconnect Feb 26, 2026
@deviantintegral
Copy link
Copy Markdown
Owner

@copilot Can you investigate and fix the test error in the CI job?

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Feb 26, 2026

@copilot Can you investigate and fix the test error in the CI job?

The CI failure was in test_tui_import_error during mutmut's stats phase. The test blanket-patched builtins.__import__ with side_effect=ImportError("no tui"), which blocked ALL imports — this broke mutmut's coverage tracking that runs during the stats collection.

The fix in c4af6e3 removes the builtins.__import__ patch since patch.dict("sys.modules", {"flameconnect.tui": None}) alone is sufficient to trigger ImportError when from flameconnect.tui import run_tui executes. All 1048 tests pass locally.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • astral.sh
    • Triggering command: /usr/bin/curl curl -LsSf REDACTED (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@deviantintegral
Copy link
Copy Markdown
Owner

@copilot mutmut is taking ~35 minutes to run. That seems much slower than other mutation testing systems I've used given the size of this project. Is there anything that can be done to speed things up?

Copy link
Copy Markdown
Contributor Author

Copilot AI commented Feb 26, 2026

@copilot mutmut is taking ~35 minutes to run. That seems much slower than other mutation testing systems I've used given the size of this project. Is there anything that can be done to...

Applied three optimizations in 77e5868:

  1. --max-children 4 (was 2) — ubuntu-24.04 runners have 4 vCPUs, so this doubles parallelism
  2. do_not_mutate — excludes */__init__.py, const.py, and exceptions.py (re-exports, constants, and trivial class declarations that don't benefit from mutation testing). Reduces from 23 to 19 mutated files.
  3. max_stack_depth = 8 — limits how deep mutmut traces call stacks when mapping tests to mutants, speeding up the stats/mapping phase

These should roughly halve the runtime. If it's still too slow, the next lever would be mutate_only_covered_lines = true which uses coverage data to skip mutating uncovered lines entirely.

Copilot AI and others added 3 commits February 26, 2026 13:28
Co-authored-by: deviantintegral <255023+deviantintegral@users.noreply.github.com>
The test patched builtins.__import__ with a blanket ImportError, blocking
ALL imports. This broke mutmut's stats phase which needs working imports
for coverage tracking. Setting sys.modules["flameconnect.tui"] = None
alone is sufficient to trigger ImportError on `from flameconnect.tui import run_tui`.

Co-authored-by: deviantintegral <255023+deviantintegral@users.noreply.github.com>
…children 4

- Exclude __init__.py, const.py, exceptions.py via do_not_mutate (re-exports,
  constants, and trivial class declarations don't benefit from mutation testing)
- Add max_stack_depth=8 to limit test-to-mutant call stack tracing depth
- Increase --max-children from 2 to 4 (GitHub Actions ubuntu-24.04 has 4 vCPUs)

Co-authored-by: deviantintegral <255023+deviantintegral@users.noreply.github.com>
@deviantintegral deviantintegral force-pushed the copilot/update-paths-to-mutate branch from 77e5868 to 56138a2 Compare February 26, 2026 18:29
@deviantintegral deviantintegral enabled auto-merge (squash) February 26, 2026 18:29
@deviantintegral deviantintegral merged commit 60ba2f8 into main Feb 26, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants