Merge pull request #22 from datalogics-kam/pdfcloud-5564-coverage

PDFCLOUD-5564 Better coverage support; fix coverage gaps

Author: datalogics-cgreen

.github/workflows/test-and-publish.yml

@@ -38,6 +38,37 @@ jobs:
         run: uvx nox --python ${{ matrix.python-version }} --session tests -- --no-parallel
         env:
           PDFREST_API_KEY: ${{ secrets.PDFREST_API_KEY }}
+      - name: Fetch base branch for diff-cover
+        if: github.event_name == 'pull_request'
+        run: |
+          if git rev-parse --is-shallow-repository | grep -q true; then
+            git fetch --no-tags --prune origin ${{ github.base_ref }} --unshallow
+          else
+            git fetch --no-tags --prune origin ${{ github.base_ref }}
+          fi
+      - name: Run diff-cover (new code must be >= 90%)
+        if: github.event_name == 'pull_request'
+        run: >
+          uv run diff-cover coverage/py${{ matrix.python-version }}/coverage.xml
+          --compare-branch origin/${{ github.base_ref }}
+          --fail-under 90
+          --format markdown:coverage/py${{ matrix.python-version }}/diff-cover.md
+      - name: Check client class function coverage
+        run: >
+          uv run python scripts/check_class_function_coverage.py
+          coverage/py${{ matrix.python-version }}/coverage.json
+          --class PdfRestClient
+          --class AsyncPdfRestClient
+          --class _FilesClient
+          --class _AsyncFilesClient
+          --fail-under 90
+          --markdown-report coverage/py${{ matrix.python-version }}/class-function-coverage.md
+      - name: Upload coverage reports
+        if: always()
+        uses: actions/upload-artifact@v4
+        with:
+          name: coverage-${{ matrix.python-version }}
+          path: coverage/py${{ matrix.python-version }}
 
   examples:
     name: Examples (Python ${{ matrix.python-version }})

.gitignore

@@ -453,3 +453,4 @@ $RECYCLE.BIN/
 *.speedscope.json
 
 !pyrightconfig.json
+/coverage/

AGENTS.md

@@ -19,15 +19,16 @@
 - `uv run pre-commit run --all-files` — enforce formatting and lint rules before
   pushing.
 - `uv run pytest` — execute the suite with the active interpreter.
-- `scripts/check_test_parity.sh` — run changed tests and report sync/async
-  parity gaps (accepts optional base/head refs, defaults to
-  `upstream/main..HEAD`).
 - `uv build` — produce wheels and sdists identical to the release workflow.
 - `uvx nox -s tests` — create matrix virtualenvs via nox and execute the pytest
   session.
 - `nox` executes pytest sessions with built-in parallelism; when invoking pytest
   directly use `pytest -n 8 --maxschedchunk 2` to mirror the parallel test
   scheduling and keep runtimes predictable.
+- Coverage reports (XML/Markdown/HTML) are produced by the nox `tests` session
+  and stored under `coverage/py<version>/` (for example,
+  `coverage/py3.12/coverage.xml`, `coverage/py3.12/coverage.md`,
+  `coverage/py3.12/html/`).
 
 ## Coding Style & Naming Conventions
 
@@ -138,6 +139,19 @@
   description with the reason and the follow-up plan; otherwise reviewers should
   block the change. Treat this as a release gate on par with unit tests.
 
+- **Client coverage criteria:** `PdfRestClient` and `AsyncPdfRestClient` are
+  customer-facing entry points and must retain high coverage. Every public
+  client method must have at least one unit test that exercises the REST call
+  path (MockTransport + request assertions), with distinct sync and async tests.
+  Optional payload branches (`pages`, `output`, `rgb_color`, etc.) need explicit
+  coverage so serialization regressions are caught.
+
+- **Class function coverage scope:** The class coverage gate targets the main
+  client-facing classes (`PdfRestClient`, `AsyncPdfRestClient`, `_FilesClient`,
+  `_AsyncFilesClient`). For these classes, underscore-prefixed methods are
+  intentionally in scope and should be covered as part of the interface
+  contract.
+
 - Write pytest tests: files named `test_*.py`, test functions `test_*`, fixtures
   in `conftest.py` where shared.
 

README.md

@@ -37,8 +37,11 @@ Run the test suite with:
 uv run pytest
 ```
 
-Check sync/async parity for changed tests (defaults to `upstream/main..HEAD`):
+Check per-function coverage for the client classes:
 
 ```bash
-scripts/check_test_parity.sh
+uvx nox -s class-coverage
 ```
+
+To reuse an existing `coverage/py<version>/coverage.json` without rerunning
+tests, add `-- --no-tests` (and optional `--coverage-json path`).

TESTING_GUIDELINES.md

@@ -13,13 +13,23 @@ iteration required.
   request customization, validation failures, file helpers, and live calls. Do
   not hide the transport behind a parameter; the test name itself should reveal
   which client is under test.
-- **Check parity regularly.** Run `scripts/check_test_parity.sh` (defaults to
-  `upstream/main..HEAD`) to spot missing sync/async counterparts, keeping
-  parameterized test IDs aligned between transports.
+- **Maintain high client coverage.** `PdfRestClient` and `AsyncPdfRestClient`
+  are the primary customer-facing entry points. Every public client method must
+  have at least one unit test that exercises the REST call path (MockTransport
+  asserting method/path/headers/body). Optional payload branches (for example,
+  `pages`, `output`, `rgb_color`, and output-prefix fields) require explicit
+  tests so serialization differences are caught early.
+- **Check client coverage regularly.** Run `uvx nox -s class-coverage` to
+  enforce minimum function-level coverage for `PdfRestClient` and
+  `AsyncPdfRestClient`.
 - **Exercise both sides of the contract.** Hermetic tests (via
   `httpx.MockTransport`) validate serialization and local validation. Live
   suites prove the server behaves the same way, including invalid literal
   handling.
+- **Know where coverage lands.** The nox `tests` session writes coverage reports
+  to `coverage/py<version>/` (XML, Markdown, and HTML). Example:
+  `coverage/py3.12/coverage.xml`, `coverage/py3.12/coverage.md`,
+  `coverage/py3.12/html/`.
 - **Reset global state per test.** Use
   `monkeypatch.delenv("PDFREST_API_KEY", raising=False)` (or `setenv`) so
   clients never inherit accidental API keys. Patch `importlib.metadata.version`

noxfile.py

@@ -17,6 +17,79 @@
 PROJECT_ROOT = Path(__file__).resolve().parent
 DEFAULT_EXAMPLE_PYTHON = "3.11"
 EXAMPLES_DIR = PROJECT_ROOT / "examples"
+DEFAULT_COVERAGE_CLASSES = (
+    "PdfRestClient",
+    "AsyncPdfRestClient",
+    "_FilesClient",
+    "_AsyncFilesClient",
+)
+
+
+def _install_test_dependencies(session: nox.Session) -> None:
+    _ = session.run_install(
+        "uv",
+        "sync",
+        "--no-default-groups",
+        "--group=dev",
+        "--reinstall-package=pdfrest",
+        f"--python={session.virtualenv.location}",
+        env={"UV_PROJECT_ENVIRONMENT": session.virtualenv.location},
+    )
+
+
+def _coverage_dir_for_session(session: nox.Session) -> Path:
+    coverage_dir = PROJECT_ROOT / "coverage" / f"py{session.python}"
+    coverage_dir.mkdir(parents=True, exist_ok=True)
+    return coverage_dir
+
+
+def _pytest_args_from_session(session: nox.Session) -> list[str]:
+    parser = argparse.ArgumentParser(add_help=False)
+    _ = parser.add_argument("--no-parallel", action="store_true")
+    _ = parser.add_argument("-n", "--workers", "--numprocesses")
+    custom, remaining = parser.parse_known_args(session.posargs)
+
+    pytest_args = list(remaining)
+
+    if custom.no_parallel:
+        return pytest_args
+    if custom.workers:
+        pytest_args[:0] = ["-n", custom.workers, "--maxschedchunk", "2"]
+    else:
+        pytest_args[:0] = ["-n", "8", "--maxschedchunk", "2"]
+
+    return pytest_args
+
+
+def _run_pytest_with_coverage(session: nox.Session, pytest_args: Iterable[str]) -> Path:
+    coverage_dir = _coverage_dir_for_session(session)
+    htmlcov_dir = coverage_dir / "html"
+    xml_report = coverage_dir / "coverage.xml"
+    md_report = coverage_dir / "coverage.md"
+    json_report = coverage_dir / "coverage.json"
+    _ = session.run(
+        "pytest",
+        "--cov=pdfrest",
+        "--cov-report=term-missing",
+        f"--cov-report=html:{htmlcov_dir}",
+        f"--cov-report=xml:{xml_report}",
+        f"--cov-report=markdown:{md_report}",
+        f"--cov-report=json:{json_report}",
+        *pytest_args,
+    )
+    return coverage_dir
+
+
+def _parse_class_values(values: Iterable[str]) -> list[str]:
+    classes: list[str] = []
+    for value in values:
+        if not value:
+            continue
+        for item in value.split(","):
+            item = item.strip()
+            if item:
+                classes.append(item)
+    return classes
 
 
 @dataclass(frozen=True)
@@ -162,40 +235,61 @@ def _infer_python_version_from_path(script: Path) -> str | None:
 
 @nox.session(name="tests", python=python_versions, reuse_venv=True)
 def tests(session: nox.Session) -> None:
-    # Define only custom flags
+    pytest_args = _pytest_args_from_session(session)
+
+    _install_test_dependencies(session)
+    _ = _run_pytest_with_coverage(session, pytest_args)
+
+
+@nox.session(name="class-coverage", python=python_versions, reuse_venv=True)
+def class_coverage(session: nox.Session) -> None:
     parser = argparse.ArgumentParser(add_help=False)
     _ = parser.add_argument("--no-parallel", action="store_true")
-    _ = parser.add_argument(
-        "-n", "--workers", "--numprocesses"
-    )  # e.g., -n 4 to set workers
+    _ = parser.add_argument("-n", "--workers", "--numprocesses")
+    _ = parser.add_argument("--no-tests", action="store_true")
+    _ = parser.add_argument("--coverage-json", type=Path, default=None)
+    _ = parser.add_argument("--markdown-report", type=Path, default=None)
+    _ = parser.add_argument("--fail-under", type=float, default=90.0)
+    _ = parser.add_argument("--class", dest="classes", action="append", default=[])
+    _ = parser.add_argument("--classes", dest="classes_csv", default="")
     custom, remaining = parser.parse_known_args(session.posargs)
 
     pytest_args = list(remaining)
+    if not custom.no_parallel:
+        if custom.workers:
+            pytest_args[:0] = ["-n", custom.workers, "--maxschedchunk", "2"]
+        else:
+            pytest_args[:0] = ["-n", "8", "--maxschedchunk", "2"]
 
-    # Default to parallel unless disabled or overridden
-    if custom.no_parallel:
-        pass
-    elif custom.workers:
-        pytest_args[:0] = ["-n", custom.workers, "--maxschedchunk", "2"]
+    if custom.no_tests:
+        coverage_dir = _coverage_dir_for_session(session)
     else:
-        pytest_args[:0] = ["-n", "8", "--maxschedchunk", "2"]
+        _install_test_dependencies(session)
+        coverage_dir = _run_pytest_with_coverage(session, pytest_args)
 
-    _ = session.run_install(
-        "uv",
-        "sync",
-        "--no-default-groups",
-        "--group=dev",
-        "--reinstall-package=pdfrest",
-        f"--python={session.virtualenv.location}",
-        env={"UV_PROJECT_ENVIRONMENT": session.virtualenv.location},
-    )
-    _ = session.run(
-        "pytest",
-        "--cov=pdfrest",
-        "--cov-report=term-missing",
-        *pytest_args,
+    coverage_json = custom.coverage_json or (coverage_dir / "coverage.json")
+    markdown_report = custom.markdown_report or (
+        coverage_dir / "class-function-coverage.md"
     )
 
+    classes = _parse_class_values([*custom.classes, custom.classes_csv])
+    if not classes:
+        classes = list(DEFAULT_COVERAGE_CLASSES)
+
+    script_args = [
+        "python",
+        str(PROJECT_ROOT / "scripts" / "check_class_function_coverage.py"),
+        str(coverage_json),
+        "--fail-under",
+        f"{custom.fail_under}",
+        "--markdown-report",
+        str(markdown_report),
+    ]
+    for class_name in classes:
+        script_args.extend(["--class", class_name])
+
+    _ = session.run(*script_args)
+
 
 @nox.session(name="examples", python=python_versions, reuse_venv=True)
 def run_examples(session: nox.Session) -> None:

pyproject.toml

@@ -34,6 +34,7 @@ dev = [
     "nox>=2025.5.1",
     "basedpyright>=1.34.0",
     "python-dotenv>=1.0.1",
+    "diff-cover>=10.2.0",
 ]
 
 [tool.pytest.ini_options]

pyrightconfig.json

@@ -22,6 +22,9 @@
     {
       "root": "src"
     },
+    {
+      "root": "scripts"
+    },
     {
       "root": "examples",
       "pythonVersion": "3.11"