|
| 1 | +# Ch27 - GitHub Code Quality: Code Health & Coverage - Delivery Assurance Guide |
| 2 | + |
| 3 | +> Audience: delivery assurance leads and authorized customer implementation owners. Pair with the customer delivery team `README.md`. |
| 4 | +
|
| 5 | +## Delivery intent |
| 6 | + |
| 7 | +This activity establishes the GitHub Code Quality product on a real customer repository. The delivery team should enable the product, receive rules-based Code Health findings, publish Cobertura coverage from CI, make an explicit threshold decision, and prepare a controlled organization rollout. The customer owns all configuration and remediation decisions; Copilot suggestions are proposed changes that still require normal pull-request review. |
| 8 | + |
| 9 | +## Delivery checkpoints |
| 10 | + |
| 11 | +| Phase | Duration | |
| 12 | +|---|---:| |
| 13 | +| Scope, availability, ownership, and baseline | ~45 min | |
| 14 | +| Enablement and first analysis | ~45 min | |
| 15 | +| Coverage generation and upload | ~75 min | |
| 16 | +| Pull-request finding triage and threshold decision | ~75 min | |
| 17 | +| Organization pilot and handover | ~40 min | |
| 18 | +| **Total** | **~5 hrs** | |
| 19 | + |
| 20 | +## Expected evidence |
| 21 | + |
| 22 | +- Code Quality is enabled in the selected customer repository and its language and runner decisions are captured. |
| 23 | +- Standard findings shows maintainability and reliability scores plus a reviewed baseline backlog. |
| 24 | +- The CI workflow creates Cobertura XML and uploads it using `contents: read` and `code-quality: write`. |
| 25 | +- A PR shows `github-code-quality[bot]` coverage feedback; capture aggregate and per-file evidence. |
| 26 | +- A rules-based PR finding has a recorded fix or a specific dismissal rationale, with Autofix review evidence where used. |
| 27 | +- The customer has an explicit informational-versus-blocking threshold decision and an organization pilot proposal. |
| 28 | + |
| 29 | +## Assurance guidance |
| 30 | + |
| 31 | +### Keep the product boundary clear |
| 32 | + |
| 33 | +Code Quality uses CodeQL technology for maintainability and reliability, but it is not the CodeQL security code-scanning activity. `ghec-ch12` is about security vulnerabilities, advanced CodeQL workflow control, and security-alert triage. This activity is about Code Health, coverage, product findings, and native Code Quality rollout. |
| 34 | + |
| 35 | +Likewise, a generic lint result, code-review bot, or organization ruleset does not enable the product. They can complement the session, but the delivery team must navigate to **Settings -> Security -> Code quality** and demonstrate actual Code Quality outputs. |
| 36 | + |
| 37 | +### Product and tenant constraints |
| 38 | + |
| 39 | +- Confirm that the tenant is GitHub Team or Enterprise Cloud, the enterprise allows Code Quality when applicable, Actions is enabled, and the repository contains a supported language. |
| 40 | +- Coverage is contingent on a real test suite producing **Cobertura XML**. Do not create fake coverage data just to satisfy the exercise. |
| 41 | +- An organization may choose a staged native repository-access rollout. Do not enable every repository without an authorized cohort and rollback decision. |
| 42 | +- Copilot review is optional. Rules-based findings and product enablement must still be demonstrable when Copilot licenses or AI features are unavailable. |
| 43 | + |
| 44 | +### Threshold and remediation guardrails |
| 45 | + |
| 46 | +- Use Error, Warning, and Note severity for rules-based Code Quality decisions only. |
| 47 | +- A threshold should reflect the customer’s repository context and team capacity. Do not prescribe a universal coverage percentage or severity gate. |
| 48 | +- Require the delivery team to explain each dismissal; a dismissal is a risk decision, not a way to hide backlog. |
| 49 | +- Review Autofix patches like any other pull-request change. Never configure automatic merge from a Code Quality suggestion. |
| 50 | + |
| 51 | +## Implementation verification |
| 52 | + |
| 53 | +Use the UI evidence first because Code Quality availability and API details are tenant-dependent. Supplement with the CLI for repository and workflow evidence: |
| 54 | + |
| 55 | +```bash |
| 56 | +ORG=<org> |
| 57 | +REPO=<repository> |
| 58 | + |
| 59 | +# Confirm the authorized repository and its default branch. |
| 60 | +gh repo view "$ORG/$REPO" --json name,visibility,defaultBranchRef |
| 61 | + |
| 62 | +# Confirm the CI workflow grants the narrowly scoped coverage upload permission. |
| 63 | +gh api "repos/$ORG/$REPO/contents/.github/workflows/ci.yml" \ |
| 64 | + -H "Accept: application/vnd.github.raw" \ |
| 65 | + | grep -E "code-quality: write|upload-code-coverage|cobertura|coverage" |
| 66 | + |
| 67 | +# Inspect the latest runs for default-branch and PR coverage evidence. |
| 68 | +gh run list --repo "$ORG/$REPO" --limit 10 |
| 69 | +``` |
| 70 | + |
| 71 | +Verify manually in GitHub: |
| 72 | + |
| 73 | +1. **Settings -> Security -> Code quality:** enabled; languages and runner match the recorded decision. |
| 74 | +2. **Security and quality -> Code quality -> Standard findings:** reliability and maintainability scores are visible, with the reviewed backlog. |
| 75 | +3. **Test pull request:** the coverage comment comes from `github-code-quality[bot]`; it includes aggregate coverage and per-file deltas. |
| 76 | +4. **Files changed:** rules-based comments from `github-code-quality[bot]` are distinguished from optional Copilot comments. |
| 77 | +5. **Checks:** the merge-block behavior matches the customer’s approved Code Quality threshold, if one was configured. |
| 78 | +6. **Organization Settings -> Security -> Code quality:** the pilot repository access selection, owners, and enforcement decision are recorded. |
| 79 | + |
| 80 | +## Delivery risks and recovery |
| 81 | + |
| 82 | +### Code Quality is unavailable or cannot be enabled |
| 83 | + |
| 84 | +**Symptom:** The Code quality setting is absent, unavailable, or cannot be saved. |
| 85 | +**Fix:** Confirm the plan, enterprise allow-list, Actions availability, organization policy, repository access, and supported language. Record the blocker and produce an authorized enablement request rather than substituting another product. |
| 86 | + |
| 87 | +### Coverage does not appear on the pull request |
| 88 | + |
| 89 | +**Symptom:** The workflow passes but there is no coverage comment or branch comparison. |
| 90 | +**Fix:** Confirm a valid Cobertura XML report exists, the upload action executed, `code-quality: write` is present, and runs exist on both the default branch and the PR branch. For forked PRs, preserve the documented upload guard instead of granting write access to untrusted code. |
| 91 | + |
| 92 | +### No quality findings appear |
| 93 | + |
| 94 | +**Symptom:** Code Quality is enabled but the selected PR has no rules-based comments. |
| 95 | +**Fix:** Verify that the changed files use a supported language and the CodeQL - Code Quality check completed. A clean PR is valid evidence; review the default-branch baseline and choose a focused, authorized change only if the customer agrees it is safe to do so. |
| 96 | + |
| 97 | +### Scores appear inconsistent with engineering judgment |
| 98 | + |
| 99 | +**Symptom:** A small repository scores highly despite limited code, or generated code lowers maintainability. |
| 100 | +**Fix:** Interpret scores in repository context. Capture the cause, exclude no code without authorization, and use the baseline and trend rather than a generic target. |
| 101 | + |
| 102 | +### A merge gate blocks unexpectedly |
| 103 | + |
| 104 | +**Symptom:** A PR remains blocked after a visible finding is addressed. |
| 105 | +**Fix:** Recheck every rules-based finding at or above the selected threshold and confirm the configured threshold. Copilot comments do not count toward the rules-based Code Quality gate. |
| 106 | + |
| 107 | +## Progressive support |
| 108 | + |
| 109 | +1. **Availability:** Ask the delivery team to find the repository’s **Settings -> Security -> Code quality** page and identify the selected languages before naming a setting for them. |
| 110 | +2. **Coverage:** Ask which test command creates the report, then have them inspect the output path before adding the upload step. |
| 111 | +3. **Triage:** Ask who authored a PR comment and what its severity means before suggesting a fix, dismissal, or Autofix review. |
| 112 | +4. **Rollout:** Ask which two repositories are safest for a pilot and what result would cause the customer to stop or expand it. |
| 113 | + |
| 114 | +## Handover questions |
| 115 | + |
| 116 | +- Which Code Quality signals will be informational, and which severity threshold—if any—will block merges for this repository? |
| 117 | +- Who owns the Standard findings backlog, and how often will that owner review it with the engineering team? |
| 118 | +- Which pilot repositories meet the rollout criteria, and what objective evidence would trigger rollback or expansion? |
| 119 | + |
| 120 | +## References for delivery leads |
| 121 | + |
| 122 | +- [Enabling GitHub Code Quality](https://docs.github.com/en/code-security/how-tos/maintain-quality-code/enable-code-quality) |
| 123 | +- [Setting up code coverage](https://docs.github.com/en/code-security/how-tos/maintain-quality-code/set-up-code-coverage) |
| 124 | +- [Preventing code quality issues from reaching the default branch](https://docs.github.com/en/code-security/tutorials/improve-code-quality/catch-issues-before-merge) |
| 125 | +- [Metrics and scores reference](https://docs.github.com/en/code-security/reference/code-quality/metrics-and-ratings) |
0 commit comments