|
1 | 1 | (chap_testCI)= |
2 | 2 | # Continuous Integration (CI) and Unit Testing |
3 | 3 |
|
4 | | -Give a tutorial on how to set up CI and coverage in a repository. Test driven development (TDD), references. Differences among unit testing, regression testing, and CI. |
| 4 | +As repositories grow, contributors need more than good intentions to keep the default branch working. They need automatic checks that run consistently for every proposed change. This is the role of {term}`continuous integration`, usually abbreviated as CI. |
| 5 | + |
| 6 | +## What CI does |
| 7 | + |
| 8 | +Continuous integration is the practice of running automated checks whenever code is pushed or a pull request is opened or updated. In GitHub-based projects, these checks are often run with GitHub Actions, but the basic idea is broader than any one platform. |
| 9 | + |
| 10 | +Typical CI jobs include: |
| 11 | + |
| 12 | +* running unit tests |
| 13 | +* running style or lint checks |
| 14 | +* building documentation |
| 15 | +* checking that notebooks or examples execute successfully |
| 16 | +* measuring code coverage |
| 17 | + |
| 18 | +The purpose of CI is not to replace human review. It is to catch routine problems quickly and consistently so reviewers can focus on design, correctness, and maintainability. |
| 19 | + |
| 20 | +## Unit testing, regression testing, and CI |
| 21 | + |
| 22 | +These ideas are related, but they are not identical. |
| 23 | + |
| 24 | +### Unit testing |
| 25 | + |
| 26 | +Unit tests check small, individual pieces of the code base. A unit test might verify that one function returns the expected output for a known input. |
| 27 | + |
| 28 | +### Regression testing |
| 29 | + |
| 30 | +Regression tests check that behavior that used to work still works after a change. A regression test may be a unit test, but it can also be a larger end-to-end or integration-style test that guards against reintroducing a known bug. |
| 31 | + |
| 32 | +### Continuous integration |
| 33 | + |
| 34 | +CI is the automation framework that runs tests and other checks whenever the repository changes. CI is the delivery mechanism, not the test itself. |
| 35 | + |
| 36 | +## Why this matters in collaborative repositories |
| 37 | + |
| 38 | +In a shared repository, a broken default branch creates friction for everyone. CI lowers that risk by making sure that each pull request is evaluated under the same rules. |
| 39 | + |
| 40 | +For PSLmodels-style collaboration, CI is especially valuable because repositories often combine code, documentation, examples, and research outputs. A pull request may look harmless in one file while breaking something important elsewhere. |
| 41 | + |
| 42 | +## What a healthy CI setup looks like |
| 43 | + |
| 44 | +A good CI setup is usually: |
| 45 | + |
| 46 | +* fast enough that contributors will actually pay attention to it |
| 47 | +* reliable enough that failures usually mean something real |
| 48 | +* broad enough to test the most important project behavior |
| 49 | +* visible in pull requests so contributors and reviewers can act on results |
| 50 | + |
| 51 | +For a small project, that might mean only a few checks. For a mature project, it might involve several operating systems, multiple Python versions, documentation builds, and separate slow-running jobs. |
| 52 | + |
| 53 | +## A simple CI path for a Python project |
| 54 | + |
| 55 | +Many Python repositories start with something like the following: |
| 56 | + |
| 57 | +1. install project dependencies |
| 58 | +2. run the test suite |
| 59 | +3. run formatting or linting checks |
| 60 | +4. optionally collect coverage |
| 61 | + |
| 62 | +If the project publishes documentation, another good early step is to add a documentation build check so broken examples or malformed Markdown are caught before merge. |
| 63 | + |
| 64 | +## Code coverage |
| 65 | + |
| 66 | +Coverage tools measure how much of your code is exercised by the test suite. Coverage is useful, but it should be interpreted carefully. |
| 67 | + |
| 68 | +Coverage can answer: |
| 69 | + |
| 70 | +* Did our tests execute this function at all? |
| 71 | +* Which files have very little test attention? |
| 72 | + |
| 73 | +Coverage cannot answer: |
| 74 | + |
| 75 | +* Are the tests meaningful? |
| 76 | +* Do the tests assert the right behavior? |
| 77 | +* Are the most important edge cases covered? |
| 78 | + |
| 79 | +High coverage is not the same thing as high quality. Still, coverage reports can be useful for spotting neglected parts of a code base. |
| 80 | + |
| 81 | +## Test-driven development |
| 82 | + |
| 83 | +Test-driven development, or TDD, is the practice of writing a failing test before implementing the code that makes it pass. Some teams use TDD heavily, while others use it selectively. |
| 84 | + |
| 85 | +Even if a project does not follow TDD strictly, it is still a strong habit to add or update tests whenever behavior changes. |
| 86 | + |
| 87 | +## Good contributor habits around CI |
| 88 | + |
| 89 | +* Run relevant tests locally before pushing, when practical. |
| 90 | +* Read the CI failure output rather than guessing. |
| 91 | +* Keep fixes for a failing check on the same pull request branch. |
| 92 | +* Treat flaky tests as real maintenance problems. |
| 93 | +* Make sure documentation changes are tested if the repo has docs automation. |
| 94 | + |
| 95 | +## Good maintainer habits around CI |
| 96 | + |
| 97 | +* Keep required checks clear and documented. |
| 98 | +* Avoid adding slow or brittle checks without strong benefit. |
| 99 | +* Make failure messages readable. |
| 100 | +* Update CI when the supported environment changes. |
| 101 | +* Keep secrets and deployment credentials out of general-purpose workflows. |
| 102 | + |
| 103 | +## Start simple |
| 104 | + |
| 105 | +Beginners sometimes think a project needs an elaborate CI system before it can benefit from automation. In practice, even a single workflow that installs dependencies and runs tests can dramatically improve collaboration quality. |
| 106 | + |
| 107 | +The best CI setup is usually one that the team understands, trusts, and maintains. |
0 commit comments