Skip to content

README.md

Latest commit

 

History

History
169 lines (131 loc) · 7.67 KB

README.md

File metadata and controls

169 lines (131 loc) · 7.67 KB
Raw

Nixpkgs Committer management script testing

The recommended way to test these scripts is to run them in a GitHub test organisation with the right setup. Since creating your own takes some time, you can just ask @infinisil to get added to his test setup instead, whose identifiers will be used here.

Setup

One-time

Per-user

Once you have the above setup (or got @infinisil to add yourself to his), you have to prepare the following:

  • Add some activity of yours to the active repo. To match Nixpkgs, it’s recommended to do this by setting the “Default commit message” for merge commits to “Pull request title”, then creating and merging a PR. You can do this from the web interface.
  • Get the GitHub CLI available (pkgs.github-cli) and authenticate it using gh auth login
  • A local Git clone of this repository with the origin remote set to the test repository: 
    git remote add upstream git@github.com:NixOS/nixpkgs-committers.git
git remote set-url origin git@github.com:infinisil-test-org/nixpkgs-committers.git

Testing sync.sh

This script has no external effects and as such can be easily tested by running:

scripts/sync.sh infinisil-test-org actors members-test

Check that it synchronises the files in the members-test directory with the team members of the actors team.

Testing nomination.sh

This script does not depend on the current repository, but has some external effects. For testing, we'll use PR #33 and issue #30.

To test:

  1. Delete all labels of the PR and reset the title:

    gh api --method DELETE /repos/infinisil-test-org/nixpkgs-committers/issues/33/labels
gh api --method PATCH /repos/infinisil-test-org/nixpkgs-committers/pulls/33 -f title="A non-conforming title"

  2. Run the script while simulating that a non-nomination PR was opened:

    scripts/nomination.sh members infinisil-test-org/nixpkgs-committers 33 30 <<< "removed members/infinisil"

    Ensure that it exits with 0 and wouldn't run any effects.

  3. Run the script while simulating that multiple users were nominated together:

    scripts/nomination.sh members infinisil-test-org/nixpkgs-committers 33 30 <<< "removed members/foo"$'\n'"added members/bar"

    Ensure that it exits with non-0 and wouldn't run any effects.

  4. Run the script simulating a successful nomination

    scripts/nomination.sh members infinisil-test-org/nixpkgs-committers 33 30 <<< "added members/infinisil"

    Ensure that it exits with 0 and would run effects to label the PR, change the title and post a comment in the issue.

  5. Rerun with effects

    PROD=1 scripts/nomination.sh members infinisil-test-org/nixpkgs-committers 33 30 <<< "added members/infinisil"

Testing retire.sh

This script has external effects and as such needs a bit more care when testing.

Setup (important!)

To avoid other users getting pings, ensure that the members-test directory contains only simulated new users and your own user (simulated to have been added over a year ago), then commit and push it for testing:

me=$(gh api /user --jq .login)
git switch -C "test-$me"
rm -rf members-test
mkdir -p members-test

touch members-test/"$me"
date +%F > "members-test/new-committer-1"
git add members-test
GIT_COMMITTER_DATE=$(date --date @0) git commit -m testing

touch "members-test/new-committer-2"
git add members-test
git commit -m testing
git push -f -u origin HEAD

Test sequence

The following sequence tests all code paths.

The CONFIRM_CUTOFF argument is passed as now so that a single run is enough to open a retirement PR. In CI it is set to 7 days ago instead, which together with the CACHE_FILE activity cache (persisted between runs as a workflow artifact) requires a week of daily runs to agree before a PR is opened.

  1. Run the script with the active repo argument to simulate CI running without inactive users:

    scripts/retire.sh infinisil-test-org active nixpkgs-committers members-test 'yesterday 1 month ago' now now

    Check that no PR would be opened.

  2. Run the previous command again with CACHE_FILE set:

    CACHE_FILE=.tmp-cache.csv scripts/retire.sh infinisil-test-org active nixpkgs-committers members-test 'yesterday 1 month ago' now now
CACHE_FILE=.tmp-cache.csv scripts/retire.sh infinisil-test-org active nixpkgs-committers members-test 'yesterday 1 month ago' now now
rm .tmp-cache.csv

    Check that the first run writes your activity to the cache file and that the second run skips the /commits query for your user because of it.

  3. Run the script with the empty repo argument to simulate CI running with inactive users:

    scripts/retire.sh infinisil-test-org empty nixpkgs-committers members-test 'yesterday 1 month ago' now now

    Check that it would only create a PR for your own user and not the "new-committer-1" or "new-committer-2" user. Also check that with a CONFIRM_CUTOFF in the past (as used in CI), no PR would be opened yet:

    scripts/retire.sh infinisil-test-org empty nixpkgs-committers members-test 'yesterday 1 month ago' now '7 days ago'

    Then run it again with PROD=1 to actually do it:

    PROD=1 scripts/retire.sh infinisil-test-org empty nixpkgs-committers members-test 'yesterday 1 month ago' now now

    Check that it created the PR appropriately, including assigning the "retirement" label. You can undo this step by closing the PR.

  4. Run it again to simulate CI running again later:

    PROD=1 scripts/retire.sh infinisil-test-org empty nixpkgs-committers members-test 'yesterday 1 month ago' now now

    Check that no other PR is opened.

  5. Run it again with now as the notice cutoff date to simulate the time interval passing:

    PROD=1 scripts/retire.sh infinisil-test-org empty nixpkgs-committers members-test now now now

    Check that it undrafted the previous PR and posted an appropriate comment.

  6. Run it again to simulate CI running again later:

    PROD=1 scripts/retire.sh infinisil-test-org empty nixpkgs-committers members-test now now now

    Check that no other PR is opened.

  7. Reset by marking the PR as a draft again, then run it again with the active repo argument to simulate activity during the time interval:

    PROD=1 scripts/retire.sh infinisil-test-org active nixpkgs-committers members-test now now now

    Check that it gets undrafted with a comment listing the new activity.

  8. Close the PR, then run the script again with no activity and for an earlier close cutoff, simulating that the retirement was delayed:

    PROD=1 scripts/retire.sh infinisil-test-org empty nixpkgs-committers members-test now '1 day ago' now

    Check that no other PR is opened.