Creates GHA workflow to notify on deletion of in-use issue labels

[ryanfkeller]

Fixes #8185

What changes did you make?

High level description

• Added a GitHub Actions workflow that runs when an issue label is deleted to check whether the label was in use by any issues
• If the label was in use:
  • Creates a "notification" issue documenting the deleted label details, the deleter, and affected issues, and adds a comment requesting information from the deleter.
  • Posts a comment to the agenda issue with details and a link to the notification issue
  • If the agenda issue is missing or closed, creates an error notification issue

The workflow will detect and notify on label removals from issues that are closed as well as open.

Implementation details

• Made new workflow flag-issues-unlabeled-after-deletion.yml with three steps:
  • Get recently unlabeled issues: Uses GraphQL query to find issues that had the deleted label removed in the last 5 minutes
  • Create notification issue: Generates notification issue with label deletion details (only if issues were affected)
  • Post to agenda: Posts to agenda issue or creates error notification if agenda posting fails
• Updated update-label-directory.yml to call the new workflow when labels are deleted
• Created utility files for template-based issue/comment creation and data centralizing:
  • Template system using markdown files with variable substitution (populate-template.js)
  • Issue and comment creation utilities (create-templated-issue.js and post-templated-comment.js)

    Note: Repo has existing templating tools (format-comment.js for single replacements, issue-template-parser.js for key-able object creation). Since I needed simple multi-variable string substitutions, I built separate utilities to avoid disrupting existing functionality

  • Los Angeles timestamp utility for consistent formatting (get-la-timestamp.js)
  • Static issue number JSON file for centralized source-of-truth (which became important during testing).

Why did you make the changes (we will use this info to test)?

• This change was made to allow the team to keep track of issues that were affected by label deletion, so that if the label was deleted by mistake, the information is conveniently accessible to restore it.
• The notification method (notification issue + post to agenda) is as specified by the team leads

---

Testing Details

Testing was performed on my fork using a separate testing branch. The test branch comparison is here.

Test setup changes (not in this PR)

• Filtered some unrelated existing workflows to only run on hackforla/website (so I would stop getting emails about them running and failing 😆)
• Updated agenda issue number in JSON to point at a test agenda issue (or at no issue for negative tests)
• Created automated testing infrastructure using Jest in a custom testing container

Automated Testing Infrastructure Details (for potential future use)

• Testing Environment:

  • Added a Docker container hfla_test based on node:20-bullseye (for npm)
    • The docker image creation also installs the octokit rest API for github scripting and prettier for style linting
    • Container was given a "profile" so it is not launched by default on docker compose up calls. This was done so it would not interfere with normal docker usage.
    • Container can be build and launched in detached mode with command docker compose up --build -d hfla_testing. Future calls can omit the --build flag and reuse the built image.
  • Added a custom jest.config.js and package.json under /docker/testing that gets copied to the root of the container on container launch for testing.
  • Tests can be launched by executing docker exec hfla_testing npm run test, which runs the "test" script defined in package.json (aka, what is stored in package.testing.json until the container is built). This script runs tests based on the jest.config.js test match.

• Test Scripts:

  • Built all JS scripts as MJS for octokit compatibility. I went with octokit because it was FAR faster than gh cli exec calls.
  • Created gh-helper.mjs as a general purpose helper for interfacing with the repo (making issues, checking labels, etc.)
  • Created flag-issues-unlabeled-after-deletion.test.mjs as the specific script for this workflow. This test includes setting up the test scenario, then verifying the proper behavior. I will note that running/debugging with this script will result in ~10 new issues in your fork per run, although the script attempts to close all issues generated during the test.

Test Evidence

Test Scenario 1: Nominal case with agenda issue open

• Created 5 reusable test issues (named 1-5)
• Between tests, deleted all labels
• Test A: Applied label test-label-A-{timestamp} to all 5 issues, then deleted label. Expected 1 notification + 1 agenda comment listing all five issues
• Test B: Created 4 labels test-label-B[0,1,2,3]-{timestamp}. Added B0 to issues [0,1], added B1 to issues [1,2,3], added B2 to issues [3,4]. Deleted labels B0, B1, and B3. Expected notifications from B0 and B1 deletions with their respective issues listed. Expected no notifications from B2 (not deleted during test) and B3 (unused)

Label	Workflow Run	Notification Issue	Agenda Comment
A	https://github.com/ryanfkeller/website/actions/runs/18106154865	ryanfkeller#1165	ryanfkeller#1072 (comment)
B0	https://github.com/ryanfkeller/website/actions/runs/18106160458	ryanfkeller#1167	ryanfkeller#1072 (comment)
B1	https://github.com/ryanfkeller/website/actions/runs/18106160505	ryanfkeller#1166	ryanfkeller#1072 (comment)
B2 (after test)	https://github.com/ryanfkeller/website/actions/runs/18106175255	X	X
B3	https://github.com/ryanfkeller/website/actions/runs/18106160719	X	X

Test Scenario 2: Scenario case with agenda issue closed

• Closed the agenda issue to test error handling

Label	Workflow Run	Notification Issue	Agenda Notification Issue
A	https://github.com/ryanfkeller/website/actions/runs/18105840443/job/51520021840	ryanfkeller#1146	ryanfkeller#1147
B0	https://github.com/ryanfkeller/website/actions/runs/18105846187	ryanfkeller#1148	ryanfkeller#1150
B1	https://github.com/ryanfkeller/website/actions/runs/18105846429	ryanfkeller#1149	ryanfkeller#1151
B2 (after test)	https://github.com/ryanfkeller/website/actions/runs/18105870328	X	X
B3	https://github.com/ryanfkeller/website/actions/runs/18105846522	X	X

Test Scenario 3: Scenario case with agenda issue missing

• Updated the agenda issue to non-existent issue "1" to test error handling

Label	Workflow Run	Notification Issue	Agenda Notification Issue
A	https://github.com/ryanfkeller/website/actions/runs/18105773502	ryanfkeller#1135	ryanfkeller#1136
B0	https://github.com/ryanfkeller/website/actions/runs/18105779842	ryanfkeller#1137	ryanfkeller#1138
B1	https://github.com/ryanfkeller/website/actions/runs/18105780215	ryanfkeller#1139	ryanfkeller#1140
B2 (after test)	https://github.com/ryanfkeller/website/actions/runs/18105801787	X	X
B3	https://github.com/ryanfkeller/website/actions/runs/18105780285	X	X

Results:

To my eye, all test scenarios produced the expected results. The new workflow correctly identified affected issues and created the appropriate notifications, both with and without an existing or open Agenda issue.

CodeQL Alerts

After the PR has been submitted and the resulting GitHub actions/checks have been completed, developers should check the PR for CodeQL alert annotations.

Check the PR's comments. If present on your PR, the CodeQL alert looks similar as shown

Please let us know that you have checked for CodeQL alerts. Please do not dismiss alerts.

• I have checked this PR for CodeQL alerts and none were found.
• I found CodeQL alert(s), and (select one):
  • I have resolved the CodeQL alert(s) as noted
  • I believe the CodeQL alert(s) is a false positive (Merge Team will evaluate)
  • I have followed the Instructions below, but I am still stuck (Merge Team will evaluate)

Instructions for resolving CodeQL alerts

If CodeQL alert/annotations appear, refer to How to Resolve CodeQL alerts.

In general, CodeQL alerts should be resolved prior to PR reviews and merging

Screenshots of Proposed Changes To The Website (if any, please do not include screenshots of code changes)

• No visual changes to the website

[github-actions]

Want to review this pull request? Take a look at this documentation for a step by step guide!

---

From your project repository, check out a new branch and test the changes.

git checkout -b ryanfkeller-flag-label-deletion-8185 gh-pages
git pull https://github.com/ryanfkeller/website.git flag-label-deletion-8185

[t-will-gillis]

available: m-f after 6
etc: 9/19 eod

[t-will-gillis]

Hey @ryanfkeller this is fantastic! All the parts seem to be working smoothly: on label deletion, an issue is generated showing the label deleted, who deleted it, and the issues affected, then a comment to the 'deleter' is posted on the issue and the info is posted as a comment to the weekly agenda. Very nice.

Things I really like:

• the addition of the note "Auto-generated by Flag Issues With Deleted Labels workflow." I think we should start using this note in similar situations.
• separating out the agenda number. It's clean and clear. Again, this should be done for many similar situations
• the modularized util files
• adding a subfolder for /templates/. Once again, this should be done to clean up some of the other workflows also
• incidentally, a PR I am working on now has a helper function that is very similar to get-la-timestamp.js except it is converting any UST to PST/PDT as opposed to returning 'now' as yours is. Assuming your PR is merged first, I will likely try to combine both

Some requests for changes:

• For the agenda issue posting, one thing that Bonnie specifically mentioned to me is that she wants an error message if the agenda issue that the workflow is trying to post to has been closed. The reason is that sometimes we close an agenda issue when it becomes overloaded with comments, and then we replace it with a new one. Eg: 7859 is a replacement for a previous agenda. Otherwise, the comment will keep posting to the closed issue and we won't know.
• Please change actions/checkout@v4 to actions/checkout@v5
• Please change actions/github-script@v7 to actions/github-script@v8
  (both updated last week)
• Suggestion to add (escaped) backticks to the issue title, line 34 of create-notification-issue.js, so the final title is formatted:

  title: `Review Needed - Label \`${labelName}\` Deleted`,
  

• In the flag-issues...yml file, could you add to the initial comment with a note that that workflow is triggered by update-label-directory.yml? Might be helpful to someone in the future.

Thanks for working on this! you obviously put a lot of time into it. I think there are some other questions I might have about how you did some thing, for example how you are using templateVars in create-notification-issue.js, and also see if you have other ideas.

[ryanfkeller]

Hi @t-will-gillis -- thank you so much for the detailed review and kind words! I really appreciate your time and feedback.

Based on your review feedback, I've made the following updates:

• Updated the "Agenda Missing" notification issue to be created when the Agenda Issue is closed in addition to missing.
• Updated the workflow actions to the latest versions respectively
• Added the back-tick formatting to the "Label Deletion" notification issue title
• Added a comment in flag-issues-unlabeled-after-deletion.yml that indicates it is launched by update-label-directory.yml

For testing these updates, I did the following:

• Updated the test scripts to expect am "Agenda Missing" notification issue if the agenda issue is closed, and to expect the "Label Deletion" notification issue will have the back-tick formatting.
• Added a new (manual) scenario in which the test suite is run when the Agenda issue is closed.
• Re-ran all test scenarios with the latest code
• Also, updated the description above of the automated test setup I made for this issue (including usage, etc.)

Absolutely looking forward to making any other changes discovered, as needed.

---

RE: templateVars,

I'll certainly be repeating info you saw/know, but just to be thorough, the premise of the template system I put in this PR is that I pass the following to my template helpers:

1. The path to the markdown template to be used
2. A dictionary that maps "variable strings" in that markdown template to the string values that I want to replace that variable string with

The "variable strings" can be literally any string. I went with JS-var-in-string style because I thought it made it easier to spot, but I could have just as easily made ${label-name} to be label-name instead. The only thing is that every instance of that text will be replaced with the mapped value, so I needed to chose something distinct enough to not show up anywhere else.

The populate-template then iterates through my templateVars dictionary, makes sure the "variable strings" are safe to be used in a regex by adding escapes to any regex special characters, then does a regex based replacement, which works for multiple instances of the same var in the template.

---

RE: other ideas,

I have a couple things I noticed as I went through that I'd be happy to chat about -- although you already mentioned several of them in your comment! Of course, I also made my whole GHA test environment and automated scripts that I'd be interested to get a temperature check on.

Thanks!

[t-will-gillis]

Hey @ryanfkeller Sorry for the delay re-reviewing.

• I did not catch this earlier, but on the main issue created and on the 'Error Posting to Agenda Issue...", could you add the labels: ['Complexity: Small', 'size: 0.5pt','Feature: Administrative','role: back end/devOps', 'Ready for product'] ?
• Minor thing, for the "Error Posting to Agenda Issue" add the backticks around the label name
  Everything else looks great- thanks again!

[ryanfkeller]

Got it! I made the changes you called out:

• Added ['Complexity: Small', 'size: 0.5pt','Feature: Administrative','role: back end/devOps', 'Ready for product'] labels to the "Label Deletion Notification" and "Agenda Missing/Closed" issues
• Updated the title of the "Agenda Missing/Closed" issue to include backtick formatting of the label name

I updated the test script, re-ran the test cases, and updated the test result links in the PR description above. Thanks!

[t-will-gillis]

Awesome- thanks @ryanfkeller for all your work on this!