Compare reference images by default

[arunkannawadi]

No description provided.

[Copilot]

Pull request overview

Updates the CI workflow so that reference image changes are detected during PRs, and attempts to generate visual comparisons and comment them back on the PR.

Changes:

• Trigger the workflow on pull_request in addition to manual dispatch.
• Detect whether the reference FITS image differs and gate subsequent steps on that result.
• Add steps to generate PNGs via ds9, push updated reference files, and comment comparison images on the PR.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[arunkannawadi]

This is that kind of a PR where AI could be useful

[Copilot]

Just as a heads up, I was blocked by some firewall rules while working on your feedback. Expand below for details.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• ds9.si.edu
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

[sidneymau]

Instead of installing ds9, I would suggest just using fits2bitmap (from astropy -- https://docs.astropy.org/en/stable/visualization/index.html#scripts)

[arunkannawadi]

Oh I'm learning so many nifty fits tools that I didn't even know existed!

[arunkannawadi]

OK, I give up on the approach where I wanted the CI workflow to push a comment with a before and after image for quick visualization. I'll rework this PR so the PNG version of the reference image also lives in the repo, and gets updated with the reference image gets updated. The before and after version will have to be from the PR diff.

[sidneymau]

Trying to catch up on everything that was tried in here... I am happy to take a stab at getting the CI to post before/after images

[arunkannawadi]

Okay, the workflow gets the job done. The difference can be seen in the Files changed. This is sufficient and in hindsight better than having multiple comments with the images. I'll clean up the PR before merging, but if you want to try something else that can be in another PR.

The only thing I dislike about using fits2bitmap is that it doesn't give a colorbar for the image

[sidneymau]

why is the current reference image blank? (https://github.com/DukeCosmology/roman_imsim_testdata/pull/16/changes#diff-b6fbf74b520e0c0ef1cae33ff3b60d8480c598eb57371e79f2984464ad4a3507)

[sidneymau]

(possibly due to nobjects = 15?)

[arunkannawadi]

It is not blank, just has fewer objects because I reduced it to 15 to trigger a difference and keep the run times shorter.

[sidneymau]

Looks good to me (I assume the plan is to revert to the full number of objects before merging?)

[arunkannawadi]

Remove the temporary trigger commit, and the bot-generated commit in response.