Add CryoSPARC patched image

[ZQyou]

• Update the Makefile to install the CryoSPARC patch.
• Update the default appVersion to use the patched image.

[treydock]

My concern here is if you built this image as 4.7.1p251124-r3 today, then next week a new patch comes out, we'd automatically build 4.7.1p251124-r3 again and overwrite the previous image but the contents of the image built would change.

I think one way to ensure we don't have images changing without changing tags, is embed the SHA256 checksum of the patch.tar.gz into the tag of the image. I see the patch 251124 but not clear that number is guaranteed to be the patch we get since we don't request patch 251124.

[ZQyou]

In my local implementation, I added a version check based on the patch file from the tarball.

According to https://cryosparc.com/updates/all, patch releases appear to be infrequent; most updates are full version releases. Therefore, I can reintroduce the patch number check, and I believe checking that number should be sufficient.

I do not expect any additional patches for version 4.7.1. The next release will likely be 4.7.2 if version 4 continues to be supported. Going forward, we plan to focus on full version updates rather than patch updates.

[treydock]

Ya, the patch number check would be good.

If future work will not include patching, what's the value in adding this patching logic for 4.7.x? Are there fixes in the patch that we need? If we don't necessarily need the patch, and future work will be full versions, maybe we don't need the patching logic? I can imagine maybe we keep patching workflows with 5.x if that's how CryoSPARC releases fixes before a release.

[ZQyou]

We don’t really know when users will need it, so I’d like to be prepared just in case. When they do need it, there may be very little time to prepare. I’d rather do it now than later.

[treydock]

All this chained commands should probably just be a script baked into the image. Something like /usr/local/sbin/cryosparc-patch.sh or something. It should start with this:

#!/bin/bash

set -e

So any failure in the script will cause the script to exit.

You may have to change the logic so that it does this:

So the normal build uses _TAG and always builds before the patch build, which might be better too to ensure the patch starts with latest build of the base image. So won't need the if/else, just have to ensure the first build is using _TAG I think.

[treydock]

If the patch self-identifies it's number, would it be better to copy this file onto the build host's /tmp and use that file's contents to generate the patch image build. So always build:

$(IMAGE):$(VERSION)-r$(REVISION)

First, then run the patch workflow after and if there is a patch file use that file's content to generate the patch image with

$(IMAGE):$(VERSION)p$(PATCH_VERSION_FROM_FILE)-r$(REVISION)

Kind of thing. So if there's a patch we build 2 images, the base and the patched image.

[ZQyou]

That was my original approach. However, I need to pass the patch version to the tag used in the push and kind-load targets. The approach I’m considering is to store this information during the build step and then reference it in later steps. This would likely require modifying the workflow or other steps, which feels overly complex. So I would prefer to keep the downstream steps as simple as possible.

Additionally, patches are required for both the master and worker nodes. For the latter, I would need to install them on our clusters. Because of this, full automation may not be necessary.

[treydock]

You could keep the logic entirely in the Makefile.

build:
    docker build ... regular image

build-patch: build
    docker build ... try and get a patch

    ... if no patch downloaded, exit ...

    ... if patch present, read `/tmp/cryosparc_master/patch` to get value like this:

    PATCH=$(shell cat $(ROOT_DIR)/tmp/patch)

This might make it easier if the regular image is:

docker-registry.osc.edu/webservices/cryosparc:4.7.1-r3

And the patch image is:

docker-registry.osc.edu/webservices/cryosparc:4.7.1-r3-p$(PATCH)

So you just append the patch detected to the end of the tag.

The build and push commands should still have access to $(ROOT_DIR)/tmp/patch to determine if a patch was downloaded.

[ZQyou]

I can update the image tag naming to account for patches. It looks like I need separate targets like this?:

_build:
   # build the base image

build: _build
   # build the patched image if patch is available

_push:
  # push the base image

push: _push
  # push the patched image if patch is available

_kind-load:
  # load the base image

kind-load: _kind-load
  # load the patched image if patch is available

[treydock]

Ya, those work though maybe build-base and build-patch then

build: build-base build-patch

As _ prefix appears like it's private when maybe shouldn't be?