Logical error causing non-compiled tests to be skipped

[robdimsdale]

The workflow to update dependencies incorrectly skips the tests of the non-compiled version of a dependency when there are also pre-compiled versions of it to test. The non-compiled dependencies are correctly added to the buildpack.toml, but they are untested.

A concrete example of this is trying to add python v3.11.x to the Python buildpack. It has both pre-compiled versions as well as a source/non-compiled version. The non-compiled version tests are incorrectly skipped, as seen in the screenshot below:

The logic for running the pre-compiled tests is found here and is as follows:

    if: ${{ needs.retrieve.outputs.compilation-length > 0 
        && (needs.get-compile-and-test.outputs.should-compile == 'true' 
            || needs.get-compile-and-test.outputs.should-test == 'true') }}

Whereas the logic for determining whether to run the non-compiled tests is found here and is as follows:

    if: ${{ needs.retrieve.outputs.length > 0
        && needs.get-compile-and-test.outputs.should-test == 'true'
           && needs.get-compile-and-test.outputs.should-compile == 'false' }}

I think the logical error in this workflow is that we use a binary value (i.e. the should-compile boolean) to represent the entire (quaternary) compilation state. The four cases are as follows:

1. Run compilation step
2. Skip compilation step
3. Run non-compilation step
4. Skip non-compilation step

I think we need two distinct, independent values - something like: run-compilation-step and run-non-compilation-step.

I think it is sufficient to keep the should-test boolean, as I don't currently see a use-case for running tests under one scenario but not the other (e.g. running tests on the pre-compilation step, but not on the non-compiled step).