Skip to content

Sync up unique cache paths for Singularity images with cwl-utils#2284

Open
adamnovak wants to merge 5 commits into
common-workflow-language:mainfrom
adamnovak:robust-sif-cache
Open

Sync up unique cache paths for Singularity images with cwl-utils#2284
adamnovak wants to merge 5 commits into
common-workflow-language:mainfrom
adamnovak:robust-sif-cache

Conversation

@adamnovak

Copy link
Copy Markdown

This and the corresponding change to cwl-utils cwl-docker-extract should solve the problem of ambiguous or different SIF file names between the two tools.

There's still the problem that cwltool supports more ways of specifying an image than cwl-utils does.

@codecov

codecov Bot commented Jun 4, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.18%. Comparing base (8949fc2) to head (d823fe6).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2284      +/-   ##
==========================================
+ Coverage   85.10%   85.18%   +0.08%     
==========================================
  Files          46       46              
  Lines        8591     8593       +2     
  Branches     2011     2010       -1     
==========================================
+ Hits         7311     7320       +9     
+ Misses        813      807       -6     
+ Partials      467      466       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@adamnovak

Copy link
Copy Markdown
Author

The linter that complains about long lines has a different line length limit than, and runs before, the linter that tells you your long lines are not wrapped exactly correctly.

@adamnovak

Copy link
Copy Markdown
Author

I'm having trouble running the offending test locally because I can't figure out how to limit the test system to a reasonable number of worker threads.

When I run tox -e py310-unit -- tests/test_singularity.py::test_singularity3_docker_image_id_in_tool on my Linux host it gets stuck after having started 130 of 160 worker processes, and I think it is probably trying to start too many.

@adamnovak

Copy link
Copy Markdown
Author

Turns out I need:

tox -e py310-unit -- tests/test_singularity.py::test_singularity3_docker_image_id_in_tool -n3

@adamnovak

Copy link
Copy Markdown
Author

OK, test_singularity3_docker_image_id_in_tool seems to be testing that you can run a workflow that asks for docker.io/debian:stable-slim, and then one in the same directory that asks for docker.io_debian:stable-slim.sif, assuming that we always replace / with exactly one _ in the image names and we can pick up the image by its relative SIF path.

Do we really need to promise that constraint?

We also have a similar test in test_singularity2_docker_image_id_in_tool that wants us to put docker.io/debian:stable-slim at debian:stable-slim.img, which I don't think we were doing before; I'm not sure the CI still attempts Singularity 2 testing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant