Merge pull request #4929 from ESMCI/fix-containers

- Migrates from conda to spack
- Fix mixed perl environment (conda/system)
- Moves to UV to manage python
- Simplifies usage and testing workflow

Author: jasonb5

.github/workflows/testing.yml

@@ -62,8 +62,8 @@ jobs:
       - name: Build and push
         uses: docker/build-push-action@v6
         with:
-          target: base
-          context: docker/
+          file: docker/Dockerfile
+          context: .
           push: true
           tags: ${{ steps.meta.outputs.tags }}
           labels: ${{ steps.meta.outputs.labels }}
@@ -123,6 +123,8 @@ jobs:
         with:
           path: cesm/cime
           submodules: "true"
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
       - name: Run tests
         shell: bash
         working-directory: cesm/cime
@@ -132,18 +134,15 @@ jobs:
           CIME_TEST_PLATFORM: "ubuntu-latest"
           SKIP_ENTRYPOINT: "true"
         run: |
-          source /opt/conda/etc/profile.d/conda.sh
-
-          conda activate cesm
-
-          mamba install -y 'python=${{ matrix.python-version }}'
-
-          pip install -r test-requirements.txt
+          # manually run the entrypoint
+          source /entrypoint.sh
 
-          # GitHub runner home is different than container
-          cp -rf /home/cime/.cime /github/home/
+          uv python install "${{ matrix.python-version }}"
+          uv venv
+          source .venv/bin/activate
+          uv pip install -r test-requirements.txt
 
-          pytest -vvv --cov=CIME --cov-branch --cov-report=xml --machine docker CIME/tests/test_unit*
+          uv run pytest -vvv --cov=CIME --cov-branch --cov-report=xml --machine docker CIME/tests/test_unit*
       - name: Upload coverage reports to Codecov
         uses: codecov/codecov-action@v4
         with:
@@ -214,6 +213,10 @@ jobs:
         with:
           path: /home/cime/inputdata
           key: inputdata-2
+      - name: Install uv
+        uses: astral-sh/setup-uv@v7
+        with:
+          python-version: "3.10"
       - name: Run tests
         shell: bash
         working-directory: ${{ matrix.model.name }}/cime
@@ -223,31 +226,18 @@ jobs:
           CIME_TEST_PLATFORM: ubuntu-latest
           SKIP_ENTRYPOINT: "true"
         run: |
-          source /opt/conda/etc/profile.d/conda.sh
-
-          conda activate ${{ matrix.model.name }}
-
-          pip install -r test-requirements.txt
-
+          # manually run the entrypoint
           source /entrypoint.sh
 
-          if [[ "${{ matrix.model.name }}" == "e3sm" ]]; then
-            fix_mct_makefiles ../externals/mct
-          fi
-
-          # GitHub runner home is different than container
-          cp -rf /home/cime/.cime /github/home/
-
-          if [[ "${CIME_MODEL}" == "e3sm" ]]; then
-            ln -sf /github/home/.cime/config_machines.v2.xml /github/home/.cime/config_machines.xml
-          else
-            ln -sf /github/home/.cime/config_machines.v3.xml /github/home/.cime/config_machines.xml
-          fi
+          uv python install 3.10
+          uv venv
+          source .venv/bin/activate
+          uv pip install -r test-requirements.txt
 
           git config --global user.name "${{ github.actor }}"
           git config --global user.email "${{ github.actor }}@users.noreply.github.com"
 
-          pytest -vvv --cov=CIME --cov-branch --cov-report=xml --machine docker --no-fortran-run --no-teardown CIME/tests/test_sys*
+          uv run pytest -vvv --cov=CIME --cov-branch --cov-report=xml --machine docker --no-fortran-run --no-teardown CIME/tests/test_sys*
       - name: Upload coverage reports to Codecov
         uses: codecov/codecov-action@v4
         with:
@@ -259,7 +249,7 @@ jobs:
       - name: Create testing log archive
         if: ${{ failure() }}
         shell: bash
-        run: tar -czvf /testing-logs-${GITHUB_RUN_NUMBER}-${{ matrix.model.name }}-${{ matrix.driver }}.tar.gz /home/cime/cases /home/cime/baselines /home/cime/archive
+        run: tar -czvf /testing-logs-${GITHUB_RUN_NUMBER}-${{ matrix.model.name }}-${{ matrix.driver }}.tar.gz /root/storage/cases /root/storage/baselines /root/storage/archive
         # How to download artifacts:
         # https://docs.github.com/en/actions/managing-workflow-runs/downloading-workflow-artifacts
       - name: Upload testing logs

CONTRIBUTING.md

@@ -20,11 +20,13 @@ For more information on contributing to open source projects,
 is a great starting point. Also, checkout the [Zen of Scientific Software Maintenance](https://jrleeman.github.io/ScientificSoftwareMaintenance/)
 for some guiding principles on how to create high quality scientific software contributions.
 
+The canonical, detailed contributing guide for this repository is included in the source tree at `doc/source/contributing-guide.rst`; please consult that file as the single source of truth for developer workflows, testing, and container usage.
+
 ## Getting Started
 
 Interested in helping extend CIME? Have code from your research that you believe others will
-find useful? Have a few minutes to tackle an issue? In this guide we will get you setup and
-integrated into contributing to CIME!
+find useful? Have a few minutes to tackle an issue? This guide will get you set up to contribute
+to CIME.
 
 ## What Can I Do?
 * Tackle any unassigned [issues](https://github.com/ESMCI/CIME/issues) you wish!
@@ -44,7 +46,7 @@ The goal is to maintain a diverse community that's pleasant for everyone.
   include multiple bug fixes in a single pull request, but they should be related.
   For unrelated changes, please submit multiple pull requests.
 * Do not commit changes to files that are irrelevant to your feature or bugfix
-  (eg: .gitignore).
+  (e.g., .gitignore).
 * Be willing to accept constructive criticism as part of issuing a pull request,
   since the CIME developers are dedicated to ensuring that new features extend the
   system robustly and do not introduce new bugs.
@@ -53,7 +55,7 @@ The goal is to maintain a diverse community that's pleasant for everyone.
 
 ## Reporting a bug
 When creating a new issue, please be as specific as possible. Include the version
-of the code you were using, as well as what operating system you are running.
+of the code you were using, as well as what operating system you are running. Include the commit SHA or tag (e.g., output of `git rev-parse HEAD`) and the branch name.
 If possible, include complete, minimal example code that reproduces the problem.
 
 ## Pull Requests
@@ -62,34 +64,18 @@ We love pull requests from everyone. Fork, then clone the repo:
 
     git clone git@github.com:your-username/CIME.git
 
-Additionally you may need to checkout the submodules with:
+You will need to initialize and update submodules:
 
     cd CIME
-    git submodule update --init
-
-You will need to install CIME dependencies and edit config files
-to tell CIME about your development machine. See the [CIME users guide](https://esmci.github.io/cime/users_guide/porting-cime.html)
-
-Run the scripts_regression_tests:
-
-    cd CIME/tests
-    python scripts_regression_tests.py
-
-Alternatively with `pytest`:
-
-    pytest CIME/tests
-
-Make your change. Add tests for your change. Make the tests pass to the same level as before your changes.
+    git submodule update --init --recursive
 
-    cd CIME/tests
-    python scripts_regression_tests.py
+From here you can edit the code and run the unit tests following this [guide](https://esmci.github.io/cime/versions/master/html/contributing-guide.html#pytest).
 
-Run [pre-commit](https://pre-commit.com/#usage) before committing changes and submitting a PR.
+When running the ``unit`` tests you can specify any valid machine e.g. docker and the tests will run.
 
-    pip install pre-commit
-    pre-commit run -a
+If you need to run the ``system`` tests you will need to have your respective model checked out and on a supported machine. Alternatively you can use CIME [container](https://esmci.github.io/cime/versions/master/html/contributing-guide.html#docker-container) which is used in our GitHub CI testing.
 
-Commit the changes you made. Chris Beams has written a [guide](https://chris.beams.io/posts/git-commit/) on how to write good commit messages.
+Before creating your PR you will need to run the code quality checkers; see this [guide](https://esmci.github.io/cime/versions/master/html/contributing-guide.html#code-quality).
 
 Push to your fork and [submit a pull request][pr].
 
@@ -102,8 +88,9 @@ We may suggest some changes or improvements or alternatives.
 Some things that will increase the chance that your pull request is accepted:
 
 * Write tests.
-* Follow [PEP8][pep8] for style. (The `flake8` utility can help with this.)
-* Write a [good commit message][commit].
+* Write documentation.
+* Follow the [Code Quality guide](https://esmci.github.io/cime/versions/master/html/contributing-guide.html#code-quality)
+* Write a good commit message, we recommend using [conventional commits](https://www.conventionalcommits.org/en/v1.0.0/)
 
 Pull requests will automatically have tests run by a Github Action. This
 includes running both the unit tests as well as `pre-commit`, which checks