new logic for tests - bad and good example; hook checking nb outputs; (#61)

* create hook checking if notebook executed and without errors

* add executable;move to correct dir

* add main

* add imports an dependencies; remove doubled code; specify exceptions

* update tests; error msg

* define func possible to move to utils; add todo flag

* add docstrings

* run check-badges only on good examples

* new structure for pre-commit hooks

* fix notebooks

* change logic of checked files

* refactor hooks using copilot

* rename notebooks

* add new tests for hooks

* add test to run notebooks; exclude bad example

* update notebooks

* pylint hints; remove some unnecessary comments,; cleanup

* change exceptions

* change exception to ValueError

* remove duplicated code

* add nu_ju to executables

* add check=True flag

* exclude bad example from run on CI workflow

* run pre-commit hook black-26.1.0

* do not fail the whole test, when error is expected

* rerun example-notebooks

* clear output for bad example

* add test for notebooks-output

* use should_fail in assertion

* move relpath function to hook that uses it; fix misuse of absolute and relative paths

* reuse ValueError for re-raising (pylint hint)

* remove fix header option

* remove header fixing at all as it is not well-designed and not essential

* pass tmp path to badges check for notebooks created during tests

Author: AgnieszkaZaba

.github/workflows/checks.yml

@@ -34,7 +34,7 @@ jobs:
           pip install pre-commit
           pre-commit clean
           pre-commit autoupdate
-          pre-commit run --all-files
+          pre-commit run --files $(git ls-files | grep -v tests/examples/bad.ipynb)
 
   selftest:
     needs: [pylint, precommit]

.pre-commit-config.yaml

@@ -2,7 +2,7 @@ default_stages: [pre-commit]
 
 repos:
   - repo: https://github.com/psf/black-pre-commit-mirror
-    rev: 25.11.0
+    rev: 26.1.0
     hooks:
       - id: black
 
@@ -13,10 +13,10 @@ repos:
       - id: end-of-file-fixer
       - id: debug-statements
 
-  - repo: https://github.com/christopher-hacker/enforce-notebook-run-order
-    rev: 2.1.1
-    hooks:
-      - id: enforce-notebook-run-order
+#  - repo: https://github.com/christopher-hacker/enforce-notebook-run-order
+#    rev: 2.1.1
+#    hooks:
+#      - id: enforce-notebook-run-order
 
   - repo: local
     hooks:
@@ -27,6 +27,7 @@ repos:
           - nbformat
         language: python
         types: [jupyter]
+        exclude: tests/examples/bad.ipynb
 
       - id: check-badges
         name: check badges
@@ -35,6 +36,24 @@ repos:
           - nbformat
           - pytest
           - gitpython
-        args: [--fix-header, --repo-name=devops_tests]
+        args: [--repo-name=devops_tests]
+        language: python
+        types: [jupyter]
+        exclude: tests/examples/bad.ipynb
+
+      - id: notebooks-output
+        name: notebooks output
+        entry: notebooks_output
+        additional_dependencies:
+          - nbformat
         language: python
         types: [jupyter]
+        exclude: tests/examples/bad.ipynb
+
+      - id: notebooks-using-jupyter-utils
+        name: notebooks using open-atmos-jupyter-utils
+        entry: notebooks_using_jupyter_utils
+        additional_dependencies:
+          - nbformat
+        language: python
+        types: [ jupyter ]

.pre-commit-hooks.yaml

@@ -13,3 +13,19 @@
   language: python
   stages: [pre-commit]
   types: [jupyter]
+
+- id: notebooks-output
+  name: notebooks output
+  description: check if notebooks are executed and without errors and warnings
+  entry: notebooks_output
+  language: python
+  stages: [pre-commit]
+  types: [jupyter]
+
+- id: notebooks-using-jupyter-utils
+  name: notebooks using open-atmos-jupyter-utils
+  description: check if notebook use show_anim() and show_plot() methods from open-atmos-jupyter-utils
+  entry: notebooks_using_jupyter_utils
+  language: python
+  stages: [pre-commit]
+  types: [jupyter]