Fix/memory leak fixes

[adammcdonagh]

This pull request introduces several improvements and fixes related to memory management, logging, and configurability for batch task handling. The main highlights are the resolution of memory leaks in both batch handlers and the logger, the addition of new environment variables for memory usage logging and polling interval, and corresponding documentation and dependency updates.

Memory management improvements:

• Fixed a memory leak in batch processing by ensuring remote handler objects are released after each task completes (src/opentaskpy/taskhandlers/batch.py).
• Fixed a small memory leak in the logger by unregistering loggers from the logger dictionary after they are closed, allowing them to be garbage collected (src/opentaskpy/otflogging.py). [1] [2] [3] [4]

Logging and monitoring enhancements:

• Added the OTF_LOG_MEMORY_USAGE environment variable to enable logging of RSS memory usage at each batch poll interval (requires psutil); this aids in diagnosing memory growth in long-running batches (src/opentaskpy/taskhandlers/batch.py, README.md, docs/usage.md, src/opentaskpy/cli/task_run.py). [1] [2] [3] [4] [5]

Configurability improvements:

• Added the OTF_BATCH_POLL_INTERVAL environment variable to allow customization of the batch status-check loop interval, with documentation warning against lowering it in production (src/opentaskpy/taskhandlers/batch.py, README.md, docs/usage.md, src/opentaskpy/cli/task_run.py). [1] [2] [3] [4] [5]

Dependency and documentation updates:

• Added psutil as an optional development dependency for memory usage logging (pyproject.toml).

CI/CD maintenance:

• Updated GitHub Actions workflow to use actions/cache@v5 for improved caching reliability (.github/workflows/test.yml). [1] [2] [3] [4] [5] [6] [7] [8]

[codecov]

Codecov Report

❌ Patch coverage is 90.00000% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.99%. Comparing base (08794e8) to head (ae3aac3).

Files with missing lines	Patch %	Lines
src/opentaskpy/taskhandlers/batch.py	84.61%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #158      +/-   ##
==========================================
- Coverage   89.03%   88.99%   -0.04%     
==========================================
  Files          22       22              
  Lines        2708     2726      +18     
==========================================
+ Hits         2411     2426      +15     
- Misses        297      300       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.