Update dependency file paths and simplify CI workflow steps (#323)

* Update dependency file paths and simplify CI workflow steps

* update install requirements

* add condition to skip dependency installation for local backend

* Update naming convention for backend-specific requirement files

Author: Borda

.github/workflows/ci-test.yml

@@ -53,11 +53,10 @@ jobs:
         uses: actions/setup-python@v6
         with:
           python-version: ${{ matrix.python-version }}
+          cache: "pip"
 
-      - name: Install dependencies
-        run: |
-          python -m pip install --upgrade pip
-          python -m pip install -e . -r tests/requirements.txt
+      - name: Install package & dependencies
+        run: python -m pip install -e . -r tests/requirements.txt
 
       - name: Lower inotify instance limit (Linux only, for Issue #24 test)
         if: runner.os == 'Linux' && matrix.backend == 'local'
@@ -75,14 +74,18 @@ jobs:
           colima start
           # For testcontainers to find the Colima socket
           sudo ln -sf $HOME/.colima/default/docker.sock /var/run/docker.sock
+
+      - name: Install other test dependencies
+        if: matrix.backend != 'local'
+        run: python -m pip install -e . -r tests/requirements_${{ matrix.backend }}.txt
+
       # ToDo: find a way to cache docker images
       #- name: Cache Container Images
       #  if: matrix.backend == 'mongodb'
       #  uses: borda/cache-container-images-action@b32a5e804cb39af3c3d134fc03ab76eac0bfcfa9
       #  with:
       #    prefix-key: "mongo-db"
       #    images: mongo:latest
-
       - name: Start MongoDB in docker
         if: matrix.backend == 'mongodb'
         run: |
@@ -93,11 +96,6 @@ jobs:
           # show running containers
           docker ps -a
 
-      - name: Install MongoDB core test dependencies
-        if: matrix.backend == 'mongodb'
-        run: |
-          python -m pip install -e . -r tests/mongodb_requirements.txt
-
       - name: Unit tests (DB)
         if: matrix.backend == 'mongodb'
         run: pytest -m "mongo" --cov=cachier --cov-report=term --cov-report=xml:cov.xml
@@ -117,11 +115,6 @@ jobs:
           sleep 10
           docker ps -a
 
-      - name: Install SQL core test dependencies (SQL/Postgres)
-        if: matrix.backend == 'postgres'
-        run: |
-          python -m pip install -e . -r tests/sql_requirements.txt
-
       - name: Unit tests (SQL/Postgres)
         if: matrix.backend == 'postgres'
         env:
@@ -138,11 +131,6 @@ jobs:
           sleep 5
           docker ps -a
 
-      - name: Install Redis core test dependencies
-        if: matrix.backend == 'redis'
-        run: |
-          python -m pip install -e . -r tests/redis_requirements.txt
-
       - name: Unit tests (Redis)
         if: matrix.backend == 'redis'
         run: pytest -m redis --cov=cachier --cov-report=term --cov-report=xml:cov.xml

AGENTS.md

@@ -36,7 +36,7 @@ cachier/
 │   └── __main__.py
 ├── tests/                 # Pytest-based tests, backend-marked
 │   ├── test_*.py
-│   └── *_requirements.txt # Backend-specific test requirements
+│   └── requirements_*.txt # Backend-specific test requirements
 ├── examples/              # Usage examples
 ├── README.rst             # Main documentation
 ├── pyproject.toml         # Build, lint, type, test config
@@ -55,7 +55,7 @@ ______________________________________________________________________
    pip install .[all]
    ```
 
-   - For backend-specific dev: see `tests/*_requirements.txt`.
+   - For backend-specific dev: see `tests/requirements_*.txt`.
 
 2. **Run tests:**
 
@@ -111,7 +111,7 @@ ______________________________________________________________________
 - **Default:** Pickle (local file cache, `~/.cachier/`)
 - **Others:** Memory, MongoDB, SQL, Redis
 - **Adding a backend:** Implement in `src/cachier/cores/`, subclass `BaseCore`, add tests with appropriate markers, update docs, and CI matrix if needed.
-- **Optional dependencies:** Code/tests must gracefully skip if backend deps are missing. Install backend-specific deps via `tests/*_requirements.txt`.
+- **Optional dependencies:** Code/tests must gracefully skip if backend deps are missing. Install backend-specific deps via `tests/requirements_*.txt`.
 - **Requirements files:** `tests/sql_requirements.txt`, `tests/redis_requirements.txt` for backend-specific dependencies.
 
 ### 3. **Decorator Usage**
@@ -125,7 +125,7 @@ ______________________________________________________________________
 - **Run all tests:** `pytest`
 - **Backend-specific:** Use markers, e.g. `pytest -m mongo`, `pytest -m redis`, `pytest -m sql`
 - **Available markers:** `mongo`, `memory`, `pickle`, `redis`, `sql`, `maxage` (see `pyproject.toml`)
-- **Requirements:** See `tests/*_requirements.txt` for backend test deps.
+- **Requirements:** See `tests/requirements_*.txt` for backend test deps.
 - **CI:** Matrix covers OS/backend combinations. Mongo/SQL/Redis require Dockerized services.
 - **Missing deps:** Tests gracefully skip if optional backend dependencies are missing.
 
@@ -406,9 +406,9 @@ ______________________________________________________________________
   pip install -e .
   pip install -r tests/requirements.txt
   # For specific backends:
-  pip install -r tests/mongodb_requirements.txt
-  pip install -r tests/redis_requirements.txt
-  pip install -r tests/sql_requirements.txt
+  pip install -r tests/requirements_mongodb.txt
+  pip install -r tests/requirements_redis.txt
+  pip install -r tests/requirements_postgres.txt
   ```
 - **Run all tests:** `pytest`
 - **Run backend-specific tests:** `pytest -m <backend>` (mongo, redis, sql, memory, pickle, maxage)
@@ -421,7 +421,7 @@ ______________________________________________________________________
 - **Build package:** `python -m build`
 - **Check docs:** `python setup.py checkdocs`
 - **Run example:** `python examples/redis_example.py`
-- **Update requirements:** Edit `tests/*_requirements.txt` as needed (sql_requirements.txt, redis_requirements.txt).
+- **Update requirements:** Edit `tests/requirements_*.txt` as needed (sql_requirements.txt, redis_requirements.txt).
 
 ### Local Testing with Docker
 
@@ -488,7 +488,7 @@ ______________________________________________________________________
 
 ### b. **Best Practices for Coding Assistance Agents**
 
-- **Always check for backend-specific requirements** before running backend tests or code (see `tests/*_requirements.txt`).
+- **Always check for backend-specific requirements** before running backend tests or code (see `tests/requirements_*.txt`).
 - **When adding a backend:** Update all relevant places (core, tests, docs, CI matrix, requirements files).
 - **When editing core logic:** Ensure all backends are still supported and tested.
 - **When updating the decorator API:** Update docstrings, README, and tests.

CLAUDE.md

@@ -36,7 +36,7 @@ cachier/
 │   └── __main__.py
 ├── tests/                 # Pytest-based tests, backend-marked
 │   ├── test_*.py
-│   └── *_requirements.txt # Backend-specific test requirements
+│   └── requirements_*.txt # Backend-specific test requirements
 ├── examples/              # Usage examples
 ├── README.rst             # Main documentation
 ├── pyproject.toml         # Build, lint, type, test config
@@ -55,7 +55,7 @@ ______________________________________________________________________
    pip install .[all]
    ```
 
-   - For backend-specific dev: see `tests/*_requirements.txt`.
+   - For backend-specific dev: see `tests/requirements_*.txt`.
 
 2. **Run tests:**
 
@@ -111,7 +111,7 @@ ______________________________________________________________________
 - **Default:** Pickle (local file cache, `~/.cachier/`)
 - **Others:** Memory, MongoDB, SQL, Redis
 - **Adding a backend:** Implement in `src/cachier/cores/`, subclass `BaseCore`, add tests with appropriate markers, update docs, and CI matrix if needed.
-- **Optional dependencies:** Code/tests must gracefully skip if backend deps are missing. Install backend-specific deps via `tests/*_requirements.txt`.
+- **Optional dependencies:** Code/tests must gracefully skip if backend deps are missing. Install backend-specific deps via `tests/requirements_*.txt`.
 - **Requirements files:** `tests/sql_requirements.txt`, `tests/redis_requirements.txt` for backend-specific dependencies.
 
 ### 3. **Decorator Usage**
@@ -125,7 +125,7 @@ ______________________________________________________________________
 - **Run all tests:** `pytest`
 - **Backend-specific:** Use markers, e.g. `pytest -m mongo`, `pytest -m redis`, `pytest -m sql`
 - **Available markers:** `mongo`, `memory`, `pickle`, `redis`, `sql`, `maxage` (see `pyproject.toml`)
-- **Requirements:** See `tests/*_requirements.txt` for backend test deps.
+- **Requirements:** See `tests/requirements_*.txt` for backend test deps.
 - **CI:** Matrix covers OS/backend combinations. Mongo/SQL/Redis require Dockerized services.
 - **Missing deps:** Tests gracefully skip if optional backend dependencies are missing.
 
@@ -406,9 +406,9 @@ ______________________________________________________________________
   pip install -e .
   pip install -r tests/requirements.txt
   # For specific backends:
-  pip install -r tests/mongodb_requirements.txt
-  pip install -r tests/redis_requirements.txt
-  pip install -r tests/sql_requirements.txt
+  pip install -r tests/requirements_mongodb.txt
+  pip install -r tests/requirements_redis.txt
+  pip install -r tests/requirements_postgres.txt
   ```
 - **Run all tests:** `pytest`
 - **Run backend-specific tests:** `pytest -m <backend>` (mongo, redis, sql, memory, pickle, maxage)
@@ -421,7 +421,7 @@ ______________________________________________________________________
 - **Build package:** `python -m build`
 - **Check docs:** `python setup.py checkdocs`
 - **Run example:** `python examples/redis_example.py`
-- **Update requirements:** Edit `tests/*_requirements.txt` as needed (sql_requirements.txt, redis_requirements.txt).
+- **Update requirements:** Edit `tests/requirements_*.txt` as needed (sql_requirements.txt, redis_requirements.txt).
 
 ### Local Testing with Docker
 
@@ -488,7 +488,7 @@ ______________________________________________________________________
 
 ### b. **Best Practices for Claude**
 
-- **Always check for backend-specific requirements** before running backend tests or code (see `tests/*_requirements.txt`).
+- **Always check for backend-specific requirements** before running backend tests or code (see `tests/requirements_*.txt`).
 - **When adding a backend:** Update all relevant places (core, tests, docs, CI matrix, requirements files).
 - **When editing core logic:** Ensure all backends are still supported and tested.
 - **When updating the decorator API:** Update docstrings, README, and tests.

scripts/README-local-testing.md

@@ -154,9 +154,9 @@ The script automatically sets the required environment variables:
 2. **Python dependencies** - Install test requirements:
    ```bash
    pip install -r tests/requirements.txt
-   pip install -r tests/mongodb_requirements.txt  # For MongoDB tests
-   pip install -r tests/redis_requirements.txt    # For Redis tests
-   pip install -r tests/sql_requirements.txt      # For SQL tests
+   pip install -r tests/requirements_mongodb.txt  # For MongoDB tests
+   pip install -r tests/requirements_redis.txt    # For Redis tests
+   pip install -r tests/requirements_postgres.txt      # For SQL tests
    ```
 
 ## Troubleshooting

tests/mongodb_requirements.txt tests/requirements_mongodb.txttests/mongodb_requirements.txt renamed to tests/requirements_mongodb.txt

@@ -1,4 +1,4 @@
-
+-r requirements.txt
 # to connect to the test mongodb server
 pymongo
 dnspython

tests/sql_requirements.txt tests/requirements_postgres.txttests/sql_requirements.txt renamed to tests/requirements_postgres.txt

@@ -1,3 +1,4 @@
+-r requirements.txt
 # for SQL core tests
 SQLAlchemy
 psycopg2-binary

tests/redis_requirements.txt tests/requirements_redis.txttests/redis_requirements.txt renamed to tests/requirements_redis.txt

@@ -1,3 +1,5 @@
+-r requirements.txt
+
 redis>=4.0.0
 pandas>=1.3.0
 birch>=0.0.35