Skip to content

Update dependency file paths and simplify CI workflow steps #323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
shaypal5 merged 4 commits into from
Jan 27, 2026
Merged

Update dependency file paths and simplify CI workflow steps #323

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f091fd6
Update dependency file paths and simplify CI workflow steps
Borda Jan 27, 2026
9572aa6
update install requirements
Borda Jan 27, 2026
cd1cb70
add condition to skip dependency installation for local backend
Borda Jan 27, 2026
36b71cd
Update naming convention for backend-specific requirement files
Borda Jan 27, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 6 additions & 17 deletions .github/workflows/ci-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,11 +53,12 @@ jobs:
uses: actions/setup-python@v6
with:
python-version: ${{ matrix.python-version }}
cache: "pip"

- name: Install dependencies
run: |
python -m pip install --upgrade pip
python -m pip install -e . -r tests/requirements.txt
python -m pip install -e .

- name: Lower inotify instance limit (Linux only, for Issue #24 test)
if: runner.os == 'Linux' && matrix.backend == 'local'
Expand All @@ -75,14 +76,17 @@ jobs:
colima start
# For testcontainers to find the Colima socket
sudo ln -sf $HOME/.colima/default/docker.sock /var/run/docker.sock

- name: Install test dependencies
run: python -m pip install -e . -r tests/requirements_${{ matrix.backend }}.txt
Copy link

Copilot AI Jan 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step has two critical issues:

  1. It runs after the "Unit tests (local)" step (lines 68-70), but the local tests need pytest and other test dependencies from tests/requirements.txt to run. In the old workflow, these were installed in the "Install dependencies" step, but now they're not installed until this step, which is too late.

  2. When backend is 'local', this step will fail because there is no tests/requirements_local.txt file.

To fix these issues, this step should:

  • Be moved before the "Unit tests (local)" step
  • Include a conditional to handle the 'local' backend, either by creating a requirements_local.txt file that references requirements.txt, or by using a conditional like if: matrix.backend != 'local' and installing tests/requirements.txt for the local backend separately.

Copilot uses AI. Check for mistakes.

# ToDo: find a way to cache docker images
#- name: Cache Container Images
# if: matrix.backend == 'mongodb'
# uses: borda/cache-container-images-action@b32a5e804cb39af3c3d134fc03ab76eac0bfcfa9
# with:
# prefix-key: "mongo-db"
# images: mongo:latest

- name: Start MongoDB in docker
if: matrix.backend == 'mongodb'
run: |
Expand All @@ -93,11 +97,6 @@ jobs:
# show running containers
docker ps -a

- name: Install MongoDB core test dependencies
if: matrix.backend == 'mongodb'
run: |
python -m pip install -e . -r tests/mongodb_requirements.txt

- name: Unit tests (DB)
if: matrix.backend == 'mongodb'
run: pytest -m "mongo" --cov=cachier --cov-report=term --cov-report=xml:cov.xml
Expand All @@ -117,11 +116,6 @@ jobs:
sleep 10
docker ps -a

- name: Install SQL core test dependencies (SQL/Postgres)
if: matrix.backend == 'postgres'
run: |
python -m pip install -e . -r tests/sql_requirements.txt

- name: Unit tests (SQL/Postgres)
if: matrix.backend == 'postgres'
env:
Expand All @@ -138,11 +132,6 @@ jobs:
sleep 5
docker ps -a

- name: Install Redis core test dependencies
if: matrix.backend == 'redis'
run: |
python -m pip install -e . -r tests/redis_requirements.txt

- name: Unit tests (Redis)
if: matrix.backend == 'redis'
run: pytest -m redis --cov=cachier --cov-report=term --cov-report=xml:cov.xml
Expand Down
6 changes: 3 additions & 3 deletions AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -406,9 +406,9 @@ ______________________________________________________________________
pip install -e .
pip install -r tests/requirements.txt
# For specific backends:
pip install -r tests/mongodb_requirements.txt
pip install -r tests/redis_requirements.txt
pip install -r tests/sql_requirements.txt
pip install -r tests/requirements_mongodb.txt
pip install -r tests/requirements_redis.txt
pip install -r tests/requirements_postgres.txt
```
- **Run all tests:** `pytest`
- **Run backend-specific tests:** `pytest -m <backend>` (mongo, redis, sql, memory, pickle, maxage)
Expand Down
6 changes: 3 additions & 3 deletions CLAUDE.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -406,9 +406,9 @@ ______________________________________________________________________
pip install -e .
pip install -r tests/requirements.txt
# For specific backends:
pip install -r tests/mongodb_requirements.txt
pip install -r tests/redis_requirements.txt
pip install -r tests/sql_requirements.txt
pip install -r tests/requirements_mongodb.txt
pip install -r tests/requirements_redis.txt
pip install -r tests/requirements_postgres.txt
```
- **Run all tests:** `pytest`
- **Run backend-specific tests:** `pytest -m <backend>` (mongo, redis, sql, memory, pickle, maxage)
Expand Down
6 changes: 3 additions & 3 deletions scripts/README-local-testing.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,9 +154,9 @@ The script automatically sets the required environment variables:
2. **Python dependencies** - Install test requirements:
```bash
pip install -r tests/requirements.txt
pip install -r tests/mongodb_requirements.txt # For MongoDB tests
pip install -r tests/redis_requirements.txt # For Redis tests
pip install -r tests/sql_requirements.txt # For SQL tests
pip install -r tests/requirements_mongodb.txt # For MongoDB tests
pip install -r tests/requirements_redis.txt # For Redis tests
pip install -r tests/requirements_postgres.txt # For SQL tests
```

## Troubleshooting
Expand Down
2 changes: 1 addition & 1 deletion tests/mongodb_requirements.txt → tests/requirements_mongodb.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@

-r requirements.txt
# to connect to the test mongodb server
pymongo
dnspython
Expand Down
1 change: 1 addition & 0 deletions tests/sql_requirements.txt → tests/requirements_postgres.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
-r requirements.txt
# for SQL core tests
SQLAlchemy
psycopg2-binary
2 changes: 2 additions & 0 deletions tests/redis_requirements.txt → tests/requirements_redis.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
-r requirements.txt

redis>=4.0.0
pandas>=1.3.0
birch>=0.0.35
Loading