Skip to content

Commit 98f137b

Browse files
authored
docs: update AGENTS workflows for s3 testing (#363)
1 parent acf18b5 commit 98f137b

1 file changed

Lines changed: 20 additions & 13 deletions

File tree

AGENTS.md

Lines changed: 20 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66

77
- **Repository:** [python-cachier/cachier](https://github.com/python-cachier/cachier)
88
- **Primary Language:** Python 3.10+
9-
- **Key Dependencies:** `portalocker`, `watchdog` (optional: `pymongo`, `sqlalchemy`, `redis`)
9+
- **Key Dependencies:** `portalocker`, `watchdog` (optional: `boto3`, `pymongo`, `sqlalchemy`, `redis`)
1010
- **Test Framework:** `pytest` with backend-specific markers
1111
- **Linting:** `ruff` (replaces black/flake8)
1212
- **Type Checking:** `mypy`
@@ -29,6 +29,7 @@ cachier/
2929
│ │ ├── mongo.py
3030
│ │ ├── sql.py
3131
│ │ ├── redis.py
32+
│ │ ├── s3.py
3233
│ │ └── base.py
3334
│ ├── config.py # Global/default config
3435
│ ├── _types.py # Type definitions
@@ -62,7 +63,7 @@ ______________________________________________________________________
6263
```bash
6364
pytest # All tests
6465
pytest -m "pickle or memory" # Basic backends only
65-
pytest -m "not (mongo or sql)" # Exclude external service backends
66+
pytest -m "not (mongo or redis or sql)" # Exclude external service backends
6667
```
6768

6869
3. **Lint and type-check:**
@@ -109,10 +110,10 @@ ______________________________________________________________________
109110
### 2. **Backends**
110111

111112
- **Default:** Pickle (local file cache, `~/.cachier/`)
112-
- **Others:** Memory, MongoDB, SQL, Redis
113+
- **Others:** Memory, MongoDB, SQL, Redis, S3
113114
- **Adding a backend:** Implement in `src/cachier/cores/`, subclass `BaseCore`, add tests with appropriate markers, update docs, and CI matrix if needed.
114115
- **Optional dependencies:** Code/tests must gracefully skip if backend deps are missing. Install backend-specific deps via `tests/requirements_*.txt`.
115-
- **Requirements files:** `tests/requirements_mongodb.txt`, `tests/requirements_postgres.txt`, `tests/requirements_redis.txt` for backend-specific dependencies.
116+
- **Requirements files:** `tests/requirements_mongodb.txt`, `tests/requirements_postgres.txt`, `tests/requirements_redis.txt`, `tests/requirements_s3.txt` for backend-specific dependencies.
116117

117118
### 3. **Decorator Usage**
118119

@@ -123,8 +124,8 @@ ______________________________________________________________________
123124
### 4. **Testing**
124125

125126
- **Run all tests:** `pytest`
126-
- **Backend-specific:** Use markers, e.g. `pytest -m mongo`, `pytest -m redis`, `pytest -m sql`
127-
- **Available markers:** `mongo`, `memory`, `pickle`, `redis`, `sql`, `maxage` (see `pyproject.toml`)
127+
- **Backend-specific:** Use markers, e.g. `pytest -m mongo`, `pytest -m redis`, `pytest -m sql`, `pytest -m s3`
128+
- **Available markers:** `mongo`, `memory`, `pickle`, `redis`, `sql`, `s3`, `maxage` (see `pyproject.toml`)
128129
- **Requirements:** See `tests/requirements_*.txt` for backend test deps.
129130
- **CI:** Matrix covers OS/backend combinations. Mongo/SQL/Redis require Dockerized services.
130131
- **Missing deps:** Tests gracefully skip if optional backend dependencies are missing.
@@ -160,7 +161,7 @@ For an up-to-date overview of the repository layout, see `README.rst` in the pro
160161

161162
### Key functionality
162163

163-
- core.py exposes the cachier decorator. It chooses a backend (pickle, mongo, memory, SQL, or Redis) and wraps the target function:
164+
- core.py exposes the cachier decorator. It chooses a backend (pickle, mongo, memory, SQL, Redis, or S3) and wraps the target function:
164165

165166
```python
166167
backend = _update_with_defaults(backend, "backend")
@@ -182,6 +183,8 @@ elif backend == "redis":
182183
redis_client=redis_client,
183184
wait_for_calc_timeout=wait_for_calc_timeout,
184185
)
186+
elif backend == "s3":
187+
core = _S3Core(...)
185188
else:
186189
raise ValueError("specified an invalid core: %s" % backend)
187190
```
@@ -204,7 +207,7 @@ class Params:
204207
allow_none: bool = False
205208
```
206209

207-
- The project supports multiple backends; each resides under src/cachier/cores/ (e.g., redis.py, mongo.py, etc.). The Redis example demonstrates how to use one backend:
210+
- The project supports multiple backends; each resides under src/cachier/cores/ (e.g., redis.py, mongo.py, s3.py). The Redis example demonstrates how to use one backend:
208211

209212
```python
210213
import time
@@ -395,9 +398,10 @@ ______________________________________________________________________
395398
pip install -r tests/requirements_mongodb.txt
396399
pip install -r tests/requirements_redis.txt
397400
pip install -r tests/requirements_postgres.txt
401+
pip install -r tests/requirements_s3.txt
398402
```
399403
- **Run all tests:** `pytest`
400-
- **Run backend-specific tests:** `pytest -m <backend>` (mongo, redis, sql, memory, pickle, maxage)
404+
- **Run backend-specific tests:** `pytest -m <backend>` (mongo, redis, sql, s3, memory, pickle, maxage)
401405
- **Run multiple backends:** `pytest -m "redis or sql"`
402406
- **Exclude backends:** `pytest -m "not mongo"`
403407
- **Lint:** `ruff check .`
@@ -407,7 +411,7 @@ ______________________________________________________________________
407411
- **Build package:** `python -m build`
408412
- **Check docs:** `python setup.py checkdocs`
409413
- **Run example:** `python examples/redis_example.py`
410-
- **Update requirements:** Edit `tests/requirements_*.txt` as needed (`requirements_mongodb.txt`, `requirements_postgres.txt`, `requirements_redis.txt`).
414+
- **Update requirements:** Edit `tests/requirements_*.txt` as needed (`requirements_mongodb.txt`, `requirements_postgres.txt`, `requirements_redis.txt`, `requirements_s3.txt`).
411415

412416
### Local Testing with Docker
413417

@@ -418,6 +422,7 @@ ______________________________________________________________________
418422
./scripts/test-local.sh mongo
419423
./scripts/test-local.sh redis
420424
./scripts/test-local.sh sql
425+
./scripts/test-local.sh s3
421426

422427
# Test multiple backends
423428
./scripts/test-local.sh mongo redis
@@ -444,6 +449,7 @@ ______________________________________________________________________
444449
- `mongo` - MongoDB backend
445450
- `redis` - Redis backend
446451
- `sql` - PostgreSQL backend
452+
- `s3` - S3 backend (no Docker)
447453
- `memory` - Memory backend (no Docker)
448454
- `pickle` - Pickle backend (no Docker)
449455
- `all` - All backends
@@ -455,8 +461,9 @@ ______________________________________________________________________
455461
- `-v, --verbose` - Verbose pytest output
456462
- `-k, --keep-running` - Keep containers running after tests
457463
- `-h, --html-coverage` - Generate HTML coverage report
464+
- `-f, --files` - Run only specific test files
458465

459-
**Note:** External backends (MongoDB, Redis, SQL) require Docker. Memory and pickle backends work without Docker.
466+
**Note:** External backends (MongoDB, Redis, SQL) require Docker. S3, memory, and pickle backends work without Docker.
460467

461468
______________________________________________________________________
462469

@@ -498,7 +505,7 @@ ______________________________________________________________________
498505

499506
## 🧪 Testing Matrix & Markers
500507

501-
- **Markers:** `@pytest.mark.<backend>` (mongo, memory, pickle, redis, sql, maxage)
508+
- **Markers:** `@pytest.mark.<backend>` (mongo, memory, pickle, redis, sql, s3, maxage)
502509
- **Backend services:** Mongo/SQL/Redis require Dockerized services for CI.
503510
- **Tests must not break if optional backend deps are missing.**
504511
- **CI matrix:** See `.github/workflows/` for details on OS/backend combinations.
@@ -567,7 +574,7 @@ ______________________________________________________________________
567574
- **Never emit warnings/errors for missing optional deps at import time.**
568575
- **All code must be Python 3.10+ compatible.**
569576
- **All new code must have full type annotations and numpy-style docstrings.**
570-
- **Backend consistency:** Ensure all backends (pickle, memory, mongo, sql, redis) are supported.\*\*
577+
- **Backend consistency:** Ensure all backends (pickle, memory, mongo, sql, redis, s3) are supported.\*\*
571578
- **Validation:** Test examples in this file work: `python -c "from cachier import cachier; ..."` should succeed.
572579
- **If you are unsure about a pattern, check the README and this file first.**
573580
- **If you are stuck, suggest opening a new chat with the latest context.**

0 commit comments

Comments
 (0)