66
77- ** Repository:** [ python-cachier/cachier] ( https://github.com/python-cachier/cachier )
88- ** Primary Language:** Python 3.10+
9- - ** Key Dependencies:** ` portalocker ` , ` watchdog ` (optional: ` pymongo ` , ` sqlalchemy ` , ` redis ` )
9+ - ** Key Dependencies:** ` portalocker ` , ` watchdog ` (optional: ` boto3 ` , ` pymongo ` , ` sqlalchemy ` , ` redis ` )
1010- ** Test Framework:** ` pytest ` with backend-specific markers
1111- ** Linting:** ` ruff ` (replaces black/flake8)
1212- ** Type Checking:** ` mypy `
@@ -29,6 +29,7 @@ cachier/
2929│ │ ├── mongo.py
3030│ │ ├── sql.py
3131│ │ ├── redis.py
32+ │ │ ├── s3.py
3233│ │ └── base.py
3334│ ├── config.py # Global/default config
3435│ ├── _types.py # Type definitions
@@ -62,7 +63,7 @@ ______________________________________________________________________
6263 ``` bash
6364 pytest # All tests
6465 pytest -m " pickle or memory" # Basic backends only
65- pytest -m " not (mongo or sql)" # Exclude external service backends
66+ pytest -m " not (mongo or redis or sql)" # Exclude external service backends
6667 ```
6768
68693 . ** Lint and type-check:**
@@ -109,10 +110,10 @@ ______________________________________________________________________
109110### 2. ** Backends**
110111
111112- ** Default:** Pickle (local file cache, ` ~/.cachier/ ` )
112- - ** Others:** Memory, MongoDB, SQL, Redis
113+ - ** Others:** Memory, MongoDB, SQL, Redis, S3
113114- ** Adding a backend:** Implement in ` src/cachier/cores/ ` , subclass ` BaseCore ` , add tests with appropriate markers, update docs, and CI matrix if needed.
114115- ** Optional dependencies:** Code/tests must gracefully skip if backend deps are missing. Install backend-specific deps via ` tests/requirements_*.txt ` .
115- - ** Requirements files:** ` tests/requirements_mongodb.txt ` , ` tests/requirements_postgres.txt ` , ` tests/requirements_redis.txt ` for backend-specific dependencies.
116+ - ** Requirements files:** ` tests/requirements_mongodb.txt ` , ` tests/requirements_postgres.txt ` , ` tests/requirements_redis.txt ` , ` tests/requirements_s3.txt ` for backend-specific dependencies.
116117
117118### 3. ** Decorator Usage**
118119
@@ -123,8 +124,8 @@ ______________________________________________________________________
123124### 4. ** Testing**
124125
125126- ** Run all tests:** ` pytest `
126- - ** Backend-specific:** Use markers, e.g. ` pytest -m mongo ` , ` pytest -m redis ` , ` pytest -m sql `
127- - ** Available markers:** ` mongo ` , ` memory ` , ` pickle ` , ` redis ` , ` sql ` , ` maxage ` (see ` pyproject.toml ` )
127+ - ** Backend-specific:** Use markers, e.g. ` pytest -m mongo ` , ` pytest -m redis ` , ` pytest -m sql ` , ` pytest -m s3 `
128+ - ** Available markers:** ` mongo ` , ` memory ` , ` pickle ` , ` redis ` , ` sql ` , ` s3 ` , ` maxage ` (see ` pyproject.toml ` )
128129- ** Requirements:** See ` tests/requirements_*.txt ` for backend test deps.
129130- ** CI:** Matrix covers OS/backend combinations. Mongo/SQL/Redis require Dockerized services.
130131- ** Missing deps:** Tests gracefully skip if optional backend dependencies are missing.
@@ -160,7 +161,7 @@ For an up-to-date overview of the repository layout, see `README.rst` in the pro
160161
161162### Key functionality
162163
163- - core.py exposes the cachier decorator. It chooses a backend (pickle, mongo, memory, SQL, or Redis ) and wraps the target function:
164+ - core.py exposes the cachier decorator. It chooses a backend (pickle, mongo, memory, SQL, Redis, or S3 ) and wraps the target function:
164165
165166``` python
166167backend = _update_with_defaults(backend, " backend" )
@@ -182,6 +183,8 @@ elif backend == "redis":
182183 redis_client = redis_client,
183184 wait_for_calc_timeout = wait_for_calc_timeout,
184185 )
186+ elif backend == " s3" :
187+ core = _S3Core(... )
185188else :
186189 raise ValueError (" specified an invalid core: %s " % backend)
187190```
@@ -204,7 +207,7 @@ class Params:
204207 allow_none: bool = False
205208```
206209
207- - The project supports multiple backends; each resides under src/cachier/cores/ (e.g., redis.py, mongo.py, etc. ). The Redis example demonstrates how to use one backend:
210+ - The project supports multiple backends; each resides under src/cachier/cores/ (e.g., redis.py, mongo.py, s3.py ). The Redis example demonstrates how to use one backend:
208211
209212``` python
210213import time
@@ -395,9 +398,10 @@ ______________________________________________________________________
395398 pip install -r tests/requirements_mongodb.txt
396399 pip install -r tests/requirements_redis.txt
397400 pip install -r tests/requirements_postgres.txt
401+ pip install -r tests/requirements_s3.txt
398402 ```
399403- ** Run all tests:** ` pytest `
400- - ** Run backend-specific tests:** ` pytest -m <backend> ` (mongo, redis, sql, memory, pickle, maxage)
404+ - ** Run backend-specific tests:** ` pytest -m <backend> ` (mongo, redis, sql, s3, memory, pickle, maxage)
401405- ** Run multiple backends:** ` pytest -m "redis or sql" `
402406- ** Exclude backends:** ` pytest -m "not mongo" `
403407- ** Lint:** ` ruff check . `
@@ -407,7 +411,7 @@ ______________________________________________________________________
407411- ** Build package:** ` python -m build `
408412- ** Check docs:** ` python setup.py checkdocs `
409413- ** Run example:** ` python examples/redis_example.py `
410- - ** Update requirements:** Edit ` tests/requirements_*.txt ` as needed (` requirements_mongodb.txt ` , ` requirements_postgres.txt ` , ` requirements_redis.txt ` ).
414+ - ** Update requirements:** Edit ` tests/requirements_*.txt ` as needed (` requirements_mongodb.txt ` , ` requirements_postgres.txt ` , ` requirements_redis.txt ` , ` requirements_s3.txt ` ).
411415
412416### Local Testing with Docker
413417
@@ -418,6 +422,7 @@ ______________________________________________________________________
418422./scripts/test-local.sh mongo
419423./scripts/test-local.sh redis
420424./scripts/test-local.sh sql
425+ ./scripts/test-local.sh s3
421426
422427# Test multiple backends
423428./scripts/test-local.sh mongo redis
@@ -444,6 +449,7 @@ ______________________________________________________________________
444449- ` mongo ` - MongoDB backend
445450- ` redis ` - Redis backend
446451- ` sql ` - PostgreSQL backend
452+ - ` s3 ` - S3 backend (no Docker)
447453- ` memory ` - Memory backend (no Docker)
448454- ` pickle ` - Pickle backend (no Docker)
449455- ` all ` - All backends
@@ -455,8 +461,9 @@ ______________________________________________________________________
455461- ` -v, --verbose ` - Verbose pytest output
456462- ` -k, --keep-running ` - Keep containers running after tests
457463- ` -h, --html-coverage ` - Generate HTML coverage report
464+ - ` -f, --files ` - Run only specific test files
458465
459- ** Note:** External backends (MongoDB, Redis, SQL) require Docker. Memory and pickle backends work without Docker.
466+ ** Note:** External backends (MongoDB, Redis, SQL) require Docker. S3, memory, and pickle backends work without Docker.
460467
461468______________________________________________________________________
462469
@@ -498,7 +505,7 @@ ______________________________________________________________________
498505
499506## 🧪 Testing Matrix & Markers
500507
501- - ** Markers:** ` @pytest.mark.<backend> ` (mongo, memory, pickle, redis, sql, maxage)
508+ - ** Markers:** ` @pytest.mark.<backend> ` (mongo, memory, pickle, redis, sql, s3, maxage)
502509- ** Backend services:** Mongo/SQL/Redis require Dockerized services for CI.
503510- ** Tests must not break if optional backend deps are missing.**
504511- ** CI matrix:** See ` .github/workflows/ ` for details on OS/backend combinations.
@@ -567,7 +574,7 @@ ______________________________________________________________________
567574- ** Never emit warnings/errors for missing optional deps at import time.**
568575- ** All code must be Python 3.10+ compatible.**
569576- ** All new code must have full type annotations and numpy-style docstrings.**
570- - ** Backend consistency:** Ensure all backends (pickle, memory, mongo, sql, redis) are supported.\*\*
577+ - ** Backend consistency:** Ensure all backends (pickle, memory, mongo, sql, redis, s3 ) are supported.\*\*
571578- ** Validation:** Test examples in this file work: ` python -c "from cachier import cachier; ..." ` should succeed.
572579- ** If you are unsure about a pattern, check the README and this file first.**
573580- ** If you are stuck, suggest opening a new chat with the latest context.**
0 commit comments