Skip to content

Commit ce7e846

Browse files
committed
codex changes
1 parent 2827e10 commit ce7e846

3 files changed

Lines changed: 452 additions & 5 deletions

File tree

README.rst

Lines changed: 14 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@ Current features
5959
* Cross-machine caching using MongoDB.
6060
* SQL-based caching using SQLAlchemy-supported databases.
6161
* Redis-based caching for high-performance scenarios.
62+
* S3-based caching for cross-machine object storage backends.
6263

6364
* Thread-safety.
6465
* **Per-call max age:** Specify a maximum age for cached values per call.
@@ -71,7 +72,6 @@ Cachier is **NOT**:
7172
Future features
7273
---------------
7374

74-
* S3 core.
7575
* Multi-core caching.
7676
* `Cache replacement policies <https://en.wikipedia.org/wiki/Cache_replacement_policies>`_
7777

@@ -580,6 +580,12 @@ Cachier supports Redis-based caching for high-performance scenarios. Redis provi
580580
- ``processing``: Boolean, is value being calculated
581581
- ``completed``: Boolean, is value calculation completed
582582

583+
**S3 Sync/Async Support:**
584+
585+
- Sync functions use direct boto3 calls.
586+
- Async functions are supported via thread-offloaded sync boto3 calls
587+
(delegated mode), not a native async client.
588+
583589
**Limitations & Notes:**
584590

585591
- Requires SQLAlchemy (install with ``pip install SQLAlchemy``)
@@ -631,6 +637,11 @@ async drivers and require the client or engine type to match the decorated funct
631637
- ``redis_client`` must be a sync client or sync callable for sync functions and
632638
an async callable returning a ``redis.asyncio.Redis`` client for async
633639
functions. Passing a sync callable to an async function raises ``TypeError``.
640+
* - **S3**
641+
- Yes
642+
- Yes (delegated)
643+
- Async support is delegated via thread-offloaded sync boto3 calls
644+
(``asyncio.to_thread``). No async S3 client is required.
634645

635646

636647
Contributing
@@ -655,13 +666,14 @@ Install in development mode with test dependencies for local cores (memory and p
655666
cd cachier
656667
pip install -e . -r tests/requirements.txt
657668
658-
Each additional core (MongoDB, Redis, SQL) requires additional dependencies. To install all dependencies for all cores, run:
669+
Each additional core (MongoDB, Redis, SQL, S3) requires additional dependencies. To install all dependencies for all cores, run:
659670

660671
.. code-block:: bash
661672
662673
pip install -r tests/requirements_mongodb.txt
663674
pip install -r tests/requirements_redis.txt
664675
pip install -r tests/requirements_postgres.txt
676+
pip install -r tests/requirements_s3.txt
665677
666678
Running the tests
667679
-----------------

src/cachier/cores/s3.py

Lines changed: 52 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
"""An S3-based caching core for cachier."""
22

3+
import asyncio
34
import pickle
45
import time
56
import warnings
@@ -28,6 +29,10 @@ class MissingS3Bucket(ValueError):
2829
class _S3Core(_BaseCore):
2930
"""S3-based core for Cachier, supporting AWS S3 and S3-compatible backends.
3031
32+
Async support in this core is delegated rather than native. Since boto3 is a
33+
synchronous client, async methods explicitly offload I/O work to a thread via
34+
``asyncio.to_thread``.
35+
3136
Parameters
3237
----------
3338
hash_func : callable, optional
@@ -367,6 +372,51 @@ def delete_stale_entries(self, stale_after: timedelta) -> None:
367372
warnings.warn(f"S3 delete_stale_entries failed: {exc}", stacklevel=2)
368373

369374
# ------------------------------------------------------------------
370-
# Async variants delegate to the thread-based defaults in _BaseCore
371-
# since boto3 is a sync library.
375+
# Async variants explicitly offload sync boto3 operations to avoid
376+
# blocking the event loop thread.
372377
# ------------------------------------------------------------------
378+
379+
async def aget_entry(self, args, kwds) -> Tuple[str, Optional[CacheEntry]]:
380+
"""Async-compatible variant of :meth:`get_entry`.
381+
382+
This method delegates to the sync implementation via
383+
``asyncio.to_thread`` because boto3 is sync-only.
384+
385+
"""
386+
return await asyncio.to_thread(self.get_entry, args, kwds)
387+
388+
async def aget_entry_by_key(self, key: str) -> Tuple[str, Optional[CacheEntry]]:
389+
"""Async-compatible variant of :meth:`get_entry_by_key`.
390+
391+
This method delegates to the sync implementation via
392+
``asyncio.to_thread`` because boto3 is sync-only.
393+
394+
"""
395+
return await asyncio.to_thread(self.get_entry_by_key, key)
396+
397+
async def aset_entry(self, key: str, func_res: Any) -> bool:
398+
"""Async-compatible variant of :meth:`set_entry`.
399+
400+
This method delegates to the sync implementation via
401+
``asyncio.to_thread`` because boto3 is sync-only.
402+
403+
"""
404+
return await asyncio.to_thread(self.set_entry, key, func_res)
405+
406+
async def amark_entry_being_calculated(self, key: str) -> None:
407+
"""Async-compatible variant of :meth:`mark_entry_being_calculated`.
408+
409+
This method delegates to the sync implementation via
410+
``asyncio.to_thread`` because boto3 is sync-only.
411+
412+
"""
413+
await asyncio.to_thread(self.mark_entry_being_calculated, key)
414+
415+
async def amark_entry_not_calculated(self, key: str) -> None:
416+
"""Async-compatible variant of :meth:`mark_entry_not_calculated`.
417+
418+
This method delegates to the sync implementation via
419+
``asyncio.to_thread`` because boto3 is sync-only.
420+
421+
"""
422+
await asyncio.to_thread(self.mark_entry_not_calculated, key)

0 commit comments

Comments
 (0)