Skip to content

Commit d645846

Browse files
[MISC] Fix Azure listings_expiry_time regression from PR #1826 (#1839)
Fix Azure listings_expiry_time regression from PR #1826 Revert listings_expiry_time back to 1 (from 0). A value of 0 breaks because fsspec's DirCache expires entries between write and immediate read, causing KeyError('/') in adlfs's _ls_containers(). Updated comment to document the issue and reference the related GitHub issue.
1 parent 410055f commit d645846

File tree

1 file changed

+4
-3
lines changed

1 file changed

+4
-3
lines changed

unstract/connectors/src/unstract/connectors/filesystems/azure_cloud_storage/azure_cloud_storage.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -41,13 +41,14 @@ def __init__(self, settings: dict[str, Any]):
4141
self.bucket = settings.get("bucket", "")
4242
# adlfs._ls_containers() unconditionally reads from DirCache after
4343
# populating it — use_listings_cache=False makes the write a no-op,
44-
# causing a KeyError. Setting listings_expiry_time=0 allows writes
45-
# but expires entries immediately on subsequent reads.
44+
# causing a KeyError. listings_expiry_time=0 also breaks because
45+
# DirCache expires entries between the write and immediate read
46+
# (even nanoseconds trigger expiry). Use 1s as the minimum safe value.
4647
# Check https://github.com/fsspec/adlfs/issues/230 for more context.
4748
self.azure_fs = AzureBlobFileSystem(
4849
account_name=account_name,
4950
credential=access_key,
50-
listings_expiry_time=0,
51+
listings_expiry_time=1,
5152
)
5253

5354
@staticmethod

0 commit comments

Comments
 (0)