Skip to content

Commit 20ae89f

Browse files
Yuri Nikonchukclaude
authored andcommitted
Restore from_uri path, minimize behavioral changes
Keep the original two-path structure in _initialize_hdfs_fs to preserve viewfs and host:port handling via from_uri. Only add replication query param to the existing URI path. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 77e0e3f commit 20ae89f

1 file changed

Lines changed: 4 additions & 4 deletions

File tree

pyiceberg/io/pyarrow.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -578,12 +578,12 @@ def _initialize_azure_fs(self) -> FileSystem:
578578
def _initialize_hdfs_fs(self, scheme: str, netloc: Optional[str]) -> FileSystem:
579579
from pyarrow.fs import HadoopFileSystem
580580

581-
hdfs_kwargs: Dict[str, Any] = {}
582581
replication = self.properties.get(HDFS_REPLICATION, "0")
583-
hdfs_kwargs["replication"] = int(replication)
584582
if netloc:
585-
hdfs_kwargs["host"] = netloc
586-
elif host := self.properties.get(HDFS_HOST):
583+
return HadoopFileSystem.from_uri(f"{scheme}://{netloc}/?replication={replication}")
584+
585+
hdfs_kwargs: Dict[str, Any] = {"replication": int(replication)}
586+
if host := self.properties.get(HDFS_HOST):
587587
hdfs_kwargs["host"] = host
588588
if port := self.properties.get(HDFS_PORT):
589589
# port should be an integer type

0 commit comments

Comments
 (0)