Commit ffaba2d
fix: use Duration strings for Polars object_store S3 timeout options
Polars' native Rust reader passes storage_options through to the object_store
crate, which requires Duration strings (e.g. '10s', '120s') rather than plain
numeric second values ('10', '120').
pyiceberg's PyArrowFileIO uses float(value) for the same keys, so the two callers
need different formats. Split into separate dicts: pyiceberg_s3_properties (plain
numbers for GlueCatalog) and polars_s3_storage_options (duration strings for
pl.scan_iceberg).
Fixes: object-store error: Generic Config error: failed to parse "120" as Duration
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>1 parent fafb9d4 commit ffaba2d
1 file changed
Lines changed: 13 additions & 9 deletions
Lines changed: 13 additions & 9 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
106 | 106 | | |
107 | 107 | | |
108 | 108 | | |
109 | | - | |
110 | | - | |
111 | | - | |
112 | | - | |
113 | | - | |
114 | | - | |
115 | | - | |
| 109 | + | |
| 110 | + | |
116 | 111 | | |
117 | 112 | | |
118 | 113 | | |
119 | 114 | | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
120 | 124 | | |
121 | 125 | | |
122 | 126 | | |
123 | | - | |
| 127 | + | |
124 | 128 | | |
125 | 129 | | |
126 | 130 | | |
127 | | - | |
| 131 | + | |
0 commit comments