Closed
Conversation
Summary of ChangesHello @kevinjqliu, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request focuses on updating core project dependencies to their latest release candidate and stable versions. Specifically, it upgrades Highlights
🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console. Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates pyiceberg-core to a release candidate and also pins the datafusion dependency. While updating dependencies for testing is a valid approach, I've found a critical issue: the uv.lock file is out of sync with the datafusion version specified in pyproject.toml. This must be resolved to ensure reproducible builds. I've also noted that the specified datafusion version does not seem to exist on PyPI, which should be double-checked. Please note that pinning to a release candidate is fine for testing, but should be addressed before merging into a release branch.
kevinjqliu
force-pushed
the
kevinjqliu/iceberg-rust-rc
branch
from
68f5ef5 to
81591d8
Compare
kevinjqliu
force-pushed
the
kevinjqliu/iceberg-rust-rc
branch
from
1102d7d to
4769c82
Compare
# Rationale for this change Currently, PyIceberg writes one manifest per snapshot operation regardless of manifest size. In order to eventually support this we need to be able to track written bytes without closing the file, so that we can roll to a new file once we hit target size. We had some of this work done in apache#650, but we can keep this simple and add writers as a follow up. The nice thing is that the underlying streams we support already have a tell() method and we just need to expose it. With this change in the follow up we can do: ``` with write_manifest(...) as writer: writer.add_entry(entry) if writer.tell() >= target_file_size: # roll to new file ``` ## Are these changes tested? Yes, added a test :) ## Are there any user-facing changes? No
<!--
Thanks for opening a pull request!
-->
<!-- In the case this PR will resolve an issue, please replace
${GITHUB_ISSUE_ID} below with the actual Github issue id. -->
<!-- Closes #${GITHUB_ISSUE_ID} -->
# Rationale for this change
Closes apache#2810, apache#393
Relates to apache#2743
This PR upgrades `pytest` to `9.0.2`. In doing so, we also had to switch
`pytest-lazy-fixture` to `pytest-lazy-fixtures` (notice the extra s)
which is a maintained version
https://github.com/dev-petrov/pytest-lazy-fixtures
Changed all references of `pytest_lazyfixture` to the new
`pytest_lazy_fixtures`
## Are these changes tested?
## Are there any user-facing changes?
<!-- In the case of user-facing changes, please add the changelog label.
-->
# Rationale for this change
Fixes a test failure that hits anyone with PyIceberg already configured.
The test uses `load_catalog("default", type="in-memory")`, which merges
your dev environment config with the test parameters. If you have a
`default` catalog defined in `~/.pyiceberg.yaml` with something like
uri: https://best-rest-catalog.com, the merge produces `{"uri":
"https://best-rest-catalog.com", "type": "in-memory"}`. SQLAlchemy sees
that https:// URI and tries to load it as a database dialect, which
fails wuth:
```
sqlalchemy.exc.NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:https
```
Therefore, this PR avoids using the local pyicberg.yaml file entirely!
## Are these changes tested?
Yes. I created a conflicting ~/.pyiceberg.yaml with a default catalog
pointing to an HTTPS REST endpoint, confirmed the old code fails, and
verified the fix bypasses the config merge and works.
## Are there any user-facing changes?
No
# Rationale for this change While reviewing some open PRs apache#3041, apache#3042, I noticed CI kept failing on the Python 3.13 job in the hive tests. Turns out [CPython 3.13.12 ](https://docs.python.org/3.13/whatsnew/changelog.html#id3)was just released and included a change python/cpython#142651 which made `Mock.call_count` thread-safe by deriving it from `len(call_args_list)`. This broke our hive test, which was resetting the counter with `mock.call_count = 0` directly. This switches to use reset_mock(), which properly clears all the internal call tracking state. ## Are these changes tested? `make test` passes ## Are there any user-facing changes? no
Bumps [cryptography](https://github.com/pyca/cryptography) from 46.0.3 to 46.0.5. <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/pyca/cryptography/blob/main/CHANGELOG.rst">cryptography's changelog</a>.</em></p> <blockquote> <p>46.0.5 - 2026-02-10</p> <pre><code> * An attacker could create a malicious public key that reveals portions of your private key when using certain uncommon elliptic curves (binary curves). This version now includes additional security checks to prevent this attack. This issue only affects binary elliptic curves, which are rarely used in real-world applications. Credit to **XlabAI Team of Tencent Xuanwu Lab and Atuin Automated Vulnerability Discovery Engine** for reporting the issue. **CVE-2026-26007** * Support for ``SECT*`` binary elliptic curves is deprecated and will be removed in the next release. <p>.. v46-0-4:</p> <p>46.0.4 - 2026-01-27<br /> </code></pre></p> <ul> <li><code>Dropped support for win_arm64 wheels</code>_.</li> <li>Updated Windows, macOS, and Linux wheels to be compiled with OpenSSL 3.5.5.</li> </ul> <p>.. _v46-0-3:</p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/pyca/cryptography/commit/06e120e682cb200e3f7050c02f0bcdac90c4c6ad"><code>06e120e</code></a> bump version for 46.0.5 release (<a href="https://redirect.github.com/pyca/cryptography/issues/14289">#14289</a>)</li> <li><a href="https://github.com/pyca/cryptography/commit/0eebb9dbb6343d9bc1d91e5a2482ed4e054a6d8c"><code>0eebb9d</code></a> EC check key on cofactor > 1 (<a href="https://redirect.github.com/pyca/cryptography/issues/14287">#14287</a>)</li> <li><a href="https://github.com/pyca/cryptography/commit/bedf6e186b814f69a3f54f51252c23a71d44ed2e"><code>bedf6e1</code></a> fix openssl version on 46 branch (<a href="https://redirect.github.com/pyca/cryptography/issues/14220">#14220</a>)</li> <li><a href="https://github.com/pyca/cryptography/commit/e6f44fc8e6391f05d719fb9d369692325b87a471"><code>e6f44fc</code></a> bump for 46.0.4 and drop win arm64 due to CI issues (<a href="https://redirect.github.com/pyca/cryptography/issues/14217">#14217</a>)</li> <li>See full diff in <a href="https://github.com/pyca/cryptography/compare/46.0.3...46.0.5">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) You can disable automated security fix PRs for this repo from the [Security Alerts page](https://github.com/apache/iceberg-python/network/alerts). </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
<!--
Thanks for opening a pull request!
-->
<!-- In the case this PR will resolve an issue, please replace
${GITHUB_ISSUE_ID} below with the actual Github issue id. -->
<!-- Closes #${GITHUB_ISSUE_ID} -->
# Rationale for this change
I've added a list of catalog-level properties to help the catalog tests.
The goal is that the catalog tests work off of "features" and don't have
exceptions for various catalogs.
As part of this, I discovered a discrepancy where HiveCatalog throws a
different error than everyone else. I'm happy to change it back.
## Are these changes tested?
Mostly a test change.
## Are there any user-facing changes?
<!-- In the case of user-facing changes, please add the changelog label.
-->
Closes apache#1820 # Rationale for this change Apache Iceberg v3 introduces native `geometry` and `geography` primitive types. This PR adds spec-compliant support for those types in PyIceberg, including: - New primitive types with parameter handling and format-version enforcement - Schema parsing and round-trip serialization - Avro mapping using WKB bytes - PyArrow / Parquet integration with version-aware fallbacks - Unit test coverage for type behavior and compatibility constraints A full design and scope discussion is available in the accompanying RFC: 📄 **RFC: [Iceberg v3 Geospatial Primitive Types](apache#3004 The RFC documents scope, non-goals, compatibility constraints, and known limitations. # Are these changes tested? Yes. - Unit tests cover: - Type creation (default and custom parameters) - `__str__` / `__repr__` round-tripping - Equality, hashing, and pickling - Format version enforcement (v3-only) - PyArrow-dependent behavior is version-gated and conditionally tested # Are there any user-facing changes? Yes. - Users may now declare `geometry` and `geography` columns in Iceberg v3 schemas - Parquet files written with PyArrow ≥ 21 preserve GEO logical types when available - Limitations (e.g. no WKB↔WKT conversion, no spatial predicates) are documented <!-- Changelog label added -->
this is the guidance from apache infra, see https://s.apache.org/max-parallel * All workflows MUST have a job concurrency level less than or equal to 20. This means a workflow cannot have more than 20 jobs running at the same time across all matrices. * All workflows SHOULD have a job concurrency level less than or equal to 15. Just because 20 is the max, doesn't mean you should strive for 20. note that `max-parallel` limits the number of simultaneous runs ([see docs](https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/run-job-variations#defining-the-maximum-number-of-concurrent-jobs)), other jobs will just wait --------- Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com> Co-authored-by: kevinjqliu <9057843+kevinjqliu@users.noreply.github.com>
Bumps [jupyterlab](https://github.com/jupyterlab/jupyterlab) from 4.5.3 to 4.5.4. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/jupyterlab/jupyterlab/releases">jupyterlab's releases</a>.</em></p> <blockquote> <h2>v4.5.4</h2> <h2>4.5.4</h2> <p>(<a href="https://github.com/jupyterlab/jupyterlab/compare/v4.5.3...19f81fcfb33889b59260da8b5aa9221379f10d11">Full Changelog</a>)</p> <h3>Bugs fixed</h3> <ul> <li>Avoid using system clipboard in Notebook widget <a href="https://redirect.github.com/jupyterlab/jupyterlab/pull/18474">#18474</a> (<a href="https://github.com/brichet"><code>@brichet</code></a>)</li> <li>Update CodeMirror versions, fixing a few selection issues <a href="https://redirect.github.com/jupyterlab/jupyterlab/pull/18466">#18466</a> (<a href="https://github.com/krassowski"><code>@krassowski</code></a>)</li> <li>Fix debugger variable panel to render value 0 properly <a href="https://redirect.github.com/jupyterlab/jupyterlab/pull/18464">#18464</a> (<a href="https://github.com/itsmevichu"><code>@itsmevichu</code></a>)</li> <li>Consider both the content type and pattern match in <code>DocumentRegistry.getFileTypeForModel()</code> <a href="https://redirect.github.com/jupyterlab/jupyterlab/pull/18409">#18409</a> (<a href="https://github.com/krassowski"><code>@krassowski</code></a>)</li> <li>false in saveAs to avoid fetching file body <a href="https://redirect.github.com/jupyterlab/jupyterlab/pull/18379">#18379</a> (<a href="https://github.com/DeborahOlaboye"><code>@DeborahOlaboye</code></a>)</li> </ul> <h3>Maintenance and upkeep improvements</h3> <ul> <li>Bump Lumino packages to the 2026.2.5 release, fixing iframe resizing <a href="https://redirect.github.com/jupyterlab/jupyterlab/pull/18440">#18440</a> (<a href="https://github.com/jasongrout"><code>@jasongrout</code></a>)</li> </ul> <h3>Documentation improvements</h3> <ul> <li>Clarify notebook and cell metadata API changes in JupyterLab 4 <a href="https://redirect.github.com/jupyterlab/jupyterlab/pull/18304">#18304</a> (<a href="https://github.com/Krish-876"><code>@Krish-876</code></a>)</li> <li>Document a change in the Code Editor source updates via <code>sharedModel</code> in JupyterLab 4 <a href="https://redirect.github.com/jupyterlab/jupyterlab/pull/18260">#18260</a> (<a href="https://github.com/Krish-876"><code>@Krish-876</code></a>)</li> <li>Fix Contents API fetch documentation of the <code>content</code> default <a href="https://redirect.github.com/jupyterlab/jupyterlab/pull/18383">#18383</a> (<a href="https://github.com/ceasermikes002"><code>@ceasermikes002</code></a>)</li> <li>Add AI rules <a href="https://redirect.github.com/jupyterlab/jupyterlab/pull/18322">#18322</a> (<a href="https://github.com/jtpio"><code>@jtpio</code></a>)</li> </ul> <h3>Contributors to this release</h3> <p>The following people contributed discussions, new ideas, code and documentation contributions, and review. See <a href="https://github-activity.readthedocs.io/en/latest/use/#how-does-this-tool-define-contributions-in-the-reports">our definition of contributors</a>.</p> <p>(<a href="https://github.com/jupyterlab/jupyterlab/graphs/contributors?from=2026-01-23&to=2026-02-11&type=c">GitHub contributors page for this release</a>)</p> <p><a href="https://github.com/brichet"><code>@brichet</code></a> (<a href="https://github.com/search?q=repo%3Ajupyterlab%2Fjupyterlab+involves%3Abrichet+updated%3A2026-01-23..2026-02-11&type=Issues">activity</a>) | <a href="https://github.com/ceasermikes002"><code>@ceasermikes002</code></a> (<a href="https://github.com/search?q=repo%3Ajupyterlab%2Fjupyterlab+involves%3Aceasermikes002+updated%3A2026-01-23..2026-02-11&type=Issues">activity</a>) | <a href="https://github.com/DeborahOlaboye"><code>@DeborahOlaboye</code></a> (<a href="https://github.com/search?q=repo%3Ajupyterlab%2Fjupyterlab+involves%3ADeborahOlaboye+updated%3A2026-01-23..2026-02-11&type=Issues">activity</a>) | <a href="https://github.com/itsmevichu"><code>@itsmevichu</code></a> (<a href="https://github.com/search?q=repo%3Ajupyterlab%2Fjupyterlab+involves%3Aitsmevichu+updated%3A2026-01-23..2026-02-11&type=Issues">activity</a>) | <a href="https://github.com/jasongrout"><code>@jasongrout</code></a> (<a href="https://github.com/search?q=repo%3Ajupyterlab%2Fjupyterlab+involves%3Ajasongrout+updated%3A2026-01-23..2026-02-11&type=Issues">activity</a>) | <a href="https://github.com/jtpio"><code>@jtpio</code></a> (<a href="https://github.com/search?q=repo%3Ajupyterlab%2Fjupyterlab+involves%3Ajtpio+updated%3A2026-01-23..2026-02-11&type=Issues">activity</a>) | <a href="https://github.com/krassowski"><code>@krassowski</code></a> (<a href="https://github.com/search?q=repo%3Ajupyterlab%2Fjupyterlab+involves%3Akrassowski+updated%3A2026-01-23..2026-02-11&type=Issues">activity</a>) | <a href="https://github.com/Krish-876"><code>@Krish-876</code></a> (<a href="https://github.com/search?q=repo%3Ajupyterlab%2Fjupyterlab+involves%3AKrish-876+updated%3A2026-01-23..2026-02-11&type=Issues">activity</a>)</p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/jupyterlab/jupyterlab/commit/2ecc1eed9944d9ad080f8c3f8206285338065af4"><code>2ecc1ee</code></a> [ci skip] Publish 4.5.4</li> <li><a href="https://github.com/jupyterlab/jupyterlab/commit/19f81fcfb33889b59260da8b5aa9221379f10d11"><code>19f81fc</code></a> Backport PR <a href="https://redirect.github.com/jupyterlab/jupyterlab/issues/18474">#18474</a> on branch 4.5.x (Avoid using system clipboard in Notebook ...</li> <li><a href="https://github.com/jupyterlab/jupyterlab/commit/b82e6d75338a27b218298e254b6d31e1d2bd2daf"><code>b82e6d7</code></a> Backport PR <a href="https://redirect.github.com/jupyterlab/jupyterlab/issues/18466">#18466</a> on branch 4.5.x (Update CodeMirror versions, fixing a few ...</li> <li><a href="https://github.com/jupyterlab/jupyterlab/commit/c70c36e3c5ece87be55a836f45a6965e75a3d84c"><code>c70c36e</code></a> Backport PR <a href="https://redirect.github.com/jupyterlab/jupyterlab/issues/18440">#18440</a> on branch 4.5.x (Bump Lumino packages to the 2026.2.5 rele...</li> <li><a href="https://github.com/jupyterlab/jupyterlab/commit/23b9ff52b4d97f33c1fce10089842f51787489ff"><code>23b9ff5</code></a> Backport PR <a href="https://redirect.github.com/jupyterlab/jupyterlab/issues/18464">#18464</a> on branch 4.5.x (Fix debugger variable panel to render val...</li> <li><a href="https://github.com/jupyterlab/jupyterlab/commit/553d941f6161e20eed5ddffa85a0da0538e05996"><code>553d941</code></a> Backport PR <a href="https://redirect.github.com/jupyterlab/jupyterlab/issues/18409">#18409</a> on branch 4.5.x (Consider both the content type and patter...</li> <li><a href="https://github.com/jupyterlab/jupyterlab/commit/d8acd39e47e71a1732bbca4757355f66b0f3197b"><code>d8acd39</code></a> Backport PR <a href="https://redirect.github.com/jupyterlab/jupyterlab/issues/18304">#18304</a> on branch 4.5.x (Clarify notebook and cell metadata API ch...</li> <li><a href="https://github.com/jupyterlab/jupyterlab/commit/e0219696094436eab6a62e4bf5784437bcf0e5a4"><code>e021969</code></a> Backport PR <a href="https://redirect.github.com/jupyterlab/jupyterlab/issues/18260">#18260</a> on branch 4.5.x (Document a change in the Code Editor sour...</li> <li><a href="https://github.com/jupyterlab/jupyterlab/commit/dc39ba5e79190d8721932fee6ece0a329655c667"><code>dc39ba5</code></a> Backport PR <a href="https://redirect.github.com/jupyterlab/jupyterlab/issues/18383">#18383</a> on branch 4.5.x (Fix Contents API fetch documentation of t...</li> <li><a href="https://github.com/jupyterlab/jupyterlab/commit/fd57db92797b12ac0787347a4aa464e71ef38ddc"><code>fd57db9</code></a> Backport PR <a href="https://redirect.github.com/jupyterlab/jupyterlab/issues/18322">#18322</a>: Add AI rules (<a href="https://redirect.github.com/jupyterlab/jupyterlab/issues/18395">#18395</a>)</li> <li>Additional commits viewable in <a href="https://github.com/jupyterlab/jupyterlab/compare/@jupyterlab/lsp@4.5.3...@jupyterlab/lsp@4.5.4">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
# Rationale for this change
This pr is a small change to correct the `python-release-docs.yaml`
python version to reference a python version. Today it uses an non
existing variable of ${{ matrix.python }.
This will evaluate to an empty sting and just use the runners system
python install which happens to work. So this change will hardcode to
use 3.12 to match `python-ci-docs.yml`.
Action warning
Link: https://github.com/apache/iceberg-python/actions/runs/22287174360
<img width="687" height="113" alt="image"
src="https://github.com/user-attachments/assets/f4e89403-20b1-4451-b220-3cfb7b84e394"
/>
## Are these changes tested?
No
## Are there any user-facing changes?
No
ubuntu-slim is a new cost-efficient runner to fit lightweight jobs. We can use this to save ASF infra usage (if possible). By *lightweight* I mean jobs that run for less than a minute and are not easily affected by other concurrently running jobs on the same host. This topic was raised in the Apache ORC project and later adopted by iceberg-cpp. We believe it could also benefit other projects under the Apache Iceberg umbrella. Refs: - https://github.blog/changelog/2025-10-28-1-vcpu-linux-runner-now-available-in-github-actions-in-public-preview/ - https://github.com/actions/runner-images/blob/main/images/ubuntu-slim/ubuntu-slim-Readme.md
Closes apache#3066 # Rationale for this change This PR addes a small remark in the documentation to `s3.profile-name` and `client.profile-name`, since they are currently only supported when using `FsspecFileIO`. Support for AWS profile for `PyArrowFileIO` is not yet implemented, see apache#3065 ## Are these changes tested? n/a, purely changes to documentation. ## Are there any user-facing changes? Yes, minor change log adjustment needed compared to apache#2948. - `client.profile-name`: Sets the AWS profile for both the catalog client and FileIO (unified configuration). (only supported by `FsspecFileIO` currently) - `s3.profile-name`: Sets the AWS profile specifically for S3 FileIO. (only supported by `FsspecFileIO` currently) <!-- In the case of user-facing changes, please add the changelog label. --> --------- Co-authored-by: Thomas Pfeiffer (ext.) <thomas.tp.pfeiffer@partner.bmwgroup.com>
Bring Iceberg Java's stale.yml workflow over to handle stale PRs in addition to the existing stale issues behavior. Borrowed from the Iceberg rust implementation: apache/iceberg-rust#2171 # Rationale for this change The iceberg-python PR list is grown over the years and needs clean up ## Are these changes tested? Existing CI enhancement ## Are there any user-facing changes? No
) <!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> # Rationale for this change Vended credentials don't work with REST catalogs following latests specs. Per [latest spec](https://github.com/apache/iceberg/blob/main/open-api/rest-catalog-open-api.yaml#L3485): > Clients must first check whether the respective credentials exist in the `storage-credentials` field before checking the `config` for credentials. This is missing in pyiceberg. REST catalogs that don't expose a config field are hence not compatible. ## Are these changes tested? Yes, ran locally against my REST catalog implementation, added tests. ## Are there any user-facing changes? No API changes. REST catalogs that return vended credentials via storage-credentials will now work correctly. <!-- In the case of user-facing changes, please add the changelog label. --> --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
<!-- Closes apache#3008 --> # Rationale for this change When REST catalog uses SigV4 signing, retries were not configured on the mounted adapter. This change adds explicit retry defaults for signed REST requests and makes retry count configurable. Changes: - Added `rest.sigv4.max-retries` catalog property. - Default retries set to 10 attempts. - Retries configured for idempotent methods (`GET`, `HEAD`, `OPTIONS`) and common throttling/transient codes (`429`, `500`, `502`, `503`, `504`). ## Are these changes tested? Yes. - Added `test_sigv4_adapter_default_retry_config`. - Added `test_sigv4_adapter_override_retry_config`. - Verified with targeted pytest run. ## Are there any user-facing changes? Yes. - New optional REST catalog property: `rest.sigv4.max-retries`. - Improved resilience for throttling/transient HTTP failures on SigV4-enabled REST catalogs. --------- Co-authored-by: Mrutunjay Kinagi <you@example.com> Co-authored-by: Kevin Liu <kevinjqliu@users.noreply.github.com> Co-authored-by: Kevin Liu <kevin.jq.liu@gmail.com>
…he#2154) <!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> <!-- Closes #${GITHUB_ISSUE_ID} --> Part of apache#818 This adds `create_view` to the REST Catalog. As part of this, it also defines the View and ViewMetadata class. # Rationale for this change PyIceberg's REST Catalog doesn't support `create_view`. Furthermore, PyIceberg doesn't support views at all. This is the first part in getting views supported. # Are these changes tested? Unit tests included # Are there any user-facing changes? Added Iceberg REST Catalog support for `create_view` <!-- In the case of user-facing changes, please add the changelog label. --> --------- Co-authored-by: Sung Yun <107272191+sungwy@users.noreply.github.com> Co-authored-by: Fokko Driesprong <fokko@apache.org>
kevinjqliu
force-pushed
the
kevinjqliu/iceberg-rust-rc
branch
from
4769c82 to
ad47b55
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
16 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Rationale for this change
Are these changes tested?
Are there any user-facing changes?