Skip to content

Commit 46602a0

Browse files
yingjianwu98yingjianwu98
authored
Merge branch 'apache:main' into main
2 parents e39bb79 + 5f0f770 commit 46602a0

File tree

23 files changed

+713
-387
lines changed

23 files changed

+713
-387
lines changed

‎.github/ISSUE_TEMPLATE/iceberg_bug_report.yml‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,8 @@ body:
99
description: What Apache Iceberg version are you using?
1010
multiple: false
1111
options:
12-
- "0.7.1 (latest release)"
12+
- "0.8.0 (latest release)"
13+
- "0.7.1"
1314
- "0.7.0"
1415
- "0.6.1"
1516
- "0.6.0"

‎dev/Dockerfile‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,7 @@ WORKDIR ${SPARK_HOME}
3939
ENV SPARK_VERSION=3.5.0
4040
ENV ICEBERG_SPARK_RUNTIME_VERSION=3.5_2.12
4141
ENV ICEBERG_VERSION=1.6.0
42-
ENV PYICEBERG_VERSION=0.7.1
42+
ENV PYICEBERG_VERSION=0.8.0
4343

4444
RUN curl --retry 3 -s -C - https://archive.apache.org/dist/spark/spark-${SPARK_VERSION}/spark-${SPARK_VERSION}-bin-hadoop3.tgz -o spark-${SPARK_VERSION}-bin-hadoop3.tgz \
4545
&& tar xzf spark-${SPARK_VERSION}-bin-hadoop3.tgz --directory /opt/spark --strip-components 1 \

‎mkdocs/docs/configuration.md‎

Lines changed: 16 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -101,18 +101,20 @@ For the FileIO there are several configuration options available:
101101

102102
<!-- markdown-link-check-disable -->
103103

104-
| Key | Example | Description |
105-
| -------------------- | ------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
104+
| Key | Example | Description |
105+
|----------------------|----------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
106106
| s3.endpoint | <https://10.0.19.25/> | Configure an alternative endpoint of the S3 service for the FileIO to access. This could be used to use S3FileIO with any s3-compatible object storage service that has a different endpoint, or access a private S3 endpoint in a virtual private cloud. |
107-
| s3.access-key-id | admin | Configure the static access key id used to access the FileIO. |
108-
| s3.secret-access-key | password | Configure the static secret access key used to access the FileIO. |
109-
| s3.session-token | AQoDYXdzEJr... | Configure the static session token used to access the FileIO. |
110-
| s3.signer | bearer | Configure the signature version of the FileIO. |
107+
| s3.access-key-id | admin | Configure the static access key id used to access the FileIO. |
108+
| s3.secret-access-key | password | Configure the static secret access key used to access the FileIO. |
109+
| s3.session-token | AQoDYXdzEJr... | Configure the static session token used to access the FileIO. |
110+
| s3.role-session-name | session | An optional identifier for the assumed role session. |
111+
| s3.role-arn | arn:aws:... | AWS Role ARN. If provided instead of access_key and secret_key, temporary credentials will be fetched by assuming this role. |
112+
| s3.signer | bearer | Configure the signature version of the FileIO. |
111113
| s3.signer.uri | <http://my.signer:8080/s3> | Configure the remote signing uri if it differs from the catalog uri. Remote signing is only implemented for `FsspecFileIO`. The final request is sent to `<s3.signer.uri>/<s3.signer.endpoint>`. |
112-
| s3.signer.endpoint | v1/main/s3-sign | Configure the remote signing endpoint. Remote signing is only implemented for `FsspecFileIO`. The final request is sent to `<s3.signer.uri>/<s3.signer.endpoint>`. (default : v1/aws/s3/sign). |
113-
| s3.region | us-west-2 | Sets the region of the bucket |
114+
| s3.signer.endpoint | v1/main/s3-sign | Configure the remote signing endpoint. Remote signing is only implemented for `FsspecFileIO`. The final request is sent to `<s3.signer.uri>/<s3.signer.endpoint>`. (default : v1/aws/s3/sign). |
115+
| s3.region | us-west-2 | Sets the region of the bucket |
114116
| s3.proxy-uri | <http://my.proxy.com:8080> | Configure the proxy server to be used by the FileIO. |
115-
| s3.connect-timeout | 60.0 | Configure socket connection timeout, in seconds. |
117+
| s3.connect-timeout | 60.0 | Configure socket connection timeout, in seconds. |
116118

117119
<!-- markdown-link-check-enable-->
118120

@@ -159,7 +161,7 @@ For the FileIO there are several configuration options available:
159161
| gcs.cache-timeout | 60 | Configure the cache expiration time in seconds for object metadata cache |
160162
| gcs.requester-pays | False | Configure whether to use requester-pays requests |
161163
| gcs.session-kwargs | {} | Configure a dict of parameters to pass on to aiohttp.ClientSession; can contain, for example, proxy settings. |
162-
| gcs.endpoint | <http://0.0.0.0:4443> | Configure an alternative endpoint for the GCS FileIO to access (format protocol://host:port) If not given, defaults to the value of environment variable "STORAGE_EMULATOR_HOST"; if that is not set either, will use the standard Google endpoint. |
164+
| gcs.service.host | <http://0.0.0.0:4443> | Configure an alternative endpoint for the GCS FileIO to access (format protocol://host:port) If not given, defaults to the value of environment variable "STORAGE_EMULATOR_HOST"; if that is not set either, will use the standard Google endpoint. |
163165
| gcs.default-location | US | Configure the default location where buckets are created, like 'US' or 'EUROPE-WEST3'. |
164166
| gcs.version-aware | False | Configure whether to support object versioning on the GCS bucket. |
165167

@@ -345,7 +347,7 @@ catalog:
345347
<!-- prettier-ignore-start -->
346348

347349
!!! warning "Removed Properties"
348-
The properties `profile_name`, `region_name`, `botocore_session`, `aws_access_key_id`, `aws_secret_access_key`, and `aws_session_token` were deprecated and removed in 0.8.0
350+
The properties `profile_name`, `region_name`, `aws_access_key_id`, `aws_secret_access_key`, and `aws_session_token` were deprecated and removed in 0.8.0
349351

350352
<!-- prettier-ignore-end -->
351353

@@ -401,7 +403,7 @@ catalog:
401403
<!-- prettier-ignore-start -->
402404

403405
!!! warning "Removed Properties"
404-
The properties `profile_name`, `region_name`, `botocore_session`, `aws_access_key_id`, `aws_secret_access_key`, and `aws_session_token` were deprecated and removed in 0.8.0
406+
The properties `profile_name`, `region_name`, `aws_access_key_id`, `aws_secret_access_key`, and `aws_session_token` were deprecated and removed in 0.8.0
405407

406408
<!-- prettier-ignore-end -->
407409

@@ -438,6 +440,8 @@ configures the AWS credentials for both Glue Catalog and S3 FileIO.
438440
| client.access-key-id | admin | Configure the static access key id used to access both the Glue/DynamoDB Catalog and the S3 FileIO |
439441
| client.secret-access-key | password | Configure the static secret access key used to access both the Glue/DynamoDB Catalog and the S3 FileIO |
440442
| client.session-token | AQoDYXdzEJr... | Configure the static session token used to access both the Glue/DynamoDB Catalog and the S3 FileIO |
443+
| client.role-session-name | session | An optional identifier for the assumed role session. |
444+
| client.role-arn | arn:aws:... | AWS Role ARN. If provided instead of access_key and secret_key, temporary credentials will be fetched by assuming this role. |
441445

442446
<!-- prettier-ignore-start -->
443447

‎mkdocs/docs/how-to-release.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -82,10 +82,10 @@ export LAST_COMMIT_ID=$(git rev-list ${GIT_TAG} 2> /dev/null | head -n 1)
8282
```
8383

8484
The `-s` option will sign the commit. If you don't have a key yet, you can find the instructions [here](http://www.apache.org/dev/openpgp.html#key-gen-generate-key). To install gpg on a M1 based Mac, a couple of additional steps are required: <https://gist.github.com/phortuin/cf24b1cca3258720c71ad42977e1ba57>.
85-
If you have not published your GPG key in [KEYS](https://dist.apache.org/repos/dist/dev/iceberg/KEYS) yet, you must publish it before sending the vote email by doing:
85+
If you have not published your GPG key in [KEYS](https://downloads.apache.org/iceberg/KEYS) yet, you must publish it before sending the vote email by doing:
8686

8787
```bash
88-
svn co https://dist.apache.org/repos/dist/dev/iceberg icebergsvn
88+
svn co https://dist.apache.org/repos/dist/release/iceberg icebergsvn
8989
cd icebergsvn
9090
echo "" >> KEYS # append a newline
9191
gpg --list-sigs <YOUR KEY ID HERE> >> KEYS # append signatures
@@ -166,7 +166,7 @@ The release tarball, signature, and checksums are here:
166166
167167
You can find the KEYS file here:
168168
169-
* https://dist.apache.org/repos/dist/dev/iceberg/KEYS
169+
* https://downloads.apache.org/iceberg/KEYS
170170
171171
Convenience binary artifacts are staged on pypi:
172172

‎mkdocs/docs/verify-release.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ After downloading the source tarball, signature, checksum, and KEYS file, here a
4040
First, import the keys.
4141

4242
```sh
43-
curl https://dist.apache.org/repos/dist/dev/iceberg/KEYS -o KEYS
43+
curl https://downloads.apache.org/iceberg/KEYS -o KEYS
4444
gpg --import KEYS
4545
```
4646

‎mkdocs/requirements.txt‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,11 @@
1818
mkdocs==1.6.1
1919
griffe==1.5.1
2020
jinja2==3.1.4
21-
mkdocstrings==0.26.2
22-
mkdocstrings-python==1.11.1
21+
mkdocstrings==0.27.0
22+
mkdocstrings-python==1.12.2
2323
mkdocs-literate-nav==0.6.1
2424
mkdocs-autorefs==1.2.0
2525
mkdocs-gen-files==0.5.0
26-
mkdocs-material==9.5.43
26+
mkdocs-material==9.5.44
2727
mkdocs-material-extensions==1.3.1
2828
mkdocs-section-index==0.3.9

0 commit comments

Comments
 (0)