Skip to content

Commit 3a3bc62

Browse files
MonkeyCanCodeebyhr
andauthored
Add more examples for pyiceberg view (#3414)
<!-- Thanks for opening a pull request! --> <!-- In the case this PR will resolve an issue, please replace ${GITHUB_ISSUE_ID} below with the actual Github issue id. --> Closes #3413 # Rationale for this change Adds more examples around view support. ## Are these changes tested? Yes. Those had been tested locally with local Polaris. ## Are there any user-facing changes? No. <!-- In the case of user-facing changes, please add the changelog label. --> --------- Co-authored-by: Yuya Ebihara <ebyhry@gmail.com>
1 parent aa39484 commit 3a3bc62

2 files changed

Lines changed: 100 additions & 3 deletions

File tree

mkdocs/docs/api.md

Lines changed: 99 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1529,15 +1529,69 @@ cleanup_old_snapshots("analytics.user_events", [12345, 67890, 11111])
15291529

15301530
## Views
15311531

1532-
PyIceberg supports view operations.
1532+
If PyIceberg is unable to automatically determine view support on your REST Catalog, you can manually specify, `"view-endpoints-supported": "true"`:
15331533

1534-
### Check if a view exists
1534+
```python
1535+
from pyiceberg.catalog import load_catalog
1536+
1537+
catalog = load_catalog(
1538+
"docs",
1539+
**{
1540+
"uri": "http://127.0.0.1:8181",
1541+
"py-io-impl": "pyiceberg.io.pyarrow.PyArrowFileIO",
1542+
"s3.endpoint": "http://127.0.0.1:9000",
1543+
"s3.access-key-id": "admin",
1544+
"s3.secret-access-key": "password",
1545+
"view-endpoints-supported": "true",
1546+
}
1547+
)
1548+
```
1549+
1550+
## Create a view
1551+
1552+
To create a view from the catalog:
15351553

15361554
```python
15371555
from pyiceberg.catalog import load_catalog
1556+
from pyiceberg.schema import Schema
1557+
from pyiceberg.types import IntegerType, NestedField
1558+
from pyiceberg.view import SQLViewRepresentation, ViewVersion
15381559

15391560
catalog = load_catalog("default")
1540-
catalog.view_exists("default.bar")
1561+
1562+
schema = Schema(NestedField(field_id=1, name="some_col", field_type=IntegerType(), required=False))
1563+
view_version = ViewVersion(
1564+
schema_id=1,
1565+
summary={"engine-name": "pyiceberg", "engine-version": "0.11.1"},
1566+
representations=[
1567+
SQLViewRepresentation(
1568+
type="sql",
1569+
sql="SELECT 1 as some_col",
1570+
dialect="spark",
1571+
)
1572+
],
1573+
default_namespace=["default"],
1574+
)
1575+
1576+
catalog.create_view(
1577+
identifier="default.some_view",
1578+
schema=schema,
1579+
view_version=view_version,
1580+
)
1581+
```
1582+
1583+
`catalog.create_view` also accepts a PyArrow schema, so the following is equivalent:
1584+
1585+
```python
1586+
import pyarrow as pa
1587+
1588+
schema = pa.schema([pa.field("some_col", pa.int32())])
1589+
1590+
catalog.create_view(
1591+
identifier="default.some_view",
1592+
schema=schema,
1593+
view_version=view_version,
1594+
)
15411595
```
15421596

15431597
## Register a view
@@ -1551,6 +1605,48 @@ catalog.register_view(
15511605
)
15521606
```
15531607

1608+
## Load a view
1609+
1610+
Loading the `some_view` view:
1611+
1612+
```python
1613+
view = catalog.load_view("default.some_view")
1614+
# Equivalent to:
1615+
view = catalog.load_view(("default", "some_view"))
1616+
# The tuple syntax can be used if the namespace or view contains a dot.
1617+
```
1618+
1619+
This returns a `View` that represents an Iceberg view. You can access the SQL representation for a specific dialect:
1620+
1621+
```python
1622+
sql_representation = view.sql_for("spark")
1623+
print(sql_representation.sql)
1624+
```
1625+
1626+
## Check if a view exists
1627+
1628+
To check whether the `some_view` view exists:
1629+
1630+
```python
1631+
catalog.view_exists("default.some_view")
1632+
```
1633+
1634+
## List views
1635+
1636+
To list views in the `default` namespace:
1637+
1638+
```python
1639+
catalog.list_views("default")
1640+
```
1641+
1642+
## Drop a view
1643+
1644+
To drop a view:
1645+
1646+
```python
1647+
catalog.drop_view("default.some_view")
1648+
```
1649+
15541650
## Table Statistics Management
15551651

15561652
Manage table statistics with operations through the `Table` API:

mkdocs/docs/configuration.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -347,6 +347,7 @@ catalog:
347347
| warehouse | myWarehouse | Warehouse location or identifier to request from the catalog service. May be used to determine server-side overrides, such as the warehouse location. |
348348
| snapshot-loading-mode | refs | The snapshots to return in the body of the metadata. Setting the value to `all` would return the full set of snapshots currently valid for the table. Setting the value to `refs` would load all snapshots referenced by branches or tags. |
349349
| `header.X-Iceberg-Access-Delegation` | `vended-credentials` | Signal to the server that the client supports delegated access via a comma-separated list of access mechanisms. The server may choose to supply access via any or none of the requested mechanisms. When using `vended-credentials`, the server provides temporary credentials to the client. When using `remote-signing`, the server signs requests on behalf of the client. (default: `vended-credentials`) |
350+
| view-endpoints-supported | false | For backwards compatibility with older REST servers. Set to `true` if the server supports view endpoints but doesn't send the `endpoints` field in the ConfigResponse. |
350351

351352
#### Headers in REST Catalog
352353

0 commit comments

Comments
 (0)