Skip to content

Commit 9922e6d

Browse files
authored
Docs: Fix the default value of use_caching to false in the rewrite_manifests and add where parameter to rewrite_position_delete_files (#13549)
1 parent 71629c1 commit 9922e6d

1 file changed

Lines changed: 4 additions & 3 deletions

File tree

docs/docs/spark-procedures.md

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -488,7 +488,7 @@ Data files in manifests are sorted by fields in the partition spec. This procedu
488488
| Argument Name | Required? | Type | Description |
489489
|---------------|-----------|------|---------------------------------------------------------------|
490490
| `table` | ✔️ | string | Name of the table to update |
491-
| `use_caching` || boolean | Use Spark caching during operation (defaults to true) |
491+
| `use_caching` || boolean | Use Spark caching during operation (defaults to false). Enabling caching can increase memory footprint on executors. |
492492
| `spec_id` || int | Spec id of the manifests to rewrite (defaults to current spec id) |
493493

494494
#### Output
@@ -505,9 +505,9 @@ Rewrite the manifests in table `db.sample` and align manifest files with table p
505505
CALL catalog_name.system.rewrite_manifests('db.sample');
506506
```
507507

508-
Rewrite the manifests in table `db.sample` and disable the use of Spark caching. This could be done to avoid memory issues on executors.
508+
Rewrite the manifests on the partition spec `1` in table `db.sample`.
509509
```sql
510-
CALL catalog_name.system.rewrite_manifests('db.sample', false);
510+
CALL catalog_name.system.rewrite_manifests(table => 'db.sample', spec_id => 1);
511511
```
512512

513513
### `rewrite_position_delete_files`
@@ -523,6 +523,7 @@ Iceberg can rewrite position delete files, which serves two purposes:
523523
|---------------|-----------|------|----------------------------------|
524524
| `table` | ✔️ | string | Name of the table to update |
525525
| `options` || map<string, string> | Options to be used for procedure |
526+
| `where` || string | predicate as a string used for filtering the files. |
526527

527528
Dangling deletes are always filtered out during rewriting.
528529

0 commit comments

Comments
 (0)