You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/src/catalog/dir/index.md
+2-2Lines changed: 2 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -112,13 +112,13 @@ The `__manifest` table has the following schema:
112
112
113
113
**Primary Key**: The `object_id` column is the [unenforced primary key](https://lance.org/format/table/#unenforced-primary-key) for the manifest table. Implementation of this spec must always enforce the primary key uniqueness using features like Lance merge insert with primary key deduplication.
114
114
115
-
**Schema Extensibility**: The `__manifest` table schema may include additional columns beyond those listed above. Extensions like [partitioned namespaces](../partitioning-spec.md) add columns for efficient filtering. Implementations should preserve unrecognized columns during updates.
115
+
**Schema Extensibility**: The `__manifest` table schema may include additional columns beyond those listed above. Implementations should preserve unrecognized columns during updates, since extensions may add columns for filtering or other metadata-driven behaviors.
116
116
117
117
### Root Namespace Properties
118
118
119
119
In V2, the root namespace is implicit and does not have a row in the `__manifest` table. Instead, root namespace properties are stored in the `__manifest` Lance table's metadata map. Properties are stored as key-value pairs where the key is the property name and the value is a UTF-8 encoded byte array.
120
120
121
-
For example, a partitioned namespace stores its `partition_spec_v1`, `partition_spec_v2`, and `schema` properties in the `__manifest` table's metadata.
121
+
For example, implementations may store catalog-level properties in the `__manifest` table's metadata.
Copy file name to clipboardExpand all lines: docs/src/namespace/supported-catalogs/lance-dir.md
+6-6Lines changed: 6 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ This document describes how the Lance Directory Catalog implements the Lance Nam
4
4
5
5
## Background
6
6
7
-
The Lance Directory Catalog is a storage-native catalog that stores tables in a directory structure on any local or remote storage system. For details on the catalog design including V1 (directory listing), V2 (manifest), and compatibility mode, see the [Directory Catalog Format Specification](../catalog/dir/index.md).
7
+
The Lance Directory Catalog is a storage-native catalog that stores tables in a directory structure on any local or remote storage system. For details on the catalog design including V1 (directory listing), V2 (manifest), and compatibility mode, see the [Directory Catalog Format Specification](../../catalog/dir/index.md).
8
8
9
9
## Implementation Configuration Properties
10
10
@@ -16,7 +16,7 @@ The **manifest_enabled** property controls whether the manifest table is used fo
16
16
17
17
The **dir_listing_enabled** property controls whether directory scanning is used for table discovery (V1). Defaults to `true`.
18
18
19
-
By default, both properties are enabled, which means the implementation operates in [Compatibility Mode](../catalog/dir/index.md#compatibility-mode).
19
+
By default, both properties are enabled, which means the implementation operates in [Compatibility Mode](../../catalog/dir/index.md#compatibility-mode).
20
20
21
21
Properties with the **storage.** prefix are passed directly to the underlying Lance ObjectStore after removing the prefix. For example, `storage.region` becomes `region` when passed to the storage layer.
22
22
@@ -173,7 +173,7 @@ In **V2**:
173
173
3. Further filter to rows where `object_id` has exactly one more level than the namespace
174
174
4. Return the list of table names
175
175
176
-
When **both V1 and V2 are enabled** (the default [Compatibility Mode](../catalog/dir/index.md#compatibility-mode)),
176
+
When **both V1 and V2 are enabled** (the default [Compatibility Mode](../../catalog/dir/index.md#compatibility-mode)),
177
177
the implementation performs both queries and merges results, with manifest entries taking precedence when duplicates exist.
178
178
179
179
**Error Handling:**
@@ -189,7 +189,7 @@ The implementation:
189
189
1. Locate the table:
190
190
- In V1, check for the `<table_name>.lance` directory
191
191
- In V2, query the manifest table for the table location
192
-
- When both V1 and V2 are enabled (the default [Compatibility Mode](../catalog/dir/index.md#compatibility-mode)),
192
+
- When both V1 and V2 are enabled (the default [Compatibility Mode](../../catalog/dir/index.md#compatibility-mode)),
193
193
first check the manifest table, then fall back to checking the `.lance` directory
194
194
2. Open the Lance table using the Lance SDK
195
195
3. Read the table metadata and return:
@@ -231,7 +231,7 @@ In **V2**:
231
231
3. Keep the table files at the storage location
232
232
4. Return the table location and properties for reference
233
233
234
-
When **both V1 and V2 are enabled** (the default [Compatibility Mode](../catalog/dir/index.md#compatibility-mode)),
234
+
When **both V1 and V2 are enabled** (the default [Compatibility Mode](../../catalog/dir/index.md#compatibility-mode)),
235
235
first check the manifest table, then fall back to checking the `.lance` directory.
236
236
If found in manifest, follow V2 behavior; otherwise follow V1 behavior.
237
237
@@ -260,7 +260,7 @@ In **V2**:
260
260
3. Delete the table directory and all its contents from storage
261
261
(failure here does not affect the success of the drop since the table is no longer reachable)
262
262
263
-
When **both V1 and V2 are enabled** (the default [Compatibility Mode](../catalog/dir/index.md#compatibility-mode)),
263
+
When **both V1 and V2 are enabled** (the default [Compatibility Mode](../../catalog/dir/index.md#compatibility-mode)),
264
264
first check the manifest table, then fall back to checking the `.lance` directory.
265
265
If found in manifest, follow V2 behavior; otherwise follow V1 behavior.
Copy file name to clipboardExpand all lines: docs/src/namespace/supported-catalogs/lance-rest.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -4,7 +4,7 @@ This document describes how the Lance REST Catalog implements the Lance Namespac
4
4
5
5
## Background
6
6
7
-
The Lance REST Catalog provides access to Lance tables via a REST API. For details on the API design, endpoints, and data models, see the [REST Catalog API Specification](../catalog/rest/index.md).
7
+
The Lance REST Catalog provides access to Lance tables via a REST API. For details on the API design, endpoints, and data models, see the [REST Catalog API Specification](../../catalog/rest/index.md).
0 commit comments