Skip to content

Commit 4a6cd87

Browse files
committed
Update split catalog namespace docs
1 parent c91adb4 commit 4a6cd87

6 files changed

Lines changed: 20 additions & 12 deletions

File tree

docs/mkdocs.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -68,4 +68,3 @@ extra:
6868

6969
extra_javascript:
7070
- https://unpkg.com/mermaid@10/dist/mermaid.min.js
71-

docs/src/catalog/dir/index.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -112,13 +112,13 @@ The `__manifest` table has the following schema:
112112

113113
**Primary Key**: The `object_id` column is the [unenforced primary key](https://lance.org/format/table/#unenforced-primary-key) for the manifest table. Implementation of this spec must always enforce the primary key uniqueness using features like Lance merge insert with primary key deduplication.
114114

115-
**Schema Extensibility**: The `__manifest` table schema may include additional columns beyond those listed above. Extensions like [partitioned namespaces](../partitioning-spec.md) add columns for efficient filtering. Implementations should preserve unrecognized columns during updates.
115+
**Schema Extensibility**: The `__manifest` table schema may include additional columns beyond those listed above. Implementations should preserve unrecognized columns during updates, since extensions may add columns for filtering or other metadata-driven behaviors.
116116

117117
### Root Namespace Properties
118118

119119
In V2, the root namespace is implicit and does not have a row in the `__manifest` table. Instead, root namespace properties are stored in the `__manifest` Lance table's metadata map. Properties are stored as key-value pairs where the key is the property name and the value is a UTF-8 encoded byte array.
120120

121-
For example, a partitioned namespace stores its `partition_spec_v1`, `partition_spec_v2`, and `schema` properties in the `__manifest` table's metadata.
121+
For example, implementations may store catalog-level properties in the `__manifest` table's metadata.
122122

123123
### Manifest Table Indexes
124124

docs/src/layout.png

-96.8 KB
Binary file not shown.

docs/src/namespace/object-relationship.md

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,16 @@ that frequently appear in other similar data systems to allow easy integration w
1111

1212
Here is an example layout of a namespace:
1313

14-
![Namespace Layout](../layout.png)
14+
```text
15+
Root namespace
16+
├── Namespace "cat2"
17+
│ └── Namespace "cat5"
18+
│ └── Table "t1"
19+
├── Namespace "cat3"
20+
└── Namespace "cat4"
21+
├── Table "t3"
22+
└── Table "t4"
23+
```
1524

1625
## Parent & Child
1726

@@ -83,4 +92,4 @@ A table `[catalog1, database2, table3]` is at level 3, and its identifier `catal
8392
If every table in the root namespace are at the same level `N`, the namespace is called **leveled**,
8493
and we say this namespace is a `N`-level namespace.
8594
For example, a [Directory Catalog](../catalog/dir/index.md) is a 1-level namespace,
86-
and a Hive 2.x namespace is a 2-level namespace.
95+
and a Hive 2.x namespace is a 2-level namespace.

docs/src/namespace/supported-catalogs/lance-dir.md

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ This document describes how the Lance Directory Catalog implements the Lance Nam
44

55
## Background
66

7-
The Lance Directory Catalog is a storage-native catalog that stores tables in a directory structure on any local or remote storage system. For details on the catalog design including V1 (directory listing), V2 (manifest), and compatibility mode, see the [Directory Catalog Format Specification](../catalog/dir/index.md).
7+
The Lance Directory Catalog is a storage-native catalog that stores tables in a directory structure on any local or remote storage system. For details on the catalog design including V1 (directory listing), V2 (manifest), and compatibility mode, see the [Directory Catalog Format Specification](../../catalog/dir/index.md).
88

99
## Implementation Configuration Properties
1010

@@ -16,7 +16,7 @@ The **manifest_enabled** property controls whether the manifest table is used fo
1616

1717
The **dir_listing_enabled** property controls whether directory scanning is used for table discovery (V1). Defaults to `true`.
1818

19-
By default, both properties are enabled, which means the implementation operates in [Compatibility Mode](../catalog/dir/index.md#compatibility-mode).
19+
By default, both properties are enabled, which means the implementation operates in [Compatibility Mode](../../catalog/dir/index.md#compatibility-mode).
2020

2121
Properties with the **storage.** prefix are passed directly to the underlying Lance ObjectStore after removing the prefix. For example, `storage.region` becomes `region` when passed to the storage layer.
2222

@@ -173,7 +173,7 @@ In **V2**:
173173
3. Further filter to rows where `object_id` has exactly one more level than the namespace
174174
4. Return the list of table names
175175

176-
When **both V1 and V2 are enabled** (the default [Compatibility Mode](../catalog/dir/index.md#compatibility-mode)),
176+
When **both V1 and V2 are enabled** (the default [Compatibility Mode](../../catalog/dir/index.md#compatibility-mode)),
177177
the implementation performs both queries and merges results, with manifest entries taking precedence when duplicates exist.
178178

179179
**Error Handling:**
@@ -189,7 +189,7 @@ The implementation:
189189
1. Locate the table:
190190
- In V1, check for the `<table_name>.lance` directory
191191
- In V2, query the manifest table for the table location
192-
- When both V1 and V2 are enabled (the default [Compatibility Mode](../catalog/dir/index.md#compatibility-mode)),
192+
- When both V1 and V2 are enabled (the default [Compatibility Mode](../../catalog/dir/index.md#compatibility-mode)),
193193
first check the manifest table, then fall back to checking the `.lance` directory
194194
2. Open the Lance table using the Lance SDK
195195
3. Read the table metadata and return:
@@ -231,7 +231,7 @@ In **V2**:
231231
3. Keep the table files at the storage location
232232
4. Return the table location and properties for reference
233233

234-
When **both V1 and V2 are enabled** (the default [Compatibility Mode](../catalog/dir/index.md#compatibility-mode)),
234+
When **both V1 and V2 are enabled** (the default [Compatibility Mode](../../catalog/dir/index.md#compatibility-mode)),
235235
first check the manifest table, then fall back to checking the `.lance` directory.
236236
If found in manifest, follow V2 behavior; otherwise follow V1 behavior.
237237

@@ -260,7 +260,7 @@ In **V2**:
260260
3. Delete the table directory and all its contents from storage
261261
(failure here does not affect the success of the drop since the table is no longer reachable)
262262

263-
When **both V1 and V2 are enabled** (the default [Compatibility Mode](../catalog/dir/index.md#compatibility-mode)),
263+
When **both V1 and V2 are enabled** (the default [Compatibility Mode](../../catalog/dir/index.md#compatibility-mode)),
264264
first check the manifest table, then fall back to checking the `.lance` directory.
265265
If found in manifest, follow V2 behavior; otherwise follow V1 behavior.
266266

docs/src/namespace/supported-catalogs/lance-rest.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ This document describes how the Lance REST Catalog implements the Lance Namespac
44

55
## Background
66

7-
The Lance REST Catalog provides access to Lance tables via a REST API. For details on the API design, endpoints, and data models, see the [REST Catalog API Specification](../catalog/rest/index.md).
7+
The Lance REST Catalog provides access to Lance tables via a REST API. For details on the API design, endpoints, and data models, see the [REST Catalog API Specification](../../catalog/rest/index.md).
88

99
## Implementation Configuration Properties
1010

0 commit comments

Comments
 (0)