Skip to content

Commit 177052d

Browse files
prrao87claude
prrao87
and
claude
authored
docs: add dedicated Branches guide, promotion via merge_insert, and SDK bumps (#297)
Split branching out of the versioning guide into a standalone, snippet-driven Branches page. It covers the full lifecycle (create, write, reopen, delete), promoting branch data back to main with merge_insert (there is no automatic merge yet), and building vector/FTS indexes on a branch in isolation. Adds OSS and Enterprise connection sections reused from the quickstart snippets. Bump test SDK pins to versions with branch support and regenerate snippets: Python lancedb 0.34.0, TypeScript and Rust 0.31.0. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
1 parent f609d88 commit 177052d

13 files changed

Lines changed: 1500 additions & 298 deletions

File tree

‎docs/docs.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -85,6 +85,7 @@
8585
"tables/schema",
8686
"tables/update",
8787
"tables/versioning",
88+
"tables/branching",
8889
"tables/consistency"
8990
]
9091
},

‎docs/snippets/tables.mdx‎

Lines changed: 33 additions & 3 deletions
Large diffs are not rendered by default.

‎docs/tables/branching.mdx‎

Lines changed: 282 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,282 @@
1+
---
2+
title: "Branches"
3+
sidebarTitle: "Branches"
4+
description: "Fork isolated, writable lines of table history in LanceDB. Run experiments, backfills, and index rebuilds without disturbing production reads on main."
5+
icon: "code-branch"
6+
---
7+
import {
8+
PyConnect,
9+
TsConnect,
10+
RsConnect,
11+
PyConnectEnterpriseQuickstart,
12+
TsConnectEnterpriseQuickstart,
13+
RsConnectEnterpriseQuickstart,
14+
} from '/snippets/connection.mdx';
15+
import {
16+
PyBranchCreate as BranchCreate,
17+
TsBranchCreate,
18+
RsBranchCreate,
19+
PyBranchWrite as BranchWrite,
20+
TsBranchWrite,
21+
RsBranchWrite,
22+
PyBranchReopen as BranchReopen,
23+
TsBranchReopen,
24+
RsBranchReopen,
25+
PyBranchDelete as BranchDelete,
26+
TsBranchDelete,
27+
RsBranchDelete,
28+
PyBranchPromote as BranchPromote,
29+
TsBranchPromote,
30+
RsBranchPromote,
31+
PyBranchIndex as BranchIndex,
32+
TsBranchIndex,
33+
RsBranchIndex,
34+
} from '/snippets/tables.mdx';
35+
36+
A branch is an isolated, writable line of history forked from `main` (or from any
37+
other branch). Anything you do on a branch (e.g., adding rows, changing the schema,
38+
building an index) stays on that branch, so `main` keeps serving production
39+
reads exactly as before. Branches are a natural fit when you want to:
40+
41+
- Experiment with a new index, schema change, or reprocessing step without
42+
affecting live queries on `main`.
43+
- Run a backfill or migration you'd like to review before promoting it.
44+
- Hand a collaborator a frozen point-in-time fork while you keep writing to `main`.
45+
46+
## How branches relate to versions and tags
47+
48+
Every LanceDB table already tracks a linear history of [versions](/tables/versioning),
49+
and you can [tag](/tables/versioning#tag-based-versioning) a version or `checkout`
50+
one to read it. Branches add the missing piece: a *separate, writable* line of
51+
history. Where a tag is a read-only label and `checkout` is a read-only view, a
52+
branch forks from a point in history and then evolves on its own. Creating or
53+
checking out a branch hands you a new table handle whose reads and writes are
54+
scoped to that branch. The [comparison table](#branches-vs-tags-vs-checkout) at
55+
the end of this page lays out when to reach for each.
56+
57+
<Note>
58+
Branches are supported on local and namespace-backed tables in LanceDB OSS, as
59+
well as on LanceDB Enterprise (remote) tables.
60+
</Note>
61+
62+
## Connect to a table
63+
64+
The branch API is identical no matter how you connect — only the connection
65+
itself differs between OSS and Enterprise. Establish a connection (`db`) and open
66+
a `table` (see [Create a table](/tables/create)), then use the same branch calls
67+
in every example that follows.
68+
69+
### LanceDB OSS
70+
71+
Point LanceDB at a local directory (or an object-storage URI) to use it as an
72+
embedded library.
73+
74+
<CodeGroup>
75+
<CodeBlock filename="Python" language="Python" icon="python">
76+
{PyConnect}
77+
</CodeBlock>
78+
79+
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
80+
{TsConnect}
81+
</CodeBlock>
82+
83+
<CodeBlock filename="Rust" language="Rust" icon="rust">
84+
{ "use lancedb::connect;\n\n" }
85+
{RsConnect}
86+
</CodeBlock>
87+
</CodeGroup>
88+
89+
### LanceDB Enterprise
90+
91+
<Badge color="red">Enterprise</Badge>
92+
93+
Branching on LanceDB Enterprise works the same way as on OSS, but you connect to your
94+
Enterprise deployment with a `db://` URI, an API key, and your
95+
region. Once you have a connection, open a table and use the same branch calls in
96+
every example that follows.
97+
98+
<CodeGroup>
99+
<CodeBlock filename="Python" language="Python" icon="python">
100+
{PyConnectEnterpriseQuickstart}
101+
</CodeBlock>
102+
103+
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
104+
{TsConnectEnterpriseQuickstart}
105+
</CodeBlock>
106+
107+
<CodeBlock filename="Rust" language="Rust" icon="rust">
108+
{RsConnectEnterpriseQuickstart}
109+
</CodeBlock>
110+
</CodeGroup>
111+
112+
## Work with branches
113+
114+
The lifecycle of a branch is short and predictable: fork it, write to it, reopen
115+
it whenever you need it, and delete it once you're done. The examples below use a
116+
small `quotes` table with three rows on `main`.
117+
118+
### Create a branch
119+
120+
Forking from `main` returns a table handle scoped to the new branch. `main` is
121+
the reserved default source, so `create` needs only a name; to fork from
122+
somewhere else, pass a branch name, a specific version, or both.
123+
124+
<CodeGroup>
125+
<CodeBlock filename="Python" language="Python" icon="python">
126+
{BranchCreate}
127+
</CodeBlock>
128+
129+
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
130+
{TsBranchCreate}
131+
</CodeBlock>
132+
133+
<CodeBlock filename="Rust" language="Rust" icon="rust">
134+
{RsBranchCreate}
135+
</CodeBlock>
136+
</CodeGroup>
137+
138+
### Write to a branch
139+
140+
Writes go through the branch handle and stay there — the `main` handle keeps
141+
reporting its original row count. Listing branches returns a mapping of each
142+
branch name to its metadata, including the version it was forked from.
143+
144+
<CodeGroup>
145+
<CodeBlock filename="Python" language="Python" icon="python">
146+
{BranchWrite}
147+
</CodeBlock>
148+
149+
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
150+
{TsBranchWrite}
151+
</CodeBlock>
152+
153+
<CodeBlock filename="Rust" language="Rust" icon="rust">
154+
{RsBranchWrite}
155+
</CodeBlock>
156+
</CodeGroup>
157+
158+
### Reopen a branch
159+
160+
A branch outlives the handle that created it. Reopen it later by name — either
161+
from an existing table handle or straight from the connection when you open the
162+
table. Both routes give you a writable handle tracking the branch's latest state.
163+
164+
<CodeGroup>
165+
<CodeBlock filename="Python" language="Python" icon="python">
166+
{BranchReopen}
167+
</CodeBlock>
168+
169+
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
170+
{TsBranchReopen}
171+
</CodeBlock>
172+
173+
<CodeBlock filename="Rust" language="Rust" icon="rust">
174+
{RsBranchReopen}
175+
</CodeBlock>
176+
</CodeGroup>
177+
178+
### Delete a branch
179+
180+
Deleting a branch removes it and its branch-local history; `main` is untouched.
181+
Delete a branch only once you've promoted anything worth keeping — see
182+
[Merging a branch into `main`](#merging-a-branch-into-main) below.
183+
184+
<CodeGroup>
185+
<CodeBlock filename="Python" language="Python" icon="python">
186+
{BranchDelete}
187+
</CodeBlock>
188+
189+
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
190+
{TsBranchDelete}
191+
</CodeBlock>
192+
193+
<CodeBlock filename="Rust" language="Rust" icon="rust">
194+
{RsBranchDelete}
195+
</CodeBlock>
196+
</CodeGroup>
197+
198+
## Merge a branch into `main`
199+
200+
<Warning>
201+
LanceDB doesn't yet support an automatic merge of a branch into `main`, with full conflict-handling.
202+
There's no single `merge()` call that reconciles the two histories for you.
203+
</Warning>
204+
205+
Until full merge support is available, you can promote a branch the same way any other data lands in a table: by
206+
running an ingestion workload against `main`. The only real question is how each
207+
branch row lines up with the row it should replace on `main`. This is exactly
208+
what [`merge_insert`](/tables/update#merge-incoming-rows-by-key) handles. Keyed on
209+
a unique column, it updates rows that already exist and inserts the ones that
210+
don't (an *upsert*), leaving everything else on `main` untouched.
211+
212+
Read the rows you want out of the branch, then upsert them into `main` on your
213+
key column — `id` in this example:
214+
215+
<CodeGroup>
216+
<CodeBlock filename="Python" language="Python" icon="python">
217+
{BranchPromote}
218+
</CodeBlock>
219+
220+
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
221+
{TsBranchPromote}
222+
</CodeBlock>
223+
224+
<CodeBlock filename="Rust" language="Rust" icon="rust">
225+
{RsBranchPromote}
226+
</CodeBlock>
227+
</CodeGroup>
228+
229+
Since this is just a keyed write against `main`, it looks like a normal
230+
ingestion job: point the job's source at the branch and its destination at
231+
`main`. Reading the whole branch and upserting it is the simplest approach and is
232+
safe to re-run, because `merge_insert` is idempotent on the key. To promote only
233+
what changed, filter the branch read down to the rows you touched before the
234+
upsert.
235+
236+
## Build indexes on a branch
237+
238+
One of the most useful things a branch buys you is a safe place to build and
239+
validate an index without affecting what's in production on the `main` branch.
240+
Fork a branch, create your vector (ANN) and full-text search (FTS)
241+
indexes on it, check recall and latency, and only then promote the change to
242+
`main`. Because the indexes live on the branch, queries against `main` never see
243+
a half-built index and are never slowed down by the build.
244+
245+
This pattern is especially valuable on <Badge color="red">Enterprise</Badge>
246+
deployments, where `main` is typically serving production traffic while you tune
247+
an index configuration on the side.
248+
249+
<CodeGroup>
250+
<CodeBlock filename="Python" language="Python" icon="python">
251+
{BranchIndex}
252+
</CodeBlock>
253+
254+
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
255+
{TsBranchIndex}
256+
</CodeBlock>
257+
258+
<CodeBlock filename="Rust" language="Rust" icon="rust">
259+
{RsBranchIndex}
260+
</CodeBlock>
261+
</CodeGroup>
262+
263+
Schema changes such as adding, altering, or dropping columns are branch-scoped in
264+
the same way, so you can stage a larger reshaping of a table and review it before
265+
it ever reaches `main`.
266+
267+
## Branches vs. tags vs. versions
268+
269+
Now that you're familiar with branches, you can see how they complement the other ways LanceDB
270+
give you to work with table history. Choose these approaches based on whether you need to
271+
*label*, *read*, or *write* data at a point in history:
272+
273+
| Feature | Writable? | Purpose |
274+
| --- | --- | --- |
275+
| **[Branch](#work-with-branches)** | ✅ Yes | Write on top of a point in history without touching `main`. |
276+
| **[Tag](/tables/versioning#tag-based-versioning)** | ❌ No | Attach a human-readable label to an existing version; protects it from cleanup. |
277+
| **[`checkout(version)`](/tables/versioning#rollback-to-previous-versions)** | ❌ No | Read a historical version of `main` without forking. Read-only until you `restore`. |
278+
279+
<Note>
280+
For linear version history — creating versions, listing them, rolling back, and
281+
tagging — see the [Versioning and Reproducibility](/tables/versioning) guide.
282+
</Note>

‎docs/tables/versioning.mdx‎

Lines changed: 6 additions & 62 deletions
Original file line numberDiff line numberDiff line change
@@ -35,9 +35,6 @@ import {
3535
PyVersioningTags as VersioningTags,
3636
TsVersioningTags as TsVersioningTags,
3737
RsVersioningTags as RsVersioningTags,
38-
PyBranches as Branches,
39-
TsBranches as TsBranches,
40-
RsBranches as RsBranches,
4138
RsVersioningMakeQuotesReader as RsVersioningMakeQuotesReader,
4239
} from '/snippets/tables.mdx';
4340

@@ -223,66 +220,13 @@ deletion, the underlying table version becomes eligible for cleanup again.
223220

224221
## Branches
225222

226-
Branches are isolated, writable lines of history forked from another branch (or
227-
a specific version). Writes to a branch do **not** affect `main`, which makes
228-
branches ideal for:
223+
Beyond linear history, LanceDB also supports **branches** — isolated, writable
224+
lines of history forked from `main` (or a specific version). Whereas tags and
225+
`checkout` give you read-only views of existing versions, a branch has its own
226+
writable history, making it ideal for experiments, backfills, and migrations
227+
that you want to keep separate from production reads on `main`.
229228

230-
- Running experiments (new indexes, schema changes, reprocessing) without
231-
disturbing production reads on `main`.
232-
- Backfills or migrations that you want to review before promoting.
233-
- Sharing a frozen point-in-time fork with a collaborator while continuing to
234-
write to `main`.
235-
236-
Unlike tags, which are read-only labels on existing versions, a branch has its
237-
own writable history. Creating or checking out a branch returns a new table
238-
handle whose reads and writes are scoped to that branch.
239-
240-
<Note>
241-
Branches are currently supported on local and namespace-backed tables in
242-
LanceDB OSS. Branch support for LanceDB Enterprise (remote) tables is not yet
243-
available.
244-
</Note>
245-
246-
### Create, Write, and Reopen a Branch
247-
248-
Create a branch from `main` (the default source) and write to it. The base table
249-
continues to point at `main` and is unaffected by writes on the branch. You can
250-
also reopen an existing branch either from a table handle or directly from the
251-
database connection when opening the table.
252-
253-
<CodeGroup>
254-
<CodeBlock filename="Python" language="Python" icon="python">
255-
{Branches}
256-
</CodeBlock>
257-
258-
<CodeBlock filename="TypeScript" language="TypeScript" icon="square-js">
259-
{TsBranches}
260-
</CodeBlock>
261-
262-
<CodeBlock filename="Rust" language="Rust" icon="rust">
263-
{RsBranches}
264-
</CodeBlock>
265-
</CodeGroup>
266-
267-
`list` returns a mapping of branch names to branch metadata, including which
268-
branch each was forked from. `main` is the reserved default branch: wherever a
269-
branch is optional, leaving it out means `main`. To fork from somewhere else,
270-
pass a branch name, a specific version, or both when creating the branch.
271-
272-
Deleting a branch removes that branch and its branch-local history; data on
273-
`main` is not affected. There is no automatic merge operation. To promote
274-
changes, copy the desired data back explicitly.
275-
276-
<Note>
277-
**Branches vs. tags vs. checkout**
278-
279-
- Use a **tag** to bookmark an existing version with a human-readable name. Tags
280-
are read-only and protect that version from cleanup.
281-
- Use **`checkout(version)`** to read from a historical version of `main`
282-
without forking. The handle is read-only until you `restore`.
283-
- Use a **branch** when you need to *write* on top of a point in history
284-
without touching `main`.
285-
</Note>
229+
Branches are covered in their own guide: see [Branches](/tables/branching).
286230

287231
## Delete Data From the Table
288232

‎pyproject.toml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description = "Add your description here"
55
readme = "README.md"
66
requires-python = ">=3.12,<3.14"
77
dependencies = [
8-
"lancedb>=0.33.0",
8+
"lancedb>=0.34.0",
99
"pyarrow>=23.0.1",
1010
"lance-namespace>=0.6.1",
1111
"pandas>=3.0.1",

0 commit comments

Comments
 (0)