Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,9 @@
[![Dev](https://img.shields.io/badge/docs-dev-blue.svg)](https://JuliaDatabases.github.io/LMDB.jl/dev)

Julia bindings for [LMDB](http://www.lmdb.tech/doc/), the Lightning
Memory-Mapped Database: An embedded, memory-mapped, ACID key-value store
developed by Symas for OpenLDAP. Small, fast, persisted to disk, and reads at
near in-memory speeds.
Memory-Mapped Database. LMDB is an embedded, memory-mapped, ACID key-value
store developed by Symas for OpenLDAP. It persists to disk while reading
at near in-memory speeds.

```julia
using Pkg; Pkg.add("LMDB")
Expand All @@ -18,15 +18,15 @@ using Pkg; Pkg.add("LMDB")

LMDB.jl exposes the same database through three surfaces:

- **High-level interface**: `LMDBDict <: AbstractDict`, an
- High-level interface: `LMDBDict <: AbstractDict`, an
`AbstractDict{K,V}` over a single LMDB file. Standard library
machinery (`merge!`, `filter!`, `pairs`, iteration, …) works out
of the box. Reach for this when you want a persistent `Dict`.
- **Julia wrappers**: `Environment`, `Transaction`, `DBI`, `Cursor`.
- Julia wrappers: `Environment`, `Transaction`, `DBI`, `Cursor`.
Julia-shaped wrappers around handles, transactions, and cursors,
with finalizers, `do`-block forms, etc. Use this when you want
explicit transactions.
- **C API**: `LMDB.mdb_*` and `LMDB.MDB_*`. Raw `ccall` bindings and
with finalizers, `do`-block forms, and so on. Use these when you
want explicit transactions.
- C API: `LMDB.mdb_*` and `LMDB.MDB_*`. Raw `ccall` bindings and
status-code constants. Use this when the Julia wrappers don't expose
a particular API or you want to inspect status codes directly.

Expand Down Expand Up @@ -86,8 +86,8 @@ end

The package decodes `String`, `Vector{T}` for any bitstype `T`, and the
primitive numeric types out of the box. To plug in a custom representation,
define a `Base.read(io::IO, ::Type{T})` method; it will be picked up by `tryget`
/ `get` / `walk(f, cur, K, V)` and the cursor accessors `key`/`value`/`item`.
define a `Base.read(io::IO, ::Type{T})` method; it will be picked up by `tryget`,
`get`, `walk(f, cur, K, V)`, and the cursor accessors `key`/`value`/`item`.
Status-code matchers live on `LMDBError`:

```julia
Expand Down
13 changes: 6 additions & 7 deletions docs/src/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,9 +4,8 @@
Memory-Mapped Database.*

LMDB is an embedded, memory-mapped, ACID key-value store developed by
Symas for OpenLDAP. It is small, fast, and persists to disk while reading
at near in-memory speeds — limited only by the size of the virtual address
space.
Symas for OpenLDAP. It persists to disk while reading at near in-memory
speeds, limited only by the size of the virtual address space.

```julia
using Pkg; Pkg.add("LMDB")
Expand All @@ -18,16 +17,16 @@ LMDB.jl exposes the same database through three surfaces:

| Surface | What it offers | When to use |
|---------|----------------|-------------|
| **High-level interface** | `LMDBDict <: AbstractDict{K,V}` | When you want a persistent `Dict`. |
| **Julia wrappers** | `Environment`, `Transaction`, `DBI`, `Cursor` | When you want explicit transactions and cursors with Julia-shaped wrappers. |
| **C API** | `LMDB.mdb_*`, `LMDB.MDB_*` | Raw `ccall` bindings and status-code constants, for custom data layouts or when you want to skip allocations on hot paths. |
| High-level interface | `LMDBDict <: AbstractDict{K,V}` | When you want a persistent `Dict`. |
| Julia wrappers | `Environment`, `Transaction`, `DBI`, `Cursor` | When you want explicit transactions and cursors with Julia-shaped wrappers. |
| C API | `LMDB.mdb_*`, `LMDB.MDB_*` | Raw `ccall` bindings and status-code constants, for custom data layouts or when you want to skip allocations on hot paths. |

`MDBValue`, `MDBArg`, and the [`MDBValueIO`](@ref LMDB.MDBValueIO)
type sit between the C API and the Julia wrappers. `MDBValueIO` is an
`IO` view over `MDB_val`; defining `Base.read(io, T)` on it is how you
teach the typed reads about a custom value type.

The Usage section starts simple and gets more involved: [Essentials](@ref)
The Usage section starts simple and gets more involved. [Essentials](@ref)
has a working example, [Dictionary interface](@ref) covers `LMDBDict`,
and [Environments](@ref), [Transactions](@ref), [Databases](@ref),
[Cursors](@ref), and [Duplicate-sort databases](@ref) cover the wrappers.
Expand Down
2 changes: 1 addition & 1 deletion docs/src/lib/cursors.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ CurrentModule = LMDB
```

A `Cursor` is a positioned iterator over the entries in a `DBI`. Cursors
are bound to a transaction; closing the txn invalidates the cursor.
are bound to a transaction. Closing the txn invalidates the cursor.

## Construction

Expand Down
2 changes: 1 addition & 1 deletion docs/src/lib/databases.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ tryget
```

`get(txn, dbi, key, T, default)` falls back to `default` if `key` is
missing — same shape as `Base.get(dict, key, default)`.
missing, matching `Base.get(dict, key, default)`.

## Writes

Expand Down
8 changes: 4 additions & 4 deletions docs/src/lib/dict.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,17 @@ Writes (`setindex!`, `delete!`, `pop!`, `empty!`) likewise. `delete!`
silently no-ops on a missing key, matching `Base.delete!`'s "if any"
contract.

Everything `AbstractDict` derives `merge!`, `merge`, `mergewith!`,
Everything `AbstractDict` derives (`merge!`, `merge`, `mergewith!`,
`filter!`, `filter`, `==`, `isequal`, `hash`, `in(::Pair, d)`,
`copy(d)` comes along for free.
`copy(d)`) comes along for free.

`LMDBDict` iterates in lexicographic key order, which is stronger than
`LMDBDict` iterates in lexicographic key order, which is stricter than
`Base.Dict`'s no-order promise.

## Lifecycle

`close(::LMDBDict)` closes the underlying env (and the default DBI).
Idempotent also called from the finalizer.
Idempotent, and also called from the finalizer.

## Prefix-scan helpers

Expand Down
6 changes: 3 additions & 3 deletions docs/src/lib/errors.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,8 @@ CurrentModule = LMDB

Every LMDB-internal error surfaces as an `LMDBError` whose `code` field is
the raw status `Cint` returned by the underlying binding. Status-code
matchers cover the three most common branches; less common codes can be
matched against `LMDB.MDB_*` constants directly.
matchers cover the most common branches; less common codes can be matched
against `LMDB.MDB_*` constants directly.

## Exception type

Expand Down Expand Up @@ -54,7 +54,7 @@ branch on `MDB_NOTFOUND`/`MDB_KEYEXIST` and friends.

At the Julia wrappers, handle methods that wrap status-returning
bindings let `LMDBError` propagate. `tryget` and `get(..., default)`
swallow `MDB_NOTFOUND` and return `nothing`/`default`;
swallow `MDB_NOTFOUND` and return `nothing` or `default`.
`delete!(txn, dbi, key)` likewise swallows `MDB_NOTFOUND` and returns
`false`.

Expand Down
10 changes: 5 additions & 5 deletions docs/src/lib/lowlevel.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ for callers that need to inspect the raw status code.

Every status-returning binding in `liblmdb.jl` is paired with an
`unchecked_*` companion at definition time. Use the bare name when any
error should propagate (the common case); use `unchecked_*` when you
error should propagate (the common case). Use `unchecked_*` when you
need to inspect the raw `Cint` yourself, for example to distinguish
`MDB_NOTFOUND` from a real error:

Expand All @@ -34,15 +34,15 @@ Bindings that return non-status data (`mdb_strerror`, `mdb_version`,
`mdb_txn_id`, `mdb_cmp`, `mdb_dcmp`, `mdb_env_get_maxkeysize`,
`mdb_env_get_userctx`, `mdb_cursor_txn`, `mdb_cursor_dbi`) and
`Cvoid`-returning ones (`mdb_env_close`, `mdb_dbi_close`,
`mdb_txn_abort`, `mdb_txn_reset`, `mdb_cursor_close`) are left bare;
there is nothing to check.
`mdb_txn_abort`, `mdb_txn_reset`, `mdb_cursor_close`) are left bare,
since there is nothing to check.

## Customisation point: `MDBValueIO`

`tryget` / `get` / `key` / `value` / `item` / typed `walk` / `pop!` /
`tryget`, `get`, `key`, `value`, `item`, typed `walk`, `pop!`, and
`replace!` all go through `read(::MDBValueIO, T)` to decode an
`MDB_val` into a Julia value. Define a `Base.read` method on
`MDBValueIO` to plug in a custom representation; see [Cursors](@ref)
`MDBValueIO` to plug in a custom representation. See [Cursors](@ref)
for a worked example.

```@docs
Expand Down
34 changes: 17 additions & 17 deletions docs/src/man/cursors.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,8 @@ CurrentModule = LMDB
```

A `Cursor` is a positioned iterator over a `DBI`. Use it for ordered
scans, range queries, or when you want to amortise the per-lookup
overhead of `mdb_get` across many keys.
scans, range queries, or to amortise the per-lookup overhead of
`mdb_get` across many keys.

## Opening a cursor

Expand All @@ -21,19 +21,19 @@ end
```

A cursor is bound to its transaction; closing the txn invalidates the
cursor. The cursor's finalizer is idempotent, so a still-open cursor
is reclaimed when GC visits it.
cursor. The cursor's finalizer is idempotent, so a still-open cursor is
reclaimed when GC visits it.

## Navigation

Each navigation function repositions the cursor and returns the new
key, or `nothing` if the move would step past the end:

```julia
seek!(cur) # MDB_FIRST first entry
seek_last!(cur) # MDB_LAST last entry
seek!(cur, key) # MDB_SET_KEY exact key match
seek_range!(cur, key) # MDB_SET_RANGE smallest key ≥ `key`
seek!(cur) # MDB_FIRST first entry
seek_last!(cur) # MDB_LAST last entry
seek!(cur, key) # MDB_SET_KEY exact key match
seek_range!(cur, key) # MDB_SET_RANGE smallest key ≥ `key`
next!(cur) # MDB_NEXT
prev!(cur) # MDB_PREV
```
Expand Down Expand Up @@ -80,24 +80,24 @@ start(env; flags = MDB_RDONLY) do txn
end
```

For the same pattern *one level up* (already wrapped, returns a
For the same pattern one level up (already wrapped, returns a
`Vector{Pair}`), use [`LMDB.scan(d; prefix)`](@ref LMDB.scan) on an
`LMDBDict`.

## [Bulk walk zero-copy iteration](@id man-cur-walk)
## [Bulk walk: zero-copy iteration](@id man-cur-walk)

`walk` runs a callback over every entry the cursor visits. It exists in
two shapes:

```julia
# Untyped receives Ref{MDB_val} pairs (zero-copy, mmap pointers)
# Untyped: receives Ref{MDB_val} pairs (zero-copy, mmap pointers)
walk(cur) do k_ref, v_ref
kv = k_ref[]; vv = v_ref[]
# kv.mv_data / vv.mv_data are mmap pointers, valid in this scope
do_something(kv.mv_size, vv.mv_size)
end

# Typed runs each ref through `read(MDBValueIO, K)` / `read(MDBValueIO, V)`
# Typed: runs each ref through `read(MDBValueIO, K)` / `read(MDBValueIO, V)`
walk(cur, String, Vector{UInt8}) do k::String, v::Vector{UInt8}
println(k, " => ", length(v), " bytes")
end
Expand All @@ -110,12 +110,12 @@ The callback can return `false` to stop iteration; any other return
(including `nothing`) continues.

Use the untyped form when you want to inspect raw byte sizes, copy
slices, or feed a custom decoder; the data pointers are into LMDB's
slices, or feed a custom decoder. The data pointers are into LMDB's
mmap and are valid only inside the callback (and only for the
surrounding txn). The typed form is the iteration analogue of
`tryget(..., T)` and works for any `T` for which `Base.read(io::IO,
::Type{T})` (or `Base.read(io::LMDB.MDBValueIO, ::Type{T})`) is
defined; see [Custom value decoding](@ref).
defined. See [Custom value decoding](@ref).

## Cursor mutation

Expand All @@ -134,14 +134,14 @@ key (1 in non-DUPSORT databases).

## Custom value decoding

`tryget` / `get` / `key` / `value` / `item` / typed `walk` all funnel
`tryget`, `get`, `key`, `value`, `item`, and typed `walk` all funnel
through `Base.read(io::IO, ::Type{T})` against an
[`MDBValueIO`](@ref LMDB.MDBValueIO). The defaults cover Base's
primitive numeric types (`Int8`/…/`Float64`, `Bool`, `Char`, `Ptr`),
`String`, and (added by this package) `Vector{E}` for any bitstype `E`.

For everything else, including `isbitstype` structs and framed
values, define a single `Base.read` method on the abstract `IO`:
values, define a single `Base.read` method on the abstract `IO`.

```julia
struct PrefixedBlob end
Expand All @@ -165,7 +165,7 @@ end
decoders end up reading like any other Julia binary parser, and the
same decoder works against any byte source. This is the analogue of
heed's `BytesDecode<'txn>` trait, expressed through Julia's existing IO
extension point rather than a bespoke trait.
extension point instead of a bespoke trait.

## Reset and renew

Expand Down
16 changes: 8 additions & 8 deletions docs/src/man/databases.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ open(txn, "users") do dbi
end
```

In practice you'll rarely *want* to close a DBI handle explicitly: the
In practice you'll rarely *want* to close a DBI handle explicitly. The
env owns it, and `mdb_dbi_close` is documented as rarely useful. The
env's finalizer cascades through any open DBI handles.

Expand Down Expand Up @@ -53,8 +53,8 @@ get(txn, dbi, key, T, default) # default on miss
```

`T` is anything `read(::LMDB.MDBValueIO, ::Type{T})` knows how to
decode `String`, `Vector{E}` for any bitstype `E`, or any bitstype
scalar:
decode: `String`, `Vector{E}` for any bitstype `E`, or any bitstype
scalar.

```julia
tryget(txn, dbi, "name", String) # → Union{String, Nothing}
Expand Down Expand Up @@ -82,7 +82,7 @@ Useful write flags:
|------|---------|
| `MDB_NOOVERWRITE` | fail with `MDB_KEYEXIST` if `key` is already present |
| `MDB_NODUPDATA` | (DUPSORT) fail if the `(key, val)` pair already exists |
| `MDB_APPEND` | append; only valid if the new key sorts after every existing key — *much* faster for sorted bulk loads |
| `MDB_APPEND` | append; only valid if the new key sorts after every existing key. Much faster for sorted bulk loads |

```julia
# Bulk import in sorted order:
Expand All @@ -98,7 +98,7 @@ end
`replace!` and `pop!` do the read-modify pair inside the same
transaction, so there is no time-of-check / time-of-use gap.

## `put_reserved!` write directly into the mmap
## `put_reserved!`: write directly into the mmap

When the value is large or assembled from multiple sources, you can
skip the intermediate `Vector{UInt8}` round-trip and write straight
Expand All @@ -115,8 +115,8 @@ end
inside the callback, and only inside the surrounding write txn; don't
escape it.

`put_reserved!` is the equivalent of heed's `Database::put_reserved`,
and is incompatible with DUPSORT.
`put_reserved!` is the equivalent of heed's `Database::put_reserved`.
It is incompatible with DUPSORT.

## Stats

Expand All @@ -137,4 +137,4 @@ drop(txn, dbi; delete = true) # delete the DB and close the handle

For named sub-DBs, `delete = true` removes the entry from the env's
main DB. For the main DB itself, `delete = true` is treated as
`delete = false` (LMDB cannot delete its own root).
`delete = false`, since LMDB cannot delete its own root.
10 changes: 5 additions & 5 deletions docs/src/man/dict.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ tasks.
## Storing and retrieving

Anything that round-trips through the package's `MDB_val` glue and
[`MDBValueIO`](@ref LMDB.MDBValueIO) works as a value type `String`,
[`MDBValueIO`](@ref LMDB.MDBValueIO) works as a value type: `String`,
`Vector{T}` for any bitstype `T`, and any bitstype scalar (`Int`,
`Float32`, `(Int, UInt32)` `Tuple`, …).

Expand Down Expand Up @@ -66,7 +66,7 @@ for (k, v) in d
end
```

Iteration is in **lexicographic key order** — strictly stronger than
Iteration is in lexicographic key order, which is stricter than
`Base.Dict`'s no-order promise. Each `for` loop opens a fresh read
transaction; the txn is committed on normal exit and aborted (via the
`Transaction` finalizer) on early break or throw.
Expand Down Expand Up @@ -97,7 +97,7 @@ filter!(((k, v),) -> v > 0, d)
## Prefix-scoped scans

For hierarchical key schemes (e.g. `"users/123/name"`), LMDB's
lexicographic order makes prefix scans cheap — they're a single
lexicographic order makes prefix scans cheap: a single
`MDB_SET_RANGE` plus iteration until the prefix stops matching.

```julia
Expand All @@ -119,15 +119,15 @@ LMDB.scan(d, prefix = "users/2/")
# "users/2/name" => "Bob"
```

For directory-style listingsleaf keys appear as-is, anything with
For directory-style listings, leaf keys appear as-is and anything with
the separator after the prefix collapses to its first segment:

```julia
LMDB.list_dirs(d, prefix = "") # ["other", "users/"]
LMDB.list_dirs(d, prefix = "users/") # ["users/1/", "users/2/"]
```

`LMDB.valuesize(d; prefix)` sums byte sizes — useful for quick storage
`LMDB.valuesize(d; prefix)` sums byte sizes. Useful for quick storage
audits without `stat`.

## When to drop down
Expand Down
Loading
Loading