Conversation
dicts.jl no longer reaches into liblmdb directly: keys/values/collect/ list_dirs/valuesize switch from LMDBIterator to a private _walk_prefix helper built on `walk` + `seek_range!`/`next!`, and haskey/get/get! use `tryget` and `get(...,default)` instead of `unchecked_mdb_get`. `get!`-with-default now writes inside the same write txn that observed the missing key, instead of opening a separate write txn after a read txn — a small atomicity improvement; the previous form raced against concurrent writers. Also: `walk` now stops if its callback returns `false` (any other return — including `nothing` — continues), which is what makes the prefix-scan loop in `_walk_prefix` work without dropping back to tier-1 cursor ops. `grep '\bmdb_' src/dicts.jl` is empty after this commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
LMDBDict now subtypes AbstractDict{K,V}, so merge!/filter!/pairs/==/
hash/lazy keys/lazy values come from the standard library. The
hand-rolled keys/values/collect overloads with a `prefix` kwarg are
gone; `keys(d)` and `values(d)` are the lazy `KeySet`/`ValueIterator`
the AbstractDict contract promises.
Iteration follows LLVM.jl's `BasicBlockInstructionSet` pattern: the
state is `(txn, cur)`, opened on the first `iterate` call (analogous
to LLVM's `LLVMGetFirstInstruction` seed) and advanced with
`MDB_NEXT` each step. Normal completion commits the txn and closes
the cursor; early `break`/throw is reclaimed by the Step-9 finalizers.
Other AbstractDict-shaped changes:
- `getindex(d, k)` throws `KeyError(k)` on miss (was `LMDBError`).
- `pop!(d, k)` throws `KeyError`; `pop!(d, k, default)` returns the
default; both are atomic in one write txn.
- `length(d)` via `mdb_stat`; `isempty`, `empty!` follow.
- `delete!(d, k)` no longer throws on missing keys (Base contract).
Prefix-scoped scans move to LMDB-namespaced helpers — `LMDB.scan(d)`,
`LMDB.scan_keys(d)`, `LMDB.scan_values(d)` — alongside `list_dirs`
and `valuesize`.
Defaults:
- `MDB_NOTLS` is now set on the LMDBDict env, matching py-lmdb. Without
it, a `length(d)` call mid-iteration tripped MDB_BAD_RSLOT.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…sor.
cur.jl loses ~100 lines of legacy iterator scaffolding that nothing in
the new API exercises:
- LMDBIterator{R} + ReturnKeys/ReturnValues/ReturnBoth/ReturnValueSize
- DirectoryLister + process_returns + init_values + arcopy
- keys(cur, T; prefix) / Base.values(cur, T; prefix) / a broken
iterate(cur, K, V) that referenced an undefined `prefix` variable
- get(cur, key, T, op) — superseded by seek!+value at the right op
`mbd_unpack` was always a typo for `mdb_unpack`; renamed in src and tests.
Cursor now stores its parent DBI alongside its parent Transaction;
database(cur) returns it directly instead of round-tripping through
`mdb_cursor_dbi` and synthesizing a fresh wrapper with an empty name.
Adds the AbstractDict gap that Base.Dict supplies but
Base.AbstractDict does not — `pop!(d::LMDBDict)` (no key form) pops
the lexicographically-first entry, throws ArgumentError if empty.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
AbstractDict's default merge! / mergewith! / filter! call d[k]=v or delete!(d, k) in a loop, and each one of those opens its own LMDB write txn (with its own commit/fsync). For a 1k-entry merge! that's 1k transactions, which dominates wall time on durable storage. Override the three to land in one txn instead.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #54 +/- ##
==========================================
+ Coverage 80.59% 83.40% +2.80%
==========================================
Files 9 9
Lines 701 717 +16
==========================================
+ Hits 565 598 +33
+ Misses 136 119 -17 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Iterator behavior inspired by LLVM.jl.