Release 4.9.1

Changes from 4.9.0 to 4.9.1

A small hot-fix release for the Arrow interop work in 4.9.0: a real
performance regression in dictionary-column export, and a clearer error
message when opening a nonexistent CTable in append mode.

Improvements

• CTable.iter_arrow_batches() (and therefore to_arrow() and the Arrow
  PyCapsule interchange, __arrow_c_stream__) no longer recomputes the
  full live-row-position array from scratch on every batch, for every
  dictionary column — an O(n_rows) scan that was repeated
  O(n_rows / batch_size) times. The position array is now computed once
  per export call instead. Measured 6-14x faster export for
  dictionary-encoded string columns (e.g. company) on a 1M-row table.
• Reminder for anyone consuming a CTable through the Arrow PyCapsule
  protocol (DuckDB, pyarrow, Polars, pandas): the raw Arrow C Stream
  interface has no column-projection pushdown, so a consumer that only
  needs a few columns still triggers export of every column in the table.
  Use CTable.select([...]) to project down to the columns you actually
  need before handing the table to the consumer, particularly if any
  column is an expensive nested/list type.

Bug fixes

• Opening a CTable with mode="a" at a path that doesn't exist yet now
  raises a clear FileNotFoundError ("mode='a' opens an existing table;
  use mode='w' to create a new one") instead of silently falling through
  and creating a new, empty table.

Release 4.9.0

Changes from 4.8.1 to 4.9.0

This release is about cooperation: CTable now speaks the tabular
ecosystem's own protocols instead of asking it to speak blosc2's. Arrow
tools (pyarrow, DuckDB, Polars, and pandas >= 3.0 via DataFrame.from_arrow())
can consume or produce a CTable directly through the Arrow PyCapsule
interface; a new utf8() string column stores text in Arrow's own
offsets+bytes layout and reads back as NumPy StringDType; and
engine=blosc2.jit now runs correctly inside pandas 3 itself. Alongside
that, CTable gained a proper missing-data story (fillna/dropna,
null-safe arithmetic and comparisons) and a pandas-3-style chaining API
(assign()/col(), UDF aggregations, CTable.apply()).

New features

Ecosystem interop

• Arrow PyCapsule interchange: CTable.__arrow_c_stream__ lets
  pyarrow, DuckDB, and Polars consume a CTable directly as a stream of
  record batches, with bounded memory — no to_arrow()/copy step
  required. pandas >= 3.0 can do the same via the new
  pandas.DataFrame.from_arrow() classmethod (the plain pd.DataFrame(t)
  constructor does not use this protocol). CTable.from_arrow() now
  accepts any object implementing the same protocol on ingest
  (single-argument form), in addition to the existing (schema, batches)
  form — including Arrow's string_view layout (Polars' default string
  export type), which raised TypeError before this release.
• blosc2.utf8(): a new column type for high-cardinality/free-text
  strings, storing each column as two companion NDArrays — int64 row
  offsets plus a UTF-8 byte blob — the same layout Arrow uses for
  large_string. A row costs exactly its encoded byte length (7-13x
  smaller uncompressed than fixed-width string() on high-cardinality
  text), and reads materialize as NumPy StringDType arrays (NumPy >=
  2.0 required; older NumPy falls back to vlstring on Arrow/Parquet
  import with a clear message). Full query surface: comparisons,
  where(), sort_by, group_by keys, fillna, and Arrow export
  (large_string, sentinel-null mask) / import (Arrow/Parquet string
  columns now default to utf8 instead of vlstring). Measured on the
  1e7-row NYC-taxi company column: ingest 3622 ms -> 598.6 ms
  (6.05x faster, only 1.22x slower than string() and 2.63x faster
  than vlstring()), full-column read 2472.6 ms -> 165.3 ms (14.96x
  faster), equality filter ~1900 ms -> 162 ms (~11.7x faster), and
  groupby-key factorization 3304 ms -> 558 ms (2.83x, down from a
  16.9x-slower initial fallback, now faster than fixed-width string()
  keys). See the new "Choosing a string column type" guide in the
  CTable reference and examples/ctable/utf8_strings.py. Known gaps:
  string-expression filters (t.where("name == 'x'")) and
  create_index on utf8 columns still raise a clear
  NotImplementedError; use fixed-width string() if you need those.
• pandas engine, pandas 3 compatible: engine=blosc2.jit for
  DataFrame.apply now returns a properly indexed DataFrame/Series
  under pandas 3.0.3's default raw=False (previously a raw NumPy
  array, so results only matched by value, never by type).
  Series.map(func, engine=blosc2.jit) is now implemented (it
  previously always raised NotImplementedError). Non-numeric columns
  now raise a clear ValueError instead of a deep numexpr error. See
  the guide "Using Blosc2 as a pandas engine" and
  bench/bench_pandas_engine.py.

pandas-style CTable API

• CTable.assign(**named_exprs): return a view with additional computed
  columns, without mutating the table or copying column data. Pairs with
  the new blosc2.col(name) — an unbound column expression that defers
  operator replay until it's bound to a table (assign(), t[...],
  where()) — to write pandas-3-style chains:
  t.assign(profit=col("revenue") - col("cost"))[col("profit") > 0].sort_by("profit", ascending=False).head(10).
• Missing data: Column.fillna(value) replaces sentinel/None
  values for scalar, dictionary, and varlen-scalar columns;
  CTable.dropna(subset=None) returns a view excluding rows where any
  nullable column (or a chosen subset) is null. Column arithmetic
  (+ - * / // % **) and comparisons (< <= > >= == !=) on nullable
  int/timestamp/bool columns now propagate nulls instead of operating on
  the raw sentinel: arithmetic promotes to float64/NaN, comparisons
  follow SQL WHERE semantics (a null operand never satisfies any
  comparison) — fixing filters like t[t.x < 0] wrongly matching null
  rows. Also fixes null detection for timestamp columns
  (is_null()/null_count()/dropna() previously missed every
  NaT).
• UDF aggregations: group_by().agg() accepts a custom callable as
  the op via the named form (output_name=(column, callable[, dtype]));
  it receives each group's live, non-null values as a 1-D NumPy array.
  CTable.apply(func, columns=None, dtype=None, engine="auto") applies
  a UDF across the table's live rows, sugar over blosc2.lazyudf().
  group_by(engine=...) now accepts "auto"/"numpy" explicitly
  alongside the existing default.
• CTable views are now read-only for value writes: Column.__setitem__
  and Column.assign() raise ValueError on a view, pointing at
  take()/copy() as the escape hatch (structural mutations already
  raised; this closes the one unguarded cell-write path).
• NDArray.iter_sorted()/argsort() on a FULL-indexed array now reads
  the sidecar range directly instead of building the full permutation —
  ~52x faster and ~193x less memory on a 20M-element array for
  iter_sorted(start=-k)-style tail queries. See the new optimization
  tip.

Improvements

• String-key group_by() (fixed-width string() keys) now factorizes
  via an exact hash of each row's raw bytes instead of a NumPy
  UTF-32 argsort, with a vectorized collision-checked verify pass
  keeping the result bit-identical: 1157 ms -> 737 ms on a 1e7-row
  benchmark. Every caller benefits automatically; no engine= switch
  involved.
• An example showing ctables and ndarrays bundled together in the same
  TreeStore.
• C-Blosc2 bumped to 3.2.3.

Bug fixes

• Fixed a @blosc2.dsl_kernel-decorated function crashing unconditionally
  when passed as a groupby UDF aggregation (g.agg(name=(col, dsl_kernel_fn))): it now runs like the equivalent undecorated callable.
• Fixed CTable.head()/tail() silently discarding row order when called
  on a lazily-sorted view (e.g. t.sort_by("col", ascending=False) on a
  view, or any .sort_by() result chained off a prior filter): they
  ignored _cached_live_positions and built a plain physical-order mask
  instead, so t.where(...).sort_by("x", ascending=False).head(10) came
  back in the wrong order.
• Fixed a NameError when nan/inf scalars appeared in lazy
  expressions; ShapeInferencer no longer ignores user-provided shapes
  for nan/inf.
• Fixed CTable.from_arrow() raising TypeError: No blosc2 spec for Arrow type DataType(string_view) on any Arrow string_view/binary_view
  column — the layout Polars exports by default through the PyCapsule
  protocol. string_view/binary_view now import exactly like
  string/large_string/binary/large_binary everywhere a column's
  Arrow type is inspected (schema inference, null-sentinel selection,
  list/struct/dictionary value types).

Release 4.8.1

Changes from 4.8.0 to 4.8.1

Improvements

• Read-only memory mapping for CTable stores: CTable.open() (and
  FileTableStorage) gain an mmap_mode="r" parameter, mirroring
  blosc2.open(). All members of a read-only store — scalar, list, varlen
  and dictionary columns alike — are then read from mapped pages; for .b2z
  archives, in place at their offsets inside the single mapped container
  file. With several concurrent readers on one file this pays off quickly:
  2.5x/4.4x/4.5x faster wall time for 1/4/8 readers in our benchmark
  (bench/optim_tips/tip_10_mmap_many_readers.py).
• Reduced memory consumption in CTable.extend() when passed an NDArray.

Bug fixes

• Fixed lifetime/use-after-free hazards around zero-copy cframes and
  vlmeta: schunk_from_cframe()/ndarray_from_cframe() with copy=False
  (the default) returned objects pointing into the caller's bytes buffer
  without keeping it alive, so a temporary cframe (e.g.
  ndarray_from_cframe(response.content)) could be reclaimed under the live
  object, corrupting reads. The buffer is now pinned on the returned object.
  Also, vlmeta read paths (__getitem__/__len__/__iter__) now raise
  ReferenceError on an orphaned owner instead of segfaulting, matching the
  write paths.
• BatchArray.delete() / ObjectArray.delete(): negative-step slices
  (e.g. del arr[3:0:-1]) deleted chunks in ascending order, shifting the
  indices of chunks still to be deleted and removing the wrong ones (or
  raising RuntimeError).
• ListArray.extend_arrow(): Arrow chunks were appended to the backend
  without flushing pending cells first, reordering unflushed rows after the
  new ones.
• Shape inference: stack() with a negative axis inserted the new dimension
  one position too early, so a lazyexpr's reported .shape disagreed with
  its computed result; vecdot() also normalized positive axes as if they
  were negative.
• Chunked matmul(): broadcast (size-1) operand batch dims were sliced with
  the result-chunk coordinates, producing empty slices when the broadcast
  dim spans several result chunks.
• DictStore.__setitem__(): overwrite semantics depended on value size —
  embedded keys refused overwrite ("already exists"), while an
  embedded-to-external overwrite double-stored the key and resurrected the
  stale embedded value after a delete. Assignments now behave uniformly
  dict-like, dropping any previous value.
• Explicitly passing cparams=None or dparams=None to NDArray
  constructors crashed; both now mean "defaults".
• Fixed a builtin-shadowing bug that made store detection in
  blosc2.open() recurse ~250 times (silently swallowed) on every
  .b2z/.b2d open; opening a .b2z is now ~10x faster under allocation
  tracing.

Documentation

• Restructured docs (#674), with a new
  Optimization tips
  section, including new tips on grouping related data into a single
  memory-mapped .b2z file and on using mmap_mode="r" with many
  concurrent readers.

Release 4.8.0

Changes from 4.7.0 to 4.8.0

Sharing containers across processes

• New locking storage parameter (and the BLOSC_LOCKING environment
  variable to enable it fleet-wide) serializes accesses to an on-disk
  SChunk/NDArray/EmbedStore/DictStore against other handles and other
  processes, via a small sidecar lock file (.b2lock). Advisory: every
  handle touching the container must opt in.
• SChunk.holding_lock() / NDArray.holding_lock(): a context manager to
  hold the exclusive lock across several operations, making a multi-step
  mutation atomic to other locked handles.
• New SChunk.refresh(), mirroring the existing NDArray.refresh().
• Fixed a data-loss bug in NDArray.append(): it read the cached,
  unrefreshed shape before computing the resize target, so under
  concurrent growth/shrink — even inside holding_lock() — another writer's
  just-appended data could be silently deleted.
• EmbedStore and DictStore (.b2d) now support cross-process writers
  under locking: transactional writes plus key-map re-sync, so readers
  follow keys added or removed by another process.
• DictStore.to_b2z() (and TreeStore, which inherits from it) now replaces
  the target file atomically, so concurrent readers always see either the old
  or the new archive, never a torn one.
• Growth-SWMR (single writer, multiple readers): a reader NDArray handle
  opened before a resize() made through another handle follows the new
  shape on its next data access, or via the new explicit NDArray.refresh().
• New user guide page,
  Sharing containers across processes,
  covering all of the above plus the caveats (NFS, mmap_mode, Windows
  in-use-file rename).

Bug fixes

• Fixed detect_aligned_chunks() (used internally to fast-path aligned
  slice reads/writes): a floor-division undercounted the chunk grid for
  arrays whose shape isn't a multiple of the chunk shape, which could
  silently return the wrong chunk's data for an otherwise-aligned slice
  with a nonzero start in an earlier dimension.

Others

• Raised the manylinux wheel baseline from manylinux2014 (CentOS 7, glibc
  2.17, GCC 10.2) to manylinux_2_28 (AlmaLinux 8, glibc 2.28, GCC 12),
  fixing a build failure with NumPy >=2.5 which requires GCC >=10.3.

Release 4.6.0

Changes from 4.5.1 to 4.6.0

CTable.sort_by(view=True): zero-copy sorted views

• CTable.sort_by() now accepts view=True, returning a lightweight
  sorted view that shares the parent's column data and gathers rows on
  demand in sorted order — no whole-table copy. This is ideal for reading a
  sorted slice of a large (possibly on-disk) table::

  t.sort_by("col", view=True)[:10]      # top-10 without materialising
  

  Sorting on a fully indexed column streams directly from the index, so the
  table is never materialised. Multi-column sorts and dotted (nested) leaf
  names are supported (e.g. t.sort_by(["trip.begin.lon", "payment.fare"], ascending=[True, False])).

where on dictionary (string) columns

• where expressions now work over dictionary-encoded (string) columns,
  including membership tests such as '"Acme" in company', so categorical
  text columns can be filtered without decoding the whole column.

b2view is now an opt-in extra

• The b2view terminal browser and its TUI stack (textual,
  textual-plotext) are no longer core dependencies: a plain
  pip install blosc2 no longer pulls them, keeping the compression library
  lean (and dropping deps that are unusable under wasm32, which has no TTY).
  Install the viewer with pip install "blosc2[tui]", or
  pip install "blosc2[hires]" to also get the high-res h view. The
  b2view command prints this hint if the dependencies are missing.

group_by: flexible aggregation naming

• CTable.group_by(...).agg() now accepts a list of (column, ops) pairs
  and explicit output names (pandas-style keyword arguments), alongside the
  existing auto-suffixed mapping; the forms can be combined::

  g.agg({"sales": ["sum", "mean"]})              # auto: sales_sum, sales_mean
  g.agg([(t.sales, ["sum", "mean"])])            # auto, but accepts Column objects
  g.agg(revenue=("sales", "sum"))                # explicit: revenue
  g.agg({"sales": "sum"}, n=("*", "size"))       # combined, with a named row count
  

  The list-of-pairs and named forms accept Column objects (t.sales), which
  the mapping form cannot because Column is unhashable and so cannot be a dict
  key.

• Aggregation ops may also be given as the matching blosc2 reduction functions
  (blosc2.sum, mean, min, max, argmin, argmax), matched by
  identity -- e.g. g.agg([(t.sales, [blosc2.sum, "mean"])]). This is a
  naming shorthand only; arbitrary/UDF callables (and look-alikes such as
  np.sum or a user function named sum) are rejected rather than silently
  misinterpreted.

group_by / group_reduce: tri-state sort=

• Vectorized dictionary group ordering: group_by() result building now
  batch-decodes dictionary (string) keys in one pass (decode_batch) instead of
  one decode() per group, making high-cardinality string group-bys dramatically
  faster (end-to-end group_by().size() dropped from seconds to milliseconds on
  ~100k-group workloads).
• sort= is now a tri-state (None / True / False) on both
  CTable.group_by() and blosc2.group_reduce():
  • True — always return groups sorted by key.
  • False — never sort; deterministic but unspecified order.
  • None (the new default) — auto: sort only when cheap. Integer and
    dictionary keys are sorted (free / vectorized); float and multi-key results,
    whose only ordering is an O(G log G) Python sort over every distinct group,
    are left unsorted to avoid a cost that can rival the grouping itself on
    high-cardinality data.
• Behavior changes (the two APIs had different prior defaults, so they move
  in opposite directions):
  • CTable.group_by() previously returned results always sorted. Under the
    new None default, float-key and multi-key group-bys are no longer
    key-sorted by default — pass sort=True to restore sorted output. This is
    a deliberate divergence from pandas (which defaults to sort=True), suited
    to blosc2's large / on-disk datasets.
  • blosc2.group_reduce() previously defaulted to sort=False (unsorted).
    Under the new None default its cheap kernels now sort by default —
    most visibly float keys, which previously came out in hash order. Integer
    keys were already ascending; the generic Python fallback stays unsorted.
    Pass sort=False to opt out.

Accelerated reductions from index summaries

• min/max on indexed Columns, and argmin/argmax inside group_by, are
  now accelerated using the index's per-block min/max summaries: when an
  index is available these reductions run from the precomputed summaries instead
  of decompressing the underlying data, which is dramatically faster on large
  columns. A fast path also builds min/max envelope plots from any index.
• The last group_by operation is memoized and reused when the same
  grouping is requested again, avoiding recomputation in interactive / repeated
  workflows (e.g. b2view).

b2view: group-by, sort, and richer plots

• Interactive group-by (G): group a CTable by a column (integer, string,
  or now float keys) directly in the viewer, with a three-list / two-column
  menu; while grouped, S/R operate on the grouped result and the data
  panel's subtitle shows a G(roup) chip. The last grouping is memoized for
  instant reuse.
• Sort by column (S): sort a CTable by a fully indexed column via a
  dropdown (R toggles reverse) as a zero-copy sort_by(view=True) that streams
  from the index — the table is never materialised, Esc restores the original
  order, and a SORTED chip shows in the status bar. Non-indexed columns can
  now be sorted too. Sort and filter are mutually exclusive; a row window
  composes over a sort, and an filter is preserved across Sort / Group.
• Better plots of grouped/sorted views: a grouped view plots bars for a
  categorical key and lines for a numeric key; numeric-key group plots
  render as stem/impulse charts rather than misleading connected lines. Bar
  plots gain an hi-res counterpart mirroring the line/scatter plots, and +/-
  zoom about the view's left edge.
• --max maximizes the current panel, and escape is now the single,
  consistent way to back out of every modal.

Other / bug fixes

• C-Blosc2 upgraded to 3.1.5.
• Open-file cache correctness: cached open handles are now validated against
  the file's fingerprint (st_mtime_ns, st_size) and cached index handles are
  released when a table closes, so a file changed underneath an open handle is no
  longer served stale.
• NumPy 2.5 compatibility: adjusted for deprecations in NumPy 2.5.
• Substantially reduced test-suite runtime, and emscripten builds no longer
  attempt to spawn subprocesses (unsupported there).

Release 4.5.1

Changes from 4.5.0 to 4.5.1

This follow-up release builds the b2view terminal viewer into a richer
data-exploration tool — a scatter plot, a searchable column picker, a
one-shot demo download, refreshed chrome, and several interaction fixes — and
upgrades the bundled C-Blosc2 to 3.1.4. WASM/Pyodide is now a fully
supported platform, and CTable.info reports per-column compressed sizes.

b2view: richer exploration

• Scatter plots: from a column plot, press s to scatter the current column
  (X) against another column (Y) chosen from a list, over the current (zoomed)
  row range; h then opens a high-resolution matplotlib scatter.
• High-res for 1-D series is now an envelope plot (matching the in-terminal
  view), and a new r key toggles between the min/max envelope and the raw
  values (strided-sampled when the range is wide).
• Searchable column picker: the c go-to-column key now opens a searchable,
  selectable list (type to filter, ↑/↓, Enter) for CTables, instead of a text
  field; N-D arrays still go by numeric index.
• Show/hide columns: / opens a searchable multi-select to pick which CTable
  columns are displayed.
• Demo download: b2view --download fetches a demo bundle
  (chicago-taxi-flat.b2z by default) into the current directory if it is not
  already there, then opens it.
• Refreshed chrome: a branded header, a left-docked filename label in the
  title, and clearer status chips.

b2view: interaction fixes

• Go-to-row/column pre-fill is now pre-selected, so the first keystroke
  replaces the current index instead of appending to it (typing a column name no
  longer produced e.g. 0payment.fare).
• Escape keeps its layered exit while a panel is maximized: with the data
  panel maximized, escape now unlocks a plot's locked row window (and clears
  filters) as documented, instead of being hijacked into restoring the panel —
  use r to restore (ESCAPE_TO_MINIMIZE = False).
• Test-suite robustness fixes (a timing flake and a Windows rendering glitch).

Other

• C-Blosc2 upgraded to 3.1.4.
• WASM/Pyodide is now a fully supported platform, with more frequent CI runs.
• CTable.info shows per-column compressed sizes (cbytes and cratio),
  and print_versions() uses clearer Python-Blosc2 / C-Blosc2 labels.

Release 4.5.0

Changes from 4.4.5 to 4.5.0

This release teaches the b2view terminal viewer to plot — peak-preserving
envelope line plots of any series, with zoom, a row-window lock, and an optional
high-resolution matplotlib view — and gives CTable a pandas-like display and
CSV experience. It also publishes WASM/Pyodide wheels to PyPI and adds
faster strided reads for NDArray and Column.

b2view: plotting and data inspection

• In-terminal plots: press p on a numeric series (a CTable column or an
  array row) to draw a braille line plot. Plots are peak-preserving min/max
  envelopes by default, so no spike or trough is hidden however large the
  series is; large local series stream their envelope exactly in bounded
  spans (only remote c2arrays fall back to a labeled strided sample).
• Zoom and row-window lock: zoom the plot into a row range and pan it; press
  v to lock the data grid to the plotted range so paging stays inside it
  (escape unlocks). The plot and high-res views honor the locked window.
• High-resolution view: h opens a high-res matplotlib image of the
  plotted range (new optional hires extra: matplotlib + textual-image).
• On-demand cell decode: enter decodes a single skipped/expensive CTable
  cell, and SChunk nodes now preview as a paged hex dump.
• Fixes and polish: row paging re-aligns to the page grid after dim-mode
  single-row scrolls; the data panel now focuses correctly with
  --path ... --panel data; status chips are branded yellow.

CTable display

• CTable.to_string() now renders the whole table by default (every row and
  every column), like pandas' DataFrame.to_string(). New max_rows and
  max_width parameters truncate on demand. Behaviour change: previously
  to_string() returned the truncated view; code that relied on that should
  pass max_rows=/max_width= (or use str()).
• The [N rows x M columns] dimensions footer now follows pandas: omitted by
  to_string() (pass show_dimensions=True to force it), and shown by
  str/repr/print only when the view is actually truncated. Previously it
  was always appended.
• repr(ctable) now shows the same truncated table as str(ctable)
  (pandas/polars convention), instead of the one-line CTable<…> summary. The
  compact summary remains available via ctable.info.
• New display options in set_printoptions: display_width controls the
  column-fitting width budget (None = auto-detect terminal, -1 = show all
  columns, positive int = fixed budget), and display_rows now accepts -1 to
  show all rows (0 still shows none).
• New blosc2.printoptions(...) context manager temporarily sets the display
  options and restores them on exit, e.g.
  with blosc2.printoptions(display_rows=-1, display_width=-1): print(t).

CTable I/O

• CTable.to_csv() now accepts no path, returning the CSV as a string like
  pandas' DataFrame.to_csv(). Passing a path still writes the file (and
  returns None); the returned string is byte-for-byte the same as the file.

Performance

• Faster strided reads: NDArray.__getitem__ gains a sparse-gather fast
  path for large strides, and Column.__getitem__ short-circuits when the
  logical positions equal the physical ones.
• Fix: a negative step in Column getitem could return []; it now
  returns the reversed selection.

Indexing

• Fix: a sidecar-handle cache collision could return the wrong SUMMARY
  index for a compact-store column.
• Cross-column index pruning is now enabled for compact CTable queries, so
  more predicates prune blocks before any data is materialized. The docs also
  note when summary indexes are not created automatically.

Packaging

• WASM/Pyodide wheels on PyPI: the main wheel build now also produces
  pyemscripten wheels for CPython 3.13 (2025 ABI) and 3.14 (2026 ABI) and
  uploads them to PyPI, so blosc2 is micropip-installable in Pyodide, and
  b2view prints a clear message instead of crashing when run under WASM.
  Known limitation: slicing an in-memory SChunk loaded from a frame fails on
  the Pyodide 0.29.x Emscripten toolchain (cp313); it works on Pyodide 314
  (cp314) and natively. See issue #664.
• cibuildwheel updated to 4.1.

Release 4.4.5

Changes from 4.4.3 to 4.4.5

Note: 4.4.4 was skipped due to a failure during the release process.

This release promotes the b2view terminal viewer to a core feature —
installed by default, with new interactive row and column filtering — and
makes BatchArray block layouts (and hence compression ratios) reproducible
across CPUs.

b2view, the terminal data viewer

• Installed by default: textual and rich are now regular
  dependencies, so the b2view CLI works out of the box (the [tui] extra
  is gone). A getting-started walkthrough was added to the docs, and the
  README now lists the CLI tools.
• Row filtering: pressing f on a CTable node opens a modal that takes
  the same string expressions as CTable.where() (dotted nested names,
  and/or) and pages through the matching view. Filters are remembered
  per node for the session, the data header shows the active filter plus
  the unfiltered total, and escape (or an empty expression) clears it.
• Column filtering: / narrows the visible columns by case-insensitive
  substring; column paging and the c goto-column modal then operate on
  that subset. Combines freely with the row filter; escape clears one
  layer per press (rows first, then columns).
• Mouse handling: the terminal owns the mouse by default, so native
  text selection/copy works like in any CLI program; --mouse lets b2view
  capture it instead (click-to-focus, wheel scrolling by half a page,
  paging at the edges).
• Navigation: ? opens a help screen listing all keys; c jumps to a
  column by index, exact name or unique name prefix; s/e jump to the
  first/last column window; row paging and jumps keep the cursor on its
  column; dim-mode index/viewport movements clamp at the boundaries instead
  of wrapping around.
• Rendering: column windows are fitted from measured rendered widths
  (and re-fitted on terminal resize and panel maximize/restore), and float
  columns use a uniform number of decimals so decimal points align down
  the column.
• Test suite: first automated tests for the TUI — Pilot-driven keyboard
  journeys against a deterministic generated store (marker tui), plus
  render unit tests. Skipped on wasm, where Textual apps cannot start
  (no termios).

BatchArray

• Reproducible block layouts: automatic variable-length block sizing
  now uses fixed byte budgets (1 MiB for clevel 1-3, 8 MiB for 4-6, 16 MiB
  for 7-8) instead of the CPU cache sizes, so the layout — and hence the
  compression ratio — no longer depends on the machine that created the
  array.

Build and docs

• Installing test dependencies: the docs now use
  pip install . --group test (a PEP 735 dependency group); the stale
  [test] extra syntax was removed.
• cibuildwheel updated to 4.0.

Release 4.4.3

Changes from 4.4.2 to 4.4.3

This is a maintenance release focused on faster CTable cold-start, printing
and groupby performance, a lighter import blosc2, new raw-storage access
for columns, and support for the new J2K/HTJ2K codec plugins.

CTable performance

• Lazy column opening in views: select() (and other view-producing
  operations) no longer open every projected column up front. A column is
  only opened from storage when the view actually reads it, so selecting and
  then touching a subset of columns — or aggregating a single one — skips
  the cold-start cost of the rest.
• Lazy index opening in queries: query planning no longer opens every
  SUMMARY-indexed column on a wide persistent table; only indexes for
  columns actually referenced by the predicate are loaded.
• Faster table printing: repr()/to_string() now memoise per-column
  sparse gathers for the duration of a render and combine the head and tail
  rows into a single sparse read per column. Each column is read from
  storage once instead of ~6 times (precision detection, width sizing and
  row rendering all hit the cache).
• Groupby with integral float keys: float key columns whose values are
  integral and fit a compact non-negative range (e.g. float32 id/second
  columns) now take the dense single-key fast path instead of the markedly
  slower generic float-hash path. Fractional or non-finite keys fall back
  automatically.
• No tempdir in read mode: opening a .b2z/.b2d store in 'r' mode
  no longer creates a temporary working directory, since nothing is ever
  written.

Lighter imports and prefetcher rework

• asyncio dependency dropped: the on-disk chunk prefetcher used by the
  UDF and numexpr fallback engines now uses plain concurrent.futures
  instead of an asyncio event loop. import blosc2 no longer pulls in
  ~30 asyncio modules, saving ~3 MB of memory footprint at import time.
• Prefetcher deadlock fixed: an exception during evaluation could leave
  the generator finalizer blocked forever in thread.join() while the
  reader thread was stuck on a full prefetch queue. A stop event now makes
  the producer bail out when its consumer goes away.

New features

• Column.raw accessor: returns the underlying storage container of a
  column (NDArray, ListArray, DictionaryColumn, …) directly. Unlike
  Column.__getitem__, which always materializes NumPy arrays, this is the
  column as a blosc2-native compressed object — usable as a lazy-expression
  operand without decompressing, and exposing storage details like schunk,
  chunks or cparams. Note that this is a physical view: fixed-width
  containers are over-allocated to chunk capacity, so slice to len(table)
  to get just the live rows, and no validity-mask or null-sentinel
  processing is applied. Raises AttributeError for computed columns,
  which have no backing storage.
• J2K and HTJ2K codec IDs: blosc2.Codec.J2K and blosc2.Codec.HTJ2K
  expose the IDs for the new JPEG 2000 codec plugins (installable with
  pip install blosc2-j2k and pip install blosc2-htj2k).

Fixes

• --float-trunc-prec and nested columns: the precision-truncation
  filter of the parquet_to_blosc2 CLI now propagates to float fields
  inside nested (struct/list) columns too.
• Guard for unsupported computed-column expressions: expressions that
  would serialize to an empty or non-round-trippable string are now rejected
  with an early, actionable ValueError at add_computed_column() time,
  instead of silently breaking on reload.

Build

• C-Blosc2 updated to 3.1.3.

Release 4.4.2

Changes from 4.4.1 to 4.4.2

This is a feature and maintenance release that promotes DSL kernels to
first-class CTable computed columns, adds a new CTable.__setitem__
assignment idiom, optimises bulk NDArray writes, and fixes several
correctness issues.

DSL kernels as first-class CTable columns

• add_computed_column() accepts DSL kernels: @blosc2.dsl_kernel-decorated
  functions can now back virtual computed columns directly, in addition to
  the existing string-expression form. The column survives save/open
  round-trips via persisted dsl_source.
• add_generated_column() accepts DSL kernels: stored generated columns
  (written during append/extend and on refresh_generated_column())
  now support DSL kernels as their transformer.
• CTable.where() accepts UDF/DSL kernels: filter predicates are no
  longer limited to expression strings — any DSL kernel can be passed directly.
• dtype inference for DSL kernels: when dtype is omitted,
  lazyudf() infers the output dtype via NumPy type promotion of the input
  column dtypes. Pass dtype explicitly for type-changing kernels
  (comparisons, casts).
• kernel_from_source() utility: new dsl_kernel.kernel_from_source()
  reconstructs a DSLKernel from its stored source text, shared by the
  CTable DSL-column loaders and the persisted LazyUDF decoder.
• Security note: .b2d files from untrusted sources that contain DSL
  computed columns execute stored Python source on open. A warning is now
  included in the documentation.

New CTable.__setitem__ column-assignment API

• t["col"] = arr: new shorthand equivalent to t["col"][:] = arr.
  Accepts any array-like including blosc2.NDArray. Raises KeyError for
  unknown columns and ValueError for views or read-only tables.

Chunked NDArray writes in extend() and Column.__setitem__

• extend({"col": ndarray}) decompresses chunk-by-chunk: when a
  blosc2.NDArray is passed as a column value to extend(), it is now
  written in chunks instead of being fully decompressed upfront. Pass
  validate=False to avoid a transient full decompression during constraint
  checking.
• col[:] = blosc2_ndarray fast path: a new no-holes fast path in
  Column.__setitem__ skips the O(n) validity-mask gather and writes the
  NDArray one chunk at a time using contiguous slice writes. Works for both
  scalar and fixed-shape ndarray columns. Falls back to a chunked fancy-index
  path when deleted rows are present.

BLOSC_ME_JIT environment variable override

• Full CLI override: BLOSC_ME_JIT now takes unconditional priority over
  both the jit= and jit_backend= keyword arguments, making it easy to
  switch JIT backends from the command line without modifying code.

Correctness fixes

• View corruption in Column.__setitem__: a None == None guard
  evaluation on view-backed columns could fire the NDArray fast path,
  bypassing physical-position remapping and silently corrupting rows. Fixed
  by explicitly checking base is None before activating the fast path.
• CTable.__setitem__ view guard: the new t["col"] = arr API now
  raises ValueError on views, matching the contract of all other mutating
  CTable methods.
• Fast path enabled for disk-opened tables: the fast path previously
  remained dormant for tables opened from disk because _last_pos starts as
  None. The guard now calls _resolve_last_pos() to lazily initialise it.
• DSL column jit_backend preserved in _empty_copy: the jit_backend
  setting was silently dropped during internal table copies; it is now
  retained.
• lazyexpr Column unwrapping: convert_inputs() now automatically
  unwraps CTable.Column objects to their backing NDArray so that shape and
  identity checks work correctly.

Documentation and examples

• Parquet-to-blosc2 walkthrough: new step-by-step tutorial added to the
  getting-started section. Thanks to @SyedIshmumAhnaf.
• CTable performance tips: new section in the overview covering when to
  prefer computed vs. generated columns, chunk sizing, and query optimisation.
• Simplified docstring examples: examples throughout ndarray.py and
  ctable.py now use blosc2.array(), blosc2.arange(), and
  blosc2.linspace() directly instead of two-step numpy-then-asarray
  patterns.
• udf-computed-col.py example: new end-to-end example demonstrating DSL
  kernel computed and generated columns.