From 026474dcf786a4be5c83e8a424987af32dedf75a Mon Sep 17 00:00:00 2001 From: hyperpolymath <6759885+hyperpolymath@users.noreply.github.com> Date: Thu, 25 Jun 2026 20:08:53 +0100 Subject: [PATCH] docs(readme): convert README.adoc -> Markdown README.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit README must be real Markdown to render in GitHub community-health, the GitHub profile, and external MCP directories (Glama) — AsciiDoc shows as raw markup there. pandoc asciidoc->GFM, badges fixed to clickable, SPDX header kept as an HTML comment, duplicate README.adoc removed. Co-Authored-By: Claude Opus 4.8 --- README.adoc | 147 ----------------------------------------- README.md | 187 ++++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 187 insertions(+), 147 deletions(-) delete mode 100644 README.adoc create mode 100644 README.md diff --git a/README.adoc b/README.adoc deleted file mode 100644 index 3b0e274..0000000 --- a/README.adoc +++ /dev/null @@ -1,147 +0,0 @@ -// SPDX-License-Identifier: CC-BY-SA-4.0 -// Copyright (c) 2026 Jonathan D.A. Jewell (hyperpolymath) -= Bqniser -Jonathan D.A. Jewell -:toc: left -:icons: font - -image:https://img.shields.io/badge/Sponsor-%E2%9D%A4-pink?logo=github[Sponsor,link=https://github.com/sponsors/hyperpolymath] - -== What Is This? - -**BQNiser** scans your existing code for array computation patterns -- loops, -maps, folds, reductions, comprehensions -- and rewrites them as optimised -https://mlochbaum.github.io/BQN/[BQN] array primitives. - -BQN (by Marshall Lochbaum) is an array language descended from APL/J/K. -It brings first-class functions, trains (`∘ ○ ⊸ ⟜`), under/structural-under -combinators (`⌾`), rank polymorphism, and leading-axis theory to array -processing. BQNiser lets you harness 10-100x speedups on array-heavy -workloads *without learning BQN yourself*. - -Where your Python loop takes 200ms to filter-then-reduce a million-row column, -a single BQN expression using `/` (replicate), `⌽` (reverse), or `⍋` (grade-up) -completes in under 2ms via https://github.com/dzaima/CBQN[CBQN]'s vectorised -runtime. - -== How It Works - -BQNiser follows the https://github.com/hyperpolymath/iseriser[-iser pattern]: -you describe *what* you want; the tool generates *everything*. - -1. **Manifest** (`bqniser.toml`) -- describe your workload, entry points, data shapes -2. **Source Analysis** -- AST-based detection of array patterns in Rust, Python, or Julia source -3. **Pattern Matching** -- map detected patterns to BQN primitives: - - Nested loops over arrays -> `¨` (each), `⌜` (table) - - Accumulate/reduce -> `´` (fold), `` ` `` (scan) - - Filter -> `/` (replicate) with boolean mask - - Sort/rank -> `⍋` (grade-up), `⍒` (grade-down) - - Reshape/transpose -> `⥊` (reshape), `⍉` (transpose) - - Concatenation -> `∾` (join) - - Index selection -> `⊏` (select) - - Reversal -> `⌽` (reverse), `⌾` (under) for structural transforms -4. **Idris2 ABI** (`src/interface/abi/`) -- formal proofs that each rewrite preserves - input-output equivalence (dependent types guarantee correctness) -5. **BQN Codegen** -- emit optimised BQN source using trains and tacit style -6. **CBQN FFI Bridge** (`src/interface/ffi/`) -- Zig-based C-ABI bridge to the - https://github.com/dzaima/CBQN[CBQN] runtime, so generated BQN runs - in-process with zero serialisation overhead - -== Key Value - -* **10-100x on array operations** -- vectorised BQN primitives beat scalar loops -* **Automatic pattern detection** -- finds optimisable code via AST analysis -* **Equivalence proofs** -- Idris2 dependent types prove rewrites preserve semantics -* **Gradual adoption** -- replace hot loops one at a time, keep the rest untouched -* **Zero BQN knowledge required** -- the tool generates all BQN and FFI code -* **Rank polymorphism** -- BQN's leading-axis theory means primitives generalise - across any number of dimensions automatically - -== BQN Primitives Reference - -BQNiser targets these core primitives and combinators: - -[cols="1,2,3"] -|=== -| Glyph | Name | Detected Pattern - -| `∾` | Join | `concat`, `append`, `extend` -| `⌽` | Reverse | `reversed()`, `[::-1]` -| `⍋` | Grade Up | `sorted()`, `argsort` -| `⍒` | Grade Down | `sorted(reverse=True)` -| `/` | Replicate | `filter`, boolean indexing -| `⊏` | Select | `arr[indices]`, `take`, `gather` -| `⥊` | Reshape | `reshape`, `flatten`, `ravel` -| `⍉` | Transpose | `transpose`, `T`, axis permutation -| `¨` | Each | `map`, list comprehension -| `⌜` | Table | nested `map`, outer product -| `´` | Fold | `reduce`, `foldl`, `inject` -| `` ` `` | Scan | `accumulate`, `scanl`, prefix sums -| `⌾` | Under | structural transforms, lens-like updates -| `∘○⊸⟜` | Trains | function composition, tacit pipelines -|=== - -== Architecture - -``` - bqniser.toml src/interface/abi/ src/interface/ffi/ - ┌──────────┐ ┌─────────────────┐ ┌─────────────────┐ - │ Manifest │──parse──→│ Idris2 ABI │──gen──→│ Zig FFI │ - │ (TOML) │ │ (Types, Layout, │ │ (CBQN embedding │ - └──────────┘ │ Foreign) │ │ C-ABI bridge) │ - │ └────────┬────────┘ └────────┬────────┘ - │ │ │ - ▼ ▼ ▼ - ┌──────────┐ ┌─────────────────┐ ┌─────────────────┐ - │ Rust CLI │──scan──→ │ Pattern Matcher │──emit─→│ BQN Codegen │ - │ (clap) │ │ (AST → BQN map)│ │ (.bqn files) │ - └──────────┘ └─────────────────┘ └─────────────────┘ -``` - -* **Rust CLI** -- parses manifest, invokes AST analysis, orchestrates generation -* **Pattern Matcher** -- maps source-level patterns to BQN primitive candidates -* **Idris2 ABI** -- proves BQN value layout (array header + shape vector + data cells), - proves rewrite equivalence for each transformation -* **BQN Codegen** -- emits idiomatic BQN using trains, under, and tacit definitions -* **Zig FFI** -- bridges CBQN's C API (`BQN_NewEval`, `BQN_Call`, value accessors), - handles BQN value lifecycle and memory management - -Part of the https://github.com/hyperpolymath/iseriser[-iser family] of -acceleration frameworks. - -== Use Cases - -* **Data processing pipelines** -- ETL stages with heavy filtering, sorting, reshaping -* **Scientific computing** -- matrix operations, statistical reductions, signal processing -* **Financial analytics** -- rolling windows, aggregations, time-series transforms -* **Compiler/transpiler backends** -- batch AST transformations as array operations -* **ETL optimisation** -- replace row-by-row processing with bulk array primitives - -== Quick Start - -```bash -# Initialise a manifest -bqniser init - -# Edit bqniser.toml to describe your workload -# (see examples/ for patterns) - -# Generate BQN + FFI artifacts -bqniser generate - -# Build everything -bqniser build - -# Run -bqniser run -``` - -== Status - -**Codebase in progress.** Architecture defined, CLI scaffolded, Idris2 ABI types -and Zig FFI bridge stubbed. Pattern detection engine and BQN codegen are the -active development frontier. - -== License - -SPDX-License-Identifier: CC-BY-SA-4.0 diff --git a/README.md b/README.md new file mode 100644 index 0000000..2d92b71 --- /dev/null +++ b/README.md @@ -0,0 +1,187 @@ + + +[![Sponsor](https://img.shields.io/badge/Sponsor-%E2%9D%A4-pink?logo=github)](https://github.com/sponsors/hyperpolymath) + +# What Is This? + +**BQNiser** scans your existing code for array computation +patterns — loops, maps, folds, reductions, comprehensions — and rewrites +them as optimised [BQN](https://mlochbaum.github.io/BQN/) array +primitives. + +BQN (by Marshall Lochbaum) is an array language descended from APL/J/K. +It brings first-class functions, trains (`∘` `○` `⊸` `⟜`), +under/structural-under combinators (`⌾`), rank polymorphism, and +leading-axis theory to array processing. BQNiser lets you harness +10-100x speedups on array-heavy workloads **without learning BQN +yourself**. + +Where your Python loop takes 200ms to filter-then-reduce a million-row +column, a single BQN expression using `/` (replicate), `⌽` (reverse), or +`⍋` (grade-up) completes in under 2ms via +[CBQN](https://github.com/dzaima/CBQN)’s vectorised runtime. + +# How It Works + +BQNiser follows the [-iser +pattern](https://github.com/hyperpolymath/iseriser): you describe +**what** you want; the tool generates **everything**. + +1. **Manifest** (`bqniser.toml`) — describe your workload, entry + points, data shapes + +2. **Source Analysis** — AST-based detection of array patterns in Rust, + Python, or Julia source + +3. **Pattern Matching** — map detected patterns to BQN primitives: + + - Nested loops over arrays → `¨` (each), `⌜` (table) + + - Accumulate/reduce → `´` (fold), `` ` `` (scan) + + - Filter → `/` (replicate) with boolean mask + + - Sort/rank → `⍋` (grade-up), `⍒` (grade-down) + + - Reshape/transpose → `⥊` (reshape), `⍉` (transpose) + + - Concatenation → `∾` (join) + + - Index selection → `⊏` (select) + + - Reversal → `⌽` (reverse), `⌾` (under) for structural transforms + +4. **Idris2 ABI** (`src/interface/abi/`) — formal proofs that each + rewrite preserves input-output equivalence (dependent types + guarantee correctness) + +5. **BQN Codegen** — emit optimised BQN source using trains and tacit + style + +6. **CBQN FFI Bridge** (`src/interface/ffi/`) — Zig-based C-ABI bridge + to the [CBQN](https://github.com/dzaima/CBQN) runtime, so generated + BQN runs in-process with zero serialisation overhead + +# Key Value + +- **10-100x on array operations** — vectorised BQN primitives beat + scalar loops + +- **Automatic pattern detection** — finds optimisable code via AST + analysis + +- **Equivalence proofs** — Idris2 dependent types prove rewrites + preserve semantics + +- **Gradual adoption** — replace hot loops one at a time, keep the rest + untouched + +- **Zero BQN knowledge required** — the tool generates all BQN and FFI + code + +- **Rank polymorphism** — BQN’s leading-axis theory means primitives + generalise across any number of dimensions automatically + +# BQN Primitives Reference + +BQNiser targets these core primitives and combinators: + +| Glyph | Name | Detected Pattern | +|---------|------------|------------------------------------------| +| `∾` | Join | `concat`, `append`, `extend` | +| `⌽` | Reverse | `reversed()`, `[::-1]` | +| `⍋` | Grade Up | `sorted()`, `argsort` | +| `⍒` | Grade Down | `sorted(reverse=True)` | +| `/` | Replicate | `filter`, boolean indexing | +| `⊏` | Select | `arr[indices]`, `take`, `gather` | +| `⥊` | Reshape | `reshape`, `flatten`, `ravel` | +| `⍉` | Transpose | `transpose`, `T`, axis permutation | +| `¨` | Each | `map`, list comprehension | +| `⌜` | Table | nested `map`, outer product | +| `´` | Fold | `reduce`, `foldl`, `inject` | +| `` ` `` | Scan | `accumulate`, `scanl`, prefix sums | +| `⌾` | Under | structural transforms, lens-like updates | +| `∘○⊸⟜` | Trains | function composition, tacit pipelines | + +# Architecture + + bqniser.toml src/interface/abi/ src/interface/ffi/ + ┌──────────┐ ┌─────────────────┐ ┌─────────────────┐ + │ Manifest │──parse──→│ Idris2 ABI │──gen──→│ Zig FFI │ + │ (TOML) │ │ (Types, Layout, │ │ (CBQN embedding │ + └──────────┘ │ Foreign) │ │ C-ABI bridge) │ + │ └────────┬────────┘ └────────┬────────┘ + │ │ │ + ▼ ▼ ▼ + ┌──────────┐ ┌─────────────────┐ ┌─────────────────┐ + │ Rust CLI │──scan──→ │ Pattern Matcher │──emit─→│ BQN Codegen │ + │ (clap) │ │ (AST → BQN map)│ │ (.bqn files) │ + └──────────┘ └─────────────────┘ └─────────────────┘ + +- **Rust CLI** — parses manifest, invokes AST analysis, orchestrates + generation + +- **Pattern Matcher** — maps source-level patterns to BQN primitive + candidates + +- **Idris2 ABI** — proves BQN value layout (array header + shape + vector + data cells), proves rewrite equivalence for each + transformation + +- **BQN Codegen** — emits idiomatic BQN using trains, under, and tacit + definitions + +- **Zig FFI** — bridges CBQN’s C API (`BQN_NewEval`, `BQN_Call`, value + accessors), handles BQN value lifecycle and memory management + +Part of the [-iser family](https://github.com/hyperpolymath/iseriser) of +acceleration frameworks. + +# Use Cases + +- **Data processing pipelines** — ETL stages with heavy filtering, + sorting, reshaping + +- **Scientific computing** — matrix operations, statistical reductions, + signal processing + +- **Financial analytics** — rolling windows, aggregations, time-series + transforms + +- **Compiler/transpiler backends** — batch AST transformations as array + operations + +- **ETL optimisation** — replace row-by-row processing with bulk array + primitives + +# Quick Start + +```bash +# Initialise a manifest +bqniser init + +# Edit bqniser.toml to describe your workload +# (see examples/ for patterns) + +# Generate BQN + FFI artifacts +bqniser generate + +# Build everything +bqniser build + +# Run +bqniser run +``` + +# Status + +**Codebase in progress.** Architecture defined, CLI scaffolded, Idris2 +ABI types and Zig FFI bridge stubbed. Pattern detection engine and BQN +codegen are the active development frontier. + +# License + +SPDX-License-Identifier: CC-BY-SA-4.0