|
| 1 | +<!-- |
| 2 | +SPDX-License-Identifier: CC-BY-SA-4.0 |
| 3 | +SPDX-FileCopyrightText: 2025-2026 Jonathan D.A. Jewell <j.d.a.jewell@open.ac.uk> |
| 4 | +--> |
| 5 | + |
| 6 | +# What Is This? |
| 7 | + |
| 8 | +Julianiser analyses existing Python and R data science code, identifies |
| 9 | +performance-critical array and dataframe operations, and generates |
| 10 | +equivalent Julia code with full type annotations — producing **drop-in |
| 11 | +replacement modules** that deliver **10-100x speedups** through Julia’s |
| 12 | +LLVM JIT compilation. |
| 13 | + |
| 14 | +Scientists keep their Python notebooks. Julia runs underneath. |
| 15 | + |
| 16 | +# How It Works |
| 17 | + |
| 18 | +Describe your data pipeline in a `julianiser.toml` manifest. Julianiser: |
| 19 | + |
| 20 | +1. **Parses** your Python functions (via AST) or R scripts (via R |
| 21 | + parser) |
| 22 | + |
| 23 | +2. **Identifies** array operations, dataframe transformations, and |
| 24 | + numeric hot paths |
| 25 | + |
| 26 | +3. **Generates** Julia equivalents with proper type annotations and |
| 27 | + broadcasting |
| 28 | + |
| 29 | +4. **Creates** a Zig FFI bridge for calling Julia from your existing |
| 30 | + code (zero-copy where possible) |
| 31 | + |
| 32 | +5. **Benchmarks** the original vs. generated code to verify speedup |
| 33 | + |
| 34 | +6. **Falls back** gracefully — if Julia is not installed, your Python/R |
| 35 | + code still runs |
| 36 | + |
| 37 | +# Key Value |
| 38 | + |
| 39 | +- **No Julia knowledge required** — Julianiser handles the translation |
| 40 | + |
| 41 | +- **10-100x speedups** on numeric, array, and dataframe code without |
| 42 | + manual rewrites |
| 43 | + |
| 44 | +- **Gradual adoption** — julianise one function at a time, benchmark |
| 45 | + each |
| 46 | + |
| 47 | +- **Drop-in modules** — generated Julia code exposes the same API as |
| 48 | + your Python/R functions |
| 49 | + |
| 50 | +- **Formally verified bridges** — Idris2 ABI proofs guarantee interface |
| 51 | + correctness |
| 52 | + |
| 53 | +# Supported Patterns |
| 54 | + |
| 55 | +Julianiser recognises and translates these common data science patterns: |
| 56 | + |
| 57 | +| Python | Julia Equivalent | Notes | |
| 58 | +|----|----|----| |
| 59 | +| `pandas.DataFrame` | `DataFrames.jl` | Column operations, groupby, joins | |
| 60 | +| `numpy` arrays | Native Julia arrays | Broadcasting, slicing, linear algebra | |
| 61 | +| `scipy.optimize` | `Optim.jl` / Julia stdlib | Minimisation, root-finding, curve fitting | |
| 62 | +| `matplotlib` / `seaborn` | `Plots.jl` / `Makie.jl` | Static and interactive plotting | |
| 63 | +| `scikit-learn` pipelines | `MLJ.jl` | Train/predict/evaluate pattern | |
| 64 | +| R `data.frame` / `tibble` | `DataFrames.jl` | dplyr-style verbs mapped to Julia | |
| 65 | +| R `apply` / `sapply` / `lapply` | Julia broadcasting / `map` | Vectorised equivalents | |
| 66 | + |
| 67 | +# Why Julia? |
| 68 | + |
| 69 | +Julia achieves C-like performance through: |
| 70 | + |
| 71 | +- **LLVM JIT compilation** — code is compiled to native machine code on |
| 72 | + first call |
| 73 | + |
| 74 | +- **Multiple dispatch** — the type system enables aggressive |
| 75 | + specialisation |
| 76 | + |
| 77 | +- **Type inference** — the compiler infers concrete types for fast code |
| 78 | + paths |
| 79 | + |
| 80 | +- **Broadcasting** — element-wise operations fuse into single loops (no |
| 81 | + temporary arrays) |
| 82 | + |
| 83 | +- **In-place operations** — `mul!`, `ldiv!`, and friends avoid |
| 84 | + allocation |
| 85 | + |
| 86 | +The "two-language problem" (prototype in Python, rewrite in C) |
| 87 | +disappears. Julia is both the prototyping language and the production |
| 88 | +language. |
| 89 | + |
| 90 | +# Architecture |
| 91 | + |
| 92 | +Follows the hyperpolymath -iser pattern (same as |
| 93 | +[Chapeliser](https://github.com/hyperpolymath/chapeliser)): |
| 94 | + |
| 95 | +- **Manifest** (`julianiser.toml`) — describe WHAT you want julianised |
| 96 | + |
| 97 | +- **Source Parser** — Python AST analysis / R parser for identifying |
| 98 | + translatable patterns |
| 99 | + |
| 100 | +- **Idris2 ABI** (`src/interface/abi/`) — formal proofs of equivalence |
| 101 | + between source and generated code |
| 102 | + |
| 103 | +- **Julia Codegen** (`src/codegen/`) — generates type-annotated Julia |
| 104 | + with proper broadcasting |
| 105 | + |
| 106 | +- **Zig FFI** (`src/interface/ffi/`) — C-ABI bridge between Julia |
| 107 | + runtime and calling code |
| 108 | + |
| 109 | +- **Benchmark Harness** — compares original Python/R vs. generated Julia |
| 110 | + performance |
| 111 | + |
| 112 | +- **Rust CLI** (`src/main.rs`) — orchestrates parsing, validation, |
| 113 | + generation, building, and benchmarking |
| 114 | + |
| 115 | +User writes zero Julia code. Julianiser generates everything. |
| 116 | + |
| 117 | +Part of the [-iser family](https://github.com/hyperpolymath/iseriser) of |
| 118 | +acceleration frameworks. |
| 119 | + |
| 120 | +# Use Cases |
| 121 | + |
| 122 | +- **Data science acceleration** — speed up pandas/numpy pipelines |
| 123 | + without rewriting |
| 124 | + |
| 125 | +- **Batch processing** — convert overnight Python ETL jobs to minutes |
| 126 | + with Julia |
| 127 | + |
| 128 | +- **Scientific computing migration** — gradually move research code from |
| 129 | + Python/R to Julia |
| 130 | + |
| 131 | +- **HPC preparation** — Julia code can target GPU (CUDA.jl) and |
| 132 | + distributed (Distributed.jl) backends |
| 133 | + |
| 134 | +# Quick Start |
| 135 | + |
| 136 | +```bash |
| 137 | +# Install julianiser |
| 138 | +cargo install julianiser |
| 139 | + |
| 140 | +# Initialise a manifest in your project |
| 141 | +julianiser init |
| 142 | + |
| 143 | +# Edit julianiser.toml to point at your Python/R code |
| 144 | +# Then generate Julia replacements |
| 145 | +julianiser generate |
| 146 | + |
| 147 | +# Benchmark original vs. generated |
| 148 | +julianiser run --benchmark |
| 149 | +``` |
| 150 | + |
| 151 | +# Status |
| 152 | + |
| 153 | +**Codebase in progress.** Architecture defined, CLI scaffolded, codegen |
| 154 | +pending. Phase 0 (scaffold) complete. Phase 1 (Python AST parser + Julia |
| 155 | +codegen) underway. |
| 156 | + |
| 157 | +# License |
| 158 | + |
| 159 | +SPDX-License-Identifier: CC-BY-SA-4.0 |
0 commit comments