Skip to content

Commit edeffb2

Browse files
hyperpolymathclaude
andcommitted
feat: implement Phase 1 — Python/R parser, Julia codegen, and benchmarks
Implement the core julianiser pipeline that analyses Python/R data-science code and generates equivalent Julia modules with type annotations: - Manifest: [project], [[sources]], [[mappings]], [julia] sections - Parser: detects pandas/numpy/scipy/dplyr/ggplot2/tidyr library calls from import statements, qualified calls, and bare function calls - Julia codegen: generates .jl modules with using statements, translated function calls, Project.toml with package UUIDs - Benchmarks: generates BenchmarkTools.jl script, shell runner comparing original vs Julia, and results.toml template - ABI: SourceLanguage, LibraryMapping, JuliaType, TranslationUnit, BenchmarkResult, DetectedCall domain types - Tests: 25 unit tests + 8 integration tests + 1 doctest (59 total) - Example: data-pipeline with Python pandas/numpy and R dplyr/ggplot2 Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
1 parent 9b15faa commit edeffb2

14 files changed

Lines changed: 3828 additions & 61 deletions

File tree

Cargo.lock

Lines changed: 907 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,3 +21,4 @@ walkdir = "2"
2121

2222
[dev-dependencies]
2323
tempfile = "3"
24+
serde_json = "1"

examples/data-pipeline/analysis.R

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# SPDX-License-Identifier: PMPL-1.0-or-later
2+
# Example R analysis script for julianiser translation demo.
3+
#
4+
# This script demonstrates common dplyr/ggplot2 patterns that julianiser
5+
# can detect and translate into Julia equivalents.
6+
7+
library(dplyr)
8+
library(ggplot2)
9+
10+
# Load and filter data.
11+
data <- read_csv("experiment_results.csv")
12+
clean_data <- filter(data, !is.na(value))
13+
14+
# dplyr pipeline: group, summarise, arrange.
15+
summary_stats <- clean_data %>%
16+
group_by(treatment) %>%
17+
summarise(
18+
mean_value = mean(value),
19+
sd_value = sd(value),
20+
n = n()
21+
) %>%
22+
arrange(desc(mean_value))
23+
24+
# Join with metadata.
25+
metadata <- read_csv("metadata.csv")
26+
enriched <- left_join(summary_stats, metadata, by = "treatment")
27+
28+
# Statistical test.
29+
model <- lm(value ~ treatment + age + gender, data = clean_data)
30+
31+
# Visualisation.
32+
p <- ggplot(clean_data, aes(x = treatment, y = value, fill = treatment)) +
33+
geom_bar(stat = "summary", fun = "mean") +
34+
geom_point(position = position_jitter(width = 0.2), alpha = 0.5) +
35+
theme(legend.position = "none")
36+
37+
write_csv(enriched, "enriched_results.csv")
38+
print(summary_stats)
Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,38 @@
1+
# SPDX-License-Identifier: PMPL-1.0-or-later
2+
# julianiser manifest — Example data pipeline translation
3+
#
4+
# This example demonstrates translating a Python pandas/numpy pipeline
5+
# and an R dplyr/ggplot2 analysis script into Julia equivalents.
6+
7+
[project]
8+
name = "data-pipeline-example"
9+
version = "0.1.0"
10+
description = "Example: translate Python and R data pipelines to Julia"
11+
12+
[[sources]]
13+
path = "pipeline.py"
14+
language = "python"
15+
16+
[[sources]]
17+
path = "analysis.R"
18+
language = "r"
19+
20+
[[mappings]]
21+
from-lib = "pandas"
22+
to-lib = "DataFrames.jl"
23+
24+
[[mappings]]
25+
from-lib = "numpy"
26+
to-lib = "Base"
27+
28+
[[mappings]]
29+
from-lib = "dplyr"
30+
to-lib = "DataFrames.jl"
31+
32+
[[mappings]]
33+
from-lib = "ggplot2"
34+
to-lib = "Plots.jl"
35+
36+
[julia]
37+
version = "1.10"
38+
packages = ["DataFrames", "CSV", "Statistics", "LinearAlgebra", "Plots", "BenchmarkTools"]

examples/data-pipeline/pipeline.py

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
# SPDX-License-Identifier: PMPL-1.0-or-later
2+
# Example Python data pipeline for julianiser translation demo.
3+
#
4+
# This script demonstrates common pandas/numpy patterns that julianiser
5+
# can detect and translate into Julia equivalents.
6+
7+
import pandas as pd
8+
import numpy as np
9+
10+
# Load data from CSV.
11+
df = pd.read_csv("sales_data.csv")
12+
13+
# Filter rows and compute derived columns.
14+
filtered = df.dropna()
15+
filtered["total"] = filtered["quantity"] * filtered["price"]
16+
17+
# Group and aggregate.
18+
summary = filtered.groupby("category").agg({"total": "sum", "quantity": "mean"})
19+
20+
# Numerical computation with numpy.
21+
prices = np.array(filtered["price"].values)
22+
quantities = np.array(filtered["quantity"].values)
23+
weighted_avg = np.mean(prices * quantities) / np.mean(quantities)
24+
25+
# Statistical summary.
26+
std_dev = np.std(prices)
27+
correlation = np.dot(prices - np.mean(prices), quantities - np.mean(quantities))
28+
29+
# Sort and output.
30+
result = summary.sort_values("total", ascending=False)
31+
result.to_csv("output.csv")
32+
33+
print(f"Weighted average price: {weighted_avg:.2f}")
34+
print(f"Price std dev: {std_dev:.2f}")
35+
print(f"Price-quantity correlation: {correlation:.4f}")

0 commit comments

Comments
 (0)