Skip to content

Commit e54fa44

Browse files
refactor(interop): remove Python from_pytorch bridge, make it pure Julia (#63)
## Summary The audit found `from_pytorch` claimed a **"built-in Python bridge"** for direct `.pt/.pth/.ckpt` import — but the bridge script (`scripts/pytorch_to_axiom_descriptor.py`) **never shipped**, so that path always threw, and shelling out to `python3` violates the estate-wide Python ban anyway. This removes the Python entirely and makes PyTorch import **pure Julia**. ## Changes - **`src/integrations/interop.jl`** — delete `_run_pytorch_bridge` (the `python3` shell-out) and `_default_pytorch_bridge_script`. `from_pytorch(path; strict)` now imports the pure-Julia `axiom.pytorch.sequential.v1` JSON descriptor. A raw `.pt/.pth/.ckpt` gets a clear `ArgumentError` (it needs a PyTorch/Python runtime; export the descriptor first) instead of a false "bridge" promise. The `bridge`/`python_cmd`/`bridge_script` params are gone. - **`test/runtests.jl` + `test/ci/interop_smoke.jl`** — the interop tests previously **wrote an inline `.py` bridge script and ran it via `python3`** (banned-language content in tests). Replaced with pure-Julia descriptor tests: write the JSON descriptor directly, load it, and assert a `.pt` path is rejected. - **`ROADMAP.adoc` + `README.adoc`** — correct the two false `[x] direct .pt bridge shipped (via scripts/pytorch_to_axiom_descriptor.py)` claims and the "built-in Python bridge (requires python3 + torch)" README text to the honest pure-Julia descriptor scope. ## Verification - Package loads; `from_pytorch` **rejects `.pt` with `ArgumentError`** and **imports the descriptor to a real `Pipeline`** (checked directly). - `grep` confirms **no dangling references** to the removed bridge params (`_run_pytorch_bridge`, `bridge_script`, `python_cmd`, `AXIOM_PYTHON`, `pytorch_to_axiom_descriptor`) anywhere in `src/` or `test/`. - Full `Pkg.test()` matrix runs here as the authoritative gate. ## Not in this change (flagged) `src/integrations/huggingface.jl:633` has a **help-string** suggesting the user run `python -c "import torch; …"` to convert a `.bin` to safetensors — not executed Python, a different feature. Left as-is pending a call on whether to reword it. A **native Julia checkpoint reader** (zip + pickle, to read raw `.pt` without Python) is possible but needs a real `.pt` fixture to build safely (which needs PyTorch) — noted as future work. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01UPFC9YQ7g9gc3VnRox42Q1 --- _Generated by [Claude Code](https://claude.ai/code/session_01UPFC9YQ7g9gc3VnRox42Q1)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent 0d263c9 commit e54fa44

5 files changed

Lines changed: 64 additions & 171 deletions

File tree

README.adoc

Lines changed: 7 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -96,20 +96,20 @@ save_certificate(cert, "fda_submission.cert")
9696

9797
[source,julia]
9898
----
99-
# Import from a PyTorch checkpoint (.pt/.pth/.ckpt) via built-in Python bridge
100-
# (requires python3 + torch in the selected runtime)
101-
model = from_pytorch("model.pt")
99+
# Import from a canonical PyTorch descriptor JSON — pure Julia, no runtime deps.
100+
model = from_pytorch("model.pytorch.json")
102101
103-
# Or import canonical descriptor JSON
104-
model2 = from_pytorch("model.pytorch.json")
102+
# Raw .pt/.pth/.ckpt files are Python-pickle and need a PyTorch/Python runtime
103+
# to read, so they are not imported directly — export your model to the
104+
# `axiom.pytorch.sequential.v1` descriptor first, then import the .json above.
105105
106106
# Export supported models to ONNX
107107
to_onnx(model, "model.onnx", input_shape=(1, 3, 224, 224))
108108
----
109109

110110
Current scope:
111111

112-
* `from_pytorch(…)`: canonical descriptor import + direct `.pt/.pth/.ckpt` bridge.
112+
* `from_pytorch(…)`: pure-Julia `axiom.pytorch.sequential.v1` descriptor import.
113113
* `to_onnx(…)`: export for `Sequential`/`Pipeline` models built from
114114
Dense/Conv/Norm/Pool + common activations.
115115

@@ -193,8 +193,7 @@ generate_grpc_proto("axiom_inference.proto")
193193

194194
[source,julia]
195195
----
196-
# PyTorch import (checkpoint bridge or canonical descriptor JSON)
197-
model = from_pytorch("model.pt")
196+
# PyTorch import — pure-Julia descriptor JSON (export from PyTorch first)
198197
model = from_pytorch("model.pytorch.json")
199198
200199
# ONNX export (Dense/Conv/Norm/Pool + common activations)

ROADMAP.adoc

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -44,7 +44,7 @@ Must progress snapshot (2026-02-16):
4444
* [x] Added non-GPU accelerator strategy checks and CI coverage: `test/ci/coprocessor_strategy.jl`.
4545
* [x] Added certificate integrity CI checks and digest-report artifacts: `test/ci/certificate_integrity.jl`, `.github/workflows/verify-certificates.yml`.
4646
* [x] Added in-tree gRPC unary protobuf binary-wire support (`application/grpc`) with JSON bridge fallback (`application/grpc+json`).
47-
* [x] Added direct `.pt/.pth/.ckpt` import bridge and expanded ONNX export coverage (Dense/Conv/Norm/Pool + activations).
47+
* [x] Added pure-Julia PyTorch **descriptor** import (`axiom.pytorch.sequential.v1`) and expanded ONNX export coverage (Dense/Conv/Norm/Pool + activations). (Direct `.pt/.pth/.ckpt` binary import is *not* shipped — it would need a PyTorch/Python runtime; export to the descriptor first.)
4848
* [x] Added consolidated readiness gate script for local/CI release checks: `scripts/readiness-check.sh`.
4949
* [x] Added REAL authenticating hybrid Ed448+Dilithium5 (ML-DSA-87) certificate
5050
signing (G01, opt-in): Rust `cdylib` shim in `crypto/` (`pqcrypto-dilithium`
@@ -88,7 +88,7 @@ Could completion gates:
8888

8989
These roadmap promises are still tracked explicitly (with current delivery state):
9090

91-
* [x] `from_pytorch(...)` import API (baseline shipped: descriptor import + direct `.pt/.pth/.ckpt` bridge + CI interop smoke via `scripts/pytorch_to_axiom_descriptor.py`).
91+
* [x] `from_pytorch(...)` import API (shipped: pure-Julia `axiom.pytorch.sequential.v1` descriptor import + CI interop smoke). Direct binary `.pt/.pth/.ckpt` import is out of scope (needs a PyTorch/Python runtime); a native Julia checkpoint reader (zip + pickle) is future work.
9292
* [x] `to_onnx(...)` export API (baseline shipped: Dense/Conv/Norm/Pool + common activations for supported `Sequential`/`Pipeline` models).
9393
* [x] Production-hardened GPU paths across CUDA/ROCm/Metal (baseline shipped: deterministic fallback CI + optional hardware smoke + extension-hook dispatch + device-range guards + runtime self-healing diagnostics + backend-specific performance evidence via `test/ci/gpu_resilience.jl` and `scripts/gpu-performance-evidence.jl`).
9494
* [ ] Non-GPU accelerators (TPU/NPU/PPU/MATH/FPGA/DSP) backend strategy (in progress: targets, detection, compiled dispatch, fallback/strategy CI, capability/evidence reporting, runtime self-healing diagnostics, resilience CI/evidence, and TPU/NPU/DSP/MATH strict-mode gating shipped via `coprocessor_capability_report`, `coprocessor_runtime_diagnostics`, `scripts/coprocessor-evidence.jl`, `test/ci/coprocessor_resilience.jl`, `scripts/coprocessor-resilience-evidence.jl`, `test/ci/tpu_required_mode.jl`, `scripts/tpu-strict-evidence.jl`, `test/ci/npu_required_mode.jl`, `scripts/npu-strict-evidence.jl`, `test/ci/dsp_required_mode.jl`, `scripts/dsp-strict-evidence.jl`, `test/ci/math_required_mode.jl`, and `scripts/math-strict-evidence.jl`; production kernels remain).

src/integrations/interop.jl

Lines changed: 21 additions & 83 deletions
Original file line numberDiff line numberDiff line change
@@ -93,54 +93,13 @@ function _nested_array_f32(value, field_name::AbstractString)
9393
_array_from_row_major(flat, shape)
9494
end
9595

96-
function _default_pytorch_bridge_script()
97-
normpath(joinpath(@__DIR__, "..", "..", "scripts", "pytorch_to_axiom_descriptor.py"))
98-
end
99-
100-
function _run_pytorch_bridge(
101-
input_path::AbstractString;
102-
python_cmd::AbstractString = get(ENV, "AXIOM_PYTHON", "python3"),
103-
bridge_script::AbstractString = _default_pytorch_bridge_script(),
104-
strict::Bool = true,
105-
)
106-
isfile(input_path) || throw(ArgumentError("Checkpoint path does not exist: $(input_path)"))
107-
isfile(bridge_script) || throw(ArgumentError("PyTorch bridge script not found: $(bridge_script)"))
108-
109-
parts = Base.shell_split(String(python_cmd))
110-
isempty(parts) && throw(ArgumentError("`python_cmd` must not be empty"))
111-
112-
tmp_path, tmp_io = mktemp()
113-
close(tmp_io)
114-
tmp_json_path = tmp_path * ".pytorch.json"
115-
mv(tmp_path, tmp_json_path; force=true)
116-
117-
cmd_parts = copy(parts)
118-
append!(cmd_parts, [
119-
String(bridge_script),
120-
"--input", String(input_path),
121-
"--output", String(tmp_json_path),
122-
strict ? "--strict" : "--no-strict",
123-
])
124-
cmd = Cmd(cmd_parts)
125-
126-
stdout_buf = IOBuffer()
127-
stderr_buf = IOBuffer()
128-
ok = success(pipeline(cmd; stdout=stdout_buf, stderr=stderr_buf))
129-
if !ok
130-
stderr_out = String(take!(stderr_buf))
131-
stdout_out = String(take!(stdout_buf))
132-
rm(tmp_json_path; force=true)
133-
throw(ErrorException(
134-
"PyTorch bridge failed for `$(input_path)`.\n" *
135-
"Command: $(cmd)\n" *
136-
(isempty(stdout_out) ? "" : "stdout:\n$(stdout_out)\n") *
137-
(isempty(stderr_out) ? "" : "stderr:\n$(stderr_out)\n")
138-
))
139-
end
140-
141-
isfile(tmp_json_path) || throw(ErrorException("PyTorch bridge did not produce descriptor output: $(tmp_json_path)"))
142-
tmp_json_path
143-
end
96+
# PyTorch import is pure Julia: `from_pytorch` reads the
97+
# `axiom.pytorch.sequential.v1` JSON descriptor directly (see below). The
98+
# previous bridge shelled out to a `python3` script to convert raw
99+
# `.pt/.pth/.ckpt` checkpoints — removed, because Python is banned estate-wide
100+
# and that bridge script never shipped (so the "direct checkpoint" path always
101+
# failed anyway). Reading a raw checkpoint (zip + pickle) natively in Julia is
102+
# tracked as future work; today, export the model to the descriptor first.
144103

145104
function _pytorch_layers(spec::Dict{String, Any})
146105
raw_layers = get(spec, "layers", get(spec, "modules", nothing))
@@ -373,54 +332,33 @@ function _pytorch_module_to_layer(spec::Dict{String, Any}; strict::Bool = true)
373332
end
374333

375334
"""
376-
from_pytorch(path::AbstractString; strict=true, bridge=true, python_cmd="python3", bridge_script=scripts/pytorch_to_axiom_descriptor.py)
377-
378-
Import a model from:
379-
- a PyTorch JSON descriptor (`axiom.pytorch.sequential.v1`)
380-
- or a `.pt`/`.pth`/`.ckpt` checkpoint via the built-in Python bridge script.
335+
from_pytorch(path::AbstractString; strict=true)
381336
382-
Bridge requirements for direct checkpoints:
383-
- Python runtime (`python3` by default, configurable via `python_cmd`)
384-
- `torch` installed in that Python environment
337+
Import a model from a PyTorch **JSON descriptor** (`axiom.pytorch.sequential.v1`).
338+
Pure Julia — no Python runtime, no external bridge.
385339
386340
Supported descriptor format:
387341
- `format = "axiom.pytorch.sequential.v1"`
388342
- `layers = [...]` where each layer is a module object (`Linear`, `ReLU`, etc.)
343+
344+
Raw `.pt`/`.pth`/`.ckpt` checkpoints are **not** loaded directly (that would
345+
require a PyTorch/Python runtime, which is out of scope). Export the model to
346+
the descriptor format first, then pass the resulting `.json` here.
389347
"""
390-
function from_pytorch(
391-
path::AbstractString;
392-
strict::Bool = true,
393-
bridge::Bool = true,
394-
python_cmd::AbstractString = get(ENV, "AXIOM_PYTHON", "python3"),
395-
bridge_script::AbstractString = _default_pytorch_bridge_script(),
396-
)
348+
function from_pytorch(path::AbstractString; strict::Bool = true)
397349
isfile(path) || throw(ArgumentError("PyTorch import path does not exist: $(path)"))
398350
ext = lowercase(splitext(String(path))[2])
399351

400352
if ext in (".pt", ".pth", ".ckpt")
401-
bridge || throw(ArgumentError(
402-
"Direct `.pt/.pth/.ckpt` loading requires bridge support. " *
403-
"Pass `bridge=true` (default) or export JSON descriptor first."
353+
throw(ArgumentError(
354+
"Direct `.pt/.pth/.ckpt` import is not supported (it needs a " *
355+
"PyTorch/Python runtime). Export the model to the " *
356+
"`axiom.pytorch.sequential.v1` JSON descriptor and pass that `.json` " *
357+
"to `from_pytorch` instead."
404358
))
405-
descriptor_path = _run_pytorch_bridge(
406-
path;
407-
python_cmd=python_cmd,
408-
bridge_script=bridge_script,
409-
strict=strict
410-
)
411-
try
412-
return from_pytorch(descriptor_path; strict=strict, bridge=false)
413-
finally
414-
rm(descriptor_path; force=true)
415-
end
416-
end
417-
418-
spec = try
419-
JSON.parse(read(path, String))
420-
catch e
421-
rethrow(e)
422359
end
423360

361+
spec = JSON.parse(read(path, String))
424362
from_pytorch(spec; strict=strict)
425363
end
426364

test/ci/interop_smoke.jl

Lines changed: 15 additions & 38 deletions
Original file line numberDiff line numberDiff line change
@@ -61,45 +61,22 @@ Random.seed!(0xA710)
6161

6262
pt_path = tempname() * ".pth"
6363
write(pt_path, "placeholder")
64-
@test_throws ArgumentError from_pytorch(pt_path; bridge=false)
64+
# Raw .pt/.pth/.ckpt import is unsupported (needs a PyTorch/Python runtime).
65+
@test_throws ArgumentError from_pytorch(pt_path)
6566

66-
bridge_script = tempname() * ".py"
67-
open(bridge_script, "w") do io
68-
write(io, """
69-
import argparse
70-
import json
71-
72-
p = argparse.ArgumentParser()
73-
p.add_argument("--input", required=True)
74-
p.add_argument("--output", required=True)
75-
p.add_argument("--strict", action="store_true")
76-
p.add_argument("--no-strict", action="store_false", dest="strict")
77-
args = p.parse_args()
78-
79-
spec = {
80-
"format": "axiom.pytorch.sequential.v1",
81-
"layers": [
67+
# Pure-Julia descriptor import — no Python, no external bridge.
68+
descriptor_path = tempname() * ".pytorch.json"
69+
write(descriptor_path, """
8270
{
83-
"type": "Linear",
84-
"in_features": 3,
85-
"out_features": 2,
86-
"weight": [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]],
87-
"bias": [0.0, 0.0]
88-
},
89-
{"type": "ReLU"}
90-
]
91-
}
92-
93-
with open(args.output, "w", encoding="utf-8") as f:
94-
json.dump(spec, f)
95-
""")
96-
end
97-
bridged = from_pytorch(
98-
pt_path;
99-
python_cmd="python3",
100-
bridge_script=bridge_script,
101-
strict=true
102-
)
71+
"format": "axiom.pytorch.sequential.v1",
72+
"layers": [
73+
{"type": "Linear", "in_features": 3, "out_features": 2,
74+
"weight": [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]], "bias": [0.0, 0.0]},
75+
{"type": "ReLU"}
76+
]
77+
}
78+
""")
79+
bridged = from_pytorch(descriptor_path; strict=true)
10380
@test bridged isa Axiom.Pipeline
10481

10582
cv_model = Sequential(
@@ -129,7 +106,7 @@ with open(args.output, "w", encoding="utf-8") as f:
129106
rm(spec_path)
130107
rm(onnx_path)
131108
rm(pt_path)
132-
rm(bridge_script)
109+
rm(descriptor_path)
133110
rm(cv_onnx_path_a)
134111
rm(cv_onnx_path_b)
135112
end

test/runtests.jl

Lines changed: 19 additions & 40 deletions
Original file line numberDiff line numberDiff line change
@@ -604,45 +604,24 @@ using JSON
604604

605605
pt_path = tempname() * ".pth"
606606
write(pt_path, "placeholder")
607-
@test_throws ArgumentError from_pytorch(pt_path; bridge=false)
608-
609-
bridge_script = tempname() * ".py"
610-
open(bridge_script, "w") do io
611-
write(io, """
612-
import argparse
613-
import json
614-
615-
p = argparse.ArgumentParser()
616-
p.add_argument("--input", required=True)
617-
p.add_argument("--output", required=True)
618-
p.add_argument("--strict", action="store_true")
619-
p.add_argument("--no-strict", action="store_false", dest="strict")
620-
args = p.parse_args()
621-
622-
spec = {
623-
"format": "axiom.pytorch.sequential.v1",
624-
"layers": [
625-
{
626-
"type": "Linear",
627-
"in_features": 3,
628-
"out_features": 2,
629-
"weight": [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]],
630-
"bias": [0.0, 0.0]
631-
},
632-
{"type": "ReLU"}
633-
]
634-
}
635-
636-
with open(args.output, "w", encoding="utf-8") as f:
637-
json.dump(spec, f)
638-
""")
639-
end
640-
bridged = from_pytorch(
641-
pt_path;
642-
python_cmd="python3",
643-
bridge_script=bridge_script,
644-
strict=true
645-
)
607+
# Raw .pt/.pth/.ckpt import is not supported (it would need a PyTorch/
608+
# Python runtime); from_pytorch rejects it with a clear message.
609+
@test_throws ArgumentError from_pytorch(pt_path)
610+
611+
# Pure-Julia descriptor import: write the axiom.pytorch.sequential.v1
612+
# JSON descriptor directly and load it — no Python, no external bridge.
613+
descriptor_path = tempname() * ".pytorch.json"
614+
write(descriptor_path, """
615+
{
616+
"format": "axiom.pytorch.sequential.v1",
617+
"layers": [
618+
{"type": "Linear", "in_features": 3, "out_features": 2,
619+
"weight": [[1.0, 0.0, 0.0], [0.0, 1.0, 0.0]], "bias": [0.0, 0.0]},
620+
{"type": "ReLU"}
621+
]
622+
}
623+
""")
624+
bridged = from_pytorch(descriptor_path; strict=true)
646625
@test bridged isa Axiom.Pipeline
647626

648627
cv_model = Sequential(
@@ -672,7 +651,7 @@ with open(args.output, "w", encoding="utf-8") as f:
672651
rm(spec_path)
673652
rm(onnx_path)
674653
rm(pt_path)
675-
rm(bridge_script)
654+
rm(descriptor_path)
676655
rm(cv_onnx_path_a)
677656
rm(cv_onnx_path_b)
678657
end

0 commit comments

Comments
 (0)