Skip to content

Allow easier customization of rust plugins#222

Open
nolag wants to merge 11 commits intomainfrom
rtinianov_multi_javy_2
Open

Allow easier customization of rust plugins#222
nolag wants to merge 11 commits intomainfrom
rtinianov_multi_javy_2

Conversation

@nolag
Copy link
Copy Markdown
Contributor

@nolag nolag commented Mar 27, 2026

No description provided.

@nolag nolag force-pushed the rtinianov_multi_javy_2 branch 2 times, most recently from da6d4f8 to 5eb9658 Compare April 8, 2026 13:29
@nolag nolag marked this pull request as ready for review April 8, 2026 13:43
@nolag nolag requested review from a team as code owners April 8, 2026 13:43
@nolag nolag enabled auto-merge April 8, 2026 13:43
Copy link
Copy Markdown
Collaborator

@ernest-nowacki ernest-nowacki left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shared some feedback - my main concern and ask would be to move the examples to their own packages under /packages following monorepo convention and make @chainlink/ paths available to be used in imports.

On top of that I also run claude review on this PR and got the following feedback:


Code Review Report: PR #222

Reviewed by: Security, Performance, Architecture
Files reviewed: 57 | Changes: +1542 / -343


Critical (0)

No critical findings.


High (3)

H1. Cross-package imports via relative filesystem paths break on publish

  • Dimension: Architecture
  • Files: cre-sdk/bin/cre-compile.ts:4, cre-sdk/scripts/src/compile-workflow.ts:4, cre-sdk/scripts/src/compile-to-wasm.ts:5
  • Three files import parseCompileFlags via ../../cre-sdk-javy-plugin/scripts/... relative paths. These won't resolve when packages are installed from npm, breaking cre-compile for end users.
  • Fix: Add a sub-path export in cre-sdk-javy-plugin/package.json and import via @chainlink/cre-sdk-javy-plugin/scripts/parse-compile-flags.

H2. reset_registry() placement defeats duplicate detection between core and extensions

  • Dimension: Architecture
  • Files: generate-host-crate.ts:119-123, javy_chainlink_sdk/src/lib.rs:102-330
  • modify_runtime() runs its own reset_registry() → register core → check_duplicates() cycle. The generated host crate then calls reset_registry() again before extensions, wiping the core export names. An extension registering "log" or "sendResponse" silently overwrites the core binding.
  • Fix: Remove reset_registry()/check_duplicates() from modify_runtime(). Let callers own the lifecycle so the registry spans both core and extension exports.

H3. Ephemeral temp directories discard all cargo compilation cache

  • Dimension: Performance
  • Files: compile-workflow.ts:72-119, build-plugin.ts:51-81
  • Every --cre-exports invocation creates a fresh temp dir, runs cargo build, then deletes everything including target/. Full Rust recompilation (30-90+ sec) on every build with zero cache reuse.
  • Fix: Set CARGO_TARGET_DIR to a stable shared location so compiled dependencies persist. Only delete the generated source files.

Medium (5)

M1. Config duplication between standalone Rust code and generated TypeScript template

  • Dimension: Architecture
  • Files: javy_chainlink_sdk/src/lib.rs:94-98, generate-host-crate.ts:114-115
  • Javy Config (event_loop, text_encoding, promise) is defined in two places that must stay in sync with no mechanism to detect drift.
  • Fix: Export config() from javy_chainlink_sdk and call it from the generated template.

M2. run() helper and Javy version duplicated 3+ times

  • Dimension: Architecture
  • Files: compile-workflow.ts:16, build-plugin.ts:16, compile-javy-sdk-plugin.ts:22
  • Nearly identical process-spawning helpers with minor variations (one uses shell: true). 'v8.1.0' appears in 6 different files.
  • Fix: Extract shared run() and JAVY_VERSION to a common utility module.

M3. Extension crate contract relies on naming convention with no compile-time enforcement

  • Dimension: Architecture
  • File: generate-host-crate.ts:98-99
  • Extensions must expose pub fn register(ctx: &Ctx<'_>) but this is only a convention. Violations produce confusing Rust errors pointing to generated code in a temp directory.
  • Fix: Define a WasmExtension trait in cre_wasm_exports, or at minimum add validation with a clear error message before attempting compilation.

M4. CRE_SDK_JAVY_PLUGIN_HOME env var undocumented with fragile relative-path fallback

  • Dimension: Architecture
  • File: compile-to-wasm.ts:81-83
  • Fallback is ../../../cre-sdk-javy-plugin (monorepo-only). No documentation, no version compatibility check.
  • Fix: Document the env var; replace the relative fallback with require.resolve('@chainlink/cre-sdk-javy-plugin/package.json').

M5. ensureJavy could run in parallel with cargo build

  • Dimension: Performance
  • File: compile-workflow.ts:78-98
  • ensureJavy() (potential download) runs sequentially after cargo build despite being independent.
  • Fix: await Promise.all([cargoPromise, ensureJavy(...)]).

Low (6)

# Dimension Finding File
L1 Correctness Unescaped paths in generated Cargo.toml — backslashes on Windows would break TOML parsing generate-host-crate.ts:79-83
L2 Correctness Temp dirs created under package root (may be read-only in pnpm/npm global installs) compile-workflow.ts:72, build-plugin.ts:51
L3 Architecture WASM output filename guessing with lib prefix fallback is fragile compile-workflow.ts:80-94
L4 Architecture zod added as production dep to javy-plugin for a single runtime utility package.json:32
L5 Performance E2E script steps 5a/5b could be parallelized simulate-rust-inject.sh
L6 Security Box::leak memory leak on error path in await_capabilities (low risk in short-lived WASM) javy_chainlink_sdk/src/lib.rs:139

Summary

Severity Count
Critical 0
High 3
Medium 5
Low 6
Dismissed 5
Active 14

Top priorities before merge:

  1. H1 — relative imports will break published packages
  2. H2reset_registry() placement defeats the duplicate detection safety net
  3. H3 — cargo cache destruction makes --cre-exports unusably slow for iterative development

The overall design is solid. The --plugin / --cre-exports split, cre_wasm_exports shared crate, and createExtensionAccessor pattern are well-conceived.

var rustAlpha: RustAlpha
}

// biome-ignore lint/suspicious/noRedeclare: global augmentation declares rustAlpha; this export is the validated accessor
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see

"Shouldn't redeclare 'rustAlpha'. Consider to delete it or rename it.biomelint/suspicious/noRedeclare" if I remove this one.

@@ -0,0 +1,19 @@
{
"name": "@chainlink/cre-rust-inject-alpha",
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: this probably doesn't matter for the purpose of this demo, but monorepo would only be able to resolve the path @chainlink/cre-rust-inject-alpha if it lives under /packages.

Copy link
Copy Markdown
Contributor Author

@nolag nolag Apr 10, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm a bit confused by that, I have this library packed and imported in the test and it works.

I'll tell cursor your comment and see what it does with it.

var rustBeta: RustBeta
}

// biome-ignore lint/suspicious/noRedeclare: global augmentation declares rustBeta; this export is the validated accessor
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// biome-ignore lint/suspicious/noRedeclare: global augmentation declares rustBeta; this export is the validated accessor
// global augmentation declares rustBeta; this export is the validated accessor

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above, I see an error in Cursor without it.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm seeing bun.lock here makes me think we actually are treating this as other package (like cre-sdk-examples itself). In that case I would try to make it clear and follow monorepo structure correctly - by moving it to top level (/packages/*).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we are, but it's only meant to be used by the test. It's demonstrating how you can import a package with rust in it (in two different ways).

I'll find time for us to sync, then move files after.

@@ -0,0 +1,24 @@
import { CronCapability, handler, Runner, type Runtime } from '@chainlink/cre-sdk'
import { z } from 'zod'
import { rustAlpha } from '../lib_alpha'
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would be able to reference it here as @chainlink/cre-rust-inject-alpha leveraging monorepo capabilities.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same deal - feels like all of those mini apps should just be their own directories inside /packages

Comment on lines +3 to +4
import { rustAlpha } from '../lib_alpha'
import { rustBeta } from '../lib_beta'
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@chainlink/ imports unlocked after moving to the packages structure.

"dependencies": {
"@bufbuild/protobuf": "2.6.3",
"@chainlink/cre-sdk": "workspace:*",
"@chainlink/cre-sdk-javy-plugin": "workspace:*",
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering while this was still a draft why we need that.

I guess now when user can extend the rust code that make more sense to list as dependency. Previously the goal was to make it invisible. Make user care only about the cre-sdk itself.

Wonder if we could actually still have it via Proxying some of the Javy parts through sdk? 🤔

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need them to be able to see it. They need to lock to an exact version of the javy plugin. That will mean changes we make to the SDK w/o the javy plugin won't require them to re-build their pre-built plugins.

@nolag
Copy link
Copy Markdown
Contributor Author

nolag commented Apr 10, 2026

Shared some feedback - my main concern and ask would be to move the examples to their own packages under /packages following monorepo convention and make @chainlink/ paths available to be used in imports.

Do you mean moving rust-inject to src/workflows? These aren't examples, they are tests. I had issues when it was in the same area because these use their own node_modules. I needed to install instead of using relative paths to ensure the scripts worked correctly with our distributions. I could update all the examples to do that then move it, would that be better?

}
`

writeFileSync(join(outDir, 'Cargo.toml'), cargoToml)
`

writeFileSync(join(outDir, 'Cargo.toml'), cargoToml)
writeFileSync(join(outDir, 'src', 'lib.rs'), libRs)
// even when the generated crate lives outside the repo (e.g. /tmp on CI).
const toolchainSrc = resolve(realPluginDir, 'src', 'javy_chainlink_sdk', 'rust-toolchain.toml')
if (existsSync(toolchainSrc)) {
writeFileSync(join(outDir, 'rust-toolchain.toml'), readFileSync(toolchainSrc))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants