Skip to content

feat: nodes chart#8

Merged
roderik merged 17 commits intomainfrom
feat/nodes
Sep 17, 2025
Merged

feat: nodes chart#8
roderik merged 17 commits intomainfrom
feat/nodes

Conversation

@roderik
Copy link
Copy Markdown
Member

@roderik roderik commented Sep 17, 2025

  • Updated biome.jsonc to include linter rules for suspicious code.
  • Adjusted package.json to ensure consistent command execution.
  • Enhanced README.md with additional options for CLI commands.
  • Modified .vscode/settings.json to enforce formatting on save and paste.
  • Updated Helm chart files for network-bootstrapper, including values and templates for deployment.
  • Refined test cases in build-command.test.ts and output.test.ts to align with new configurations.

Summary by Sourcery

Add a new ‘network-nodes’ Helm chart and update the CLI to use a dedicated generate subcommand while removing RPC node support, and adjust scripts, tests, and documentation accordingly.

New Features:

  • Add a generate subcommand to the CLI and remove the RPC node count option from the root command
  • Introduce a new umbrella Helm chart “network” that includes “network-bootstrapper” and a new “network-nodes” chart for deploying validator and RPC StatefulSets

Enhancements:

  • Refactor the CLI build command and output module to drop RPC node handling and streamline spec generation
  • Update package.json helm and packaging scripts to point at the new chart names and paths

Build:

  • Add biome.jsonc linter rules and .vscode settings to enforce formatting on save

Documentation:

  • Revise README.md to reflect the new CLI command structure and document the umbrella and nodes charts

Tests:

  • Adjust unit tests for the CLI and output modules to remove RPC node expectations and target the generate subcommand
  • Add Helm test hooks and templates in the network-nodes chart

- Updated biome.jsonc to include linter rules for suspicious code.
- Adjusted package.json to ensure consistent command execution.
- Enhanced README.md with additional options for CLI commands.
- Modified .vscode/settings.json to enforce formatting on save and paste.
- Updated Helm chart files for network-bootstrapper, including values and templates for deployment.
- Refined test cases in build-command.test.ts and output.test.ts to align with new configurations.
@github-actions github-actions Bot added the qa:running QA workflow is currently running label Sep 17, 2025
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Sep 17, 2025

To view in Slack, search for: 1758112557.400819

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary of Changes

Hello @roderik, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors the project's Helm charts and command-line interface to streamline the deployment and management of Besu-based blockchain networks. It introduces a new, more modular Helm chart structure that separates the network bootstrapping logic from the actual node deployment, enhancing organization and maintainability. Concurrently, the CLI has been simplified by removing the explicit distinction between RPC and validator nodes during the generation process, focusing solely on validator nodes. These changes aim to improve the clarity, flexibility, and overall developer experience for deploying and configuring blockchain networks on Kubernetes.

Highlights

  • CLI Command Refactoring: The network-bootstrapper CLI command has been refactored to introduce a generate subcommand, making the CLI more modular. The concept of rpcNodes has been removed from the CLI options and internal logic, simplifying node generation to focus solely on validators.
  • Helm Chart Restructuring: The existing network-bootstrapper Helm chart has been renamed and moved to become a subchart within a new top-level network Helm chart. This new structure provides a more organized way to deploy blockchain networks.
  • New network-nodes Helm Subchart: A new Helm subchart named network-nodes has been introduced. This subchart includes templates for deploying Besu nodes (both RPC and validator types) as StatefulSets, along with associated services, configmaps, and ingress/route configurations, allowing for flexible node deployment.
  • Development Tooling Updates: biome.jsonc was updated to include linter rules for suspicious code and exclude .yaml files from linting. Additionally, .vscode/settings.json was modified to disable Prettier and enforce formatting on save and paste, aligning with Biome's formatting practices.
  • README and package.json Updates: The README.md has been updated to reflect the new CLI usage, including the generate subcommand and the removal of RPC node options. Correspondingly, package.json scripts have been adjusted to point to the new Helm chart structure and CLI command for packaging and deployment.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

@github-actions github-actions Bot added status:ready-for-review Pull request is ready for review qa:success QA workflow passed successfully and removed qa:running QA workflow is currently running labels Sep 17, 2025
Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a significant and well-structured refactoring by creating a new parent Helm chart named network and updating the CLI to use a generate subcommand. These changes improve the project's modularity and extensibility. However, I've identified a critical issue where the new StatefulSet for RPC nodes depends on secrets that are no longer generated by the bootstrapper, which will prevent the RPC nodes from deploying successfully. Additionally, there are a few other issues related to incorrect health probes, insecure default configurations, and dependency management in the Helm charts that should be addressed.

Comment on lines +160 to +173
- name: besu-private-key
projected:
{{- if gt $rpcReplicaBudget 0 }}
sources:
{{- range $i, $_ := until $rpcReplicaBudget }}
- secret:
name: {{ printf "besu-node-rpc-node-%d-private-key" (add $i 1) }}
items:
- key: privateKey
path: {{ printf "%s-rpc-%d/privateKey" (include "nodes.fullname" $root) $i }}
{{- end }}
{{- else }}
sources: []
{{- end }}
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

This StatefulSet for RPC nodes is configured to mount private keys from secrets named besu-node-rpc-node-<n>-private-key. However, the bootstrapper tool has been updated in this PR to no longer generate keys or secrets for RPC nodes. This will cause the RPC node pods to fail during startup as the required secrets will be missing. You should either update the bootstrapper to generate these keys again or remove the private key dependency from the RPC nodes if they are not required to have a unique identity.

Comment on lines +253 to +260
livenessProbe:
httpGet:
path: /
port: http
readinessProbe:
httpGet:
path: /
port: http
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The liveness and readiness probes are configured to use a port named http, but no such port is defined in the service or statefulset templates. The RPC port is named json-rpc. This mismatch will cause the probes to fail.

Additionally, checking the root path (/) is a very basic health check. Besu exposes more specific health endpoints. It's recommended to use more robust health checks by targeting Besu's built-in health endpoints like /liveness and /readiness.

livenessProbe:
  httpGet:
    path: /liveness
    port: json-rpc
readinessProbe:
  httpGet:
    path: /readiness
    port: json-rpc

Comment thread charts/network/Chart.yaml
Comment on lines +12 to +15
- name: network-bootstrapper
version: "*"
- name: network-nodes
version: "*"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Using a wildcard * for dependency versions is not recommended as it can introduce breaking changes unexpectedly. It is best practice to pin dependencies to a specific chart version or use a version range (e.g., ~0.1.0 or ^0.1.0) to ensure stable and predictable deployments.

dependencies:
  - name: network-bootstrapper
    version: "0.1.0"
  - name: network-nodes
    version: "0.1.0"

- PLUGINS
corsOrigins:
- all
authenticationEnabled: false
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Disabling authentication for HTTP-RPC (http.authenticationEnabled: false) and WebSocket-RPC (ws.authenticationEnabled: false on line 135) can pose a security risk, especially if these endpoints are exposed externally. While this might be acceptable for local development, it's recommended to enable authentication for any production-like environment. Consider setting the default to true or adding a prominent warning in the documentation about the security implications of leaving it disabled.

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey there - I've reviewed your changes - here's some feedback:

  • This PR bundles a major CLI refactor (removing the rpc-nodes option and adding a subcommand) with extensive Helm chart restructuring—consider splitting those into separate PRs to simplify review and minimize risk.
  • The removal of the --rpc-nodes flag is a breaking change; consider providing a deprecation alias or migration notice so existing users aren’t surprised.
  • After splitting the charts into network, network-bootstrapper, and network-nodes, double-check CI/CD scripts and README entries to ensure the new chart paths and dependencies are correctly referenced.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- This PR bundles a major CLI refactor (removing the rpc-nodes option and adding a subcommand) with extensive Helm chart restructuring—consider splitting those into separate PRs to simplify review and minimize risk.
- The removal of the `--rpc-nodes` flag is a breaking change; consider providing a deprecation alias or migration notice so existing users aren’t surprised.
- After splitting the charts into `network`, `network-bootstrapper`, and `network-nodes`, double-check CI/CD scripts and README entries to ensure the new chart paths and dependencies are correctly referenced.

## Individual Comments

### Comment 1
<location> `charts/network/charts/network-nodes/templates/configmap.yaml:15` </location>
<code_context>
+{{- $dataPath = $mountPath }}
+{{- end }}
+data:
+  config.toml: |
+    genesis-file = "/etc/besu/genesis.json"
+    logging = "{{ .Values.config.logging }}"
</code_context>

<issue_to_address>
**nitpick:** ConfigMap template uses mustToJson for lists; ensure compatibility with Besu config format.

Verify that mustToJson outputs list and array values in a format compatible with Besu's config requirements.
</issue_to_address>

### Comment 2
<location> `src/cli/build-command.test.ts:364` </location>
<code_context>
     };

     const command = createCliCommand(deps);
+    const generate = command.commands.find(
+      (child) => child.name() === "generate"
+    );
</code_context>

<issue_to_address>
**issue (complexity):** Consider extracting the generate command builder and handler into a separate exported function to simplify test setup and invocation.

You can avoid all of the “find the nested generate sub‐command” and per‐child exitOverride() boilerplate in your tests by pulling the generate builder (and its handler) out into its own exported function.  Then tests can import and drive just that one command directly.

--- build-command.ts ---
```ts
import yargs from "yargs";
import type { Argv } from "yargs";
import type { BootstrapDependencies, CliOptions } from "./build-command";
import { runBootstrap } from "./build-command";

// 1) extract generate builder + handler
export function buildGenerateCommand(y: Argv<CliOptions>) {
  return y
    .scriptName("network-bootstrapper generate")
    .option("validators", {
      type: "number",
      describe: "number of validator nodes",
      default: 4,
    })
    // … all of your other options here …
    .option("outputType", {
      choices: ["screen", "file", "kubernetes"] as const,
      describe: "where to send output",
      default: "screen",
      coerce: (val) => val.replace(/^"(.*)"$/, "$1"),
    })
    .strict();
}

export function createCliCommand(deps?: BootstrapDependencies) {
  return yargs<CliOptions>()
    .scriptName("network-bootstrapper")
    .command(
      "generate",
      "Generate keys, genesis and allocations",
      (y) => buildGenerateCommand(y),
      (argv) => runBootstrap(argv, deps!)
    )
    .help();
}
```

--- in your tests ---
```ts
import yargs from "yargs";
import { buildGenerateCommand } from "./build-command";

// now tests only the one command
test("rejects bad numeric args", async () => {
  const cmd = buildGenerateCommand(yargs().exitProcess(false));
  cmd.exitOverride();

  await expect(
    cmd.parseAsync(["--chain-id", "0"])
  ).rejects.toThrow("Chain ID must be a positive integer.");
});

test("strips quotes and sets outputType", async () => {
  let captured: string | undefined;
  // stub deps so handler doesn't actually run
  const fakeDeps = { /**/ };
  // attach a fake handler in your test
  const cmd = buildGenerateCommand(
    yargs().exitProcess(false).middleware((argv) => { captured = argv.outputType; })
  );
  cmd.exitOverride();

  await expect(cmd.parseAsync(['--outputType="kubernetes"'])).resolves.toBeDefined();
  expect(captured).toBe("kubernetes");
});
```

Benefits:
- no more walking command.commands
- no more per‐child exitOverride()
- tests import exactly the bits they care about
- coverage is identical, functionality stays the same.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

{{- $dataPath = $mountPath }}
{{- end }}
data:
config.toml: |
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nitpick: ConfigMap template uses mustToJson for lists; ensure compatibility with Besu config format.

Verify that mustToJson outputs list and array values in a format compatible with Besu's config requirements.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Codex Review: Here are some suggestions.

Reply with @codex fix comments to fix any unresolved comments.

About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you open a pull request for review, mark a draft as ready, or comment "@codex review". If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex fix this CI failure" or "@codex address that feedback".

Comment thread src/cli/build-command.ts
Comment on lines 162 to 166
const payload: OutputPayload = {
faucet,
genesis,
rpcNodes,
validators,
};
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[P1] Generate command no longer creates RPC node secrets

The generate action removes all handling of RPC nodes and only produces validator keys (e.g. CliOptions and runBootstrap now omit the rpcNodes count, and the payload at lines 19‑166 includes only validators). However, the newly added network-nodes chart still mounts secrets named besu-node-rpc-node-<n>-private-key for every replica (charts/network/charts/network-nodes/templates/statefulset-rpc.yaml lines 160‑168 with default rpcReplicaCount: 2). Deploying the chart will therefore create RPC pods that reference non‑existent secrets and fail to start. Either restore RPC generation in the CLI/job or make the chart tolerant of no RPC keys being generated.

Useful? React with 👍 / 👎.

Comment thread src/cli/output.ts
Comment on lines 16 to 17
validators: readonly IndexedNode[];
};
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Potential bug: The CLI no longer generates secrets for RPC nodes, but the Helm chart's RPC StatefulSet still tries to mount them, causing deployment failure.
  • Description: The CLI tool's logic for generating secrets for RPC nodes was removed in src/cli/output.ts. However, the Helm chart still contains the statefulset-rpc.yaml template, which attempts to deploy RPC nodes by default. These nodes are configured to mount secrets (e.g., besu-node-rpc-node-&lt;n&gt;-private-key) that are no longer being created. This mismatch will cause the RPC pods to fail during startup with "secret not found" errors, preventing the RPC node component from becoming operational.

  • Suggested fix: To resolve this, either remove the RPC StatefulSet template (charts/network/charts/network-nodes/templates/statefulset-rpc.yaml) and its associated configurations from values.yaml, or reinstate the RPC node secret generation logic in the CLI tool.
    severity: 0.85, confidence: 0.95

Did we get this right? 👍 / 👎 to inform future reviews.

- Updated README.md to include additional options for the generate command, detailing parameters for static nodes and consensus settings.
- Refined Helm chart files for network-bootstrapper, ensuring accurate configuration for static node settings and genesis parameters.
- Adjusted test cases in build-command.test.ts to align with new CLI options and expected outputs.
@github-actions github-actions Bot added qa:running QA workflow is currently running qa:success QA workflow passed successfully and removed qa:success QA workflow passed successfully qa:running QA workflow is currently running labels Sep 17, 2025
- Enhanced README.md to include detailed descriptions of new CLI options for static node configuration, including domain, namespace, and ports.
- Updated Helm chart files to ensure accurate representation of static node settings in the network-bootstrapper.
- Adjusted test cases in build-command.test.ts to reflect the new CLI options and expected outputs.
@github-actions github-actions Bot added qa:running QA workflow is currently running qa:success QA workflow passed successfully and removed qa:success QA workflow passed successfully qa:running QA workflow is currently running labels Sep 17, 2025
- Updated the `deriveNodeId` function to correctly handle public key prefixes and lengths.
- Adjusted test cases in `build-command.test.ts` and `output.test.ts` to reflect changes in node ID derivation logic.
- Ensured consistency in handling public keys across CLI commands.
@github-actions github-actions Bot added qa:running QA workflow is currently running qa:success QA workflow passed successfully and removed qa:success QA workflow passed successfully qa:running QA workflow is currently running labels Sep 17, 2025
- Enhanced `values.yaml` to include default Prometheus annotations for pod metrics scraping.
- Updated liveness and readiness probe configurations to use specific paths and ports for the Besu application.
- Ensured consistency in pod annotations and labels for Kubernetes deployments.
- Modified `README.md` to reflect changes in liveness and readiness probe configurations, including updated paths and ports for the Besu application.
- Enhanced pod annotations for Prometheus metrics scraping.
- Ensured consistency in Helm chart values for network nodes, including ingress and persistence settings.
- Updated `values.yaml` to change the validator pod secret naming from 1-indexed to 0-indexed.
- Adjusted related references in `build-command.ts` and `output.test.ts` to align with the new naming convention for validator secrets.
- Ensured consistency across the Helm chart for network nodes regarding secret management.
- Corrected the formatting of configuration values in `README.md` for network nodes, ensuring clarity and consistency.
- Adjusted the `fullnameOverride` value to reflect the correct naming convention.
- Ensured that all configuration options are accurately documented for user reference.
- Updated the Chart.lock file to reflect the latest dependencies for the network-bootstrapper and network-nodes.
- Ensured that the lock file is in sync with the current dependency versions and configurations.
- Updated the Chart.lock file to ensure it reflects the latest dependencies for the network-bootstrapper and network-nodes.
- Adjusted service configurations in `values.yaml` to correct the `fullnameOverride` for the bootstrapper and ensure consistency in service types across templates.
- Made minor formatting adjustments in service templates to align with the updated configurations.
… configuration values

- Corrected formatting and descriptions of configuration values in `README.md` for both network-bootstrapper and network-nodes.
- Ensured clarity and consistency in the documentation of parameters such as `clusterDomain`, `defaultStaticNodeDiscoveryPort`, and `fullnameOverride`.
- Updated image settings to reflect accurate container image configurations for the network bootstrapper workload.
@github-actions github-actions Bot added qa:running QA workflow is currently running qa:success QA workflow passed successfully and removed qa:success QA workflow passed successfully qa:running QA workflow is currently running labels Sep 17, 2025
…nodes

- Revised comments and descriptions in `values.yaml` for the network-bootstrapper to enhance clarity and consistency.
- Updated the structure of configuration values to align with best practices, ensuring accurate documentation for users.
- Made minor adjustments to the `statefulset-validator.yaml` template to address YAML parsing errors and improve overall configuration integrity.
…nd network-nodes

- Revised the README.md to improve clarity and consistency in the documentation of configuration values for both network-bootstrapper and network-nodes.
- Updated the structure of configuration values in values.yaml to align with best practices, ensuring accurate documentation for users.
- Made minor adjustments to enhance the overall integrity and readability of the configuration files.
- Added a comment in `values.yaml` to clarify that the parent chart does not define additional values and to guide users on overriding subchart defaults.
- Made minor adjustments to the `README.md` for network nodes to ensure clarity and consistency in the documentation of configuration values.
- Enhanced overall integrity and readability of the configuration files.
@github-actions github-actions Bot added qa:running QA workflow is currently running qa:success QA workflow passed successfully and removed qa:success QA workflow passed successfully qa:running QA workflow is currently running labels Sep 17, 2025
- Revised the README.md to improve clarity and consistency in the documentation of configuration values for the network-bootstrapper.
- Updated the structure of configuration values in values.yaml to align with best practices, ensuring accurate documentation for users.
- Enhanced overall integrity and readability of the configuration files, including detailed descriptions for settings like `clusterDomain`, `defaultStaticNodeDiscoveryPort`, and `fullnameOverride`.
@github-actions github-actions Bot added qa:running QA workflow is currently running qa:success QA workflow passed successfully and removed qa:success QA workflow passed successfully qa:running QA workflow is currently running labels Sep 17, 2025
- Fixed a YAML parsing error in the `test-connection.yaml` template for network nodes, ensuring proper structure and syntax.
- Updated the pod configuration to correctly reference the service port and include necessary annotations for Helm hooks.
…onfiguration

- Revised the README.md to enhance clarity and consistency in documenting configuration values for the network-bootstrapper, including detailed descriptions for `clusterDomain`, `defaultStaticNodeDiscoveryPort`, and `fullnameOverride`.
- Updated the structure of configuration values in values.yaml to align with best practices, ensuring accurate documentation for users.
- Improved overall integrity and readability of the configuration files, addressing YAML parsing errors and enhancing user guidance.
@github-actions github-actions Bot added qa:running QA workflow is currently running qa:success QA workflow passed successfully feat New feature and removed qa:success QA workflow passed successfully qa:running QA workflow is currently running labels Sep 17, 2025
@roderik roderik merged commit 3ad607e into main Sep 17, 2025
7 checks passed
@roderik roderik deleted the feat/nodes branch September 17, 2025 15:42
@github-actions github-actions Bot added status:merged Pull request has been merged and removed status:ready-for-review Pull request is ready for review labels Sep 17, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feat New feature qa:success QA workflow passed successfully status:merged Pull request has been merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant