feat: fork mainnet/testnet chain data into the local devnet

.changeset/devnet-fork.md

@@ -0,0 +1,10 @@
+---
+'@offckb/cli': minor
+---
+
+Add `offckb devnet fork` to fork an existing mainnet/testnet data directory into the local devnet ([Devnet From Existing Data](https://docs.nervos.org/docs/node/devnet-from-existing-data) flow), plus fork-aware system scripts and local-first debugging.
+
+- `offckb devnet fork --from <dir> [--source mainnet|testnet] [--spec-file <path>] [--force]` copies the source chain data, imports the matching chain spec, patches it for local mining (Dummy PoW, `cellbase_maturity = 0`, correct `genesis_epoch_length` per chain), verifies the genesis hash, and records the fork state. The first `offckb node` run boots with `--skip-spec-check --overwrite-spec` automatically; `offckb clean` resets back to a pure devnet.
+- Devnet system scripts now self-identify the chain via the genesis hash in `ckb list-hashes`: on a mainnet/testnet fork, genesis scripts come from the chain's own spec and post-genesis deployments (sudt/xudt/omnilock/spore/…) are filled from the well-known static records, so `system-scripts`, transfers, deploys and fee estimation keep working on a fork.
+- The devnet ccc client follows the forked chain too: a mainnet fork uses the `ckb` address prefix.
+- `offckb debug` falls back to fetching the transaction from the node when it is not in the local proxy cache, and the tx dumper now embeds full header objects in `mock_info.header_deps` (previously bare hashes, which broke debugging for transactions with header deps).

README.md

@@ -18,7 +18,7 @@ CKB local development network for your first try.
 
 There are BREAKING CHANGES between v0.3.x and v0.4.x, make sure to read the [migration guide](/docs/migration.md) before upgrading.
 
-----
+---
 
 - [OffCKB](#offckb)
 - [Install](#install)
@@ -92,7 +92,7 @@ _Use `offckb [command] -h` to learn more about a specific command._
 ## Get started
 
 ### 1. Run a Local CKB Devnet {#running-ckb}
-    
+
 Start a local blockchain with one command:
 
 ```sh
@@ -151,7 +151,7 @@ offckb node --daemon --json
 Each log line is a single JSON object:
 
 ```json
-{"level":"info","message":"Launching CKB devnet Node...","timestamp":"2026-07-07T07:10:00.000Z"}
+{ "level": "info", "message": "Launching CKB devnet Node...", "timestamp": "2026-07-07T07:10:00.000Z" }
 ```
 
 **RPC & Proxy RPC**
@@ -163,23 +163,26 @@ When the Devnet starts:
 
 The proxy RPC server forwards all requests to the RPC server and record every requests while automatically dumping failed transactions for easier debugging.
 
-You can also start a proxy RPC server for public networks: 
+You can also start a proxy RPC server for public networks:
 
 ```sh
 offckb node --network <testnet or mainnet>
 ```
+
 Using a proxy RPC server for Testnet/Mainnet is especially helpful for debugging transactions, since failed transactions are dumped automatically.
 
 **Watch Network with TUI**
 
 Once you start the CKB Node, you can use `offckb status --network devnet/testnet/mainnet` to start a CKB-TUI interface to monitor the CKB network from your node.
 
 ### 2. Create a New Contract Project {#create-project}
-    
+
 Generate a ready-to-use smart-contract project in JS/TS using templates:
+
 ```sh
 offckb create <your-project-name> -c <your-contract-name>
 ```
+
 - The `-c` option is optional, if not provided, the contract name defaults to `hello-world`.
 
 **Note for Windows Users:**
@@ -210,10 +213,11 @@ To run mock tests in the generated project, you need to manually install `ckb-de
 After completing these steps, `npm run test` should pass without mock test failures.
 
 ### 3. Deploy Your Contract {#deploy-contract}
-    
+
 ```sh
 offckb deploy --network <devnet/testnet> --target <path-to-your-contract-binary-file-or-folder> --output <output-folder-path>
 ```
+
 - Deployment info is written to the `output-folder-path` you specify.
 
 **Upgradable Scripts with `--type-id`**
@@ -224,14 +228,14 @@ offckb deploy --type-id --network <devnet/testnet>
 ```
 
 - **Important**: Upgrades are keyed by the contract‘s artifact name.
-    - If you plan to upgrade with `--type-id`, do not rename your contract artifact (e.g. keep `hello-world.bc`).
-    - Renaming it makes the offckb unable to find the previous Type ID info from the `output-folder-path` and will create a new Type ID.
+  - If you plan to upgrade with `--type-id`, do not rename your contract artifact (e.g. keep `hello-world.bc`).
+  - Renaming it makes the offckb unable to find the previous Type ID info from the `output-folder-path` and will create a new Type ID.
 
 ### 4. Debug Your Contract {#debug-contract}
-    
-When you interact with the CKB Devnet through the Proxy RPC server (localhost:28114), any failed transactions are automatically dumped and recorded for debugging. 
-    
-**Debug a Transaction:** 
+
+When you interact with the CKB Devnet through the Proxy RPC server (localhost:28114), any failed transactions are automatically dumped and recorded for debugging.
+
+**Debug a Transaction:**
 
 ```sh
 offckb debug --tx-hash <transaction-hash> --network <devnet/testnet>
@@ -283,9 +287,9 @@ offckb debug --tx-hash <tx-hash> --single-script input[0].lock
 ```
 
 All debug utilities are powered by [ckb-debugger](https://github.com/nervosnetwork/ckb-standalone-debugger/tree/develop/ckb-debugger).
-    
+
 ### 5. Explore Built-in Scripts {#explore-scripts}
-    
+
 Print all the predefined Scripts for the local blockchain:
 
 ```sh
@@ -311,9 +315,9 @@ offckb system-scripts --export-style ccc
 ```sh
 offckb system-scripts --output <output-file-path>
 ```
-    
+
 ### 6. Tweak Devnet Config {#tweak-devnet-config}
-    
+
 By default, OffCKB use a fixed Devnet config. You can customize it, for example by modifying the default log level (`warn,ckb-script=debug`).
 
 1. Open the interactive Devnet config editor:
@@ -370,6 +374,26 @@ Pay attention to the `devnet.configPath` and `devnet.dataPath`.
 2. After modifications, run `offckb clean -d` to remove the chain data if needed while keeping the updated config files.
 3. Restart local blockchain by running `offckb node`
 
+### 7. Fork Mainnet/Testnet Into Your Devnet {#fork-devnet}
+
+You can fork an existing Mainnet/Testnet data directory into your local devnet, so it keeps the real on-chain state (deployed contracts, cells) while mining locally with Dummy PoW. This implements the same flow as [Devnet From Existing Data](https://docs.nervos.org/docs/node/devnet-from-existing-data).
+
+```sh
+offckb devnet fork --from /path/to/ckb-data
+offckb node
+```
+
+- `--from` points at the directory the source node runs with (`-C`), which must contain `data/db`. Stop the source node first.
+- The source chain is auto-detected from the source `ckb.toml`; pass `--source mainnet|testnet` when it cannot be detected, and `--spec-file <path>` to use a local chain spec (e.g. offline).
+- The command copies the chain `data/` (your original data is never modified), imports the matching chain spec, patches it for local mining (Dummy PoW, `cellbase_maturity = 0`), and verifies the genesis hash.
+- The first `offckb node` run automatically boots with `--skip-spec-check --overwrite-spec`; later runs are normal.
+- Forking replaces the current devnet; use `--force` to replace an existing devnet/fork, or `offckb clean` to reset back to a pure devnet.
+
+On a forked devnet, `offckb system-scripts`, transfers, deploys and `offckb debug --tx-hash <hash>` work against the real source-chain state, e.g. debugging a failed mainnet transaction fully locally.
+
+> [!CAUTION]
+> CKB transactions carry no chain id, so a transaction built on a mainnet fork that spends copied mainnet cells is also valid on mainnet (CKB provides no replay protection). offckb's own flows only use dev keys and fork-mined cells, which cannot replay. Never sign transactions with real mainnet keys against a fork unless you intend to broadcast them yourself.
+
 ## Config Setting
 
 ### List All Settings
@@ -455,4 +479,3 @@ npm install -g @offckb/cli
 ## Contributing
 
 check [development doc](/docs/develop.md)
-

src/cli.ts

@@ -12,6 +12,7 @@ import { udtIssue, udtDestroy, UdtIssueOption, UdtDestroyOption } from './cmd/ud
 import { createScriptProject, CreateScriptProjectOptions } from './cmd/create';
 import { Config, ConfigItem } from './cmd/config';
 import { devnetConfig } from './cmd/devnet-config';
+import { devnetFork } from './cmd/devnet-fork';
 import { debugSingleScript, debugTransaction, parseSingleScriptOption } from './cmd/debug';
 import { printSystemScripts } from './cmd/system-scripts';
 import { transferAll } from './cmd/transfer-all';
@@ -234,6 +235,15 @@ devnetCommand
   )
   .action(devnetConfig);
 
+devnetCommand
+  .command('fork')
+  .description('Fork an existing mainnet/testnet chain data directory into the local devnet')
+  .requiredOption('--from <dir>', 'Path to the source CKB node directory (the one passed to ckb -C)')
+  .option('--source <source>', 'Source chain: mainnet or testnet (auto-detected from the source ckb.toml when omitted)')
+  .option('--spec-file <path>', 'Use a local chain spec file instead of downloading it')
+  .option('--force', 'Replace the existing devnet (or a previous fork)')
+  .action(devnetFork);
+
 program.parse(process.argv);
 
 // If no command is specified, display help

src/cmd/debug.ts

@@ -6,6 +6,7 @@ import path from 'path';
 import { cccA } from '@ckb-ccc/core/advanced';
 import { Network } from '../type/base';
 import { encodeBinPathForTerminal } from '../util/encoding';
+import { callJsonRpc } from '../util/json-rpc';
 import { logger } from '../util/logger';
 
 export async function debugTransaction(txHash: string, network: Network) {
@@ -87,15 +88,33 @@ export async function buildTxFileOptionBy(txHash: string, network: Network) {
   if (!fs.existsSync(outputFilePath)) {
     const rpc = settings[network].rpcUrl;
     const txJsonFilePath = buildTransactionJsonFilePath(network, txHash);
-    if (!fs.existsSync(outputFilePath)) {
-      fs.mkdirSync(path.dirname(outputFilePath), { recursive: true });
+    if (!fs.existsSync(txJsonFilePath)) {
+      await fetchTransactionIntoCache(rpc, txHash, txJsonFilePath);
     }
     await dumpTransaction({ rpc, txJsonFilePath, outputFilePath });
   }
   const opt = `--tx-file ${encodeBinPathForTerminal(outputFilePath)}`;
   return opt;
 }
 
+// Fallback for transactions that never went through the local RPC proxy
+// (e.g. historical transactions on a forked devnet): pull the transaction
+// from the node and cache it in the same JSON-RPC format the proxy stores.
+async function fetchTransactionIntoCache(rpc: string, txHash: string, txJsonFilePath: string) {
+  logger.info(`Transaction ${txHash} not found in local cache, fetching from ${rpc} ..`);
+  const result = await callJsonRpc(rpc, 'get_transaction', [txHash]).catch((error: Error) => {
+    throw new Error(`Failed to fetch transaction ${txHash} from ${rpc}: ${error.message}`);
+  });
+  if (!result?.transaction) {
+    throw new Error(
+      `Transaction ${txHash} not found on ${rpc}. ` +
+        `Check the hash and the --network option, or send the transaction through the offckb RPC proxy first.`,
+    );
+  }
+  fs.mkdirSync(path.dirname(txJsonFilePath), { recursive: true });
+  fs.writeFileSync(txJsonFilePath, JSON.stringify(result.transaction, null, 2));
+}
+
 export function buildTransactionJsonFilePath(network: Network, txHash: string) {
   const settings = readSettings();
   if (network === Network.devnet) {

src/cmd/devnet-fork.ts

@@ -0,0 +1,16 @@
+import { forkDevnet, ForkOptions } from '../devnet/fork';
+import { logger } from '../util/logger';
+
+export async function devnetFork(options: ForkOptions) {
+  if (options.source && options.source !== 'mainnet' && options.source !== 'testnet') {
+    logger.error(`Invalid --source value: ${options.source}. Expected mainnet or testnet.`);
+    process.exit(1);
+  }
+
+  try {
+    await forkDevnet(options);
+  } catch (error) {
+    logger.error((error as Error).message);
+    process.exit(1);
+  }
+}

src/cmd/node.ts

@@ -1,11 +1,13 @@
-import { exec, spawn } from 'child_process';
+import { exec, spawn, ChildProcess } from 'child_process';
 import * as fs from 'fs';
 import * as path from 'path';
 import { initChainIfNeeded } from '../node/init-chain';
 import { installCKBBinary } from '../node/install';
 import { getCKBBinaryPath, readSettings } from '../cfg/setting';
 import { encodeBinPathForTerminal } from '../util/encoding';
 import { createRPCProxy } from '../tools/rpc-proxy';
+import { markForkFirstRunComplete, readForkState } from '../devnet/fork';
+import { callJsonRpc } from '../util/json-rpc';
 import { Network } from '../type/base';
 import { logger } from '../util/logger';
 
@@ -66,7 +68,15 @@ export async function nodeDevnet({ version, binaryPath, daemon }: NodeProp) {
   await initChainIfNeeded();
   const devnetConfigPath = encodeBinPathForTerminal(settings.devnet.configPath);
 
-  const ckbCmd = `${ckbBinPath} run -C ${devnetConfigPath}`;
+  // A forked devnet must boot once with --skip-spec-check --overwrite-spec so
+  // the imported (and patched) spec replaces the source chain's stored spec.
+  const forkState = readForkState(settings.devnet.configPath);
+  const firstRunFlags = forkState?.firstRunPending ? ' --skip-spec-check --overwrite-spec' : '';
+  if (forkState?.firstRunPending) {
+    logger.info(`Forked devnet (${forkState.source}) detected, first run uses --skip-spec-check --overwrite-spec.`);
+  }
+
+  const ckbCmd = `${ckbBinPath} run -C ${devnetConfigPath}${firstRunFlags}`;
   const minerCmd = `${ckbBinPath} miner -C ${devnetConfigPath}`;
   logger.info(`Launching CKB devnet Node...`);
   try {
@@ -81,6 +91,17 @@ export async function nodeDevnet({ version, binaryPath, daemon }: NodeProp) {
       logger.error(['CKB error:', data.toString()]);
     });
 
+    if (forkState?.firstRunPending) {
+      // Only clear the flag once the spawned node is actually up and reports
+      // the fork's genesis; if startup fails, the next run retries the flags.
+      void clearForkFirstRunWhenNodeUp(
+        ckbProcess,
+        settings.devnet.rpcUrl,
+        settings.devnet.configPath,
+        forkState.genesisHash,
+      );
+    }
+
     // Start the second command after 3 seconds
     setTimeout(async () => {
       try {
@@ -115,6 +136,51 @@ function resolveDaemonPaths() {
   return { logDir, logFile, pidFile };
 }
 
+// Poll the devnet RPC until the spawned node answers with the fork's genesis
+// hash, then mark the first run as done so subsequent `offckb node` runs boot
+// normally. Two guards against clearing the flag on the wrong signal:
+//   - the poll aborts when the spawned ckb process exits (e.g. failed boot),
+//   - an answering node is only trusted when its genesis matches the fork
+//     state — an unrelated node occupying the port must not clear the flag.
+async function clearForkFirstRunWhenNodeUp(
+  ckbProcess: ChildProcess,
+  rpcUrl: string,
+  configPath: string,
+  expectedGenesisHash: string,
+) {
+  let processExited = false;
+  const markExited = () => {
+    processExited = true;
+  };
+  ckbProcess.once('exit', markExited);
+  ckbProcess.once('error', markExited);
+
+  const timeoutMs = 10 * 60 * 1000; // large forks take a while to boot
+  const start = Date.now();
+  while (!processExited && Date.now() - start < timeoutMs) {
+    try {
+      const genesisHash = String(await callJsonRpc(rpcUrl, 'get_block_hash', ['0x0'], 5000)).toLowerCase();
+      if (genesisHash !== expectedGenesisHash.toLowerCase()) {
+        logger.warn(
+          `A node is answering at ${rpcUrl} but reports a different genesis (${genesisHash}); ` +
+            'leaving the first-run flags in place.',
+        );
+        return;
+      }
+      markForkFirstRunComplete(configPath);
+      logger.success('Forked devnet is up; first-run spec flags cleared.');
+      return;
+    } catch {
+      await new Promise((resolve) => setTimeout(resolve, 1000));
+    }
+  }
+  if (processExited) {
+    logger.warn('The CKB process exited before the forked devnet came up; first-run flags will be retried next time.');
+  } else {
+    logger.warn('Timed out waiting for the forked devnet to start; first-run flags will be retried next time.');
+  }
+}
+
 function readPidFile(pidFile: string): PidMetadata | null {
   let raw: string;
   try {