dash(oracle): embedded ORACLE-SHADOW validator + graduation gate (first slice)

Run c2pool ALONGSIDE a Dash Core daemon and use dashd as a CONTINUOUS REALTIME
ORACLE for the daemonless embedded DIP4 CbTx/template arm. On every new tip the
shadow (OBSERVE-only, --embedded-oracle-shadow):

  1. builds the embedded template via the SAME path the serve arm uses
     (NodeCoinState::select_work) -- Embedded arm, or records a FALL-CLOSED
     serve-gap when the arm is not armed;
  2. VERDICT: assembles the embedded block and submits dashd
     getblocktemplate{mode:proposal} (NodeRPC::propose_block_hex) --
     TestBlockValidity is the authoritative, mempool-independent pass/fail;
  3. DIAGNOSIS: regime-aware field compare vs dashd getblocktemplate.

Regime model (adversarial-critique determinism table): EQUALITY (nHeight, CbTx
version, bits, mintime, payee identity, platform burn, normalized subsidy, MN/
quorum roots on empty templates) COUNT; creditPool is an INVARIANT checked
against the normalized base (committed - platformReward - ownDelta ==
creditPool(N-1)) -- the exact check that caught the seed bug 3x; bestCL is a
CONSTRAINT (range, sig-equality only when CL heights coincide); fee/mempool
fields are informational and NEVER counted (the separate-mempool false-positive
the naive compare would cause).

Graduation ledger (JSONL divergence log + JSON state under the data-dir,
/embedded_oracle endpoint): GRADUATED == N consecutive served+proposal-accepted
clean blocks AND >=K served coverage of each height class (dkg_window,
superblock, post_restart_cold, quorum_rotation, credit_pool_transition,
non_empty_template) AND >=1 reorg AND serve-rate floor. Selection-bias fix:
fall-closed classes are recorded and declared as residual serve-gaps (disabling
dashd removes the fallback those classes rely on). Ledger hard-keyed to
{c2pool_commit, comparator_version, dashd_version, net}; revocable by chain-drift
sentinels (unknown CbTx version / payload parse fail).

Soundness grounded on the embedded arm's INDEPENDENT chain-state derivation, NOT
peer diversity (DASH has no CoinPeerManager). Graduation scope = chain-state
equivalence over the served domain (reward-safe daemonless block construction);
full network-standalone operation needs the separate CoinPeerManager gate.

Consensus-neutral: the serve/consensus/share path (work_source.cpp,
coinbase_builder.hpp, share_check.hpp, pplns.hpp, node_coin_state.hpp,
embedded_gbt.hpp) is UNTOUCHED -- the shadow is a separate new_tip subscriber
that only reads. web_server gains an additive /embedded_oracle reporting route.

Builds c2pool-dash + test_dash_embedded_oracle_shadow (11 KATs, green) pinning
the regime model, the fee-difference-is-not-a-divergence keystone, the creditPool
invariant, the bestCL constraint, and the graduation predicate.

Design doc: frstrtr/the docs/c2pool-dash-embedded-oracle-shadow-validator.md.

Author: integrator

.github/workflows/build.yml

@@ -138,7 +138,7 @@ jobs:
                     core_test sharechain_test share_test btc_share_test \
                     test_threading test_weights \
                     test_header_chain test_mempool test_template_builder \
-                    test_doge_chain test_compact_blocks test_dash_x11_kat test_dash_header_chain test_dash_block_replay test_dash_conformance test_dash_subsidy test_dash_mempool test_dash_simplifiedmns test_dash_quorum test_dash_quorum_root test_dash_mn_state test_dash_embedded_gbt test_dash_underfill_guard test_dash_smldiff test_dash_p2p_messages test_dash_p2p_connection test_dash_p2p_node test_dash_node_interface test_dash_config test_dash_broadcaster test_dash_broadcaster_full test_dash_share_hash_link test_dash_block_relay test_dash_rpc_request test_dash_rpc_conf test_dash_block_producer test_dash_embedded_relay_e2e test_dash_block_relay_plan test_dash_version_activation_latch test_dash_block_relay_binding test_dash_block_relay_dual_arm test_dash_coinbase_parity test_dash_coinbase_muldiv test_dash_donation_combined test_dash_g3_assembled test_dash_work_target test_dash_work_job_targets test_dash_stratum_binding test_dash_job_notify_roundtrip test_dash_cb_payee test_dash_stratum_notify_roundtrip test_dash_poolnode_messages test_dash_peer test_dash_share_tracker test_dash_node test_dash_share_messages test_dash_stratum_extranonce_split test_dash_stratum_submit_reassembly test_dash_difficulty_parity test_dash_auto_ratchet test_dash_min_protocol_gate test_dash_work_source test_dash_node_coin_state test_dash_coin_state_maintainer test_dash_node_embedded_wire test_dash_node_reception_wire test_dash_get_work test_dash_stratum_work_source \
+                    test_doge_chain test_compact_blocks test_dash_x11_kat test_dash_header_chain test_dash_block_replay test_dash_conformance test_dash_subsidy test_dash_mempool test_dash_simplifiedmns test_dash_quorum test_dash_quorum_root test_dash_mn_state test_dash_embedded_gbt test_dash_underfill_guard test_dash_smldiff test_dash_p2p_messages test_dash_p2p_connection test_dash_p2p_node test_dash_node_interface test_dash_config test_dash_broadcaster test_dash_broadcaster_full test_dash_share_hash_link test_dash_block_relay test_dash_rpc_request test_dash_rpc_conf test_dash_block_producer test_dash_embedded_relay_e2e test_dash_block_relay_plan test_dash_version_activation_latch test_dash_block_relay_binding test_dash_block_relay_dual_arm test_dash_coinbase_parity test_dash_coinbase_muldiv test_dash_donation_combined test_dash_g3_assembled test_dash_work_target test_dash_work_job_targets test_dash_stratum_binding test_dash_job_notify_roundtrip test_dash_cb_payee test_dash_stratum_notify_roundtrip test_dash_poolnode_messages test_dash_peer test_dash_share_tracker test_dash_node test_dash_share_messages test_dash_stratum_extranonce_split test_dash_stratum_submit_reassembly test_dash_difficulty_parity test_dash_auto_ratchet test_dash_min_protocol_gate test_dash_work_source test_dash_node_coin_state test_dash_coin_state_maintainer test_dash_node_embedded_wire test_dash_node_reception_wire test_dash_get_work test_dash_stratum_work_source test_dash_embedded_oracle_shadow \
                     test_multiaddress_pplns test_pplns_stress \
                     test_hash_link test_decay_pplns \
                     test_pplns_consensus \
@@ -320,7 +320,7 @@ jobs:
                     test_threading test_weights \
                     test_header_chain test_mempool test_template_builder \
                     test_doge_chain test_compact_blocks test_dash_x11_kat \
-                    test_dash_header_chain test_dash_block_replay test_dash_conformance test_dash_subsidy test_dash_mempool test_dash_simplifiedmns test_dash_quorum test_dash_quorum_root test_dash_mn_state test_dash_embedded_gbt test_dash_underfill_guard test_dash_smldiff test_dash_p2p_messages test_dash_p2p_connection test_dash_p2p_node test_dash_node_interface test_dash_config test_dash_broadcaster test_dash_broadcaster_full test_dash_share_hash_link test_dash_block_relay test_dash_rpc_request test_dash_rpc_conf test_dash_block_producer test_dash_embedded_relay_e2e test_dash_block_relay_plan test_dash_version_activation_latch test_dash_block_relay_binding test_dash_block_relay_dual_arm test_dash_coinbase_parity test_dash_coinbase_muldiv test_dash_donation_combined test_dash_g3_assembled test_dash_work_target test_dash_work_job_targets test_dash_stratum_binding test_dash_job_notify_roundtrip test_dash_cb_payee test_dash_stratum_notify_roundtrip test_dash_poolnode_messages test_dash_peer test_dash_share_tracker test_dash_node test_dash_share_messages test_dash_stratum_extranonce_split test_dash_stratum_submit_reassembly test_dash_difficulty_parity test_dash_auto_ratchet test_dash_min_protocol_gate test_dash_work_source test_dash_node_coin_state test_dash_coin_state_maintainer test_dash_node_embedded_wire test_dash_node_reception_wire test_dash_get_work test_dash_stratum_work_source \
+                    test_dash_header_chain test_dash_block_replay test_dash_conformance test_dash_subsidy test_dash_mempool test_dash_simplifiedmns test_dash_quorum test_dash_quorum_root test_dash_mn_state test_dash_embedded_gbt test_dash_underfill_guard test_dash_smldiff test_dash_p2p_messages test_dash_p2p_connection test_dash_p2p_node test_dash_node_interface test_dash_config test_dash_broadcaster test_dash_broadcaster_full test_dash_share_hash_link test_dash_block_relay test_dash_rpc_request test_dash_rpc_conf test_dash_block_producer test_dash_embedded_relay_e2e test_dash_block_relay_plan test_dash_version_activation_latch test_dash_block_relay_binding test_dash_block_relay_dual_arm test_dash_coinbase_parity test_dash_coinbase_muldiv test_dash_donation_combined test_dash_g3_assembled test_dash_work_target test_dash_work_job_targets test_dash_stratum_binding test_dash_job_notify_roundtrip test_dash_cb_payee test_dash_stratum_notify_roundtrip test_dash_poolnode_messages test_dash_peer test_dash_share_tracker test_dash_node test_dash_share_messages test_dash_stratum_extranonce_split test_dash_stratum_submit_reassembly test_dash_difficulty_parity test_dash_auto_ratchet test_dash_min_protocol_gate test_dash_work_source test_dash_node_coin_state test_dash_coin_state_maintainer test_dash_node_embedded_wire test_dash_node_reception_wire test_dash_get_work test_dash_stratum_work_source test_dash_embedded_oracle_shadow \
                     test_hash_link test_decay_pplns \
                     test_pplns_consensus \
                     test_v36_script_sorting test_v36_cross_impl_refhash \

src/c2pool/main_dash.cpp

@@ -63,6 +63,7 @@
 #include <impl/dash/coin/live_feed.hpp>          // E2a live-feed bridge (raw wire events -> derived ingest events)
 #include <impl/dash/coin/mempool_ingest.hpp>     // wire_mempool_ingest (leg 1)
 #include <impl/dash/coin/tip_ingest.hpp>         // wire_tip_ingest (leg 2)
+#include <impl/dash/coin/embedded_oracle_shadow.hpp> // dash::coin::EmbeddedOracleShadow — per-block dashd cross-check (OBSERVE-only)
 #include <impl/dash/coin/block_connect_ingest.hpp>   // wire_block_connect_ingest (leg 3)
 #include <impl/dash/coin/mn_list_ingest.hpp>     // wire_mn_list_ingest (leg 4)
 #include <impl/dash/coin/mn_seed.hpp>            // E2c: RPC protx-list MN-set seed (parse_protx_list_seed)
@@ -192,7 +193,8 @@ void print_banner(const char* argv0)
         << "           [--listen [HOST:]PORT] [--addnode HOST:PORT]... [--connect HOST:PORT]...\n"
         << "           [--stratum [HOST:]PORT] [--coin-p2p-connect HOST:PORT]...\n"
         << "           [--web-port PORT] [--web-host ADDR] [--dashboard-dir PATH]\n"
-        << "           [--embedded-utxo]\n"
+        << "           [--embedded-utxo] [--embedded-oracle-shadow]\n"
+        << "           [--oracle-graduation-blocks N] [--oracle-class-coverage K]\n"
         << "           [--give-author PCT] [-f|--fee PCT] [--node-owner-address ADDR]\n"
         << "           [--redistribute pplns|fee|boost|donate]\n"
         << "       " << argv0 << " --mine-block [--coin-rpc H:P] [--coin-rpc-auth PATH]\n"
@@ -212,6 +214,12 @@ void print_banner(const char* argv0)
         << "        --web-port PORT (alias --http-port, default 8080) serves the FULL\n"
         << "        c2pool web dashboard + JSON API on --web-host (default 0.0.0.0)\n"
         << "        from --dashboard-dir (default web-static); --web-port 0 disables.\n"
+        << "        --embedded-oracle-shadow runs the OBSERVE-only per-block dashd\n"
+        << "        cross-check: dashd getblocktemplate{mode:proposal} is the VERDICT,\n"
+        << "        regime-aware field-compare is the DIAGNOSIS; a persisted graduation\n"
+        << "        ledger + /embedded_oracle verdict signal when the embedded arm is\n"
+        << "        proven equivalent (safe to disable dashd, served domain). Needs the\n"
+        << "        dashd RPC arm; never changes serving. N/K tune the graduation gate.\n"
         << "        Live sharechain tip/stats, pool hashrate and per-share difficulty\n"
         << "        are bound to the REAL DASH tracker; local hashrate comes from the\n"
         << "        DASH stratum acceptor. If stratum and web ports collide the web\n"
@@ -380,7 +388,10 @@ int run_node(bool testnet, const std::string& rpc_endpoint,
              double dev_donation, double node_owner_fee,
              const std::string& node_owner_address,
              const std::string& redistribute_mode,
-             bool no_p2p_relay)
+             bool no_p2p_relay,
+             bool embedded_oracle_shadow = false,
+             uint64_t oracle_grad_blocks = 5000,
+             uint64_t oracle_class_coverage = 20)
 {
     namespace io = boost::asio;
 
@@ -536,6 +547,12 @@ int run_node(bool testnet, const std::string& rpc_endpoint,
     // serve X11 miners from the LTC work source. The dashboard is told the
     // miner-facing port through mining_interface->set_worker_port() (display)
     // and fed real stratum rates from the DASH acceptor after it starts.
+    // Embedded ORACLE-SHADOW validator (--embedded-oracle-shadow). Declared here
+    // (before web_server) so the /embedded_oracle endpoint closure can read it;
+    // constructed + subscribed to the tip event later in the coin_p2p block, once
+    // node_coin_state exists. OBSERVE-only: stats_json() touches no serving state.
+    std::shared_ptr<dash::coin::EmbeddedOracleShadow> oracle_shadow;
+
     std::unique_ptr<core::WebServer> web_server;
     auto enhanced_node = std::make_shared<dash::EnhancedDashNode>(testnet);
     if (web_port != 0) {
@@ -568,6 +585,16 @@ int run_node(bool testnet, const std::string& rpc_endpoint,
         // resolved), not an ICoinNode, so tell the dashboard RPC is present.
         mi->set_coin_rpc_available(static_cast<bool>(rpc));
 
+        // ── /embedded_oracle stats endpoint (OBSERVE-only) ────────────────
+        // JSON coverage ledger + objective GRADUATION verdict (safe-to-disable-
+        // dashd gate). Reads the shadow validator's own structures; never the
+        // serving path. Empty/disabled shape when --embedded-oracle-shadow off.
+        mi->set_embedded_oracle_fn([&oracle_shadow]() -> nlohmann::json {
+            if (oracle_shadow) return oracle_shadow->stats_json();
+            return nlohmann::json{{"mode", "disabled"},
+                {"note", "run with --embedded-oracle-shadow to enable"}};
+        });
+
         web_server->set_dashboard_dir(dashboard_dir);
         // Explicitly DISABLE the WebServer's own stratum acceptor. Its ctor
         // defaults stratum_port_ to (web_port + 10) (web_server.cpp:8330/8344/
@@ -1278,6 +1305,96 @@ int run_node(bool testnet, const std::string& rpc_endpoint,
         // header chain), NOT the raw wire.
         coin_feed_subs.push_back(
             c2pool::dash::wire_tip_ingest(coin_state, *maintainer));
+
+        // ── Embedded ORACLE-SHADOW: per-block dashd cross-check (OBSERVE-only) ─
+        // Subscribed to the SAME new_tip event AFTER wire_tip_ingest, so the
+        // maintainer has already republished the embedded bundle for this tip
+        // before the shadow reads it. The shadow builds the embedded template
+        // via node_coin_state.select_work (the SAME build path the serve arm
+        // uses) and cross-checks it against a fresh dashd getblocktemplate,
+        // gating divergence/graduation on the DETERMINISTIC field set only (the
+        // two nodes have intentionally different mempools/peers). It NEVER
+        // changes the serving decision. Requires the alongside dashd RPC arm.
+        if (embedded_oracle_shadow) {
+            if (!rpc) {
+                std::cout << "[run] --embedded-oracle-shadow given but the dashd "
+                             "RPC arm is UNARMED (need dash.conf creds / --coin-rpc); "
+                             "the shadow has no oracle to cross-check against -- "
+                             "disabled.\n";
+            } else {
+                dash::coin::EmbeddedOracleShadow::Config oc;
+                oc.testnet = testnet;
+                oc.grad.consecutive_clean_target  = oracle_grad_blocks;
+                oc.grad.per_class_coverage_target = oracle_class_coverage;
+#ifdef C2POOL_VERSION
+                oc.c2pool_commit = C2POOL_VERSION;
+#endif
+                try { oc.dashd_version =
+                    rpc->getnetworkinfo().value("subversion", std::string{}); }
+                catch (...) { /* best-effort ledger identity */ }
+                const auto oracle_dir =
+                    (core::filesystem::config_path() / net_subdir).string();
+                oc.divergence_ledger_path =
+                    oracle_dir + "/embedded_oracle_divergence.jsonl";
+                oc.graduation_state_path =
+                    oracle_dir + "/embedded_oracle_graduation.json";
+
+                // Proposal VERDICT leg: assemble the embedded block (pool-only
+                // coinbase, no miner payout -- consensus-irrelevant to dashd
+                // TestBlockValidity) with the SAME SSOTs the --mine-block path
+                // uses, and submit getblocktemplate{mode:proposal}. "" = dashd
+                // ACCEPTED; else the reject reason. This is the authoritative,
+                // mempool-independent per-height verdict (§6 condition 2).
+                auto proposal_fn =
+                    [testnet, rp = rpc.get()](const dash::coin::DashWorkData& wd)
+                        -> dash::coin::ProposalResult {
+                    dash::coin::ProposalResult r;
+                    r.attempted = true;
+                    try {
+                        const core::CoinParams params = dash::make_coin_params(testnet);
+                        std::map<std::vector<unsigned char>, uint64_t> empty_weights;
+                        uint160 zero_pkh;   // pool-only coinbase (no finder)
+                        auto tx_outs = dash::coinbase::compute_dash_payouts(
+                            wd.m_coinbase_value, wd.m_packed_payments, zero_pkh,
+                            empty_weights, /*total_weight=*/0, params);
+                        auto layout = dash::coinbase::build(
+                            wd, tx_outs, /*pool_tag=*/"c2pool", params,
+                            /*ref_hash=*/uint256::ZERO);
+                        // nonce 0 + curtime: proposal mode skips PoW; validity is
+                        // structure + payee + CbTx + tx-set (TestBlockValidity).
+                        auto block = dash::coin::serialize_full_block(
+                            wd, layout.bytes, /*nonce=*/0,
+                            wd.m_curtime ? wd.m_curtime
+                                         : static_cast<uint32_t>(std::time(nullptr)));
+                        const std::string reason = rp->propose_block_hex(HexStr(block));
+                        r.accepted = reason.empty();
+                        r.reason = reason;
+                    } catch (const std::exception& e) {
+                        r.accepted = false; r.reason = std::string("assemble:") + e.what();
+                    }
+                    return r;
+                };
+
+                oracle_shadow = std::make_shared<dash::coin::EmbeddedOracleShadow>(
+                    node_coin_state,
+                    [rp = rpc.get()]() { return rp->getwork(); },
+                    std::move(proposal_fn),
+                    std::move(oc));
+                auto* shadow = oracle_shadow.get();
+                coin_feed_subs.push_back(
+                    coin_state.new_tip.subscribe(
+                        [shadow](const ::dash::interfaces::TipAdvance& t) {
+                            shadow->on_new_tip(t.prev_height + 1, t.prev_hash);
+                        }));
+                std::cout << "[run] --embedded-oracle-shadow ARMED: per-block dashd "
+                             "PROPOSAL verdict + regime-aware field diagnosis (N="
+                          << oracle_grad_blocks << " K=" << oracle_class_coverage
+                          << "); ledger at " << oracle_dir
+                          << "/embedded_oracle_*.{jsonl,json}; verdict at "
+                             "/embedded_oracle\n";
+            }
+        }
+
         // Leg 3 (block connect): Node::block_connected -> maintainer
         // .on_block_connected (MnStateMachine::apply_block, folds DIP3 special
         // txs into the DMN set). block_connected is fired by the live-feed
@@ -1742,6 +1859,9 @@ int main(int argc, char** argv)
     std::string stratum_host = "0.0.0.0";      // --stratum [HOST:]PORT bind interface (default all)
     uint16_t    stratum_port = 0;              // 0 disables the Stratum accept-loop; --stratum sets it
     bool embedded_utxo = false;                // --embedded-utxo: arm the E2b UTXO/fee lane (opt-in)
+    bool embedded_oracle_shadow = false;       // --embedded-oracle-shadow: per-block dashd cross-check (OBSERVE-only)
+    uint64_t oracle_grad_blocks = 5000;        // --oracle-graduation-blocks N (consecutive clean)
+    uint64_t oracle_class_coverage = 20;       // --oracle-class-coverage K (per height class)
     double dev_donation = 0.1;                 // --give-author (donation_percentage; README default 0.1%)
     double node_owner_fee = 0.0;               // -f / --fee (node_owner_fee; default 0)
     std::string node_owner_address;            // --node-owner-address (fee destination)
@@ -1786,6 +1906,12 @@ int main(int argc, char** argv)
             no_p2p_relay = true;
         else if (std::strcmp(argv[i], "--embedded-utxo") == 0)
             embedded_utxo = true;
+        else if (std::strcmp(argv[i], "--embedded-oracle-shadow") == 0)
+            embedded_oracle_shadow = true;
+        else if (std::strcmp(argv[i], "--oracle-graduation-blocks") == 0 && i + 1 < argc)
+            oracle_grad_blocks = std::strtoull(argv[++i], nullptr, 10);
+        else if (std::strcmp(argv[i], "--oracle-class-coverage") == 0 && i + 1 < argc)
+            oracle_class_coverage = std::strtoull(argv[++i], nullptr, 10);
         else if ((std::strcmp(argv[i], "--give-author") == 0 ||
                   std::strcmp(argv[i], "--dev-donation") == 0) && i + 1 < argc)
             dev_donation = std::strtod(argv[++i], nullptr);
@@ -1886,7 +2012,9 @@ int main(int argc, char** argv)
                         stratum_host, stratum_port, web_host, web_port,
                         dashboard_dir, coin_p2p_targets,
                         embedded_utxo, dev_donation, node_owner_fee,
-                        node_owner_address, redistribute_mode, no_p2p_relay);
+                        node_owner_address, redistribute_mode, no_p2p_relay,
+                        embedded_oracle_shadow, oracle_grad_blocks,
+                        oracle_class_coverage);
     }
     return run_selftest();
 }