performance: reduce PoW checks on blockindex load (#36)

* performance: reduce PoW checks on blockindex load

When the blockindex is constructed from disk on startup, currently
each header is subjected to a PoW check. This is relatively cheap
for Bitcoin's SHA256D, but less optimal when having to perform
FactorN's ghash function.

This changes the frequency at which the PoW check is performed to
be configurable with the startup argument idxpowcheckrate,
specifying a denominator at which PoW is checked during initial
blockindex construction when the node starts, and sets the default
value to 100 - meaning 1 in 100 blockheaders that are loaded from
disk will be selected randomly to undergo a PoW check.

More paranoid node operators can lower the value, at the cost of
startup performance.

Note: All leveldb records that are read by the iterator are still
subjected to internal integrity checks.

* lint: allow -idxpowcheckrate to be undocumented for now

Author: patricklodder

src/init.cpp

@@ -366,7 +366,7 @@ void SetupServerArgs(ArgsManager& argsman)
 
     // Hidden Options
     std::vector<std::string> hidden_args = {
-        "-dbcrashratio", "-forcecompactdb",
+        "-dbcrashratio", "-forcecompactdb", "-idxpowcheckrate",
         // GUI args. These will be overwritten by SetupUIArgs for the GUI
         "-choosedatadir", "-lang=<lang>", "-min", "-resetguisettings", "-splash", "-uiplatform"};
 

src/txdb.cpp

@@ -277,6 +277,13 @@ bool CBlockTreeDB::LoadBlockIndexGuts(const Consensus::Params& consensusParams,
 
     pcursor->Seek(std::make_pair(DB_BLOCK_INDEX, uint256()));
 
+    // The denominator rate at which we will check PoW when re-constructing
+    // m_block_index from disk. Higher values reduce the amount of checks
+    // performed, i.e. `100` means that 1 in 100 headers will be randomly
+    // selected for a PoW check, and `1` means every header will be checked.
+    int powcheck_rate = gArgs.GetArg("-idxpowcheckrate", nDefaultCheckPoWRate);
+    static FastRandomContext rngPoWCheck;
+
     // Load m_block_index
     while (pcursor->Valid()) {
         if (ShutdownRequested()) return false;
@@ -301,8 +308,12 @@ bool CBlockTreeDB::LoadBlockIndexGuts(const Consensus::Params& consensusParams,
                 pindexNew->wOffset        = diskindex.wOffset;
                 pindexNew->nTx            = diskindex.nTx;
 
-                if (!CheckProofOfWork( pindexNew->GetBlockHeader(), consensusParams))
+                // Randomly check PoW. This does not affect integrity, as every
+                // record is integrity-checked by leveldb; see also the
+                // CDBWrapper::CDBWrapper implementation
+                if (rngPoWCheck.randrange(powcheck_rate) == 0 && !CheckProofOfWork(pindexNew->GetBlockHeader(), consensusParams)) {
                     return error("%s: CheckProofOfWork failed: %s", __func__, pindexNew->ToString());
+                }
 
                 pcursor->Next();
             } else {

src/txdb.h

@@ -38,6 +38,8 @@ static const int64_t nMaxTxIndexCache = 1024;
 static const int64_t max_filter_index_cache = 1024;
 //! Max memory allocated to coin DB specific cache (MiB)
 static const int64_t nMaxCoinsDBCache = 8;
+//!Default rate of checking pow on index load
+static const int nDefaultCheckPoWRate = 100;
 
 // Actually declared in validation.cpp; can't include because of circular dependency.
 extern RecursiveMutex cs_main;

test/lint/check-doc.py

@@ -23,7 +23,7 @@
 CMD_GREP_WALLET_HIDDEN_ARGS = r"git grep --function-context 'void DummyWalletInit::AddWalletOptions' -- {}".format(CMD_ROOT_DIR)
 CMD_GREP_DOCS = r"git grep --perl-regexp '{}' {}".format(REGEX_DOC, CMD_ROOT_DIR)
 # list unsupported, deprecated and duplicate args as they need no documentation
-SET_DOC_OPTIONAL = set(['-h', '-help', '-dbcrashratio', '-forcecompactdb', '-zapwallettxes'])
+SET_DOC_OPTIONAL = set(['-h', '-help', '-dbcrashratio', '-forcecompactdb', '-zapwallettxes', '-idxpowcheckrate'])
 
 
 def lint_missing_argument_documentation():