db: verify the tail of the chain on startup and back out of a bad tip (#61)

A node that accepted blocks its current rules would reject -- because it
was running an older build, or because the tip it landed on was never valid
-- had no way to notice. It kept the branch, kept building on it, and kept
mining coins the rest of the network throws away. The only cure was
deleting blk*.dat by hand, which requires knowing something is wrong. A
node here once mined a 323-block fork for five days that way.

On load, walk back from the tip running CheckBlock on each block. If one
fails, move the best chain back to its parent, disconnect what came after,
and let the node re-download from peers. Satoshi added this after the block
74638 incident for the same reason: so it would not have needed anyone to
delete files.

Only the last 288 blocks are checked by default -- about ten hours at a
two-minute target -- because every check costs a RandomX hash.
/checkblocks=N changes it, 0 does the whole chain.

The repair deliberately runs in LoadBlockIndex() in main.cpp rather than in
CTxDB::LoadBlockIndex where the detection happens. CTxDB::LoadBlockIndex
walks the index with a cursor it never closes, so it still holds read locks
when it returns, and a write transaction opened alongside it waits on them
forever. That is measured, not guessed: the first version came up, printed
the diagnosis, and hung with nothing in any log. Closing that cursor is a
separate fix worth making on its own.

A failed rollback reports and starts anyway. Refusing to start would leave
the operator with no node to read the message from.

What this catches is what CheckBlock catches: proof of work, merkle root,
duplicate transactions, sigops, timestamps. It does not re-verify
signatures or spends, which would mean rebuilding the transaction index.

Verified by forcing a block at tip-5 to fail: the node reported the bad
block, moved back six, stayed up, and re-synced past its old tip. Without
the forced failure it verifies 288 and says so.

1 files changed, 75 insertions, 0 deletions

diff --git a/src/main.cpp b/src/main.cpp
index 15805ca..dbb2c41 100644
--- a/src/main.cpp
+++ b/src/main.cpp
@@ -49,6 +49,15 @@ uint256 hashGenesisMerkleRoot("0x1a45b4482532abb29b10e234d3f13132230525a339ecea9
 CBlockIndex* pindexGenesisBlock = NULL;
 int nBestHeight = -1;
 uint256 hashBestChain = 0;
+// Blocks at the tip to re-verify on startup. 0 means the whole chain. 288 is
+// about ten hours at a two-minute target -- far enough back to catch a bad tip,
+// short enough that startup does not pay a RandomX hash per block for the whole
+// history. /checkblocks=N overrides it.
+int nCheckBlocksOnLoad = 288;
+// Set by CTxDB::LoadBlockIndex when the tail of the chain fails verification.
+// The repair runs in LoadBlockIndex() below, once that read-only handle and its
+// cursor are closed.
+CBlockIndex* pindexBadChainFork = NULL;
 CBlockIndex* pindexBest = NULL;
 
 map<uint256, CBlock*> mapOrphanBlocks;
@@ -1577,6 +1586,72 @@ bool LoadBlockIndex(bool fAllowNew)
         return false;
     txdb.Close();
 
+    // Repair a bad tip found during verification.
+    //
+    // This has to happen after the handle above is closed. CTxDB::LoadBlockIndex
+    // walks the index with a cursor it never closes, so it is still holding read
+    // locks -- a write transaction opened alongside it waits on them forever.
+    // Measured, not guessed: the node came up, printed the diagnosis, and hung
+    // there with nothing in any log.
+    if (pindexBadChainFork)
+    {
+        CBlockIndex* pindexFork = pindexBadChainFork;
+        pindexBadChainFork = NULL;
+
+        printf("LoadBlockIndex() : *** moving best chain back to height %d, discarding %d block(s)\n",
+               pindexFork->nHeight, nBestHeight - pindexFork->nHeight);
+
+        vector<CBlockIndex*> vDisconnect;
+        for (CBlockIndex* pindex = pindexBest; pindex && pindex != pindexFork; pindex = pindex->pprev)
+            vDisconnect.push_back(pindex);
+
+        bool fRolledBack = false;
+        try
+        {
+            CTxDB txdbWrite;
+            txdbWrite.TxnBegin();
+            foreach(CBlockIndex* pindex, vDisconnect)
+            {
+                CBlock block;
+                if (!block.ReadFromDisk(pindex->nFile, pindex->nBlockPos, true))
+                    throw runtime_error("ReadFromDisk for disconnect failed");
+                if (!block.DisconnectBlock(txdbWrite, pindex))
+                    throw runtime_error("DisconnectBlock failed");
+            }
+            if (!txdbWrite.WriteHashBestChain(pindexFork->GetBlockHash()))
+                throw runtime_error("WriteHashBestChain failed");
+            txdbWrite.TxnCommit();
+            txdbWrite.Close();
+            fRolledBack = true;
+        }
+        catch (const std::exception& e)
+        {
+            printf("LoadBlockIndex() : rollback failed: %s\n", e.what());
+        }
+        catch (...)
+        {
+            printf("LoadBlockIndex() : rollback failed\n");
+        }
+
+        if (fRolledBack)
+        {
+            foreach(CBlockIndex* pindex, vDisconnect)
+                if (pindex->pprev)
+                    pindex->pprev->pnext = NULL;
+
+            pindexBest    = pindexFork;
+            hashBestChain = pindexBest->GetBlockHash();
+            nBestHeight   = pindexBest->nHeight;
+            printf("LoadBlockIndex() : now at height %d, will re-download from peers\n", nBestHeight);
+        }
+        else
+        {
+            // Start anyway. The operator needs a running node to read this from.
+            printf("LoadBlockIndex() : *** still on the bad chain at height %d -- back up the data directory and report this\n",
+                   nBestHeight);
+        }
+    }
+
     //
     // Init with genesis block
     //