Bring main onto the line this project actually ships

main had drifted to a history with no ancestor in common with the tree we
build and deploy. It sat at be63e83 (2026-08-09) with 28 commits; the shipped
line, deploy/soak-2026-08-21, had 383. The two shared nothing -- git
merge-base returns no result at all -- so anyone cloning this repository
landed on a default branch that was neither current nor related to the
running node.

The 28 commits on main were audited before this merge: every one is already
present in the tree by content, including the two that are hardest to see
because they were rewritten afterwards -- the payout reorg-safety work
(PayoutBlockStillOnChain, since evolved to re-check on every retry rather
than once at maturity) and the nostr relay certificate verification
(SSL_set1_host, now with a Windows root-store path). Nothing on main is lost
by this merge.

Recorded as a merge rather than a force-push: main only ever moves forward,
its old history stays reachable as the first parent, and this is revertible.
The tree is taken wholesale from the merged branch, so main's content is now
exactly what was built and tested.

What this brings onto main:

- compactblock: BIP152-style compact block relay ported from upstream 1.2.23,
  primitives and unit test only, with the RAND_bytes failure path closed and
  the reconstruction allocation bounded.
- upstream/main 17741d9: isolated testnet parameters and genesis, optional
  2 MB large pages for RandomX. Mainnet genesis verified byte-identical
  through upstream's BuildGenesisBlock() refactor.
- deploy: every image built on bookworm so binaries run on Debian 12, and the
  container build now honours the RandomX and libsecp256k1 pins it had been
  silently ignoring since before 1.2.15.

Gates at this tree: 15/15 selftests, compactblock's 17 checks, consensus
constants unchanged against e4f898c apart from the pre-existing
MAX_BLOCK_SIZE_GEN, mainnet genesis intact, money-log check ok.

1 files changed, 159 insertions, 0 deletions

diff --git a/docs/storage.md b/docs/storage.md
new file mode 100644
index 0000000..c7de4e7
--- /dev/null
+++ b/docs/storage.md
@@ -0,0 +1,159 @@
+# Bitflash Storage Inventory
+
+This document records the Berkeley DB surface that still exists in Bitflash.
+It is a migration map, not a promise that all stores should move at once.
+
+The immediate priority is `wallet.dat`: it holds spend authority and recovery
+metadata. `blkindex.dat` can be rebuilt from block files, so it is important for
+startup correctness but less dangerous than wallet storage.
+
+## Berkeley DB Environment
+
+All `CDB` stores use one Berkeley DB environment rooted at:
+
+```text
+<datadir>/database/
+```
+
+The environment creates log files such as:
+
+```text
+database/log.0000000001
+```
+
+Those logs are part of the legacy BDB risk model. They are not user-facing
+wallet files, but they can contain historical records while the environment is
+live. Wallet encryption already rewrites `wallet.dat` and purges BDB environment
+logs after the encrypted file is installed.
+
+## Common Records
+
+Every `CDB` file can carry:
+
+| Key | Value | Purpose |
+|---|---|---|
+| `"version"` | `int` | Database format version written when a DB is opened in create mode. |
+
+Keys and values are serialized with the existing `CDataStream` format. A future
+SQLite migration must preserve byte-for-byte value semantics until a higher
+level schema migration is intentionally introduced.
+
+## `wallet.dat`
+
+`wallet.dat` is accessed through `CWalletDB`.
+Read-only tooling should go through `ScanWalletRecords()` instead of opening a
+Berkeley DB cursor directly. That keeps migration checks, audits, and future
+backends behind one record-stream boundary. Wallet rewrite code should also use
+that boundary when it needs to copy existing records into a newly written
+wallet file.
+
+| Key shape | Value | Secret? | Meaning |
+|---|---|---:|---|
+| `("name", address)` | `string` | No | Address book label. |
+| `("tx", txid)` | `CWalletTx` | No | Wallet transaction state, including spend flags and metadata. |
+| `("key", pubkey)` | `CPrivKey` | Yes | Plain private key record for unencrypted wallets or old backups. |
+| `"defaultkey"` | `vector<unsigned char>` | No | Default public key shown by the wallet. |
+| `"hdmaster"` | `vector<unsigned char>` | Yes | Plain BIP32 master private key material. |
+| `"hdchaincode"` | `vector<unsigned char>` | Yes | Plain BIP32 chain code. |
+| `"hdnext"` | `unsigned int` | No | Legacy deterministic derivation next index. |
+| `"hdschema"` | `int` | No | Deterministic derivation schema (`legacy` or `bip44`). |
+| `"hdcointype"` | `unsigned int` | No | BIP44 coin type. Bitflash is `4346950`. |
+| `"hdreceivenext"` | `unsigned int` | No | BIP44 external receive-chain next index. |
+| `"hdchangenext"` | `unsigned int` | No | BIP44 internal change-chain next index. |
+| `("mkey", id)` | `CWalletMasterKey` | Sensitive | Encrypted wallet master-key metadata. |
+| `("ckey", pubkey)` | `vector<unsigned char>` | Yes | Private key encrypted under the wallet master key. |
+| `"cryptedhdmaster"` | `vector<unsigned char>` | Yes | Encrypted BIP32 master private key material. |
+| `"cryptedhdchaincode"` | `vector<unsigned char>` | Yes | Encrypted BIP32 chain code. |
+| `"walletminversion"` | `int` | No | Minimum wallet format version required to open this wallet safely. |
+| `("pool", index)` | `vector<unsigned char>` | No | Public key reserved in the key pool. The private key lives in `key`/`ckey`. |
+| `("setting", name)` | typed setting value | Usually no | Wallet/UI settings such as mining mode, fees, and backup timestamps. |
+
+### Wallet Migration Rules
+
+A future wallet backend must keep these properties:
+
+1. Unknown future wallet formats fail closed with a clear message.
+2. Encrypted wallets must encrypt private keys and HD seed material.
+3. A locked encrypted wallet may expose public keys and balances, but not
+   private keys or decrypted HD seed material.
+4. `-backupwallet` must produce a wallet file that opens in a different datadir
+   without copying `database/`, and must not leave a half-written final backup
+   path if the copy or install step fails.
+5. `-recoveryaudit` must distinguish:
+   - no recovery phrase installed;
+   - recovery phrase present;
+   - recovery phrase encrypted and locked, requiring unlock before coverage can
+     be audited.
+
+## SQLite Wallet Store
+
+Since v1.2.20 SQLite is the default wallet backend. It is a byte-preserving
+record store, not a new wallet format -- the same serialized `CDataStream` bytes
+`wallet.dat` uses:
+
+```sql
+CREATE TABLE wallet_records (
+  key BLOB PRIMARY KEY NOT NULL,
+  value BLOB NOT NULL
+);
+PRAGMA user_version=1;
+```
+
+A fresh data directory starts on `wallet.sqlite`. An existing `wallet.dat` keeps
+running on Berkeley DB until it is converted: the GUI offers a one-click
+conversion on first run, and `-walletbackend=sqlite` / `-walletbackend=bdb`
+(recorded in a plain-text `wallet-backend` marker file in the datadir) select a
+backend explicitly. Conversion never modifies `wallet.dat`; it stays as a
+fallback, and the change takes effect on restart.
+
+Every wallet operation runs on either backend: read, write, erase, key-pool and
+HD-seed writes (the two halves of the seed bracketed in one SQLite transaction),
+`-backupwallet` (checkpoint then copy the self-contained file), `-encryptwallet`
+(an atomic rebuild-and-swap in one transaction), and the `-walletstorageaudit` /
+`-walletstoragecheck` diagnostics, which read the active backend's file.
+
+The migration tooling remains available for scripted use. `-walletsqliteexport=FILE`
+streams every raw `wallet.dat` record through `ScanWalletRecords()` into a SQLite
+file in one transaction, refusing to overwrite. `-walletsqliteverify=FILE`
+compares the two streams and prints only counts. `-walletsqliteloadcheck=FILE`
+parses an export with the same record types the loader uses.
+`-walletsqliterestore=FILE` rebuilds a `wallet.dat` from an export in an empty
+data directory. Failed exports and restores remove their incomplete output.
+
+## `blkindex.dat`
+
+`blkindex.dat` is accessed through `CTxDB`.
+
+| Key shape | Value | Rebuildable? | Meaning |
+|---|---|---:|---|
+| `("tx", txid)` | `CTxIndex` | Yes | Transaction disk position and spent-output index state. |
+| `("owner", hash160, CDiskTxPos)` | `int height` | Yes | Legacy owner transaction index cursor range. |
+| `("blockindex", blockhash)` | `CDiskBlockIndex` | Yes | Block index entry. |
+| `"hashBestChain"` | `uint256` | Yes | Current best-chain tip hash. |
+
+`blkindex.dat` can be rebuilt from `blk*.dat` plus consensus validation. A
+future migration may therefore prefer a rebuild path over a byte-preserving
+conversion path.
+
+## Other Legacy Stores
+
+None. Upstream still documents `reviews.dat` (`CReviewDB`) and `market.dat`
+(`CMarketDB`) here; this fork deleted the 2009 marketplace outright, along
+with `CReviewDB`, `market.cpp` and `market.h`, so neither file is opened or
+created by this tree. A datadir carrying them from an older build is simply
+ignored.
+
+The old IP address database (`addr.dat`) has been removed. Peer discovery is
+handled by Nostr, `.btf`, rendezvous relays, and direct onion peers.
+
+## Migration Order (shipped in v1.2.20)
+
+1. Storage self-tests were added around Berkeley DB behavior first.
+2. The SQLite backend landed behind an explicit flag, then behind a
+   `wallet-backend` marker and a GUI conversion wizard.
+3. BDB-to-SQLite migration is a one-way copy that leaves `wallet.dat` as a backup.
+4. SQLite became the default for new wallets only after cross-version restore,
+   encryption, backup, and recovery-audit tests passed -- and a live coin-proof
+   of receive, spend, restart, and encrypt on real mainnet funds.
+5. Treat `blkindex.dat` separately; it is safer to rebuild than to migrate under
+   pressure.