upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--docs/storage.md67
-rw-r--r--src/main_gui.cpp16
-rw-r--r--src/walletcmd.h6
3 files changed, 43 insertions, 46 deletions
diff --git a/docs/storage.md b/docs/storage.md
index 08c4a01..b52a955 100644
--- a/docs/storage.md
+++ b/docs/storage.md
@@ -85,10 +85,11 @@ A future wallet backend must keep these properties:
85 - recovery phrase encrypted and locked, requiring unlock before coverage can 85 - recovery phrase encrypted and locked, requiring unlock before coverage can
86 be audited. 86 be audited.
87 87
88## Experimental SQLite Wallet Store 88## SQLite Wallet Store
89 89
90The SQLite wallet path starts as a byte-preserving record store, not as a new 90Since v1.2.20 SQLite is the default wallet backend. It is a byte-preserving
91wallet format: 91record store, not a new wallet format -- the same serialized `CDataStream` bytes
92`wallet.dat` uses:
92 93
93```sql 94```sql
94CREATE TABLE wallet_records ( 95CREATE TABLE wallet_records (
@@ -98,32 +99,26 @@ CREATE TABLE wallet_records (
98PRAGMA user_version=1; 99PRAGMA user_version=1;
99``` 100```
100 101
101Keys and values are the same serialized `CDataStream` bytes used by `wallet.dat`. 102A fresh data directory starts on `wallet.sqlite`. An existing `wallet.dat` keeps
102The first implementation is exercised only by `-selftest=wallet-sqlite`; the 103running on Berkeley DB until it is converted: the GUI offers a one-click
103runtime wallet still opens Berkeley DB. The purpose is to prove schema creation, 104conversion on first run, and `-walletbackend=sqlite` / `-walletbackend=bdb`
104raw record write/read, close/reopen behavior, and value preservation before any 105(recorded in a plain-text `wallet-backend` marker file in the datadir) select a
105BDB-to-SQLite migration or runtime backend flag exists. 106backend explicitly. Conversion never modifies `wallet.dat`; it stays as a
106 107fallback, and the change takes effect on restart.
107`-walletsqliteexport=FILE` is the next staging tool. It streams every raw 108
108`wallet.dat` record through `ScanWalletRecords()` and writes those same key and 109Every wallet operation runs on either backend: read, write, erase, key-pool and
109value bytes into the SQLite table inside one transaction. It refuses to 110HD-seed writes (the two halves of the seed bracketed in one SQLite transaction),
110overwrite an existing export and still does not change the runtime wallet 111`-backupwallet` (checkpoint then copy the self-contained file), `-encryptwallet`
111backend. `CWalletDBSQLite::ScanRecords()` can stream those exported records back 112(an atomic rebuild-and-swap in one transaction), and the `-walletstorageaudit` /
112through the same visitor shape, so tests can compare BDB and SQLite record 113`-walletstoragecheck` diagnostics, which read the active backend's file.
113streams without introducing a SQLite runtime wallet loader yet. 114
114`-walletsqliteverify=FILE` compares the current `wallet.dat` stream with a 115The migration tooling remains available for scripted use. `-walletsqliteexport=FILE`
115SQLite export and prints only record counts and mismatch counts. 116streams every raw `wallet.dat` record through `ScanWalletRecords()` into a SQLite
116`-walletsqliterestore=FILE` rebuilds `wallet.dat` from a SQLite export only in 117file in one transaction, refusing to overwrite. `-walletsqliteverify=FILE`
117an empty data directory, then the verifier can prove the restored BDB file still 118compares the two streams and prints only counts. `-walletsqliteloadcheck=FILE`
118matches the export byte-for-byte. Failed exports remove the incomplete SQLite 119parses an export with the same record types the loader uses.
119file and failed restores remove the incomplete `wallet.dat`. 120`-walletsqliterestore=FILE` rebuilds a `wallet.dat` from an export in an empty
120`-walletsqliteloadcheck=FILE` parses a SQLite export using the same record 121data directory. Failed exports and restores remove their incomplete output.
121types the wallet loader understands, so a migration can prove the export is not
122just byte-preserving but loader-compatible before runtime activation.
123`-walletsqlite=FILE` is the first runtime staging flag: it loads a SQLite export
124through the startup wallet path, initializes the in-memory wallet state, reports
125success, and exits before network, mining, GUI, or wallet mutation begins. It is
126deliberately read-only until the write path is routed to SQLite as well.
127 122
128## `blkindex.dat` 123## `blkindex.dat`
129 124
@@ -150,12 +145,14 @@ conversion path.
150The old IP address database (`addr.dat`) has been removed. Peer discovery is 145The old IP address database (`addr.dat`) has been removed. Peer discovery is
151handled by Nostr, `.btf`, rendezvous relays, and direct onion peers. 146handled by Nostr, `.btf`, rendezvous relays, and direct onion peers.
152 147
153## Recommended Migration Order 148## Migration Order (shipped in v1.2.20)
154 149
1551. Keep adding storage self-tests around current BDB behavior. 1501. Storage self-tests were added around Berkeley DB behavior first.
1562. Add an experimental wallet backend behind an explicit flag. 1512. The SQLite backend landed behind an explicit flag, then behind a
1573. Implement BDB-to-new-wallet migration as a one-way copy with a backup. 152 `wallet-backend` marker and a GUI conversion wizard.
1584. Make the new wallet backend default only after cross-version restore, 1533. BDB-to-SQLite migration is a one-way copy that leaves `wallet.dat` as a backup.
159 encryption, backup, and recovery-audit tests pass. 1544. SQLite became the default for new wallets only after cross-version restore,
155 encryption, backup, and recovery-audit tests passed -- and a live coin-proof
156 of receive, spend, restart, and encrypt on real mainnet funds.
1605. Treat `blkindex.dat` separately; it is safer to rebuild than to migrate under 1575. Treat `blkindex.dat` separately; it is safer to rebuild than to migrate under
161 pressure. 158 pressure.
diff --git a/src/main_gui.cpp b/src/main_gui.cpp
index 55ca8c6..943799f 100644
--- a/src/main_gui.cpp
+++ b/src/main_gui.cpp
@@ -182,7 +182,7 @@ static void PrintUsage()
182 printf(" /walletstoragecheck (fail closed on inconsistent or unsafe\n"); 182 printf(" /walletstoragecheck (fail closed on inconsistent or unsafe\n");
183 printf(" wallet.dat storage state)\n"); 183 printf(" wallet.dat storage state)\n");
184 printf(" /walletsqliteexport=FILE (copy raw wallet.dat records into an\n"); 184 printf(" /walletsqliteexport=FILE (copy raw wallet.dat records into an\n");
185 printf(" experimental SQLite store, then exit)\n"); 185 printf(" SQLite store, then exit)\n");
186 printf(" /walletsqliteverify=FILE (compare wallet.dat with a SQLite export,\n"); 186 printf(" /walletsqliteverify=FILE (compare wallet.dat with a SQLite export,\n");
187 printf(" printing counts only, then exit)\n"); 187 printf(" printing counts only, then exit)\n");
188 printf(" /walletsqliterestore=FILE (rebuild wallet.dat from a SQLite export in\n"); 188 printf(" /walletsqliterestore=FILE (rebuild wallet.dat from a SQLite export in\n");
@@ -680,17 +680,17 @@ int main(int argc, char* argv[])
680 AttachTerminal(); 680 AttachTerminal();
681 FatalStartupError(fHeadlessStartup, 681 FatalStartupError(fHeadlessStartup,
682 strprintf("Unknown wallet backend '%s'.", strWalletBackend.c_str()), 682 strprintf("Unknown wallet backend '%s'.", strWalletBackend.c_str()),
683 "Use -walletbackend=sqlite for the experimental SQLite backend, " 683 "Use -walletbackend=sqlite for the SQLite backend, or "
684 "or omit -walletbackend (or -walletbackend=bdb) to use the " 684 "-walletbackend=bdb for the Berkeley DB wallet.dat.");
685 "default Berkeley DB wallet.dat.");
686 return 1; 685 return 1;
687 } 686 }
688 687
689 if (strWalletBackend == "sqlite") 688 if (strWalletBackend == "sqlite")
690 { 689 {
691 // Opt-in, experimental. The node runs on <datadir>/wallet.sqlite and 690 // Explicit opt-in via the flag. The node runs on
692 // never opens wallet.dat, but the user has to have exported one 691 // <datadir>/wallet.sqlite and never opens wallet.dat, but the user
693 // first -- there is no silent migration, and wallet.dat is left 692 // has to have exported one first -- there is no silent migration,
693 // and wallet.dat is left
694 // exactly where it is so a plain restart goes back to it. 694 // exactly where it is so a plain restart goes back to it.
695 AttachTerminal(); 695 AttachTerminal();
696 string strSQLitePath = GetAppDir() + "/wallet.sqlite"; 696 string strSQLitePath = GetAppDir() + "/wallet.sqlite";
@@ -724,7 +724,7 @@ int main(int argc, char* argv[])
724 } 724 }
725 725
726 fprintf(stderr, 726 fprintf(stderr,
727 "Wallet backend: SQLite (experimental)\n" 727 "Wallet backend: SQLite\n"
728 " %s\n" 728 " %s\n"
729 "Your Berkeley DB wallet.dat is left untouched. Restart without\n" 729 "Your Berkeley DB wallet.dat is left untouched. Restart without\n"
730 "-walletbackend=sqlite to go back to it.\n", 730 "-walletbackend=sqlite to go back to it.\n",
diff --git a/src/walletcmd.h b/src/walletcmd.h
index 38cbfdb..7013d44 100644
--- a/src/walletcmd.h
+++ b/src/walletcmd.h
@@ -70,9 +70,9 @@ int CmdWalletStorageAudit(const std::string& strJsonOut = "");
70// or migration-unsafe mix of storage records. 70// or migration-unsafe mix of storage records.
71int CmdWalletStorageCheck(); 71int CmdWalletStorageCheck();
72 72
73// Diagnostic/migration staging: copy every raw wallet.dat record into an 73// Copy every raw wallet.dat record into a SQLite key/value store, preserving
74// experimental SQLite key/value store, preserving serialized bytes. This does 74// serialized bytes. A migration/scripting tool; it does not change which backend
75// not change which backend the runtime wallet opens. 75// the runtime wallet opens.
76int CmdWalletSQLiteExport(const std::string& strDest); 76int CmdWalletSQLiteExport(const std::string& strDest);
77 77
78// Export the currently-loaded wallet to a SQLite file (shared by the command 78// Export the currently-loaded wallet to a SQLite file (shared by the command