upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/src/rpc.cpp
diff options
context:
space:
mode:
authorbitflash-maintainer <noreply@bitflash.invalid>2026-08-23 05:21:31 +0000
committerbitflash-maintainer <noreply@bitflash.invalid>2026-08-23 05:21:31 +0000
commita9b4f8940907c2bf51e431172b628de89734a908 (patch)
treeff0a0b19d7d7754b809e35c6eae43dae7a32e1e4 /src/rpc.cpp
parent6d4c3f153cf44cefead39b21191dfb33491bfa5a (diff)
parentbc3de301d9ed0e5fb5e2034be2e066d77cf23804 (diff)
Merge upstream v1.2.18 into the fork
Merges upstream tag v1.2.18 (bc3de30) into our 24 commits. The merge base is a259215, upstream's own commit, whose src/ tree is byte-identical to 68de575 -- our commits were replayed onto a259215 first so that git had a real base to work from. That produced 28 conflict hunks in 9 files instead of the 21 conflict events / 51 hunks a direct cherry-pick onto the tag would have forced, most of which were artifacts of lifting our commits away from their own predecessors. No consensus change. main.cpp's GetBlockValue, GetNextWorkRequired, ConnectInputs, ConnectBlock, CheckBlock, AcceptBlock and AcceptTransaction are untouched by this merge; script.cpp, serialize.h and randomx_pow.h are identical on both sides. The frozen-chain oracle reproduces the pinned UTXO root 116a2b6b... byte for byte, and the merged binary loads the same 12,586-block chain to the same tip aac622f0a3b2f8... with all blocks re-verified and none rejected. Which side won, per file and per hunk. src/rpc.cpp -- 15 hunks. 1. RebuildJob transaction selection. THEIRS for the multi-pass loop and the nFees fix, OURS for the cap. Their vfAlreadyAdded/fFoundSomething loop lands dependent transactions that our single pass dropped deterministically, and their nFees is accumulated across the scan instead of being redeclared inside it -- ours passed a hardcoded 0 to GetBlockValue, so every pool block built here discarded its fees. But their bound is MAX_SIZE/2 = 16 MB, and MAX_BLOCK_SIZE is 1 MB, so their template can exceed what their own CheckBlock accepts. Cap kept at MAX_BLOCK_SIZE/2, with our inner break so a single pass cannot overshoot by more than one transaction. Our RebuildJob sync gating and deaf-node detection are kept; upstream has none. 2. struct PendingPayout. OURS for the key, THEIRS for paidTxids. Their id is the string "block-<height>", which is height keying -- the exact bug 44849e6 fixed after a 280-block reorg left ~14.4k BTF owed against blocks that no longer existed. Height keying also lets the crash-recovery probe match the previous round's transaction after a reorg and silently skip a miner, so their id is dropped entirely and blockHash stays the key. paidTxids is taken because it is what makes an entry survivable. Their nextAttempt and our retryAfter are the same concept under two names; ours is kept and theirs dropped. matureAtHeight gained an explicit = 0, which neither side had in our style of initialiser. 3./4. Ledger writer and loader. Mechanical follow-ons of 2. The loader keeps ours and takes their tolerance for an entry with no usable height, guarded so it cannot fire on a well-formed entry. One redundancy was removed rather than merged: their unguarded `if (obj.contains("blockHeight")) get<int>()` sat two lines above our is_number-guarded version, and an unguarded get<int>() on a malformed value throws out of the whole loop, losing every entry in the file. Verified against the live production pending_payouts.json (2 entries, keys amounts/blockHash/blockHeight/matureAtHeight): loads both. A file in the new shape, with retryAfter and paidTxids, also loads. 5. Insertion point after the loader. BOTH. Our PayingRecord/gPaying in-flight ledger and their PoolRoundProof/PoolRoundToJson are disjoint features that landed at the same offset. Both kept, in full. 6. Writer tail. THEIRS, copied into our writer. Checking fclose() matters here because fclose is where the buffered write actually reaches the OS: renaming a short .tmp over a good file is precisely the truncation the atomic write exists to prevent. The Windows remove()-before-rename() is required because rename() there will not replace an existing file. 7. Loader bodies. BOTH -- LoadPaying and LoadPoolRounds are different functions that happened to share a common tail. 8. Function block, 111 vs 36 lines. BOTH, with one deliberate deletion. PayoutBlockStillOnChain, PayoutComment, RecordPoolRound, MarkPoolRoundPaid all kept. WalletAlreadyPaid is DELETED and ReconcilePayouts rewritten, because our reconciliation and their FindRecordedPoolPayout are two answers to the same crash-recovery question and running both would give the same debt two authorities. Their wallet probe wins: it matches structured metadata rather than a comment string, so it can tell two rounds paying the same address the same amount apart. ReconcilePayouts keeps only the job that is still real -- dropping in-flight records the ledger already shows paid, and arming each restored record for its one wallet probe. It no longer touches gPendingPayouts at all; under the new model dropping ledger entries there would reintroduce the loss this merge adopts their model to prevent. Their gPayoutExecMutex/gPayoutThreadRunning re-declarations were dropped as duplicates of ours, and gPayoutExecQueue was not taken: gPaying is the exec queue in this tree, at finer granularity. 9. QueuePayouts signature. THEIRS for the round-proof arguments, OURS for the type of blockHash. It stays a uint256 and is converted to hex only where the round proof needs text -- a ledger key that has been through ToString() is a key that can be compared wrong. 10. Maturity policy. OURS. matureAtHeight = blockHeight + COINBASE_SPENDABLE_DEPTH (120), not + COINBASE_MATURITY (100). At depth 100 the block's own coinbase is still 20 blocks short of spendable, so the payment must come from unrelated mature coin and the pool runs a permanent float. Our own history contradicts itself here -- d6972a0 argued for COINBASE_MATURITY citing "96% of 934 payments", 44849e6 reversed it on the grounds that the measurement was taken while the node was alone on a private fork paying itself. HEAD's position is kept. This is a considered disagreement with upstream, not an oversight. The live production ledger confirms it: both entries have matureAtHeight - blockHeight = 120. 11. Comment above FlushMaturePayouts. Rewritten to describe the merged design. 12. FlushMaturePayouts, the central difference. THEIRS for the model. The ledger entry now survives on disk until every recipient has a txid, instead of being erased as soon as its payments were handed off -- the old ordering left a window where the debt was described by nothing at all, and a container restart on 2026-07-30 silently dropped ~1,156 BTF through it. Our gPaying is kept on top as the in-flight working set, because it carries the per-recipient retry and deferral state a per-round nextAttempt cannot express. It is explicitly a cache derived from the ledger, not a second record of debt: if it were lost the next flush would rebuild it. 13. PayoutThreadFn, 79 vs 70 lines. OURS for the loop and the reorg re-check, THEIRS for the per-recipient dedupe. The re-check on every attempt is load-bearing and upstream has no equivalent -- it decides once, at maturity, and a payment can sit through hours of retries during which the reorg that kills its funding block can land. Their paidTxids check is taken, against the ledger rather than against the batch. Our deferral policy is kept: waiting for coin is not a failure and must not consume the retry budget. Two fixes upstream does not have are folded in here rather than deferred to a follow-up commit: - FindRecordedPoolPayout runs once per record, on its first attempt, not on every attempt. It is a linear scan of mapWallet under cs_mapWallet, per recipient; upstream repeats it for every retry, which with 2000 rotating payout addresses and a budget of 1440 retries is a full wallet scan per address per minute for a day. The question it answers can only change across a restart, and every restart re-arms the flag. - The ledger file write no longer happens under gPayoutLedgerMutex in the per-recipient loop. SavePendingPayoutsLocked was split into a serialise step that needs the lock and a write step that does not, so the mutex is held only long enough to read the structure. The orphan scan in FlushMaturePayouts was likewise restructured so the ledger mutex is not held across PayoutBlockStillOnChain's cs_main acquisition. One case the merged model created and neither side handled: under survives-until-fully-paid, our GIVING UP path dropped the in-flight record but left the recipient owed in the ledger forever, so every later flush would explode it again -- a forfeit that silently becomes an infinite retry. ForfeitPayoutRecipient writes the amount off so the entry can retire. 14. QueuePayouts call site. Follow-on of 9. 15. ThreadRPCServer startup. BOTH. LoadPoolRounds added; LoadPaying and ReconcilePayouts stay, and stay before the payout thread starts. Crash safety, stated as the property that must hold: after kill -9 at any point, restart pays each recipient exactly once. Before SendMoney, the ledger has no txid and the payment is retried. Between SendMoney and the ledger write, the wallet holds the transaction and the first-attempt probe finds it. After the ledger write, paidTxids suppresses it. If the in-flight file is lost entirely, the ledger rebuilds it and the probe still covers the middle window. src/nostr.cpp -- 3 hunks. THEIRS as the base, with two things restored from ours. Theirs loads the Windows ROOT certificate store, which ours lacks and genuinely needs, since OpenSSL's default verify paths are usually empty there and our version would fail closed. But the plan's claim that theirs is a superset of our 06bceb4 is FALSE: upstream's nostr.cpp contains no min_proto_version, no TLS1_2, no SSL_OP_NO_*, and no set_options anywhere, so taking it wholesale would have removed the TLS-version floor outright. Restored. Second, their SSL_get_verify_result check sits after a successful handshake, where with SSL_VERIFY_PEER set it is nearly dead code -- a bad certificate aborts inside SSL_connect. The reporting belongs in the failure branch, which is where the merge put it; their post-handshake duplicate was dropped. Our relay-list and socket-pooling hunks auto-merged and were verified intact. src/net.cpp -- 1 hunk. THEIRS. Their guard, hSocket == INVALID_SOCKET || pnode->hSocket != hSocket, subsumes ours and additionally catches descriptor recycling, which ours did not; our version also referenced a pnode that only existed in the select() loop. Our other db34011 hunk, nulling pnode->hSocket in the poll-failure branch, auto-merged and was NOT re-applied -- plan section 7 Batch D says to re-apply it, and that instruction is stale. Two pieces of residue the merge left behind were cleaned up: a comment still describing select() semantics, and a pnode->hSocket != INVALID_SOCKET guard made redundant by a continue two lines above it. src/main.cpp -- 2 hunks. h1 BOTH, pure add/add adjacency between our ParticipantHashThreadCount and their SecureClear forward declarations. h2 OURS, and their handler discarded: 44004e3 no-ops checkorder/submitorder but keeps market.cpp and market.h, whereas 11766be deletes the marketplace outright including CRequestTracker, subscribe/publish and mapProducts. Ours is strictly more. src/main_gui.cpp -- 2 hunks. BOTH, additive on both sides. Our /payoutaddr, /payoutfile and /hashthreads alongside their /stratumbridge and /stratumbridgeport. src/btfrv.cpp -- 1 hunk, not predicted by the plan. False adjacency: our INVALID_SOCKET check and their #endif landed at the same offset. Ordering is load-bearing -- our check goes inside the #else arm, before the #endif, or the non-proxy build silently loses it. The BITFLASH_NODE_BUILD arm already returns early, so both paths now have the check. Makefile -- 2 hunks plus a hazard with no conflict marker. THEIRS for both dependency-fetch recipes: checkout --detach FETCH_HEAD followed by test "$(git rev-parse HEAD)" = "$(..._COMMIT)" asserts the pin actually landed, where ours only checked it out. Everything of ours that they lack is kept: the appimagetool 1.9.1 release pin with a sha256 check, where upstream fetches the mutable "continuous" tag and executes it undigested, and the move of both the tool and the AppDir out of /tmp, where any local user could pre-place an executable and have it run as whoever runs make. The hazard: both sides added a pin block in a different place, so git merged both, outside any conflict region. SECP256K1_COMMIT and RANDOMX_COMMIT ended up assigned twice, near L16 and near L44, and GNU Make takes the last := -- a naive resolution would have silently built against our old secp256k1 pin while appearing to adopt upstream's. The duplicate block was deleted by hand. Each of the four variables is now assigned exactly once, and make -n resolves SECP256K1_COMMIT to 7fecac74 with 6e2c8bc4 appearing nowhere. Also applied here, from plan phase 4: SECP256K1_COMMIT moved to upstream's 7fecac74aed8e1fd9078380d67dd04663705c989 so signature validation matches the binaries the rest of the network runs. secp256k1 was rebuilt from scratch against the new pin on the build host and the checked-out dependency commit verified. RANDOMX_COMMIT already matched at 1e9d4b2d and is unchanged. src/Makefile, src/makefile.mingw -- 1 hunk each, both the HEADERS list. Ours drops market.h, theirs adds proxy.h; the resolution does both. Taking theirs wholesale would have left the Makefiles referencing a header git had already deleted in our favour, silently, without a conflict. Files the plan predicted would conflict and did not: net.h, main.h and db.cpp auto-merged. Verified rather than assumed. net.h carries both our BTF_MSG_WAIT_MAX_SECS work, including the constructor fix from 2d84982 that starts nMsgWaitSince and nMsgWaitSize at 0, and their comment rewrite with extern int nPeersWatched. The dead members that fix removed are gone from every file. Deviations from MERGE-1.2.18-PLAN.md. - Section 6's mechanism was not used. Replayed onto the real merge base and merged, rather than cherry-picking onto the tag. Same result, 28 hunks instead of 51. - Section 2 calls 09ff575 a superset of our 06bceb4. It is not; upstream has no TLS-version floor at all. Ours restored on top of theirs. - Section 7 Batch D says to re-apply our db34011 line into their poll-failure branch. It auto-merged; re-applying would have duplicated it. - Section 2 says do not take the stratum bridge. It is taken, and left off. fStratumBridge defaults false in main.cpp, the thread is only spawned when /stratumbridge is passed, and the listener binds 127.0.0.1 only. Excising it would cost real divergence and future conflict for no gain while it is disabled. Its defects -- select()/FD_SET on a node that can hold more than 1024 descriptors, and an uncapped thread per connection -- are real only when it is enabled and are being reported upstream separately. - Section 7's staged batches were not run as separate gated commits. The conflict surface is interleaved within single functions, chiefly the payout ledger, and splitting it would have meant committing states that do not build. The merge is one commit, as briefed. - Batch F's two fixes are folded into hunk 13 rather than deferred, because they change the same lines the resolution rewrites. - The plan did not anticipate the btfrv.cpp conflict, the duplicate pin block, the market.h trap in the two Makefiles, or the give-up/infinite-retry interaction created by adopting upstream's ledger model.
Diffstat (limited to 'src/rpc.cpp')
-rw-r--r--src/rpc.cpp1152
1 files changed, 990 insertions, 162 deletions
diff --git a/src/rpc.cpp b/src/rpc.cpp
index 03d1f44..8af2fbe 100644
--- a/src/rpc.cpp
+++ b/src/rpc.cpp
@@ -236,35 +236,73 @@ static bool RebuildJob()
236 CBlock block; 236 CBlock block;
237 block.vtx.push_back(txNew); 237 block.vtx.push_back(txNew);
238 238
239 // Fill the block the same way BitcoinMiner() does, because this is the
240 // same job: a block that gets mined and submitted for real.
241 //
242 // It used to differ in two ways, and both were bugs.
243 //
244 // One pass over mapTransactions is not enough. That map is keyed by
245 // hash, so it has no dependency order, and a transaction spending the
246 // still-unconfirmed output of another (change from a recent send, most
247 // commonly) fails ConnectInputs whenever its parent happens to sit
248 // later in map order. A single pass then drops it -- and because map
249 // order does not change between rebuilds, every later rebuild drops it
250 // again at the same point. Deterministic exclusion, not bad luck: if
251 // pool operators mine most of the blocks, a transaction of that shape
252 // can stay unconfirmed for as long as its parent does.
253 //
254 // Diagnosis and fix both come from Vic-Nas, who found it in his fork
255 // of this code and described it exactly. The fork has since been
256 // deleted, so this carries it back rather than losing it.
257 int64 nFees = 0;
239 { 258 {
240 CTxDB txdb("r"); 259 CTxDB txdb("r");
241 CRITICAL_BLOCK(cs_mapTransactions) 260 CRITICAL_BLOCK(cs_mapTransactions)
242 { 261 {
243 map<uint256, CTxIndex> pool; 262 map<uint256, CTxIndex> pool;
263 vector<char> vfAlreadyAdded(mapTransactions.size());
264 bool fFoundSomething = true;
244 unsigned int sz = 0; 265 unsigned int sz = 0;
245 for (auto& kv : mapTransactions) { 266 // The cap is MAX_BLOCK_SIZE/2, not MAX_SIZE/2. MAX_SIZE is the
246 CTransaction& tx = kv.second; 267 // 32 MB serialisation ceiling, so half of it is 16 MB -- this
247 if (tx.IsCoinBase() || !tx.IsFinal()) continue; 268 // builder would happily assemble a template that its own
248 int64 nFees = 0; 269 // CheckBlock() rejects at the 1 MB consensus cap. This is the
249 map<uint256, CTxIndex> tmp(pool); 270 // pool's job builder, so a block over the cap is not one
250 if (!tx.ConnectInputs(txdb, tmp, CDiskTxPos(1,1,1), 0, nFees, 271 // wasted solve, it is every miner on the pool working a
251 false, true, tx.GetMinFee(block.vtx.size() < 100))) 272 // template the network will reject. The inner break bounds the
252 continue; 273 // overshoot to a single transaction within a pass.
253 pool = tmp; 274 while (fFoundSomething && sz < MAX_BLOCK_SIZE / 2)
254 block.vtx.push_back(tx); 275 {
255 sz += ::GetSerializeSize(tx, SER_NETWORK); 276 fFoundSomething = false;
256 // Must stay under the consensus cap CheckBlock() enforces. 277 unsigned int n = 0;
257 // This is the pool's job builder, so a block over the cap 278 for (map<uint256, CTransaction>::iterator mi = mapTransactions.begin();
258 // is not just one wasted solve -- it is every miner on the 279 mi != mapTransactions.end(); ++mi, ++n)
259 // pool working a template the network will reject. The 280 {
260 // transaction is already pushed above, so this can overshoot 281 if (vfAlreadyAdded[n])
261 // by one; half the cap leaves room for that. 282 continue;
262 if (sz > MAX_BLOCK_SIZE / 2) break; 283 CTransaction& tx = (*mi).second;
284 if (tx.IsCoinBase() || !tx.IsFinal()) continue;
285 map<uint256, CTxIndex> tmp(pool);
286 if (!tx.ConnectInputs(txdb, tmp, CDiskTxPos(1,1,1), 0, nFees,
287 false, true, tx.GetMinFee(block.vtx.size() < 100)))
288 continue;
289 swap(pool, tmp);
290 block.vtx.push_back(tx);
291 sz += ::GetSerializeSize(tx, SER_NETWORK);
292 vfAlreadyAdded[n] = true;
293 fFoundSomething = true;
294 if (sz > MAX_BLOCK_SIZE / 2) break;
295 }
263 } 296 }
264 } 297 }
265 } 298 }
266 299
267 block.vtx[0].vout[0].nValue = block.GetBlockValue(pindexPrev->nHeight + 1, 0); 300 // And the fees actually go to the coinbase now. nFees was declared
301 // inside the scan loop, so every transaction reset it and the total was
302 // thrown away, while this line passed a hardcoded 0. A pool operator
303 // collected fees into a block and paid none of them to itself. Solo
304 // mining has always passed nFees here; only this builder did not.
305 block.vtx[0].vout[0].nValue = block.GetBlockValue(pindexPrev->nHeight + 1, nFees);
268 block.hashPrevBlock = pindexPrev->GetBlockHash(); 306 block.hashPrevBlock = pindexPrev->GetBlockHash();
269 block.hashMerkleRoot = block.BuildMerkleTree(); 307 block.hashMerkleRoot = block.BuildMerkleTree();
270 block.nTime = max((unsigned int)(pindexPrev->GetMedianTimePast() + 1), 308 block.nTime = max((unsigned int)(pindexPrev->GetMedianTimePast() + 1),
@@ -297,26 +335,110 @@ struct PendingPayout {
297 // supposed to pay it: a 280-block reorg once left ~14.4k BTF of debt 335 // supposed to pay it: a 280-block reorg once left ~14.4k BTF of debt
298 // owed against blocks that no longer existed, which then drained the 336 // owed against blocks that no longer existed, which then drained the
299 // wallet paying fork-era miners out of real-chain coin. 337 // wallet paying fork-era miners out of real-chain coin.
338 //
339 // Upstream keys these on the string "block-<height>". That is the same
340 // bug this fork already paid for, and height keying additionally lets
341 // FindRecordedPoolPayout match the *previous* round's transaction after a
342 // reorg and silently skip a miner, so their `id` is deliberately dropped.
300 int blockHeight = 0; 343 int blockHeight = 0;
301 // Must be explicitly zeroed: uint256's default constructor is empty 344 // Must be explicitly zeroed: uint256's default constructor is empty
302 // (uint256.h:514), so leaving this unassigned gives every ledger entry 345 // (uint256.h:514), so leaving this unassigned gives every ledger entry
303 // loaded from an older file a garbage hash, which then matches no block 346 // loaded from an older file a garbage hash, which then matches no block
304 // and gets voided as "orphaned". That mistake voided a live payout. 347 // and gets voided as "orphaned". That mistake voided a live payout.
305 uint256 blockHash = 0; // zero = entry predates hash keying 348 uint256 blockHash = 0; // zero = entry predates hash keying
306 int matureAtHeight; 349 int matureAtHeight = 0;
307 std::map<std::string, int64> amounts; // address -> satoshis 350 std::map<std::string, int64> amounts; // address -> satoshis
351 // From upstream: which recipients already have a transaction. This is what
352 // lets an entry stay on disk until every address in `amounts` is paid,
353 // instead of being erased the moment the batch is handed off.
354 std::map<std::string, std::string> paidTxids; // address -> txid
308 int retries = 0; // exec-thread only: SendMoney attempts so far 355 int retries = 0; // exec-thread only: SendMoney attempts so far
309 int deferrals = 0; // exec-thread only: waits for spendable coin 356 int deferrals = 0; // exec-thread only: waits for spendable coin
357 // Upstream's `nextAttempt` is this field under another name; ours is kept
358 // so the retry/deferral policy above stays coherent with it.
310 int64 retryAfter = 0; // exec-thread only: unix time, 0 = ready now 359 int64 retryAfter = 0; // exec-thread only: unix time, 0 = ready now
311}; 360};
312static std::vector<PendingPayout> gPendingPayouts; 361static std::vector<PendingPayout> gPendingPayouts;
362static std::mutex gPayoutLedgerMutex;
363
364struct PoolRoundProof {
365 std::string id;
366 int blockHeight;
367 int matureAtHeight;
368 int64 createdAt;
369 std::string blockHash;
370 std::string coinbaseTxid;
371 std::string foundBy;
372 double feePercent;
373 uint64 totalShares;
374 int64 coinbaseValue;
375 int64 subsidy;
376 int64 minerReward;
377 int64 operatorReward;
378 int skippedDust;
379 int64 forfeitedBadAddress;
380 std::map<std::string, uint64> shares;
381 std::map<std::string, int64> payouts;
382 std::map<std::string, std::string> paidTxids;
383
384 PoolRoundProof()
385 : blockHeight(0), matureAtHeight(0), createdAt(0), feePercent(0.0),
386 totalShares(0), coinbaseValue(0), subsidy(0), minerReward(0), operatorReward(0),
387 skippedDust(0), forfeitedBadAddress(0) {}
388};
389static std::vector<PoolRoundProof> gPoolRounds;
390static std::mutex gPoolRoundsMutex;
391static const size_t MAX_POOL_ROUNDS = 200;
313 392
314static std::string PayoutFilePath() 393static std::string PayoutFilePath()
315{ 394{
316 return GetAppDir() + "/pending_payouts.json"; 395 return GetAppDir() + "/pending_payouts.json";
317} 396}
318 397
319static void SavePendingPayouts() 398static std::string PoolRoundsFilePath()
399{
400 if (!strPoolRoundsFile.empty())
401 return strPoolRoundsFile;
402 return GetAppDir() + "/pool_rounds.json";
403}
404
405// Identity of a payout round. Used as the pool-round proof id and, more
406// importantly, as the `pool_payout_id` written into wallet transaction
407// metadata and matched by FindRecordedPoolPayout().
408//
409// Derived from the block HASH, not from its height alone. Upstream uses
410// "block-<height>", which names two different blocks identically across a
411// reorg -- and because this id is what the crash-recovery probe matches on, a
412// height-keyed id lets that probe find the *previous* round's transaction and
413// conclude a miner has already been paid when they have not. The height stays
414// in the string for readability only.
415//
416// A zero hash means the entry predates hash keying; those keep the old form so
417// anything already recorded against them still matches.
418static std::string PayoutIdForBlock(int blockHeight, const uint256& blockHash)
419{
420 if (blockHash == 0)
421 return strprintf("block-%d", blockHeight);
422 return strprintf("block-%d-%s", blockHeight,
423 blockHash.ToString().substr(0, 16).c_str());
424}
425
426static bool PayoutFullyPaid(const PendingPayout& pp)
427{
428 for (const auto& kv : pp.amounts)
429 if (pp.paidTxids.count(kv.first) == 0)
430 return false;
431 return true;
432}
433
434// Serialise the ledger. Caller must hold gPayoutLedgerMutex.
435//
436// Split from the file write so the payout thread can hold the mutex just long
437// enough to read the structure and then do the I/O with it released. Upstream
438// writes the file inside the per-recipient loop with gPayoutLedgerMutex held,
439// while the event loop's FlushMaturePayouts blocks on that same mutex -- one
440// slow fsync per recipient, on the thread the miners' I/O shares.
441static std::string SerializePendingPayoutsLocked()
320{ 442{
321 nlohmann::json arr = nlohmann::json::array(); 443 nlohmann::json arr = nlohmann::json::array();
322 for (const auto& pp : gPendingPayouts) { 444 for (const auto& pp : gPendingPayouts) {
@@ -324,21 +446,49 @@ static void SavePendingPayouts()
324 obj["blockHeight"] = pp.blockHeight; 446 obj["blockHeight"] = pp.blockHeight;
325 obj["blockHash"] = pp.blockHash.ToString(); 447 obj["blockHash"] = pp.blockHash.ToString();
326 obj["matureAtHeight"] = pp.matureAtHeight; 448 obj["matureAtHeight"] = pp.matureAtHeight;
449 obj["retryAfter"] = pp.retryAfter;
327 nlohmann::json amounts = nlohmann::json::object(); 450 nlohmann::json amounts = nlohmann::json::object();
328 for (const auto& kv : pp.amounts) 451 for (const auto& kv : pp.amounts)
329 amounts[kv.first] = kv.second; 452 amounts[kv.first] = kv.second;
330 obj["amounts"] = amounts; 453 obj["amounts"] = amounts;
454 nlohmann::json paid = nlohmann::json::object();
455 for (const auto& kv : pp.paidTxids)
456 paid[kv.first] = kv.second;
457 obj["paidTxids"] = paid;
331 arr.push_back(obj); 458 arr.push_back(obj);
332 } 459 }
460 return arr.dump(2);
461}
462
463// Write the serialised ledger. Holds no lock; safe to call with the mutex
464// released. Atomic tmp+rename so a crash mid-write cannot truncate the record
465// of who is still owed money.
466static void WritePendingPayoutsFile(const std::string& s)
467{
333 std::string path = PayoutFilePath(); 468 std::string path = PayoutFilePath();
334 FILE* f = fopen(path.c_str(), "w"); 469 std::string tmp = path + ".tmp";
335 if (f) { 470 FILE* f = fopen(tmp.c_str(), "w");
336 std::string s = arr.dump(2); 471 if (!f) {
337 fwrite(s.c_str(), 1, s.size(), f); 472 LogPrint("payout", "[payout] WARNING: could not write %s\n", tmp.c_str());
338 fclose(f); 473 return;
339 } else {
340 LogPrint("payout", "[payout] WARNING: could not write %s\n", path.c_str());
341 } 474 }
475 fwrite(s.c_str(), 1, s.size(), f);
476 if (fclose(f) != 0) {
477 LogPrint("payout", "[payout] WARNING: could not close %s\n", tmp.c_str());
478 return;
479 }
480#ifdef _WIN32
481 remove(path.c_str());
482#endif
483 if (rename(tmp.c_str(), path.c_str()) != 0)
484 LogPrint("payout", "[payout] WARNING: could not replace %s\n", path.c_str());
485}
486
487// Convenience for the cold paths (startup, queueing) where holding the mutex
488// across the write costs nothing. Caller must hold gPayoutLedgerMutex.
489static void SavePendingPayoutsLocked()
490{
491 WritePendingPayoutsFile(SerializePendingPayoutsLocked());
342} 492}
343 493
344static void LoadPendingPayouts() 494static void LoadPendingPayouts()
@@ -354,6 +504,7 @@ static void LoadPendingPayouts()
354 fclose(f); 504 fclose(f);
355 try { 505 try {
356 nlohmann::json arr = nlohmann::json::parse(s); 506 nlohmann::json arr = nlohmann::json::parse(s);
507 std::lock_guard<std::mutex> lk(gPayoutLedgerMutex);
357 for (const auto& obj : arr) { 508 for (const auto& obj : arr) {
358 PendingPayout pp; 509 PendingPayout pp;
359 pp.matureAtHeight = obj["matureAtHeight"].get<int>(); 510 pp.matureAtHeight = obj["matureAtHeight"].get<int>();
@@ -364,8 +515,24 @@ static void LoadPendingPayouts()
364 pp.blockHash.SetHex(obj["blockHash"].get<std::string>()); 515 pp.blockHash.SetHex(obj["blockHash"].get<std::string>());
365 if (obj.contains("blockHeight") && obj["blockHeight"].is_number()) 516 if (obj.contains("blockHeight") && obj["blockHeight"].is_number())
366 pp.blockHeight = obj["blockHeight"].get<int>(); 517 pp.blockHeight = obj["blockHeight"].get<int>();
518 // Upstream's tolerance for an entry with no usable height, kept
519 // because it costs nothing and an entry that cannot name its block
520 // is worse than one derived from its maturity. COINBASE_MATURITY
521 // and not COINBASE_SPENDABLE_DEPTH: this reverses whatever the
522 // writer of *that* file used, and only files old enough to lack
523 // blockHeight are affected -- they were written when maturity was
524 // still COINBASE_MATURITY.
525 if (pp.blockHeight == 0 && pp.matureAtHeight > COINBASE_MATURITY)
526 pp.blockHeight = pp.matureAtHeight - COINBASE_MATURITY;
527 // Absent in every file written before this merge; default 0 means
528 // "ready now", which is the right answer for a restored entry.
529 if (obj.contains("retryAfter") && obj["retryAfter"].is_number())
530 pp.retryAfter = obj["retryAfter"].get<int64>();
367 for (auto it = obj["amounts"].begin(); it != obj["amounts"].end(); ++it) 531 for (auto it = obj["amounts"].begin(); it != obj["amounts"].end(); ++it)
368 pp.amounts[it.key()] = it.value().get<int64>(); 532 pp.amounts[it.key()] = it.value().get<int64>();
533 if (obj.contains("paidTxids") && obj["paidTxids"].is_object())
534 for (auto it = obj["paidTxids"].begin(); it != obj["paidTxids"].end(); ++it)
535 pp.paidTxids[it.key()] = it.value().get<std::string>();
369 gPendingPayouts.push_back(pp); 536 gPendingPayouts.push_back(pp);
370 } 537 }
371 LogPrint("payout", "[payout] loaded %zu pending payout(s) from disk\n", 538 LogPrint("payout", "[payout] loaded %zu pending payout(s) from disk\n",
@@ -396,6 +563,17 @@ struct PayingRecord {
396 int retries = 0; 563 int retries = 0;
397 int deferrals = 0; 564 int deferrals = 0;
398 int64 retryAfter = 0; 565 int64 retryAfter = 0;
566 // Ask the wallet whether this payment already went out, once, before the
567 // first send attempt. Upstream asks on every attempt: FindRecordedPoolPayout
568 // is a linear scan of mapWallet under cs_mapWallet, per recipient, so with
569 // 2000 rotating payout addresses and a retry budget of 1440 that is a
570 // scan per address per minute for a day. The question it answers -- "did a
571 // previous process send this before it died?" -- can only change across a
572 // restart, and every restart re-sets this flag (ReconcilePayouts), so
573 // asking once per record is the whole of the answer.
574 //
575 // Not persisted: a record read back from disk is by definition a restart.
576 bool fProbeWallet = true;
399}; 577};
400static std::vector<PayingRecord> gPaying; // guarded by gPayoutExecMutex 578static std::vector<PayingRecord> gPaying; // guarded by gPayoutExecMutex
401static std::mutex gPayoutExecMutex; 579static std::mutex gPayoutExecMutex;
@@ -432,8 +610,19 @@ static void SavePayingLocked()
432 } 610 }
433 std::string s = arr.dump(2); 611 std::string s = arr.dump(2);
434 fwrite(s.c_str(), 1, s.size(), f); 612 fwrite(s.c_str(), 1, s.size(), f);
435 fflush(f); 613 // fclose() is where buffered data actually reaches the OS, so a failure
436 fclose(f); 614 // here means the .tmp is short -- renaming it over the real file would
615 // destroy the good copy. Upstream's check, taken because a truncated
616 // list of who is owed money is exactly what this function exists to
617 // prevent. rename() on Windows will not replace an existing file, hence
618 // the remove() first.
619 if (fclose(f) != 0) {
620 LogPrint("payout", "[payout] WARNING: could not close %s\n", tmp.c_str());
621 return;
622 }
623#ifdef _WIN32
624 remove(path.c_str());
625#endif
437 if (rename(tmp.c_str(), path.c_str()) != 0) 626 if (rename(tmp.c_str(), path.c_str()) != 0)
438 LogPrint("payout", "[payout] WARNING: could not rename %s -> %s\n", 627 LogPrint("payout", "[payout] WARNING: could not rename %s -> %s\n",
439 tmp.c_str(), path.c_str()); 628 tmp.c_str(), path.c_str());
@@ -475,6 +664,143 @@ static void LoadPaying()
475 } 664 }
476} 665}
477 666
667// ---------------------------------------------------------------------------
668// Pool round proofs (upstream cbdd06d). Disjoint from the in-flight ledger
669// above -- this is the operator's published record of how a round was split,
670// not an obligation to pay. Both sides added code at this offset; both are
671// kept.
672// ---------------------------------------------------------------------------
673static nlohmann::json PoolRoundToJson(const PoolRoundProof& r)
674{
675 nlohmann::json j;
676 j["id"] = r.id;
677 j["blockHeight"] = r.blockHeight;
678 j["matureAtHeight"] = r.matureAtHeight;
679 j["createdAt"] = r.createdAt;
680 j["blockHash"] = r.blockHash;
681 j["coinbaseTxid"] = r.coinbaseTxid;
682 j["foundBy"] = r.foundBy;
683 j["feePercent"] = r.feePercent;
684 j["totalShares"] = r.totalShares;
685 j["coinbaseValueSatoshis"] = r.coinbaseValue;
686 j["coinbaseValueBtf"] = FormatMoney(r.coinbaseValue);
687 j["subsidySatoshis"] = r.subsidy;
688 j["subsidyBtf"] = FormatMoney(r.subsidy);
689 j["minerRewardSatoshis"] = r.minerReward;
690 j["minerRewardBtf"] = FormatMoney(r.minerReward);
691 j["operatorRewardSatoshis"] = r.operatorReward;
692 j["operatorRewardBtf"] = FormatMoney(r.operatorReward);
693 j["skippedDust"] = r.skippedDust;
694 j["forfeitedBadAddressSatoshis"] = r.forfeitedBadAddress;
695 j["forfeitedBadAddressBtf"] = FormatMoney(r.forfeitedBadAddress);
696
697 j["shares"] = nlohmann::json::object();
698 for (const auto& kv : r.shares)
699 j["shares"][kv.first] = kv.second;
700
701 j["payouts"] = nlohmann::json::object();
702 for (const auto& kv : r.payouts) {
703 nlohmann::json p;
704 p["amountSatoshis"] = kv.second;
705 p["amountBtf"] = FormatMoney(kv.second);
706 std::map<std::string, std::string>::const_iterator tx = r.paidTxids.find(kv.first);
707 if (tx != r.paidTxids.end())
708 p["txid"] = tx->second;
709 else
710 p["txid"] = nullptr;
711 j["payouts"][kv.first] = p;
712 }
713 return j;
714}
715
716static void SavePoolRoundsLocked()
717{
718 nlohmann::json root;
719 root["schema"] = "bitflash-pool-rounds-1";
720 root["updatedAt"] = GetTime();
721 root["rounds"] = nlohmann::json::array();
722 for (const PoolRoundProof& r : gPoolRounds)
723 root["rounds"].push_back(PoolRoundToJson(r));
724
725 std::string path = PoolRoundsFilePath();
726 std::string tmp = path + ".tmp";
727 FILE* f = fopen(tmp.c_str(), "w");
728 if (!f) {
729 LogPrint("payout", "[payout] WARNING: could not write %s\n", tmp.c_str());
730 return;
731 }
732 std::string s = root.dump(2);
733 fwrite(s.c_str(), 1, s.size(), f);
734 if (fclose(f) != 0) {
735 LogPrint("payout", "[payout] WARNING: could not close %s\n", tmp.c_str());
736 return;
737 }
738#ifdef _WIN32
739 remove(path.c_str());
740#endif
741 if (rename(tmp.c_str(), path.c_str()) != 0)
742 LogPrint("payout", "[payout] WARNING: could not replace %s\n", path.c_str());
743}
744
745static void LoadPoolRounds()
746{
747 std::string path = PoolRoundsFilePath();
748 FILE* f = fopen(path.c_str(), "r");
749 if (!f) return;
750 fseek(f, 0, SEEK_END);
751 long sz = ftell(f);
752 fseek(f, 0, SEEK_SET);
753 std::string s(sz, '\0');
754 fread(&s[0], 1, sz, f);
755 fclose(f);
756
757 try {
758 nlohmann::json root = nlohmann::json::parse(s);
759 nlohmann::json arr = root.is_array() ? root : root["rounds"];
760 std::lock_guard<std::mutex> lk(gPoolRoundsMutex);
761 for (const auto& obj : arr) {
762 PoolRoundProof r;
763 if (obj.contains("id") && obj["id"].is_string()) r.id = obj["id"].get<std::string>();
764 if (obj.contains("blockHeight")) r.blockHeight = obj["blockHeight"].get<int>();
765 if (obj.contains("matureAtHeight")) r.matureAtHeight = obj["matureAtHeight"].get<int>();
766 if (obj.contains("createdAt")) r.createdAt = obj["createdAt"].get<int64>();
767 if (obj.contains("blockHash") && obj["blockHash"].is_string()) r.blockHash = obj["blockHash"].get<std::string>();
768 if (obj.contains("coinbaseTxid") && obj["coinbaseTxid"].is_string()) r.coinbaseTxid = obj["coinbaseTxid"].get<std::string>();
769 if (obj.contains("foundBy") && obj["foundBy"].is_string()) r.foundBy = obj["foundBy"].get<std::string>();
770 if (obj.contains("feePercent")) r.feePercent = obj["feePercent"].get<double>();
771 if (obj.contains("totalShares")) r.totalShares = obj["totalShares"].get<uint64>();
772 if (obj.contains("coinbaseValueSatoshis")) r.coinbaseValue = obj["coinbaseValueSatoshis"].get<int64>();
773 if (obj.contains("subsidySatoshis")) r.subsidy = obj["subsidySatoshis"].get<int64>();
774 if (obj.contains("minerRewardSatoshis")) r.minerReward = obj["minerRewardSatoshis"].get<int64>();
775 if (obj.contains("operatorRewardSatoshis")) r.operatorReward = obj["operatorRewardSatoshis"].get<int64>();
776 if (obj.contains("skippedDust")) r.skippedDust = obj["skippedDust"].get<int>();
777 if (obj.contains("forfeitedBadAddressSatoshis")) r.forfeitedBadAddress = obj["forfeitedBadAddressSatoshis"].get<int64>();
778 if (obj.contains("shares") && obj["shares"].is_object())
779 for (auto it = obj["shares"].begin(); it != obj["shares"].end(); ++it)
780 r.shares[it.key()] = it.value().get<uint64>();
781 if (obj.contains("payouts") && obj["payouts"].is_object())
782 for (auto it = obj["payouts"].begin(); it != obj["payouts"].end(); ++it) {
783 if (it.value().is_object()) {
784 r.payouts[it.key()] = it.value().value("amountSatoshis", (int64)0);
785 if (it.value().contains("txid") && it.value()["txid"].is_string())
786 r.paidTxids[it.key()] = it.value()["txid"].get<std::string>();
787 } else if (it.value().is_number_integer()) {
788 r.payouts[it.key()] = it.value().get<int64>();
789 }
790 }
791 if (!r.id.empty())
792 gPoolRounds.push_back(r);
793 }
794 while (gPoolRounds.size() > MAX_POOL_ROUNDS)
795 gPoolRounds.erase(gPoolRounds.begin());
796 LogPrint("payout", "[payout] loaded %zu pool round proof(s) from disk\n",
797 gPoolRounds.size());
798 } catch (const std::exception& e) {
799 LogPrint("payout", "[payout] WARNING: failed to parse %s: %s\n",
800 path.c_str(), e.what());
801 }
802}
803
478// Is the block that funds this payout still on the main chain? 804// Is the block that funds this payout still on the main chain?
479// 805//
480// A zero hash means the entry was written before payouts recorded one, so 806// A zero hash means the entry was written before payouts recorded one, so
@@ -499,85 +825,55 @@ static std::string PayoutComment(int blockHeight)
499 return strprintf("Pool payout height %d", blockHeight); 825 return strprintf("Pool payout height %d", blockHeight);
500} 826}
501 827
502// Did we already send this payment before dying? Looks for a wallet 828// Startup reconciliation.
503// transaction of ours carrying this payout's comment and paying exactly this
504// amount to this address.
505// 829//
506// Biased towards reporting "already paid" on any doubt: skipping a payment a 830// This used to be the crash-recovery mechanism: it scanned the wallet with
507// miner is owed is visible and recoverable from the log, while paying twice 831// WalletAlreadyPaid() and then dropped ledger entries that had already been
508// is neither. 832// exploded into in-flight records. Neither job is right any more.
509static bool WalletAlreadyPaid(const PayingRecord& r) 833//
510{ 834// The ledger is now authoritative and an entry survives until every recipient
511 uint160 h160; 835// has a txid recorded against it (upstream cbdd06d), so dropping ledger
512 if (!AddressToHash160(r.address, h160)) 836// entries here would reintroduce exactly the loss this merge is adopting that
513 return false; 837// model to prevent. And the wallet scan is now FindRecordedPoolPayout(), done
514 CScript want; 838// once per record on its first attempt, which is the same question asked in
515 want << OP_DUP << OP_HASH160 << h160 << OP_EQUALVERIFY << OP_CHECKSIG; 839// the one place that can act on the answer. Running both would be two
516 const std::string comment = PayoutComment(r.blockHeight); 840// authorities for the same debt.
517 841//
518 CRITICAL_BLOCK(cs_mapWallet) 842// What is left is the useful part: the in-flight list restored from disk may
519 { 843// contain records the ledger already shows as paid (we died between the send
520 for (std::map<uint256, CWalletTx>::iterator it = mapWallet.begin(); 844// and the in-flight file being rewritten). The ledger wins.
521 it != mapWallet.end(); ++it) {
522 CWalletTx& wtx = it->second;
523 // The comment is written only by the payout path, so its presence
524 // already identifies the transaction as one of ours.
525 std::map<std::string, std::string>::const_iterator ci =
526 wtx.mapValue.find("comment");
527 if (ci == wtx.mapValue.end() || ci->second != comment)
528 continue;
529 foreach (const CTxOut& txout, wtx.vout)
530 if (txout.nValue == r.amount && txout.scriptPubKey == want)
531 return true;
532 }
533 }
534 return false;
535}
536
537// Startup reconciliation. Drops in-flight records the wallet shows were
538// already sent, and drops ledger entries that were already exploded into
539// records (which happens if we died between writing the records and rewriting
540// the ledger -- both files then describe the same debt).
541static void ReconcilePayouts() 845static void ReconcilePayouts()
542{ 846{
543 std::lock_guard<std::mutex> lk(gPayoutExecMutex); 847 std::lock_guard<std::mutex> exec(gPayoutExecMutex);
848 std::lock_guard<std::mutex> ledger(gPayoutLedgerMutex);
544 849
545 size_t settled = 0; 850 size_t settled = 0;
546 int64 settledAmount = 0; 851 int64 settledAmount = 0;
547 for (auto it = gPaying.begin(); it != gPaying.end(); ) { 852 for (auto it = gPaying.begin(); it != gPaying.end(); ) {
548 if (WalletAlreadyPaid(*it)) { 853 bool paid = false;
854 for (const PendingPayout& pp : gPendingPayouts)
855 if (pp.blockHash == it->blockHash && pp.paidTxids.count(it->address)) {
856 paid = true;
857 break;
858 }
859 if (paid) {
549 settledAmount += it->amount; 860 settledAmount += it->amount;
550 settled++; 861 settled++;
551 it = gPaying.erase(it); 862 it = gPaying.erase(it);
552 } else { 863 } else {
864 // Every restored record gets one wallet probe before it is allowed
865 // to send: this is precisely the window where the wallet may hold
866 // a transaction the ledger has not recorded yet.
867 it->fProbeWallet = true;
553 ++it; 868 ++it;
554 } 869 }
555 } 870 }
556 if (settled) { 871 if (settled) {
557 LogPrint("payout", "[payout] %zu in-flight payment(s) totalling %s were" 872 LogPrint("payout", "[payout] %zu in-flight payment(s) totalling %s are"
558 " already sent before the last shutdown -- not resending\n", 873 " already recorded paid in the ledger -- not resending\n",
559 settled, FormatMoney(settledAmount).c_str()); 874 settled, FormatMoney(settledAmount).c_str());
560 SavePayingLocked();
561 }
562
563 size_t dropped = 0;
564 for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) {
565 bool exploded = false;
566 for (const auto& r : gPaying)
567 if (r.matureAtHeight == it->matureAtHeight && r.blockHash == it->blockHash) {
568 exploded = true;
569 break;
570 }
571 if (exploded) {
572 LogPrint("payout", "[payout] ledger entry for block %d is already"
573 " in flight -- dropping the duplicate\n", it->blockHeight);
574 it = gPendingPayouts.erase(it);
575 dropped++;
576 } else {
577 ++it;
578 }
579 } 875 }
580 if (dropped) SavePendingPayouts(); 876 SavePayingLocked();
581 877
582 if (!gPaying.empty()) { 878 if (!gPaying.empty()) {
583 int64 owed = 0; 879 int64 owed = 0;
@@ -587,12 +883,100 @@ static void ReconcilePayouts()
587 } 883 }
588} 884}
589 885
886static void RecordPoolRound(const PoolRoundProof& round)
887{
888 std::lock_guard<std::mutex> lk(gPoolRoundsMutex);
889 for (PoolRoundProof& existing : gPoolRounds) {
890 if (existing.id == round.id) {
891 existing = round;
892 SavePoolRoundsLocked();
893 return;
894 }
895 }
896 gPoolRounds.push_back(round);
897 while (gPoolRounds.size() > MAX_POOL_ROUNDS)
898 gPoolRounds.erase(gPoolRounds.begin());
899 SavePoolRoundsLocked();
900}
901
902static void MarkPoolRoundPaid(const std::string& id,
903 const std::string& address,
904 const std::string& txid)
905{
906 std::lock_guard<std::mutex> lk(gPoolRoundsMutex);
907 for (PoolRoundProof& r : gPoolRounds) {
908 if (r.id != id)
909 continue;
910 r.paidTxids[address] = txid;
911 SavePoolRoundsLocked();
912 return;
913 }
914}
915
916// Upstream's gPayoutExecQueue/gPayoutExecMutex/gPayoutThreadRunning are
917// declared above with gPaying: the in-flight list IS the exec queue in this
918// tree, at per-recipient rather than per-round granularity, so a second queue
919// would be a second answer to "who is owed money". Only the dedupe set is new.
920//
921// Ids of rounds currently exploded into gPaying, so a flush cannot queue the
922// same round twice. Guarded by gPayoutExecMutex.
923static std::set<std::string> gQueuedPayouts;
924
925static bool FindRecordedPoolPayout(const std::string& id,
926 const std::string& address,
927 int64 amount,
928 std::string& txidRet)
929{
930 CRITICAL_BLOCK(cs_mapWallet)
931 {
932 for (map<uint256, CWalletTx>::const_iterator it = mapWallet.begin();
933 it != mapWallet.end(); ++it)
934 {
935 const CWalletTx& wtx = it->second;
936 map<string, string>::const_iterator idIt = wtx.mapValue.find("pool_payout_id");
937 if (idIt == wtx.mapValue.end() || idIt->second != id)
938 continue;
939 map<string, string>::const_iterator addrIt = wtx.mapValue.find("pool_payout_address");
940 if (addrIt == wtx.mapValue.end() || addrIt->second != address)
941 continue;
942 map<string, string>::const_iterator amtIt = wtx.mapValue.find("pool_payout_amount");
943 if (amtIt == wtx.mapValue.end() || amtIt->second != strprintf("%lld", amount))
944 continue;
945 txidRet = wtx.GetHash().GetHex();
946 return true;
947 }
948 }
949 return false;
950}
951
952// blockHash stays a uint256 and not upstream's display string: it is the key
953// this fork's payout ledger is built on, and a key that has been through
954// ToString() is a key that can be compared wrong. It is converted to text only
955// where the round proof needs text. The other three arguments are upstream's,
956// carried for the round-proof data.
590static void QueuePayouts(int blockHeight, 957static void QueuePayouts(int blockHeight,
591 const uint256& blockHash, 958 const uint256& blockHash,
959 const std::string& coinbaseTxid,
960 int64 coinbaseValue,
961 const std::string& foundBy,
592 const std::map<std::string, uint64>& shareCount, 962 const std::map<std::string, uint64>& shareCount,
593 uint64 total) 963 uint64 total)
594{ 964{
965 PoolRoundProof round;
966 round.id = PayoutIdForBlock(blockHeight, blockHash);
967 round.blockHeight = blockHeight;
968 round.matureAtHeight = blockHeight + COINBASE_SPENDABLE_DEPTH;
969 round.createdAt = GetTime();
970 round.blockHash = blockHash.ToString();
971 round.coinbaseTxid = coinbaseTxid;
972 round.coinbaseValue = coinbaseValue;
973 round.foundBy = foundBy;
974 round.feePercent = dPoolFeePercent;
975 round.totalShares = total;
976 round.shares = shareCount;
977
595 if (total == 0) { 978 if (total == 0) {
979 RecordPoolRound(round);
596 LogPrint("payout", "[payout] block %d: zero shares recorded -- nothing to pay out\n", 980 LogPrint("payout", "[payout] block %d: zero shares recorded -- nothing to pay out\n",
597 blockHeight); 981 blockHeight);
598 return; 982 return;
@@ -608,6 +992,10 @@ static void QueuePayouts(int blockHeight,
608 int64 nSubsidy = 50 * COIN; 992 int64 nSubsidy = 50 * COIN;
609 nSubsidy >>= (blockHeight / 210000); // halving schedule 993 nSubsidy >>= (blockHeight / 210000); // halving schedule
610 int64 minerReward = (int64)((double)nSubsidy * (1.0 - feeFrac)); 994 int64 minerReward = (int64)((double)nSubsidy * (1.0 - feeFrac));
995 int64 operatorCut = nSubsidy - minerReward;
996 round.subsidy = nSubsidy;
997 round.minerReward = minerReward;
998 round.operatorReward = operatorCut;
611 999
612 PendingPayout pp; 1000 PendingPayout pp;
613 pp.blockHeight = blockHeight; 1001 pp.blockHeight = blockHeight;
@@ -621,7 +1009,9 @@ static void QueuePayouts(int blockHeight,
621 // alone on a private fork paying itself, so it says nothing about a pool 1009 // alone on a private fork paying itself, so it says nothing about a pool
622 // competing on the real chain. Waiting the extra 20 blocks makes each 1010 // competing on the real chain. Waiting the extra 20 blocks makes each
623 // payout funded by construction, and puts the maturity gate past the 1011 // payout funded by construction, and puts the maturity gate past the
624 // reorg depth that orphaned 280 blocks of obligations. 1012 // reorg depth that orphaned 280 blocks of obligations. Upstream uses
1013 // COINBASE_MATURITY here; this is a considered disagreement, not an
1014 // oversight -- see the merge commit message.
625 pp.matureAtHeight = blockHeight + COINBASE_SPENDABLE_DEPTH; 1015 pp.matureAtHeight = blockHeight + COINBASE_SPENDABLE_DEPTH;
626 1016
627 int skipped = 0; 1017 int skipped = 0;
@@ -643,12 +1033,18 @@ static void QueuePayouts(int blockHeight,
643 continue; 1033 continue;
644 } 1034 }
645 pp.amounts[kv.first] = amount; 1035 pp.amounts[kv.first] = amount;
1036 round.payouts[kv.first] = amount;
646 } 1037 }
1038 round.skippedDust = skipped;
1039 round.forfeitedBadAddress = burned;
647 1040
648 gPendingPayouts.push_back(pp); 1041 {
649 SavePendingPayouts(); 1042 std::lock_guard<std::mutex> lk(gPayoutLedgerMutex);
1043 gPendingPayouts.push_back(pp);
1044 SavePendingPayoutsLocked();
1045 }
1046 RecordPoolRound(round);
650 1047
651 int64 operatorCut = nSubsidy - minerReward;
652 LogPrint("payout", 1048 LogPrint("payout",
653 "[payout] block %d: subsidy %s BTF, miners get %s" 1049 "[payout] block %d: subsidy %s BTF, miners get %s"
654 " (fee %.2f%% = %s to operator)," 1050 " (fee %.2f%% = %s to operator),"
@@ -664,66 +1060,110 @@ static void QueuePayouts(int blockHeight,
664 : ""); 1060 : "");
665} 1061}
666 1062
667// Called from event loop: move matured entries into the in-flight list (fast). 1063// Called from the event loop: explode matured ledger entries into per-recipient
1064// in-flight records (fast; no wallet or disk work beyond one small write).
1065//
1066// The ledger entry SURVIVES here. That is upstream's model and it is the
1067// crash-safe one: this fork used to erase the entry as soon as its payments
1068// were handed off, so a death in that window left the debt described by
1069// nothing at all -- a container restart on 2026-07-30 silently dropped
1070// ~1,156 BTF exactly that way. An entry is removed only once every recipient
1071// in it has a txid recorded (PayoutFullyPaid), which the payout thread does.
668// 1072//
669// Ordering matters and is deliberate: the in-flight records are written and 1073// gPaying is kept on top of that as the in-flight working set: it carries the
670// fsync'd to their own file *before* the ledger entry is erased, so the debt 1074// per-recipient retry/deferral state that a per-round `nextAttempt` cannot
671// is always described by at least one of the two files. Dying in the window 1075// express, and it is what the payout thread iterates. It is a cache of work
672// where both describe it is handled at startup by ReconcilePayouts, which 1076// derived from the ledger, never an independent record of debt -- if it were
673// drops the ledger entry it can see is already in flight. 1077// lost entirely the ledger would rebuild it on the next flush.
674static void FlushMaturePayouts() 1078static void FlushMaturePayouts()
675{ 1079{
676 // Void obligations whose funding block lost a reorg. The coin that was 1080 // Which funding blocks are still on the main chain? Asked with no payout
677 // meant to pay them does not exist, so paying anyway spends real-chain 1081 // lock held: PayoutBlockStillOnChain takes cs_main, and cs_main is held by
678 // coin the pool owes to somebody else. 1082 // ProcessBlock and SendMoney for long stretches. Holding the ledger mutex
679 bool voided = false; 1083 // across it would let the event loop park the payout thread behind a chain
680 for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) { 1084 // lock it has no interest in. (It is not a deadlock either way -- nothing
681 if (PayoutBlockStillOnChain(it->blockHash)) { ++it; continue; } 1085 // takes the ledger mutex while holding cs_main -- but it is exactly the
682 int64 total = 0; 1086 // kind of stall this merge is removing from the ledger path.)
683 for (auto& kv : it->amounts) total += kv.second; 1087 std::vector<uint256> orphaned;
684 LogPrint("payout", "[payout] block %d (%s) was orphaned -- voiding %zu" 1088 {
685 " payment(s) totalling %s\n", 1089 std::vector<uint256> hashes;
686 it->blockHeight, it->blockHash.ToString().substr(0, 14).c_str(), 1090 {
687 it->amounts.size(), FormatMoney(total).c_str()); 1091 std::lock_guard<std::mutex> ledger(gPayoutLedgerMutex);
688 it = gPendingPayouts.erase(it); 1092 for (const PendingPayout& pp : gPendingPayouts)
689 voided = true; 1093 hashes.push_back(pp.blockHash);
1094 }
1095 for (const uint256& h : hashes)
1096 if (!PayoutBlockStillOnChain(h))
1097 orphaned.push_back(h);
690 } 1098 }
691 if (voided) SavePendingPayouts();
692 1099
693 std::vector<PendingPayout> ready; 1100 std::vector<PendingPayout> ready;
694 for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) { 1101 {
695 if (nBestHeight < it->matureAtHeight) { ++it; continue; } 1102 std::lock_guard<std::mutex> ledger(gPayoutLedgerMutex);
696 LogPrint("payout", "[payout] block height %d matured -- queuing %zu payments\n", 1103
697 it->matureAtHeight, it->amounts.size()); 1104 // Void obligations whose funding block lost a reorg. The coin that was
698 ready.push_back(*it); // copy first; erase after save 1105 // meant to pay them does not exist, so paying anyway spends real-chain
699 ++it; 1106 // coin the pool owes to somebody else.
700 } 1107 bool voided = false;
701 if (!ready.empty()) {
702 // Record the individual payments first, and only then drop the ledger
703 // entries that produced them.
704 {
705 std::lock_guard<std::mutex> lk(gPayoutExecMutex);
706 for (const auto& pp : ready) {
707 for (const auto& kv : pp.amounts) {
708 PayingRecord r;
709 r.blockHeight = pp.blockHeight;
710 r.blockHash = pp.blockHash;
711 r.matureAtHeight = pp.matureAtHeight;
712 r.address = kv.first;
713 r.amount = kv.second;
714 gPaying.push_back(r);
715 }
716 }
717 SavePayingLocked();
718 }
719 for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) { 1108 for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) {
720 if (nBestHeight >= it->matureAtHeight) 1109 if (std::find(orphaned.begin(), orphaned.end(), it->blockHash) == orphaned.end()) {
721 it = gPendingPayouts.erase(it);
722 else
723 ++it; 1110 ++it;
1111 continue;
1112 }
1113 int64 total = 0;
1114 for (auto& kv : it->amounts) total += kv.second;
1115 LogPrint("payout", "[payout] block %d (%s) was orphaned -- voiding %zu"
1116 " payment(s) totalling %s\n",
1117 it->blockHeight, it->blockHash.ToString().substr(0, 14).c_str(),
1118 it->amounts.size(), FormatMoney(total).c_str());
1119 it = gPendingPayouts.erase(it);
1120 voided = true;
724 } 1121 }
725 SavePendingPayouts(); 1122 if (voided) SavePendingPayoutsLocked();
1123
1124 int64 now = GetTime();
1125 for (const PendingPayout& pp : gPendingPayouts) {
1126 if (nBestHeight < pp.matureAtHeight)
1127 continue;
1128 if (pp.retryAfter > now)
1129 continue;
1130 if (PayoutFullyPaid(pp))
1131 continue;
1132 ready.push_back(pp);
1133 }
1134 }
1135
1136 if (ready.empty())
1137 return;
1138
1139 std::lock_guard<std::mutex> queue(gPayoutExecMutex);
1140 size_t added = 0;
1141 for (const PendingPayout& pp : ready) {
1142 const std::string id = PayoutIdForBlock(pp.blockHeight, pp.blockHash);
1143 if (gQueuedPayouts.count(id))
1144 continue; // already exploded into gPaying and still in flight
1145 size_t n = 0;
1146 for (const auto& kv : pp.amounts) {
1147 // Upstream's per-recipient dedupe: a recipient already carrying a
1148 // txid is done, even though the entry as a whole is not.
1149 if (pp.paidTxids.count(kv.first))
1150 continue;
1151 PayingRecord r;
1152 r.blockHeight = pp.blockHeight;
1153 r.blockHash = pp.blockHash;
1154 r.matureAtHeight = pp.matureAtHeight;
1155 r.address = kv.first;
1156 r.amount = kv.second;
1157 gPaying.push_back(r);
1158 n++;
1159 }
1160 gQueuedPayouts.insert(id);
1161 added += n;
1162 LogPrint("payout", "[payout] block height %d matured -- queuing payout %s"
1163 " (%zu of %zu recipients still owed)\n",
1164 pp.matureAtHeight, id.c_str(), n, pp.amounts.size());
726 } 1165 }
1166 if (added) SavePayingLocked();
727} 1167}
728 1168
729// Identity of a payment, for finding its record again after SendMoney. 1169// Identity of a payment, for finding its record again after SendMoney.
@@ -737,13 +1177,24 @@ static bool SamePayment(const PayingRecord& a, const PayingRecord& b)
737} 1177}
738 1178
739// Settle a payment: it is either done or abandoned, either way it stops being 1179// Settle a payment: it is either done or abandoned, either way it stops being
740// owed. Removing the record and rewriting the file is what makes the payment 1180// in flight. Note this only retires the working record -- whether the miner is
741// not happen twice after a restart. 1181// still OWED the money is the ledger's business, and the ledger is updated
1182// separately by RecordPayoutTxid().
742static void ForgetPayment(const PayingRecord& r) 1183static void ForgetPayment(const PayingRecord& r)
743{ 1184{
744 std::lock_guard<std::mutex> lk(gPayoutExecMutex); 1185 std::lock_guard<std::mutex> lk(gPayoutExecMutex);
745 for (auto it = gPaying.begin(); it != gPaying.end(); ++it) 1186 for (auto it = gPaying.begin(); it != gPaying.end(); ++it)
746 if (SamePayment(*it, r)) { gPaying.erase(it); break; } 1187 if (SamePayment(*it, r)) { gPaying.erase(it); break; }
1188
1189 // When the last record for a round is gone the round is no longer in
1190 // flight, so a later flush may explode it again if the ledger still shows
1191 // recipients owed (a give-up, for instance, leaves the entry standing).
1192 bool stillInFlight = false;
1193 for (const auto& e : gPaying)
1194 if (e.blockHash == r.blockHash) { stillInFlight = true; break; }
1195 if (!stillInFlight)
1196 gQueuedPayouts.erase(PayoutIdForBlock(r.blockHeight, r.blockHash));
1197
747 SavePayingLocked(); 1198 SavePayingLocked();
748} 1199}
749 1200
@@ -753,14 +1204,77 @@ static void ReschedulePayment(const PayingRecord& r)
753 std::lock_guard<std::mutex> lk(gPayoutExecMutex); 1204 std::lock_guard<std::mutex> lk(gPayoutExecMutex);
754 for (auto& e : gPaying) 1205 for (auto& e : gPaying)
755 if (SamePayment(e, r)) { 1206 if (SamePayment(e, r)) {
756 e.retries = r.retries; 1207 e.retries = r.retries;
757 e.deferrals = r.deferrals; 1208 e.deferrals = r.deferrals;
758 e.retryAfter = r.retryAfter; 1209 e.retryAfter = r.retryAfter;
1210 // Carry the cleared probe flag back, or every retry re-scans the
1211 // whole wallet -- the cost this merge is specifically removing.
1212 e.fProbeWallet = r.fProbeWallet;
759 break; 1213 break;
760 } 1214 }
761 SavePayingLocked(); 1215 SavePayingLocked();
762} 1216}
763 1217
1218// Record that a recipient has been paid, in the ledger, which is the only
1219// durable statement that the debt is settled.
1220//
1221// The file write happens with gPayoutLedgerMutex RELEASED. Upstream calls
1222// SavePendingPayoutsLocked() inside the per-recipient loop with the mutex
1223// held, so every recipient costs the event loop a full serialise+fsync it has
1224// to wait for in FlushMaturePayouts.
1225static void RecordPayoutTxid(const PayingRecord& r, const std::string& txid)
1226{
1227 std::string blob;
1228 bool complete = false;
1229 {
1230 std::lock_guard<std::mutex> lk(gPayoutLedgerMutex);
1231 for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ++it) {
1232 if (it->blockHash != r.blockHash)
1233 continue;
1234 it->paidTxids[r.address] = txid;
1235 it->retryAfter = 0;
1236 if (PayoutFullyPaid(*it)) {
1237 LogPrint("payout", "[payout] payout for block %d complete --"
1238 " removing from ledger\n", it->blockHeight);
1239 gPendingPayouts.erase(it);
1240 complete = true;
1241 }
1242 break;
1243 }
1244 blob = SerializePendingPayoutsLocked();
1245 }
1246 WritePendingPayoutsFile(blob);
1247 // The round proof records a txid per address, so this is per recipient and
1248 // not only on completion.
1249 MarkPoolRoundPaid(PayoutIdForBlock(r.blockHeight, r.blockHash), r.address, txid);
1250 if (complete)
1251 LogPrint("payout", "[payout] block %d fully settled\n", r.blockHeight);
1252}
1253
1254// Write a recipient off. Only reached after the full retry budget against a
1255// block that is still on the main chain with coin available to spend, i.e. a
1256// wallet fault rather than a maturity or reorg question.
1257//
1258// The amount is removed from the entry rather than marked paid, so the money
1259// is not reported as sent. Once nothing is left owed the entry retires.
1260static void ForfeitPayoutRecipient(const PayingRecord& r)
1261{
1262 std::string blob;
1263 {
1264 std::lock_guard<std::mutex> lk(gPayoutLedgerMutex);
1265 for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ++it) {
1266 if (it->blockHash != r.blockHash)
1267 continue;
1268 it->amounts.erase(r.address);
1269 if (it->amounts.empty() || PayoutFullyPaid(*it))
1270 gPendingPayouts.erase(it);
1271 break;
1272 }
1273 blob = SerializePendingPayoutsLocked();
1274 }
1275 WritePendingPayoutsFile(blob);
1276}
1277
764// Payout execution thread: works through gPaying, calls SendMoney. 1278// Payout execution thread: works through gPaying, calls SendMoney.
765// Runs independently of the event loop so slow wallet operations 1279// Runs independently of the event loop so slow wallet operations
766// (SendMoney takes cs_main) never stall miner I/O. 1280// (SendMoney takes cs_main) never stall miner I/O.
@@ -780,9 +1294,12 @@ static void PayoutThreadFn(void*)
780 } 1294 }
781 1295
782 for (PayingRecord r : batch) { 1296 for (PayingRecord r : batch) {
1297 const std::string id = PayoutIdForBlock(r.blockHeight, r.blockHash);
1298
783 // Re-check here and not only at maturity: a payment can sit in this 1299 // Re-check here and not only at maturity: a payment can sit in this
784 // list for hours across retries, and the reorg that kills its 1300 // list for hours across retries, and the reorg that kills its
785 // funding block can land at any point in that window. 1301 // funding block can land at any point in that window. Upstream has
1302 // no equivalent -- it decides once, when the entry matures.
786 if (!PayoutBlockStillOnChain(r.blockHash)) { 1303 if (!PayoutBlockStillOnChain(r.blockHash)) {
787 LogPrint("payout", "[payout] block %d (%s) orphaned while queued" 1304 LogPrint("payout", "[payout] block %d (%s) orphaned while queued"
788 " -- dropping %s to %s\n", 1305 " -- dropping %s to %s\n",
@@ -792,6 +1309,25 @@ static void PayoutThreadFn(void*)
792 continue; 1309 continue;
793 } 1310 }
794 1311
1312 // Upstream's per-recipient dedupe, against the ledger rather than
1313 // against the batch: if a txid is already recorded for this
1314 // address, the money went out and nothing more is owed.
1315 {
1316 bool alreadyPaid = false;
1317 {
1318 std::lock_guard<std::mutex> lk(gPayoutLedgerMutex);
1319 for (const PendingPayout& pp : gPendingPayouts)
1320 if (pp.blockHash == r.blockHash && pp.paidTxids.count(r.address)) {
1321 alreadyPaid = true;
1322 break;
1323 }
1324 }
1325 if (alreadyPaid) {
1326 ForgetPayment(r);
1327 continue;
1328 }
1329 }
1330
795 uint160 h160; 1331 uint160 h160;
796 if (!AddressToHash160(r.address, h160)) { 1332 if (!AddressToHash160(r.address, h160)) {
797 // Should not happen -- bad addresses are filtered in QueuePayouts 1333 // Should not happen -- bad addresses are filtered in QueuePayouts
@@ -803,6 +1339,22 @@ static void PayoutThreadFn(void*)
803 CScript sc; 1339 CScript sc;
804 sc << OP_DUP << OP_HASH160 << h160 << OP_EQUALVERIFY << OP_CHECKSIG; 1340 sc << OP_DUP << OP_HASH160 << h160 << OP_EQUALVERIFY << OP_CHECKSIG;
805 1341
1342 // Did a previous process send this and die before recording it?
1343 // Only asked once per record -- see PayingRecord::fProbeWallet.
1344 if (r.fProbeWallet) {
1345 std::string recovered;
1346 r.fProbeWallet = false;
1347 if (FindRecordedPoolPayout(id, r.address, r.amount, recovered)) {
1348 LogPrint("payout", "[payout] recovered recorded payout %s to %s"
1349 " from wallet tx %s -- not resending\n",
1350 FormatMoney(r.amount).c_str(), r.address.c_str(),
1351 recovered.c_str());
1352 RecordPayoutTxid(r, recovered);
1353 ForgetPayment(r);
1354 continue;
1355 }
1356 }
1357
806 // Waiting for coin to mature is not a failure, so it must not 1358 // Waiting for coin to mature is not a failure, so it must not
807 // consume the retry budget. Burning a retry every 60s means a 1359 // consume the retry budget. Burning a retry every 60s means a
808 // payment the pool fully intends to honour can hit GIVING UP 1360 // payment the pool fully intends to honour can hit GIVING UP
@@ -825,11 +1377,22 @@ static void PayoutThreadFn(void*)
825 1377
826 CWalletTx wtx; 1378 CWalletTx wtx;
827 wtx.mapValue["comment"] = PayoutComment(r.blockHeight); 1379 wtx.mapValue["comment"] = PayoutComment(r.blockHeight);
1380 // Upstream's structured metadata. This is what makes the wallet
1381 // probe above possible at all -- the old comment-only form could
1382 // not distinguish two rounds paying the same address the same
1383 // amount. Keyed by the hash-derived id, so a reorg cannot make one
1384 // round's transaction answer for another's.
1385 wtx.mapValue["pool_payout_id"] = id;
1386 wtx.mapValue["pool_payout_address"] = r.address;
1387 wtx.mapValue["pool_payout_amount"] = strprintf("%lld", r.amount);
828 1388
829 if (SendMoney(sc, r.amount, wtx)) { 1389 if (SendMoney(sc, r.amount, wtx)) {
830 // The wallet holds the transaction before this returns, so a 1390 // The wallet holds the transaction before this returns, so a
831 // crash here is recoverable: ReconcilePayouts finds it by 1391 // crash between here and the ledger write is recoverable: the
832 // comment+address+amount and does not resend. 1392 // probe above finds it by id+address+amount and does not
1393 // resend. Ledger first, in-flight record second -- the debt is
1394 // described by at least one of them at every instant.
1395 RecordPayoutTxid(r, wtx.GetHash().GetHex());
833 ForgetPayment(r); 1396 ForgetPayment(r);
834 LogPrint("payout", "[payout] paid %s to %s (tx %s)\n", 1397 LogPrint("payout", "[payout] paid %s to %s (tx %s)\n",
835 FormatMoney(r.amount).c_str(), r.address.c_str(), 1398 FormatMoney(r.amount).c_str(), r.address.c_str(),
@@ -857,6 +1420,12 @@ static void PayoutThreadFn(void*)
857 FormatMoney(r.amount).c_str(), r.address.c_str(), 1420 FormatMoney(r.amount).c_str(), r.address.c_str(),
858 r.retries, r.blockHeight, 1421 r.retries, r.blockHeight,
859 FormatMoney(nSpendable).c_str()); 1422 FormatMoney(nSpendable).c_str());
1423 // Write the debt off in the ledger as well as dropping the
1424 // in-flight record. Under the survives-until-fully-paid model
1425 // the entry would otherwise never complete, and every later
1426 // flush would explode it again -- a forfeit that silently
1427 // becomes an infinite retry.
1428 ForfeitPayoutRecipient(r);
860 ForgetPayment(r); 1429 ForgetPayment(r);
861 } 1430 }
862 } 1431 }
@@ -881,6 +1450,7 @@ static StatsSnapshot gStats;
881static std::vector<PoolWorkerStatView> gWorkerStats; 1450static std::vector<PoolWorkerStatView> gWorkerStats;
882static std::vector<PendingPayoutView> gPayoutViews; 1451static std::vector<PendingPayoutView> gPayoutViews;
883static std::mutex gStatsMutex; 1452static std::mutex gStatsMutex;
1453static int64 gLastPoolStatusWrite = 0;
884 1454
885void GetPoolOperatorStats(int& authorizedMiners, int& blocksFound, uint64& roundShares, double& totalHashRate) 1455void GetPoolOperatorStats(int& authorizedMiners, int& blocksFound, uint64& roundShares, double& totalHashRate)
886{ 1456{
@@ -903,6 +1473,107 @@ void GetPendingPayouts(std::vector<PendingPayoutView>& out)
903 out = gPayoutViews; 1473 out = gPayoutViews;
904} 1474}
905 1475
1476static std::string PoolStatusFilePath()
1477{
1478 if (!strPoolStatusFile.empty())
1479 return strPoolStatusFile;
1480 return GetAppDir() + "/pool_status.json";
1481}
1482
1483static void WritePoolStatusJsonLocked()
1484{
1485 nlohmann::json j;
1486 j["schema"] = "bitflash-pool-status-1";
1487 j["network"] = "bitflash";
1488 j["updatedAt"] = GetTime();
1489
1490 j["operator"] = {
1491 {"name", strPoolName},
1492 {"btfAddress", BtfLocalAddress()},
1493 {"feePercent", dPoolFeePercent},
1494 {"dashboardUrl", strPoolDashboardUrl},
1495 {"roundsFile", "pool_rounds.json"}
1496 };
1497
1498 j["node"] = {
1499 {"height", nBestHeight},
1500 {"poolRunning", (bool)gPoolRunning},
1501 {"mode", nMineMode}
1502 };
1503
1504 j["pool"] = {
1505 {"authorizedMiners", gStats.authorizedMiners},
1506 {"blocksFoundSession", gStats.blocksFound},
1507 {"roundShares", gStats.roundShares},
1508 {"hashRate", gStats.totalHashRate}
1509 };
1510
1511 j["workers"] = nlohmann::json::array();
1512 for (const PoolWorkerStatView& row : gWorkerStats) {
1513 j["workers"].push_back({
1514 {"address", row.address},
1515 {"worker", row.worker},
1516 {"totalShares", row.totalShares},
1517 {"roundShares", row.roundShares},
1518 {"lastSeen", row.lastSeen},
1519 {"hashRate", row.hashRate}
1520 });
1521 }
1522
1523 j["pendingPayouts"] = nlohmann::json::array();
1524 for (const PendingPayoutView& row : gPayoutViews) {
1525 j["pendingPayouts"].push_back({
1526 {"matureAtHeight", row.matureAtHeight},
1527 {"recipients", row.recipients},
1528 {"totalAmountSatoshis", row.totalAmount},
1529 {"totalAmountBtf", FormatMoney(row.totalAmount)}
1530 });
1531 }
1532
1533 j["recentRounds"] = nlohmann::json::array();
1534 {
1535 std::lock_guard<std::mutex> lk(gPoolRoundsMutex);
1536 size_t start = gPoolRounds.size() > 10 ? gPoolRounds.size() - 10 : 0;
1537 for (size_t i = start; i < gPoolRounds.size(); i++) {
1538 const PoolRoundProof& r = gPoolRounds[i];
1539 int unpaid = 0;
1540 for (const auto& kv : r.payouts)
1541 if (r.paidTxids.count(kv.first) == 0)
1542 unpaid++;
1543 j["recentRounds"].push_back({
1544 {"id", r.id},
1545 {"blockHeight", r.blockHeight},
1546 {"blockHash", r.blockHash},
1547 {"coinbaseTxid", r.coinbaseTxid},
1548 {"totalShares", r.totalShares},
1549 {"payouts", (int)r.payouts.size()},
1550 {"unpaidPayouts", unpaid},
1551 {"coinbaseValueBtf", FormatMoney(r.coinbaseValue)},
1552 {"minerRewardBtf", FormatMoney(r.minerReward)}
1553 });
1554 }
1555 }
1556
1557 std::string path = PoolStatusFilePath();
1558 std::string tmp = path + ".tmp";
1559 FILE* f = fopen(tmp.c_str(), "w");
1560 if (!f) {
1561 LogPrint("pool", "[pool] WARNING: could not write status file %s\n", tmp.c_str());
1562 return;
1563 }
1564 std::string s = j.dump(2);
1565 fwrite(s.c_str(), 1, s.size(), f);
1566 if (fclose(f) != 0) {
1567 LogPrint("pool", "[pool] WARNING: could not close status file %s\n", tmp.c_str());
1568 return;
1569 }
1570#ifdef _WIN32
1571 remove(path.c_str());
1572#endif
1573 if (rename(tmp.c_str(), path.c_str()) != 0)
1574 LogPrint("pool", "[pool] WARNING: could not replace status file %s\n", path.c_str());
1575}
1576
906// --------------------------------------------------------------------------- 1577// ---------------------------------------------------------------------------
907// Miner state -- event loop thread only 1578// Miner state -- event loop thread only
908// --------------------------------------------------------------------------- 1579// ---------------------------------------------------------------------------
@@ -1453,7 +2124,11 @@ static bool HandleLine(Miner* m, const std::string& rawLine,
1453 int foundHeight = gCurrentJob.height; 2124 int foundHeight = gCurrentJob.height;
1454 // b is the block we submitted, untouched by ProcessBlock (which 2125 // b is the block we submitted, untouched by ProcessBlock (which
1455 // took a copy), so its hash is what the payout must be keyed on. 2126 // took a copy), so its hash is what the payout must be keyed on.
1456 QueuePayouts(foundHeight, b.GetHash(), roundShareCount, roundShareTotal); 2127 // Passed as a uint256, not hex: the ledger key must stay a real
2128 // hash. The remaining arguments are upstream's round-proof data.
2129 QueuePayouts(foundHeight, b.GetHash(), b.vtx[0].GetHash().GetHex(),
2130 b.vtx[0].vout[0].nValue, m->address,
2131 roundShareCount, roundShareTotal);
1457 // Reset round shares 2132 // Reset round shares
1458 roundShareCount.clear(); 2133 roundShareCount.clear();
1459 roundShareTotal = 0; 2134 roundShareTotal = 0;
@@ -1479,6 +2154,147 @@ static bool HandleLine(Miner* m, const std::string& rawLine,
1479 return true; 2154 return true;
1480} 2155}
1481 2156
2157int RunPoolStratumSelfTest()
2158{
2159 int nFail = 0;
2160 auto check = [&](bool ok, const char* msg) {
2161 fprintf(stdout, " %s %s\n", ok ? "ok" : "FAIL", msg);
2162 if (!ok) nFail++;
2163 };
2164
2165 fprintf(stdout, "pool-stratum self-test\n");
2166
2167 StratumJob savedJob = gCurrentJob;
2168 bool savedHaveJob = gHaveJob;
2169
2170 CTransaction coinbase;
2171 coinbase.vin.resize(1);
2172 coinbase.vin[0].prevout.SetNull();
2173 coinbase.vin[0].scriptSig << 1 << (CBigNum)1;
2174 coinbase.vout.resize(1);
2175 coinbase.vout[0].nValue = 50 * COIN;
2176
2177 CBlock block;
2178 block.vtx.push_back(coinbase);
2179 block.nVersion = 1;
2180 block.nBits = 0x1f0fffff;
2181 block.nTime = 1;
2182 block.nNonce = 1;
2183 block.hashMerkleRoot = block.BuildMerkleTree();
2184
2185 gCurrentJob.jobId = "selftest";
2186 gCurrentJob.block = block;
2187 gCurrentJob.target = 0; // accepted share, not a block candidate
2188 gCurrentJob.height = 1;
2189 gHaveJob = true;
2190
2191 Miner miner(INVALID_SOCKET);
2192 miner.authorised = true;
2193 miner.address = "selftest-address";
2194 miner.worker = "worker";
2195 miner.shareTarget = ~uint256(0); // every RandomX hash is a valid share
2196 miner.difficulty = 0.001;
2197 miner.vardiffWindowStart = GetTime();
2198
2199 std::map<std::string, uint64> roundShareCount;
2200 uint64 roundShareTotal = 0;
2201 int blocksFound = 0;
2202
2203 uint256 targetFromNotify;
2204 CBlock minerBlock = BuildMinerBlock(gCurrentJob, miner.extranonce2);
2205 json params = MakeNotifyParams(gCurrentJob, minerBlock, miner.shareTarget, true);
2206 std::vector<unsigned char> targetBytes = FromHex(params[3].get<std::string>());
2207 if (targetBytes.size() == 32)
2208 memcpy(&targetFromNotify, &targetBytes[0], 32);
2209 check(targetFromNotify == miner.shareTarget,
2210 "mining.notify exposes the same share target the pool validates");
2211
2212 json submit = {
2213 {"id", 7},
2214 {"method", "mining.submit"},
2215 {"params", json::array({miner.address, "selftest", ToHex(&minerBlock.nNonce, 4)})}
2216 };
2217 HandleLine(&miner, submit.dump(), roundShareCount, roundShareTotal, blocksFound);
2218
2219 check(roundShareTotal == 1, "fake low-difficulty share increments the round total");
2220 check(roundShareCount[miner.address] == 1, "fake share is credited to the submitting address");
2221 check(miner.sessionShares == 1 && miner.roundShares == 1 && miner.totalShares == 1,
2222 "miner share counters advance together");
2223 check(blocksFound == 0, "share below block target is not counted as a found block");
2224
2225 json stale = {
2226 {"id", 8},
2227 {"method", "mining.submit"},
2228 {"params", json::array({miner.address, "stale", ToHex(&minerBlock.nNonce, 4)})}
2229 };
2230 HandleLine(&miner, stale.dump(), roundShareCount, roundShareTotal, blocksFound);
2231 check(roundShareTotal == 1, "stale submit does not increment shares");
2232
2233 std::vector<PoolRoundProof> savedRounds = gPoolRounds;
2234 std::string savedRoundsFile = strPoolRoundsFile;
2235 std::string proofPath = "pool_rounds_selftest.json";
2236 remove(proofPath.c_str());
2237 remove((proofPath + ".tmp").c_str());
2238 gPoolRounds.clear();
2239 strPoolRoundsFile = proofPath;
2240
2241 PoolRoundProof proof;
2242 proof.id = "selftest-round";
2243 proof.blockHeight = 42;
2244 proof.matureAtHeight = 42 + COINBASE_MATURITY;
2245 proof.createdAt = GetTime();
2246 proof.blockHash = block.GetHash().GetHex();
2247 proof.coinbaseTxid = coinbase.GetHash().GetHex();
2248 proof.foundBy = miner.address;
2249 proof.feePercent = 1.25;
2250 proof.totalShares = 3;
2251 proof.coinbaseValue = 50 * COIN;
2252 proof.subsidy = 50 * COIN;
2253 proof.minerReward = 49 * COIN;
2254 proof.operatorReward = 1 * COIN;
2255 proof.shares[miner.address] = 3;
2256 proof.payouts[miner.address] = 49 * COIN;
2257 RecordPoolRound(proof);
2258 MarkPoolRoundPaid(proof.id, miner.address, "selftest-txid");
2259
2260 bool proofOk = false;
2261 FILE* pf = fopen(proofPath.c_str(), "r");
2262 if (pf) {
2263 fseek(pf, 0, SEEK_END);
2264 long sz = ftell(pf);
2265 fseek(pf, 0, SEEK_SET);
2266 std::string s(sz, '\0');
2267 fread(&s[0], 1, sz, pf);
2268 fclose(pf);
2269 try {
2270 json root = json::parse(s);
2271 proofOk =
2272 root.value("schema", "") == "bitflash-pool-rounds-1" &&
2273 root["rounds"].is_array() &&
2274 root["rounds"].size() == 1 &&
2275 root["rounds"][0].value("id", "") == proof.id &&
2276 root["rounds"][0]["shares"].value(miner.address, (uint64)0) == 3 &&
2277 root["rounds"][0]["payouts"][miner.address].value("txid", "") == "selftest-txid";
2278 } catch (...) {
2279 proofOk = false;
2280 }
2281 }
2282 check(proofOk, "pool round proof ledger is written and paid txid is recorded");
2283
2284 remove(proofPath.c_str());
2285 remove((proofPath + ".tmp").c_str());
2286 strPoolRoundsFile = savedRoundsFile;
2287 gPoolRounds = savedRounds;
2288
2289 gCurrentJob = savedJob;
2290 gHaveJob = savedHaveJob;
2291
2292 fprintf(stdout, "%s (%d failure%s)\n", nFail == 0 ? "ALL TESTS PASSED" : "TESTS FAILED",
2293 nFail, nFail == 1 ? "" : "s");
2294 fflush(stdout);
2295 return nFail == 0 ? 0 : 1;
2296}
2297
1482// --------------------------------------------------------------------------- 2298// ---------------------------------------------------------------------------
1483// Stats refresh -- called every 2s from event loop 2299// Stats refresh -- called every 2s from event loop
1484// --------------------------------------------------------------------------- 2300// ---------------------------------------------------------------------------
@@ -1512,16 +2328,27 @@ static void RefreshStats(int blocksFoundThisSession, uint64 roundShareTotal)
1512 : a.lastSeen > b.lastSeen; 2328 : a.lastSeen > b.lastSeen;
1513 }); 2329 });
1514 2330
1515 // gPendingPayouts is event-loop-only data (no lock needed here since
1516 // RefreshStats is always called from the event loop thread).
1517 gPayoutViews.clear(); 2331 gPayoutViews.clear();
2332 std::lock_guard<std::mutex> ledger(gPayoutLedgerMutex);
1518 for (const PendingPayout& pp : gPendingPayouts) { 2333 for (const PendingPayout& pp : gPendingPayouts) {
1519 PendingPayoutView pv; 2334 PendingPayoutView pv;
1520 pv.matureAtHeight = pp.matureAtHeight; 2335 pv.matureAtHeight = pp.matureAtHeight;
1521 pv.recipients = (int)pp.amounts.size(); 2336 pv.recipients = 0;
1522 pv.totalAmount = 0; 2337 pv.totalAmount = 0;
1523 for (auto& kv : pp.amounts) pv.totalAmount += kv.second; 2338 for (auto& kv : pp.amounts) {
1524 gPayoutViews.push_back(pv); 2339 if (pp.paidTxids.count(kv.first))
2340 continue;
2341 pv.recipients++;
2342 pv.totalAmount += kv.second;
2343 }
2344 if (pv.recipients > 0)
2345 gPayoutViews.push_back(pv);
2346 }
2347
2348 int64 now = GetTime();
2349 if (now - gLastPoolStatusWrite >= 10) {
2350 WritePoolStatusJsonLocked();
2351 gLastPoolStatusWrite = now;
1525 } 2352 }
1526} 2353}
1527 2354
@@ -1901,10 +2728,11 @@ void ThreadRPCServer(void*)
1901 } 2728 }
1902 2729
1903 LoadPendingPayouts(); 2730 LoadPendingPayouts();
2731 LoadPoolRounds();
1904 LoadPaying(); 2732 LoadPaying();
1905 // Must run before the payout thread starts, so nothing is resent that the 2733 // Must run before the payout thread starts, so nothing is resent that the
1906 // wallet already shows as sent, and nothing is owed twice because we died 2734 // ledger already shows as sent, and every restored in-flight record is
1907 // between recording the payments and rewriting the ledger. 2735 // armed for its one wallet probe before it is allowed to send.
1908 ReconcilePayouts(); 2736 ReconcilePayouts();
1909 2737
1910 _beginthread(AcceptLoopFn, 0, NULL); 2738 _beginthread(AcceptLoopFn, 0, NULL);