diff options
| author | bitflash-maintainer <noreply@bitflash.invalid> | 2026-08-23 05:21:31 +0000 |
|---|---|---|
| committer | bitflash-maintainer <noreply@bitflash.invalid> | 2026-08-23 05:21:31 +0000 |
| commit | a9b4f8940907c2bf51e431172b628de89734a908 (patch) | |
| tree | ff0a0b19d7d7754b809e35c6eae43dae7a32e1e4 /src/rpc.cpp | |
| parent | 6d4c3f153cf44cefead39b21191dfb33491bfa5a (diff) | |
| parent | bc3de301d9ed0e5fb5e2034be2e066d77cf23804 (diff) | |
Merge upstream v1.2.18 into the fork
Merges upstream tag v1.2.18 (bc3de30) into our 24 commits. The merge base is
a259215, upstream's own commit, whose src/ tree is byte-identical to 68de575 --
our commits were replayed onto a259215 first so that git had a real base to work
from. That produced 28 conflict hunks in 9 files instead of the 21 conflict
events / 51 hunks a direct cherry-pick onto the tag would have forced, most of
which were artifacts of lifting our commits away from their own predecessors.
No consensus change. main.cpp's GetBlockValue, GetNextWorkRequired,
ConnectInputs, ConnectBlock, CheckBlock, AcceptBlock and AcceptTransaction are
untouched by this merge; script.cpp, serialize.h and randomx_pow.h are
identical on both sides. The frozen-chain oracle reproduces the pinned UTXO root
116a2b6b... byte for byte, and the merged binary loads the same 12,586-block
chain to the same tip aac622f0a3b2f8... with all blocks re-verified and none
rejected.
Which side won, per file and per hunk.
src/rpc.cpp -- 15 hunks.
1. RebuildJob transaction selection. THEIRS for the multi-pass loop and the
nFees fix, OURS for the cap. Their vfAlreadyAdded/fFoundSomething loop lands
dependent transactions that our single pass dropped deterministically, and
their nFees is accumulated across the scan instead of being redeclared inside
it -- ours passed a hardcoded 0 to GetBlockValue, so every pool block built
here discarded its fees. But their bound is MAX_SIZE/2 = 16 MB, and
MAX_BLOCK_SIZE is 1 MB, so their template can exceed what their own
CheckBlock accepts. Cap kept at MAX_BLOCK_SIZE/2, with our inner break so a
single pass cannot overshoot by more than one transaction. Our RebuildJob
sync gating and deaf-node detection are kept; upstream has none.
2. struct PendingPayout. OURS for the key, THEIRS for paidTxids. Their id is
the string "block-<height>", which is height keying -- the exact bug 44849e6
fixed after a 280-block reorg left ~14.4k BTF owed against blocks that no
longer existed. Height keying also lets the crash-recovery probe match the
previous round's transaction after a reorg and silently skip a miner, so
their id is dropped entirely and blockHash stays the key. paidTxids is taken
because it is what makes an entry survivable. Their nextAttempt and our
retryAfter are the same concept under two names; ours is kept and theirs
dropped. matureAtHeight gained an explicit = 0, which neither side had in
our style of initialiser.
3./4. Ledger writer and loader. Mechanical follow-ons of 2. The loader keeps
ours and takes their tolerance for an entry with no usable height, guarded so
it cannot fire on a well-formed entry. One redundancy was removed rather than
merged: their unguarded `if (obj.contains("blockHeight")) get<int>()` sat two
lines above our is_number-guarded version, and an unguarded get<int>() on a
malformed value throws out of the whole loop, losing every entry in the file.
Verified against the live production pending_payouts.json (2 entries, keys
amounts/blockHash/blockHeight/matureAtHeight): loads both. A file in the new
shape, with retryAfter and paidTxids, also loads.
5. Insertion point after the loader. BOTH. Our PayingRecord/gPaying in-flight
ledger and their PoolRoundProof/PoolRoundToJson are disjoint features that
landed at the same offset. Both kept, in full.
6. Writer tail. THEIRS, copied into our writer. Checking fclose() matters here
because fclose is where the buffered write actually reaches the OS: renaming
a short .tmp over a good file is precisely the truncation the atomic write
exists to prevent. The Windows remove()-before-rename() is required because
rename() there will not replace an existing file.
7. Loader bodies. BOTH -- LoadPaying and LoadPoolRounds are different functions
that happened to share a common tail.
8. Function block, 111 vs 36 lines. BOTH, with one deliberate deletion.
PayoutBlockStillOnChain, PayoutComment, RecordPoolRound, MarkPoolRoundPaid
all kept. WalletAlreadyPaid is DELETED and ReconcilePayouts rewritten,
because our reconciliation and their FindRecordedPoolPayout are two answers
to the same crash-recovery question and running both would give the same debt
two authorities. Their wallet probe wins: it matches structured metadata
rather than a comment string, so it can tell two rounds paying the same
address the same amount apart. ReconcilePayouts keeps only the job that is
still real -- dropping in-flight records the ledger already shows paid, and
arming each restored record for its one wallet probe. It no longer touches
gPendingPayouts at all; under the new model dropping ledger entries there
would reintroduce the loss this merge adopts their model to prevent. Their
gPayoutExecMutex/gPayoutThreadRunning re-declarations were dropped as
duplicates of ours, and gPayoutExecQueue was not taken: gPaying is the exec
queue in this tree, at finer granularity.
9. QueuePayouts signature. THEIRS for the round-proof arguments, OURS for the
type of blockHash. It stays a uint256 and is converted to hex only where the
round proof needs text -- a ledger key that has been through ToString() is a
key that can be compared wrong.
10. Maturity policy. OURS. matureAtHeight = blockHeight +
COINBASE_SPENDABLE_DEPTH (120), not + COINBASE_MATURITY (100). At depth 100
the block's own coinbase is still 20 blocks short of spendable, so the
payment must come from unrelated mature coin and the pool runs a permanent
float. Our own history contradicts itself here -- d6972a0 argued for
COINBASE_MATURITY citing "96% of 934 payments", 44849e6 reversed it on the
grounds that the measurement was taken while the node was alone on a private
fork paying itself. HEAD's position is kept. This is a considered
disagreement with upstream, not an oversight. The live production ledger
confirms it: both entries have matureAtHeight - blockHeight = 120.
11. Comment above FlushMaturePayouts. Rewritten to describe the merged design.
12. FlushMaturePayouts, the central difference. THEIRS for the model. The ledger
entry now survives on disk until every recipient has a txid, instead of
being erased as soon as its payments were handed off -- the old ordering
left a window where the debt was described by nothing at all, and a
container restart on 2026-07-30 silently dropped ~1,156 BTF through it. Our
gPaying is kept on top as the in-flight working set, because it carries the
per-recipient retry and deferral state a per-round nextAttempt cannot
express. It is explicitly a cache derived from the ledger, not a second
record of debt: if it were lost the next flush would rebuild it.
13. PayoutThreadFn, 79 vs 70 lines. OURS for the loop and the reorg re-check,
THEIRS for the per-recipient dedupe. The re-check on every attempt is
load-bearing and upstream has no equivalent -- it decides once, at maturity,
and a payment can sit through hours of retries during which the reorg that
kills its funding block can land. Their paidTxids check is taken, against
the ledger rather than against the batch. Our deferral policy is kept:
waiting for coin is not a failure and must not consume the retry budget.
Two fixes upstream does not have are folded in here rather than deferred to
a follow-up commit:
- FindRecordedPoolPayout runs once per record, on its first attempt, not on
every attempt. It is a linear scan of mapWallet under cs_mapWallet, per
recipient; upstream repeats it for every retry, which with 2000 rotating
payout addresses and a budget of 1440 retries is a full wallet scan per
address per minute for a day. The question it answers can only change
across a restart, and every restart re-arms the flag.
- The ledger file write no longer happens under gPayoutLedgerMutex in the
per-recipient loop. SavePendingPayoutsLocked was split into a serialise
step that needs the lock and a write step that does not, so the mutex is
held only long enough to read the structure. The orphan scan in
FlushMaturePayouts was likewise restructured so the ledger mutex is not
held across PayoutBlockStillOnChain's cs_main acquisition.
One case the merged model created and neither side handled: under
survives-until-fully-paid, our GIVING UP path dropped the in-flight record
but left the recipient owed in the ledger forever, so every later flush
would explode it again -- a forfeit that silently becomes an infinite retry.
ForfeitPayoutRecipient writes the amount off so the entry can retire.
14. QueuePayouts call site. Follow-on of 9.
15. ThreadRPCServer startup. BOTH. LoadPoolRounds added; LoadPaying and
ReconcilePayouts stay, and stay before the payout thread starts.
Crash safety, stated as the property that must hold: after kill -9 at any
point, restart pays each recipient exactly once. Before SendMoney, the
ledger has no txid and the payment is retried. Between SendMoney and the
ledger write, the wallet holds the transaction and the first-attempt probe
finds it. After the ledger write, paidTxids suppresses it. If the in-flight
file is lost entirely, the ledger rebuilds it and the probe still covers the
middle window.
src/nostr.cpp -- 3 hunks. THEIRS as the base, with two things restored from
ours. Theirs loads the Windows ROOT certificate store, which ours lacks and
genuinely needs, since OpenSSL's default verify paths are usually empty there
and our version would fail closed. But the plan's claim that theirs is a
superset of our 06bceb4 is FALSE: upstream's nostr.cpp contains no
min_proto_version, no TLS1_2, no SSL_OP_NO_*, and no set_options anywhere, so
taking it wholesale would have removed the TLS-version floor outright. Restored.
Second, their SSL_get_verify_result check sits after a successful handshake,
where with SSL_VERIFY_PEER set it is nearly dead code -- a bad certificate
aborts inside SSL_connect. The reporting belongs in the failure branch, which is
where the merge put it; their post-handshake duplicate was dropped. Our
relay-list and socket-pooling hunks auto-merged and were verified intact.
src/net.cpp -- 1 hunk. THEIRS. Their guard, hSocket == INVALID_SOCKET ||
pnode->hSocket != hSocket, subsumes ours and additionally catches descriptor
recycling, which ours did not; our version also referenced a pnode that only
existed in the select() loop. Our other db34011 hunk, nulling pnode->hSocket in
the poll-failure branch, auto-merged and was NOT re-applied -- plan section 7
Batch D says to re-apply it, and that instruction is stale. Two pieces of
residue the merge left behind were cleaned up: a comment still describing
select() semantics, and a pnode->hSocket != INVALID_SOCKET guard made redundant
by a continue two lines above it.
src/main.cpp -- 2 hunks. h1 BOTH, pure add/add adjacency between our
ParticipantHashThreadCount and their SecureClear forward declarations. h2 OURS,
and their handler discarded: 44004e3 no-ops checkorder/submitorder but keeps
market.cpp and market.h, whereas 11766be deletes the marketplace outright
including CRequestTracker, subscribe/publish and mapProducts. Ours is strictly
more.
src/main_gui.cpp -- 2 hunks. BOTH, additive on both sides. Our /payoutaddr,
/payoutfile and /hashthreads alongside their /stratumbridge and
/stratumbridgeport.
src/btfrv.cpp -- 1 hunk, not predicted by the plan. False adjacency: our
INVALID_SOCKET check and their #endif landed at the same offset. Ordering is
load-bearing -- our check goes inside the #else arm, before the #endif, or the
non-proxy build silently loses it. The BITFLASH_NODE_BUILD arm already returns
early, so both paths now have the check.
Makefile -- 2 hunks plus a hazard with no conflict marker. THEIRS for both
dependency-fetch recipes: checkout --detach FETCH_HEAD followed by test
"$(git rev-parse HEAD)" = "$(..._COMMIT)" asserts the pin actually landed, where
ours only checked it out. Everything of ours that they lack is kept: the
appimagetool 1.9.1 release pin with a sha256 check, where upstream fetches the
mutable "continuous" tag and executes it undigested, and the move of both the
tool and the AppDir out of /tmp, where any local user could pre-place an
executable and have it run as whoever runs make.
The hazard: both sides added a pin block in a different place, so git merged
both, outside any conflict region. SECP256K1_COMMIT and RANDOMX_COMMIT ended up
assigned twice, near L16 and near L44, and GNU Make takes the last := -- a naive
resolution would have silently built against our old secp256k1 pin while
appearing to adopt upstream's. The duplicate block was deleted by hand. Each of
the four variables is now assigned exactly once, and make -n resolves
SECP256K1_COMMIT to 7fecac74 with 6e2c8bc4 appearing nowhere.
Also applied here, from plan phase 4: SECP256K1_COMMIT moved to upstream's
7fecac74aed8e1fd9078380d67dd04663705c989 so signature validation matches the
binaries the rest of the network runs. secp256k1 was rebuilt from scratch
against the new pin on the build host and the checked-out dependency commit
verified. RANDOMX_COMMIT already matched at 1e9d4b2d and is unchanged.
src/Makefile, src/makefile.mingw -- 1 hunk each, both the HEADERS list. Ours
drops market.h, theirs adds proxy.h; the resolution does both. Taking theirs
wholesale would have left the Makefiles referencing a header git had already
deleted in our favour, silently, without a conflict.
Files the plan predicted would conflict and did not: net.h, main.h and db.cpp
auto-merged. Verified rather than assumed. net.h carries both our
BTF_MSG_WAIT_MAX_SECS work, including the constructor fix from 2d84982 that
starts nMsgWaitSince and nMsgWaitSize at 0, and their comment rewrite with
extern int nPeersWatched. The dead members that fix removed are gone from every
file.
Deviations from MERGE-1.2.18-PLAN.md.
- Section 6's mechanism was not used. Replayed onto the real merge base and
merged, rather than cherry-picking onto the tag. Same result, 28 hunks
instead of 51.
- Section 2 calls 09ff575 a superset of our 06bceb4. It is not; upstream has no
TLS-version floor at all. Ours restored on top of theirs.
- Section 7 Batch D says to re-apply our db34011 line into their poll-failure
branch. It auto-merged; re-applying would have duplicated it.
- Section 2 says do not take the stratum bridge. It is taken, and left off.
fStratumBridge defaults false in main.cpp, the thread is only spawned when
/stratumbridge is passed, and the listener binds 127.0.0.1 only. Excising it
would cost real divergence and future conflict for no gain while it is
disabled. Its defects -- select()/FD_SET on a node that can hold more than
1024 descriptors, and an uncapped thread per connection -- are real only when
it is enabled and are being reported upstream separately.
- Section 7's staged batches were not run as separate gated commits. The
conflict surface is interleaved within single functions, chiefly the payout
ledger, and splitting it would have meant committing states that do not
build. The merge is one commit, as briefed.
- Batch F's two fixes are folded into hunk 13 rather than deferred, because
they change the same lines the resolution rewrites.
- The plan did not anticipate the btfrv.cpp conflict, the duplicate pin block,
the market.h trap in the two Makefiles, or the give-up/infinite-retry
interaction created by adopting upstream's ledger model.
Diffstat (limited to 'src/rpc.cpp')
| -rw-r--r-- | src/rpc.cpp | 1152 |
1 files changed, 990 insertions, 162 deletions
diff --git a/src/rpc.cpp b/src/rpc.cpp index 03d1f44..8af2fbe 100644 --- a/src/rpc.cpp +++ b/src/rpc.cpp | |||
| @@ -236,35 +236,73 @@ static bool RebuildJob() | |||
| 236 | CBlock block; | 236 | CBlock block; |
| 237 | block.vtx.push_back(txNew); | 237 | block.vtx.push_back(txNew); |
| 238 | 238 | ||
| 239 | // Fill the block the same way BitcoinMiner() does, because this is the | ||
| 240 | // same job: a block that gets mined and submitted for real. | ||
| 241 | // | ||
| 242 | // It used to differ in two ways, and both were bugs. | ||
| 243 | // | ||
| 244 | // One pass over mapTransactions is not enough. That map is keyed by | ||
| 245 | // hash, so it has no dependency order, and a transaction spending the | ||
| 246 | // still-unconfirmed output of another (change from a recent send, most | ||
| 247 | // commonly) fails ConnectInputs whenever its parent happens to sit | ||
| 248 | // later in map order. A single pass then drops it -- and because map | ||
| 249 | // order does not change between rebuilds, every later rebuild drops it | ||
| 250 | // again at the same point. Deterministic exclusion, not bad luck: if | ||
| 251 | // pool operators mine most of the blocks, a transaction of that shape | ||
| 252 | // can stay unconfirmed for as long as its parent does. | ||
| 253 | // | ||
| 254 | // Diagnosis and fix both come from Vic-Nas, who found it in his fork | ||
| 255 | // of this code and described it exactly. The fork has since been | ||
| 256 | // deleted, so this carries it back rather than losing it. | ||
| 257 | int64 nFees = 0; | ||
| 239 | { | 258 | { |
| 240 | CTxDB txdb("r"); | 259 | CTxDB txdb("r"); |
| 241 | CRITICAL_BLOCK(cs_mapTransactions) | 260 | CRITICAL_BLOCK(cs_mapTransactions) |
| 242 | { | 261 | { |
| 243 | map<uint256, CTxIndex> pool; | 262 | map<uint256, CTxIndex> pool; |
| 263 | vector<char> vfAlreadyAdded(mapTransactions.size()); | ||
| 264 | bool fFoundSomething = true; | ||
| 244 | unsigned int sz = 0; | 265 | unsigned int sz = 0; |
| 245 | for (auto& kv : mapTransactions) { | 266 | // The cap is MAX_BLOCK_SIZE/2, not MAX_SIZE/2. MAX_SIZE is the |
| 246 | CTransaction& tx = kv.second; | 267 | // 32 MB serialisation ceiling, so half of it is 16 MB -- this |
| 247 | if (tx.IsCoinBase() || !tx.IsFinal()) continue; | 268 | // builder would happily assemble a template that its own |
| 248 | int64 nFees = 0; | 269 | // CheckBlock() rejects at the 1 MB consensus cap. This is the |
| 249 | map<uint256, CTxIndex> tmp(pool); | 270 | // pool's job builder, so a block over the cap is not one |
| 250 | if (!tx.ConnectInputs(txdb, tmp, CDiskTxPos(1,1,1), 0, nFees, | 271 | // wasted solve, it is every miner on the pool working a |
| 251 | false, true, tx.GetMinFee(block.vtx.size() < 100))) | 272 | // template the network will reject. The inner break bounds the |
| 252 | continue; | 273 | // overshoot to a single transaction within a pass. |
| 253 | pool = tmp; | 274 | while (fFoundSomething && sz < MAX_BLOCK_SIZE / 2) |
| 254 | block.vtx.push_back(tx); | 275 | { |
| 255 | sz += ::GetSerializeSize(tx, SER_NETWORK); | 276 | fFoundSomething = false; |
| 256 | // Must stay under the consensus cap CheckBlock() enforces. | 277 | unsigned int n = 0; |
| 257 | // This is the pool's job builder, so a block over the cap | 278 | for (map<uint256, CTransaction>::iterator mi = mapTransactions.begin(); |
| 258 | // is not just one wasted solve -- it is every miner on the | 279 | mi != mapTransactions.end(); ++mi, ++n) |
| 259 | // pool working a template the network will reject. The | 280 | { |
| 260 | // transaction is already pushed above, so this can overshoot | 281 | if (vfAlreadyAdded[n]) |
| 261 | // by one; half the cap leaves room for that. | 282 | continue; |
| 262 | if (sz > MAX_BLOCK_SIZE / 2) break; | 283 | CTransaction& tx = (*mi).second; |
| 284 | if (tx.IsCoinBase() || !tx.IsFinal()) continue; | ||
| 285 | map<uint256, CTxIndex> tmp(pool); | ||
| 286 | if (!tx.ConnectInputs(txdb, tmp, CDiskTxPos(1,1,1), 0, nFees, | ||
| 287 | false, true, tx.GetMinFee(block.vtx.size() < 100))) | ||
| 288 | continue; | ||
| 289 | swap(pool, tmp); | ||
| 290 | block.vtx.push_back(tx); | ||
| 291 | sz += ::GetSerializeSize(tx, SER_NETWORK); | ||
| 292 | vfAlreadyAdded[n] = true; | ||
| 293 | fFoundSomething = true; | ||
| 294 | if (sz > MAX_BLOCK_SIZE / 2) break; | ||
| 295 | } | ||
| 263 | } | 296 | } |
| 264 | } | 297 | } |
| 265 | } | 298 | } |
| 266 | 299 | ||
| 267 | block.vtx[0].vout[0].nValue = block.GetBlockValue(pindexPrev->nHeight + 1, 0); | 300 | // And the fees actually go to the coinbase now. nFees was declared |
| 301 | // inside the scan loop, so every transaction reset it and the total was | ||
| 302 | // thrown away, while this line passed a hardcoded 0. A pool operator | ||
| 303 | // collected fees into a block and paid none of them to itself. Solo | ||
| 304 | // mining has always passed nFees here; only this builder did not. | ||
| 305 | block.vtx[0].vout[0].nValue = block.GetBlockValue(pindexPrev->nHeight + 1, nFees); | ||
| 268 | block.hashPrevBlock = pindexPrev->GetBlockHash(); | 306 | block.hashPrevBlock = pindexPrev->GetBlockHash(); |
| 269 | block.hashMerkleRoot = block.BuildMerkleTree(); | 307 | block.hashMerkleRoot = block.BuildMerkleTree(); |
| 270 | block.nTime = max((unsigned int)(pindexPrev->GetMedianTimePast() + 1), | 308 | block.nTime = max((unsigned int)(pindexPrev->GetMedianTimePast() + 1), |
| @@ -297,26 +335,110 @@ struct PendingPayout { | |||
| 297 | // supposed to pay it: a 280-block reorg once left ~14.4k BTF of debt | 335 | // supposed to pay it: a 280-block reorg once left ~14.4k BTF of debt |
| 298 | // owed against blocks that no longer existed, which then drained the | 336 | // owed against blocks that no longer existed, which then drained the |
| 299 | // wallet paying fork-era miners out of real-chain coin. | 337 | // wallet paying fork-era miners out of real-chain coin. |
| 338 | // | ||
| 339 | // Upstream keys these on the string "block-<height>". That is the same | ||
| 340 | // bug this fork already paid for, and height keying additionally lets | ||
| 341 | // FindRecordedPoolPayout match the *previous* round's transaction after a | ||
| 342 | // reorg and silently skip a miner, so their `id` is deliberately dropped. | ||
| 300 | int blockHeight = 0; | 343 | int blockHeight = 0; |
| 301 | // Must be explicitly zeroed: uint256's default constructor is empty | 344 | // Must be explicitly zeroed: uint256's default constructor is empty |
| 302 | // (uint256.h:514), so leaving this unassigned gives every ledger entry | 345 | // (uint256.h:514), so leaving this unassigned gives every ledger entry |
| 303 | // loaded from an older file a garbage hash, which then matches no block | 346 | // loaded from an older file a garbage hash, which then matches no block |
| 304 | // and gets voided as "orphaned". That mistake voided a live payout. | 347 | // and gets voided as "orphaned". That mistake voided a live payout. |
| 305 | uint256 blockHash = 0; // zero = entry predates hash keying | 348 | uint256 blockHash = 0; // zero = entry predates hash keying |
| 306 | int matureAtHeight; | 349 | int matureAtHeight = 0; |
| 307 | std::map<std::string, int64> amounts; // address -> satoshis | 350 | std::map<std::string, int64> amounts; // address -> satoshis |
| 351 | // From upstream: which recipients already have a transaction. This is what | ||
| 352 | // lets an entry stay on disk until every address in `amounts` is paid, | ||
| 353 | // instead of being erased the moment the batch is handed off. | ||
| 354 | std::map<std::string, std::string> paidTxids; // address -> txid | ||
| 308 | int retries = 0; // exec-thread only: SendMoney attempts so far | 355 | int retries = 0; // exec-thread only: SendMoney attempts so far |
| 309 | int deferrals = 0; // exec-thread only: waits for spendable coin | 356 | int deferrals = 0; // exec-thread only: waits for spendable coin |
| 357 | // Upstream's `nextAttempt` is this field under another name; ours is kept | ||
| 358 | // so the retry/deferral policy above stays coherent with it. | ||
| 310 | int64 retryAfter = 0; // exec-thread only: unix time, 0 = ready now | 359 | int64 retryAfter = 0; // exec-thread only: unix time, 0 = ready now |
| 311 | }; | 360 | }; |
| 312 | static std::vector<PendingPayout> gPendingPayouts; | 361 | static std::vector<PendingPayout> gPendingPayouts; |
| 362 | static std::mutex gPayoutLedgerMutex; | ||
| 363 | |||
| 364 | struct PoolRoundProof { | ||
| 365 | std::string id; | ||
| 366 | int blockHeight; | ||
| 367 | int matureAtHeight; | ||
| 368 | int64 createdAt; | ||
| 369 | std::string blockHash; | ||
| 370 | std::string coinbaseTxid; | ||
| 371 | std::string foundBy; | ||
| 372 | double feePercent; | ||
| 373 | uint64 totalShares; | ||
| 374 | int64 coinbaseValue; | ||
| 375 | int64 subsidy; | ||
| 376 | int64 minerReward; | ||
| 377 | int64 operatorReward; | ||
| 378 | int skippedDust; | ||
| 379 | int64 forfeitedBadAddress; | ||
| 380 | std::map<std::string, uint64> shares; | ||
| 381 | std::map<std::string, int64> payouts; | ||
| 382 | std::map<std::string, std::string> paidTxids; | ||
| 383 | |||
| 384 | PoolRoundProof() | ||
| 385 | : blockHeight(0), matureAtHeight(0), createdAt(0), feePercent(0.0), | ||
| 386 | totalShares(0), coinbaseValue(0), subsidy(0), minerReward(0), operatorReward(0), | ||
| 387 | skippedDust(0), forfeitedBadAddress(0) {} | ||
| 388 | }; | ||
| 389 | static std::vector<PoolRoundProof> gPoolRounds; | ||
| 390 | static std::mutex gPoolRoundsMutex; | ||
| 391 | static const size_t MAX_POOL_ROUNDS = 200; | ||
| 313 | 392 | ||
| 314 | static std::string PayoutFilePath() | 393 | static std::string PayoutFilePath() |
| 315 | { | 394 | { |
| 316 | return GetAppDir() + "/pending_payouts.json"; | 395 | return GetAppDir() + "/pending_payouts.json"; |
| 317 | } | 396 | } |
| 318 | 397 | ||
| 319 | static void SavePendingPayouts() | 398 | static std::string PoolRoundsFilePath() |
| 399 | { | ||
| 400 | if (!strPoolRoundsFile.empty()) | ||
| 401 | return strPoolRoundsFile; | ||
| 402 | return GetAppDir() + "/pool_rounds.json"; | ||
| 403 | } | ||
| 404 | |||
| 405 | // Identity of a payout round. Used as the pool-round proof id and, more | ||
| 406 | // importantly, as the `pool_payout_id` written into wallet transaction | ||
| 407 | // metadata and matched by FindRecordedPoolPayout(). | ||
| 408 | // | ||
| 409 | // Derived from the block HASH, not from its height alone. Upstream uses | ||
| 410 | // "block-<height>", which names two different blocks identically across a | ||
| 411 | // reorg -- and because this id is what the crash-recovery probe matches on, a | ||
| 412 | // height-keyed id lets that probe find the *previous* round's transaction and | ||
| 413 | // conclude a miner has already been paid when they have not. The height stays | ||
| 414 | // in the string for readability only. | ||
| 415 | // | ||
| 416 | // A zero hash means the entry predates hash keying; those keep the old form so | ||
| 417 | // anything already recorded against them still matches. | ||
| 418 | static std::string PayoutIdForBlock(int blockHeight, const uint256& blockHash) | ||
| 419 | { | ||
| 420 | if (blockHash == 0) | ||
| 421 | return strprintf("block-%d", blockHeight); | ||
| 422 | return strprintf("block-%d-%s", blockHeight, | ||
| 423 | blockHash.ToString().substr(0, 16).c_str()); | ||
| 424 | } | ||
| 425 | |||
| 426 | static bool PayoutFullyPaid(const PendingPayout& pp) | ||
| 427 | { | ||
| 428 | for (const auto& kv : pp.amounts) | ||
| 429 | if (pp.paidTxids.count(kv.first) == 0) | ||
| 430 | return false; | ||
| 431 | return true; | ||
| 432 | } | ||
| 433 | |||
| 434 | // Serialise the ledger. Caller must hold gPayoutLedgerMutex. | ||
| 435 | // | ||
| 436 | // Split from the file write so the payout thread can hold the mutex just long | ||
| 437 | // enough to read the structure and then do the I/O with it released. Upstream | ||
| 438 | // writes the file inside the per-recipient loop with gPayoutLedgerMutex held, | ||
| 439 | // while the event loop's FlushMaturePayouts blocks on that same mutex -- one | ||
| 440 | // slow fsync per recipient, on the thread the miners' I/O shares. | ||
| 441 | static std::string SerializePendingPayoutsLocked() | ||
| 320 | { | 442 | { |
| 321 | nlohmann::json arr = nlohmann::json::array(); | 443 | nlohmann::json arr = nlohmann::json::array(); |
| 322 | for (const auto& pp : gPendingPayouts) { | 444 | for (const auto& pp : gPendingPayouts) { |
| @@ -324,21 +446,49 @@ static void SavePendingPayouts() | |||
| 324 | obj["blockHeight"] = pp.blockHeight; | 446 | obj["blockHeight"] = pp.blockHeight; |
| 325 | obj["blockHash"] = pp.blockHash.ToString(); | 447 | obj["blockHash"] = pp.blockHash.ToString(); |
| 326 | obj["matureAtHeight"] = pp.matureAtHeight; | 448 | obj["matureAtHeight"] = pp.matureAtHeight; |
| 449 | obj["retryAfter"] = pp.retryAfter; | ||
| 327 | nlohmann::json amounts = nlohmann::json::object(); | 450 | nlohmann::json amounts = nlohmann::json::object(); |
| 328 | for (const auto& kv : pp.amounts) | 451 | for (const auto& kv : pp.amounts) |
| 329 | amounts[kv.first] = kv.second; | 452 | amounts[kv.first] = kv.second; |
| 330 | obj["amounts"] = amounts; | 453 | obj["amounts"] = amounts; |
| 454 | nlohmann::json paid = nlohmann::json::object(); | ||
| 455 | for (const auto& kv : pp.paidTxids) | ||
| 456 | paid[kv.first] = kv.second; | ||
| 457 | obj["paidTxids"] = paid; | ||
| 331 | arr.push_back(obj); | 458 | arr.push_back(obj); |
| 332 | } | 459 | } |
| 460 | return arr.dump(2); | ||
| 461 | } | ||
| 462 | |||
| 463 | // Write the serialised ledger. Holds no lock; safe to call with the mutex | ||
| 464 | // released. Atomic tmp+rename so a crash mid-write cannot truncate the record | ||
| 465 | // of who is still owed money. | ||
| 466 | static void WritePendingPayoutsFile(const std::string& s) | ||
| 467 | { | ||
| 333 | std::string path = PayoutFilePath(); | 468 | std::string path = PayoutFilePath(); |
| 334 | FILE* f = fopen(path.c_str(), "w"); | 469 | std::string tmp = path + ".tmp"; |
| 335 | if (f) { | 470 | FILE* f = fopen(tmp.c_str(), "w"); |
| 336 | std::string s = arr.dump(2); | 471 | if (!f) { |
| 337 | fwrite(s.c_str(), 1, s.size(), f); | 472 | LogPrint("payout", "[payout] WARNING: could not write %s\n", tmp.c_str()); |
| 338 | fclose(f); | 473 | return; |
| 339 | } else { | ||
| 340 | LogPrint("payout", "[payout] WARNING: could not write %s\n", path.c_str()); | ||
| 341 | } | 474 | } |
| 475 | fwrite(s.c_str(), 1, s.size(), f); | ||
| 476 | if (fclose(f) != 0) { | ||
| 477 | LogPrint("payout", "[payout] WARNING: could not close %s\n", tmp.c_str()); | ||
| 478 | return; | ||
| 479 | } | ||
| 480 | #ifdef _WIN32 | ||
| 481 | remove(path.c_str()); | ||
| 482 | #endif | ||
| 483 | if (rename(tmp.c_str(), path.c_str()) != 0) | ||
| 484 | LogPrint("payout", "[payout] WARNING: could not replace %s\n", path.c_str()); | ||
| 485 | } | ||
| 486 | |||
| 487 | // Convenience for the cold paths (startup, queueing) where holding the mutex | ||
| 488 | // across the write costs nothing. Caller must hold gPayoutLedgerMutex. | ||
| 489 | static void SavePendingPayoutsLocked() | ||
| 490 | { | ||
| 491 | WritePendingPayoutsFile(SerializePendingPayoutsLocked()); | ||
| 342 | } | 492 | } |
| 343 | 493 | ||
| 344 | static void LoadPendingPayouts() | 494 | static void LoadPendingPayouts() |
| @@ -354,6 +504,7 @@ static void LoadPendingPayouts() | |||
| 354 | fclose(f); | 504 | fclose(f); |
| 355 | try { | 505 | try { |
| 356 | nlohmann::json arr = nlohmann::json::parse(s); | 506 | nlohmann::json arr = nlohmann::json::parse(s); |
| 507 | std::lock_guard<std::mutex> lk(gPayoutLedgerMutex); | ||
| 357 | for (const auto& obj : arr) { | 508 | for (const auto& obj : arr) { |
| 358 | PendingPayout pp; | 509 | PendingPayout pp; |
| 359 | pp.matureAtHeight = obj["matureAtHeight"].get<int>(); | 510 | pp.matureAtHeight = obj["matureAtHeight"].get<int>(); |
| @@ -364,8 +515,24 @@ static void LoadPendingPayouts() | |||
| 364 | pp.blockHash.SetHex(obj["blockHash"].get<std::string>()); | 515 | pp.blockHash.SetHex(obj["blockHash"].get<std::string>()); |
| 365 | if (obj.contains("blockHeight") && obj["blockHeight"].is_number()) | 516 | if (obj.contains("blockHeight") && obj["blockHeight"].is_number()) |
| 366 | pp.blockHeight = obj["blockHeight"].get<int>(); | 517 | pp.blockHeight = obj["blockHeight"].get<int>(); |
| 518 | // Upstream's tolerance for an entry with no usable height, kept | ||
| 519 | // because it costs nothing and an entry that cannot name its block | ||
| 520 | // is worse than one derived from its maturity. COINBASE_MATURITY | ||
| 521 | // and not COINBASE_SPENDABLE_DEPTH: this reverses whatever the | ||
| 522 | // writer of *that* file used, and only files old enough to lack | ||
| 523 | // blockHeight are affected -- they were written when maturity was | ||
| 524 | // still COINBASE_MATURITY. | ||
| 525 | if (pp.blockHeight == 0 && pp.matureAtHeight > COINBASE_MATURITY) | ||
| 526 | pp.blockHeight = pp.matureAtHeight - COINBASE_MATURITY; | ||
| 527 | // Absent in every file written before this merge; default 0 means | ||
| 528 | // "ready now", which is the right answer for a restored entry. | ||
| 529 | if (obj.contains("retryAfter") && obj["retryAfter"].is_number()) | ||
| 530 | pp.retryAfter = obj["retryAfter"].get<int64>(); | ||
| 367 | for (auto it = obj["amounts"].begin(); it != obj["amounts"].end(); ++it) | 531 | for (auto it = obj["amounts"].begin(); it != obj["amounts"].end(); ++it) |
| 368 | pp.amounts[it.key()] = it.value().get<int64>(); | 532 | pp.amounts[it.key()] = it.value().get<int64>(); |
| 533 | if (obj.contains("paidTxids") && obj["paidTxids"].is_object()) | ||
| 534 | for (auto it = obj["paidTxids"].begin(); it != obj["paidTxids"].end(); ++it) | ||
| 535 | pp.paidTxids[it.key()] = it.value().get<std::string>(); | ||
| 369 | gPendingPayouts.push_back(pp); | 536 | gPendingPayouts.push_back(pp); |
| 370 | } | 537 | } |
| 371 | LogPrint("payout", "[payout] loaded %zu pending payout(s) from disk\n", | 538 | LogPrint("payout", "[payout] loaded %zu pending payout(s) from disk\n", |
| @@ -396,6 +563,17 @@ struct PayingRecord { | |||
| 396 | int retries = 0; | 563 | int retries = 0; |
| 397 | int deferrals = 0; | 564 | int deferrals = 0; |
| 398 | int64 retryAfter = 0; | 565 | int64 retryAfter = 0; |
| 566 | // Ask the wallet whether this payment already went out, once, before the | ||
| 567 | // first send attempt. Upstream asks on every attempt: FindRecordedPoolPayout | ||
| 568 | // is a linear scan of mapWallet under cs_mapWallet, per recipient, so with | ||
| 569 | // 2000 rotating payout addresses and a retry budget of 1440 that is a | ||
| 570 | // scan per address per minute for a day. The question it answers -- "did a | ||
| 571 | // previous process send this before it died?" -- can only change across a | ||
| 572 | // restart, and every restart re-sets this flag (ReconcilePayouts), so | ||
| 573 | // asking once per record is the whole of the answer. | ||
| 574 | // | ||
| 575 | // Not persisted: a record read back from disk is by definition a restart. | ||
| 576 | bool fProbeWallet = true; | ||
| 399 | }; | 577 | }; |
| 400 | static std::vector<PayingRecord> gPaying; // guarded by gPayoutExecMutex | 578 | static std::vector<PayingRecord> gPaying; // guarded by gPayoutExecMutex |
| 401 | static std::mutex gPayoutExecMutex; | 579 | static std::mutex gPayoutExecMutex; |
| @@ -432,8 +610,19 @@ static void SavePayingLocked() | |||
| 432 | } | 610 | } |
| 433 | std::string s = arr.dump(2); | 611 | std::string s = arr.dump(2); |
| 434 | fwrite(s.c_str(), 1, s.size(), f); | 612 | fwrite(s.c_str(), 1, s.size(), f); |
| 435 | fflush(f); | 613 | // fclose() is where buffered data actually reaches the OS, so a failure |
| 436 | fclose(f); | 614 | // here means the .tmp is short -- renaming it over the real file would |
| 615 | // destroy the good copy. Upstream's check, taken because a truncated | ||
| 616 | // list of who is owed money is exactly what this function exists to | ||
| 617 | // prevent. rename() on Windows will not replace an existing file, hence | ||
| 618 | // the remove() first. | ||
| 619 | if (fclose(f) != 0) { | ||
| 620 | LogPrint("payout", "[payout] WARNING: could not close %s\n", tmp.c_str()); | ||
| 621 | return; | ||
| 622 | } | ||
| 623 | #ifdef _WIN32 | ||
| 624 | remove(path.c_str()); | ||
| 625 | #endif | ||
| 437 | if (rename(tmp.c_str(), path.c_str()) != 0) | 626 | if (rename(tmp.c_str(), path.c_str()) != 0) |
| 438 | LogPrint("payout", "[payout] WARNING: could not rename %s -> %s\n", | 627 | LogPrint("payout", "[payout] WARNING: could not rename %s -> %s\n", |
| 439 | tmp.c_str(), path.c_str()); | 628 | tmp.c_str(), path.c_str()); |
| @@ -475,6 +664,143 @@ static void LoadPaying() | |||
| 475 | } | 664 | } |
| 476 | } | 665 | } |
| 477 | 666 | ||
| 667 | // --------------------------------------------------------------------------- | ||
| 668 | // Pool round proofs (upstream cbdd06d). Disjoint from the in-flight ledger | ||
| 669 | // above -- this is the operator's published record of how a round was split, | ||
| 670 | // not an obligation to pay. Both sides added code at this offset; both are | ||
| 671 | // kept. | ||
| 672 | // --------------------------------------------------------------------------- | ||
| 673 | static nlohmann::json PoolRoundToJson(const PoolRoundProof& r) | ||
| 674 | { | ||
| 675 | nlohmann::json j; | ||
| 676 | j["id"] = r.id; | ||
| 677 | j["blockHeight"] = r.blockHeight; | ||
| 678 | j["matureAtHeight"] = r.matureAtHeight; | ||
| 679 | j["createdAt"] = r.createdAt; | ||
| 680 | j["blockHash"] = r.blockHash; | ||
| 681 | j["coinbaseTxid"] = r.coinbaseTxid; | ||
| 682 | j["foundBy"] = r.foundBy; | ||
| 683 | j["feePercent"] = r.feePercent; | ||
| 684 | j["totalShares"] = r.totalShares; | ||
| 685 | j["coinbaseValueSatoshis"] = r.coinbaseValue; | ||
| 686 | j["coinbaseValueBtf"] = FormatMoney(r.coinbaseValue); | ||
| 687 | j["subsidySatoshis"] = r.subsidy; | ||
| 688 | j["subsidyBtf"] = FormatMoney(r.subsidy); | ||
| 689 | j["minerRewardSatoshis"] = r.minerReward; | ||
| 690 | j["minerRewardBtf"] = FormatMoney(r.minerReward); | ||
| 691 | j["operatorRewardSatoshis"] = r.operatorReward; | ||
| 692 | j["operatorRewardBtf"] = FormatMoney(r.operatorReward); | ||
| 693 | j["skippedDust"] = r.skippedDust; | ||
| 694 | j["forfeitedBadAddressSatoshis"] = r.forfeitedBadAddress; | ||
| 695 | j["forfeitedBadAddressBtf"] = FormatMoney(r.forfeitedBadAddress); | ||
| 696 | |||
| 697 | j["shares"] = nlohmann::json::object(); | ||
| 698 | for (const auto& kv : r.shares) | ||
| 699 | j["shares"][kv.first] = kv.second; | ||
| 700 | |||
| 701 | j["payouts"] = nlohmann::json::object(); | ||
| 702 | for (const auto& kv : r.payouts) { | ||
| 703 | nlohmann::json p; | ||
| 704 | p["amountSatoshis"] = kv.second; | ||
| 705 | p["amountBtf"] = FormatMoney(kv.second); | ||
| 706 | std::map<std::string, std::string>::const_iterator tx = r.paidTxids.find(kv.first); | ||
| 707 | if (tx != r.paidTxids.end()) | ||
| 708 | p["txid"] = tx->second; | ||
| 709 | else | ||
| 710 | p["txid"] = nullptr; | ||
| 711 | j["payouts"][kv.first] = p; | ||
| 712 | } | ||
| 713 | return j; | ||
| 714 | } | ||
| 715 | |||
| 716 | static void SavePoolRoundsLocked() | ||
| 717 | { | ||
| 718 | nlohmann::json root; | ||
| 719 | root["schema"] = "bitflash-pool-rounds-1"; | ||
| 720 | root["updatedAt"] = GetTime(); | ||
| 721 | root["rounds"] = nlohmann::json::array(); | ||
| 722 | for (const PoolRoundProof& r : gPoolRounds) | ||
| 723 | root["rounds"].push_back(PoolRoundToJson(r)); | ||
| 724 | |||
| 725 | std::string path = PoolRoundsFilePath(); | ||
| 726 | std::string tmp = path + ".tmp"; | ||
| 727 | FILE* f = fopen(tmp.c_str(), "w"); | ||
| 728 | if (!f) { | ||
| 729 | LogPrint("payout", "[payout] WARNING: could not write %s\n", tmp.c_str()); | ||
| 730 | return; | ||
| 731 | } | ||
| 732 | std::string s = root.dump(2); | ||
| 733 | fwrite(s.c_str(), 1, s.size(), f); | ||
| 734 | if (fclose(f) != 0) { | ||
| 735 | LogPrint("payout", "[payout] WARNING: could not close %s\n", tmp.c_str()); | ||
| 736 | return; | ||
| 737 | } | ||
| 738 | #ifdef _WIN32 | ||
| 739 | remove(path.c_str()); | ||
| 740 | #endif | ||
| 741 | if (rename(tmp.c_str(), path.c_str()) != 0) | ||
| 742 | LogPrint("payout", "[payout] WARNING: could not replace %s\n", path.c_str()); | ||
| 743 | } | ||
| 744 | |||
| 745 | static void LoadPoolRounds() | ||
| 746 | { | ||
| 747 | std::string path = PoolRoundsFilePath(); | ||
| 748 | FILE* f = fopen(path.c_str(), "r"); | ||
| 749 | if (!f) return; | ||
| 750 | fseek(f, 0, SEEK_END); | ||
| 751 | long sz = ftell(f); | ||
| 752 | fseek(f, 0, SEEK_SET); | ||
| 753 | std::string s(sz, '\0'); | ||
| 754 | fread(&s[0], 1, sz, f); | ||
| 755 | fclose(f); | ||
| 756 | |||
| 757 | try { | ||
| 758 | nlohmann::json root = nlohmann::json::parse(s); | ||
| 759 | nlohmann::json arr = root.is_array() ? root : root["rounds"]; | ||
| 760 | std::lock_guard<std::mutex> lk(gPoolRoundsMutex); | ||
| 761 | for (const auto& obj : arr) { | ||
| 762 | PoolRoundProof r; | ||
| 763 | if (obj.contains("id") && obj["id"].is_string()) r.id = obj["id"].get<std::string>(); | ||
| 764 | if (obj.contains("blockHeight")) r.blockHeight = obj["blockHeight"].get<int>(); | ||
| 765 | if (obj.contains("matureAtHeight")) r.matureAtHeight = obj["matureAtHeight"].get<int>(); | ||
| 766 | if (obj.contains("createdAt")) r.createdAt = obj["createdAt"].get<int64>(); | ||
| 767 | if (obj.contains("blockHash") && obj["blockHash"].is_string()) r.blockHash = obj["blockHash"].get<std::string>(); | ||
| 768 | if (obj.contains("coinbaseTxid") && obj["coinbaseTxid"].is_string()) r.coinbaseTxid = obj["coinbaseTxid"].get<std::string>(); | ||
| 769 | if (obj.contains("foundBy") && obj["foundBy"].is_string()) r.foundBy = obj["foundBy"].get<std::string>(); | ||
| 770 | if (obj.contains("feePercent")) r.feePercent = obj["feePercent"].get<double>(); | ||
| 771 | if (obj.contains("totalShares")) r.totalShares = obj["totalShares"].get<uint64>(); | ||
| 772 | if (obj.contains("coinbaseValueSatoshis")) r.coinbaseValue = obj["coinbaseValueSatoshis"].get<int64>(); | ||
| 773 | if (obj.contains("subsidySatoshis")) r.subsidy = obj["subsidySatoshis"].get<int64>(); | ||
| 774 | if (obj.contains("minerRewardSatoshis")) r.minerReward = obj["minerRewardSatoshis"].get<int64>(); | ||
| 775 | if (obj.contains("operatorRewardSatoshis")) r.operatorReward = obj["operatorRewardSatoshis"].get<int64>(); | ||
| 776 | if (obj.contains("skippedDust")) r.skippedDust = obj["skippedDust"].get<int>(); | ||
| 777 | if (obj.contains("forfeitedBadAddressSatoshis")) r.forfeitedBadAddress = obj["forfeitedBadAddressSatoshis"].get<int64>(); | ||
| 778 | if (obj.contains("shares") && obj["shares"].is_object()) | ||
| 779 | for (auto it = obj["shares"].begin(); it != obj["shares"].end(); ++it) | ||
| 780 | r.shares[it.key()] = it.value().get<uint64>(); | ||
| 781 | if (obj.contains("payouts") && obj["payouts"].is_object()) | ||
| 782 | for (auto it = obj["payouts"].begin(); it != obj["payouts"].end(); ++it) { | ||
| 783 | if (it.value().is_object()) { | ||
| 784 | r.payouts[it.key()] = it.value().value("amountSatoshis", (int64)0); | ||
| 785 | if (it.value().contains("txid") && it.value()["txid"].is_string()) | ||
| 786 | r.paidTxids[it.key()] = it.value()["txid"].get<std::string>(); | ||
| 787 | } else if (it.value().is_number_integer()) { | ||
| 788 | r.payouts[it.key()] = it.value().get<int64>(); | ||
| 789 | } | ||
| 790 | } | ||
| 791 | if (!r.id.empty()) | ||
| 792 | gPoolRounds.push_back(r); | ||
| 793 | } | ||
| 794 | while (gPoolRounds.size() > MAX_POOL_ROUNDS) | ||
| 795 | gPoolRounds.erase(gPoolRounds.begin()); | ||
| 796 | LogPrint("payout", "[payout] loaded %zu pool round proof(s) from disk\n", | ||
| 797 | gPoolRounds.size()); | ||
| 798 | } catch (const std::exception& e) { | ||
| 799 | LogPrint("payout", "[payout] WARNING: failed to parse %s: %s\n", | ||
| 800 | path.c_str(), e.what()); | ||
| 801 | } | ||
| 802 | } | ||
| 803 | |||
| 478 | // Is the block that funds this payout still on the main chain? | 804 | // Is the block that funds this payout still on the main chain? |
| 479 | // | 805 | // |
| 480 | // A zero hash means the entry was written before payouts recorded one, so | 806 | // A zero hash means the entry was written before payouts recorded one, so |
| @@ -499,85 +825,55 @@ static std::string PayoutComment(int blockHeight) | |||
| 499 | return strprintf("Pool payout height %d", blockHeight); | 825 | return strprintf("Pool payout height %d", blockHeight); |
| 500 | } | 826 | } |
| 501 | 827 | ||
| 502 | // Did we already send this payment before dying? Looks for a wallet | 828 | // Startup reconciliation. |
| 503 | // transaction of ours carrying this payout's comment and paying exactly this | ||
| 504 | // amount to this address. | ||
| 505 | // | 829 | // |
| 506 | // Biased towards reporting "already paid" on any doubt: skipping a payment a | 830 | // This used to be the crash-recovery mechanism: it scanned the wallet with |
| 507 | // miner is owed is visible and recoverable from the log, while paying twice | 831 | // WalletAlreadyPaid() and then dropped ledger entries that had already been |
| 508 | // is neither. | 832 | // exploded into in-flight records. Neither job is right any more. |
| 509 | static bool WalletAlreadyPaid(const PayingRecord& r) | 833 | // |
| 510 | { | 834 | // The ledger is now authoritative and an entry survives until every recipient |
| 511 | uint160 h160; | 835 | // has a txid recorded against it (upstream cbdd06d), so dropping ledger |
| 512 | if (!AddressToHash160(r.address, h160)) | 836 | // entries here would reintroduce exactly the loss this merge is adopting that |
| 513 | return false; | 837 | // model to prevent. And the wallet scan is now FindRecordedPoolPayout(), done |
| 514 | CScript want; | 838 | // once per record on its first attempt, which is the same question asked in |
| 515 | want << OP_DUP << OP_HASH160 << h160 << OP_EQUALVERIFY << OP_CHECKSIG; | 839 | // the one place that can act on the answer. Running both would be two |
| 516 | const std::string comment = PayoutComment(r.blockHeight); | 840 | // authorities for the same debt. |
| 517 | 841 | // | |
| 518 | CRITICAL_BLOCK(cs_mapWallet) | 842 | // What is left is the useful part: the in-flight list restored from disk may |
| 519 | { | 843 | // contain records the ledger already shows as paid (we died between the send |
| 520 | for (std::map<uint256, CWalletTx>::iterator it = mapWallet.begin(); | 844 | // and the in-flight file being rewritten). The ledger wins. |
| 521 | it != mapWallet.end(); ++it) { | ||
| 522 | CWalletTx& wtx = it->second; | ||
| 523 | // The comment is written only by the payout path, so its presence | ||
| 524 | // already identifies the transaction as one of ours. | ||
| 525 | std::map<std::string, std::string>::const_iterator ci = | ||
| 526 | wtx.mapValue.find("comment"); | ||
| 527 | if (ci == wtx.mapValue.end() || ci->second != comment) | ||
| 528 | continue; | ||
| 529 | foreach (const CTxOut& txout, wtx.vout) | ||
| 530 | if (txout.nValue == r.amount && txout.scriptPubKey == want) | ||
| 531 | return true; | ||
| 532 | } | ||
| 533 | } | ||
| 534 | return false; | ||
| 535 | } | ||
| 536 | |||
| 537 | // Startup reconciliation. Drops in-flight records the wallet shows were | ||
| 538 | // already sent, and drops ledger entries that were already exploded into | ||
| 539 | // records (which happens if we died between writing the records and rewriting | ||
| 540 | // the ledger -- both files then describe the same debt). | ||
| 541 | static void ReconcilePayouts() | 845 | static void ReconcilePayouts() |
| 542 | { | 846 | { |
| 543 | std::lock_guard<std::mutex> lk(gPayoutExecMutex); | 847 | std::lock_guard<std::mutex> exec(gPayoutExecMutex); |
| 848 | std::lock_guard<std::mutex> ledger(gPayoutLedgerMutex); | ||
| 544 | 849 | ||
| 545 | size_t settled = 0; | 850 | size_t settled = 0; |
| 546 | int64 settledAmount = 0; | 851 | int64 settledAmount = 0; |
| 547 | for (auto it = gPaying.begin(); it != gPaying.end(); ) { | 852 | for (auto it = gPaying.begin(); it != gPaying.end(); ) { |
| 548 | if (WalletAlreadyPaid(*it)) { | 853 | bool paid = false; |
| 854 | for (const PendingPayout& pp : gPendingPayouts) | ||
| 855 | if (pp.blockHash == it->blockHash && pp.paidTxids.count(it->address)) { | ||
| 856 | paid = true; | ||
| 857 | break; | ||
| 858 | } | ||
| 859 | if (paid) { | ||
| 549 | settledAmount += it->amount; | 860 | settledAmount += it->amount; |
| 550 | settled++; | 861 | settled++; |
| 551 | it = gPaying.erase(it); | 862 | it = gPaying.erase(it); |
| 552 | } else { | 863 | } else { |
| 864 | // Every restored record gets one wallet probe before it is allowed | ||
| 865 | // to send: this is precisely the window where the wallet may hold | ||
| 866 | // a transaction the ledger has not recorded yet. | ||
| 867 | it->fProbeWallet = true; | ||
| 553 | ++it; | 868 | ++it; |
| 554 | } | 869 | } |
| 555 | } | 870 | } |
| 556 | if (settled) { | 871 | if (settled) { |
| 557 | LogPrint("payout", "[payout] %zu in-flight payment(s) totalling %s were" | 872 | LogPrint("payout", "[payout] %zu in-flight payment(s) totalling %s are" |
| 558 | " already sent before the last shutdown -- not resending\n", | 873 | " already recorded paid in the ledger -- not resending\n", |
| 559 | settled, FormatMoney(settledAmount).c_str()); | 874 | settled, FormatMoney(settledAmount).c_str()); |
| 560 | SavePayingLocked(); | ||
| 561 | } | ||
| 562 | |||
| 563 | size_t dropped = 0; | ||
| 564 | for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) { | ||
| 565 | bool exploded = false; | ||
| 566 | for (const auto& r : gPaying) | ||
| 567 | if (r.matureAtHeight == it->matureAtHeight && r.blockHash == it->blockHash) { | ||
| 568 | exploded = true; | ||
| 569 | break; | ||
| 570 | } | ||
| 571 | if (exploded) { | ||
| 572 | LogPrint("payout", "[payout] ledger entry for block %d is already" | ||
| 573 | " in flight -- dropping the duplicate\n", it->blockHeight); | ||
| 574 | it = gPendingPayouts.erase(it); | ||
| 575 | dropped++; | ||
| 576 | } else { | ||
| 577 | ++it; | ||
| 578 | } | ||
| 579 | } | 875 | } |
| 580 | if (dropped) SavePendingPayouts(); | 876 | SavePayingLocked(); |
| 581 | 877 | ||
| 582 | if (!gPaying.empty()) { | 878 | if (!gPaying.empty()) { |
| 583 | int64 owed = 0; | 879 | int64 owed = 0; |
| @@ -587,12 +883,100 @@ static void ReconcilePayouts() | |||
| 587 | } | 883 | } |
| 588 | } | 884 | } |
| 589 | 885 | ||
| 886 | static void RecordPoolRound(const PoolRoundProof& round) | ||
| 887 | { | ||
| 888 | std::lock_guard<std::mutex> lk(gPoolRoundsMutex); | ||
| 889 | for (PoolRoundProof& existing : gPoolRounds) { | ||
| 890 | if (existing.id == round.id) { | ||
| 891 | existing = round; | ||
| 892 | SavePoolRoundsLocked(); | ||
| 893 | return; | ||
| 894 | } | ||
| 895 | } | ||
| 896 | gPoolRounds.push_back(round); | ||
| 897 | while (gPoolRounds.size() > MAX_POOL_ROUNDS) | ||
| 898 | gPoolRounds.erase(gPoolRounds.begin()); | ||
| 899 | SavePoolRoundsLocked(); | ||
| 900 | } | ||
| 901 | |||
| 902 | static void MarkPoolRoundPaid(const std::string& id, | ||
| 903 | const std::string& address, | ||
| 904 | const std::string& txid) | ||
| 905 | { | ||
| 906 | std::lock_guard<std::mutex> lk(gPoolRoundsMutex); | ||
| 907 | for (PoolRoundProof& r : gPoolRounds) { | ||
| 908 | if (r.id != id) | ||
| 909 | continue; | ||
| 910 | r.paidTxids[address] = txid; | ||
| 911 | SavePoolRoundsLocked(); | ||
| 912 | return; | ||
| 913 | } | ||
| 914 | } | ||
| 915 | |||
| 916 | // Upstream's gPayoutExecQueue/gPayoutExecMutex/gPayoutThreadRunning are | ||
| 917 | // declared above with gPaying: the in-flight list IS the exec queue in this | ||
| 918 | // tree, at per-recipient rather than per-round granularity, so a second queue | ||
| 919 | // would be a second answer to "who is owed money". Only the dedupe set is new. | ||
| 920 | // | ||
| 921 | // Ids of rounds currently exploded into gPaying, so a flush cannot queue the | ||
| 922 | // same round twice. Guarded by gPayoutExecMutex. | ||
| 923 | static std::set<std::string> gQueuedPayouts; | ||
| 924 | |||
| 925 | static bool FindRecordedPoolPayout(const std::string& id, | ||
| 926 | const std::string& address, | ||
| 927 | int64 amount, | ||
| 928 | std::string& txidRet) | ||
| 929 | { | ||
| 930 | CRITICAL_BLOCK(cs_mapWallet) | ||
| 931 | { | ||
| 932 | for (map<uint256, CWalletTx>::const_iterator it = mapWallet.begin(); | ||
| 933 | it != mapWallet.end(); ++it) | ||
| 934 | { | ||
| 935 | const CWalletTx& wtx = it->second; | ||
| 936 | map<string, string>::const_iterator idIt = wtx.mapValue.find("pool_payout_id"); | ||
| 937 | if (idIt == wtx.mapValue.end() || idIt->second != id) | ||
| 938 | continue; | ||
| 939 | map<string, string>::const_iterator addrIt = wtx.mapValue.find("pool_payout_address"); | ||
| 940 | if (addrIt == wtx.mapValue.end() || addrIt->second != address) | ||
| 941 | continue; | ||
| 942 | map<string, string>::const_iterator amtIt = wtx.mapValue.find("pool_payout_amount"); | ||
| 943 | if (amtIt == wtx.mapValue.end() || amtIt->second != strprintf("%lld", amount)) | ||
| 944 | continue; | ||
| 945 | txidRet = wtx.GetHash().GetHex(); | ||
| 946 | return true; | ||
| 947 | } | ||
| 948 | } | ||
| 949 | return false; | ||
| 950 | } | ||
| 951 | |||
| 952 | // blockHash stays a uint256 and not upstream's display string: it is the key | ||
| 953 | // this fork's payout ledger is built on, and a key that has been through | ||
| 954 | // ToString() is a key that can be compared wrong. It is converted to text only | ||
| 955 | // where the round proof needs text. The other three arguments are upstream's, | ||
| 956 | // carried for the round-proof data. | ||
| 590 | static void QueuePayouts(int blockHeight, | 957 | static void QueuePayouts(int blockHeight, |
| 591 | const uint256& blockHash, | 958 | const uint256& blockHash, |
| 959 | const std::string& coinbaseTxid, | ||
| 960 | int64 coinbaseValue, | ||
| 961 | const std::string& foundBy, | ||
| 592 | const std::map<std::string, uint64>& shareCount, | 962 | const std::map<std::string, uint64>& shareCount, |
| 593 | uint64 total) | 963 | uint64 total) |
| 594 | { | 964 | { |
| 965 | PoolRoundProof round; | ||
| 966 | round.id = PayoutIdForBlock(blockHeight, blockHash); | ||
| 967 | round.blockHeight = blockHeight; | ||
| 968 | round.matureAtHeight = blockHeight + COINBASE_SPENDABLE_DEPTH; | ||
| 969 | round.createdAt = GetTime(); | ||
| 970 | round.blockHash = blockHash.ToString(); | ||
| 971 | round.coinbaseTxid = coinbaseTxid; | ||
| 972 | round.coinbaseValue = coinbaseValue; | ||
| 973 | round.foundBy = foundBy; | ||
| 974 | round.feePercent = dPoolFeePercent; | ||
| 975 | round.totalShares = total; | ||
| 976 | round.shares = shareCount; | ||
| 977 | |||
| 595 | if (total == 0) { | 978 | if (total == 0) { |
| 979 | RecordPoolRound(round); | ||
| 596 | LogPrint("payout", "[payout] block %d: zero shares recorded -- nothing to pay out\n", | 980 | LogPrint("payout", "[payout] block %d: zero shares recorded -- nothing to pay out\n", |
| 597 | blockHeight); | 981 | blockHeight); |
| 598 | return; | 982 | return; |
| @@ -608,6 +992,10 @@ static void QueuePayouts(int blockHeight, | |||
| 608 | int64 nSubsidy = 50 * COIN; | 992 | int64 nSubsidy = 50 * COIN; |
| 609 | nSubsidy >>= (blockHeight / 210000); // halving schedule | 993 | nSubsidy >>= (blockHeight / 210000); // halving schedule |
| 610 | int64 minerReward = (int64)((double)nSubsidy * (1.0 - feeFrac)); | 994 | int64 minerReward = (int64)((double)nSubsidy * (1.0 - feeFrac)); |
| 995 | int64 operatorCut = nSubsidy - minerReward; | ||
| 996 | round.subsidy = nSubsidy; | ||
| 997 | round.minerReward = minerReward; | ||
| 998 | round.operatorReward = operatorCut; | ||
| 611 | 999 | ||
| 612 | PendingPayout pp; | 1000 | PendingPayout pp; |
| 613 | pp.blockHeight = blockHeight; | 1001 | pp.blockHeight = blockHeight; |
| @@ -621,7 +1009,9 @@ static void QueuePayouts(int blockHeight, | |||
| 621 | // alone on a private fork paying itself, so it says nothing about a pool | 1009 | // alone on a private fork paying itself, so it says nothing about a pool |
| 622 | // competing on the real chain. Waiting the extra 20 blocks makes each | 1010 | // competing on the real chain. Waiting the extra 20 blocks makes each |
| 623 | // payout funded by construction, and puts the maturity gate past the | 1011 | // payout funded by construction, and puts the maturity gate past the |
| 624 | // reorg depth that orphaned 280 blocks of obligations. | 1012 | // reorg depth that orphaned 280 blocks of obligations. Upstream uses |
| 1013 | // COINBASE_MATURITY here; this is a considered disagreement, not an | ||
| 1014 | // oversight -- see the merge commit message. | ||
| 625 | pp.matureAtHeight = blockHeight + COINBASE_SPENDABLE_DEPTH; | 1015 | pp.matureAtHeight = blockHeight + COINBASE_SPENDABLE_DEPTH; |
| 626 | 1016 | ||
| 627 | int skipped = 0; | 1017 | int skipped = 0; |
| @@ -643,12 +1033,18 @@ static void QueuePayouts(int blockHeight, | |||
| 643 | continue; | 1033 | continue; |
| 644 | } | 1034 | } |
| 645 | pp.amounts[kv.first] = amount; | 1035 | pp.amounts[kv.first] = amount; |
| 1036 | round.payouts[kv.first] = amount; | ||
| 646 | } | 1037 | } |
| 1038 | round.skippedDust = skipped; | ||
| 1039 | round.forfeitedBadAddress = burned; | ||
| 647 | 1040 | ||
| 648 | gPendingPayouts.push_back(pp); | 1041 | { |
| 649 | SavePendingPayouts(); | 1042 | std::lock_guard<std::mutex> lk(gPayoutLedgerMutex); |
| 1043 | gPendingPayouts.push_back(pp); | ||
| 1044 | SavePendingPayoutsLocked(); | ||
| 1045 | } | ||
| 1046 | RecordPoolRound(round); | ||
| 650 | 1047 | ||
| 651 | int64 operatorCut = nSubsidy - minerReward; | ||
| 652 | LogPrint("payout", | 1048 | LogPrint("payout", |
| 653 | "[payout] block %d: subsidy %s BTF, miners get %s" | 1049 | "[payout] block %d: subsidy %s BTF, miners get %s" |
| 654 | " (fee %.2f%% = %s to operator)," | 1050 | " (fee %.2f%% = %s to operator)," |
| @@ -664,66 +1060,110 @@ static void QueuePayouts(int blockHeight, | |||
| 664 | : ""); | 1060 | : ""); |
| 665 | } | 1061 | } |
| 666 | 1062 | ||
| 667 | // Called from event loop: move matured entries into the in-flight list (fast). | 1063 | // Called from the event loop: explode matured ledger entries into per-recipient |
| 1064 | // in-flight records (fast; no wallet or disk work beyond one small write). | ||
| 1065 | // | ||
| 1066 | // The ledger entry SURVIVES here. That is upstream's model and it is the | ||
| 1067 | // crash-safe one: this fork used to erase the entry as soon as its payments | ||
| 1068 | // were handed off, so a death in that window left the debt described by | ||
| 1069 | // nothing at all -- a container restart on 2026-07-30 silently dropped | ||
| 1070 | // ~1,156 BTF exactly that way. An entry is removed only once every recipient | ||
| 1071 | // in it has a txid recorded (PayoutFullyPaid), which the payout thread does. | ||
| 668 | // | 1072 | // |
| 669 | // Ordering matters and is deliberate: the in-flight records are written and | 1073 | // gPaying is kept on top of that as the in-flight working set: it carries the |
| 670 | // fsync'd to their own file *before* the ledger entry is erased, so the debt | 1074 | // per-recipient retry/deferral state that a per-round `nextAttempt` cannot |
| 671 | // is always described by at least one of the two files. Dying in the window | 1075 | // express, and it is what the payout thread iterates. It is a cache of work |
| 672 | // where both describe it is handled at startup by ReconcilePayouts, which | 1076 | // derived from the ledger, never an independent record of debt -- if it were |
| 673 | // drops the ledger entry it can see is already in flight. | 1077 | // lost entirely the ledger would rebuild it on the next flush. |
| 674 | static void FlushMaturePayouts() | 1078 | static void FlushMaturePayouts() |
| 675 | { | 1079 | { |
| 676 | // Void obligations whose funding block lost a reorg. The coin that was | 1080 | // Which funding blocks are still on the main chain? Asked with no payout |
| 677 | // meant to pay them does not exist, so paying anyway spends real-chain | 1081 | // lock held: PayoutBlockStillOnChain takes cs_main, and cs_main is held by |
| 678 | // coin the pool owes to somebody else. | 1082 | // ProcessBlock and SendMoney for long stretches. Holding the ledger mutex |
| 679 | bool voided = false; | 1083 | // across it would let the event loop park the payout thread behind a chain |
| 680 | for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) { | 1084 | // lock it has no interest in. (It is not a deadlock either way -- nothing |
| 681 | if (PayoutBlockStillOnChain(it->blockHash)) { ++it; continue; } | 1085 | // takes the ledger mutex while holding cs_main -- but it is exactly the |
| 682 | int64 total = 0; | 1086 | // kind of stall this merge is removing from the ledger path.) |
| 683 | for (auto& kv : it->amounts) total += kv.second; | 1087 | std::vector<uint256> orphaned; |
| 684 | LogPrint("payout", "[payout] block %d (%s) was orphaned -- voiding %zu" | 1088 | { |
| 685 | " payment(s) totalling %s\n", | 1089 | std::vector<uint256> hashes; |
| 686 | it->blockHeight, it->blockHash.ToString().substr(0, 14).c_str(), | 1090 | { |
| 687 | it->amounts.size(), FormatMoney(total).c_str()); | 1091 | std::lock_guard<std::mutex> ledger(gPayoutLedgerMutex); |
| 688 | it = gPendingPayouts.erase(it); | 1092 | for (const PendingPayout& pp : gPendingPayouts) |
| 689 | voided = true; | 1093 | hashes.push_back(pp.blockHash); |
| 1094 | } | ||
| 1095 | for (const uint256& h : hashes) | ||
| 1096 | if (!PayoutBlockStillOnChain(h)) | ||
| 1097 | orphaned.push_back(h); | ||
| 690 | } | 1098 | } |
| 691 | if (voided) SavePendingPayouts(); | ||
| 692 | 1099 | ||
| 693 | std::vector<PendingPayout> ready; | 1100 | std::vector<PendingPayout> ready; |
| 694 | for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) { | 1101 | { |
| 695 | if (nBestHeight < it->matureAtHeight) { ++it; continue; } | 1102 | std::lock_guard<std::mutex> ledger(gPayoutLedgerMutex); |
| 696 | LogPrint("payout", "[payout] block height %d matured -- queuing %zu payments\n", | 1103 | |
| 697 | it->matureAtHeight, it->amounts.size()); | 1104 | // Void obligations whose funding block lost a reorg. The coin that was |
| 698 | ready.push_back(*it); // copy first; erase after save | 1105 | // meant to pay them does not exist, so paying anyway spends real-chain |
| 699 | ++it; | 1106 | // coin the pool owes to somebody else. |
| 700 | } | 1107 | bool voided = false; |
| 701 | if (!ready.empty()) { | ||
| 702 | // Record the individual payments first, and only then drop the ledger | ||
| 703 | // entries that produced them. | ||
| 704 | { | ||
| 705 | std::lock_guard<std::mutex> lk(gPayoutExecMutex); | ||
| 706 | for (const auto& pp : ready) { | ||
| 707 | for (const auto& kv : pp.amounts) { | ||
| 708 | PayingRecord r; | ||
| 709 | r.blockHeight = pp.blockHeight; | ||
| 710 | r.blockHash = pp.blockHash; | ||
| 711 | r.matureAtHeight = pp.matureAtHeight; | ||
| 712 | r.address = kv.first; | ||
| 713 | r.amount = kv.second; | ||
| 714 | gPaying.push_back(r); | ||
| 715 | } | ||
| 716 | } | ||
| 717 | SavePayingLocked(); | ||
| 718 | } | ||
| 719 | for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) { | 1108 | for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) { |
| 720 | if (nBestHeight >= it->matureAtHeight) | 1109 | if (std::find(orphaned.begin(), orphaned.end(), it->blockHash) == orphaned.end()) { |
| 721 | it = gPendingPayouts.erase(it); | ||
| 722 | else | ||
| 723 | ++it; | 1110 | ++it; |
| 1111 | continue; | ||
| 1112 | } | ||
| 1113 | int64 total = 0; | ||
| 1114 | for (auto& kv : it->amounts) total += kv.second; | ||
| 1115 | LogPrint("payout", "[payout] block %d (%s) was orphaned -- voiding %zu" | ||
| 1116 | " payment(s) totalling %s\n", | ||
| 1117 | it->blockHeight, it->blockHash.ToString().substr(0, 14).c_str(), | ||
| 1118 | it->amounts.size(), FormatMoney(total).c_str()); | ||
| 1119 | it = gPendingPayouts.erase(it); | ||
| 1120 | voided = true; | ||
| 724 | } | 1121 | } |
| 725 | SavePendingPayouts(); | 1122 | if (voided) SavePendingPayoutsLocked(); |
| 1123 | |||
| 1124 | int64 now = GetTime(); | ||
| 1125 | for (const PendingPayout& pp : gPendingPayouts) { | ||
| 1126 | if (nBestHeight < pp.matureAtHeight) | ||
| 1127 | continue; | ||
| 1128 | if (pp.retryAfter > now) | ||
| 1129 | continue; | ||
| 1130 | if (PayoutFullyPaid(pp)) | ||
| 1131 | continue; | ||
| 1132 | ready.push_back(pp); | ||
| 1133 | } | ||
| 1134 | } | ||
| 1135 | |||
| 1136 | if (ready.empty()) | ||
| 1137 | return; | ||
| 1138 | |||
| 1139 | std::lock_guard<std::mutex> queue(gPayoutExecMutex); | ||
| 1140 | size_t added = 0; | ||
| 1141 | for (const PendingPayout& pp : ready) { | ||
| 1142 | const std::string id = PayoutIdForBlock(pp.blockHeight, pp.blockHash); | ||
| 1143 | if (gQueuedPayouts.count(id)) | ||
| 1144 | continue; // already exploded into gPaying and still in flight | ||
| 1145 | size_t n = 0; | ||
| 1146 | for (const auto& kv : pp.amounts) { | ||
| 1147 | // Upstream's per-recipient dedupe: a recipient already carrying a | ||
| 1148 | // txid is done, even though the entry as a whole is not. | ||
| 1149 | if (pp.paidTxids.count(kv.first)) | ||
| 1150 | continue; | ||
| 1151 | PayingRecord r; | ||
| 1152 | r.blockHeight = pp.blockHeight; | ||
| 1153 | r.blockHash = pp.blockHash; | ||
| 1154 | r.matureAtHeight = pp.matureAtHeight; | ||
| 1155 | r.address = kv.first; | ||
| 1156 | r.amount = kv.second; | ||
| 1157 | gPaying.push_back(r); | ||
| 1158 | n++; | ||
| 1159 | } | ||
| 1160 | gQueuedPayouts.insert(id); | ||
| 1161 | added += n; | ||
| 1162 | LogPrint("payout", "[payout] block height %d matured -- queuing payout %s" | ||
| 1163 | " (%zu of %zu recipients still owed)\n", | ||
| 1164 | pp.matureAtHeight, id.c_str(), n, pp.amounts.size()); | ||
| 726 | } | 1165 | } |
| 1166 | if (added) SavePayingLocked(); | ||
| 727 | } | 1167 | } |
| 728 | 1168 | ||
| 729 | // Identity of a payment, for finding its record again after SendMoney. | 1169 | // Identity of a payment, for finding its record again after SendMoney. |
| @@ -737,13 +1177,24 @@ static bool SamePayment(const PayingRecord& a, const PayingRecord& b) | |||
| 737 | } | 1177 | } |
| 738 | 1178 | ||
| 739 | // Settle a payment: it is either done or abandoned, either way it stops being | 1179 | // Settle a payment: it is either done or abandoned, either way it stops being |
| 740 | // owed. Removing the record and rewriting the file is what makes the payment | 1180 | // in flight. Note this only retires the working record -- whether the miner is |
| 741 | // not happen twice after a restart. | 1181 | // still OWED the money is the ledger's business, and the ledger is updated |
| 1182 | // separately by RecordPayoutTxid(). | ||
| 742 | static void ForgetPayment(const PayingRecord& r) | 1183 | static void ForgetPayment(const PayingRecord& r) |
| 743 | { | 1184 | { |
| 744 | std::lock_guard<std::mutex> lk(gPayoutExecMutex); | 1185 | std::lock_guard<std::mutex> lk(gPayoutExecMutex); |
| 745 | for (auto it = gPaying.begin(); it != gPaying.end(); ++it) | 1186 | for (auto it = gPaying.begin(); it != gPaying.end(); ++it) |
| 746 | if (SamePayment(*it, r)) { gPaying.erase(it); break; } | 1187 | if (SamePayment(*it, r)) { gPaying.erase(it); break; } |
| 1188 | |||
| 1189 | // When the last record for a round is gone the round is no longer in | ||
| 1190 | // flight, so a later flush may explode it again if the ledger still shows | ||
| 1191 | // recipients owed (a give-up, for instance, leaves the entry standing). | ||
| 1192 | bool stillInFlight = false; | ||
| 1193 | for (const auto& e : gPaying) | ||
| 1194 | if (e.blockHash == r.blockHash) { stillInFlight = true; break; } | ||
| 1195 | if (!stillInFlight) | ||
| 1196 | gQueuedPayouts.erase(PayoutIdForBlock(r.blockHeight, r.blockHash)); | ||
| 1197 | |||
| 747 | SavePayingLocked(); | 1198 | SavePayingLocked(); |
| 748 | } | 1199 | } |
| 749 | 1200 | ||
| @@ -753,14 +1204,77 @@ static void ReschedulePayment(const PayingRecord& r) | |||
| 753 | std::lock_guard<std::mutex> lk(gPayoutExecMutex); | 1204 | std::lock_guard<std::mutex> lk(gPayoutExecMutex); |
| 754 | for (auto& e : gPaying) | 1205 | for (auto& e : gPaying) |
| 755 | if (SamePayment(e, r)) { | 1206 | if (SamePayment(e, r)) { |
| 756 | e.retries = r.retries; | 1207 | e.retries = r.retries; |
| 757 | e.deferrals = r.deferrals; | 1208 | e.deferrals = r.deferrals; |
| 758 | e.retryAfter = r.retryAfter; | 1209 | e.retryAfter = r.retryAfter; |
| 1210 | // Carry the cleared probe flag back, or every retry re-scans the | ||
| 1211 | // whole wallet -- the cost this merge is specifically removing. | ||
| 1212 | e.fProbeWallet = r.fProbeWallet; | ||
| 759 | break; | 1213 | break; |
| 760 | } | 1214 | } |
| 761 | SavePayingLocked(); | 1215 | SavePayingLocked(); |
| 762 | } | 1216 | } |
| 763 | 1217 | ||
| 1218 | // Record that a recipient has been paid, in the ledger, which is the only | ||
| 1219 | // durable statement that the debt is settled. | ||
| 1220 | // | ||
| 1221 | // The file write happens with gPayoutLedgerMutex RELEASED. Upstream calls | ||
| 1222 | // SavePendingPayoutsLocked() inside the per-recipient loop with the mutex | ||
| 1223 | // held, so every recipient costs the event loop a full serialise+fsync it has | ||
| 1224 | // to wait for in FlushMaturePayouts. | ||
| 1225 | static void RecordPayoutTxid(const PayingRecord& r, const std::string& txid) | ||
| 1226 | { | ||
| 1227 | std::string blob; | ||
| 1228 | bool complete = false; | ||
| 1229 | { | ||
| 1230 | std::lock_guard<std::mutex> lk(gPayoutLedgerMutex); | ||
| 1231 | for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ++it) { | ||
| 1232 | if (it->blockHash != r.blockHash) | ||
| 1233 | continue; | ||
| 1234 | it->paidTxids[r.address] = txid; | ||
| 1235 | it->retryAfter = 0; | ||
| 1236 | if (PayoutFullyPaid(*it)) { | ||
| 1237 | LogPrint("payout", "[payout] payout for block %d complete --" | ||
| 1238 | " removing from ledger\n", it->blockHeight); | ||
| 1239 | gPendingPayouts.erase(it); | ||
| 1240 | complete = true; | ||
| 1241 | } | ||
| 1242 | break; | ||
| 1243 | } | ||
| 1244 | blob = SerializePendingPayoutsLocked(); | ||
| 1245 | } | ||
| 1246 | WritePendingPayoutsFile(blob); | ||
| 1247 | // The round proof records a txid per address, so this is per recipient and | ||
| 1248 | // not only on completion. | ||
| 1249 | MarkPoolRoundPaid(PayoutIdForBlock(r.blockHeight, r.blockHash), r.address, txid); | ||
| 1250 | if (complete) | ||
| 1251 | LogPrint("payout", "[payout] block %d fully settled\n", r.blockHeight); | ||
| 1252 | } | ||
| 1253 | |||
| 1254 | // Write a recipient off. Only reached after the full retry budget against a | ||
| 1255 | // block that is still on the main chain with coin available to spend, i.e. a | ||
| 1256 | // wallet fault rather than a maturity or reorg question. | ||
| 1257 | // | ||
| 1258 | // The amount is removed from the entry rather than marked paid, so the money | ||
| 1259 | // is not reported as sent. Once nothing is left owed the entry retires. | ||
| 1260 | static void ForfeitPayoutRecipient(const PayingRecord& r) | ||
| 1261 | { | ||
| 1262 | std::string blob; | ||
| 1263 | { | ||
| 1264 | std::lock_guard<std::mutex> lk(gPayoutLedgerMutex); | ||
| 1265 | for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ++it) { | ||
| 1266 | if (it->blockHash != r.blockHash) | ||
| 1267 | continue; | ||
| 1268 | it->amounts.erase(r.address); | ||
| 1269 | if (it->amounts.empty() || PayoutFullyPaid(*it)) | ||
| 1270 | gPendingPayouts.erase(it); | ||
| 1271 | break; | ||
| 1272 | } | ||
| 1273 | blob = SerializePendingPayoutsLocked(); | ||
| 1274 | } | ||
| 1275 | WritePendingPayoutsFile(blob); | ||
| 1276 | } | ||
| 1277 | |||
| 764 | // Payout execution thread: works through gPaying, calls SendMoney. | 1278 | // Payout execution thread: works through gPaying, calls SendMoney. |
| 765 | // Runs independently of the event loop so slow wallet operations | 1279 | // Runs independently of the event loop so slow wallet operations |
| 766 | // (SendMoney takes cs_main) never stall miner I/O. | 1280 | // (SendMoney takes cs_main) never stall miner I/O. |
| @@ -780,9 +1294,12 @@ static void PayoutThreadFn(void*) | |||
| 780 | } | 1294 | } |
| 781 | 1295 | ||
| 782 | for (PayingRecord r : batch) { | 1296 | for (PayingRecord r : batch) { |
| 1297 | const std::string id = PayoutIdForBlock(r.blockHeight, r.blockHash); | ||
| 1298 | |||
| 783 | // Re-check here and not only at maturity: a payment can sit in this | 1299 | // Re-check here and not only at maturity: a payment can sit in this |
| 784 | // list for hours across retries, and the reorg that kills its | 1300 | // list for hours across retries, and the reorg that kills its |
| 785 | // funding block can land at any point in that window. | 1301 | // funding block can land at any point in that window. Upstream has |
| 1302 | // no equivalent -- it decides once, when the entry matures. | ||
| 786 | if (!PayoutBlockStillOnChain(r.blockHash)) { | 1303 | if (!PayoutBlockStillOnChain(r.blockHash)) { |
| 787 | LogPrint("payout", "[payout] block %d (%s) orphaned while queued" | 1304 | LogPrint("payout", "[payout] block %d (%s) orphaned while queued" |
| 788 | " -- dropping %s to %s\n", | 1305 | " -- dropping %s to %s\n", |
| @@ -792,6 +1309,25 @@ static void PayoutThreadFn(void*) | |||
| 792 | continue; | 1309 | continue; |
| 793 | } | 1310 | } |
| 794 | 1311 | ||
| 1312 | // Upstream's per-recipient dedupe, against the ledger rather than | ||
| 1313 | // against the batch: if a txid is already recorded for this | ||
| 1314 | // address, the money went out and nothing more is owed. | ||
| 1315 | { | ||
| 1316 | bool alreadyPaid = false; | ||
| 1317 | { | ||
| 1318 | std::lock_guard<std::mutex> lk(gPayoutLedgerMutex); | ||
| 1319 | for (const PendingPayout& pp : gPendingPayouts) | ||
| 1320 | if (pp.blockHash == r.blockHash && pp.paidTxids.count(r.address)) { | ||
| 1321 | alreadyPaid = true; | ||
| 1322 | break; | ||
| 1323 | } | ||
| 1324 | } | ||
| 1325 | if (alreadyPaid) { | ||
| 1326 | ForgetPayment(r); | ||
| 1327 | continue; | ||
| 1328 | } | ||
| 1329 | } | ||
| 1330 | |||
| 795 | uint160 h160; | 1331 | uint160 h160; |
| 796 | if (!AddressToHash160(r.address, h160)) { | 1332 | if (!AddressToHash160(r.address, h160)) { |
| 797 | // Should not happen -- bad addresses are filtered in QueuePayouts | 1333 | // Should not happen -- bad addresses are filtered in QueuePayouts |
| @@ -803,6 +1339,22 @@ static void PayoutThreadFn(void*) | |||
| 803 | CScript sc; | 1339 | CScript sc; |
| 804 | sc << OP_DUP << OP_HASH160 << h160 << OP_EQUALVERIFY << OP_CHECKSIG; | 1340 | sc << OP_DUP << OP_HASH160 << h160 << OP_EQUALVERIFY << OP_CHECKSIG; |
| 805 | 1341 | ||
| 1342 | // Did a previous process send this and die before recording it? | ||
| 1343 | // Only asked once per record -- see PayingRecord::fProbeWallet. | ||
| 1344 | if (r.fProbeWallet) { | ||
| 1345 | std::string recovered; | ||
| 1346 | r.fProbeWallet = false; | ||
| 1347 | if (FindRecordedPoolPayout(id, r.address, r.amount, recovered)) { | ||
| 1348 | LogPrint("payout", "[payout] recovered recorded payout %s to %s" | ||
| 1349 | " from wallet tx %s -- not resending\n", | ||
| 1350 | FormatMoney(r.amount).c_str(), r.address.c_str(), | ||
| 1351 | recovered.c_str()); | ||
| 1352 | RecordPayoutTxid(r, recovered); | ||
| 1353 | ForgetPayment(r); | ||
| 1354 | continue; | ||
| 1355 | } | ||
| 1356 | } | ||
| 1357 | |||
| 806 | // Waiting for coin to mature is not a failure, so it must not | 1358 | // Waiting for coin to mature is not a failure, so it must not |
| 807 | // consume the retry budget. Burning a retry every 60s means a | 1359 | // consume the retry budget. Burning a retry every 60s means a |
| 808 | // payment the pool fully intends to honour can hit GIVING UP | 1360 | // payment the pool fully intends to honour can hit GIVING UP |
| @@ -825,11 +1377,22 @@ static void PayoutThreadFn(void*) | |||
| 825 | 1377 | ||
| 826 | CWalletTx wtx; | 1378 | CWalletTx wtx; |
| 827 | wtx.mapValue["comment"] = PayoutComment(r.blockHeight); | 1379 | wtx.mapValue["comment"] = PayoutComment(r.blockHeight); |
| 1380 | // Upstream's structured metadata. This is what makes the wallet | ||
| 1381 | // probe above possible at all -- the old comment-only form could | ||
| 1382 | // not distinguish two rounds paying the same address the same | ||
| 1383 | // amount. Keyed by the hash-derived id, so a reorg cannot make one | ||
| 1384 | // round's transaction answer for another's. | ||
| 1385 | wtx.mapValue["pool_payout_id"] = id; | ||
| 1386 | wtx.mapValue["pool_payout_address"] = r.address; | ||
| 1387 | wtx.mapValue["pool_payout_amount"] = strprintf("%lld", r.amount); | ||
| 828 | 1388 | ||
| 829 | if (SendMoney(sc, r.amount, wtx)) { | 1389 | if (SendMoney(sc, r.amount, wtx)) { |
| 830 | // The wallet holds the transaction before this returns, so a | 1390 | // The wallet holds the transaction before this returns, so a |
| 831 | // crash here is recoverable: ReconcilePayouts finds it by | 1391 | // crash between here and the ledger write is recoverable: the |
| 832 | // comment+address+amount and does not resend. | 1392 | // probe above finds it by id+address+amount and does not |
| 1393 | // resend. Ledger first, in-flight record second -- the debt is | ||
| 1394 | // described by at least one of them at every instant. | ||
| 1395 | RecordPayoutTxid(r, wtx.GetHash().GetHex()); | ||
| 833 | ForgetPayment(r); | 1396 | ForgetPayment(r); |
| 834 | LogPrint("payout", "[payout] paid %s to %s (tx %s)\n", | 1397 | LogPrint("payout", "[payout] paid %s to %s (tx %s)\n", |
| 835 | FormatMoney(r.amount).c_str(), r.address.c_str(), | 1398 | FormatMoney(r.amount).c_str(), r.address.c_str(), |
| @@ -857,6 +1420,12 @@ static void PayoutThreadFn(void*) | |||
| 857 | FormatMoney(r.amount).c_str(), r.address.c_str(), | 1420 | FormatMoney(r.amount).c_str(), r.address.c_str(), |
| 858 | r.retries, r.blockHeight, | 1421 | r.retries, r.blockHeight, |
| 859 | FormatMoney(nSpendable).c_str()); | 1422 | FormatMoney(nSpendable).c_str()); |
| 1423 | // Write the debt off in the ledger as well as dropping the | ||
| 1424 | // in-flight record. Under the survives-until-fully-paid model | ||
| 1425 | // the entry would otherwise never complete, and every later | ||
| 1426 | // flush would explode it again -- a forfeit that silently | ||
| 1427 | // becomes an infinite retry. | ||
| 1428 | ForfeitPayoutRecipient(r); | ||
| 860 | ForgetPayment(r); | 1429 | ForgetPayment(r); |
| 861 | } | 1430 | } |
| 862 | } | 1431 | } |
| @@ -881,6 +1450,7 @@ static StatsSnapshot gStats; | |||
| 881 | static std::vector<PoolWorkerStatView> gWorkerStats; | 1450 | static std::vector<PoolWorkerStatView> gWorkerStats; |
| 882 | static std::vector<PendingPayoutView> gPayoutViews; | 1451 | static std::vector<PendingPayoutView> gPayoutViews; |
| 883 | static std::mutex gStatsMutex; | 1452 | static std::mutex gStatsMutex; |
| 1453 | static int64 gLastPoolStatusWrite = 0; | ||
| 884 | 1454 | ||
| 885 | void GetPoolOperatorStats(int& authorizedMiners, int& blocksFound, uint64& roundShares, double& totalHashRate) | 1455 | void GetPoolOperatorStats(int& authorizedMiners, int& blocksFound, uint64& roundShares, double& totalHashRate) |
| 886 | { | 1456 | { |
| @@ -903,6 +1473,107 @@ void GetPendingPayouts(std::vector<PendingPayoutView>& out) | |||
| 903 | out = gPayoutViews; | 1473 | out = gPayoutViews; |
| 904 | } | 1474 | } |
| 905 | 1475 | ||
| 1476 | static std::string PoolStatusFilePath() | ||
| 1477 | { | ||
| 1478 | if (!strPoolStatusFile.empty()) | ||
| 1479 | return strPoolStatusFile; | ||
| 1480 | return GetAppDir() + "/pool_status.json"; | ||
| 1481 | } | ||
| 1482 | |||
| 1483 | static void WritePoolStatusJsonLocked() | ||
| 1484 | { | ||
| 1485 | nlohmann::json j; | ||
| 1486 | j["schema"] = "bitflash-pool-status-1"; | ||
| 1487 | j["network"] = "bitflash"; | ||
| 1488 | j["updatedAt"] = GetTime(); | ||
| 1489 | |||
| 1490 | j["operator"] = { | ||
| 1491 | {"name", strPoolName}, | ||
| 1492 | {"btfAddress", BtfLocalAddress()}, | ||
| 1493 | {"feePercent", dPoolFeePercent}, | ||
| 1494 | {"dashboardUrl", strPoolDashboardUrl}, | ||
| 1495 | {"roundsFile", "pool_rounds.json"} | ||
| 1496 | }; | ||
| 1497 | |||
| 1498 | j["node"] = { | ||
| 1499 | {"height", nBestHeight}, | ||
| 1500 | {"poolRunning", (bool)gPoolRunning}, | ||
| 1501 | {"mode", nMineMode} | ||
| 1502 | }; | ||
| 1503 | |||
| 1504 | j["pool"] = { | ||
| 1505 | {"authorizedMiners", gStats.authorizedMiners}, | ||
| 1506 | {"blocksFoundSession", gStats.blocksFound}, | ||
| 1507 | {"roundShares", gStats.roundShares}, | ||
| 1508 | {"hashRate", gStats.totalHashRate} | ||
| 1509 | }; | ||
| 1510 | |||
| 1511 | j["workers"] = nlohmann::json::array(); | ||
| 1512 | for (const PoolWorkerStatView& row : gWorkerStats) { | ||
| 1513 | j["workers"].push_back({ | ||
| 1514 | {"address", row.address}, | ||
| 1515 | {"worker", row.worker}, | ||
| 1516 | {"totalShares", row.totalShares}, | ||
| 1517 | {"roundShares", row.roundShares}, | ||
| 1518 | {"lastSeen", row.lastSeen}, | ||
| 1519 | {"hashRate", row.hashRate} | ||
| 1520 | }); | ||
| 1521 | } | ||
| 1522 | |||
| 1523 | j["pendingPayouts"] = nlohmann::json::array(); | ||
| 1524 | for (const PendingPayoutView& row : gPayoutViews) { | ||
| 1525 | j["pendingPayouts"].push_back({ | ||
| 1526 | {"matureAtHeight", row.matureAtHeight}, | ||
| 1527 | {"recipients", row.recipients}, | ||
| 1528 | {"totalAmountSatoshis", row.totalAmount}, | ||
| 1529 | {"totalAmountBtf", FormatMoney(row.totalAmount)} | ||
| 1530 | }); | ||
| 1531 | } | ||
| 1532 | |||
| 1533 | j["recentRounds"] = nlohmann::json::array(); | ||
| 1534 | { | ||
| 1535 | std::lock_guard<std::mutex> lk(gPoolRoundsMutex); | ||
| 1536 | size_t start = gPoolRounds.size() > 10 ? gPoolRounds.size() - 10 : 0; | ||
| 1537 | for (size_t i = start; i < gPoolRounds.size(); i++) { | ||
| 1538 | const PoolRoundProof& r = gPoolRounds[i]; | ||
| 1539 | int unpaid = 0; | ||
| 1540 | for (const auto& kv : r.payouts) | ||
| 1541 | if (r.paidTxids.count(kv.first) == 0) | ||
| 1542 | unpaid++; | ||
| 1543 | j["recentRounds"].push_back({ | ||
| 1544 | {"id", r.id}, | ||
| 1545 | {"blockHeight", r.blockHeight}, | ||
| 1546 | {"blockHash", r.blockHash}, | ||
| 1547 | {"coinbaseTxid", r.coinbaseTxid}, | ||
| 1548 | {"totalShares", r.totalShares}, | ||
| 1549 | {"payouts", (int)r.payouts.size()}, | ||
| 1550 | {"unpaidPayouts", unpaid}, | ||
| 1551 | {"coinbaseValueBtf", FormatMoney(r.coinbaseValue)}, | ||
| 1552 | {"minerRewardBtf", FormatMoney(r.minerReward)} | ||
| 1553 | }); | ||
| 1554 | } | ||
| 1555 | } | ||
| 1556 | |||
| 1557 | std::string path = PoolStatusFilePath(); | ||
| 1558 | std::string tmp = path + ".tmp"; | ||
| 1559 | FILE* f = fopen(tmp.c_str(), "w"); | ||
| 1560 | if (!f) { | ||
| 1561 | LogPrint("pool", "[pool] WARNING: could not write status file %s\n", tmp.c_str()); | ||
| 1562 | return; | ||
| 1563 | } | ||
| 1564 | std::string s = j.dump(2); | ||
| 1565 | fwrite(s.c_str(), 1, s.size(), f); | ||
| 1566 | if (fclose(f) != 0) { | ||
| 1567 | LogPrint("pool", "[pool] WARNING: could not close status file %s\n", tmp.c_str()); | ||
| 1568 | return; | ||
| 1569 | } | ||
| 1570 | #ifdef _WIN32 | ||
| 1571 | remove(path.c_str()); | ||
| 1572 | #endif | ||
| 1573 | if (rename(tmp.c_str(), path.c_str()) != 0) | ||
| 1574 | LogPrint("pool", "[pool] WARNING: could not replace status file %s\n", path.c_str()); | ||
| 1575 | } | ||
| 1576 | |||
| 906 | // --------------------------------------------------------------------------- | 1577 | // --------------------------------------------------------------------------- |
| 907 | // Miner state -- event loop thread only | 1578 | // Miner state -- event loop thread only |
| 908 | // --------------------------------------------------------------------------- | 1579 | // --------------------------------------------------------------------------- |
| @@ -1453,7 +2124,11 @@ static bool HandleLine(Miner* m, const std::string& rawLine, | |||
| 1453 | int foundHeight = gCurrentJob.height; | 2124 | int foundHeight = gCurrentJob.height; |
| 1454 | // b is the block we submitted, untouched by ProcessBlock (which | 2125 | // b is the block we submitted, untouched by ProcessBlock (which |
| 1455 | // took a copy), so its hash is what the payout must be keyed on. | 2126 | // took a copy), so its hash is what the payout must be keyed on. |
| 1456 | QueuePayouts(foundHeight, b.GetHash(), roundShareCount, roundShareTotal); | 2127 | // Passed as a uint256, not hex: the ledger key must stay a real |
| 2128 | // hash. The remaining arguments are upstream's round-proof data. | ||
| 2129 | QueuePayouts(foundHeight, b.GetHash(), b.vtx[0].GetHash().GetHex(), | ||
| 2130 | b.vtx[0].vout[0].nValue, m->address, | ||
| 2131 | roundShareCount, roundShareTotal); | ||
| 1457 | // Reset round shares | 2132 | // Reset round shares |
| 1458 | roundShareCount.clear(); | 2133 | roundShareCount.clear(); |
| 1459 | roundShareTotal = 0; | 2134 | roundShareTotal = 0; |
| @@ -1479,6 +2154,147 @@ static bool HandleLine(Miner* m, const std::string& rawLine, | |||
| 1479 | return true; | 2154 | return true; |
| 1480 | } | 2155 | } |
| 1481 | 2156 | ||
| 2157 | int RunPoolStratumSelfTest() | ||
| 2158 | { | ||
| 2159 | int nFail = 0; | ||
| 2160 | auto check = [&](bool ok, const char* msg) { | ||
| 2161 | fprintf(stdout, " %s %s\n", ok ? "ok" : "FAIL", msg); | ||
| 2162 | if (!ok) nFail++; | ||
| 2163 | }; | ||
| 2164 | |||
| 2165 | fprintf(stdout, "pool-stratum self-test\n"); | ||
| 2166 | |||
| 2167 | StratumJob savedJob = gCurrentJob; | ||
| 2168 | bool savedHaveJob = gHaveJob; | ||
| 2169 | |||
| 2170 | CTransaction coinbase; | ||
| 2171 | coinbase.vin.resize(1); | ||
| 2172 | coinbase.vin[0].prevout.SetNull(); | ||
| 2173 | coinbase.vin[0].scriptSig << 1 << (CBigNum)1; | ||
| 2174 | coinbase.vout.resize(1); | ||
| 2175 | coinbase.vout[0].nValue = 50 * COIN; | ||
| 2176 | |||
| 2177 | CBlock block; | ||
| 2178 | block.vtx.push_back(coinbase); | ||
| 2179 | block.nVersion = 1; | ||
| 2180 | block.nBits = 0x1f0fffff; | ||
| 2181 | block.nTime = 1; | ||
| 2182 | block.nNonce = 1; | ||
| 2183 | block.hashMerkleRoot = block.BuildMerkleTree(); | ||
| 2184 | |||
| 2185 | gCurrentJob.jobId = "selftest"; | ||
| 2186 | gCurrentJob.block = block; | ||
| 2187 | gCurrentJob.target = 0; // accepted share, not a block candidate | ||
| 2188 | gCurrentJob.height = 1; | ||
| 2189 | gHaveJob = true; | ||
| 2190 | |||
| 2191 | Miner miner(INVALID_SOCKET); | ||
| 2192 | miner.authorised = true; | ||
| 2193 | miner.address = "selftest-address"; | ||
| 2194 | miner.worker = "worker"; | ||
| 2195 | miner.shareTarget = ~uint256(0); // every RandomX hash is a valid share | ||
| 2196 | miner.difficulty = 0.001; | ||
| 2197 | miner.vardiffWindowStart = GetTime(); | ||
| 2198 | |||
| 2199 | std::map<std::string, uint64> roundShareCount; | ||
| 2200 | uint64 roundShareTotal = 0; | ||
| 2201 | int blocksFound = 0; | ||
| 2202 | |||
| 2203 | uint256 targetFromNotify; | ||
| 2204 | CBlock minerBlock = BuildMinerBlock(gCurrentJob, miner.extranonce2); | ||
| 2205 | json params = MakeNotifyParams(gCurrentJob, minerBlock, miner.shareTarget, true); | ||
| 2206 | std::vector<unsigned char> targetBytes = FromHex(params[3].get<std::string>()); | ||
| 2207 | if (targetBytes.size() == 32) | ||
| 2208 | memcpy(&targetFromNotify, &targetBytes[0], 32); | ||
| 2209 | check(targetFromNotify == miner.shareTarget, | ||
| 2210 | "mining.notify exposes the same share target the pool validates"); | ||
| 2211 | |||
| 2212 | json submit = { | ||
| 2213 | {"id", 7}, | ||
| 2214 | {"method", "mining.submit"}, | ||
| 2215 | {"params", json::array({miner.address, "selftest", ToHex(&minerBlock.nNonce, 4)})} | ||
| 2216 | }; | ||
| 2217 | HandleLine(&miner, submit.dump(), roundShareCount, roundShareTotal, blocksFound); | ||
| 2218 | |||
| 2219 | check(roundShareTotal == 1, "fake low-difficulty share increments the round total"); | ||
| 2220 | check(roundShareCount[miner.address] == 1, "fake share is credited to the submitting address"); | ||
| 2221 | check(miner.sessionShares == 1 && miner.roundShares == 1 && miner.totalShares == 1, | ||
| 2222 | "miner share counters advance together"); | ||
| 2223 | check(blocksFound == 0, "share below block target is not counted as a found block"); | ||
| 2224 | |||
| 2225 | json stale = { | ||
| 2226 | {"id", 8}, | ||
| 2227 | {"method", "mining.submit"}, | ||
| 2228 | {"params", json::array({miner.address, "stale", ToHex(&minerBlock.nNonce, 4)})} | ||
| 2229 | }; | ||
| 2230 | HandleLine(&miner, stale.dump(), roundShareCount, roundShareTotal, blocksFound); | ||
| 2231 | check(roundShareTotal == 1, "stale submit does not increment shares"); | ||
| 2232 | |||
| 2233 | std::vector<PoolRoundProof> savedRounds = gPoolRounds; | ||
| 2234 | std::string savedRoundsFile = strPoolRoundsFile; | ||
| 2235 | std::string proofPath = "pool_rounds_selftest.json"; | ||
| 2236 | remove(proofPath.c_str()); | ||
| 2237 | remove((proofPath + ".tmp").c_str()); | ||
| 2238 | gPoolRounds.clear(); | ||
| 2239 | strPoolRoundsFile = proofPath; | ||
| 2240 | |||
| 2241 | PoolRoundProof proof; | ||
| 2242 | proof.id = "selftest-round"; | ||
| 2243 | proof.blockHeight = 42; | ||
| 2244 | proof.matureAtHeight = 42 + COINBASE_MATURITY; | ||
| 2245 | proof.createdAt = GetTime(); | ||
| 2246 | proof.blockHash = block.GetHash().GetHex(); | ||
| 2247 | proof.coinbaseTxid = coinbase.GetHash().GetHex(); | ||
| 2248 | proof.foundBy = miner.address; | ||
| 2249 | proof.feePercent = 1.25; | ||
| 2250 | proof.totalShares = 3; | ||
| 2251 | proof.coinbaseValue = 50 * COIN; | ||
| 2252 | proof.subsidy = 50 * COIN; | ||
| 2253 | proof.minerReward = 49 * COIN; | ||
| 2254 | proof.operatorReward = 1 * COIN; | ||
| 2255 | proof.shares[miner.address] = 3; | ||
| 2256 | proof.payouts[miner.address] = 49 * COIN; | ||
| 2257 | RecordPoolRound(proof); | ||
| 2258 | MarkPoolRoundPaid(proof.id, miner.address, "selftest-txid"); | ||
| 2259 | |||
| 2260 | bool proofOk = false; | ||
| 2261 | FILE* pf = fopen(proofPath.c_str(), "r"); | ||
| 2262 | if (pf) { | ||
| 2263 | fseek(pf, 0, SEEK_END); | ||
| 2264 | long sz = ftell(pf); | ||
| 2265 | fseek(pf, 0, SEEK_SET); | ||
| 2266 | std::string s(sz, '\0'); | ||
| 2267 | fread(&s[0], 1, sz, pf); | ||
| 2268 | fclose(pf); | ||
| 2269 | try { | ||
| 2270 | json root = json::parse(s); | ||
| 2271 | proofOk = | ||
| 2272 | root.value("schema", "") == "bitflash-pool-rounds-1" && | ||
| 2273 | root["rounds"].is_array() && | ||
| 2274 | root["rounds"].size() == 1 && | ||
| 2275 | root["rounds"][0].value("id", "") == proof.id && | ||
| 2276 | root["rounds"][0]["shares"].value(miner.address, (uint64)0) == 3 && | ||
| 2277 | root["rounds"][0]["payouts"][miner.address].value("txid", "") == "selftest-txid"; | ||
| 2278 | } catch (...) { | ||
| 2279 | proofOk = false; | ||
| 2280 | } | ||
| 2281 | } | ||
| 2282 | check(proofOk, "pool round proof ledger is written and paid txid is recorded"); | ||
| 2283 | |||
| 2284 | remove(proofPath.c_str()); | ||
| 2285 | remove((proofPath + ".tmp").c_str()); | ||
| 2286 | strPoolRoundsFile = savedRoundsFile; | ||
| 2287 | gPoolRounds = savedRounds; | ||
| 2288 | |||
| 2289 | gCurrentJob = savedJob; | ||
| 2290 | gHaveJob = savedHaveJob; | ||
| 2291 | |||
| 2292 | fprintf(stdout, "%s (%d failure%s)\n", nFail == 0 ? "ALL TESTS PASSED" : "TESTS FAILED", | ||
| 2293 | nFail, nFail == 1 ? "" : "s"); | ||
| 2294 | fflush(stdout); | ||
| 2295 | return nFail == 0 ? 0 : 1; | ||
| 2296 | } | ||
| 2297 | |||
| 1482 | // --------------------------------------------------------------------------- | 2298 | // --------------------------------------------------------------------------- |
| 1483 | // Stats refresh -- called every 2s from event loop | 2299 | // Stats refresh -- called every 2s from event loop |
| 1484 | // --------------------------------------------------------------------------- | 2300 | // --------------------------------------------------------------------------- |
| @@ -1512,16 +2328,27 @@ static void RefreshStats(int blocksFoundThisSession, uint64 roundShareTotal) | |||
| 1512 | : a.lastSeen > b.lastSeen; | 2328 | : a.lastSeen > b.lastSeen; |
| 1513 | }); | 2329 | }); |
| 1514 | 2330 | ||
| 1515 | // gPendingPayouts is event-loop-only data (no lock needed here since | ||
| 1516 | // RefreshStats is always called from the event loop thread). | ||
| 1517 | gPayoutViews.clear(); | 2331 | gPayoutViews.clear(); |
| 2332 | std::lock_guard<std::mutex> ledger(gPayoutLedgerMutex); | ||
| 1518 | for (const PendingPayout& pp : gPendingPayouts) { | 2333 | for (const PendingPayout& pp : gPendingPayouts) { |
| 1519 | PendingPayoutView pv; | 2334 | PendingPayoutView pv; |
| 1520 | pv.matureAtHeight = pp.matureAtHeight; | 2335 | pv.matureAtHeight = pp.matureAtHeight; |
| 1521 | pv.recipients = (int)pp.amounts.size(); | 2336 | pv.recipients = 0; |
| 1522 | pv.totalAmount = 0; | 2337 | pv.totalAmount = 0; |
| 1523 | for (auto& kv : pp.amounts) pv.totalAmount += kv.second; | 2338 | for (auto& kv : pp.amounts) { |
| 1524 | gPayoutViews.push_back(pv); | 2339 | if (pp.paidTxids.count(kv.first)) |
| 2340 | continue; | ||
| 2341 | pv.recipients++; | ||
| 2342 | pv.totalAmount += kv.second; | ||
| 2343 | } | ||
| 2344 | if (pv.recipients > 0) | ||
| 2345 | gPayoutViews.push_back(pv); | ||
| 2346 | } | ||
| 2347 | |||
| 2348 | int64 now = GetTime(); | ||
| 2349 | if (now - gLastPoolStatusWrite >= 10) { | ||
| 2350 | WritePoolStatusJsonLocked(); | ||
| 2351 | gLastPoolStatusWrite = now; | ||
| 1525 | } | 2352 | } |
| 1526 | } | 2353 | } |
| 1527 | 2354 | ||
| @@ -1901,10 +2728,11 @@ void ThreadRPCServer(void*) | |||
| 1901 | } | 2728 | } |
| 1902 | 2729 | ||
| 1903 | LoadPendingPayouts(); | 2730 | LoadPendingPayouts(); |
| 2731 | LoadPoolRounds(); | ||
| 1904 | LoadPaying(); | 2732 | LoadPaying(); |
| 1905 | // Must run before the payout thread starts, so nothing is resent that the | 2733 | // Must run before the payout thread starts, so nothing is resent that the |
| 1906 | // wallet already shows as sent, and nothing is owed twice because we died | 2734 | // ledger already shows as sent, and every restored in-flight record is |
| 1907 | // between recording the payments and rewriting the ledger. | 2735 | // armed for its one wallet probe before it is allowed to send. |
| 1908 | ReconcilePayouts(); | 2736 | ReconcilePayouts(); |
| 1909 | 2737 | ||
| 1910 | _beginthread(AcceptLoopFn, 0, NULL); | 2738 | _beginthread(AcceptLoopFn, 0, NULL); |