upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/src/Makefile
diff options
context:
space:
mode:
authorbitflash-maintainer <noreply@bitflash.invalid>2026-08-23 05:21:31 +0000
committerbitflash-maintainer <noreply@bitflash.invalid>2026-08-23 05:21:31 +0000
commita9b4f8940907c2bf51e431172b628de89734a908 (patch)
treeff0a0b19d7d7754b809e35c6eae43dae7a32e1e4 /src/Makefile
parent6d4c3f153cf44cefead39b21191dfb33491bfa5a (diff)
parentbc3de301d9ed0e5fb5e2034be2e066d77cf23804 (diff)
Merge upstream v1.2.18 into the fork
Merges upstream tag v1.2.18 (bc3de30) into our 24 commits. The merge base is a259215, upstream's own commit, whose src/ tree is byte-identical to 68de575 -- our commits were replayed onto a259215 first so that git had a real base to work from. That produced 28 conflict hunks in 9 files instead of the 21 conflict events / 51 hunks a direct cherry-pick onto the tag would have forced, most of which were artifacts of lifting our commits away from their own predecessors. No consensus change. main.cpp's GetBlockValue, GetNextWorkRequired, ConnectInputs, ConnectBlock, CheckBlock, AcceptBlock and AcceptTransaction are untouched by this merge; script.cpp, serialize.h and randomx_pow.h are identical on both sides. The frozen-chain oracle reproduces the pinned UTXO root 116a2b6b... byte for byte, and the merged binary loads the same 12,586-block chain to the same tip aac622f0a3b2f8... with all blocks re-verified and none rejected. Which side won, per file and per hunk. src/rpc.cpp -- 15 hunks. 1. RebuildJob transaction selection. THEIRS for the multi-pass loop and the nFees fix, OURS for the cap. Their vfAlreadyAdded/fFoundSomething loop lands dependent transactions that our single pass dropped deterministically, and their nFees is accumulated across the scan instead of being redeclared inside it -- ours passed a hardcoded 0 to GetBlockValue, so every pool block built here discarded its fees. But their bound is MAX_SIZE/2 = 16 MB, and MAX_BLOCK_SIZE is 1 MB, so their template can exceed what their own CheckBlock accepts. Cap kept at MAX_BLOCK_SIZE/2, with our inner break so a single pass cannot overshoot by more than one transaction. Our RebuildJob sync gating and deaf-node detection are kept; upstream has none. 2. struct PendingPayout. OURS for the key, THEIRS for paidTxids. Their id is the string "block-<height>", which is height keying -- the exact bug 44849e6 fixed after a 280-block reorg left ~14.4k BTF owed against blocks that no longer existed. Height keying also lets the crash-recovery probe match the previous round's transaction after a reorg and silently skip a miner, so their id is dropped entirely and blockHash stays the key. paidTxids is taken because it is what makes an entry survivable. Their nextAttempt and our retryAfter are the same concept under two names; ours is kept and theirs dropped. matureAtHeight gained an explicit = 0, which neither side had in our style of initialiser. 3./4. Ledger writer and loader. Mechanical follow-ons of 2. The loader keeps ours and takes their tolerance for an entry with no usable height, guarded so it cannot fire on a well-formed entry. One redundancy was removed rather than merged: their unguarded `if (obj.contains("blockHeight")) get<int>()` sat two lines above our is_number-guarded version, and an unguarded get<int>() on a malformed value throws out of the whole loop, losing every entry in the file. Verified against the live production pending_payouts.json (2 entries, keys amounts/blockHash/blockHeight/matureAtHeight): loads both. A file in the new shape, with retryAfter and paidTxids, also loads. 5. Insertion point after the loader. BOTH. Our PayingRecord/gPaying in-flight ledger and their PoolRoundProof/PoolRoundToJson are disjoint features that landed at the same offset. Both kept, in full. 6. Writer tail. THEIRS, copied into our writer. Checking fclose() matters here because fclose is where the buffered write actually reaches the OS: renaming a short .tmp over a good file is precisely the truncation the atomic write exists to prevent. The Windows remove()-before-rename() is required because rename() there will not replace an existing file. 7. Loader bodies. BOTH -- LoadPaying and LoadPoolRounds are different functions that happened to share a common tail. 8. Function block, 111 vs 36 lines. BOTH, with one deliberate deletion. PayoutBlockStillOnChain, PayoutComment, RecordPoolRound, MarkPoolRoundPaid all kept. WalletAlreadyPaid is DELETED and ReconcilePayouts rewritten, because our reconciliation and their FindRecordedPoolPayout are two answers to the same crash-recovery question and running both would give the same debt two authorities. Their wallet probe wins: it matches structured metadata rather than a comment string, so it can tell two rounds paying the same address the same amount apart. ReconcilePayouts keeps only the job that is still real -- dropping in-flight records the ledger already shows paid, and arming each restored record for its one wallet probe. It no longer touches gPendingPayouts at all; under the new model dropping ledger entries there would reintroduce the loss this merge adopts their model to prevent. Their gPayoutExecMutex/gPayoutThreadRunning re-declarations were dropped as duplicates of ours, and gPayoutExecQueue was not taken: gPaying is the exec queue in this tree, at finer granularity. 9. QueuePayouts signature. THEIRS for the round-proof arguments, OURS for the type of blockHash. It stays a uint256 and is converted to hex only where the round proof needs text -- a ledger key that has been through ToString() is a key that can be compared wrong. 10. Maturity policy. OURS. matureAtHeight = blockHeight + COINBASE_SPENDABLE_DEPTH (120), not + COINBASE_MATURITY (100). At depth 100 the block's own coinbase is still 20 blocks short of spendable, so the payment must come from unrelated mature coin and the pool runs a permanent float. Our own history contradicts itself here -- d6972a0 argued for COINBASE_MATURITY citing "96% of 934 payments", 44849e6 reversed it on the grounds that the measurement was taken while the node was alone on a private fork paying itself. HEAD's position is kept. This is a considered disagreement with upstream, not an oversight. The live production ledger confirms it: both entries have matureAtHeight - blockHeight = 120. 11. Comment above FlushMaturePayouts. Rewritten to describe the merged design. 12. FlushMaturePayouts, the central difference. THEIRS for the model. The ledger entry now survives on disk until every recipient has a txid, instead of being erased as soon as its payments were handed off -- the old ordering left a window where the debt was described by nothing at all, and a container restart on 2026-07-30 silently dropped ~1,156 BTF through it. Our gPaying is kept on top as the in-flight working set, because it carries the per-recipient retry and deferral state a per-round nextAttempt cannot express. It is explicitly a cache derived from the ledger, not a second record of debt: if it were lost the next flush would rebuild it. 13. PayoutThreadFn, 79 vs 70 lines. OURS for the loop and the reorg re-check, THEIRS for the per-recipient dedupe. The re-check on every attempt is load-bearing and upstream has no equivalent -- it decides once, at maturity, and a payment can sit through hours of retries during which the reorg that kills its funding block can land. Their paidTxids check is taken, against the ledger rather than against the batch. Our deferral policy is kept: waiting for coin is not a failure and must not consume the retry budget. Two fixes upstream does not have are folded in here rather than deferred to a follow-up commit: - FindRecordedPoolPayout runs once per record, on its first attempt, not on every attempt. It is a linear scan of mapWallet under cs_mapWallet, per recipient; upstream repeats it for every retry, which with 2000 rotating payout addresses and a budget of 1440 retries is a full wallet scan per address per minute for a day. The question it answers can only change across a restart, and every restart re-arms the flag. - The ledger file write no longer happens under gPayoutLedgerMutex in the per-recipient loop. SavePendingPayoutsLocked was split into a serialise step that needs the lock and a write step that does not, so the mutex is held only long enough to read the structure. The orphan scan in FlushMaturePayouts was likewise restructured so the ledger mutex is not held across PayoutBlockStillOnChain's cs_main acquisition. One case the merged model created and neither side handled: under survives-until-fully-paid, our GIVING UP path dropped the in-flight record but left the recipient owed in the ledger forever, so every later flush would explode it again -- a forfeit that silently becomes an infinite retry. ForfeitPayoutRecipient writes the amount off so the entry can retire. 14. QueuePayouts call site. Follow-on of 9. 15. ThreadRPCServer startup. BOTH. LoadPoolRounds added; LoadPaying and ReconcilePayouts stay, and stay before the payout thread starts. Crash safety, stated as the property that must hold: after kill -9 at any point, restart pays each recipient exactly once. Before SendMoney, the ledger has no txid and the payment is retried. Between SendMoney and the ledger write, the wallet holds the transaction and the first-attempt probe finds it. After the ledger write, paidTxids suppresses it. If the in-flight file is lost entirely, the ledger rebuilds it and the probe still covers the middle window. src/nostr.cpp -- 3 hunks. THEIRS as the base, with two things restored from ours. Theirs loads the Windows ROOT certificate store, which ours lacks and genuinely needs, since OpenSSL's default verify paths are usually empty there and our version would fail closed. But the plan's claim that theirs is a superset of our 06bceb4 is FALSE: upstream's nostr.cpp contains no min_proto_version, no TLS1_2, no SSL_OP_NO_*, and no set_options anywhere, so taking it wholesale would have removed the TLS-version floor outright. Restored. Second, their SSL_get_verify_result check sits after a successful handshake, where with SSL_VERIFY_PEER set it is nearly dead code -- a bad certificate aborts inside SSL_connect. The reporting belongs in the failure branch, which is where the merge put it; their post-handshake duplicate was dropped. Our relay-list and socket-pooling hunks auto-merged and were verified intact. src/net.cpp -- 1 hunk. THEIRS. Their guard, hSocket == INVALID_SOCKET || pnode->hSocket != hSocket, subsumes ours and additionally catches descriptor recycling, which ours did not; our version also referenced a pnode that only existed in the select() loop. Our other db34011 hunk, nulling pnode->hSocket in the poll-failure branch, auto-merged and was NOT re-applied -- plan section 7 Batch D says to re-apply it, and that instruction is stale. Two pieces of residue the merge left behind were cleaned up: a comment still describing select() semantics, and a pnode->hSocket != INVALID_SOCKET guard made redundant by a continue two lines above it. src/main.cpp -- 2 hunks. h1 BOTH, pure add/add adjacency between our ParticipantHashThreadCount and their SecureClear forward declarations. h2 OURS, and their handler discarded: 44004e3 no-ops checkorder/submitorder but keeps market.cpp and market.h, whereas 11766be deletes the marketplace outright including CRequestTracker, subscribe/publish and mapProducts. Ours is strictly more. src/main_gui.cpp -- 2 hunks. BOTH, additive on both sides. Our /payoutaddr, /payoutfile and /hashthreads alongside their /stratumbridge and /stratumbridgeport. src/btfrv.cpp -- 1 hunk, not predicted by the plan. False adjacency: our INVALID_SOCKET check and their #endif landed at the same offset. Ordering is load-bearing -- our check goes inside the #else arm, before the #endif, or the non-proxy build silently loses it. The BITFLASH_NODE_BUILD arm already returns early, so both paths now have the check. Makefile -- 2 hunks plus a hazard with no conflict marker. THEIRS for both dependency-fetch recipes: checkout --detach FETCH_HEAD followed by test "$(git rev-parse HEAD)" = "$(..._COMMIT)" asserts the pin actually landed, where ours only checked it out. Everything of ours that they lack is kept: the appimagetool 1.9.1 release pin with a sha256 check, where upstream fetches the mutable "continuous" tag and executes it undigested, and the move of both the tool and the AppDir out of /tmp, where any local user could pre-place an executable and have it run as whoever runs make. The hazard: both sides added a pin block in a different place, so git merged both, outside any conflict region. SECP256K1_COMMIT and RANDOMX_COMMIT ended up assigned twice, near L16 and near L44, and GNU Make takes the last := -- a naive resolution would have silently built against our old secp256k1 pin while appearing to adopt upstream's. The duplicate block was deleted by hand. Each of the four variables is now assigned exactly once, and make -n resolves SECP256K1_COMMIT to 7fecac74 with 6e2c8bc4 appearing nowhere. Also applied here, from plan phase 4: SECP256K1_COMMIT moved to upstream's 7fecac74aed8e1fd9078380d67dd04663705c989 so signature validation matches the binaries the rest of the network runs. secp256k1 was rebuilt from scratch against the new pin on the build host and the checked-out dependency commit verified. RANDOMX_COMMIT already matched at 1e9d4b2d and is unchanged. src/Makefile, src/makefile.mingw -- 1 hunk each, both the HEADERS list. Ours drops market.h, theirs adds proxy.h; the resolution does both. Taking theirs wholesale would have left the Makefiles referencing a header git had already deleted in our favour, silently, without a conflict. Files the plan predicted would conflict and did not: net.h, main.h and db.cpp auto-merged. Verified rather than assumed. net.h carries both our BTF_MSG_WAIT_MAX_SECS work, including the constructor fix from 2d84982 that starts nMsgWaitSince and nMsgWaitSize at 0, and their comment rewrite with extern int nPeersWatched. The dead members that fix removed are gone from every file. Deviations from MERGE-1.2.18-PLAN.md. - Section 6's mechanism was not used. Replayed onto the real merge base and merged, rather than cherry-picking onto the tag. Same result, 28 hunks instead of 51. - Section 2 calls 09ff575 a superset of our 06bceb4. It is not; upstream has no TLS-version floor at all. Ours restored on top of theirs. - Section 7 Batch D says to re-apply our db34011 line into their poll-failure branch. It auto-merged; re-applying would have duplicated it. - Section 2 says do not take the stratum bridge. It is taken, and left off. fStratumBridge defaults false in main.cpp, the thread is only spawned when /stratumbridge is passed, and the listener binds 127.0.0.1 only. Excising it would cost real divergence and future conflict for no gain while it is disabled. Its defects -- select()/FD_SET on a node that can hold more than 1024 descriptors, and an uncapped thread per connection -- are real only when it is enabled and are being reported upstream separately. - Section 7's staged batches were not run as separate gated commits. The conflict surface is interleaved within single functions, chiefly the payout ledger, and splitting it would have meant committing states that do not build. The merge is one commit, as briefed. - Batch F's two fixes are folded into hunk 13 rather than deferred, because they change the same lines the resolution rewrites. - The plan did not anticipate the btfrv.cpp conflict, the duplicate pin block, the market.h trap in the two Makefiles, or the give-up/infinite-retry interaction created by adopting upstream's ledger model.
Diffstat (limited to 'src/Makefile')
-rw-r--r--src/Makefile60
1 files changed, 56 insertions, 4 deletions
diff --git a/src/Makefile b/src/Makefile
index 171afe0..1166daf 100644
--- a/src/Makefile
+++ b/src/Makefile
@@ -2,6 +2,7 @@
2# Called by: make linux (root Makefile) 2# Called by: make linux (root Makefile)
3 3
4CXX ?= g++ 4CXX ?= g++
5FUZZ_CXX ?= clang++
5 6
6SECP_INC ?= /usr/local/include 7SECP_INC ?= /usr/local/include
7SECP_LIB ?= /usr/local/lib 8SECP_LIB ?= /usr/local/lib
@@ -29,10 +30,13 @@ WARNFLAGS = -Wformat -Werror=format \
29 -Wno-narrowing -Wno-unused-result 30 -Wno-narrowing -Wno-unused-result
30 31
31CXXFLAGS = -std=gnu++14 -O2 $(WARNFLAGS) -fpermissive -pthread \ 32CXXFLAGS = -std=gnu++14 -O2 $(WARNFLAGS) -fpermissive -pthread \
32 -DSECP256K1_STATIC \ 33 -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD \
33 -I$(SECP_INC) -I$(RANDOMX_DIR)/src \ 34 -I$(SECP_INC) -I$(RANDOMX_DIR)/src \
34 -I/usr/include/nlohmann \ 35 -I/usr/include/nlohmann \
35 -I$(IMGUI_DIR) 36 -I$(IMGUI_DIR)
37FUZZ_CXXFLAGS = $(filter-out -O2,$(CXXFLAGS)) -O1 -g -fno-omit-frame-pointer \
38 -fsanitize=fuzzer-no-link,address,undefined
39FUZZ_LDFLAGS = -fsanitize=fuzzer,address,undefined
36 40
37LIBS = -L$(SECP_LIB) -lsecp256k1 \ 41LIBS = -L$(SECP_LIB) -lsecp256k1 \
38 -L$(RANDOMX_DIR)/build -lrandomx \ 42 -L$(RANDOMX_DIR)/build -lrandomx \
@@ -49,7 +53,7 @@ LIBS = -L$(SECP_LIB) -lsecp256k1 \
49HEADERS = headers_core.h headers.h util.h main.h serialize.h uint256.h \ 53HEADERS = headers_core.h headers.h util.h main.h serialize.h uint256.h \
50 key.h bignum.h script.h db.h base58.h net.h nostr.h \ 54 key.h bignum.h script.h db.h base58.h net.h nostr.h \
51 randomx_pow.h btfaddr.h btfchan.h btfrv.h btftunnel.h \ 55 randomx_pow.h btfaddr.h btfchan.h btfrv.h btftunnel.h \
52 selftest.h walletcmd.h compat.h sockcount.h font_roboto.h 56 selftest.h walletcmd.h compat.h sockcount.h proxy.h font_roboto.h
53 57
54IMGUI_OBJS = obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o \ 58IMGUI_OBJS = obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o \
55 obj/imgui_widgets.o obj/imgui_impl_glfw.o obj/imgui_impl_opengl3.o 59 obj/imgui_widgets.o obj/imgui_impl_glfw.o obj/imgui_impl_opengl3.o
@@ -57,7 +61,7 @@ IMGUI_OBJS = obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o \
57NODE_OBJS = obj/util.o obj/script.o obj/db.o obj/net.o obj/main.o \ 61NODE_OBJS = obj/util.o obj/script.o obj/db.o obj/net.o obj/main.o \
58 obj/sha.o obj/nostr.o obj/randomx_pow.o \ 62 obj/sha.o obj/nostr.o obj/randomx_pow.o \
59 obj/btfaddr.o obj/btfchan.o obj/btfrv.o obj/btftunnel.o obj/rpc.o \ 63 obj/btfaddr.o obj/btfchan.o obj/btfrv.o obj/btftunnel.o obj/rpc.o \
60 obj/selftest.o obj/bip32.o obj/walletcmd.o 64 obj/selftest.o obj/bip32.o obj/walletcmd.o obj/proxy.o
61 65
62GUI_OBJS = obj/gui.o obj/main_gui.o 66GUI_OBJS = obj/gui.o obj/main_gui.o
63 67
@@ -117,6 +121,10 @@ NODE_LIBS = -L$(SECP_LIB) -lsecp256k1 \
117NODE_ONLY_OBJS = $(patsubst obj/%,obj-node/%,$(NODE_OBJS)) \ 121NODE_ONLY_OBJS = $(patsubst obj/%,obj-node/%,$(NODE_OBJS)) \
118 obj-node/main_gui.o obj-node/headless.o 122 obj-node/main_gui.o obj-node/headless.o
119 123
124FUZZ_BASE_OBJS = $(patsubst obj/%,obj-fuzz/%,$(NODE_OBJS))
125FUZZ_NET_MESSAGE_OBJS = $(FUZZ_BASE_OBJS) obj-fuzz/fuzz_net_message.o
126FUZZ_SCRIPT_OBJS = $(FUZZ_BASE_OBJS) obj-fuzz/fuzz_script.o
127
120obj-node/%.o: %.cpp $(HEADERS) 128obj-node/%.o: %.cpp $(HEADERS)
121 @mkdir -p obj-node 129 @mkdir -p obj-node
122 $(CXX) -c $(CXXFLAGS) -DBITFLASH_NO_GUI -o $@ $< 130 $(CXX) -c $(CXXFLAGS) -DBITFLASH_NO_GUI -o $@ $<
@@ -129,8 +137,47 @@ bitflash-node: $(NODE_ONLY_OBJS)
129 $(CXX) $(CXXFLAGS) -o $@ $^ $(NODE_LIBS) 137 $(CXX) $(CXXFLAGS) -o $@ $^ $(NODE_LIBS)
130 @echo "Built: bitflash-node (headless, no libGL needed)" 138 @echo "Built: bitflash-node (headless, no libGL needed)"
131 139
140obj-fuzz/%.o: %.cpp $(HEADERS)
141 @mkdir -p obj-fuzz
142 $(FUZZ_CXX) -c $(FUZZ_CXXFLAGS) -DBITFLASH_FUZZ -o $@ $<
143
144obj-fuzz/sha.o: sha.cpp sha.h
145 @mkdir -p obj-fuzz
146 $(FUZZ_CXX) -c $(FUZZ_CXXFLAGS) -DBITFLASH_FUZZ -O1 -o $@ $<
147
148obj-fuzz/fuzz_net_message.o: fuzz_net_message.cpp $(HEADERS)
149 @mkdir -p obj-fuzz
150 $(FUZZ_CXX) -c $(FUZZ_CXXFLAGS) -DBITFLASH_FUZZ -o $@ $<
151
152obj-fuzz/fuzz_script.o: fuzz_script.cpp $(HEADERS)
153 @mkdir -p obj-fuzz
154 $(FUZZ_CXX) -c $(FUZZ_CXXFLAGS) -DBITFLASH_FUZZ -o $@ $<
155
156fuzz-net-message: $(FUZZ_NET_MESSAGE_OBJS)
157 $(FUZZ_CXX) $(filter-out -fsanitize=fuzzer-no-link%,$(FUZZ_CXXFLAGS)) \
158 $(FUZZ_LDFLAGS) -o $@ $^ $(NODE_LIBS)
159 @echo "Built: fuzz-net-message"
160
161fuzz-script: $(FUZZ_SCRIPT_OBJS)
162 $(FUZZ_CXX) $(filter-out -fsanitize=fuzzer-no-link%,$(FUZZ_CXXFLAGS)) \
163 $(FUZZ_LDFLAGS) -o $@ $^ $(NODE_LIBS)
164 @echo "Built: fuzz-script"
165
166fuzz-net-message-smoke: fuzz-net-message
167 @mkdir -p ../work/fuzz-corpus/net-message
168 @printf '\277\040\134\375ping\000\000\000\000\000\000\000\000\000\000\000\000' > ../work/fuzz-corpus/net-message/ping-header
169 @printf '\277\040\134\375version\000\000\000\000\000\000\000\000\000' > ../work/fuzz-corpus/net-message/version-header
170 ./fuzz-net-message -runs=128 ../work/fuzz-corpus/net-message
171
172fuzz-script-smoke: fuzz-script
173 @mkdir -p ../work/fuzz-corpus/script
174 @printf '\000\121' > ../work/fuzz-corpus/script/op-true
175 @printf '\001\166\251\024\001\002\003\004\005\006\007\010\011\012\013\014\015\016\017\020\021\022\023\024\210\254' > ../work/fuzz-corpus/script/p2pkh-shape
176 @printf '\000\115\377\377' > ../work/fuzz-corpus/script/truncated-pushdata2
177 ./fuzz-script -runs=128 ../work/fuzz-corpus/script
178
132clean: 179clean:
133 rm -rf obj obj-node bitflash bitflash-node test_bip32 180 rm -rf obj obj-node obj-fuzz bitflash bitflash-node fuzz-net-message fuzz-script test_bip32
134 181
135test_bip32: test_bip32.cpp bip32.cpp bip32.h bip39_english.h 182test_bip32: test_bip32.cpp bip32.cpp bip32.h bip39_english.h
136 $(CXX) $(CXXFLAGS) -o $@ test_bip32.cpp bip32.cpp -lssl -lcrypto 183 $(CXX) $(CXXFLAGS) -o $@ test_bip32.cpp bip32.cpp -lssl -lcrypto
@@ -144,3 +191,8 @@ tests: test_bip32 bitflash-node
144 ./bitflash-node -selftest=wallet-encrypt 191 ./bitflash-node -selftest=wallet-encrypt
145 ./bitflash-node -selftest=net-message 192 ./bitflash-node -selftest=net-message
146 ./bitflash-node -selftest=consensus-limits 193 ./bitflash-node -selftest=consensus-limits
194 ./bitflash-node -selftest=pool-stratum
195 ./bitflash-node -selftest=parse-money
196 ./bitflash-node -selftest=socks5-proxy
197
198.PHONY: tests fuzz-net-message-smoke fuzz-script-smoke