diff options
| author | bitflash-maintainer <noreply@bitflash.invalid> | 2026-08-23 05:21:31 +0000 |
|---|---|---|
| committer | bitflash-maintainer <noreply@bitflash.invalid> | 2026-08-23 05:21:31 +0000 |
| commit | a9b4f8940907c2bf51e431172b628de89734a908 (patch) | |
| tree | ff0a0b19d7d7754b809e35c6eae43dae7a32e1e4 | |
| parent | 6d4c3f153cf44cefead39b21191dfb33491bfa5a (diff) | |
| parent | bc3de301d9ed0e5fb5e2034be2e066d77cf23804 (diff) | |
Merge upstream v1.2.18 into the fork
Merges upstream tag v1.2.18 (bc3de30) into our 24 commits. The merge base is
a259215, upstream's own commit, whose src/ tree is byte-identical to 68de575 --
our commits were replayed onto a259215 first so that git had a real base to work
from. That produced 28 conflict hunks in 9 files instead of the 21 conflict
events / 51 hunks a direct cherry-pick onto the tag would have forced, most of
which were artifacts of lifting our commits away from their own predecessors.
No consensus change. main.cpp's GetBlockValue, GetNextWorkRequired,
ConnectInputs, ConnectBlock, CheckBlock, AcceptBlock and AcceptTransaction are
untouched by this merge; script.cpp, serialize.h and randomx_pow.h are
identical on both sides. The frozen-chain oracle reproduces the pinned UTXO root
116a2b6b... byte for byte, and the merged binary loads the same 12,586-block
chain to the same tip aac622f0a3b2f8... with all blocks re-verified and none
rejected.
Which side won, per file and per hunk.
src/rpc.cpp -- 15 hunks.
1. RebuildJob transaction selection. THEIRS for the multi-pass loop and the
nFees fix, OURS for the cap. Their vfAlreadyAdded/fFoundSomething loop lands
dependent transactions that our single pass dropped deterministically, and
their nFees is accumulated across the scan instead of being redeclared inside
it -- ours passed a hardcoded 0 to GetBlockValue, so every pool block built
here discarded its fees. But their bound is MAX_SIZE/2 = 16 MB, and
MAX_BLOCK_SIZE is 1 MB, so their template can exceed what their own
CheckBlock accepts. Cap kept at MAX_BLOCK_SIZE/2, with our inner break so a
single pass cannot overshoot by more than one transaction. Our RebuildJob
sync gating and deaf-node detection are kept; upstream has none.
2. struct PendingPayout. OURS for the key, THEIRS for paidTxids. Their id is
the string "block-<height>", which is height keying -- the exact bug 44849e6
fixed after a 280-block reorg left ~14.4k BTF owed against blocks that no
longer existed. Height keying also lets the crash-recovery probe match the
previous round's transaction after a reorg and silently skip a miner, so
their id is dropped entirely and blockHash stays the key. paidTxids is taken
because it is what makes an entry survivable. Their nextAttempt and our
retryAfter are the same concept under two names; ours is kept and theirs
dropped. matureAtHeight gained an explicit = 0, which neither side had in
our style of initialiser.
3./4. Ledger writer and loader. Mechanical follow-ons of 2. The loader keeps
ours and takes their tolerance for an entry with no usable height, guarded so
it cannot fire on a well-formed entry. One redundancy was removed rather than
merged: their unguarded `if (obj.contains("blockHeight")) get<int>()` sat two
lines above our is_number-guarded version, and an unguarded get<int>() on a
malformed value throws out of the whole loop, losing every entry in the file.
Verified against the live production pending_payouts.json (2 entries, keys
amounts/blockHash/blockHeight/matureAtHeight): loads both. A file in the new
shape, with retryAfter and paidTxids, also loads.
5. Insertion point after the loader. BOTH. Our PayingRecord/gPaying in-flight
ledger and their PoolRoundProof/PoolRoundToJson are disjoint features that
landed at the same offset. Both kept, in full.
6. Writer tail. THEIRS, copied into our writer. Checking fclose() matters here
because fclose is where the buffered write actually reaches the OS: renaming
a short .tmp over a good file is precisely the truncation the atomic write
exists to prevent. The Windows remove()-before-rename() is required because
rename() there will not replace an existing file.
7. Loader bodies. BOTH -- LoadPaying and LoadPoolRounds are different functions
that happened to share a common tail.
8. Function block, 111 vs 36 lines. BOTH, with one deliberate deletion.
PayoutBlockStillOnChain, PayoutComment, RecordPoolRound, MarkPoolRoundPaid
all kept. WalletAlreadyPaid is DELETED and ReconcilePayouts rewritten,
because our reconciliation and their FindRecordedPoolPayout are two answers
to the same crash-recovery question and running both would give the same debt
two authorities. Their wallet probe wins: it matches structured metadata
rather than a comment string, so it can tell two rounds paying the same
address the same amount apart. ReconcilePayouts keeps only the job that is
still real -- dropping in-flight records the ledger already shows paid, and
arming each restored record for its one wallet probe. It no longer touches
gPendingPayouts at all; under the new model dropping ledger entries there
would reintroduce the loss this merge adopts their model to prevent. Their
gPayoutExecMutex/gPayoutThreadRunning re-declarations were dropped as
duplicates of ours, and gPayoutExecQueue was not taken: gPaying is the exec
queue in this tree, at finer granularity.
9. QueuePayouts signature. THEIRS for the round-proof arguments, OURS for the
type of blockHash. It stays a uint256 and is converted to hex only where the
round proof needs text -- a ledger key that has been through ToString() is a
key that can be compared wrong.
10. Maturity policy. OURS. matureAtHeight = blockHeight +
COINBASE_SPENDABLE_DEPTH (120), not + COINBASE_MATURITY (100). At depth 100
the block's own coinbase is still 20 blocks short of spendable, so the
payment must come from unrelated mature coin and the pool runs a permanent
float. Our own history contradicts itself here -- d6972a0 argued for
COINBASE_MATURITY citing "96% of 934 payments", 44849e6 reversed it on the
grounds that the measurement was taken while the node was alone on a private
fork paying itself. HEAD's position is kept. This is a considered
disagreement with upstream, not an oversight. The live production ledger
confirms it: both entries have matureAtHeight - blockHeight = 120.
11. Comment above FlushMaturePayouts. Rewritten to describe the merged design.
12. FlushMaturePayouts, the central difference. THEIRS for the model. The ledger
entry now survives on disk until every recipient has a txid, instead of
being erased as soon as its payments were handed off -- the old ordering
left a window where the debt was described by nothing at all, and a
container restart on 2026-07-30 silently dropped ~1,156 BTF through it. Our
gPaying is kept on top as the in-flight working set, because it carries the
per-recipient retry and deferral state a per-round nextAttempt cannot
express. It is explicitly a cache derived from the ledger, not a second
record of debt: if it were lost the next flush would rebuild it.
13. PayoutThreadFn, 79 vs 70 lines. OURS for the loop and the reorg re-check,
THEIRS for the per-recipient dedupe. The re-check on every attempt is
load-bearing and upstream has no equivalent -- it decides once, at maturity,
and a payment can sit through hours of retries during which the reorg that
kills its funding block can land. Their paidTxids check is taken, against
the ledger rather than against the batch. Our deferral policy is kept:
waiting for coin is not a failure and must not consume the retry budget.
Two fixes upstream does not have are folded in here rather than deferred to
a follow-up commit:
- FindRecordedPoolPayout runs once per record, on its first attempt, not on
every attempt. It is a linear scan of mapWallet under cs_mapWallet, per
recipient; upstream repeats it for every retry, which with 2000 rotating
payout addresses and a budget of 1440 retries is a full wallet scan per
address per minute for a day. The question it answers can only change
across a restart, and every restart re-arms the flag.
- The ledger file write no longer happens under gPayoutLedgerMutex in the
per-recipient loop. SavePendingPayoutsLocked was split into a serialise
step that needs the lock and a write step that does not, so the mutex is
held only long enough to read the structure. The orphan scan in
FlushMaturePayouts was likewise restructured so the ledger mutex is not
held across PayoutBlockStillOnChain's cs_main acquisition.
One case the merged model created and neither side handled: under
survives-until-fully-paid, our GIVING UP path dropped the in-flight record
but left the recipient owed in the ledger forever, so every later flush
would explode it again -- a forfeit that silently becomes an infinite retry.
ForfeitPayoutRecipient writes the amount off so the entry can retire.
14. QueuePayouts call site. Follow-on of 9.
15. ThreadRPCServer startup. BOTH. LoadPoolRounds added; LoadPaying and
ReconcilePayouts stay, and stay before the payout thread starts.
Crash safety, stated as the property that must hold: after kill -9 at any
point, restart pays each recipient exactly once. Before SendMoney, the
ledger has no txid and the payment is retried. Between SendMoney and the
ledger write, the wallet holds the transaction and the first-attempt probe
finds it. After the ledger write, paidTxids suppresses it. If the in-flight
file is lost entirely, the ledger rebuilds it and the probe still covers the
middle window.
src/nostr.cpp -- 3 hunks. THEIRS as the base, with two things restored from
ours. Theirs loads the Windows ROOT certificate store, which ours lacks and
genuinely needs, since OpenSSL's default verify paths are usually empty there
and our version would fail closed. But the plan's claim that theirs is a
superset of our 06bceb4 is FALSE: upstream's nostr.cpp contains no
min_proto_version, no TLS1_2, no SSL_OP_NO_*, and no set_options anywhere, so
taking it wholesale would have removed the TLS-version floor outright. Restored.
Second, their SSL_get_verify_result check sits after a successful handshake,
where with SSL_VERIFY_PEER set it is nearly dead code -- a bad certificate
aborts inside SSL_connect. The reporting belongs in the failure branch, which is
where the merge put it; their post-handshake duplicate was dropped. Our
relay-list and socket-pooling hunks auto-merged and were verified intact.
src/net.cpp -- 1 hunk. THEIRS. Their guard, hSocket == INVALID_SOCKET ||
pnode->hSocket != hSocket, subsumes ours and additionally catches descriptor
recycling, which ours did not; our version also referenced a pnode that only
existed in the select() loop. Our other db34011 hunk, nulling pnode->hSocket in
the poll-failure branch, auto-merged and was NOT re-applied -- plan section 7
Batch D says to re-apply it, and that instruction is stale. Two pieces of
residue the merge left behind were cleaned up: a comment still describing
select() semantics, and a pnode->hSocket != INVALID_SOCKET guard made redundant
by a continue two lines above it.
src/main.cpp -- 2 hunks. h1 BOTH, pure add/add adjacency between our
ParticipantHashThreadCount and their SecureClear forward declarations. h2 OURS,
and their handler discarded: 44004e3 no-ops checkorder/submitorder but keeps
market.cpp and market.h, whereas 11766be deletes the marketplace outright
including CRequestTracker, subscribe/publish and mapProducts. Ours is strictly
more.
src/main_gui.cpp -- 2 hunks. BOTH, additive on both sides. Our /payoutaddr,
/payoutfile and /hashthreads alongside their /stratumbridge and
/stratumbridgeport.
src/btfrv.cpp -- 1 hunk, not predicted by the plan. False adjacency: our
INVALID_SOCKET check and their #endif landed at the same offset. Ordering is
load-bearing -- our check goes inside the #else arm, before the #endif, or the
non-proxy build silently loses it. The BITFLASH_NODE_BUILD arm already returns
early, so both paths now have the check.
Makefile -- 2 hunks plus a hazard with no conflict marker. THEIRS for both
dependency-fetch recipes: checkout --detach FETCH_HEAD followed by test
"$(git rev-parse HEAD)" = "$(..._COMMIT)" asserts the pin actually landed, where
ours only checked it out. Everything of ours that they lack is kept: the
appimagetool 1.9.1 release pin with a sha256 check, where upstream fetches the
mutable "continuous" tag and executes it undigested, and the move of both the
tool and the AppDir out of /tmp, where any local user could pre-place an
executable and have it run as whoever runs make.
The hazard: both sides added a pin block in a different place, so git merged
both, outside any conflict region. SECP256K1_COMMIT and RANDOMX_COMMIT ended up
assigned twice, near L16 and near L44, and GNU Make takes the last := -- a naive
resolution would have silently built against our old secp256k1 pin while
appearing to adopt upstream's. The duplicate block was deleted by hand. Each of
the four variables is now assigned exactly once, and make -n resolves
SECP256K1_COMMIT to 7fecac74 with 6e2c8bc4 appearing nowhere.
Also applied here, from plan phase 4: SECP256K1_COMMIT moved to upstream's
7fecac74aed8e1fd9078380d67dd04663705c989 so signature validation matches the
binaries the rest of the network runs. secp256k1 was rebuilt from scratch
against the new pin on the build host and the checked-out dependency commit
verified. RANDOMX_COMMIT already matched at 1e9d4b2d and is unchanged.
src/Makefile, src/makefile.mingw -- 1 hunk each, both the HEADERS list. Ours
drops market.h, theirs adds proxy.h; the resolution does both. Taking theirs
wholesale would have left the Makefiles referencing a header git had already
deleted in our favour, silently, without a conflict.
Files the plan predicted would conflict and did not: net.h, main.h and db.cpp
auto-merged. Verified rather than assumed. net.h carries both our
BTF_MSG_WAIT_MAX_SECS work, including the constructor fix from 2d84982 that
starts nMsgWaitSince and nMsgWaitSize at 0, and their comment rewrite with
extern int nPeersWatched. The dead members that fix removed are gone from every
file.
Deviations from MERGE-1.2.18-PLAN.md.
- Section 6's mechanism was not used. Replayed onto the real merge base and
merged, rather than cherry-picking onto the tag. Same result, 28 hunks
instead of 51.
- Section 2 calls 09ff575 a superset of our 06bceb4. It is not; upstream has no
TLS-version floor at all. Ours restored on top of theirs.
- Section 7 Batch D says to re-apply our db34011 line into their poll-failure
branch. It auto-merged; re-applying would have duplicated it.
- Section 2 says do not take the stratum bridge. It is taken, and left off.
fStratumBridge defaults false in main.cpp, the thread is only spawned when
/stratumbridge is passed, and the listener binds 127.0.0.1 only. Excising it
would cost real divergence and future conflict for no gain while it is
disabled. Its defects -- select()/FD_SET on a node that can hold more than
1024 descriptors, and an uncapped thread per connection -- are real only when
it is enabled and are being reported upstream separately.
- Section 7's staged batches were not run as separate gated commits. The
conflict surface is interleaved within single functions, chiefly the payout
ledger, and splitting it would have meant committing states that do not
build. The merge is one commit, as briefed.
- Batch F's two fixes are folded into hunk 13 rather than deferred, because
they change the same lines the resolution rewrites.
- The plan did not anticipate the btfrv.cpp conflict, the duplicate pin block,
the market.h trap in the two Makefiles, or the give-up/infinite-retry
interaction created by adopting upstream's ledger model.
52 files changed, 5651 insertions, 462 deletions
diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..f4045f4 --- /dev/null +++ b/.github/workflows/ci.yml | |||
| @@ -0,0 +1,71 @@ | |||
| 1 | name: CI | ||
| 2 | |||
| 3 | on: | ||
| 4 | push: | ||
| 5 | pull_request: | ||
| 6 | |||
| 7 | permissions: | ||
| 8 | contents: read | ||
| 9 | |||
| 10 | concurrency: | ||
| 11 | group: ci-${{ github.workflow }}-${{ github.ref }} | ||
| 12 | cancel-in-progress: true | ||
| 13 | |||
| 14 | jobs: | ||
| 15 | linux-headless: | ||
| 16 | name: Linux headless selftests | ||
| 17 | runs-on: ubuntu-latest | ||
| 18 | steps: | ||
| 19 | - name: Check out repository | ||
| 20 | uses: actions/checkout@v5 | ||
| 21 | |||
| 22 | - name: Install Linux dependencies | ||
| 23 | run: make deps-linux | ||
| 24 | |||
| 25 | - name: Run Linux selftests | ||
| 26 | run: make tests | ||
| 27 | |||
| 28 | - name: Run Python tool regressions | ||
| 29 | run: python3 tests/scripts/test_bitflash_tools.py | ||
| 30 | |||
| 31 | - name: Run P2P message fuzz smoke | ||
| 32 | run: make fuzz-net-message-smoke | ||
| 33 | |||
| 34 | - name: Run script fuzz smoke | ||
| 35 | run: make fuzz-script-smoke | ||
| 36 | |||
| 37 | windows-ucrt64: | ||
| 38 | name: Windows UCRT64 selftests | ||
| 39 | runs-on: windows-latest | ||
| 40 | defaults: | ||
| 41 | run: | ||
| 42 | shell: msys2 {0} | ||
| 43 | steps: | ||
| 44 | - name: Check out repository | ||
| 45 | uses: actions/checkout@v5 | ||
| 46 | |||
| 47 | - name: Set up MSYS2 | ||
| 48 | uses: msys2/setup-msys2@v2 | ||
| 49 | with: | ||
| 50 | msystem: UCRT64 | ||
| 51 | update: true | ||
| 52 | install: >- | ||
| 53 | git | ||
| 54 | make | ||
| 55 | mingw-w64-ucrt-x86_64-gcc | ||
| 56 | mingw-w64-ucrt-x86_64-glfw | ||
| 57 | mingw-w64-ucrt-x86_64-openssl | ||
| 58 | mingw-w64-ucrt-x86_64-db | ||
| 59 | mingw-w64-ucrt-x86_64-libsodium | ||
| 60 | mingw-w64-ucrt-x86_64-nlohmann-json | ||
| 61 | mingw-w64-ucrt-x86_64-boost | ||
| 62 | mingw-w64-ucrt-x86_64-cmake | ||
| 63 | autoconf | ||
| 64 | automake | ||
| 65 | libtool | ||
| 66 | |||
| 67 | - name: Build Windows dependencies | ||
| 68 | run: make deps-windows | ||
| 69 | |||
| 70 | - name: Run Windows selftests | ||
| 71 | run: cd src && make -f makefile.mingw tests | ||
| @@ -10,7 +10,7 @@ | |||
| 10 | ROOT := $(shell pwd) | 10 | ROOT := $(shell pwd) |
| 11 | NPROC := $(shell nproc 2>/dev/null || echo 2) | 11 | NPROC := $(shell nproc 2>/dev/null || echo 2) |
| 12 | SUDO := $(shell [ "$$(id -u)" = "0" ] && echo "" || echo "sudo") | 12 | SUDO := $(shell [ "$$(id -u)" = "0" ] && echo "" || echo "sudo") |
| 13 | VERSION := 1.2.15 | 13 | VERSION := 1.2.18 |
| 14 | 14 | ||
| 15 | # ---- Pinned third-party sources ------------------------------------------- | 15 | # ---- Pinned third-party sources ------------------------------------------- |
| 16 | # | 16 | # |
| @@ -34,9 +34,16 @@ VERSION := 1.2.15 | |||
| 34 | # resync before shipping the result. | 34 | # resync before shipping the result. |
| 35 | # | 35 | # |
| 36 | # RandomX pin is the commit this tree has been building and syncing against | 36 | # RandomX pin is the commit this tree has been building and syncing against |
| 37 | # (post-v2.0.1 master, 2026-05-24) -- chosen to change nothing today. | 37 | # (post-v2.0.1 master, 2026-05-24) -- identical to upstream's pin. |
| 38 | # | ||
| 39 | # libsecp256k1 is pinned to upstream v1.2.18's commit rather than the one this | ||
| 40 | # fork carried, so signature validation matches the binaries the rest of the | ||
| 41 | # network runs. Consensus code must not differ from the network by accident. | ||
| 42 | # There must be exactly ONE assignment of each of these variables in this file: | ||
| 43 | # upstream defines them near the top too, and GNU Make takes the last `:=`, so | ||
| 44 | # a second block silently decides the build. | ||
| 38 | SECP256K1_REPO := https://github.com/bitcoin-core/secp256k1 | 45 | SECP256K1_REPO := https://github.com/bitcoin-core/secp256k1 |
| 39 | SECP256K1_COMMIT := 6e2c8bc4ecdc6e71dbe7a368f360d8d453ce435d | 46 | SECP256K1_COMMIT := 7fecac74aed8e1fd9078380d67dd04663705c989 |
| 40 | RANDOMX_REPO := https://github.com/tevador/RandomX | 47 | RANDOMX_REPO := https://github.com/tevador/RandomX |
| 41 | RANDOMX_COMMIT := 1e9d4b2df63fa6edf46b06789486e23c0fcaa55a | 48 | RANDOMX_COMMIT := 1e9d4b2df63fa6edf46b06789486e23c0fcaa55a |
| 42 | 49 | ||
| @@ -71,7 +78,7 @@ deps-apt: | |||
| 71 | @if command -v apt-get >/dev/null 2>&1; then \ | 78 | @if command -v apt-get >/dev/null 2>&1; then \ |
| 72 | $(SUDO) apt-get install -y build-essential cmake git pkg-config autoconf \ | 79 | $(SUDO) apt-get install -y build-essential cmake git pkg-config autoconf \ |
| 73 | libtool libssl-dev libdb5.3++-dev libsodium-dev nlohmann-json3-dev \ | 80 | libtool libssl-dev libdb5.3++-dev libsodium-dev nlohmann-json3-dev \ |
| 74 | libboost-system-dev libglfw3-dev libgl-dev python3-pil; \ | 81 | libboost-system-dev libglfw3-dev libgl-dev python3-pil clang; \ |
| 75 | else \ | 82 | else \ |
| 76 | echo "Not an apt system — install: g++ cmake git autoconf libtool"; \ | 83 | echo "Not an apt system — install: g++ cmake git autoconf libtool"; \ |
| 77 | echo " libssl libdb++ libsodium nlohmann-json boost glfw3 opengl python3-pil"; \ | 84 | echo " libssl libdb++ libsodium nlohmann-json boost glfw3 opengl python3-pil"; \ |
| @@ -85,8 +92,10 @@ deps-secp256k1: | |||
| 85 | cd $(ROOT)/secp256k1-build && \ | 92 | cd $(ROOT)/secp256k1-build && \ |
| 86 | git remote add origin $(SECP256K1_REPO) && \ | 93 | git remote add origin $(SECP256K1_REPO) && \ |
| 87 | git fetch -q --depth 1 origin $(SECP256K1_COMMIT) && \ | 94 | git fetch -q --depth 1 origin $(SECP256K1_COMMIT) && \ |
| 88 | git checkout -q $(SECP256K1_COMMIT) && \ | 95 | git checkout -q --detach FETCH_HEAD && \ |
| 89 | ./autogen.sh && \ | 96 | test "$$(git rev-parse HEAD)" = "$(SECP256K1_COMMIT)" && \ |
| 97 | echo "==> libsecp256k1 pinned at $(SECP256K1_COMMIT)" && \ | ||
| 98 | cd $(ROOT)/secp256k1-build && ./autogen.sh && \ | ||
| 90 | ./configure --enable-module-schnorrsig --enable-module-extrakeys \ | 99 | ./configure --enable-module-schnorrsig --enable-module-extrakeys \ |
| 91 | --disable-shared --with-pic --disable-benchmark --disable-tests && \ | 100 | --disable-shared --with-pic --disable-benchmark --disable-tests && \ |
| 92 | make -j$(NPROC) && $(SUDO) make install && $(SUDO) ldconfig; \ | 101 | make -j$(NPROC) && $(SUDO) make install && $(SUDO) ldconfig; \ |
| @@ -103,7 +112,9 @@ deps-randomx: | |||
| 103 | cd $(HOME)/RandomX && \ | 112 | cd $(HOME)/RandomX && \ |
| 104 | git remote add origin $(RANDOMX_REPO) && \ | 113 | git remote add origin $(RANDOMX_REPO) && \ |
| 105 | git fetch -q --depth 1 origin $(RANDOMX_COMMIT) && \ | 114 | git fetch -q --depth 1 origin $(RANDOMX_COMMIT) && \ |
| 106 | git checkout -q $(RANDOMX_COMMIT); \ | 115 | git checkout -q --detach FETCH_HEAD && \ |
| 116 | test "$$(git rev-parse HEAD)" = "$(RANDOMX_COMMIT)" && \ | ||
| 117 | echo "==> RandomX pinned at $(RANDOMX_COMMIT)" && \ | ||
| 107 | mkdir -p $(HOME)/RandomX/build; \ | 118 | mkdir -p $(HOME)/RandomX/build; \ |
| 108 | cd $(HOME)/RandomX/build && cmake .. -DCMAKE_BUILD_TYPE=Release && \ | 119 | cd $(HOME)/RandomX/build && cmake .. -DCMAKE_BUILD_TYPE=Release && \ |
| 109 | make -j$(NPROC) randomx; \ | 120 | make -j$(NPROC) randomx; \ |
| @@ -182,8 +193,14 @@ deps-windows: | |||
| 182 | autoconf automake libtool | 193 | autoconf automake libtool |
| 183 | @if [ ! -f deps/lib/libsecp256k1.a ]; then \ | 194 | @if [ ! -f deps/lib/libsecp256k1.a ]; then \ |
| 184 | rm -rf secp256k1-build; \ | 195 | rm -rf secp256k1-build; \ |
| 185 | git clone --depth 1 https://github.com/bitcoin-core/secp256k1 secp256k1-build; \ | 196 | git init -q secp256k1-build; \ |
| 186 | cd secp256k1-build && ./autogen.sh && \ | 197 | cd secp256k1-build && \ |
| 198 | git remote add origin $(SECP256K1_REPO) && \ | ||
| 199 | git fetch -q --depth 1 origin $(SECP256K1_COMMIT) && \ | ||
| 200 | git checkout -q --detach FETCH_HEAD && \ | ||
| 201 | test "$$(git rev-parse HEAD)" = "$(SECP256K1_COMMIT)" && \ | ||
| 202 | echo "==> secp256k1 pinned at $(SECP256K1_COMMIT)" && \ | ||
| 203 | cd $(ROOT)/secp256k1-build && ./autogen.sh && \ | ||
| 187 | ./configure --prefix=$(ROOT)/deps \ | 204 | ./configure --prefix=$(ROOT)/deps \ |
| 188 | --enable-module-schnorrsig --enable-module-extrakeys \ | 205 | --enable-module-schnorrsig --enable-module-extrakeys \ |
| 189 | --disable-shared --with-pic --disable-benchmark --disable-tests && \ | 206 | --disable-shared --with-pic --disable-benchmark --disable-tests && \ |
| @@ -194,9 +211,15 @@ deps-windows: | |||
| 194 | fi | 211 | fi |
| 195 | @if [ ! -f RandomX/build/librandomx.a ]; then \ | 212 | @if [ ! -f RandomX/build/librandomx.a ]; then \ |
| 196 | rm -rf RandomX; \ | 213 | rm -rf RandomX; \ |
| 197 | git clone --depth 1 https://github.com/tevador/RandomX; \ | 214 | git init -q RandomX; \ |
| 198 | mkdir -p RandomX/build; \ | 215 | cd RandomX && \ |
| 199 | cd RandomX/build && cmake .. -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release && \ | 216 | git remote add origin $(RANDOMX_REPO) && \ |
| 217 | git fetch -q --depth 1 origin $(RANDOMX_COMMIT) && \ | ||
| 218 | git checkout -q --detach FETCH_HEAD && \ | ||
| 219 | test "$$(git rev-parse HEAD)" = "$(RANDOMX_COMMIT)" && \ | ||
| 220 | echo "==> RandomX pinned at $(RANDOMX_COMMIT)" && \ | ||
| 221 | mkdir -p $(ROOT)/RandomX/build; \ | ||
| 222 | cd $(ROOT)/RandomX/build && cmake .. -G "Unix Makefiles" -DCMAKE_BUILD_TYPE=Release && \ | ||
| 200 | make randomx; \ | 223 | make randomx; \ |
| 201 | echo "==> RandomX done"; \ | 224 | echo "==> RandomX done"; \ |
| 202 | else \ | 225 | else \ |
| @@ -224,6 +247,12 @@ clean: | |||
| 224 | tests: | 247 | tests: |
| 225 | $(MAKE) -C src -f Makefile tests | 248 | $(MAKE) -C src -f Makefile tests |
| 226 | 249 | ||
| 250 | fuzz-net-message-smoke: deps-linux | ||
| 251 | $(MAKE) -C src -f Makefile fuzz-net-message-smoke | ||
| 252 | |||
| 253 | fuzz-script-smoke: deps-linux | ||
| 254 | $(MAKE) -C src -f Makefile fuzz-script-smoke | ||
| 255 | |||
| 227 | checksums: | 256 | checksums: |
| 228 | ./scripts/make-release-checksums.sh | 257 | ./scripts/make-release-checksums.sh |
| 229 | 258 | ||
| @@ -234,5 +263,5 @@ verify-release: | |||
| 234 | ./scripts/verify-release.sh $(if $(TAG),$(TAG),latest) | 263 | ./scripts/verify-release.sh $(if $(TAG),$(TAG),latest) |
| 235 | 264 | ||
| 236 | .PHONY: linux windows clean appimage \ | 265 | .PHONY: linux windows clean appimage \ |
| 237 | tests checksums sign-checksums verify-release \ | 266 | tests fuzz-net-message-smoke fuzz-script-smoke checksums sign-checksums verify-release \ |
| 238 | deps-linux deps-windows deps-apt deps-secp256k1 deps-randomx | 267 | deps-linux deps-windows deps-apt deps-secp256k1 deps-randomx |
| @@ -36,7 +36,8 @@ Worth being precise, because the difference matters if you are relying on it. | |||
| 36 | 36 | ||
| 37 | **The node still listens on 8433.** Nothing dials by IP any more, but the listener is still there, so anyone who already knows your address and can reach that port may connect directly. If that matters to you, firewall it. | 37 | **The node still listens on 8433.** Nothing dials by IP any more, but the listener is still there, so anyone who already knows your address and can reach that port may connect directly. If that matters to you, firewall it. |
| 38 | 38 | ||
| 39 | This is unlinkability between peers, not anonymity against a network observer. It is not Tor. | 39 | This is unlinkability between peers, not anonymity against a network observer. |
| 40 | For Tor routing, start the node with `-tor`; see [Tor mode](docs/tor.md). | ||
| 40 | 41 | ||
| 41 | --- | 42 | --- |
| 42 | 43 | ||
| @@ -73,6 +74,45 @@ scripts/verify-release.sh latest | |||
| 73 | See [release verification](docs/release-verification.md) for the full release | 74 | See [release verification](docs/release-verification.md) for the full release |
| 74 | audit flow and the maintainer signing step. | 75 | audit flow and the maintainer signing step. |
| 75 | 76 | ||
| 77 | Bitflash also ships a deterministic UTXO-set commitment tool: | ||
| 78 | |||
| 79 | ```bash | ||
| 80 | python3 scripts/verify-utxo-set.py --out utxo-report.json --json | ||
| 81 | ``` | ||
| 82 | |||
| 83 | It reconstructs the best chain from local block files, computes the current | ||
| 84 | UTXO root and supply, and writes canonical JSON that independent auditors can | ||
| 85 | compare or timestamp externally with OpenTimestamps. See | ||
| 86 | [UTXO-set commitments](docs/utxo-commitment.md). | ||
| 87 | |||
| 88 | The same root can prove one unspent output without sharing the full UTXO set: | ||
| 89 | |||
| 90 | ```bash | ||
| 91 | python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json | ||
| 92 | python3 scripts/verify-utxo-proof.py utxo-proof.json | ||
| 93 | ``` | ||
| 94 | |||
| 95 | See [UTXO inclusion proofs](docs/utxo-proofs.md). | ||
| 96 | |||
| 97 | Bitflash also ships a deterministic fair-launch verifier: | ||
| 98 | |||
| 99 | ```bash | ||
| 100 | python3 scripts/verify-fair-launch.py --max-blocks 1000 --out bitflash-fair-launch-report.json --json | ||
| 101 | ``` | ||
| 102 | |||
| 103 | The report reads the local chain directly, checks the genesis launch baseline, | ||
| 104 | and produces canonical JSON whose hash can be compared by independent auditors | ||
| 105 | or timestamped externally with OpenTimestamps. See | ||
| 106 | [fair-launch verification](docs/fair-launch.md). | ||
| 107 | |||
| 108 | For a static local block explorer, generate HTML and JSON from either a data | ||
| 109 | directory or explicit block files: | ||
| 110 | |||
| 111 | ```bash | ||
| 112 | python3 scripts/build-explorer.py --datadir ~/.bitflash ./explorer-out | ||
| 113 | python3 scripts/build-explorer.py ~/.bitflash/blk0001.dat ~/.bitflash/blk0002.dat ./explorer-out | ||
| 114 | ``` | ||
| 115 | |||
| 76 | **Keep your node current.** Consensus rules have changed since the first | 116 | **Keep your node current.** Consensus rules have changed since the first |
| 77 | releases — 1.2.1 fixed a bug that let anyone spend anyone's coins, and 1.2.2 | 117 | releases — 1.2.1 fixed a bug that let anyone spend anyone's coins, and 1.2.2 |
| 78 | added a per-block signature-operation cap. A node on an older build will accept | 118 | added a per-block signature-operation cap. A node on an older build will accept |
| @@ -219,15 +259,37 @@ Open **Options** from the menu bar. Under Mining Mode: | |||
| 219 | 259 | ||
| 220 | Pool announcements are published on Nostr with live status fields, so external tools can query the latest pool state by `.btf` address. | 260 | Pool announcements are published on Nostr with live status fields, so external tools can query the latest pool state by `.btf` address. |
| 221 | 261 | ||
| 262 | Pool operators also write a local `pool_status.json` every ten seconds and a | ||
| 263 | `pool_rounds.json` proof ledger whenever they find blocks or pay miners. By | ||
| 264 | default both are created in the data directory; use `-poolstatusfile=PATH` and | ||
| 265 | `-poolroundsfile=PATH` to write them somewhere a dashboard or web sync can read. | ||
| 266 | The files are informational only: nodes still discover pools through Nostr and | ||
| 267 | `.btf`, not through any web domain. | ||
| 268 | |||
| 222 | **Participant** — mine to someone else's pool. Enter or select the pool's `.btf` address and enable Start Mining. | 269 | **Participant** — mine to someone else's pool. Enter or select the pool's `.btf` address and enable Start Mining. |
| 223 | 270 | ||
| 224 | SRBMiner and XMRig connect to operator pools through the `.btf` path: | 271 | The built-in participant miner connects to operator pools through the `.btf` |
| 272 | path. External miners can use Bitflash as a local Stratum bridge: | ||
| 225 | 273 | ||
| 226 | ```bash | 274 | ```bash |
| 227 | SRBMiner-MULTI --algorithm randomx --pool POOL_BTF_ADDRESS --wallet YOUR_BTF_ADDRESS --password x | 275 | ./bitflash -nogui -stratumbridge=POOL_BTF_ADDRESS -stratumbridgeport=3333 |
| 228 | xmrig -a rx/0 -o POOL_BTF_ADDRESS -u YOUR_BTF_ADDRESS -p x | 276 | SRBMiner-MULTI --algorithm randomx --pool 127.0.0.1:3333 --wallet YOUR_BTF_ADDRESS --password x |
| 277 | xmrig -a rx/0 -o 127.0.0.1:3333 -u YOUR_BTF_ADDRESS -p x | ||
| 229 | ``` | 278 | ``` |
| 230 | 279 | ||
| 280 | Estimate rewards and electricity cost locally: | ||
| 281 | |||
| 282 | ```bash | ||
| 283 | python3 scripts/bitflash-mining-calculator.py \ | ||
| 284 | --hashrate 21 --hashrate-unit kh/s \ | ||
| 285 | --network-hashrate 1.56 --network-hashrate-unit mh/s \ | ||
| 286 | --watts 140 --kwh-cost 0.10 \ | ||
| 287 | --price 0.001 --pool-fee 1 | ||
| 288 | ``` | ||
| 289 | |||
| 290 | See [mining calculator](docs/mining-calculator.md). Price is manual until | ||
| 291 | Bitflash has a reliable public market. | ||
| 292 | |||
| 231 | --- | 293 | --- |
| 232 | 294 | ||
| 233 | ## Headless / server mode | 295 | ## Headless / server mode |
| @@ -236,7 +298,11 @@ xmrig -a rx/0 -o POOL_BTF_ADDRESS -u YOUR_BTF_ADDRESS -p x | |||
| 236 | ./bitflash -nogui # node only | 298 | ./bitflash -nogui # node only |
| 237 | ./bitflash -nogui -gen # node + solo mining | 299 | ./bitflash -nogui -gen # node + solo mining |
| 238 | ./bitflash -nogui -gen -operator # pool operator | 300 | ./bitflash -nogui -gen -operator # pool operator |
| 301 | ./bitflash -nogui -gen -operator \ | ||
| 302 | -poolstatusfile=/var/www/pool_status.json \ | ||
| 303 | -poolroundsfile=/var/www/pool_rounds.json | ||
| 239 | ./bitflash -nogui -gen -participant=POOL_BTF_ADDRESS # mine to pool | 304 | ./bitflash -nogui -gen -participant=POOL_BTF_ADDRESS # mine to pool |
| 305 | ./bitflash -nogui -stratumbridge=POOL_BTF_ADDRESS # local bridge for XMRig/SRBMiner | ||
| 240 | ``` | 306 | ``` |
| 241 | 307 | ||
| 242 | Every option takes `-` or `/`. **Under MSYS2 use the `-` form** — the shell | 308 | Every option takes `-` or `/`. **Under MSYS2 use the `-` form** — the shell |
| @@ -253,6 +319,8 @@ Other options worth knowing: | |||
| 253 | ```bash | 319 | ```bash |
| 254 | -datadir=PATH # wallet and chain data elsewhere | 320 | -datadir=PATH # wallet and chain data elsewhere |
| 255 | -port=N # P2P listen port, default 8433 | 321 | -port=N # P2P listen port, default 8433 |
| 322 | -socks=HOST:PORT # SOCKS5 for outbound Nostr and .btf rendezvous dials | ||
| 323 | -tor[=HOST:PORT] # Tor mode; default local Tor SOCKS5 proxy is 127.0.0.1:9050 | ||
| 256 | -debug # verbose log; without it debug.log is nearly silent | 324 | -debug # verbose log; without it debug.log is nearly silent |
| 257 | -help # full list | 325 | -help # full list |
| 258 | ``` | 326 | ``` |
| @@ -261,10 +329,20 @@ Other options worth knowing: | |||
| 261 | needed — the data directory takes an exclusive lock, so a second node pointed at | 329 | needed — the data directory takes an exclusive lock, so a second node pointed at |
| 262 | the same one will refuse to start. | 330 | the same one will refuse to start. |
| 263 | 331 | ||
| 332 | `-socks=127.0.0.1:9050` routes outbound Nostr discovery and `.btf` rendezvous | ||
| 333 | dials through a local SOCKS5 proxy such as Tor. When it is set, the node skips | ||
| 334 | plain HTTP external-IP probes instead of leaking a direct request outside the | ||
| 335 | proxy. IPv6 proxy endpoints use brackets, for example `-socks=[::1]:9050`. | ||
| 336 | |||
| 337 | `-tor` is a privacy shorthand for the usual local Tor SOCKS5 listener at | ||
| 338 | `127.0.0.1:9050`. It routes outbound Nostr discovery and `.btf` rendezvous | ||
| 339 | dials through Tor, refuses direct `.onion` resolution, and keeps external-IP | ||
| 340 | probes disabled. Use `-tor=HOST:PORT` when Tor listens somewhere else. | ||
| 341 | |||
| 264 | ### When something looks wrong | 342 | ### When something looks wrong |
| 265 | 343 | ||
| 266 | **Menu > Diagnostics**, and the same report in `debug.log` every ten minutes: | 344 | **Menu > Diagnostics**, and the same report in `debug.log` every ten minutes: |
| 267 | peers held against peers `select()` is actually watching, blocks received and how | 345 | peers held against peers the socket loop is actually watching, blocks received and how |
| 268 | many arrived without a parent, the proof-of-work mode with the live miner thread | 346 | many arrived without a parent, the proof-of-work mode with the live miner thread |
| 269 | count, sockets by the part of the program that opened them, and per peer how long | 347 | count, sockets by the part of the program that opened them, and per peer how long |
| 270 | since the last message each way. There is a copy button — if you open an issue, | 348 | since the last message each way. There is a copy button — if you open an issue, |
diff --git a/docs/dependencies.md b/docs/dependencies.md new file mode 100644 index 0000000..9fd45b6 --- /dev/null +++ b/docs/dependencies.md | |||
| @@ -0,0 +1,18 @@ | |||
| 1 | # Build Dependencies | ||
| 2 | |||
| 3 | Bitflash links consensus-adjacent cryptographic and proof-of-work code from | ||
| 4 | external repositories. Fresh builds must not follow upstream `master` silently: | ||
| 5 | a changed upstream branch can break reproducible builds or change behavior in | ||
| 6 | code that every node relies on. | ||
| 7 | |||
| 8 | The top-level `Makefile` pins source dependencies by full commit hash: | ||
| 9 | |||
| 10 | | dependency | repository | pinned commit | | ||
| 11 | |---|---|---| | ||
| 12 | | libsecp256k1 | https://github.com/bitcoin-core/secp256k1 | `7fecac74aed8e1fd9078380d67dd04663705c989` | | ||
| 13 | | RandomX | https://github.com/tevador/RandomX | `1e9d4b2df63fa6edf46b06789486e23c0fcaa55a` | | ||
| 14 | |||
| 15 | When updating either dependency, rebuild from a clean dependency directory and | ||
| 16 | run the full selftest suite before opening the PR. The PR should explain why the | ||
| 17 | dependency changed and whether the update can affect consensus, addresses, | ||
| 18 | signatures, proof-of-work validation, or mining. | ||
diff --git a/docs/fair-launch.md b/docs/fair-launch.md new file mode 100644 index 0000000..9603910 --- /dev/null +++ b/docs/fair-launch.md | |||
| @@ -0,0 +1,70 @@ | |||
| 1 | # Bitflash fair-launch verifier | ||
| 2 | |||
| 3 | Bitflash ships a deterministic fair-launch audit script: | ||
| 4 | |||
| 5 | ```bash | ||
| 6 | python3 scripts/verify-fair-launch.py --max-blocks 1000 --out bitflash-fair-launch-report.json --json | ||
| 7 | ``` | ||
| 8 | |||
| 9 | The verifier reads `blk*.dat` directly from the local Bitflash data directory. | ||
| 10 | It does not use Berkeley DB, does not call the node binary, and does not need | ||
| 11 | network access. It uses the same shared Python chain parser as the UTXO | ||
| 12 | commitment and proof tools, so block serialization fixes land in one place. | ||
| 13 | |||
| 14 | The report proves the launch baseline that can be checked from the chain: | ||
| 15 | |||
| 16 | - the genesis block hash matches the source constant; | ||
| 17 | - the genesis merkle root matches the source constant; | ||
| 18 | - the genesis transaction is a single coinbase transaction; | ||
| 19 | - the genesis output total is exactly `50.00000000 BTF`; | ||
| 20 | - the scanned block range and mined coinbase total are reproducible. | ||
| 21 | |||
| 22 | For machines that do not have a synced data directory yet, the source constants | ||
| 23 | can be checked without disk blocks: | ||
| 24 | |||
| 25 | ```bash | ||
| 26 | python3 scripts/verify-fair-launch.py --reconstruct-genesis | ||
| 27 | ``` | ||
| 28 | |||
| 29 | ## Deterministic output | ||
| 30 | |||
| 31 | Use `--json` or `--out` for canonical JSON. The JSON is written with sorted | ||
| 32 | keys, fixed separators, ASCII escaping, and one trailing newline. It deliberately | ||
| 33 | does not include local paths or wall-clock generation time, so two auditors | ||
| 34 | scanning the same block range produce the same bytes. | ||
| 35 | |||
| 36 | That makes the report shareable: | ||
| 37 | |||
| 38 | ```bash | ||
| 39 | python3 scripts/verify-fair-launch.py --max-blocks 1000 --out report.json --json | ||
| 40 | sha256sum report.json | ||
| 41 | ``` | ||
| 42 | |||
| 43 | If another auditor runs the same command against the same chain prefix, the | ||
| 44 | `sha256sum` should match. | ||
| 45 | |||
| 46 | ## OpenTimestamps | ||
| 47 | |||
| 48 | OpenTimestamps is optional and external. It anchors the report in Bitcoin | ||
| 49 | without adding any bridge or dependency to Bitflash consensus. | ||
| 50 | |||
| 51 | After generating a canonical JSON report: | ||
| 52 | |||
| 53 | ```bash | ||
| 54 | ots stamp report.json | ||
| 55 | ots verify report.json.ots | ||
| 56 | ``` | ||
| 57 | |||
| 58 | The `.ots` file proves that the exact report bytes existed no later than the | ||
| 59 | Bitcoin block that confirms the timestamp. | ||
| 60 | |||
| 61 | ## What this does not claim | ||
| 62 | |||
| 63 | This is not a retroactive genesis commitment. The Bitflash genesis block already | ||
| 64 | exists, so adding a new on-chain commitment to genesis would create a different | ||
| 65 | chain. This verifier instead audits the launch state that is already on-chain | ||
| 66 | and makes the result reproducible and timestampable. | ||
| 67 | |||
| 68 | It also does not prove real-world miner identity. It proves the genesis state, | ||
| 69 | the absence of non-coinbase premine in genesis, and the deterministic coinbase | ||
| 70 | totals over the scanned prefix. | ||
diff --git a/docs/mining-calculator.md b/docs/mining-calculator.md new file mode 100644 index 0000000..352eea1 --- /dev/null +++ b/docs/mining-calculator.md | |||
| @@ -0,0 +1,86 @@ | |||
| 1 | # Bitflash mining calculator | ||
| 2 | |||
| 3 | `scripts/bitflash-mining-calculator.py` is a local WhatToMine-style calculator | ||
| 4 | for Bitflash. | ||
| 5 | |||
| 6 | It does not fetch market prices, does not talk to exchanges, and does not | ||
| 7 | change consensus. Price is a manual input until Bitflash has a reliable public | ||
| 8 | market. | ||
| 9 | |||
| 10 | ## Examples | ||
| 11 | |||
| 12 | Estimate from network hashrate: | ||
| 13 | |||
| 14 | ```bash | ||
| 15 | python3 scripts/bitflash-mining-calculator.py \ | ||
| 16 | --hashrate 21 --hashrate-unit kh/s \ | ||
| 17 | --network-hashrate 1.56 --network-hashrate-unit mh/s \ | ||
| 18 | --watts 140 --kwh-cost 0.10 \ | ||
| 19 | --price 0.001 --pool-fee 1 | ||
| 20 | ``` | ||
| 21 | |||
| 22 | Estimate from Bitflash difficulty: | ||
| 23 | |||
| 24 | ```bash | ||
| 25 | python3 scripts/bitflash-mining-calculator.py \ | ||
| 26 | --hashrate 21 --hashrate-unit kh/s \ | ||
| 27 | --difficulty 45600 \ | ||
| 28 | --watts 140 --kwh-cost 0.10 | ||
| 29 | ``` | ||
| 30 | |||
| 31 | Machine-readable output: | ||
| 32 | |||
| 33 | ```bash | ||
| 34 | python3 scripts/bitflash-mining-calculator.py \ | ||
| 35 | --hashrate 21 --hashrate-unit kh/s \ | ||
| 36 | --network-hashrate 1.56 --network-hashrate-unit mh/s \ | ||
| 37 | --json | ||
| 38 | ``` | ||
| 39 | |||
| 40 | ## Bitflash Difficulty Scale | ||
| 41 | |||
| 42 | Bitflash does not use Bitcoin's SHA256 difficulty-1 constant for pool shares. | ||
| 43 | The chain's easiest RandomX target is `bnProofOfWorkLimit = ~uint256(0) >> 12`, | ||
| 44 | so difficulty 1 expects about `4096` hashes. | ||
| 45 | |||
| 46 | The calculator uses: | ||
| 47 | |||
| 48 | ```text | ||
| 49 | network_hashrate = difficulty * 4096 / block_time_seconds | ||
| 50 | ``` | ||
| 51 | |||
| 52 | At the default two-minute block target: | ||
| 53 | |||
| 54 | ```text | ||
| 55 | network_hashrate = difficulty * 4096 / 120 | ||
| 56 | ``` | ||
| 57 | |||
| 58 | If you already know network hashrate, pass `--network-hashrate` directly and | ||
| 59 | the script derives the equivalent Bitflash difficulty for display. | ||
| 60 | |||
| 61 | ## Reward Estimate | ||
| 62 | |||
| 63 | The expected daily reward is: | ||
| 64 | |||
| 65 | ```text | ||
| 66 | blocks_per_day = 86400 / block_time_seconds | ||
| 67 | miner_share = miner_hashrate / network_hashrate | ||
| 68 | gross_btf_per_day = blocks_per_day * block_reward * miner_share | ||
| 69 | net_btf_per_day = gross_btf_per_day * (1 - pool_fee_percent / 100) | ||
| 70 | ``` | ||
| 71 | |||
| 72 | Electricity is: | ||
| 73 | |||
| 74 | ```text | ||
| 75 | electricity_per_day = watts / 1000 * 24 * kwh_cost | ||
| 76 | ``` | ||
| 77 | |||
| 78 | This is an expectation, not a promise. Solo mining variance can be very large, | ||
| 79 | especially when the miner has a small share of the network. | ||
| 80 | |||
| 81 | ## Future Website Use | ||
| 82 | |||
| 83 | The script is intentionally standalone so the same math can later back a | ||
| 84 | website widget or status dashboard. Until a public price exists, a website | ||
| 85 | calculator should keep price as a manual field and label fiat profit as | ||
| 86 | speculative. | ||
diff --git a/docs/pool-directory.md b/docs/pool-directory.md new file mode 100644 index 0000000..76269b9 --- /dev/null +++ b/docs/pool-directory.md | |||
| @@ -0,0 +1,43 @@ | |||
| 1 | # Public Pool Directory | ||
| 2 | |||
| 3 | `pool.bitflash.network` should be a public directory, not a network dependency. | ||
| 4 | Nodes discover pools through Nostr and `.btf`; the website only helps humans | ||
| 5 | compare operators. | ||
| 6 | |||
| 7 | Operators can publish the `pool_status.json` and `pool_rounds.json` files | ||
| 8 | produced by: | ||
| 9 | |||
| 10 | ```bash | ||
| 11 | bitflash -nogui -gen -operator \ | ||
| 12 | -poolstatusfile=/var/www/pool_status.json \ | ||
| 13 | -poolroundsfile=/var/www/pool_rounds.json | ||
| 14 | ``` | ||
| 15 | |||
| 16 | Build a static directory from one or more status files: | ||
| 17 | |||
| 18 | ```bash | ||
| 19 | python3 scripts/build-pool-index.py --out-dir public/pool \ | ||
| 20 | https://operator-one.example/pool_status.json \ | ||
| 21 | https://operator-two.example/pool_status.json | ||
| 22 | ``` | ||
| 23 | |||
| 24 | The generated output contains: | ||
| 25 | |||
| 26 | - `index.html`: human-readable ranking page. | ||
| 27 | - `pools.json`: machine-readable API for `api.bitflash.network/pools.json`. | ||
| 28 | |||
| 29 | When a source is an HTTP(S) `pool_status.json`, the directory links to a | ||
| 30 | `pool_rounds.json` proof file next to it. That file records the block height, | ||
| 31 | block hash, coinbase txid, coinbase value, pool fee, total shares, per-address | ||
| 32 | share counts, computed payouts, and payout txids once paid. | ||
| 33 | |||
| 34 | Ranking order is intentionally simple: | ||
| 35 | |||
| 36 | 1. online pools first; | ||
| 37 | 2. higher reported hashrate; | ||
| 38 | 3. more authorized miners; | ||
| 39 | 4. lower fee; | ||
| 40 | 5. name as a stable tie-breaker. | ||
| 41 | |||
| 42 | An entry is considered online when the source says the pool is running and its | ||
| 43 | status file is no more than 120 seconds old. | ||
diff --git a/docs/release-verification.md b/docs/release-verification.md index ecacd5f..32de74f 100644 --- a/docs/release-verification.md +++ b/docs/release-verification.md | |||
| @@ -3,60 +3,82 @@ | |||
| 3 | Bitflash releases publish binaries and a `SHA256SUMS` file. The checksum file | 3 | Bitflash releases publish binaries and a `SHA256SUMS` file. The checksum file |
| 4 | proves that the file you downloaded matches the file the release page names. | 4 | proves that the file you downloaded matches the file the release page names. |
| 5 | A detached signature, `SHA256SUMS.asc`, proves that the checksum file itself was | 5 | A detached signature, `SHA256SUMS.asc`, proves that the checksum file itself was |
| 6 | signed by a trusted release key. | 6 | signed by the Bitflash release key. |
| 7 | 7 | ||
| 8 | That gives users two separate checks: | 8 | That gives users two separate checks: |
| 9 | 9 | ||
| 10 | 1. `gpg --verify SHA256SUMS.asc SHA256SUMS` checks who signed the checksums. | 10 | 1. `gpg --verify SHA256SUMS.asc SHA256SUMS` — who signed the checksums. |
| 11 | 2. `sha256sum -c SHA256SUMS` checks the binaries against those checksums. | 11 | 2. `sha256sum -c SHA256SUMS` — the binaries against those checksums. |
| 12 | 12 | ||
| 13 | ## User Check | 13 | But the first check is only worth something if you require a **specific** key. |
| 14 | A good signature by *some* key proves nothing; anyone can make one. | ||
| 14 | 15 | ||
| 15 | From a shell with `curl` or `wget` and `sha256sum`: | 16 | ## The release signing key |
| 16 | 17 | ||
| 17 | ```bash | 18 | ```text |
| 18 | scripts/verify-release.sh latest | 19 | fingerprint 910A 2B4C CA87 9E81 FB4B 2AEA 1D4A 53D3 B78A A4B8 |
| 20 | uid Bitflash Releases (bitflash.network release signing) | ||
| 21 | <releases@bitflash.network> | ||
| 19 | ``` | 22 | ``` |
| 20 | 23 | ||
| 21 | For a specific release: | 24 | The public key is published at |
| 25 | `https://releases.bitflash.network/bitflash-release-key.asc` and embedded in | ||
| 26 | `scripts/verify-release.sh`, so the tool's trust anchor lives in this | ||
| 27 | repository rather than on the server it verifies. | ||
| 28 | |||
| 29 | ## User check | ||
| 30 | |||
| 31 | From a shell with `curl` or `wget`, `sha256sum` and `gpg`: | ||
| 22 | 32 | ||
| 23 | ```bash | 33 | ```bash |
| 24 | scripts/verify-release.sh v1.2.13 | 34 | scripts/verify-release.sh latest |
| 25 | ``` | 35 | ``` |
| 26 | 36 | ||
| 27 | For release audits, require the signature: | 37 | For a specific release: |
| 28 | 38 | ||
| 29 | ```bash | 39 | ```bash |
| 30 | scripts/verify-release.sh v1.2.13 --require-signature | 40 | scripts/verify-release.sh v1.2.15 |
| 31 | ``` | 41 | ``` |
| 32 | 42 | ||
| 33 | Older releases may not have `SHA256SUMS.asc`. In that case the script warns and | 43 | The script downloads the release, imports the pinned key into a throwaway |
| 34 | still checks file integrity. New release audits should use `--require-signature`. | 44 | keyring, and **refuses anything that is not a valid signature by the key above** |
| 35 | 45 | before checking the hashes. Releases from before signing was introduced ship | |
| 36 | ## Maintainer Flow | 46 | only `SHA256SUMS`; without `--require-signature` those are checked for integrity |
| 47 | and reported as unsigned. | ||
| 37 | 48 | ||
| 38 | After building release assets in the repository root: | 49 | By hand, if you would rather not run the script: |
| 39 | 50 | ||
| 40 | ```bash | 51 | ```bash |
| 41 | scripts/make-release-checksums.sh --sign --local-user RELEASE_KEY_ID | 52 | curl -O https://releases.bitflash.network/bitflash-release-key.asc |
| 53 | gpg --import bitflash-release-key.asc | ||
| 54 | gpg --verify SHA256SUMS.asc SHA256SUMS # must name the fingerprint above | ||
| 55 | sha256sum -c SHA256SUMS | ||
| 42 | ``` | 56 | ``` |
| 43 | 57 | ||
| 44 | Upload all built assets plus: | 58 | ## Maintainer flow |
| 45 | 59 | ||
| 46 | ```text | 60 | The signing key lives on a dedicated location, not in the build tree. After |
| 47 | SHA256SUMS | 61 | building the release assets in the repository root, write and sign the |
| 48 | SHA256SUMS.asc | 62 | checksums: |
| 63 | |||
| 64 | ```bash | ||
| 65 | sha256sum Bitflash-*.zip Bitflash-*.AppImage bitflash-node-* > SHA256SUMS | ||
| 66 | gpg --armor --detach-sign --output SHA256SUMS.asc SHA256SUMS | ||
| 49 | ``` | 67 | ``` |
| 50 | 68 | ||
| 51 | Keep the private signing key offline or on a dedicated release machine. Publish | 69 | Publish, alongside the assets, `SHA256SUMS` and `SHA256SUMS.asc`, and update |
| 52 | the public key fingerprint in the release notes and keep using the same key for | 70 | `latest.txt` on the releases host to the new tag. Keep using the same key; if it |
| 53 | future releases unless there is a clearly announced rotation. | 71 | is ever rotated, announce the new fingerprint clearly and update the embedded |
| 72 | key in `verify-release.sh`. | ||
| 54 | 73 | ||
| 55 | ## Why This Matters | 74 | ## Why this matters |
| 56 | 75 | ||
| 57 | `SHA256SUMS` alone protects against a broken download, but not against someone | 76 | `SHA256SUMS` alone protects against a broken download, but not against someone |
| 58 | replacing both a binary and the checksum file. Signing `SHA256SUMS` means an | 77 | replacing both a binary and the checksum file. Signing `SHA256SUMS`, and |
| 59 | attacker must also have the release signing key to make the replacement verify. | 78 | requiring a pinned key, means an attacker must also hold the release signing key |
| 79 | to make the replacement verify. Serving from bitflash.network makes this the | ||
| 80 | guarantee that matters: without a pinned key, "trust the site" is the only | ||
| 81 | thing standing behind a download. | ||
| 60 | 82 | ||
| 61 | This is not reproducible builds yet. It is the smaller, immediate step that | 83 | This is not reproducible builds yet. It is the smaller, immediate step that |
| 62 | makes every release asset auditable by users before they run it. | 84 | makes every release asset auditable before anyone runs it. |
diff --git a/docs/rendezvous.md b/docs/rendezvous.md new file mode 100644 index 0000000..ac8e598 --- /dev/null +++ b/docs/rendezvous.md | |||
| @@ -0,0 +1,288 @@ | |||
| 1 | # Bitflash `.btf` Rendezvous Protocol | ||
| 2 | |||
| 3 | This is the specification of how Bitflash nodes reach each other, and how miners | ||
| 4 | reach pools, without revealing an IP address and without any central directory. | ||
| 5 | It documents the protocol as implemented in `src/btfaddr.cpp`, `src/btfchan.cpp`, | ||
| 6 | `src/btfrv.cpp`, `src/btftunnel.cpp` and `src/nostr.cpp`. | ||
| 7 | |||
| 8 | The design is ported from the Itzal hidden service design. A `.btf` address names | ||
| 9 | a public key, a node proves ownership of that key by signing a service | ||
| 10 | descriptor, other nodes find the descriptor over Nostr, and the two sides then | ||
| 11 | meet through an untrusted relay over an end to end encrypted tunnel. The relay | ||
| 12 | carries ciphertext only and never learns the plaintext. | ||
| 13 | |||
| 14 | ## 1. Overview | ||
| 15 | |||
| 16 | A connection to a `.btf` service has three layers: | ||
| 17 | |||
| 18 | 1. **Discovery (Nostr).** The service publishes a signed descriptor as a Nostr | ||
| 19 | event. A caller fetches it by the service public key and verifies the | ||
| 20 | signature. The descriptor says which rendezvous relay the service is waiting | ||
| 21 | at and which encryption key to use. | ||
| 22 | 2. **Rendezvous (TCP).** Caller and service both connect out to the same relay | ||
| 23 | and are paired by public key. Neither needs an inbound port or a public IP. | ||
| 24 | After pairing the relay is a blind byte pipe. | ||
| 25 | 3. **Encrypted channel (X25519 + XChaCha20-Poly1305).** Over the paired pipe the | ||
| 26 | caller and service run an ephemeral to static handshake and then exchange | ||
| 27 | authenticated frames. The relay sees only ciphertext. | ||
| 28 | |||
| 29 | The caller ends up holding an ordinary local socket that behaves like a direct | ||
| 30 | TCP connection to the peer. The node treats it exactly like any other peer | ||
| 31 | socket. | ||
| 32 | |||
| 33 | ## 2. Identities and keys | ||
| 34 | |||
| 35 | A node holds two independent keypairs. | ||
| 36 | |||
| 37 | - **Signing / identity key**: a secp256k1 x-only key (BIP340 Schnorr). This is | ||
| 38 | the `.btf` identity and also the node's Nostr identity. Persisted in | ||
| 39 | `nostr.key`. | ||
| 40 | - **Encryption key**: an X25519 (Curve25519) keypair. Persisted in | ||
| 41 | `btf_enc.key`. Its public half is published in the descriptor as `enc` and is | ||
| 42 | used for the tunnel key agreement. | ||
| 43 | |||
| 44 | The identity key names the service and authenticates the descriptor. The | ||
| 45 | encryption key secures the tunnel. They are separate so the long term identity | ||
| 46 | never performs Diffie-Hellman and the encryption key can be rotated in a new | ||
| 47 | signed descriptor. | ||
| 48 | |||
| 49 | ## 3. The `.btf` address | ||
| 50 | |||
| 51 | An address is the identity public key plus a short checksum, Base32 encoded, with | ||
| 52 | the suffix `.btf`. | ||
| 53 | |||
| 54 | ``` | ||
| 55 | label = Base32( pubkey[32] || checksum[2] ) | ||
| 56 | address = label + ".btf" | ||
| 57 | ``` | ||
| 58 | |||
| 59 | - `pubkey` is the 32 byte secp256k1 x-only public key. | ||
| 60 | - `checksum` is the first two bytes of `SHA256(".btf-checksum-v1" || pubkey)`. | ||
| 61 | - Base32 uses the lowercase alphabet `abcdefghijklmnopqrstuvwxyz234567`, no | ||
| 62 | padding. The 34 byte payload encodes to 55 characters. | ||
| 63 | - Parsing is case insensitive, trims surrounding whitespace, and strips a | ||
| 64 | trailing dot. A wrong checksum is rejected. | ||
| 65 | |||
| 66 | The address is self certifying: it *is* the public key, so no registry has to be | ||
| 67 | trusted to map a name to a key. | ||
| 68 | |||
| 69 | ## 4. Discovery over Nostr | ||
| 70 | |||
| 71 | Nodes publish and read three kinds of parameterized replaceable Nostr events. | ||
| 72 | Parameterized replaceable means the newest event for a given author and `d` tag | ||
| 73 | supersedes the older one, so a descriptor is always the latest state. | ||
| 74 | |||
| 75 | | Purpose | kind | `d` tag | | ||
| 76 | |---|---|---| | ||
| 77 | | Service descriptor | 38501 | `btf-descriptor-2` | | ||
| 78 | | Rendezvous relay announcement | 38502 | `btf-relay-2` | | ||
| 79 | | Pool announcement | 38503 | `btf-pool-2` | | ||
| 80 | |||
| 81 | The Nostr event is signed with the node's identity key, so the event author | ||
| 82 | equals the `.btf` public key. That binds the descriptor to the address at the | ||
| 83 | Nostr layer, and the descriptor is *also* signed inside its own content (see | ||
| 84 | below), so a relay operator cannot forge or swap a descriptor for an address it | ||
| 85 | does not control. | ||
| 86 | |||
| 87 | **Resolving an address.** To reach `X.btf`, a caller parses the address to the 32 | ||
| 88 | byte key and sends a Nostr `REQ` filtered by `authors = [X]`, `kinds = [38501]`, | ||
| 89 | `#d = ["btf-descriptor-2"]`, `limit = 1`. It verifies the returned descriptor and | ||
| 90 | uses its `meeting_node` and `enc`. | ||
| 91 | |||
| 92 | **Relay auto discovery.** A relay operator runs a rendezvous relay and publishes a | ||
| 93 | kind 38502 event carrying `host:port`. Every node that reads it adds the relay to | ||
| 94 | the set it can register at and dial through. No seed list edit or maintainer | ||
| 95 | approval is needed. A small curated seed list is still shipped for bootstrap: | ||
| 96 | |||
| 97 | ``` | ||
| 98 | 92.246.128.180:8434 | ||
| 99 | 31.44.4.249:8434 | ||
| 100 | 90.156.222.107:8434 | ||
| 101 | ``` | ||
| 102 | |||
| 103 | The default rendezvous port is `8434`. | ||
| 104 | |||
| 105 | ## 5. The service descriptor | ||
| 106 | |||
| 107 | The descriptor is a JSON object, signed with the identity key over a domain | ||
| 108 | separated digest of its own fields. | ||
| 109 | |||
| 110 | ```json | ||
| 111 | { | ||
| 112 | "v": 2, | ||
| 113 | "pubkey": "<hex 32-byte x-only identity key>", | ||
| 114 | "enc": "<hex 32-byte X25519 public key>", | ||
| 115 | "meeting_node": "host:port", | ||
| 116 | "created": <unix seconds>, | ||
| 117 | "sig": "<hex 64-byte BIP340 Schnorr signature>" | ||
| 118 | } | ||
| 119 | ``` | ||
| 120 | |||
| 121 | The signed digest is: | ||
| 122 | |||
| 123 | ``` | ||
| 124 | digest = SHA256( ".btf-descriptor-v2" | ||
| 125 | || pubkey[32] | ||
| 126 | || enc (raw bytes of the string) | ||
| 127 | || meeting_node (raw bytes of the string) | ||
| 128 | || created (8 bytes, big endian) ) | ||
| 129 | ``` | ||
| 130 | |||
| 131 | The signature is a deterministic BIP340 Schnorr signature (no auxiliary | ||
| 132 | randomness) over `digest` by the identity key. | ||
| 133 | |||
| 134 | **Verification** (untrusted input, must never throw): parse the JSON, require all | ||
| 135 | fields with the right types, confirm `pubkey` equals the address being resolved, | ||
| 136 | recompute the digest, and verify the Schnorr signature. Any failure rejects the | ||
| 137 | descriptor. | ||
| 138 | |||
| 139 | `created` lets a reader prefer the freshest descriptor when more than one is | ||
| 140 | seen. A node only publishes a descriptor once it has actually registered at a | ||
| 141 | `meeting_node`, so it never advertises a relay it cannot be reached at. | ||
| 142 | |||
| 143 | ## 6. Rendezvous transport | ||
| 144 | |||
| 145 | The relay is a small TCP server. Every connection opens with a fixed 33 byte | ||
| 146 | header: | ||
| 147 | |||
| 148 | ``` | ||
| 149 | byte 0 role: 'S' (service) or 'C' (client) | ||
| 150 | bytes 1..32 target public key (the .btf identity, 32 bytes) | ||
| 151 | ``` | ||
| 152 | |||
| 153 | **Service registration (`S`).** The service connects out to the relay and sends | ||
| 154 | `'S' || own_pubkey`. The relay stores the connection in a table keyed by the | ||
| 155 | public key and leaves it open, waiting. There is one slot per key: a new | ||
| 156 | registration for the same key replaces and closes the old one. The service does | ||
| 157 | not block waiting to be paired at registration time; it is considered registered | ||
| 158 | the moment the relay reads the header, which is what lets it publish a descriptor | ||
| 159 | pointing at this relay. | ||
| 160 | |||
| 161 | **Client dial (`C`).** The caller connects out to the same relay and sends | ||
| 162 | `'C' || target_pubkey`. The relay looks up the waiting service: | ||
| 163 | |||
| 164 | - If present, it removes it from the table, writes a single `0x01` byte to both | ||
| 165 | sockets, and then forwards raw bytes in both directions until either side | ||
| 166 | closes. From this point the relay is a blind pipe. | ||
| 167 | - If absent, it writes `0x00` to the caller and closes. | ||
| 168 | |||
| 169 | Both sides connect *outbound* to the relay, so neither needs a public IP or an | ||
| 170 | open inbound port. The relay never sees the identity of the caller (the caller | ||
| 171 | sends no key of its own, only the target it wants). | ||
| 172 | |||
| 173 | **Liveness.** A registration socket is idle by design and can outlive the network | ||
| 174 | path that created it (a NAT drops the idle mapping, the relay restarts, a route | ||
| 175 | changes) with no FIN and nothing readable. TCP keepalive was measured and does | ||
| 176 | not reliably detect this. The service therefore imposes its own timeout on the | ||
| 177 | wait for the pairing byte and re-registers on expiry. Expiring is not a failure; | ||
| 178 | it costs one reconnect per interval and returns the loop to a known state. | ||
| 179 | |||
| 180 | ## 7. The encrypted channel | ||
| 181 | |||
| 182 | Once paired through the relay, the caller and service run an end to end handshake | ||
| 183 | and then exchange authenticated frames. The primitive is libsodium | ||
| 184 | `crypto_box_curve25519xchacha20poly1305` (X25519 key agreement, XChaCha20-Poly1305 | ||
| 185 | AEAD), the same construction as Rust's `crypto_box` ChaChaBox. | ||
| 186 | |||
| 187 | **Handshake (ephemeral to static).** | ||
| 188 | |||
| 189 | 1. The caller generates a fresh ephemeral X25519 keypair, computes | ||
| 190 | `shared = X25519(eph_sk, service_enc_pub)` reduced to a box key | ||
| 191 | (`crypto_box_beforenm`), and sends the 32 byte ephemeral public key over the | ||
| 192 | paired pipe. | ||
| 193 | 2. The service reads the ephemeral public key and computes the same | ||
| 194 | `shared = X25519(enc_sk, eph_pub)`. | ||
| 195 | |||
| 196 | Because the caller uses a new ephemeral key per connection, each session has an | ||
| 197 | independent key and forward secrecy against later compromise of the service | ||
| 198 | encryption key. | ||
| 199 | |||
| 200 | **Framing.** Every message is length prefixed: | ||
| 201 | |||
| 202 | ``` | ||
| 203 | frame = len[4] (big endian, length of the body that follows) | ||
| 204 | body = nonce[24] || ciphertext || mac[16] | ||
| 205 | ``` | ||
| 206 | |||
| 207 | The nonce is 24 random bytes per frame (XChaCha20), the MAC is 16 bytes | ||
| 208 | (Poly1305). A body shorter than `nonce + mac` or larger than 1 MiB is rejected. | ||
| 209 | Decryption that fails authentication drops the frame and tears the tunnel down. | ||
| 210 | |||
| 211 | The tunnel is transparent: the application is handed a local socket pair end and | ||
| 212 | reads and writes plaintext. Two pump threads move data between that socket and | ||
| 213 | the relay socket, encrypting outbound and decrypting inbound. | ||
| 214 | |||
| 215 | ## 8. End to end connection flow | ||
| 216 | |||
| 217 | To dial `X.btf`: | ||
| 218 | |||
| 219 | 1. Parse `X.btf` to the 32 byte identity key. | ||
| 220 | 2. Fetch and verify the descriptor over Nostr (kind 38501). Obtain | ||
| 221 | `meeting_node` and `enc`. | ||
| 222 | 3. Connect to `meeting_node`, send `'C' || X`, and wait for `0x01`. | ||
| 223 | 4. Generate an ephemeral X25519 key, derive the shared key against `enc`, send | ||
| 224 | the ephemeral public key. | ||
| 225 | 5. Exchange XChaCha20-Poly1305 frames. Hand the application a local socket that | ||
| 226 | speaks the ordinary peer protocol. | ||
| 227 | |||
| 228 | A service, symmetrically: registers at a relay (`'S' || own_pubkey`), publishes a | ||
| 229 | descriptor pointing at that relay, waits for `0x01`, reads the caller ephemeral | ||
| 230 | public key, derives the shared key with its encryption secret, and serves the | ||
| 231 | tunnel. | ||
| 232 | |||
| 233 | ## 9. Security properties | ||
| 234 | |||
| 235 | - **Address unforgeability.** The address is the identity key and the descriptor | ||
| 236 | is Schnorr signed by it, and the Nostr event is signed by it. A relay or Nostr | ||
| 237 | operator cannot publish a working descriptor for an address it does not own. | ||
| 238 | - **Relay confidentiality.** The relay forwards ciphertext and never holds the | ||
| 239 | channel key. It cannot read or modify traffic undetected; a tampered frame | ||
| 240 | fails the Poly1305 tag and the tunnel closes. | ||
| 241 | - **Caller anonymity to the service and relay.** The caller presents no identity | ||
| 242 | key, only the target it wants to reach. It connects outbound, so its address is | ||
| 243 | not published anywhere. | ||
| 244 | - **Forward secrecy.** The caller's per connection ephemeral key means a later | ||
| 245 | compromise of the service encryption key does not decrypt past sessions. | ||
| 246 | - **IP privacy.** Neither side needs a public IP or an inbound port. The observable | ||
| 247 | network fact is that each side has a TCP connection to a relay. | ||
| 248 | |||
| 249 | ## 10. Threat model and limitations | ||
| 250 | |||
| 251 | - **The relay is trusted for availability, not for confidentiality.** It can | ||
| 252 | refuse to pair, drop a connection, or go offline. This is why relays are | ||
| 253 | plural, auto discovered, and re-registration is periodic. It cannot read | ||
| 254 | traffic. | ||
| 255 | - **Traffic analysis is not addressed.** A network observer, or the relay, can | ||
| 256 | see connection timing and byte volumes. There is no padding or mixing. | ||
| 257 | - **Single slot registration causes churn.** One waiting slot per key per relay | ||
| 258 | means a race or a stale registration can briefly make a service unreachable at | ||
| 259 | that relay. Multiple relays and the wait timeout bound the damage, but this is | ||
| 260 | a known source of connection churn. | ||
| 261 | - **Descriptor freshness is best effort.** `created` is a self reported | ||
| 262 | timestamp; readers prefer the newest descriptor they have seen but there is no | ||
| 263 | global clock. | ||
| 264 | - **Discovery depends on public Nostr relays.** If the relays a node uses are all | ||
| 265 | unreachable, discovery stalls until one is reachable again. Relay TLS is | ||
| 266 | verified, which closes a man in the middle path on discovery, but availability | ||
| 267 | still rests on the relay set. | ||
| 268 | |||
| 269 | ## 11. Wire reference | ||
| 270 | |||
| 271 | | Item | Value | | ||
| 272 | |---|---| | ||
| 273 | | `.btf` Base32 alphabet | `abcdefghijklmnopqrstuvwxyz234567`, no padding | | ||
| 274 | | Address payload | `pubkey[32] || checksum[2]`, 55 Base32 chars | | ||
| 275 | | Address checksum | `SHA256(".btf-checksum-v1" || pubkey)[0..2]` | | ||
| 276 | | Descriptor domain sep | `.btf-descriptor-v2` | | ||
| 277 | | Descriptor version | `2` | | ||
| 278 | | Descriptor signature | BIP340 Schnorr, deterministic (no aux rand) | | ||
| 279 | | Nostr descriptor event | kind `38501`, `d = btf-descriptor-2` | | ||
| 280 | | Nostr relay event | kind `38502`, `d = btf-relay-2` | | ||
| 281 | | Nostr pool event | kind `38503`, `d = btf-pool-2` | | ||
| 282 | | Rendezvous port (default) | `8434` | | ||
| 283 | | Rendezvous header | `role[1] ('S'/'C') || pubkey[32]`, 33 bytes | | ||
| 284 | | Rendezvous pairing byte | `0x01` paired, `0x00` no service | | ||
| 285 | | Channel primitive | `crypto_box_curve25519xchacha20poly1305` | | ||
| 286 | | Nonce / MAC / key sizes | 24 / 16 / 32 bytes | | ||
| 287 | | Frame | `len[4 big endian] || nonce[24] || ct || mac[16]` | | ||
| 288 | | Max frame body | 1 MiB | | ||
diff --git a/docs/social.png b/docs/social.png index 19e2a3b..9a08b02 100644 --- a/docs/social.png +++ b/docs/social.png | |||
| Binary files differ | |||
diff --git a/docs/tor.md b/docs/tor.md new file mode 100644 index 0000000..087c822 --- /dev/null +++ b/docs/tor.md | |||
| @@ -0,0 +1,84 @@ | |||
| 1 | # Tor mode | ||
| 2 | |||
| 3 | Bitflash can route outbound discovery traffic through a local Tor SOCKS5 proxy. | ||
| 4 | This is transport plumbing only: it does not change consensus, wallet keys, | ||
| 5 | mining, `.btf` addresses, or the encrypted rendezvous protocol. | ||
| 6 | |||
| 7 | ## Client mode | ||
| 8 | |||
| 9 | Start Tor locally, then start Bitflash with: | ||
| 10 | |||
| 11 | ```bash | ||
| 12 | bitflash -tor | ||
| 13 | ``` | ||
| 14 | |||
| 15 | `-tor` is shorthand for: | ||
| 16 | |||
| 17 | ```bash | ||
| 18 | bitflash -socks=127.0.0.1:9050 | ||
| 19 | ``` | ||
| 20 | |||
| 21 | If Tor listens somewhere else: | ||
| 22 | |||
| 23 | ```bash | ||
| 24 | bitflash -tor=127.0.0.1:9150 | ||
| 25 | bitflash -tor=[::1]:9050 | ||
| 26 | ``` | ||
| 27 | |||
| 28 | When Tor mode is enabled: | ||
| 29 | |||
| 30 | - outbound Nostr relay dials use Tor; | ||
| 31 | - outbound `.btf` rendezvous relay dials use Tor; | ||
| 32 | - destination names are sent to Tor as SOCKS5 domain-name CONNECT requests, so | ||
| 33 | the local DNS resolver does not see them; | ||
| 34 | - direct `.onion` dials are refused unless a SOCKS5/Tor proxy is enabled; | ||
| 35 | - plain HTTP external-IP probes are skipped. | ||
| 36 | |||
| 37 | ## Onion rendezvous relay | ||
| 38 | |||
| 39 | An operator can expose a Bitflash rendezvous relay as a Tor hidden service. The | ||
| 40 | node does not create hidden services itself; Tor owns that configuration. | ||
| 41 | |||
| 42 | Example `torrc`: | ||
| 43 | |||
| 44 | ```text | ||
| 45 | HiddenServiceDir /var/lib/tor/bitflash-rendezvous/ | ||
| 46 | HiddenServicePort 8434 127.0.0.1:8434 | ||
| 47 | ``` | ||
| 48 | |||
| 49 | After restarting Tor, read the generated onion name: | ||
| 50 | |||
| 51 | ```bash | ||
| 52 | sudo cat /var/lib/tor/bitflash-rendezvous/hostname | ||
| 53 | ``` | ||
| 54 | |||
| 55 | Then run the relay on localhost/VPS as usual and announce the onion endpoint from | ||
| 56 | nodes that use Tor: | ||
| 57 | |||
| 58 | ```bash | ||
| 59 | bitflash -tor -rvrelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 | ||
| 60 | bitflash -tor -announcerelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 | ||
| 61 | ``` | ||
| 62 | |||
| 63 | For a pool operator that wants the pool descriptor to point at an onion | ||
| 64 | rendezvous relay, use both: | ||
| 65 | |||
| 66 | ```bash | ||
| 67 | bitflash -operator -tor \ | ||
| 68 | -rvrelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 \ | ||
| 69 | -announcerelay=exampleexampleexampleexampleexampleexampleexampleexample.onion:8434 | ||
| 70 | ``` | ||
| 71 | |||
| 72 | ## Full-node hidden service | ||
| 73 | |||
| 74 | The legacy P2P listener still binds to port `8433`. You can expose it through | ||
| 75 | Tor manually: | ||
| 76 | |||
| 77 | ```text | ||
| 78 | HiddenServiceDir /var/lib/tor/bitflash-node/ | ||
| 79 | HiddenServicePort 8433 127.0.0.1:8433 | ||
| 80 | ``` | ||
| 81 | |||
| 82 | Current automatic peer discovery is `.btf`/rendezvous based, not onion-address | ||
| 83 | gossip. Treat direct full-node onion access as an operator/admin tool unless a | ||
| 84 | future release adds first-class onion peer advertisements. | ||
diff --git a/docs/utxo-commitment.md b/docs/utxo-commitment.md new file mode 100644 index 0000000..ab43deb --- /dev/null +++ b/docs/utxo-commitment.md | |||
| @@ -0,0 +1,125 @@ | |||
| 1 | # Bitflash UTXO-set commitment | ||
| 2 | |||
| 3 | `scripts/verify-utxo-set.py` reconstructs a deterministic commitment to the | ||
| 4 | current UTXO set from local `blk*.dat` files. | ||
| 5 | |||
| 6 | This is intentionally not a consensus rule yet. It is an audit tool: independent | ||
| 7 | operators can run it against the same chain prefix and compare the resulting | ||
| 8 | root and canonical JSON hash. | ||
| 9 | |||
| 10 | ```bash | ||
| 11 | python3 scripts/verify-utxo-set.py --max-blocks 1000 --out utxo-report.json --json | ||
| 12 | ``` | ||
| 13 | |||
| 14 | Use `--max-blocks 0` or omit the option to scan every available block file: | ||
| 15 | |||
| 16 | ```bash | ||
| 17 | python3 scripts/verify-utxo-set.py --out utxo-report.json --json | ||
| 18 | ``` | ||
| 19 | |||
| 20 | ## What It Verifies | ||
| 21 | |||
| 22 | The verifier: | ||
| 23 | |||
| 24 | - parses Bitflash block files directly with the shared Python chain parser; | ||
| 25 | - streams `blk*.dat` records instead of loading full block files into memory; | ||
| 26 | - reconstructs the best chain rooted at the Bitflash genesis block; | ||
| 27 | - ignores side blocks that may also be present in `blk*.dat`; | ||
| 28 | - verifies every scanned block merkle root; | ||
| 29 | - applies spends and creates outputs in deterministic order; | ||
| 30 | - reports current UTXO count, current supply, mined coinbase total, fees, and | ||
| 31 | historical duplicate-output overwrites; | ||
| 32 | - computes a deterministic UTXO root. | ||
| 33 | |||
| 34 | ## Commitment Algorithm | ||
| 35 | |||
| 36 | The UTXO map is keyed by: | ||
| 37 | |||
| 38 | ```text | ||
| 39 | txid:vout | ||
| 40 | ``` | ||
| 41 | |||
| 42 | Keys are sorted lexicographically. Each leaf is: | ||
| 43 | |||
| 44 | ```text | ||
| 45 | SHA256("BTFUTXO1|" || outpoint || "|" || value_satoshis || "|" || | ||
| 46 | script_pub_key_hex || "|" || created_height || "|" || created_txid) | ||
| 47 | ``` | ||
| 48 | |||
| 49 | The tree is then reduced pairwise. If a level has an odd number of hashes, the | ||
| 50 | last hash is duplicated. Each parent is: | ||
| 51 | |||
| 52 | ```text | ||
| 53 | SHA256("BTFNODE1|" || left_hash || right_hash) | ||
| 54 | ``` | ||
| 55 | |||
| 56 | The final hash is the UTXO-set root. | ||
| 57 | |||
| 58 | ## Inclusion Proofs | ||
| 59 | |||
| 60 | After a root has been computed, `scripts/prove-utxo.py` can create a compact | ||
| 61 | inclusion proof for one unspent output: | ||
| 62 | |||
| 63 | ```bash | ||
| 64 | python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json | ||
| 65 | python3 scripts/verify-utxo-proof.py utxo-proof.json | ||
| 66 | ``` | ||
| 67 | |||
| 68 | The verifier does not need local block files. It recomputes the target leaf, | ||
| 69 | walks the sibling path, and compares the result with the expected root. See | ||
| 70 | [UTXO inclusion proofs](utxo-proofs.md). | ||
| 71 | |||
| 72 | ## Deterministic Output | ||
| 73 | |||
| 74 | `--json` and `--out` write canonical JSON with sorted keys, fixed separators, | ||
| 75 | ASCII escaping, and one trailing newline. It contains no local path and no | ||
| 76 | generation timestamp, so two auditors scanning the same chain prefix should | ||
| 77 | produce byte-identical reports. | ||
| 78 | |||
| 79 | ```bash | ||
| 80 | python3 scripts/verify-utxo-set.py --out utxo-report.json --json | ||
| 81 | sha256sum utxo-report.json | ||
| 82 | ``` | ||
| 83 | |||
| 84 | ## OpenTimestamps | ||
| 85 | |||
| 86 | The canonical JSON can be timestamped in Bitcoin without adding any dependency | ||
| 87 | to Bitflash consensus: | ||
| 88 | |||
| 89 | ```bash | ||
| 90 | ots stamp utxo-report.json | ||
| 91 | ots verify utxo-report.json.ots | ||
| 92 | ``` | ||
| 93 | |||
| 94 | ## Historical Duplicates | ||
| 95 | |||
| 96 | Bitflash inherited early Bitcoin behavior around duplicate transaction IDs. | ||
| 97 | The verifier reports duplicate-output overwrites as warnings and uses | ||
| 98 | last-write-wins semantics for the UTXO map, matching the legacy index model. | ||
| 99 | Those warnings are part of the report so auditors can see the historical debt | ||
| 100 | instead of relying on a polished summary. | ||
| 101 | |||
| 102 | For synthetic chains or future clean consensus ranges, auditors can make | ||
| 103 | duplicates a hard failure: | ||
| 104 | |||
| 105 | ```bash | ||
| 106 | python3 scripts/verify-utxo-set.py --strict-duplicates | ||
| 107 | ``` | ||
| 108 | |||
| 109 | The default remains compatible with the current historical chain because it has | ||
| 110 | known legacy duplicate-output overwrites. | ||
| 111 | |||
| 112 | ## Shared Parser | ||
| 113 | |||
| 114 | The audit scripts share `scripts/bitflash_chain.py` for block parsing, | ||
| 115 | main-chain selection, UTXO application, canonical JSON, and genesis | ||
| 116 | reconstruction. That keeps `verify-utxo-set.py`, `verify-fair-launch.py`, | ||
| 117 | `prove-utxo.py`, and the explorer on the same serialization rules. | ||
| 118 | |||
| 119 | ## Future Consensus Path | ||
| 120 | |||
| 121 | This tool is the low-risk first step toward consensus commitments. A future | ||
| 122 | hard-fork design can commit this root, or a Utreexo-style accumulator root, in | ||
| 123 | coinbase or block headers after a fixed activation height. Until then this | ||
| 124 | script provides reproducible supply and state commitments without changing what | ||
| 125 | nodes accept. | ||
diff --git a/docs/utxo-proofs.md b/docs/utxo-proofs.md new file mode 100644 index 0000000..4947c3c --- /dev/null +++ b/docs/utxo-proofs.md | |||
| @@ -0,0 +1,61 @@ | |||
| 1 | # Bitflash UTXO inclusion proofs | ||
| 2 | |||
| 3 | `scripts/prove-utxo.py` creates a deterministic inclusion proof for one | ||
| 4 | currently unspent output. `scripts/verify-utxo-proof.py` verifies that proof | ||
| 5 | against the UTXO root produced by `scripts/verify-utxo-set.py`. | ||
| 6 | |||
| 7 | This is intentionally outside consensus. It turns the audit root into something | ||
| 8 | useful for a specific coin without changing what nodes accept. | ||
| 9 | |||
| 10 | ## Create A Proof | ||
| 11 | |||
| 12 | Use an unspent output in `txid:vout` form: | ||
| 13 | |||
| 14 | ```bash | ||
| 15 | python3 scripts/prove-utxo.py TXID:VOUT --out utxo-proof.json --json | ||
| 16 | ``` | ||
| 17 | |||
| 18 | The proof contains: | ||
| 19 | |||
| 20 | - the target UTXO; | ||
| 21 | - its sorted leaf index; | ||
| 22 | - the leaf hash; | ||
| 23 | - the Merkle sibling path; | ||
| 24 | - the UTXO root and chain tip used to create it. | ||
| 25 | |||
| 26 | It does not contain the full UTXO set. | ||
| 27 | |||
| 28 | ## Verify A Proof | ||
| 29 | |||
| 30 | Verify against the root embedded in the proof: | ||
| 31 | |||
| 32 | ```bash | ||
| 33 | python3 scripts/verify-utxo-proof.py utxo-proof.json | ||
| 34 | ``` | ||
| 35 | |||
| 36 | Or verify against a root copied from another machine, dashboard, block explorer, | ||
| 37 | or future node export: | ||
| 38 | |||
| 39 | ```bash | ||
| 40 | python3 scripts/verify-utxo-proof.py utxo-proof.json \ | ||
| 41 | --root ROOT_FROM_REPORT_OR_EXPORT | ||
| 42 | ``` | ||
| 43 | |||
| 44 | The verifier does not need local `blk*.dat` files. It only needs the proof and | ||
| 45 | the expected root. | ||
| 46 | |||
| 47 | ## Scope | ||
| 48 | |||
| 49 | This first proof format supports inclusion only. Absence and range proofs are | ||
| 50 | left for a later design because they require a more careful commitment scheme | ||
| 51 | and are easier to get subtly wrong. | ||
| 52 | |||
| 53 | The proof uses the exact same leaf and node domains as the UTXO commitment: | ||
| 54 | |||
| 55 | ```text | ||
| 56 | BTFUTXO1 | ||
| 57 | BTFNODE1 | ||
| 58 | ``` | ||
| 59 | |||
| 60 | That keeps the verifier language-neutral and gives future implementations a | ||
| 61 | small, stable target. | ||
diff --git a/scripts/bitflash-mining-calculator.py b/scripts/bitflash-mining-calculator.py new file mode 100644 index 0000000..1fcc1a9 --- /dev/null +++ b/scripts/bitflash-mining-calculator.py | |||
| @@ -0,0 +1,206 @@ | |||
| 1 | #!/usr/bin/env python3 | ||
| 2 | """Estimate Bitflash mining rewards and electricity cost. | ||
| 3 | |||
| 4 | This is a local WhatToMine-style calculator for Bitflash. It does not fetch | ||
| 5 | prices, does not talk to the node, and does not affect consensus. | ||
| 6 | """ | ||
| 7 | |||
| 8 | import argparse | ||
| 9 | import json | ||
| 10 | import math | ||
| 11 | import sys | ||
| 12 | |||
| 13 | |||
| 14 | HASHES_AT_DIFF_1 = 4096.0 | ||
| 15 | DEFAULT_BLOCK_TIME = 120.0 | ||
| 16 | DEFAULT_BLOCK_REWARD = 50.0 | ||
| 17 | |||
| 18 | |||
| 19 | UNITS = { | ||
| 20 | "h": 1.0, | ||
| 21 | "h/s": 1.0, | ||
| 22 | "hs": 1.0, | ||
| 23 | "kh": 1_000.0, | ||
| 24 | "kh/s": 1_000.0, | ||
| 25 | "khs": 1_000.0, | ||
| 26 | "mh": 1_000_000.0, | ||
| 27 | "mh/s": 1_000_000.0, | ||
| 28 | "mhs": 1_000_000.0, | ||
| 29 | "gh": 1_000_000_000.0, | ||
| 30 | "gh/s": 1_000_000_000.0, | ||
| 31 | "ghs": 1_000_000_000.0, | ||
| 32 | } | ||
| 33 | |||
| 34 | |||
| 35 | def parse_hashrate(value, unit): | ||
| 36 | try: | ||
| 37 | n = float(value) | ||
| 38 | except ValueError: | ||
| 39 | raise SystemExit("invalid hashrate: %s" % value) | ||
| 40 | mul = UNITS.get(unit.lower()) | ||
| 41 | if mul is None: | ||
| 42 | raise SystemExit("unknown hashrate unit: %s" % unit) | ||
| 43 | if n < 0: | ||
| 44 | raise SystemExit("hashrate must not be negative") | ||
| 45 | return n * mul | ||
| 46 | |||
| 47 | |||
| 48 | def fmt_hashrate(hps): | ||
| 49 | units = [("GH/s", 1_000_000_000.0), ("MH/s", 1_000_000.0), ("kH/s", 1_000.0), ("H/s", 1.0)] | ||
| 50 | for name, div in units: | ||
| 51 | if hps >= div: | ||
| 52 | return "%.4g %s" % (hps / div, name) | ||
| 53 | return "%.4g H/s" % hps | ||
| 54 | |||
| 55 | |||
| 56 | def fmt_money(x): | ||
| 57 | return "%.8f" % x | ||
| 58 | |||
| 59 | |||
| 60 | def estimate(args): | ||
| 61 | miner_hps = parse_hashrate(args.hashrate, args.hashrate_unit) | ||
| 62 | if args.network_hashrate is not None: | ||
| 63 | network_hps = parse_hashrate(args.network_hashrate, args.network_hashrate_unit) | ||
| 64 | difficulty = network_hps * args.block_time / HASHES_AT_DIFF_1 | ||
| 65 | network_source = "network_hashrate" | ||
| 66 | else: | ||
| 67 | if args.difficulty is None: | ||
| 68 | raise SystemExit("provide either --network-hashrate or --difficulty") | ||
| 69 | difficulty = float(args.difficulty) | ||
| 70 | if difficulty <= 0: | ||
| 71 | raise SystemExit("difficulty must be positive") | ||
| 72 | network_hps = difficulty * HASHES_AT_DIFF_1 / args.block_time | ||
| 73 | network_source = "difficulty" | ||
| 74 | |||
| 75 | if network_hps <= 0: | ||
| 76 | raise SystemExit("network hashrate must be positive") | ||
| 77 | |||
| 78 | block_reward = float(args.block_reward) | ||
| 79 | block_time = float(args.block_time) | ||
| 80 | pool_fee_percent = float(args.pool_fee) | ||
| 81 | price = float(args.price) | ||
| 82 | watts = float(args.watts) | ||
| 83 | kwh_cost = float(args.kwh_cost) | ||
| 84 | if block_reward < 0 or block_time <= 0 or pool_fee_percent < 0 or price < 0 or watts < 0 or kwh_cost < 0: | ||
| 85 | raise SystemExit("numeric inputs must be non-negative; block time must be positive") | ||
| 86 | |||
| 87 | share = miner_hps / network_hps | ||
| 88 | blocks_per_day = 86400.0 / block_time | ||
| 89 | gross_btf_day = blocks_per_day * block_reward * share | ||
| 90 | net_btf_day = gross_btf_day * max(0.0, 1.0 - pool_fee_percent / 100.0) | ||
| 91 | revenue_day = net_btf_day * price | ||
| 92 | cost_day = watts / 1000.0 * 24.0 * kwh_cost | ||
| 93 | profit_day = revenue_day - cost_day | ||
| 94 | expected_seconds_per_block = math.inf if miner_hps <= 0 else difficulty * HASHES_AT_DIFF_1 / miner_hps | ||
| 95 | |||
| 96 | periods = { | ||
| 97 | "hour": 1.0 / 24.0, | ||
| 98 | "day": 1.0, | ||
| 99 | "week": 7.0, | ||
| 100 | "month_30d": 30.0, | ||
| 101 | } | ||
| 102 | rows = {} | ||
| 103 | for name, days in periods.items(): | ||
| 104 | rows[name] = { | ||
| 105 | "btf": net_btf_day * days, | ||
| 106 | "revenue": revenue_day * days, | ||
| 107 | "electricity_cost": cost_day * days, | ||
| 108 | "profit": profit_day * days, | ||
| 109 | } | ||
| 110 | |||
| 111 | return { | ||
| 112 | "schema": "bitflash-mining-calculator-1", | ||
| 113 | "network_source": network_source, | ||
| 114 | "inputs": { | ||
| 115 | "miner_hashrate_hps": miner_hps, | ||
| 116 | "miner_hashrate": fmt_hashrate(miner_hps), | ||
| 117 | "network_hashrate_hps": network_hps, | ||
| 118 | "network_hashrate": fmt_hashrate(network_hps), | ||
| 119 | "difficulty": difficulty, | ||
| 120 | "block_time_seconds": block_time, | ||
| 121 | "block_reward_btf": block_reward, | ||
| 122 | "pool_fee_percent": pool_fee_percent, | ||
| 123 | "price_per_btf": price, | ||
| 124 | "power_watts": watts, | ||
| 125 | "electricity_per_kwh": kwh_cost, | ||
| 126 | }, | ||
| 127 | "estimates": { | ||
| 128 | "miner_network_share": share, | ||
| 129 | "blocks_per_day": blocks_per_day, | ||
| 130 | "expected_time_to_solo_block_seconds": expected_seconds_per_block, | ||
| 131 | "expected_time_to_solo_block_hours": expected_seconds_per_block / 3600.0, | ||
| 132 | "gross_btf_per_day": gross_btf_day, | ||
| 133 | "net_btf_per_day": net_btf_day, | ||
| 134 | "periods": rows, | ||
| 135 | }, | ||
| 136 | } | ||
| 137 | |||
| 138 | |||
| 139 | def print_text(result): | ||
| 140 | i = result["inputs"] | ||
| 141 | e = result["estimates"] | ||
| 142 | print("Bitflash mining calculator") | ||
| 143 | print("schema: %s" % result["schema"]) | ||
| 144 | print() | ||
| 145 | print("Network") | ||
| 146 | print(" source: %s" % result["network_source"]) | ||
| 147 | print(" hashrate: %s" % i["network_hashrate"]) | ||
| 148 | print(" difficulty: %.8f" % i["difficulty"]) | ||
| 149 | print(" block time: %.0fs" % i["block_time_seconds"]) | ||
| 150 | print(" reward: %s BTF" % fmt_money(i["block_reward_btf"])) | ||
| 151 | print() | ||
| 152 | print("Miner") | ||
| 153 | print(" hashrate: %s" % i["miner_hashrate"]) | ||
| 154 | print(" share: %.8f%%" % (e["miner_network_share"] * 100.0)) | ||
| 155 | if math.isinf(e["expected_time_to_solo_block_seconds"]): | ||
| 156 | print(" solo block: never") | ||
| 157 | else: | ||
| 158 | print(" solo block: %.2f hours expected average" % e["expected_time_to_solo_block_hours"]) | ||
| 159 | print() | ||
| 160 | print("Economics") | ||
| 161 | print(" price: $%.8f / BTF" % i["price_per_btf"]) | ||
| 162 | print(" power: %.2f W" % i["power_watts"]) | ||
| 163 | print(" kWh cost: $%.4f" % i["electricity_per_kwh"]) | ||
| 164 | print(" pool fee: %.2f%%" % i["pool_fee_percent"]) | ||
| 165 | print() | ||
| 166 | print("Estimated rewards") | ||
| 167 | print(" %-9s %16s %12s %12s %12s" % ("period", "BTF", "revenue", "power", "profit")) | ||
| 168 | for name in ["hour", "day", "week", "month_30d"]: | ||
| 169 | row = e["periods"][name] | ||
| 170 | print(" %-9s %16s $%11.4f $%11.4f $%11.4f" % ( | ||
| 171 | name, | ||
| 172 | fmt_money(row["btf"]), | ||
| 173 | row["revenue"], | ||
| 174 | row["electricity_cost"], | ||
| 175 | row["profit"], | ||
| 176 | )) | ||
| 177 | print() | ||
| 178 | print("Note: this is an expectation, not a promise. Solo mining variance can be huge.") | ||
| 179 | |||
| 180 | |||
| 181 | def main(argv): | ||
| 182 | ap = argparse.ArgumentParser(description=__doc__) | ||
| 183 | ap.add_argument("--hashrate", required=True, help="miner hashrate number") | ||
| 184 | ap.add_argument("--hashrate-unit", default="kh/s", choices=sorted(UNITS), help="miner hashrate unit") | ||
| 185 | ap.add_argument("--network-hashrate", help="network hashrate number") | ||
| 186 | ap.add_argument("--network-hashrate-unit", default="kh/s", choices=sorted(UNITS), help="network hashrate unit") | ||
| 187 | ap.add_argument("--difficulty", type=float, help="Bitflash difficulty; diff 1 expects 4096 hashes") | ||
| 188 | ap.add_argument("--block-time", type=float, default=DEFAULT_BLOCK_TIME, help="target block time in seconds") | ||
| 189 | ap.add_argument("--block-reward", type=float, default=DEFAULT_BLOCK_REWARD, help="block reward in BTF") | ||
| 190 | ap.add_argument("--pool-fee", type=float, default=0.0, help="pool fee percent") | ||
| 191 | ap.add_argument("--price", type=float, default=0.0, help="price per BTF in USD or your chosen fiat unit") | ||
| 192 | ap.add_argument("--watts", type=float, default=0.0, help="miner or whole-system power in watts") | ||
| 193 | ap.add_argument("--kwh-cost", type=float, default=0.0, help="electricity cost per kWh") | ||
| 194 | ap.add_argument("--json", action="store_true", help="print JSON") | ||
| 195 | args = ap.parse_args(argv) | ||
| 196 | |||
| 197 | result = estimate(args) | ||
| 198 | if args.json: | ||
| 199 | print(json.dumps(result, sort_keys=True, indent=2)) | ||
| 200 | else: | ||
| 201 | print_text(result) | ||
| 202 | return 0 | ||
| 203 | |||
| 204 | |||
| 205 | if __name__ == "__main__": | ||
| 206 | raise SystemExit(main(sys.argv[1:])) | ||
diff --git a/scripts/bitflash_chain.py b/scripts/bitflash_chain.py new file mode 100644 index 0000000..4a3231c --- /dev/null +++ b/scripts/bitflash_chain.py | |||
| @@ -0,0 +1,450 @@ | |||
| 1 | #!/usr/bin/env python3 | ||
| 2 | """Shared Bitflash block/chain parser for deterministic audit tools.""" | ||
| 3 | |||
| 4 | import datetime as _dt | ||
| 5 | import hashlib | ||
| 6 | import json | ||
| 7 | import mmap | ||
| 8 | import os | ||
| 9 | import struct | ||
| 10 | from pathlib import Path | ||
| 11 | |||
| 12 | |||
| 13 | MAGIC = bytes([0xBF, 0x20, 0x5C, 0xFD]) | ||
| 14 | MAX_BLOCK_PAYLOAD = 32 * 1024 * 1024 | ||
| 15 | UTXO_LEAF_DOMAIN = "BTFUTXO1" | ||
| 16 | UTXO_NODE_DOMAIN_TAG = "BTFNODE1" | ||
| 17 | UTXO_NODE_DOMAIN = (UTXO_NODE_DOMAIN_TAG + "|").encode("ascii") | ||
| 18 | COIN = 100_000_000 | ||
| 19 | GENESIS_HASH = "5bd7cb255d814e48cebcdfb72da4dc87b34bd774227f8ceb546c5640f4bdc169" | ||
| 20 | GENESIS_MERKLE = "1a45b4482532abb29b10e234d3f13132230525a339ecea91658ffa675a5b1325" | ||
| 21 | GENESIS_TIME = 1_753_315_200 | ||
| 22 | GENESIS_BITS = 0x1F0FFFFF | ||
| 23 | GENESIS_NONCE = 3141 | ||
| 24 | GENESIS_TEXT = "Bitflash 24/Jul/2026 Fair launch: one CPU one vote, no premine" | ||
| 25 | |||
| 26 | |||
| 27 | class ParseError(Exception): | ||
| 28 | pass | ||
| 29 | |||
| 30 | |||
| 31 | class Reader: | ||
| 32 | def __init__(self, data): | ||
| 33 | self.data = data | ||
| 34 | self.pos = 0 | ||
| 35 | |||
| 36 | def take(self, n): | ||
| 37 | if self.pos + n > len(self.data): | ||
| 38 | raise ParseError("unexpected end of data") | ||
| 39 | out = self.data[self.pos:self.pos + n] | ||
| 40 | self.pos += n | ||
| 41 | return out | ||
| 42 | |||
| 43 | def u8(self): | ||
| 44 | return self.take(1)[0] | ||
| 45 | |||
| 46 | def u32(self): | ||
| 47 | return struct.unpack("<I", self.take(4))[0] | ||
| 48 | |||
| 49 | def i32(self): | ||
| 50 | return struct.unpack("<i", self.take(4))[0] | ||
| 51 | |||
| 52 | def i64(self): | ||
| 53 | return struct.unpack("<q", self.take(8))[0] | ||
| 54 | |||
| 55 | |||
| 56 | def sha256(data): | ||
| 57 | return hashlib.sha256(data).digest() | ||
| 58 | |||
| 59 | |||
| 60 | def sha256d(data): | ||
| 61 | return hashlib.sha256(hashlib.sha256(data).digest()).digest() | ||
| 62 | |||
| 63 | |||
| 64 | def hash_hex(serialized): | ||
| 65 | return sha256d(serialized)[::-1].hex() | ||
| 66 | |||
| 67 | |||
| 68 | def compact_size(r): | ||
| 69 | first = r.u8() | ||
| 70 | if first < 253: | ||
| 71 | return first | ||
| 72 | if first == 253: | ||
| 73 | return struct.unpack("<H", r.take(2))[0] | ||
| 74 | if first == 254: | ||
| 75 | return r.u32() | ||
| 76 | return struct.unpack("<Q", r.take(8))[0] | ||
| 77 | |||
| 78 | |||
| 79 | def read_varbytes(r): | ||
| 80 | return r.take(compact_size(r)) | ||
| 81 | |||
| 82 | |||
| 83 | def ser_compact_size(n): | ||
| 84 | if n < 253: | ||
| 85 | return bytes([n]) | ||
| 86 | if n <= 0xFFFF: | ||
| 87 | return b"\xfd" + struct.pack("<H", n) | ||
| 88 | if n <= 0xFFFFFFFF: | ||
| 89 | return b"\xfe" + struct.pack("<I", n) | ||
| 90 | return b"\xff" + struct.pack("<Q", n) | ||
| 91 | |||
| 92 | |||
| 93 | def ser_varbytes(b): | ||
| 94 | return ser_compact_size(len(b)) + b | ||
| 95 | |||
| 96 | |||
| 97 | def ser_tx(tx): | ||
| 98 | out = bytearray() | ||
| 99 | out += struct.pack("<i", tx["version"]) | ||
| 100 | out += ser_compact_size(len(tx["vin"])) | ||
| 101 | for txin in tx["vin"]: | ||
| 102 | out += bytes.fromhex(txin["prevout_hash"])[::-1] | ||
| 103 | out += struct.pack("<I", txin["prevout_n"]) | ||
| 104 | out += ser_varbytes(bytes.fromhex(txin["script_sig_hex"])) | ||
| 105 | out += struct.pack("<I", txin["sequence"]) | ||
| 106 | out += ser_compact_size(len(tx["vout"])) | ||
| 107 | for txout in tx["vout"]: | ||
| 108 | out += struct.pack("<q", txout["value_satoshis"]) | ||
| 109 | out += ser_varbytes(bytes.fromhex(txout["script_pub_key_hex"])) | ||
| 110 | out += struct.pack("<I", tx["lock_time"]) | ||
| 111 | return bytes(out) | ||
| 112 | |||
| 113 | |||
| 114 | def parse_tx(r): | ||
| 115 | start = r.pos | ||
| 116 | tx = {"version": r.i32(), "vin": [], "vout": [], "lock_time": 0} | ||
| 117 | for _ in range(compact_size(r)): | ||
| 118 | tx["vin"].append({ | ||
| 119 | "prevout_hash": r.take(32)[::-1].hex(), | ||
| 120 | "prevout_n": r.u32(), | ||
| 121 | "script_sig_hex": read_varbytes(r).hex(), | ||
| 122 | "sequence": r.u32(), | ||
| 123 | }) | ||
| 124 | for _ in range(compact_size(r)): | ||
| 125 | value = r.i64() | ||
| 126 | tx["vout"].append({ | ||
| 127 | "value_satoshis": value, | ||
| 128 | "value_btf": format_money(value), | ||
| 129 | "script_pub_key_hex": read_varbytes(r).hex(), | ||
| 130 | }) | ||
| 131 | tx["lock_time"] = r.u32() | ||
| 132 | raw = r.data[start:r.pos] | ||
| 133 | tx["txid"] = hash_hex(raw) | ||
| 134 | return tx | ||
| 135 | |||
| 136 | |||
| 137 | def merkle_root(txids): | ||
| 138 | if not txids: | ||
| 139 | return "0" * 64 | ||
| 140 | layer = [bytes.fromhex(x)[::-1] for x in txids] | ||
| 141 | while len(layer) > 1: | ||
| 142 | nxt = [] | ||
| 143 | for i in range(0, len(layer), 2): | ||
| 144 | left = layer[i] | ||
| 145 | right = layer[i + 1] if i + 1 < len(layer) else left | ||
| 146 | nxt.append(sha256d(left + right)) | ||
| 147 | layer = nxt | ||
| 148 | return layer[0][::-1].hex() | ||
| 149 | |||
| 150 | |||
| 151 | def parse_block(payload, file_name="", offset=0): | ||
| 152 | r = Reader(payload) | ||
| 153 | header = r.take(80) | ||
| 154 | version, = struct.unpack("<i", header[0:4]) | ||
| 155 | prev = header[4:36][::-1].hex() | ||
| 156 | merkle = header[36:68][::-1].hex() | ||
| 157 | ntime, bits, nonce = struct.unpack("<III", header[68:80]) | ||
| 158 | txs = [parse_tx(r) for _ in range(compact_size(r))] | ||
| 159 | if r.pos != len(payload): | ||
| 160 | raise ParseError("trailing bytes inside block") | ||
| 161 | txids = [tx["txid"] for tx in txs] | ||
| 162 | return { | ||
| 163 | "hash": hash_hex(header), | ||
| 164 | "version": version, | ||
| 165 | "previous_hash": prev, | ||
| 166 | "merkle_root": merkle, | ||
| 167 | "computed_merkle_root": merkle_root(txids), | ||
| 168 | "time": ntime, | ||
| 169 | "time_utc": utc(ntime), | ||
| 170 | "bits": "0x%08x" % bits, | ||
| 171 | "nonce": nonce, | ||
| 172 | "tx_count": len(txs), | ||
| 173 | "transactions": txs, | ||
| 174 | "source_file": file_name, | ||
| 175 | "source_offset": offset, | ||
| 176 | "size": len(payload), | ||
| 177 | } | ||
| 178 | |||
| 179 | |||
| 180 | def iter_block_files(datadir): | ||
| 181 | for path in sorted(Path(datadir).glob("blk*.dat")): | ||
| 182 | yield path | ||
| 183 | |||
| 184 | |||
| 185 | def iter_blocks_from_file(path): | ||
| 186 | path = Path(path) | ||
| 187 | with path.open("rb") as f: | ||
| 188 | f.seek(0, os.SEEK_END) | ||
| 189 | file_size = f.tell() | ||
| 190 | if file_size == 0: | ||
| 191 | return | ||
| 192 | f.seek(0) | ||
| 193 | mm = mmap.mmap(f.fileno(), 0, access=mmap.ACCESS_READ) | ||
| 194 | try: | ||
| 195 | pos = 0 | ||
| 196 | while True: | ||
| 197 | offset = mm.find(MAGIC, pos) | ||
| 198 | if offset < 0 or offset + 8 > file_size: | ||
| 199 | break | ||
| 200 | size = struct.unpack("<I", mm[offset + 4:offset + 8])[0] | ||
| 201 | payload_offset = offset + 8 | ||
| 202 | end = payload_offset + size | ||
| 203 | if size == 0 or size > MAX_BLOCK_PAYLOAD or end > file_size: | ||
| 204 | pos = offset + 1 | ||
| 205 | continue | ||
| 206 | payload = mm[payload_offset:end] | ||
| 207 | try: | ||
| 208 | yield parse_block(payload, path.name, payload_offset) | ||
| 209 | except ParseError: | ||
| 210 | pos = offset + 1 | ||
| 211 | continue | ||
| 212 | pos = end | ||
| 213 | finally: | ||
| 214 | mm.close() | ||
| 215 | |||
| 216 | |||
| 217 | def read_blocks_from_files(paths, max_blocks=0): | ||
| 218 | blocks = [] | ||
| 219 | for path in paths: | ||
| 220 | for block in iter_blocks_from_file(path): | ||
| 221 | blocks.append(block) | ||
| 222 | if max_blocks and len(blocks) >= max_blocks: | ||
| 223 | return blocks | ||
| 224 | return blocks | ||
| 225 | |||
| 226 | |||
| 227 | def read_blocks(datadir, max_blocks=0): | ||
| 228 | return read_blocks_from_files(iter_block_files(datadir), max_blocks) | ||
| 229 | |||
| 230 | |||
| 231 | def select_main_chain(blocks): | ||
| 232 | by_hash = {} | ||
| 233 | for block in blocks: | ||
| 234 | by_hash.setdefault(block["hash"], block) | ||
| 235 | if GENESIS_HASH not in by_hash: | ||
| 236 | raise ParseError("genesis block not found in block files") | ||
| 237 | |||
| 238 | children = {} | ||
| 239 | for block in by_hash.values(): | ||
| 240 | children.setdefault(block["previous_hash"], []).append(block) | ||
| 241 | for v in children.values(): | ||
| 242 | v.sort(key=lambda b: (b["hash"], b["source_file"], b["source_offset"])) | ||
| 243 | |||
| 244 | best = [] | ||
| 245 | stack = [(by_hash[GENESIS_HASH], [by_hash[GENESIS_HASH]])] | ||
| 246 | while stack: | ||
| 247 | block, path = stack.pop() | ||
| 248 | if len(path) > len(best) or (len(path) == len(best) and block["hash"] < best[-1]["hash"]): | ||
| 249 | best = path | ||
| 250 | for child in children.get(block["hash"], []): | ||
| 251 | stack.append((child, path + [child])) | ||
| 252 | return best | ||
| 253 | |||
| 254 | |||
| 255 | def is_coinbase(tx): | ||
| 256 | return ( | ||
| 257 | len(tx["vin"]) == 1 | ||
| 258 | and tx["vin"][0]["prevout_hash"] == "0" * 64 | ||
| 259 | and tx["vin"][0]["prevout_n"] == 0xFFFFFFFF | ||
| 260 | ) | ||
| 261 | |||
| 262 | |||
| 263 | def utxo_leaf_hash(outpoint, utxo): | ||
| 264 | payload = "|".join([ | ||
| 265 | UTXO_LEAF_DOMAIN, | ||
| 266 | outpoint, | ||
| 267 | str(utxo["value_satoshis"]), | ||
| 268 | utxo["script_pub_key_hex"], | ||
| 269 | str(utxo["created_height"]), | ||
| 270 | utxo["created_txid"], | ||
| 271 | ]).encode("ascii") | ||
| 272 | return sha256(payload) | ||
| 273 | |||
| 274 | |||
| 275 | def utxo_root(utxos): | ||
| 276 | leaves = [utxo_leaf_hash(k, utxos[k]) for k in sorted(utxos)] | ||
| 277 | if not leaves: | ||
| 278 | return sha256(b"BTFUTXO1|empty").hex() | ||
| 279 | layer = leaves | ||
| 280 | while len(layer) > 1: | ||
| 281 | nxt = [] | ||
| 282 | for i in range(0, len(layer), 2): | ||
| 283 | left = layer[i] | ||
| 284 | right = layer[i + 1] if i + 1 < len(layer) else left | ||
| 285 | nxt.append(sha256(UTXO_NODE_DOMAIN + left + right)) | ||
| 286 | layer = nxt | ||
| 287 | return layer[0].hex() | ||
| 288 | |||
| 289 | |||
| 290 | def apply_blocks(blocks, strict_duplicates=False): | ||
| 291 | utxos = {} | ||
| 292 | spent = 0 | ||
| 293 | missing_spends = [] | ||
| 294 | duplicate_outputs = [] | ||
| 295 | total_coinbase = 0 | ||
| 296 | total_fees = 0 | ||
| 297 | normal_txs = 0 | ||
| 298 | |||
| 299 | for height, block in enumerate(blocks): | ||
| 300 | if block["computed_merkle_root"] != block["merkle_root"]: | ||
| 301 | raise ParseError("bad merkle root at height %d" % height) | ||
| 302 | for tx in block["transactions"]: | ||
| 303 | coinbase = is_coinbase(tx) | ||
| 304 | input_value = 0 | ||
| 305 | output_value = sum(out["value_satoshis"] for out in tx["vout"]) | ||
| 306 | if coinbase: | ||
| 307 | total_coinbase += output_value | ||
| 308 | else: | ||
| 309 | normal_txs += 1 | ||
| 310 | for txin in tx["vin"]: | ||
| 311 | prev = "%s:%d" % (txin["prevout_hash"], txin["prevout_n"]) | ||
| 312 | prev_utxo = utxos.pop(prev, None) | ||
| 313 | if prev_utxo is None: | ||
| 314 | missing_spends.append({ | ||
| 315 | "height": height, | ||
| 316 | "txid": tx["txid"], | ||
| 317 | "missing_outpoint": prev, | ||
| 318 | }) | ||
| 319 | else: | ||
| 320 | spent += prev_utxo["value_satoshis"] | ||
| 321 | input_value += prev_utxo["value_satoshis"] | ||
| 322 | if input_value >= output_value: | ||
| 323 | total_fees += input_value - output_value | ||
| 324 | for n, out in enumerate(tx["vout"]): | ||
| 325 | key = "%s:%d" % (tx["txid"], n) | ||
| 326 | if key in utxos: | ||
| 327 | if strict_duplicates: | ||
| 328 | raise ParseError("duplicate output at height %d: %s" % (height, key)) | ||
| 329 | duplicate_outputs.append({"height": height, "outpoint": key}) | ||
| 330 | utxos[key] = { | ||
| 331 | "txid": tx["txid"], | ||
| 332 | "vout": n, | ||
| 333 | "value_satoshis": out["value_satoshis"], | ||
| 334 | "value_btf": out["value_btf"], | ||
| 335 | "script_pub_key_hex": out["script_pub_key_hex"], | ||
| 336 | "created_height": height, | ||
| 337 | "created_block_hash": block["hash"], | ||
| 338 | "created_txid": tx["txid"], | ||
| 339 | "coinbase": coinbase, | ||
| 340 | } | ||
| 341 | |||
| 342 | total_unspent = sum(u["value_satoshis"] for u in utxos.values()) | ||
| 343 | return { | ||
| 344 | "utxos": utxos, | ||
| 345 | "stats": { | ||
| 346 | "utxo_count": len(utxos), | ||
| 347 | "spent_outputs": spent, | ||
| 348 | "total_unspent_satoshis": total_unspent, | ||
| 349 | "total_unspent_btf": format_money(total_unspent), | ||
| 350 | "total_coinbase_satoshis": total_coinbase, | ||
| 351 | "total_coinbase_btf": format_money(total_coinbase), | ||
| 352 | "coinbase_minus_unspent_satoshis": total_coinbase - total_unspent, | ||
| 353 | "coinbase_minus_unspent_btf": format_money(total_coinbase - total_unspent), | ||
| 354 | "total_fees_satoshis": total_fees, | ||
| 355 | "total_fees_btf": format_money(total_fees), | ||
| 356 | "normal_transactions": normal_txs, | ||
| 357 | "missing_spends": missing_spends, | ||
| 358 | "duplicate_outputs": duplicate_outputs, | ||
| 359 | }, | ||
| 360 | } | ||
| 361 | |||
| 362 | |||
| 363 | def genesis_transaction(): | ||
| 364 | script_sig = ( | ||
| 365 | bytes.fromhex("04ffff001d") | ||
| 366 | + b"\x01\x04" | ||
| 367 | + bytes([len(GENESIS_TEXT)]) | ||
| 368 | + GENESIS_TEXT.encode("ascii") | ||
| 369 | ) | ||
| 370 | pubkey_num = bytes.fromhex( | ||
| 371 | "5f1df16b2b704c8a578d0bbaf74d385cde12c11ee50455f3c438ef4c3fbcf649" | ||
| 372 | "b6de611feae06279a60939e028a8d65c10b73071a6f16719274855feb0fd8a6704" | ||
| 373 | ) | ||
| 374 | script_pub_key = bytes([65]) + pubkey_num[::-1] + b"\xac" | ||
| 375 | tx = { | ||
| 376 | "version": 1, | ||
| 377 | "vin": [{ | ||
| 378 | "prevout_hash": "0" * 64, | ||
| 379 | "prevout_n": 0xFFFFFFFF, | ||
| 380 | "script_sig_hex": script_sig.hex(), | ||
| 381 | "sequence": 0xFFFFFFFF, | ||
| 382 | }], | ||
| 383 | "vout": [{ | ||
| 384 | "value_satoshis": 50 * COIN, | ||
| 385 | "value_btf": format_money(50 * COIN), | ||
| 386 | "script_pub_key_hex": script_pub_key.hex(), | ||
| 387 | }], | ||
| 388 | "lock_time": 0, | ||
| 389 | } | ||
| 390 | tx["txid"] = hash_hex(ser_tx(tx)) | ||
| 391 | return tx | ||
| 392 | |||
| 393 | |||
| 394 | def reconstructed_genesis(): | ||
| 395 | tx = genesis_transaction() | ||
| 396 | header = ( | ||
| 397 | struct.pack("<i", 1) | ||
| 398 | + bytes(32) | ||
| 399 | + bytes.fromhex(tx["txid"])[::-1] | ||
| 400 | + struct.pack("<III", GENESIS_TIME, GENESIS_BITS, GENESIS_NONCE) | ||
| 401 | ) | ||
| 402 | return { | ||
| 403 | "hash": hash_hex(header), | ||
| 404 | "version": 1, | ||
| 405 | "previous_hash": "0" * 64, | ||
| 406 | "merkle_root": tx["txid"], | ||
| 407 | "computed_merkle_root": tx["txid"], | ||
| 408 | "time": GENESIS_TIME, | ||
| 409 | "time_utc": utc(GENESIS_TIME), | ||
| 410 | "bits": "0x%08x" % GENESIS_BITS, | ||
| 411 | "nonce": GENESIS_NONCE, | ||
| 412 | "tx_count": 1, | ||
| 413 | "transactions": [tx], | ||
| 414 | "source_file": "reconstructed-from-source-constants", | ||
| 415 | "source_offset": 0, | ||
| 416 | "size": 80 + len(ser_compact_size(1)) + len(ser_tx(tx)), | ||
| 417 | } | ||
| 418 | |||
| 419 | |||
| 420 | def ser_block(block): | ||
| 421 | header = ( | ||
| 422 | struct.pack("<i", block["version"]) | ||
| 423 | + bytes.fromhex(block["previous_hash"])[::-1] | ||
| 424 | + bytes.fromhex(block["merkle_root"])[::-1] | ||
| 425 | + struct.pack("<III", block["time"], int(block["bits"], 16), block["nonce"]) | ||
| 426 | ) | ||
| 427 | txs = block["transactions"] | ||
| 428 | return header + ser_compact_size(len(txs)) + b"".join(ser_tx(tx) for tx in txs) | ||
| 429 | |||
| 430 | |||
| 431 | def format_money(value): | ||
| 432 | sign = "-" if value < 0 else "" | ||
| 433 | value = abs(value) | ||
| 434 | return "%s%d.%08d" % (sign, value // COIN, value % COIN) | ||
| 435 | |||
| 436 | |||
| 437 | def utc(ts): | ||
| 438 | return _dt.datetime.fromtimestamp(ts, _dt.timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ") | ||
| 439 | |||
| 440 | |||
| 441 | def default_datadir(): | ||
| 442 | if os.name == "nt" and os.environ.get("APPDATA"): | ||
| 443 | return os.path.join(os.environ["APPDATA"], "Bitflash") | ||
| 444 | if os.environ.get("HOME"): | ||
| 445 | return os.path.join(os.environ["HOME"], ".bitflash") | ||
| 446 | return "." | ||
| 447 | |||
| 448 | |||
| 449 | def canonical_bytes(report): | ||
| 450 | return (json.dumps(report, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii") | ||
diff --git a/scripts/build-explorer.py b/scripts/build-explorer.py new file mode 100755 index 0000000..97a7f4e --- /dev/null +++ b/scripts/build-explorer.py | |||
| @@ -0,0 +1,336 @@ | |||
| 1 | #!/usr/bin/env python3 | ||
| 2 | """Build a static Bitflash block explorer from local blk*.dat files.""" | ||
| 3 | |||
| 4 | import argparse | ||
| 5 | import hashlib | ||
| 6 | import json | ||
| 7 | import sys | ||
| 8 | from pathlib import Path | ||
| 9 | |||
| 10 | import bitflash_chain as chain | ||
| 11 | |||
| 12 | |||
| 13 | ADDRESS_VERSION = 25 | ||
| 14 | B58 = "123456789ABCDEFGHJKLMNPQRSTUVWXYZabcdefghijkmnopqrstuvwxyz" | ||
| 15 | |||
| 16 | |||
| 17 | def b58check(payload): | ||
| 18 | v = bytes([ADDRESS_VERSION]) + payload | ||
| 19 | v += chain.sha256d(v)[:4] | ||
| 20 | n = int.from_bytes(v, "big") | ||
| 21 | s = "" | ||
| 22 | while n > 0: | ||
| 23 | n, r = divmod(n, 58) | ||
| 24 | s = B58[r] + s | ||
| 25 | for byte in v: | ||
| 26 | if byte == 0: | ||
| 27 | s = "1" + s | ||
| 28 | else: | ||
| 29 | break | ||
| 30 | return s | ||
| 31 | |||
| 32 | |||
| 33 | def hash160(b): | ||
| 34 | return hashlib.new("ripemd160", hashlib.sha256(b).digest()).digest() | ||
| 35 | |||
| 36 | |||
| 37 | def spk_info(script_hex): | ||
| 38 | try: | ||
| 39 | spk = bytes.fromhex(script_hex) | ||
| 40 | except ValueError: | ||
| 41 | return {"type": "malformed", "hex": script_hex[:80]} | ||
| 42 | if ( | ||
| 43 | len(spk) == 25 and spk[0] == 0x76 and spk[1] == 0xA9 | ||
| 44 | and spk[2] == 0x14 and spk[23] == 0x88 and spk[24] == 0xAC | ||
| 45 | ): | ||
| 46 | return {"type": "p2pkh", "address": b58check(spk[3:23])} | ||
| 47 | if len(spk) >= 35 and spk[-1] == 0xAC: | ||
| 48 | if spk[0] == 0x41 and len(spk) == 67: | ||
| 49 | pk = spk[1:66] | ||
| 50 | return {"type": "p2pk", "pubkey": pk.hex(), "address": b58check(hash160(pk))} | ||
| 51 | if spk[0] == 0x21 and len(spk) == 35: | ||
| 52 | pk = spk[1:34] | ||
| 53 | return {"type": "p2pk", "pubkey": pk.hex(), "address": b58check(hash160(pk))} | ||
| 54 | return {"type": "other", "hex": script_hex[:80]} | ||
| 55 | |||
| 56 | |||
| 57 | def explorer_tx(tx): | ||
| 58 | vin = [] | ||
| 59 | for txin in tx["vin"]: | ||
| 60 | coinbase = ( | ||
| 61 | txin["prevout_hash"] == "0" * 64 | ||
| 62 | and txin["prevout_n"] == 0xFFFFFFFF | ||
| 63 | ) | ||
| 64 | vin.append({ | ||
| 65 | "coinbase": coinbase, | ||
| 66 | "prev": None if coinbase else txin["prevout_hash"], | ||
| 67 | "vout": None if coinbase else txin["prevout_n"], | ||
| 68 | }) | ||
| 69 | vout = [] | ||
| 70 | for out in tx["vout"]: | ||
| 71 | vout.append(dict({"value": out["value_satoshis"]}, **spk_info(out["script_pub_key_hex"]))) | ||
| 72 | return {"txid": tx["txid"], "vin": vin, "vout": vout, "size": len(chain.ser_tx(tx))} | ||
| 73 | |||
| 74 | |||
| 75 | def parse_args(argv): | ||
| 76 | ap = argparse.ArgumentParser(description=__doc__) | ||
| 77 | ap.add_argument("--datadir", help="Bitflash data directory containing blk*.dat") | ||
| 78 | ap.add_argument("--max-blocks", type=int, default=0, | ||
| 79 | help="number of blocks to scan; 0 means all block files") | ||
| 80 | ap.add_argument("paths", nargs="+", | ||
| 81 | help="with --datadir: <out-dir>; otherwise: <blk*.dat>... <out-dir>") | ||
| 82 | args = ap.parse_args(argv) | ||
| 83 | if args.datadir: | ||
| 84 | if len(args.paths) != 1: | ||
| 85 | raise SystemExit("usage with --datadir: build-explorer.py --datadir DATADIR <out-dir>") | ||
| 86 | args.block_files = None | ||
| 87 | args.out_dir = args.paths[0] | ||
| 88 | else: | ||
| 89 | if len(args.paths) < 2: | ||
| 90 | raise SystemExit("usage: build-explorer.py <blk*.dat>... <out-dir>") | ||
| 91 | args.block_files = args.paths[:-1] | ||
| 92 | args.out_dir = args.paths[-1] | ||
| 93 | return args | ||
| 94 | |||
| 95 | |||
| 96 | def load_blocks(args): | ||
| 97 | if args.datadir: | ||
| 98 | return chain.read_blocks(args.datadir, args.max_blocks) | ||
| 99 | return chain.read_blocks_from_files(args.block_files, args.max_blocks) | ||
| 100 | |||
| 101 | |||
| 102 | def build_explorer(raw_blocks, out_dir): | ||
| 103 | try: | ||
| 104 | blocks = chain.select_main_chain(raw_blocks) | ||
| 105 | except chain.ParseError as e: | ||
| 106 | raise SystemExit(str(e)) | ||
| 107 | out = Path(out_dir) | ||
| 108 | (out / "block").mkdir(parents=True, exist_ok=True) | ||
| 109 | |||
| 110 | summaries = [] | ||
| 111 | for height, block in enumerate(blocks): | ||
| 112 | txs = [explorer_tx(tx) for tx in block["transactions"]] | ||
| 113 | detail = { | ||
| 114 | "height": height, | ||
| 115 | "hash": block["hash"], | ||
| 116 | "prev": block["previous_hash"], | ||
| 117 | "merkleRoot": block["merkle_root"], | ||
| 118 | "version": block["version"], | ||
| 119 | "time": block["time"], | ||
| 120 | "bits": int(block["bits"], 16), | ||
| 121 | "nonce": block["nonce"], | ||
| 122 | "size": block["size"], | ||
| 123 | "txs": txs, | ||
| 124 | } | ||
| 125 | cb = txs[0]["vout"][0] if txs and txs[0]["vout"] else {} | ||
| 126 | summaries.append({ | ||
| 127 | "height": height, | ||
| 128 | "hash": block["hash"], | ||
| 129 | "time": block["time"], | ||
| 130 | "nTx": len(txs), | ||
| 131 | "size": block["size"], | ||
| 132 | "cbValue": cb.get("value", 0), | ||
| 133 | "cbAddr": cb.get("address", ""), | ||
| 134 | }) | ||
| 135 | (out / "block" / ("%d.json" % height)).write_text( | ||
| 136 | json.dumps(detail, sort_keys=True, separators=(",", ":")) + "\n", | ||
| 137 | encoding="ascii", | ||
| 138 | ) | ||
| 139 | |||
| 140 | summaries.sort(key=lambda x: -x["height"]) | ||
| 141 | (out / "blocks.json").write_text( | ||
| 142 | json.dumps({ | ||
| 143 | "network": "bitflash", | ||
| 144 | "tipHeight": len(blocks) - 1, | ||
| 145 | "count": len(blocks), | ||
| 146 | "blocks": summaries, | ||
| 147 | }, sort_keys=True, separators=(",", ":")) + "\n", | ||
| 148 | encoding="ascii", | ||
| 149 | ) | ||
| 150 | (out / "index.html").write_text(INDEX_HTML, encoding="utf-8") | ||
| 151 | return len(blocks), len(blocks) - 1 | ||
| 152 | |||
| 153 | |||
| 154 | def main(argv): | ||
| 155 | args = parse_args(argv) | ||
| 156 | raw_blocks = load_blocks(args) | ||
| 157 | if not raw_blocks: | ||
| 158 | raise SystemExit("no Bitflash blocks parsed") | ||
| 159 | count, tip = build_explorer(raw_blocks, args.out_dir) | ||
| 160 | print("explorer built: %d main-chain blocks, tip %d, into %s" % (count, tip, args.out_dir)) | ||
| 161 | return 0 | ||
| 162 | |||
| 163 | |||
| 164 | INDEX_HTML = r"""<!doctype html> | ||
| 165 | <html lang="en"> | ||
| 166 | <head> | ||
| 167 | <meta charset="utf-8"> | ||
| 168 | <meta name="viewport" content="width=device-width, initial-scale=1"> | ||
| 169 | <title>Bitflash Block Explorer</title> | ||
| 170 | <style> | ||
| 171 | :root { color-scheme: light dark; --bg:#fff; --fg:#111; --mut:#666; --line:#e2e2e2; --card:#fafafa; --acc:#2b6cb0; } | ||
| 172 | @media (prefers-color-scheme: dark){ :root{ --bg:#0e0f11; --fg:#e6e6e6; --mut:#9aa0a6; --line:#26282c; --card:#16181b; --acc:#63b3ed; } } | ||
| 173 | * { box-sizing: border-box; } | ||
| 174 | body { font: 15px/1.5 system-ui, sans-serif; margin:0; background:var(--bg); color:var(--fg); } | ||
| 175 | header { padding:20px 24px; border-bottom:1px solid var(--line); display:flex; gap:16px; align-items:baseline; flex-wrap:wrap; } | ||
| 176 | h1 { font-size:18px; margin:0; } | ||
| 177 | h2 { font-size:16px; } | ||
| 178 | .mut { color:var(--mut); } | ||
| 179 | .wrap { padding:16px 24px; max-width:1100px; margin:0 auto; } | ||
| 180 | input { width:100%; max-width:420px; padding:9px 12px; border:1px solid var(--line); border-radius:8px; background:var(--card); color:var(--fg); font-size:14px; } | ||
| 181 | table { width:100%; border-collapse:collapse; margin-top:12px; } | ||
| 182 | th,td { text-align:left; padding:9px 10px; border-bottom:1px solid var(--line); font-variant-numeric:tabular-nums; } | ||
| 183 | th { font-size:12px; text-transform:uppercase; letter-spacing:.05em; color:var(--mut); } | ||
| 184 | tr.blk { cursor:pointer; } | ||
| 185 | tr.blk:hover td { background:var(--card); } | ||
| 186 | code { font-size:12px; word-break:break-all; } | ||
| 187 | .scroll { overflow-x:auto; } | ||
| 188 | .card { border:1px solid var(--line); border-radius:10px; background:var(--card); padding:14px 16px; margin-top:12px; } | ||
| 189 | .kv { display:grid; grid-template-columns:150px 1fr; gap:4px 12px; font-size:13px; } | ||
| 190 | .kv div:nth-child(odd){ color:var(--mut); } | ||
| 191 | .tx { border-top:1px solid var(--line); padding:10px 0; } | ||
| 192 | .io { display:grid; grid-template-columns:1fr 1fr; gap:16px; } | ||
| 193 | @media (max-width:640px){ .io{ grid-template-columns:1fr; } .kv{ grid-template-columns:110px 1fr; } } | ||
| 194 | .pill { display:inline-block; font-size:11px; padding:1px 7px; border-radius:99px; border:1px solid var(--line); color:var(--mut); } | ||
| 195 | button.link { background:none; border:none; color:var(--acc); cursor:pointer; font:inherit; padding:0; } | ||
| 196 | .note { color:var(--mut); font-size:13px; max-width:820px; } | ||
| 197 | </style> | ||
| 198 | </head> | ||
| 199 | <body> | ||
| 200 | <header> | ||
| 201 | <h1>Bitflash Block Explorer</h1> | ||
| 202 | <span class="mut" id="tip"></span> | ||
| 203 | <span style="flex:1"></span> | ||
| 204 | <button class="link" id="home" style="display:none">back to blocks</button> | ||
| 205 | </header> | ||
| 206 | <div class="wrap"> | ||
| 207 | <p class="note">Read-only view of the main chain: blocks, their transactions, and the coinbase payout. There is no mempool and no per-address history here.</p> | ||
| 208 | <div id="listview"> | ||
| 209 | <input id="q" placeholder="Search by height or block hash"> | ||
| 210 | <div class="scroll"><table> | ||
| 211 | <thead><tr><th>Height</th><th>Time (UTC)</th><th>Txs</th><th>Size</th><th>Coinbase</th><th>To</th></tr></thead> | ||
| 212 | <tbody id="rows"></tbody> | ||
| 213 | </table></div> | ||
| 214 | <p class="mut" id="more"></p> | ||
| 215 | </div> | ||
| 216 | <div id="detail" style="display:none"></div> | ||
| 217 | </div> | ||
| 218 | <script> | ||
| 219 | var SUM = [], SHOWN = 200; | ||
| 220 | function btf(sat){ return (sat/1e8).toFixed(8).replace(/0+$/,'').replace(/\.$/,'')+" BTF"; } | ||
| 221 | function ts(t){ return new Date(t*1000).toISOString().replace('T',' ').replace('.000Z',' UTC'); } | ||
| 222 | function short(h){ return h ? h.slice(0,10)+"..."+h.slice(-6) : "-"; } | ||
| 223 | function E(tag, cls, text){ | ||
| 224 | var e = document.createElement(tag); | ||
| 225 | if(cls) e.className = cls; | ||
| 226 | if(text !== undefined) e.textContent = text; | ||
| 227 | return e; | ||
| 228 | } | ||
| 229 | function code(text){ return E('code', '', text); } | ||
| 230 | function clear(e){ while(e.firstChild) e.removeChild(e.firstChild); } | ||
| 231 | function append(parent){ | ||
| 232 | for(var i=1;i<arguments.length;i++) parent.appendChild(arguments[i]); | ||
| 233 | return parent; | ||
| 234 | } | ||
| 235 | function row(cells, cls){ | ||
| 236 | var tr = E('tr', cls || ''); | ||
| 237 | cells.forEach(function(c){ append(tr, E('td', '', c)); }); | ||
| 238 | return tr; | ||
| 239 | } | ||
| 240 | function renderList(filter){ | ||
| 241 | var q=(filter||"").trim().toLowerCase(), rows=document.getElementById('rows'); | ||
| 242 | clear(rows); | ||
| 243 | var list = q ? SUM.filter(function(b){ return String(b.height)===q || b.hash.indexOf(q)===0; }) : SUM.slice(0,SHOWN); | ||
| 244 | if(!list.length){ | ||
| 245 | var empty = E('tr'); | ||
| 246 | var td = E('td', 'mut', 'no match'); | ||
| 247 | td.colSpan = 6; | ||
| 248 | append(empty, td); append(rows, empty); | ||
| 249 | } | ||
| 250 | list.forEach(function(b){ | ||
| 251 | var tr = row([String(b.height), ts(b.time), String(b.nTx), b.size+' B', btf(b.cbValue), ''], 'blk'); | ||
| 252 | tr.dataset.h = b.height; | ||
| 253 | clear(tr.children[5]); | ||
| 254 | append(tr.children[5], code(b.cbAddr || '-')); | ||
| 255 | tr.onclick = function(){ openBlock(tr.dataset.h); }; | ||
| 256 | append(rows, tr); | ||
| 257 | }); | ||
| 258 | document.getElementById('more').textContent = (!q && SUM.length>SHOWN) ? ("showing latest "+SHOWN+" of "+SUM.length+" blocks; search to find any") : ""; | ||
| 259 | } | ||
| 260 | function kv(parent, key, value, isCode){ | ||
| 261 | append(parent, E('div', '', key)); | ||
| 262 | append(parent, isCode ? append(E('div'), code(value)) : E('div', '', value)); | ||
| 263 | } | ||
| 264 | function ioSide(title, items, render){ | ||
| 265 | var side = E('div'); | ||
| 266 | append(side, E('div', 'mut', title)); | ||
| 267 | items.forEach(function(item){ append(side, render(item)); }); | ||
| 268 | return side; | ||
| 269 | } | ||
| 270 | function openBlock(h){ | ||
| 271 | fetch('block/'+h+'.json').then(function(r){return r.json();}).then(function(b){ | ||
| 272 | var detail = document.getElementById('detail'); | ||
| 273 | clear(detail); | ||
| 274 | append(detail, E('h2', '', 'Block '+b.height)); | ||
| 275 | var card = E('div', 'card'), grid = E('div', 'kv'); | ||
| 276 | kv(grid, 'Height', String(b.height)); | ||
| 277 | kv(grid, 'Hash', b.hash, true); | ||
| 278 | var prev = E('div'); | ||
| 279 | if(b.height > 0){ | ||
| 280 | var btn = E('button', 'link'); | ||
| 281 | append(btn, code(short(b.prev))); | ||
| 282 | btn.onclick = function(){ openBlock(b.height - 1); }; | ||
| 283 | append(prev, btn); | ||
| 284 | } else { | ||
| 285 | append(prev, code(short(b.prev))); | ||
| 286 | } | ||
| 287 | append(grid, E('div', '', 'Previous'), prev); | ||
| 288 | kv(grid, 'Merkle root', b.merkleRoot, true); | ||
| 289 | kv(grid, 'Time (UTC)', ts(b.time)); | ||
| 290 | kv(grid, 'Bits / Nonce', b.bits.toString(16)+' / '+b.nonce); | ||
| 291 | kv(grid, 'Size / Txs', b.size+' B / '+b.txs.length); | ||
| 292 | append(card, grid); append(detail, card); | ||
| 293 | b.txs.forEach(function(t, i){ | ||
| 294 | var tx = E('div', 'tx'); | ||
| 295 | var head = E('div'); | ||
| 296 | append(head, code(t.txid)); | ||
| 297 | if(i === 0) append(head, E('span', 'pill', 'coinbase')); | ||
| 298 | var io = E('div', 'io'); | ||
| 299 | append(io, ioSide('Inputs', t.vin, function(v){ | ||
| 300 | var d = E('div'); | ||
| 301 | if(v.coinbase) append(d, E('span', 'pill', 'coinbase')); | ||
| 302 | else append(d, code(short(v.prev)+':'+v.vout)); | ||
| 303 | return d; | ||
| 304 | })); | ||
| 305 | append(io, ioSide('Outputs', t.vout, function(o){ | ||
| 306 | var d = E('div'); | ||
| 307 | var to = o.address ? o.address : (o.pubkey ? 'pubkey '+o.pubkey.slice(0,16)+'...' : o.type); | ||
| 308 | append(d, document.createTextNode(btf(o.value)+' -> '), code(to)); | ||
| 309 | return d; | ||
| 310 | })); | ||
| 311 | append(tx, head, io); append(detail, tx); | ||
| 312 | }); | ||
| 313 | document.getElementById('listview').style.display='none'; | ||
| 314 | detail.style.display='block'; | ||
| 315 | document.getElementById('home').style.display='inline'; | ||
| 316 | window.scrollTo(0,0); | ||
| 317 | }); | ||
| 318 | } | ||
| 319 | document.getElementById('home').onclick=function(){ | ||
| 320 | document.getElementById('detail').style.display='none'; | ||
| 321 | document.getElementById('listview').style.display='block'; | ||
| 322 | this.style.display='none'; | ||
| 323 | }; | ||
| 324 | document.getElementById('q').oninput=function(){ renderList(this.value); }; | ||
| 325 | fetch('blocks.json').then(function(r){return r.json();}).then(function(d){ | ||
| 326 | SUM=d.blocks; document.getElementById('tip').textContent='tip height '+d.tipHeight+' - '+d.count+' blocks'; | ||
| 327 | renderList(''); | ||
| 328 | }); | ||
| 329 | </script> | ||
| 330 | </body> | ||
| 331 | </html> | ||
| 332 | """ | ||
| 333 | |||
| 334 | |||
| 335 | if __name__ == "__main__": | ||
| 336 | raise SystemExit(main(sys.argv[1:])) | ||
diff --git a/scripts/build-pool-index.py b/scripts/build-pool-index.py new file mode 100644 index 0000000..d3eb386 --- /dev/null +++ b/scripts/build-pool-index.py | |||
| @@ -0,0 +1,174 @@ | |||
| 1 | #!/usr/bin/env python3 | ||
| 2 | # | ||
| 3 | # Build a static public pool directory from one or more Bitflash pool_status.json | ||
| 4 | # files. Inputs may be local paths or http(s) URLs. | ||
| 5 | |||
| 6 | import argparse | ||
| 7 | import html | ||
| 8 | import json | ||
| 9 | import os | ||
| 10 | import time | ||
| 11 | import urllib.parse | ||
| 12 | import urllib.request | ||
| 13 | |||
| 14 | |||
| 15 | def load_json(source): | ||
| 16 | if source.startswith("http://") or source.startswith("https://"): | ||
| 17 | with urllib.request.urlopen(source, timeout=10) as r: | ||
| 18 | return json.loads(r.read().decode("utf-8")) | ||
| 19 | with open(source, "r", encoding="utf-8-sig") as f: | ||
| 20 | return json.load(f) | ||
| 21 | |||
| 22 | |||
| 23 | def normalize_pool(source, obj, now): | ||
| 24 | operator = obj.get("operator", {}) | ||
| 25 | pool = obj.get("pool", {}) | ||
| 26 | node = obj.get("node", {}) | ||
| 27 | rounds_file = operator.get("roundsFile") or "pool_rounds.json" | ||
| 28 | rounds_url = "" | ||
| 29 | if source.startswith("http://") or source.startswith("https://"): | ||
| 30 | rounds_url = urllib.parse.urljoin(source, rounds_file) | ||
| 31 | updated = int(obj.get("updatedAt") or 0) | ||
| 32 | age = max(0, now - updated) if updated else None | ||
| 33 | online = bool(node.get("poolRunning")) and age is not None and age <= 120 | ||
| 34 | return { | ||
| 35 | "source": source, | ||
| 36 | "name": operator.get("name") or "Bitflash Pool", | ||
| 37 | "btfAddress": operator.get("btfAddress") or "", | ||
| 38 | "feePercent": float(operator.get("feePercent") or 0.0), | ||
| 39 | "dashboardUrl": operator.get("dashboardUrl") or "", | ||
| 40 | "height": int(node.get("height") or 0), | ||
| 41 | "online": online, | ||
| 42 | "ageSeconds": age, | ||
| 43 | "authorizedMiners": int(pool.get("authorizedMiners") or 0), | ||
| 44 | "blocksFoundSession": int(pool.get("blocksFoundSession") or 0), | ||
| 45 | "roundShares": int(pool.get("roundShares") or 0), | ||
| 46 | "hashRate": float(pool.get("hashRate") or 0.0), | ||
| 47 | "roundsUrl": rounds_url, | ||
| 48 | "recentRounds": obj.get("recentRounds", []), | ||
| 49 | } | ||
| 50 | |||
| 51 | |||
| 52 | def rank_key(pool): | ||
| 53 | return ( | ||
| 54 | 1 if pool["online"] else 0, | ||
| 55 | pool["hashRate"], | ||
| 56 | pool["authorizedMiners"], | ||
| 57 | -pool["feePercent"], | ||
| 58 | pool["name"].lower(), | ||
| 59 | ) | ||
| 60 | |||
| 61 | |||
| 62 | def render_html(pools, generated_at): | ||
| 63 | rows = [] | ||
| 64 | for p in pools: | ||
| 65 | dash = p["dashboardUrl"] | ||
| 66 | rounds = p.get("roundsUrl", "") | ||
| 67 | # Only linkify http(s). The dashboard URL comes from an operator's | ||
| 68 | # self-published status file, so a "javascript:" or "data:" scheme | ||
| 69 | # would become a clickable script-injection on this public page. | ||
| 70 | dash_lower = dash.lower() | ||
| 71 | if dash_lower.startswith("http://") or dash_lower.startswith("https://"): | ||
| 72 | dash_html = '<a href="{0}" rel="noopener noreferrer">dashboard</a>'.format( | ||
| 73 | html.escape(dash, quote=True)) | ||
| 74 | else: | ||
| 75 | dash_html = "-" | ||
| 76 | rounds_lower = rounds.lower() | ||
| 77 | if rounds_lower.startswith("http://") or rounds_lower.startswith("https://"): | ||
| 78 | rounds_html = '<a href="{0}" rel="noopener noreferrer">proof</a>'.format( | ||
| 79 | html.escape(rounds, quote=True)) | ||
| 80 | else: | ||
| 81 | rounds_html = "-" | ||
| 82 | rows.append( | ||
| 83 | "<tr>" | ||
| 84 | "<td>{status}</td>" | ||
| 85 | "<td><strong>{name}</strong><br><code>{addr}</code></td>" | ||
| 86 | "<td>{fee:.2f}%</td>" | ||
| 87 | "<td>{miners}</td>" | ||
| 88 | "<td>{hashrate:.2f}</td>" | ||
| 89 | "<td>{blocks}</td>" | ||
| 90 | "<td>{height}</td>" | ||
| 91 | "<td>{age}</td>" | ||
| 92 | "<td>{dash}</td>" | ||
| 93 | "<td>{rounds}</td>" | ||
| 94 | "</tr>".format( | ||
| 95 | status="online" if p["online"] else "stale", | ||
| 96 | name=html.escape(p["name"]), | ||
| 97 | addr=html.escape(p["btfAddress"]), | ||
| 98 | fee=p["feePercent"], | ||
| 99 | miners=p["authorizedMiners"], | ||
| 100 | hashrate=p["hashRate"], | ||
| 101 | blocks=p["blocksFoundSession"], | ||
| 102 | height=p["height"], | ||
| 103 | age="{}s".format(p["ageSeconds"]) if p["ageSeconds"] is not None else "-", | ||
| 104 | dash=dash_html, | ||
| 105 | rounds=rounds_html, | ||
| 106 | ) | ||
| 107 | ) | ||
| 108 | |||
| 109 | return """<!doctype html> | ||
| 110 | <html lang="en"> | ||
| 111 | <head> | ||
| 112 | <meta charset="utf-8"> | ||
| 113 | <meta name="viewport" content="width=device-width, initial-scale=1"> | ||
| 114 | <title>Bitflash Pools</title> | ||
| 115 | <style> | ||
| 116 | body {{ font: 15px/1.45 system-ui, sans-serif; margin: 32px; color: #111; }} | ||
| 117 | table {{ border-collapse: collapse; width: 100%; }} | ||
| 118 | th, td {{ border-bottom: 1px solid #ddd; padding: 10px; text-align: left; vertical-align: top; }} | ||
| 119 | th {{ font-size: 12px; text-transform: uppercase; letter-spacing: .08em; }} | ||
| 120 | code {{ font-size: 12px; word-break: break-all; }} | ||
| 121 | .note {{ color: #555; max-width: 780px; }} | ||
| 122 | </style> | ||
| 123 | </head> | ||
| 124 | <body> | ||
| 125 | <h1>Bitflash Pools</h1> | ||
| 126 | <p class="note">This directory is informational. Bitflash nodes discover pools through Nostr and .btf; this page only helps humans compare public operators.</p> | ||
| 127 | <table> | ||
| 128 | <thead> | ||
| 129 | <tr><th>Status</th><th>Pool</th><th>Fee</th><th>Miners</th><th>H/s</th><th>Blocks</th><th>Height</th><th>Age</th><th>Link</th><th>Proof</th></tr> | ||
| 130 | </thead> | ||
| 131 | <tbody> | ||
| 132 | {rows} | ||
| 133 | </tbody> | ||
| 134 | </table> | ||
| 135 | <p class="note">Generated at {generated_at}. JSON: <a href="pools.json">pools.json</a>.</p> | ||
| 136 | </body> | ||
| 137 | </html> | ||
| 138 | """.format(rows="\n ".join(rows), generated_at=generated_at) | ||
| 139 | |||
| 140 | |||
| 141 | def main(): | ||
| 142 | ap = argparse.ArgumentParser() | ||
| 143 | ap.add_argument("--out-dir", default="pool-site") | ||
| 144 | ap.add_argument("sources", nargs="+", help="pool_status.json paths or URLs") | ||
| 145 | args = ap.parse_args() | ||
| 146 | |||
| 147 | now = int(time.time()) | ||
| 148 | pools = [] | ||
| 149 | errors = [] | ||
| 150 | for source in args.sources: | ||
| 151 | try: | ||
| 152 | pools.append(normalize_pool(source, load_json(source), now)) | ||
| 153 | except Exception as e: | ||
| 154 | errors.append({"source": source, "error": str(e)}) | ||
| 155 | |||
| 156 | pools.sort(key=rank_key, reverse=True) | ||
| 157 | os.makedirs(args.out_dir, exist_ok=True) | ||
| 158 | |||
| 159 | generated_at = time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime(now)) | ||
| 160 | with open(os.path.join(args.out_dir, "pools.json"), "w", encoding="utf-8") as f: | ||
| 161 | json.dump({ | ||
| 162 | "schema": "bitflash-pool-directory-1", | ||
| 163 | "generatedAt": generated_at, | ||
| 164 | "pools": pools, | ||
| 165 | "errors": errors, | ||
| 166 | }, f, indent=2, sort_keys=True) | ||
| 167 | f.write("\n") | ||
| 168 | |||
| 169 | with open(os.path.join(args.out_dir, "index.html"), "w", encoding="utf-8") as f: | ||
| 170 | f.write(render_html(pools, generated_at)) | ||
| 171 | |||
| 172 | |||
| 173 | if __name__ == "__main__": | ||
| 174 | main() | ||
diff --git a/scripts/prove-utxo.py b/scripts/prove-utxo.py new file mode 100644 index 0000000..105e677 --- /dev/null +++ b/scripts/prove-utxo.py | |||
| @@ -0,0 +1,186 @@ | |||
| 1 | #!/usr/bin/env python3 | ||
| 2 | """Create a Bitflash UTXO inclusion proof from local blk*.dat files. | ||
| 3 | |||
| 4 | The proof is deliberately outside consensus. It proves that one txid:vout is | ||
| 5 | included in the deterministic UTXO root produced by verify-utxo-set.py. | ||
| 6 | """ | ||
| 7 | |||
| 8 | import argparse | ||
| 9 | import hashlib | ||
| 10 | import json | ||
| 11 | import sys | ||
| 12 | from pathlib import Path | ||
| 13 | |||
| 14 | import bitflash_chain as chain | ||
| 15 | |||
| 16 | SCHEMA = "bitflash-utxo-inclusion-proof-1" | ||
| 17 | |||
| 18 | |||
| 19 | def canonical_bytes(obj): | ||
| 20 | return (json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii") | ||
| 21 | |||
| 22 | |||
| 23 | def split_outpoint(outpoint): | ||
| 24 | parts = outpoint.split(":") | ||
| 25 | if len(parts) != 2 or len(parts[0]) != 64: | ||
| 26 | raise SystemExit("outpoint must be txid:vout") | ||
| 27 | try: | ||
| 28 | vout = int(parts[1], 10) | ||
| 29 | except ValueError: | ||
| 30 | raise SystemExit("vout must be a decimal integer") | ||
| 31 | if vout < 0: | ||
| 32 | raise SystemExit("vout must not be negative") | ||
| 33 | return parts[0].lower(), vout | ||
| 34 | |||
| 35 | |||
| 36 | def build_merkle_proof(keys, leaf_hashes, index): | ||
| 37 | current_index = index | ||
| 38 | layer = list(leaf_hashes) | ||
| 39 | proof = [] | ||
| 40 | level = 0 | ||
| 41 | while len(layer) > 1: | ||
| 42 | if current_index % 2 == 0: | ||
| 43 | sibling_index = current_index + 1 | ||
| 44 | duplicated = sibling_index >= len(layer) | ||
| 45 | sibling = layer[current_index] if duplicated else layer[sibling_index] | ||
| 46 | side = "right" | ||
| 47 | else: | ||
| 48 | sibling_index = current_index - 1 | ||
| 49 | duplicated = False | ||
| 50 | sibling = layer[sibling_index] | ||
| 51 | side = "left" | ||
| 52 | proof.append({ | ||
| 53 | "level": level, | ||
| 54 | "side": side, | ||
| 55 | "hash": sibling.hex(), | ||
| 56 | "duplicated": duplicated, | ||
| 57 | }) | ||
| 58 | |||
| 59 | next_layer = [] | ||
| 60 | for i in range(0, len(layer), 2): | ||
| 61 | left = layer[i] | ||
| 62 | right = layer[i + 1] if i + 1 < len(layer) else left | ||
| 63 | next_layer.append(hashlib.sha256(chain.UTXO_NODE_DOMAIN + left + right).digest()) | ||
| 64 | layer = next_layer | ||
| 65 | current_index //= 2 | ||
| 66 | level += 1 | ||
| 67 | return proof, layer[0].hex() | ||
| 68 | |||
| 69 | |||
| 70 | def build_proof(args): | ||
| 71 | txid, vout = split_outpoint(args.outpoint) | ||
| 72 | outpoint = "%s:%d" % (txid, vout) | ||
| 73 | |||
| 74 | raw_blocks = chain.read_blocks(args.datadir, args.max_blocks) | ||
| 75 | if not raw_blocks: | ||
| 76 | raise SystemExit("no Bitflash blocks found in %s" % args.datadir) | ||
| 77 | try: | ||
| 78 | blocks = chain.select_main_chain(raw_blocks) | ||
| 79 | state = chain.apply_blocks(blocks, strict_duplicates=args.strict_duplicates) | ||
| 80 | except chain.ParseError as e: | ||
| 81 | raise SystemExit(str(e)) | ||
| 82 | utxos = state["utxos"] | ||
| 83 | stats = state["stats"] | ||
| 84 | if outpoint not in utxos: | ||
| 85 | raise SystemExit("outpoint is not unspent in the scanned chain: %s" % outpoint) | ||
| 86 | |||
| 87 | keys = sorted(utxos) | ||
| 88 | leaf_hashes = [chain.utxo_leaf_hash(k, utxos[k]) for k in keys] | ||
| 89 | index = keys.index(outpoint) | ||
| 90 | proof_path, root = build_merkle_proof(keys, leaf_hashes, index) | ||
| 91 | direct_root = chain.utxo_root(utxos) | ||
| 92 | if root != direct_root: | ||
| 93 | raise SystemExit("internal proof root mismatch") | ||
| 94 | |||
| 95 | tip = blocks[-1] | ||
| 96 | utxo = utxos[outpoint] | ||
| 97 | report = { | ||
| 98 | "schema": SCHEMA, | ||
| 99 | "deterministic": True, | ||
| 100 | "chain": { | ||
| 101 | "name": "Bitflash", | ||
| 102 | "ticker": "BTF", | ||
| 103 | "message_start_hex": chain.MAGIC.hex(), | ||
| 104 | "coin": chain.COIN, | ||
| 105 | }, | ||
| 106 | "commitment": { | ||
| 107 | "algorithm": "sorted-outpoint-merkle-sha256", | ||
| 108 | "leaf_domain": chain.UTXO_LEAF_DOMAIN, | ||
| 109 | "node_domain": chain.UTXO_NODE_DOMAIN_TAG, | ||
| 110 | "root": root, | ||
| 111 | "leaf_count": len(keys), | ||
| 112 | }, | ||
| 113 | "target": { | ||
| 114 | "outpoint": outpoint, | ||
| 115 | "txid": txid, | ||
| 116 | "vout": vout, | ||
| 117 | "leaf_index": index, | ||
| 118 | "leaf_hash": leaf_hashes[index].hex(), | ||
| 119 | "utxo": utxo, | ||
| 120 | }, | ||
| 121 | "proof": proof_path, | ||
| 122 | "scanned": { | ||
| 123 | "raw_block_count": len(raw_blocks), | ||
| 124 | "side_blocks_ignored": len(raw_blocks) - len(blocks), | ||
| 125 | "main_chain_block_count": len(blocks), | ||
| 126 | "first_height": 0, | ||
| 127 | "last_height": len(blocks) - 1, | ||
| 128 | "tip_hash": tip["hash"], | ||
| 129 | "tip_time": tip["time"], | ||
| 130 | "tip_time_utc": tip["time_utc"], | ||
| 131 | }, | ||
| 132 | "warnings": { | ||
| 133 | "legacy_duplicate_output_overwrites": len(stats["duplicate_outputs"]), | ||
| 134 | }, | ||
| 135 | "verdict": "UTXO_INCLUSION_PROOF_CREATED", | ||
| 136 | } | ||
| 137 | return report | ||
| 138 | |||
| 139 | |||
| 140 | def print_text(proof): | ||
| 141 | data_hash = hashlib.sha256(canonical_bytes(proof)).hexdigest() | ||
| 142 | target = proof["target"] | ||
| 143 | utxo = target["utxo"] | ||
| 144 | print("Bitflash UTXO inclusion proof") | ||
| 145 | print("schema: %s" % proof["schema"]) | ||
| 146 | print("canonical proof sha256: %s" % data_hash) | ||
| 147 | print() | ||
| 148 | print("Target") | ||
| 149 | print(" outpoint: %s" % target["outpoint"]) | ||
| 150 | print(" value: %s BTF" % utxo["value_btf"]) | ||
| 151 | print(" height: %d" % utxo["created_height"]) | ||
| 152 | print(" index: %d of %d" % (target["leaf_index"], proof["commitment"]["leaf_count"])) | ||
| 153 | print() | ||
| 154 | print("Commitment") | ||
| 155 | print(" root: %s" % proof["commitment"]["root"]) | ||
| 156 | print(" path: %d sibling hashes" % len(proof["proof"])) | ||
| 157 | print(" tip: %s" % proof["scanned"]["tip_hash"]) | ||
| 158 | print() | ||
| 159 | print("Verdict: %s" % proof["verdict"]) | ||
| 160 | |||
| 161 | |||
| 162 | def main(argv): | ||
| 163 | ap = argparse.ArgumentParser(description=__doc__) | ||
| 164 | ap.add_argument("outpoint", help="unspent output to prove, as txid:vout") | ||
| 165 | ap.add_argument("--datadir", default=chain.default_datadir(), help="Bitflash data directory") | ||
| 166 | ap.add_argument("--max-blocks", type=int, default=0, | ||
| 167 | help="number of blocks to scan; 0 means all block files") | ||
| 168 | ap.add_argument("--strict-duplicates", action="store_true", | ||
| 169 | help="fail instead of using legacy last-write-wins duplicate output handling") | ||
| 170 | ap.add_argument("--json", action="store_true", help="print canonical JSON") | ||
| 171 | ap.add_argument("--out", help="write canonical JSON proof to this path") | ||
| 172 | args = ap.parse_args(argv) | ||
| 173 | |||
| 174 | proof = build_proof(args) | ||
| 175 | data = canonical_bytes(proof) | ||
| 176 | if args.out: | ||
| 177 | Path(args.out).write_bytes(data) | ||
| 178 | if args.json: | ||
| 179 | sys.stdout.buffer.write(data) | ||
| 180 | else: | ||
| 181 | print_text(proof) | ||
| 182 | return 0 | ||
| 183 | |||
| 184 | |||
| 185 | if __name__ == "__main__": | ||
| 186 | raise SystemExit(main(sys.argv[1:])) | ||
diff --git a/scripts/verify-fair-launch.py b/scripts/verify-fair-launch.py new file mode 100644 index 0000000..c33c8be --- /dev/null +++ b/scripts/verify-fair-launch.py | |||
| @@ -0,0 +1,166 @@ | |||
| 1 | #!/usr/bin/env python3 | ||
| 2 | """Produce a deterministic Bitflash fair-launch audit report.""" | ||
| 3 | |||
| 4 | import argparse | ||
| 5 | import hashlib | ||
| 6 | import sys | ||
| 7 | from pathlib import Path | ||
| 8 | |||
| 9 | import bitflash_chain as chain | ||
| 10 | |||
| 11 | |||
| 12 | def build_report(blocks, source_mode): | ||
| 13 | genesis = blocks[0] | ||
| 14 | launch_outputs = genesis["transactions"][0]["vout"] if genesis["transactions"] else [] | ||
| 15 | coinbase_outputs = [] | ||
| 16 | total_mined = 0 | ||
| 17 | non_coinbase_txs = 0 | ||
| 18 | |||
| 19 | for height, block in enumerate(blocks): | ||
| 20 | if not block["transactions"]: | ||
| 21 | continue | ||
| 22 | cb = block["transactions"][0] | ||
| 23 | if chain.is_coinbase(cb): | ||
| 24 | amount = sum(out["value_satoshis"] for out in cb["vout"]) | ||
| 25 | total_mined += amount | ||
| 26 | coinbase_outputs.append({ | ||
| 27 | "height": height, | ||
| 28 | "block_hash": block["hash"], | ||
| 29 | "coinbase_txid": cb["txid"], | ||
| 30 | "value_satoshis": amount, | ||
| 31 | "value_btf": chain.format_money(amount), | ||
| 32 | "outputs": cb["vout"], | ||
| 33 | }) | ||
| 34 | non_coinbase_txs += sum(1 for tx in block["transactions"] if not chain.is_coinbase(tx)) | ||
| 35 | |||
| 36 | checks = { | ||
| 37 | "genesis_hash_matches_source": genesis["hash"] == chain.GENESIS_HASH, | ||
| 38 | "genesis_merkle_matches_source": genesis["merkle_root"] == chain.GENESIS_MERKLE, | ||
| 39 | "computed_merkle_matches_header": genesis["computed_merkle_root"] == genesis["merkle_root"], | ||
| 40 | "genesis_has_single_transaction": genesis["tx_count"] == 1, | ||
| 41 | "genesis_transaction_is_coinbase": bool(genesis["transactions"] and chain.is_coinbase(genesis["transactions"][0])), | ||
| 42 | "genesis_supply_is_50_btf": sum(out["value_satoshis"] for out in launch_outputs) == 50 * chain.COIN, | ||
| 43 | } | ||
| 44 | |||
| 45 | report = { | ||
| 46 | "schema": "bitflash-fair-launch-report-1", | ||
| 47 | "deterministic": True, | ||
| 48 | "source_mode": source_mode, | ||
| 49 | "chain": { | ||
| 50 | "name": "Bitflash", | ||
| 51 | "ticker": "BTF", | ||
| 52 | "message_start_hex": chain.MAGIC.hex(), | ||
| 53 | "coin": chain.COIN, | ||
| 54 | }, | ||
| 55 | "source_constants": { | ||
| 56 | "genesis_hash": chain.GENESIS_HASH, | ||
| 57 | "genesis_merkle_root": chain.GENESIS_MERKLE, | ||
| 58 | "genesis_time": chain.GENESIS_TIME, | ||
| 59 | "genesis_time_utc": chain.utc(chain.GENESIS_TIME), | ||
| 60 | "genesis_bits": "0x%08x" % chain.GENESIS_BITS, | ||
| 61 | "genesis_nonce": chain.GENESIS_NONCE, | ||
| 62 | "genesis_message": chain.GENESIS_TEXT, | ||
| 63 | }, | ||
| 64 | "scanned": { | ||
| 65 | "block_count": len(blocks), | ||
| 66 | "first_height": 0, | ||
| 67 | "last_height": len(blocks) - 1, | ||
| 68 | "non_coinbase_transactions": non_coinbase_txs, | ||
| 69 | "total_coinbase_satoshis": total_mined, | ||
| 70 | "total_coinbase_btf": chain.format_money(total_mined), | ||
| 71 | }, | ||
| 72 | "genesis": { | ||
| 73 | "hash": genesis["hash"], | ||
| 74 | "merkle_root": genesis["merkle_root"], | ||
| 75 | "computed_merkle_root": genesis["computed_merkle_root"], | ||
| 76 | "time": genesis["time"], | ||
| 77 | "time_utc": genesis["time_utc"], | ||
| 78 | "bits": genesis["bits"], | ||
| 79 | "nonce": genesis["nonce"], | ||
| 80 | "tx_count": genesis["tx_count"], | ||
| 81 | "coinbase_txid": genesis["transactions"][0]["txid"] if genesis["transactions"] else "", | ||
| 82 | "outputs": launch_outputs, | ||
| 83 | }, | ||
| 84 | "coinbase_outputs": coinbase_outputs, | ||
| 85 | "checks": checks, | ||
| 86 | "verdict": "FAIR_LAUNCH_BASELINE_VERIFIED" if all(checks.values()) else "CHECKS_FAILED", | ||
| 87 | "external_timestamping": { | ||
| 88 | "status": "not_embedded", | ||
| 89 | "recommended": "OpenTimestamps", | ||
| 90 | "instructions": [ | ||
| 91 | "Save the canonical JSON report.", | ||
| 92 | "Run: ots stamp bitflash-fair-launch-report.json", | ||
| 93 | "Later verify with: ots verify bitflash-fair-launch-report.json.ots", | ||
| 94 | ], | ||
| 95 | }, | ||
| 96 | } | ||
| 97 | report["attestation_payload_sha256"] = hashlib.sha256(chain.canonical_bytes(report)).hexdigest() | ||
| 98 | return report | ||
| 99 | |||
| 100 | |||
| 101 | def print_text(report): | ||
| 102 | checks = report["checks"] | ||
| 103 | print("Bitflash fair-launch verifier") | ||
| 104 | print("schema: %s" % report["schema"]) | ||
| 105 | print("source: %s" % report["source_mode"]) | ||
| 106 | print("attestation payload sha256: %s" % report["attestation_payload_sha256"]) | ||
| 107 | print() | ||
| 108 | print("Genesis") | ||
| 109 | print(" hash: %s" % report["genesis"]["hash"]) | ||
| 110 | print(" merkle: %s" % report["genesis"]["merkle_root"]) | ||
| 111 | print(" time: %s" % report["genesis"]["time_utc"]) | ||
| 112 | print(" txs: %d" % report["genesis"]["tx_count"]) | ||
| 113 | print(" supply: %s BTF" % chain.format_money(sum(o["value_satoshis"] for o in report["genesis"]["outputs"]))) | ||
| 114 | print() | ||
| 115 | print("Scanned") | ||
| 116 | print(" blocks: %d" % report["scanned"]["block_count"]) | ||
| 117 | print(" height: %d..%d" % (report["scanned"]["first_height"], report["scanned"]["last_height"])) | ||
| 118 | print(" minted: %s BTF" % report["scanned"]["total_coinbase_btf"]) | ||
| 119 | print(" normal transactions in range: %d" % report["scanned"]["non_coinbase_transactions"]) | ||
| 120 | print() | ||
| 121 | print("Checks") | ||
| 122 | for key in sorted(checks): | ||
| 123 | print(" %s %s" % ("OK " if checks[key] else "FAIL", key)) | ||
| 124 | print() | ||
| 125 | print("Verdict: %s" % report["verdict"]) | ||
| 126 | print() | ||
| 127 | print("OpenTimestamps") | ||
| 128 | print(" canonical JSON can be stamped later with: ots stamp <report.json>") | ||
| 129 | |||
| 130 | |||
| 131 | def main(argv): | ||
| 132 | ap = argparse.ArgumentParser(description=__doc__) | ||
| 133 | ap.add_argument("--datadir", default=chain.default_datadir(), help="Bitflash data directory") | ||
| 134 | ap.add_argument("--max-blocks", type=int, default=100, help="number of blocks to scan from blk*.dat") | ||
| 135 | ap.add_argument("--json", action="store_true", help="print canonical JSON") | ||
| 136 | ap.add_argument("--out", help="write canonical JSON to this path") | ||
| 137 | ap.add_argument("--reconstruct-genesis", action="store_true", | ||
| 138 | help="do not read disk; reconstruct only the genesis block from source constants") | ||
| 139 | args = ap.parse_args(argv) | ||
| 140 | |||
| 141 | if args.reconstruct_genesis: | ||
| 142 | blocks = [chain.reconstructed_genesis()] | ||
| 143 | source_mode = "reconstructed-genesis" | ||
| 144 | else: | ||
| 145 | blocks = chain.read_blocks(args.datadir, args.max_blocks) | ||
| 146 | if not blocks: | ||
| 147 | raise SystemExit("no Bitflash blocks found in %s; use --reconstruct-genesis for a source-only check" % args.datadir) | ||
| 148 | source_mode = "local-block-files" | ||
| 149 | |||
| 150 | try: | ||
| 151 | report = build_report(blocks, source_mode) | ||
| 152 | except chain.ParseError as e: | ||
| 153 | raise SystemExit(str(e)) | ||
| 154 | data = chain.canonical_bytes(report) | ||
| 155 | |||
| 156 | if args.out: | ||
| 157 | Path(args.out).write_bytes(data) | ||
| 158 | if args.json: | ||
| 159 | sys.stdout.buffer.write(data) | ||
| 160 | else: | ||
| 161 | print_text(report) | ||
| 162 | return 0 if report["verdict"] == "FAIR_LAUNCH_BASELINE_VERIFIED" else 1 | ||
| 163 | |||
| 164 | |||
| 165 | if __name__ == "__main__": | ||
| 166 | raise SystemExit(main(sys.argv[1:])) | ||
diff --git a/scripts/verify-release.sh b/scripts/verify-release.sh index 23e71d1..4fd4558 100755 --- a/scripts/verify-release.sh +++ b/scripts/verify-release.sh | |||
| @@ -1,23 +1,50 @@ | |||
| 1 | #!/usr/bin/env bash | 1 | #!/usr/bin/env bash |
| 2 | set -euo pipefail | 2 | set -euo pipefail |
| 3 | 3 | ||
| 4 | repo="Bitflash-sh/bitflash" | 4 | # Verifies a Bitflash release: the checksum file's signature against a pinned |
| 5 | tag="${1:-latest}" | 5 | # key, then the assets against the checksums. |
| 6 | # | ||
| 7 | # The signing key is embedded below, not fetched. That is the whole point of a | ||
| 8 | # pinned key: the trust anchor ships with this script (in git), so it does not | ||
| 9 | # depend on the same server that serves the release. A server that is | ||
| 10 | # compromised can swap the binaries and the checksums, but it cannot forge a | ||
| 11 | # signature by this key. | ||
| 12 | |||
| 13 | host="https://releases.bitflash.network" | ||
| 6 | require_signature=0 | 14 | require_signature=0 |
| 7 | 15 | ||
| 16 | # The Bitflash release signing key. Verification requires this exact key -- | ||
| 17 | # gpg's own "good signature" is not enough, because a good signature by *some* | ||
| 18 | # key proves nothing. See issue #138. | ||
| 19 | RELEASE_KEY_FP="910A2B4CCA879E81FB4B2AEA1D4A53D3B78AA4B8" | ||
| 20 | read -r -d '' RELEASE_KEY <<'KEY' || true | ||
| 21 | -----BEGIN PGP PUBLIC KEY BLOCK----- | ||
| 22 | |||
| 23 | mDMEanSOmRYJKwYBBAHaRw8BAQdAXj24i2BYUcAmOpoXwjE4ZTynHsifCZ+niwQK | ||
| 24 | HqO0nMa0UEJpdGZsYXNoIFJlbGVhc2VzIChiaXRmbGFzaC5uZXR3b3JrIHJlbGVh | ||
| 25 | c2Ugc2lnbmluZykgPHJlbGVhc2VzQGJpdGZsYXNoLm5ldHdvcms+iJAEExYKADgW | ||
| 26 | IQSRCitMyoeegftLKuodSlPTt4qkuAUCanSOmQIbAwULCQgHAgYVCgkICwIEFgID | ||
| 27 | AQIeAQIXgAAKCRAdSlPTt4qkuIi9AP96X3OcstiCTTgKOr16Vhds3LyuZ19q3Apq | ||
| 28 | 4KAcNuOUMAD+NqgJ1CqkLGPXfZLCtYBCRHXNYWxMMEk+ViLcRtdLMg4= | ||
| 29 | =mxic | ||
| 30 | -----END PGP PUBLIC KEY BLOCK----- | ||
| 31 | KEY | ||
| 32 | |||
| 8 | usage() { | 33 | usage() { |
| 9 | cat <<'EOF' | 34 | cat <<'EOF' |
| 10 | Usage: scripts/verify-release.sh [tag|latest] [--require-signature] | 35 | Usage: scripts/verify-release.sh [tag|latest] [--require-signature] |
| 11 | 36 | ||
| 12 | Downloads the release checksum file and every asset named in it, then verifies: | 37 | Downloads the release checksum file and every asset named in it, then verifies: |
| 13 | 1. SHA256SUMS.asc, if present, against SHA256SUMS with gpg. | 38 | 1. SHA256SUMS.asc against SHA256SUMS, requiring the pinned Bitflash release |
| 39 | key (fingerprint 910A 2B4C CA87 9E81 FB4B 2AEA 1D4A 53D3 B78A A4B8). | ||
| 14 | 2. SHA256SUMS against the downloaded assets. | 40 | 2. SHA256SUMS against the downloaded assets. |
| 15 | 41 | ||
| 16 | Use --require-signature for release audits. Without it, older releases that | 42 | Releases from before signing was introduced ship only SHA256SUMS; without |
| 17 | ship only SHA256SUMS are checked for integrity and reported as unsigned. | 43 | --require-signature those are checked for integrity and reported as unsigned. |
| 18 | EOF | 44 | EOF |
| 19 | } | 45 | } |
| 20 | 46 | ||
| 47 | tag="${1:-latest}" | ||
| 21 | for arg in "${@:2}"; do | 48 | for arg in "${@:2}"; do |
| 22 | case "$arg" in | 49 | case "$arg" in |
| 23 | --require-signature) require_signature=1 ;; | 50 | --require-signature) require_signature=1 ;; |
| @@ -25,19 +52,9 @@ for arg in "${@:2}"; do | |||
| 25 | *) echo "unknown option: $arg" >&2; usage >&2; exit 2 ;; | 52 | *) echo "unknown option: $arg" >&2; usage >&2; exit 2 ;; |
| 26 | esac | 53 | esac |
| 27 | done | 54 | done |
| 55 | if [ "$tag" = "-h" ] || [ "$tag" = "--help" ]; then usage; exit 0; fi | ||
| 28 | 56 | ||
| 29 | if [ "$tag" = "-h" ] || [ "$tag" = "--help" ]; then | 57 | need() { command -v "$1" >/dev/null 2>&1 || { echo "missing required command: $1" >&2; exit 1; }; } |
| 30 | usage | ||
| 31 | exit 0 | ||
| 32 | fi | ||
| 33 | |||
| 34 | need() { | ||
| 35 | if ! command -v "$1" >/dev/null 2>&1; then | ||
| 36 | echo "missing required command: $1" >&2 | ||
| 37 | exit 1 | ||
| 38 | fi | ||
| 39 | } | ||
| 40 | |||
| 41 | need sed | 58 | need sed |
| 42 | need awk | 59 | need awk |
| 43 | 60 | ||
| @@ -46,39 +63,25 @@ if command -v sha256sum >/dev/null 2>&1; then | |||
| 46 | elif command -v shasum >/dev/null 2>&1; then | 63 | elif command -v shasum >/dev/null 2>&1; then |
| 47 | sha_check=(shasum -a 256 -c SHA256SUMS) | 64 | sha_check=(shasum -a 256 -c SHA256SUMS) |
| 48 | else | 65 | else |
| 49 | echo "missing required command: sha256sum or shasum" >&2 | 66 | echo "missing required command: sha256sum or shasum" >&2; exit 1 |
| 50 | exit 1 | ||
| 51 | fi | 67 | fi |
| 52 | 68 | ||
| 53 | download() { | 69 | download() { |
| 54 | local url="$1" | 70 | if command -v curl >/dev/null 2>&1; then curl -fsSL "$1" -o "$2" |
| 55 | local out="$2" | 71 | elif command -v wget >/dev/null 2>&1; then wget -q "$1" -O "$2" |
| 56 | if command -v curl >/dev/null 2>&1; then | 72 | else echo "missing required command: curl or wget" >&2; exit 1; fi |
| 57 | curl -fsSL "$url" -o "$out" | ||
| 58 | elif command -v wget >/dev/null 2>&1; then | ||
| 59 | wget -q "$url" -O "$out" | ||
| 60 | else | ||
| 61 | echo "missing required command: curl or wget" >&2 | ||
| 62 | exit 1 | ||
| 63 | fi | ||
| 64 | } | 73 | } |
| 65 | 74 | ||
| 66 | if [ "$tag" = "latest" ]; then | 75 | if [ "$tag" = "latest" ]; then |
| 67 | tag="$(download "https://api.github.com/repos/$repo/releases/latest" - | | 76 | tag="$(download "$host/latest.txt" - 2>/dev/null | tr -d '[:space:]' || true)" |
| 68 | sed -n 's/.*"tag_name"[[:space:]]*:[[:space:]]*"\([^"]*\)".*/\1/p' | | 77 | [ -n "$tag" ] || { echo "could not resolve the latest release tag from $host/latest.txt" >&2; exit 1; } |
| 69 | head -1)" | ||
| 70 | if [ -z "$tag" ]; then | ||
| 71 | echo "could not resolve latest release tag" >&2 | ||
| 72 | exit 1 | ||
| 73 | fi | ||
| 74 | fi | 78 | fi |
| 75 | 79 | ||
| 76 | workdir="$(mktemp -d "${TMPDIR:-/tmp}/bitflash-release-verify.XXXXXX")" | 80 | workdir="$(mktemp -d "${TMPDIR:-/tmp}/bitflash-release-verify.XXXXXX")" |
| 77 | cleanup() { rm -rf "$workdir"; } | 81 | cleanup() { rm -rf "$workdir"; } |
| 78 | trap cleanup EXIT | 82 | trap cleanup EXIT |
| 79 | 83 | ||
| 80 | base_url="https://github.com/$repo/releases/download/$tag" | 84 | base_url="$host/$tag" |
| 81 | |||
| 82 | echo "Verifying Bitflash $tag" | 85 | echo "Verifying Bitflash $tag" |
| 83 | echo "Working directory: $workdir" | 86 | echo "Working directory: $workdir" |
| 84 | cd "$workdir" | 87 | cd "$workdir" |
| @@ -86,27 +89,32 @@ cd "$workdir" | |||
| 86 | download "$base_url/SHA256SUMS" SHA256SUMS | 89 | download "$base_url/SHA256SUMS" SHA256SUMS |
| 87 | 90 | ||
| 88 | if download "$base_url/SHA256SUMS.asc" SHA256SUMS.asc 2>/dev/null; then | 91 | if download "$base_url/SHA256SUMS.asc" SHA256SUMS.asc 2>/dev/null; then |
| 89 | if ! command -v gpg >/dev/null 2>&1; then | 92 | need gpg |
| 90 | echo "SHA256SUMS.asc exists, but gpg is not installed" >&2 | 93 | # Verify in a throwaway keyring holding ONLY the pinned key, and require that |
| 94 | # the signature validates against it with the pinned fingerprint. This is the | ||
| 95 | # difference between "signed" and "signed by Bitflash". | ||
| 96 | keyring="$workdir/keyring" | ||
| 97 | mkdir -p "$keyring"; chmod 700 "$keyring" | ||
| 98 | printf '%s\n' "$RELEASE_KEY" | GNUPGHOME="$keyring" gpg --batch --quiet --import 2>/dev/null | ||
| 99 | status="$(GNUPGHOME="$keyring" gpg --batch --status-fd=1 --verify SHA256SUMS.asc SHA256SUMS 2>/dev/null || true)" | ||
| 100 | if printf '%s' "$status" | grep -q "VALIDSIG.*$RELEASE_KEY_FP"; then | ||
| 101 | echo "Signature OK -- signed by the pinned Bitflash release key" | ||
| 102 | else | ||
| 103 | echo "SIGNATURE REJECTED: not a valid signature by the pinned release key" >&2 | ||
| 104 | echo " expected key: $RELEASE_KEY_FP" >&2 | ||
| 91 | exit 1 | 105 | exit 1 |
| 92 | fi | 106 | fi |
| 93 | gpg --verify SHA256SUMS.asc SHA256SUMS | ||
| 94 | echo "Signature OK" | ||
| 95 | else | 107 | else |
| 96 | if [ "$require_signature" -eq 1 ]; then | 108 | if [ "$require_signature" -eq 1 ]; then |
| 97 | echo "release does not publish SHA256SUMS.asc" >&2 | 109 | echo "release does not publish SHA256SUMS.asc" >&2; exit 1 |
| 98 | exit 1 | ||
| 99 | fi | 110 | fi |
| 100 | echo "WARNING: no SHA256SUMS.asc found; checking hashes only" | 111 | echo "WARNING: no SHA256SUMS.asc found; checking hashes only (unsigned release)" |
| 101 | fi | 112 | fi |
| 102 | 113 | ||
| 103 | awk '{print $2}' SHA256SUMS | while IFS= read -r file; do | 114 | awk '{print $2}' SHA256SUMS | while IFS= read -r file; do |
| 104 | file="${file#\*}" | 115 | file="${file#\*}"; file="${file#./}" |
| 105 | file="${file#./}" | ||
| 106 | [ -z "$file" ] && continue | 116 | [ -z "$file" ] && continue |
| 107 | case "$file" in | 117 | case "$file" in */*|..*) echo "refusing unexpected checksum path: $file" >&2; exit 1 ;; esac |
| 108 | */*|..*) echo "refusing unexpected checksum path: $file" >&2; exit 1 ;; | ||
| 109 | esac | ||
| 110 | echo "Downloading $file" | 118 | echo "Downloading $file" |
| 111 | download "$base_url/$file" "$file" | 119 | download "$base_url/$file" "$file" |
| 112 | done | 120 | done |
diff --git a/scripts/verify-utxo-proof.py b/scripts/verify-utxo-proof.py new file mode 100644 index 0000000..50d9356 --- /dev/null +++ b/scripts/verify-utxo-proof.py | |||
| @@ -0,0 +1,146 @@ | |||
| 1 | #!/usr/bin/env python3 | ||
| 2 | """Verify a Bitflash UTXO inclusion proof against a UTXO root.""" | ||
| 3 | |||
| 4 | import argparse | ||
| 5 | import hashlib | ||
| 6 | import json | ||
| 7 | import sys | ||
| 8 | from pathlib import Path | ||
| 9 | |||
| 10 | import bitflash_chain as chain | ||
| 11 | |||
| 12 | SCHEMA = "bitflash-utxo-inclusion-proof-1" | ||
| 13 | |||
| 14 | |||
| 15 | def canonical_bytes(obj): | ||
| 16 | return (json.dumps(obj, sort_keys=True, separators=(",", ":"), ensure_ascii=True) + "\n").encode("ascii") | ||
| 17 | |||
| 18 | |||
| 19 | def load_proof(path): | ||
| 20 | try: | ||
| 21 | return json.loads(Path(path).read_text(encoding="ascii")) | ||
| 22 | except Exception as e: | ||
| 23 | raise SystemExit("could not read proof: %s" % e) | ||
| 24 | |||
| 25 | |||
| 26 | def verify(proof, expected_root=None): | ||
| 27 | failures = [] | ||
| 28 | if proof.get("schema") != SCHEMA: | ||
| 29 | failures.append("unexpected schema") | ||
| 30 | commitment = proof.get("commitment", {}) | ||
| 31 | if commitment.get("algorithm") != "sorted-outpoint-merkle-sha256": | ||
| 32 | failures.append("unexpected commitment algorithm") | ||
| 33 | if commitment.get("leaf_domain") != chain.UTXO_LEAF_DOMAIN: | ||
| 34 | failures.append("unexpected leaf domain") | ||
| 35 | if commitment.get("node_domain") != chain.UTXO_NODE_DOMAIN_TAG: | ||
| 36 | failures.append("unexpected node domain") | ||
| 37 | |||
| 38 | target = proof.get("target", {}) | ||
| 39 | outpoint = target.get("outpoint", "") | ||
| 40 | utxo = target.get("utxo", {}) | ||
| 41 | if outpoint != "%s:%s" % (target.get("txid"), target.get("vout")): | ||
| 42 | failures.append("target outpoint does not match txid:vout") | ||
| 43 | |||
| 44 | try: | ||
| 45 | current = chain.utxo_leaf_hash(outpoint, utxo) | ||
| 46 | leaf_ok = target.get("leaf_hash") == current.hex() | ||
| 47 | except Exception as e: | ||
| 48 | failures.append("could not hash target leaf: %s" % e) | ||
| 49 | current = b"\x00" * 32 | ||
| 50 | leaf_ok = False | ||
| 51 | |||
| 52 | if not leaf_ok: | ||
| 53 | failures.append("target leaf hash mismatch") | ||
| 54 | |||
| 55 | for i, step in enumerate(proof.get("proof", [])): | ||
| 56 | try: | ||
| 57 | sibling = bytes.fromhex(step["hash"]) | ||
| 58 | except Exception: | ||
| 59 | failures.append("bad sibling hash at proof level %d" % i) | ||
| 60 | sibling = b"\x00" * 32 | ||
| 61 | if len(sibling) != 32: | ||
| 62 | failures.append("bad sibling hash length at proof level %d" % i) | ||
| 63 | side = step.get("side") | ||
| 64 | if side == "right": | ||
| 65 | current = chain.sha256(chain.UTXO_NODE_DOMAIN + current + sibling) | ||
| 66 | elif side == "left": | ||
| 67 | current = chain.sha256(chain.UTXO_NODE_DOMAIN + sibling + current) | ||
| 68 | else: | ||
| 69 | failures.append("bad sibling side at proof level %d" % i) | ||
| 70 | |||
| 71 | computed_root = current.hex() | ||
| 72 | root = expected_root or commitment.get("root") | ||
| 73 | if computed_root != root: | ||
| 74 | failures.append("computed root does not match expected root") | ||
| 75 | |||
| 76 | return { | ||
| 77 | "schema": "bitflash-utxo-proof-verification-1", | ||
| 78 | "deterministic": True, | ||
| 79 | "target": { | ||
| 80 | "outpoint": outpoint, | ||
| 81 | "value_satoshis": utxo.get("value_satoshis"), | ||
| 82 | "value_btf": utxo.get("value_btf"), | ||
| 83 | "created_height": utxo.get("created_height"), | ||
| 84 | }, | ||
| 85 | "commitment": { | ||
| 86 | "expected_root": root, | ||
| 87 | "computed_root": computed_root, | ||
| 88 | "leaf_hash": target.get("leaf_hash"), | ||
| 89 | "path_length": len(proof.get("proof", [])), | ||
| 90 | }, | ||
| 91 | "checks": { | ||
| 92 | "schema_is_supported": proof.get("schema") == SCHEMA, | ||
| 93 | "leaf_hash_matches_target": leaf_ok, | ||
| 94 | "root_matches": computed_root == root, | ||
| 95 | }, | ||
| 96 | "failures": failures, | ||
| 97 | "verdict": "UTXO_PROOF_VERIFIED" if not failures else "UTXO_PROOF_FAILED", | ||
| 98 | } | ||
| 99 | |||
| 100 | |||
| 101 | def print_text(result): | ||
| 102 | data_hash = hashlib.sha256(canonical_bytes(result)).hexdigest() | ||
| 103 | print("Bitflash UTXO proof verifier") | ||
| 104 | print("schema: %s" % result["schema"]) | ||
| 105 | print("canonical verification sha256: %s" % data_hash) | ||
| 106 | print() | ||
| 107 | print("Target") | ||
| 108 | print(" outpoint: %s" % result["target"]["outpoint"]) | ||
| 109 | print(" value: %s BTF" % result["target"]["value_btf"]) | ||
| 110 | print(" height: %s" % result["target"]["created_height"]) | ||
| 111 | print() | ||
| 112 | print("Commitment") | ||
| 113 | print(" expected: %s" % result["commitment"]["expected_root"]) | ||
| 114 | print(" computed: %s" % result["commitment"]["computed_root"]) | ||
| 115 | print(" path: %d sibling hashes" % result["commitment"]["path_length"]) | ||
| 116 | print() | ||
| 117 | print("Checks") | ||
| 118 | for key in sorted(result["checks"]): | ||
| 119 | print(" %s %s" % ("OK " if result["checks"][key] else "FAIL", key)) | ||
| 120 | if result["failures"]: | ||
| 121 | print() | ||
| 122 | print("Failures") | ||
| 123 | for failure in result["failures"]: | ||
| 124 | print(" - %s" % failure) | ||
| 125 | print() | ||
| 126 | print("Verdict: %s" % result["verdict"]) | ||
| 127 | |||
| 128 | |||
| 129 | def main(argv): | ||
| 130 | ap = argparse.ArgumentParser(description=__doc__) | ||
| 131 | ap.add_argument("proof", help="canonical JSON proof from prove-utxo.py") | ||
| 132 | ap.add_argument("--root", help="expected UTXO root; defaults to root embedded in proof") | ||
| 133 | ap.add_argument("--json", action="store_true", help="print canonical JSON") | ||
| 134 | args = ap.parse_args(argv) | ||
| 135 | |||
| 136 | result = verify(load_proof(args.proof), args.root) | ||
| 137 | data = canonical_bytes(result) | ||
| 138 | if args.json: | ||
| 139 | sys.stdout.buffer.write(data) | ||
| 140 | else: | ||
| 141 | print_text(result) | ||
| 142 | return 0 if result["verdict"] == "UTXO_PROOF_VERIFIED" else 1 | ||
| 143 | |||
| 144 | |||
| 145 | if __name__ == "__main__": | ||
| 146 | raise SystemExit(main(sys.argv[1:])) | ||
diff --git a/scripts/verify-utxo-set.py b/scripts/verify-utxo-set.py new file mode 100644 index 0000000..b5bf3d5 --- /dev/null +++ b/scripts/verify-utxo-set.py | |||
| @@ -0,0 +1,131 @@ | |||
| 1 | #!/usr/bin/env python3 | ||
| 2 | """Build a deterministic Bitflash UTXO-set commitment from local blk*.dat.""" | ||
| 3 | |||
| 4 | import argparse | ||
| 5 | import hashlib | ||
| 6 | import sys | ||
| 7 | from pathlib import Path | ||
| 8 | |||
| 9 | import bitflash_chain as chain | ||
| 10 | |||
| 11 | |||
| 12 | def build_report(raw_blocks, include_utxos=False, strict_duplicates=False): | ||
| 13 | blocks = chain.select_main_chain(raw_blocks) | ||
| 14 | state = chain.apply_blocks(blocks, strict_duplicates=strict_duplicates) | ||
| 15 | utxos = state["utxos"] | ||
| 16 | stats = state["stats"] | ||
| 17 | tip = blocks[-1] | ||
| 18 | root = chain.utxo_root(utxos) | ||
| 19 | checks = { | ||
| 20 | "genesis_hash_matches_source": blocks[0]["hash"] == chain.GENESIS_HASH, | ||
| 21 | "genesis_merkle_matches_source": blocks[0]["merkle_root"] == chain.GENESIS_MERKLE, | ||
| 22 | "all_scanned_merkle_roots_match": all(b["computed_merkle_root"] == b["merkle_root"] for b in blocks), | ||
| 23 | "no_missing_spends": len(stats["missing_spends"]) == 0, | ||
| 24 | } | ||
| 25 | warnings = { | ||
| 26 | "legacy_duplicate_output_overwrites": len(stats["duplicate_outputs"]), | ||
| 27 | } | ||
| 28 | report = { | ||
| 29 | "schema": "bitflash-utxo-set-commitment-1", | ||
| 30 | "deterministic": True, | ||
| 31 | "chain": { | ||
| 32 | "name": "Bitflash", | ||
| 33 | "ticker": "BTF", | ||
| 34 | "message_start_hex": chain.MAGIC.hex(), | ||
| 35 | "coin": chain.COIN, | ||
| 36 | }, | ||
| 37 | "commitment": { | ||
| 38 | "algorithm": "sorted-outpoint-merkle-sha256", | ||
| 39 | "leaf_domain": "BTFUTXO1", | ||
| 40 | "node_domain": "BTFNODE1", | ||
| 41 | "root": root, | ||
| 42 | }, | ||
| 43 | "scanned": { | ||
| 44 | "raw_block_count": len(raw_blocks), | ||
| 45 | "side_blocks_ignored": len(raw_blocks) - len(blocks), | ||
| 46 | "main_chain_block_count": len(blocks), | ||
| 47 | "first_height": 0, | ||
| 48 | "last_height": len(blocks) - 1, | ||
| 49 | "tip_hash": tip["hash"], | ||
| 50 | "tip_time": tip["time"], | ||
| 51 | "tip_time_utc": tip["time_utc"], | ||
| 52 | }, | ||
| 53 | "stats": stats, | ||
| 54 | "checks": checks, | ||
| 55 | "warnings": warnings, | ||
| 56 | "verdict": "UTXO_COMMITMENT_VERIFIED" if all(checks.values()) else "CHECKS_FAILED", | ||
| 57 | } | ||
| 58 | if include_utxos: | ||
| 59 | report["utxos"] = {k: utxos[k] for k in sorted(utxos)} | ||
| 60 | return report | ||
| 61 | |||
| 62 | |||
| 63 | def print_text(report): | ||
| 64 | data_hash = hashlib.sha256(chain.canonical_bytes(report)).hexdigest() | ||
| 65 | print("Bitflash UTXO-set commitment verifier") | ||
| 66 | print("schema: %s" % report["schema"]) | ||
| 67 | print("canonical report sha256: %s" % data_hash) | ||
| 68 | print() | ||
| 69 | print("Commitment") | ||
| 70 | print(" root: %s" % report["commitment"]["root"]) | ||
| 71 | print(" algo: %s" % report["commitment"]["algorithm"]) | ||
| 72 | print() | ||
| 73 | print("Scanned") | ||
| 74 | print(" main chain blocks: %d" % report["scanned"]["main_chain_block_count"]) | ||
| 75 | print(" raw disk blocks: %d" % report["scanned"]["raw_block_count"]) | ||
| 76 | print(" side ignored: %d" % report["scanned"]["side_blocks_ignored"]) | ||
| 77 | print(" height: %d..%d" % (report["scanned"]["first_height"], report["scanned"]["last_height"])) | ||
| 78 | print(" tip: %s" % report["scanned"]["tip_hash"]) | ||
| 79 | print() | ||
| 80 | print("Supply") | ||
| 81 | print(" utxos: %d" % report["stats"]["utxo_count"]) | ||
| 82 | print(" supply: %s BTF" % report["stats"]["total_unspent_btf"]) | ||
| 83 | print(" mined: %s BTF" % report["stats"]["total_coinbase_btf"]) | ||
| 84 | print(" delta: %s BTF" % report["stats"]["coinbase_minus_unspent_btf"]) | ||
| 85 | print(" fees: %s BTF" % report["stats"]["total_fees_btf"]) | ||
| 86 | if report["warnings"]["legacy_duplicate_output_overwrites"]: | ||
| 87 | print(" legacy duplicate overwrites: %d" % report["warnings"]["legacy_duplicate_output_overwrites"]) | ||
| 88 | print() | ||
| 89 | print("Checks") | ||
| 90 | for key in sorted(report["checks"]): | ||
| 91 | print(" %s %s" % ("OK " if report["checks"][key] else "FAIL", key)) | ||
| 92 | print() | ||
| 93 | print("Verdict: %s" % report["verdict"]) | ||
| 94 | print() | ||
| 95 | print("OpenTimestamps") | ||
| 96 | print(" canonical JSON can be stamped later with: ots stamp <report.json>") | ||
| 97 | |||
| 98 | |||
| 99 | def main(argv): | ||
| 100 | ap = argparse.ArgumentParser(description=__doc__) | ||
| 101 | ap.add_argument("--datadir", default=chain.default_datadir(), help="Bitflash data directory") | ||
| 102 | ap.add_argument("--max-blocks", type=int, default=0, | ||
| 103 | help="number of blocks to scan; 0 means all block files") | ||
| 104 | ap.add_argument("--include-utxos", action="store_true", | ||
| 105 | help="include the full sorted UTXO map in JSON output") | ||
| 106 | ap.add_argument("--strict-duplicates", action="store_true", | ||
| 107 | help="fail instead of using legacy last-write-wins duplicate output handling") | ||
| 108 | ap.add_argument("--json", action="store_true", help="print canonical JSON") | ||
| 109 | ap.add_argument("--out", help="write canonical JSON to this path") | ||
| 110 | args = ap.parse_args(argv) | ||
| 111 | |||
| 112 | blocks = chain.read_blocks(args.datadir, args.max_blocks) | ||
| 113 | if not blocks: | ||
| 114 | raise SystemExit("no Bitflash blocks found in %s" % args.datadir) | ||
| 115 | try: | ||
| 116 | report = build_report(blocks, include_utxos=args.include_utxos, | ||
| 117 | strict_duplicates=args.strict_duplicates) | ||
| 118 | except chain.ParseError as e: | ||
| 119 | raise SystemExit(str(e)) | ||
| 120 | data = chain.canonical_bytes(report) | ||
| 121 | if args.out: | ||
| 122 | Path(args.out).write_bytes(data) | ||
| 123 | if args.json: | ||
| 124 | sys.stdout.buffer.write(data) | ||
| 125 | else: | ||
| 126 | print_text(report) | ||
| 127 | return 0 if report["verdict"] == "UTXO_COMMITMENT_VERIFIED" else 1 | ||
| 128 | |||
| 129 | |||
| 130 | if __name__ == "__main__": | ||
| 131 | raise SystemExit(main(sys.argv[1:])) | ||
diff --git a/src/Makefile b/src/Makefile index 171afe0..1166daf 100644 --- a/src/Makefile +++ b/src/Makefile | |||
| @@ -2,6 +2,7 @@ | |||
| 2 | # Called by: make linux (root Makefile) | 2 | # Called by: make linux (root Makefile) |
| 3 | 3 | ||
| 4 | CXX ?= g++ | 4 | CXX ?= g++ |
| 5 | FUZZ_CXX ?= clang++ | ||
| 5 | 6 | ||
| 6 | SECP_INC ?= /usr/local/include | 7 | SECP_INC ?= /usr/local/include |
| 7 | SECP_LIB ?= /usr/local/lib | 8 | SECP_LIB ?= /usr/local/lib |
| @@ -29,10 +30,13 @@ WARNFLAGS = -Wformat -Werror=format \ | |||
| 29 | -Wno-narrowing -Wno-unused-result | 30 | -Wno-narrowing -Wno-unused-result |
| 30 | 31 | ||
| 31 | CXXFLAGS = -std=gnu++14 -O2 $(WARNFLAGS) -fpermissive -pthread \ | 32 | CXXFLAGS = -std=gnu++14 -O2 $(WARNFLAGS) -fpermissive -pthread \ |
| 32 | -DSECP256K1_STATIC \ | 33 | -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD \ |
| 33 | -I$(SECP_INC) -I$(RANDOMX_DIR)/src \ | 34 | -I$(SECP_INC) -I$(RANDOMX_DIR)/src \ |
| 34 | -I/usr/include/nlohmann \ | 35 | -I/usr/include/nlohmann \ |
| 35 | -I$(IMGUI_DIR) | 36 | -I$(IMGUI_DIR) |
| 37 | FUZZ_CXXFLAGS = $(filter-out -O2,$(CXXFLAGS)) -O1 -g -fno-omit-frame-pointer \ | ||
| 38 | -fsanitize=fuzzer-no-link,address,undefined | ||
| 39 | FUZZ_LDFLAGS = -fsanitize=fuzzer,address,undefined | ||
| 36 | 40 | ||
| 37 | LIBS = -L$(SECP_LIB) -lsecp256k1 \ | 41 | LIBS = -L$(SECP_LIB) -lsecp256k1 \ |
| 38 | -L$(RANDOMX_DIR)/build -lrandomx \ | 42 | -L$(RANDOMX_DIR)/build -lrandomx \ |
| @@ -49,7 +53,7 @@ LIBS = -L$(SECP_LIB) -lsecp256k1 \ | |||
| 49 | HEADERS = headers_core.h headers.h util.h main.h serialize.h uint256.h \ | 53 | HEADERS = headers_core.h headers.h util.h main.h serialize.h uint256.h \ |
| 50 | key.h bignum.h script.h db.h base58.h net.h nostr.h \ | 54 | key.h bignum.h script.h db.h base58.h net.h nostr.h \ |
| 51 | randomx_pow.h btfaddr.h btfchan.h btfrv.h btftunnel.h \ | 55 | randomx_pow.h btfaddr.h btfchan.h btfrv.h btftunnel.h \ |
| 52 | selftest.h walletcmd.h compat.h sockcount.h font_roboto.h | 56 | selftest.h walletcmd.h compat.h sockcount.h proxy.h font_roboto.h |
| 53 | 57 | ||
| 54 | IMGUI_OBJS = obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o \ | 58 | IMGUI_OBJS = obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o \ |
| 55 | obj/imgui_widgets.o obj/imgui_impl_glfw.o obj/imgui_impl_opengl3.o | 59 | obj/imgui_widgets.o obj/imgui_impl_glfw.o obj/imgui_impl_opengl3.o |
| @@ -57,7 +61,7 @@ IMGUI_OBJS = obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o \ | |||
| 57 | NODE_OBJS = obj/util.o obj/script.o obj/db.o obj/net.o obj/main.o \ | 61 | NODE_OBJS = obj/util.o obj/script.o obj/db.o obj/net.o obj/main.o \ |
| 58 | obj/sha.o obj/nostr.o obj/randomx_pow.o \ | 62 | obj/sha.o obj/nostr.o obj/randomx_pow.o \ |
| 59 | obj/btfaddr.o obj/btfchan.o obj/btfrv.o obj/btftunnel.o obj/rpc.o \ | 63 | obj/btfaddr.o obj/btfchan.o obj/btfrv.o obj/btftunnel.o obj/rpc.o \ |
| 60 | obj/selftest.o obj/bip32.o obj/walletcmd.o | 64 | obj/selftest.o obj/bip32.o obj/walletcmd.o obj/proxy.o |
| 61 | 65 | ||
| 62 | GUI_OBJS = obj/gui.o obj/main_gui.o | 66 | GUI_OBJS = obj/gui.o obj/main_gui.o |
| 63 | 67 | ||
| @@ -117,6 +121,10 @@ NODE_LIBS = -L$(SECP_LIB) -lsecp256k1 \ | |||
| 117 | NODE_ONLY_OBJS = $(patsubst obj/%,obj-node/%,$(NODE_OBJS)) \ | 121 | NODE_ONLY_OBJS = $(patsubst obj/%,obj-node/%,$(NODE_OBJS)) \ |
| 118 | obj-node/main_gui.o obj-node/headless.o | 122 | obj-node/main_gui.o obj-node/headless.o |
| 119 | 123 | ||
| 124 | FUZZ_BASE_OBJS = $(patsubst obj/%,obj-fuzz/%,$(NODE_OBJS)) | ||
| 125 | FUZZ_NET_MESSAGE_OBJS = $(FUZZ_BASE_OBJS) obj-fuzz/fuzz_net_message.o | ||
| 126 | FUZZ_SCRIPT_OBJS = $(FUZZ_BASE_OBJS) obj-fuzz/fuzz_script.o | ||
| 127 | |||
| 120 | obj-node/%.o: %.cpp $(HEADERS) | 128 | obj-node/%.o: %.cpp $(HEADERS) |
| 121 | @mkdir -p obj-node | 129 | @mkdir -p obj-node |
| 122 | $(CXX) -c $(CXXFLAGS) -DBITFLASH_NO_GUI -o $@ $< | 130 | $(CXX) -c $(CXXFLAGS) -DBITFLASH_NO_GUI -o $@ $< |
| @@ -129,8 +137,47 @@ bitflash-node: $(NODE_ONLY_OBJS) | |||
| 129 | $(CXX) $(CXXFLAGS) -o $@ $^ $(NODE_LIBS) | 137 | $(CXX) $(CXXFLAGS) -o $@ $^ $(NODE_LIBS) |
| 130 | @echo "Built: bitflash-node (headless, no libGL needed)" | 138 | @echo "Built: bitflash-node (headless, no libGL needed)" |
| 131 | 139 | ||
| 140 | obj-fuzz/%.o: %.cpp $(HEADERS) | ||
| 141 | @mkdir -p obj-fuzz | ||
| 142 | $(FUZZ_CXX) -c $(FUZZ_CXXFLAGS) -DBITFLASH_FUZZ -o $@ $< | ||
| 143 | |||
| 144 | obj-fuzz/sha.o: sha.cpp sha.h | ||
| 145 | @mkdir -p obj-fuzz | ||
| 146 | $(FUZZ_CXX) -c $(FUZZ_CXXFLAGS) -DBITFLASH_FUZZ -O1 -o $@ $< | ||
| 147 | |||
| 148 | obj-fuzz/fuzz_net_message.o: fuzz_net_message.cpp $(HEADERS) | ||
| 149 | @mkdir -p obj-fuzz | ||
| 150 | $(FUZZ_CXX) -c $(FUZZ_CXXFLAGS) -DBITFLASH_FUZZ -o $@ $< | ||
| 151 | |||
| 152 | obj-fuzz/fuzz_script.o: fuzz_script.cpp $(HEADERS) | ||
| 153 | @mkdir -p obj-fuzz | ||
| 154 | $(FUZZ_CXX) -c $(FUZZ_CXXFLAGS) -DBITFLASH_FUZZ -o $@ $< | ||
| 155 | |||
| 156 | fuzz-net-message: $(FUZZ_NET_MESSAGE_OBJS) | ||
| 157 | $(FUZZ_CXX) $(filter-out -fsanitize=fuzzer-no-link%,$(FUZZ_CXXFLAGS)) \ | ||
| 158 | $(FUZZ_LDFLAGS) -o $@ $^ $(NODE_LIBS) | ||
| 159 | @echo "Built: fuzz-net-message" | ||
| 160 | |||
| 161 | fuzz-script: $(FUZZ_SCRIPT_OBJS) | ||
| 162 | $(FUZZ_CXX) $(filter-out -fsanitize=fuzzer-no-link%,$(FUZZ_CXXFLAGS)) \ | ||
| 163 | $(FUZZ_LDFLAGS) -o $@ $^ $(NODE_LIBS) | ||
| 164 | @echo "Built: fuzz-script" | ||
| 165 | |||
| 166 | fuzz-net-message-smoke: fuzz-net-message | ||
| 167 | @mkdir -p ../work/fuzz-corpus/net-message | ||
| 168 | @printf '\277\040\134\375ping\000\000\000\000\000\000\000\000\000\000\000\000' > ../work/fuzz-corpus/net-message/ping-header | ||
| 169 | @printf '\277\040\134\375version\000\000\000\000\000\000\000\000\000' > ../work/fuzz-corpus/net-message/version-header | ||
| 170 | ./fuzz-net-message -runs=128 ../work/fuzz-corpus/net-message | ||
| 171 | |||
| 172 | fuzz-script-smoke: fuzz-script | ||
| 173 | @mkdir -p ../work/fuzz-corpus/script | ||
| 174 | @printf '\000\121' > ../work/fuzz-corpus/script/op-true | ||
| 175 | @printf '\001\166\251\024\001\002\003\004\005\006\007\010\011\012\013\014\015\016\017\020\021\022\023\024\210\254' > ../work/fuzz-corpus/script/p2pkh-shape | ||
| 176 | @printf '\000\115\377\377' > ../work/fuzz-corpus/script/truncated-pushdata2 | ||
| 177 | ./fuzz-script -runs=128 ../work/fuzz-corpus/script | ||
| 178 | |||
| 132 | clean: | 179 | clean: |
| 133 | rm -rf obj obj-node bitflash bitflash-node test_bip32 | 180 | rm -rf obj obj-node obj-fuzz bitflash bitflash-node fuzz-net-message fuzz-script test_bip32 |
| 134 | 181 | ||
| 135 | test_bip32: test_bip32.cpp bip32.cpp bip32.h bip39_english.h | 182 | test_bip32: test_bip32.cpp bip32.cpp bip32.h bip39_english.h |
| 136 | $(CXX) $(CXXFLAGS) -o $@ test_bip32.cpp bip32.cpp -lssl -lcrypto | 183 | $(CXX) $(CXXFLAGS) -o $@ test_bip32.cpp bip32.cpp -lssl -lcrypto |
| @@ -144,3 +191,8 @@ tests: test_bip32 bitflash-node | |||
| 144 | ./bitflash-node -selftest=wallet-encrypt | 191 | ./bitflash-node -selftest=wallet-encrypt |
| 145 | ./bitflash-node -selftest=net-message | 192 | ./bitflash-node -selftest=net-message |
| 146 | ./bitflash-node -selftest=consensus-limits | 193 | ./bitflash-node -selftest=consensus-limits |
| 194 | ./bitflash-node -selftest=pool-stratum | ||
| 195 | ./bitflash-node -selftest=parse-money | ||
| 196 | ./bitflash-node -selftest=socks5-proxy | ||
| 197 | |||
| 198 | .PHONY: tests fuzz-net-message-smoke fuzz-script-smoke | ||
diff --git a/src/bignum.h b/src/bignum.h index 9da104b..6a295b2 100644 --- a/src/bignum.h +++ b/src/bignum.h | |||
| @@ -155,7 +155,9 @@ public: | |||
| 155 | for (int i = 0; i < 8; i++) | 155 | for (int i = 0; i < 8; i++) |
| 156 | { | 156 | { |
| 157 | unsigned char c = (n >> 56) & 0xff; | 157 | unsigned char c = (n >> 56) & 0xff; |
| 158 | n <<= 8; | 158 | n = (int64)((uint64)n << 8); // shift in unsigned space; <<= on a |
| 159 | // signed int64 with the top byte set | ||
| 160 | // is overflow UB (fuzzer/UBSan finding) | ||
| 159 | if (fLeadingZeroes) | 161 | if (fLeadingZeroes) |
| 160 | { | 162 | { |
| 161 | if (c == 0) | 163 | if (c == 0) |
diff --git a/src/btfrv.cpp b/src/btfrv.cpp index 905717f..133eb89 100644 --- a/src/btfrv.cpp +++ b/src/btfrv.cpp | |||
| @@ -5,6 +5,9 @@ | |||
| 5 | // Windows/Winsock implementation (Layer 6 will #ifdef this for Linux/BSD sockets). | 5 | // Windows/Winsock implementation (Layer 6 will #ifdef this for Linux/BSD sockets). |
| 6 | 6 | ||
| 7 | #include "btfrv.h" | 7 | #include "btfrv.h" |
| 8 | #ifdef BITFLASH_NODE_BUILD | ||
| 9 | #include "proxy.h" | ||
| 10 | #endif | ||
| 8 | 11 | ||
| 9 | // Portable sockets: Winsock on Windows, BSD sockets on Linux/POSIX (for the | 12 | // Portable sockets: Winsock on Windows, BSD sockets on Linux/POSIX (for the |
| 10 | // headless relay `bitflashd` that runs on the VPS). | 13 | // headless relay `bitflashd` that runs on the VPS). |
| @@ -92,6 +95,11 @@ static bool WriteN(SOCKET s, const void* buf, int n) | |||
| 92 | 95 | ||
| 93 | static SOCKET ConnectTo(const char* host, unsigned short port) | 96 | static SOCKET ConnectTo(const char* host, unsigned short port) |
| 94 | { | 97 | { |
| 98 | #ifdef BITFLASH_NODE_BUILD | ||
| 99 | SOCKET s = BtfConnectSocket(host, port, SOCK_RV_DIAL, 10); | ||
| 100 | if (s == INVALID_SOCKET) | ||
| 101 | return INVALID_SOCKET; | ||
| 102 | #else | ||
| 95 | struct addrinfo hints, *res = NULL; | 103 | struct addrinfo hints, *res = NULL; |
| 96 | memset(&hints, 0, sizeof(hints)); | 104 | memset(&hints, 0, sizeof(hints)); |
| 97 | hints.ai_family = AF_UNSPEC; // IPv4 or IPv6 (some relays are v6-only) | 105 | hints.ai_family = AF_UNSPEC; // IPv4 or IPv6 (some relays are v6-only) |
| @@ -130,6 +138,7 @@ static SOCKET ConnectTo(const char* host, unsigned short port) | |||
| 130 | } | 138 | } |
| 131 | freeaddrinfo(res); | 139 | freeaddrinfo(res); |
| 132 | if (s == INVALID_SOCKET) return INVALID_SOCKET; | 140 | if (s == INVALID_SOCKET) return INVALID_SOCKET; |
| 141 | #endif | ||
| 133 | 142 | ||
| 134 | // Clear the timeout -- the socket is now a live tunnel and must not | 143 | // Clear the timeout -- the socket is now a live tunnel and must not |
| 135 | // time out when idle. recv() will block until data arrives or peer closes. | 144 | // time out when idle. recv() will block until data arrives or peer closes. |
| @@ -872,9 +872,9 @@ bool CWalletDB::LoadWallet(vector<unsigned char>& vchDefaultKeyRet) | |||
| 872 | } | 872 | } |
| 873 | if (!fHaveStoredHDCoinType) | 873 | if (!fHaveStoredHDCoinType) |
| 874 | { | 874 | { |
| 875 | nHDCoinType = HD_BIP44_COIN_TYPE_BITFLASH_PROVISIONAL; | 875 | nHDCoinType = HD_BIP44_COIN_TYPE_BITFLASH; |
| 876 | printf("LoadWallet: deterministic seed has no BIP44 coin type metadata; " | 876 | printf("LoadWallet: deterministic seed has no BIP44 coin type metadata; " |
| 877 | "using provisional Bitflash coin type %u\n", nHDCoinType); | 877 | "using Bitflash coin type %u\n", nHDCoinType); |
| 878 | } | 878 | } |
| 879 | } | 879 | } |
| 880 | else | 880 | else |
| @@ -883,7 +883,7 @@ bool CWalletDB::LoadWallet(vector<unsigned char>& vchDefaultKeyRet) | |||
| 883 | nHDNext = 0; | 883 | nHDNext = 0; |
| 884 | nHDReceiveNext = 0; | 884 | nHDReceiveNext = 0; |
| 885 | nHDChangeNext = 0; | 885 | nHDChangeNext = 0; |
| 886 | nHDCoinType = HD_BIP44_COIN_TYPE_BITFLASH_PROVISIONAL; | 886 | nHDCoinType = HD_BIP44_COIN_TYPE_BITFLASH; |
| 887 | } | 887 | } |
| 888 | 888 | ||
| 889 | // fGenerateBitcoins and nMineMode only mean anything together, and a | 889 | // fGenerateBitcoins and nMineMode only mean anything together, and a |
| @@ -1396,6 +1396,13 @@ static bool EncryptPrivateKeyForWallet(const CKeyingMaterial& vchMasterKey, | |||
| 1396 | return true; | 1396 | return true; |
| 1397 | } | 1397 | } |
| 1398 | 1398 | ||
| 1399 | static void SecureClearBytes(vector<unsigned char>& vch) | ||
| 1400 | { | ||
| 1401 | if (!vch.empty()) | ||
| 1402 | OPENSSL_cleanse(&vch[0], vch.size()); | ||
| 1403 | vch.clear(); | ||
| 1404 | } | ||
| 1405 | |||
| 1399 | bool EncryptWallet(const string& strPassphrase, string& strBackupRet, string& strErrorRet) | 1406 | bool EncryptWallet(const string& strPassphrase, string& strBackupRet, string& strErrorRet) |
| 1400 | { | 1407 | { |
| 1401 | strBackupRet.clear(); | 1408 | strBackupRet.clear(); |
| @@ -1415,6 +1422,7 @@ bool EncryptWallet(const string& strPassphrase, string& strBackupRet, string& st | |||
| 1415 | map<vector<unsigned char>, CPrivKey> mapPlainKeys; | 1422 | map<vector<unsigned char>, CPrivKey> mapPlainKeys; |
| 1416 | vector<unsigned char> vchPlainHDMaster; | 1423 | vector<unsigned char> vchPlainHDMaster; |
| 1417 | vector<unsigned char> vchPlainHDChainCode; | 1424 | vector<unsigned char> vchPlainHDChainCode; |
| 1425 | bool fHadHDSeed = false; | ||
| 1418 | unsigned int nHDNextSnapshot = 0; | 1426 | unsigned int nHDNextSnapshot = 0; |
| 1419 | { | 1427 | { |
| 1420 | CRITICAL_BLOCK(cs_mapKeys) | 1428 | CRITICAL_BLOCK(cs_mapKeys) |
| @@ -1425,6 +1433,7 @@ bool EncryptWallet(const string& strPassphrase, string& strBackupRet, string& st | |||
| 1425 | { | 1433 | { |
| 1426 | vchPlainHDMaster = vchHDMaster; | 1434 | vchPlainHDMaster = vchHDMaster; |
| 1427 | vchPlainHDChainCode = vchHDChainCode; | 1435 | vchPlainHDChainCode = vchHDChainCode; |
| 1436 | fHadHDSeed = !vchPlainHDMaster.empty() || !vchPlainHDChainCode.empty(); | ||
| 1428 | nHDNextSnapshot = nHDNext; | 1437 | nHDNextSnapshot = nHDNext; |
| 1429 | } | 1438 | } |
| 1430 | } | 1439 | } |
| @@ -1475,11 +1484,13 @@ bool EncryptWallet(const string& strPassphrase, string& strBackupRet, string& st | |||
| 1475 | 1484 | ||
| 1476 | vector<unsigned char> vchCryptedHDMasterNew; | 1485 | vector<unsigned char> vchCryptedHDMasterNew; |
| 1477 | vector<unsigned char> vchCryptedHDChainCodeNew; | 1486 | vector<unsigned char> vchCryptedHDChainCodeNew; |
| 1478 | if (!vchPlainHDMaster.empty() || !vchPlainHDChainCode.empty()) | 1487 | if (fHadHDSeed) |
| 1479 | { | 1488 | { |
| 1480 | if (vchPlainHDMaster.size() != 32 || vchPlainHDChainCode.size() != 32) | 1489 | if (vchPlainHDMaster.size() != 32 || vchPlainHDChainCode.size() != 32) |
| 1481 | { | 1490 | { |
| 1482 | strErrorRet = "wallet has an incomplete HD seed"; | 1491 | strErrorRet = "wallet has an incomplete HD seed"; |
| 1492 | SecureClearBytes(vchPlainHDMaster); | ||
| 1493 | SecureClearBytes(vchPlainHDChainCode); | ||
| 1483 | return false; | 1494 | return false; |
| 1484 | } | 1495 | } |
| 1485 | if (!EncryptSecret(vchMasterKey, vchPlainHDMaster, | 1496 | if (!EncryptSecret(vchMasterKey, vchPlainHDMaster, |
| @@ -1488,8 +1499,12 @@ bool EncryptWallet(const string& strPassphrase, string& strBackupRet, string& st | |||
| 1488 | WalletSecretIV("hdchaincode"), vchCryptedHDChainCodeNew)) | 1499 | WalletSecretIV("hdchaincode"), vchCryptedHDChainCodeNew)) |
| 1489 | { | 1500 | { |
| 1490 | strErrorRet = "could not encrypt the HD seed"; | 1501 | strErrorRet = "could not encrypt the HD seed"; |
| 1502 | SecureClearBytes(vchPlainHDMaster); | ||
| 1503 | SecureClearBytes(vchPlainHDChainCode); | ||
| 1491 | return false; | 1504 | return false; |
| 1492 | } | 1505 | } |
| 1506 | SecureClearBytes(vchPlainHDMaster); | ||
| 1507 | SecureClearBytes(vchPlainHDChainCode); | ||
| 1493 | } | 1508 | } |
| 1494 | 1509 | ||
| 1495 | vector<pair<int64, vector<unsigned char> > > vNewPool; | 1510 | vector<pair<int64, vector<unsigned char> > > vNewPool; |
| @@ -1498,7 +1513,7 @@ bool EncryptWallet(const string& strPassphrase, string& strBackupRet, string& st | |||
| 1498 | for (int i = 0; i < KEYPOOL_SIZE; i++) | 1513 | for (int i = 0; i < KEYPOOL_SIZE; i++) |
| 1499 | { | 1514 | { |
| 1500 | CKey key; | 1515 | CKey key; |
| 1501 | if (!vchPlainHDMaster.empty()) | 1516 | if (fHadHDSeed) |
| 1502 | { | 1517 | { |
| 1503 | string strDeriveError; | 1518 | string strDeriveError; |
| 1504 | if (!DeriveHDKey(nHDNextNew, key, strDeriveError)) | 1519 | if (!DeriveHDKey(nHDNextNew, key, strDeriveError)) |
diff --git a/src/fuzz_net_message.cpp b/src/fuzz_net_message.cpp new file mode 100644 index 0000000..1104d90 --- /dev/null +++ b/src/fuzz_net_message.cpp | |||
| @@ -0,0 +1,32 @@ | |||
| 1 | // Copyright (c) 2026 Bitflash developers | ||
| 2 | // Distributed under the MIT/X11 software license. | ||
| 3 | // | ||
| 4 | // libFuzzer harness for the P2P message envelope parser. It feeds arbitrary | ||
| 5 | // receive-buffer bytes into ProcessMessages() through a dummy CNode, without | ||
| 6 | // opening sockets or starting node threads. | ||
| 7 | |||
| 8 | #include "headers_core.h" | ||
| 9 | |||
| 10 | #include <stdint.h> | ||
| 11 | #include <stddef.h> | ||
| 12 | |||
| 13 | // Stand-ins normally supplied by the GUI/headless entry objects. | ||
| 14 | map<string,string> mapAddressBook; | ||
| 15 | bool gPoolServerRunning = false; | ||
| 16 | void MainFrameRepaint() {} | ||
| 17 | |||
| 18 | extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) | ||
| 19 | { | ||
| 20 | // Keep this first target focused on parser behavior, not multi-megabyte | ||
| 21 | // allocation pressure. The protocol size guards have their own selftest. | ||
| 22 | if (size > 4096) | ||
| 23 | return 0; | ||
| 24 | |||
| 25 | CNode node(INVALID_SOCKET, CAddress("127.0.0.1")); | ||
| 26 | node.nVersion = 0; // exercise pre-version gating plus the version parser | ||
| 27 | if (size > 0) | ||
| 28 | node.vRecv.write((const char*)data, (int)size); | ||
| 29 | |||
| 30 | ProcessMessages(&node); | ||
| 31 | return 0; | ||
| 32 | } | ||
diff --git a/src/fuzz_script.cpp b/src/fuzz_script.cpp new file mode 100644 index 0000000..28c84f6 --- /dev/null +++ b/src/fuzz_script.cpp | |||
| @@ -0,0 +1,59 @@ | |||
| 1 | // Copyright (c) 2026 Bitflash developers | ||
| 2 | // Distributed under the MIT/X11 software license. | ||
| 3 | // | ||
| 4 | // libFuzzer harness for raw script parsing and evaluation. It keeps the | ||
| 5 | // target deliberately small: arbitrary bytes become one CScript, then the | ||
| 6 | // parser, standard-script extractors and EvalScript all see that same input. | ||
| 7 | |||
| 8 | #include "headers_core.h" | ||
| 9 | |||
| 10 | #include <stdint.h> | ||
| 11 | #include <stddef.h> | ||
| 12 | |||
| 13 | // Stand-ins normally supplied by the GUI/headless entry objects. | ||
| 14 | map<string,string> mapAddressBook; | ||
| 15 | bool gPoolServerRunning = false; | ||
| 16 | void MainFrameRepaint() {} | ||
| 17 | |||
| 18 | extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t size) | ||
| 19 | { | ||
| 20 | if (size < 1 || size > 4096) | ||
| 21 | return 0; | ||
| 22 | |||
| 23 | try | ||
| 24 | { | ||
| 25 | int nHashType = data[0]; | ||
| 26 | const unsigned char* pbegin = data + 1; | ||
| 27 | const unsigned char* pend = data + size; | ||
| 28 | CScript script(pbegin, pend); | ||
| 29 | |||
| 30 | CScript::const_iterator pc = script.begin(); | ||
| 31 | opcodetype opcode; | ||
| 32 | vector<unsigned char> vchPush; | ||
| 33 | while (pc < script.end()) | ||
| 34 | { | ||
| 35 | if (!script.GetOp(pc, opcode, vchPush)) | ||
| 36 | break; | ||
| 37 | } | ||
| 38 | |||
| 39 | script.GetSigOpCount(); | ||
| 40 | script.ToString(); | ||
| 41 | |||
| 42 | vector<unsigned char> vchPubKey; | ||
| 43 | ExtractPubKey(script, false, vchPubKey); | ||
| 44 | uint160 hash160; | ||
| 45 | ExtractHash160(script, hash160); | ||
| 46 | |||
| 47 | CTransaction tx; | ||
| 48 | tx.vin.push_back(CTxIn(COutPoint(), CScript())); | ||
| 49 | tx.vout.push_back(CTxOut(0, CScript())); | ||
| 50 | |||
| 51 | vector<vector<unsigned char> > stack; | ||
| 52 | EvalScript(script, tx, 0, nHashType, &stack); | ||
| 53 | } | ||
| 54 | catch (...) | ||
| 55 | { | ||
| 56 | } | ||
| 57 | |||
| 58 | return 0; | ||
| 59 | } | ||
diff --git a/src/gui.cpp b/src/gui.cpp index 8f1d204..8dcb854 100644 --- a/src/gui.cpp +++ b/src/gui.cpp | |||
| @@ -1250,7 +1250,7 @@ static void DrawWalletSafetyDialog() | |||
| 1250 | if (g_recoveryAudit.fHaveSeed) | 1250 | if (g_recoveryAudit.fHaveSeed) |
| 1251 | { | 1251 | { |
| 1252 | ImGui::Text("Derivation schema: %s", HDKeySchemaName(g_recoveryAudit.nSchema).c_str()); | 1252 | ImGui::Text("Derivation schema: %s", HDKeySchemaName(g_recoveryAudit.nSchema).c_str()); |
| 1253 | ImGui::Text("BIP44 coin type: %u (provisional BITFLASH)", g_recoveryAudit.nCoinType); | 1253 | ImGui::Text("BIP44 coin type: %u (SLIP-0044 BITFLASH)", g_recoveryAudit.nCoinType); |
| 1254 | } | 1254 | } |
| 1255 | ImGui::Text("Phrase-backed spendable balance: %s BTF", | 1255 | ImGui::Text("Phrase-backed spendable balance: %s BTF", |
| 1256 | FmtMoney(g_recoveryAudit.nRecoverableCredit).c_str()); | 1256 | FmtMoney(g_recoveryAudit.nRecoverableCredit).c_str()); |
diff --git a/src/main.cpp b/src/main.cpp index 77aba89..252911b 100644 --- a/src/main.cpp +++ b/src/main.cpp | |||
| @@ -109,7 +109,11 @@ string strPayoutFile; // participant: /payoutfile with one addres | |||
| 109 | int nHashThreads = 0; // participant: /hashthreads override (0 = cores-1) | 109 | int nHashThreads = 0; // participant: /hashthreads override (0 = cores-1) |
| 110 | string strPoolName = "Bitflash Pool"; | 110 | string strPoolName = "Bitflash Pool"; |
| 111 | string strPoolDashboardUrl; | 111 | string strPoolDashboardUrl; |
| 112 | string strPoolStatusFile; | ||
| 113 | string strPoolRoundsFile; | ||
| 112 | double dPoolFeePercent = 0.0; | 114 | double dPoolFeePercent = 0.0; |
| 115 | bool fStratumBridge = false; | ||
| 116 | int nStratumBridgePort = 3333; | ||
| 113 | 117 | ||
| 114 | static std::atomic<uint64> gParticipantSharesSent{0}; | 118 | static std::atomic<uint64> gParticipantSharesSent{0}; |
| 115 | static std::atomic<uint64> gParticipantSharesAccepted{0}; | 119 | static std::atomic<uint64> gParticipantSharesAccepted{0}; |
| @@ -190,6 +194,9 @@ int ParticipantHashThreadCount() | |||
| 190 | return nWorkers - 1; | 194 | return nWorkers - 1; |
| 191 | } | 195 | } |
| 192 | 196 | ||
| 197 | static void SecureClearBytes(vector<unsigned char>& vch); | ||
| 198 | static void SecureClearKeyingMaterial(CKeyingMaterial& vch); | ||
| 199 | |||
| 193 | bool AddKey(const CKey& key) | 200 | bool AddKey(const CKey& key) |
| 194 | { | 201 | { |
| 195 | if (IsWalletEncrypted()) | 202 | if (IsWalletEncrypted()) |
| @@ -199,9 +206,13 @@ bool AddKey(const CKey& key) | |||
| 199 | vector<unsigned char> vchCryptedSecret; | 206 | vector<unsigned char> vchCryptedSecret; |
| 200 | if (IsWalletLocked()) | 207 | if (IsWalletLocked()) |
| 201 | return error("AddKey() : wallet is locked\n"); | 208 | return error("AddKey() : wallet is locked\n"); |
| 202 | if (!EncryptSecret(vWalletMasterKey, vector<unsigned char>(vchPrivKey.begin(), vchPrivKey.end()), | 209 | vector<unsigned char> vchPlainSecret(vchPrivKey.begin(), vchPrivKey.end()); |
| 203 | WalletKeyIV(key.GetPubKey()), vchCryptedSecret)) | 210 | if (!EncryptSecret(vWalletMasterKey, vchPlainSecret, WalletKeyIV(key.GetPubKey()), vchCryptedSecret)) |
| 211 | { | ||
| 212 | SecureClearBytes(vchPlainSecret); | ||
| 204 | return error("AddKey() : encrypting key failed\n"); | 213 | return error("AddKey() : encrypting key failed\n"); |
| 214 | } | ||
| 215 | SecureClearBytes(vchPlainSecret); | ||
| 205 | if (!AddCryptedKey(key.GetPubKey(), vchCryptedSecret)) | 216 | if (!AddCryptedKey(key.GetPubKey(), vchCryptedSecret)) |
| 206 | return false; | 217 | return false; |
| 207 | return CWalletDB().WriteCryptedKey(key.GetPubKey(), vchCryptedSecret); | 218 | return CWalletDB().WriteCryptedKey(key.GetPubKey(), vchCryptedSecret); |
| @@ -225,17 +236,31 @@ bool IsWalletLocked() | |||
| 225 | return fWalletEncrypted && fWalletLocked; | 236 | return fWalletEncrypted && fWalletLocked; |
| 226 | } | 237 | } |
| 227 | 238 | ||
| 239 | static void SecureClearBytes(vector<unsigned char>& vch) | ||
| 240 | { | ||
| 241 | if (!vch.empty()) | ||
| 242 | OPENSSL_cleanse(&vch[0], vch.size()); | ||
| 243 | vch.clear(); | ||
| 244 | } | ||
| 245 | |||
| 246 | static void SecureClearKeyingMaterial(CKeyingMaterial& vch) | ||
| 247 | { | ||
| 248 | if (!vch.empty()) | ||
| 249 | OPENSSL_cleanse(&vch[0], vch.size()); | ||
| 250 | vch.clear(); | ||
| 251 | } | ||
| 252 | |||
| 228 | void LockWallet() | 253 | void LockWallet() |
| 229 | { | 254 | { |
| 230 | if (!fWalletEncrypted) | 255 | if (!fWalletEncrypted) |
| 231 | return; | 256 | return; |
| 232 | CRITICAL_BLOCK(cs_mapKeys) | 257 | CRITICAL_BLOCK(cs_mapKeys) |
| 233 | { | 258 | { |
| 234 | vWalletMasterKey.clear(); | 259 | SecureClearKeyingMaterial(vWalletMasterKey); |
| 235 | if (!vchCryptedHDMaster.empty()) | 260 | if (!vchCryptedHDMaster.empty()) |
| 236 | vchHDMaster.clear(); | 261 | SecureClearBytes(vchHDMaster); |
| 237 | if (!vchCryptedHDChainCode.empty()) | 262 | if (!vchCryptedHDChainCode.empty()) |
| 238 | vchHDChainCode.clear(); | 263 | SecureClearBytes(vchHDChainCode); |
| 239 | fWalletLocked = true; | 264 | fWalletLocked = true; |
| 240 | } | 265 | } |
| 241 | } | 266 | } |
| @@ -260,14 +285,14 @@ bool DeriveWalletPassphraseKey(const string& strPassphrase, | |||
| 260 | chKey, chIV); | 285 | chKey, chIV); |
| 261 | if (n != 32) | 286 | if (n != 32) |
| 262 | { | 287 | { |
| 263 | memset(chKey, 0, sizeof(chKey)); | 288 | OPENSSL_cleanse(chKey, sizeof(chKey)); |
| 264 | memset(chIV, 0, sizeof(chIV)); | 289 | OPENSSL_cleanse(chIV, sizeof(chIV)); |
| 265 | return false; | 290 | return false; |
| 266 | } | 291 | } |
| 267 | vchKeyRet.assign(chKey, chKey + sizeof(chKey)); | 292 | vchKeyRet.assign(chKey, chKey + sizeof(chKey)); |
| 268 | vchIVRet.assign(chIV, chIV + sizeof(chIV)); | 293 | vchIVRet.assign(chIV, chIV + sizeof(chIV)); |
| 269 | memset(chKey, 0, sizeof(chKey)); | 294 | OPENSSL_cleanse(chKey, sizeof(chKey)); |
| 270 | memset(chIV, 0, sizeof(chIV)); | 295 | OPENSSL_cleanse(chIV, sizeof(chIV)); |
| 271 | return true; | 296 | return true; |
| 272 | } | 297 | } |
| 273 | 298 | ||
| @@ -328,6 +353,8 @@ bool DecryptSecret(const CKeyingMaterial& vchKey, | |||
| 328 | vchPlaintextRet.swap(vchOut); | 353 | vchPlaintextRet.swap(vchOut); |
| 329 | fOk = true; | 354 | fOk = true; |
| 330 | } | 355 | } |
| 356 | if (!fOk) | ||
| 357 | SecureClearBytes(vchOut); | ||
| 331 | EVP_CIPHER_CTX_free(ctx); | 358 | EVP_CIPHER_CTX_free(ctx); |
| 332 | return fOk; | 359 | return fOk; |
| 333 | } | 360 | } |
| @@ -403,6 +430,7 @@ bool GetWalletPrivKey(const vector<unsigned char>& vchPubKey, | |||
| 403 | } | 430 | } |
| 404 | 431 | ||
| 405 | CPrivKey vchPrivKey(vchPlaintext.begin(), vchPlaintext.end()); | 432 | CPrivKey vchPrivKey(vchPlaintext.begin(), vchPlaintext.end()); |
| 433 | SecureClearBytes(vchPlaintext); | ||
| 406 | CKey key; | 434 | CKey key; |
| 407 | if (!key.SetPrivKey(vchPrivKey) || key.GetPubKey() != vchPubKey) | 435 | if (!key.SetPrivKey(vchPrivKey) || key.GetPubKey() != vchPubKey) |
| 408 | { | 436 | { |
| @@ -437,11 +465,20 @@ bool UnlockWallet(const string& strPassphrase, string& strErrorRet) | |||
| 437 | vector<unsigned char> vchMasterPlain; | 465 | vector<unsigned char> vchMasterPlain; |
| 438 | if (!DecryptSecret(vchPassKey, kMasterKey.vchCryptedKey, | 466 | if (!DecryptSecret(vchPassKey, kMasterKey.vchCryptedKey, |
| 439 | vchPassIV, vchMasterPlain)) | 467 | vchPassIV, vchMasterPlain)) |
| 468 | { | ||
| 469 | SecureClearBytes(vchPassIV); | ||
| 440 | continue; | 470 | continue; |
| 471 | } | ||
| 441 | if (vchMasterPlain.size() != 32) | 472 | if (vchMasterPlain.size() != 32) |
| 473 | { | ||
| 474 | SecureClearBytes(vchMasterPlain); | ||
| 475 | SecureClearBytes(vchPassIV); | ||
| 442 | continue; | 476 | continue; |
| 477 | } | ||
| 443 | 478 | ||
| 444 | CKeyingMaterial vchCandidate(vchMasterPlain.begin(), vchMasterPlain.end()); | 479 | CKeyingMaterial vchCandidate(vchMasterPlain.begin(), vchMasterPlain.end()); |
| 480 | SecureClearBytes(vchMasterPlain); | ||
| 481 | SecureClearBytes(vchPassIV); | ||
| 445 | bool fValid = mapCryptedKeys.empty(); | 482 | bool fValid = mapCryptedKeys.empty(); |
| 446 | for (map<vector<unsigned char>, vector<unsigned char> >::iterator mci = mapCryptedKeys.begin(); | 483 | for (map<vector<unsigned char>, vector<unsigned char> >::iterator mci = mapCryptedKeys.begin(); |
| 447 | mci != mapCryptedKeys.end(); ++mci) | 484 | mci != mapCryptedKeys.end(); ++mci) |
| @@ -450,6 +487,7 @@ bool UnlockWallet(const string& strPassphrase, string& strErrorRet) | |||
| 450 | if (!DecryptSecret(vchCandidate, (*mci).second, WalletKeyIV((*mci).first), vchPlaintext)) | 487 | if (!DecryptSecret(vchCandidate, (*mci).second, WalletKeyIV((*mci).first), vchPlaintext)) |
| 451 | break; | 488 | break; |
| 452 | CPrivKey vchPrivKey(vchPlaintext.begin(), vchPlaintext.end()); | 489 | CPrivKey vchPrivKey(vchPlaintext.begin(), vchPlaintext.end()); |
| 490 | SecureClearBytes(vchPlaintext); | ||
| 453 | CKey key; | 491 | CKey key; |
| 454 | if (key.SetPrivKey(vchPrivKey) && key.GetPubKey() == (*mci).first) | 492 | if (key.SetPrivKey(vchPrivKey) && key.GetPubKey() == (*mci).first) |
| 455 | fValid = true; | 493 | fValid = true; |
| @@ -465,10 +503,12 @@ bool UnlockWallet(const string& strPassphrase, string& strErrorRet) | |||
| 465 | WalletSecretIV("hdmaster"), vchPlainHDMaster) || | 503 | WalletSecretIV("hdmaster"), vchPlainHDMaster) || |
| 466 | vchPlainHDMaster.size() != 32) | 504 | vchPlainHDMaster.size() != 32) |
| 467 | { | 505 | { |
| 506 | SecureClearBytes(vchPlainHDMaster); | ||
| 468 | strErrorRet = "could not decrypt HD master key"; | 507 | strErrorRet = "could not decrypt HD master key"; |
| 469 | return false; | 508 | return false; |
| 470 | } | 509 | } |
| 471 | vchHDMaster = vchPlainHDMaster; | 510 | vchHDMaster = vchPlainHDMaster; |
| 511 | SecureClearBytes(vchPlainHDMaster); | ||
| 472 | } | 512 | } |
| 473 | if (!vchCryptedHDChainCode.empty()) | 513 | if (!vchCryptedHDChainCode.empty()) |
| 474 | { | 514 | { |
| @@ -477,10 +517,12 @@ bool UnlockWallet(const string& strPassphrase, string& strErrorRet) | |||
| 477 | WalletSecretIV("hdchaincode"), vchPlainHDChainCode) || | 517 | WalletSecretIV("hdchaincode"), vchPlainHDChainCode) || |
| 478 | vchPlainHDChainCode.size() != 32) | 518 | vchPlainHDChainCode.size() != 32) |
| 479 | { | 519 | { |
| 520 | SecureClearBytes(vchPlainHDChainCode); | ||
| 480 | strErrorRet = "could not decrypt HD chain code"; | 521 | strErrorRet = "could not decrypt HD chain code"; |
| 481 | return false; | 522 | return false; |
| 482 | } | 523 | } |
| 483 | vchHDChainCode = vchPlainHDChainCode; | 524 | vchHDChainCode = vchPlainHDChainCode; |
| 525 | SecureClearBytes(vchPlainHDChainCode); | ||
| 484 | } | 526 | } |
| 485 | fWalletLocked = false; | 527 | fWalletLocked = false; |
| 486 | return true; | 528 | return true; |
| @@ -625,7 +667,7 @@ unsigned int nHDNext = 0; | |||
| 625 | int nHDKeySchema = HD_SCHEMA_NONE; | 667 | int nHDKeySchema = HD_SCHEMA_NONE; |
| 626 | unsigned int nHDReceiveNext = 0; | 668 | unsigned int nHDReceiveNext = 0; |
| 627 | unsigned int nHDChangeNext = 0; | 669 | unsigned int nHDChangeNext = 0; |
| 628 | unsigned int nHDCoinType = HD_BIP44_COIN_TYPE_BITFLASH_PROVISIONAL; | 670 | unsigned int nHDCoinType = HD_BIP44_COIN_TYPE_BITFLASH; |
| 629 | 671 | ||
| 630 | string HDKeySchemaName(int nSchema) | 672 | string HDKeySchemaName(int nSchema) |
| 631 | { | 673 | { |
| @@ -781,7 +823,7 @@ bool SetHDSeedFromMnemonic(const string& strMnemonic, string& strErrorRet) | |||
| 781 | 823 | ||
| 782 | if (!CWalletDB().WriteHDMaster(master.privateKey, master.chainCode) || | 824 | if (!CWalletDB().WriteHDMaster(master.privateKey, master.chainCode) || |
| 783 | !CWalletDB().WriteHDSchema(HD_SCHEMA_BIP44) || | 825 | !CWalletDB().WriteHDSchema(HD_SCHEMA_BIP44) || |
| 784 | !CWalletDB().WriteHDCoinType(HD_BIP44_COIN_TYPE_BITFLASH_PROVISIONAL) || | 826 | !CWalletDB().WriteHDCoinType(HD_BIP44_COIN_TYPE_BITFLASH) || |
| 785 | !CWalletDB().WriteHDNext(0) || | 827 | !CWalletDB().WriteHDNext(0) || |
| 786 | !CWalletDB().WriteHDReceiveNext(1) || | 828 | !CWalletDB().WriteHDReceiveNext(1) || |
| 787 | !CWalletDB().WriteHDChangeNext(0)) | 829 | !CWalletDB().WriteHDChangeNext(0)) |
| @@ -794,7 +836,7 @@ bool SetHDSeedFromMnemonic(const string& strMnemonic, string& strErrorRet) | |||
| 794 | nHDKeySchema = HD_SCHEMA_BIP44; | 836 | nHDKeySchema = HD_SCHEMA_BIP44; |
| 795 | nHDReceiveNext = 1; | 837 | nHDReceiveNext = 1; |
| 796 | nHDChangeNext = 0; | 838 | nHDChangeNext = 0; |
| 797 | nHDCoinType = HD_BIP44_COIN_TYPE_BITFLASH_PROVISIONAL; | 839 | nHDCoinType = HD_BIP44_COIN_TYPE_BITFLASH; |
| 798 | 840 | ||
| 799 | // Index 0 is spoken for before it is derived, so a failure below cannot | 841 | // Index 0 is spoken for before it is derived, so a failure below cannot |
| 800 | // leave the pool free to hand it out as an ordinary key. The cost of | 842 | // leave the pool free to hand it out as an ordinary key. The cost of |
| @@ -4128,6 +4170,190 @@ connected: | |||
| 4128 | return true; | 4170 | return true; |
| 4129 | } | 4171 | } |
| 4130 | 4172 | ||
| 4173 | // --------------------------------------------------------------------------- | ||
| 4174 | // Local Stratum bridge -- lets external miners speak plain Stratum to | ||
| 4175 | // 127.0.0.1 while Bitflash opens the .btf tunnel to the selected pool. | ||
| 4176 | // --------------------------------------------------------------------------- | ||
| 4177 | |||
| 4178 | struct StratumBridgeClientCtx | ||
| 4179 | { | ||
| 4180 | SOCKET miner; | ||
| 4181 | std::string pool; | ||
| 4182 | }; | ||
| 4183 | |||
| 4184 | static bool BridgeSendAll(SOCKET s, const char* p, int n) | ||
| 4185 | { | ||
| 4186 | int off = 0; | ||
| 4187 | while (off < n) | ||
| 4188 | { | ||
| 4189 | int r = send(s, p + off, n - off, 0); | ||
| 4190 | if (r <= 0) | ||
| 4191 | return false; | ||
| 4192 | off += r; | ||
| 4193 | } | ||
| 4194 | return true; | ||
| 4195 | } | ||
| 4196 | |||
| 4197 | static void BridgePump(SOCKET a, SOCKET b) | ||
| 4198 | { | ||
| 4199 | char buf[8192]; | ||
| 4200 | while (!fShutdown) | ||
| 4201 | { | ||
| 4202 | fd_set rd; | ||
| 4203 | FD_ZERO(&rd); | ||
| 4204 | FD_SET(a, &rd); | ||
| 4205 | FD_SET(b, &rd); | ||
| 4206 | SOCKET maxfd = a > b ? a : b; | ||
| 4207 | struct timeval tv = {1, 0}; | ||
| 4208 | int r = select((int)maxfd + 1, &rd, NULL, NULL, &tv); | ||
| 4209 | if (r <= 0) | ||
| 4210 | continue; | ||
| 4211 | |||
| 4212 | if (FD_ISSET(a, &rd)) | ||
| 4213 | { | ||
| 4214 | int n = recv(a, buf, sizeof(buf), 0); | ||
| 4215 | if (n <= 0 || !BridgeSendAll(b, buf, n)) | ||
| 4216 | break; | ||
| 4217 | } | ||
| 4218 | if (FD_ISSET(b, &rd)) | ||
| 4219 | { | ||
| 4220 | int n = recv(b, buf, sizeof(buf), 0); | ||
| 4221 | if (n <= 0 || !BridgeSendAll(a, buf, n)) | ||
| 4222 | break; | ||
| 4223 | } | ||
| 4224 | } | ||
| 4225 | } | ||
| 4226 | |||
| 4227 | static void ThreadStratumBridgeClient(void* arg) | ||
| 4228 | { | ||
| 4229 | StratumBridgeClientCtx* ctx = (StratumBridgeClientCtx*)arg; | ||
| 4230 | SOCKET miner = ctx->miner; | ||
| 4231 | std::string pool = ctx->pool; | ||
| 4232 | delete ctx; | ||
| 4233 | |||
| 4234 | unsigned char target_pubkey[32]; | ||
| 4235 | if (!btf::ParseAddress(pool, target_pubkey)) | ||
| 4236 | { | ||
| 4237 | LogPrint("worker", "[bridge] invalid pool .btf address: %s\n", pool.c_str()); | ||
| 4238 | BtfCloseSocket(miner); | ||
| 4239 | return; | ||
| 4240 | } | ||
| 4241 | |||
| 4242 | std::string meetingHostPort; | ||
| 4243 | unsigned char service_enc_pub[32]; | ||
| 4244 | if (!BtfResolve(pool, meetingHostPort, service_enc_pub)) | ||
| 4245 | { | ||
| 4246 | LogPrint("worker", "[bridge] failed to resolve pool %s\n", pool.c_str()); | ||
| 4247 | BtfCloseSocket(miner); | ||
| 4248 | return; | ||
| 4249 | } | ||
| 4250 | |||
| 4251 | size_t colon = meetingHostPort.rfind(':'); | ||
| 4252 | if (colon == std::string::npos) | ||
| 4253 | { | ||
| 4254 | LogPrint("worker", "[bridge] bad pool endpoint '%s'\n", meetingHostPort.c_str()); | ||
| 4255 | BtfCloseSocket(miner); | ||
| 4256 | return; | ||
| 4257 | } | ||
| 4258 | std::string host = meetingHostPort.substr(0, colon); | ||
| 4259 | int port = atoi(meetingHostPort.substr(colon + 1).c_str()); | ||
| 4260 | if (port <= 0 || port > 65535) | ||
| 4261 | { | ||
| 4262 | LogPrint("worker", "[bridge] bad pool port in '%s'\n", meetingHostPort.c_str()); | ||
| 4263 | BtfCloseSocket(miner); | ||
| 4264 | return; | ||
| 4265 | } | ||
| 4266 | |||
| 4267 | SOCKET poolSock = btf::BtfClientTunnel(host.c_str(), (unsigned short)port, | ||
| 4268 | target_pubkey, service_enc_pub); | ||
| 4269 | if (poolSock == INVALID_SOCKET) | ||
| 4270 | { | ||
| 4271 | LogPrint("worker", "[bridge] tunnel connect failed for pool %s via %s\n", | ||
| 4272 | pool.c_str(), meetingHostPort.c_str()); | ||
| 4273 | BtfCloseSocket(miner); | ||
| 4274 | return; | ||
| 4275 | } | ||
| 4276 | |||
| 4277 | LogPrint("worker", "[bridge] miner connected to pool %s via %s\n", | ||
| 4278 | pool.c_str(), meetingHostPort.c_str()); | ||
| 4279 | BridgePump(miner, poolSock); | ||
| 4280 | BtfCloseSocket(poolSock); | ||
| 4281 | BtfCloseSocket(miner); | ||
| 4282 | LogPrint("worker", "[bridge] miner disconnected from pool %s\n", pool.c_str()); | ||
| 4283 | } | ||
| 4284 | |||
| 4285 | void ThreadStratumBridge(void*) | ||
| 4286 | { | ||
| 4287 | if (strParticipantPool.empty()) | ||
| 4288 | { | ||
| 4289 | printf("Stratum bridge: no pool selected. Use /stratumbridge=POOL_BTF_ADDRESS\n"); | ||
| 4290 | return; | ||
| 4291 | } | ||
| 4292 | |||
| 4293 | SOCKET listener = BtfSocketTag(socket(AF_INET, SOCK_STREAM, IPPROTO_TCP), | ||
| 4294 | SOCK_STRATUM_BRIDGE); | ||
| 4295 | if (listener == INVALID_SOCKET) | ||
| 4296 | { | ||
| 4297 | printf("Stratum bridge: could not create listen socket\n"); | ||
| 4298 | return; | ||
| 4299 | } | ||
| 4300 | |||
| 4301 | int yes = 1; | ||
| 4302 | setsockopt(listener, SOL_SOCKET, SO_REUSEADDR, (char*)&yes, sizeof(yes)); | ||
| 4303 | |||
| 4304 | sockaddr_in addr; | ||
| 4305 | memset(&addr, 0, sizeof(addr)); | ||
| 4306 | addr.sin_family = AF_INET; | ||
| 4307 | addr.sin_addr.s_addr = htonl(0x7f000001); | ||
| 4308 | addr.sin_port = htons((unsigned short)nStratumBridgePort); | ||
| 4309 | |||
| 4310 | if (bind(listener, (sockaddr*)&addr, sizeof(addr)) != 0 || | ||
| 4311 | listen(listener, 16) != 0) | ||
| 4312 | { | ||
| 4313 | printf("Stratum bridge: could not listen on 127.0.0.1:%d\n", | ||
| 4314 | nStratumBridgePort); | ||
| 4315 | BtfCloseSocket(listener); | ||
| 4316 | return; | ||
| 4317 | } | ||
| 4318 | |||
| 4319 | printf("Stratum bridge: listening on 127.0.0.1:%d for pool %s\n", | ||
| 4320 | nStratumBridgePort, strParticipantPool.c_str()); | ||
| 4321 | |||
| 4322 | while (!fShutdown) | ||
| 4323 | { | ||
| 4324 | fd_set rd; | ||
| 4325 | FD_ZERO(&rd); | ||
| 4326 | FD_SET(listener, &rd); | ||
| 4327 | struct timeval tv = {1, 0}; | ||
| 4328 | if (select((int)listener + 1, &rd, NULL, NULL, &tv) <= 0) | ||
| 4329 | continue; | ||
| 4330 | |||
| 4331 | sockaddr_in cli; | ||
| 4332 | #ifdef _WIN32 | ||
| 4333 | int len = sizeof(cli); | ||
| 4334 | #else | ||
| 4335 | socklen_t len = sizeof(cli); | ||
| 4336 | #endif | ||
| 4337 | SOCKET miner = BtfSocketTag(accept(listener, (sockaddr*)&cli, &len), | ||
| 4338 | SOCK_STRATUM_BRIDGE); | ||
| 4339 | if (miner == INVALID_SOCKET) | ||
| 4340 | continue; | ||
| 4341 | |||
| 4342 | StratumBridgeClientCtx* ctx = new StratumBridgeClientCtx(); | ||
| 4343 | ctx->miner = miner; | ||
| 4344 | ctx->pool = strParticipantPool; | ||
| 4345 | if (_beginthread(ThreadStratumBridgeClient, 0, ctx) == (uintptr_t)-1) | ||
| 4346 | { | ||
| 4347 | delete ctx; | ||
| 4348 | BtfCloseSocket(miner); | ||
| 4349 | LogPrint("worker", "[bridge] failed to spawn client thread\n"); | ||
| 4350 | } | ||
| 4351 | } | ||
| 4352 | |||
| 4353 | BtfCloseSocket(listener); | ||
| 4354 | printf("Stratum bridge: stopped\n"); | ||
| 4355 | } | ||
| 4356 | |||
| 4131 | bool BitcoinMiner(int nThreadId) | 4357 | bool BitcoinMiner(int nThreadId) |
| 4132 | { | 4358 | { |
| 4133 | // Relay mode: never mines, just relays/syncs. (Shouldn't normally get here | 4359 | // Relay mode: never mines, just relays/syncs. (Shouldn't normally get here |
| @@ -97,7 +97,11 @@ extern bool fPoolMineBlind; // /poolmineblind: mine on even when deaf to | |||
| 97 | extern int nHashThreads; // participant: /hashthreads override (0 = cores-1) | 97 | extern int nHashThreads; // participant: /hashthreads override (0 = cores-1) |
| 98 | extern string strPoolName; // operator: announced pool name | 98 | extern string strPoolName; // operator: announced pool name |
| 99 | extern string strPoolDashboardUrl; // operator: optional dashboard URL | 99 | extern string strPoolDashboardUrl; // operator: optional dashboard URL |
| 100 | extern string strPoolStatusFile; // operator: optional JSON status output path | ||
| 101 | extern string strPoolRoundsFile; // operator: optional public round proof path | ||
| 100 | extern double dPoolFeePercent; // operator: announced fee percent | 102 | extern double dPoolFeePercent; // operator: announced fee percent |
| 103 | extern bool fStratumBridge; // participant helper: local Stratum bridge | ||
| 104 | extern int nStratumBridgePort; // local bridge listen port, default 3333 | ||
| 101 | extern bool fSoloMineTest; // /solomine: mine without requiring a peer (local test) | 105 | extern bool fSoloMineTest; // /solomine: mine without requiring a peer (local test) |
| 102 | // Threads to hash with; 0 = decide from the hardware. Set by /genproclimit. | 106 | // Threads to hash with; 0 = decide from the hardware. Set by /genproclimit. |
| 103 | extern int nMinerThreads; | 107 | extern int nMinerThreads; |
| @@ -226,9 +230,9 @@ static const unsigned int HD_BIP44_PURPOSE = 44; | |||
| 226 | static const unsigned int HD_BIP44_ACCOUNT = 0; | 230 | static const unsigned int HD_BIP44_ACCOUNT = 0; |
| 227 | static const unsigned int HD_BIP44_CHAIN_RECEIVE = 0; | 231 | static const unsigned int HD_BIP44_CHAIN_RECEIVE = 0; |
| 228 | static const unsigned int HD_BIP44_CHAIN_CHANGE = 1; | 232 | static const unsigned int HD_BIP44_CHAIN_CHANGE = 1; |
| 229 | // Provisional until Bitflash has an official SLIP-0044 assignment. | 233 | // Official SLIP-0044 assignment (registered 2026-08-07): |
| 230 | // Proposed registry row: 4346950 | BITFLASH | Bitflash. | 234 | // 4346950 | BITFLASH | Bitflash (4346950 == 0x425446 == "BTF") |
| 231 | static const unsigned int HD_BIP44_COIN_TYPE_BITFLASH_PROVISIONAL = 4346950; | 235 | static const unsigned int HD_BIP44_COIN_TYPE_BITFLASH = 4346950; |
| 232 | extern int nHDKeySchema; | 236 | extern int nHDKeySchema; |
| 233 | extern unsigned int nHDReceiveNext; // BIP44 external chain | 237 | extern unsigned int nHDReceiveNext; // BIP44 external chain |
| 234 | extern unsigned int nHDChangeNext; // BIP44 internal chain | 238 | extern unsigned int nHDChangeNext; // BIP44 internal chain |
| @@ -285,6 +289,7 @@ void LimitOrphanBlocks(unsigned int nMaxOrphans); | |||
| 285 | // distinguishable, which the dedup filter would otherwise collapse into one. | 289 | // distinguishable, which the dedup filter would otherwise collapse into one. |
| 286 | bool BitcoinMiner(int nThreadId = 1); | 290 | bool BitcoinMiner(int nThreadId = 1); |
| 287 | void ThreadRPCServer(void* parg); // rpc.cpp -- .btf pool server | 291 | void ThreadRPCServer(void* parg); // rpc.cpp -- .btf pool server |
| 292 | void ThreadStratumBridge(void* parg); // main.cpp -- local Stratum -> .btf bridge | ||
| 288 | void GetParticipantMiningStats(uint64& sharesSent, uint64& sharesAccepted, double& hashRate); | 293 | void GetParticipantMiningStats(uint64& sharesSent, uint64& sharesAccepted, double& hashRate); |
| 289 | void SetParticipantMiningStatus(const std::string& status); | 294 | void SetParticipantMiningStatus(const std::string& status); |
| 290 | std::string GetParticipantMiningStatus(); | 295 | std::string GetParticipantMiningStatus(); |
| @@ -325,7 +330,7 @@ struct WalletRecoveryAudit | |||
| 325 | nDerivedKnown = 0; | 330 | nDerivedKnown = 0; |
| 326 | nReceiveNext = 0; | 331 | nReceiveNext = 0; |
| 327 | nChangeNext = 0; | 332 | nChangeNext = 0; |
| 328 | nCoinType = HD_BIP44_COIN_TYPE_BITFLASH_PROVISIONAL; | 333 | nCoinType = HD_BIP44_COIN_TYPE_BITFLASH; |
| 329 | nRecoverableTx = 0; | 334 | nRecoverableTx = 0; |
| 330 | nLegacyTx = 0; | 335 | nLegacyTx = 0; |
| 331 | nRecoverableImmatureTx = 0; | 336 | nRecoverableImmatureTx = 0; |
diff --git a/src/main_gui.cpp b/src/main_gui.cpp index bee4b95..e941522 100644 --- a/src/main_gui.cpp +++ b/src/main_gui.cpp | |||
| @@ -1,6 +1,7 @@ | |||
| 1 | // Bitflash entry point -- starts node threads then runs GUI (or headless). | 1 | // Bitflash entry point -- starts node threads then runs GUI (or headless). |
| 2 | 2 | ||
| 3 | #include "headers_core.h" | 3 | #include "headers_core.h" |
| 4 | #include "proxy.h" | ||
| 4 | #include "selftest.h" | 5 | #include "selftest.h" |
| 5 | #include "walletcmd.h" | 6 | #include "walletcmd.h" |
| 6 | #include <thread> // hardware_concurrency, to sanity-check /genproclimit | 7 | #include <thread> // hardware_concurrency, to sanity-check /genproclimit |
| @@ -110,7 +111,8 @@ static void PrintUsage() | |||
| 110 | printf(" /debug\n"); | 111 | printf(" /debug\n"); |
| 111 | printf(" /gen\n"); | 112 | printf(" /gen\n"); |
| 112 | printf(" /nogui or /daemon\n"); | 113 | printf(" /nogui or /daemon\n"); |
| 113 | printf(" /selftest=wallet-keypool, wallet-hd, wallet-format, wallet-crypto, or wallet-encrypt\n"); | 114 | printf(" /selftest=wallet-keypool, wallet-hd, wallet-format, wallet-crypto, wallet-encrypt,\n"); |
| 115 | printf(" net-message, consensus-limits, pool-stratum, parse-money, or socks5-proxy\n"); | ||
| 114 | printf("\n"); | 116 | printf("\n"); |
| 115 | printf("Mining mode:\n"); | 117 | printf("Mining mode:\n"); |
| 116 | printf(" /operator\n"); | 118 | printf(" /operator\n"); |
| @@ -118,6 +120,8 @@ static void PrintUsage() | |||
| 118 | printf(" /payoutaddr=BTF_ADDRESS (participant: pay shares to this address, no wallet needed)\n"); | 120 | printf(" /payoutaddr=BTF_ADDRESS (participant: pay shares to this address, no wallet needed)\n"); |
| 119 | printf(" /payoutfile=FILE (participant: rotate payout addresses from file)\n"); | 121 | printf(" /payoutfile=FILE (participant: rotate payout addresses from file)\n"); |
| 120 | printf(" /hashthreads=N (participant: mining threads, default cores-1)\n"); | 122 | printf(" /hashthreads=N (participant: mining threads, default cores-1)\n"); |
| 123 | printf(" /stratumbridge=POOL_BTF_ADDRESS\n"); | ||
| 124 | printf(" /stratumbridgeport=N (default 3333; listen on 127.0.0.1)\n"); | ||
| 121 | printf(" /solomine\n"); | 125 | printf(" /solomine\n"); |
| 122 | printf(" /genproclimit=N (mining threads; 0 or absent = every core but one)\n"); | 126 | printf(" /genproclimit=N (mining threads; 0 or absent = every core but one)\n"); |
| 123 | printf(" /checkblocks=N (blocks re-verified at startup, default 288, 0 = all)\n"); | 127 | printf(" /checkblocks=N (blocks re-verified at startup, default 288, 0 = all)\n"); |
| @@ -126,6 +130,8 @@ static void PrintUsage() | |||
| 126 | printf(" /poolname=NAME\n"); | 130 | printf(" /poolname=NAME\n"); |
| 127 | printf(" /poolfee=PCT\n"); | 131 | printf(" /poolfee=PCT\n"); |
| 128 | printf(" /pooldashboard=URL (alias: /pooldash=URL)\n"); | 132 | printf(" /pooldashboard=URL (alias: /pooldash=URL)\n"); |
| 133 | printf(" /poolstatusfile=PATH (write pool status JSON for dashboards)\n"); | ||
| 134 | printf(" /poolroundsfile=PATH (write public pool round proofs JSON)\n"); | ||
| 129 | printf("\n"); | 135 | printf("\n"); |
| 130 | printf(".btf and rendezvous:\n"); | 136 | printf(".btf and rendezvous:\n"); |
| 131 | printf(" /connectbtf=PEER_BTF_ADDRESS\n"); | 137 | printf(" /connectbtf=PEER_BTF_ADDRESS\n"); |
| @@ -134,6 +140,8 @@ static void PrintUsage() | |||
| 134 | printf("\n"); | 140 | printf("\n"); |
| 135 | printf("Network:\n"); | 141 | printf("Network:\n"); |
| 136 | printf(" /port=N (P2P listen port, default 8433)\n"); | 142 | printf(" /port=N (P2P listen port, default 8433)\n"); |
| 143 | printf(" /socks=HOST:PORT (SOCKS5 proxy for outbound Nostr and .btf relay dials)\n"); | ||
| 144 | printf(" /tor[=HOST:PORT] (Tor mode; default SOCKS5 proxy is 127.0.0.1:9050)\n"); | ||
| 137 | printf(" /btfseed=ADDRESS:ENCHEX (extra bootstrap peer, repeatable)\n"); | 145 | printf(" /btfseed=ADDRESS:ENCHEX (extra bootstrap peer, repeatable)\n"); |
| 138 | printf("\n"); | 146 | printf("\n"); |
| 139 | printf("Wallet:\n"); | 147 | printf("Wallet:\n"); |
| @@ -240,6 +248,22 @@ static void ParseStartupArguments(int argc, char* argv[]) | |||
| 240 | if (!hashThreads.empty()) | 248 | if (!hashThreads.empty()) |
| 241 | nHashThreads = atoi(hashThreads.c_str()); | 249 | nHashThreads = atoi(hashThreads.c_str()); |
| 242 | 250 | ||
| 251 | if (arg(argc,argv,"/stratumbridge") || arg(argc,argv,"-stratumbridge")) | ||
| 252 | { | ||
| 253 | fStratumBridge = true; | ||
| 254 | string bridgePool = argval2(argc, argv, "/stratumbridge", "-stratumbridge"); | ||
| 255 | if (!bridgePool.empty()) | ||
| 256 | strParticipantPool = bridgePool; | ||
| 257 | } | ||
| 258 | |||
| 259 | string bridgePort = argval2(argc, argv, "/stratumbridgeport", "-stratumbridgeport"); | ||
| 260 | if (!bridgePort.empty()) | ||
| 261 | { | ||
| 262 | int nPort = atoi(bridgePort.c_str()); | ||
| 263 | if (nPort > 0 && nPort <= 65535) | ||
| 264 | nStratumBridgePort = nPort; | ||
| 265 | } | ||
| 266 | |||
| 243 | string poolName = argval2(argc, argv, "/poolname", "-poolname"); | 267 | string poolName = argval2(argc, argv, "/poolname", "-poolname"); |
| 244 | if (!poolName.empty()) | 268 | if (!poolName.empty()) |
| 245 | strPoolName = poolName; | 269 | strPoolName = poolName; |
| @@ -260,6 +284,14 @@ static void ParseStartupArguments(int argc, char* argv[]) | |||
| 260 | if (!poolFee.empty()) | 284 | if (!poolFee.empty()) |
| 261 | dPoolFeePercent = atof(poolFee.c_str()); | 285 | dPoolFeePercent = atof(poolFee.c_str()); |
| 262 | 286 | ||
| 287 | string poolStatusFile = argval2(argc, argv, "/poolstatusfile", "-poolstatusfile"); | ||
| 288 | if (!poolStatusFile.empty()) | ||
| 289 | strPoolStatusFile = poolStatusFile; | ||
| 290 | |||
| 291 | string poolRoundsFile = argval2(argc, argv, "/poolroundsfile", "-poolroundsfile"); | ||
| 292 | if (!poolRoundsFile.empty()) | ||
| 293 | strPoolRoundsFile = poolRoundsFile; | ||
| 294 | |||
| 263 | string btfConnect = argval2(argc, argv, "/connectbtf", "-connectbtf"); | 295 | string btfConnect = argval2(argc, argv, "/connectbtf", "-connectbtf"); |
| 264 | if (!btfConnect.empty()) | 296 | if (!btfConnect.empty()) |
| 265 | strBtfConnect = btfConnect; | 297 | strBtfConnect = btfConnect; |
| @@ -287,6 +319,36 @@ static void ParseStartupArguments(int argc, char* argv[]) | |||
| 287 | if (!announceRelay.empty()) | 319 | if (!announceRelay.empty()) |
| 288 | strBtfAnnounceRelay = announceRelay; | 320 | strBtfAnnounceRelay = announceRelay; |
| 289 | 321 | ||
| 322 | bool fTorMode = arg(argc, argv, "/tor") || arg(argc, argv, "-tor"); | ||
| 323 | if (fTorMode) | ||
| 324 | { | ||
| 325 | string socksProxy = argval2(argc, argv, "/socks", "-socks"); | ||
| 326 | if (!socksProxy.empty()) | ||
| 327 | fprintf(stderr, "Warning: /tor takes precedence over /socks=%s\n", | ||
| 328 | socksProxy.c_str()); | ||
| 329 | |||
| 330 | string torProxy = argval2(argc, argv, "/tor", "-tor"); | ||
| 331 | string err; | ||
| 332 | if (!BtfEnableTorProxy(torProxy, err)) | ||
| 333 | fprintf(stderr, "Ignoring /tor=%s: %s\n", torProxy.c_str(), err.c_str()); | ||
| 334 | else | ||
| 335 | fprintf(stderr, "Tor mode enabled through SOCKS5 proxy %s\n", | ||
| 336 | BtfSocks5ProxyName().c_str()); | ||
| 337 | } | ||
| 338 | else | ||
| 339 | { | ||
| 340 | string socksProxy = argval2(argc, argv, "/socks", "-socks"); | ||
| 341 | if (!socksProxy.empty()) | ||
| 342 | { | ||
| 343 | string err; | ||
| 344 | if (!BtfSetSocks5Proxy(socksProxy, err)) | ||
| 345 | fprintf(stderr, "Ignoring /socks=%s: %s\n", socksProxy.c_str(), err.c_str()); | ||
| 346 | else | ||
| 347 | fprintf(stderr, "SOCKS5 proxy enabled for outbound discovery/relay dials: %s\n", | ||
| 348 | BtfSocks5ProxyName().c_str()); | ||
| 349 | } | ||
| 350 | } | ||
| 351 | |||
| 290 | // /btfseed=ADDRESS:ENCHEX -- extra bootstrap peers, repeatable. Useful for | 352 | // /btfseed=ADDRESS:ENCHEX -- extra bootstrap peers, repeatable. Useful for |
| 291 | // testing the seed path and for private networks that ship no compiled list. | 353 | // testing the seed path and for private networks that ship no compiled list. |
| 292 | for (int i = 1; i < argc; i++) | 354 | for (int i = 1; i < argc; i++) |
| @@ -368,6 +430,36 @@ static void FatalStartupError(bool fHeadless, const string& strWhat, const strin | |||
| 368 | #endif | 430 | #endif |
| 369 | } | 431 | } |
| 370 | 432 | ||
| 433 | #ifdef _WIN32 | ||
| 434 | // Windows equivalent of the SIGINT/SIGTERM handler the headless loop installs on | ||
| 435 | // POSIX. Without it a headless node on Windows had no way to reach the clean | ||
| 436 | // StopNode()+DBFlush() path: Ctrl-C, closing the console window, a service stop | ||
| 437 | // and OS shutdown all killed it mid-write, and a wallet.dat left unflushed by | ||
| 438 | // Berkeley DB will not reopen. For CLOSE/LOGOFF/SHUTDOWN the OS terminates us | ||
| 439 | // shortly after the handler returns, so we set fShutdown and then block until the | ||
| 440 | // main loop signals the flush is done -- returning any sooner throws the flush | ||
| 441 | // away, which is the whole bug. (A `taskkill /F` is still an uncatchable hard | ||
| 442 | // kill; nothing can help there, and nothing should pretend to.) | ||
| 443 | static volatile bool g_fHeadlessShutdownDone = false; | ||
| 444 | static BOOL WINAPI HeadlessConsoleCtrlHandler(DWORD dwCtrlType) | ||
| 445 | { | ||
| 446 | switch (dwCtrlType) | ||
| 447 | { | ||
| 448 | case CTRL_C_EVENT: | ||
| 449 | case CTRL_BREAK_EVENT: | ||
| 450 | case CTRL_CLOSE_EVENT: | ||
| 451 | case CTRL_LOGOFF_EVENT: | ||
| 452 | case CTRL_SHUTDOWN_EVENT: | ||
| 453 | fShutdown = true; | ||
| 454 | while (!g_fHeadlessShutdownDone) | ||
| 455 | Sleep(50); | ||
| 456 | return TRUE; | ||
| 457 | default: | ||
| 458 | return FALSE; | ||
| 459 | } | ||
| 460 | } | ||
| 461 | #endif | ||
| 462 | |||
| 371 | int main(int argc, char* argv[]) | 463 | int main(int argc, char* argv[]) |
| 372 | { | 464 | { |
| 373 | if (arg(argc,argv,"/help") || arg(argc,argv,"-help") || | 465 | if (arg(argc,argv,"/help") || arg(argc,argv,"-help") || |
| @@ -676,6 +768,10 @@ int main(int argc, char* argv[]) | |||
| 676 | if (_beginthread(ThreadRPCServer, 0, NULL) == (uintptr_t)-1) | 768 | if (_beginthread(ThreadRPCServer, 0, NULL) == (uintptr_t)-1) |
| 677 | printf("Error: _beginthread(ThreadRPCServer) failed\n"); | 769 | printf("Error: _beginthread(ThreadRPCServer) failed\n"); |
| 678 | } | 770 | } |
| 771 | if (fStratumBridge) { | ||
| 772 | if (_beginthread(ThreadStratumBridge, 0, NULL) == (uintptr_t)-1) | ||
| 773 | printf("Error: _beginthread(ThreadStratumBridge) failed\n"); | ||
| 774 | } | ||
| 679 | if (fGenerateBitcoins) | 775 | if (fGenerateBitcoins) |
| 680 | StartMinerThreads(); | 776 | StartMinerThreads(); |
| 681 | 777 | ||
| @@ -688,7 +784,9 @@ int main(int argc, char* argv[]) | |||
| 688 | #endif | 784 | #endif |
| 689 | 785 | ||
| 690 | if (fHeadless) { | 786 | if (fHeadless) { |
| 691 | #ifndef _WIN32 | 787 | #ifdef _WIN32 |
| 788 | SetConsoleCtrlHandler(HeadlessConsoleCtrlHandler, TRUE); | ||
| 789 | #else | ||
| 692 | auto sig=[](int){fShutdown=true;}; | 790 | auto sig=[](int){fShutdown=true;}; |
| 693 | signal(SIGINT,sig); signal(SIGTERM,sig); | 791 | signal(SIGINT,sig); signal(SIGTERM,sig); |
| 694 | #endif | 792 | #endif |
| @@ -700,6 +798,11 @@ int main(int argc, char* argv[]) | |||
| 700 | // called here, which is why a wallet.dat copied elsewhere would not | 798 | // called here, which is why a wallet.dat copied elsewhere would not |
| 701 | // open -- issue #40. Not optional. | 799 | // open -- issue #40. Not optional. |
| 702 | DBFlush(true); | 800 | DBFlush(true); |
| 801 | #ifdef _WIN32 | ||
| 802 | // wallet.dat is safely flushed and closed now; let a pending console | ||
| 803 | // control handler (CLOSE/LOGOFF/SHUTDOWN) return so the OS can finish. | ||
| 804 | g_fHeadlessShutdownDone = true; | ||
| 805 | #endif | ||
| 703 | return 0; | 806 | return 0; |
| 704 | } | 807 | } |
| 705 | 808 | ||
diff --git a/src/makefile.mingw b/src/makefile.mingw index 72f8bbf..8b49cac 100644 --- a/src/makefile.mingw +++ b/src/makefile.mingw | |||
| @@ -14,14 +14,9 @@ WARNFLAGS = -Wformat -Werror=format \ | |||
| 14 | -Wno-deprecated-declarations -Wno-write-strings \ | 14 | -Wno-deprecated-declarations -Wno-write-strings \ |
| 15 | -Wno-narrowing -Wno-unused-result | 15 | -Wno-narrowing -Wno-unused-result |
| 16 | 16 | ||
| 17 | # FD_SETSIZE has to be defined before winsock2.h and it has to be the same | ||
| 18 | # number in every translation unit -- fd_set carries an array of that length, | ||
| 19 | # so disagreeing about it means disagreeing about sizeof(fd_set). That is why | ||
| 20 | # it lives here as a -D and not as a #define in a header. Windows defaults to | ||
| 21 | # 64, which capped this node at 63 watchable peers no matter how many it had. | ||
| 22 | CXXFLAGS = -std=gnu++14 -O2 $(WARNFLAGS) -fpermissive -pthread \ | 17 | CXXFLAGS = -std=gnu++14 -O2 $(WARNFLAGS) -fpermissive -pthread \ |
| 23 | -DWIN32 -DNOPCH -DFD_SETSIZE=1024 \ | 18 | -DWIN32 -DNOPCH \ |
| 24 | -DSECP256K1_STATIC \ | 19 | -DSECP256K1_STATIC -DBITFLASH_NODE_BUILD \ |
| 25 | -I$(DEPS)/include -I$(RANDOMX)/src \ | 20 | -I$(DEPS)/include -I$(RANDOMX)/src \ |
| 26 | -I$(DEPS)/include/nlohmann \ | 21 | -I$(DEPS)/include/nlohmann \ |
| 27 | -IC:/msys64/ucrt64/include \ | 22 | -IC:/msys64/ucrt64/include \ |
| @@ -42,7 +37,7 @@ IMGUI_OBJS = obj/imgui.o obj/imgui_draw.o obj/imgui_tables.o \ | |||
| 42 | NODE_OBJS = obj/util.o obj/script.o obj/db.o obj/net.o obj/main.o \ | 37 | NODE_OBJS = obj/util.o obj/script.o obj/db.o obj/net.o obj/main.o \ |
| 43 | obj/sha.o obj/nostr.o obj/randomx_pow.o \ | 38 | obj/sha.o obj/nostr.o obj/randomx_pow.o \ |
| 44 | obj/btfaddr.o obj/btfchan.o obj/btfrv.o obj/btftunnel.o obj/rpc.o \ | 39 | obj/btfaddr.o obj/btfchan.o obj/btfrv.o obj/btftunnel.o obj/rpc.o \ |
| 45 | obj/selftest.o obj/bip32.o obj/walletcmd.o | 40 | obj/selftest.o obj/bip32.o obj/walletcmd.o obj/proxy.o |
| 46 | 41 | ||
| 47 | GUI_OBJS = obj/gui.o obj/main_gui.o obj/bitflash_res.o | 42 | GUI_OBJS = obj/gui.o obj/main_gui.o obj/bitflash_res.o |
| 48 | 43 | ||
| @@ -56,7 +51,7 @@ GUI_OBJS = obj/gui.o obj/main_gui.o obj/bitflash_res.o | |||
| 56 | HEADERS = headers_core.h headers.h util.h main.h serialize.h uint256.h \ | 51 | HEADERS = headers_core.h headers.h util.h main.h serialize.h uint256.h \ |
| 57 | key.h bignum.h script.h db.h base58.h net.h nostr.h \ | 52 | key.h bignum.h script.h db.h base58.h net.h nostr.h \ |
| 58 | randomx_pow.h btfaddr.h btfchan.h btfrv.h btftunnel.h \ | 53 | randomx_pow.h btfaddr.h btfchan.h btfrv.h btftunnel.h \ |
| 59 | selftest.h walletcmd.h sockcount.h compat.h font_roboto.h | 54 | selftest.h walletcmd.h sockcount.h proxy.h compat.h font_roboto.h |
| 60 | 55 | ||
| 61 | all: bitflash.exe | 56 | all: bitflash.exe |
| 62 | 57 | ||
| @@ -116,3 +111,5 @@ tests: test_bip32.exe bitflash.exe | |||
| 116 | ./bitflash.exe -selftest=wallet-encrypt | 111 | ./bitflash.exe -selftest=wallet-encrypt |
| 117 | ./bitflash.exe -selftest=net-message | 112 | ./bitflash.exe -selftest=net-message |
| 118 | ./bitflash.exe -selftest=consensus-limits | 113 | ./bitflash.exe -selftest=consensus-limits |
| 114 | ./bitflash.exe -selftest=pool-stratum | ||
| 115 | ./bitflash.exe -selftest=socks5-proxy | ||
diff --git a/src/net.cpp b/src/net.cpp index 34c608c..bb39122 100644 --- a/src/net.cpp +++ b/src/net.cpp | |||
| @@ -13,15 +13,32 @@ | |||
| 13 | #include <thread> // hardware_concurrency, for the miner thread count | 13 | #include <thread> // hardware_concurrency, for the miner thread count |
| 14 | #ifdef _WIN32 | 14 | #ifdef _WIN32 |
| 15 | #include <winsock2.h> | 15 | #include <winsock2.h> |
| 16 | #else | ||
| 17 | #include <poll.h> | ||
| 16 | #endif | 18 | #endif |
| 17 | #include <openssl/rand.h> | 19 | #include <openssl/rand.h> |
| 18 | #include "btfaddr.h" | 20 | #include "btfaddr.h" |
| 19 | #include "btftunnel.h" | 21 | #include "btftunnel.h" |
| 22 | #include "proxy.h" | ||
| 20 | 23 | ||
| 21 | void ThreadReconnectCachedBtfPeers(void* parg); | 24 | void ThreadReconnectCachedBtfPeers(void* parg); |
| 22 | void ThreadMessageHandler2(void* parg); | 25 | void ThreadMessageHandler2(void* parg); |
| 23 | void ThreadSocketHandler2(void* parg); | 26 | void ThreadSocketHandler2(void* parg); |
| 24 | 27 | ||
| 28 | #ifdef _WIN32 | ||
| 29 | typedef WSAPOLLFD BtfPollFd; | ||
| 30 | static int BtfPoll(BtfPollFd* pfd, unsigned int nfd, int nTimeoutMs) | ||
| 31 | { | ||
| 32 | return WSAPoll(pfd, nfd, nTimeoutMs); | ||
| 33 | } | ||
| 34 | #else | ||
| 35 | typedef struct pollfd BtfPollFd; | ||
| 36 | static int BtfPoll(BtfPollFd* pfd, unsigned int nfd, int nTimeoutMs) | ||
| 37 | { | ||
| 38 | return poll(pfd, (nfds_t)nfd, nTimeoutMs); | ||
| 39 | } | ||
| 40 | #endif | ||
| 41 | |||
| 25 | 42 | ||
| 26 | 43 | ||
| 27 | 44 | ||
| @@ -153,7 +170,7 @@ static const char* pszSockSite[SOCK_SITES] = { | |||
| 153 | "external-ip probe", "listen socket", "inbound accept", | 170 | "external-ip probe", "listen socket", "inbound accept", |
| 154 | "rendezvous dial", "rendezvous listen", "rendezvous accept", | 171 | "rendezvous dial", "rendezvous listen", "rendezvous accept", |
| 155 | "loopback pair listener", "loopback pair app end", "loopback pair pump end", | 172 | "loopback pair listener", "loopback pair app end", "loopback pair pump end", |
| 156 | "nostr relay" | 173 | "nostr relay", "stratum bridge" |
| 157 | }; | 174 | }; |
| 158 | 175 | ||
| 159 | static string SockAccountingText() | 176 | static string SockAccountingText() |
| @@ -274,8 +291,8 @@ string GetDiagnosticsText() | |||
| 274 | 291 | ||
| 275 | // The number that would have made the deafness obvious. Anything held but | 292 | // The number that would have made the deafness obvious. Anything held but |
| 276 | // not watched is a socket this node will never read again. | 293 | // not watched is a socket this node will never read again. |
| 277 | str += strprintf(" peers watched %d of %d that select() can hold%s\n", | 294 | str += strprintf(" peers watched %d of %d held by poll%s\n", |
| 278 | nPeersWatched, (int)FD_SETSIZE - 1, | 295 | nPeersWatched, nHeld, |
| 279 | nHeld > nPeersWatched ? " <-- NOT ALL PEERS ARE BEING READ" : ""); | 296 | nHeld > nPeersWatched ? " <-- NOT ALL PEERS ARE BEING READ" : ""); |
| 280 | 297 | ||
| 281 | if (nBlocksReceived > 0) | 298 | if (nBlocksReceived > 0) |
| @@ -361,6 +378,9 @@ string GetDiagnosticsText() | |||
| 361 | 378 | ||
| 362 | bool GetMyExternalIP(unsigned int& ipRet) | 379 | bool GetMyExternalIP(unsigned int& ipRet) |
| 363 | { | 380 | { |
| 381 | if (BtfSocks5ProxyEnabled()) | ||
| 382 | return error("GetMyExternalIP() skipped while SOCKS5 proxy is enabled\n"); | ||
| 383 | |||
| 364 | // Try several plain-text IP echo services in order. | 384 | // Try several plain-text IP echo services in order. |
| 365 | // Each returns just the IPv4 address as the first line of the HTTP body. | 385 | // Each returns just the IPv4 address as the first line of the HTTP body. |
| 366 | struct { const char* host; const char* path; } services[] = { | 386 | struct { const char* host; const char* path; } services[] = { |
| @@ -1127,7 +1147,7 @@ void CNode::Disconnect() | |||
| 1127 | 1147 | ||
| 1128 | // Invalidate before anything else can look at it. This used to close the | 1148 | // Invalidate before anything else can look at it. This used to close the |
| 1129 | // socket and leave the handle number sitting in the object, so ~CNode | 1149 | // socket and leave the handle number sitting in the object, so ~CNode |
| 1130 | // closed it a second time and the select loop kept arming it in the | 1150 | // closed it a second time and the socket loop kept arming it in the |
| 1131 | // meantime. See the note on ~CNode in net.h. | 1151 | // meantime. See the note on ~CNode in net.h. |
| 1132 | BtfCloseSocket(hSocket); | 1152 | BtfCloseSocket(hSocket); |
| 1133 | hSocket = INVALID_SOCKET; | 1153 | hSocket = INVALID_SOCKET; |
| @@ -1276,52 +1296,49 @@ void ThreadSocketHandler2(void* parg) | |||
| 1276 | // | 1296 | // |
| 1277 | // Find which sockets have data to receive | 1297 | // Find which sockets have data to receive |
| 1278 | // | 1298 | // |
| 1279 | struct timeval timeout; | 1299 | static const int nPollTimeoutMs = 50; // frequency to poll pnode->vSend |
| 1280 | timeout.tv_sec = 0; | 1300 | |
| 1281 | timeout.tv_usec = 50000; // frequency to poll pnode->vSend | 1301 | struct NodePollResult |
| 1282 | 1302 | { | |
| 1283 | fd_set fdsetRecv; | 1303 | CNode* pnode; |
| 1284 | fd_set fdsetSend; | 1304 | SOCKET hSocket; |
| 1285 | FD_ZERO(&fdsetRecv); | 1305 | bool fRecv; |
| 1286 | FD_ZERO(&fdsetSend); | 1306 | bool fSend; |
| 1287 | SOCKET hSocketMax = 0; | 1307 | bool fInvalid; |
| 1288 | FD_SET(hListenSocket, &fdsetRecv); | 1308 | NodePollResult(CNode* pnodeIn, SOCKET hSocketIn) |
| 1289 | hSocketMax = max(hSocketMax, hListenSocket); | 1309 | : pnode(pnodeIn), hSocket(hSocketIn), fRecv(false), fSend(false), fInvalid(false) {} |
| 1290 | // FD_SET drops silently once the set is full, so count what goes in | 1310 | }; |
| 1291 | // and say so. A peer that never makes it into the set is never read, | 1311 | |
| 1292 | // and that is indistinguishable from a peer with nothing to say -- | 1312 | vector<BtfPollFd> vPoll; |
| 1293 | // the failure this whole path used to hide. | 1313 | vector<NodePollResult> vPollNodes; |
| 1294 | unsigned int nWatched = 1; // the listen socket | 1314 | |
| 1315 | BtfPollFd listenPoll; | ||
| 1316 | memset(&listenPoll, 0, sizeof(listenPoll)); | ||
| 1317 | listenPoll.fd = hListenSocket; | ||
| 1318 | listenPoll.events = POLLIN; | ||
| 1319 | vPoll.push_back(listenPoll); | ||
| 1320 | |||
| 1295 | CRITICAL_BLOCK(cs_vNodes) | 1321 | CRITICAL_BLOCK(cs_vNodes) |
| 1296 | { | 1322 | { |
| 1297 | foreach(CNode* pnode, vNodes) | 1323 | foreach(CNode* pnode, vNodes) |
| 1298 | { | 1324 | { |
| 1299 | if (nWatched >= FD_SETSIZE) | ||
| 1300 | break; | ||
| 1301 | // A disconnected node stays in vNodes until its references go. | 1325 | // A disconnected node stays in vNodes until its references go. |
| 1302 | // Arming its closed handle asks select() to watch a number the | 1326 | // Do not watch a closed handle number: the OS may already have |
| 1303 | // operating system may already have given to somebody else. | 1327 | // handed it to a different connection. |
| 1304 | if (pnode->hSocket == INVALID_SOCKET) | 1328 | if (pnode->hSocket == INVALID_SOCKET) |
| 1305 | continue; | 1329 | continue; |
| 1306 | FD_SET(pnode->hSocket, &fdsetRecv); | 1330 | |
| 1307 | nWatched++; | 1331 | BtfPollFd pfd; |
| 1308 | hSocketMax = max(hSocketMax, pnode->hSocket); | 1332 | memset(&pfd, 0, sizeof(pfd)); |
| 1333 | pfd.fd = pnode->hSocket; | ||
| 1334 | pfd.events = POLLIN; | ||
| 1309 | TRY_CRITICAL_BLOCK(pnode->cs_vSend) | 1335 | TRY_CRITICAL_BLOCK(pnode->cs_vSend) |
| 1310 | if (!pnode->vSend.empty()) | 1336 | if (!pnode->vSend.empty()) |
| 1311 | FD_SET(pnode->hSocket, &fdsetSend); | 1337 | pfd.events |= POLLOUT; |
| 1312 | } | 1338 | vPoll.push_back(pfd); |
| 1313 | if (vNodes.size() + 1 > (size_t)FD_SETSIZE) | 1339 | vPollNodes.push_back(NodePollResult(pnode, pnode->hSocket)); |
| 1314 | { | ||
| 1315 | static int64 nLastWarned = 0; | ||
| 1316 | if (GetTime() - nLastWarned > 60) | ||
| 1317 | { | ||
| 1318 | nLastWarned = GetTime(); | ||
| 1319 | printf("WARNING: %d peers but select() can only watch %d -- %d are not being read\n", | ||
| 1320 | (int)vNodes.size(), (int)FD_SETSIZE - 1, | ||
| 1321 | (int)vNodes.size() - (int)FD_SETSIZE + 1); | ||
| 1322 | } | ||
| 1323 | } | 1340 | } |
| 1324 | nPeersWatched = (int)nWatched - 1; // less the listen socket | 1341 | nPeersWatched = (int)vPollNodes.size(); |
| 1325 | } | 1342 | } |
| 1326 | 1343 | ||
| 1327 | // Say the state of the node out loud now and then, so a log pulled off | 1344 | // Say the state of the node out loud now and then, so a log pulled off |
| @@ -1342,32 +1359,17 @@ void ThreadSocketHandler2(void* parg) | |||
| 1342 | } | 1359 | } |
| 1343 | 1360 | ||
| 1344 | vfThreadRunning[0] = false; | 1361 | vfThreadRunning[0] = false; |
| 1345 | int nSelect = select(hSocketMax + 1, &fdsetRecv, &fdsetSend, NULL, &timeout); | 1362 | int nPoll = BtfPoll(&vPoll[0], (unsigned int)vPoll.size(), nPollTimeoutMs); |
| 1346 | vfThreadRunning[0] = true; | 1363 | vfThreadRunning[0] = true; |
| 1347 | CheckForShutdown(0); | 1364 | CheckForShutdown(0); |
| 1348 | if (nSelect == SOCKET_ERROR) | 1365 | if (nPoll == SOCKET_ERROR) |
| 1349 | { | 1366 | { |
| 1350 | // This used to answer a failed select() by marking every | ||
| 1351 | // descriptor from 0 to hSocketMax ready, which is code written for | ||
| 1352 | // POSIX, where a descriptor is a small integer. A Windows SOCKET is | ||
| 1353 | // a kernel handle in the thousands, so the loop iterated over | ||
| 1354 | // numbers that are not sockets and told the rest of the function | ||
| 1355 | // that every peer had data. One bad handle in the set makes | ||
| 1356 | // select() fail on every pass, so a single closed socket put the | ||
| 1357 | // whole thread into that state permanently. | ||
| 1358 | // | ||
| 1359 | // Find the bad handle instead and let it be dropped. | ||
| 1360 | int nErr = WSAGetLastError(); | 1367 | int nErr = WSAGetLastError(); |
| 1361 | LogPrint("net", "select failed: %d\n", nErr); | 1368 | LogPrint("net", "poll failed: %d\n", nErr); |
| 1362 | FD_ZERO(&fdsetRecv); | ||
| 1363 | FD_ZERO(&fdsetSend); | ||
| 1364 | CRITICAL_BLOCK(cs_vNodes) | 1369 | CRITICAL_BLOCK(cs_vNodes) |
| 1365 | { | 1370 | { |
| 1366 | foreach(CNode* pnode, vNodes) | 1371 | foreach(CNode* pnode, vNodes) |
| 1367 | { | 1372 | { |
| 1368 | // Already disconnected: its handle is gone on purpose, and | ||
| 1369 | // reporting it here would name the wrong node for a select() | ||
| 1370 | // failure it did not cause. | ||
| 1371 | if (pnode->hSocket == INVALID_SOCKET) | 1373 | if (pnode->hSocket == INVALID_SOCKET) |
| 1372 | continue; | 1374 | continue; |
| 1373 | int nType = 0; | 1375 | int nType = 0; |
| @@ -1376,24 +1378,27 @@ void ThreadSocketHandler2(void* parg) | |||
| 1376 | #else | 1378 | #else |
| 1377 | socklen_t nTypeLen = sizeof(nType); | 1379 | socklen_t nTypeLen = sizeof(nType); |
| 1378 | #endif | 1380 | #endif |
| 1379 | if (pnode->hSocket != INVALID_SOCKET && | 1381 | if (getsockopt(pnode->hSocket, SOL_SOCKET, SO_TYPE, (char*)&nType, &nTypeLen) != 0) |
| 1380 | getsockopt(pnode->hSocket, SOL_SOCKET, SO_TYPE, (char*)&nType, &nTypeLen) != 0) | ||
| 1381 | { | 1382 | { |
| 1382 | printf("dropping node %s: its socket is no longer valid\n", | 1383 | printf("dropping node %s: its socket is no longer valid\n", |
| 1383 | pnode->addr.ToString().c_str()); | 1384 | pnode->addr.ToString().c_str()); |
| 1384 | pnode->fDisconnect = true; | 1385 | pnode->fDisconnect = true; |
| 1385 | 1386 | ||
| 1386 | // Stop referring to it, or it goes back into the next | 1387 | // Stop referring to it, or it goes into the next poll |
| 1387 | // select() set and fails it again. | 1388 | // set and fails it again. |
| 1388 | // | 1389 | // |
| 1389 | // Marking fDisconnect is not enough: a node is only | 1390 | // Marking fDisconnect is not enough: a node is only |
| 1390 | // removed from vNodes once it has drained or once | 1391 | // removed from vNodes once it has drained or once |
| 1391 | // DISCONNECT_DRAIN_SECS has passed, so one dead | 1392 | // DISCONNECT_DRAIN_SECS has passed, so one dead |
| 1392 | // descriptor kept failing select() for a full minute -- | 1393 | // descriptor kept failing the wait for a full minute -- |
| 1393 | // and a failed select() zeroes both sets above, which | 1394 | // and a failed wait skips the service loop below |
| 1394 | // means NO peer is read on that pass. Measured on a | 1395 | // entirely, which means NO peer is read on that pass. |
| 1395 | // wedged pool: 1,027 consecutive passes over 61 | 1396 | // Measured on a wedged pool under the old select() |
| 1396 | // seconds, one bad socket, the entire node deaf. | 1397 | // implementation: 1,027 consecutive passes over 61 |
| 1398 | // seconds, one bad socket, the entire node deaf. poll() | ||
| 1399 | // reports a bad descriptor as POLLNVAL on that entry | ||
| 1400 | // rather than failing the whole call, so this path is | ||
| 1401 | // now the backstop rather than the common case. | ||
| 1397 | // | 1402 | // |
| 1398 | // Not closed, only forgotten. getsockopt just told us | 1403 | // Not closed, only forgotten. getsockopt just told us |
| 1399 | // this descriptor is not ours any more; closing it now | 1404 | // this descriptor is not ours any more; closing it now |
| @@ -1402,7 +1407,17 @@ void ThreadSocketHandler2(void* parg) | |||
| 1402 | } | 1407 | } |
| 1403 | } | 1408 | } |
| 1404 | } | 1409 | } |
| 1405 | Sleep(timeout.tv_usec/1000); | 1410 | Sleep(nPollTimeoutMs); |
| 1411 | } | ||
| 1412 | else | ||
| 1413 | { | ||
| 1414 | for (size_t i = 0; i < vPollNodes.size(); i++) | ||
| 1415 | { | ||
| 1416 | short revents = vPoll[i + 1].revents; // poll[0] is the listener | ||
| 1417 | vPollNodes[i].fRecv = (revents & (POLLIN | POLLERR | POLLHUP)) != 0; | ||
| 1418 | vPollNodes[i].fSend = (revents & POLLOUT) != 0; | ||
| 1419 | vPollNodes[i].fInvalid = (revents & POLLNVAL) != 0; | ||
| 1420 | } | ||
| 1406 | } | 1421 | } |
| 1407 | RandAddSeed(); | 1422 | RandAddSeed(); |
| 1408 | 1423 | ||
| @@ -1418,7 +1433,7 @@ void ThreadSocketHandler2(void* parg) | |||
| 1418 | // | 1433 | // |
| 1419 | // Accept new connections | 1434 | // Accept new connections |
| 1420 | // | 1435 | // |
| 1421 | if (FD_ISSET(hListenSocket, &fdsetRecv)) | 1436 | if (nPoll != SOCKET_ERROR && (vPoll[0].revents & POLLIN)) |
| 1422 | { | 1437 | { |
| 1423 | struct sockaddr_in sockaddr; | 1438 | struct sockaddr_in sockaddr; |
| 1424 | #ifdef _WIN32 | 1439 | #ifdef _WIN32 |
| @@ -1437,7 +1452,7 @@ void ThreadSocketHandler2(void* parg) | |||
| 1437 | { | 1452 | { |
| 1438 | // Refuse rather than accept a connection this node cannot | 1453 | // Refuse rather than accept a connection this node cannot |
| 1439 | // watch. Without this the listen socket kept taking peers -- | 1454 | // watch. Without this the listen socket kept taking peers -- |
| 1440 | // it is always in the select set -- while everything past the | 1455 | // it is always watched -- while everything past the |
| 1441 | // limit sat open and unread, which fed on itself: the deafer | 1456 | // limit sat open and unread, which fed on itself: the deafer |
| 1442 | // the node got, the more connections it collected. | 1457 | // the node got, the more connections it collected. |
| 1443 | unsigned int nNodes = 0; | 1458 | unsigned int nNodes = 0; |
| @@ -1464,22 +1479,28 @@ void ThreadSocketHandler2(void* parg) | |||
| 1464 | // | 1479 | // |
| 1465 | // Service each socket | 1480 | // Service each socket |
| 1466 | // | 1481 | // |
| 1467 | vector<CNode*> vNodesCopy; | 1482 | for (size_t i = 0; i < vPollNodes.size(); i++) |
| 1468 | CRITICAL_BLOCK(cs_vNodes) | ||
| 1469 | vNodesCopy = vNodes; | ||
| 1470 | foreach(CNode* pnode, vNodesCopy) | ||
| 1471 | { | 1483 | { |
| 1472 | CheckForShutdown(0); | 1484 | CheckForShutdown(0); |
| 1473 | SOCKET hSocket = pnode->hSocket; | 1485 | NodePollResult& poll = vPollNodes[i]; |
| 1474 | // Forgotten above because it was already invalid. FD_ISSET on it | 1486 | CNode* pnode = poll.pnode; |
| 1475 | // is meaningless and recv()/send() would be operating on -1. | 1487 | SOCKET hSocket = poll.hSocket; |
| 1476 | if (hSocket == INVALID_SOCKET) | 1488 | |
| 1489 | if (hSocket == INVALID_SOCKET || pnode->hSocket != hSocket) | ||
| 1477 | continue; | 1490 | continue; |
| 1478 | 1491 | ||
| 1492 | if (poll.fInvalid) | ||
| 1493 | { | ||
| 1494 | LogPrint("net", "poll reported invalid socket for %s\n", | ||
| 1495 | pnode->addr.ToString().c_str()); | ||
| 1496 | pnode->fDisconnect = true; | ||
| 1497 | continue; | ||
| 1498 | } | ||
| 1499 | |||
| 1479 | // | 1500 | // |
| 1480 | // Receive | 1501 | // Receive |
| 1481 | // | 1502 | // |
| 1482 | if (FD_ISSET(hSocket, &fdsetRecv)) | 1503 | if (poll.fRecv) |
| 1483 | { | 1504 | { |
| 1484 | TRY_CRITICAL_BLOCK(pnode->cs_vRecv) | 1505 | TRY_CRITICAL_BLOCK(pnode->cs_vRecv) |
| 1485 | { | 1506 | { |
| @@ -1519,7 +1540,7 @@ void ThreadSocketHandler2(void* parg) | |||
| 1519 | // | 1540 | // |
| 1520 | // Send | 1541 | // Send |
| 1521 | // | 1542 | // |
| 1522 | if (FD_ISSET(hSocket, &fdsetSend)) | 1543 | if (poll.fSend) |
| 1523 | { | 1544 | { |
| 1524 | TRY_CRITICAL_BLOCK(pnode->cs_vSend) | 1545 | TRY_CRITICAL_BLOCK(pnode->cs_vSend) |
| 1525 | { | 1546 | { |
| @@ -95,19 +95,16 @@ static const unsigned int MAX_PROTOCOL_MESSAGE_SIZE = MAX_SIZE; | |||
| 95 | 95 | ||
| 96 | // Ceiling on simultaneous connections. | 96 | // Ceiling on simultaneous connections. |
| 97 | // | 97 | // |
| 98 | // ThreadSocketHandler watches every peer through one select(), and select() | 98 | // ThreadSocketHandler watches every peer in one socket loop. This used to be a |
| 99 | // cannot watch more descriptors than FD_SETSIZE. On Windows FD_SET simply | 99 | // select() loop, which made FD_SETSIZE the real limit and silently starved |
| 100 | // stops adding once the set is full -- no error, no log -- so every socket | 100 | // sockets past that bitmap. It is now poll()/WSAPoll(), so the ceiling below is |
| 101 | // past the limit stays open and is never read again. Nothing capped vNodes, | 101 | // a deliberate resource bound rather than a descriptor-set accident. |
| 102 | // so a node accepted connections it had no way to service, and because new | ||
| 103 | // entries go on the end of vNodes it was always the newest connection that | ||
| 104 | // starved, including the ones dialed to fix connectivity. | ||
| 105 | // | 102 | // |
| 106 | // Measured on two nodes before this ceiling existed: 277 and 90 CNode objects | 103 | // Measured on two nodes before this ceiling existed: 277 and 90 CNode objects |
| 107 | // against 93 and 90 live sockets, and a status bar reporting hundreds of | 104 | // against 93 and 90 live sockets, and a status bar reporting hundreds of |
| 108 | // peers for a node that was really talking to thirty. | 105 | // peers for a node that was really talking to thirty. |
| 109 | // | 106 | // |
| 110 | // 125 is Bitcoin's number and sits far below FD_SETSIZE on both platforms. | 107 | // 125 is Bitcoin's number and remains a conservative public-node default. |
| 111 | static const unsigned int MAX_CONNECTIONS = 125; | 108 | static const unsigned int MAX_CONNECTIONS = 125; |
| 112 | 109 | ||
| 113 | // A node marked for disconnect is normally held until its buffers drain, so | 110 | // A node marked for disconnect is normally held until its buffers drain, so |
| @@ -175,7 +172,7 @@ static const int BTF_BEHIND_WARN_INTERVAL_SECS = 5 * 60; | |||
| 175 | // | 172 | // |
| 176 | // These counters exist so it can. Cheap to keep, and they name the failures | 173 | // These counters exist so it can. Cheap to keep, and they name the failures |
| 177 | // that actually happened rather than the ones that sound impressive. | 174 | // that actually happened rather than the ones that sound impressive. |
| 178 | extern int nPeersWatched; // peers in the last select() set | 175 | extern int nPeersWatched; // peers in the last poll() set |
| 179 | extern int64 nBlocksReceived; // blocks handed to ProcessBlock | 176 | extern int64 nBlocksReceived; // blocks handed to ProcessBlock |
| 180 | extern int64 nBlocksWithoutParent; // ...of those, how many arrived orphaned | 177 | extern int64 nBlocksWithoutParent; // ...of those, how many arrived orphaned |
| 181 | 178 | ||
diff --git a/src/nostr.cpp b/src/nostr.cpp index 3f43302..32cea18 100644 --- a/src/nostr.cpp +++ b/src/nostr.cpp | |||
| @@ -7,6 +7,10 @@ | |||
| 7 | #include <openssl/ssl.h> | 7 | #include <openssl/ssl.h> |
| 8 | #include <openssl/err.h> | 8 | #include <openssl/err.h> |
| 9 | #include <openssl/x509.h> | 9 | #include <openssl/x509.h> |
| 10 | #include <openssl/x509v3.h> | ||
| 11 | #ifdef _WIN32 | ||
| 12 | #include <wincrypt.h> | ||
| 13 | #endif | ||
| 10 | // We link libsecp256k1 statically (without this the header assumes DLL import). | 14 | // We link libsecp256k1 statically (without this the header assumes DLL import). |
| 11 | // Both makefiles also pass -DSECP256K1_STATIC; the guard keeps that from | 15 | // Both makefiles also pass -DSECP256K1_STATIC; the guard keeps that from |
| 12 | // warning while leaving this file correct if built without the flag. | 16 | // warning while leaving this file correct if built without the flag. |
| @@ -26,6 +30,7 @@ | |||
| 26 | 30 | ||
| 27 | #include "btfaddr.h" | 31 | #include "btfaddr.h" |
| 28 | #include "btfchan.h" | 32 | #include "btfchan.h" |
| 33 | #include "proxy.h" | ||
| 29 | 34 | ||
| 30 | using json = nlohmann::json; | 35 | using json = nlohmann::json; |
| 31 | 36 | ||
| @@ -468,6 +473,41 @@ static bool BuildSignedEvent(CNostrKey& key, int kind, const json& tags, | |||
| 468 | static const uint64 MAX_WS_FRAME = 1 << 20; // 1 MiB per frame | 473 | static const uint64 MAX_WS_FRAME = 1 << 20; // 1 MiB per frame |
| 469 | static const uint64 MAX_WS_MESSAGE = 4 << 20; // 4 MiB reassembled | 474 | static const uint64 MAX_WS_MESSAGE = 4 << 20; // 4 MiB reassembled |
| 470 | 475 | ||
| 476 | static int BtfConnectedNodeCount() | ||
| 477 | { | ||
| 478 | int nNodes = 0; | ||
| 479 | CRITICAL_BLOCK(cs_vNodes) | ||
| 480 | { | ||
| 481 | nNodes = (int)vNodes.size(); | ||
| 482 | } | ||
| 483 | return nNodes; | ||
| 484 | } | ||
| 485 | |||
| 486 | #ifdef _WIN32 | ||
| 487 | static bool BtfLoadWindowsRootCertificates(SSL_CTX* sslctx) | ||
| 488 | { | ||
| 489 | HCERTSTORE hStore = CertOpenSystemStoreA(0, "ROOT"); | ||
| 490 | if (!hStore) | ||
| 491 | return false; | ||
| 492 | |||
| 493 | bool fLoaded = false; | ||
| 494 | X509_STORE* x509store = SSL_CTX_get_cert_store(sslctx); | ||
| 495 | PCCERT_CONTEXT pCert = NULL; | ||
| 496 | while ((pCert = CertEnumCertificatesInStore(hStore, pCert)) != NULL) | ||
| 497 | { | ||
| 498 | const unsigned char* p = pCert->pbCertEncoded; | ||
| 499 | X509* x509 = d2i_X509(NULL, &p, pCert->cbCertEncoded); | ||
| 500 | if (!x509) | ||
| 501 | continue; | ||
| 502 | if (X509_STORE_add_cert(x509store, x509) == 1) | ||
| 503 | fLoaded = true; | ||
| 504 | X509_free(x509); | ||
| 505 | } | ||
| 506 | CertCloseStore(hStore, 0); | ||
| 507 | return fLoaded; | ||
| 508 | } | ||
| 509 | #endif | ||
| 510 | |||
| 471 | class CWebSocket | 511 | class CWebSocket |
| 472 | { | 512 | { |
| 473 | public: | 513 | public: |
| @@ -523,53 +563,8 @@ public: | |||
| 523 | size_t colon = host.find(':'); | 563 | size_t colon = host.find(':'); |
| 524 | if (colon != string::npos) { port = atoi(host.substr(colon + 1).c_str()); host = host.substr(0, colon); } | 564 | if (colon != string::npos) { port = atoi(host.substr(colon + 1).c_str()); host = host.substr(0, colon); } |
| 525 | 565 | ||
| 526 | // Resolve and connect (TCP). AF_UNSPEC so we get both A and AAAA | 566 | hSocket = BtfConnectSocket(host, (unsigned short)port, SOCK_NOSTR, timeoutSec); |
| 527 | // records back and try them in the order the resolver prefers, | 567 | if (hSocket == INVALID_SOCKET) |
| 528 | // rather than hard-failing whenever the IPv4 path to a relay is | ||
| 529 | // filtered/rate-limited (increasingly common for CDN-fronted | ||
| 530 | // relays) while a working IPv6 route exists right there in the | ||
| 531 | // same getaddrinfo() result. | ||
| 532 | struct addrinfo hints, *res = NULL, *rp = NULL; | ||
| 533 | memset(&hints, 0, sizeof(hints)); | ||
| 534 | hints.ai_family = AF_UNSPEC; | ||
| 535 | hints.ai_socktype = SOCK_STREAM; | ||
| 536 | char portstr[16]; sprintf(portstr, "%d", port); | ||
| 537 | if (getaddrinfo(host.c_str(), portstr, &hints, &res) != 0 || !res) | ||
| 538 | return error("Nostr: getaddrinfo %s failed", host.c_str()); | ||
| 539 | |||
| 540 | bool fConnected = false; | ||
| 541 | for (rp = res; rp != NULL; rp = rp->ai_next) | ||
| 542 | { | ||
| 543 | hSocket = BtfSocketTag(socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol), SOCK_NOSTR); | ||
| 544 | if (hSocket == INVALID_SOCKET) | ||
| 545 | continue; | ||
| 546 | |||
| 547 | // Socket receive/send timeout: Windows takes a DWORD of | ||
| 548 | // milliseconds; POSIX takes a struct timeval. | ||
| 549 | #ifdef _WIN32 | ||
| 550 | DWORD tv = timeoutSec * 1000; | ||
| 551 | setsockopt(hSocket, SOL_SOCKET, SO_RCVTIMEO, (char*)&tv, sizeof(tv)); | ||
| 552 | setsockopt(hSocket, SOL_SOCKET, SO_SNDTIMEO, (char*)&tv, sizeof(tv)); | ||
| 553 | #else | ||
| 554 | struct timeval tv; | ||
| 555 | tv.tv_sec = timeoutSec; | ||
| 556 | tv.tv_usec = 0; | ||
| 557 | setsockopt(hSocket, SOL_SOCKET, SO_RCVTIMEO, (char*)&tv, sizeof(tv)); | ||
| 558 | setsockopt(hSocket, SOL_SOCKET, SO_SNDTIMEO, (char*)&tv, sizeof(tv)); | ||
| 559 | #endif | ||
| 560 | |||
| 561 | if (connect(hSocket, rp->ai_addr, (int)rp->ai_addrlen) == 0) | ||
| 562 | { | ||
| 563 | fConnected = true; | ||
| 564 | break; | ||
| 565 | } | ||
| 566 | |||
| 567 | BtfCloseSocket(hSocket); | ||
| 568 | hSocket = INVALID_SOCKET; | ||
| 569 | } | ||
| 570 | freeaddrinfo(res); | ||
| 571 | |||
| 572 | if (!fConnected) | ||
| 573 | return error("Nostr: connect %s:%d failed", host.c_str(), port); | 568 | return error("Nostr: connect %s:%d failed", host.c_str(), port); |
| 574 | 569 | ||
| 575 | // TLS | 570 | // TLS |
| @@ -585,22 +580,35 @@ public: | |||
| 585 | // so a MITM cannot forge them, but it could drop the rendezvous | 580 | // so a MITM cannot forge them, but it could drop the rendezvous |
| 586 | // registration (leaving the pool unreachable to miners) or answer | 581 | // registration (leaving the pool unreachable to miners) or answer |
| 587 | // subscriptions with whatever it liked. | 582 | // subscriptions with whatever it liked. |
| 588 | if (!SSL_CTX_set_default_verify_paths(sslctx)) | ||
| 589 | return error("Nostr: no system CA store available, refusing %s", host.c_str()); | ||
| 590 | SSL_CTX_set_verify(sslctx, SSL_VERIFY_PEER, NULL); | 583 | SSL_CTX_set_verify(sslctx, SSL_VERIFY_PEER, NULL); |
| 584 | |||
| 585 | // OpenSSL's default verify paths are usually empty on Windows, so | ||
| 586 | // the system ROOT store is loaded too -- without it verification | ||
| 587 | // fails closed there and every relay becomes unreachable. | ||
| 588 | bool fTrustLoaded = (SSL_CTX_set_default_verify_paths(sslctx) == 1); | ||
| 589 | #ifdef _WIN32 | ||
| 590 | fTrustLoaded = BtfLoadWindowsRootCertificates(sslctx) || fTrustLoaded; | ||
| 591 | #endif | ||
| 592 | if (!fTrustLoaded) | ||
| 593 | return error("Nostr: failed to load TLS trust store"); | ||
| 594 | |||
| 595 | // Floor the protocol version. Verification says who the peer is, | ||
| 596 | // not how weak a cipher suite we will agree to; without this a | ||
| 597 | // relay offering nothing better still gets TLS 1.0. | ||
| 591 | SSL_CTX_set_min_proto_version(sslctx, TLS1_2_VERSION); | 598 | SSL_CTX_set_min_proto_version(sslctx, TLS1_2_VERSION); |
| 592 | 599 | ||
| 593 | ssl = SSL_new(sslctx); | 600 | ssl = SSL_new(sslctx); |
| 594 | if (!ssl) return false; | 601 | if (!ssl) return false; |
| 595 | SSL_set_fd(ssl, (int)hSocket); | 602 | SSL_set_fd(ssl, (int)hSocket); |
| 596 | SSL_set_tlsext_host_name(ssl, host.c_str()); // SNI | 603 | if (SSL_set_tlsext_host_name(ssl, host.c_str()) != 1) // SNI |
| 604 | return error("Nostr: failed to set TLS SNI for %s", host.c_str()); | ||
| 597 | 605 | ||
| 598 | // Verifying the chain is not enough on its own: without this the | 606 | // Verifying the chain is not enough on its own: without this the |
| 599 | // relay could present any certificate a trusted CA ever issued, | 607 | // relay could present any certificate a trusted CA ever issued, |
| 600 | // for any name. SSL_set1_host binds the check to the host we asked | 608 | // for any name. SSL_set1_host binds the check to the host we asked |
| 601 | // for. | 609 | // for. |
| 602 | if (!SSL_set1_host(ssl, host.c_str())) | 610 | if (SSL_set1_host(ssl, host.c_str()) != 1) |
| 603 | return error("Nostr: cannot pin verification to %s", host.c_str()); | 611 | return error("Nostr: failed to set TLS hostname for %s", host.c_str()); |
| 604 | 612 | ||
| 605 | if (SSL_connect(ssl) != 1) | 613 | if (SSL_connect(ssl) != 1) |
| 606 | { | 614 | { |
| @@ -1071,6 +1079,7 @@ static bool ResolveDescriptor(CWebSocket& ws, void* ctx, const string& btfAddr, | |||
| 1071 | json filter = json::object(); | 1079 | json filter = json::object(); |
| 1072 | filter["authors"] = json::array({ HexEncode(pk, 32) }); | 1080 | filter["authors"] = json::array({ HexEncode(pk, 32) }); |
| 1073 | filter["kinds"] = json::array({ BTF_DESC_KIND }); | 1081 | filter["kinds"] = json::array({ BTF_DESC_KIND }); |
| 1082 | filter["#d"] = json::array({ BTF_DESC_DTAG }); | ||
| 1074 | filter["limit"] = 1; | 1083 | filter["limit"] = 1; |
| 1075 | json req = json::array({ "REQ", "btf-resolve", filter }); | 1084 | json req = json::array({ "REQ", "btf-resolve", filter }); |
| 1076 | if (!ws.SendText(req.dump())) | 1085 | if (!ws.SendText(req.dump())) |
| @@ -1207,6 +1216,7 @@ bool BtfResolveMany(const std::vector<std::string>& btfAddrs, | |||
| 1207 | json filter = json::object(); | 1216 | json filter = json::object(); |
| 1208 | filter["authors"] = authors; | 1217 | filter["authors"] = authors; |
| 1209 | filter["kinds"] = json::array({ BTF_DESC_KIND }); | 1218 | filter["kinds"] = json::array({ BTF_DESC_KIND }); |
| 1219 | filter["#d"] = json::array({ BTF_DESC_DTAG }); | ||
| 1210 | filter["limit"] = (int)authors.size(); | 1220 | filter["limit"] = (int)authors.size(); |
| 1211 | json req = json::array({ "REQ", "btf-resolve-many", filter }); | 1221 | json req = json::array({ "REQ", "btf-resolve-many", filter }); |
| 1212 | if (!ws.SendText(req.dump())) | 1222 | if (!ws.SendText(req.dump())) |
| @@ -1617,8 +1627,9 @@ static void ConnectDiscoveredBtfPeers() | |||
| 1617 | // plus a random sample, not one or the other. | 1627 | // plus a random sample, not one or the other. |
| 1618 | int64 now = GetTime(); | 1628 | int64 now = GetTime(); |
| 1619 | int nBudget = (int)BTF_DIALS_PER_PASS; | 1629 | int nBudget = (int)BTF_DIALS_PER_PASS; |
| 1630 | int nNodeCount = BtfConnectedNodeCount(); | ||
| 1620 | { | 1631 | { |
| 1621 | int nRoom = (int)BTF_TARGET_CONN - (int)vNodes.size(); | 1632 | int nRoom = (int)BTF_TARGET_CONN - nNodeCount; |
| 1622 | if (nRoom < nBudget) nBudget = nRoom; | 1633 | if (nRoom < nBudget) nBudget = nRoom; |
| 1623 | } | 1634 | } |
| 1624 | if (nBudget < 0) nBudget = 0; | 1635 | if (nBudget < 0) nBudget = 0; |
| @@ -1632,7 +1643,6 @@ static void ConnectDiscoveredBtfPeers() | |||
| 1632 | // Reliability slice: walk the tiered (good -> untried -> bad) order. | 1643 | // Reliability slice: walk the tiered (good -> untried -> bad) order. |
| 1633 | for (const string& addr : peers) | 1644 | for (const string& addr : peers) |
| 1634 | { | 1645 | { |
| 1635 | if ((int)vNodes.size() >= (int)BTF_TARGET_CONN) break; | ||
| 1636 | if ((int)candidates.size() >= nSortedSlots) break; | 1646 | if ((int)candidates.size() >= nSortedSlots) break; |
| 1637 | map<string, int64>::iterator bi = g_btfPeerBackoff.find(addr); | 1647 | map<string, int64>::iterator bi = g_btfPeerBackoff.find(addr); |
| 1638 | if (bi != g_btfPeerBackoff.end() && now < bi->second) | 1648 | if (bi != g_btfPeerBackoff.end() && now < bi->second) |
| @@ -1657,7 +1667,7 @@ static void ConnectDiscoveredBtfPeers() | |||
| 1657 | int nWant = min(nRandomSlots, (int)pool.size()); | 1667 | int nWant = min(nRandomSlots, (int)pool.size()); |
| 1658 | for (int i = 0; i < nWant; i++) | 1668 | for (int i = 0; i < nWant; i++) |
| 1659 | { | 1669 | { |
| 1660 | if ((int)vNodes.size() + (int)candidates.size() >= (int)BTF_TARGET_CONN) break; | 1670 | if (nNodeCount + (int)candidates.size() >= (int)BTF_TARGET_CONN) break; |
| 1661 | if ((int)candidates.size() >= nBudget) break; | 1671 | if ((int)candidates.size() >= nBudget) break; |
| 1662 | size_t j = i + (size_t)GetRand((uint64)(pool.size() - i)); | 1672 | size_t j = i + (size_t)GetRand((uint64)(pool.size() - i)); |
| 1663 | std::swap(pool[i], pool[j]); | 1673 | std::swap(pool[i], pool[j]); |
| @@ -1712,12 +1722,10 @@ static void ConnectDiscoveredBtfPeers() | |||
| 1712 | // not the shared Nostr relays, so there's no thundering-herd effect here. | 1722 | // not the shared Nostr relays, so there's no thundering-herd effect here. |
| 1713 | // Each thread writes its result back via shared state guarded by cs_btfPeerState. | 1723 | // Each thread writes its result back via shared state guarded by cs_btfPeerState. |
| 1714 | struct DialCtx { string addr; BtfResolvedPeer rp; }; | 1724 | struct DialCtx { string addr; BtfResolvedPeer rp; }; |
| 1715 | vector<DialCtx*> ctxs; | ||
| 1716 | for (const string& addr : toDial) | 1725 | for (const string& addr : toDial) |
| 1717 | { | 1726 | { |
| 1718 | DialCtx* ctx = new DialCtx{addr, resolved[addr]}; | 1727 | DialCtx* ctx = new DialCtx{addr, resolved[addr]}; |
| 1719 | ctxs.push_back(ctx); | 1728 | uintptr_t tid = _beginthread([](void* arg) { |
| 1720 | _beginthread([](void* arg) { | ||
| 1721 | DialCtx* ctx = (DialCtx*)arg; | 1729 | DialCtx* ctx = (DialCtx*)arg; |
| 1722 | string addr = ctx->addr; | 1730 | string addr = ctx->addr; |
| 1723 | string meeting = ctx->rp.meetingHostPort; | 1731 | string meeting = ctx->rp.meetingHostPort; |
| @@ -1750,6 +1758,21 @@ static void ConnectDiscoveredBtfPeers() | |||
| 1750 | } | 1758 | } |
| 1751 | } | 1759 | } |
| 1752 | }, 0, ctx); | 1760 | }, 0, ctx); |
| 1761 | if (tid == (uintptr_t)-1) | ||
| 1762 | { | ||
| 1763 | LogPrint("nostr", "Nostr: failed to create .btf dial thread for %s\n", addr.c_str()); | ||
| 1764 | CRITICAL_BLOCK(cs_btfPeerState) | ||
| 1765 | { | ||
| 1766 | int n = ++g_btfPeerFails[addr]; | ||
| 1767 | int64 cap = (n >= BTF_CHRONIC_FAIL_THRESHOLD) ? BTF_BACKOFF_CAP_SECS_DEAD | ||
| 1768 | : BTF_BACKOFF_CAP_SECS; | ||
| 1769 | int64 secs = 30; | ||
| 1770 | for (int k = 1; k < n && secs < cap; k++) secs *= 2; | ||
| 1771 | if (secs > cap) secs = cap; | ||
| 1772 | g_btfPeerBackoff[addr] = GetTime() + secs; | ||
| 1773 | } | ||
| 1774 | delete ctx; | ||
| 1775 | } | ||
| 1753 | } | 1776 | } |
| 1754 | 1777 | ||
| 1755 | // Wait long enough for all parallel dials to finish (each has a 10s socket | 1778 | // Wait long enough for all parallel dials to finish (each has a 10s socket |
| @@ -1852,7 +1875,7 @@ void ThreadNostrSeed(void* parg) | |||
| 1852 | 1875 | ||
| 1853 | // Re-discover periodically. Sleep less aggressively when | 1876 | // Re-discover periodically. Sleep less aggressively when |
| 1854 | // we have no connections yet so we find peers faster on startup. | 1877 | // we have no connections yet so we find peers faster on startup. |
| 1855 | int nSleepSecs = vNodes.empty() ? 20 : 60; | 1878 | int nSleepSecs = (BtfConnectedNodeCount() == 0) ? 20 : 60; |
| 1856 | for (int i = 0; i < nSleepSecs && !fShutdown; i++) | 1879 | for (int i = 0; i < nSleepSecs && !fShutdown; i++) |
| 1857 | Sleep(1000); | 1880 | Sleep(1000); |
| 1858 | } | 1881 | } |
diff --git a/src/proxy.cpp b/src/proxy.cpp new file mode 100644 index 0000000..7d30911 --- /dev/null +++ b/src/proxy.cpp | |||
| @@ -0,0 +1,377 @@ | |||
| 1 | // Copyright (c) 2026 Bitflash developers | ||
| 2 | // Distributed under the MIT/X11 software license. | ||
| 3 | |||
| 4 | #include "headers_core.h" | ||
| 5 | #include "proxy.h" | ||
| 6 | #include "sockcount.h" | ||
| 7 | |||
| 8 | #include <errno.h> | ||
| 9 | |||
| 10 | static CCriticalSection cs_socks5; | ||
| 11 | static bool g_fSocks5Proxy = false; | ||
| 12 | static bool g_fTorProxy = false; | ||
| 13 | static std::string g_socks5Host; | ||
| 14 | static unsigned short g_socks5Port = 0; | ||
| 15 | static const char* DEFAULT_TOR_SOCKS5_PROXY = "127.0.0.1:9050"; | ||
| 16 | |||
| 17 | static void SetSocketTimeout(SOCKET s, int nTimeoutSecs) | ||
| 18 | { | ||
| 19 | if (nTimeoutSecs <= 0) | ||
| 20 | return; | ||
| 21 | #ifdef _WIN32 | ||
| 22 | DWORD tv = (DWORD)nTimeoutSecs * 1000; | ||
| 23 | setsockopt(s, SOL_SOCKET, SO_RCVTIMEO, (const char*)&tv, sizeof(tv)); | ||
| 24 | setsockopt(s, SOL_SOCKET, SO_SNDTIMEO, (const char*)&tv, sizeof(tv)); | ||
| 25 | #else | ||
| 26 | struct timeval tv; | ||
| 27 | tv.tv_sec = nTimeoutSecs; | ||
| 28 | tv.tv_usec = 0; | ||
| 29 | setsockopt(s, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv)); | ||
| 30 | setsockopt(s, SOL_SOCKET, SO_SNDTIMEO, &tv, sizeof(tv)); | ||
| 31 | #endif | ||
| 32 | } | ||
| 33 | |||
| 34 | static bool ReadN(SOCKET s, void* buf, int n) | ||
| 35 | { | ||
| 36 | char* p = (char*)buf; | ||
| 37 | int off = 0; | ||
| 38 | while (off < n) | ||
| 39 | { | ||
| 40 | int r = recv(s, p + off, n - off, 0); | ||
| 41 | if (r <= 0) | ||
| 42 | return false; | ||
| 43 | off += r; | ||
| 44 | } | ||
| 45 | return true; | ||
| 46 | } | ||
| 47 | |||
| 48 | static bool WriteN(SOCKET s, const void* buf, int n) | ||
| 49 | { | ||
| 50 | const char* p = (const char*)buf; | ||
| 51 | int off = 0; | ||
| 52 | while (off < n) | ||
| 53 | { | ||
| 54 | int r = send(s, p + off, n - off, 0); | ||
| 55 | if (r <= 0) | ||
| 56 | return false; | ||
| 57 | off += r; | ||
| 58 | } | ||
| 59 | return true; | ||
| 60 | } | ||
| 61 | |||
| 62 | static bool ValidProxyHostChar(char c) | ||
| 63 | { | ||
| 64 | return isalnum((unsigned char)c) || c == '.' || c == '-' || c == '_'; | ||
| 65 | } | ||
| 66 | |||
| 67 | static bool ValidProxyIpv6HostChar(char c) | ||
| 68 | { | ||
| 69 | return isxdigit((unsigned char)c) || c == ':' || c == '.'; | ||
| 70 | } | ||
| 71 | |||
| 72 | static std::string LowerProxyHost(std::string host) | ||
| 73 | { | ||
| 74 | if (!host.empty() && host[host.size() - 1] == '.') | ||
| 75 | host.erase(host.size() - 1); | ||
| 76 | for (size_t i = 0; i < host.size(); i++) | ||
| 77 | host[i] = (char)tolower((unsigned char)host[i]); | ||
| 78 | return host; | ||
| 79 | } | ||
| 80 | |||
| 81 | bool BtfIsTorOnionHost(const std::string& host) | ||
| 82 | { | ||
| 83 | std::string lower = LowerProxyHost(host); | ||
| 84 | static const char* suffix = ".onion"; | ||
| 85 | size_t suffixLen = strlen(suffix); | ||
| 86 | return lower.size() > suffixLen && | ||
| 87 | lower.compare(lower.size() - suffixLen, suffixLen, suffix) == 0; | ||
| 88 | } | ||
| 89 | |||
| 90 | static bool ParsePort(const std::string& port, unsigned short& portOut, | ||
| 91 | std::string& errOut) | ||
| 92 | { | ||
| 93 | if (port.empty()) | ||
| 94 | { | ||
| 95 | errOut = "port is empty"; | ||
| 96 | return false; | ||
| 97 | } | ||
| 98 | for (size_t i = 0; i < port.size(); i++) | ||
| 99 | { | ||
| 100 | if (!isdigit((unsigned char)port[i])) | ||
| 101 | { | ||
| 102 | errOut = "port is not numeric"; | ||
| 103 | return false; | ||
| 104 | } | ||
| 105 | } | ||
| 106 | |||
| 107 | errno = 0; | ||
| 108 | char* end = NULL; | ||
| 109 | long nPort = strtol(port.c_str(), &end, 10); | ||
| 110 | if (errno == ERANGE || end == NULL || *end != '\0' || nPort <= 0 || nPort > 65535) | ||
| 111 | { | ||
| 112 | errOut = "port is out of range"; | ||
| 113 | return false; | ||
| 114 | } | ||
| 115 | portOut = (unsigned short)nPort; | ||
| 116 | return true; | ||
| 117 | } | ||
| 118 | |||
| 119 | bool BtfParseSocks5Proxy(const std::string& spec, std::string& hostOut, | ||
| 120 | unsigned short& portOut, std::string& errOut) | ||
| 121 | { | ||
| 122 | hostOut.clear(); | ||
| 123 | portOut = 0; | ||
| 124 | errOut.clear(); | ||
| 125 | |||
| 126 | std::string host; | ||
| 127 | std::string port; | ||
| 128 | bool fBracketedIpv6 = false; | ||
| 129 | if (!spec.empty() && spec[0] == '[') | ||
| 130 | { | ||
| 131 | size_t close = spec.find(']'); | ||
| 132 | if (close == std::string::npos || close == 1 || close + 1 >= spec.size() || spec[close + 1] != ':') | ||
| 133 | { | ||
| 134 | errOut = "expected [IPv6]:PORT"; | ||
| 135 | return false; | ||
| 136 | } | ||
| 137 | host = spec.substr(1, close - 1); | ||
| 138 | port = spec.substr(close + 2); | ||
| 139 | fBracketedIpv6 = true; | ||
| 140 | } | ||
| 141 | else | ||
| 142 | { | ||
| 143 | size_t colon = spec.rfind(':'); | ||
| 144 | if (colon == std::string::npos || colon == 0 || colon + 1 >= spec.size()) | ||
| 145 | { | ||
| 146 | errOut = "expected HOST:PORT"; | ||
| 147 | return false; | ||
| 148 | } | ||
| 149 | host = spec.substr(0, colon); | ||
| 150 | port = spec.substr(colon + 1); | ||
| 151 | if (host.find(':') != std::string::npos) | ||
| 152 | { | ||
| 153 | errOut = "IPv6 proxy hosts must use [ADDR]:PORT"; | ||
| 154 | return false; | ||
| 155 | } | ||
| 156 | } | ||
| 157 | |||
| 158 | if (host.size() > 255) | ||
| 159 | { | ||
| 160 | errOut = "host is too long"; | ||
| 161 | return false; | ||
| 162 | } | ||
| 163 | if (fBracketedIpv6) | ||
| 164 | { | ||
| 165 | bool fHasColon = false; | ||
| 166 | for (size_t i = 0; i < host.size(); i++) | ||
| 167 | { | ||
| 168 | fHasColon = fHasColon || host[i] == ':'; | ||
| 169 | if (!ValidProxyIpv6HostChar(host[i])) | ||
| 170 | { | ||
| 171 | errOut = "IPv6 proxy host contains unsupported characters"; | ||
| 172 | return false; | ||
| 173 | } | ||
| 174 | } | ||
| 175 | if (!fHasColon) | ||
| 176 | { | ||
| 177 | errOut = "bracketed proxy host is not IPv6"; | ||
| 178 | return false; | ||
| 179 | } | ||
| 180 | } | ||
| 181 | else | ||
| 182 | { | ||
| 183 | for (size_t i = 0; i < host.size(); i++) | ||
| 184 | { | ||
| 185 | if (!ValidProxyHostChar(host[i])) | ||
| 186 | { | ||
| 187 | errOut = "host contains unsupported characters"; | ||
| 188 | return false; | ||
| 189 | } | ||
| 190 | } | ||
| 191 | } | ||
| 192 | if (!ParsePort(port, portOut, errOut)) | ||
| 193 | return false; | ||
| 194 | |||
| 195 | hostOut = host; | ||
| 196 | return true; | ||
| 197 | } | ||
| 198 | |||
| 199 | static bool SetSocks5Proxy(const std::string& spec, bool fTorMode, | ||
| 200 | std::string& errOut) | ||
| 201 | { | ||
| 202 | std::string host; | ||
| 203 | unsigned short port = 0; | ||
| 204 | if (!BtfParseSocks5Proxy(spec, host, port, errOut)) | ||
| 205 | return false; | ||
| 206 | CRITICAL_BLOCK(cs_socks5) | ||
| 207 | { | ||
| 208 | g_socks5Host = host; | ||
| 209 | g_socks5Port = port; | ||
| 210 | g_fSocks5Proxy = true; | ||
| 211 | g_fTorProxy = fTorMode; | ||
| 212 | } | ||
| 213 | return true; | ||
| 214 | } | ||
| 215 | |||
| 216 | bool BtfSetSocks5Proxy(const std::string& spec, std::string& errOut) | ||
| 217 | { | ||
| 218 | return SetSocks5Proxy(spec, false, errOut); | ||
| 219 | } | ||
| 220 | |||
| 221 | bool BtfEnableTorProxy(const std::string& spec, std::string& errOut) | ||
| 222 | { | ||
| 223 | return SetSocks5Proxy(spec.empty() ? DEFAULT_TOR_SOCKS5_PROXY : spec, true, errOut); | ||
| 224 | } | ||
| 225 | |||
| 226 | void BtfClearSocks5Proxy() | ||
| 227 | { | ||
| 228 | CRITICAL_BLOCK(cs_socks5) | ||
| 229 | { | ||
| 230 | g_fSocks5Proxy = false; | ||
| 231 | g_fTorProxy = false; | ||
| 232 | g_socks5Host.clear(); | ||
| 233 | g_socks5Port = 0; | ||
| 234 | } | ||
| 235 | } | ||
| 236 | |||
| 237 | bool BtfSocks5ProxyEnabled() | ||
| 238 | { | ||
| 239 | CRITICAL_BLOCK(cs_socks5) | ||
| 240 | return g_fSocks5Proxy; | ||
| 241 | return false; | ||
| 242 | } | ||
| 243 | |||
| 244 | bool BtfTorProxyEnabled() | ||
| 245 | { | ||
| 246 | CRITICAL_BLOCK(cs_socks5) | ||
| 247 | return g_fSocks5Proxy && g_fTorProxy; | ||
| 248 | return false; | ||
| 249 | } | ||
| 250 | |||
| 251 | std::string BtfSocks5ProxyName() | ||
| 252 | { | ||
| 253 | CRITICAL_BLOCK(cs_socks5) | ||
| 254 | { | ||
| 255 | if (!g_fSocks5Proxy) | ||
| 256 | return ""; | ||
| 257 | if (g_socks5Host.find(':') != std::string::npos) | ||
| 258 | return strprintf("[%s]:%u", g_socks5Host.c_str(), (unsigned)g_socks5Port); | ||
| 259 | return strprintf("%s:%u", g_socks5Host.c_str(), (unsigned)g_socks5Port); | ||
| 260 | } | ||
| 261 | return ""; | ||
| 262 | } | ||
| 263 | |||
| 264 | static SOCKET ConnectDirect(const std::string& host, unsigned short port, | ||
| 265 | int nSockSite, int nTimeoutSecs) | ||
| 266 | { | ||
| 267 | struct addrinfo hints, *res = NULL, *rp = NULL; | ||
| 268 | memset(&hints, 0, sizeof(hints)); | ||
| 269 | hints.ai_family = AF_UNSPEC; | ||
| 270 | hints.ai_socktype = SOCK_STREAM; | ||
| 271 | char portstr[16]; | ||
| 272 | sprintf(portstr, "%u", (unsigned)port); | ||
| 273 | if (getaddrinfo(host.c_str(), portstr, &hints, &res) != 0 || !res) | ||
| 274 | return INVALID_SOCKET; | ||
| 275 | |||
| 276 | SOCKET s = INVALID_SOCKET; | ||
| 277 | for (rp = res; rp != NULL; rp = rp->ai_next) | ||
| 278 | { | ||
| 279 | s = BtfSocketTag(socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol), nSockSite); | ||
| 280 | if (s == INVALID_SOCKET) | ||
| 281 | continue; | ||
| 282 | SetSocketTimeout(s, nTimeoutSecs); | ||
| 283 | if (connect(s, rp->ai_addr, (int)rp->ai_addrlen) == 0) | ||
| 284 | break; | ||
| 285 | BtfCloseSocket(s); | ||
| 286 | s = INVALID_SOCKET; | ||
| 287 | } | ||
| 288 | freeaddrinfo(res); | ||
| 289 | return s; | ||
| 290 | } | ||
| 291 | |||
| 292 | static bool Socks5Connect(SOCKET s, const std::string& destHost, | ||
| 293 | unsigned short destPort) | ||
| 294 | { | ||
| 295 | if (destHost.empty() || destHost.size() > 255) | ||
| 296 | return false; | ||
| 297 | |||
| 298 | unsigned char hello[3] = { 0x05, 0x01, 0x00 }; // SOCKS5, one method, no auth | ||
| 299 | if (!WriteN(s, hello, sizeof(hello))) | ||
| 300 | return false; | ||
| 301 | unsigned char choice[2] = { 0, 0 }; | ||
| 302 | if (!ReadN(s, choice, sizeof(choice)) || choice[0] != 0x05 || choice[1] != 0x00) | ||
| 303 | return false; | ||
| 304 | |||
| 305 | std::vector<unsigned char> req; | ||
| 306 | req.reserve(7 + destHost.size()); | ||
| 307 | req.push_back(0x05); // version | ||
| 308 | req.push_back(0x01); // CONNECT | ||
| 309 | req.push_back(0x00); // reserved | ||
| 310 | req.push_back(0x03); // domain name; proxy resolves, avoiding local DNS leak | ||
| 311 | req.push_back((unsigned char)destHost.size()); | ||
| 312 | req.insert(req.end(), destHost.begin(), destHost.end()); | ||
| 313 | req.push_back((unsigned char)(destPort >> 8)); | ||
| 314 | req.push_back((unsigned char)(destPort & 0xff)); | ||
| 315 | if (!WriteN(s, &req[0], (int)req.size())) | ||
| 316 | return false; | ||
| 317 | |||
| 318 | unsigned char rep[4] = {0,0,0,0}; | ||
| 319 | if (!ReadN(s, rep, sizeof(rep)) || rep[0] != 0x05) | ||
| 320 | return false; | ||
| 321 | if (rep[1] != 0x00) | ||
| 322 | { | ||
| 323 | error("SOCKS5 CONNECT to %s:%u failed with reply 0x%02x\n", | ||
| 324 | destHost.c_str(), (unsigned)destPort, (unsigned)rep[1]); | ||
| 325 | return false; | ||
| 326 | } | ||
| 327 | int nAddr = 0; | ||
| 328 | if (rep[3] == 0x01) nAddr = 4; | ||
| 329 | else if (rep[3] == 0x04) nAddr = 16; | ||
| 330 | else if (rep[3] == 0x03) | ||
| 331 | { | ||
| 332 | unsigned char len = 0; | ||
| 333 | if (!ReadN(s, &len, 1)) | ||
| 334 | return false; | ||
| 335 | nAddr = len; | ||
| 336 | } | ||
| 337 | else | ||
| 338 | return false; | ||
| 339 | |||
| 340 | std::vector<unsigned char> discard(nAddr + 2); | ||
| 341 | return ReadN(s, &discard[0], (int)discard.size()); | ||
| 342 | } | ||
| 343 | |||
| 344 | SOCKET BtfConnectSocket(const std::string& destHost, unsigned short destPort, | ||
| 345 | int nSockSite, int nTimeoutSecs) | ||
| 346 | { | ||
| 347 | std::string proxyHost; | ||
| 348 | unsigned short proxyPort = 0; | ||
| 349 | bool fProxy = false; | ||
| 350 | CRITICAL_BLOCK(cs_socks5) | ||
| 351 | { | ||
| 352 | fProxy = g_fSocks5Proxy; | ||
| 353 | proxyHost = g_socks5Host; | ||
| 354 | proxyPort = g_socks5Port; | ||
| 355 | } | ||
| 356 | |||
| 357 | if (!fProxy) | ||
| 358 | { | ||
| 359 | if (BtfIsTorOnionHost(destHost)) | ||
| 360 | { | ||
| 361 | error("Refusing direct .onion connection to %s:%u; use /tor or /socks\n", | ||
| 362 | destHost.c_str(), (unsigned)destPort); | ||
| 363 | return INVALID_SOCKET; | ||
| 364 | } | ||
| 365 | return ConnectDirect(destHost, destPort, nSockSite, nTimeoutSecs); | ||
| 366 | } | ||
| 367 | |||
| 368 | SOCKET s = ConnectDirect(proxyHost, proxyPort, nSockSite, nTimeoutSecs); | ||
| 369 | if (s == INVALID_SOCKET) | ||
| 370 | return INVALID_SOCKET; | ||
| 371 | if (!Socks5Connect(s, destHost, destPort)) | ||
| 372 | { | ||
| 373 | BtfCloseSocket(s); | ||
| 374 | return INVALID_SOCKET; | ||
| 375 | } | ||
| 376 | return s; | ||
| 377 | } | ||
diff --git a/src/proxy.h b/src/proxy.h new file mode 100644 index 0000000..6429cf2 --- /dev/null +++ b/src/proxy.h | |||
| @@ -0,0 +1,34 @@ | |||
| 1 | // Copyright (c) 2026 Bitflash developers | ||
| 2 | // Distributed under the MIT/X11 software license. | ||
| 3 | // | ||
| 4 | // Optional SOCKS5 outbound proxy support. This is transport plumbing only: | ||
| 5 | // no consensus, wallet, mining, or .btf identity data depends on it. | ||
| 6 | |||
| 7 | #ifndef BITFLASH_PROXY_H | ||
| 8 | #define BITFLASH_PROXY_H | ||
| 9 | |||
| 10 | #ifdef _WIN32 | ||
| 11 | #ifndef WIN32_LEAN_AND_MEAN | ||
| 12 | #define WIN32_LEAN_AND_MEAN 1 | ||
| 13 | #endif | ||
| 14 | #include <winsock2.h> | ||
| 15 | #else | ||
| 16 | #include "compat.h" | ||
| 17 | #endif | ||
| 18 | |||
| 19 | #include <string> | ||
| 20 | |||
| 21 | bool BtfParseSocks5Proxy(const std::string& spec, std::string& hostOut, | ||
| 22 | unsigned short& portOut, std::string& errOut); | ||
| 23 | bool BtfSetSocks5Proxy(const std::string& spec, std::string& errOut); | ||
| 24 | bool BtfEnableTorProxy(const std::string& spec, std::string& errOut); | ||
| 25 | void BtfClearSocks5Proxy(); | ||
| 26 | bool BtfSocks5ProxyEnabled(); | ||
| 27 | bool BtfTorProxyEnabled(); | ||
| 28 | bool BtfIsTorOnionHost(const std::string& host); | ||
| 29 | std::string BtfSocks5ProxyName(); | ||
| 30 | |||
| 31 | SOCKET BtfConnectSocket(const std::string& destHost, unsigned short destPort, | ||
| 32 | int nSockSite, int nTimeoutSecs); | ||
| 33 | |||
| 34 | #endif | ||
diff --git a/src/randomx_pow.cpp b/src/randomx_pow.cpp index d617ba2..1d058d4 100644 --- a/src/randomx_pow.cpp +++ b/src/randomx_pow.cpp | |||
| @@ -19,6 +19,7 @@ static randomx_cache* g_cache = NULL; | |||
| 19 | static randomx_dataset* g_dataset = NULL; | 19 | static randomx_dataset* g_dataset = NULL; |
| 20 | static randomx_vm* g_vmVerify = NULL; | 20 | static randomx_vm* g_vmVerify = NULL; |
| 21 | static CCriticalSection g_csVerify; | 21 | static CCriticalSection g_csVerify; |
| 22 | static std::mutex g_csInit; | ||
| 22 | static bool g_fInit = false; | 23 | static bool g_fInit = false; |
| 23 | static bool g_fFast = false; | 24 | static bool g_fFast = false; |
| 24 | 25 | ||
| @@ -28,6 +29,10 @@ bool RandomXInit() | |||
| 28 | if (g_fInit) | 29 | if (g_fInit) |
| 29 | return true; | 30 | return true; |
| 30 | 31 | ||
| 32 | std::lock_guard<std::mutex> lock(g_csInit); | ||
| 33 | if (g_fInit) | ||
| 34 | return true; | ||
| 35 | |||
| 31 | // Detect the best flags for this CPU (JIT, hardware AES, Argon2). | 36 | // Detect the best flags for this CPU (JIT, hardware AES, Argon2). |
| 32 | g_flags = randomx_get_flags(); | 37 | g_flags = randomx_get_flags(); |
| 33 | 38 | ||
diff --git a/src/rc/bitcoin.ico b/src/rc/bitcoin.ico index b768077..33323de 100644 --- a/src/rc/bitcoin.ico +++ b/src/rc/bitcoin.ico | |||
| Binary files differ | |||
diff --git a/src/readme.txt b/src/readme.txt index 6af1694..794fe20 100644 --- a/src/readme.txt +++ b/src/readme.txt | |||
| @@ -42,14 +42,30 @@ The resulting executable is src/bitflash.exe. | |||
| 42 | Command-line options (in addition to the original ones) | 42 | Command-line options (in addition to the original ones) |
| 43 | ------------------------------------------------------- | 43 | ------------------------------------------------------- |
| 44 | /port=N listen on P2P port N (default 8433) | 44 | /port=N listen on P2P port N (default 8433) |
| 45 | /socks=HOST:PORT | ||
| 46 | route outbound Nostr and .btf rendezvous dials through a | ||
| 47 | SOCKS5 proxy, e.g. /socks=127.0.0.1:9050 for local Tor | ||
| 48 | or /socks=[::1]:9050 for an IPv6 loopback proxy | ||
| 49 | /tor[=HOST:PORT] | ||
| 50 | Tor mode for outbound Nostr and .btf rendezvous dials; | ||
| 51 | defaults to the local Tor SOCKS5 proxy at 127.0.0.1:9050 | ||
| 45 | /solomine mine without requiring a connected peer (local testing) | 52 | /solomine mine without requiring a connected peer (local testing) |
| 46 | /operator operator mode (run pool server) | 53 | /operator operator mode (run pool server) |
| 47 | /participant=ADDR | 54 | /participant=ADDR |
| 48 | participant mode; mine to pool .btf address ADDR | 55 | participant mode; mine to pool .btf address ADDR |
| 56 | /stratumbridge=ADDR | ||
| 57 | listen on 127.0.0.1:3333 and bridge external Stratum miners | ||
| 58 | to pool .btf address ADDR | ||
| 59 | /stratumbridgeport=N | ||
| 60 | local Stratum bridge port (default 3333) | ||
| 49 | /poolname=NAME operator announcement pool name | 61 | /poolname=NAME operator announcement pool name |
| 50 | /poolfee=PCT operator announcement fee percent (example: 0.75) | 62 | /poolfee=PCT operator announcement fee percent (example: 0.75) |
| 51 | /pooldashboard=URL | 63 | /pooldashboard=URL |
| 52 | operator announcement dashboard URL | 64 | operator announcement dashboard URL |
| 65 | /poolstatusfile=PATH | ||
| 66 | write pool status JSON for dashboards | ||
| 67 | /poolroundsfile=PATH | ||
| 68 | write public pool round proofs JSON | ||
| 53 | /connectbtf=ADDR | 69 | /connectbtf=ADDR |
| 54 | keep an outbound .btf connection to ADDR | 70 | keep an outbound .btf connection to ADDR |
| 55 | /rvrelay=HOST:PORT | 71 | /rvrelay=HOST:PORT |
diff --git a/src/rpc.cpp b/src/rpc.cpp index 03d1f44..8af2fbe 100644 --- a/src/rpc.cpp +++ b/src/rpc.cpp | |||
| @@ -236,35 +236,73 @@ static bool RebuildJob() | |||
| 236 | CBlock block; | 236 | CBlock block; |
| 237 | block.vtx.push_back(txNew); | 237 | block.vtx.push_back(txNew); |
| 238 | 238 | ||
| 239 | // Fill the block the same way BitcoinMiner() does, because this is the | ||
| 240 | // same job: a block that gets mined and submitted for real. | ||
| 241 | // | ||
| 242 | // It used to differ in two ways, and both were bugs. | ||
| 243 | // | ||
| 244 | // One pass over mapTransactions is not enough. That map is keyed by | ||
| 245 | // hash, so it has no dependency order, and a transaction spending the | ||
| 246 | // still-unconfirmed output of another (change from a recent send, most | ||
| 247 | // commonly) fails ConnectInputs whenever its parent happens to sit | ||
| 248 | // later in map order. A single pass then drops it -- and because map | ||
| 249 | // order does not change between rebuilds, every later rebuild drops it | ||
| 250 | // again at the same point. Deterministic exclusion, not bad luck: if | ||
| 251 | // pool operators mine most of the blocks, a transaction of that shape | ||
| 252 | // can stay unconfirmed for as long as its parent does. | ||
| 253 | // | ||
| 254 | // Diagnosis and fix both come from Vic-Nas, who found it in his fork | ||
| 255 | // of this code and described it exactly. The fork has since been | ||
| 256 | // deleted, so this carries it back rather than losing it. | ||
| 257 | int64 nFees = 0; | ||
| 239 | { | 258 | { |
| 240 | CTxDB txdb("r"); | 259 | CTxDB txdb("r"); |
| 241 | CRITICAL_BLOCK(cs_mapTransactions) | 260 | CRITICAL_BLOCK(cs_mapTransactions) |
| 242 | { | 261 | { |
| 243 | map<uint256, CTxIndex> pool; | 262 | map<uint256, CTxIndex> pool; |
| 263 | vector<char> vfAlreadyAdded(mapTransactions.size()); | ||
| 264 | bool fFoundSomething = true; | ||
| 244 | unsigned int sz = 0; | 265 | unsigned int sz = 0; |
| 245 | for (auto& kv : mapTransactions) { | 266 | // The cap is MAX_BLOCK_SIZE/2, not MAX_SIZE/2. MAX_SIZE is the |
| 246 | CTransaction& tx = kv.second; | 267 | // 32 MB serialisation ceiling, so half of it is 16 MB -- this |
| 247 | if (tx.IsCoinBase() || !tx.IsFinal()) continue; | 268 | // builder would happily assemble a template that its own |
| 248 | int64 nFees = 0; | 269 | // CheckBlock() rejects at the 1 MB consensus cap. This is the |
| 249 | map<uint256, CTxIndex> tmp(pool); | 270 | // pool's job builder, so a block over the cap is not one |
| 250 | if (!tx.ConnectInputs(txdb, tmp, CDiskTxPos(1,1,1), 0, nFees, | 271 | // wasted solve, it is every miner on the pool working a |
| 251 | false, true, tx.GetMinFee(block.vtx.size() < 100))) | 272 | // template the network will reject. The inner break bounds the |
| 252 | continue; | 273 | // overshoot to a single transaction within a pass. |
| 253 | pool = tmp; | 274 | while (fFoundSomething && sz < MAX_BLOCK_SIZE / 2) |
| 254 | block.vtx.push_back(tx); | 275 | { |
| 255 | sz += ::GetSerializeSize(tx, SER_NETWORK); | 276 | fFoundSomething = false; |
| 256 | // Must stay under the consensus cap CheckBlock() enforces. | 277 | unsigned int n = 0; |
| 257 | // This is the pool's job builder, so a block over the cap | 278 | for (map<uint256, CTransaction>::iterator mi = mapTransactions.begin(); |
| 258 | // is not just one wasted solve -- it is every miner on the | 279 | mi != mapTransactions.end(); ++mi, ++n) |
| 259 | // pool working a template the network will reject. The | 280 | { |
| 260 | // transaction is already pushed above, so this can overshoot | 281 | if (vfAlreadyAdded[n]) |
| 261 | // by one; half the cap leaves room for that. | 282 | continue; |
| 262 | if (sz > MAX_BLOCK_SIZE / 2) break; | 283 | CTransaction& tx = (*mi).second; |
| 284 | if (tx.IsCoinBase() || !tx.IsFinal()) continue; | ||
| 285 | map<uint256, CTxIndex> tmp(pool); | ||
| 286 | if (!tx.ConnectInputs(txdb, tmp, CDiskTxPos(1,1,1), 0, nFees, | ||
| 287 | false, true, tx.GetMinFee(block.vtx.size() < 100))) | ||
| 288 | continue; | ||
| 289 | swap(pool, tmp); | ||
| 290 | block.vtx.push_back(tx); | ||
| 291 | sz += ::GetSerializeSize(tx, SER_NETWORK); | ||
| 292 | vfAlreadyAdded[n] = true; | ||
| 293 | fFoundSomething = true; | ||
| 294 | if (sz > MAX_BLOCK_SIZE / 2) break; | ||
| 295 | } | ||
| 263 | } | 296 | } |
| 264 | } | 297 | } |
| 265 | } | 298 | } |
| 266 | 299 | ||
| 267 | block.vtx[0].vout[0].nValue = block.GetBlockValue(pindexPrev->nHeight + 1, 0); | 300 | // And the fees actually go to the coinbase now. nFees was declared |
| 301 | // inside the scan loop, so every transaction reset it and the total was | ||
| 302 | // thrown away, while this line passed a hardcoded 0. A pool operator | ||
| 303 | // collected fees into a block and paid none of them to itself. Solo | ||
| 304 | // mining has always passed nFees here; only this builder did not. | ||
| 305 | block.vtx[0].vout[0].nValue = block.GetBlockValue(pindexPrev->nHeight + 1, nFees); | ||
| 268 | block.hashPrevBlock = pindexPrev->GetBlockHash(); | 306 | block.hashPrevBlock = pindexPrev->GetBlockHash(); |
| 269 | block.hashMerkleRoot = block.BuildMerkleTree(); | 307 | block.hashMerkleRoot = block.BuildMerkleTree(); |
| 270 | block.nTime = max((unsigned int)(pindexPrev->GetMedianTimePast() + 1), | 308 | block.nTime = max((unsigned int)(pindexPrev->GetMedianTimePast() + 1), |
| @@ -297,26 +335,110 @@ struct PendingPayout { | |||
| 297 | // supposed to pay it: a 280-block reorg once left ~14.4k BTF of debt | 335 | // supposed to pay it: a 280-block reorg once left ~14.4k BTF of debt |
| 298 | // owed against blocks that no longer existed, which then drained the | 336 | // owed against blocks that no longer existed, which then drained the |
| 299 | // wallet paying fork-era miners out of real-chain coin. | 337 | // wallet paying fork-era miners out of real-chain coin. |
| 338 | // | ||
| 339 | // Upstream keys these on the string "block-<height>". That is the same | ||
| 340 | // bug this fork already paid for, and height keying additionally lets | ||
| 341 | // FindRecordedPoolPayout match the *previous* round's transaction after a | ||
| 342 | // reorg and silently skip a miner, so their `id` is deliberately dropped. | ||
| 300 | int blockHeight = 0; | 343 | int blockHeight = 0; |
| 301 | // Must be explicitly zeroed: uint256's default constructor is empty | 344 | // Must be explicitly zeroed: uint256's default constructor is empty |
| 302 | // (uint256.h:514), so leaving this unassigned gives every ledger entry | 345 | // (uint256.h:514), so leaving this unassigned gives every ledger entry |
| 303 | // loaded from an older file a garbage hash, which then matches no block | 346 | // loaded from an older file a garbage hash, which then matches no block |
| 304 | // and gets voided as "orphaned". That mistake voided a live payout. | 347 | // and gets voided as "orphaned". That mistake voided a live payout. |
| 305 | uint256 blockHash = 0; // zero = entry predates hash keying | 348 | uint256 blockHash = 0; // zero = entry predates hash keying |
| 306 | int matureAtHeight; | 349 | int matureAtHeight = 0; |
| 307 | std::map<std::string, int64> amounts; // address -> satoshis | 350 | std::map<std::string, int64> amounts; // address -> satoshis |
| 351 | // From upstream: which recipients already have a transaction. This is what | ||
| 352 | // lets an entry stay on disk until every address in `amounts` is paid, | ||
| 353 | // instead of being erased the moment the batch is handed off. | ||
| 354 | std::map<std::string, std::string> paidTxids; // address -> txid | ||
| 308 | int retries = 0; // exec-thread only: SendMoney attempts so far | 355 | int retries = 0; // exec-thread only: SendMoney attempts so far |
| 309 | int deferrals = 0; // exec-thread only: waits for spendable coin | 356 | int deferrals = 0; // exec-thread only: waits for spendable coin |
| 357 | // Upstream's `nextAttempt` is this field under another name; ours is kept | ||
| 358 | // so the retry/deferral policy above stays coherent with it. | ||
| 310 | int64 retryAfter = 0; // exec-thread only: unix time, 0 = ready now | 359 | int64 retryAfter = 0; // exec-thread only: unix time, 0 = ready now |
| 311 | }; | 360 | }; |
| 312 | static std::vector<PendingPayout> gPendingPayouts; | 361 | static std::vector<PendingPayout> gPendingPayouts; |
| 362 | static std::mutex gPayoutLedgerMutex; | ||
| 363 | |||
| 364 | struct PoolRoundProof { | ||
| 365 | std::string id; | ||
| 366 | int blockHeight; | ||
| 367 | int matureAtHeight; | ||
| 368 | int64 createdAt; | ||
| 369 | std::string blockHash; | ||
| 370 | std::string coinbaseTxid; | ||
| 371 | std::string foundBy; | ||
| 372 | double feePercent; | ||
| 373 | uint64 totalShares; | ||
| 374 | int64 coinbaseValue; | ||
| 375 | int64 subsidy; | ||
| 376 | int64 minerReward; | ||
| 377 | int64 operatorReward; | ||
| 378 | int skippedDust; | ||
| 379 | int64 forfeitedBadAddress; | ||
| 380 | std::map<std::string, uint64> shares; | ||
| 381 | std::map<std::string, int64> payouts; | ||
| 382 | std::map<std::string, std::string> paidTxids; | ||
| 383 | |||
| 384 | PoolRoundProof() | ||
| 385 | : blockHeight(0), matureAtHeight(0), createdAt(0), feePercent(0.0), | ||
| 386 | totalShares(0), coinbaseValue(0), subsidy(0), minerReward(0), operatorReward(0), | ||
| 387 | skippedDust(0), forfeitedBadAddress(0) {} | ||
| 388 | }; | ||
| 389 | static std::vector<PoolRoundProof> gPoolRounds; | ||
| 390 | static std::mutex gPoolRoundsMutex; | ||
| 391 | static const size_t MAX_POOL_ROUNDS = 200; | ||
| 313 | 392 | ||
| 314 | static std::string PayoutFilePath() | 393 | static std::string PayoutFilePath() |
| 315 | { | 394 | { |
| 316 | return GetAppDir() + "/pending_payouts.json"; | 395 | return GetAppDir() + "/pending_payouts.json"; |
| 317 | } | 396 | } |
| 318 | 397 | ||
| 319 | static void SavePendingPayouts() | 398 | static std::string PoolRoundsFilePath() |
| 399 | { | ||
| 400 | if (!strPoolRoundsFile.empty()) | ||
| 401 | return strPoolRoundsFile; | ||
| 402 | return GetAppDir() + "/pool_rounds.json"; | ||
| 403 | } | ||
| 404 | |||
| 405 | // Identity of a payout round. Used as the pool-round proof id and, more | ||
| 406 | // importantly, as the `pool_payout_id` written into wallet transaction | ||
| 407 | // metadata and matched by FindRecordedPoolPayout(). | ||
| 408 | // | ||
| 409 | // Derived from the block HASH, not from its height alone. Upstream uses | ||
| 410 | // "block-<height>", which names two different blocks identically across a | ||
| 411 | // reorg -- and because this id is what the crash-recovery probe matches on, a | ||
| 412 | // height-keyed id lets that probe find the *previous* round's transaction and | ||
| 413 | // conclude a miner has already been paid when they have not. The height stays | ||
| 414 | // in the string for readability only. | ||
| 415 | // | ||
| 416 | // A zero hash means the entry predates hash keying; those keep the old form so | ||
| 417 | // anything already recorded against them still matches. | ||
| 418 | static std::string PayoutIdForBlock(int blockHeight, const uint256& blockHash) | ||
| 419 | { | ||
| 420 | if (blockHash == 0) | ||
| 421 | return strprintf("block-%d", blockHeight); | ||
| 422 | return strprintf("block-%d-%s", blockHeight, | ||
| 423 | blockHash.ToString().substr(0, 16).c_str()); | ||
| 424 | } | ||
| 425 | |||
| 426 | static bool PayoutFullyPaid(const PendingPayout& pp) | ||
| 427 | { | ||
| 428 | for (const auto& kv : pp.amounts) | ||
| 429 | if (pp.paidTxids.count(kv.first) == 0) | ||
| 430 | return false; | ||
| 431 | return true; | ||
| 432 | } | ||
| 433 | |||
| 434 | // Serialise the ledger. Caller must hold gPayoutLedgerMutex. | ||
| 435 | // | ||
| 436 | // Split from the file write so the payout thread can hold the mutex just long | ||
| 437 | // enough to read the structure and then do the I/O with it released. Upstream | ||
| 438 | // writes the file inside the per-recipient loop with gPayoutLedgerMutex held, | ||
| 439 | // while the event loop's FlushMaturePayouts blocks on that same mutex -- one | ||
| 440 | // slow fsync per recipient, on the thread the miners' I/O shares. | ||
| 441 | static std::string SerializePendingPayoutsLocked() | ||
| 320 | { | 442 | { |
| 321 | nlohmann::json arr = nlohmann::json::array(); | 443 | nlohmann::json arr = nlohmann::json::array(); |
| 322 | for (const auto& pp : gPendingPayouts) { | 444 | for (const auto& pp : gPendingPayouts) { |
| @@ -324,21 +446,49 @@ static void SavePendingPayouts() | |||
| 324 | obj["blockHeight"] = pp.blockHeight; | 446 | obj["blockHeight"] = pp.blockHeight; |
| 325 | obj["blockHash"] = pp.blockHash.ToString(); | 447 | obj["blockHash"] = pp.blockHash.ToString(); |
| 326 | obj["matureAtHeight"] = pp.matureAtHeight; | 448 | obj["matureAtHeight"] = pp.matureAtHeight; |
| 449 | obj["retryAfter"] = pp.retryAfter; | ||
| 327 | nlohmann::json amounts = nlohmann::json::object(); | 450 | nlohmann::json amounts = nlohmann::json::object(); |
| 328 | for (const auto& kv : pp.amounts) | 451 | for (const auto& kv : pp.amounts) |
| 329 | amounts[kv.first] = kv.second; | 452 | amounts[kv.first] = kv.second; |
| 330 | obj["amounts"] = amounts; | 453 | obj["amounts"] = amounts; |
| 454 | nlohmann::json paid = nlohmann::json::object(); | ||
| 455 | for (const auto& kv : pp.paidTxids) | ||
| 456 | paid[kv.first] = kv.second; | ||
| 457 | obj["paidTxids"] = paid; | ||
| 331 | arr.push_back(obj); | 458 | arr.push_back(obj); |
| 332 | } | 459 | } |
| 460 | return arr.dump(2); | ||
| 461 | } | ||
| 462 | |||
| 463 | // Write the serialised ledger. Holds no lock; safe to call with the mutex | ||
| 464 | // released. Atomic tmp+rename so a crash mid-write cannot truncate the record | ||
| 465 | // of who is still owed money. | ||
| 466 | static void WritePendingPayoutsFile(const std::string& s) | ||
| 467 | { | ||
| 333 | std::string path = PayoutFilePath(); | 468 | std::string path = PayoutFilePath(); |
| 334 | FILE* f = fopen(path.c_str(), "w"); | 469 | std::string tmp = path + ".tmp"; |
| 335 | if (f) { | 470 | FILE* f = fopen(tmp.c_str(), "w"); |
| 336 | std::string s = arr.dump(2); | 471 | if (!f) { |
| 337 | fwrite(s.c_str(), 1, s.size(), f); | 472 | LogPrint("payout", "[payout] WARNING: could not write %s\n", tmp.c_str()); |
| 338 | fclose(f); | 473 | return; |
| 339 | } else { | ||
| 340 | LogPrint("payout", "[payout] WARNING: could not write %s\n", path.c_str()); | ||
| 341 | } | 474 | } |
| 475 | fwrite(s.c_str(), 1, s.size(), f); | ||
| 476 | if (fclose(f) != 0) { | ||
| 477 | LogPrint("payout", "[payout] WARNING: could not close %s\n", tmp.c_str()); | ||
| 478 | return; | ||
| 479 | } | ||
| 480 | #ifdef _WIN32 | ||
| 481 | remove(path.c_str()); | ||
| 482 | #endif | ||
| 483 | if (rename(tmp.c_str(), path.c_str()) != 0) | ||
| 484 | LogPrint("payout", "[payout] WARNING: could not replace %s\n", path.c_str()); | ||
| 485 | } | ||
| 486 | |||
| 487 | // Convenience for the cold paths (startup, queueing) where holding the mutex | ||
| 488 | // across the write costs nothing. Caller must hold gPayoutLedgerMutex. | ||
| 489 | static void SavePendingPayoutsLocked() | ||
| 490 | { | ||
| 491 | WritePendingPayoutsFile(SerializePendingPayoutsLocked()); | ||
| 342 | } | 492 | } |
| 343 | 493 | ||
| 344 | static void LoadPendingPayouts() | 494 | static void LoadPendingPayouts() |
| @@ -354,6 +504,7 @@ static void LoadPendingPayouts() | |||
| 354 | fclose(f); | 504 | fclose(f); |
| 355 | try { | 505 | try { |
| 356 | nlohmann::json arr = nlohmann::json::parse(s); | 506 | nlohmann::json arr = nlohmann::json::parse(s); |
| 507 | std::lock_guard<std::mutex> lk(gPayoutLedgerMutex); | ||
| 357 | for (const auto& obj : arr) { | 508 | for (const auto& obj : arr) { |
| 358 | PendingPayout pp; | 509 | PendingPayout pp; |
| 359 | pp.matureAtHeight = obj["matureAtHeight"].get<int>(); | 510 | pp.matureAtHeight = obj["matureAtHeight"].get<int>(); |
| @@ -364,8 +515,24 @@ static void LoadPendingPayouts() | |||
| 364 | pp.blockHash.SetHex(obj["blockHash"].get<std::string>()); | 515 | pp.blockHash.SetHex(obj["blockHash"].get<std::string>()); |
| 365 | if (obj.contains("blockHeight") && obj["blockHeight"].is_number()) | 516 | if (obj.contains("blockHeight") && obj["blockHeight"].is_number()) |
| 366 | pp.blockHeight = obj["blockHeight"].get<int>(); | 517 | pp.blockHeight = obj["blockHeight"].get<int>(); |
| 518 | // Upstream's tolerance for an entry with no usable height, kept | ||
| 519 | // because it costs nothing and an entry that cannot name its block | ||
| 520 | // is worse than one derived from its maturity. COINBASE_MATURITY | ||
| 521 | // and not COINBASE_SPENDABLE_DEPTH: this reverses whatever the | ||
| 522 | // writer of *that* file used, and only files old enough to lack | ||
| 523 | // blockHeight are affected -- they were written when maturity was | ||
| 524 | // still COINBASE_MATURITY. | ||
| 525 | if (pp.blockHeight == 0 && pp.matureAtHeight > COINBASE_MATURITY) | ||
| 526 | pp.blockHeight = pp.matureAtHeight - COINBASE_MATURITY; | ||
| 527 | // Absent in every file written before this merge; default 0 means | ||
| 528 | // "ready now", which is the right answer for a restored entry. | ||
| 529 | if (obj.contains("retryAfter") && obj["retryAfter"].is_number()) | ||
| 530 | pp.retryAfter = obj["retryAfter"].get<int64>(); | ||
| 367 | for (auto it = obj["amounts"].begin(); it != obj["amounts"].end(); ++it) | 531 | for (auto it = obj["amounts"].begin(); it != obj["amounts"].end(); ++it) |
| 368 | pp.amounts[it.key()] = it.value().get<int64>(); | 532 | pp.amounts[it.key()] = it.value().get<int64>(); |
| 533 | if (obj.contains("paidTxids") && obj["paidTxids"].is_object()) | ||
| 534 | for (auto it = obj["paidTxids"].begin(); it != obj["paidTxids"].end(); ++it) | ||
| 535 | pp.paidTxids[it.key()] = it.value().get<std::string>(); | ||
| 369 | gPendingPayouts.push_back(pp); | 536 | gPendingPayouts.push_back(pp); |
| 370 | } | 537 | } |
| 371 | LogPrint("payout", "[payout] loaded %zu pending payout(s) from disk\n", | 538 | LogPrint("payout", "[payout] loaded %zu pending payout(s) from disk\n", |
| @@ -396,6 +563,17 @@ struct PayingRecord { | |||
| 396 | int retries = 0; | 563 | int retries = 0; |
| 397 | int deferrals = 0; | 564 | int deferrals = 0; |
| 398 | int64 retryAfter = 0; | 565 | int64 retryAfter = 0; |
| 566 | // Ask the wallet whether this payment already went out, once, before the | ||
| 567 | // first send attempt. Upstream asks on every attempt: FindRecordedPoolPayout | ||
| 568 | // is a linear scan of mapWallet under cs_mapWallet, per recipient, so with | ||
| 569 | // 2000 rotating payout addresses and a retry budget of 1440 that is a | ||
| 570 | // scan per address per minute for a day. The question it answers -- "did a | ||
| 571 | // previous process send this before it died?" -- can only change across a | ||
| 572 | // restart, and every restart re-sets this flag (ReconcilePayouts), so | ||
| 573 | // asking once per record is the whole of the answer. | ||
| 574 | // | ||
| 575 | // Not persisted: a record read back from disk is by definition a restart. | ||
| 576 | bool fProbeWallet = true; | ||
| 399 | }; | 577 | }; |
| 400 | static std::vector<PayingRecord> gPaying; // guarded by gPayoutExecMutex | 578 | static std::vector<PayingRecord> gPaying; // guarded by gPayoutExecMutex |
| 401 | static std::mutex gPayoutExecMutex; | 579 | static std::mutex gPayoutExecMutex; |
| @@ -432,8 +610,19 @@ static void SavePayingLocked() | |||
| 432 | } | 610 | } |
| 433 | std::string s = arr.dump(2); | 611 | std::string s = arr.dump(2); |
| 434 | fwrite(s.c_str(), 1, s.size(), f); | 612 | fwrite(s.c_str(), 1, s.size(), f); |
| 435 | fflush(f); | 613 | // fclose() is where buffered data actually reaches the OS, so a failure |
| 436 | fclose(f); | 614 | // here means the .tmp is short -- renaming it over the real file would |
| 615 | // destroy the good copy. Upstream's check, taken because a truncated | ||
| 616 | // list of who is owed money is exactly what this function exists to | ||
| 617 | // prevent. rename() on Windows will not replace an existing file, hence | ||
| 618 | // the remove() first. | ||
| 619 | if (fclose(f) != 0) { | ||
| 620 | LogPrint("payout", "[payout] WARNING: could not close %s\n", tmp.c_str()); | ||
| 621 | return; | ||
| 622 | } | ||
| 623 | #ifdef _WIN32 | ||
| 624 | remove(path.c_str()); | ||
| 625 | #endif | ||
| 437 | if (rename(tmp.c_str(), path.c_str()) != 0) | 626 | if (rename(tmp.c_str(), path.c_str()) != 0) |
| 438 | LogPrint("payout", "[payout] WARNING: could not rename %s -> %s\n", | 627 | LogPrint("payout", "[payout] WARNING: could not rename %s -> %s\n", |
| 439 | tmp.c_str(), path.c_str()); | 628 | tmp.c_str(), path.c_str()); |
| @@ -475,6 +664,143 @@ static void LoadPaying() | |||
| 475 | } | 664 | } |
| 476 | } | 665 | } |
| 477 | 666 | ||
| 667 | // --------------------------------------------------------------------------- | ||
| 668 | // Pool round proofs (upstream cbdd06d). Disjoint from the in-flight ledger | ||
| 669 | // above -- this is the operator's published record of how a round was split, | ||
| 670 | // not an obligation to pay. Both sides added code at this offset; both are | ||
| 671 | // kept. | ||
| 672 | // --------------------------------------------------------------------------- | ||
| 673 | static nlohmann::json PoolRoundToJson(const PoolRoundProof& r) | ||
| 674 | { | ||
| 675 | nlohmann::json j; | ||
| 676 | j["id"] = r.id; | ||
| 677 | j["blockHeight"] = r.blockHeight; | ||
| 678 | j["matureAtHeight"] = r.matureAtHeight; | ||
| 679 | j["createdAt"] = r.createdAt; | ||
| 680 | j["blockHash"] = r.blockHash; | ||
| 681 | j["coinbaseTxid"] = r.coinbaseTxid; | ||
| 682 | j["foundBy"] = r.foundBy; | ||
| 683 | j["feePercent"] = r.feePercent; | ||
| 684 | j["totalShares"] = r.totalShares; | ||
| 685 | j["coinbaseValueSatoshis"] = r.coinbaseValue; | ||
| 686 | j["coinbaseValueBtf"] = FormatMoney(r.coinbaseValue); | ||
| 687 | j["subsidySatoshis"] = r.subsidy; | ||
| 688 | j["subsidyBtf"] = FormatMoney(r.subsidy); | ||
| 689 | j["minerRewardSatoshis"] = r.minerReward; | ||
| 690 | j["minerRewardBtf"] = FormatMoney(r.minerReward); | ||
| 691 | j["operatorRewardSatoshis"] = r.operatorReward; | ||
| 692 | j["operatorRewardBtf"] = FormatMoney(r.operatorReward); | ||
| 693 | j["skippedDust"] = r.skippedDust; | ||
| 694 | j["forfeitedBadAddressSatoshis"] = r.forfeitedBadAddress; | ||
| 695 | j["forfeitedBadAddressBtf"] = FormatMoney(r.forfeitedBadAddress); | ||
| 696 | |||
| 697 | j["shares"] = nlohmann::json::object(); | ||
| 698 | for (const auto& kv : r.shares) | ||
| 699 | j["shares"][kv.first] = kv.second; | ||
| 700 | |||
| 701 | j["payouts"] = nlohmann::json::object(); | ||
| 702 | for (const auto& kv : r.payouts) { | ||
| 703 | nlohmann::json p; | ||
| 704 | p["amountSatoshis"] = kv.second; | ||
| 705 | p["amountBtf"] = FormatMoney(kv.second); | ||
| 706 | std::map<std::string, std::string>::const_iterator tx = r.paidTxids.find(kv.first); | ||
| 707 | if (tx != r.paidTxids.end()) | ||
| 708 | p["txid"] = tx->second; | ||
| 709 | else | ||
| 710 | p["txid"] = nullptr; | ||
| 711 | j["payouts"][kv.first] = p; | ||
| 712 | } | ||
| 713 | return j; | ||
| 714 | } | ||
| 715 | |||
| 716 | static void SavePoolRoundsLocked() | ||
| 717 | { | ||
| 718 | nlohmann::json root; | ||
| 719 | root["schema"] = "bitflash-pool-rounds-1"; | ||
| 720 | root["updatedAt"] = GetTime(); | ||
| 721 | root["rounds"] = nlohmann::json::array(); | ||
| 722 | for (const PoolRoundProof& r : gPoolRounds) | ||
| 723 | root["rounds"].push_back(PoolRoundToJson(r)); | ||
| 724 | |||
| 725 | std::string path = PoolRoundsFilePath(); | ||
| 726 | std::string tmp = path + ".tmp"; | ||
| 727 | FILE* f = fopen(tmp.c_str(), "w"); | ||
| 728 | if (!f) { | ||
| 729 | LogPrint("payout", "[payout] WARNING: could not write %s\n", tmp.c_str()); | ||
| 730 | return; | ||
| 731 | } | ||
| 732 | std::string s = root.dump(2); | ||
| 733 | fwrite(s.c_str(), 1, s.size(), f); | ||
| 734 | if (fclose(f) != 0) { | ||
| 735 | LogPrint("payout", "[payout] WARNING: could not close %s\n", tmp.c_str()); | ||
| 736 | return; | ||
| 737 | } | ||
| 738 | #ifdef _WIN32 | ||
| 739 | remove(path.c_str()); | ||
| 740 | #endif | ||
| 741 | if (rename(tmp.c_str(), path.c_str()) != 0) | ||
| 742 | LogPrint("payout", "[payout] WARNING: could not replace %s\n", path.c_str()); | ||
| 743 | } | ||
| 744 | |||
| 745 | static void LoadPoolRounds() | ||
| 746 | { | ||
| 747 | std::string path = PoolRoundsFilePath(); | ||
| 748 | FILE* f = fopen(path.c_str(), "r"); | ||
| 749 | if (!f) return; | ||
| 750 | fseek(f, 0, SEEK_END); | ||
| 751 | long sz = ftell(f); | ||
| 752 | fseek(f, 0, SEEK_SET); | ||
| 753 | std::string s(sz, '\0'); | ||
| 754 | fread(&s[0], 1, sz, f); | ||
| 755 | fclose(f); | ||
| 756 | |||
| 757 | try { | ||
| 758 | nlohmann::json root = nlohmann::json::parse(s); | ||
| 759 | nlohmann::json arr = root.is_array() ? root : root["rounds"]; | ||
| 760 | std::lock_guard<std::mutex> lk(gPoolRoundsMutex); | ||
| 761 | for (const auto& obj : arr) { | ||
| 762 | PoolRoundProof r; | ||
| 763 | if (obj.contains("id") && obj["id"].is_string()) r.id = obj["id"].get<std::string>(); | ||
| 764 | if (obj.contains("blockHeight")) r.blockHeight = obj["blockHeight"].get<int>(); | ||
| 765 | if (obj.contains("matureAtHeight")) r.matureAtHeight = obj["matureAtHeight"].get<int>(); | ||
| 766 | if (obj.contains("createdAt")) r.createdAt = obj["createdAt"].get<int64>(); | ||
| 767 | if (obj.contains("blockHash") && obj["blockHash"].is_string()) r.blockHash = obj["blockHash"].get<std::string>(); | ||
| 768 | if (obj.contains("coinbaseTxid") && obj["coinbaseTxid"].is_string()) r.coinbaseTxid = obj["coinbaseTxid"].get<std::string>(); | ||
| 769 | if (obj.contains("foundBy") && obj["foundBy"].is_string()) r.foundBy = obj["foundBy"].get<std::string>(); | ||
| 770 | if (obj.contains("feePercent")) r.feePercent = obj["feePercent"].get<double>(); | ||
| 771 | if (obj.contains("totalShares")) r.totalShares = obj["totalShares"].get<uint64>(); | ||
| 772 | if (obj.contains("coinbaseValueSatoshis")) r.coinbaseValue = obj["coinbaseValueSatoshis"].get<int64>(); | ||
| 773 | if (obj.contains("subsidySatoshis")) r.subsidy = obj["subsidySatoshis"].get<int64>(); | ||
| 774 | if (obj.contains("minerRewardSatoshis")) r.minerReward = obj["minerRewardSatoshis"].get<int64>(); | ||
| 775 | if (obj.contains("operatorRewardSatoshis")) r.operatorReward = obj["operatorRewardSatoshis"].get<int64>(); | ||
| 776 | if (obj.contains("skippedDust")) r.skippedDust = obj["skippedDust"].get<int>(); | ||
| 777 | if (obj.contains("forfeitedBadAddressSatoshis")) r.forfeitedBadAddress = obj["forfeitedBadAddressSatoshis"].get<int64>(); | ||
| 778 | if (obj.contains("shares") && obj["shares"].is_object()) | ||
| 779 | for (auto it = obj["shares"].begin(); it != obj["shares"].end(); ++it) | ||
| 780 | r.shares[it.key()] = it.value().get<uint64>(); | ||
| 781 | if (obj.contains("payouts") && obj["payouts"].is_object()) | ||
| 782 | for (auto it = obj["payouts"].begin(); it != obj["payouts"].end(); ++it) { | ||
| 783 | if (it.value().is_object()) { | ||
| 784 | r.payouts[it.key()] = it.value().value("amountSatoshis", (int64)0); | ||
| 785 | if (it.value().contains("txid") && it.value()["txid"].is_string()) | ||
| 786 | r.paidTxids[it.key()] = it.value()["txid"].get<std::string>(); | ||
| 787 | } else if (it.value().is_number_integer()) { | ||
| 788 | r.payouts[it.key()] = it.value().get<int64>(); | ||
| 789 | } | ||
| 790 | } | ||
| 791 | if (!r.id.empty()) | ||
| 792 | gPoolRounds.push_back(r); | ||
| 793 | } | ||
| 794 | while (gPoolRounds.size() > MAX_POOL_ROUNDS) | ||
| 795 | gPoolRounds.erase(gPoolRounds.begin()); | ||
| 796 | LogPrint("payout", "[payout] loaded %zu pool round proof(s) from disk\n", | ||
| 797 | gPoolRounds.size()); | ||
| 798 | } catch (const std::exception& e) { | ||
| 799 | LogPrint("payout", "[payout] WARNING: failed to parse %s: %s\n", | ||
| 800 | path.c_str(), e.what()); | ||
| 801 | } | ||
| 802 | } | ||
| 803 | |||
| 478 | // Is the block that funds this payout still on the main chain? | 804 | // Is the block that funds this payout still on the main chain? |
| 479 | // | 805 | // |
| 480 | // A zero hash means the entry was written before payouts recorded one, so | 806 | // A zero hash means the entry was written before payouts recorded one, so |
| @@ -499,85 +825,55 @@ static std::string PayoutComment(int blockHeight) | |||
| 499 | return strprintf("Pool payout height %d", blockHeight); | 825 | return strprintf("Pool payout height %d", blockHeight); |
| 500 | } | 826 | } |
| 501 | 827 | ||
| 502 | // Did we already send this payment before dying? Looks for a wallet | 828 | // Startup reconciliation. |
| 503 | // transaction of ours carrying this payout's comment and paying exactly this | ||
| 504 | // amount to this address. | ||
| 505 | // | 829 | // |
| 506 | // Biased towards reporting "already paid" on any doubt: skipping a payment a | 830 | // This used to be the crash-recovery mechanism: it scanned the wallet with |
| 507 | // miner is owed is visible and recoverable from the log, while paying twice | 831 | // WalletAlreadyPaid() and then dropped ledger entries that had already been |
| 508 | // is neither. | 832 | // exploded into in-flight records. Neither job is right any more. |
| 509 | static bool WalletAlreadyPaid(const PayingRecord& r) | 833 | // |
| 510 | { | 834 | // The ledger is now authoritative and an entry survives until every recipient |
| 511 | uint160 h160; | 835 | // has a txid recorded against it (upstream cbdd06d), so dropping ledger |
| 512 | if (!AddressToHash160(r.address, h160)) | 836 | // entries here would reintroduce exactly the loss this merge is adopting that |
| 513 | return false; | 837 | // model to prevent. And the wallet scan is now FindRecordedPoolPayout(), done |
| 514 | CScript want; | 838 | // once per record on its first attempt, which is the same question asked in |
| 515 | want << OP_DUP << OP_HASH160 << h160 << OP_EQUALVERIFY << OP_CHECKSIG; | 839 | // the one place that can act on the answer. Running both would be two |
| 516 | const std::string comment = PayoutComment(r.blockHeight); | 840 | // authorities for the same debt. |
| 517 | 841 | // | |
| 518 | CRITICAL_BLOCK(cs_mapWallet) | 842 | // What is left is the useful part: the in-flight list restored from disk may |
| 519 | { | 843 | // contain records the ledger already shows as paid (we died between the send |
| 520 | for (std::map<uint256, CWalletTx>::iterator it = mapWallet.begin(); | 844 | // and the in-flight file being rewritten). The ledger wins. |
| 521 | it != mapWallet.end(); ++it) { | ||
| 522 | CWalletTx& wtx = it->second; | ||
| 523 | // The comment is written only by the payout path, so its presence | ||
| 524 | // already identifies the transaction as one of ours. | ||
| 525 | std::map<std::string, std::string>::const_iterator ci = | ||
| 526 | wtx.mapValue.find("comment"); | ||
| 527 | if (ci == wtx.mapValue.end() || ci->second != comment) | ||
| 528 | continue; | ||
| 529 | foreach (const CTxOut& txout, wtx.vout) | ||
| 530 | if (txout.nValue == r.amount && txout.scriptPubKey == want) | ||
| 531 | return true; | ||
| 532 | } | ||
| 533 | } | ||
| 534 | return false; | ||
| 535 | } | ||
| 536 | |||
| 537 | // Startup reconciliation. Drops in-flight records the wallet shows were | ||
| 538 | // already sent, and drops ledger entries that were already exploded into | ||
| 539 | // records (which happens if we died between writing the records and rewriting | ||
| 540 | // the ledger -- both files then describe the same debt). | ||
| 541 | static void ReconcilePayouts() | 845 | static void ReconcilePayouts() |
| 542 | { | 846 | { |
| 543 | std::lock_guard<std::mutex> lk(gPayoutExecMutex); | 847 | std::lock_guard<std::mutex> exec(gPayoutExecMutex); |
| 848 | std::lock_guard<std::mutex> ledger(gPayoutLedgerMutex); | ||
| 544 | 849 | ||
| 545 | size_t settled = 0; | 850 | size_t settled = 0; |
| 546 | int64 settledAmount = 0; | 851 | int64 settledAmount = 0; |
| 547 | for (auto it = gPaying.begin(); it != gPaying.end(); ) { | 852 | for (auto it = gPaying.begin(); it != gPaying.end(); ) { |
| 548 | if (WalletAlreadyPaid(*it)) { | 853 | bool paid = false; |
| 854 | for (const PendingPayout& pp : gPendingPayouts) | ||
| 855 | if (pp.blockHash == it->blockHash && pp.paidTxids.count(it->address)) { | ||
| 856 | paid = true; | ||
| 857 | break; | ||
| 858 | } | ||
| 859 | if (paid) { | ||
| 549 | settledAmount += it->amount; | 860 | settledAmount += it->amount; |
| 550 | settled++; | 861 | settled++; |
| 551 | it = gPaying.erase(it); | 862 | it = gPaying.erase(it); |
| 552 | } else { | 863 | } else { |
| 864 | // Every restored record gets one wallet probe before it is allowed | ||
| 865 | // to send: this is precisely the window where the wallet may hold | ||
| 866 | // a transaction the ledger has not recorded yet. | ||
| 867 | it->fProbeWallet = true; | ||
| 553 | ++it; | 868 | ++it; |
| 554 | } | 869 | } |
| 555 | } | 870 | } |
| 556 | if (settled) { | 871 | if (settled) { |
| 557 | LogPrint("payout", "[payout] %zu in-flight payment(s) totalling %s were" | 872 | LogPrint("payout", "[payout] %zu in-flight payment(s) totalling %s are" |
| 558 | " already sent before the last shutdown -- not resending\n", | 873 | " already recorded paid in the ledger -- not resending\n", |
| 559 | settled, FormatMoney(settledAmount).c_str()); | 874 | settled, FormatMoney(settledAmount).c_str()); |
| 560 | SavePayingLocked(); | ||
| 561 | } | ||
| 562 | |||
| 563 | size_t dropped = 0; | ||
| 564 | for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) { | ||
| 565 | bool exploded = false; | ||
| 566 | for (const auto& r : gPaying) | ||
| 567 | if (r.matureAtHeight == it->matureAtHeight && r.blockHash == it->blockHash) { | ||
| 568 | exploded = true; | ||
| 569 | break; | ||
| 570 | } | ||
| 571 | if (exploded) { | ||
| 572 | LogPrint("payout", "[payout] ledger entry for block %d is already" | ||
| 573 | " in flight -- dropping the duplicate\n", it->blockHeight); | ||
| 574 | it = gPendingPayouts.erase(it); | ||
| 575 | dropped++; | ||
| 576 | } else { | ||
| 577 | ++it; | ||
| 578 | } | ||
| 579 | } | 875 | } |
| 580 | if (dropped) SavePendingPayouts(); | 876 | SavePayingLocked(); |
| 581 | 877 | ||
| 582 | if (!gPaying.empty()) { | 878 | if (!gPaying.empty()) { |
| 583 | int64 owed = 0; | 879 | int64 owed = 0; |
| @@ -587,12 +883,100 @@ static void ReconcilePayouts() | |||
| 587 | } | 883 | } |
| 588 | } | 884 | } |
| 589 | 885 | ||
| 886 | static void RecordPoolRound(const PoolRoundProof& round) | ||
| 887 | { | ||
| 888 | std::lock_guard<std::mutex> lk(gPoolRoundsMutex); | ||
| 889 | for (PoolRoundProof& existing : gPoolRounds) { | ||
| 890 | if (existing.id == round.id) { | ||
| 891 | existing = round; | ||
| 892 | SavePoolRoundsLocked(); | ||
| 893 | return; | ||
| 894 | } | ||
| 895 | } | ||
| 896 | gPoolRounds.push_back(round); | ||
| 897 | while (gPoolRounds.size() > MAX_POOL_ROUNDS) | ||
| 898 | gPoolRounds.erase(gPoolRounds.begin()); | ||
| 899 | SavePoolRoundsLocked(); | ||
| 900 | } | ||
| 901 | |||
| 902 | static void MarkPoolRoundPaid(const std::string& id, | ||
| 903 | const std::string& address, | ||
| 904 | const std::string& txid) | ||
| 905 | { | ||
| 906 | std::lock_guard<std::mutex> lk(gPoolRoundsMutex); | ||
| 907 | for (PoolRoundProof& r : gPoolRounds) { | ||
| 908 | if (r.id != id) | ||
| 909 | continue; | ||
| 910 | r.paidTxids[address] = txid; | ||
| 911 | SavePoolRoundsLocked(); | ||
| 912 | return; | ||
| 913 | } | ||
| 914 | } | ||
| 915 | |||
| 916 | // Upstream's gPayoutExecQueue/gPayoutExecMutex/gPayoutThreadRunning are | ||
| 917 | // declared above with gPaying: the in-flight list IS the exec queue in this | ||
| 918 | // tree, at per-recipient rather than per-round granularity, so a second queue | ||
| 919 | // would be a second answer to "who is owed money". Only the dedupe set is new. | ||
| 920 | // | ||
| 921 | // Ids of rounds currently exploded into gPaying, so a flush cannot queue the | ||
| 922 | // same round twice. Guarded by gPayoutExecMutex. | ||
| 923 | static std::set<std::string> gQueuedPayouts; | ||
| 924 | |||
| 925 | static bool FindRecordedPoolPayout(const std::string& id, | ||
| 926 | const std::string& address, | ||
| 927 | int64 amount, | ||
| 928 | std::string& txidRet) | ||
| 929 | { | ||
| 930 | CRITICAL_BLOCK(cs_mapWallet) | ||
| 931 | { | ||
| 932 | for (map<uint256, CWalletTx>::const_iterator it = mapWallet.begin(); | ||
| 933 | it != mapWallet.end(); ++it) | ||
| 934 | { | ||
| 935 | const CWalletTx& wtx = it->second; | ||
| 936 | map<string, string>::const_iterator idIt = wtx.mapValue.find("pool_payout_id"); | ||
| 937 | if (idIt == wtx.mapValue.end() || idIt->second != id) | ||
| 938 | continue; | ||
| 939 | map<string, string>::const_iterator addrIt = wtx.mapValue.find("pool_payout_address"); | ||
| 940 | if (addrIt == wtx.mapValue.end() || addrIt->second != address) | ||
| 941 | continue; | ||
| 942 | map<string, string>::const_iterator amtIt = wtx.mapValue.find("pool_payout_amount"); | ||
| 943 | if (amtIt == wtx.mapValue.end() || amtIt->second != strprintf("%lld", amount)) | ||
| 944 | continue; | ||
| 945 | txidRet = wtx.GetHash().GetHex(); | ||
| 946 | return true; | ||
| 947 | } | ||
| 948 | } | ||
| 949 | return false; | ||
| 950 | } | ||
| 951 | |||
| 952 | // blockHash stays a uint256 and not upstream's display string: it is the key | ||
| 953 | // this fork's payout ledger is built on, and a key that has been through | ||
| 954 | // ToString() is a key that can be compared wrong. It is converted to text only | ||
| 955 | // where the round proof needs text. The other three arguments are upstream's, | ||
| 956 | // carried for the round-proof data. | ||
| 590 | static void QueuePayouts(int blockHeight, | 957 | static void QueuePayouts(int blockHeight, |
| 591 | const uint256& blockHash, | 958 | const uint256& blockHash, |
| 959 | const std::string& coinbaseTxid, | ||
| 960 | int64 coinbaseValue, | ||
| 961 | const std::string& foundBy, | ||
| 592 | const std::map<std::string, uint64>& shareCount, | 962 | const std::map<std::string, uint64>& shareCount, |
| 593 | uint64 total) | 963 | uint64 total) |
| 594 | { | 964 | { |
| 965 | PoolRoundProof round; | ||
| 966 | round.id = PayoutIdForBlock(blockHeight, blockHash); | ||
| 967 | round.blockHeight = blockHeight; | ||
| 968 | round.matureAtHeight = blockHeight + COINBASE_SPENDABLE_DEPTH; | ||
| 969 | round.createdAt = GetTime(); | ||
| 970 | round.blockHash = blockHash.ToString(); | ||
| 971 | round.coinbaseTxid = coinbaseTxid; | ||
| 972 | round.coinbaseValue = coinbaseValue; | ||
| 973 | round.foundBy = foundBy; | ||
| 974 | round.feePercent = dPoolFeePercent; | ||
| 975 | round.totalShares = total; | ||
| 976 | round.shares = shareCount; | ||
| 977 | |||
| 595 | if (total == 0) { | 978 | if (total == 0) { |
| 979 | RecordPoolRound(round); | ||
| 596 | LogPrint("payout", "[payout] block %d: zero shares recorded -- nothing to pay out\n", | 980 | LogPrint("payout", "[payout] block %d: zero shares recorded -- nothing to pay out\n", |
| 597 | blockHeight); | 981 | blockHeight); |
| 598 | return; | 982 | return; |
| @@ -608,6 +992,10 @@ static void QueuePayouts(int blockHeight, | |||
| 608 | int64 nSubsidy = 50 * COIN; | 992 | int64 nSubsidy = 50 * COIN; |
| 609 | nSubsidy >>= (blockHeight / 210000); // halving schedule | 993 | nSubsidy >>= (blockHeight / 210000); // halving schedule |
| 610 | int64 minerReward = (int64)((double)nSubsidy * (1.0 - feeFrac)); | 994 | int64 minerReward = (int64)((double)nSubsidy * (1.0 - feeFrac)); |
| 995 | int64 operatorCut = nSubsidy - minerReward; | ||
| 996 | round.subsidy = nSubsidy; | ||
| 997 | round.minerReward = minerReward; | ||
| 998 | round.operatorReward = operatorCut; | ||
| 611 | 999 | ||
| 612 | PendingPayout pp; | 1000 | PendingPayout pp; |
| 613 | pp.blockHeight = blockHeight; | 1001 | pp.blockHeight = blockHeight; |
| @@ -621,7 +1009,9 @@ static void QueuePayouts(int blockHeight, | |||
| 621 | // alone on a private fork paying itself, so it says nothing about a pool | 1009 | // alone on a private fork paying itself, so it says nothing about a pool |
| 622 | // competing on the real chain. Waiting the extra 20 blocks makes each | 1010 | // competing on the real chain. Waiting the extra 20 blocks makes each |
| 623 | // payout funded by construction, and puts the maturity gate past the | 1011 | // payout funded by construction, and puts the maturity gate past the |
| 624 | // reorg depth that orphaned 280 blocks of obligations. | 1012 | // reorg depth that orphaned 280 blocks of obligations. Upstream uses |
| 1013 | // COINBASE_MATURITY here; this is a considered disagreement, not an | ||
| 1014 | // oversight -- see the merge commit message. | ||
| 625 | pp.matureAtHeight = blockHeight + COINBASE_SPENDABLE_DEPTH; | 1015 | pp.matureAtHeight = blockHeight + COINBASE_SPENDABLE_DEPTH; |
| 626 | 1016 | ||
| 627 | int skipped = 0; | 1017 | int skipped = 0; |
| @@ -643,12 +1033,18 @@ static void QueuePayouts(int blockHeight, | |||
| 643 | continue; | 1033 | continue; |
| 644 | } | 1034 | } |
| 645 | pp.amounts[kv.first] = amount; | 1035 | pp.amounts[kv.first] = amount; |
| 1036 | round.payouts[kv.first] = amount; | ||
| 646 | } | 1037 | } |
| 1038 | round.skippedDust = skipped; | ||
| 1039 | round.forfeitedBadAddress = burned; | ||
| 647 | 1040 | ||
| 648 | gPendingPayouts.push_back(pp); | 1041 | { |
| 649 | SavePendingPayouts(); | 1042 | std::lock_guard<std::mutex> lk(gPayoutLedgerMutex); |
| 1043 | gPendingPayouts.push_back(pp); | ||
| 1044 | SavePendingPayoutsLocked(); | ||
| 1045 | } | ||
| 1046 | RecordPoolRound(round); | ||
| 650 | 1047 | ||
| 651 | int64 operatorCut = nSubsidy - minerReward; | ||
| 652 | LogPrint("payout", | 1048 | LogPrint("payout", |
| 653 | "[payout] block %d: subsidy %s BTF, miners get %s" | 1049 | "[payout] block %d: subsidy %s BTF, miners get %s" |
| 654 | " (fee %.2f%% = %s to operator)," | 1050 | " (fee %.2f%% = %s to operator)," |
| @@ -664,66 +1060,110 @@ static void QueuePayouts(int blockHeight, | |||
| 664 | : ""); | 1060 | : ""); |
| 665 | } | 1061 | } |
| 666 | 1062 | ||
| 667 | // Called from event loop: move matured entries into the in-flight list (fast). | 1063 | // Called from the event loop: explode matured ledger entries into per-recipient |
| 1064 | // in-flight records (fast; no wallet or disk work beyond one small write). | ||
| 1065 | // | ||
| 1066 | // The ledger entry SURVIVES here. That is upstream's model and it is the | ||
| 1067 | // crash-safe one: this fork used to erase the entry as soon as its payments | ||
| 1068 | // were handed off, so a death in that window left the debt described by | ||
| 1069 | // nothing at all -- a container restart on 2026-07-30 silently dropped | ||
| 1070 | // ~1,156 BTF exactly that way. An entry is removed only once every recipient | ||
| 1071 | // in it has a txid recorded (PayoutFullyPaid), which the payout thread does. | ||
| 668 | // | 1072 | // |
| 669 | // Ordering matters and is deliberate: the in-flight records are written and | 1073 | // gPaying is kept on top of that as the in-flight working set: it carries the |
| 670 | // fsync'd to their own file *before* the ledger entry is erased, so the debt | 1074 | // per-recipient retry/deferral state that a per-round `nextAttempt` cannot |
| 671 | // is always described by at least one of the two files. Dying in the window | 1075 | // express, and it is what the payout thread iterates. It is a cache of work |
| 672 | // where both describe it is handled at startup by ReconcilePayouts, which | 1076 | // derived from the ledger, never an independent record of debt -- if it were |
| 673 | // drops the ledger entry it can see is already in flight. | 1077 | // lost entirely the ledger would rebuild it on the next flush. |
| 674 | static void FlushMaturePayouts() | 1078 | static void FlushMaturePayouts() |
| 675 | { | 1079 | { |
| 676 | // Void obligations whose funding block lost a reorg. The coin that was | 1080 | // Which funding blocks are still on the main chain? Asked with no payout |
| 677 | // meant to pay them does not exist, so paying anyway spends real-chain | 1081 | // lock held: PayoutBlockStillOnChain takes cs_main, and cs_main is held by |
| 678 | // coin the pool owes to somebody else. | 1082 | // ProcessBlock and SendMoney for long stretches. Holding the ledger mutex |
| 679 | bool voided = false; | 1083 | // across it would let the event loop park the payout thread behind a chain |
| 680 | for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) { | 1084 | // lock it has no interest in. (It is not a deadlock either way -- nothing |
| 681 | if (PayoutBlockStillOnChain(it->blockHash)) { ++it; continue; } | 1085 | // takes the ledger mutex while holding cs_main -- but it is exactly the |
| 682 | int64 total = 0; | 1086 | // kind of stall this merge is removing from the ledger path.) |
| 683 | for (auto& kv : it->amounts) total += kv.second; | 1087 | std::vector<uint256> orphaned; |
| 684 | LogPrint("payout", "[payout] block %d (%s) was orphaned -- voiding %zu" | 1088 | { |
| 685 | " payment(s) totalling %s\n", | 1089 | std::vector<uint256> hashes; |
| 686 | it->blockHeight, it->blockHash.ToString().substr(0, 14).c_str(), | 1090 | { |
| 687 | it->amounts.size(), FormatMoney(total).c_str()); | 1091 | std::lock_guard<std::mutex> ledger(gPayoutLedgerMutex); |
| 688 | it = gPendingPayouts.erase(it); | 1092 | for (const PendingPayout& pp : gPendingPayouts) |
| 689 | voided = true; | 1093 | hashes.push_back(pp.blockHash); |
| 1094 | } | ||
| 1095 | for (const uint256& h : hashes) | ||
| 1096 | if (!PayoutBlockStillOnChain(h)) | ||
| 1097 | orphaned.push_back(h); | ||
| 690 | } | 1098 | } |
| 691 | if (voided) SavePendingPayouts(); | ||
| 692 | 1099 | ||
| 693 | std::vector<PendingPayout> ready; | 1100 | std::vector<PendingPayout> ready; |
| 694 | for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) { | 1101 | { |
| 695 | if (nBestHeight < it->matureAtHeight) { ++it; continue; } | 1102 | std::lock_guard<std::mutex> ledger(gPayoutLedgerMutex); |
| 696 | LogPrint("payout", "[payout] block height %d matured -- queuing %zu payments\n", | 1103 | |
| 697 | it->matureAtHeight, it->amounts.size()); | 1104 | // Void obligations whose funding block lost a reorg. The coin that was |
| 698 | ready.push_back(*it); // copy first; erase after save | 1105 | // meant to pay them does not exist, so paying anyway spends real-chain |
| 699 | ++it; | 1106 | // coin the pool owes to somebody else. |
| 700 | } | 1107 | bool voided = false; |
| 701 | if (!ready.empty()) { | ||
| 702 | // Record the individual payments first, and only then drop the ledger | ||
| 703 | // entries that produced them. | ||
| 704 | { | ||
| 705 | std::lock_guard<std::mutex> lk(gPayoutExecMutex); | ||
| 706 | for (const auto& pp : ready) { | ||
| 707 | for (const auto& kv : pp.amounts) { | ||
| 708 | PayingRecord r; | ||
| 709 | r.blockHeight = pp.blockHeight; | ||
| 710 | r.blockHash = pp.blockHash; | ||
| 711 | r.matureAtHeight = pp.matureAtHeight; | ||
| 712 | r.address = kv.first; | ||
| 713 | r.amount = kv.second; | ||
| 714 | gPaying.push_back(r); | ||
| 715 | } | ||
| 716 | } | ||
| 717 | SavePayingLocked(); | ||
| 718 | } | ||
| 719 | for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) { | 1108 | for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ) { |
| 720 | if (nBestHeight >= it->matureAtHeight) | 1109 | if (std::find(orphaned.begin(), orphaned.end(), it->blockHash) == orphaned.end()) { |
| 721 | it = gPendingPayouts.erase(it); | ||
| 722 | else | ||
| 723 | ++it; | 1110 | ++it; |
| 1111 | continue; | ||
| 1112 | } | ||
| 1113 | int64 total = 0; | ||
| 1114 | for (auto& kv : it->amounts) total += kv.second; | ||
| 1115 | LogPrint("payout", "[payout] block %d (%s) was orphaned -- voiding %zu" | ||
| 1116 | " payment(s) totalling %s\n", | ||
| 1117 | it->blockHeight, it->blockHash.ToString().substr(0, 14).c_str(), | ||
| 1118 | it->amounts.size(), FormatMoney(total).c_str()); | ||
| 1119 | it = gPendingPayouts.erase(it); | ||
| 1120 | voided = true; | ||
| 724 | } | 1121 | } |
| 725 | SavePendingPayouts(); | 1122 | if (voided) SavePendingPayoutsLocked(); |
| 1123 | |||
| 1124 | int64 now = GetTime(); | ||
| 1125 | for (const PendingPayout& pp : gPendingPayouts) { | ||
| 1126 | if (nBestHeight < pp.matureAtHeight) | ||
| 1127 | continue; | ||
| 1128 | if (pp.retryAfter > now) | ||
| 1129 | continue; | ||
| 1130 | if (PayoutFullyPaid(pp)) | ||
| 1131 | continue; | ||
| 1132 | ready.push_back(pp); | ||
| 1133 | } | ||
| 1134 | } | ||
| 1135 | |||
| 1136 | if (ready.empty()) | ||
| 1137 | return; | ||
| 1138 | |||
| 1139 | std::lock_guard<std::mutex> queue(gPayoutExecMutex); | ||
| 1140 | size_t added = 0; | ||
| 1141 | for (const PendingPayout& pp : ready) { | ||
| 1142 | const std::string id = PayoutIdForBlock(pp.blockHeight, pp.blockHash); | ||
| 1143 | if (gQueuedPayouts.count(id)) | ||
| 1144 | continue; // already exploded into gPaying and still in flight | ||
| 1145 | size_t n = 0; | ||
| 1146 | for (const auto& kv : pp.amounts) { | ||
| 1147 | // Upstream's per-recipient dedupe: a recipient already carrying a | ||
| 1148 | // txid is done, even though the entry as a whole is not. | ||
| 1149 | if (pp.paidTxids.count(kv.first)) | ||
| 1150 | continue; | ||
| 1151 | PayingRecord r; | ||
| 1152 | r.blockHeight = pp.blockHeight; | ||
| 1153 | r.blockHash = pp.blockHash; | ||
| 1154 | r.matureAtHeight = pp.matureAtHeight; | ||
| 1155 | r.address = kv.first; | ||
| 1156 | r.amount = kv.second; | ||
| 1157 | gPaying.push_back(r); | ||
| 1158 | n++; | ||
| 1159 | } | ||
| 1160 | gQueuedPayouts.insert(id); | ||
| 1161 | added += n; | ||
| 1162 | LogPrint("payout", "[payout] block height %d matured -- queuing payout %s" | ||
| 1163 | " (%zu of %zu recipients still owed)\n", | ||
| 1164 | pp.matureAtHeight, id.c_str(), n, pp.amounts.size()); | ||
| 726 | } | 1165 | } |
| 1166 | if (added) SavePayingLocked(); | ||
| 727 | } | 1167 | } |
| 728 | 1168 | ||
| 729 | // Identity of a payment, for finding its record again after SendMoney. | 1169 | // Identity of a payment, for finding its record again after SendMoney. |
| @@ -737,13 +1177,24 @@ static bool SamePayment(const PayingRecord& a, const PayingRecord& b) | |||
| 737 | } | 1177 | } |
| 738 | 1178 | ||
| 739 | // Settle a payment: it is either done or abandoned, either way it stops being | 1179 | // Settle a payment: it is either done or abandoned, either way it stops being |
| 740 | // owed. Removing the record and rewriting the file is what makes the payment | 1180 | // in flight. Note this only retires the working record -- whether the miner is |
| 741 | // not happen twice after a restart. | 1181 | // still OWED the money is the ledger's business, and the ledger is updated |
| 1182 | // separately by RecordPayoutTxid(). | ||
| 742 | static void ForgetPayment(const PayingRecord& r) | 1183 | static void ForgetPayment(const PayingRecord& r) |
| 743 | { | 1184 | { |
| 744 | std::lock_guard<std::mutex> lk(gPayoutExecMutex); | 1185 | std::lock_guard<std::mutex> lk(gPayoutExecMutex); |
| 745 | for (auto it = gPaying.begin(); it != gPaying.end(); ++it) | 1186 | for (auto it = gPaying.begin(); it != gPaying.end(); ++it) |
| 746 | if (SamePayment(*it, r)) { gPaying.erase(it); break; } | 1187 | if (SamePayment(*it, r)) { gPaying.erase(it); break; } |
| 1188 | |||
| 1189 | // When the last record for a round is gone the round is no longer in | ||
| 1190 | // flight, so a later flush may explode it again if the ledger still shows | ||
| 1191 | // recipients owed (a give-up, for instance, leaves the entry standing). | ||
| 1192 | bool stillInFlight = false; | ||
| 1193 | for (const auto& e : gPaying) | ||
| 1194 | if (e.blockHash == r.blockHash) { stillInFlight = true; break; } | ||
| 1195 | if (!stillInFlight) | ||
| 1196 | gQueuedPayouts.erase(PayoutIdForBlock(r.blockHeight, r.blockHash)); | ||
| 1197 | |||
| 747 | SavePayingLocked(); | 1198 | SavePayingLocked(); |
| 748 | } | 1199 | } |
| 749 | 1200 | ||
| @@ -753,14 +1204,77 @@ static void ReschedulePayment(const PayingRecord& r) | |||
| 753 | std::lock_guard<std::mutex> lk(gPayoutExecMutex); | 1204 | std::lock_guard<std::mutex> lk(gPayoutExecMutex); |
| 754 | for (auto& e : gPaying) | 1205 | for (auto& e : gPaying) |
| 755 | if (SamePayment(e, r)) { | 1206 | if (SamePayment(e, r)) { |
| 756 | e.retries = r.retries; | 1207 | e.retries = r.retries; |
| 757 | e.deferrals = r.deferrals; | 1208 | e.deferrals = r.deferrals; |
| 758 | e.retryAfter = r.retryAfter; | 1209 | e.retryAfter = r.retryAfter; |
| 1210 | // Carry the cleared probe flag back, or every retry re-scans the | ||
| 1211 | // whole wallet -- the cost this merge is specifically removing. | ||
| 1212 | e.fProbeWallet = r.fProbeWallet; | ||
| 759 | break; | 1213 | break; |
| 760 | } | 1214 | } |
| 761 | SavePayingLocked(); | 1215 | SavePayingLocked(); |
| 762 | } | 1216 | } |
| 763 | 1217 | ||
| 1218 | // Record that a recipient has been paid, in the ledger, which is the only | ||
| 1219 | // durable statement that the debt is settled. | ||
| 1220 | // | ||
| 1221 | // The file write happens with gPayoutLedgerMutex RELEASED. Upstream calls | ||
| 1222 | // SavePendingPayoutsLocked() inside the per-recipient loop with the mutex | ||
| 1223 | // held, so every recipient costs the event loop a full serialise+fsync it has | ||
| 1224 | // to wait for in FlushMaturePayouts. | ||
| 1225 | static void RecordPayoutTxid(const PayingRecord& r, const std::string& txid) | ||
| 1226 | { | ||
| 1227 | std::string blob; | ||
| 1228 | bool complete = false; | ||
| 1229 | { | ||
| 1230 | std::lock_guard<std::mutex> lk(gPayoutLedgerMutex); | ||
| 1231 | for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ++it) { | ||
| 1232 | if (it->blockHash != r.blockHash) | ||
| 1233 | continue; | ||
| 1234 | it->paidTxids[r.address] = txid; | ||
| 1235 | it->retryAfter = 0; | ||
| 1236 | if (PayoutFullyPaid(*it)) { | ||
| 1237 | LogPrint("payout", "[payout] payout for block %d complete --" | ||
| 1238 | " removing from ledger\n", it->blockHeight); | ||
| 1239 | gPendingPayouts.erase(it); | ||
| 1240 | complete = true; | ||
| 1241 | } | ||
| 1242 | break; | ||
| 1243 | } | ||
| 1244 | blob = SerializePendingPayoutsLocked(); | ||
| 1245 | } | ||
| 1246 | WritePendingPayoutsFile(blob); | ||
| 1247 | // The round proof records a txid per address, so this is per recipient and | ||
| 1248 | // not only on completion. | ||
| 1249 | MarkPoolRoundPaid(PayoutIdForBlock(r.blockHeight, r.blockHash), r.address, txid); | ||
| 1250 | if (complete) | ||
| 1251 | LogPrint("payout", "[payout] block %d fully settled\n", r.blockHeight); | ||
| 1252 | } | ||
| 1253 | |||
| 1254 | // Write a recipient off. Only reached after the full retry budget against a | ||
| 1255 | // block that is still on the main chain with coin available to spend, i.e. a | ||
| 1256 | // wallet fault rather than a maturity or reorg question. | ||
| 1257 | // | ||
| 1258 | // The amount is removed from the entry rather than marked paid, so the money | ||
| 1259 | // is not reported as sent. Once nothing is left owed the entry retires. | ||
| 1260 | static void ForfeitPayoutRecipient(const PayingRecord& r) | ||
| 1261 | { | ||
| 1262 | std::string blob; | ||
| 1263 | { | ||
| 1264 | std::lock_guard<std::mutex> lk(gPayoutLedgerMutex); | ||
| 1265 | for (auto it = gPendingPayouts.begin(); it != gPendingPayouts.end(); ++it) { | ||
| 1266 | if (it->blockHash != r.blockHash) | ||
| 1267 | continue; | ||
| 1268 | it->amounts.erase(r.address); | ||
| 1269 | if (it->amounts.empty() || PayoutFullyPaid(*it)) | ||
| 1270 | gPendingPayouts.erase(it); | ||
| 1271 | break; | ||
| 1272 | } | ||
| 1273 | blob = SerializePendingPayoutsLocked(); | ||
| 1274 | } | ||
| 1275 | WritePendingPayoutsFile(blob); | ||
| 1276 | } | ||
| 1277 | |||
| 764 | // Payout execution thread: works through gPaying, calls SendMoney. | 1278 | // Payout execution thread: works through gPaying, calls SendMoney. |
| 765 | // Runs independently of the event loop so slow wallet operations | 1279 | // Runs independently of the event loop so slow wallet operations |
| 766 | // (SendMoney takes cs_main) never stall miner I/O. | 1280 | // (SendMoney takes cs_main) never stall miner I/O. |
| @@ -780,9 +1294,12 @@ static void PayoutThreadFn(void*) | |||
| 780 | } | 1294 | } |
| 781 | 1295 | ||
| 782 | for (PayingRecord r : batch) { | 1296 | for (PayingRecord r : batch) { |
| 1297 | const std::string id = PayoutIdForBlock(r.blockHeight, r.blockHash); | ||
| 1298 | |||
| 783 | // Re-check here and not only at maturity: a payment can sit in this | 1299 | // Re-check here and not only at maturity: a payment can sit in this |
| 784 | // list for hours across retries, and the reorg that kills its | 1300 | // list for hours across retries, and the reorg that kills its |
| 785 | // funding block can land at any point in that window. | 1301 | // funding block can land at any point in that window. Upstream has |
| 1302 | // no equivalent -- it decides once, when the entry matures. | ||
| 786 | if (!PayoutBlockStillOnChain(r.blockHash)) { | 1303 | if (!PayoutBlockStillOnChain(r.blockHash)) { |
| 787 | LogPrint("payout", "[payout] block %d (%s) orphaned while queued" | 1304 | LogPrint("payout", "[payout] block %d (%s) orphaned while queued" |
| 788 | " -- dropping %s to %s\n", | 1305 | " -- dropping %s to %s\n", |
| @@ -792,6 +1309,25 @@ static void PayoutThreadFn(void*) | |||
| 792 | continue; | 1309 | continue; |
| 793 | } | 1310 | } |
| 794 | 1311 | ||
| 1312 | // Upstream's per-recipient dedupe, against the ledger rather than | ||
| 1313 | // against the batch: if a txid is already recorded for this | ||
| 1314 | // address, the money went out and nothing more is owed. | ||
| 1315 | { | ||
| 1316 | bool alreadyPaid = false; | ||
| 1317 | { | ||
| 1318 | std::lock_guard<std::mutex> lk(gPayoutLedgerMutex); | ||
| 1319 | for (const PendingPayout& pp : gPendingPayouts) | ||
| 1320 | if (pp.blockHash == r.blockHash && pp.paidTxids.count(r.address)) { | ||
| 1321 | alreadyPaid = true; | ||
| 1322 | break; | ||
| 1323 | } | ||
| 1324 | } | ||
| 1325 | if (alreadyPaid) { | ||
| 1326 | ForgetPayment(r); | ||
| 1327 | continue; | ||
| 1328 | } | ||
| 1329 | } | ||
| 1330 | |||
| 795 | uint160 h160; | 1331 | uint160 h160; |
| 796 | if (!AddressToHash160(r.address, h160)) { | 1332 | if (!AddressToHash160(r.address, h160)) { |
| 797 | // Should not happen -- bad addresses are filtered in QueuePayouts | 1333 | // Should not happen -- bad addresses are filtered in QueuePayouts |
| @@ -803,6 +1339,22 @@ static void PayoutThreadFn(void*) | |||
| 803 | CScript sc; | 1339 | CScript sc; |
| 804 | sc << OP_DUP << OP_HASH160 << h160 << OP_EQUALVERIFY << OP_CHECKSIG; | 1340 | sc << OP_DUP << OP_HASH160 << h160 << OP_EQUALVERIFY << OP_CHECKSIG; |
| 805 | 1341 | ||
| 1342 | // Did a previous process send this and die before recording it? | ||
| 1343 | // Only asked once per record -- see PayingRecord::fProbeWallet. | ||
| 1344 | if (r.fProbeWallet) { | ||
| 1345 | std::string recovered; | ||
| 1346 | r.fProbeWallet = false; | ||
| 1347 | if (FindRecordedPoolPayout(id, r.address, r.amount, recovered)) { | ||
| 1348 | LogPrint("payout", "[payout] recovered recorded payout %s to %s" | ||
| 1349 | " from wallet tx %s -- not resending\n", | ||
| 1350 | FormatMoney(r.amount).c_str(), r.address.c_str(), | ||
| 1351 | recovered.c_str()); | ||
| 1352 | RecordPayoutTxid(r, recovered); | ||
| 1353 | ForgetPayment(r); | ||
| 1354 | continue; | ||
| 1355 | } | ||
| 1356 | } | ||
| 1357 | |||
| 806 | // Waiting for coin to mature is not a failure, so it must not | 1358 | // Waiting for coin to mature is not a failure, so it must not |
| 807 | // consume the retry budget. Burning a retry every 60s means a | 1359 | // consume the retry budget. Burning a retry every 60s means a |
| 808 | // payment the pool fully intends to honour can hit GIVING UP | 1360 | // payment the pool fully intends to honour can hit GIVING UP |
| @@ -825,11 +1377,22 @@ static void PayoutThreadFn(void*) | |||
| 825 | 1377 | ||
| 826 | CWalletTx wtx; | 1378 | CWalletTx wtx; |
| 827 | wtx.mapValue["comment"] = PayoutComment(r.blockHeight); | 1379 | wtx.mapValue["comment"] = PayoutComment(r.blockHeight); |
| 1380 | // Upstream's structured metadata. This is what makes the wallet | ||
| 1381 | // probe above possible at all -- the old comment-only form could | ||
| 1382 | // not distinguish two rounds paying the same address the same | ||
| 1383 | // amount. Keyed by the hash-derived id, so a reorg cannot make one | ||
| 1384 | // round's transaction answer for another's. | ||
| 1385 | wtx.mapValue["pool_payout_id"] = id; | ||
| 1386 | wtx.mapValue["pool_payout_address"] = r.address; | ||
| 1387 | wtx.mapValue["pool_payout_amount"] = strprintf("%lld", r.amount); | ||
| 828 | 1388 | ||
| 829 | if (SendMoney(sc, r.amount, wtx)) { | 1389 | if (SendMoney(sc, r.amount, wtx)) { |
| 830 | // The wallet holds the transaction before this returns, so a | 1390 | // The wallet holds the transaction before this returns, so a |
| 831 | // crash here is recoverable: ReconcilePayouts finds it by | 1391 | // crash between here and the ledger write is recoverable: the |
| 832 | // comment+address+amount and does not resend. | 1392 | // probe above finds it by id+address+amount and does not |
| 1393 | // resend. Ledger first, in-flight record second -- the debt is | ||
| 1394 | // described by at least one of them at every instant. | ||
| 1395 | RecordPayoutTxid(r, wtx.GetHash().GetHex()); | ||
| 833 | ForgetPayment(r); | 1396 | ForgetPayment(r); |
| 834 | LogPrint("payout", "[payout] paid %s to %s (tx %s)\n", | 1397 | LogPrint("payout", "[payout] paid %s to %s (tx %s)\n", |
| 835 | FormatMoney(r.amount).c_str(), r.address.c_str(), | 1398 | FormatMoney(r.amount).c_str(), r.address.c_str(), |
| @@ -857,6 +1420,12 @@ static void PayoutThreadFn(void*) | |||
| 857 | FormatMoney(r.amount).c_str(), r.address.c_str(), | 1420 | FormatMoney(r.amount).c_str(), r.address.c_str(), |
| 858 | r.retries, r.blockHeight, | 1421 | r.retries, r.blockHeight, |
| 859 | FormatMoney(nSpendable).c_str()); | 1422 | FormatMoney(nSpendable).c_str()); |
| 1423 | // Write the debt off in the ledger as well as dropping the | ||
| 1424 | // in-flight record. Under the survives-until-fully-paid model | ||
| 1425 | // the entry would otherwise never complete, and every later | ||
| 1426 | // flush would explode it again -- a forfeit that silently | ||
| 1427 | // becomes an infinite retry. | ||
| 1428 | ForfeitPayoutRecipient(r); | ||
| 860 | ForgetPayment(r); | 1429 | ForgetPayment(r); |
| 861 | } | 1430 | } |
| 862 | } | 1431 | } |
| @@ -881,6 +1450,7 @@ static StatsSnapshot gStats; | |||
| 881 | static std::vector<PoolWorkerStatView> gWorkerStats; | 1450 | static std::vector<PoolWorkerStatView> gWorkerStats; |
| 882 | static std::vector<PendingPayoutView> gPayoutViews; | 1451 | static std::vector<PendingPayoutView> gPayoutViews; |
| 883 | static std::mutex gStatsMutex; | 1452 | static std::mutex gStatsMutex; |
| 1453 | static int64 gLastPoolStatusWrite = 0; | ||
| 884 | 1454 | ||
| 885 | void GetPoolOperatorStats(int& authorizedMiners, int& blocksFound, uint64& roundShares, double& totalHashRate) | 1455 | void GetPoolOperatorStats(int& authorizedMiners, int& blocksFound, uint64& roundShares, double& totalHashRate) |
| 886 | { | 1456 | { |
| @@ -903,6 +1473,107 @@ void GetPendingPayouts(std::vector<PendingPayoutView>& out) | |||
| 903 | out = gPayoutViews; | 1473 | out = gPayoutViews; |
| 904 | } | 1474 | } |
| 905 | 1475 | ||
| 1476 | static std::string PoolStatusFilePath() | ||
| 1477 | { | ||
| 1478 | if (!strPoolStatusFile.empty()) | ||
| 1479 | return strPoolStatusFile; | ||
| 1480 | return GetAppDir() + "/pool_status.json"; | ||
| 1481 | } | ||
| 1482 | |||
| 1483 | static void WritePoolStatusJsonLocked() | ||
| 1484 | { | ||
| 1485 | nlohmann::json j; | ||
| 1486 | j["schema"] = "bitflash-pool-status-1"; | ||
| 1487 | j["network"] = "bitflash"; | ||
| 1488 | j["updatedAt"] = GetTime(); | ||
| 1489 | |||
| 1490 | j["operator"] = { | ||
| 1491 | {"name", strPoolName}, | ||
| 1492 | {"btfAddress", BtfLocalAddress()}, | ||
| 1493 | {"feePercent", dPoolFeePercent}, | ||
| 1494 | {"dashboardUrl", strPoolDashboardUrl}, | ||
| 1495 | {"roundsFile", "pool_rounds.json"} | ||
| 1496 | }; | ||
| 1497 | |||
| 1498 | j["node"] = { | ||
| 1499 | {"height", nBestHeight}, | ||
| 1500 | {"poolRunning", (bool)gPoolRunning}, | ||
| 1501 | {"mode", nMineMode} | ||
| 1502 | }; | ||
| 1503 | |||
| 1504 | j["pool"] = { | ||
| 1505 | {"authorizedMiners", gStats.authorizedMiners}, | ||
| 1506 | {"blocksFoundSession", gStats.blocksFound}, | ||
| 1507 | {"roundShares", gStats.roundShares}, | ||
| 1508 | {"hashRate", gStats.totalHashRate} | ||
| 1509 | }; | ||
| 1510 | |||
| 1511 | j["workers"] = nlohmann::json::array(); | ||
| 1512 | for (const PoolWorkerStatView& row : gWorkerStats) { | ||
| 1513 | j["workers"].push_back({ | ||
| 1514 | {"address", row.address}, | ||
| 1515 | {"worker", row.worker}, | ||
| 1516 | {"totalShares", row.totalShares}, | ||
| 1517 | {"roundShares", row.roundShares}, | ||
| 1518 | {"lastSeen", row.lastSeen}, | ||
| 1519 | {"hashRate", row.hashRate} | ||
| 1520 | }); | ||
| 1521 | } | ||
| 1522 | |||
| 1523 | j["pendingPayouts"] = nlohmann::json::array(); | ||
| 1524 | for (const PendingPayoutView& row : gPayoutViews) { | ||
| 1525 | j["pendingPayouts"].push_back({ | ||
| 1526 | {"matureAtHeight", row.matureAtHeight}, | ||
| 1527 | {"recipients", row.recipients}, | ||
| 1528 | {"totalAmountSatoshis", row.totalAmount}, | ||
| 1529 | {"totalAmountBtf", FormatMoney(row.totalAmount)} | ||
| 1530 | }); | ||
| 1531 | } | ||
| 1532 | |||
| 1533 | j["recentRounds"] = nlohmann::json::array(); | ||
| 1534 | { | ||
| 1535 | std::lock_guard<std::mutex> lk(gPoolRoundsMutex); | ||
| 1536 | size_t start = gPoolRounds.size() > 10 ? gPoolRounds.size() - 10 : 0; | ||
| 1537 | for (size_t i = start; i < gPoolRounds.size(); i++) { | ||
| 1538 | const PoolRoundProof& r = gPoolRounds[i]; | ||
| 1539 | int unpaid = 0; | ||
| 1540 | for (const auto& kv : r.payouts) | ||
| 1541 | if (r.paidTxids.count(kv.first) == 0) | ||
| 1542 | unpaid++; | ||
| 1543 | j["recentRounds"].push_back({ | ||
| 1544 | {"id", r.id}, | ||
| 1545 | {"blockHeight", r.blockHeight}, | ||
| 1546 | {"blockHash", r.blockHash}, | ||
| 1547 | {"coinbaseTxid", r.coinbaseTxid}, | ||
| 1548 | {"totalShares", r.totalShares}, | ||
| 1549 | {"payouts", (int)r.payouts.size()}, | ||
| 1550 | {"unpaidPayouts", unpaid}, | ||
| 1551 | {"coinbaseValueBtf", FormatMoney(r.coinbaseValue)}, | ||
| 1552 | {"minerRewardBtf", FormatMoney(r.minerReward)} | ||
| 1553 | }); | ||
| 1554 | } | ||
| 1555 | } | ||
| 1556 | |||
| 1557 | std::string path = PoolStatusFilePath(); | ||
| 1558 | std::string tmp = path + ".tmp"; | ||
| 1559 | FILE* f = fopen(tmp.c_str(), "w"); | ||
| 1560 | if (!f) { | ||
| 1561 | LogPrint("pool", "[pool] WARNING: could not write status file %s\n", tmp.c_str()); | ||
| 1562 | return; | ||
| 1563 | } | ||
| 1564 | std::string s = j.dump(2); | ||
| 1565 | fwrite(s.c_str(), 1, s.size(), f); | ||
| 1566 | if (fclose(f) != 0) { | ||
| 1567 | LogPrint("pool", "[pool] WARNING: could not close status file %s\n", tmp.c_str()); | ||
| 1568 | return; | ||
| 1569 | } | ||
| 1570 | #ifdef _WIN32 | ||
| 1571 | remove(path.c_str()); | ||
| 1572 | #endif | ||
| 1573 | if (rename(tmp.c_str(), path.c_str()) != 0) | ||
| 1574 | LogPrint("pool", "[pool] WARNING: could not replace status file %s\n", path.c_str()); | ||
| 1575 | } | ||
| 1576 | |||
| 906 | // --------------------------------------------------------------------------- | 1577 | // --------------------------------------------------------------------------- |
| 907 | // Miner state -- event loop thread only | 1578 | // Miner state -- event loop thread only |
| 908 | // --------------------------------------------------------------------------- | 1579 | // --------------------------------------------------------------------------- |
| @@ -1453,7 +2124,11 @@ static bool HandleLine(Miner* m, const std::string& rawLine, | |||
| 1453 | int foundHeight = gCurrentJob.height; | 2124 | int foundHeight = gCurrentJob.height; |
| 1454 | // b is the block we submitted, untouched by ProcessBlock (which | 2125 | // b is the block we submitted, untouched by ProcessBlock (which |
| 1455 | // took a copy), so its hash is what the payout must be keyed on. | 2126 | // took a copy), so its hash is what the payout must be keyed on. |
| 1456 | QueuePayouts(foundHeight, b.GetHash(), roundShareCount, roundShareTotal); | 2127 | // Passed as a uint256, not hex: the ledger key must stay a real |
| 2128 | // hash. The remaining arguments are upstream's round-proof data. | ||
| 2129 | QueuePayouts(foundHeight, b.GetHash(), b.vtx[0].GetHash().GetHex(), | ||
| 2130 | b.vtx[0].vout[0].nValue, m->address, | ||
| 2131 | roundShareCount, roundShareTotal); | ||
| 1457 | // Reset round shares | 2132 | // Reset round shares |
| 1458 | roundShareCount.clear(); | 2133 | roundShareCount.clear(); |
| 1459 | roundShareTotal = 0; | 2134 | roundShareTotal = 0; |
| @@ -1479,6 +2154,147 @@ static bool HandleLine(Miner* m, const std::string& rawLine, | |||
| 1479 | return true; | 2154 | return true; |
| 1480 | } | 2155 | } |
| 1481 | 2156 | ||
| 2157 | int RunPoolStratumSelfTest() | ||
| 2158 | { | ||
| 2159 | int nFail = 0; | ||
| 2160 | auto check = [&](bool ok, const char* msg) { | ||
| 2161 | fprintf(stdout, " %s %s\n", ok ? "ok" : "FAIL", msg); | ||
| 2162 | if (!ok) nFail++; | ||
| 2163 | }; | ||
| 2164 | |||
| 2165 | fprintf(stdout, "pool-stratum self-test\n"); | ||
| 2166 | |||
| 2167 | StratumJob savedJob = gCurrentJob; | ||
| 2168 | bool savedHaveJob = gHaveJob; | ||
| 2169 | |||
| 2170 | CTransaction coinbase; | ||
| 2171 | coinbase.vin.resize(1); | ||
| 2172 | coinbase.vin[0].prevout.SetNull(); | ||
| 2173 | coinbase.vin[0].scriptSig << 1 << (CBigNum)1; | ||
| 2174 | coinbase.vout.resize(1); | ||
| 2175 | coinbase.vout[0].nValue = 50 * COIN; | ||
| 2176 | |||
| 2177 | CBlock block; | ||
| 2178 | block.vtx.push_back(coinbase); | ||
| 2179 | block.nVersion = 1; | ||
| 2180 | block.nBits = 0x1f0fffff; | ||
| 2181 | block.nTime = 1; | ||
| 2182 | block.nNonce = 1; | ||
| 2183 | block.hashMerkleRoot = block.BuildMerkleTree(); | ||
| 2184 | |||
| 2185 | gCurrentJob.jobId = "selftest"; | ||
| 2186 | gCurrentJob.block = block; | ||
| 2187 | gCurrentJob.target = 0; // accepted share, not a block candidate | ||
| 2188 | gCurrentJob.height = 1; | ||
| 2189 | gHaveJob = true; | ||
| 2190 | |||
| 2191 | Miner miner(INVALID_SOCKET); | ||
| 2192 | miner.authorised = true; | ||
| 2193 | miner.address = "selftest-address"; | ||
| 2194 | miner.worker = "worker"; | ||
| 2195 | miner.shareTarget = ~uint256(0); // every RandomX hash is a valid share | ||
| 2196 | miner.difficulty = 0.001; | ||
| 2197 | miner.vardiffWindowStart = GetTime(); | ||
| 2198 | |||
| 2199 | std::map<std::string, uint64> roundShareCount; | ||
| 2200 | uint64 roundShareTotal = 0; | ||
| 2201 | int blocksFound = 0; | ||
| 2202 | |||
| 2203 | uint256 targetFromNotify; | ||
| 2204 | CBlock minerBlock = BuildMinerBlock(gCurrentJob, miner.extranonce2); | ||
| 2205 | json params = MakeNotifyParams(gCurrentJob, minerBlock, miner.shareTarget, true); | ||
| 2206 | std::vector<unsigned char> targetBytes = FromHex(params[3].get<std::string>()); | ||
| 2207 | if (targetBytes.size() == 32) | ||
| 2208 | memcpy(&targetFromNotify, &targetBytes[0], 32); | ||
| 2209 | check(targetFromNotify == miner.shareTarget, | ||
| 2210 | "mining.notify exposes the same share target the pool validates"); | ||
| 2211 | |||
| 2212 | json submit = { | ||
| 2213 | {"id", 7}, | ||
| 2214 | {"method", "mining.submit"}, | ||
| 2215 | {"params", json::array({miner.address, "selftest", ToHex(&minerBlock.nNonce, 4)})} | ||
| 2216 | }; | ||
| 2217 | HandleLine(&miner, submit.dump(), roundShareCount, roundShareTotal, blocksFound); | ||
| 2218 | |||
| 2219 | check(roundShareTotal == 1, "fake low-difficulty share increments the round total"); | ||
| 2220 | check(roundShareCount[miner.address] == 1, "fake share is credited to the submitting address"); | ||
| 2221 | check(miner.sessionShares == 1 && miner.roundShares == 1 && miner.totalShares == 1, | ||
| 2222 | "miner share counters advance together"); | ||
| 2223 | check(blocksFound == 0, "share below block target is not counted as a found block"); | ||
| 2224 | |||
| 2225 | json stale = { | ||
| 2226 | {"id", 8}, | ||
| 2227 | {"method", "mining.submit"}, | ||
| 2228 | {"params", json::array({miner.address, "stale", ToHex(&minerBlock.nNonce, 4)})} | ||
| 2229 | }; | ||
| 2230 | HandleLine(&miner, stale.dump(), roundShareCount, roundShareTotal, blocksFound); | ||
| 2231 | check(roundShareTotal == 1, "stale submit does not increment shares"); | ||
| 2232 | |||
| 2233 | std::vector<PoolRoundProof> savedRounds = gPoolRounds; | ||
| 2234 | std::string savedRoundsFile = strPoolRoundsFile; | ||
| 2235 | std::string proofPath = "pool_rounds_selftest.json"; | ||
| 2236 | remove(proofPath.c_str()); | ||
| 2237 | remove((proofPath + ".tmp").c_str()); | ||
| 2238 | gPoolRounds.clear(); | ||
| 2239 | strPoolRoundsFile = proofPath; | ||
| 2240 | |||
| 2241 | PoolRoundProof proof; | ||
| 2242 | proof.id = "selftest-round"; | ||
| 2243 | proof.blockHeight = 42; | ||
| 2244 | proof.matureAtHeight = 42 + COINBASE_MATURITY; | ||
| 2245 | proof.createdAt = GetTime(); | ||
| 2246 | proof.blockHash = block.GetHash().GetHex(); | ||
| 2247 | proof.coinbaseTxid = coinbase.GetHash().GetHex(); | ||
| 2248 | proof.foundBy = miner.address; | ||
| 2249 | proof.feePercent = 1.25; | ||
| 2250 | proof.totalShares = 3; | ||
| 2251 | proof.coinbaseValue = 50 * COIN; | ||
| 2252 | proof.subsidy = 50 * COIN; | ||
| 2253 | proof.minerReward = 49 * COIN; | ||
| 2254 | proof.operatorReward = 1 * COIN; | ||
| 2255 | proof.shares[miner.address] = 3; | ||
| 2256 | proof.payouts[miner.address] = 49 * COIN; | ||
| 2257 | RecordPoolRound(proof); | ||
| 2258 | MarkPoolRoundPaid(proof.id, miner.address, "selftest-txid"); | ||
| 2259 | |||
| 2260 | bool proofOk = false; | ||
| 2261 | FILE* pf = fopen(proofPath.c_str(), "r"); | ||
| 2262 | if (pf) { | ||
| 2263 | fseek(pf, 0, SEEK_END); | ||
| 2264 | long sz = ftell(pf); | ||
| 2265 | fseek(pf, 0, SEEK_SET); | ||
| 2266 | std::string s(sz, '\0'); | ||
| 2267 | fread(&s[0], 1, sz, pf); | ||
| 2268 | fclose(pf); | ||
| 2269 | try { | ||
| 2270 | json root = json::parse(s); | ||
| 2271 | proofOk = | ||
| 2272 | root.value("schema", "") == "bitflash-pool-rounds-1" && | ||
| 2273 | root["rounds"].is_array() && | ||
| 2274 | root["rounds"].size() == 1 && | ||
| 2275 | root["rounds"][0].value("id", "") == proof.id && | ||
| 2276 | root["rounds"][0]["shares"].value(miner.address, (uint64)0) == 3 && | ||
| 2277 | root["rounds"][0]["payouts"][miner.address].value("txid", "") == "selftest-txid"; | ||
| 2278 | } catch (...) { | ||
| 2279 | proofOk = false; | ||
| 2280 | } | ||
| 2281 | } | ||
| 2282 | check(proofOk, "pool round proof ledger is written and paid txid is recorded"); | ||
| 2283 | |||
| 2284 | remove(proofPath.c_str()); | ||
| 2285 | remove((proofPath + ".tmp").c_str()); | ||
| 2286 | strPoolRoundsFile = savedRoundsFile; | ||
| 2287 | gPoolRounds = savedRounds; | ||
| 2288 | |||
| 2289 | gCurrentJob = savedJob; | ||
| 2290 | gHaveJob = savedHaveJob; | ||
| 2291 | |||
| 2292 | fprintf(stdout, "%s (%d failure%s)\n", nFail == 0 ? "ALL TESTS PASSED" : "TESTS FAILED", | ||
| 2293 | nFail, nFail == 1 ? "" : "s"); | ||
| 2294 | fflush(stdout); | ||
| 2295 | return nFail == 0 ? 0 : 1; | ||
| 2296 | } | ||
| 2297 | |||
| 1482 | // --------------------------------------------------------------------------- | 2298 | // --------------------------------------------------------------------------- |
| 1483 | // Stats refresh -- called every 2s from event loop | 2299 | // Stats refresh -- called every 2s from event loop |
| 1484 | // --------------------------------------------------------------------------- | 2300 | // --------------------------------------------------------------------------- |
| @@ -1512,16 +2328,27 @@ static void RefreshStats(int blocksFoundThisSession, uint64 roundShareTotal) | |||
| 1512 | : a.lastSeen > b.lastSeen; | 2328 | : a.lastSeen > b.lastSeen; |
| 1513 | }); | 2329 | }); |
| 1514 | 2330 | ||
| 1515 | // gPendingPayouts is event-loop-only data (no lock needed here since | ||
| 1516 | // RefreshStats is always called from the event loop thread). | ||
| 1517 | gPayoutViews.clear(); | 2331 | gPayoutViews.clear(); |
| 2332 | std::lock_guard<std::mutex> ledger(gPayoutLedgerMutex); | ||
| 1518 | for (const PendingPayout& pp : gPendingPayouts) { | 2333 | for (const PendingPayout& pp : gPendingPayouts) { |
| 1519 | PendingPayoutView pv; | 2334 | PendingPayoutView pv; |
| 1520 | pv.matureAtHeight = pp.matureAtHeight; | 2335 | pv.matureAtHeight = pp.matureAtHeight; |
| 1521 | pv.recipients = (int)pp.amounts.size(); | 2336 | pv.recipients = 0; |
| 1522 | pv.totalAmount = 0; | 2337 | pv.totalAmount = 0; |
| 1523 | for (auto& kv : pp.amounts) pv.totalAmount += kv.second; | 2338 | for (auto& kv : pp.amounts) { |
| 1524 | gPayoutViews.push_back(pv); | 2339 | if (pp.paidTxids.count(kv.first)) |
| 2340 | continue; | ||
| 2341 | pv.recipients++; | ||
| 2342 | pv.totalAmount += kv.second; | ||
| 2343 | } | ||
| 2344 | if (pv.recipients > 0) | ||
| 2345 | gPayoutViews.push_back(pv); | ||
| 2346 | } | ||
| 2347 | |||
| 2348 | int64 now = GetTime(); | ||
| 2349 | if (now - gLastPoolStatusWrite >= 10) { | ||
| 2350 | WritePoolStatusJsonLocked(); | ||
| 2351 | gLastPoolStatusWrite = now; | ||
| 1525 | } | 2352 | } |
| 1526 | } | 2353 | } |
| 1527 | 2354 | ||
| @@ -1901,10 +2728,11 @@ void ThreadRPCServer(void*) | |||
| 1901 | } | 2728 | } |
| 1902 | 2729 | ||
| 1903 | LoadPendingPayouts(); | 2730 | LoadPendingPayouts(); |
| 2731 | LoadPoolRounds(); | ||
| 1904 | LoadPaying(); | 2732 | LoadPaying(); |
| 1905 | // Must run before the payout thread starts, so nothing is resent that the | 2733 | // Must run before the payout thread starts, so nothing is resent that the |
| 1906 | // wallet already shows as sent, and nothing is owed twice because we died | 2734 | // ledger already shows as sent, and every restored in-flight record is |
| 1907 | // between recording the payments and rewriting the ledger. | 2735 | // armed for its one wallet probe before it is allowed to send. |
| 1908 | ReconcilePayouts(); | 2736 | ReconcilePayouts(); |
| 1909 | 2737 | ||
| 1910 | _beginthread(AcceptLoopFn, 0, NULL); | 2738 | _beginthread(AcceptLoopFn, 0, NULL); |
diff --git a/src/selftest.cpp b/src/selftest.cpp index d12c96b..eed6779 100644 --- a/src/selftest.cpp +++ b/src/selftest.cpp | |||
| @@ -7,9 +7,13 @@ | |||
| 7 | 7 | ||
| 8 | #include "headers_core.h" | 8 | #include "headers_core.h" |
| 9 | #include "bip32.h" | 9 | #include "bip32.h" |
| 10 | #include "proxy.h" | ||
| 10 | #include "selftest.h" | 11 | #include "selftest.h" |
| 12 | #include "sockcount.h" | ||
| 11 | #include "walletcmd.h" | 13 | #include "walletcmd.h" |
| 12 | 14 | ||
| 15 | extern int RunPoolStratumSelfTest(); | ||
| 16 | |||
| 13 | // Test results go to the terminal, not to debug.log. | 17 | // Test results go to the terminal, not to debug.log. |
| 14 | // | 18 | // |
| 15 | // printf in this tree is OutputDebugStringF, which writes into the data | 19 | // printf in this tree is OutputDebugStringF, which writes into the data |
| @@ -348,17 +352,17 @@ static int RunWalletHDSelfTest() | |||
| 348 | nFail += Check(HaveHDSeed(), "the wallet reports having a seed") ? 0 : 1; | 352 | nFail += Check(HaveHDSeed(), "the wallet reports having a seed") ? 0 : 1; |
| 349 | nFail += Check(nHDKeySchema == HD_SCHEMA_BIP44, | 353 | nFail += Check(nHDKeySchema == HD_SCHEMA_BIP44, |
| 350 | "a new recovery phrase records the BIP44 HD schema") ? 0 : 1; | 354 | "a new recovery phrase records the BIP44 HD schema") ? 0 : 1; |
| 351 | nFail += Check(nHDCoinType == HD_BIP44_COIN_TYPE_BITFLASH_PROVISIONAL, | 355 | nFail += Check(nHDCoinType == HD_BIP44_COIN_TYPE_BITFLASH, |
| 352 | "a new recovery phrase records the provisional BIP44 coin type") ? 0 : 1; | 356 | "a new recovery phrase records the SLIP-0044 BIP44 coin type") ? 0 : 1; |
| 353 | nFail += Check(nHDReceiveNext == 1 && nHDChangeNext == 0, | 357 | nFail += Check(nHDReceiveNext == 1 && nHDChangeNext == 0, |
| 354 | "BIP44 receive/change counters reserve the default receive key") ? 0 : 1; | 358 | "BIP44 receive/change counters reserve the default receive key") ? 0 : 1; |
| 355 | std::vector<unsigned int> vBIP44Path = HDBIP44Path(HD_BIP44_COIN_TYPE_BITFLASH_PROVISIONAL, | 359 | std::vector<unsigned int> vBIP44Path = HDBIP44Path(HD_BIP44_COIN_TYPE_BITFLASH, |
| 356 | HD_BIP44_ACCOUNT, | 360 | HD_BIP44_ACCOUNT, |
| 357 | HD_BIP44_CHAIN_RECEIVE, | 361 | HD_BIP44_CHAIN_RECEIVE, |
| 358 | 0); | 362 | 0); |
| 359 | nFail += Check(vBIP44Path.size() == 5 && | 363 | nFail += Check(vBIP44Path.size() == 5 && |
| 360 | vBIP44Path[0] == (HD_BIP44_PURPOSE | bitflash::BIP32_HARDENED) && | 364 | vBIP44Path[0] == (HD_BIP44_PURPOSE | bitflash::BIP32_HARDENED) && |
| 361 | vBIP44Path[1] == (HD_BIP44_COIN_TYPE_BITFLASH_PROVISIONAL | bitflash::BIP32_HARDENED) && | 365 | vBIP44Path[1] == (HD_BIP44_COIN_TYPE_BITFLASH | bitflash::BIP32_HARDENED) && |
| 362 | vBIP44Path[2] == (HD_BIP44_ACCOUNT | bitflash::BIP32_HARDENED) && | 366 | vBIP44Path[2] == (HD_BIP44_ACCOUNT | bitflash::BIP32_HARDENED) && |
| 363 | vBIP44Path[3] == HD_BIP44_CHAIN_RECEIVE && | 367 | vBIP44Path[3] == HD_BIP44_CHAIN_RECEIVE && |
| 364 | vBIP44Path[4] == 0, | 368 | vBIP44Path[4] == 0, |
| @@ -380,7 +384,7 @@ static int RunWalletHDSelfTest() | |||
| 380 | hdParent.chainCode = vchHDChainCode; | 384 | hdParent.chainCode = vchHDChainCode; |
| 381 | bitflash::BIP32PrivateNode hdBIP44ReceiveChild; | 385 | bitflash::BIP32PrivateNode hdBIP44ReceiveChild; |
| 382 | if (!bitflash::BIP32DerivePath(hdParent, | 386 | if (!bitflash::BIP32DerivePath(hdParent, |
| 383 | HDBIP44Path(HD_BIP44_COIN_TYPE_BITFLASH_PROVISIONAL, | 387 | HDBIP44Path(HD_BIP44_COIN_TYPE_BITFLASH, |
| 384 | HD_BIP44_ACCOUNT, | 388 | HD_BIP44_ACCOUNT, |
| 385 | HD_BIP44_CHAIN_RECEIVE, | 389 | HD_BIP44_CHAIN_RECEIVE, |
| 386 | 0), | 390 | 0), |
| @@ -568,7 +572,7 @@ static int RunWalletHDSelfTest() | |||
| 568 | nFail += Check(audit.fHaveSeed, "the recovery audit reports the phrase") ? 0 : 1; | 572 | nFail += Check(audit.fHaveSeed, "the recovery audit reports the phrase") ? 0 : 1; |
| 569 | nFail += Check(audit.nSchema == HD_SCHEMA_BIP44, | 573 | nFail += Check(audit.nSchema == HD_SCHEMA_BIP44, |
| 570 | "the recovery audit reports the derivation schema") ? 0 : 1; | 574 | "the recovery audit reports the derivation schema") ? 0 : 1; |
| 571 | nFail += Check(audit.nCoinType == HD_BIP44_COIN_TYPE_BITFLASH_PROVISIONAL, | 575 | nFail += Check(audit.nCoinType == HD_BIP44_COIN_TYPE_BITFLASH, |
| 572 | "the recovery audit reports the BIP44 coin type") ? 0 : 1; | 576 | "the recovery audit reports the BIP44 coin type") ? 0 : 1; |
| 573 | nFail += Check(audit.nReceiveNext == nHDReceiveNext && audit.nChangeNext == 1, | 577 | nFail += Check(audit.nReceiveNext == nHDReceiveNext && audit.nChangeNext == 1, |
| 574 | "the recovery audit reports receive/change counters") ? 0 : 1; | 578 | "the recovery audit reports receive/change counters") ? 0 : 1; |
| @@ -1241,6 +1245,314 @@ static int RunConsensusLimitsSelfTest() | |||
| 1241 | return nFail == 0 ? 0 : 1; | 1245 | return nFail == 0 ? 0 : 1; |
| 1242 | } | 1246 | } |
| 1243 | 1247 | ||
| 1248 | static int RunParseMoneySelfTest() | ||
| 1249 | { | ||
| 1250 | printf("parse-money self-test\n"); | ||
| 1251 | int nFail = 0; | ||
| 1252 | struct Case { const char* in; bool ok; int64 val; }; | ||
| 1253 | Case cases[] = { | ||
| 1254 | {"0.5", true, 50000000LL}, | ||
| 1255 | {"1", true, 100000000LL}, | ||
| 1256 | {"1.23456789", true, 123456789LL}, | ||
| 1257 | {"0.00000001", true, 1LL}, | ||
| 1258 | {"10.00000000", true, 1000000000LL}, | ||
| 1259 | {"1,000", true, 100000000000LL}, | ||
| 1260 | {"1.5 ", true, 150000000LL}, | ||
| 1261 | {"0.123456789", false, 0LL}, | ||
| 1262 | {"1.2.3", false, 0LL}, | ||
| 1263 | {"abc", false, 0LL}, | ||
| 1264 | {"-1", false, 0LL}, | ||
| 1265 | {"1e5", false, 0LL}, | ||
| 1266 | }; | ||
| 1267 | for (size_t i = 0; i < sizeof(cases) / sizeof(cases[0]); i++) | ||
| 1268 | { | ||
| 1269 | int64 v = -1; | ||
| 1270 | bool r = ParseMoney(cases[i].in, v); | ||
| 1271 | bool pass = (r == cases[i].ok) && (!r || v == cases[i].val); | ||
| 1272 | char desc[160]; | ||
| 1273 | snprintf(desc, sizeof(desc), "ParseMoney(\"%s\") %s and value matches", | ||
| 1274 | cases[i].in, cases[i].ok ? "accepts" : "rejects"); | ||
| 1275 | nFail += Check(pass, desc) ? 0 : 1; | ||
| 1276 | } | ||
| 1277 | printf("%s (%d failure%s)\n", nFail == 0 ? "ALL TESTS PASSED" : "TESTS FAILED", | ||
| 1278 | nFail, nFail == 1 ? "" : "s"); | ||
| 1279 | fflush(stdout); | ||
| 1280 | return nFail == 0 ? 0 : 1; | ||
| 1281 | } | ||
| 1282 | |||
| 1283 | static bool SelfTestReadN(SOCKET s, void* buf, int n) | ||
| 1284 | { | ||
| 1285 | char* p = (char*)buf; | ||
| 1286 | int off = 0; | ||
| 1287 | while (off < n) | ||
| 1288 | { | ||
| 1289 | int r = recv(s, p + off, n - off, 0); | ||
| 1290 | if (r <= 0) | ||
| 1291 | return false; | ||
| 1292 | off += r; | ||
| 1293 | } | ||
| 1294 | return true; | ||
| 1295 | } | ||
| 1296 | |||
| 1297 | static bool SelfTestWriteN(SOCKET s, const void* buf, int n) | ||
| 1298 | { | ||
| 1299 | const char* p = (const char*)buf; | ||
| 1300 | int off = 0; | ||
| 1301 | while (off < n) | ||
| 1302 | { | ||
| 1303 | int r = send(s, p + off, n - off, 0); | ||
| 1304 | if (r <= 0) | ||
| 1305 | return false; | ||
| 1306 | off += r; | ||
| 1307 | } | ||
| 1308 | return true; | ||
| 1309 | } | ||
| 1310 | |||
| 1311 | static void SelfTestSetSocketTimeout(SOCKET s, int nTimeoutSecs) | ||
| 1312 | { | ||
| 1313 | #ifdef _WIN32 | ||
| 1314 | DWORD tv = (DWORD)nTimeoutSecs * 1000; | ||
| 1315 | setsockopt(s, SOL_SOCKET, SO_RCVTIMEO, (const char*)&tv, sizeof(tv)); | ||
| 1316 | setsockopt(s, SOL_SOCKET, SO_SNDTIMEO, (const char*)&tv, sizeof(tv)); | ||
| 1317 | #else | ||
| 1318 | struct timeval tv; | ||
| 1319 | tv.tv_sec = nTimeoutSecs; | ||
| 1320 | tv.tv_usec = 0; | ||
| 1321 | setsockopt(s, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv)); | ||
| 1322 | setsockopt(s, SOL_SOCKET, SO_SNDTIMEO, &tv, sizeof(tv)); | ||
| 1323 | #endif | ||
| 1324 | } | ||
| 1325 | |||
| 1326 | #ifdef _WIN32 | ||
| 1327 | struct SelfTestWinsock | ||
| 1328 | { | ||
| 1329 | bool fStarted; | ||
| 1330 | SelfTestWinsock() : fStarted(false) | ||
| 1331 | { | ||
| 1332 | WSADATA wsadata; | ||
| 1333 | fStarted = (WSAStartup(MAKEWORD(2,2), &wsadata) == 0); | ||
| 1334 | } | ||
| 1335 | ~SelfTestWinsock() | ||
| 1336 | { | ||
| 1337 | if (fStarted) | ||
| 1338 | WSACleanup(); | ||
| 1339 | } | ||
| 1340 | }; | ||
| 1341 | #endif | ||
| 1342 | |||
| 1343 | static bool RunSocks5HandshakeProbe(std::string& errOut) | ||
| 1344 | { | ||
| 1345 | errOut.clear(); | ||
| 1346 | #ifdef _WIN32 | ||
| 1347 | SelfTestWinsock winsock; | ||
| 1348 | if (!winsock.fStarted) | ||
| 1349 | { | ||
| 1350 | errOut = "could not start Winsock"; | ||
| 1351 | return false; | ||
| 1352 | } | ||
| 1353 | #endif | ||
| 1354 | |||
| 1355 | SOCKET listener = socket(AF_INET, SOCK_STREAM, IPPROTO_TCP); | ||
| 1356 | if (listener == INVALID_SOCKET) | ||
| 1357 | { | ||
| 1358 | errOut = "could not create listener"; | ||
| 1359 | return false; | ||
| 1360 | } | ||
| 1361 | SelfTestSetSocketTimeout(listener, 5); | ||
| 1362 | |||
| 1363 | struct sockaddr_in addr; | ||
| 1364 | memset(&addr, 0, sizeof(addr)); | ||
| 1365 | addr.sin_family = AF_INET; | ||
| 1366 | addr.sin_addr.s_addr = htonl(INADDR_LOOPBACK); | ||
| 1367 | addr.sin_port = 0; | ||
| 1368 | if (bind(listener, (struct sockaddr*)&addr, sizeof(addr)) != 0 || | ||
| 1369 | listen(listener, 1) != 0) | ||
| 1370 | { | ||
| 1371 | closesocket(listener); | ||
| 1372 | errOut = "could not bind/listen on loopback"; | ||
| 1373 | return false; | ||
| 1374 | } | ||
| 1375 | |||
| 1376 | #ifdef _WIN32 | ||
| 1377 | int addrLen = sizeof(addr); | ||
| 1378 | #else | ||
| 1379 | socklen_t addrLen = sizeof(addr); | ||
| 1380 | #endif | ||
| 1381 | if (getsockname(listener, (struct sockaddr*)&addr, &addrLen) != 0) | ||
| 1382 | { | ||
| 1383 | closesocket(listener); | ||
| 1384 | errOut = "could not read listener port"; | ||
| 1385 | return false; | ||
| 1386 | } | ||
| 1387 | unsigned short proxyPort = ntohs(addr.sin_port); | ||
| 1388 | |||
| 1389 | bool fServerOk = false; | ||
| 1390 | std::string serverErr; | ||
| 1391 | std::string requestedHost; | ||
| 1392 | unsigned short requestedPort = 0; | ||
| 1393 | std::thread server([&]() { | ||
| 1394 | SOCKET s = accept(listener, NULL, NULL); | ||
| 1395 | if (s == INVALID_SOCKET) | ||
| 1396 | { | ||
| 1397 | serverErr = "accept failed"; | ||
| 1398 | return; | ||
| 1399 | } | ||
| 1400 | SelfTestSetSocketTimeout(s, 5); | ||
| 1401 | |||
| 1402 | unsigned char hello[3] = {0,0,0}; | ||
| 1403 | if (!SelfTestReadN(s, hello, sizeof(hello)) || | ||
| 1404 | hello[0] != 0x05 || hello[1] != 0x01 || hello[2] != 0x00) | ||
| 1405 | { | ||
| 1406 | serverErr = "bad SOCKS5 greeting"; | ||
| 1407 | closesocket(s); | ||
| 1408 | return; | ||
| 1409 | } | ||
| 1410 | unsigned char choice[2] = {0x05, 0x00}; | ||
| 1411 | if (!SelfTestWriteN(s, choice, sizeof(choice))) | ||
| 1412 | { | ||
| 1413 | serverErr = "could not write method choice"; | ||
| 1414 | closesocket(s); | ||
| 1415 | return; | ||
| 1416 | } | ||
| 1417 | |||
| 1418 | unsigned char hdr[5] = {0,0,0,0,0}; | ||
| 1419 | if (!SelfTestReadN(s, hdr, sizeof(hdr)) || | ||
| 1420 | hdr[0] != 0x05 || hdr[1] != 0x01 || hdr[2] != 0x00 || hdr[3] != 0x03) | ||
| 1421 | { | ||
| 1422 | serverErr = "CONNECT did not use domain-name address type"; | ||
| 1423 | closesocket(s); | ||
| 1424 | return; | ||
| 1425 | } | ||
| 1426 | |||
| 1427 | unsigned char len = hdr[4]; | ||
| 1428 | std::vector<unsigned char> rest((size_t)len + 2); | ||
| 1429 | if (!SelfTestReadN(s, &rest[0], (int)rest.size())) | ||
| 1430 | { | ||
| 1431 | serverErr = "could not read CONNECT target"; | ||
| 1432 | closesocket(s); | ||
| 1433 | return; | ||
| 1434 | } | ||
| 1435 | requestedHost.assign((const char*)&rest[0], (size_t)len); | ||
| 1436 | requestedPort = ((unsigned short)rest[len] << 8) | rest[len + 1]; | ||
| 1437 | |||
| 1438 | unsigned char reply[10] = {0x05, 0x00, 0x00, 0x01, 0,0,0,0, 0,0}; | ||
| 1439 | if (!SelfTestWriteN(s, reply, sizeof(reply))) | ||
| 1440 | { | ||
| 1441 | serverErr = "could not write CONNECT reply"; | ||
| 1442 | closesocket(s); | ||
| 1443 | return; | ||
| 1444 | } | ||
| 1445 | fServerOk = true; | ||
| 1446 | closesocket(s); | ||
| 1447 | }); | ||
| 1448 | |||
| 1449 | std::string err; | ||
| 1450 | BtfClearSocks5Proxy(); | ||
| 1451 | bool fClientOk = BtfSetSocks5Proxy(strprintf("127.0.0.1:%u", (unsigned)proxyPort), err); | ||
| 1452 | SOCKET client = INVALID_SOCKET; | ||
| 1453 | if (fClientOk) | ||
| 1454 | client = BtfConnectSocket("relay.example", 443, SOCK_NOSTR, 5); | ||
| 1455 | if (client != INVALID_SOCKET) | ||
| 1456 | BtfCloseSocket(client); | ||
| 1457 | BtfClearSocks5Proxy(); | ||
| 1458 | closesocket(listener); | ||
| 1459 | server.join(); | ||
| 1460 | |||
| 1461 | if (!fClientOk) | ||
| 1462 | errOut = strprintf("could not enable proxy: %s", err.c_str()); | ||
| 1463 | else if (client == INVALID_SOCKET) | ||
| 1464 | errOut = "client SOCKS5 connect failed"; | ||
| 1465 | else if (!fServerOk) | ||
| 1466 | errOut = serverErr.empty() ? "mock server failed" : serverErr; | ||
| 1467 | else if (requestedHost != "relay.example" || requestedPort != 443) | ||
| 1468 | errOut = strprintf("unexpected CONNECT target %s:%u", requestedHost.c_str(), (unsigned)requestedPort); | ||
| 1469 | else | ||
| 1470 | return true; | ||
| 1471 | return false; | ||
| 1472 | } | ||
| 1473 | |||
| 1474 | static int RunSocks5ProxySelfTest() | ||
| 1475 | { | ||
| 1476 | printf("socks5-proxy self-test\n"); | ||
| 1477 | int nFail = 0; | ||
| 1478 | std::string host, err; | ||
| 1479 | unsigned short port = 0; | ||
| 1480 | |||
| 1481 | nFail += Check(BtfParseSocks5Proxy("127.0.0.1:9050", host, port, err) && | ||
| 1482 | host == "127.0.0.1" && port == 9050, | ||
| 1483 | "parses local Tor proxy endpoint") ? 0 : 1; | ||
| 1484 | nFail += Check(BtfParseSocks5Proxy("localhost:9050", host, port, err) && | ||
| 1485 | host == "localhost" && port == 9050, | ||
| 1486 | "parses hostname proxy endpoint") ? 0 : 1; | ||
| 1487 | nFail += Check(BtfParseSocks5Proxy("[::1]:9050", host, port, err) && | ||
| 1488 | host == "::1" && port == 9050, | ||
| 1489 | "parses bracketed IPv6 proxy endpoint") ? 0 : 1; | ||
| 1490 | nFail += Check(!BtfParseSocks5Proxy("::1:9050", host, port, err), | ||
| 1491 | "rejects unbracketed IPv6 proxy endpoint") ? 0 : 1; | ||
| 1492 | nFail += Check(!BtfParseSocks5Proxy("localhost", host, port, err), | ||
| 1493 | "rejects missing port") ? 0 : 1; | ||
| 1494 | nFail += Check(!BtfParseSocks5Proxy("localhost:0", host, port, err), | ||
| 1495 | "rejects zero port") ? 0 : 1; | ||
| 1496 | nFail += Check(!BtfParseSocks5Proxy("localhost:70000", host, port, err), | ||
| 1497 | "rejects out-of-range port") ? 0 : 1; | ||
| 1498 | nFail += Check(!BtfParseSocks5Proxy("localhost:999999999999999999999999999999", host, port, err), | ||
| 1499 | "rejects overflowing port") ? 0 : 1; | ||
| 1500 | nFail += Check(!BtfParseSocks5Proxy("user:pass@localhost:9050", host, port, err), | ||
| 1501 | "rejects unsupported authenticated proxy syntax") ? 0 : 1; | ||
| 1502 | nFail += Check(BtfIsTorOnionHost("abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcd.onion") && | ||
| 1503 | BtfIsTorOnionHost("Relay.Example.ONION.") && | ||
| 1504 | !BtfIsTorOnionHost("relay.example") && | ||
| 1505 | !BtfIsTorOnionHost(".onion"), | ||
| 1506 | "detects .onion hosts without accepting lookalikes") ? 0 : 1; | ||
| 1507 | |||
| 1508 | BtfClearSocks5Proxy(); | ||
| 1509 | nFail += Check(!BtfSocks5ProxyEnabled(), "starts disabled after clear") ? 0 : 1; | ||
| 1510 | nFail += Check(BtfSetSocks5Proxy("127.0.0.1:9050", err) && | ||
| 1511 | BtfSocks5ProxyEnabled() && | ||
| 1512 | BtfSocks5ProxyName() == "127.0.0.1:9050", | ||
| 1513 | "enables parsed SOCKS5 proxy") ? 0 : 1; | ||
| 1514 | BtfClearSocks5Proxy(); | ||
| 1515 | nFail += Check(!BtfSocks5ProxyEnabled(), "clear disables proxy") ? 0 : 1; | ||
| 1516 | |||
| 1517 | nFail += Check(BtfSetSocks5Proxy("[::1]:9050", err) && | ||
| 1518 | BtfSocks5ProxyEnabled() && | ||
| 1519 | BtfSocks5ProxyName() == "[::1]:9050", | ||
| 1520 | "enables bracketed IPv6 proxy endpoint") ? 0 : 1; | ||
| 1521 | BtfClearSocks5Proxy(); | ||
| 1522 | |||
| 1523 | nFail += Check(BtfEnableTorProxy("", err) && | ||
| 1524 | BtfSocks5ProxyEnabled() && | ||
| 1525 | BtfTorProxyEnabled() && | ||
| 1526 | BtfSocks5ProxyName() == "127.0.0.1:9050", | ||
| 1527 | "enables Tor mode on the default local SOCKS5 endpoint") ? 0 : 1; | ||
| 1528 | BtfClearSocks5Proxy(); | ||
| 1529 | |||
| 1530 | nFail += Check(BtfEnableTorProxy("[::1]:9050", err) && | ||
| 1531 | BtfSocks5ProxyEnabled() && | ||
| 1532 | BtfTorProxyEnabled() && | ||
| 1533 | BtfSocks5ProxyName() == "[::1]:9050", | ||
| 1534 | "enables Tor mode on a custom IPv6 endpoint") ? 0 : 1; | ||
| 1535 | BtfClearSocks5Proxy(); | ||
| 1536 | |||
| 1537 | nFail += Check(BtfSetSocks5Proxy("127.0.0.1:9051", err) && | ||
| 1538 | BtfSocks5ProxyEnabled() && | ||
| 1539 | !BtfTorProxyEnabled(), | ||
| 1540 | "plain SOCKS5 mode is distinct from Tor mode") ? 0 : 1; | ||
| 1541 | BtfClearSocks5Proxy(); | ||
| 1542 | |||
| 1543 | std::string handshakeErr; | ||
| 1544 | bool fHandshakeOk = RunSocks5HandshakeProbe(handshakeErr); | ||
| 1545 | nFail += Check(fHandshakeOk, handshakeErr.empty() ? | ||
| 1546 | "completes SOCKS5 domain-name CONNECT handshake" : | ||
| 1547 | strprintf("completes SOCKS5 domain-name CONNECT handshake (%s)", | ||
| 1548 | handshakeErr.c_str()).c_str()) ? 0 : 1; | ||
| 1549 | |||
| 1550 | printf("%s (%d failure%s)\n", nFail == 0 ? "ALL TESTS PASSED" : "TESTS FAILED", | ||
| 1551 | nFail, nFail == 1 ? "" : "s"); | ||
| 1552 | fflush(stdout); | ||
| 1553 | return nFail == 0 ? 0 : 1; | ||
| 1554 | } | ||
| 1555 | |||
| 1244 | int RunSelfTest(const std::string& name) | 1556 | int RunSelfTest(const std::string& name) |
| 1245 | { | 1557 | { |
| 1246 | AttachTerminal(); | 1558 | AttachTerminal(); |
| @@ -1259,8 +1571,14 @@ int RunSelfTest(const std::string& name) | |||
| 1259 | return RunNetMessageSelfTest(); | 1571 | return RunNetMessageSelfTest(); |
| 1260 | if (name == "consensus-limits") | 1572 | if (name == "consensus-limits") |
| 1261 | return RunConsensusLimitsSelfTest(); | 1573 | return RunConsensusLimitsSelfTest(); |
| 1574 | if (name == "pool-stratum") | ||
| 1575 | return RunPoolStratumSelfTest(); | ||
| 1576 | if (name == "parse-money") | ||
| 1577 | return RunParseMoneySelfTest(); | ||
| 1578 | if (name == "socks5-proxy") | ||
| 1579 | return RunSocks5ProxySelfTest(); | ||
| 1262 | 1580 | ||
| 1263 | printf("Unknown self-test '%s'\n", name.c_str()); | 1581 | printf("Unknown self-test '%s'\n", name.c_str()); |
| 1264 | printf("Known self-tests: wallet-keypool, wallet-hd, wallet-format, wallet-crypto, wallet-encrypt, net-message, consensus-limits\n"); | 1582 | printf("Known self-tests: wallet-keypool, wallet-hd, wallet-format, wallet-crypto, wallet-encrypt, net-message, consensus-limits, pool-stratum, parse-money, socks5-proxy\n"); |
| 1265 | return 1; | 1583 | return 1; |
| 1266 | } | 1584 | } |
diff --git a/src/sockcount.h b/src/sockcount.h index 96b6efb..5147f55 100644 --- a/src/sockcount.h +++ b/src/sockcount.h | |||
| @@ -63,6 +63,7 @@ enum | |||
| 63 | SOCK_PAIR_APP, // loopback pair: the app end handed to CNode | 63 | SOCK_PAIR_APP, // loopback pair: the app end handed to CNode |
| 64 | SOCK_PAIR_PUMP, // loopback pair: the pump end | 64 | SOCK_PAIR_PUMP, // loopback pair: the pump end |
| 65 | SOCK_NOSTR, // outbound to a Nostr relay | 65 | SOCK_NOSTR, // outbound to a Nostr relay |
| 66 | SOCK_STRATUM_BRIDGE, // local Stratum bridge listener/miner sockets | ||
| 66 | SOCK_SITES | 67 | SOCK_SITES |
| 67 | }; | 68 | }; |
| 68 | 69 | ||
diff --git a/src/util.cpp b/src/util.cpp index d52ca88..0cd0133 100644 --- a/src/util.cpp +++ b/src/util.cpp | |||
| @@ -337,7 +337,7 @@ string FormatMoney(int64 n, bool fPlus) | |||
| 337 | bool ParseMoney(const char* pszIn, int64& nRet) | 337 | bool ParseMoney(const char* pszIn, int64& nRet) |
| 338 | { | 338 | { |
| 339 | string strWhole; | 339 | string strWhole; |
| 340 | int64 nCents = 0; | 340 | int64 nUnits = 0; // fractional part, scaled to COIN (8 decimals) |
| 341 | const char* p = pszIn; | 341 | const char* p = pszIn; |
| 342 | while (isspace(*p)) | 342 | while (isspace(*p)) |
| 343 | p++; | 343 | p++; |
| @@ -348,12 +348,16 @@ bool ParseMoney(const char* pszIn, int64& nRet) | |||
| 348 | if (*p == '.') | 348 | if (*p == '.') |
| 349 | { | 349 | { |
| 350 | p++; | 350 | p++; |
| 351 | if (!isdigit(p[0]) || !isdigit(p[1])) | 351 | int64 nMult = COIN / 10; // weight of the first decimal digit |
| 352 | while (isdigit(*p) && nMult > 0) | ||
| 353 | { | ||
| 354 | nUnits += nMult * (*p - '0'); | ||
| 355 | nMult /= 10; | ||
| 356 | p++; | ||
| 357 | } | ||
| 358 | // reject more than 8 fractional digits rather than silently truncate | ||
| 359 | if (isdigit(*p)) | ||
| 352 | return false; | 360 | return false; |
| 353 | nCents = atoi64(p); | ||
| 354 | if (nCents < 0 || nCents > 99) | ||
| 355 | return false; | ||
| 356 | p += 2; | ||
| 357 | break; | 361 | break; |
| 358 | } | 362 | } |
| 359 | if (isspace(*p)) | 363 | if (isspace(*p)) |
| @@ -365,13 +369,12 @@ bool ParseMoney(const char* pszIn, int64& nRet) | |||
| 365 | for (; *p; p++) | 369 | for (; *p; p++) |
| 366 | if (!isspace(*p)) | 370 | if (!isspace(*p)) |
| 367 | return false; | 371 | return false; |
| 368 | if (strWhole.size() > 17) | 372 | if (strWhole.size() > 10) // 10 whole digits * COIN stays within int64 |
| 369 | return false; | 373 | return false; |
| 370 | int64 nWhole = atoi64(strWhole); | 374 | int64 nWhole = atoi64(strWhole); |
| 371 | int64 nValue = nWhole * 100 + nCents; | 375 | int64 nValue = nWhole * COIN + nUnits; |
| 372 | if (nValue / 100 != nWhole) | 376 | if (nValue < 0 || (nValue - nUnits) / COIN != nWhole) |
| 373 | return false; | 377 | return false; |
| 374 | nValue *= CENT; | ||
| 375 | nRet = nValue; | 378 | nRet = nValue; |
| 376 | return true; | 379 | return true; |
| 377 | } | 380 | } |
diff --git a/src/walletcmd.cpp b/src/walletcmd.cpp index 06f35a5..0ea2251 100644 --- a/src/walletcmd.cpp +++ b/src/walletcmd.cpp | |||
| @@ -354,7 +354,7 @@ int CmdRecoveryAudit() | |||
| 354 | if (audit.fHaveSeed) | 354 | if (audit.fHaveSeed) |
| 355 | { | 355 | { |
| 356 | printf(" derivation schema: %s\n", HDKeySchemaName(audit.nSchema).c_str()); | 356 | printf(" derivation schema: %s\n", HDKeySchemaName(audit.nSchema).c_str()); |
| 357 | printf(" BIP44 coin type: %u (provisional BITFLASH)\n", audit.nCoinType); | 357 | printf(" BIP44 coin type: %u (SLIP-0044 BITFLASH)\n", audit.nCoinType); |
| 358 | printf(" derived keys known to this wallet: %u\n", audit.nDerivedKnown); | 358 | printf(" derived keys known to this wallet: %u\n", audit.nDerivedKnown); |
| 359 | printf(" receive/change counters: %u/%u\n", | 359 | printf(" receive/change counters: %u/%u\n", |
| 360 | audit.nReceiveNext, audit.nChangeNext); | 360 | audit.nReceiveNext, audit.nChangeNext); |
diff --git a/tests/fixtures/chain-tools/explorer-blocks.json b/tests/fixtures/chain-tools/explorer-blocks.json new file mode 100644 index 0000000..795d6e4 --- /dev/null +++ b/tests/fixtures/chain-tools/explorer-blocks.json | |||
| @@ -0,0 +1 @@ | |||
| {"blocks":[{"cbAddr":"BDnXDAAvEameyQk12FrtvBiPBp9MsfRmUA","cbValue":5000000000,"hash":"7c0e819cec25ebe29452049f1b35479526ba679bc2c1b00407417109384dea76","height":2,"nTx":2,"size":331,"time":1753315440},{"cbAddr":"B61KYnjpfXHN2w2e1nR869HneK4P1Gz4va","cbValue":5000000000,"hash":"17bdb455d41592adbff4aa97b7fca2091fef69600e67306baca829ab5a588b30","height":1,"nTx":2,"size":331,"time":1753315320},{"cbAddr":"BDU4zj6apurZ7YWNxsndaTt75PMgzvwbN4","cbValue":5000000000,"hash":"5bd7cb255d814e48cebcdfb72da4dc87b34bd774227f8ceb546c5640f4bdc169","height":0,"nTx":1,"size":278,"time":1753315200}],"count":3,"network":"bitflash","tipHeight":2} | |||
diff --git a/tests/fixtures/chain-tools/fair-launch-report.json b/tests/fixtures/chain-tools/fair-launch-report.json new file mode 100644 index 0000000..d35f30b --- /dev/null +++ b/tests/fixtures/chain-tools/fair-launch-report.json | |||
| @@ -0,0 +1 @@ | |||
| {"attestation_payload_sha256":"bf192bdbbb84c69d15bb68cf133abd474d0e1021600febc68e9f1a9de844b2af","chain":{"coin":100000000,"message_start_hex":"bf205cfd","name":"Bitflash","ticker":"BTF"},"checks":{"computed_merkle_matches_header":true,"genesis_has_single_transaction":true,"genesis_hash_matches_source":true,"genesis_merkle_matches_source":true,"genesis_supply_is_50_btf":true,"genesis_transaction_is_coinbase":true},"coinbase_outputs":[{"block_hash":"5bd7cb255d814e48cebcdfb72da4dc87b34bd774227f8ceb546c5640f4bdc169","coinbase_txid":"1a45b4482532abb29b10e234d3f13132230525a339ecea91658ffa675a5b1325","height":0,"outputs":[{"script_pub_key_hex":"4104678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5fac","value_btf":"50.00000000","value_satoshis":5000000000}],"value_btf":"50.00000000","value_satoshis":5000000000},{"block_hash":"17bdb455d41592adbff4aa97b7fca2091fef69600e67306baca829ab5a588b30","coinbase_txid":"d6967c16013f1da136183f962aefd2f15e59b6836043f8ef49aadb92f38dca05","height":1,"outputs":[{"script_pub_key_hex":"76a914111111111111111111111111111111111111111188ac","value_btf":"50.00000000","value_satoshis":5000000000}],"value_btf":"50.00000000","value_satoshis":5000000000},{"block_hash":"7c0e819cec25ebe29452049f1b35479526ba679bc2c1b00407417109384dea76","coinbase_txid":"d55af2c80e8b70a9ae8f362a8aac5448e7925267c7f737d1fa0d227fd472a657","height":2,"outputs":[{"script_pub_key_hex":"76a914666666666666666666666666666666666666666688ac","value_btf":"50.00000000","value_satoshis":5000000000}],"value_btf":"50.00000000","value_satoshis":5000000000}],"deterministic":true,"external_timestamping":{"instructions":["Save the canonical JSON report.","Run: ots stamp bitflash-fair-launch-report.json","Later verify with: ots verify bitflash-fair-launch-report.json.ots"],"recommended":"OpenTimestamps","status":"not_embedded"},"genesis":{"bits":"0x1f0fffff","coinbase_txid":"1a45b4482532abb29b10e234d3f13132230525a339ecea91658ffa675a5b1325","computed_merkle_root":"1a45b4482532abb29b10e234d3f13132230525a339ecea91658ffa675a5b1325","hash":"5bd7cb255d814e48cebcdfb72da4dc87b34bd774227f8ceb546c5640f4bdc169","merkle_root":"1a45b4482532abb29b10e234d3f13132230525a339ecea91658ffa675a5b1325","nonce":3141,"outputs":[{"script_pub_key_hex":"4104678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5fac","value_btf":"50.00000000","value_satoshis":5000000000}],"time":1753315200,"time_utc":"2025-07-24T00:00:00Z","tx_count":1},"scanned":{"block_count":3,"first_height":0,"last_height":2,"non_coinbase_transactions":2,"total_coinbase_btf":"150.00000000","total_coinbase_satoshis":15000000000},"schema":"bitflash-fair-launch-report-1","source_constants":{"genesis_bits":"0x1f0fffff","genesis_hash":"5bd7cb255d814e48cebcdfb72da4dc87b34bd774227f8ceb546c5640f4bdc169","genesis_merkle_root":"1a45b4482532abb29b10e234d3f13132230525a339ecea91658ffa675a5b1325","genesis_message":"Bitflash 24/Jul/2026 Fair launch: one CPU one vote, no premine","genesis_nonce":3141,"genesis_time":1753315200,"genesis_time_utc":"2025-07-24T00:00:00Z"},"source_mode":"local-block-files","verdict":"FAIR_LAUNCH_BASELINE_VERIFIED"} | |||
diff --git a/tests/fixtures/chain-tools/utxo-proof-verification.json b/tests/fixtures/chain-tools/utxo-proof-verification.json new file mode 100644 index 0000000..a40c673 --- /dev/null +++ b/tests/fixtures/chain-tools/utxo-proof-verification.json | |||
| @@ -0,0 +1 @@ | |||
| {"checks":{"leaf_hash_matches_target":true,"root_matches":true,"schema_is_supported":true},"commitment":{"computed_root":"6e49b47ffccb178b62402235343ae7c55ec4a03effe19debb8c057e8ffc5b3c0","expected_root":"6e49b47ffccb178b62402235343ae7c55ec4a03effe19debb8c057e8ffc5b3c0","leaf_hash":"da964462812e40c918d6349baa57aee1dc838416eca44ccd31b782f0d1542d7b","path_length":3},"deterministic":true,"failures":[],"schema":"bitflash-utxo-proof-verification-1","target":{"created_height":2,"outpoint":"218609b2969dc0b6ae1c378ae255bbed863902987df4c943200e3c291b0e4c2a:1","value_btf":"17.50000000","value_satoshis":1750000000},"verdict":"UTXO_PROOF_VERIFIED"} | |||
diff --git a/tests/fixtures/chain-tools/utxo-proof.json b/tests/fixtures/chain-tools/utxo-proof.json new file mode 100644 index 0000000..154c45b --- /dev/null +++ b/tests/fixtures/chain-tools/utxo-proof.json | |||
| @@ -0,0 +1 @@ | |||
| {"chain":{"coin":100000000,"message_start_hex":"bf205cfd","name":"Bitflash","ticker":"BTF"},"commitment":{"algorithm":"sorted-outpoint-merkle-sha256","leaf_count":5,"leaf_domain":"BTFUTXO1","node_domain":"BTFNODE1","root":"6e49b47ffccb178b62402235343ae7c55ec4a03effe19debb8c057e8ffc5b3c0"},"deterministic":true,"proof":[{"duplicated":false,"hash":"7a4f4f13aaf91f8b6b32d1fb22feaa3d0398782504915e9c156e60fdeca1255d","level":0,"side":"left"},{"duplicated":false,"hash":"5f91d5ba472b7d6ab96e9ce99d689896dad3c264d3abbce4c77f257687608943","level":1,"side":"right"},{"duplicated":false,"hash":"f041ba6a82fbc6759f0632f412dd8aab971ac1317358d6f204af3e57da4d12de","level":2,"side":"right"}],"scanned":{"first_height":0,"last_height":2,"main_chain_block_count":3,"raw_block_count":3,"side_blocks_ignored":0,"tip_hash":"7c0e819cec25ebe29452049f1b35479526ba679bc2c1b00407417109384dea76","tip_time":1753315440,"tip_time_utc":"2025-07-24T00:04:00Z"},"schema":"bitflash-utxo-inclusion-proof-1","target":{"leaf_hash":"da964462812e40c918d6349baa57aee1dc838416eca44ccd31b782f0d1542d7b","leaf_index":1,"outpoint":"218609b2969dc0b6ae1c378ae255bbed863902987df4c943200e3c291b0e4c2a:1","txid":"218609b2969dc0b6ae1c378ae255bbed863902987df4c943200e3c291b0e4c2a","utxo":{"coinbase":false,"created_block_hash":"7c0e819cec25ebe29452049f1b35479526ba679bc2c1b00407417109384dea76","created_height":2,"created_txid":"218609b2969dc0b6ae1c378ae255bbed863902987df4c943200e3c291b0e4c2a","script_pub_key_hex":"76a914555555555555555555555555555555555555555588ac","txid":"218609b2969dc0b6ae1c378ae255bbed863902987df4c943200e3c291b0e4c2a","value_btf":"17.50000000","value_satoshis":1750000000,"vout":1},"vout":1},"verdict":"UTXO_INCLUSION_PROOF_CREATED","warnings":{"legacy_duplicate_output_overwrites":0}} | |||
diff --git a/tests/fixtures/chain-tools/utxo-report.json b/tests/fixtures/chain-tools/utxo-report.json new file mode 100644 index 0000000..c6e98ee --- /dev/null +++ b/tests/fixtures/chain-tools/utxo-report.json | |||
| @@ -0,0 +1 @@ | |||
| {"chain":{"coin":100000000,"message_start_hex":"bf205cfd","name":"Bitflash","ticker":"BTF"},"checks":{"all_scanned_merkle_roots_match":true,"genesis_hash_matches_source":true,"genesis_merkle_matches_source":true,"no_missing_spends":true},"commitment":{"algorithm":"sorted-outpoint-merkle-sha256","leaf_domain":"BTFUTXO1","node_domain":"BTFNODE1","root":"6e49b47ffccb178b62402235343ae7c55ec4a03effe19debb8c057e8ffc5b3c0"},"deterministic":true,"scanned":{"first_height":0,"last_height":2,"main_chain_block_count":3,"raw_block_count":3,"side_blocks_ignored":0,"tip_hash":"7c0e819cec25ebe29452049f1b35479526ba679bc2c1b00407417109384dea76","tip_time":1753315440,"tip_time_utc":"2025-07-24T00:04:00Z"},"schema":"bitflash-utxo-set-commitment-1","stats":{"coinbase_minus_unspent_btf":"1.50000000","coinbase_minus_unspent_satoshis":150000000,"duplicate_outputs":[],"missing_spends":[],"normal_transactions":2,"spent_outputs":8000000000,"total_coinbase_btf":"150.00000000","total_coinbase_satoshis":15000000000,"total_fees_btf":"1.50000000","total_fees_satoshis":150000000,"total_unspent_btf":"148.50000000","total_unspent_satoshis":14850000000,"utxo_count":5},"utxos":{"218609b2969dc0b6ae1c378ae255bbed863902987df4c943200e3c291b0e4c2a:0":{"coinbase":false,"created_block_hash":"7c0e819cec25ebe29452049f1b35479526ba679bc2c1b00407417109384dea76","created_height":2,"created_txid":"218609b2969dc0b6ae1c378ae255bbed863902987df4c943200e3c291b0e4c2a","script_pub_key_hex":"76a914444444444444444444444444444444444444444488ac","txid":"218609b2969dc0b6ae1c378ae255bbed863902987df4c943200e3c291b0e4c2a","value_btf":"12.00000000","value_satoshis":1200000000,"vout":0},"218609b2969dc0b6ae1c378ae255bbed863902987df4c943200e3c291b0e4c2a:1":{"coinbase":false,"created_block_hash":"7c0e819cec25ebe29452049f1b35479526ba679bc2c1b00407417109384dea76","created_height":2,"created_txid":"218609b2969dc0b6ae1c378ae255bbed863902987df4c943200e3c291b0e4c2a","script_pub_key_hex":"76a914555555555555555555555555555555555555555588ac","txid":"218609b2969dc0b6ae1c378ae255bbed863902987df4c943200e3c291b0e4c2a","value_btf":"17.50000000","value_satoshis":1750000000,"vout":1},"24bddfded9e22c2f98d9369e9ac00a4eccc8dcf05a2d1b9e12d94fd9f62ababc:1":{"coinbase":false,"created_block_hash":"17bdb455d41592adbff4aa97b7fca2091fef69600e67306baca829ab5a588b30","created_height":1,"created_txid":"24bddfded9e22c2f98d9369e9ac00a4eccc8dcf05a2d1b9e12d94fd9f62ababc","script_pub_key_hex":"76a914333333333333333333333333333333333333333388ac","txid":"24bddfded9e22c2f98d9369e9ac00a4eccc8dcf05a2d1b9e12d94fd9f62ababc","value_btf":"19.00000000","value_satoshis":1900000000,"vout":1},"d55af2c80e8b70a9ae8f362a8aac5448e7925267c7f737d1fa0d227fd472a657:0":{"coinbase":true,"created_block_hash":"7c0e819cec25ebe29452049f1b35479526ba679bc2c1b00407417109384dea76","created_height":2,"created_txid":"d55af2c80e8b70a9ae8f362a8aac5448e7925267c7f737d1fa0d227fd472a657","script_pub_key_hex":"76a914666666666666666666666666666666666666666688ac","txid":"d55af2c80e8b70a9ae8f362a8aac5448e7925267c7f737d1fa0d227fd472a657","value_btf":"50.00000000","value_satoshis":5000000000,"vout":0},"d6967c16013f1da136183f962aefd2f15e59b6836043f8ef49aadb92f38dca05:0":{"coinbase":true,"created_block_hash":"17bdb455d41592adbff4aa97b7fca2091fef69600e67306baca829ab5a588b30","created_height":1,"created_txid":"d6967c16013f1da136183f962aefd2f15e59b6836043f8ef49aadb92f38dca05","script_pub_key_hex":"76a914111111111111111111111111111111111111111188ac","txid":"d6967c16013f1da136183f962aefd2f15e59b6836043f8ef49aadb92f38dca05","value_btf":"50.00000000","value_satoshis":5000000000,"vout":0}},"verdict":"UTXO_COMMITMENT_VERIFIED","warnings":{"legacy_duplicate_output_overwrites":0}} | |||
diff --git a/tests/scripts/test_bitflash_tools.py b/tests/scripts/test_bitflash_tools.py new file mode 100644 index 0000000..4b18d1e --- /dev/null +++ b/tests/scripts/test_bitflash_tools.py | |||
| @@ -0,0 +1,283 @@ | |||
| 1 | #!/usr/bin/env python3 | ||
| 2 | """Regression tests for deterministic Bitflash Python tooling.""" | ||
| 3 | |||
| 4 | import json | ||
| 5 | import os | ||
| 6 | import struct | ||
| 7 | import subprocess | ||
| 8 | import sys | ||
| 9 | import tempfile | ||
| 10 | from pathlib import Path | ||
| 11 | |||
| 12 | |||
| 13 | ROOT = Path(__file__).resolve().parents[2] | ||
| 14 | SCRIPTS = ROOT / "scripts" | ||
| 15 | FIXTURES = ROOT / "tests" / "fixtures" / "chain-tools" | ||
| 16 | sys.path.insert(0, str(SCRIPTS)) | ||
| 17 | |||
| 18 | import bitflash_chain as chain # noqa: E402 | ||
| 19 | |||
| 20 | |||
| 21 | def run(args, **kwargs): | ||
| 22 | env = dict(os.environ) | ||
| 23 | env["PYTHONIOENCODING"] = "utf-8" | ||
| 24 | return subprocess.run( | ||
| 25 | [sys.executable] + [str(a) for a in args], | ||
| 26 | cwd=str(ROOT), | ||
| 27 | env=env, | ||
| 28 | check=True, | ||
| 29 | text=True, | ||
| 30 | stdout=subprocess.PIPE, | ||
| 31 | stderr=subprocess.PIPE, | ||
| 32 | **kwargs | ||
| 33 | ) | ||
| 34 | |||
| 35 | |||
| 36 | def write_block_file(path, blocks): | ||
| 37 | with Path(path).open("wb") as f: | ||
| 38 | f.write(b"noise") | ||
| 39 | f.write(chain.MAGIC) | ||
| 40 | f.write(struct.pack("<I", chain.MAX_BLOCK_PAYLOAD + 1)) | ||
| 41 | f.write(b"still-noise") | ||
| 42 | for block in blocks: | ||
| 43 | payload = chain.ser_block(block) | ||
| 44 | f.write(chain.MAGIC) | ||
| 45 | f.write(struct.pack("<I", len(payload))) | ||
| 46 | f.write(payload) | ||
| 47 | |||
| 48 | |||
| 49 | def p2pkh_script(byte): | ||
| 50 | return "76a914%s88ac" % (bytes([byte]) * 20).hex() | ||
| 51 | |||
| 52 | |||
| 53 | def make_tx(vin, vout): | ||
| 54 | tx = { | ||
| 55 | "version": 1, | ||
| 56 | "vin": vin, | ||
| 57 | "vout": vout, | ||
| 58 | "lock_time": 0, | ||
| 59 | } | ||
| 60 | tx["txid"] = chain.hash_hex(chain.ser_tx(tx)) | ||
| 61 | return tx | ||
| 62 | |||
| 63 | |||
| 64 | def make_coinbase(label, script_byte, value=50 * chain.COIN): | ||
| 65 | return make_tx( | ||
| 66 | [{ | ||
| 67 | "prevout_hash": "0" * 64, | ||
| 68 | "prevout_n": 0xFFFFFFFF, | ||
| 69 | "script_sig_hex": label.encode("ascii").hex(), | ||
| 70 | "sequence": 0xFFFFFFFF, | ||
| 71 | }], | ||
| 72 | [{ | ||
| 73 | "value_satoshis": value, | ||
| 74 | "value_btf": chain.format_money(value), | ||
| 75 | "script_pub_key_hex": p2pkh_script(script_byte), | ||
| 76 | }], | ||
| 77 | ) | ||
| 78 | |||
| 79 | |||
| 80 | def make_spend(prev_txid, prev_n, outputs, label): | ||
| 81 | return make_tx( | ||
| 82 | [{ | ||
| 83 | "prevout_hash": prev_txid, | ||
| 84 | "prevout_n": prev_n, | ||
| 85 | "script_sig_hex": label.encode("ascii").hex(), | ||
| 86 | "sequence": 0xFFFFFFFF, | ||
| 87 | }], | ||
| 88 | [{ | ||
| 89 | "value_satoshis": value, | ||
| 90 | "value_btf": chain.format_money(value), | ||
| 91 | "script_pub_key_hex": p2pkh_script(script_byte), | ||
| 92 | } for value, script_byte in outputs], | ||
| 93 | ) | ||
| 94 | |||
| 95 | |||
| 96 | def make_block(prev_hash, txs, ntime, nonce): | ||
| 97 | merkle = chain.merkle_root([tx["txid"] for tx in txs]) | ||
| 98 | header = ( | ||
| 99 | struct.pack("<i", 1) | ||
| 100 | + bytes.fromhex(prev_hash)[::-1] | ||
| 101 | + bytes.fromhex(merkle)[::-1] | ||
| 102 | + struct.pack("<III", ntime, chain.GENESIS_BITS, nonce) | ||
| 103 | ) | ||
| 104 | block = { | ||
| 105 | "hash": chain.hash_hex(header), | ||
| 106 | "version": 1, | ||
| 107 | "previous_hash": prev_hash, | ||
| 108 | "merkle_root": merkle, | ||
| 109 | "computed_merkle_root": merkle, | ||
| 110 | "time": ntime, | ||
| 111 | "time_utc": chain.utc(ntime), | ||
| 112 | "bits": "0x%08x" % chain.GENESIS_BITS, | ||
| 113 | "nonce": nonce, | ||
| 114 | "tx_count": len(txs), | ||
| 115 | "transactions": txs, | ||
| 116 | "source_file": "synthetic", | ||
| 117 | "source_offset": 0, | ||
| 118 | "size": 80 + len(chain.ser_compact_size(len(txs))) + sum(len(chain.ser_tx(tx)) for tx in txs), | ||
| 119 | } | ||
| 120 | return block | ||
| 121 | |||
| 122 | |||
| 123 | def fixture_chain(): | ||
| 124 | genesis = chain.reconstructed_genesis() | ||
| 125 | spend1 = make_spend( | ||
| 126 | chain.GENESIS_MERKLE, | ||
| 127 | 0, | ||
| 128 | [(30 * chain.COIN, 0x22), (19 * chain.COIN, 0x33)], | ||
| 129 | "spend genesis with 1 BTF fee", | ||
| 130 | ) | ||
| 131 | block1 = make_block( | ||
| 132 | genesis["hash"], | ||
| 133 | [make_coinbase("fixture coinbase 1", 0x11), spend1], | ||
| 134 | chain.GENESIS_TIME + 120, | ||
| 135 | 4001, | ||
| 136 | ) | ||
| 137 | spend2 = make_spend( | ||
| 138 | spend1["txid"], | ||
| 139 | 0, | ||
| 140 | [(12 * chain.COIN, 0x44), (17 * chain.COIN + 50_000_000, 0x55)], | ||
| 141 | "split spend with 0.5 BTF fee", | ||
| 142 | ) | ||
| 143 | block2 = make_block( | ||
| 144 | block1["hash"], | ||
| 145 | [make_coinbase("fixture coinbase 2", 0x66), spend2], | ||
| 146 | chain.GENESIS_TIME + 240, | ||
| 147 | 4002, | ||
| 148 | ) | ||
| 149 | return [genesis, block1, block2], { | ||
| 150 | "genesis_outpoint": "%s:0" % chain.GENESIS_MERKLE, | ||
| 151 | "first_spend_txid": spend1["txid"], | ||
| 152 | "proof_outpoint": "%s:1" % spend2["txid"], | ||
| 153 | } | ||
| 154 | |||
| 155 | |||
| 156 | def compare_golden(name, actual, update): | ||
| 157 | path = FIXTURES / name | ||
| 158 | if update: | ||
| 159 | path.parent.mkdir(parents=True, exist_ok=True) | ||
| 160 | path.write_bytes(actual) | ||
| 161 | return | ||
| 162 | expected = path.read_bytes() | ||
| 163 | if actual != expected: | ||
| 164 | raise AssertionError("golden mismatch: %s" % name) | ||
| 165 | |||
| 166 | |||
| 167 | def assert_true(cond, msg): | ||
| 168 | if not cond: | ||
| 169 | raise AssertionError(msg) | ||
| 170 | |||
| 171 | |||
| 172 | def run_genesis_smoke(): | ||
| 173 | genesis = chain.reconstructed_genesis() | ||
| 174 | outpoint = "%s:0" % chain.GENESIS_MERKLE | ||
| 175 | |||
| 176 | with tempfile.TemporaryDirectory(prefix="bitflash-tool-tests-") as tmp: | ||
| 177 | tmp = Path(tmp) | ||
| 178 | datadir = tmp / "datadir" | ||
| 179 | datadir.mkdir() | ||
| 180 | blk1 = datadir / "blk0001.dat" | ||
| 181 | blk2 = datadir / "blk0002.dat" | ||
| 182 | write_block_file(blk1, [genesis]) | ||
| 183 | blk2.write_bytes(b"") | ||
| 184 | |||
| 185 | fair = run([SCRIPTS / "verify-fair-launch.py", "--datadir", datadir, | ||
| 186 | "--max-blocks", "1", "--json"]).stdout | ||
| 187 | fair_report = json.loads(fair) | ||
| 188 | assert_true(fair_report["verdict"] == "FAIR_LAUNCH_BASELINE_VERIFIED", | ||
| 189 | "fair launch verifier failed synthetic genesis") | ||
| 190 | |||
| 191 | utxo = run([SCRIPTS / "verify-utxo-set.py", "--datadir", datadir, | ||
| 192 | "--json"]).stdout | ||
| 193 | utxo_report = json.loads(utxo) | ||
| 194 | assert_true(utxo_report["verdict"] == "UTXO_COMMITMENT_VERIFIED", | ||
| 195 | "UTXO verifier failed synthetic genesis") | ||
| 196 | |||
| 197 | proof_path = tmp / "proof.json" | ||
| 198 | run([SCRIPTS / "prove-utxo.py", outpoint, "--datadir", datadir, | ||
| 199 | "--out", proof_path, "--json"]) | ||
| 200 | proof_check = run([SCRIPTS / "verify-utxo-proof.py", proof_path, | ||
| 201 | "--root", utxo_report["commitment"]["root"], "--json"]).stdout | ||
| 202 | proof_report = json.loads(proof_check) | ||
| 203 | assert_true(proof_report["verdict"] == "UTXO_PROOF_VERIFIED", | ||
| 204 | "UTXO proof verifier rejected synthetic proof") | ||
| 205 | |||
| 206 | explorer_dir = tmp / "explorer" | ||
| 207 | run([SCRIPTS / "build-explorer.py", blk1, blk2, explorer_dir]) | ||
| 208 | blocks = json.loads((explorer_dir / "blocks.json").read_text(encoding="ascii")) | ||
| 209 | assert_true(blocks["tipHeight"] == 0 and blocks["count"] == 1, | ||
| 210 | "explorer did not build the synthetic chain") | ||
| 211 | index_html = (explorer_dir / "index.html").read_text(encoding="utf-8") | ||
| 212 | assert_true("innerHTML" not in index_html, "explorer viewer still uses innerHTML") | ||
| 213 | |||
| 214 | try: | ||
| 215 | chain.apply_blocks([genesis, genesis], strict_duplicates=True) | ||
| 216 | except chain.ParseError: | ||
| 217 | pass | ||
| 218 | else: | ||
| 219 | raise AssertionError("strict duplicate output mode did not fail") | ||
| 220 | |||
| 221 | |||
| 222 | def run_golden_fixture(update): | ||
| 223 | blocks, meta = fixture_chain() | ||
| 224 | |||
| 225 | with tempfile.TemporaryDirectory(prefix="bitflash-tool-golden-") as tmp: | ||
| 226 | tmp = Path(tmp) | ||
| 227 | datadir = tmp / "datadir" | ||
| 228 | datadir.mkdir() | ||
| 229 | blk1 = datadir / "blk0001.dat" | ||
| 230 | blk2 = datadir / "blk0002.dat" | ||
| 231 | write_block_file(blk1, blocks[:2]) | ||
| 232 | write_block_file(blk2, blocks[2:]) | ||
| 233 | |||
| 234 | fair = run([SCRIPTS / "verify-fair-launch.py", "--datadir", datadir, | ||
| 235 | "--max-blocks", "3", "--json"]).stdout.encode("ascii") | ||
| 236 | compare_golden("fair-launch-report.json", fair, update) | ||
| 237 | fair_report = json.loads(fair) | ||
| 238 | assert_true(fair_report["scanned"]["non_coinbase_transactions"] == 2, | ||
| 239 | "fair-launch fixture did not count normal transactions") | ||
| 240 | |||
| 241 | utxo = run([SCRIPTS / "verify-utxo-set.py", "--datadir", datadir, | ||
| 242 | "--include-utxos", "--strict-duplicates", "--json"]).stdout.encode("ascii") | ||
| 243 | compare_golden("utxo-report.json", utxo, update) | ||
| 244 | utxo_report = json.loads(utxo) | ||
| 245 | assert_true(utxo_report["stats"]["normal_transactions"] == 2, | ||
| 246 | "UTXO fixture did not apply normal transactions") | ||
| 247 | assert_true(utxo_report["stats"]["total_fees_btf"] == "1.50000000", | ||
| 248 | "UTXO fixture fee total changed") | ||
| 249 | assert_true(meta["genesis_outpoint"] not in utxo_report["utxos"], | ||
| 250 | "spent genesis output still appears in the UTXO map") | ||
| 251 | |||
| 252 | proof_path = tmp / "fixture-proof.json" | ||
| 253 | proof = run([SCRIPTS / "prove-utxo.py", meta["proof_outpoint"], | ||
| 254 | "--datadir", datadir, "--strict-duplicates", | ||
| 255 | "--out", proof_path, "--json"]).stdout.encode("ascii") | ||
| 256 | compare_golden("utxo-proof.json", proof, update) | ||
| 257 | |||
| 258 | proof_check = run([SCRIPTS / "verify-utxo-proof.py", proof_path, | ||
| 259 | "--root", utxo_report["commitment"]["root"], "--json"]).stdout.encode("ascii") | ||
| 260 | compare_golden("utxo-proof-verification.json", proof_check, update) | ||
| 261 | proof_report = json.loads(proof_check) | ||
| 262 | assert_true(proof_report["verdict"] == "UTXO_PROOF_VERIFIED", | ||
| 263 | "fixture UTXO proof did not verify") | ||
| 264 | |||
| 265 | explorer_dir = tmp / "explorer" | ||
| 266 | run([SCRIPTS / "build-explorer.py", blk1, blk2, explorer_dir]) | ||
| 267 | compare_golden("explorer-blocks.json", | ||
| 268 | (explorer_dir / "blocks.json").read_bytes(), update) | ||
| 269 | detail = json.loads((explorer_dir / "block" / "1.json").read_text(encoding="ascii")) | ||
| 270 | assert_true(len(detail["txs"]) == 2, | ||
| 271 | "explorer fixture block did not include the normal transaction") | ||
| 272 | |||
| 273 | |||
| 274 | def main(): | ||
| 275 | update = "--update-goldens" in sys.argv[1:] | ||
| 276 | run_genesis_smoke() | ||
| 277 | run_golden_fixture(update) | ||
| 278 | print("Bitflash script tool regressions passed") | ||
| 279 | return 0 | ||
| 280 | |||
| 281 | |||
| 282 | if __name__ == "__main__": | ||
| 283 | raise SystemExit(main()) | ||