| Age | Commit message (Collapse) | Author |
|---|
|
Splitting SavePendingPayoutsLocked() into a serialise step that needs
gPayoutLedgerMutex and a write step that does not left the write step with
no serialisation at all. The event loop wrote pending_payouts.json from
QueuePayouts and the orphan-void path with the ledger mutex HELD; the payout
thread wrote it from RecordPayoutTxid and ForfeitPayoutRecipient with the
mutex RELEASED. Both fopen()ed the same "<path>.tmp" with independent handles
and both rename()d it.
Two ways that ends, both live money:
- the renamed file is a mixture of two images. LoadPendingPayouts throws
inside its parse loop, logs one warning, and the node comes up with a
nearly empty ledger -- every obligation not yet exploded into an in-flight
record is simply gone, with nothing on disk saying anyone was owed it.
- a lost update: the payout thread's older snapshot lands after the event
loop's newer one and silently erases a whole round that had just been
queued.
Neither parent had this. Upstream writes under the mutex everywhere; the
pre-merge tree had one writer thread and no ledger mutex. The merge invented
the window while fixing a real problem -- the event loop should not block in
FlushMaturePayouts behind a per-recipient fsync on the payout thread -- so
putting the write back under the ledger mutex is not the fix.
Instead: a per-file write mutex that is never held together with a data
mutex, plus a version on every snapshot.
- serialising happens under the data mutex and stamps a monotonically
increasing sequence number (gPayoutLedgerSeq).
- publishing happens with the data mutex released, under the file mutex,
and drops any blob whose sequence number is not strictly newer than the
one already on disk.
Because each blob is a whole-state dump of a single in-memory structure and
the sequence number is assigned under the mutex guarding that structure, a
higher sequence number is by construction a later state -- so discarding an
older blob loses nothing, and the file can only move forwards. Writers may
queue and complete in any order and the published file is still the newest
state anyone has serialised. A failed write deliberately does not advance the
published sequence, so the next write still tries.
The same treatment is applied to paying_payouts.json and pool_rounds.json.
Neither had the interleaving race -- gPayoutExecMutex and gPoolRoundsMutex
already serialise their writers -- but both did their I/O under a mutex the
event loop takes (FlushMaturePayouts, RecordPoolRound), so both were a place
for the event loop to wait on the payout thread's disk. They are now written
outside those mutexes on the same versioned terms, which is also what makes
adding fsync to these paths safe.
pool_status.json keeps a plain atomic write with no version guard: RefreshStats
is its only caller and only ever runs on the event loop, so it has a single
writer by construction. It shares the same WriteFileAtomic helper so the
durability rules live in one place.
No on-disk format change.
|
|
'make -C src tests' builds src/test_bip32 in the tree and it was showing up as
untracked; src/obj-node/ was only covered incidentally by the *.o rule.
|
|
VERSION names every release artifact -- the AppImage, the windows zip and the
node binary. This tree is upstream v1.2.18 plus 24 commits of our own and a
payout ledger that is neither side's, so a binary built here must not be
mistakable for the upstream release of that number.
|
|
Merges upstream tag v1.2.18 (bc3de30) into our 24 commits. The merge base is
a259215, upstream's own commit, whose src/ tree is byte-identical to 68de575 --
our commits were replayed onto a259215 first so that git had a real base to work
from. That produced 28 conflict hunks in 9 files instead of the 21 conflict
events / 51 hunks a direct cherry-pick onto the tag would have forced, most of
which were artifacts of lifting our commits away from their own predecessors.
No consensus change. main.cpp's GetBlockValue, GetNextWorkRequired,
ConnectInputs, ConnectBlock, CheckBlock, AcceptBlock and AcceptTransaction are
untouched by this merge; script.cpp, serialize.h and randomx_pow.h are
identical on both sides. The frozen-chain oracle reproduces the pinned UTXO root
116a2b6b... byte for byte, and the merged binary loads the same 12,586-block
chain to the same tip aac622f0a3b2f8... with all blocks re-verified and none
rejected.
Which side won, per file and per hunk.
src/rpc.cpp -- 15 hunks.
1. RebuildJob transaction selection. THEIRS for the multi-pass loop and the
nFees fix, OURS for the cap. Their vfAlreadyAdded/fFoundSomething loop lands
dependent transactions that our single pass dropped deterministically, and
their nFees is accumulated across the scan instead of being redeclared inside
it -- ours passed a hardcoded 0 to GetBlockValue, so every pool block built
here discarded its fees. But their bound is MAX_SIZE/2 = 16 MB, and
MAX_BLOCK_SIZE is 1 MB, so their template can exceed what their own
CheckBlock accepts. Cap kept at MAX_BLOCK_SIZE/2, with our inner break so a
single pass cannot overshoot by more than one transaction. Our RebuildJob
sync gating and deaf-node detection are kept; upstream has none.
2. struct PendingPayout. OURS for the key, THEIRS for paidTxids. Their id is
the string "block-<height>", which is height keying -- the exact bug 44849e6
fixed after a 280-block reorg left ~14.4k BTF owed against blocks that no
longer existed. Height keying also lets the crash-recovery probe match the
previous round's transaction after a reorg and silently skip a miner, so
their id is dropped entirely and blockHash stays the key. paidTxids is taken
because it is what makes an entry survivable. Their nextAttempt and our
retryAfter are the same concept under two names; ours is kept and theirs
dropped. matureAtHeight gained an explicit = 0, which neither side had in
our style of initialiser.
3./4. Ledger writer and loader. Mechanical follow-ons of 2. The loader keeps
ours and takes their tolerance for an entry with no usable height, guarded so
it cannot fire on a well-formed entry. One redundancy was removed rather than
merged: their unguarded `if (obj.contains("blockHeight")) get<int>()` sat two
lines above our is_number-guarded version, and an unguarded get<int>() on a
malformed value throws out of the whole loop, losing every entry in the file.
Verified against the live production pending_payouts.json (2 entries, keys
amounts/blockHash/blockHeight/matureAtHeight): loads both. A file in the new
shape, with retryAfter and paidTxids, also loads.
5. Insertion point after the loader. BOTH. Our PayingRecord/gPaying in-flight
ledger and their PoolRoundProof/PoolRoundToJson are disjoint features that
landed at the same offset. Both kept, in full.
6. Writer tail. THEIRS, copied into our writer. Checking fclose() matters here
because fclose is where the buffered write actually reaches the OS: renaming
a short .tmp over a good file is precisely the truncation the atomic write
exists to prevent. The Windows remove()-before-rename() is required because
rename() there will not replace an existing file.
7. Loader bodies. BOTH -- LoadPaying and LoadPoolRounds are different functions
that happened to share a common tail.
8. Function block, 111 vs 36 lines. BOTH, with one deliberate deletion.
PayoutBlockStillOnChain, PayoutComment, RecordPoolRound, MarkPoolRoundPaid
all kept. WalletAlreadyPaid is DELETED and ReconcilePayouts rewritten,
because our reconciliation and their FindRecordedPoolPayout are two answers
to the same crash-recovery question and running both would give the same debt
two authorities. Their wallet probe wins: it matches structured metadata
rather than a comment string, so it can tell two rounds paying the same
address the same amount apart. ReconcilePayouts keeps only the job that is
still real -- dropping in-flight records the ledger already shows paid, and
arming each restored record for its one wallet probe. It no longer touches
gPendingPayouts at all; under the new model dropping ledger entries there
would reintroduce the loss this merge adopts their model to prevent. Their
gPayoutExecMutex/gPayoutThreadRunning re-declarations were dropped as
duplicates of ours, and gPayoutExecQueue was not taken: gPaying is the exec
queue in this tree, at finer granularity.
9. QueuePayouts signature. THEIRS for the round-proof arguments, OURS for the
type of blockHash. It stays a uint256 and is converted to hex only where the
round proof needs text -- a ledger key that has been through ToString() is a
key that can be compared wrong.
10. Maturity policy. OURS. matureAtHeight = blockHeight +
COINBASE_SPENDABLE_DEPTH (120), not + COINBASE_MATURITY (100). At depth 100
the block's own coinbase is still 20 blocks short of spendable, so the
payment must come from unrelated mature coin and the pool runs a permanent
float. Our own history contradicts itself here -- d6972a0 argued for
COINBASE_MATURITY citing "96% of 934 payments", 44849e6 reversed it on the
grounds that the measurement was taken while the node was alone on a private
fork paying itself. HEAD's position is kept. This is a considered
disagreement with upstream, not an oversight. The live production ledger
confirms it: both entries have matureAtHeight - blockHeight = 120.
11. Comment above FlushMaturePayouts. Rewritten to describe the merged design.
12. FlushMaturePayouts, the central difference. THEIRS for the model. The ledger
entry now survives on disk until every recipient has a txid, instead of
being erased as soon as its payments were handed off -- the old ordering
left a window where the debt was described by nothing at all, and a
container restart on 2026-07-30 silently dropped ~1,156 BTF through it. Our
gPaying is kept on top as the in-flight working set, because it carries the
per-recipient retry and deferral state a per-round nextAttempt cannot
express. It is explicitly a cache derived from the ledger, not a second
record of debt: if it were lost the next flush would rebuild it.
13. PayoutThreadFn, 79 vs 70 lines. OURS for the loop and the reorg re-check,
THEIRS for the per-recipient dedupe. The re-check on every attempt is
load-bearing and upstream has no equivalent -- it decides once, at maturity,
and a payment can sit through hours of retries during which the reorg that
kills its funding block can land. Their paidTxids check is taken, against
the ledger rather than against the batch. Our deferral policy is kept:
waiting for coin is not a failure and must not consume the retry budget.
Two fixes upstream does not have are folded in here rather than deferred to
a follow-up commit:
- FindRecordedPoolPayout runs once per record, on its first attempt, not on
every attempt. It is a linear scan of mapWallet under cs_mapWallet, per
recipient; upstream repeats it for every retry, which with 2000 rotating
payout addresses and a budget of 1440 retries is a full wallet scan per
address per minute for a day. The question it answers can only change
across a restart, and every restart re-arms the flag.
- The ledger file write no longer happens under gPayoutLedgerMutex in the
per-recipient loop. SavePendingPayoutsLocked was split into a serialise
step that needs the lock and a write step that does not, so the mutex is
held only long enough to read the structure. The orphan scan in
FlushMaturePayouts was likewise restructured so the ledger mutex is not
held across PayoutBlockStillOnChain's cs_main acquisition.
One case the merged model created and neither side handled: under
survives-until-fully-paid, our GIVING UP path dropped the in-flight record
but left the recipient owed in the ledger forever, so every later flush
would explode it again -- a forfeit that silently becomes an infinite retry.
ForfeitPayoutRecipient writes the amount off so the entry can retire.
14. QueuePayouts call site. Follow-on of 9.
15. ThreadRPCServer startup. BOTH. LoadPoolRounds added; LoadPaying and
ReconcilePayouts stay, and stay before the payout thread starts.
Crash safety, stated as the property that must hold: after kill -9 at any
point, restart pays each recipient exactly once. Before SendMoney, the
ledger has no txid and the payment is retried. Between SendMoney and the
ledger write, the wallet holds the transaction and the first-attempt probe
finds it. After the ledger write, paidTxids suppresses it. If the in-flight
file is lost entirely, the ledger rebuilds it and the probe still covers the
middle window.
src/nostr.cpp -- 3 hunks. THEIRS as the base, with two things restored from
ours. Theirs loads the Windows ROOT certificate store, which ours lacks and
genuinely needs, since OpenSSL's default verify paths are usually empty there
and our version would fail closed. But the plan's claim that theirs is a
superset of our 06bceb4 is FALSE: upstream's nostr.cpp contains no
min_proto_version, no TLS1_2, no SSL_OP_NO_*, and no set_options anywhere, so
taking it wholesale would have removed the TLS-version floor outright. Restored.
Second, their SSL_get_verify_result check sits after a successful handshake,
where with SSL_VERIFY_PEER set it is nearly dead code -- a bad certificate
aborts inside SSL_connect. The reporting belongs in the failure branch, which is
where the merge put it; their post-handshake duplicate was dropped. Our
relay-list and socket-pooling hunks auto-merged and were verified intact.
src/net.cpp -- 1 hunk. THEIRS. Their guard, hSocket == INVALID_SOCKET ||
pnode->hSocket != hSocket, subsumes ours and additionally catches descriptor
recycling, which ours did not; our version also referenced a pnode that only
existed in the select() loop. Our other db34011 hunk, nulling pnode->hSocket in
the poll-failure branch, auto-merged and was NOT re-applied -- plan section 7
Batch D says to re-apply it, and that instruction is stale. Two pieces of
residue the merge left behind were cleaned up: a comment still describing
select() semantics, and a pnode->hSocket != INVALID_SOCKET guard made redundant
by a continue two lines above it.
src/main.cpp -- 2 hunks. h1 BOTH, pure add/add adjacency between our
ParticipantHashThreadCount and their SecureClear forward declarations. h2 OURS,
and their handler discarded: 44004e3 no-ops checkorder/submitorder but keeps
market.cpp and market.h, whereas 11766be deletes the marketplace outright
including CRequestTracker, subscribe/publish and mapProducts. Ours is strictly
more.
src/main_gui.cpp -- 2 hunks. BOTH, additive on both sides. Our /payoutaddr,
/payoutfile and /hashthreads alongside their /stratumbridge and
/stratumbridgeport.
src/btfrv.cpp -- 1 hunk, not predicted by the plan. False adjacency: our
INVALID_SOCKET check and their #endif landed at the same offset. Ordering is
load-bearing -- our check goes inside the #else arm, before the #endif, or the
non-proxy build silently loses it. The BITFLASH_NODE_BUILD arm already returns
early, so both paths now have the check.
Makefile -- 2 hunks plus a hazard with no conflict marker. THEIRS for both
dependency-fetch recipes: checkout --detach FETCH_HEAD followed by test
"$(git rev-parse HEAD)" = "$(..._COMMIT)" asserts the pin actually landed, where
ours only checked it out. Everything of ours that they lack is kept: the
appimagetool 1.9.1 release pin with a sha256 check, where upstream fetches the
mutable "continuous" tag and executes it undigested, and the move of both the
tool and the AppDir out of /tmp, where any local user could pre-place an
executable and have it run as whoever runs make.
The hazard: both sides added a pin block in a different place, so git merged
both, outside any conflict region. SECP256K1_COMMIT and RANDOMX_COMMIT ended up
assigned twice, near L16 and near L44, and GNU Make takes the last := -- a naive
resolution would have silently built against our old secp256k1 pin while
appearing to adopt upstream's. The duplicate block was deleted by hand. Each of
the four variables is now assigned exactly once, and make -n resolves
SECP256K1_COMMIT to 7fecac74 with 6e2c8bc4 appearing nowhere.
Also applied here, from plan phase 4: SECP256K1_COMMIT moved to upstream's
7fecac74aed8e1fd9078380d67dd04663705c989 so signature validation matches the
binaries the rest of the network runs. secp256k1 was rebuilt from scratch
against the new pin on the build host and the checked-out dependency commit
verified. RANDOMX_COMMIT already matched at 1e9d4b2d and is unchanged.
src/Makefile, src/makefile.mingw -- 1 hunk each, both the HEADERS list. Ours
drops market.h, theirs adds proxy.h; the resolution does both. Taking theirs
wholesale would have left the Makefiles referencing a header git had already
deleted in our favour, silently, without a conflict.
Files the plan predicted would conflict and did not: net.h, main.h and db.cpp
auto-merged. Verified rather than assumed. net.h carries both our
BTF_MSG_WAIT_MAX_SECS work, including the constructor fix from 2d84982 that
starts nMsgWaitSince and nMsgWaitSize at 0, and their comment rewrite with
extern int nPeersWatched. The dead members that fix removed are gone from every
file.
Deviations from MERGE-1.2.18-PLAN.md.
- Section 6's mechanism was not used. Replayed onto the real merge base and
merged, rather than cherry-picking onto the tag. Same result, 28 hunks
instead of 51.
- Section 2 calls 09ff575 a superset of our 06bceb4. It is not; upstream has no
TLS-version floor at all. Ours restored on top of theirs.
- Section 7 Batch D says to re-apply our db34011 line into their poll-failure
branch. It auto-merged; re-applying would have duplicated it.
- Section 2 says do not take the stratum bridge. It is taken, and left off.
fStratumBridge defaults false in main.cpp, the thread is only spawned when
/stratumbridge is passed, and the listener binds 127.0.0.1 only. Excising it
would cost real divergence and future conflict for no gain while it is
disabled. Its defects -- select()/FD_SET on a node that can hold more than
1024 descriptors, and an uncapped thread per connection -- are real only when
it is enabled and are being reported upstream separately.
- Section 7's staged batches were not run as separate gated commits. The
conflict surface is interleaved within single functions, chiefly the payout
ledger, and splitting it would have meant committing states that do not
build. The merge is one commit, as briefed.
- Batch F's two fixes are folded into hunk 13 rather than deferred, because
they change the same lines the resolution rewrites.
- The plan did not anticipate the btfrv.cpp conflict, the duplicate pin block,
the market.h trap in the two Makefiles, or the give-up/infinite-retry
interaction created by adopting upstream's ledger model.
|
|
CNode's constructor never wrote nMsgWaitSince or nMsgWaitSize, so every peer
began life with indeterminate values in the pair ProcessMessages uses to
decide whether that peer has stopped sending, and the first partial message
it read was measured against them.
The guard restarts the clock only when nMsgWaitSince is 0 or the buffer has
grown since the last look. Garbage in nMsgWaitSince is neither, so the very
first look fell through to the timeout branch. A peer that merely split a
message across two TCP segments -- header in one, body in the next, which is
ordinary on any real link -- could be dropped as unreadable on first contact,
before it had sent anything wrong and often before its version message. It
was never reproducible either: what happened depended on what had last used
that heap. The log reported the stall as roughly the age of the epoch:
peer 127.0.0.1:8433 announced a 100 byte 'block' and has sent nothing more
for 1786593923 s (0 of 100 payload bytes) -- dropping, this peer is
unreadable
Every node is running this.
Introduced in 7f6e5ad, which added the pair to the class and to
ProcessMessages but not to the constructor; the constructor went on
initialising nIncompleteMessageStart, nIncompleteMessageSize and
strIncompleteMessageCommand, the upstream members our implementation
supersedes. 5490d66 removed their last reader from ProcessMessages, and
nothing has read or written them since, so they go, with
BTF_INCOMPLETE_MESSAGE_TIMEOUT_SECS -- decoy state in a networking struct is
how this happened, and the merge ahead of us would carry it further. The
initialiser list now follows declaration order, so a member missing from it
is visible.
The net-message selftest was asserting on those dead members too, which is
why it could not have caught this: it read state no code path writes, and its
stale-message case passed only when the garbage in nMsgWaitSince happened to
look old. Rewired onto nMsgWaitSince / nMsgWaitSize / BTF_MSG_WAIT_MAX_SECS,
and given the two cases it never covered -- a new peer's clock must start
stopped, and a peer that is still delivering payload must not be dropped
however old its clock is.
Measured on the build host, 20 runs each: before, net-message failed 20/20
("a fresh incomplete message does not disconnect immediately" failed 17/20 by
itself). After, 20/20 pass. Rewired selftest against a reverted constructor
fix: fails 20/20, deterministically, where the old suite caught it by luck or
not at all.
|
|
StartMinerThreads() sizes its pool with MinerThreadCount(), which is right for
solo mining -- each thread hashes a template of its own -- and wrong for
participant mode, where every thread entering BitcoinMiner() is a whole miner:
its own Stratum session, its own payout address off the rotation, its own pool
of /hashthreads workers. The two counts multiplied instead of dividing.
Measured on the pool box with /hashthreads=2: a container with no cpuset logged
"Mining with 11 thread(s) of 12 core(s)" -- 11 sessions, 22 hash threads on 12
cores, 1058% CPU, load 22.68 -- while the same binary under --cpuset-cpus 10,11
logged one session and 199%. The second miner (8 cores, /hashthreads=1) ran 7
sessions and went 74C to 89C. No code differed between those runs. Only the
cpuset did, which means the cpuset was holding the thread count down by
accident and every deployment without one was oversubscribed.
Participant mode now starts exactly one session, so /hashthreads means what it
says instead of being multiplied by the core count. The worker count moves to
ParticipantHashThreadCount(), next to MinerThreadCount() where the two are
easy to confuse, so what is printed at startup and what actually hashes cannot
drift apart -- and it IS printed now. It was only ever visible under -debug,
which is how eleven sessions ran unnoticed.
Same host, same cpuset (0-3), same arguments, measured as a delta over 20 s
more than four minutes after start:
before 399% CPU, 6-7 threads hot, 3 sessions
after 201% CPU, exactly 2 threads at 99.9%, 1 session
Solo mining is untouched: /gen without /participant still starts
MinerThreadCount() threads.
(cherry picked from commit 45bc9ed5bc132712092f3e7902fef746c6cae898)
|
|
The two dependency clones were `git clone --depth 1 <url>` with no
revision, which resolves to whatever the default branch points at the
moment the build happens to run.
For RandomX that is the proof of work. Its master is not frozen -- v2.0
changed the algorithm in March -- so two builds of the same Bitflash
commit, made weeks apart, can hash the same header differently. That is
not a supply-chain worry, it is a chain split delivered as a build
artifact, with no version recorded anywhere to explain it afterwards. For
libsecp256k1 it is signature verification, the rule deciding who may
spend.
Both now fetch a full commit id, which is the only form here that
verifies what it got, since git resolves the object by its own hash. The
RandomX pin is the commit this tree has been building and syncing
against, so today's behaviour is unchanged; secp256k1 is pinned to the
v0.8.0 release commit.
appimagetool came from the "continuous" tag, which upstream moves in
place on every push, fetched over the network and executed unverified.
Pinned to a numbered release and checked by digest before it is allowed
to run.
It also ran from /tmp on a fixed name, guarded by `[ ! -x
/tmp/appimagetool ]` -- an executable already sitting there is taken as
the tool, so any local account could leave one and have it run as
whoever built. The AppDir had the same shape one step later, through
symlinks on the rm -rf. Both moved under the build tree, which the build
already owns, and cleaned by `make clean`.
(cherry picked from commit 664aebca08d17a9dbae12c25f21cd8ef2abe7aa6)
|
|
Four inherited handlers took attacker-controlled input and wrote to disk
with nothing authorising it. None of them had a caller left anywhere in
the tree, so all four were reachable only from the outside.
checkorder called GenerateNewKey() -- AddKey(), a wallet.dat write -- for
any peer that asked, and handed back the pubkey. Its comment claims the
cache keeps one key per ip, and on a 2009 network of routable addresses
it did. Not here: inbound peers arrive through a rendezvous tunnel and
are tagged by BtfMarkerAddr() over three random bytes, so addr.ip is
fresh every connection and the cache never hits. Reconnect in a loop and
the node mints a key per connection, none of them in any backup taken
beforehand, which for a pre-BIP32 wallet is how coins go missing. The map
holding them was static and never trimmed.
submitorder AddToWallet()'d a CWalletTx from any peer that sent one. It
had to be a valid transaction; it did not have to be ours.
review checked a signature the sender made with its own key, which proves
possession and authorises nothing, then appended to reviews.dat by
reading the whole vector, pushing one entry and writing it all back --
quadratic disk on an unbounded map<string,string> off the socket -- then
walked a link graph the same peer supplied and relayed the message to
every other node.
reply dispatched into CRequestTracker. Nothing ever called PushRequest,
so the table was permanently empty and the handler was a parser and
nothing else.
Removed with the subsystem behind them: market.cpp, CProduct, CReview,
CUser, CReviewDB, the advert publish templates and the subscribe channel
machinery, none of which had an inbound handler either. MSG_REVIEW,
MSG_PRODUCT and MSG_TABLE stay in the inv enum, unserved: CInv puts type
on the wire as an integer and MSG_TX and MSG_BLOCK are shared with every
other node.
Unknown commands are already ignored for extensibility, so peers still
running these lose nothing but an answer they never asked for. Verified
by a clean build of bitflash-node.
(cherry picked from commit ad9c393866c007a115f32d70a9017ce91066a7d1)
|
|
std::thread::hardware_concurrency() reports the MACHINE's core count. Inside a
container that is the wrong number: with --cpuset-cpus 0-5 on a 12-core host it
still answers 12, and every caller that sized a pool from it oversubscribed 2x.
Measured on the pool box:
- the pool node logged "Mining with 11 thread(s) of 12 core(s)" while
confined to 6 CPUs
- a burst miner launched with /hashthreads=2 filled the ~2 GB RandomX
dataset with 12 threads, alongside the pool node in the same 8 GB container
AvailableCoreCount() asks sched_getaffinity what the cpuset actually permits,
which is the number all of these meant. It cannot make the dataset build slower:
the CPU budget is identical either way, so fewer threads on the same cores is
the same work without the thrash. Falls back to hardware_concurrency where the
call is unavailable, and never returns 0 because callers subtract from it.
Not the cause of the burst miner dying -- that was a corrupt Berkeley DB
environment in the worker datadir -- but it is why the failure landed on the
dataset build, and it is wrong on its own terms.
(cherry picked from commit b3bee16f3eef8982bff1174bd9d0a9c21473732c)
|
|
Upstream #109's message-stall rework, hand-merged so it keeps the progress
timing this tree already had.
The old path consumed the header, found the payload short, inserted the header
back at the front of vRecv, then Sleep(100) inside the shared message thread.
A peer that announces a large payload and drips bytes made every pass pay an
O(n) insert and a 100 ms stall -- one peer slowing the thread that serves all
of them. Now the header is read from a copy and only skipped once the whole
body has arrived, so there is nothing to put back and nothing to sleep for.
Kept OURS, not upstream's, on the timeout: upstream restarts its clock only
when the (size, command) pair changes, which drops a peer sending one big
message slowly but honestly. This times the STALL -- the clock restarts on any
progress -- which is the distinction that matters on a slow link.
Also from #109: a header could announce 256 MB while serialize.h refuses
anything over MAX_SIZE (32 MB), so the check admitted an allocation the parser
would reject anyway and burned a peer slot for two minutes doing it.
MAX_PROTOCOL_MESSAGE_SIZE ties the two together. 8x less a peer can make us
hold, and this node takes public Stratum.
And #103's other half: ~CNode closed its socket with a raw closesocket(),
invisible to the accounting, and left hSocket set afterwards -- which is how a
descriptor number that now belongs to somebody else gets closed a second time.
(cherry picked from commit 2f25e2e07985e9bc732cadfbb3c9ac233895a814)
|
|
CheckBlock() validated block size against MAX_SIZE, the serializer's generic
32 MB guard, not a block rule. MAX_BLOCK_SIZE did not exist in this tree at
all: Bitcoin's 1 MB cap arrived in 0.3.x, after the 0.1.0 this fork came from.
A miner could produce a 32 MB block and every node would download, verify and
store it. Upstream #111 (from #110 by mtg0001).
The cap is a tightening, so a node that does not upgrade is the permissive one.
Measured upstream over 5709 blocks: largest 34,290 bytes, median 216, so 1 MB
rejects nothing that exists.
Upstream shipped the validation half only. Both places that ASSEMBLE a block
still filled to MAX_SIZE/2 (16 MB), which was consistent against a 32 MB
validation cap and is inverted by a 1 MB one -- the node would solve a block
and then reject it itself. Fixed both:
main.cpp CreateNewBlock, the node's own miner
rpc.cpp RebuildJob, the POOL's job builder -- the one that matters here,
because every miner on the pool would be working a template the
network rejects, and this pool is ~90% of hashrate
Half the cap in both, preserving the original margin: each test runs against a
transaction that is already counted, so the loop can overshoot by one.
(cherry picked from commit a3be9f04903447fa2741aff4eb529fe45f916877)
|
|
From the first of two wedges captured on the pool. It is a different
proximate fault from the second one, and this is the part that turns a
single bad socket into a node-wide outage.
A failed select() zeroes both descriptor sets, so no peer is serviced on
that pass. That is correct as far as it goes. What was missing is that
nothing stopped the offending descriptor coming straight back:
- the invalid socket was only marked fDisconnect, and a node is not
removed from vNodes until it has drained or DISCONNECT_DRAIN_SECS has
elapsed -- and it cannot drain, because its socket is gone;
- the select set was rebuilt from vNodes with no check, so the same dead
descriptor went back in on the very next pass and failed select()
again.
Measured in the capture: 1,027 consecutive 'its socket is no longer valid'
passes across 61 seconds -- the full drain timeout -- one bad socket, and
for the whole of it the node read nothing from anybody. The peer that
caused it had 57 bytes unsent, so it took the slow path out.
Now the descriptor is forgotten as soon as getsockopt says it is not ours:
select() succeeds again on the next pass and every other peer is served
normally while the dead node ages out as before.
Deliberately forgotten rather than closed. getsockopt has just said this
descriptor is no longer ours; calling close() on it is precisely how an
unrelated socket gets closed instead, which is the fault fixed in e742d0b.
(cherry picked from commit f14c513b4e39d6203b73584f6dc01acbb5a9546e)
|
|
Found auditing the previous commit for the push. Timing from the first
rewind would drop a peer whose message is large and merely slow, which
would be a fault of our own making rather than one we fixed.
Restart the clock whenever the buffer has grown by even one byte since the
last look. A transfer still making progress is never dropped however long
it takes; only one that has stopped growing while the node goes on
expecting more is, which is the actual failure.
(cherry picked from commit 5dddb26bae02013ce87719b11953aadf0bfa2594)
|
|
THIS IS THE WEDGE, and the note asking for this fix has been in the file
since Bitcoin 0.1: 'need a mechanism to give up waiting for overlong
message size error'.
ProcessMessages reads a header, finds the body has not all arrived, pushes
the header back onto the front of vRecv and breaks. Correct for a message
still in flight. For one that never completes it is permanent: every later
pass re-reads the same header and breaks in the same place, so that peer is
never parsed again for the life of the process -- no blocks, no inv, not
even its version message.
Nothing rescues it:
- a header may claim up to 256 MB (0x10000000) and still pass
CMessageHeader::IsValid(), so a garbage or truncated header is accepted
and then waited on forever;
- nLastRecv is refreshed by any byte that arrives (net.cpp), whether or
not it parses, so the inactivity timeout that drops a silent peer never
fires on a peer that is merely unreadable;
- the socket stays ESTABLISHED and select() keeps offering it, so nothing
at the socket layer notices either.
It also explains the signature that made no sense. A peer stuck before its
version message is parsed keeps nStartingHeight == -1, so a node with a full
vNodes still reports GetPeerMedianHeight() == -1 -- which is exactly what
was observed, and exactly what disabled the pool's behind-the-network gate
and let it fork.
Evidence, from two wedges captured on the pool by the watchdog:
MESSAGE-BREAK 2 ran 2014 times in the deaf window against 607 in the healthy
window before it, while blocks received went to zero and the node kept
processing miner traffic normally throughout.
Now: a peer that announces a body and does not deliver it within two minutes
is dropped, and says so in the log unconditionally. Two minutes is far
beyond any legitimate header-to-body gap and well inside
BTF_RECV_TIMEOUT_SECS.
(cherry picked from commit ca1e99288e03ac7beb2b97cf615a3255b4635d95)
|
|
THE WEDGE.
CNode::Disconnect() closed hSocket and left the member holding the stale
number. ~CNode's only guard is 'hSocket != INVALID_SOCKET' and nothing in
the tree ever assigned that sentinel, so the destructor closed the same
number a second time.
The two closes are five minutes apart by design: ThreadSocketHandler2
disconnects a node, sets nReleaseTime to GetTime() + 5*60, and parks it in
vNodesDisconnected; GetRefCount() reports it busy until that expires, so
'delete pnode' -- and the second close -- cannot run before then.
Linux hands out the lowest free descriptor. Measured on the pool: 202 open
descriptors with a high-water mark of 216, so a freed number is reissued
within seconds -- to a Nostr relay TLS socket, a rendezvous socket, a
tunnel end, or a miner connection. Five minutes later the destructor
closes it. The victim gets no error and no log line. It simply stops.
That is the whole signature, including the part that never made sense:
.btf block delivery and Nostr descriptor lookup share no lock, no queue
and no thread, so they should not fail together. They share the file
descriptor table, and a stray close is drawn from all of it.
It also explains why #78 did not help and may have hurt. #78 drops stuck
nodes -- it fired 41 times in one window -- and every drop is another
Disconnect(), which is another delayed close armed against an innocent
socket.
Second bug, same symptom, different subsystem: the pooled relay sockets.
ResolveDescriptor returned the instant a descriptor verified, leaving the
relay's EOSE unread on a socket that goes straight back into g_relayPool,
so the next caller read our leftovers as its own answer -- permanently one
message out of step, and no code path anywhere inspected the subscription
id that would have caught it. Worse, BtfResolve set lease.fOk = true on
every failure path including a dead RecvText, labelling it 'exchange
completed, just no match'. fOk = false is the only thing that ever closes
a pooled socket or clears fConnected, so a relay connection that died was
pinned in the pool, reported healthy, and reused for the life of the
process. That is 'no Nostr descriptor found' for a descriptor that nak can
see, cleared only by a restart.
Now: read to EOSE before returning, match the subscription id, and keep a
pooled socket only when the relay actually finished the conversation.
(cherry picked from commit ed4640c306cc46dbdbd92cb6dc55e7528651269c)
|
|
The peer-height gate replaced a tip-age check that was wrong in both
directions. It is a strict improvement and it is still blind in the one
case that matters most.
When the node goes deaf -- peers still in vNodes, nothing arriving -- no
peer announces anything, GetPeerMedianHeight() returns -1, and the
fallback mines our best chain. That fallback is right for a node that has
not been told yet and wrong for a node that has stopped being able to
hear, and a height nobody sent cannot tell those apart. A wedged node is
exactly a node with no peer height information, so the gate was silent
for the whole of heights 3966-4085: 120 blocks, 5890.50 BTF voided.
Time can tell them apart. GetPeerHeightAgeSecs() measures how long since
any peer reported a height, counting from first use rather than the epoch
so a starting node does not refuse jobs before its first handshake. Past
20 minutes the pool stops issuing jobs and says so unconditionally in the
log. /poolmineblind overrides it: without an escape hatch a genuinely
isolated pool that is 90% of the hashrate could never restart its own
chain, which is the deadlock the tip-age check used to cause.
Also stop nStartingHeight from going stale. It was a snapshot taken once
during the version handshake and never touched again, so on a long-lived
connection the median is built from heights the peers had hours ago, and
the behind-the-network check drifts toward 'everyone is behind us' and
quietly stops firing. A peer that hands us a block is demonstrably at
least at that block's height.
(cherry picked from commit 0800c18faf703b4acde1b131d0b742d37a86d92b)
|
|
The sync gate compared the tip timestamp against 30 minutes. That cannot
distinguish "we are not synced" from "the network is slow", and it failed
in both directions on the same day.
A pool mining its own fork keeps its tip fresh, so the check passed for
the whole of heights 3966-4085 -- 120 consecutive blocks, all orphaned,
5890.50 BTF of payouts voided, and nothing in the log to say so. Then,
once the pool stopped finding blocks, the same check refused to issue
jobs against a tip that was the network's own best. The pool is ~90% of
the hashrate, so nobody else could move the chain either: stale tip, no
jobs; no jobs, stale tip.
Gate on the height peers announce in their version message instead.
Median, so one peer cannot stall us with an absurd claim, and a block of
slack to match WarnIfBehind. Peers predating the announcement report -1
and are ignored; with no reports at all we mine our best chain, which is
what the node does anyway without a pool attached.
(cherry picked from commit 4e76e34f37a82c0885f75db7dfb9e49db410171d)
|
|
The exec queue was memory-only, and FlushMaturePayouts erased a matured entry
from pending_payouts.json *before* handing it over. Any restart in that window
lost those payments with no record that anyone was owed anything. It was not
hypothetical: a container restart on 2026-07-30 silently dropped a backlog of
~1,156 BTF, and there was nothing left to reconstruct it from.
Re-queueing blindly would have double-paid whoever had already been sent to,
so the payment needs to be idempotent rather than merely persistent.
In-flight work is now a persisted list of per-payment records
(paying_payouts.json), one per (block, address) -- the granularity SendMoney
works at, and therefore the granularity at which a payment can be half-done
when the process dies. Records are written, via a temp file and rename so a
crash cannot truncate them, *before* the ledger entry is erased, so the debt
is always described by at least one of the two files. Each attempt updates the
record (retries, deferrals, backoff); settling it removes the record and
rewrites the file, which is what stops a restart from paying twice.
On startup ReconcilePayouts runs before the payout thread:
- a record whose payment the wallet already shows -- matched on the payout
comment plus the exact amount and destination -- is dropped, not resent.
SendMoney commits the transaction to the wallet before it returns, so a
crash immediately after sending is recoverable.
- a ledger entry that has already been exploded into records is dropped,
which is the crash-in-the-window case where both files describe one debt.
The wallet check is deliberately biased towards "already paid": skipping a
payment is visible in the log and recoverable, paying twice is neither.
Verified live: sampling paying_payouts.json during a payout burst caught 3
records / 18.56 BTF mid-flight, cleared once paid, and the paid lines now
carry the txid.
(cherry picked from commit d0248f2e65dbe745f2e9b1d09e1f53a4354d3640)
|
|
Two fixes.
uint256's default constructor is empty (uint256.h:514), so the blockHash
field added in 6963862 was uninitialised garbage on every PendingPayout
built without an explicit hash -- which is every entry loaded from a ledger
written before that commit. Those entries were meant to be grandfathered
past the orphan check on a zero hash; instead each carried a random value
that matched no block in mapBlockIndex, and the first maturity sweep after
deploy voided the whole loaded ledger: 100 entries, 4950 BTF. Only one
"was orphaned -- voiding" line reached the log because LogPrint collapsed
the other 99 as duplicates, so the destruction was nearly silent. Nothing
had been paid at that point and the file was restored from a pre-deploy
backup, so no obligation was lost. Assigning 0 explicitly is all that was
missing.
Separately, four of the twelve relays are dead. Probed all twelve from two
independent networks (a residential line and the pool's VPS) over v4 and v6,
with DNS confirmed against 1.1.1.1:
relay.current.fyi, relay.nostr.bg -- NXDOMAIN, no A and no AAAA. The
domains are gone. (A getaddrinfo call on the VPS reports an address for
them, but that is the VPS's own IPv6 handed back by its local resolver,
not a real host.)
relay.nostr.band, nostr.fmt.wiz.biz -- DNS resolves and TCP connects, but
the TLS handshake never completes, from either network. Whether they are
down or refusing non-browser clients, this node cannot use them.
The remaining eight all complete a WebSocket upgrade. A dead relay is not
free: each holds a connect/retry slot and delays discovery for no added
redundancy.
(cherry picked from commit 144bc96f150d23507d57d29cb339f177dcd6b092)
|
|
A 280-block reorg (heights 3075-3354) orphaned 292 of the pool's 707 blocks
and destroyed 14,600 BTF of coinbase. The payout ledger did not notice: it
records only matureAtHeight, and a reorg replaces the block at a height
without changing the height, so ~14.4k BTF of obligations outlived the coin
that was supposed to fund them. The pool then paid fork-era miners out of
real-chain coin until the wallet was empty -- 17,827 BTF paid against 17,900
BTF of matured supply, leaving under 1.1 BTF spendable and every subsequent
payment failing.
Three changes, all in the payout path:
- Record the funding block's hash and height, persist both, and void an
obligation whose block is no longer in the main chain. Checked both at
maturity and again in the exec queue, since an entry can sit there for
hours across retries and the reorg can land at any point in that window.
Entries loaded from a file written before this are grandfathered rather
than voided on a missing field.
- Stop spending the retry budget on maturity. SelectCoins failing for want
of spendable coin is not an error, but it consumed one of 1440 retries
every 60s, so a payment the pool fully intended to honour would hit
GIVING UP 24h later and the miner would silently never be paid. Those
attempts now defer without incrementing retries, logged hourly rather
than every minute. GIVING UP now means what it says: 1440 real failures
against a block still on-chain with the money available.
- Schedule at COINBASE_SPENDABLE_DEPTH instead of COINBASE_MATURITY. At
depth 100 a block's own coinbase is 20 blocks short of spendable, so its
payout had to come from unrelated mature coin and the pool carried a
permanent ~19-block float. The "96% of 934 payments went through at depth
100" note that justified the shorter wait was measured while this node was
alone on a private fork paying itself. The extra 20 blocks also put the
maturity gate past the reorg that caused all of the above.
(cherry picked from commit 696386220a3d2fac88cb8045931392f4909e0985)
|
|
SendMoney's "oversized transaction that requires a transaction fee of
0.00" is not an oversized transaction and there is no fee. That branch is
taken when nValue + nFeeRequired > GetBalance() -- an empty spendable
balance -- and the fee reads 0.00 because CreateTransaction failed before
setting one. 3,267 of these in debug.log sent me looking for a
transaction-size problem that does not exist. Report what is short, by how
much, and against what balance.
Name COINBASE_SPENDABLE_DEPTH in main.h and derive GetBlocksToMaturity
from it, because the distinction it captures is genuinely surprising:
CMerkleTx::GetCredit values a mined coinbase at zero until
COINBASE_MATURITY+20 deep, not COINBASE_MATURITY.
I first changed the payout gate to that deeper value, on the theory that
each payout was firing 20 blocks before its own coinbase could be spent.
Measuring it refuted that: across 934 payments, 96% succeeded at depth
100, the moment they were queued, which is only possible because a payout
is funded from whatever coin is already mature rather than from its own
block. The gate stays at COINBASE_MATURITY, with a comment recording the
measurement so the next person does not re-make the same change.
The real shortfall is structural and not addressed here: the pool pays out
~99% of each block's subsidy while only matured coinbases are spendable,
so it runs with almost no buffer, and any orphaned block it queued a
payout for drains that buffer permanently. The 24h retry loop absorbs this
-- 0 payouts forfeited -- but the buffer is what needs fixing.
(cherry picked from commit e28df01f90d621b0cd818f28d70a20b8ab5d1a8d)
|
|
Upstream #30 split rendezvous registration from waiting for a dial, so
RvServiceRegister now returns as soon as the relay lists the service and
no longer consumes the relay's 0x01 pairing byte. It updated its own
caller in net.cpp and left this one behind.
AcceptOneFn went straight from RvServiceRegister to BtfServiceWrap, so
BtfServiceWrap read that 0x01 as the first byte of the miner's 32-byte
ephemeral key and every byte after it was shifted by one. The X25519
handshake has no confirmation step, so it completed against a garbage
shared secret, the tunnel was handed to the Stratum event loop, and
nothing failed loudly: the pool logged a miner arriving and then dropped
it as "never authorized", while the miner logged a pool that closed on it
with gotSubscribeAck=0. Loopback miners were unaffected, which is exactly
what made it look like a relay problem.
Block in RvServiceWaitPaired before wrapping, and close and re-register if
the relay drops us first.
test_rvaccept.cpp covers both shapes against a real bitflashd on loopback:
the pre-fix order establishes a tunnel that carries no bytes, the fixed
order round-trips. Also corrects the btftunnel.h contract comment, which
still described the pre-#30 behaviour and is what the old code was written
against -- upstream's own test_relayproc.cpp, test_btftunnel.cpp and
test_btfrv.cpp are still on that stale contract and fail the same way.
(cherry picked from commit 3e3a36b2bf1e6ffb4861507089b743688619377f)
|
|
SSL_CTX_new leaves verification at SSL_VERIFY_NONE and loads no trust
store, and nothing here ever changed that. SNI was set, a certificate was
received, and it was then ignored: every wss:// relay connection this node
has ever made would have completed against an attacker's certificate.
Signed events limit the damage -- a MITM cannot forge an announcement
attributable to our key -- but it can drop the rendezvous registration so
miners never find the pool, censor subscriptions, or answer them with
arbitrary content, which is the input the allocation bounds exist to
contain.
Load the system trust store, require SSL_VERIFY_PEER, and bind the check
to the host we dialed with SSL_set1_host, since chain validation alone
would accept any certificate a trusted CA ever issued for any name. Floor
the protocol at TLS 1.2. Verification failures report the specific X509
reason rather than a generic handshake error, and a missing CA store is a
hard failure instead of a silent downgrade to trusting everything.
(cherry picked from commit 6020740e68463deb1cc3c5b477372fcd1a245adb)
|
|
The min-2-threads floor applied even when the user explicitly asked
for 1 thread, so /hashthreads=1 silently spawned 2.
(cherry picked from commit a67b9e84f4f7607d307081b4a9873f5d9e78cf8a)
|
|
Descriptor (38501) still published so miners can resolve the pool;
only the public pool listing event (38503) is suppressed.
(cherry picked from commit 17bbdae1ec97945c37c1d5b2486e5c01a4d7ad87)
|
|
A pool that just found its first blocks has zero mature balance until
100 blocks later; orphaned coinbases (observed: reorgs orphaned our
blocks 2651-2655) make that worse. 30 one-minute retries forfeited
payments that would have succeeded hours later.
(cherry picked from commit 49e80023ca23baee8a4b80cfb891e7a358dc39ba)
|
|
A pool started with /rvrelay=ONE still spawned an accept thread for every
Nostr-discovered relay (BtfAllRelays always merged g_discoveredRelays), and
each thread optimistically overwrote the advertised meeting node before
registering -- so a discovered Tailscale-only relay (a mesh host)
flapped the published .btf descriptor and public miners could not connect.
fBtfRelayPinned (set only when /rvrelay parses) makes BtfAllRelays return
exactly the pinned list. Also validate the /rvrelay value at parse time:
with the pin in place no discovered relay can mask a typo.
(cherry picked from commit 0a4d2b02fa5382c5e788b5f0e2f8a844695dec47)
|
|
- RebuildJob: no jobs with zero peers or a tip older than 30min;
mining unsynced produced orphaned blocks and payouts that can
never be paid (observed in production: 8 orphan 'found blocks')
- payout exec: on SendMoney failure requeue with 60s backoff, up to
30 attempts, instead of silently forfeiting the payment
- nostr: add oxtr.dev, nos.social, fmt.wiz.biz, current.fyi, nostr.bg
(12 relays total) for announcement redundancy
(cherry picked from commit f1986cd7e4a6f69469a2256eb96ff03f3f146792)
|
|
app icon)
|
|
BITFLASH_NODE_BUILD)
|
|
poll-socket-handler into main
|
|
# Conflicts:
# src/makefile.mingw
|
|
tor-mode into main
|
|
socks5-proxy into main
|
|
tool-fixtures into main
|
|
|
|
|
|
tooling-hardening into main
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
parsemoney-full-decimals into main
|
|
The legacy parser only accepted two decimal places (cents) and rejected
0.5, leaving six of the eight coin decimals unreachable by -sendto and the
send window. Parse up to 8 fractional digits Bitcoin-style, reject >8 to
avoid silent truncation. Adds the parse-money selftest (12 cases).
|
|
naming' (#22) from slip44-official-naming into main
|
|
The coin type 4346950 (0x425446 = BTF) was merged into the SLIP-0044
registry on 2026-08-07. Rename HD_BIP44_COIN_TYPE_BITFLASH_PROVISIONAL to
HD_BIP44_COIN_TYPE_BITFLASH and update comments, logs and UI strings.
Value is unchanged, so derivation paths and addresses are identical.
|
|
utxo-inclusion-proofs into main
|
|
|
|
from utxo-set-commitment into main
|
|
mining-calculator into main
|
|
|
|
|
|
fair-launch-verifier into main
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The headless loop installed SIGINT/SIGTERM handlers only under #ifndef _WIN32,
so a headless node on Windows never reached the clean StopNode()+DBFlush() path:
Ctrl-C, closing the console window, a service stop and OS shutdown all killed it
mid-write, and a Berkeley DB wallet.dat left unflushed will not reopen.
Install a SetConsoleCtrlHandler that mirrors the POSIX path: set fShutdown and,
because the OS terminates us shortly after the handler returns on
CLOSE/LOGOFF/SHUTDOWN, block until the main loop signals the flush is done.
A 'taskkill /F' is still an uncatchable hard kill by design.
Closes #16.
|
|
|
|
setint64() extracted big-endian bytes with a signed left shift (n <<= 8).
When the top byte is set that overflows int64 -- undefined behaviour, which
the script fuzzer flagged under UBSan (bignum.h:158). The result was correct
on two's-complement hardware, so this is a latent UB inherited from the
original 0.1 code, not a behaviour change. Do the shift on a uint64.
|
|
A pool's dashboard URL comes from its self-published status file. Escaping
it for HTML is not enough: a javascript: or data: scheme would still render
as a clickable script-injection on the public directory page. Only emit a
link for http(s) schemes; anything else shows as no link.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
verify-release.sh embeds the Bitflash release signing key and now requires that
the checksum signature validate against that exact key, not merely against some
key in the caller's keyring (#138). It also fetches releases from
releases.bitflash.network instead of the retired GitHub host, resolving 'latest'
from latest.txt.
Release signing key fingerprint:
910A 2B4C CA87 9E81 FB4B 2AEA 1D4A 53D3 B78A A4B8
|
|
* fuzz: add P2P message parser harness
* fuzz: avoid duplicate legacy header includes
* fuzz: seed the corpus with our network magic, not Bitcoin's
The two seed inputs began f5 be b6 d9, which is Bitcoin's message
start. Ours is bf 20 5c fd. CMessageHeader::IsValid rejects on that
field first, so both seeds died at the first check and the smoke run
exercised the reject path and nothing else.
libFuzzer would have to mutate its way onto four exact bytes before
reaching any parser, which does not happen in 128 runs. The job would
have gone green while covering none of what it was added to cover.
---------
Co-authored-by: Gilcimar Duarte <ti@gpm.infra>
Co-authored-by: Bitflash-sh <btf01@proton.me>
|
|
Co-authored-by: Gilcimar Duarte <ti@gpm.infra>
|
|
Co-authored-by: Gilcimar Duarte <ti@gpm.infra>
|
|
Two bugs in RebuildJob(), the operator's Stratum template builder. It
is a separate path from BitcoinMiner() and had drifted from it.
A single pass over mapTransactions is not enough. That map is keyed by
hash and carries no dependency order, so a transaction spending another
still-unconfirmed transaction's output -- change from a recent send,
usually -- fails ConnectInputs whenever its parent sits later in map
order. One pass drops it, and map order does not change between
rebuilds, so every later rebuild drops it at the same point. That is
deterministic exclusion rather than bad luck: where pool operators mine
most blocks, a transaction of that shape can stay unconfirmed for as
long as its parent is. Now it retries until a pass adds nothing, which
is what the solo builder has always done.
And the fees reached nobody. nFees was declared inside the scan loop, so
each transaction reset it and the running total was discarded, while the
coinbase line passed a hardcoded 0. An operator collected fees into a
block and paid none of them out. The block stayed valid, since paying
less than allowed always is, so nothing ever complained.
Both the diagnosis and the fix are Vic-Nas's, from his fork of this
code. He deleted the fork, so this brings the work back rather than
letting it go with it. His description of the failure was precise enough
that finding the same two defects here took one read of our RebuildJob.
|
|
docs/ has held the README's images since the fair-launch release. I
added doc/ yesterday for the derivation spec without looking, and #108
then correctly used docs/ -- leaving the repository with both, one
letter apart, which is the kind of thing that quietly costs someone ten
minutes a year forever.
The mistake was mine, so the move is mine: derivation.md and
verify-derivation.py go to docs/, and the two README links follow.
No content changes. The verifier still reproduces the wallet from its
new path, which is the only thing here that can actually break.
|
|
Co-authored-by: Gilcimar Duarte <ti@gpm.infra>
|
|
Every fatal startup path wrote to stderr and to printf, and on Windows
neither reaches anybody. printf is remapped to OutputDebugStringF, so it
lands in debug.log inside the data directory. A -mwindows binary started
by double-clicking has no stderr at all, and AttachTerminal() cannot
help: it attaches to a parent console, which a double-clicked exe does
not have.
So Bitflash exited silently. Someone whose wallet.dat needs a newer
build -- exactly what the wallet format guard was added to tell them --
double-clicked and watched nothing happen. The guard was correct and
mute, which is the failure it was written to prevent.
Fatal startup errors now go through one reporter: debug.log as before,
stderr as before, and a message box when there is a person in front of
the screen. Applied to the three that mean "your data will not open":
the block index failing to load, the block index throwing, and
wallet.dat failing or throwing.
The box is deliberately not shown for a headless node. Those run under
scheduled tasks and services, where a modal dialog waits forever for a
click nobody is there to make -- a hung process instead of an exit code.
Headless keeps stderr and the log.
LoadWallet() now also reports why it refused, in a sentence, instead of
only logging it. The unsupported-format case is the one that has to be
understood by whoever reads it, and "LoadWallet failed" is not that.
Verified by breaking a throwaway wallet on purpose and starting both
ways. Headless: exit code 1, no dialog, no process left behind. Windowed
with no console: a box naming the file, the Berkeley DB error, the usual
cause, and where to look for more.
|
|
(#134)
* wallet: refuse future encrypted wallet formats
* wallet: add encrypted key unlock core
* wallet: add encryptwallet rewrite command
* wallet: read passphrases from stdin or files
* wallet: keep the derivation across encryption, and take the logs with it
Two things stopped the encryption stack from being safe to merge. Both
were found against a wallet holding real coin, not in the self-tests.
1. Encrypting turned a BIP44 wallet into a legacy one.
LoadWallet() ends by resetting the schema and the derivation counters
when no seed is in memory. That was written when "no seed after loading"
could only mean the wallet had none. An encrypted wallet has a seed --
it just has not been decrypted yet -- so the reset threw away metadata
that had been read correctly a moment earlier. Unlocking later restores
the seed and nothing restores the rest.
Measured on the wallet from the #124 proof: after encrypting, index 1
moved off BAtgSgra2X8Dqo5PJzPCLJhPwrrC2thYVJ, the address actually
holding 1.00 BTF, and the audit went from "1.00 covered by the phrase"
to "0.00 covered, 1.00 wallet.dat-only". Encrypt your wallet and the
program told you your recovery phrase no longer covered your money.
Now the condition also accepts a wallet whose seed is present but
encrypted. Same wallet, before and after: schema bip44, same four
derived addresses, 1.00 BTF still reported as covered.
2. The plaintext keys stayed behind in the Berkeley DB logs.
wallet.dat comes out clean -- verified by scanning the raw file for the
118 bytes of a known private key. database/log.0000000002 did not: it
held the whole key, and still did after a full node restart, because
every record this wallet ever wrote passed through the log.
That defeats the feature for the case it exists for, since our own
backup advice is to copy the whole directory rather than wallet.dat
alone. log_archive() cannot fix it: it only drops logs Berkeley DB
considers obsolete and never the one it is writing to, and with one 10
MB log per directory that is precisely where the keys are. They also did
not get there at encryption time -- they accumulated over the wallet's
whole life -- so nothing done during the rewrite can archive them out.
So the encrypt command now closes the environment and deletes the logs
on its way out, after the flush that makes every .dat self-contained.
This is not secure erasure; the blocks are unlinked, not overwritten.
It closes the case of someone reading a copied data directory.
Getting there uncovered a third bug, older than both.
DBFlush(true) is not safe to call twice. The first call closes the
environment, and txn_checkpoint() on a closed environment segfaults
instead of returning an error, so the try/catch around it never sees
anything. EncryptWallet() already flushes before renaming the wallet,
and the command flushes again on the way out -- that pair crashed every
single time. Guarded now.
The same function has been calling log_archive() with the address of an
uninitialised pointer since 0.1.0, on every clean shutdown, in a path
whose catch(...) cannot catch a structured fault on Windows.
DB_ARCH_REMOVE requires that argument to be NULL. Fixed while here,
because the encryption cleanup depends on that call not being a coin
flip.
---------
Co-authored-by: Gilcimar Duarte <ti@gpm.infra>
|
|
The twelve words were only as good as this binary staying alive. The
derivation path, the address version byte and the public-key encoding
existed nowhere outside src/, so anyone holding a phrase and no working
Bitflash had the keys and no way to find them.
doc/derivation.md states all of it: BIP-39 to seed, BIP-32 to children,
both layouts this wallet has used (legacy m/i' and BIP-44
m/44'/4346950'/0'/{0,1}/i), secp256k1 with UNCOMPRESSED public keys, and
Base58Check with version byte 25.
The uncompressed part is the trap worth naming. Hashing the same key in
compressed form yields a different, perfectly valid-looking address that
holds nothing, and nothing in the source says out loud which one is
used -- it falls out of OpenSSL's default in code inherited from 0.1.0.
doc/verify-derivation.py implements the document from scratch, sharing
no code with this project and importing nothing outside the standard
library, then checks it against addresses the wallet actually produced.
Both tables were generated by a binary first: the legacy one by the
published v1.2.15, which knows only that layout, and the BIP-44 one by
the current tree. The script reproduces all eight.
That is the point of shipping the checker rather than only the prose: if
the two ever drift, the script fails and says which side is wrong.
|
|
The right icon has been in the exe the whole time. GLFW never found it.
GLFW registers its own window class and fills hIcon with
LoadImageW(module, L"GLFW_ICON", ...), falling back to IDI_APPLICATION
when that exact resource name is missing. Ours was named "bitflash", so
the lookup failed and every window this program has opened wore the
generic Windows icon -- in the title bar and in the taskbar -- while the
correct artwork sat unused a few kilobytes away.
One line, verified by looking at it: rebuilt, ran the window, and zoomed
in on both places the user pointed at. The title bar and the taskbar
button now show the green BF coin. Checked the shell path too, since it
resolves icons differently: ExtractAssociatedIcon on the new exe still
returns the coin, so renaming the resource did not cost the file icon in
Explorer.
|
|
* Add BIP32 path derivation foundation
* Add HD wallet schema metadata
* Add provisional BIP44 coin type metadata
* Activate BIP44 wallet derivation
* wallet: scan BIP44 restore depth per branch
* wallet: widen default BIP44 restore lookahead
---------
Co-authored-by: Gilcimar Duarte <ti@gpm.infra>
|
|
|
|
CmdSendTo() has been in walletcmd.cpp since #115 and nothing declared
it or called it. The flag was dead: `bitflash -sendto=ADDRESS,AMOUNT`
started a node and said nothing, because an unrecognised argument is
not an error here.
The v1.2.14 notes announce it under "A headless node can finally
spend", and the headless Linux binary is exactly the one that does not
have it -- `-help` there lists /newaddress and stops. The published
Windows binary does have it, because it was built from a working tree
that carried these two files uncommitted. So the artifacts disagree
with each other and neither matches the tag.
This adds only what was missing: the declaration in walletcmd.h, the
dispatch in main(), and the -help line. No behaviour change to
CmdSendTo itself.
Checked on a build of this branch, against a throwaway data directory:
-help lists /sendto=ADDRESS,AMOUNT
-sendto=NOTANADDRESS,1 "Not a valid address", exit 1
-sendto=<valid>,-5 "Not a valid amount", exit 1
-sendto=<valid>,999 refused on an empty wallet, exit 1
All five self-tests still pass.
|
|
Co-authored-by: Gilcimar Duarte <ti@gpm.infra>
|
|
Creating a recovery phrase from the window aborted with an ImGui
assertion on the success path:
Expression: (g.DisabledStackSize > 0) && "Calling EndDisabled() too many times!"
File: imgui/imgui.cpp Line: 8435
The Finish button is wrapped in BeginDisabled/EndDisabled guarded by
!g_phraseWrittenDown, and its own click handler sets g_phraseWrittenDown
to false. So the pair was opened while the checkbox was ticked and
closed after the handler had unticked it: BeginDisabled was never
called, EndDisabled was, and ImGui asserted on the unmatched close.
It fires on the only path that installs a seed -- tick the box, press
Finish -- so every phrase created from the window has hit it since the
feature shipped in 1.2.12. Three releases carried it because nothing
tests the window.
Reproduced on the published 1.2.14 binary and confirmed fixed on this
build, driving the real window: Wallet Safety -> Create recovery phrase
-> tick -> Finish. The old build raises the assertion; the fixed build
closes the dialog, reports "This wallet has a recovery phrase", and the
address changes to a derived one.
The seed itself was already written before the assertion, so the wallet
was never left half-installed -- and ImGui's IM_ASSERT_USER_ERROR_RET
returns before touching its state, so answering Ignore was safe. Abort
was not: it kills the process with no DBFlush.
The restore dialog next door already does this correctly, copying the
condition into a local first. This makes the create dialog match.
|
|
A block had no size limit, spending could move coins onto a key your recovery phrase cannot reproduce, and one peer could hold up the whole node with a message it never finished sending. All three are fixed here, and all three were found by auditing the tree rather than by anyone hitting them.
## Blocks now have a size limit — and this one is consensus
`CheckBlock()` measured a block against `MAX_SIZE`, the serializer's generic 32 MB guard. There was no `MAX_BLOCK_SIZE` anywhere in the tree: Bitcoin's 1 MB cap arrived in 0.3.x, after the 0.1.0 this project came from, so the inheritance was simply missing. A miner could produce a 32 MB block and every node would download, verify and store it.
The cap is now **1 MB**, chosen against the chain rather than tradition. Reading all 5709 blocks out of `blk0001.dat`:
| | |
|---|---|
| largest block ever mined | 34,290 bytes |
| median | 216 bytes |
| p99 | 14,451 bytes |
1 MB leaves **29× of headroom** over the largest block this chain has ever carried, so nothing existing is rejected. A node built with the rule then synced genesis → 5706 on the live network with zero rejections, which is the same test that once caught BIP30 rejecting a real block at height 859.
**Because it is consensus, it matters that you upgrade.** Until most nodes have it, a block above 1 MB would be accepted by old nodes and rejected by new ones — which is how a chain splits. Nobody is mining such blocks today; the gap is worth closing before somebody does.
## Spending could put your change on a key the phrase cannot recover
When a wallet spends, the leftover goes back to you as *change*. This tree sent change to **the same key as one of the coins being spent** — 0.1.0 behaviour, and a privacy problem long tracked as #23.
With recovery phrases it became a way to lose money. If the coin being spent sat on a key created *before* the seed — the wallet's original default address, an imported key, anything older than `-newphrase` — then the change went back to that same pre-seed key. So spending quietly moved your balance onto a key the twelve words do not reproduce, in a wallet you believed the words covered.
Change now comes from a newly derived, phrase-backed key. **If that derivation fails, the transaction is refused rather than created**, because a failed send is recoverable and unrecoverable change is not. Wallets with no seed keep the old behaviour exactly.
Restore also now scans past the pre-generated key-pool window before accepting an empty batch as the end, so a restore cannot stop short while used addresses sit just beyond it.
## One peer could hold up the whole node
`ProcessMessages` runs in a single thread that serves every peer. When a message header announced more bytes than had arrived, the old code consumed the header, inserted it back at the *front* of the receive buffer — an O(n) memmove on a buffer that could hold 256 MB — and then slept 100 ms **inside that shared thread**. A peer that announced a large message and dripped bytes could make every pass pay both costs, forever, while looking perfectly active to the inactivity reaper.
The parser now reads the header from a copy without consuming it, never sleeps in the shared thread, and disconnects a peer that sits on an incomplete message for two minutes. The wire cap drops from 256 MB to `MAX_SIZE`.
## The rendezvous holds more than one registration
A node's registration at a rendezvous relay was consumed by the first caller, and registering again *replaced* the previous one — so a node was reachable to exactly one peer at a time and unreachable for the whole gap between being paired and registering again. Measured on a live node: descriptors resolved 53 times out of 54, its own registrations paired 11 out of 12, and **half of every outbound dial failed**.
All three relays now keep a queue per node and pair with the oldest registration still alive. **This is already deployed** — you get it without upgrading. The matching node-side change, which parks spare registrations, is deliberately **not** in this release: it measured worse in the field than the code it replaced and will not ship until that is explained.
## Restoring from a phrase now looks far enough
Found by proving the change fix above with real coin, and caused by it.
Restoring derived forward in batches of a hundred and stopped at the first batch that turned up nothing. The wallet digs a wider gap than that by itself: a restore leaves the derivation counter at the depth it scanned, the key pool derives a hundred more, and change takes the index after those. So the address used *after* a restore can sit two hundred indices past the last one used before it, with nothing in between.
On a real wallet: receive, restore, spend once — the outputs land at indices 201 and 302. Restoring again reported
```
Restored: 1 transaction(s) across 201 derived addresses
```
and the spend was invisible, both the payment and the change. The money was derived and covered by the phrase; the search stopped short of it. Three quiet batches instead of one, and the same wallet reports `2 transaction(s) across 601 derived addresses`.
Change returning to an input's key used to hide this, because that key sat at a low index the scan had already passed. Fixing recoverability moved change out to where the search stopped looking, so the two had to be fixed together.
The gap is self-inflicted and grows with each restore, so a long enough history can still outrun a fixed number of quiet batches. `-restoredepth=N` looks as far as you ask. The structural fix is tracked separately.
## A headless node can finally spend
`SendMoney()` has been in this tree since 0.1.0 and only the window ever called it, so a node without a screen could be paid and could never pay — it could hold a balance with no way to move it. Found while trying to prove the change fix, which needs a spend.
```
bitflash -sendto=ADDRESS,AMOUNT
```
Refuses an address that does not decode, an amount that is zero, negative or not a number, and a wallet whose chain is not loaded — all before touching the wallet. A transaction created while the node is stopped is not broadcast until the node starts.
## Also
- `-help` printed nothing at all on Windows. This tree remaps `printf` to the debug-log writer, and a `-mwindows` binary has no console, so the one command whose job is to tell you something returned 0 in silence.
- The README described a program from several releases ago — deterministic restore listed as an open issue, mining mode described as forgotten between restarts, and every flag written in the slash form that MSYS2 rewrites into a path.
- Diagnostics now report why a socket close failed, per error code and per site, and flag a handle handed back while some part of the program still claims it.
## Downloads
| File | Platform | Notes |
|------|----------|-------|
| `Bitflash-1.2.14-windows.zip` | Windows x86_64 | Extract and run |
| `Bitflash-1.2.14-x86_64.AppImage` | Linux x86_64 | Desktop/GUI — `chmod +x` and run |
| `bitflash-node-1.2.14-x86_64` | Linux x86_64 | Headless server node, no libGL or FUSE |
Verify with `sha256sum -c SHA256SUMS`.
Everything in [v1.2.13](https://github.com/Bitflash-sh/bitflash/releases/tag/v1.2.13) is included.
|
|
Restoring from a phrase gave up after the first batch of a hundred addresses that turned up nothing, and the wallet digs a wider gap than that by itself: a restore leaves nHDNext at the depth it scanned, the key pool derives KEYPOOL_SIZE more, and change takes the index after those. Measured on a real wallet with real coin -- restore, spend once, and the outputs land at indices 201 and 302; restoring again found only the first transaction and reported the spend as gone. With -restoredepth=500 the same wallet found both, which is what said this was reach and not loss.
Three quiet batches instead of one: 2 transaction(s) across 601 derived addresses on the same wallet, chain and phrase.
Worse after #109, not before it: change used to return to an input's key, a low index the scan had already passed. Making change recoverable moved it out to where the scan stopped looking.
Does not fix the gap growing with every restore -- pulling nHDNext back to the last used index is the structural answer and needs its own testing.
|
|
#113 added three accept threads and called the extras harmless against an un-upgraded relay. They are not: an old relay closes the previous registration whenever the same node registers again, so the threads evict each other in a loop. Measured on the live network -- the node whose registrations landed on the un-upgraded New Jersey relay made 189 registrations in fifteen minutes with zero pairings, unreachable through it and hammering it, while the node on the upgraded relay made 9 with 7 pairings. One relay of three is upgraded, so this would hit two thirds of every node's registrations.
The relay half of #113 stays; it is compatible in both directions and is the prerequisite for raising this again, which the comment now states.
|
|
A registration was consumed by the first caller and registering again replaced the previous one, so a node was reachable to one peer at a time and unreachable for the gap between being paired and registering again. Measured on a live node: descriptors resolved 53/54, our own registrations paired 11/12, nothing paired and went silent -- and half of every outbound dial failed, 15 of 30 in eleven minutes.
The relay keeps a queue per pubkey capped at four, pairs with the oldest still-alive registration and discards the dead; the node runs three accept threads, each holding its own registration. The wire protocol does not change, so either side can be deployed first.
Bench with both relays built from source and waves of concurrent dialers: old+1 65%, new+1 58%, old+3 68%, new+3 98%. Each half alone lands within noise of doing nothing.
|
|
CheckBlock() validated block size against MAX_SIZE, the serializer's generic 32 MB guard rather than a block rule; MAX_BLOCK_SIZE did not exist in this tree at all, because Bitcoin's 1 MB cap arrived in 0.3.x, after the 0.1.0 this fork came from. A miner could produce a 32 MB block and every node would download, verify and store it.
Original work by mtg0001 in #110, rebased over #109 with the self-test conflict resolved, plus a comment explaining why the inherited count-against-bytes test stays.
Measured before choosing the number: 5709 blocks read from blk0001.dat give a largest block of 34,290 bytes and a median of 216, so 1 MB leaves 29x of headroom and rejects nothing that exists. A node built from this branch then synced genesis to 5706 on the live network with zero size-limit rejections and zero CheckBlock errors -- the same test that caught BIP30 rejecting a real block at height 859.
|
|
Change from an HD wallet now goes to a newly derived phrase-backed key instead of reusing a spent input's key. Reusing an input key silently moved value back to a key the phrase cannot reproduce; if derivation fails the transaction is refused rather than creating unrecoverable change. Legacy wallets with no seed keep the 0.1.0 behaviour. Restore also scans past the pre-generated key-pool window before treating an empty batch as the end.
The message parser no longer consumes a header it cannot complete: it reads from a copy, so the O(n) reinsert and the 100 ms sleep inside the shared message thread are gone, and a peer that sits on an incomplete payload is disconnected after two minutes. The wire cap drops from 256 MB to MAX_SIZE. New net-message self-test covers complete, incomplete, stale and oversized headers.
Verified here: clean rebuild, all self-tests pass, lock order checked (no path takes cs_keyPool before cs_mapWallet), and a node built from this branch synced from genesis to the chain tip on the live network with zero rejections.
|
|
The deaf-node paragraph cited the bench A/B -- two nodes on one machine for half an hour. Both mining machines have since run 1.2.13 in production: zero spurious disconnections against the 741 the unpatched side logged, and 13 and 18 peers held where the unpatched node held 11, ten of them inbound. Also says what the pre-1.2.11 socket pile cost and what returns the ports.
|