| Age | Commit message (Collapse) | Author |
|---|
|
Splitting SavePendingPayoutsLocked() into a serialise step that needs
gPayoutLedgerMutex and a write step that does not left the write step with
no serialisation at all. The event loop wrote pending_payouts.json from
QueuePayouts and the orphan-void path with the ledger mutex HELD; the payout
thread wrote it from RecordPayoutTxid and ForfeitPayoutRecipient with the
mutex RELEASED. Both fopen()ed the same "<path>.tmp" with independent handles
and both rename()d it.
Two ways that ends, both live money:
- the renamed file is a mixture of two images. LoadPendingPayouts throws
inside its parse loop, logs one warning, and the node comes up with a
nearly empty ledger -- every obligation not yet exploded into an in-flight
record is simply gone, with nothing on disk saying anyone was owed it.
- a lost update: the payout thread's older snapshot lands after the event
loop's newer one and silently erases a whole round that had just been
queued.
Neither parent had this. Upstream writes under the mutex everywhere; the
pre-merge tree had one writer thread and no ledger mutex. The merge invented
the window while fixing a real problem -- the event loop should not block in
FlushMaturePayouts behind a per-recipient fsync on the payout thread -- so
putting the write back under the ledger mutex is not the fix.
Instead: a per-file write mutex that is never held together with a data
mutex, plus a version on every snapshot.
- serialising happens under the data mutex and stamps a monotonically
increasing sequence number (gPayoutLedgerSeq).
- publishing happens with the data mutex released, under the file mutex,
and drops any blob whose sequence number is not strictly newer than the
one already on disk.
Because each blob is a whole-state dump of a single in-memory structure and
the sequence number is assigned under the mutex guarding that structure, a
higher sequence number is by construction a later state -- so discarding an
older blob loses nothing, and the file can only move forwards. Writers may
queue and complete in any order and the published file is still the newest
state anyone has serialised. A failed write deliberately does not advance the
published sequence, so the next write still tries.
The same treatment is applied to paying_payouts.json and pool_rounds.json.
Neither had the interleaving race -- gPayoutExecMutex and gPoolRoundsMutex
already serialise their writers -- but both did their I/O under a mutex the
event loop takes (FlushMaturePayouts, RecordPoolRound), so both were a place
for the event loop to wait on the payout thread's disk. They are now written
outside those mutexes on the same versioned terms, which is also what makes
adding fsync to these paths safe.
pool_status.json keeps a plain atomic write with no version guard: RefreshStats
is its only caller and only ever runs on the event loop, so it has a single
writer by construction. It shares the same WriteFileAtomic helper so the
durability rules live in one place.
No on-disk format change.
|
|
'make -C src tests' builds src/test_bip32 in the tree and it was showing up as
untracked; src/obj-node/ was only covered incidentally by the *.o rule.
|
|
VERSION names every release artifact -- the AppImage, the windows zip and the
node binary. This tree is upstream v1.2.18 plus 24 commits of our own and a
payout ledger that is neither side's, so a binary built here must not be
mistakable for the upstream release of that number.
|
|
Merges upstream tag v1.2.18 (bc3de30) into our 24 commits. The merge base is
a259215, upstream's own commit, whose src/ tree is byte-identical to 68de575 --
our commits were replayed onto a259215 first so that git had a real base to work
from. That produced 28 conflict hunks in 9 files instead of the 21 conflict
events / 51 hunks a direct cherry-pick onto the tag would have forced, most of
which were artifacts of lifting our commits away from their own predecessors.
No consensus change. main.cpp's GetBlockValue, GetNextWorkRequired,
ConnectInputs, ConnectBlock, CheckBlock, AcceptBlock and AcceptTransaction are
untouched by this merge; script.cpp, serialize.h and randomx_pow.h are
identical on both sides. The frozen-chain oracle reproduces the pinned UTXO root
116a2b6b... byte for byte, and the merged binary loads the same 12,586-block
chain to the same tip aac622f0a3b2f8... with all blocks re-verified and none
rejected.
Which side won, per file and per hunk.
src/rpc.cpp -- 15 hunks.
1. RebuildJob transaction selection. THEIRS for the multi-pass loop and the
nFees fix, OURS for the cap. Their vfAlreadyAdded/fFoundSomething loop lands
dependent transactions that our single pass dropped deterministically, and
their nFees is accumulated across the scan instead of being redeclared inside
it -- ours passed a hardcoded 0 to GetBlockValue, so every pool block built
here discarded its fees. But their bound is MAX_SIZE/2 = 16 MB, and
MAX_BLOCK_SIZE is 1 MB, so their template can exceed what their own
CheckBlock accepts. Cap kept at MAX_BLOCK_SIZE/2, with our inner break so a
single pass cannot overshoot by more than one transaction. Our RebuildJob
sync gating and deaf-node detection are kept; upstream has none.
2. struct PendingPayout. OURS for the key, THEIRS for paidTxids. Their id is
the string "block-<height>", which is height keying -- the exact bug 44849e6
fixed after a 280-block reorg left ~14.4k BTF owed against blocks that no
longer existed. Height keying also lets the crash-recovery probe match the
previous round's transaction after a reorg and silently skip a miner, so
their id is dropped entirely and blockHash stays the key. paidTxids is taken
because it is what makes an entry survivable. Their nextAttempt and our
retryAfter are the same concept under two names; ours is kept and theirs
dropped. matureAtHeight gained an explicit = 0, which neither side had in
our style of initialiser.
3./4. Ledger writer and loader. Mechanical follow-ons of 2. The loader keeps
ours and takes their tolerance for an entry with no usable height, guarded so
it cannot fire on a well-formed entry. One redundancy was removed rather than
merged: their unguarded `if (obj.contains("blockHeight")) get<int>()` sat two
lines above our is_number-guarded version, and an unguarded get<int>() on a
malformed value throws out of the whole loop, losing every entry in the file.
Verified against the live production pending_payouts.json (2 entries, keys
amounts/blockHash/blockHeight/matureAtHeight): loads both. A file in the new
shape, with retryAfter and paidTxids, also loads.
5. Insertion point after the loader. BOTH. Our PayingRecord/gPaying in-flight
ledger and their PoolRoundProof/PoolRoundToJson are disjoint features that
landed at the same offset. Both kept, in full.
6. Writer tail. THEIRS, copied into our writer. Checking fclose() matters here
because fclose is where the buffered write actually reaches the OS: renaming
a short .tmp over a good file is precisely the truncation the atomic write
exists to prevent. The Windows remove()-before-rename() is required because
rename() there will not replace an existing file.
7. Loader bodies. BOTH -- LoadPaying and LoadPoolRounds are different functions
that happened to share a common tail.
8. Function block, 111 vs 36 lines. BOTH, with one deliberate deletion.
PayoutBlockStillOnChain, PayoutComment, RecordPoolRound, MarkPoolRoundPaid
all kept. WalletAlreadyPaid is DELETED and ReconcilePayouts rewritten,
because our reconciliation and their FindRecordedPoolPayout are two answers
to the same crash-recovery question and running both would give the same debt
two authorities. Their wallet probe wins: it matches structured metadata
rather than a comment string, so it can tell two rounds paying the same
address the same amount apart. ReconcilePayouts keeps only the job that is
still real -- dropping in-flight records the ledger already shows paid, and
arming each restored record for its one wallet probe. It no longer touches
gPendingPayouts at all; under the new model dropping ledger entries there
would reintroduce the loss this merge adopts their model to prevent. Their
gPayoutExecMutex/gPayoutThreadRunning re-declarations were dropped as
duplicates of ours, and gPayoutExecQueue was not taken: gPaying is the exec
queue in this tree, at finer granularity.
9. QueuePayouts signature. THEIRS for the round-proof arguments, OURS for the
type of blockHash. It stays a uint256 and is converted to hex only where the
round proof needs text -- a ledger key that has been through ToString() is a
key that can be compared wrong.
10. Maturity policy. OURS. matureAtHeight = blockHeight +
COINBASE_SPENDABLE_DEPTH (120), not + COINBASE_MATURITY (100). At depth 100
the block's own coinbase is still 20 blocks short of spendable, so the
payment must come from unrelated mature coin and the pool runs a permanent
float. Our own history contradicts itself here -- d6972a0 argued for
COINBASE_MATURITY citing "96% of 934 payments", 44849e6 reversed it on the
grounds that the measurement was taken while the node was alone on a private
fork paying itself. HEAD's position is kept. This is a considered
disagreement with upstream, not an oversight. The live production ledger
confirms it: both entries have matureAtHeight - blockHeight = 120.
11. Comment above FlushMaturePayouts. Rewritten to describe the merged design.
12. FlushMaturePayouts, the central difference. THEIRS for the model. The ledger
entry now survives on disk until every recipient has a txid, instead of
being erased as soon as its payments were handed off -- the old ordering
left a window where the debt was described by nothing at all, and a
container restart on 2026-07-30 silently dropped ~1,156 BTF through it. Our
gPaying is kept on top as the in-flight working set, because it carries the
per-recipient retry and deferral state a per-round nextAttempt cannot
express. It is explicitly a cache derived from the ledger, not a second
record of debt: if it were lost the next flush would rebuild it.
13. PayoutThreadFn, 79 vs 70 lines. OURS for the loop and the reorg re-check,
THEIRS for the per-recipient dedupe. The re-check on every attempt is
load-bearing and upstream has no equivalent -- it decides once, at maturity,
and a payment can sit through hours of retries during which the reorg that
kills its funding block can land. Their paidTxids check is taken, against
the ledger rather than against the batch. Our deferral policy is kept:
waiting for coin is not a failure and must not consume the retry budget.
Two fixes upstream does not have are folded in here rather than deferred to
a follow-up commit:
- FindRecordedPoolPayout runs once per record, on its first attempt, not on
every attempt. It is a linear scan of mapWallet under cs_mapWallet, per
recipient; upstream repeats it for every retry, which with 2000 rotating
payout addresses and a budget of 1440 retries is a full wallet scan per
address per minute for a day. The question it answers can only change
across a restart, and every restart re-arms the flag.
- The ledger file write no longer happens under gPayoutLedgerMutex in the
per-recipient loop. SavePendingPayoutsLocked was split into a serialise
step that needs the lock and a write step that does not, so the mutex is
held only long enough to read the structure. The orphan scan in
FlushMaturePayouts was likewise restructured so the ledger mutex is not
held across PayoutBlockStillOnChain's cs_main acquisition.
One case the merged model created and neither side handled: under
survives-until-fully-paid, our GIVING UP path dropped the in-flight record
but left the recipient owed in the ledger forever, so every later flush
would explode it again -- a forfeit that silently becomes an infinite retry.
ForfeitPayoutRecipient writes the amount off so the entry can retire.
14. QueuePayouts call site. Follow-on of 9.
15. ThreadRPCServer startup. BOTH. LoadPoolRounds added; LoadPaying and
ReconcilePayouts stay, and stay before the payout thread starts.
Crash safety, stated as the property that must hold: after kill -9 at any
point, restart pays each recipient exactly once. Before SendMoney, the
ledger has no txid and the payment is retried. Between SendMoney and the
ledger write, the wallet holds the transaction and the first-attempt probe
finds it. After the ledger write, paidTxids suppresses it. If the in-flight
file is lost entirely, the ledger rebuilds it and the probe still covers the
middle window.
src/nostr.cpp -- 3 hunks. THEIRS as the base, with two things restored from
ours. Theirs loads the Windows ROOT certificate store, which ours lacks and
genuinely needs, since OpenSSL's default verify paths are usually empty there
and our version would fail closed. But the plan's claim that theirs is a
superset of our 06bceb4 is FALSE: upstream's nostr.cpp contains no
min_proto_version, no TLS1_2, no SSL_OP_NO_*, and no set_options anywhere, so
taking it wholesale would have removed the TLS-version floor outright. Restored.
Second, their SSL_get_verify_result check sits after a successful handshake,
where with SSL_VERIFY_PEER set it is nearly dead code -- a bad certificate
aborts inside SSL_connect. The reporting belongs in the failure branch, which is
where the merge put it; their post-handshake duplicate was dropped. Our
relay-list and socket-pooling hunks auto-merged and were verified intact.
src/net.cpp -- 1 hunk. THEIRS. Their guard, hSocket == INVALID_SOCKET ||
pnode->hSocket != hSocket, subsumes ours and additionally catches descriptor
recycling, which ours did not; our version also referenced a pnode that only
existed in the select() loop. Our other db34011 hunk, nulling pnode->hSocket in
the poll-failure branch, auto-merged and was NOT re-applied -- plan section 7
Batch D says to re-apply it, and that instruction is stale. Two pieces of
residue the merge left behind were cleaned up: a comment still describing
select() semantics, and a pnode->hSocket != INVALID_SOCKET guard made redundant
by a continue two lines above it.
src/main.cpp -- 2 hunks. h1 BOTH, pure add/add adjacency between our
ParticipantHashThreadCount and their SecureClear forward declarations. h2 OURS,
and their handler discarded: 44004e3 no-ops checkorder/submitorder but keeps
market.cpp and market.h, whereas 11766be deletes the marketplace outright
including CRequestTracker, subscribe/publish and mapProducts. Ours is strictly
more.
src/main_gui.cpp -- 2 hunks. BOTH, additive on both sides. Our /payoutaddr,
/payoutfile and /hashthreads alongside their /stratumbridge and
/stratumbridgeport.
src/btfrv.cpp -- 1 hunk, not predicted by the plan. False adjacency: our
INVALID_SOCKET check and their #endif landed at the same offset. Ordering is
load-bearing -- our check goes inside the #else arm, before the #endif, or the
non-proxy build silently loses it. The BITFLASH_NODE_BUILD arm already returns
early, so both paths now have the check.
Makefile -- 2 hunks plus a hazard with no conflict marker. THEIRS for both
dependency-fetch recipes: checkout --detach FETCH_HEAD followed by test
"$(git rev-parse HEAD)" = "$(..._COMMIT)" asserts the pin actually landed, where
ours only checked it out. Everything of ours that they lack is kept: the
appimagetool 1.9.1 release pin with a sha256 check, where upstream fetches the
mutable "continuous" tag and executes it undigested, and the move of both the
tool and the AppDir out of /tmp, where any local user could pre-place an
executable and have it run as whoever runs make.
The hazard: both sides added a pin block in a different place, so git merged
both, outside any conflict region. SECP256K1_COMMIT and RANDOMX_COMMIT ended up
assigned twice, near L16 and near L44, and GNU Make takes the last := -- a naive
resolution would have silently built against our old secp256k1 pin while
appearing to adopt upstream's. The duplicate block was deleted by hand. Each of
the four variables is now assigned exactly once, and make -n resolves
SECP256K1_COMMIT to 7fecac74 with 6e2c8bc4 appearing nowhere.
Also applied here, from plan phase 4: SECP256K1_COMMIT moved to upstream's
7fecac74aed8e1fd9078380d67dd04663705c989 so signature validation matches the
binaries the rest of the network runs. secp256k1 was rebuilt from scratch
against the new pin on the build host and the checked-out dependency commit
verified. RANDOMX_COMMIT already matched at 1e9d4b2d and is unchanged.
src/Makefile, src/makefile.mingw -- 1 hunk each, both the HEADERS list. Ours
drops market.h, theirs adds proxy.h; the resolution does both. Taking theirs
wholesale would have left the Makefiles referencing a header git had already
deleted in our favour, silently, without a conflict.
Files the plan predicted would conflict and did not: net.h, main.h and db.cpp
auto-merged. Verified rather than assumed. net.h carries both our
BTF_MSG_WAIT_MAX_SECS work, including the constructor fix from 2d84982 that
starts nMsgWaitSince and nMsgWaitSize at 0, and their comment rewrite with
extern int nPeersWatched. The dead members that fix removed are gone from every
file.
Deviations from MERGE-1.2.18-PLAN.md.
- Section 6's mechanism was not used. Replayed onto the real merge base and
merged, rather than cherry-picking onto the tag. Same result, 28 hunks
instead of 51.
- Section 2 calls 09ff575 a superset of our 06bceb4. It is not; upstream has no
TLS-version floor at all. Ours restored on top of theirs.
- Section 7 Batch D says to re-apply our db34011 line into their poll-failure
branch. It auto-merged; re-applying would have duplicated it.
- Section 2 says do not take the stratum bridge. It is taken, and left off.
fStratumBridge defaults false in main.cpp, the thread is only spawned when
/stratumbridge is passed, and the listener binds 127.0.0.1 only. Excising it
would cost real divergence and future conflict for no gain while it is
disabled. Its defects -- select()/FD_SET on a node that can hold more than
1024 descriptors, and an uncapped thread per connection -- are real only when
it is enabled and are being reported upstream separately.
- Section 7's staged batches were not run as separate gated commits. The
conflict surface is interleaved within single functions, chiefly the payout
ledger, and splitting it would have meant committing states that do not
build. The merge is one commit, as briefed.
- Batch F's two fixes are folded into hunk 13 rather than deferred, because
they change the same lines the resolution rewrites.
- The plan did not anticipate the btfrv.cpp conflict, the duplicate pin block,
the market.h trap in the two Makefiles, or the give-up/infinite-retry
interaction created by adopting upstream's ledger model.
|
|
CNode's constructor never wrote nMsgWaitSince or nMsgWaitSize, so every peer
began life with indeterminate values in the pair ProcessMessages uses to
decide whether that peer has stopped sending, and the first partial message
it read was measured against them.
The guard restarts the clock only when nMsgWaitSince is 0 or the buffer has
grown since the last look. Garbage in nMsgWaitSince is neither, so the very
first look fell through to the timeout branch. A peer that merely split a
message across two TCP segments -- header in one, body in the next, which is
ordinary on any real link -- could be dropped as unreadable on first contact,
before it had sent anything wrong and often before its version message. It
was never reproducible either: what happened depended on what had last used
that heap. The log reported the stall as roughly the age of the epoch:
peer 127.0.0.1:8433 announced a 100 byte 'block' and has sent nothing more
for 1786593923 s (0 of 100 payload bytes) -- dropping, this peer is
unreadable
Every node is running this.
Introduced in 7f6e5ad, which added the pair to the class and to
ProcessMessages but not to the constructor; the constructor went on
initialising nIncompleteMessageStart, nIncompleteMessageSize and
strIncompleteMessageCommand, the upstream members our implementation
supersedes. 5490d66 removed their last reader from ProcessMessages, and
nothing has read or written them since, so they go, with
BTF_INCOMPLETE_MESSAGE_TIMEOUT_SECS -- decoy state in a networking struct is
how this happened, and the merge ahead of us would carry it further. The
initialiser list now follows declaration order, so a member missing from it
is visible.
The net-message selftest was asserting on those dead members too, which is
why it could not have caught this: it read state no code path writes, and its
stale-message case passed only when the garbage in nMsgWaitSince happened to
look old. Rewired onto nMsgWaitSince / nMsgWaitSize / BTF_MSG_WAIT_MAX_SECS,
and given the two cases it never covered -- a new peer's clock must start
stopped, and a peer that is still delivering payload must not be dropped
however old its clock is.
Measured on the build host, 20 runs each: before, net-message failed 20/20
("a fresh incomplete message does not disconnect immediately" failed 17/20 by
itself). After, 20/20 pass. Rewired selftest against a reverted constructor
fix: fails 20/20, deterministically, where the old suite caught it by luck or
not at all.
|
|
StartMinerThreads() sizes its pool with MinerThreadCount(), which is right for
solo mining -- each thread hashes a template of its own -- and wrong for
participant mode, where every thread entering BitcoinMiner() is a whole miner:
its own Stratum session, its own payout address off the rotation, its own pool
of /hashthreads workers. The two counts multiplied instead of dividing.
Measured on the pool box with /hashthreads=2: a container with no cpuset logged
"Mining with 11 thread(s) of 12 core(s)" -- 11 sessions, 22 hash threads on 12
cores, 1058% CPU, load 22.68 -- while the same binary under --cpuset-cpus 10,11
logged one session and 199%. The second miner (8 cores, /hashthreads=1) ran 7
sessions and went 74C to 89C. No code differed between those runs. Only the
cpuset did, which means the cpuset was holding the thread count down by
accident and every deployment without one was oversubscribed.
Participant mode now starts exactly one session, so /hashthreads means what it
says instead of being multiplied by the core count. The worker count moves to
ParticipantHashThreadCount(), next to MinerThreadCount() where the two are
easy to confuse, so what is printed at startup and what actually hashes cannot
drift apart -- and it IS printed now. It was only ever visible under -debug,
which is how eleven sessions ran unnoticed.
Same host, same cpuset (0-3), same arguments, measured as a delta over 20 s
more than four minutes after start:
before 399% CPU, 6-7 threads hot, 3 sessions
after 201% CPU, exactly 2 threads at 99.9%, 1 session
Solo mining is untouched: /gen without /participant still starts
MinerThreadCount() threads.
(cherry picked from commit 45bc9ed5bc132712092f3e7902fef746c6cae898)
|
|
The two dependency clones were `git clone --depth 1 <url>` with no
revision, which resolves to whatever the default branch points at the
moment the build happens to run.
For RandomX that is the proof of work. Its master is not frozen -- v2.0
changed the algorithm in March -- so two builds of the same Bitflash
commit, made weeks apart, can hash the same header differently. That is
not a supply-chain worry, it is a chain split delivered as a build
artifact, with no version recorded anywhere to explain it afterwards. For
libsecp256k1 it is signature verification, the rule deciding who may
spend.
Both now fetch a full commit id, which is the only form here that
verifies what it got, since git resolves the object by its own hash. The
RandomX pin is the commit this tree has been building and syncing
against, so today's behaviour is unchanged; secp256k1 is pinned to the
v0.8.0 release commit.
appimagetool came from the "continuous" tag, which upstream moves in
place on every push, fetched over the network and executed unverified.
Pinned to a numbered release and checked by digest before it is allowed
to run.
It also ran from /tmp on a fixed name, guarded by `[ ! -x
/tmp/appimagetool ]` -- an executable already sitting there is taken as
the tool, so any local account could leave one and have it run as
whoever built. The AppDir had the same shape one step later, through
symlinks on the rm -rf. Both moved under the build tree, which the build
already owns, and cleaned by `make clean`.
(cherry picked from commit 664aebca08d17a9dbae12c25f21cd8ef2abe7aa6)
|
|
Four inherited handlers took attacker-controlled input and wrote to disk
with nothing authorising it. None of them had a caller left anywhere in
the tree, so all four were reachable only from the outside.
checkorder called GenerateNewKey() -- AddKey(), a wallet.dat write -- for
any peer that asked, and handed back the pubkey. Its comment claims the
cache keeps one key per ip, and on a 2009 network of routable addresses
it did. Not here: inbound peers arrive through a rendezvous tunnel and
are tagged by BtfMarkerAddr() over three random bytes, so addr.ip is
fresh every connection and the cache never hits. Reconnect in a loop and
the node mints a key per connection, none of them in any backup taken
beforehand, which for a pre-BIP32 wallet is how coins go missing. The map
holding them was static and never trimmed.
submitorder AddToWallet()'d a CWalletTx from any peer that sent one. It
had to be a valid transaction; it did not have to be ours.
review checked a signature the sender made with its own key, which proves
possession and authorises nothing, then appended to reviews.dat by
reading the whole vector, pushing one entry and writing it all back --
quadratic disk on an unbounded map<string,string> off the socket -- then
walked a link graph the same peer supplied and relayed the message to
every other node.
reply dispatched into CRequestTracker. Nothing ever called PushRequest,
so the table was permanently empty and the handler was a parser and
nothing else.
Removed with the subsystem behind them: market.cpp, CProduct, CReview,
CUser, CReviewDB, the advert publish templates and the subscribe channel
machinery, none of which had an inbound handler either. MSG_REVIEW,
MSG_PRODUCT and MSG_TABLE stay in the inv enum, unserved: CInv puts type
on the wire as an integer and MSG_TX and MSG_BLOCK are shared with every
other node.
Unknown commands are already ignored for extensibility, so peers still
running these lose nothing but an answer they never asked for. Verified
by a clean build of bitflash-node.
(cherry picked from commit ad9c393866c007a115f32d70a9017ce91066a7d1)
|
|
std::thread::hardware_concurrency() reports the MACHINE's core count. Inside a
container that is the wrong number: with --cpuset-cpus 0-5 on a 12-core host it
still answers 12, and every caller that sized a pool from it oversubscribed 2x.
Measured on the pool box:
- the pool node logged "Mining with 11 thread(s) of 12 core(s)" while
confined to 6 CPUs
- a burst miner launched with /hashthreads=2 filled the ~2 GB RandomX
dataset with 12 threads, alongside the pool node in the same 8 GB container
AvailableCoreCount() asks sched_getaffinity what the cpuset actually permits,
which is the number all of these meant. It cannot make the dataset build slower:
the CPU budget is identical either way, so fewer threads on the same cores is
the same work without the thrash. Falls back to hardware_concurrency where the
call is unavailable, and never returns 0 because callers subtract from it.
Not the cause of the burst miner dying -- that was a corrupt Berkeley DB
environment in the worker datadir -- but it is why the failure landed on the
dataset build, and it is wrong on its own terms.
(cherry picked from commit b3bee16f3eef8982bff1174bd9d0a9c21473732c)
|
|
Upstream #109's message-stall rework, hand-merged so it keeps the progress
timing this tree already had.
The old path consumed the header, found the payload short, inserted the header
back at the front of vRecv, then Sleep(100) inside the shared message thread.
A peer that announces a large payload and drips bytes made every pass pay an
O(n) insert and a 100 ms stall -- one peer slowing the thread that serves all
of them. Now the header is read from a copy and only skipped once the whole
body has arrived, so there is nothing to put back and nothing to sleep for.
Kept OURS, not upstream's, on the timeout: upstream restarts its clock only
when the (size, command) pair changes, which drops a peer sending one big
message slowly but honestly. This times the STALL -- the clock restarts on any
progress -- which is the distinction that matters on a slow link.
Also from #109: a header could announce 256 MB while serialize.h refuses
anything over MAX_SIZE (32 MB), so the check admitted an allocation the parser
would reject anyway and burned a peer slot for two minutes doing it.
MAX_PROTOCOL_MESSAGE_SIZE ties the two together. 8x less a peer can make us
hold, and this node takes public Stratum.
And #103's other half: ~CNode closed its socket with a raw closesocket(),
invisible to the accounting, and left hSocket set afterwards -- which is how a
descriptor number that now belongs to somebody else gets closed a second time.
(cherry picked from commit 2f25e2e07985e9bc732cadfbb3c9ac233895a814)
|
|
CheckBlock() validated block size against MAX_SIZE, the serializer's generic
32 MB guard, not a block rule. MAX_BLOCK_SIZE did not exist in this tree at
all: Bitcoin's 1 MB cap arrived in 0.3.x, after the 0.1.0 this fork came from.
A miner could produce a 32 MB block and every node would download, verify and
store it. Upstream #111 (from #110 by mtg0001).
The cap is a tightening, so a node that does not upgrade is the permissive one.
Measured upstream over 5709 blocks: largest 34,290 bytes, median 216, so 1 MB
rejects nothing that exists.
Upstream shipped the validation half only. Both places that ASSEMBLE a block
still filled to MAX_SIZE/2 (16 MB), which was consistent against a 32 MB
validation cap and is inverted by a 1 MB one -- the node would solve a block
and then reject it itself. Fixed both:
main.cpp CreateNewBlock, the node's own miner
rpc.cpp RebuildJob, the POOL's job builder -- the one that matters here,
because every miner on the pool would be working a template the
network rejects, and this pool is ~90% of hashrate
Half the cap in both, preserving the original margin: each test runs against a
transaction that is already counted, so the loop can overshoot by one.
(cherry picked from commit a3be9f04903447fa2741aff4eb529fe45f916877)
|
|
From the first of two wedges captured on the pool. It is a different
proximate fault from the second one, and this is the part that turns a
single bad socket into a node-wide outage.
A failed select() zeroes both descriptor sets, so no peer is serviced on
that pass. That is correct as far as it goes. What was missing is that
nothing stopped the offending descriptor coming straight back:
- the invalid socket was only marked fDisconnect, and a node is not
removed from vNodes until it has drained or DISCONNECT_DRAIN_SECS has
elapsed -- and it cannot drain, because its socket is gone;
- the select set was rebuilt from vNodes with no check, so the same dead
descriptor went back in on the very next pass and failed select()
again.
Measured in the capture: 1,027 consecutive 'its socket is no longer valid'
passes across 61 seconds -- the full drain timeout -- one bad socket, and
for the whole of it the node read nothing from anybody. The peer that
caused it had 57 bytes unsent, so it took the slow path out.
Now the descriptor is forgotten as soon as getsockopt says it is not ours:
select() succeeds again on the next pass and every other peer is served
normally while the dead node ages out as before.
Deliberately forgotten rather than closed. getsockopt has just said this
descriptor is no longer ours; calling close() on it is precisely how an
unrelated socket gets closed instead, which is the fault fixed in e742d0b.
(cherry picked from commit f14c513b4e39d6203b73584f6dc01acbb5a9546e)
|
|
Found auditing the previous commit for the push. Timing from the first
rewind would drop a peer whose message is large and merely slow, which
would be a fault of our own making rather than one we fixed.
Restart the clock whenever the buffer has grown by even one byte since the
last look. A transfer still making progress is never dropped however long
it takes; only one that has stopped growing while the node goes on
expecting more is, which is the actual failure.
(cherry picked from commit 5dddb26bae02013ce87719b11953aadf0bfa2594)
|
|
THIS IS THE WEDGE, and the note asking for this fix has been in the file
since Bitcoin 0.1: 'need a mechanism to give up waiting for overlong
message size error'.
ProcessMessages reads a header, finds the body has not all arrived, pushes
the header back onto the front of vRecv and breaks. Correct for a message
still in flight. For one that never completes it is permanent: every later
pass re-reads the same header and breaks in the same place, so that peer is
never parsed again for the life of the process -- no blocks, no inv, not
even its version message.
Nothing rescues it:
- a header may claim up to 256 MB (0x10000000) and still pass
CMessageHeader::IsValid(), so a garbage or truncated header is accepted
and then waited on forever;
- nLastRecv is refreshed by any byte that arrives (net.cpp), whether or
not it parses, so the inactivity timeout that drops a silent peer never
fires on a peer that is merely unreadable;
- the socket stays ESTABLISHED and select() keeps offering it, so nothing
at the socket layer notices either.
It also explains the signature that made no sense. A peer stuck before its
version message is parsed keeps nStartingHeight == -1, so a node with a full
vNodes still reports GetPeerMedianHeight() == -1 -- which is exactly what
was observed, and exactly what disabled the pool's behind-the-network gate
and let it fork.
Evidence, from two wedges captured on the pool by the watchdog:
MESSAGE-BREAK 2 ran 2014 times in the deaf window against 607 in the healthy
window before it, while blocks received went to zero and the node kept
processing miner traffic normally throughout.
Now: a peer that announces a body and does not deliver it within two minutes
is dropped, and says so in the log unconditionally. Two minutes is far
beyond any legitimate header-to-body gap and well inside
BTF_RECV_TIMEOUT_SECS.
(cherry picked from commit ca1e99288e03ac7beb2b97cf615a3255b4635d95)
|
|
THE WEDGE.
CNode::Disconnect() closed hSocket and left the member holding the stale
number. ~CNode's only guard is 'hSocket != INVALID_SOCKET' and nothing in
the tree ever assigned that sentinel, so the destructor closed the same
number a second time.
The two closes are five minutes apart by design: ThreadSocketHandler2
disconnects a node, sets nReleaseTime to GetTime() + 5*60, and parks it in
vNodesDisconnected; GetRefCount() reports it busy until that expires, so
'delete pnode' -- and the second close -- cannot run before then.
Linux hands out the lowest free descriptor. Measured on the pool: 202 open
descriptors with a high-water mark of 216, so a freed number is reissued
within seconds -- to a Nostr relay TLS socket, a rendezvous socket, a
tunnel end, or a miner connection. Five minutes later the destructor
closes it. The victim gets no error and no log line. It simply stops.
That is the whole signature, including the part that never made sense:
.btf block delivery and Nostr descriptor lookup share no lock, no queue
and no thread, so they should not fail together. They share the file
descriptor table, and a stray close is drawn from all of it.
It also explains why #78 did not help and may have hurt. #78 drops stuck
nodes -- it fired 41 times in one window -- and every drop is another
Disconnect(), which is another delayed close armed against an innocent
socket.
Second bug, same symptom, different subsystem: the pooled relay sockets.
ResolveDescriptor returned the instant a descriptor verified, leaving the
relay's EOSE unread on a socket that goes straight back into g_relayPool,
so the next caller read our leftovers as its own answer -- permanently one
message out of step, and no code path anywhere inspected the subscription
id that would have caught it. Worse, BtfResolve set lease.fOk = true on
every failure path including a dead RecvText, labelling it 'exchange
completed, just no match'. fOk = false is the only thing that ever closes
a pooled socket or clears fConnected, so a relay connection that died was
pinned in the pool, reported healthy, and reused for the life of the
process. That is 'no Nostr descriptor found' for a descriptor that nak can
see, cleared only by a restart.
Now: read to EOSE before returning, match the subscription id, and keep a
pooled socket only when the relay actually finished the conversation.
(cherry picked from commit ed4640c306cc46dbdbd92cb6dc55e7528651269c)
|
|
The peer-height gate replaced a tip-age check that was wrong in both
directions. It is a strict improvement and it is still blind in the one
case that matters most.
When the node goes deaf -- peers still in vNodes, nothing arriving -- no
peer announces anything, GetPeerMedianHeight() returns -1, and the
fallback mines our best chain. That fallback is right for a node that has
not been told yet and wrong for a node that has stopped being able to
hear, and a height nobody sent cannot tell those apart. A wedged node is
exactly a node with no peer height information, so the gate was silent
for the whole of heights 3966-4085: 120 blocks, 5890.50 BTF voided.
Time can tell them apart. GetPeerHeightAgeSecs() measures how long since
any peer reported a height, counting from first use rather than the epoch
so a starting node does not refuse jobs before its first handshake. Past
20 minutes the pool stops issuing jobs and says so unconditionally in the
log. /poolmineblind overrides it: without an escape hatch a genuinely
isolated pool that is 90% of the hashrate could never restart its own
chain, which is the deadlock the tip-age check used to cause.
Also stop nStartingHeight from going stale. It was a snapshot taken once
during the version handshake and never touched again, so on a long-lived
connection the median is built from heights the peers had hours ago, and
the behind-the-network check drifts toward 'everyone is behind us' and
quietly stops firing. A peer that hands us a block is demonstrably at
least at that block's height.
(cherry picked from commit 0800c18faf703b4acde1b131d0b742d37a86d92b)
|
|
The sync gate compared the tip timestamp against 30 minutes. That cannot
distinguish "we are not synced" from "the network is slow", and it failed
in both directions on the same day.
A pool mining its own fork keeps its tip fresh, so the check passed for
the whole of heights 3966-4085 -- 120 consecutive blocks, all orphaned,
5890.50 BTF of payouts voided, and nothing in the log to say so. Then,
once the pool stopped finding blocks, the same check refused to issue
jobs against a tip that was the network's own best. The pool is ~90% of
the hashrate, so nobody else could move the chain either: stale tip, no
jobs; no jobs, stale tip.
Gate on the height peers announce in their version message instead.
Median, so one peer cannot stall us with an absurd claim, and a block of
slack to match WarnIfBehind. Peers predating the announcement report -1
and are ignored; with no reports at all we mine our best chain, which is
what the node does anyway without a pool attached.
(cherry picked from commit 4e76e34f37a82c0885f75db7dfb9e49db410171d)
|
|
The exec queue was memory-only, and FlushMaturePayouts erased a matured entry
from pending_payouts.json *before* handing it over. Any restart in that window
lost those payments with no record that anyone was owed anything. It was not
hypothetical: a container restart on 2026-07-30 silently dropped a backlog of
~1,156 BTF, and there was nothing left to reconstruct it from.
Re-queueing blindly would have double-paid whoever had already been sent to,
so the payment needs to be idempotent rather than merely persistent.
In-flight work is now a persisted list of per-payment records
(paying_payouts.json), one per (block, address) -- the granularity SendMoney
works at, and therefore the granularity at which a payment can be half-done
when the process dies. Records are written, via a temp file and rename so a
crash cannot truncate them, *before* the ledger entry is erased, so the debt
is always described by at least one of the two files. Each attempt updates the
record (retries, deferrals, backoff); settling it removes the record and
rewrites the file, which is what stops a restart from paying twice.
On startup ReconcilePayouts runs before the payout thread:
- a record whose payment the wallet already shows -- matched on the payout
comment plus the exact amount and destination -- is dropped, not resent.
SendMoney commits the transaction to the wallet before it returns, so a
crash immediately after sending is recoverable.
- a ledger entry that has already been exploded into records is dropped,
which is the crash-in-the-window case where both files describe one debt.
The wallet check is deliberately biased towards "already paid": skipping a
payment is visible in the log and recoverable, paying twice is neither.
Verified live: sampling paying_payouts.json during a payout burst caught 3
records / 18.56 BTF mid-flight, cleared once paid, and the paid lines now
carry the txid.
(cherry picked from commit d0248f2e65dbe745f2e9b1d09e1f53a4354d3640)
|
|
Two fixes.
uint256's default constructor is empty (uint256.h:514), so the blockHash
field added in 6963862 was uninitialised garbage on every PendingPayout
built without an explicit hash -- which is every entry loaded from a ledger
written before that commit. Those entries were meant to be grandfathered
past the orphan check on a zero hash; instead each carried a random value
that matched no block in mapBlockIndex, and the first maturity sweep after
deploy voided the whole loaded ledger: 100 entries, 4950 BTF. Only one
"was orphaned -- voiding" line reached the log because LogPrint collapsed
the other 99 as duplicates, so the destruction was nearly silent. Nothing
had been paid at that point and the file was restored from a pre-deploy
backup, so no obligation was lost. Assigning 0 explicitly is all that was
missing.
Separately, four of the twelve relays are dead. Probed all twelve from two
independent networks (a residential line and the pool's VPS) over v4 and v6,
with DNS confirmed against 1.1.1.1:
relay.current.fyi, relay.nostr.bg -- NXDOMAIN, no A and no AAAA. The
domains are gone. (A getaddrinfo call on the VPS reports an address for
them, but that is the VPS's own IPv6 handed back by its local resolver,
not a real host.)
relay.nostr.band, nostr.fmt.wiz.biz -- DNS resolves and TCP connects, but
the TLS handshake never completes, from either network. Whether they are
down or refusing non-browser clients, this node cannot use them.
The remaining eight all complete a WebSocket upgrade. A dead relay is not
free: each holds a connect/retry slot and delays discovery for no added
redundancy.
(cherry picked from commit 144bc96f150d23507d57d29cb339f177dcd6b092)
|
|
A 280-block reorg (heights 3075-3354) orphaned 292 of the pool's 707 blocks
and destroyed 14,600 BTF of coinbase. The payout ledger did not notice: it
records only matureAtHeight, and a reorg replaces the block at a height
without changing the height, so ~14.4k BTF of obligations outlived the coin
that was supposed to fund them. The pool then paid fork-era miners out of
real-chain coin until the wallet was empty -- 17,827 BTF paid against 17,900
BTF of matured supply, leaving under 1.1 BTF spendable and every subsequent
payment failing.
Three changes, all in the payout path:
- Record the funding block's hash and height, persist both, and void an
obligation whose block is no longer in the main chain. Checked both at
maturity and again in the exec queue, since an entry can sit there for
hours across retries and the reorg can land at any point in that window.
Entries loaded from a file written before this are grandfathered rather
than voided on a missing field.
- Stop spending the retry budget on maturity. SelectCoins failing for want
of spendable coin is not an error, but it consumed one of 1440 retries
every 60s, so a payment the pool fully intended to honour would hit
GIVING UP 24h later and the miner would silently never be paid. Those
attempts now defer without incrementing retries, logged hourly rather
than every minute. GIVING UP now means what it says: 1440 real failures
against a block still on-chain with the money available.
- Schedule at COINBASE_SPENDABLE_DEPTH instead of COINBASE_MATURITY. At
depth 100 a block's own coinbase is 20 blocks short of spendable, so its
payout had to come from unrelated mature coin and the pool carried a
permanent ~19-block float. The "96% of 934 payments went through at depth
100" note that justified the shorter wait was measured while this node was
alone on a private fork paying itself. The extra 20 blocks also put the
maturity gate past the reorg that caused all of the above.
(cherry picked from commit 696386220a3d2fac88cb8045931392f4909e0985)
|
|
SendMoney's "oversized transaction that requires a transaction fee of
0.00" is not an oversized transaction and there is no fee. That branch is
taken when nValue + nFeeRequired > GetBalance() -- an empty spendable
balance -- and the fee reads 0.00 because CreateTransaction failed before
setting one. 3,267 of these in debug.log sent me looking for a
transaction-size problem that does not exist. Report what is short, by how
much, and against what balance.
Name COINBASE_SPENDABLE_DEPTH in main.h and derive GetBlocksToMaturity
from it, because the distinction it captures is genuinely surprising:
CMerkleTx::GetCredit values a mined coinbase at zero until
COINBASE_MATURITY+20 deep, not COINBASE_MATURITY.
I first changed the payout gate to that deeper value, on the theory that
each payout was firing 20 blocks before its own coinbase could be spent.
Measuring it refuted that: across 934 payments, 96% succeeded at depth
100, the moment they were queued, which is only possible because a payout
is funded from whatever coin is already mature rather than from its own
block. The gate stays at COINBASE_MATURITY, with a comment recording the
measurement so the next person does not re-make the same change.
The real shortfall is structural and not addressed here: the pool pays out
~99% of each block's subsidy while only matured coinbases are spendable,
so it runs with almost no buffer, and any orphaned block it queued a
payout for drains that buffer permanently. The 24h retry loop absorbs this
-- 0 payouts forfeited -- but the buffer is what needs fixing.
(cherry picked from commit e28df01f90d621b0cd818f28d70a20b8ab5d1a8d)
|
|
Upstream #30 split rendezvous registration from waiting for a dial, so
RvServiceRegister now returns as soon as the relay lists the service and
no longer consumes the relay's 0x01 pairing byte. It updated its own
caller in net.cpp and left this one behind.
AcceptOneFn went straight from RvServiceRegister to BtfServiceWrap, so
BtfServiceWrap read that 0x01 as the first byte of the miner's 32-byte
ephemeral key and every byte after it was shifted by one. The X25519
handshake has no confirmation step, so it completed against a garbage
shared secret, the tunnel was handed to the Stratum event loop, and
nothing failed loudly: the pool logged a miner arriving and then dropped
it as "never authorized", while the miner logged a pool that closed on it
with gotSubscribeAck=0. Loopback miners were unaffected, which is exactly
what made it look like a relay problem.
Block in RvServiceWaitPaired before wrapping, and close and re-register if
the relay drops us first.
test_rvaccept.cpp covers both shapes against a real bitflashd on loopback:
the pre-fix order establishes a tunnel that carries no bytes, the fixed
order round-trips. Also corrects the btftunnel.h contract comment, which
still described the pre-#30 behaviour and is what the old code was written
against -- upstream's own test_relayproc.cpp, test_btftunnel.cpp and
test_btfrv.cpp are still on that stale contract and fail the same way.
(cherry picked from commit 3e3a36b2bf1e6ffb4861507089b743688619377f)
|
|
SSL_CTX_new leaves verification at SSL_VERIFY_NONE and loads no trust
store, and nothing here ever changed that. SNI was set, a certificate was
received, and it was then ignored: every wss:// relay connection this node
has ever made would have completed against an attacker's certificate.
Signed events limit the damage -- a MITM cannot forge an announcement
attributable to our key -- but it can drop the rendezvous registration so
miners never find the pool, censor subscriptions, or answer them with
arbitrary content, which is the input the allocation bounds exist to
contain.
Load the system trust store, require SSL_VERIFY_PEER, and bind the check
to the host we dialed with SSL_set1_host, since chain validation alone
would accept any certificate a trusted CA ever issued for any name. Floor
the protocol at TLS 1.2. Verification failures report the specific X509
reason rather than a generic handshake error, and a missing CA store is a
hard failure instead of a silent downgrade to trusting everything.
(cherry picked from commit 6020740e68463deb1cc3c5b477372fcd1a245adb)
|
|
The min-2-threads floor applied even when the user explicitly asked
for 1 thread, so /hashthreads=1 silently spawned 2.
(cherry picked from commit a67b9e84f4f7607d307081b4a9873f5d9e78cf8a)
|
|
Descriptor (38501) still published so miners can resolve the pool;
only the public pool listing event (38503) is suppressed.
(cherry picked from commit 17bbdae1ec97945c37c1d5b2486e5c01a4d7ad87)
|
|
A pool that just found its first blocks has zero mature balance until
100 blocks later; orphaned coinbases (observed: reorgs orphaned our
blocks 2651-2655) make that worse. 30 one-minute retries forfeited
payments that would have succeeded hours later.
(cherry picked from commit 49e80023ca23baee8a4b80cfb891e7a358dc39ba)
|
|
A pool started with /rvrelay=ONE still spawned an accept thread for every
Nostr-discovered relay (BtfAllRelays always merged g_discoveredRelays), and
each thread optimistically overwrote the advertised meeting node before
registering -- so a discovered Tailscale-only relay (a mesh host)
flapped the published .btf descriptor and public miners could not connect.
fBtfRelayPinned (set only when /rvrelay parses) makes BtfAllRelays return
exactly the pinned list. Also validate the /rvrelay value at parse time:
with the pin in place no discovered relay can mask a typo.
(cherry picked from commit 0a4d2b02fa5382c5e788b5f0e2f8a844695dec47)
|
|
- RebuildJob: no jobs with zero peers or a tip older than 30min;
mining unsynced produced orphaned blocks and payouts that can
never be paid (observed in production: 8 orphan 'found blocks')
- payout exec: on SendMoney failure requeue with 60s backoff, up to
30 attempts, instead of silently forfeiting the payment
- nostr: add oxtr.dev, nos.social, fmt.wiz.biz, current.fyi, nostr.bg
(12 relays total) for announcement redundancy
(cherry picked from commit f1986cd7e4a6f69469a2256eb96ff03f3f146792)
|
|
testnet-consensus into main
|
|
rate' (#69) from randomx-large-pages into main
|
|
The reservation instruction printed when a large-page allocation falls back was
Linux-only (sysctl vm.nr_hugepages). Route it through a small helper so a
Windows build points at the "Lock pages in memory" privilege instead.
|
|
RandomX reads its 256 MB cache, 2 GB dataset and per-VM scratchpads at random,
so most accesses miss the TLB on 4 KB pages. Requesting 2 MB pages recovers
roughly a tenth of the hash rate. Each allocation tries large pages first and
falls back to normal pages when the pool is not reserved, so nothing regresses
when hugepages are unavailable. -nolargepages opts out, and the diagnostics
report which allocations got large pages.
|
|
|
|
testnet-prep into main
|
|
|
|
|
|
StartMinerThreads() sizes its pool with MinerThreadCount(), which is right for
solo mining -- each thread hashes a template of its own -- and wrong for
participant mode, where every thread entering BitcoinMiner() is a whole miner:
its own Stratum session, its own payout address off the rotation, its own pool
of /hashthreads workers. The two counts multiplied instead of dividing.
Measured on the pool box with /hashthreads=2: a container with no cpuset logged
"Mining with 11 thread(s) of 12 core(s)" -- 11 sessions, 22 hash threads on 12
cores, 1058% CPU, load 22.68 -- while the same binary under --cpuset-cpus 10,11
logged one session and 199%. The second miner (8 cores, /hashthreads=1) ran 7
sessions and went 74C to 89C. No code differed between those runs. Only the
cpuset did, which means the cpuset was holding the thread count down by
accident and every deployment without one was oversubscribed.
Participant mode now starts exactly one session, so /hashthreads means what it
says instead of being multiplied by the core count. The worker count moves to
ParticipantHashThreadCount(), next to MinerThreadCount() where the two are
easy to confuse, so what is printed at startup and what actually hashes cannot
drift apart -- and it IS printed now. It was only ever visible under -debug,
which is how eleven sessions ran unnoticed.
Same host, same cpuset (0-3), same arguments, measured as a delta over 20 s
more than four minutes after start:
before 399% CPU, 6-7 threads hot, 3 sessions
after 201% CPU, exactly 2 threads at 99.9%, 1 session
Solo mining is untouched: /gen without /participant still starts
MinerThreadCount() threads.
(cherry picked from commit 45bc9ed5bc132712092f3e7902fef746c6cae898)
|
|
The two dependency clones were `git clone --depth 1 <url>` with no
revision, which resolves to whatever the default branch points at the
moment the build happens to run.
For RandomX that is the proof of work. Its master is not frozen -- v2.0
changed the algorithm in March -- so two builds of the same Bitflash
commit, made weeks apart, can hash the same header differently. That is
not a supply-chain worry, it is a chain split delivered as a build
artifact, with no version recorded anywhere to explain it afterwards. For
libsecp256k1 it is signature verification, the rule deciding who may
spend.
Both now fetch a full commit id, which is the only form here that
verifies what it got, since git resolves the object by its own hash. The
RandomX pin is the commit this tree has been building and syncing
against, so today's behaviour is unchanged; secp256k1 is pinned to the
v0.8.0 release commit.
appimagetool came from the "continuous" tag, which upstream moves in
place on every push, fetched over the network and executed unverified.
Pinned to a numbered release and checked by digest before it is allowed
to run.
It also ran from /tmp on a fixed name, guarded by `[ ! -x
/tmp/appimagetool ]` -- an executable already sitting there is taken as
the tool, so any local account could leave one and have it run as
whoever built. The AppDir had the same shape one step later, through
symlinks on the rm -rf. Both moved under the build tree, which the build
already owns, and cleaned by `make clean`.
(cherry picked from commit 664aebca08d17a9dbae12c25f21cd8ef2abe7aa6)
|
|
Four inherited handlers took attacker-controlled input and wrote to disk
with nothing authorising it. None of them had a caller left anywhere in
the tree, so all four were reachable only from the outside.
checkorder called GenerateNewKey() -- AddKey(), a wallet.dat write -- for
any peer that asked, and handed back the pubkey. Its comment claims the
cache keeps one key per ip, and on a 2009 network of routable addresses
it did. Not here: inbound peers arrive through a rendezvous tunnel and
are tagged by BtfMarkerAddr() over three random bytes, so addr.ip is
fresh every connection and the cache never hits. Reconnect in a loop and
the node mints a key per connection, none of them in any backup taken
beforehand, which for a pre-BIP32 wallet is how coins go missing. The map
holding them was static and never trimmed.
submitorder AddToWallet()'d a CWalletTx from any peer that sent one. It
had to be a valid transaction; it did not have to be ours.
review checked a signature the sender made with its own key, which proves
possession and authorises nothing, then appended to reviews.dat by
reading the whole vector, pushing one entry and writing it all back --
quadratic disk on an unbounded map<string,string> off the socket -- then
walked a link graph the same peer supplied and relayed the message to
every other node.
reply dispatched into CRequestTracker. Nothing ever called PushRequest,
so the table was permanently empty and the handler was a parser and
nothing else.
Removed with the subsystem behind them: market.cpp, CProduct, CReview,
CUser, CReviewDB, the advert publish templates and the subscribe channel
machinery, none of which had an inbound handler either. MSG_REVIEW,
MSG_PRODUCT and MSG_TABLE stay in the inv enum, unserved: CInv puts type
on the wire as an integer and MSG_TX and MSG_BLOCK are shared with every
other node.
Unknown commands are already ignored for extensibility, so peers still
running these lose nothing but an answer they never asked for. Verified
by a clean build of bitflash-node.
(cherry picked from commit ad9c393866c007a115f32d70a9017ce91066a7d1)
|
|
std::thread::hardware_concurrency() reports the MACHINE's core count. Inside a
container that is the wrong number: with --cpuset-cpus 0-5 on a 12-core host it
still answers 12, and every caller that sized a pool from it oversubscribed 2x.
Measured on the pool box:
- the pool node logged "Mining with 11 thread(s) of 12 core(s)" while
confined to 6 CPUs
- a burst miner launched with /hashthreads=2 filled the ~2 GB RandomX
dataset with 12 threads, alongside the pool node in the same 8 GB container
AvailableCoreCount() asks sched_getaffinity what the cpuset actually permits,
which is the number all of these meant. It cannot make the dataset build slower:
the CPU budget is identical either way, so fewer threads on the same cores is
the same work without the thrash. Falls back to hardware_concurrency where the
call is unavailable, and never returns 0 because callers subtract from it.
Not the cause of the burst miner dying -- that was a corrupt Berkeley DB
environment in the worker datadir -- but it is why the failure landed on the
dataset build, and it is wrong on its own terms.
(cherry picked from commit b3bee16f3eef8982bff1174bd9d0a9c21473732c)
|
|
Upstream #109's message-stall rework, hand-merged so it keeps the progress
timing this tree already had.
The old path consumed the header, found the payload short, inserted the header
back at the front of vRecv, then Sleep(100) inside the shared message thread.
A peer that announces a large payload and drips bytes made every pass pay an
O(n) insert and a 100 ms stall -- one peer slowing the thread that serves all
of them. Now the header is read from a copy and only skipped once the whole
body has arrived, so there is nothing to put back and nothing to sleep for.
Kept OURS, not upstream's, on the timeout: upstream restarts its clock only
when the (size, command) pair changes, which drops a peer sending one big
message slowly but honestly. This times the STALL -- the clock restarts on any
progress -- which is the distinction that matters on a slow link.
Also from #109: a header could announce 256 MB while serialize.h refuses
anything over MAX_SIZE (32 MB), so the check admitted an allocation the parser
would reject anyway and burned a peer slot for two minutes doing it.
MAX_PROTOCOL_MESSAGE_SIZE ties the two together. 8x less a peer can make us
hold, and this node takes public Stratum.
And #103's other half: ~CNode closed its socket with a raw closesocket(),
invisible to the accounting, and left hSocket set afterwards -- which is how a
descriptor number that now belongs to somebody else gets closed a second time.
(cherry picked from commit 2f25e2e07985e9bc732cadfbb3c9ac233895a814)
|
|
CheckBlock() validated block size against MAX_SIZE, the serializer's generic
32 MB guard, not a block rule. MAX_BLOCK_SIZE did not exist in this tree at
all: Bitcoin's 1 MB cap arrived in 0.3.x, after the 0.1.0 this fork came from.
A miner could produce a 32 MB block and every node would download, verify and
store it. Upstream #111 (from #110 by mtg0001).
The cap is a tightening, so a node that does not upgrade is the permissive one.
Measured upstream over 5709 blocks: largest 34,290 bytes, median 216, so 1 MB
rejects nothing that exists.
Upstream shipped the validation half only. Both places that ASSEMBLE a block
still filled to MAX_SIZE/2 (16 MB), which was consistent against a 32 MB
validation cap and is inverted by a 1 MB one -- the node would solve a block
and then reject it itself. Fixed both:
main.cpp CreateNewBlock, the node's own miner
rpc.cpp RebuildJob, the POOL's job builder -- the one that matters here,
because every miner on the pool would be working a template the
network rejects, and this pool is ~90% of hashrate
Half the cap in both, preserving the original margin: each test runs against a
transaction that is already counted, so the loop can overshoot by one.
(cherry picked from commit a3be9f04903447fa2741aff4eb529fe45f916877)
|
|
From the first of two wedges captured on the pool. It is a different
proximate fault from the second one, and this is the part that turns a
single bad socket into a node-wide outage.
A failed select() zeroes both descriptor sets, so no peer is serviced on
that pass. That is correct as far as it goes. What was missing is that
nothing stopped the offending descriptor coming straight back:
- the invalid socket was only marked fDisconnect, and a node is not
removed from vNodes until it has drained or DISCONNECT_DRAIN_SECS has
elapsed -- and it cannot drain, because its socket is gone;
- the select set was rebuilt from vNodes with no check, so the same dead
descriptor went back in on the very next pass and failed select()
again.
Measured in the capture: 1,027 consecutive 'its socket is no longer valid'
passes across 61 seconds -- the full drain timeout -- one bad socket, and
for the whole of it the node read nothing from anybody. The peer that
caused it had 57 bytes unsent, so it took the slow path out.
Now the descriptor is forgotten as soon as getsockopt says it is not ours:
select() succeeds again on the next pass and every other peer is served
normally while the dead node ages out as before.
Deliberately forgotten rather than closed. getsockopt has just said this
descriptor is no longer ours; calling close() on it is precisely how an
unrelated socket gets closed instead, which is the fault fixed in e742d0b.
(cherry picked from commit f14c513b4e39d6203b73584f6dc01acbb5a9546e)
|
|
Found auditing the previous commit for the push. Timing from the first
rewind would drop a peer whose message is large and merely slow, which
would be a fault of our own making rather than one we fixed.
Restart the clock whenever the buffer has grown by even one byte since the
last look. A transfer still making progress is never dropped however long
it takes; only one that has stopped growing while the node goes on
expecting more is, which is the actual failure.
(cherry picked from commit 5dddb26bae02013ce87719b11953aadf0bfa2594)
|
|
THIS IS THE WEDGE, and the note asking for this fix has been in the file
since Bitcoin 0.1: 'need a mechanism to give up waiting for overlong
message size error'.
ProcessMessages reads a header, finds the body has not all arrived, pushes
the header back onto the front of vRecv and breaks. Correct for a message
still in flight. For one that never completes it is permanent: every later
pass re-reads the same header and breaks in the same place, so that peer is
never parsed again for the life of the process -- no blocks, no inv, not
even its version message.
Nothing rescues it:
- a header may claim up to 256 MB (0x10000000) and still pass
CMessageHeader::IsValid(), so a garbage or truncated header is accepted
and then waited on forever;
- nLastRecv is refreshed by any byte that arrives (net.cpp), whether or
not it parses, so the inactivity timeout that drops a silent peer never
fires on a peer that is merely unreadable;
- the socket stays ESTABLISHED and select() keeps offering it, so nothing
at the socket layer notices either.
It also explains the signature that made no sense. A peer stuck before its
version message is parsed keeps nStartingHeight == -1, so a node with a full
vNodes still reports GetPeerMedianHeight() == -1 -- which is exactly what
was observed, and exactly what disabled the pool's behind-the-network gate
and let it fork.
Evidence, from two wedges captured on the pool by the watchdog:
MESSAGE-BREAK 2 ran 2014 times in the deaf window against 607 in the healthy
window before it, while blocks received went to zero and the node kept
processing miner traffic normally throughout.
Now: a peer that announces a body and does not deliver it within two minutes
is dropped, and says so in the log unconditionally. Two minutes is far
beyond any legitimate header-to-body gap and well inside
BTF_RECV_TIMEOUT_SECS.
(cherry picked from commit ca1e99288e03ac7beb2b97cf615a3255b4635d95)
|
|
THE WEDGE.
CNode::Disconnect() closed hSocket and left the member holding the stale
number. ~CNode's only guard is 'hSocket != INVALID_SOCKET' and nothing in
the tree ever assigned that sentinel, so the destructor closed the same
number a second time.
The two closes are five minutes apart by design: ThreadSocketHandler2
disconnects a node, sets nReleaseTime to GetTime() + 5*60, and parks it in
vNodesDisconnected; GetRefCount() reports it busy until that expires, so
'delete pnode' -- and the second close -- cannot run before then.
Linux hands out the lowest free descriptor. Measured on the pool: 202 open
descriptors with a high-water mark of 216, so a freed number is reissued
within seconds -- to a Nostr relay TLS socket, a rendezvous socket, a
tunnel end, or a miner connection. Five minutes later the destructor
closes it. The victim gets no error and no log line. It simply stops.
That is the whole signature, including the part that never made sense:
.btf block delivery and Nostr descriptor lookup share no lock, no queue
and no thread, so they should not fail together. They share the file
descriptor table, and a stray close is drawn from all of it.
It also explains why #78 did not help and may have hurt. #78 drops stuck
nodes -- it fired 41 times in one window -- and every drop is another
Disconnect(), which is another delayed close armed against an innocent
socket.
Second bug, same symptom, different subsystem: the pooled relay sockets.
ResolveDescriptor returned the instant a descriptor verified, leaving the
relay's EOSE unread on a socket that goes straight back into g_relayPool,
so the next caller read our leftovers as its own answer -- permanently one
message out of step, and no code path anywhere inspected the subscription
id that would have caught it. Worse, BtfResolve set lease.fOk = true on
every failure path including a dead RecvText, labelling it 'exchange
completed, just no match'. fOk = false is the only thing that ever closes
a pooled socket or clears fConnected, so a relay connection that died was
pinned in the pool, reported healthy, and reused for the life of the
process. That is 'no Nostr descriptor found' for a descriptor that nak can
see, cleared only by a restart.
Now: read to EOSE before returning, match the subscription id, and keep a
pooled socket only when the relay actually finished the conversation.
(cherry picked from commit ed4640c306cc46dbdbd92cb6dc55e7528651269c)
|
|
The peer-height gate replaced a tip-age check that was wrong in both
directions. It is a strict improvement and it is still blind in the one
case that matters most.
When the node goes deaf -- peers still in vNodes, nothing arriving -- no
peer announces anything, GetPeerMedianHeight() returns -1, and the
fallback mines our best chain. That fallback is right for a node that has
not been told yet and wrong for a node that has stopped being able to
hear, and a height nobody sent cannot tell those apart. A wedged node is
exactly a node with no peer height information, so the gate was silent
for the whole of heights 3966-4085: 120 blocks, 5890.50 BTF voided.
Time can tell them apart. GetPeerHeightAgeSecs() measures how long since
any peer reported a height, counting from first use rather than the epoch
so a starting node does not refuse jobs before its first handshake. Past
20 minutes the pool stops issuing jobs and says so unconditionally in the
log. /poolmineblind overrides it: without an escape hatch a genuinely
isolated pool that is 90% of the hashrate could never restart its own
chain, which is the deadlock the tip-age check used to cause.
Also stop nStartingHeight from going stale. It was a snapshot taken once
during the version handshake and never touched again, so on a long-lived
connection the median is built from heights the peers had hours ago, and
the behind-the-network check drifts toward 'everyone is behind us' and
quietly stops firing. A peer that hands us a block is demonstrably at
least at that block's height.
(cherry picked from commit 0800c18faf703b4acde1b131d0b742d37a86d92b)
|
|
The sync gate compared the tip timestamp against 30 minutes. That cannot
distinguish "we are not synced" from "the network is slow", and it failed
in both directions on the same day.
A pool mining its own fork keeps its tip fresh, so the check passed for
the whole of heights 3966-4085 -- 120 consecutive blocks, all orphaned,
5890.50 BTF of payouts voided, and nothing in the log to say so. Then,
once the pool stopped finding blocks, the same check refused to issue
jobs against a tip that was the network's own best. The pool is ~90% of
the hashrate, so nobody else could move the chain either: stale tip, no
jobs; no jobs, stale tip.
Gate on the height peers announce in their version message instead.
Median, so one peer cannot stall us with an absurd claim, and a block of
slack to match WarnIfBehind. Peers predating the announcement report -1
and are ignored; with no reports at all we mine our best chain, which is
what the node does anyway without a pool attached.
(cherry picked from commit 4e76e34f37a82c0885f75db7dfb9e49db410171d)
|
|
The exec queue was memory-only, and FlushMaturePayouts erased a matured entry
from pending_payouts.json *before* handing it over. Any restart in that window
lost those payments with no record that anyone was owed anything. It was not
hypothetical: a container restart on 2026-07-30 silently dropped a backlog of
~1,156 BTF, and there was nothing left to reconstruct it from.
Re-queueing blindly would have double-paid whoever had already been sent to,
so the payment needs to be idempotent rather than merely persistent.
In-flight work is now a persisted list of per-payment records
(paying_payouts.json), one per (block, address) -- the granularity SendMoney
works at, and therefore the granularity at which a payment can be half-done
when the process dies. Records are written, via a temp file and rename so a
crash cannot truncate them, *before* the ledger entry is erased, so the debt
is always described by at least one of the two files. Each attempt updates the
record (retries, deferrals, backoff); settling it removes the record and
rewrites the file, which is what stops a restart from paying twice.
On startup ReconcilePayouts runs before the payout thread:
- a record whose payment the wallet already shows -- matched on the payout
comment plus the exact amount and destination -- is dropped, not resent.
SendMoney commits the transaction to the wallet before it returns, so a
crash immediately after sending is recoverable.
- a ledger entry that has already been exploded into records is dropped,
which is the crash-in-the-window case where both files describe one debt.
The wallet check is deliberately biased towards "already paid": skipping a
payment is visible in the log and recoverable, paying twice is neither.
Verified live: sampling paying_payouts.json during a payout burst caught 3
records / 18.56 BTF mid-flight, cleared once paid, and the paid lines now
carry the txid.
(cherry picked from commit d0248f2e65dbe745f2e9b1d09e1f53a4354d3640)
|
|
Two fixes.
uint256's default constructor is empty (uint256.h:514), so the blockHash
field added in 6963862 was uninitialised garbage on every PendingPayout
built without an explicit hash -- which is every entry loaded from a ledger
written before that commit. Those entries were meant to be grandfathered
past the orphan check on a zero hash; instead each carried a random value
that matched no block in mapBlockIndex, and the first maturity sweep after
deploy voided the whole loaded ledger: 100 entries, 4950 BTF. Only one
"was orphaned -- voiding" line reached the log because LogPrint collapsed
the other 99 as duplicates, so the destruction was nearly silent. Nothing
had been paid at that point and the file was restored from a pre-deploy
backup, so no obligation was lost. Assigning 0 explicitly is all that was
missing.
Separately, four of the twelve relays are dead. Probed all twelve from two
independent networks (a residential line and the pool's VPS) over v4 and v6,
with DNS confirmed against 1.1.1.1:
relay.current.fyi, relay.nostr.bg -- NXDOMAIN, no A and no AAAA. The
domains are gone. (A getaddrinfo call on the VPS reports an address for
them, but that is the VPS's own IPv6 handed back by its local resolver,
not a real host.)
relay.nostr.band, nostr.fmt.wiz.biz -- DNS resolves and TCP connects, but
the TLS handshake never completes, from either network. Whether they are
down or refusing non-browser clients, this node cannot use them.
The remaining eight all complete a WebSocket upgrade. A dead relay is not
free: each holds a connect/retry slot and delays discovery for no added
redundancy.
(cherry picked from commit 144bc96f150d23507d57d29cb339f177dcd6b092)
|
|
A 280-block reorg (heights 3075-3354) orphaned 292 of the pool's 707 blocks
and destroyed 14,600 BTF of coinbase. The payout ledger did not notice: it
records only matureAtHeight, and a reorg replaces the block at a height
without changing the height, so ~14.4k BTF of obligations outlived the coin
that was supposed to fund them. The pool then paid fork-era miners out of
real-chain coin until the wallet was empty -- 17,827 BTF paid against 17,900
BTF of matured supply, leaving under 1.1 BTF spendable and every subsequent
payment failing.
Three changes, all in the payout path:
- Record the funding block's hash and height, persist both, and void an
obligation whose block is no longer in the main chain. Checked both at
maturity and again in the exec queue, since an entry can sit there for
hours across retries and the reorg can land at any point in that window.
Entries loaded from a file written before this are grandfathered rather
than voided on a missing field.
- Stop spending the retry budget on maturity. SelectCoins failing for want
of spendable coin is not an error, but it consumed one of 1440 retries
every 60s, so a payment the pool fully intended to honour would hit
GIVING UP 24h later and the miner would silently never be paid. Those
attempts now defer without incrementing retries, logged hourly rather
than every minute. GIVING UP now means what it says: 1440 real failures
against a block still on-chain with the money available.
- Schedule at COINBASE_SPENDABLE_DEPTH instead of COINBASE_MATURITY. At
depth 100 a block's own coinbase is 20 blocks short of spendable, so its
payout had to come from unrelated mature coin and the pool carried a
permanent ~19-block float. The "96% of 934 payments went through at depth
100" note that justified the shorter wait was measured while this node was
alone on a private fork paying itself. The extra 20 blocks also put the
maturity gate past the reorg that caused all of the above.
(cherry picked from commit 696386220a3d2fac88cb8045931392f4909e0985)
|
|
SendMoney's "oversized transaction that requires a transaction fee of
0.00" is not an oversized transaction and there is no fee. That branch is
taken when nValue + nFeeRequired > GetBalance() -- an empty spendable
balance -- and the fee reads 0.00 because CreateTransaction failed before
setting one. 3,267 of these in debug.log sent me looking for a
transaction-size problem that does not exist. Report what is short, by how
much, and against what balance.
Name COINBASE_SPENDABLE_DEPTH in main.h and derive GetBlocksToMaturity
from it, because the distinction it captures is genuinely surprising:
CMerkleTx::GetCredit values a mined coinbase at zero until
COINBASE_MATURITY+20 deep, not COINBASE_MATURITY.
I first changed the payout gate to that deeper value, on the theory that
each payout was firing 20 blocks before its own coinbase could be spent.
Measuring it refuted that: across 934 payments, 96% succeeded at depth
100, the moment they were queued, which is only possible because a payout
is funded from whatever coin is already mature rather than from its own
block. The gate stays at COINBASE_MATURITY, with a comment recording the
measurement so the next person does not re-make the same change.
The real shortfall is structural and not addressed here: the pool pays out
~99% of each block's subsidy while only matured coinbases are spendable,
so it runs with almost no buffer, and any orphaned block it queued a
payout for drains that buffer permanently. The 24h retry loop absorbs this
-- 0 payouts forfeited -- but the buffer is what needs fixing.
(cherry picked from commit e28df01f90d621b0cd818f28d70a20b8ab5d1a8d)
|
|
Upstream #30 split rendezvous registration from waiting for a dial, so
RvServiceRegister now returns as soon as the relay lists the service and
no longer consumes the relay's 0x01 pairing byte. It updated its own
caller in net.cpp and left this one behind.
AcceptOneFn went straight from RvServiceRegister to BtfServiceWrap, so
BtfServiceWrap read that 0x01 as the first byte of the miner's 32-byte
ephemeral key and every byte after it was shifted by one. The X25519
handshake has no confirmation step, so it completed against a garbage
shared secret, the tunnel was handed to the Stratum event loop, and
nothing failed loudly: the pool logged a miner arriving and then dropped
it as "never authorized", while the miner logged a pool that closed on it
with gotSubscribeAck=0. Loopback miners were unaffected, which is exactly
what made it look like a relay problem.
Block in RvServiceWaitPaired before wrapping, and close and re-register if
the relay drops us first.
test_rvaccept.cpp covers both shapes against a real bitflashd on loopback:
the pre-fix order establishes a tunnel that carries no bytes, the fixed
order round-trips. Also corrects the btftunnel.h contract comment, which
still described the pre-#30 behaviour and is what the old code was written
against -- upstream's own test_relayproc.cpp, test_btftunnel.cpp and
test_btfrv.cpp are still on that stale contract and fail the same way.
(cherry picked from commit 3e3a36b2bf1e6ffb4861507089b743688619377f)
|
|
SSL_CTX_new leaves verification at SSL_VERIFY_NONE and loads no trust
store, and nothing here ever changed that. SNI was set, a certificate was
received, and it was then ignored: every wss:// relay connection this node
has ever made would have completed against an attacker's certificate.
Signed events limit the damage -- a MITM cannot forge an announcement
attributable to our key -- but it can drop the rendezvous registration so
miners never find the pool, censor subscriptions, or answer them with
arbitrary content, which is the input the allocation bounds exist to
contain.
Load the system trust store, require SSL_VERIFY_PEER, and bind the check
to the host we dialed with SSL_set1_host, since chain validation alone
would accept any certificate a trusted CA ever issued for any name. Floor
the protocol at TLS 1.2. Verification failures report the specific X509
reason rather than a generic handshake error, and a missing CA store is a
hard failure instead of a silent downgrade to trusting everything.
(cherry picked from commit 6020740e68463deb1cc3c5b477372fcd1a245adb)
|
|
The min-2-threads floor applied even when the user explicitly asked
for 1 thread, so /hashthreads=1 silently spawned 2.
(cherry picked from commit a67b9e84f4f7607d307081b4a9873f5d9e78cf8a)
|
|
Descriptor (38501) still published so miners can resolve the pool;
only the public pool listing event (38503) is suppressed.
(cherry picked from commit 17bbdae1ec97945c37c1d5b2486e5c01a4d7ad87)
|
|
A pool that just found its first blocks has zero mature balance until
100 blocks later; orphaned coinbases (observed: reorgs orphaned our
blocks 2651-2655) make that worse. 30 one-minute retries forfeited
payments that would have succeeded hours later.
(cherry picked from commit 49e80023ca23baee8a4b80cfb891e7a358dc39ba)
|
|
A pool started with /rvrelay=ONE still spawned an accept thread for every
Nostr-discovered relay (BtfAllRelays always merged g_discoveredRelays), and
each thread optimistically overwrote the advertised meeting node before
registering -- so a discovered Tailscale-only relay (a mesh host)
flapped the published .btf descriptor and public miners could not connect.
fBtfRelayPinned (set only when /rvrelay parses) makes BtfAllRelays return
exactly the pinned list. Also validate the /rvrelay value at parse time:
with the pin in place no discovered relay can mask a typo.
(cherry picked from commit 0a4d2b02fa5382c5e788b5f0e2f8a844695dec47)
|
|
- RebuildJob: no jobs with zero peers or a tip older than 30min;
mining unsynced produced orphaned blocks and payouts that can
never be paid (observed in production: 8 orphan 'found blocks')
- payout exec: on SendMoney failure requeue with 60s backoff, up to
30 attempts, instead of silently forfeiting the payment
- nostr: add oxtr.dev, nos.social, fmt.wiz.biz, current.fyi, nostr.bg
(12 relays total) for announcement redundancy
(cherry picked from commit f1986cd7e4a6f69469a2256eb96ff03f3f146792)
|
|
Match the wording already used on the site and in the manual: the closing
line no longer calls the software experimental, keeping only the young-network
and stay-current caution.
|
|
- name the -walletbackend=sqlite|bdb override and the wallet-backend
marker in the wallet section
- add -nomanagedtor to the Tor flag list and note that the desktop app
starts a managed Tor automatically, while a headless node asks for it
|
|
- describe the recovery phrase as a BIP44 tree on m/44'/4346950'/0'/...,
coin type 4346950 = 0x425446 "BTF", registered in SLIP-0044
- note the SQLite default in "The file" and that -backupwallet writes a
matching self-contained copy; scope the Berkeley DB recovery notes to a
legacy wallet.dat
- at-a-glance: name BIP44 and add a wallet-storage row
|
|
- add a Tor badge to the header row
- point Download and the quick-start link at releases.bitflash.network;
the relative ../../releases/latest 404s outside GitHub
- windows quick start: the published package bundles Tor and starts a
managed Tor automatically, with -nomanagedtor to turn it off
- note the single-file SQLite wallet default (1.2.20) and link
docs/storage.md
- bump the checksum-mirror example to v1.2.20
|
|
main
|
|
Name the release artifacts for v1.2.20. Matches BITFLASH_VERSION_STRING.
|
|
from fix-selftest-exitcode into main
|
|
The Linux branch of the child-process helper returned system()s raw wait
status instead of the exit code, so an exit code of 2 read back as 512.
Every assertion that checks a specific code (== 2), like the locked
recovery-audit test, failed only on Linux while the node itself was
correct. Windows was unaffected (_spawnv returns the code directly).
Unwrap with WIFEXITED/WEXITSTATUS so both platforms return the same value.
Test harness only; no change to node behavior.
|
|
experimental' (#64) from remove-sqlite-experimental into main
|
|
Drop the "experimental" label from the wallet code and docs now that
SQLite is the default backend for new wallets, is coin-proven, and carries
encrypt, backup, atomic seed writes, and the storage diagnostics.
- main_gui.cpp: -help text, the unknown-backend error (SQLite is no longer
"experimental" and Berkeley DB is no longer "the default"), and the
startup banner ("Wallet backend: SQLite").
- walletcmd.h: the export command comment.
- docs/storage.md: rewrite the SQLite section from staging-only ("runtime
still opens Berkeley DB", "no runtime backend flag exists yet") to the
shipped state -- default backend, conversion wizard, per-backend ops --
and reframe the migration order as shipped in v1.2.20.
|
|
wizard' (#63) from wallet-wizard-restart into main
|
|
The About dialog had "v1.1.0" hardcoded and no home for the release
version, so it drifted. Add a single BITFLASH_VERSION_STRING in
headers_core.h (bumped at release, unrelated to the protocol nVersion
fields) and show it in About -- now v1.2.20.
Fill the empty line under the version with a clickable
https://bitflash.network that opens in the browser (ShellExecute on
Windows, xdg-open elsewhere), and grow the dialog to fit it.
|
|
Follow-up to the Restart Now button. Addresses the edge cases the review
flagged, and drops the fragile batch layer entirely:
- Windows no longer writes a .bat and shells through cmd (which re-parsed
the arguments and could leave a stray file on failure). It builds the
command line once, quoted correctly (only backslashes before a quote are
doubled, so ordinary paths are not mangled), and CreateProcess launches
the new instance directly.
- The delay moved into the new instance: a -restartwait flag makes it wait
~2.5s before it opens the network port and the block index, so the old
process is gone first. No shell delay hack. Verified: -restartwait adds
the wait (0.16s -> 2.67s).
- Linux re-execs /proc/self/exe (absolute) and only falls back to argv[0],
which may be relative or a bare PATH name that execv cannot resolve.
- This instance now exits only if the relaunch actually started; on failure
it keeps running and tells the user to restart manually, so a failed
relaunch never leaves them with no node.
GUI/process wiring only; wallet code unchanged, suites still green.
|
|
gaps
Two things that read wrong once the wallet is on SQLite:
- The Wallet Safety backup dialog defaulted the file name to .dat, but
BackupWallet copies wallet.sqlite under the SQLite backend -- so the
suggested name put SQLite contents in a wallet-backup-*.dat file. The
default now matches the live backend (.sqlite on SQLite, .dat on BDB).
- The recovery audit called coins on non-phrase keys "wallet.dat-only" and
told the user to "keep wallet.dat backups". Under SQLite those coins live
in wallet.sqlite, not wallet.dat. Reworded to the backend-neutral truth:
"not phrase-backed" balance, and "keep a file backup of this wallet" --
accurate on either backend. Same wording in the CLI -recoveryaudit and
the GUI Wallet Safety panel.
Display/naming only; audit numbers and BackupWallet are unchanged. All 13
wallet suites still pass.
|
|
After converting to SQLite the wizard said "restart to apply" but offered
no way to do it, so a user could close the dialog and keep mining on the
old Berkeley DB wallet without realizing the switch had not taken effect --
and coins mined in the meantime land only in wallet.dat, not the SQLite
wallet they think they are now on.
The post-conversion screen now offers "Restart Now", which relaunches
Bitflash with the same command line and datadir (a short delay lets this
instance release the network port and block-index files first) and asks
this one to exit, so mining resumes on the SQLite wallet. "Later" keeps
the old behaviour. A dimmed line warns that mining before the restart
still credits the old wallet.
wallet.dat is still never touched. The conversion action itself is
unchanged (wallet-convert self-test still green); this is GUI wiring only.
|
|
first-run conversion wizard' (#62) from wallet-sqlite-default into main
|
|
A first-run dialog for an existing wallet.dat with no backend choice yet:
"Convert to SQLite" exports the wallet and records the sqlite marker so
the next start opens it; "Keep Berkeley DB" records that choice so the
offer does not return; "Decide later" leaves no marker and asks again.
wallet.dat is never touched, and the change takes effect on restart --
matching the restart-to-apply model.
The conversion itself is DoConvertWalletToSQLite(), a headless-testable
function the button calls: it reuses the export path (extracted as
ExportActiveWalletToSQLite, shared with -walletsqliteexport, which keeps
its own specific messages), refuses if already on SQLite or if a
wallet.sqlite exists, and writes the marker only after a verified export.
New wallet-convert self-test proves the action end to end: build a
Berkeley DB wallet, convert, and confirm wallet.sqlite + the marker are
written, wallet.dat is left untouched, a second convert refuses, and a
flagless restart opens the converted SQLite wallet with its phrase intact.
All wallet suites pass. The modal itself is thin UI over this tested
action.
|
|
A fresh install with no wallet.dat now starts on the SQLite backend and
creates wallet.sqlite, instead of a Berkeley DB wallet.dat. The choice is
recorded in a plain-text <datadir>/wallet-backend marker so it survives a
restart without a command-line flag.
Backend resolution when no -walletbackend is given: an existing
wallet.sqlite (or a "sqlite" marker) stays on SQLite; an existing
wallet.dat stays on Berkeley DB -- nothing is converted behind the user;
a fresh datadir starts on SQLite. An explicit -walletbackend=bdb or a
"bdb" marker forces Berkeley DB and overrides the default.
Adds CreateNewSQLiteWallet() (WalletSQLiteRuntimeOpen gains fMustExist so
the create path can make a new file) which mints the first default key
straight into the SQLite store via the same FinishLoadedWallet() call the
Berkeley DB create path uses, then cleans up on failure. Marker read/write
helpers are plain text, hand-editable.
New wallet-backend-default self-test proves the resolution in child
processes (fresh->SQLite+marker, existing wallet.dat->BDB, "bdb"
marker->BDB, explicit flag->wallet.dat). The three wallet.dat-specific
suites now ask for -walletbackend=bdb explicitly, since a flagless fresh
datadir is SQLite now. All wallet suites pass.
|
|
atomicity, one-shot commands)' (#61) from wallet-sqlite-encrypt into main
|
|
-walletstorageaudit, -walletstoragecheck and -backupwallet all read
wallet.dat regardless of the selected backend. Under -walletbackend=sqlite
that is wrong, and for backup it is dangerous: it silently backs up the
stale plaintext wallet.dat instead of the live wallet.sqlite.
- ScanWalletRecords() routes to the active SQLite store instead of a
Berkeley DB cursor (it returned an error under the runtime before). This
is the single choke point audit and check scan through.
- WalletSQLiteRuntimeOpenReadOnly() opens wallet.sqlite for scanning
without loading or validating records, so the pre-load diagnostics can
inspect a wallet the loader would refuse. main_gui opens it for
audit/check when the SQLite backend is selected (and rejects an unknown
backend value there too).
- BackupWallet() copies wallet.sqlite (checkpoint, then the temp/old/rename
dance; a checkpointed SQLite file is self-contained, no lsn_reset).
wallet-sqlite-encrypt self-test proves routing on a datadir where the
SQLite wallet is encrypted and wallet.dat beside it is still plaintext:
audit with the flag reports the encrypted seed, without it reports the
plaintext one; the SQLite backup carries no plaintext key and loadchecks
as a standalone wallet.
|
|
The SQLite runtime wrote each record as its own autocommit, so a group
that must land together -- hdmaster and hdchaincode, the two halves of an
HD seed -- could be left half-written by a failure or crash between them,
the incomplete-seed state -walletstoragecheck exists to catch. Berkeley DB
was no worse (it is not transactional here either) but no better.
Add WalletSQLiteRuntimeBeginTxn/CommitTxn/RollbackTxn -- no-ops that
succeed when the SQLite backend is not active, so Berkeley DB keeps its
existing per-write behaviour -- and wrap WriteHDMaster in them so the seed
lands all-or-nothing under SQLite. The one BDB-atomic wallet rewrite, the
encrypt path, is already matched by its own transaction in the SQLite
encrypt branch; every other wallet multi-write is ordering-based on both
backends by design.
wallet-sqlite self-test now proves the bracket: a rolled-back runtime
write leaves no record, a committed one persists, and WriteHDMaster lands
both halves of the seed.
|
|
EncryptWallet() had only a Berkeley DB path: it rebuilt a wallet.dat temp
file and swapped it in. Under -walletbackend=sqlite that was wrong -- it
would leave the live wallet.sqlite unencrypted and write a divergent
wallet.dat instead. Add a SQLite persistence branch that rebuilds a fresh
wallet.sqlite in one transaction (public records copied from the live
store, encrypted key material serialized byte-for-byte the way the BDB
path writes it), checkpoints it, and swaps it in with a .before-encrypt
backup, mirroring the BDB rename dance. A fresh file rather than an
in-place rewrite keeps the change atomic and leaves no plaintext key in a
freed page.
CmdEncryptWallet skips PurgeDbEnvironmentLogs under SQLite (no Berkeley DB
environment behind the wallet) and reports the backend it used.
New wallet-sqlite-encrypt self-test drives the real command path in child
processes: export a wallet to SQLite, encrypt it through
-walletbackend=sqlite, and prove it comes out encrypted, unlockable with
the right passphrase, refusing the wrong one, with its recovery phrase
intact and the original wallet.dat left untouched.
|
|
net-per-ip-connlimit into main
|
|
from wallet-sqlite-activate into main
|
|
Review follow-up. A managed-Tor node receives every inbound onion peer
over its local hidden-service listener, so they all accept as 127.0.0.1.
Capping loopback would cap onion peers as one group and refuse the 33rd
legitimate onion peer -- the wrong limit for a network moving toward
onion-only. Exempt loopback (127.0.0.0/8): the global MAX_CONNECTIONS
still bounds the total, and the direct-IP flood this defends against
never arrives over loopback.
Bench: a 40-connection flood from 127.0.0.1 is now held in full (no
per-IP refusal), while a flood from the machine's LAN address still caps
at exactly 32.
|
|
Review follow-up on the SQLite activation. Two guards, both fail-closed;
neither is a correctness fix -- FinishLoadedWallet already refuses a
missing default key -- but they make the refusal say what happened:
- an empty or non-wallet wallet.sqlite is refused with "no default key
record; it may be empty or not a wallet export", instead of the generic
"no usable default public key" that reads as corruption;
- a misspelled -walletbackend value (e.g. sqlit) is refused instead of
silently falling through to Berkeley DB; -walletbackend=bdb is accepted
as the explicit default.
|
|
A single IP could hold every inbound slot up to MAX_CONNECTIONS -- which
is what a connection-exhaustion flood did to the rendezvous relays, one
address parking hundreds of sockets and starving real peers. Add
MAX_CONNECTIONS_PER_IP (32): the accept path counts existing connections
from the same address and refuses a new one past the cap. The relays'
firewall connlimit does the same a layer lower; this makes every build
carry the defense, including future poll()-based relays with no FD ceiling.
|
|
Adds -walletbackend=sqlite: an opt-in flag that runs the node on
<datadir>/wallet.sqlite instead of wallet.dat. It requires an export to
exist first (no silent migration) and prints a step-by-step guide when
one does not; wallet.dat is never opened or touched, so a plain restart
goes back to Berkeley DB.
- LoadWalletFromSQLiteRuntime opens the export read-write, keeps the
handle open as the active backend, loads records into memory through
the same visitor/reconcile/finish path as the read-only staging load,
and never fabricates a default key.
- Shutdown checkpoints and closes the SQLite backend; a kill before that
loses nothing because WAL + synchronous=FULL already made every
committed write durable.
- Self-test proves runtime writes persist across a close and reopen.
BDB stays the default: nothing activates the SQLite runtime unless the
flag is given. One-shot audit/recovery commands still use Berkeley DB.
|
|
from wallet-sqlite-runtime-write into main
|
|
|
|
wallet-sqlite-typed-load-proof into main
|
|
|
|
wallet-storage-backend-stage into main
|
|
|
|
wallet-sqlite-runtime into main
|
|
|
|
from wallet-sqlite-loadcheck into main
|
|
|
|
wallet-sqlite-cleanup-failures into main
|
|
|
|
from wallet-sqlite-restore into main
|