upleb.uk

Public git repos — served from a NIP-34 GRASP relay at git.upleb.uk

summaryrefslogtreecommitdiff
path: root/src/nostr.cpp
diff options
context:
space:
mode:
authorBitflash <btf01@proton.me>2026-07-28 12:55:04 -0300
committerGitHub <noreply@github.com>2026-07-28 12:55:04 -0300
commit663784eae9b04ec71e3e1eebf445910e5a8add42 (patch)
tree1482b0778a11c1b46cfbd476a5d91e12ef31204b /src/nostr.cpp
parent1d0314c0d220231afc56193ba66b70234fa24092 (diff)
Merge PR #6 (large refactor), preserving main's fixes (#9)
* Add Linux GUI installer via Wine * Fix launcher creation and add BTF_RELAY env var support * Pool server, Linux GUI, unified binary, README update * Root Makefile: make linux / make windows, cleaner README * Remove headless.cpp * Fix XPM color count mismatch causing toolbar image errors on Linux * XPM icons: same code on Windows and Linux, no ifdefs around image loading * Fix GTK icon crash and Linux printf format * btfrv: 10s socket timeout on relay connects * Faster peer discovery: more dials per pass, shorter backoff and sleep * Parallel btf peer connections * Fix GetMyExternalIP - was disabled, now uses modern IP echo services * Fix tunnel disconnect on idle and restore external IP detection * Filter stale Nostr descriptors * Sort peers by freshness, fresh first * Pool: add .btf Stratum listener — miners connect via rendezvous, no port forwarding needed * btf pool: self-replacing listeners, one per relay, no queuing between miners * Mining modes: solo/operator/participant — pool server on-demand, participant Stratum client * Status bar: show external IP and .btf address * README: clean rewrite, concept-first, minimal * make linux produces AppImage * make linux produces AppImage * Pool dashboard: connect instructions, tabs, share bars * Perf fixes: GetMyExternalIP threaded, btf listener counted * Perf fixes: GetMyExternalIP threaded, btf listener counted * Fix pool payouts: defer until coinbase maturity * Pool: clean mode switching, IP warning * Pool: clean mode switching * Fix gPoolRunning: remove static * AppImage: use real icon from bitcoin.ico * Fix exit crash; Mining menu * Fix exit crash; Mining menu * Fix exit crash; Mining menu * Fix exit crash; Mining menu * Fix exit crash; Mining menu * Windows build: use mingw32-make * Windows build: simpler make invocation * Add deps-windows target * Windows build fixes * Windows build fixes * Windows makefile: fix obj dir * Windows: output to repo root * Windows: bundle DLLs into zip * Replace wxWidgets with ImGui+GLFW: same build on Linux and Windows * Fix %I64d that slipped back through rebase (main.h, net.h) * MinGW: add _mkdir shim for non-MSVC Windows; fix %I64d * Fix gPoolRunning extern outside ifndef _WIN32 * Remove -lboost_system: boost is header-only in this codebase * Status bar: copy buttons for IP and .btf address * Windows: add app icon via windres, strip wx stuff from rc file * Clean GUI: single Options menu item, bigger font, better spacing, cleaner options dialog; update README * GUI: embed Roboto font at 16px for clear readability on all platforms * Windows: windres depends on obj dir target * Fix balance hidden off-screen: put on own line below address bar * Windows: inline mkdir in every rule, remove dir target that confused make * Windows: move all target to top so it's the default rule * Fix pool server not starting: always start when operator selected, re-sync dialog state on open * Write debug.log to data directory (AppData/Roaming/Bitflash on Windows) * Participant miner: use wallet address as username so payouts go to wallet * Add .gitattributes to lock LF line endings, prevent CRLF drift * Refactor pool mining to btf-only transport * Persist pool settings and startup wiring * Prune stale peers and expose pool queries * Fix build regressions and add CLI mode/arg parity * Fix rpc miner stats declaration order * Centralize pool runtime extern declarations * Separate operator/participant dashboards and restore peer stats * Add live telemetry strip and clearer per-mode runtime metrics * Reduce mining dashboard clutter and remove manual pool entry * Count only authorized pool miners * Seed initial pool job on startup * Use a real share target for pool work * Switch pool accounting to round-based shares * Fix nostr pool announcer stats call sites * Fix pool stats definition order * Improve pool worker status display * Add mutex include for participant status * Replace ImClamp in dashboard layout * Clarify participant work status * Log participant status transitions * Reduce peer discovery log noise * Add participant status badge * Track participant auth acknowledgements * Log and validate pool job delivery * Stop persisting mining mode/pool settings across restarts LoadWallet() runs after CLI flags are parsed in main_gui.cpp, and was restoring nMineMode/strParticipantPool/strPoolName/strPoolDashboardUrl/ dPoolFeePercent/fGenerateBitcoins from wallet.dat every time, silently overwriting whatever was just set on the command line or in the current Options session -- no error, no log line. This is why pool name/address could show correctly in the UI but stay stale in the logs (UI held the in-memory value just set; logs eventually reflected whatever LoadWallet clobbered it back to), and why mode changed behavior across restarts. Mode/pool config is now decided fresh every launch: CLI flags if given, otherwise the compiled-in default. Nothing is written back to wallet.dat for these settings anymore, so there's nothing left to go stale or fight over load order. * Add explicit Relay mode as the new startup default 4th mining mode: node runs/syncs, never mines, no pool server -- an explicit option instead of an implicit side-effect of 'Solo + never click Start Mining'. Now the compiled-in default (was Solo). - main.h: MINE_RELAY = 0, renumbered other modes (nothing compares nMineMode against raw integers elsewhere, verified). - main.cpp: default nMineMode = MINE_RELAY; BitcoinMiner() early-returns for Relay, mirroring the existing Participant-mode guard. - gui.cpp: added Relay radio button, status label, and telemetry-strip text. Start/Stop Mining button is hidden for Relay (meaningless -- it never mines by definition); OK forces fGenerateBitcoins off if you switch into Relay while previously mining. * Announce pool changes immediately instead of waiting up to 60s ThreadBtfPoolAnnouncer re-read strPoolName/dPoolFeePercent/etc live, but only once per iteration of its 60s sleep loop, so a pool name/fee change in Options could take up to a minute to reach other nodes. - nostr.h/.cpp: add gAnnounceNow flag; the announcer's sleep loop now wakes early and publishes immediately when it's set, then clears it. - gui.cpp: Options OK handler sets gAnnounceNow when operator pool settings actually changed (name/dashboard/fee/mode), so a saved change is announced right away instead of on the next tick. * Give each pool miner a distinct extranonce2 so jobs no longer overlap mining.notify previously broadcast one identical 80-byte header (baked from a single server-side extraNonce) to every connected miner, so any two miners on the same job searched the exact same nNonce space instead of disjoint ranges. Each Miner connection is now assigned a unique extranonce2 on connect. BuildMinerBlock() takes the shared job template and appends that miner's extranonce2 to the coinbase scriptSig, recomputing the merkle root, so every miner gets its own header variant of the same job. mining.submit validates and, on a full block, assembles the block using that same miner-specific variant (via m->extranonce2), not the shared template. No client-side protocol change needed: main.cpp's participant miner already just takes whatever header bytes mining.notify hands it and submits (jobId, nonce) back, so it works unmodified against per-miner headers. Still simplified vs standard Stratum (no coinb1/coinb2/merkle-branch wire split, no mining.extranonce.subscribe reply payload) since this protocol was already a custom simplification pre-existing this fix. Syntax-checked with g++ -fsyntax-only (full link needs secp256k1/ RandomX/GLFW not present in this sandbox). * Fix debug.log never being written on the real Windows build printf-to-debug.log routing was gated on: #if defined(_WINDOWS) || !defined(_WIN32) _WINDOWS is only ever defined by the legacy, unused MSVC makefile.vc. The actual Windows build (makefile.mingw) never defines it, and _WIN32 IS defined there (mingw builtin), so on real Windows this condition was always false || false = false. printf() fell through to plain libc printf (stdout only), so debug.log was never created, and output was only visible if the process happened to be run attached to a console. OutputDebugStringF() was already platform-safe internally (always writes debug.log; only echoes to console when not on Windows), so the outer condition was pure dead weight. Made the routing unconditional. * Reduce base UI font size one more step (16px -> 14px) * Remove Start/Stop Mining button; mining state follows selected mode The button let a user configure a pool then click a separate toggle, creating states where the button's label/action didn't match what the selected mode actually needed. OK now starts/stops the miner thread itself based on nMineMode: Relay clears fGenerateBitcoins, the other three modes set it and spawn ThreadBitcoinMiner if it isn't already running (guarded by vfThreadRunning[3]). * Fix participant miner stuck waiting for notify after buffered multi-line recv Root cause of the operator/participant pairing successfully but no work ever flowing: PoolParticipantMiner()'s receive loop only pulled one line out of the socket per select() wakeup. The server sends the authorize reply, mining.set_difficulty, and mining.notify back-to-back on the same connection, and these often land in a single TCP segment — one recv() inside StratumRecvLine() buffers all three lines into 'buf', but only the first was extracted and processed. The remaining lines (frequently mining.notify itself) sat in 'buf' indefinitely, because select() only reports readability of the OS socket, not whether our own buffer still holds unprocessed complete lines — and the server had nothing further to send once the initial burst was over, so select() never fired again. Fix: drain any complete line already sitting in 'buf' first; only fall back to select()+recv() when 'buf' has no complete line left. * Fix mining.submit rejecting every share due to wrong param-count check Server required params.size() >= 4 for mining.submit, but the client only ever sends 3: [username, jobId, nonceHex] (indices 0-2, matching what the handler actually reads: p[1]=jobId, p[2]=nonceHex). Every submission was bounced with 'bad params' before its nonce was ever checked against the job, producing an endless share-sent/share-rejected loop with no other visible cause. Likely a leftover from the standard 5-field Stratum mining.submit shape, which this simplified 3-field protocol never matched. Threshold corrected to size() >= 3. * Fix mining.submit rejecting legitimate shares that weren't also winning blocks Pool share difficulty is intentionally ~256x easier than full network difficulty (MakeShareTarget() shifts the block target left 8 bits), so miners are expected to submit many shares per actual block found. But the handler replied to the miner based on ProcessBlock()/CheckBlock() success — validation against the much harder network target — rather than on whether the share met pool difficulty. Every legitimate share that wasn't also a winning block (the vast majority, by design) failed CheckBlock()'s nBits check and got reported to the miner as rejected. Now: reply(true) as soon as a submission clears pool-share difficulty. Only if it ALSO clears full network difficulty (job.target, not job.shareTarget) does it get submitted as a block — with no second reply sent, since the share was already acknowledged. * Fix dead-peer dial priority: freshness sort was undone by full shuffle ConnectDiscoveredBtfPeers sorted candidates newest-first, then immediately shuffled the whole list, making the freshness sort a no-op — every pass was fully random. Meanwhile g_btfPeerFails was tracked but never used to prioritize, so a peer that had failed 20 times in a row got equal footing with an untried or previously-successful peer; the only effect was backoff, capped at 15 minutes, so dead descriptors cycled back into the dial pool constantly. - Track successful connections (g_btfPeerEverConnected) and use it, plus fail history, to order candidates in tiers: known-good, then untried, then known-bad (worst offenders last). Freshness order is preserved within each tier instead of being discarded. - Raise the backoff cap for chronic failures (6+ in a row) from 15 minutes to 6 hours, so descriptors that are clearly dead stop consuming dial budget every pass while still getting retried eventually. * Batch-resolve .btf descriptors to stop hammering public relays during dial passes Root cause of 'stuck at 1 peer' despite good peer discovery: each parallel dial thread in ConnectDiscoveredBtfPeers called ConnectNodeBtf, which itself walked all configured relays (relay.damus.io, nos.lol, relay.nostr.band, relay.snort.social) to resolve that one peer's descriptor. With up to BTF_DIALS_PER_PASS (12) threads dialing at once, a single pass could fire up to 12 * nNostrRelays near-simultaneous fresh connections at the same handful of public relays -- easily enough to trip rate limits or get handshakes refused (exactly what the logs showed: 'handshake refused by relay.damus.io', repeated connect failures to relay.nostr.band / relay.snort.social). Peers that were genuinely reachable looked dead simply because their descriptor couldn't be fetched under that load, and then got penalized as known-bad by the dial-priority tiering from the previous fix. - Add BtfResolveMany: resolves every still-unresolved candidate's descriptor in a single Nostr REQ per relay (multiple authors in one filter), instead of one relay connection per candidate. A 12-candidate pass now costs at most a few relay round-trips total instead of dozens. - Add ConnectNodeBtfResolved / ConnectNodeBtfTail (net.cpp): given an already-resolved descriptor, dial straight to the peer's rendezvous node with zero further relay contact. ConnectDiscoveredBtfPeers now batch- resolves all candidates up front, then fans out tunnel-only connect threads in parallel -- no thundering herd against the relays. - Candidates that fail to resolve are recorded as a failure immediately (same backoff/fail-count bookkeeping as before) instead of spawning a thread with nothing to dial. * Stop reconnecting to relays constantly: pool one persistent connection per relay Real root cause of the ongoing 'handshake refused' / 'connect failed' spam and peer count stuck at 0-1: every relay-touching operation (startup self- test, per-pass seeding, batch descriptor resolve, pool announce, pool query) opened its own brand-new CWebSocket every single time it ran. Two independent threads (ThreadNostrSeed and ThreadBtfPoolAnnouncer) each reconnect from scratch on their own ~60s cycle, so a node could fire well over a dozen fresh TCP+TLS+WS handshakes at the same 4 public relays inside any given minute. Public relays -- relay.damus.io especially -- rate-limit new connections per source IP fairly aggressively, so this node was effectively DoS-ing itself: peers were genuinely reachable, but their descriptors couldn't be fetched because the relay had already started refusing this node's own repeated reconnects. - Add a small per-relay connection pool (g_relayPool) and an RAII CRelayLease that reuses a relay's live socket across calls instead of reconnecting every time, only reconnecting when a socket actually drops or was never established. - Route every relay call site through it: the startup self-test, the self-test loop, SeedFromRelay, BtfResolve, BtfResolveMany, BtfPublishPoolAnnouncement, BtfQueryPoolAnnouncement. - Only log 'connected to <relay>' on an actual fresh (re)connect, not on every pass that's just reusing an already-live socket, so the log reflects what's really happening instead of implying constant churn. This cuts fresh relay connections from several per minute down to essentially one per relay for the life of the process (until a relay actually drops the connection), which should make the rate-limit-driven refusals go away and let descriptor resolution -- and therefore dialing -- actually succeed at the rate the peer list implies it should. * Fix wallet crash on rejected send; remove legacy IP-based peer discovery - SendMoney() no longer throws when a committed tx is rejected by AcceptTransaction() (e.g. raced input already spent). It now unwinds the wallet state via RollbackTransactionSpent() and returns false, which the GUI already handles as 'Transaction failed.' - Removed the dead IP-based peer discovery/connect path now that discovery is entirely Nostr/.btf: CAddrDB/addr.dat, mapAddresses, AddAddress, ConnectNode(CAddress), FindNode(CAddress), ConnectSocket (SOCKS4 proxy helper), addrProxy, ThreadOpenConnections/2 class-C dialer and its thread spawn, the addr/getaddr P2P handlers, and the dead vAddrToSend/setAddrKnown fields on CNode. - Deleted irc.cpp/irc.h (already-disabled legacy seed code) and scrubbed references from headers_core.h and all four makefiles. * Raise peer connection target and add dial diversity; fix IPv4-only relay connect - BTF_TARGET_CONN: 8 -> 24. With ~100 known .btf peers, average degree needs only to exceed ln(N)~=4.6 for the network to stay connected with overwhelming probability (Erdos-Renyi); 24 is a healthy margin. - ConnectDiscoveredBtfPeers now splits each pass's dial budget ~70/30 between the existing reliability-tiered order and a random sample from the full fresh peer pool. Pure reliability bias means every node converges on the same 'known good' hub set, which correlates peer sets across the network -- a relay outage then drops the same edges everywhere at once instead of independent ones. The random slice keeps a floor of uncorrelated edges. - CWebSocket::Connect (relay websocket) now resolves with AF_UNSPEC and tries every returned address (IPv6 or IPv4) in turn instead of hard-failing on AF_INET-only resolution, which was one plausible cause of the relay.nostr.band:443 connect failures (CDN-fronted relays increasingly rate-limit/filter the v4 path). * Remove accidentally committed build binary; ignore src/bitflash * Fix pool participants getting stuck after the first job JobWatcherFn only rebuilt/broadcast a new Stratum job when a new block arrived on this node or no job existed yet. Once a job was built after startup, gHaveJob stayed true forever unless *this* node found the block, so with no new block (slow interval, or this node just not seeing one) every connected miner sat hashing the exact same job -- stale timestamp, stale mempool snapshot -- indefinitely. From the miner's side that's indistinguishable from being stuck: it worked at connect (gets the one-time job sent on authorize) and then nothing ever arrives again. Added a 60s periodic refresh so the job (and therefore mining.notify) gets rebroadcast on a timer too, same as a normal stratum pool, not only on new-block/no-job events. * Diversify Nostr discovery relay list Went from 4 relays to 7, and reduced Cloudflare concentration: previously 2 of 4 (damus.io, snort.social) sat behind the same CDN, so a Cloudflare- side issue could take out half the list even though they're nominally independent relays. Kept damus.io (most established) as the one CF-fronted entry and filled the rest with different hosts (nos.lol, nostr.band on Hetzner, nostr21.com, offchain.pub, nostr.mom, primal.net) so peer discovery doesn't have a single shared upstream as a point of failure. 7 relays is within the commonly recommended 5-10 range for redundancy. * Fix data race in RebuildJob: chain-state reads weren't holding cs_main pindexBest, nBestHeight, GetNextWorkRequired() and GetMedianTimePast() were all read without cs_main. A block being accepted concurrently by the P2P thread could interleave with this: pindexPrev captured from the old best block, but nBestHeight already bumped to the new one by the time job.height is computed, or pindexBest reassigned mid-read. That mismatch would make QueuePayouts() schedule a payout for the wrong maturity height for whatever block gets found against that job. Wrapped the whole chain-state snapshot in one CRITICAL_BLOCK(cs_main), consistent with how the rest of the codebase touches this state. * add operator fee * Add categorized, burst-collapsing logging for pool/worker/payout flow debug.log was growing unboundedly because every printf() call wrote unconditionally, so loops that log once per item (discovered peers, connect attempts) produced a near-identical line per address with no cap. Add LogPrint(category, ...): a drop-in printf replacement that tags each call with a subsystem (pool/worker/payout/net/nostr) and collapses repeated calls from the same call site into one line per 3s window plus a '(xN similar in 3s)' summary, so log growth is bounded by elapsed time instead of event count. Wire it through the whole pool -> work -> payout pipeline (rpc.cpp operator side, main.cpp participant/worker side, net.cpp/nostr.cpp peer & relay discovery), and along the way fix several previously *silent* failure points that produced exactly the reported symptom (worker stuck asking for work, nothing logged): - Relay listener registration failing: no log at all, so a pool with every relay down looked identical to a healthy idle pool. - No relays configured: silent infinite 5s retry loop. - Initial job seed failing at pool startup: silent. - JobWatcher: added a stuck-job watchdog that, after 15s with no job available, logs a specific diagnosis (no chain tip yet vs. RebuildJob() failing) instead of retrying forever in silence. - QueuePayouts(): a block found with zero recorded shares queued no payout and logged nothing. - FlushMaturePayouts(): an unparseable payout address was skipped with no log. - Participant miner: the 'waiting for pool notify' stall was only ever shown as a transient GUI status string, never written to debug.log. Now logs on a 15s cadence while stalled, and after 120s proactively drops the tunnel so the existing reconnect-in-10s loop gets a fresh attempt instead of hanging indefinitely. - Participant miner: share rejections were only a transient GUI status string; now logged with the pool's rejection reason. All edited files verified with g++ -fsyntax-only (gnu++14). * Fix two bugs causing worker stuck + log bloat 1. Worker stuck asking for work (main.cpp): The participant miner checked msg["result"].is_boolean() to detect the mining.subscribe acknowledgement (id==1), but the pool sends an array ([[["mining.notify",sid],sid,4]]) as per Stratum spec, not a boolean. is_boolean() always returned false, so gotSubscribeAck was never set true. This didn't directly block authorize (id==2 path is independent), but it did mean the status never advanced through "subscribed", leaving the progress logging misleading. More importantly, the same wrong assumption meant any pool that also gated job dispatch on a prior subscribe confirmation would leave the worker stuck forever with no diagnosis. Fix: accept any non-null result for id==1 (the array is the session/extranonce info; its contents don't matter for our use case). 2. Log bloat from ProcessMessages (main.cpp): ProcessMessages() used a bare printf() which fired unconditionally for every P2P message received -- the bulk of debug.log was this one line repeating thousands of times. Convert to LogPrint("net", ...) so it is gated behind fDebug (off by default) and burst-collapsed by the dedup introduced in the previous commit. * Fix log bloat, em-dashes, dead code, status bar, Connected/Detected peers Log bloat: - Gate all per-message/per-block/per-loop printf in main.cpp, net.h, net.cpp behind LogAcceptsCategory("net") -- only visible with -debug. This includes: received:/sending: hex dumps, got inventory, askfor, getblocks, getdata, received block, AddToWallet, RelayWalletTransactions, ProcessMessage FAILED, mining loop N-transactions line, retarget block, socket recv/send errors. AddToBlockIndex new best and ProcessBlock ACCEPTED intentionally left always-on (one line per block). - PoolParticipantMiner: status -> now LogPrint(worker) not printf. Em-dashes: - Replace all UTF-8 em-dashes (0xE2 0x80 0x94) with ASCII '--' in every source file (rpc.cpp, gui.cpp, main.cpp, main_gui.cpp, net.cpp, nostr.cpp, util.cpp, btfrv.cpp, db.cpp, db.h, main.h, headers.h, headers_core.h, util.h). 40 occurrences across 14 files. Dead code: - Remove unused gJobSeq atomic from rpc.cpp (written, never read). - Fix minerReward: was computed but payout loop used blockReward directly (operator fee was advertised but never deducted). Now payout correctly uses minerReward = blockReward * (1 - feeFrac). Status bar / peers: - Add GetDiscoveredPeerCount() to nostr.cpp/nostr.h (reads g_btfPeers size under cs_btfPeers). - Replace scrollable inline status text with DrawStatusBar(): a separate ImGui overlay window pinned to bottom of screen (NoScrollbar, NoBringToDisplayFront). Labels changed from 'Peers N' to 'Connected peers: N Detected peers: N'. * Rewrite gui.cpp: proper fixed status bar, scrollable content child Layout architecture: - DrawMainWindow() covers the full display width, height minus STATUS_BAR_H (22px). NoScrollbar + NoScrollWithMouse on the outer window so the chrome (menu, address/balance strip, telemetry strip, separators) never scrolls. - A BeginChild("##scroll") fills the remaining height after the fixed chrome, sized to GetContentRegionAvail().y -- which is already the space between the cursor and the bottom of the outer window. All mode-specific content (transaction table, participant pool list, operator workers/payouts) lives inside this child. Content that overflows gets a scrollbar; nothing bleeds past the outer window edge. - DrawStatusBar() is a separate overlay window pinned to the bottom 22px, drawn last so it is always on top. NoBringToDisplayFront is NOT used here -- drawing last is sufficient and avoids interaction issues with dialogs. - Status bar background is fully opaque (not alpha-blended) so it never shows content behind it. Labels and content: - 'Peers N' replaced with 'Connected peers: N Detected peers: N' using GetDiscoveredPeerCount(). - Mining indicator: '* Mode' instead of the 'filled circle' Unicode glyph that renders as boxes on some Linux font configs. - FmtAge used consistently; 'blocks deep' shortened to 'N deep' in tx list. - Options dialog: em-dashes removed from RadioButton labels (ASCII '--'). - Removed usedY dead variable (GetCursorPosY() result was unused). * Fix blocking bugs, dead code, pool/participant flow correctness Bugs fixed: StratumRecvLine blocking (main.cpp): The participant miner's recv loop called recv() with no timeout after select() said the socket was readable. If the server sent a partial line (split across two TCP segments), the second recv() would block indefinitely waiting for the rest -- the classic 'stuck asking for work' hang even though the pool was alive and had sent data. Fix: single non-blocking recv() pass (MSG_DONTWAIT on Linux, FIONREAD+recv on Windows). If no complete line is in the buffer yet, return false; the outer select() loop retries when more data arrives. BroadcastJob holding mutex during send() (rpc.cpp): BroadcastJob() held gMinersMutex across all SendLine() calls. send() can block if a slow miner's TCP receive buffer is full. With the mutex held, MinerThreadFn could not acquire it to remove a disconnecting miner, causing a deadlock-adjacent stall: broadcast blocked on slow miner, disconnect blocked on broadcast, new jobs not delivered. Fix: snapshot {fd, authorised, address, extranonce2} under the mutex, release it, then send to each fd outside the lock. Dead code removed (rpc.cpp): - gRpcUser, gRpcPassword: defined, never read or set by anything. - dHashesPerSec, nHPSTimerStart: leftover Bitcoin 0.1 stubs, unused. - IsInitialBlockDownload(): stub that returned pindexBest==NULL, never called anywhere in this codebase. - struct Share: declared with three fields, never instantiated. - Duplicate comment block before payout section. Other cleanup (main.cpp): - STUCK_NO_JOB_TIMEOUT_SECS moved from inside the loop body to file scope (was 'static const' inside a loop -- correct but confusing). - pblock->print() in solo miner gated behind LogAcceptsCategory("net") -- was unconditionally dumping a full block to stdout on every solve. - ERROR in BitcoinMiner printf converted to LogPrint("net"). * Pool/participant: fix recv blocking, BroadcastJob mutex, listener spawn, escaped em-dash main.cpp -- StratumRecvLine rewrite: The previous version called recv() in a while loop -- if select() said the socket was readable but only a partial line had arrived, the second recv() call would block indefinitely waiting for the rest of the line. This was the root cause of the recurring stuck-worker bug. Rewrite: single non-blocking recv (MSG_DONTWAIT on Linux, FIONREAD+recv on Windows). If no complete newline is in the buffer yet, return false immediately; the outer 50ms select() loop retries when more data arrives. This means the hash loop always stays responsive regardless of network fragmentation. rpc.cpp -- BroadcastJob mutex held across send(): BroadcastJob held gMinersMutex for the entire duration of sending jobs to all connected miners. send() can block if a slow miner's TCP buffer is full. With the mutex held, MinerThreadFn could not acquire it to clean up a disconnecting miner, stalling the whole broadcast. Fix: snapshot {fd, authorised, address, extranonce2} under the mutex, release it, then send to each fd without holding any lock. rpc.cpp -- LaunchThread failure not handled in BtfStratumAcceptOneFn: The replacement listener thread was launched via LaunchThread() which wraps _beginthread() but discarded the return value. If _beginthread fails (thread limit, OOM), the listener count stayed at 1 with no thread behind it -- that relay would get no new listener until the 30s outer loop in BtfStratumAcceptFn fired. Fix: call _beginthread() directly, check the return value, decrement and log on failure. rpc.cpp -- escaped em-dash in log string: BtfStratumAcceptFn had a literal \xe2\x80\x94 in a LogPrint format string that survived the em-dash sweep (it was source-encoded, not a UTF-8 byte in the file). Replaced with '--'. * Rewrite pool server: single event loop, no per-miner threads Replace the old architecture (one thread per connected miner + separate JobWatcher thread + BtfStratumAcceptFn thread with listener count tracking) with a clean two-layer design: Accept layer (one thread per relay, AcceptOneFn): Each relay gets exactly one persistent thread. It loops: calls blocking RvServiceRegister (waits for a miner to knock), then BtfServiceWrap (echan handshake), then pushes the finished plain socket to a small lock-protected queue, then loops immediately to accept the next miner. No listener count tracking. No replacement thread spawning. No races around spawn failure. The relay always has exactly one pending accept. Event loop (PoolEventLoop, runs on the ThreadRPCServer thread): Single thread owns all connected miner sockets. Each tick: 1. Drain the accept queue -- pick up any newly-connected miners. 2. Check for a new block -- rebuild job and broadcast if so. 3. No-job watchdog -- log and retry if job is missing too long. 4. Periodic job refresh -- restamp nTime every 60s. 5. select() over all miner fds with 50ms timeout. 6. For each readable fd: non-blocking RecvLine, drain all complete lines, dispatch each to HandleLine, collect dead fds. 7. Refresh GUI stats snapshot every 2s. Because the event loop is the only writer to gMiners, no mutex is needed on the miner list. The only mutexes remaining are: - gStatsMutex: guards the GUI-visible snapshot (StatsSnapshot, gWorkerStats, gPayoutViews) written by the event loop and read by GUI threads via GetPoolOperatorStats/GetPoolWorkerStats/GetPendingPayouts. - gReadyMutex: guards the accept queue between AcceptOneFn threads and the event loop. Other improvements in this rewrite: - gPendingPayouts is event-loop-only (no mutex needed). - Round share tracking lives in the event loop (roundShareCount map + roundShareTotal counter) -- zeroed on block found, no per-miner stat mutex needed mid-round. - QueuePayouts takes the share snapshot as a parameter instead of reaching into gMinerStats under a separate mutex. - BroadcastJob no longer snapshots miners to avoid a mutex -- it runs directly in the event loop with exclusive access to gMiners. - AcceptOneFn backs off 30s on relay registration failure instead of returning -- one relay going down doesn't kill the accept thread. - No goto. No per-thread dynamic allocation for listener contexts on each accepted connection. * Thorough logging and correctness fixes across the full pool/participant flow Every failure point in the operator->miner->payout chain now logs exactly what broke, with enough context to diagnose without a debugger. Operator side (rpc.cpp): RebuildJob: logs 'no chain tip (pindexBest is null)' on failure instead of silently returning false. HandleLine: - JSON parse errors logged with the raw line (first 80 chars) - subscribe: logs fd and session id - authorize: logs address, worker, and whether the address is valid - submit: logs stale-job case with submitted vs current job id; logs high-hash with actual hash and target values; logs valid shares with nonce and running session count; logs BLOCK CANDIDATE separately from BLOCK ACCEPTED/REJECTED so a ProcessBlock failure is unambiguous - unknown method logged (reveals miner software quirks) - reply() logs send failures with method name QueuePayouts: logs total payout amount, miner count, fee applied, and how many miners were below dust threshold. FlushMaturePayouts: logs 'insufficient funds or wallet locked?' on SendMoney failure instead of just 'FAILED'. BroadcastJob: logs job id and miner count on every broadcast. Event loop: logs when each new miner socket enters the loop; logs disconnect with session and round share counts. No-job watchdog: RebuildJob retried every 5s (not every 50ms tick), eliminating cs_main hammering when there is no chain tip. AcceptLoopFn: re-polls BtfAllRelays every 60s so relays discovered after startup are picked up; logs each new relay thread spawned. AcceptOneFn: logs relay name on start/stop; logs BtfServiceWrap (echan handshake) failure separately from RvServiceRegister failure. RecvLine: disconnected out-param distinguishes 'socket closed' from 'no complete line yet', preventing incorrect disconnection on partial recv. Participant side (main.cpp): StratumRecvLine: same disconnected out-param fix as RecvLine in rpc.cpp. Previously a false return could mean either 'disconnect' or 'no data yet', causing spurious reconnects when a partial line arrived. PoolParticipantMiner rewritten with step-by-step logging: Step 1 resolve: logs specific failure (no descriptor vs bad format) Step 2 tunnel: logs relay and failure reason Step 3 address: warns explicitly if payout address cannot be determined Step 4 subscribe/authorize: logs each send and each response Step 5 RandomX: logs VM mode Step 6 main loop: - malformed mining.notify (wrong param count or short header/target) logged with sizes instead of silently ignored - authorize rejected: logs reason, does not disconnect (some pools reject bad addresses but still send work) - share accepted/rejected: logs share id and rejection reason - unknown server method logged - pool disconnect logged with handshake state at that moment (gotSubscribeAck, gotAuthorizeAck, haveJob) so you know exactly how far the connection got - nonce space exhausted logged (was silent before) - on exit: logs total hashes, shares sent, shares accepted * Fix build error: ImGuiWindowFlags_NoBringToDisplayFront -> NoBringToFrontOnFocus NoBringToDisplayFront does not exist in this version of ImGui (v1.89.x). The correct flag is ImGuiWindowFlags_NoBringToFrontOnFocus (bit 13). Used in two places: ##statusbar and ##main windows. Also remove dead variable usedY (GetCursorPosY() result assigned but never read) and replace the confusing 'ImGuiWindowFlags_NoScrollbar * 0' expression (evaluates to 0 but looks like a mistake) with a plain 0 in the BeginChild call. * Remove duplicate status logging from SetParticipantMiningStatus Every transition in PoolParticipantMiner already has a direct LogPrint with full context (addresses, job ids, nonces, error reasons). The LogPrint inside SetParticipantMiningStatus was firing on top of those, producing redundant 'PoolParticipantMiner: status -> ...' lines after every meaningful log entry. SetParticipantMiningStatus now just updates the GUI string silently. * Implement vardiff: stable share rate instead of fixed difficulty Previously the pool set shareTarget = blockTarget >> 8 for every miner forever. This meant at low network difficulty a fast miner found shares every few nonces (dozens per second), while at high difficulty a slow miner might go hours without one -- both are bad for fair payout accounting and pool stability. Pool (rpc.cpp): Per-miner difficulty that adjusts to target one share every 15 seconds. After every 5 shares, check actual seconds/share vs target. If the ratio is off by more than 50% (VARDIFF_RETARGET_RATIO), recompute: new_difficulty = current_difficulty / (actual_secs_per_share / target) i.e. shares coming too fast -> difficulty goes up; too slow -> down. Only apply if the change exceeds 10% to avoid jitter. Difficulty is bounded to [0.001, 1e12]. Each miner gets their own shareTarget derived from their difficulty via ShareTargetFromDifficulty(), which maps Stratum difficulty to a uint256 target using the Bitcoin difficulty-1 base (0x1d00ffff compact). New difficulty is sent to the miner via mining.set_difficulty. Participant (main.cpp): Apply the received difficulty value from mining.set_difficulty. Previously the handler was a no-op ('acknowledge only'); now it parses the difficulty double and logs the change. The share target used for comparison comes from the pool's mining.notify p[3] field, which the pool updates after each vardiff adjustment -- this is the standard approach and matches how cpuminer, xmrig etc. handle it. With vardiff, share rate is roughly constant regardless of miner speed or network difficulty. Payouts are proportional to shares submitted, so each miner's payout accurately reflects their CPU contribution. * Fix share target mismatch: pool rejected shares it should have accepted Root cause: the pool put job.shareTarget (from blockTarget<<8, a fixed per-job value) into mining.notify p[3], but checked m->shareTarget (from ShareTargetFromDifficulty, the vardiff value) on submit. With difficulty=1.0, ShareTargetFromDifficulty gives a much HARDER target than blockTarget<<8, so shares that the participant thought were valid (passed the easier notify target) were rejected by the pool (failed the harder difficulty-1 target). Fix: MakeNotifyParams now takes the per-miner shareTarget as an explicit parameter. SendJob passes m->shareTarget. Both sides -- the participant reading p[3] and HandleLine checking m->shareTarget on submit -- now use exactly the same value. Single source of truth. Cleanup: - Remove shareTarget field from StratumJob (was only set, never read after the MakeNotifyParams call was fixed). - Remove MakeShareTarget() function (was only used to populate that now-dead field). - Participant set_difficulty log: 'initial' vs 'updated' instead of 'unchanged' (first connect sends 1.0 which matches the default 1.0, so it was misleadingly logged as 'unchanged'). * Fix double broadcast, correct block reward calculation, improve payout log Double broadcast (event loop vs HandleLine): When a pool miner finds a block, HandleLine calls RebuildJob() + BroadcastJob(true) immediately. The event loop then detects nBestHeight changed (step 2) and was calling RebuildJob() + BroadcastJob(true) again -- miners received duplicate job broadcasts. Fix: step 2 now checks gHaveJob before rebuilding. If HandleLine already built and broadcast a job for this height (gHaveJob=true), step 2 just logs 'job already broadcast by submit handler' and skips. If the block came from an external peer (not our miner), gHaveJob is false and step 2 handles it as before. Block reward calculation in QueuePayouts: Was calling CBlock().GetBlockValue(0) on an empty block object. GetBlockValue uses nBestHeight globally, which by the time QueuePayouts runs has already been incremented to the height AFTER the found block. Fix: compute subsidy directly as '50 COIN >> (blockHeight / 210000)' using the actual found height. At current scale this makes no difference (same subsidy for thousands of blocks) but is now correct by construction. Payout log now shows: - Full block subsidy - Miner share after fee - Operator fee amount in BTF (not just percentage) - Number of recipients, maturity height, dust-filtered count Example: '[payout] block 2096: subsidy 50.00 BTF, miners get 49.00 (fee 2.00% = 1.00 to operator), 3 recipients, matures height 2196' * Fix 6 bugs found in full flow re-read Bug 1 -- Double broadcast (re-applied fix that was lost): When HandleLine accepted a block it called RebuildJob()+BroadcastJob(true). The event loop then detected nBestHeight changed and did it again. Miners received two back-to-back mining.notify clean=true, wasting their work. Fix: step 2 of event loop checks gHaveJob before rebuilding. If HandleLine already built a fresh job (gHaveJob=true), step 2 just logs and skips. Bug 3 -- Disconnect log showed global totals, not session totals: gParticipantSharesSent/Accepted accumulate since node start, not since the last connect. Added sessionSharesSent/sessionSharesAccepted local counters that reset each time PoolParticipantMiner() is called. Bug 4 -- Dead function ParticipantShareTargetFromDiff (main.cpp): Defined but never called anywhere. Removed. Bug 5 -- Invalid-address miner shares silently burned: If a miner authorized with an invalid address, their share of the block reward was neither paid out nor logged -- it just vanished. QueuePayouts now validates addresses up front, logs the forfeited amount explicitly ('X BTF forfeited (bad address)'), and excludes those entries so the payout log is accurate. Bug 6 -- SendMoney called from event loop (stalls miner I/O): FlushMaturePayouts called SendMoney which takes cs_main. If the P2P thread held cs_main (processing a block), the event loop would stall for seconds, causing all connected miners to time out waiting for responses. Fix: FlushMaturePayouts now only moves matured entries to gPayoutExecQueue (fast, no locks except a brief mutex). A new PayoutThreadFn thread drains the queue and calls SendMoney independently, so the event loop is never blocked by wallet I/O. Bug 7 -- Dead goto + dead HandleLine return-false path: HandleLine always returns true; the goto next_miner path was never reached. Removed goto and simplified the select drain loop. * Fix share-target scale mismatch and dead pool hashrate reporting - ShareTargetFromDifficulty() based Stratum difficulty 1 on Bitcoin's SHA256 diff-1 constant (0x1d00ffff, ~2^32 hashes/share) instead of this chain's actual PoW floor (bnProofOfWorkLimit = ~uint256(0)>>12, ~2^12 hashes/share). That's a ~2^20 (~1,048,576x) mismatch: at the supposedly-easy difficulty of 1.0, workers were being asked for shares roughly a million times harder than the chain's own easiest target. Vardiff never gets a chance to correct this because it only retargets after real shares come in, and none were coming in. Fixed by basing the diff-1 reference on bnProofOfWorkLimit instead. - Pool hashrate was never computed anywhere despite the data (per-miner difficulty + share timing) being available: ThreadBtfPoolAnnouncer in nostr.cpp hardcoded ann.hashRate = 0.0, PoolWorkerStatView had no hashrate field, and the GUI workers table had no column for it. Added an EWMA hashrate estimate per miner (updated on each accepted share from expected-hashes-at-difficulty / time-since-last-share), threaded it through GetPoolOperatorStats/GetPoolWorkerStats, the pool announcement payload, and the GUI (new Hashrate column + totals in the telemetry line). Verified: full clean build via src/Makefile (secp256k1 + RandomX deps built from source, g++ 13, all four touched files compile and link). * Remove stale wxWidgets-era build files and orphaned assets These predate the ImGui/GLFW migration (83af065) and are dead weight: - src/makefile.linux, build-linux.sh: superseded by src/Makefile + root Makefile (make linux). The old ones reference obj/uibase.o and obj/ui.o, which don't exist -- they never worked post-migration. - src/makefile.vc: MSVC build for the old wx UI (obj\ui.obj, obj\uibase.obj, obj\irc.obj) -- irc.cpp doesn't even exist anymore, and nothing invokes this makefile. Windows builds go through makefile.mingw via the root Makefile's target. - src/ui.rc, src/uiproject.fbp: wxFormBuilder project/resource files for the old wx UI. The current Windows resource file is bitflash.rc. - src/rc/{addressbook,send}*.bmp, check.ico, bitcoin_bc_original.ico: icons used only by the old wx UI. bitcoin.ico (used by bitflash.rc) is kept. Verified nothing includes uibase.h/ui.h/irc.h anywhere, and did a full clean rebuild afterward (secp256k1 + RandomX from source, src/Makefile) to confirm removal doesn't break the real build. * Mine with RandomX fast (2GB dataset) mode instead of light mode forever RandomXInitDataset() was fully implemented and ready to switch mining into fast mode, but nothing ever called it -- not in PoolParticipantMiner() (pool worker), not in the solo BitcoinMiner() loop. RandomXCreateMinerVM() falls back to light mode whenever the dataset isn't built, so every miner, participant or solo, has always been hashing in light/cache-only mode. Light mode exists for cheap single-hash verification (which is why the pool's own share verification correctly stays on it) -- it recomputes dataset items on the fly for every hash, which is dramatically slower than fast mode's direct dataset lookups. Combined with the share-target scale bug fixed in e87a905, this compounds into exactly the symptom reported: a participant sitting there hashing for many minutes per share instead of the vardiff-targeted ~15s, even after the target fix landed. Fix: build the ~2GB dataset once (multi-threaded, sized to hardware_concurrency) before mining starts, in both the participant and solo paths, so RandomXCreateMinerVM() actually gets to use fast mode. Verified: full clean rebuild via src/Makefile. * Add vardiff stall watchdog: force difficulty down on zero shares, not just share count ShareTargetFromDifficulty already fixed the diff-1 scale mismatch (chain floor vs Bitcoin's SHA256 diff-1), so difficulty 1 correctly means ~4096 expected hashes on this chain, not ~2^32. That part was already correct. But the normal vardiff retarget in HandleLine only runs after VARDIFF_SHARES_PER_CHECK (5) shares arrive. A miner that never finds even one share -- e.g. stuck in RandomX light/cache mode instead of the fast 2GB dataset, or just slower hardware than the target assumes -- never reaches that check, so it can sit at its initial/current difficulty indefinitely with vardiff having no samples to act on. Add a time-based fallback in PoolEventLoop: if a miner's vardiff window has been open for VARDIFF_STALL_SECS (90s, 6x the 15s target) with zero shares, force difficulty down by VARDIFF_STALL_CUT (4x) regardless of share count, and re-notify via mining.set_difficulty. Once a miner is already at VARDIFF_MIN, log it plainly as a hashrate/connectivity issue instead of a difficulty issue, so it's not confused with the same-looking but different problem this was actually fixed for before. * Fix PrintException rethrow killing the whole node on any transient exception The uploaded participant log shows the real bug behind the mining stalls: the node process itself was dying and restarting from scratch every minute or two ('UNKNOWN EXCEPTION ... in ThreadMessageHandler()' / 'ThreadSocketHandler()'), wiping the RandomX dataset/cache and losing all progress toward a share -- not a difficulty problem. Every background thread (ThreadMessageHandler, ThreadSocketHandler, ThreadBtfConnect, the Nostr helpers) wraps its real work in a 'try { Foo2() } catch(...) { log, sleep 5s, loop again }' self-healing retry pattern specifically so one bad exception doesn't take the whole node down. But PrintException(), called from inside every one of those catch blocks, ended with 'if (!fShutdown) throw;' -- re-throwing the just- caught exception during ordinary operation. Nothing above the retry loop catches it, so it escapes the thread and std::terminate()s the entire process on the very first transient error (a flaky relay connect, a malformed message, whatever). The comment's intent (suppress noise while already shutting down) was inverted from what the surrounding retry loops actually need. Removed the rethrow. Threads now log the exception and let their own retry loop handle it, as designed. * Push a fresh job immediately after any difficulty change, not just on next broadcast mining.set_difficulty alone doesn't change what the worker actually hashes against -- the participant only reads its share target from mining.notify's params[3] (see main.cpp's mining.notify handler); the set_difficulty message just updates a locally-logged currentDifficulty value. So both the normal vardiff retarget (in HandleLine, on 5 shares) and the new stall watchdog were updating m->shareTarget and telling the worker about the new difficulty, but the worker kept hashing against its old target until the next natural job broadcast -- a new block, or the 60s trickle refresh. That could delay any difficulty correction by up to a minute for no reason. Call SendJob(m, false) right after each set_difficulty send so the updated shareTarget reaches the worker in the very next tick instead of waiting on the next unrelated broadcast. * Build RandomX dataset before stratum handshake, not after Worker was sending mining.subscribe/authorize, then blocking for the duration of the synchronous ~2GB RandomX dataset build, and only then reading the socket. The pool's replies (subscribe ack, authorize ack, set_difficulty, first job) sat unread in the OS receive buffer for that entire window -- nothing was lost, but with the build taking anywhere from seconds to a couple minutes depending on core count, the worker looked and logged as permanently stuck on 'waiting for pool response' when it just wasn't listening yet. Move the dataset build ahead of the subscribe/authorize sends so the socket is actively being read the moment the pool replies. Also add missing RandomXDestroyMinerVM cleanup on the two early-return paths that now happen after VM creation. * Reset round shares on externally-found blocks, not just our own Round shares were only cleared in HandleLine() after our own miner's block got accepted. When the chain height advanced because someone else found the block, the main loop rebuilt the job but left roundShareCount/roundShareTotal/Miner::roundShares untouched, so they kept accumulating across rounds instead of resetting. * Show pool hashrate in participant view; show immature coinbase value with maturity countdown * Persist pending payouts to disk -- survive operator restarts * Fix log escapes, stall timeout 90->45s, save-before-exec ordering, Immature tx status * Sort transactions by nTimeReceived not chain median time -- new txs always on top * Sort txs by chain depth not nTimeReceived -- survives resync, unconfirmed always first * Fix orphan block false-positive payout; meaningful tx status instead of raw depth * Show Orphaned status in red for orphaned coinbases; distinguish from genuinely unconfirmed --------- Co-authored-by: Vic-Nas <nasci.victorio@gmail.com> Co-authored-by: Bitflash Pool Dev <pool@bitflash> Co-authored-by: Victorio Nascimento <155357371+Vic-Nas@users.noreply.github.com> Co-authored-by: Claude <claude@anthropic.com>
Diffstat (limited to 'src/nostr.cpp')
-rw-r--r--src/nostr.cpp518
1 files changed, 450 insertions, 68 deletions
diff --git a/src/nostr.cpp b/src/nostr.cpp
index c2ccc4c..009edd1 100644
--- a/src/nostr.cpp
+++ b/src/nostr.cpp
@@ -45,7 +45,12 @@ static const int64 BTF_POOL_MAX_AGE = 3 * 60; // seconds
45 45
46// Anonymous auto-discovery: how many peers to keep connected via .btf rendezvous 46// Anonymous auto-discovery: how many peers to keep connected via .btf rendezvous
47// before we stop dialing more, and how many new dials to attempt per cycle. 47// before we stop dialing more, and how many new dials to attempt per cycle.
48static const unsigned int BTF_TARGET_CONN = 8; 48// Target outbound peer count. With ~100 known .btf peers on the network,
49// an Erdos-Renyi random graph stays connected with overwhelming probability
50// once average degree exceeds ln(N) (~4.6 here); 24 is ~5x past that
51// threshold, so this is plenty *provided* selection isn't fully correlated
52// across nodes -- see the diversity slice in ConnectDiscoveredBtfPeers.
53static const unsigned int BTF_TARGET_CONN = 24;
49static const int BTF_DIALS_PER_PASS = 12; // try more peers per Nostr cycle 54static const int BTF_DIALS_PER_PASS = 12; // try more peers per Nostr cycle
50 55
51// Nostr descriptor freshness cutoff: skip peers whose descriptor hasn't been 56// Nostr descriptor freshness cutoff: skip peers whose descriptor hasn't been
@@ -58,11 +63,27 @@ static map<string, int64> g_btfPeers;
58static CCriticalSection cs_btfPeers; 63static CCriticalSection cs_btfPeers;
59 64
60// Public discovery relays. Tunable. 65// Public discovery relays. Tunable.
66//
67// Picked for: free/public (no NIP-42 auth or payment wall), track record of
68// high uptime, and *infrastructure* diversity, not just different hostnames.
69// relay.damus.io, relay.snort.social and relay.primal.net are all fronted by
70// Cloudflare -- different operators, same upstream, so a Cloudflare-side
71// incident (or a Cloudflare IP range getting rate-limited/blocked on some
72// network) can take out several of them at once even though they look like
73// independent relays. Kept only one Cloudflare-fronted relay (damus.io, the
74// most established) and filled the rest with relays on different hosts
75// (Hetzner, various independents) so no single upstream provider is a
76// single point of failure for peer discovery. 7 relays lands inside the
77// commonly recommended 5-10 range for redundancy without keeping an
78// excessive number of persistent sockets open per node.
61const char* pszNostrRelays[] = { 79const char* pszNostrRelays[] = {
62 "wss://relay.damus.io", 80 "wss://relay.damus.io",
63 "wss://nos.lol", 81 "wss://nos.lol",
64 "wss://relay.nostr.band", 82 "wss://relay.nostr.band",
65 "wss://relay.snort.social", 83 "wss://nostr21.com",
84 "wss://offchain.pub",
85 "wss://nostr.mom",
86 "wss://relay.primal.net",
66}; 87};
67const int nNostrRelays = ARRAYLEN(pszNostrRelays); 88const int nNostrRelays = ARRAYLEN(pszNostrRelays);
68 89
@@ -317,6 +338,13 @@ std::string BtfLocalAddress()
317 return g_nostrKey.BtfAddress(); 338 return g_nostrKey.BtfAddress();
318} 339}
319 340
341int GetDiscoveredPeerCount()
342{
343 int n = 0;
344 CRITICAL_BLOCK(cs_btfPeers) { n = (int)g_btfPeers.size(); }
345 return n;
346}
347
320void BtfGetPoolAnnouncements(std::vector<BtfPoolAnnouncement>& out) 348void BtfGetPoolAnnouncements(std::vector<BtfPoolAnnouncement>& out)
321{ 349{
322 out.clear(); 350 out.clear();
@@ -425,38 +453,55 @@ public:
425 size_t colon = host.find(':'); 453 size_t colon = host.find(':');
426 if (colon != string::npos) { port = atoi(host.substr(colon + 1).c_str()); host = host.substr(0, colon); } 454 if (colon != string::npos) { port = atoi(host.substr(colon + 1).c_str()); host = host.substr(0, colon); }
427 455
428 // Resolve and connect (TCP) 456 // Resolve and connect (TCP). AF_UNSPEC so we get both A and AAAA
429 struct addrinfo hints, *res = NULL; 457 // records back and try them in the order the resolver prefers,
458 // rather than hard-failing whenever the IPv4 path to a relay is
459 // filtered/rate-limited (increasingly common for CDN-fronted
460 // relays) while a working IPv6 route exists right there in the
461 // same getaddrinfo() result.
462 struct addrinfo hints, *res = NULL, *rp = NULL;
430 memset(&hints, 0, sizeof(hints)); 463 memset(&hints, 0, sizeof(hints));
431 hints.ai_family = AF_INET; 464 hints.ai_family = AF_UNSPEC;
432 hints.ai_socktype = SOCK_STREAM; 465 hints.ai_socktype = SOCK_STREAM;
433 char portstr[16]; sprintf(portstr, "%d", port); 466 char portstr[16]; sprintf(portstr, "%d", port);
434 if (getaddrinfo(host.c_str(), portstr, &hints, &res) != 0 || !res) 467 if (getaddrinfo(host.c_str(), portstr, &hints, &res) != 0 || !res)
435 return error("Nostr: getaddrinfo %s failed", host.c_str()); 468 return error("Nostr: getaddrinfo %s failed", host.c_str());
436 hSocket = socket(res->ai_family, res->ai_socktype, res->ai_protocol);
437 if (hSocket == INVALID_SOCKET) { freeaddrinfo(res); return false; }
438 469
439 // Socket receive/send timeout: Windows takes a DWORD of milliseconds; 470 bool fConnected = false;
440 // POSIX takes a struct timeval. 471 for (rp = res; rp != NULL; rp = rp->ai_next)
472 {
473 hSocket = socket(rp->ai_family, rp->ai_socktype, rp->ai_protocol);
474 if (hSocket == INVALID_SOCKET)
475 continue;
476
477 // Socket receive/send timeout: Windows takes a DWORD of
478 // milliseconds; POSIX takes a struct timeval.
441#ifdef _WIN32 479#ifdef _WIN32
442 DWORD tv = timeoutSec * 1000; 480 DWORD tv = timeoutSec * 1000;
443 setsockopt(hSocket, SOL_SOCKET, SO_RCVTIMEO, (char*)&tv, sizeof(tv)); 481 setsockopt(hSocket, SOL_SOCKET, SO_RCVTIMEO, (char*)&tv, sizeof(tv));
444 setsockopt(hSocket, SOL_SOCKET, SO_SNDTIMEO, (char*)&tv, sizeof(tv)); 482 setsockopt(hSocket, SOL_SOCKET, SO_SNDTIMEO, (char*)&tv, sizeof(tv));
445#else 483#else
446 struct timeval tv; 484 struct timeval tv;
447 tv.tv_sec = timeoutSec; 485 tv.tv_sec = timeoutSec;
448 tv.tv_usec = 0; 486 tv.tv_usec = 0;
449 setsockopt(hSocket, SOL_SOCKET, SO_RCVTIMEO, (char*)&tv, sizeof(tv)); 487 setsockopt(hSocket, SOL_SOCKET, SO_RCVTIMEO, (char*)&tv, sizeof(tv));
450 setsockopt(hSocket, SOL_SOCKET, SO_SNDTIMEO, (char*)&tv, sizeof(tv)); 488 setsockopt(hSocket, SOL_SOCKET, SO_SNDTIMEO, (char*)&tv, sizeof(tv));
451#endif 489#endif
452 490
453 if (connect(hSocket, res->ai_addr, (int)res->ai_addrlen) != 0) 491 if (connect(hSocket, rp->ai_addr, (int)rp->ai_addrlen) == 0)
454 { 492 {
455 freeaddrinfo(res); 493 fConnected = true;
456 return error("Nostr: connect %s:%d failed", host.c_str(), port); 494 break;
495 }
496
497 closesocket(hSocket);
498 hSocket = INVALID_SOCKET;
457 } 499 }
458 freeaddrinfo(res); 500 freeaddrinfo(res);
459 501
502 if (!fConnected)
503 return error("Nostr: connect %s:%d failed", host.c_str(), port);
504
460 // TLS 505 // TLS
461 if (fTls) 506 if (fTls)
462 { 507 {
@@ -598,6 +643,88 @@ private:
598}; 643};
599 644
600 645
646// Persistent relay connections. Public relays (relay.damus.io in particular)
647// rate-limit new connections per source IP fairly aggressively. Before this,
648// every single operation -- self-test, per-pass seeding, batch descriptor
649// resolve, pool announce, pool query -- opened its own fresh CWebSocket,
650// meaning a single node could fire a dozen-plus brand-new TCP+TLS+WS
651// handshakes at the same 4 relays within any given minute (ThreadNostrSeed's
652// own loop plus the separate ThreadBtfPoolAnnouncer thread, each reconnecting
653// from scratch every cycle). That's exactly what was showing up as constant
654// "handshake refused by relay.damus.io" and "connect ... failed" -- not
655// necessarily dead relays, just this node getting rate-limited by its own
656// reconnect churn.
657//
658// Now every relay-touching call reuses one long-lived socket per relay via
659// this pool, reconnecting only when a socket actually drops. That cuts fresh
660// connections from several per minute down to essentially one per relay for
661// the life of the process.
662struct RelayConn
663{
664 CWebSocket ws;
665 CCriticalSection cs; // serializes use of this relay's socket
666 bool fConnected;
667 RelayConn() : fConnected(false) {}
668};
669static map<string, RelayConn*> g_relayPool;
670static CCriticalSection cs_relayPool;
671
672static RelayConn* GetRelayConn(const string& relay)
673{
674 CRITICAL_BLOCK(cs_relayPool)
675 {
676 map<string, RelayConn*>::iterator it = g_relayPool.find(relay);
677 if (it != g_relayPool.end())
678 return it->second;
679 RelayConn* rc = new RelayConn();
680 g_relayPool[relay] = rc;
681 return rc;
682 }
683 return NULL; // unreachable
684}
685
686// RAII lease on a relay's pooled connection: connects (if not already
687// connected) on construction, and on destruction either keeps the socket
688// alive for the next caller (if the exchange succeeded) or closes it so the
689// next caller reconnects instead of reusing a socket that may be wedged.
690// Caller must set fOk = true once its exchange with the relay completed
691// cleanly.
692class CRelayLease
693{
694public:
695 RelayConn* rc;
696 bool fOk;
697 bool fJustConnected; // true if this lease had to (re)connect, false if reusing a live socket
698
699 CRelayLease(const string& relay, int timeoutSec) : rc(NULL), fOk(false), fJustConnected(false)
700 {
701 rc = GetRelayConn(relay);
702 rc->cs.Enter();
703 if (!rc->fConnected)
704 {
705 rc->ws.Close();
706 rc->fConnected = rc->ws.Connect(relay, timeoutSec);
707 fJustConnected = rc->fConnected;
708 }
709 }
710 ~CRelayLease()
711 {
712 if (!fOk)
713 {
714 rc->ws.Close();
715 rc->fConnected = false;
716 }
717 rc->cs.Leave();
718 }
719 bool Connected() const { return rc->fConnected; }
720 CWebSocket& Ws() { return rc->ws; }
721
722private:
723 CRelayLease(const CRelayLease&);
724 CRelayLease& operator=(const CRelayLease&);
725};
726
727
601// Publish this node's self-certifying `.btf` descriptor: "reach me (this key) 728// Publish this node's self-certifying `.btf` descriptor: "reach me (this key)
602// via <meeting_node>, encrypting to my x25519 key <enc>". Signed by the node 729// via <meeting_node>, encrypting to my x25519 key <enc>". Signed by the node
603// key so only the address's owner can publish it (no hijacking). 730// key so only the address's owner can publish it (no hijacking).
@@ -673,11 +800,11 @@ bool BtfPublishPoolAnnouncement(const BtfPoolAnnouncement& ann)
673 { 800 {
674 try 801 try
675 { 802 {
676 CWebSocket ws; 803 CRelayLease lease(pszNostrRelays[i], 10);
677 if (!ws.Connect(pszNostrRelays[i], 10)) 804 if (!lease.Connected())
678 continue; 805 continue;
679 if (PublishPoolAnnouncementOnRelay(ws, g_nostrKey, ann)) 806 if (PublishPoolAnnouncementOnRelay(lease.Ws(), g_nostrKey, ann))
680 fPublished = true; 807 fPublished = lease.fOk = true;
681 } 808 }
682 CATCH_PRINT_EXCEPTION("BtfPublishPoolAnnouncement") 809 CATCH_PRINT_EXCEPTION("BtfPublishPoolAnnouncement")
683 } 810 }
@@ -699,9 +826,10 @@ bool BtfQueryPoolAnnouncement(const std::string& poolBtfAddr, BtfPoolAnnouncemen
699 { 826 {
700 try 827 try
701 { 828 {
702 CWebSocket ws; 829 CRelayLease lease(pszNostrRelays[i], 10);
703 if (!ws.Connect(pszNostrRelays[i], 10)) 830 if (!lease.Connected())
704 continue; 831 continue;
832 CWebSocket& ws = lease.Ws();
705 833
706 json filter = json::object(); 834 json filter = json::object();
707 filter["authors"] = json::array({ HexEncode(pk, 32) }); 835 filter["authors"] = json::array({ HexEncode(pk, 32) });
@@ -756,6 +884,7 @@ bool BtfQueryPoolAnnouncement(const std::string& poolBtfAddr, BtfPoolAnnouncemen
756 else if (type == "EOSE") 884 else if (type == "EOSE")
757 break; 885 break;
758 } 886 }
887 lease.fOk = true;
759 } 888 }
760 CATCH_PRINT_EXCEPTION("BtfQueryPoolAnnouncement") 889 CATCH_PRINT_EXCEPTION("BtfQueryPoolAnnouncement")
761 } 890 }
@@ -855,12 +984,16 @@ bool BtfResolve(const std::string& btfAddr, std::string& meetingHostPort,
855 { 984 {
856 try 985 try
857 { 986 {
858 CWebSocket ws; 987 CRelayLease lease(pszNostrRelays[i], 10);
859 if (!ws.Connect(pszNostrRelays[i], 10)) 988 if (!lease.Connected())
860 continue; 989 continue;
861 btf::Descriptor d; 990 btf::Descriptor d;
862 if (!ResolveDescriptor(ws, g_nostrKey.ctx, btfAddr, d)) 991 if (!ResolveDescriptor(lease.Ws(), g_nostrKey.ctx, btfAddr, d))
992 {
993 lease.fOk = true; // exchange completed, just no match
863 continue; 994 continue;
995 }
996 lease.fOk = true;
864 if (d.meeting_node.empty() || d.meeting_node == "rendezvous-pending") 997 if (d.meeting_node.empty() || d.meeting_node == "rendezvous-pending")
865 continue; // descriptor predates the owner's relay config 998 continue; // descriptor predates the owner's relay config
866 if (!HexDecode(d.enc, enc_pub, 32)) 999 if (!HexDecode(d.enc, enc_pub, 32))
@@ -873,14 +1006,115 @@ bool BtfResolve(const std::string& btfAddr, std::string& meetingHostPort,
873 return false; 1006 return false;
874} 1007}
875 1008
1009// Resolve MANY .btf addresses in one pass. A single relay connection carries
1010// a REQ with every still-unresolved address's pubkey in "authors", instead of
1011// opening one relay connection per address. This matters because callers like
1012// ConnectDiscoveredBtfPeers dial many candidates in parallel; without
1013// batching, each of those dial threads independently walks the same handful
1014// of public relays on its own, so a pass of N candidates can fire up to
1015// N * nNostrRelays near-simultaneous connections at the same few relays --
1016// enough to get rate-limited or have handshakes refused, which then looks
1017// like "dead peers" even though the descriptors are perfectly resolvable.
1018bool BtfResolveMany(const std::vector<std::string>& btfAddrs,
1019 std::map<std::string, BtfResolvedPeer>& out)
1020{
1021 if (!EnsureNostrKey() || btfAddrs.empty())
1022 return false;
1023
1024 map<string, string> hexToAddr; // pubkey hex -> .btf addr, for matching events back
1025 set<string> pending;
1026 for (const string& a : btfAddrs)
1027 {
1028 unsigned char pk[32];
1029 if (!btf::ParseAddress(a, pk))
1030 continue;
1031 hexToAddr[HexEncode(pk, 32)] = a;
1032 pending.insert(a);
1033 }
1034 if (pending.empty())
1035 return false;
1036
1037 for (int i = 0; i < nNostrRelays && !fShutdown && !pending.empty(); i++)
1038 {
1039 try
1040 {
1041 CRelayLease lease(pszNostrRelays[i], 10);
1042 if (!lease.Connected())
1043 continue;
1044 CWebSocket& ws = lease.Ws();
1045
1046 json authors = json::array();
1047 for (auto& kv : hexToAddr)
1048 if (pending.count(kv.second))
1049 authors.push_back(kv.first);
1050 if (authors.empty())
1051 {
1052 lease.fOk = true;
1053 break;
1054 }
1055
1056 json filter = json::object();
1057 filter["authors"] = authors;
1058 filter["kinds"] = json::array({ BTF_DESC_KIND });
1059 filter["limit"] = (int)authors.size();
1060 json req = json::array({ "REQ", "btf-resolve-many", filter });
1061 if (!ws.SendText(req.dump()))
1062 continue;
1063
1064 for (;;)
1065 {
1066 string msg;
1067 if (!ws.RecvText(msg))
1068 break;
1069 json j;
1070 try { j = json::parse(msg); } catch (...) { continue; }
1071 if (!j.is_array() || j.empty()) continue;
1072 string t = j[0].get<string>();
1073 if (t == "EVENT" && j.size() >= 3)
1074 {
1075 const json& ev = j[2];
1076 if (!ev.contains("pubkey") || !ev.contains("content"))
1077 continue;
1078 string hexpk = ev["pubkey"].get<string>();
1079 auto ai = hexToAddr.find(hexpk);
1080 if (ai == hexToAddr.end() || !pending.count(ai->second))
1081 continue; // not one we're waiting on (or already resolved)
1082
1083 unsigned char pk[32];
1084 if (!HexDecode(hexpk, pk, 32))
1085 continue;
1086 btf::Descriptor d;
1087 if (!btf::VerifyDescriptor(g_nostrKey.ctx, ev["content"].get<string>(), pk, d))
1088 continue;
1089 if (d.meeting_node.empty() || d.meeting_node == "rendezvous-pending")
1090 continue;
1091 BtfResolvedPeer rp;
1092 if (!HexDecode(d.enc, rp.enc_pub, 32))
1093 continue;
1094 rp.meetingHostPort = d.meeting_node;
1095 out[ai->second] = rp;
1096 pending.erase(ai->second);
1097 }
1098 else if (t == "EOSE")
1099 break;
1100 }
1101 lease.fOk = true;
1102 }
1103 CATCH_PRINT_EXCEPTION("BtfResolveMany")
1104 }
1105 return !out.empty();
1106}
1107
876 1108
877// Connect to a relay: publish our .btf descriptor and collect relay data. 1109// Connect to a relay: publish our .btf descriptor and collect relay data.
878static bool SeedFromRelay(CNostrKey& key, const string& relay) 1110static bool SeedFromRelay(CNostrKey& key, const string& relay)
879{ 1111{
880 CWebSocket ws; 1112 CRelayLease lease(relay, 10);
881 if (!ws.Connect(relay, 10)) 1113 if (!lease.Connected())
882 return false; 1114 return false;
883 printf("Nostr: connected to %s\n", relay.c_str()); 1115 CWebSocket& ws = lease.Ws();
1116 if (lease.fJustConnected)
1117 LogPrint("nostr", "Nostr: connected to %s\n", relay.c_str());
884 1118
885 // Publish our self-certifying .btf descriptor (rendezvous discovery) 1119 // Publish our self-certifying .btf descriptor (rendezvous discovery)
886 PublishDescriptor(ws, key); 1120 PublishDescriptor(ws, key);
@@ -930,7 +1164,7 @@ static bool SeedFromRelay(CNostrKey& key, const string& relay)
930 else if (createdAt > it->second) it->second = createdAt; // refresh 1164 else if (createdAt > it->second) it->second = createdAt; // refresh
931 } 1165 }
932 if (fNew && fDebug) 1166 if (fNew && fDebug)
933 printf("Nostr: discovered .btf peer %s\n", peerAddr.c_str()); 1167 LogPrint("nostr", "Nostr: discovered .btf peer %s\n", peerAddr.c_str());
934 if (++nPeers > 500) break; 1168 if (++nPeers > 500) break;
935 } 1169 }
936 else if (type == "EOSE") 1170 else if (type == "EOSE")
@@ -974,7 +1208,7 @@ static bool SeedFromRelay(CNostrKey& key, const string& relay)
974 if (g_discoveredRelays.size() < BTF_MAX_DISCOVERED_RELAYS) 1208 if (g_discoveredRelays.size() < BTF_MAX_DISCOVERED_RELAYS)
975 fNew = g_discoveredRelays.insert(r).second; 1209 fNew = g_discoveredRelays.insert(r).second;
976 if (fNew && fDebug) 1210 if (fNew && fDebug)
977 printf("Nostr: discovered relay %s\n", r.c_str()); 1211 LogPrint("nostr", "Nostr: discovered relay %s\n", r.c_str());
978 } 1212 }
979 else if (type == "EOSE") 1213 else if (type == "EOSE")
980 break; 1214 break;
@@ -1009,12 +1243,15 @@ static bool SeedFromRelay(CNostrKey& key, const string& relay)
1009 } 1243 }
1010 } 1244 }
1011 } 1245 }
1246 lease.fOk = true;
1012 return true; 1247 return true;
1013} 1248}
1014 1249
1250volatile bool gAnnounceNow = false;
1251
1015void ThreadBtfPoolAnnouncer(void*) 1252void ThreadBtfPoolAnnouncer(void*)
1016{ 1253{
1017 printf("ThreadBtfPoolAnnouncer started\n"); 1254 LogPrint("pool", "ThreadBtfPoolAnnouncer started\n");
1018 while (!fShutdown) 1255 while (!fShutdown)
1019 { 1256 {
1020 if (nMineMode == MINE_OPERATOR && gPoolRunning) 1257 if (nMineMode == MINE_OPERATOR && gPoolRunning)
@@ -1025,15 +1262,17 @@ void ThreadBtfPoolAnnouncer(void*)
1025 ann.dashboardUrl = strPoolDashboardUrl; 1262 ann.dashboardUrl = strPoolDashboardUrl;
1026 ann.feePercent = dPoolFeePercent; 1263 ann.feePercent = dPoolFeePercent;
1027 uint64 roundShares = 0; 1264 uint64 roundShares = 0;
1028 GetPoolOperatorStats(ann.connectedMiners, ann.blocksFound, roundShares); 1265 double totalHashRate = 0.0;
1029 ann.hashRate = 0.0; 1266 GetPoolOperatorStats(ann.connectedMiners, ann.blocksFound, roundShares, totalHashRate);
1267 ann.hashRate = totalHashRate;
1030 ann.createdAt = GetTime(); 1268 ann.createdAt = GetTime();
1031 if (!ann.btfAddress.empty() && !ann.poolName.empty()) 1269 if (!ann.btfAddress.empty() && !ann.poolName.empty())
1032 BtfPublishPoolAnnouncement(ann); 1270 BtfPublishPoolAnnouncement(ann);
1033 } 1271 }
1034 1272
1035 for (int i = 0; i < 60 && !fShutdown; i++) 1273 for (int i = 0; i < 60 && !fShutdown && !gAnnounceNow; i++)
1036 Sleep(1000); 1274 Sleep(1000);
1275 gAnnounceNow = false;
1037 } 1276 }
1038} 1277}
1039 1278
@@ -1042,10 +1281,29 @@ void ThreadBtfPoolAnnouncer(void*)
1042// dead/stale descriptors (e.g. abandoned test identities that linger on the 1281// dead/stale descriptors (e.g. abandoned test identities that linger on the
1043// relays) from consuming the whole per-pass dial budget and starving the live 1282// relays) from consuming the whole per-pass dial budget and starving the live
1044// peers. Backoff grows on repeated failures, capped. 1283// peers. Backoff grows on repeated failures, capped.
1284//
1285// g_btfPeerFails also doubles as the priority signal for dial ordering: a peer
1286// we've connected to before (present in g_btfPeerEverConnected) is known-good
1287// and goes first; a peer we've never tried goes next; a peer with a growing
1288// fail count is known-bad and goes last, worst offenders furthest back. This
1289// replaces the old behavior where the whole candidate list was shuffled right
1290// after being sorted by freshness, which made the freshness sort a no-op and
1291// gave dead peers equal footing with everyone else.
1045static map<string, int64> g_btfPeerBackoff; // addr -> next-try time 1292static map<string, int64> g_btfPeerBackoff; // addr -> next-try time
1046static map<string, int> g_btfPeerFails; // addr -> consecutive failures 1293static map<string, int> g_btfPeerFails; // addr -> consecutive failures
1294static set<string> g_btfPeerEverConnected; // addr -> has succeeded at least once
1047static CCriticalSection cs_btfPeerState; 1295static CCriticalSection cs_btfPeerState;
1048 1296
1297// Backoff cap for repeatedly-dead peers. The old 15-minute cap meant a
1298// descriptor that has failed many times in a row cycles back into the dial
1299// pool just as often as a peer we've never tried, wasting dial budget on
1300// known-dead addresses every pass. Once a peer has failed enough times to
1301// be "chronic", stretch the cap much further so it consumes budget far less
1302// often while still getting retried eventually in case it comes back online.
1303static const int64 BTF_BACKOFF_CAP_SECS = 15 * 60; // 15 min: initial ramp cap
1304static const int64 BTF_BACKOFF_CAP_SECS_DEAD = 6 * 60 * 60; // 6 hr: cap for chronic failures
1305static const int BTF_CHRONIC_FAIL_THRESHOLD = 6; // fails before "chronic" cap kicks in
1306
1049// Dial the .btf peers discovered on the relays, up to a target connection count. 1307// Dial the .btf peers discovered on the relays, up to a target connection count.
1050// Connections are attempted in parallel so a single slow/dead relay doesn't 1308// Connections are attempted in parallel so a single slow/dead relay doesn't
1051// stall the entire pass. Each candidate gets its own thread; all threads start 1309// stall the entire pass. Each candidate gets its own thread; all threads start
@@ -1060,23 +1318,18 @@ static void ConnectDiscoveredBtfPeers()
1060 for (auto& kv : g_btfPeers) 1318 for (auto& kv : g_btfPeers)
1061 peersWithAge.push_back({kv.second, kv.first}); 1319 peersWithAge.push_back({kv.second, kv.first});
1062 } 1320 }
1063 // Sort newest descriptor first — recently-active nodes are much more
1064 // likely to be online right now. Stale ones stay in the list as fallback.
1065 sort(peersWithAge.begin(), peersWithAge.end(),
1066 [](const pair<int64,string>& a, const pair<int64,string>& b){ return a.first > b.first; });
1067
1068 int64 cutoff = GetTime() - BTF_PEER_MAX_AGE; 1321 int64 cutoff = GetTime() - BTF_PEER_MAX_AGE;
1069 int nFresh = 0; 1322 int nFresh = 0;
1070 vector<string> peers; 1323 vector<pair<int64,string>> fresh; // (createdAt, addr), newest first
1071 for (auto& p : peersWithAge) 1324 for (auto& p : peersWithAge)
1072 { 1325 {
1073 if (p.first < cutoff) 1326 if (p.first < cutoff)
1074 continue; 1327 continue;
1075 peers.push_back(p.second); 1328 fresh.push_back(p);
1076 nFresh++; 1329 nFresh++;
1077 } 1330 }
1078 if (!peers.empty()) 1331 if (!fresh.empty())
1079 printf("Nostr: %d fresh peers, %d stale (of %d total)\n", nFresh, nTotal-nFresh, nTotal); 1332 LogPrint("nostr", "Nostr: %d fresh peers, %d stale (of %d total)\n", nFresh, nTotal-nFresh, nTotal);
1080 1333
1081 // Drop stale peers so we stop retrying dead descriptors until they refresh. 1334 // Drop stale peers so we stop retrying dead descriptors until they refresh.
1082 { 1335 {
@@ -1092,42 +1345,164 @@ static void ConnectDiscoveredBtfPeers()
1092 } 1345 }
1093 } 1346 }
1094 1347
1095 // Shuffle so we don't always retry the same dead peers at the front. 1348 // Sort newest descriptor first -- recently-active nodes are much more
1096 for (size_t i = peers.size(); i > 1; i--) 1349 // likely to be online right now.
1097 swap(peers[i - 1], peers[(size_t)GetRand(i)]); 1350 sort(fresh.begin(), fresh.end(),
1351 [](const pair<int64,string>& a, const pair<int64,string>& b){ return a.first > b.first; });
1352
1353 // Reorder into tiers: known-good (succeeded before) first, untried next,
1354 // known-bad (has a fail count) last -- worst offenders furthest back.
1355 // Freshness order is preserved *within* each tier, so recency still
1356 // matters, but a peer we know is dead no longer gets equal footing with
1357 // a peer we've never tried or one that's worked before.
1358 vector<string> tierGood, tierUntried, tierBad;
1359 CRITICAL_BLOCK(cs_btfPeerState)
1360 {
1361 for (auto& p : fresh)
1362 {
1363 const string& addr = p.second;
1364 if (g_btfPeerEverConnected.count(addr))
1365 tierGood.push_back(addr);
1366 else if (g_btfPeerFails.find(addr) == g_btfPeerFails.end())
1367 tierUntried.push_back(addr);
1368 else
1369 tierBad.push_back(addr);
1370 }
1371 // Within known-bad, put fewer-fails peers ahead of chronic offenders.
1372 sort(tierBad.begin(), tierBad.end(),
1373 [](const string& a, const string& b){ return g_btfPeerFails[a] < g_btfPeerFails[b]; });
1374 }
1375
1376 vector<string> peers;
1377 peers.reserve(fresh.size());
1378 peers.insert(peers.end(), tierGood.begin(), tierGood.end());
1379 peers.insert(peers.end(), tierUntried.begin(), tierUntried.end());
1380 peers.insert(peers.end(), tierBad.begin(), tierBad.end());
1098 1381
1099 // Pick candidates for this pass (skip those in backoff, cap at budget). 1382 // Pick candidates for this pass (skip those in backoff, cap at budget).
1383 //
1384 // The tiered order above is deliberately biased toward peers we know
1385 // are good -- that's the right default, since dialing untested/bad
1386 // peers wastes attempts. But if every node on the network applies the
1387 // same bias, everyone converges on the same small set of "known good"
1388 // peers as neighbors: that correlates peer sets across the network,
1389 // which is exactly what standard random-graph connectivity results
1390 // assume you avoid. If that shared hub set has a bad day (e.g. one
1391 // relay outage), many nodes lose the same edges at once instead of
1392 // losing independent, uncorrelated edges.
1393 //
1394 // So: fill most of the pass from the reliable/tiered order, but reserve
1395 // a slice for peers picked at random from the *entire* fresh pool
1396 // (not just the front of the sorted list). Same pattern Bitcoin's own
1397 // peer manager and Kademlia-style DHTs use -- sticky reliable peers
1398 // plus a random sample, not one or the other.
1100 int64 now = GetTime(); 1399 int64 now = GetTime();
1400 int nBudget = (int)BTF_DIALS_PER_PASS;
1401 {
1402 int nRoom = (int)BTF_TARGET_CONN - (int)vNodes.size();
1403 if (nRoom < nBudget) nBudget = nRoom;
1404 }
1405 if (nBudget < 0) nBudget = 0;
1406 int nRandomSlots = (nBudget > 0) ? max(1, nBudget * 3 / 10) : 0;
1407 int nSortedSlots = nBudget - nRandomSlots;
1408
1101 vector<string> candidates; 1409 vector<string> candidates;
1410 set<string> setChosen;
1102 CRITICAL_BLOCK(cs_btfPeerState) 1411 CRITICAL_BLOCK(cs_btfPeerState)
1103 { 1412 {
1413 // Reliability slice: walk the tiered (good -> untried -> bad) order.
1104 for (const string& addr : peers) 1414 for (const string& addr : peers)
1105 { 1415 {
1106 if ((int)vNodes.size() >= (int)BTF_TARGET_CONN) break; 1416 if ((int)vNodes.size() >= (int)BTF_TARGET_CONN) break;
1107 if ((int)candidates.size() >= BTF_DIALS_PER_PASS) break; 1417 if ((int)candidates.size() >= nSortedSlots) break;
1108 map<string, int64>::iterator bi = g_btfPeerBackoff.find(addr); 1418 map<string, int64>::iterator bi = g_btfPeerBackoff.find(addr);
1109 if (bi != g_btfPeerBackoff.end() && now < bi->second) 1419 if (bi != g_btfPeerBackoff.end() && now < bi->second)
1110 continue; 1420 continue;
1111 candidates.push_back(addr); 1421 candidates.push_back(addr);
1422 setChosen.insert(addr);
1423 }
1424
1425 // Diversity slice: random sample from the full fresh pool, so the
1426 // dial isn't limited to whatever the tiered sort put up front.
1427 vector<string> pool;
1428 for (auto& p : fresh)
1429 {
1430 const string& addr = p.second;
1431 if (setChosen.count(addr))
1432 continue;
1433 map<string, int64>::iterator bi = g_btfPeerBackoff.find(addr);
1434 if (bi != g_btfPeerBackoff.end() && now < bi->second)
1435 continue;
1436 pool.push_back(addr);
1437 }
1438 int nWant = min(nRandomSlots, (int)pool.size());
1439 for (int i = 0; i < nWant; i++)
1440 {
1441 if ((int)vNodes.size() + (int)candidates.size() >= (int)BTF_TARGET_CONN) break;
1442 if ((int)candidates.size() >= nBudget) break;
1443 size_t j = i + (size_t)GetRand((uint64)(pool.size() - i));
1444 std::swap(pool[i], pool[j]);
1445 candidates.push_back(pool[i]);
1446 setChosen.insert(pool[i]);
1112 } 1447 }
1113 } 1448 }
1114 1449
1115 if (candidates.empty()) return; 1450 if (candidates.empty()) return;
1116 1451
1117 // Launch one thread per candidate — all connect attempts run in parallel. 1452 // Resolve every candidate's descriptor in one batched pass (a handful of
1453 // relay round-trips total, one REQ per relay covering all candidates)
1454 // instead of each dial thread independently walking the relay list on
1455 // its own. That per-thread approach meant a 12-candidate pass could fire
1456 // up to 12 * nNostrRelays near-simultaneous connections at the same
1457 // handful of public relays -- enough to trip rate limits or get
1458 // handshakes refused, which then looks exactly like "dead peers" even
1459 // though the descriptors were perfectly resolvable.
1460 map<string, BtfResolvedPeer> resolved;
1461 BtfResolveMany(candidates, resolved);
1462
1463 // Candidates whose descriptor didn't resolve this pass are recorded as a
1464 // failure right away (no point spawning a tunnel thread with nothing to
1465 // dial), same bookkeeping as a failed connect attempt.
1466 vector<string> toDial;
1467 CRITICAL_BLOCK(cs_btfPeerState)
1468 {
1469 for (const string& addr : candidates)
1470 {
1471 if (resolved.count(addr))
1472 {
1473 toDial.push_back(addr);
1474 continue;
1475 }
1476 int n = ++g_btfPeerFails[addr];
1477 int64 cap = (n >= BTF_CHRONIC_FAIL_THRESHOLD) ? BTF_BACKOFF_CAP_SECS_DEAD
1478 : BTF_BACKOFF_CAP_SECS;
1479 int64 secs = 30;
1480 for (int k = 1; k < n && secs < cap; k++) secs *= 2;
1481 if (secs > cap) secs = cap;
1482 g_btfPeerBackoff[addr] = GetTime() + secs;
1483 }
1484 }
1485 if (toDial.empty()) return;
1486
1487 // Launch one thread per resolved candidate -- all connect attempts run in
1488 // parallel, but purely against each peer's own rendezvous meeting node,
1489 // not the shared Nostr relays, so there's no thundering-herd effect here.
1118 // Each thread writes its result back via shared state guarded by cs_btfPeerState. 1490 // Each thread writes its result back via shared state guarded by cs_btfPeerState.
1119 struct DialCtx { string addr; }; 1491 struct DialCtx { string addr; BtfResolvedPeer rp; };
1120 vector<DialCtx*> ctxs; 1492 vector<DialCtx*> ctxs;
1121 for (const string& addr : candidates) 1493 for (const string& addr : toDial)
1122 { 1494 {
1123 DialCtx* ctx = new DialCtx{addr}; 1495 DialCtx* ctx = new DialCtx{addr, resolved[addr]};
1124 ctxs.push_back(ctx); 1496 ctxs.push_back(ctx);
1125 _beginthread([](void* arg) { 1497 _beginthread([](void* arg) {
1126 DialCtx* ctx = (DialCtx*)arg; 1498 DialCtx* ctx = (DialCtx*)arg;
1127 string addr = ctx->addr; 1499 string addr = ctx->addr;
1500 string meeting = ctx->rp.meetingHostPort;
1501 unsigned char enc_pub[32];
1502 memcpy(enc_pub, ctx->rp.enc_pub, 32);
1128 delete ctx; 1503 delete ctx;
1129 1504
1130 CNode* pnode = ConnectNodeBtf(addr); 1505 CNode* pnode = ConnectNodeBtfResolved(addr, meeting, enc_pub);
1131 CRITICAL_BLOCK(cs_btfPeerState) 1506 CRITICAL_BLOCK(cs_btfPeerState)
1132 { 1507 {
1133 if (pnode) 1508 if (pnode)
@@ -1138,13 +1513,16 @@ static void ConnectDiscoveredBtfPeers()
1138 pnode->Release(); 1513 pnode->Release();
1139 g_btfPeerBackoff.erase(addr); 1514 g_btfPeerBackoff.erase(addr);
1140 g_btfPeerFails.erase(addr); 1515 g_btfPeerFails.erase(addr);
1516 g_btfPeerEverConnected.insert(addr);
1141 } 1517 }
1142 else 1518 else
1143 { 1519 {
1144 int n = ++g_btfPeerFails[addr]; 1520 int n = ++g_btfPeerFails[addr];
1521 int64 cap = (n >= BTF_CHRONIC_FAIL_THRESHOLD) ? BTF_BACKOFF_CAP_SECS_DEAD
1522 : BTF_BACKOFF_CAP_SECS;
1145 int64 secs = 30; 1523 int64 secs = 30;
1146 for (int k = 1; k < n && secs < 15*60; k++) secs *= 2; 1524 for (int k = 1; k < n && secs < cap; k++) secs *= 2;
1147 if (secs > 15*60) secs = 15*60; 1525 if (secs > cap) secs = cap;
1148 g_btfPeerBackoff[addr] = GetTime() + secs; 1526 g_btfPeerBackoff[addr] = GetTime() + secs;
1149 } 1527 }
1150 } 1528 }
@@ -1160,16 +1538,16 @@ static void ConnectDiscoveredBtfPeers()
1160 1538
1161void ThreadNostrSeed(void* parg) 1539void ThreadNostrSeed(void* parg)
1162{ 1540{
1163 printf("ThreadNostrSeed started\n"); 1541 LogPrint("nostr", "ThreadNostrSeed started\n");
1164 1542
1165 if (!EnsureNostrKey()) 1543 if (!EnsureNostrKey())
1166 { 1544 {
1167 printf("Nostr: failed to initialize secp256k1 key\n"); 1545 LogPrint("nostr", "Nostr: failed to initialize secp256k1 key\n");
1168 return; 1546 return;
1169 } 1547 }
1170 CNostrKey& key = g_nostrKey; 1548 CNostrKey& key = g_nostrKey;
1171 printf("Nostr: node pubkey = %s\n", key.PubKeyHex().c_str()); 1549 LogPrint("nostr", "Nostr: node pubkey = %s\n", key.PubKeyHex().c_str());
1172 printf("Nostr: node .btf address = %s\n", key.BtfAddress().c_str()); 1550 LogPrint("nostr", "Nostr: node .btf address = %s\n", key.BtfAddress().c_str());
1173 1551
1174 // One-time self-test of the rendezvous: publish our descriptor to a relay, 1552 // One-time self-test of the rendezvous: publish our descriptor to a relay,
1175 // then resolve our OWN .btf address back and verify it self-certifies. Proves 1553 // then resolve our OWN .btf address back and verify it self-certifies. Proves
@@ -1178,17 +1556,20 @@ void ThreadNostrSeed(void* parg)
1178 { 1556 {
1179 try 1557 try
1180 { 1558 {
1181 CWebSocket ws; 1559 CRelayLease lease(pszNostrRelays[i], 10);
1182 if (!ws.Connect(pszNostrRelays[i], 10)) continue; 1560 if (!lease.Connected()) continue;
1561 CWebSocket& ws = lease.Ws();
1183 PublishDescriptor(ws, key); 1562 PublishDescriptor(ws, key);
1184 Sleep(700); // let the relay store the replaceable event 1563 Sleep(700); // let the relay store the replaceable event
1185 btf::Descriptor d; 1564 btf::Descriptor d;
1186 if (ResolveDescriptor(ws, key.ctx, key.BtfAddress(), d)) 1565 if (ResolveDescriptor(ws, key.ctx, key.BtfAddress(), d))
1187 { 1566 {
1188 printf("Nostr: .btf self-resolve OK via %s (meeting_node=%s, enc=%s...)\n", 1567 LogPrint("nostr", "Nostr: .btf self-resolve OK via %s (meeting_node=%s, enc=%s...)\n",
1189 pszNostrRelays[i], d.meeting_node.c_str(), d.enc.substr(0, 16).c_str()); 1568 pszNostrRelays[i], d.meeting_node.c_str(), d.enc.substr(0, 16).c_str());
1569 lease.fOk = true;
1190 break; 1570 break;
1191 } 1571 }
1572 lease.fOk = true;
1192 } 1573 }
1193 CATCH_PRINT_EXCEPTION("btf self-test") 1574 CATCH_PRINT_EXCEPTION("btf self-test")
1194 } 1575 }
@@ -1223,8 +1604,9 @@ void ThreadNostrSeed(void* parg)
1223 ann.dashboardUrl = strPoolDashboardUrl; 1604 ann.dashboardUrl = strPoolDashboardUrl;
1224 ann.feePercent = dPoolFeePercent; 1605 ann.feePercent = dPoolFeePercent;
1225 uint64 roundShares = 0; 1606 uint64 roundShares = 0;
1226 GetPoolOperatorStats(ann.connectedMiners, ann.blocksFound, roundShares); 1607 double totalHashRate = 0.0;
1227 ann.hashRate = 0.0; 1608 GetPoolOperatorStats(ann.connectedMiners, ann.blocksFound, roundShares, totalHashRate);
1609 ann.hashRate = totalHashRate;
1228 ann.createdAt = GetTime(); 1610 ann.createdAt = GetTime();
1229 if (!ann.btfAddress.empty() && !ann.poolName.empty()) 1611 if (!ann.btfAddress.empty() && !ann.poolName.empty())
1230 BtfPublishPoolAnnouncement(ann); 1612 BtfPublishPoolAnnouncement(ann);