net: a refused thread spawn drops one peer, not the node

std::thread throws std::system_error when pthread_create is refused
(RLIMIT_NPROC, a container's pids cgroup, memory pressure), and nothing
caught it, so the node died by std::terminate. Threads are spawned per
peer connection, so the pressure that triggers this scales with peer
count, which is attacker-influenced. Observed killing a healthy
participant worker mid-mining.

Every spawn site in the node build, and what a failure now does:

- btftunnel.cpp: both tunnel pumps are guarded. The inbound pump starts
  first because it owns cleanup of the pump end and rendezvous socket;
  if the second spawn is refused, shutdown() wakes the first out of its
  read and it cleans up itself -- closing those descriptors from the
  failure path instead would race it onto reused descriptors. Both
  paths return INVALID_SOCKET, which every caller already treats as
  "this connection failed".
- btfrv.cpp: the relay's accept loop drops the connection it cannot
  serve and stops accepting for a moment -- longer after consecutive
  failures, capped at 5s -- instead of dying at an arbitrary one. The
  per-pair second pump now runs on the handling thread itself, leaving
  exactly one spawn to guard and nothing running yet to race.
- main.cpp: a participant that cannot spawn its hash workers stops the
  ones that did start, drops the pool connection, and returns to the
  reconnect loop, the same way an ordinary disconnect does.
- randomx_pow.cpp: dataset init joins the workers it got, releases the
  dataset, and stays in light mode -- the same trade the allocation
  failure path above it already makes.
- rpc.cpp, net.cpp: the remaining unchecked _beginthread calls (which
  report failure by value, they cannot throw) now say precisely what
  did not start. The pool's accept and payout threads failing silently
  left the pool running deaf -- nobody accepts, nobody gets paid, and
  nothing in the log says why.

Left alone deliberately: the per-relay accept spawns (rpc.cpp) already
check, log and retry on the next poll; startup-critical spawns in
net.cpp keep failing hard; the selftest-only std::thread uses may abort
a selftest run, which is acceptable where dying over one peer is not.

Verified: full node rebuilds clean and links; pool-stratum selftest
passes; a scratch harness under RLIMIT_NPROC=1 confirms _beginthread
reports -1 and std::thread throws system_error, the premises of every
path above.

6 files changed, 157 insertions, 22 deletions

diff --git a/src/btfrv.cpp b/src/btfrv.cpp
index 133eb89..8ab735c 100644
--- a/src/btfrv.cpp
+++ b/src/btfrv.cpp
@@ -40,6 +40,8 @@ typedef int SOCKET;
 #include <thread>
 #include <mutex>
 #include <atomic>
+#include <chrono>
+#include <system_error>
 
 namespace btf
 {
@@ -209,8 +211,28 @@ static void HandleIncoming(SOCKET s)
         bool a = WriteN(svc, &ok, 1);
         bool b = WriteN(s, &ok, 1);
         if (!a || !b) { CLOSESOCK(svc); CLOSESOCK(s); return; }
-        std::thread(Forward, s, svc).detach();
-        std::thread(Forward, svc, s).detach();
+        // One thread pumps s -> svc; this thread pumps svc -> s. Spawning a
+        // thread per direction left no clean way to fail the second spawn:
+        // std::thread throws std::system_error when pthread_create is refused
+        // (NPROC limit, container pids cgroup, memory pressure), the first
+        // pump was already detached, and closing its sockets from here would
+        // race it -- so nothing caught the throw and it reached
+        // std::terminate, killing the relay over one pair it could not serve.
+        // Running one direction on this thread leaves exactly one spawn to
+        // guard, and a failure here has nothing running yet to race.
+        try
+        {
+            std::thread(Forward, s, svc).detach();
+        }
+        catch (const std::system_error& e)
+        {
+            fprintf(stderr, "rendezvous relay: cannot spawn forward thread (%s), pair dropped\n",
+                    e.what());
+            CLOSESOCK(svc);
+            CLOSESOCK(s);
+            return;
+        }
+        Forward(svc, s);
     }
     else
     {
@@ -236,6 +258,7 @@ bool RvRelayRun(unsigned short port)
     if (bind(g_listen, (struct sockaddr*)&addr, sizeof(addr)) != 0) { CLOSESOCK(g_listen); return false; }
     if (listen(g_listen, 16) != 0) { CLOSESOCK(g_listen); return false; }
 
+    int nSpawnFails = 0;
     while (!g_stop)
     {
         struct sockaddr_in cli;
@@ -246,7 +269,27 @@ bool RvRelayRun(unsigned short port)
             if (g_stop) break;
             continue;
         }
-        std::thread(HandleIncoming, s).detach();
+        try
+        {
+            std::thread(HandleIncoming, s).detach();
+            nSpawnFails = 0;
+        }
+        catch (const std::system_error& e)
+        {
+            // A refused pthread_create is backpressure, not a reason to die:
+            // the condition behind it (NPROC, pids cgroup, memory) does not
+            // clear in milliseconds, and the connection count feeding it is
+            // attacker-influenced. Drop this one peer, stop accepting for a
+            // moment -- longer each time, capped -- and keep serving the
+            // connections already established.
+            fprintf(stderr, "rendezvous relay: cannot spawn connection thread (%s), connection dropped\n",
+                    e.what());
+            CLOSESOCK(s);
+            nSpawnFails++;
+            int nWaitMs = nSpawnFails < 5 ? nSpawnFails * 1000 : 5000;
+            for (int i = 0; i < nWaitMs && !g_stop; i += 100)
+                std::this_thread::sleep_for(std::chrono::milliseconds(100));
+        }
     }
     CLOSESOCK(g_listen);
     g_listen = INVALID_SOCKET;
diff --git a/src/btftunnel.cpp b/src/btftunnel.cpp
index 63d178b..2b3f818 100644
--- a/src/btftunnel.cpp
+++ b/src/btftunnel.cpp
@@ -25,8 +25,10 @@ typedef int SOCKET;
 #endif
 #include "sockcount.h"
 #include <cstring>
+#include <cstdio>
 #include <vector>
 #include <thread>
+#include <system_error>
 
 namespace btf
 {
@@ -121,18 +123,60 @@ static void PumpIn(SOCKET pumpEnd, RvSocket rv, std::vector<unsigned char> key)
 
 // Start the two pumps and hand back the app-facing socket. When both pumps
 // finish, close the pump end and the rendezvous socket.
+//
+// Every tunnel is two threads, and a tunnel is one peer connection (a .btf
+// dial or accept, a pool miner, a participant's pool session) -- so the
+// thread count scales with peer count, which is attacker-influenced. When
+// pthread_create is refused std::thread throws std::system_error; uncaught it
+// reached std::terminate and took the whole process down over one connection
+// it could not serve. Both spawns are guarded, and the failure paths return
+// INVALID_SOCKET, which every caller already handles by dropping just that
+// connection.
 static btf_socket_t StartTunnel(RvSocket rv, const unsigned char shared[32])
 {
     btf_socket_t appEnd, pumpEnd;
     if (!MakeLocalPair(appEnd, pumpEnd)) { RvClose(rv); return INVALID_SOCKET; }
 
     std::vector<unsigned char> key(shared, shared + 32);
-    std::thread(PumpOut, pumpEnd, rv, key).detach();
-    std::thread([pumpEnd, rv, key]() {
-        PumpIn(pumpEnd, rv, key);
+
+    // The inbound pump starts first because it owns the cleanup: it closes
+    // pumpEnd and rv when it exits. That ownership is what makes the second
+    // spawn's failure path below race-free.
+    try
+    {
+        std::thread([pumpEnd, rv, key]() {
+            PumpIn(pumpEnd, rv, key);
+            BtfCloseSocket(pumpEnd);
+            RvClose(rv);
+        }).detach();
+    }
+    catch (const std::system_error& e)
+    {
+        // Nothing is running yet, so closing everything here is safe.
+        fprintf(stderr, "btf tunnel: cannot spawn inbound pump (%s), tunnel dropped\n",
+                e.what());
         BtfCloseSocket(pumpEnd);
+        BtfCloseSocket(appEnd);
         RvClose(rv);
-    }).detach();
+        return INVALID_SOCKET;
+    }
+
+    try
+    {
+        std::thread(PumpOut, pumpEnd, rv, key).detach();
+    }
+    catch (const std::system_error& e)
+    {
+        // The inbound pump is parked in RvReadN(rv). shutdown() wakes it, and
+        // it closes pumpEnd and rv itself -- closing them here instead would
+        // race exactly that, and a reused descriptor is how an unrelated
+        // socket gets closed.
+        fprintf(stderr, "btf tunnel: cannot spawn outbound pump (%s), tunnel dropped\n",
+                e.what());
+        shutdown((SOCKET)rv, SD_BOTH);
+        BtfCloseSocket(appEnd);
+        return INVALID_SOCKET;
+    }
     return appEnd;
 }
 
diff --git a/src/main.cpp b/src/main.cpp
index 7b3e555..d154523 100644
--- a/src/main.cpp
+++ b/src/main.cpp
@@ -14,6 +14,7 @@
 #include <atomic>
 #include <mutex>
 #include <thread>
+#include <system_error>
 #include "bip32.h"
 #include "btfaddr.h"
 #include "btftunnel.h"
@@ -3899,9 +3900,32 @@ connected:
     std::vector<ParticipantShareHit> shareHits;
     std::mutex shareHitsMtx;
     std::vector<std::thread> hashThreads;
-    for (int i = 0; i < nWorkers; i++)
-        hashThreads.emplace_back(ParticipantHashWorker, &jobShared, i, nWorkers,
-                                 &shareHits, &shareHitsMtx);
+    try
+    {
+        for (int i = 0; i < nWorkers; i++)
+            hashThreads.emplace_back(ParticipantHashWorker, &jobShared, i, nWorkers,
+                                     &shareHits, &shareHitsMtx);
+    }
+    catch (const std::system_error& e)
+    {
+        // A refused pthread_create used to reach std::terminate here and
+        // kill the worker mid-session -- observed twice on an otherwise
+        // healthy participant. Stop the workers that did start the same way
+        // the disconnect path below does, drop this pool connection, and let
+        // the caller's reconnect loop try again once the pressure clears.
+        LogPrint("worker", "[worker] cannot spawn hash threads (%s)"
+                 " -- dropping pool connection\n", e.what());
+        SetParticipantMiningStatus("hash thread spawn failed");
+        {
+            std::lock_guard<std::mutex> lk(jobShared.mtx);
+            jobShared.stop = true;
+            jobShared.haveJob = false;
+        }
+        for (std::thread& t : hashThreads)
+            t.join();
+        closesocket(s);
+        return false;
+    }
     LogPrint("worker", "[worker] spawned %d hash threads on shared RandomX dataset\n",
              nWorkers);
     const uint64 sessionStartHashes = gParticipantHashes.load();
diff --git a/src/net.cpp b/src/net.cpp
index fed6123..16038c8 100644
--- a/src/net.cpp
+++ b/src/net.cpp
@@ -2090,7 +2090,7 @@ bool StartNode(string& strError)
     if (addrIncoming.ip)
         addrLocalHost.ip = addrIncoming.ip;
 
-    _beginthread([](void*) {
+    if (_beginthread([](void*) {
         unsigned int ip = addrLocalHost.ip;
         if (GetMyExternalIP(ip)) {
             addrLocalHost.ip = ip;
@@ -2098,7 +2098,8 @@ bool StartNode(string& strError)
             CWalletDB().WriteSetting("addrIncoming", addrIncoming);
             printf("External IP updated: %s\n", addrLocalHost.ToStringIP().c_str());
         }
-    }, 0, NULL);
+    }, 0, NULL) == (uintptr_t)-1)
+        printf("Error: _beginthread(GetMyExternalIP) failed\n");
 
     // Peers that answered last time, dialled straight away. The relays are the
     // only way to find anyone otherwise, and reaching them plus walking their
diff --git a/src/randomx_pow.cpp b/src/randomx_pow.cpp
index 1d058d4..9c59364 100644
--- a/src/randomx_pow.cpp
+++ b/src/randomx_pow.cpp
@@ -8,6 +8,7 @@
 #include <thread>
 #include <vector>
 #include <mutex>
+#include <system_error>
 
 // Fixed key (seed) that determines the RandomX cache/dataset. Keeping it
 // constant avoids per-epoch "reseed" logic; for the CPU+RAM fairness goal
@@ -101,13 +102,28 @@ bool RandomXInitDataset(int nThreads)
 
     std::vector<std::thread> workers;
     unsigned long per = total / nThreads;
-    for (int i = 0; i < nThreads; i++)
+    try
     {
-        unsigned long start = i * per;
-        unsigned long count = (i == nThreads - 1) ? (total - start) : per;
-        workers.emplace_back([start, count]() {
-            randomx_init_dataset(g_dataset, g_cache, start, count);
-        });
+        for (int i = 0; i < nThreads; i++)
+        {
+            unsigned long start = i * per;
+            unsigned long count = (i == nThreads - 1) ? (total - start) : per;
+            workers.emplace_back([start, count]() {
+                randomx_init_dataset(g_dataset, g_cache, start, count);
+            });
+        }
+    }
+    catch (const std::system_error& e)
+    {
+        // Same refusal as the allocation failure above: light mode is slow
+        // but alive, std::terminate is neither. Join the workers that did
+        // start before releasing the dataset out from under them.
+        for (auto& w : workers) w.join();
+        randomx_release_dataset(g_dataset);
+        g_dataset = NULL;
+        printf("RandomX: cannot spawn dataset init threads (%s), staying in light mode\n",
+               e.what());
+        return false;
     }
     for (auto& w : workers) w.join();
 
diff --git a/src/rpc.cpp b/src/rpc.cpp
index a4b53ca..13cb6c9 100644
--- a/src/rpc.cpp
+++ b/src/rpc.cpp
@@ -5077,10 +5077,17 @@ void ThreadRPCServer(void*)
     // armed for its one wallet probe before it is allowed to send.
     ReconcilePayouts();
 
-    _beginthread(AcceptLoopFn, 0, NULL);
-    _beginthread(LocalAcceptFn, 0, NULL);
-    _beginthread(ThreadBtfPoolAnnouncer, 0, NULL);
-    _beginthread(PayoutThreadFn, 0, NULL);
+    // _beginthread cannot throw (it reports -1), but an unchecked failure
+    // leaves the pool running deaf: nobody accepts, announces, or pays, and
+    // nothing in the log says why.
+    if (_beginthread(AcceptLoopFn, 0, NULL) == (uintptr_t)-1)
+        LogPrint("pool", "[pool] FATAL: accept loop thread failed to spawn -- miners cannot connect\n");
+    if (_beginthread(LocalAcceptFn, 0, NULL) == (uintptr_t)-1)
+        LogPrint("pool", "[pool] local accept thread failed to spawn\n");
+    if (_beginthread(ThreadBtfPoolAnnouncer, 0, NULL) == (uintptr_t)-1)
+        LogPrint("pool", "[pool] announcer thread failed to spawn -- pool will not be advertised\n");
+    if (_beginthread(PayoutThreadFn, 0, NULL) == (uintptr_t)-1)
+        LogPrint("pool", "[pool] FATAL: payout thread failed to spawn -- no one gets paid\n");
 
     PoolEventLoop(); // blocks until gPoolRunning = false or fShutdown