From 9f7141b28e2da9db7d7cb099d1f13f28742ff0fa Mon Sep 17 00:00:00 2001 From: Bitflash Date: Thu, 30 Jul 2026 13:38:12 -0300 Subject: rendezvous: bound the wait for a dial, so a dead path is noticed (#51) A node registers at a relay and waits to be dialled. The wait had no bound, and ConnectTo() clears the socket timeout on purpose -- a registration is idle by design and may sit for a long time. What that cannot survive is the path dying with neither end saying so, which is the ordinary case and not the exotic one: a NAT drops the idle mapping, a relay restarts, a route changes. No FIN arrives, nothing becomes readable, and recv() never returns. The thread stays alive and asleep, so ThreadBtfAccept never reaches the `continue` that would register again. The node is unreachable and every outward sign says it is fine. Caught on the bootstrap seed: 27 minutes without a block, eight behind the network, relay reachable the whole time, and a restart fixing it instantly. Nothing in the log marked the moment it went deaf. I tried TCP keepalive first and it does not work here. With probes every 5s and a 2-probe limit, a socket whose replies were being dropped was still ESTAB after four minutes with the probe counter stuck at zero. It is measured, not assumed, and that approach is not in this change. So the bound is ours to enforce, at the one place that knows what waiting means. RvServiceWaitPaired takes a timeout; expiring is not an error but "nobody came, go round again", which the loop above already handles. Cost is one reconnect per relay per five minutes. Verified by killing a registration's path silently -- dropping only the relay's replies, so no RST or FIN is ever seen. With a 60s bound for the test: rendezvous: no dial at 92.246.128.180:8434 within 60s (or it dropped us), re-registering Registrations went 1 -> 2 and the node carried on. Before this, the same treatment left it blocked past four minutes with no log line at all. The socket goes back to blocking once a dial lands, because from that point it is a data tunnel and a tunnel is allowed to sit quiet between messages. --- src/btfrv.cpp | 51 +++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 49 insertions(+), 2 deletions(-) (limited to 'src/btfrv.cpp') diff --git a/src/btfrv.cpp b/src/btfrv.cpp index 63e092e..7efd38b 100644 --- a/src/btfrv.cpp +++ b/src/btfrv.cpp @@ -258,10 +258,57 @@ RvSocket RvServiceRegister(const char* relay_host, unsigned short port, return (RvSocket)s; } -bool RvServiceWaitPaired(RvSocket s) +bool RvServiceWaitPaired(RvSocket s, int nTimeoutSecs) { + // A registration socket is idle on purpose -- we announce ourselves and + // wait, possibly for a long time, for someone to dial. ConnectTo() clears + // the socket timeout for exactly that reason, and with no timeout at all + // this recv() has no way to end when the path dies without either side + // saying so: a NAT drops the idle mapping, a relay restarts, a route + // changes. No FIN arrives, nothing is readable, and the call never + // returns. The thread stays alive and asleep, so ThreadBtfAccept never + // reaches the `continue` that would register again, and the node goes deaf + // while every outward sign says it is healthy. + // + // Seen in production on the bootstrap seed: 27 minutes without a block, + // eight behind the network, relay reachable throughout, fixed instantly by + // a restart. + // + // TCP keepalive is the obvious answer and it does not work here -- I + // measured it. With probes every 5s and a 2-probe limit, a socket whose + // replies were being dropped was still ESTAB after four minutes, the probe + // counter stuck at zero. So the timeout is ours to enforce, at the one + // place that knows what waiting means. + // + // Expiring is not a failure. It costs one reconnect per interval and + // returns the loop to a known state. + if (nTimeoutSecs > 0) + { +#ifdef _WIN32 + DWORD tv = (DWORD)nTimeoutSecs * 1000; + setsockopt((SOCKET)s, SOL_SOCKET, SO_RCVTIMEO, (const char*)&tv, sizeof(tv)); +#else + struct timeval tv; tv.tv_sec = nTimeoutSecs; tv.tv_usec = 0; + setsockopt((SOCKET)s, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv)); +#endif + } + unsigned char paired = 0; - return ReadN((SOCKET)s, &paired, 1) && paired == 0x01; + bool fOk = ReadN((SOCKET)s, &paired, 1) && paired == 0x01; + + // Hand back a socket that blocks again: from here it is a data tunnel, and + // a tunnel legitimately sits quiet between messages. + if (fOk && nTimeoutSecs > 0) + { +#ifdef _WIN32 + DWORD tvOff = 0; + setsockopt((SOCKET)s, SOL_SOCKET, SO_RCVTIMEO, (const char*)&tvOff, sizeof(tvOff)); +#else + struct timeval tvOff; tvOff.tv_sec = 0; tvOff.tv_usec = 0; + setsockopt((SOCKET)s, SOL_SOCKET, SO_RCVTIMEO, &tvOff, sizeof(tvOff)); +#endif + } + return fOk; } RvSocket RvClientConnect(const char* relay_host, unsigned short port, -- cgit v1.2.3