rendezvous: register and wait for a dial as separate steps (#30)

A node could not become reachable. RvServiceRegister connected, sent its
'S'+pubkey header, and then blocked reading the relay's 0x01 byte -- which the
relay only writes when a client pairs with the service. So the call did not
return when the node was registered; it returned when somebody had already
dialled it.

ThreadBtfAccept called BtfSetActiveRelay after that. The result is a deadlock:

  - to be dialled, a node must publish a descriptor naming its meeting node
  - to publish, BtfActiveRelay() must be set
  - it is only set once somebody has dialled

Before #26 this was masked. BtfActiveRelay() fell back to
vBtfMeetingRelays[0], so a node always published something, and ThreadBtfAccept
picks its relay at random -- so it worked whenever the two happened to match,
roughly one time in the number of relays. That is the flakiness this project has
been chasing: not a bad relay, an advertisement that usually named the wrong one.

#26 removed the fallback, which was right on its own terms and turned an
unreliable heuristic into a reliable deadlock. This is the other half of that
change and they belong together.

RvServiceRegister now returns as soon as the relay has the service listed.
RvServiceWaitPaired does the blocking read. ThreadBtfAccept registers, advertises,
then waits. RvClientConnect is untouched -- a dialer blocking until it is paired
is correct.

Registration failure also had no log line at all, which is why this stayed
invisible: a node that never registered looked identical to one waiting quietly
for a peer. There are now lines for registering, failing over, and being dropped.

Verified: a fresh node reports `registered at 90.156.222.107:8434, waiting for a
dial` and then self-resolves with `meeting_node=90.156.222.107:8434` -- the relay
it is actually on. Two nodes on one machine reach each other.

3 files changed, 44 insertions, 7 deletions

diff --git a/src/btfrv.cpp b/src/btfrv.cpp
index 0d02ca8..63e092e 100644
--- a/src/btfrv.cpp
+++ b/src/btfrv.cpp
@@ -250,11 +250,20 @@ RvSocket RvServiceRegister(const char* relay_host, unsigned short port,
     hdr[0] = (unsigned char)'S';
     memcpy(hdr + 1, my_pubkey, 32);
     if (!WriteN(s, hdr, 33)) { CLOSESOCK(s); return RV_INVALID; }
-    unsigned char paired = 0;
-    if (!ReadN(s, &paired, 1) || paired != 0x01) { CLOSESOCK(s); return RV_INVALID; }
+    // Return here: registered, not yet paired. The relay lists the service as
+    // soon as it reads this header. The 0x01 that used to be consumed below
+    // only arrives when somebody dials us, so blocking on it made "registered"
+    // and "already in use" indistinguishable to the caller -- and left the node
+    // unable to advertise a meeting node until after it had been reached there.
     return (RvSocket)s;
 }
 
+bool RvServiceWaitPaired(RvSocket s)
+{
+    unsigned char paired = 0;
+    return ReadN((SOCKET)s, &paired, 1) && paired == 0x01;
+}
+
 RvSocket RvClientConnect(const char* relay_host, unsigned short port,
                          const unsigned char target_pubkey[32])
 {
diff --git a/src/btfrv.h b/src/btfrv.h
index c776201..9d6756e 100644
--- a/src/btfrv.h
+++ b/src/btfrv.h
@@ -37,11 +37,20 @@ bool RvRelayRun(unsigned short port);
 void RvRelayStop();
 
 // ---- service side (the node being reached by its .btf address) ----
-// Register at the relay under `my_pubkey` (32 bytes) and block until a client is
-// paired to us. Returns the paired socket, or RV_INVALID on error.
+// Register at the relay under `my_pubkey` (32 bytes). Returns as soon as the
+// relay has us listed, WITHOUT waiting for anyone to dial -- the caller needs
+// that moment to start advertising this relay as its meeting node. Waiting for
+// the dial was folded into this call before, which deadlocked discovery: a node
+// could not be advertised until it had been dialled, and could not be dialled
+// until it was advertised. Returns RV_INVALID on error.
 RvSocket RvServiceRegister(const char* relay_host, unsigned short port,
                            const unsigned char my_pubkey[32]);
 
+// Block on a registered service socket until a client is paired to us. Returns
+// false if the relay drops us or errors, in which case the caller should close
+// the socket and register again.
+bool RvServiceWaitPaired(RvSocket s);
+
 // ---- client side (the node dialing a .btf address) ----
 // Connect through the relay to the service registered under `target_pubkey`.
 // Returns the paired socket, or RV_INVALID on error.
diff --git a/src/net.cpp b/src/net.cpp
index 7b2ef83..b5d12d0 100644
--- a/src/net.cpp
+++ b/src/net.cpp
@@ -554,7 +554,10 @@ void ThreadBtfAccept(void* parg)
         string strHost = strMeeting.substr(0, colon);
         int nPort = atoi(strMeeting.substr(colon + 1).c_str());
 
-        // Blocks until a client is paired to us (or the relay drops us)
+        // Returns as soon as the relay has us listed -- it does not wait for a
+        // dial. That distinction is the whole point: we have to be advertised
+        // before anyone can dial us, so registering and waiting cannot be the
+        // same call.
         btf::RvSocket rv = btf::RvServiceRegister(strHost.c_str(), (unsigned short)nPort, pk);
         if (fShutdown)
         {
@@ -564,13 +567,29 @@ void ThreadBtfAccept(void* parg)
         }
         if (rv == btf::RV_INVALID)
         {
+            LogPrint("net", "rendezvous: could not register at %s, trying another\n",
+                     strMeeting.c_str());
             iRelay++;       // this relay is down/attacked -> try the next one
             Sleep(3000);
             continue;
         }
-        // Registered OK: advertise THIS relay in our descriptor so clients dial
-        // us here, and keep using it (iRelay unchanged) until it fails.
+
+        // Registered. Advertise THIS relay now, while we are listed and before
+        // anybody dials -- a descriptor naming it is what makes a dial possible
+        // at all. Keep using it (iRelay unchanged) until it fails.
         BtfSetActiveRelay(strMeeting);
+        LogPrint("net", "rendezvous: registered at %s, waiting for a dial\n",
+                 strMeeting.c_str());
+
+        // Now block for someone to arrive.
+        if (!btf::RvServiceWaitPaired(rv))
+        {
+            btf::RvClose(rv);
+            LogPrint("net", "rendezvous: %s dropped us before any dial, re-registering\n",
+                     strMeeting.c_str());
+            continue;
+        }
+
         btf_socket_t hSocket = btf::BtfServiceWrap(rv, sk);
         if (hSocket == INVALID_SOCKET)
             continue;