Remember .btf peers that answered, and dial them first on start (#25)

* net: remember .btf peers that answered, and dial them first on start

Discovery depends entirely on the Nostr relays. A node that has been up
for hours knows exactly which peers respond, and throws all of it away on
exit; the next start reaches for the relays again and walks the same
descriptor list, of which roughly 96% is dead. That is what makes a
restart take minutes before the first tunnel opens -- measured today at
about seven.

Every peer that completes a tunnel is now written to btfpeers.json with
its meeting node and encryption key, which is everything needed to redial
it. On start a thread tries those before the relays have answered
anything, stopping once eight are connected. Entries older than a week
are dropped, and the file holds at most fifty.

Cache only: nothing here changes how a first-ever run bootstraps, and a
missing or corrupt file just falls through to relay discovery.

* net: exchange signed .btf descriptors between peers

Discovery has depended entirely on the Nostr relays. Two connected nodes never
told each other who else existed, so a relay outage left a running network
unable to grow, and a node that found one stale peer had no way to learn there
was anything better. The old `addr` gossip went out with the IRC seed.

Peers now trade signed descriptors over a new "btfpeers" message: our own
first, then peers that actually answered us. The receiver checks each Schnorr
signature against the key the `.btf` address decodes to, so the sender is
trusted for nothing -- forging an entry needs somebody else's secret key. That
leaves flooding with self-generated keys, which the per-message caps and the
"peers that answered" ordering are there to blunt.

Unknown commands are ignored (main.cpp), so nodes without this just drop the
message; nothing here is a protocol break.

The cache from the peer-cache change now keeps each peer's descriptor verbatim
alongside the parsed fields, because that signature is what makes a peer
relayable. Entries without one -- resolved through Nostr, or written before
this -- stay dialable, just not passed on. A dial never clears a descriptor
already on file.

* main_gui: implement /port, which net.cpp already documented

net.cpp has said nListenPort is "tunable via /port" since it was written, but
nothing ever parsed the option. Two nodes could not share a machine, which is
exactly what testing peer exchange needs. Sets addrLocalHost.port too, so we
do not advertise a port we never bound.

6 files changed, 386 insertions, 0 deletions

diff --git a/src/main.cpp b/src/main.cpp
index a75bf54..bb317d2 100644
--- a/src/main.cpp
+++ b/src/main.cpp
@@ -1877,6 +1877,14 @@ bool ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv)
             pfrom->PushMessage("getblocks", CBlockLocator(pindexBest), uint256(0));
         }
 
+        // Introduce ourselves and hand over the .btf peers that answered us.
+        // Everything in here is self-certifying, so this is safe to send to a
+        // node we know nothing about, and safe for it to act on.
+        vector<string> vPexOut;
+        BtfPexCollect(vPexOut);
+        if (!vPexOut.empty())
+            pfrom->PushMessage("btfpeers", vPexOut);
+
         if (LogAcceptsCategory("net")) printf("version addrMe = %s\n", addrMe.ToString().c_str());
     }
 
@@ -1888,6 +1896,29 @@ bool ProcessMessage(CNode* pfrom, string strCommand, CDataStream& vRecv)
     }
 
 
+    else if (strCommand == "btfpeers")
+    {
+        // Peer exchange. Nothing here is taken on trust: BtfPexAccept checks
+        // every descriptor's signature against the key its .btf address
+        // decodes to, and drops the rest.
+        int64 nNow = GetTime();
+        if (nNow - pfrom->nLastPexRecv < PEX_MIN_INTERVAL)
+        {
+            if (LogAcceptsCategory("net"))
+                printf("btfpeers: ignoring, %s is sending them too fast\n",
+                       pfrom->addr.ToString().c_str());
+            return true;
+        }
+        pfrom->nLastPexRecv = nNow;
+
+        vector<string> vDesc;
+        vRecv >> vDesc;
+        int nKept = BtfPexAccept(vDesc);
+        LogPrint("net", "btfpeers: kept %d of %zu descriptor(s) from %s\n",
+                 nKept, vDesc.size(), pfrom->addr.ToString().c_str());
+    }
+
+
     else if (strCommand == "inv")
     {
         vector<CInv> vInv;
diff --git a/src/main_gui.cpp b/src/main_gui.cpp
index 842b0d5..edc772d 100644
--- a/src/main_gui.cpp
+++ b/src/main_gui.cpp
@@ -58,6 +58,7 @@ static void PrintUsage()
     printf("  /connectbtf=PEER_BTF_ADDRESS\n");
     printf("  /rvrelay=HOST:PORT\n");
     printf("  /announcerelay=HOST:PORT\n");
+    printf("  /port=N                  (P2P listen port, default 8433)\n");
     printf("\n");
     printf("Each option also accepts '-' instead of '/'.\n");
 }
@@ -113,6 +114,23 @@ static void ParseStartupArguments(int argc, char* argv[])
     string announceRelay = argval2(argc, argv, "/announcerelay", "-announcerelay");
     if (!announceRelay.empty())
         strBtfAnnounceRelay = announceRelay;
+
+    // net.cpp has claimed nListenPort was "tunable via /port" since it was
+    // written, but nothing ever read the option. Without it a second node
+    // cannot start on a machine that already runs one -- which is exactly what
+    // testing peer exchange needs.
+    string strPort = argval2(argc, argv, "/port", "-port");
+    if (!strPort.empty())
+    {
+        int nPort = atoi(strPort.c_str());
+        if (nPort <= 0 || nPort > 65535)
+            fprintf(stderr, "Ignoring /port=%s: not a port number\n", strPort.c_str());
+        else
+        {
+            nListenPort = htons((unsigned short)nPort);
+            addrLocalHost.port = nListenPort;  // or we advertise a port we never bound
+        }
+    }
 }
 
 int main(int argc, char* argv[])
diff --git a/src/net.cpp b/src/net.cpp
index ab07f94..dfeb4a2 100644
--- a/src/net.cpp
+++ b/src/net.cpp
@@ -2,6 +2,13 @@
 // Distributed under the MIT/X11 software license, see the accompanying
 // file license.txt or http://www.opensource.org/licenses/mit-license.php.
 
+#pragma push_macro("snprintf")
+#undef snprintf
+#include <nlohmann/json.hpp>
+#pragma pop_macro("snprintf")
+#ifdef snprintf
+#undef snprintf
+#endif
 #include "headers.h"
 #ifdef _WIN32
 #include <winsock2.h>
@@ -10,6 +17,7 @@
 #include "btfaddr.h"
 #include "btftunnel.h"
 
+void ThreadReconnectCachedBtfPeers(void* parg);
 void ThreadMessageHandler2(void* parg);
 void ThreadSocketHandler2(void* parg);
 
@@ -259,6 +267,269 @@ static CAddress BtfMarkerAddr(const unsigned char b5[5])
 // ConnectNodeBtfResolved (which takes an already-resolved descriptor, so it
 // makes no Nostr relay calls at all -- see nostr.cpp's BtfResolveMany for why
 // that matters when dialing many candidates in parallel).
+//
+// Peers we have actually reached, remembered across restarts.
+//
+// Discovery is otherwise entirely dependent on the Nostr relays: a node that
+// has been running for hours and knows exactly which peers answer throws all
+// of it away on exit, and the next start walks the same ~96%-dead descriptor
+// list again. Reconnecting from this file skips the relays completely --
+// address, meeting node and encryption key are all we need.
+//
+static const size_t MAX_CACHED_BTF_PEERS = 50;
+static const int64  CACHED_BTF_PEER_TTL  = 7 * 24 * 60 * 60; // a week
+
+struct CachedBtfPeer
+{
+    string btfAddr;
+    string meeting;
+    string encHex;
+    int64  lastSeen;
+    // The peer's own signed descriptor, as it announced itself. Kept verbatim
+    // because that signature is what lets us hand this peer on to somebody
+    // else: the receiver checks it against the key the address decodes to and
+    // never has to take our word for anything. Empty for entries learned
+    // before peer exchange existed, or resolved through Nostr -- those are
+    // still dialable, just not relayable.
+    string desc;
+};
+
+static string BtfPeerCachePath() { return GetAppDir() + "/btfpeers.json"; }
+
+static string BytesToHex(const unsigned char* p, size_t n)
+{
+    static const char* h = "0123456789abcdef";
+    string s;
+    s.reserve(n * 2);
+    for (size_t i = 0; i < n; i++) { s += h[p[i] >> 4]; s += h[p[i] & 0xf]; }
+    return s;
+}
+
+static bool HexToBytes(const string& s, unsigned char* out, size_t n)
+{
+    if (s.size() != n * 2) return false;
+    for (size_t i = 0; i < n; i++)
+    {
+        unsigned int b;
+        if (sscanf(s.c_str() + i * 2, "%2x", &b) != 1) return false;
+        out[i] = (unsigned char)b;
+    }
+    return true;
+}
+
+void LoadCachedBtfPeers(vector<CachedBtfPeer>& out)
+{
+    out.clear();
+    FILE* f = fopen(BtfPeerCachePath().c_str(), "r");
+    if (!f) return;
+    string body;
+    char buf[4096];
+    size_t r;
+    while ((r = fread(buf, 1, sizeof(buf), f)) > 0) body.append(buf, r);
+    fclose(f);
+    try
+    {
+        nlohmann::json arr = nlohmann::json::parse(body);
+        if (!arr.is_array()) return;
+        int64 nNow = GetTime();
+        for (const auto& o : arr)
+        {
+            if (!o.is_object()) continue;
+            if (!o.contains("btf") || !o["btf"].is_string()) continue;
+            if (!o.contains("meeting") || !o["meeting"].is_string()) continue;
+            if (!o.contains("enc") || !o["enc"].is_string()) continue;
+            CachedBtfPeer p;
+            p.btfAddr  = o["btf"].get<string>();
+            p.meeting  = o["meeting"].get<string>();
+            p.encHex   = o["enc"].get<string>();
+            p.lastSeen = o.value("seen", (int64)0);
+            if (o.contains("desc") && o["desc"].is_string())
+                p.desc = o["desc"].get<string>();
+            if (nNow - p.lastSeen > CACHED_BTF_PEER_TTL) continue; // long gone
+            out.push_back(p);
+        }
+    }
+    catch (...)
+    {
+        LogPrint("net", "btfpeers: cache unreadable, starting empty\n");
+        out.clear();
+    }
+}
+
+static CCriticalSection cs_btfPeerCache;
+
+// strDesc is the peer's own signed descriptor when we have it (it announced
+// itself over peer exchange), "" when we only resolved it through Nostr. An
+// empty one never clears a descriptor already on file: dialing a peer we first
+// learned about by exchange must not cost us the ability to pass it on.
+static void RememberBtfPeer(const string& strBtfAddr, const string& strMeeting,
+                            const unsigned char enc_pub[32],
+                            const string& strDesc = string())
+{
+    CRITICAL_BLOCK(cs_btfPeerCache)
+    {
+        vector<CachedBtfPeer> peers;
+        LoadCachedBtfPeers(peers);
+
+        string strKeepDesc = strDesc;
+        for (size_t i = 0; i < peers.size(); i++)
+            if (peers[i].btfAddr == strBtfAddr)
+            {
+                if (strKeepDesc.empty())
+                    strKeepDesc = peers[i].desc;
+                peers.erase(peers.begin() + i);
+                break;
+            }
+
+        CachedBtfPeer p;
+        p.btfAddr  = strBtfAddr;
+        p.meeting  = strMeeting;
+        p.encHex   = BytesToHex(enc_pub, 32);
+        p.lastSeen = GetTime();
+        p.desc     = strKeepDesc;
+        peers.insert(peers.begin(), p); // most recent first
+
+        if (peers.size() > MAX_CACHED_BTF_PEERS)
+            peers.resize(MAX_CACHED_BTF_PEERS);
+
+        nlohmann::json arr = nlohmann::json::array();
+        foreach(const CachedBtfPeer& q, peers)
+        {
+            nlohmann::json o;
+            o["btf"]     = q.btfAddr;
+            o["meeting"] = q.meeting;
+            o["enc"]     = q.encHex;
+            o["seen"]    = q.lastSeen;
+            if (!q.desc.empty())
+                o["desc"] = q.desc;
+            arr.push_back(o);
+        }
+        FILE* f = fopen(BtfPeerCachePath().c_str(), "w");
+        if (f)
+        {
+            string s = arr.dump(2);
+            fwrite(s.c_str(), 1, s.size(), f);
+            fclose(f);
+        }
+    }
+}
+
+// Dial everything we reached last time, before the relays have said anything.
+void ThreadReconnectCachedBtfPeers(void* parg)
+{
+    vector<CachedBtfPeer> peers;
+    LoadCachedBtfPeers(peers);
+    if (peers.empty())
+    {
+        LogPrint("net", "btfpeers: no cache yet, waiting on relay discovery\n");
+        return;
+    }
+    LogPrint("net", "btfpeers: trying %zu remembered peer(s) before discovery\n",
+             peers.size());
+
+    int nConnected = 0;
+    foreach(const CachedBtfPeer& p, peers)
+    {
+        if (fShutdown) return;
+        unsigned char enc[32];
+        if (!HexToBytes(p.encHex, enc, 32)) continue;
+        if (ConnectNodeBtfResolved(p.btfAddr, p.meeting, enc))
+        {
+            nConnected++;
+            if (nConnected >= 8) break; // enough to bootstrap; the rest can wait
+        }
+    }
+    LogPrint("net", "btfpeers: %d of %zu remembered peer(s) answered\n",
+             nConnected, peers.size());
+}
+
+//
+// .btf peer exchange
+//
+// Until now discovery depended entirely on the Nostr relays: two connected
+// nodes never told each other who else existed, so a relay outage left a
+// running network unable to grow, and a node that found one stale peer had no
+// way to learn there was anything better.
+//
+// What travels here is each peer's own signed descriptor. The receiver checks
+// the Schnorr signature against the key the `.btf` address decodes to, so the
+// sender is trusted for nothing -- forging an entry would need somebody else's
+// secret key. That leaves flooding: valid descriptors for keys the sender
+// generated itself. The caps below bound one message, and the cache prefers
+// peers that actually answered, so a flood costs a Sybil more than it costs us.
+//
+
+void BtfPexCollect(vector<string>& vDescOut)
+{
+    vDescOut.clear();
+
+    // Ours first -- the node on the other end may know nobody but us.
+    string strMine = BtfLocalDescriptor();
+    if (!strMine.empty())
+        vDescOut.push_back(strMine);
+
+    vector<CachedBtfPeer> peers;
+    CRITICAL_BLOCK(cs_btfPeerCache)
+        LoadCachedBtfPeers(peers);
+
+    // Most recently seen first: these answered us, they are not names copied
+    // off a relay listing.
+    foreach(const CachedBtfPeer& p, peers)
+    {
+        if (vDescOut.size() >= MAX_PEX_DESCRIPTORS) break;
+        if (p.desc.empty()) continue;                          // nothing provable to pass on
+        if (p.desc.size() > MAX_PEX_DESCRIPTOR_BYTES) continue;
+        vDescOut.push_back(p.desc);
+    }
+}
+
+int BtfPexAccept(const vector<string>& vDesc)
+{
+    void* ctx = BtfSecpContext();
+    if (!ctx)
+        return 0;
+
+    string strSelf = BtfLocalAddress();
+    int nKept = 0;
+    unsigned int nSeen = 0;
+
+    foreach(const string& strDesc, vDesc)
+    {
+        if (++nSeen > MAX_PEX_DESCRIPTORS) break;
+        if (strDesc.empty() || strDesc.size() > MAX_PEX_DESCRIPTOR_BYTES) continue;
+
+        // The descriptor names the key it belongs to, and taking that key from
+        // the blob is safe precisely because the signature must verify under
+        // it: nobody can produce a valid descriptor for a key they don't hold.
+        unsigned char pubkey[32];
+        try
+        {
+            nlohmann::json v = nlohmann::json::parse(strDesc);
+            if (!v.is_object() || !v.contains("pubkey") || !v["pubkey"].is_string())
+                continue;
+            if (!HexToBytes(v["pubkey"].get<string>(), pubkey, 32))
+                continue;
+        }
+        catch (...) { continue; }   // malformed JSON from an untrusted peer
+
+        btf::Descriptor d;
+        if (!btf::VerifyDescriptor(ctx, strDesc, pubkey, d))
+            continue;
+
+        string strAddr = btf::Address(pubkey);
+        if (strAddr.empty() || strAddr == strSelf)
+            continue;               // ourselves, nothing to learn
+
+        unsigned char enc[32];
+        if (!HexToBytes(d.enc, enc, 32))
+            continue;
+
+        RememberBtfPeer(strAddr, d.meeting_node, enc, strDesc);
+        nKept++;
+    }
+    return nKept;
+}
+
 static CNode* ConnectNodeBtfTail(const string& strBtfAddr, const unsigned char pk[32],
                                   const string& strMeeting, const unsigned char enc_pub[32])
 {
@@ -289,6 +560,9 @@ static CNode* ConnectNodeBtfTail(const string& strBtfAddr, const unsigned char p
     if (fDebug)
         LogPrint("net", "connected %s via rendezvous %s\n", strBtfAddr.c_str(), strMeeting.c_str());
 
+    // This one answered -- worth trying first next time we start.
+    RememberBtfPeer(strBtfAddr, strMeeting, enc_pub);
+
     // Add node
     pnode = new CNode(hSocket, addr, false);
     pnode->AddRef();
@@ -951,6 +1225,12 @@ bool StartNode(string& strError)
         }
     }, 0, NULL);
 
+    // Peers that answered last time, dialled straight away. The relays are the
+    // only way to find anyone otherwise, and reaching them plus walking their
+    // descriptor list is what makes a restart take minutes.
+    if (_beginthread(ThreadReconnectCachedBtfPeers, 0, NULL) == -1)
+        printf("Error: _beginthread(ThreadReconnectCachedBtfPeers) failed\n");
+
     // Peer discovery over Nostr relays (replaces the old IRC seed)
     if (_beginthread(ThreadNostrSeed, 0, NULL) == -1)
         printf("Error: _beginthread(ThreadNostrSeed) failed\n");
diff --git a/src/net.h b/src/net.h
index f028c3b..7d70721 100644
--- a/src/net.h
+++ b/src/net.h
@@ -29,6 +29,21 @@ CNode* ConnectNodeBtfResolved(const string& strBtfAddr, const string& strMeeting
 void ThreadBtfAccept(void* parg);
 void ThreadBtfConnect(void* parg);
 extern string strBtfConnect;
+
+// --- .btf peer exchange ---------------------------------------------------
+// Peers gossip signed descriptors so discovery survives a relay outage. Both
+// caps apply to a single "btfpeers" message; the generic header limit is 256 MB,
+// which is no protection at all for something a stranger can send unprompted.
+static const unsigned int MAX_PEX_DESCRIPTORS      = 20;
+static const unsigned int MAX_PEX_DESCRIPTOR_BYTES = 1024;
+// Ignore a peer's exchange more often than this. One announcement per side per
+// connection is the intended traffic; anything faster is someone else's idea.
+static const int64        PEX_MIN_INTERVAL         = 60;
+
+// Descriptors to hand a peer: ours first, then peers that answered us.
+void BtfPexCollect(std::vector<std::string>& vDescOut);
+// Verify descriptors a peer sent and remember the good ones. Returns how many.
+int  BtfPexAccept(const std::vector<std::string>& vDesc);
 void AbandonRequests(void (*fn)(void*, CDataStream&), void* param1);
 bool AnySubscribed(unsigned int nChannel);
 void ThreadBitcoinMiner(void* parg);
@@ -467,6 +482,9 @@ public:
     // publish and subscription
     vector<char> vfSubscribe;
 
+    // Last "btfpeers" we accepted from this node, to rate-limit the exchange.
+    int64 nLastPexRecv;
+
 
     CNode(SOCKET hSocketIn, CAddress addrIn, bool fInboundIn=false)
     {
@@ -483,6 +501,7 @@ public:
         fDisconnect = false;
         nRefCount = 0;
         nReleaseTime = 0;
+        nLastPexRecv = 0;
         vfSubscribe.assign(256, false);
 
         // Push a version message
diff --git a/src/nostr.cpp b/src/nostr.cpp
index ec02c2e..482a835 100644
--- a/src/nostr.cpp
+++ b/src/nostr.cpp
@@ -342,6 +342,36 @@ std::string BtfLocalAddress()
     return g_nostrKey.BtfAddress();
 }
 
+// This node's own descriptor, signed and self-certifying -- the same blob
+// PublishDescriptor puts on Nostr, but handed straight to a peer instead.
+//
+// Reads strBtfActiveRelay directly rather than going through BtfActiveRelay():
+// only a rendezvous we actually registered at is worth announcing. Handing out
+// a meeting node we are not reachable at would make every peer that receives
+// it burn a dial, and over peer exchange that mistake travels further than it
+// does over Nostr.
+std::string BtfLocalDescriptor()
+{
+    if (!EnsureNostrKey())
+        return "";
+    string meeting;
+    CRITICAL_BLOCK(cs_activeRelay)
+        meeting = strBtfActiveRelay;
+    if (meeting.empty())
+        return "";
+    return btf::SignDescriptor(g_nostrKey.ctx, g_nostrKey.seckey,
+                               g_nostrKey.EncPubHex(), meeting, (uint64_t)GetTime());
+}
+
+// The shared secp256k1 context, so net.cpp can verify descriptors that arrive
+// from peers. NULL if the identity could not be loaded.
+void* BtfSecpContext()
+{
+    if (!EnsureNostrKey())
+        return NULL;
+    return (void*)g_nostrKey.ctx;
+}
+
 int GetDiscoveredPeerCount()
 {
     int n = 0;
diff --git a/src/nostr.h b/src/nostr.h
index 04149e3..b5eb5e3 100644
--- a/src/nostr.h
+++ b/src/nostr.h
@@ -82,6 +82,14 @@ bool BtfGetIdentity(unsigned char pubkey[32], unsigned char enc_sk[32]);
 // This node's printable .btf address, or "" if the identity failed to load.
 std::string BtfLocalAddress();
 
+// This node's own signed descriptor, ready to hand to a peer over the wire.
+// "" until ThreadBtfAccept has registered at a rendezvous.
+std::string BtfLocalDescriptor();
+
+// The shared secp256k1 context, for btf::VerifyDescriptor (which takes it as
+// void* to keep headers free of secp). NULL if the identity failed to load.
+void* BtfSecpContext();
+
 // Resolve a .btf address via the public discovery relays: fetch the owner's
 // self-certified descriptor and return its meeting node ("host:port") and
 // x25519 public key for the end-to-end channel.