net: give up waiting for a message body that is never going to arrive

THIS IS THE WEDGE, and the note asking for this fix has been in the file
since Bitcoin 0.1: 'need a mechanism to give up waiting for overlong
message size error'.

ProcessMessages reads a header, finds the body has not all arrived, pushes
the header back onto the front of vRecv and breaks. Correct for a message
still in flight. For one that never completes it is permanent: every later
pass re-reads the same header and breaks in the same place, so that peer is
never parsed again for the life of the process -- no blocks, no inv, not
even its version message.

Nothing rescues it:
  - a header may claim up to 256 MB (0x10000000) and still pass
    CMessageHeader::IsValid(), so a garbage or truncated header is accepted
    and then waited on forever;
  - nLastRecv is refreshed by any byte that arrives (net.cpp), whether or
    not it parses, so the inactivity timeout that drops a silent peer never
    fires on a peer that is merely unreadable;
  - the socket stays ESTABLISHED and select() keeps offering it, so nothing
    at the socket layer notices either.

It also explains the signature that made no sense. A peer stuck before its
version message is parsed keeps nStartingHeight == -1, so a node with a full
vNodes still reports GetPeerMedianHeight() == -1 -- which is exactly what
was observed, and exactly what disabled the pool's behind-the-network gate
and let it fork.

Evidence, from two wedges captured on the pool by the watchdog:
MESSAGE-BREAK 2 ran 2014 times in the deaf window against 607 in the healthy
window before it, while blocks received went to zero and the node kept
processing miner traffic normally throughout.

Now: a peer that announces a body and does not deliver it within two minutes
is dropped, and says so in the log unconditionally. Two minutes is far
beyond any legitimate header-to-body gap and well inside
BTF_RECV_TIMEOUT_SECS.

(cherry picked from commit ca1e99288e03ac7beb2b97cf615a3255b4635d95)

1 files changed, 23 insertions, 0 deletions

diff --git a/src/net.h b/src/net.h
index e4cfa69..112426e 100644
--- a/src/net.h
+++ b/src/net.h
@@ -127,6 +127,25 @@ static const int64        DISCONNECT_DRAIN_SECS    = 60;
 // 3966-4085 -- 120 blocks, 5890.50 BTF -- were mined in exactly this state.
 static const int64        BTF_NO_PEER_HEIGHT_MAX_SECS = 20 * 60;
 
+// How long a peer may leave this node waiting for the rest of a message it
+// announced in a header.
+//
+// ProcessMessages rewinds the header and breaks when the body has not all
+// arrived, which is correct for a message still in flight and unbounded for
+// one that never completes: the same header is re-read, the loop breaks
+// again, and that peer is never parsed again for the life of the process.
+// Satoshi left the note -- "need a mechanism to give up waiting for overlong
+// message size error" -- and this is it.
+//
+// It never self-corrects, because any byte that arrives refreshes nLastRecv,
+// so the inactivity timeout that would otherwise drop the peer never fires.
+// The socket stays healthy, select() keeps reporting it, and the node goes
+// silently deaf one peer at a time.
+//
+// A header may legitimately outrun its body by a round trip or two; two
+// minutes is far beyond that, and well inside BTF_RECV_TIMEOUT_SECS.
+static const int64        BTF_MSG_WAIT_MAX_SECS = 2 * 60;
+
 // Send a ping after this long with nothing to say, so a peer running the
 // inactivity check above does not mistake a quiet node for a dead one. Must
 // stay well under BTF_RECV_TIMEOUT_SECS.
@@ -651,6 +670,10 @@ public:
     string strBtfAddr;
     string strBtfMeeting;
 
+    // When this peer's message loop first rewound a header to wait for the
+    // rest of a message, or 0 when it is not waiting. See ProcessMessages.
+    int64 nMsgWaitSince;
+
     // Height this peer announced in its version message, or -1 if it sent a
     // version message that predates the field. Knowing how far along everyone
     // else is, is the only way a node can tell "in sync" from "not hearing".