1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
|
Return-Path: <gmaxwell@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
[172.17.192.35])
by mail.linuxfoundation.org (Postfix) with ESMTPS id 53DBDB15
for <bitcoin-dev@lists.linuxfoundation.org>;
Fri, 26 Feb 2016 05:56:58 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-io0-f180.google.com (mail-io0-f180.google.com
[209.85.223.180])
by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 12DFB11F
for <bitcoin-dev@lists.linuxfoundation.org>;
Fri, 26 Feb 2016 05:56:57 +0000 (UTC)
Received: by mail-io0-f180.google.com with SMTP id z135so113231744iof.0
for <bitcoin-dev@lists.linuxfoundation.org>;
Thu, 25 Feb 2016 21:56:57 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
h=mime-version:sender:in-reply-to:references:date:message-id:subject
:from:to:cc; bh=c3JO4WRDiqq9k2kYop9lmo0rUjWj8tp4URO2QjNk0mc=;
b=IOPCve0z3V9+K8M1YT+haaOkxH9LfF/5AwcsmmyDNd/ykvinFk3MgFLQ2CpofV7nTX
RVGjZOoGvioJhPiJTLtXsVkMb9AjBvRxceW+rZrwgC8OIBN77YkMDR6LD0SkQePwuYe2
6FlaD0fIdMkx2lyldPA3VgpO7YmjAV1l5OIdZtu6WIePrkKB/BpHonTqmYfDSS95FFC1
d6IXQhfNRaIilPa7xg9Zd3Mc16R7fqIzRGgwbIRCQcKZsylDRiEYeLJCQ5PDJr5OuuB0
w/kMHSsHdAui3NWDQugd+PqeX7ifBrMeWcvfWdU1EBGBeKKv6zKO3oKaO1uGFtDJhQDO
w3ww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20130820;
h=x-gm-message-state:mime-version:sender:in-reply-to:references:date
:message-id:subject:from:to:cc;
bh=c3JO4WRDiqq9k2kYop9lmo0rUjWj8tp4URO2QjNk0mc=;
b=dVVMjpKbU32zH5f8O3IploixxerhU5yratMKvZ6ER7OPtVnd1SXc24i10Z+W2hM3m5
ai3U/nchejZgB/7Pa1WaXqPYDSbdGWMBo/n0Db7+fHr2ChbsF1n/cyxAks8Qt5RfUDwP
WLRdeRh7B+vPNSHTPC7EiPBo3SfbTaT4gp3f44wPwHasFnjKqeNTCF1aVdp8PnfnramI
QWAezfqp1/TbS7JIXCeJck9wZ5b1vv8oOxvxp2Nt/3xGBKgD1Bb89YTPcO2AovxPXOx4
0QQasfJUKs4q5ZHhoX8NLtiiYM2m6+n47EgubaB3ehPXhv9PZww37Xm9ogGCIChhet/K
QeEw==
X-Gm-Message-State: AG10YOTbXz+/ui4TBmvD1sS+h45s4Nu6er7FBS4R8FVSYresFUJyROF1EaffDN3r7/AkxJ1Za8lxnvf2YHMPVg==
MIME-Version: 1.0
X-Received: by 10.107.132.142 with SMTP id o14mr6500401ioi.75.1456466216559;
Thu, 25 Feb 2016 21:56:56 -0800 (PST)
Sender: gmaxwell@gmail.com
Received: by 10.107.132.75 with HTTP; Thu, 25 Feb 2016 21:56:56 -0800 (PST)
In-Reply-To: <B186E7A6-0FD4-4C82-B42F-7EE61D420A7E@toom.im>
References: <B186E7A6-0FD4-4C82-B42F-7EE61D420A7E@toom.im>
Date: Fri, 26 Feb 2016 05:56:56 +0000
X-Google-Sender-Auth: CjOlxStZYW3hGY077rOMO8OsLAI
Message-ID: <CAAS2fgTTUjVUx0GQYed-tWnH4RmS0tpv2yrpCOGJSeW8kJkYiw@mail.gmail.com>
From: Gregory Maxwell <greg@xiph.org>
To: Jonathan Toomim <j@toom.im>
Content-Type: text/plain; charset=UTF-8
X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIM_SIGNED,
DKIM_VALID, FREEMAIL_FROM, RCVD_IN_DNSWL_LOW autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
smtp1.linux-foundation.org
Cc: Bitcoin Dev <bitcoin-dev@lists.linuxfoundation.org>
Subject: Re: [bitcoin-dev] INV overhead and batched INVs to reduce full node
traffic
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Development Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Fri, 26 Feb 2016 05:56:58 -0000
On Fri, Feb 26, 2016 at 5:35 AM, Jonathan Toomim via bitcoin-dev
<bitcoin-dev@lists.linuxfoundation.org> wrote:
> An improvement that I've been thinking about implementing (after
> Blocktorrent) is an option for batched INVs. Including the hashes for two
> txes per IP packet instead of one would increase the INV size to 229 bytes
> for 64 bytes of payload -- that is, you add 36 bytes to the packet for every
> 32 bytes of actual payload. This is a marginal efficiency of 88.8% for each
> hash after the first. This is *much* better.
>
> Waiting a short period of time to accumulate several hashes together and
> send them as a batched INV could easily reduce the traffic of running
> bitcoin nodes by a factor of 2,
Copying my response to you from BitcoinTalk
(https://bitcointalk.org/index.php?topic=1377345.msg14013294#msg14013294):
Uh. Bitcoin has done this since the very early days. The batching was
temporarily somewhat hobbled between 0.10 and 0.12 (especially when
you had any abusive frequently pinging peers attached), but is now
fully functional again and it now manages to batch many transactions
per INV pretty effectively. Turn on net message debugging and you'll
see the many INVs that are much larger than the minimum. The average
batching size (ignoring the trickle cut-through) is about 5 seconds
long-- and usually gets about 10 transactions per INV. My measurements
were with these fixes in effect; I expect the blocksonly savings would
have been higher otherwise.
2016-02-26 05:47:08 sending: inv (1261 bytes) peer=33900
2016-02-26 05:47:08 sending: inv (109 bytes) peer=32460
2016-02-26 05:47:08 sending: inv (37 bytes) peer=34501
2016-02-26 05:47:08 sending: inv (217 bytes) peer=33897
2016-02-26 05:47:08 sending: inv (145 bytes) peer=41863
2016-02-26 05:47:08 sending: inv (37 bytes) peer=35725
2016-02-26 05:47:08 sending: inv (73 bytes) peer=20567
2016-02-26 05:47:08 sending: inv (289 bytes) peer=44703
2016-02-26 05:47:08 sending: inv (73 bytes) peer=13408
2016-02-26 05:47:09 sending: inv (649 bytes) peer=41279
2016-02-26 05:47:09 sending: inv (145 bytes) peer=42612
2016-02-26 05:47:09 sending: inv (325 bytes) peer=34525
2016-02-26 05:47:09 sending: inv (181 bytes) peer=41174
2016-02-26 05:47:09 sending: inv (469 bytes) peer=41460
2016-02-26 05:47:10 sending: inv (973 bytes) peer=133
2016-02-26 05:47:10 sending: inv (361 bytes) peer=20541
Twiddling here doesn't change the asymptotic efficiency though; which
is what my post is about.
[I'm also somewhat surprised that you were unaware of this; one of the
patches "classic" was talking about patching out was the one restoring
the batching... due to a transaction deanonymization service (or
troll) claiming it interfered with their operations.]
|
