Date: Tue, 3 Apr 2018 19:10:56 -0700
From: Gleb Naumenko <naumenko.gs@gmail.com>
To: Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>, 
	Jim Posen <jim.posen@gmail.com>
Message-ID: <a1de9445-731f-4f94-b5b9-6a94588ba12e@Spark>
In-Reply-To: <CADZtCSjU78fO4bKcyc-sTT_H_wuugAd5PF3Ncom-4F2uFk_AnQ@mail.gmail.com>
References: <9ab6e32e-db51-4ce4-8f3c-3a77f7b1f9bd@Spark>
	<CADZtCSjU78fO4bKcyc-sTT_H_wuugAd5PF3Ncom-4F2uFk_AnQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="5ac43479_614fd4a1_7d7e"
Subject: Re: [bitcoin-dev] Low-bandwidth transaction relay
Precedence: list

--5ac43479_614fd4a1_7d7e
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Yeah, sure.

> How much bandwidth is consumed by redundant tx INVs currently=3F
Currently, for an average public-IP node all INVs consume 0.05 Mbps or 54=
0 megabytes per day. This number is based on current ratio public-IP node=
s:private-IP nodes and transaction rate. This number is a sum of both inc=
oming and outgoing aspects. Thus redundant INV=E2=80=99s on average consu=
me 0.044 Mbps or 475 megabytes per day.

> What is this as a % of overall bandwidth usage=3F
This is hard to estimate because overall bandwidth includes helping other=
 nodes to bootstrap from scratch. If we don=E2=80=99t consider this aspec=
t, my very rough estimate, and a short experiment shows that INV=E2=80=99=
s are around 50% of overall bandwidth (it also depends on different facto=
rs like your hardware comparing to other public-IP nodes). I=E2=80=99m go=
ing to double-check this number soon.

> How would filtering txs through N=3D2 links affect network propagation=3F=

Yes, network propagation for a new protocol definitely worth measuring. I=
=E2=80=99m going to look at it in the near future.

> Do you propose setting filters on inbound peers as well=3F
This is a good question.
I think some filter may be applied to inbound connections. Theoretically,=
 a symmetrical filter does not make much sense =E2=80=94 it might be even=
tually the same filter for all of the connections except first 8 outgoing=
 ones, so it=E2=80=99s better to use independent filters.
However, I=E2=80=99m not entirely sure it is needed. =46ilters on inbound=
 peers will reduce a download aspect. It might be much less critical than=
 upload (if we assume that private-IP nodes hear about transactions later=
 because those have much fewer connections). I think this question needs =
another experiment.

On Apr 3, 2018, 10:45 AM -0700, Jim Posen <jim.posen=40gmail.com>, wrote:=

> Hey. This idea sounds quite interesting. It'd be helpful to see some mo=
re numbers to evaluate it.
>
> - How much bandwidth is consumed by redundant tx INVs currently=3F What=
 is this as a % of overall bandwidth usage=3F
> - How would filtering txs through N=3D2 links affect network propagatio=
n=3F This probably requires simulation to determine.
> - Do you propose setting filters on inbound peers as well=3F
>
> > On Mon, Apr 2, 2018 at 3:18 PM, Gleb Naumenko via bitcoin-dev <bitcoi=
n-dev=40lists.linuxfoundation.org> wrote:
> > > Hi all,
> > > I have a couple of ideas regarding transaction relay protocol and w=
anted to share it with and probably get some feedback.
> > >
> > > I did some emulation and simulation and found out that around 90% o=
f INV messages sent by public-IP nodes are idle (duplicate), obviously be=
cause each node creates 8 connections.=C2=A0 I also realized that sending=
 INV messages is a significant part of the overall bandwidth consumed by =
a public-IP node. At a larger scale, this will result in people not able =
to run a public-IP node.
> > >
> > > My idea is in some sense similar to BIP37 but applied to public-IP =
nodes. Here I want to emphasize that all the nodes will still receive *al=
l* of the transactions. A new protocol should also keep the same zero-tru=
st, robustness, decentralization guarantees and latency.
> > >
> > > Idea: while joining the network, a new node agrees on some filter w=
ith each of 8 nodes it connects to. So that NewNode <-> Node=5FA will be =
used to relay only a subset of transactions, NewNode <-> Node=5FB for ano=
ther subset. This will significantly decrease the redundancy.
> > >
> > > To keep the guarantees, I would keep some redundancy (for example, =
each transaction INV is sent over 2 links).
> > >
> > > To make it robust to attacks, I have 2 extensions in my mind:
> > > 1. Set reconciliation (for a subset of transactions) with *other* n=
odes. Getting a bloom filter of a subset of the mempool transactions from=
 Node=5FB may help to figure out whether Node=5FA is malicious, very slow=
, etc.
> > > 2. Rotating the filters every N minutes (N < 10)
> > >
> > > I can see some issues with latency here, but I believe this problem=
 has a solution.
> > >
> > > =46eedback is appreciated=21
> > >
> > > If you want to look at a draft of the proposal =E2=80=94 please let=
 me know.
> > > If there were any similar ideas =E2=80=94 please let me know.
> > >
> > > Best,
> > > Gleb
> > >
> > >
> > > =5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F
> > > bitcoin-dev mailing list
> > > bitcoin-dev=40lists.linuxfoundation.org
> > > https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
> > >
>

--5ac43479_614fd4a1_7d7e
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

<html xmlns=3D=22http://www.w3.org/1999/xhtml=22>
<head>
<title></title>
</head>
<body>
<div name=3D=22messageBodySection=22 style=3D=22font-size: 14px; font-fam=
ily: -apple-system, BlinkMacSystem=46ont, sans-serif;=22>
<div>Yeah, sure.</div>
<div><br /></div>
<div>&gt; How much bandwidth is consumed by redundant tx INVs currently=3F=
</div>
<div>Currently, for an average public-IP node all INVs consume 0.05 Mbps =
or 540 megabytes per day. This number is based on current ratio public-IP=
 nodes:private-IP nodes and transaction rate. This number is a sum of bot=
h incoming and outgoing aspects. Thus redundant INV=E2=80=99s on average =
consume 0.044 Mbps or 475 megabytes per day.</div>
<div><br /></div>
<div>&gt; What is this as a % of overall bandwidth usage=3F</div>
<div>This is hard to estimate because overall bandwidth includes helping =
other nodes to bootstrap from scratch. If we don=E2=80=99t consider this =
aspect, my very rough estimate, and a short experiment shows that INV=E2=80=
=99s are around 50% of overall bandwidth (it also depends on different fa=
ctors like your hardware comparing to other public-IP nodes). I=E2=80=99m=
 going to double-check this number soon.</div>
<div><br /></div>
<div>&gt; How would filtering txs through N=3D2 links affect network prop=
agation=3F</div>
<div>Yes, network propagation for a new protocol definitely worth measuri=
ng. I=E2=80=99m going to look at it in the near future.</div>
<div><br /></div>
<div>&gt; Do you propose setting filters on inbound peers as well=3F</div=
>
<div>This is a good question.&=23160;</div>
<div>I think some filter may be applied to inbound connections. Theoretic=
ally, a symmetrical filter does not make much sense =E2=80=94 it might be=
 eventually the same filter for all of the connections except first 8 out=
going ones, so it=E2=80=99s better to use independent filters.</div>
<div>However, I=E2=80=99m not entirely sure it is needed. =46ilters on in=
bound peers will reduce a download aspect. It might be much less critical=
 than upload (if we assume that private-IP nodes hear about transactions =
later because those have much fewer connections). I think this question n=
eeds another experiment.</div>
</div>
<div name=3D=22messageReplySection=22 style=3D=22font-size: 14px; font-fa=
mily: -apple-system, BlinkMacSystem=46ont, sans-serif;=22><br />
On Apr 3, 2018, 10:45 AM -0700, Jim Posen &lt;jim.posen=40gmail.com&gt;, =
wrote:<br />
<blockquote type=3D=22cite=22 style=3D=22margin: 5px 5px; padding-left: 1=
0px; border-left: thin solid =231abc9c;=22>
<div dir=3D=22ltr=22>Hey. This idea sounds quite interesting. It'd be hel=
pful to see some more numbers to evaluate it.
<div><br /></div>
<div>- How much bandwidth is consumed by redundant tx INVs currently=3F W=
hat is this as a % of overall bandwidth usage=3F</div>
<div>- How would filtering txs through N=3D2 links affect network propaga=
tion=3F This probably requires simulation to determine.</div>
<div>- Do you propose setting filters on inbound peers as well=3F</div>
</div>
<div class=3D=22gmail=5Fextra=22><br />
<div class=3D=22gmail=5Fquote=22>On Mon, Apr 2, 2018 at 3:18 PM, Gleb Nau=
menko via bitcoin-dev <span dir=3D=22ltr=22>&lt;<a href=3D=22mailto:bitco=
in-dev=40lists.linuxfoundation.org=22 target=3D=22=5Fblank=22>bitcoin-dev=
=40lists.linuxfoundation.org</a>&gt;</span> wrote:<br />
<blockquote class=3D=22gmail=5Fquote=22 style=3D=22margin: 5px 5px; paddi=
ng-left: 10px; border-left: thin solid =23e67e22;=22>
<div>
<div name=3D=22messageBodySection=22 style=3D=22font-size:14px;font-famil=
y:-apple-system,BlinkMacSystem=46ont,sans-serif=22>
<div>Hi all,</div>
<div>I have a couple of ideas regarding transaction relay protocol and wa=
nted to share it with and probably get some feedback.</div>
<div><br /></div>
<div>I did some emulation and simulation and found out that around 90% of=
 INV messages sent by public-IP nodes are idle (duplicate), obviously bec=
ause each node creates 8 connections.&=23160; I also realized that sendin=
g INV messages is a significant part of the overall bandwidth consumed by=
 a public-IP node. At a larger scale, this will result in people not able=
 to run a public-IP node.</div>
<div><br /></div>
<div>My idea is in some sense similar to BIP37 but applied to public-IP n=
odes. Here I want to emphasize that all the nodes will still receive *all=
* of the transactions. A new protocol should also keep the same zero-trus=
t, robustness, decentralization guarantees and latency.</div>
<div><br /></div>
<div>Idea: while joining the network, a new node agrees on some filter wi=
th each of 8 nodes it connects to. So that NewNode &lt;-&gt; Node=5FA wil=
l be used to relay only a subset of transactions, NewNode &lt;-&gt; Node=5F=
B for another subset. This will significantly decrease the redundancy.</d=
iv>
<div><br /></div>
<div>To keep the guarantees, I would keep some redundancy (for example, e=
ach transaction INV is sent over 2 links).</div>
<div><br /></div>
<div>To make it robust to attacks, I have 2 extensions in my mind:&=23160=
;</div>
<div>1. Set reconciliation (for a subset of transactions) with *other* no=
des. Getting a bloom filter of a subset of the mempool transactions from =
Node=5FB may help to figure out whether Node=5FA is malicious, very slow,=
 etc.</div>
<div>2. Rotating the filters every N minutes (N &lt; 10)</div>
<div><br /></div>
<div>I can see some issues with latency here, but I believe this problem =
has a solution.</div>
<div><br /></div>
<div>=46eedback is appreciated=21</div>
<div><br /></div>
<div>If you want to look at a draft of the proposal =E2=80=94 please let =
me know.</div>
<div>If there were any similar ideas =E2=80=94 please let me know.</div>
<div><br /></div>
<div>Best,</div>
<div>Gleb</div>
</div>
<div name=3D=22messageReplySection=22 style=3D=22font-size:14px;font-fami=
ly:-apple-system,BlinkMacSystem=46ont,sans-serif=22><br />
<div></div>
</div>
</div>
<br />
=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
=5F=5F=5F=5F=5F<wbr />=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=5F=
<br />
bitcoin-dev mailing list<br />
<a href=3D=22mailto:bitcoin-dev=40lists.linuxfoundation.org=22>bitcoin-de=
v=40lists.<wbr />linuxfoundation.org</a><br />
<a href=3D=22https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-d=
ev=22 rel=3D=22noreferrer=22 target=3D=22=5Fblank=22>https://lists.linuxf=
oundation.<wbr />org/mailman/listinfo/bitcoin-<wbr />dev</a><br />
<br /></blockquote>
</div>
<br /></div>
</blockquote>
<div></div>
</div>
</body>
</html>

--5ac43479_614fd4a1_7d7e--