path: root/86/b39bc5537352d934028ebffcf7eb24acacce50

Return-Path: <jim.posen@gmail.com>
Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org
	[172.17.192.35])
	by mail.linuxfoundation.org (Postfix) with ESMTPS id 4E01CC00
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Tue,  3 Apr 2018 17:45:36 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.7.6
Received: from mail-it0-f41.google.com (mail-it0-f41.google.com
	[209.85.214.41])
	by smtp1.linuxfoundation.org (Postfix) with ESMTPS id AD07B5E2
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Tue,  3 Apr 2018 17:45:35 +0000 (UTC)
Received: by mail-it0-f41.google.com with SMTP id u62-v6so13081743ita.5
	for <bitcoin-dev@lists.linuxfoundation.org>;
	Tue, 03 Apr 2018 10:45:35 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
	h=mime-version:in-reply-to:references:from:date:message-id:subject:to; 
	bh=o3z2ae3HBiQX3ecB1hlSIw5bOVxLBb4IACYVmOK6s+I=;
	b=gndpKkQBGw/dNUUb+nSJdob5l6w6q0jwBNWPWPBQlWQNtaL13IOYljQ9qqiF+bXP8k
	aHR2J+ZFR4H40PdtPZpo774DCV0aHt3yWVYc7mn71ld+h3u3kAScsuXuCYJcMJGOZRL8
	LzKn0X8y50RSAi1CDTCfkmiNnlXu6n+KrcUFZpNeEAKmoTAkLRiM0reIL4qrmuqdWfUL
	vYG4B01YxWbYJK7QTIfxXMYGsJB1w5ZtkFs3KXPltHkw0JZ5nblAjG27CbTlwuM9d3AH
	JxI6lC3vd0hdtNCKc70I4Te5RwfFKGLMqvB7TI3NHS99pzLr3WRmw1mFxG55wZQZkMUm
	HRiQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
	d=1e100.net; s=20161025;
	h=x-gm-message-state:mime-version:in-reply-to:references:from:date
	:message-id:subject:to;
	bh=o3z2ae3HBiQX3ecB1hlSIw5bOVxLBb4IACYVmOK6s+I=;
	b=jAe/0GubgicCw9N/Ww8zI5+JaaJ7QezzFuhYGqS1CJ6WfzkR/xqueKYWRSFgp06PrA
	ix/KQy7cH9odFD1kKi1qb9BLGK+jD3yoWwUBXuvc8KD/LIL/8Y3nB7ywKm/Sux1Hf/1Y
	7+mWE3/KQiM5gjVKdjd7mBb+nwKSNXes4t/Ubq5vmCiRuygYXJCXmXQIjdy4n8sqSz9m
	n3REOd3eZmE8Lxi2pibFvsaq/Geu2TE5/HtyDCL8FlWP5Y+RdfKGmLfPR6E28BREhlTM
	zRStqx05TxAdah2qJVCbTEAWZjxy5BJ/s8KVWMPxZFdloEhEY74h0/kRJLGiqiYHvZEx
	5jyQ==
X-Gm-Message-State: ALQs6tB7ad4BfLTjCYcTIneYdG4hFxz9G70ikX5FuOyh+JJCIAY6Hc2f
	qzX1rsq50xlvwrUkvJ0NTEzx4efLcgVavjX6sY4jag==
X-Google-Smtp-Source: AIpwx484AB4B8aKhs0Er6nrH1FaLiaNuyEduiFFreNRuQeE9BJg4OPI9e9qc77yCOhhb6VKy4JTEgD04bnr+EAa6fJA=
X-Received: by 2002:a24:7541:: with SMTP id
	y62-v6mr5901360itc.119.1522777534910; 
	Tue, 03 Apr 2018 10:45:34 -0700 (PDT)
MIME-Version: 1.0
Received: by 10.107.52.80 with HTTP; Tue, 3 Apr 2018 10:45:34 -0700 (PDT)
In-Reply-To: <9ab6e32e-db51-4ce4-8f3c-3a77f7b1f9bd@Spark>
References: <9ab6e32e-db51-4ce4-8f3c-3a77f7b1f9bd@Spark>
From: Jim Posen <jim.posen@gmail.com>
Date: Tue, 3 Apr 2018 10:45:34 -0700
Message-ID: <CADZtCSjU78fO4bKcyc-sTT_H_wuugAd5PF3Ncom-4F2uFk_AnQ@mail.gmail.com>
To: Gleb Naumenko <naumenko.gs@gmail.com>, 
	Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: multipart/alternative; boundary="0000000000000a55f40568f54641"
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HTML_MESSAGE,
	RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1
X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on
	smtp1.linux-foundation.org
X-Mailman-Approved-At: Tue, 03 Apr 2018 17:52:54 +0000
Subject: Re: [bitcoin-dev] Low-bandwidth transaction relay
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
	<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Tue, 03 Apr 2018 17:45:36 -0000

--0000000000000a55f40568f54641
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hey. This idea sounds quite interesting. It'd be helpful to see some more
numbers to evaluate it.

- How much bandwidth is consumed by redundant tx INVs currently? What is
this as a % of overall bandwidth usage?
- How would filtering txs through N=3D2 links affect network propagation?
This probably requires simulation to determine.
- Do you propose setting filters on inbound peers as well?

On Mon, Apr 2, 2018 at 3:18 PM, Gleb Naumenko via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> Hi all,
> I have a couple of ideas regarding transaction relay protocol and wanted
> to share it with and probably get some feedback.
>
> I did some emulation and simulation and found out that around 90% of INV
> messages sent by public-IP nodes are idle (duplicate), obviously because
> each node creates 8 connections.  I also realized that sending INV messag=
es
> is a significant part of the overall bandwidth consumed by a public-IP
> node. At a larger scale, this will result in people not able to run a
> public-IP node.
>
> My idea is in some sense similar to BIP37 but applied to public-IP nodes.
> Here I want to emphasize that all the nodes will still receive *all* of t=
he
> transactions. A new protocol should also keep the same zero-trust,
> robustness, decentralization guarantees and latency.
>
> Idea: while joining the network, a new node agrees on some filter with
> each of 8 nodes it connects to. So that NewNode <-> Node_A will be used t=
o
> relay only a subset of transactions, NewNode <-> Node_B for another subse=
t.
> This will significantly decrease the redundancy.
>
> To keep the guarantees, I would keep some redundancy (for example, each
> transaction INV is sent over 2 links).
>
> To make it robust to attacks, I have 2 extensions in my mind:
> 1. Set reconciliation (for a subset of transactions) with *other* nodes.
> Getting a bloom filter of a subset of the mempool transactions from Node_=
B
> may help to figure out whether Node_A is malicious, very slow, etc.
> 2. Rotating the filters every N minutes (N < 10)
>
> I can see some issues with latency here, but I believe this problem has a
> solution.
>
> Feedback is appreciated!
>
> If you want to look at a draft of the proposal =E2=80=94 please let me kn=
ow.
> If there were any similar ideas =E2=80=94 please let me know.
>
> Best,
> Gleb
>
>
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
>
>

--0000000000000a55f40568f54641
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr">Hey. This idea sounds quite interesting. It&#39;d be helpf=
ul to see some more numbers to evaluate it.<div><br></div><div>- How much b=
andwidth is consumed by redundant tx INVs currently? What is this as a % of=
 overall bandwidth usage?</div><div>- How would filtering txs through N=3D2=
 links affect network propagation? This probably requires simulation to det=
ermine.</div><div>- Do you propose setting filters on inbound peers as well=
?</div></div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On M=
on, Apr 2, 2018 at 3:18 PM, Gleb Naumenko via bitcoin-dev <span dir=3D"ltr"=
>&lt;<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org" target=3D"_bl=
ank">bitcoin-dev@lists.linuxfoundation.org</a>&gt;</span> wrote:<br><blockq=
uote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc =
solid;padding-left:1ex">



<div>
<div name=3D"messageBodySection" style=3D"font-size:14px;font-family:-apple=
-system,BlinkMacSystemFont,sans-serif">
<div>Hi all,</div>
<div>I have a couple of ideas regarding transaction relay protocol and want=
ed to share it with and probably get some feedback.</div>
<div><br></div>
<div>I did some emulation and simulation and found out that around 90% of I=
NV messages sent by public-IP nodes are idle (duplicate), obviously because=
 each node creates 8 connections.=C2=A0 I also realized that sending INV me=
ssages is a significant part of the overall bandwidth consumed by a public-=
IP node. At a larger scale, this will result in people not able to run a pu=
blic-IP node.</div>
<div><br></div>
<div>My idea is in some sense similar to BIP37 but applied to public-IP nod=
es. Here I want to emphasize that all the nodes will still receive *all* of=
 the transactions. A new protocol should also keep the same zero-trust, rob=
ustness, decentralization guarantees and latency.</div>
<div><br></div>
<div>Idea: while joining the network, a new node agrees on some filter with=
 each of 8 nodes it connects to. So that NewNode &lt;-&gt; Node_A will be u=
sed to relay only a subset of transactions, NewNode &lt;-&gt; Node_B for an=
other subset. This will significantly decrease the redundancy.</div>
<div><br></div>
<div>To keep the guarantees, I would keep some redundancy (for example, eac=
h transaction INV is sent over 2 links).</div>
<div><br></div>
<div>To make it robust to attacks, I have 2 extensions in my mind:=C2=A0</d=
iv>
<div>1. Set reconciliation (for a subset of transactions) with *other* node=
s. Getting a bloom filter of a subset of the mempool transactions from Node=
_B may help to figure out whether Node_A is malicious, very slow, etc.</div=
>
<div>2. Rotating the filters every N minutes (N &lt; 10)</div>
<div><br></div>
<div>I can see some issues with latency here, but I believe this problem ha=
s a solution.</div>
<div><br></div>
<div>Feedback is appreciated!</div>
<div><br></div>
<div>If you want to look at a draft of the proposal =E2=80=94 please let me=
 know.</div>
<div>If there were any similar ideas =E2=80=94 please let me know.</div>
<div><br></div>
<div>Best,</div>
<div>Gleb</div>
</div>
<div name=3D"messageReplySection" style=3D"font-size:14px;font-family:-appl=
e-system,BlinkMacSystemFont,sans-serif"><br>
<div></div>
</div>
</div>

<br>______________________________<wbr>_________________<br>
bitcoin-dev mailing list<br>
<a href=3D"mailto:bitcoin-dev@lists.linuxfoundation.org">bitcoin-dev@lists.=
<wbr>linuxfoundation.org</a><br>
<a href=3D"https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev" =
rel=3D"noreferrer" target=3D"_blank">https://lists.linuxfoundation.<wbr>org=
/mailman/listinfo/bitcoin-<wbr>dev</a><br>
<br></blockquote></div><br></div>

--0000000000000a55f40568f54641--