Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192] helo=mx.sourceforge.net) by sfs-ml-2.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1UpYEh-0001Z3-Hs for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 06:20:47 +0000 X-ACL-Warn: Received: from nm37-vm0.bullet.mail.bf1.yahoo.com ([72.30.238.200]) by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:AES256-SHA:256) (Exim 4.76) id 1UpYEd-0005mb-Jp for bitcoin-development@lists.sourceforge.net; Thu, 20 Jun 2013 06:20:47 +0000 Received: from [98.139.212.150] by nm37.bullet.mail.bf1.yahoo.com with NNFMP; 20 Jun 2013 06:20:38 -0000 Received: from [98.139.212.224] by tm7.bullet.mail.bf1.yahoo.com with NNFMP; 20 Jun 2013 06:20:38 -0000 Received: from [127.0.0.1] by omp1033.mail.bf1.yahoo.com with NNFMP; 20 Jun 2013 06:20:38 -0000 X-Yahoo-Newman-Property: ymail-3 X-Yahoo-Newman-Id: 20455.959.bm@omp1033.mail.bf1.yahoo.com Received: (qmail 57913 invoked by uid 60001); 20 Jun 2013 06:20:37 -0000 X-YMail-OSG: ejngBSQVM1lG5._ms2oyJ0MQZ0t7pTfAQQrS1N3BvcGutfc KtCKJHaYjWMw1BoBjRTUFyR.d_BvYjo8JIlQFje.b4mTB37a8BsBDDOqfNV2 hXxlaDDsrxhOkHaJkevbP42n_tdH0k_n8j6DSd_l3tsWTL3HHc1NYF9OtSew tPlOPJptDtKlGsF7gy2Aw04orIYvKUkBE6moPfc7YY_1PFqu..VQWNA_h1No ZVAenCB7asMbY2LXVPGLxPPo7zYMY55k5WELNSXlQOoi7XudSOnpN9iFgqhg dtEGoSwX6EF2lujnZjEF.J4dNrOdo0eW911BL53hLBG88PzRg1dA2NzDbk3n LItU5wiLgivrVrWDO9e2yY0R_vCgWyM.URKKaA3DQMaCJppxPJ6qirqLz3IB q2H6wVNjRrehnYcajvO9dOj6GMx.U7ZulYBcNTTj3rHx43yTczONpx8EcoHc X_C7JdK6oeB4heOBtNaqkPf7Wxl4eQU_uk93JIMPDeLkaljBl3TZm3T98D92 QeNZ_l1PvYsM0JDCdQYhwwZmOZoz9X9.LvFVa2D99DmQsI4EKX4e9mg9A5uE 6S2BkMf7Nt2ohq1SwXIKgmHFxD_67k1K_nczipUBUgaX_WVH92DrHXYBM9q8 d Received: from [87.160.177.196] by web162701.mail.bf1.yahoo.com via HTTP; Wed, 19 Jun 2013 23:20:37 PDT X-Rocket-MIMEInfo: 002.001, SSBuZXZlciBzYWlkIHRoYXQgQml0Y29pbiBtZXNzYWdlIGZpZWxkIGxlbmd0aHMgc2hvdWxkIGFsd2F5cyBiZSB0aGUgc2FtZS4gQnV0IGJlZm9yZSB0aGlzIGNoYW5nZSB0aGV5IGNlcnRhaW5seSB3ZXJlIGNvbnN0YW50IHBlciBwcm90b2NvbCB2ZXJzaW9uLiBBbGwgSSdtIHNheWluZyBpcyB0aGF0IG9wdGlvbmFsIGxlbmd0aHMgc2hvdWxkbid0IGJlIHVzZWQgKGEgZmllbGQgZXhpc3RzIG9yIG5vdCkgYW5kIGZvciBldmVyeSBmaWVsZCBjaGFuZ2UsIHRoZSBwcm90b2NvbCB2ZXJzaW9uIHNob3VsZCBiZSABMAEBAQE- X-Mailer: YahooMailWebService/0.8.147.553 References: Message-ID: <1371709237.57104.YahooMailNeo@web162701.mail.bf1.yahoo.com> Date: Wed, 19 Jun 2013 23:20:37 -0700 (PDT) From: Turkey Breast To: "bitcoin-development@lists.sourceforge.net" In-Reply-To: MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="-350752386-1724672238-1371709237=:57104" X-Spam-Score: -0.1 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (turkeybreast[at]yahoo.com) -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [72.30.238.200 listed in list.dnswl.org] -1.3 RP_MATCHES_RCVD Envelope sender domain matches handover relay domain 1.0 HTML_MESSAGE BODY: HTML included in message 0.3 HTML_FONT_FACE_BAD BODY: HTML font face is not a word -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1UpYEd-0005mb-Jp Subject: Re: [Bitcoin-development] Missing fRelayTxes in version message X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list Reply-To: Turkey Breast List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 20 Jun 2013 06:20:47 -0000 ---350752386-1724672238-1371709237=:57104 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable I never said that Bitcoin message field lengths should always be the same. = But before this change they certainly were constant per protocol version. A= ll I'm saying is that optional lengths shouldn't be used (a field exists or= not) and for every field change, the protocol version should be upgraded.= =0A=0ANow that fRelayTxes is part of the protocol, the version number shoul= d be upgraded to reflect this fact.=0A=0A=0A=0A____________________________= ____=0A From: Mike Hearn =0ATo: Paul Lyon =0ACc: Turkey Breast ; "bitcoin-development@lis= ts.sourceforge.net" =0ASent: We= dnesday, June 19, 2013 3:20 PM=0ASubject: Re: [Bitcoin-development] Missing= fRelayTxes in version message=0A =0A=0A=0AIf you want to criticise the Bit= coin protocol for sloppyness, the variable length of some messages isn't wh= ere I'd start.=0A=0ANote that ping has the same issue, its length has chang= ed over time to include the nonce.=0A=0AIf your parser can't handle that ki= nd of thing, you need to fix it. The protocol has always worked that way.= =0A=0A=0A=0A=0AOn Wed, Jun 19, 2013 at 3:03 PM, Paul Lyon wrote:=0A=0AI=E2=80=99m also running into this exact same issue with my = parser, now I understand why the relay field behavior I was seeing=C2=A0doe= sn=E2=80=99t match the wiki.=0A>=C2=A0=0A>So to parse a version message, yo= u can=E2=80=99t rely on the protocol version? You have to know how long the= payload is, and then parse the message accordingly? I agree with Turkey Br= east, this seems a bit sloppy to me.=0A>=C2=A0=0A>Paul=0A>=C2=A0=0A>P.S. I= =E2=80=99ve never used a dev mailing list before and I want to get involved= with the Bitcoin dev community, so let me know if I=E2=80=99m horribly vio= lating any=C2=A0mailing list etiquette. =F0=9F=98=8A=0A>=C2=A0=0A>From:=C2= =A0Mike Hearn=0A>Sent:=C2=A0=E2=80=8EWednesday=E2=80=8E, =E2=80=8EJune=E2= =80=8E =E2=80=8E19=E2=80=8E, =E2=80=8E2013 =E2=80=8E7=E2=80=8E:=E2=80=8E43= =E2=80=8E =E2=80=8EAM=0A>To:=C2=A0Turkey Breast=0A>Cc:=C2=A0bitcoin-develop= ment@lists.sourceforge.net=0A>=C2=A0=0A>Bitcoin-Qt on master does send it n= ow although it doesn't affect anything, but as old pre-filtering versions w= ill continue to exist, you'll always have to be able to deserialize version= messages without it.=0A>=0A>=0A>Bitcoin version messages have always had v= ariable length, look at how the code is written in main.cpp. If you didn't = experience issues until now all it means is that no sufficiently old nodes = were talking to yours.=0A>=0A>=0A>The standard does not say it should appea= r. Read it again - BIP 37 says about the new version message field:=0A>If f= alse then broadcast transactions will not be announced until a filter{load,= add,clear} command is received. If missing or true, no change in protocol b= ehaviour occurs.=0A> =0A>=0A>=0A>=0A>On Wed, Jun 19, 2013 at 12:33 PM, Turk= ey Breast wrote:=0A>=0A>It's a problem if you work= with iterators to deserialize the byte stream. Even failing that, it's jus= t sloppy programming. What happens in the future when new fields are added = to the version message? It's not a big deal to say that this protocol versi= on has X number of fields, that (higher) protocol version message has X + N= number of fields. Deterministic number of fields per protocol version is s= ensical and how Bitcoin has been for a long time.=0A>>=0A>>=0A>>And yes, it= was a problem for me that caused a lot of confusion why this byte didn't e= xist in many version messages despite the standard saying it should and the= code in bitcoind indicating it should. Nowhere was this written. It doesn'= t help other implementations to have an unclear behaviour that depends on s= ome magic from one implementation.=0A>>=0A>>=0A>>=0A>>=0A>>________________= ________________=0A>> From: Mike Hearn =0A>>To: Turkey Bre= ast =0A>>Cc: "bitcoin-development@lists.sourceforg= e.net" =0A>>Sent: Wednesday, Ju= ne 19, 2013 11:39 AM=0A>>=0A>>Subject: Re: [Bitcoin-development] Missing fR= elayTxes in version message=0A>> =0A>>=0A>>=0A>>It has to be optional becau= se old clients don't send it, obviously.=0A>>=0A>>=0A>>Why is this even an = issue? There's no problem with variable length messages in any codebase tha= t I'm aware of. Is this solving some actual problem?=0A>>=0A>>=0A>>=0A>>On = Wed, Jun 19, 2013 at 12:30 AM, Turkey Breast wrote= :=0A>>=0A>>That's me. I never said to make all messages fixed length. I sai= d to make a fixed number of fields per protocol. So given a protocol versio= n number, you know the number of fields in a message. This is not only easi= er for parsing messages, but just good practice. I don't see why a 1 byte f= lag needs to be optional anyway.=0A>>>=0A>>>=0A>>>=0A>>>=0A>>>_____________= ___________________=0A>>> From: Mike Hearn =0A>>>To: Turke= y Breast =0A>>>Cc: Bitcoin Dev =0A>>>Sent: Tuesday, June 18, 2013 9:48 PM=0A>>>Su= bject: Re: [Bitcoin-development] Missing fRelayTxes in version message=0A>>= > =0A>>>=0A>>>=0A>>>It's not a bug (although there was recently a change to= make bitcoind/qt always send this field anyway).=C2=A0=0A>>>=0A>>>=0A>>>I = don't know where Amir is going with BIP 60. Version messages have always be= en variable length. There's nothing inherent in the Bitcoin protocol that s= ays all messages are fixed length, indeed, tx messages are allowed to have = arbitrary data appended after them that gets relayed.=0A>>>=0A>>>=0A>>>=0A>= >>On Tue, Jun 18, 2013 at 7:45 PM, Turkey Breast w= rote:=0A>>>=0A>>>See this BIP. I'm not sure if this is a bug or what, but i= t would be good if messages always had a fixed number of fields per protoco= l version.=0A>>>>=0A>>>>=0A>>>>https://en.bitcoin.it/wiki/BIP_0060#Code_Upd= ates=0A>>>>=0A>>>>=0A>>>>This BIP details everything that needs to be done = and proposes a protocol upgrade.=0A>>>>=0A>>>>-----------------------------= -------------------------------------------------=0A>>>>This SF.net email i= s sponsored by Windows:=0A>>>>=0A>>>>Build for Windows Store.=0A>>>>=0A>>>>= http://p.sf.net/sfu/windows-dev2dev=0A>>>>_________________________________= ______________=0A>>>>Bitcoin-development mailing list=0A>>>>Bitcoin-develop= ment@lists.sourceforge.net=0A>>>>https://lists.sourceforge.net/lists/listin= fo/bitcoin-development=0A>>>>=0A>>>>=0A>>>=0A>>>=0A>>>=0A>>>---------------= ---------------------------------------------------------------=0A>>>This S= F.net email is sponsored by Windows:=0A>>>=0A>>>Build for Windows Store.=0A= >>>=0A>>>http://p.sf.net/sfu/windows-dev2dev=0A>>>_________________________= ______________________=0A>>>Bitcoin-development mailing list=0A>>>Bitcoin-d= evelopment@lists.sourceforge.net=0A>>>https://lists.sourceforge.net/lists/l= istinfo/bitcoin-development=0A>>>=0A>>>=0A>>=0A>>=0A>>=0A>>----------------= --------------------------------------------------------------=0A>>This SF.= net email is sponsored by Windows:=0A>>=0A>>Build for Windows Store.=0A>>= =0A>>http://p.sf.net/sfu/windows-dev2dev=0A>>______________________________= _________________=0A>>Bitcoin-development mailing list=0A>>Bitcoin-developm= ent@lists.sourceforge.net=0A>>https://lists.sourceforge.net/lists/listinfo/= bitcoin-development=0A>>=0A>>=0A> ---350752386-1724672238-1371709237=:57104 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
I never said that Bit= coin message field lengths should always be the same. But before this chang= e they certainly were constant per protocol version. All I'm saying is that= optional lengths shouldn't be used (a field exists or not) and for every f= ield change, the protocol version should be upgraded.

Now that fRela= yTxes is part of the protocol, the version number should be upgraded to ref= lect this fact.

From: Mike Hearn <mike@plan99.net>
<= span style=3D"font-weight: bold;">To: Paul Lyon <pmlyon@hotma= il.ca>
Cc: Turkey Breast <= ;turkeybreast@yahoo.com>; "bitcoin-development@lists.sourceforge.net" &l= t;bitcoin-development@lists.sourceforge.net>
Sent: Wednesday, June 19, 2013 3:20 PM
Subject: Re: [Bitcoin-developme= nt] Missing fRelayTxes in version message

If you want= to criticise the Bitcoin protocol for sloppyness, the variable length of s= ome messages isn't where I'd start.

Note that ping has t= he same issue, its length has changed over time to include the nonce.
= =0A

If your parser can't handle that kind of = thing, you need to fix it. The protocol has always worked that way.


<= br>
=0AOn Wed, Jun 19, 2013 at 3:03 = PM, Paul Lyon <pmlyo= n@hotmail.ca> wrote:
=0A
=0A
I=E2=80=99m also running into this exact same issue with my pa= rser, now I understand why the relay field behavior I was seeing doesn= =E2=80=99t match the wiki.
 
So to parse a version= message, you can=E2=80=99t rely on the protocol version? You have to know = how long the payload is, and then parse the message accordingly? I agree wi= th Turkey Breast, this seems a bit sloppy to me.
=0A
 
<= div>Paul
 
P.S. I=E2=80=99ve never used a dev mail= ing list before and I want to get involved with the Bitcoin dev community, = so let me know if I=E2=80=99m horribly violating any mailing list etiq= uette. =F0=9F=98=8A
=0A
 
From: Mike Hearn
=0ASent: =E2=80=8EWednesday=E2=80=8E, =E2= =80=8EJune=E2=80=8E =E2=80=8E19=E2=80=8E, =E2=80=8E2013 =E2=80=8E7=E2=80=8E= :=E2=80=8E43=E2=80=8E =E2=80=8EAM
To: Turkey Breast
Cc= : bitcoin-development@lists.sourceforge.net=0A
 
Bitcoin-Qt on master does send it now although it doesn't affect anyth= ing, but as old pre-filtering versions will continue to exist, you'll alway= s have to be able to deserialize version messages without it.
=0A=0A
Bitcoin version messages have always had variable length, look = at how the code is written in main.cpp. If you didn't experience issues unt= il now all it means is that no sufficiently old nodes were talking to yours= .
=0A=0A

The standard does not say it should appear= . Read it again - BIP 37 says about the new version message field:
=0A=0A
If false then broadcast transacti= ons will not be announced until a filter{load,add,clear} command is receive= d. If missing or true, no change in protocol behaviour occurs.
= =0A=0A


On Wed, Jun 19, 20= 13 at 12:33 PM, Turkey Breast <turkeybreast@y= ahoo.com> wrote:
=0A=0A
=0A
It's a problem if you work with iterators to deseria= lize the byte stream. Even failing that, it's just sloppy programming. What= happens in the future when new fields are added to the version message? It= 's not a big deal to say that this protocol version has X number of fields,= that (higher) protocol version message has X + N number of fields. Determi= nistic number of fields per protocol version is sensical and how Bitcoin ha= s been for a long time.
=0A=0A

=0A=0AAnd yes, it was a problem for me=0A th= at caused a lot of confusion why this byte didn't exist in many version mes= sages despite the standard saying it should and the code in bitcoind indica= ting it should. Nowhere was this written. It doesn't help other implementat= ions to have an unclear behaviour that depends on some magic from one imple= mentation.
=0A=0A

=0A=0A From: Mike Hearn <mike@plan99.net>
= To: Turkey Breast <turkeybreast@yahoo.com>
=0A=0A
Cc: "bitcoin-development@lists.sourceforge.net" <bitcoin-develo= pment@lists.sourceforge.net>
=0A=0A Sent: Wednesday, June 19, 2013 11:39 AM

<= b>Subject: Re: [Bitcoin-develo= pment] Missing fRelayTxes in version message
=0A=0A
=

=0A
It has to be optional be= cause old clients don't send it, obviously.

Why is this = even an issue? There's no problem with variable length messages in any code= base that I'm aware of. Is this solving some actual problem?
=0A=0A=0A=


On Wed, Jun 19, 2013 at 12:30 AM, Turkey Breast <turkeybreast@yahoo.com> wrote:=0A=0A
=0A
That's me. I never said to make all messages = fixed length. I said to make a fixed number of fields per protocol. So give= n a protocol version number, you know the number of fields in a message. Th= is is not only easier for parsing messages, but just good practice. I don't= see why a 1 byte flag needs to be optional anyway.
=0A=0A=0A

=0A
From: Mike Hearn <mike@plan99.net>
=0A To: Turkey Breast <turkeybre= ast@yahoo.com>
=0A=0ACc:=0A Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
<= b>Sent: Tuesday, June 18, 2013= 9:48 PM
=0A=0A=0A Subject:<= /b> Re: [Bitcoin-development] Missing fRelayTxes in version message

=0A
It's not a bu= g (although there was recently a change to make bitcoind/qt always send thi= s field anyway). 

I don't know where Amir is going = with BIP 60. Version messages have always been variable length. There's not= hing inherent in the Bitcoin protocol that says all messages are fixed leng= th, indeed, tx messages are allowed to have arbitrary data appended after t= hem that gets relayed.
=0A=0A=0A=0A


On Tue, Jun= 18, 2013 at 7:45 PM, Turkey Breast <turkeybr= east@yahoo.com> wrote:
=0A=0A
=0A
See this BIP= . I'm not sure if this is a bug or what, but it would be good if messages a= lways had a fixed number of fields per protocol version.
=0A=0A=0A
=0A=0A=0A=0A

=0A=0A=0A=0AThis BIP details everything= that needs to be done and proposes a protocol upgrade.

---------------------------------------------------------------------= ---------
=0AThis SF.net email is sponsored by Windows:<= br>=0A
=0ABuild for Windows Store.
=0A
=0Ahttp://p.sf.net/sfu/windows-dev2dev
__= _____________________________________________
=0ABitcoin-development mai= ling list
=0ABitcoin-development@lists.sourceforge.net
=0Ahttps://lists.sourceforge.net/lists/listinfo/bitcoin-d= evelopment
=0A=0A

=0A


<= /div>

--------------------------= ----------------------------------------------------
=0AThis SF.net emai= l is sponsored by Windows:
=0A
=0ABuild for Windows Store.
=0A
= =0Ahttp://p.sf.net/sf= u/windows-dev2dev
_______________________________________________=0ABitcoin-development mailing list
=0ABitcoin-development@lists.sourceforge.net=
=0Ahttps://lists.sourceforge.n= et/lists/listinfo/bitcoin-development
=0A=0A
<= br>
=0A


--------------------------------------------------------------------------= ----
=0AThis SF.net email is sponsored by Windows:
=0A
=0ABuild fo= r Windows Store.
=0A
=0Ahttp://p.sf.net/sfu/windows-dev2dev
_______________________= ________________________
=0ABitcoin-development mailing list
=0ABitcoin-develo= pment@lists.sourceforge.net
=0A= https://lists.sourceforge.net/lists/listinfo/bitcoin-development
=0A= =0A

=0A



---350752386-1724672238-1371709237=:57104--