Received-SPF: pass (sog-mx-1.v43.ch3.sourceforge.com: domain of bitpay.com
	designates 209.85.214.169 as permitted sender)
	client-ip=209.85.214.169; envelope-from=jgarzik@bitpay.com;
	helo=mail-ob0-f169.google.com; 
MIME-Version: 1.0
In-Reply-To: <CALYO6Xs_20YpKeqmtu8N6Vt2uCSV4hM6S=6=zLhfBb_GCyuikg@mail.gmail.com>
References: <CALYO6Xt-jTYwpywUaH-s4YPYyGUp1_BLSEswscnwX+Vu166Lcw@mail.gmail.com>
	<alpine.DEB.2.10.1501281419110.21680@nzrgulfg.ivfhpber.pbz>
	<CALYO6Xv=k+Ztvke90SDB91StFBL7C0U49ufMD-WjG91uHLshFg@mail.gmail.com>
	<CANEZrP3PCHaTO3-HA3GHFxwuJJpW2dbvPuV4R1sFPcFW49uGgw@mail.gmail.com>
	<CAJHLa0Mu3Mjn=N-fTQ_fjwp+NUpfBqpdnXZiHoKz1s3tcZa+Cg@mail.gmail.com>
	<CALYO6Xs_20YpKeqmtu8N6Vt2uCSV4hM6S=6=zLhfBb_GCyuikg@mail.gmail.com>
From: Jeff Garzik <jgarzik@bitpay.com>
Date: Wed, 28 Jan 2015 12:29:53 -0500
Message-ID: <CAJHLa0MCyzm_t47R5Z5MPL9ruqM=uq15u26W3dwRsBy57K11=w@mail.gmail.com>
To: Nicolas DORIER <nicolas.dorier@gmail.com>
Content-Type: multipart/alternative; boundary=001a11331f505c8c39050db9b93f
Cc: Bitcoin Dev <bitcoin-development@lists.sourceforge.net>
Subject: Re: [Bitcoin-development] BIP70: why Google Protocol Buffers for
	encoding?
Precedence: list

--001a11331f505c8c39050db9b93f
Content-Type: text/plain; charset=UTF-8

It is not "fear", it is field experience.

JSON has proven to be a bug generator for the reasons already stated.

JSON does not include type marshalling and input validation.
Protobufs/msgpack/etc. engineered those to occur automatically, because
that is an area shown by field experience to be a constant source of bugs
and inconsistent parsing/validation behavior.




On Wed, Jan 28, 2015 at 11:52 AM, Nicolas DORIER <nicolas.dorier@gmail.com>
wrote:

> For the number of field there is in the spec, I don't consider having a
> JSON to schama really worthwhile.
> If you fear it is error prone, then we should provide some testing data
> for the BIP70. (Which I already did for protobuf, but was rejected, because
> deemed no useful thanks to the code generator... But such code generator
> gave me inconsistencies with gavin's implementation for example)
>
> Why do you think type support is very useful in our case ? we have 3
> types, and dealing only with bytes, int, and string.
> It cost me more time to find a suitable cross plateform lib for protobuf
> (in c#, that works in ios and winrt) than I would by just coding the json
> wrapper classes by hand. (JSON libs are more wildspread and supported than
> protobuf)
>
> 2015-01-28 17:04 GMT+01:00 Jeff Garzik <jgarzik@bitpay.com>:
>
>> Not to mention the tiresome and error-prone task of writing your own
>> JSON-to-schema marshalling code -- or something equivalent to the protobufs
>> compiler and libs for JSON.
>>
>> protobufs -- and its modern competitors such as msgpack -- natively
>> provide type support in a way that must be hacked into JSON or XML.
>>
>> The protobuf/msgpack design is engineered to avoid bugs routinely found
>> in JSON parsing code; due to the amount of code & effort involved in JSON
>> input sanity checking, bugs and inconsistencies inevitable arise.  We have
>> seen this in bitcoind with JSON-RPC.
>>
>>
>>
>> On Wed, Jan 28, 2015 at 10:42 AM, Mike Hearn <mike@plan99.net> wrote:
>>
>>> On the other hand, if you charge the developer (and not the plateform)
>>>> to check certificate validity, it means that you have to develop a
>>>> different codebase for all plateform you are targeting, because each
>>>> plateform store trusted root certificate in a different manner with
>>>> different APIs, and also have different types representing a X509
>>>> Certificate.
>>>>
>>>
>>> That's what cross-platform abstraction libraries are for. Both Java and
>>> Qt provide a key store library that can load from either the OS root store
>>> or a custom one. If your chosen app platform doesn't, OK, then you'll have
>>> to make or find one yourself. Perhaps contribute it upstream or make it a
>>> library. But that's not a limitation of BIP70.
>>>
>>> Just as a reminder, there is no obligation to use the OS root store. You
>>> can (and quite possibly should) take a snapshot of the Mozilla/Apple/MSFT
>>> etc stores and load it in your app. We do this in bitcoinj by default to
>>> avoid cases where BIP70 requests work on some platforms and not others,
>>> although the developer can easily override this and use the OS root store
>>> instead.
>>>
>>> Of all possible solutions, using a third party service to convert things
>>> to JSON is one of the least obvious and highest effort. I don't know anyone
>>> else who arrived at such a conclusion and respectfully disagree that this
>>> is a problem with the design choices in BIP70. It sounds like a bizarre
>>> hack around lack of features in whatever runtime you're using.
>>>
>>>
>>>
>>> ------------------------------------------------------------------------------
>>> Dive into the World of Parallel Programming. The Go Parallel Website,
>>> sponsored by Intel and developed in partnership with Slashdot Media, is
>>> your
>>> hub for all things parallel software development, from weekly thought
>>> leadership blogs to news, videos, case studies, tutorials and more. Take
>>> a
>>> look and join the conversation now. http://goparallel.sourceforge.net/
>>> _______________________________________________
>>> Bitcoin-development mailing list
>>> Bitcoin-development@lists.sourceforge.net
>>> https://lists.sourceforge.net/lists/listinfo/bitcoin-development
>>>
>>>
>>
>>
>> --
>> Jeff Garzik
>> Bitcoin core developer and open source evangelist
>> BitPay, Inc.      https://bitpay.com/
>>
>
>


-- 
Jeff Garzik
Bitcoin core developer and open source evangelist
BitPay, Inc.      https://bitpay.com/

--001a11331f505c8c39050db9b93f
Content-Type: text/html; charset=UTF-8
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div><div>It is not &quot;fear&quot;, it is field experien=
ce.<br><br></div>JSON has proven to be a bug generator for the reasons alre=
ady stated.<br><br></div>JSON does not include type marshalling and input v=
alidation.=C2=A0 Protobufs/msgpack/etc. engineered those to occur automatic=
ally, because that is an area shown by field experience to be a constant so=
urce of bugs and inconsistent parsing/validation behavior.<br><br><br><br><=
/div><div class=3D"gmail_extra"><br><div class=3D"gmail_quote">On Wed, Jan =
28, 2015 at 11:52 AM, Nicolas DORIER <span dir=3D"ltr">&lt;<a href=3D"mailt=
o:nicolas.dorier@gmail.com" target=3D"_blank">nicolas.dorier@gmail.com</a>&=
gt;</span> wrote:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 =
0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div>F=
or the number of field there is in the spec, I don&#39;t consider having a =
JSON to schama really worthwhile.<br></div>If you fear it is error prone, t=
hen we should provide some testing data for the BIP70. (Which I already did=
 for protobuf, but was rejected, because deemed no useful thanks to the cod=
e generator... But such code generator gave me inconsistencies with gavin&#=
39;s implementation for example)<br><div><div><div><br></div><div>Why do yo=
u think type support is very useful in our case ? we have 3 types, and deal=
ing only with bytes, int, and string.<br></div><div>It cost me more time to=
 find a suitable cross plateform lib for protobuf (in c#, that works in ios=
 and winrt) than I would by just coding the json wrapper classes by hand. (=
JSON libs are more wildspread and supported than protobuf)<br></div><div><d=
iv class=3D"h5"><div><div><div class=3D"gmail_extra"><br><div class=3D"gmai=
l_quote">2015-01-28 17:04 GMT+01:00 Jeff Garzik <span dir=3D"ltr">&lt;<a hr=
ef=3D"mailto:jgarzik@bitpay.com" target=3D"_blank">jgarzik@bitpay.com</a>&g=
t;</span>:<br><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;=
border-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div><div>Not=
 to mention the tiresome and error-prone task of writing your own JSON-to-s=
chema marshalling code -- or something equivalent to the protobufs compiler=
 and libs for JSON.<br><br></div>protobufs -- and its modern competitors su=
ch as msgpack -- natively provide type support in a way that must be hacked=
 into JSON or XML.<br><br></div>The protobuf/msgpack design is engineered t=
o avoid bugs routinely found in JSON parsing code; due to the amount of cod=
e &amp; effort involved in JSON input sanity checking, bugs and inconsisten=
cies inevitable arise.=C2=A0 We have seen this in bitcoind with JSON-RPC.<b=
r><br><div><br></div></div><div class=3D"gmail_extra"><br><div class=3D"gma=
il_quote"><div><div>On Wed, Jan 28, 2015 at 10:42 AM, Mike Hearn <span dir=
=3D"ltr">&lt;<a href=3D"mailto:mike@plan99.net" target=3D"_blank">mike@plan=
99.net</a>&gt;</span> wrote:<br></div></div><blockquote class=3D"gmail_quot=
e" style=3D"margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">=
<div><div><div dir=3D"ltr"><div class=3D"gmail_extra"><div class=3D"gmail_q=
uote"><span><blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;bo=
rder-left:1px #ccc solid;padding-left:1ex"><div dir=3D"ltr"><div><div>On th=
e other hand, if you charge the
 developer (and not the plateform) to check certificate validity, it=20
means that you have to develop a different codebase for all plateform=20
you are targeting, because each plateform store trusted root certificate
 in a different manner with different APIs, and also have different=20
types representing a X509 Certificate.<br></div></div></div></blockquote><d=
iv><br></div></span><div>That&#39;s what cross-platform abstraction librari=
es are for. Both Java and Qt provide a key store library that can load from=
 either the OS root store or a custom one. If your chosen app platform does=
n&#39;t, OK, then you&#39;ll have to make or find one yourself. Perhaps con=
tribute it upstream or make it a library. But that&#39;s not a limitation o=
f BIP70.</div><div><br></div><div>Just as a reminder, there is no obligatio=
n to use the OS root store. You can (and quite possibly should) take a snap=
shot of the Mozilla/Apple/MSFT etc stores and load it in your app. We do th=
is in bitcoinj by default to avoid cases where BIP70 requests work on some =
platforms and not others, although the developer can easily override this a=
nd use the OS root store instead.</div><div>=C2=A0</div><div>Of all possibl=
e solutions, using a third party service to convert things to JSON is one o=
f the least obvious and highest effort. I don&#39;t know anyone else who ar=
rived at such a conclusion and respectfully disagree that this is a problem=
 with the design choices in BIP70. It sounds like a bizarre hack around lac=
k of features in whatever runtime you&#39;re using.</div><div><br></div></d=
iv></div></div>
<br></div></div>-----------------------------------------------------------=
-------------------<br>
Dive into the World of Parallel Programming. The Go Parallel Website,<br>
sponsored by Intel and developed in partnership with Slashdot Media, is you=
r<br>
hub for all things parallel software development, from weekly thought<br>
leadership blogs to news, videos, case studies, tutorials and more. Take a<=
br>
look and join the conversation now. <a href=3D"http://goparallel.sourceforg=
e.net/" target=3D"_blank">http://goparallel.sourceforge.net/</a><br>_______=
________________________________________<br>
Bitcoin-development mailing list<br>
<a href=3D"mailto:Bitcoin-development@lists.sourceforge.net" target=3D"_bla=
nk">Bitcoin-development@lists.sourceforge.net</a><br>
<a href=3D"https://lists.sourceforge.net/lists/listinfo/bitcoin-development=
" target=3D"_blank">https://lists.sourceforge.net/lists/listinfo/bitcoin-de=
velopment</a><br>
<br></blockquote></div><span><font color=3D"#888888"><br><br clear=3D"all">=
<br>-- <br><div>Jeff Garzik<br>Bitcoin core developer and open source evang=
elist<br>BitPay, Inc. =C2=A0 =C2=A0 =C2=A0<a href=3D"https://bitpay.com/" t=
arget=3D"_blank">https://bitpay.com/</a></div>
</font></span></div>
</blockquote></div><br></div></div></div></div></div></div></div></div>
</blockquote></div><br><br clear=3D"all"><br>-- <br><div class=3D"gmail_sig=
nature">Jeff Garzik<br>Bitcoin core developer and open source evangelist<br=
>BitPay, Inc. =C2=A0 =C2=A0 =C2=A0<a href=3D"https://bitpay.com/" target=3D=
"_blank">https://bitpay.com/</a></div>
</div>

--001a11331f505c8c39050db9b93f--