Received: from sog-mx-4.v43.ch3.sourceforge.com ([172.29.43.194] helo=mx.sourceforge.net) by sfs-ml-1.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1SgR7r-0001YC-3r for bitcoin-development@lists.sourceforge.net; Mon, 18 Jun 2012 01:51:31 +0000 Received: from mail-ob0-f175.google.com ([209.85.214.175]) by sog-mx-4.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1SgR7p-0007MC-LN for bitcoin-development@lists.sourceforge.net; Mon, 18 Jun 2012 01:51:31 +0000 Received: by obhx4 with SMTP id x4so8739540obh.34 for ; Sun, 17 Jun 2012 18:51:24 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:x-originating-ip:in-reply-to:references:date :message-id:subject:from:to:content-type:x-gm-message-state; bh=fFt3UvlWRxnARdHWgOuOYR+Ja6gAn2e7KhTBSRMg2eM=; b=R+4GKeUHuCMy/NLUYHRgmUf9+IUfH5hXtEQCIjvHZr/WHAdKc9mv8pUlEraC5SfbOP rupBcc6pwi0Kt0rturt1KiifqUm8Z4ISe+NaWeWLH6n+jyDD8gXiGehSdPdaPL2v9Zfw p8lyUtdAupUaRb+SNISYlN23VMbOwe4y82TpYpI6KB+yr/MRcUy5zinzpCduP1Pt4cxu ePEokMYtV5hVqFKLFSTdzJDh0m0FrDNf9tWHFhGZio/AmT/jLwqWW8svemNgsPAsVhiE GhkKlCqN2TiiVrUsBND3jXEo3CBzVpAcBIIFZy6jLkW89VAURynqhfX8nSmvbL5dfBxb 8uXg== MIME-Version: 1.0 Received: by 10.60.28.162 with SMTP id c2mr13863433oeh.3.1339982859697; Sun, 17 Jun 2012 18:27:39 -0700 (PDT) Received: by 10.76.101.15 with HTTP; Sun, 17 Jun 2012 18:27:39 -0700 (PDT) X-Originating-IP: [50.0.36.54] In-Reply-To: <1339950641.22050.YahooMailNeo@web121005.mail.ne1.yahoo.com> References: <1339810493.15660.YahooMailNeo@web121004.mail.ne1.yahoo.com> <201206160916.24485.andyparkins@gmail.com> <1339950641.22050.YahooMailNeo@web121005.mail.ne1.yahoo.com> Date: Sun, 17 Jun 2012 18:27:39 -0700 Message-ID: From: Mark Friedenbach To: "bitcoin-development@lists.sourceforge.net" Content-Type: multipart/alternative; boundary=e89a8ff1c7625b7f7904c2b51217 X-Gm-Message-State: ALoCoQmRW/FaEzYCBeywlPv5tqKzoga+hAOjWqkY1eZB6irGmQLI4p2BuPKATdghYKJth2DtVGRS X-Spam-Score: 1.0 (+) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. 1.0 HTML_MESSAGE BODY: HTML included in message X-Headers-End: 1SgR7p-0007MC-LN Subject: Re: [Bitcoin-development] Proposed new P2P command and response: getcmds, cmdlist X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Jun 2012 01:51:31 -0000 --e89a8ff1c7625b7f7904c2b51217 Content-Type: text/plain; charset=UTF-8 Sorry for the duplication Amir, I meant to send this to everyone: BitTorrent might be an example to look to here. It's a peer-to-peer network that has undergone many significant protocol upgrades over the years while maintaining compatibility. More recent clients have had the ability to expose the capabilities of connected peers and modify behavior accordingly, and overall it has worked very well. Capability-based systems do work, and provide an excellent means of trying out new algorithms, adding new features for upgraded clients, and when necessary reverting protocol changes (by depreciating or removing extensions). The problem with OpenGL was and continues to be that the two superpowers of that industry develop and maintain competing proposals for similar functionality, which are thrust upon developers which must support both if they want access to the latest and greatest features, until such time that the ARB arbitrarily choses one to standardize upon (in the process creating yet another extension of the form ARB_* that may be different and must be explicitly supported by developers). I think the BitTorrent example shows that a loosely organized, open-source community *can* maintain a capability-based extension system without falling into capability-hell. Mark On Sun, Jun 17, 2012 at 9:30 AM, Amir Taaki wrote: > As the only person to have created and maintaining a full reimplementation > of the Bitcoin protocol/standard, I do think Bitcoin is filled with > arbitrary endianness and magic numbers. However it is a tiny and simple > protocol. > > The big problem is not implementing the Bitcoin protocol, but the fact > that once you have created a codebase, you want to perfect and fine-tune > the design. During the meantime, the Bitcoin protocol is being changed. > Change to the Bitcoin protocol is far more damaging to people that want to > implement the protocol than any issues with the current protocol. > > That's not to say, I disagree with changes to the protocol. I think > changes should be a lot more conservative and have a longer time frame than > they do currently. Usually changes suddenly get added to the Satoshi client > and I notice them in the commit log or announcements. Then it's like "oh I > have to add this" and I spend a week working to implement the change > without proper consideration or reflection which ends up with me having to > compromise on design choices. That is to remain compatible with the > protocol. > > However it is not my intent to slow down progress so I usually try to > hedge against that kind of feeling towards conservatism. > > > > ----- Original Message ----- > From: Jeff Garzik > To: Wladimir > Cc: bitcoin-development@lists.sourceforge.net > Sent: Sunday, June 17, 2012 5:19 PM > Subject: Re: [Bitcoin-development] Proposed new P2P command and response: > getcmds, cmdlist > > On Sat, Jun 16, 2012 at 4:42 AM, Wladimir wrote: > > Which is a perfectly reasonable requirement. However, one could simply > > standardize what a 'thin client' and what a 'thick client' does and > offers > > (at a certain version level), without having to explicitly enumerate > > everything over the protocol. > > > > This also makes it easier to deprecate (lack of) certain features later > on. > > You can simply drop support for protocol versions before a certain number > > (which has happened before). With the extension system this is much > harder, > > which likely means you keep certain workarounds forever. > > > > Letting the node know of each others capabilities at connection time > helps > > somewhat. It'd allow refusing clients that do not implement a certain > > feature. Then again, to me it's unclear what this wins compared to > > incremental protocol versions with clear requirements. > > > > I'm just afraid that the currently simple P2P protocol will turn into a > zoo > > of complicated (and potentially buggy/insecure) interactions. > > What is missing here is some perspective on the current situation. It > is -very- easy to make a protocol change and bump PROTOCOL_VERSION in > the Satoshi client. > > But for anyone maintaining a non-Satoshi codebase, the P2P protocol is > already filled with all sorts of magic numbers, arbitrarily versioned > binary data structures.. already an unfriendly zoo of complicated and > potentially buggy interactions. There is scant, incomplete > documentation on the wiki -- the Satoshi source code is really the > only true reference. > > I see these problems personally, trying to keep ArtForz' half-a-node > running on mainnet (distributed as 'blkmond' with pushpool). > > In an era of HTTP and JSON, NFS and iSCSI, bitcoin's P2P protocol is > woefully backwards, fragile, limited and inflexible when it comes to > parameter/extension exchange and negotiation. Even iSCSI, that which > is implemented on hard drive firmware, has the ability to exchange > key=value parameters between local and remote sides of the RPC > connection. > > Calling the current P2P protocol "simple" belies all the > implementation details you absolutely -must- get right, to run on > mainnet today. Satoshi client devs almost never see the fragility and > complexity inherent in the current legacy codebase, built up over > time. > > -- > Jeff Garzik > exMULTI, Inc. > jgarzik@exmulti.com > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > > > > ------------------------------------------------------------------------------ > Live Security Virtual Conference > Exclusive live event will cover all the ways today's security and > threat landscape has changed and how IT managers can respond. Discussions > will include endpoint security, mobile security and the latest in malware > threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ > _______________________________________________ > Bitcoin-development mailing list > Bitcoin-development@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/bitcoin-development > --e89a8ff1c7625b7f7904c2b51217 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Sorry for the dupl= ication Amir, I meant to send this to everyone:

= BitTorrent might be an example to look to here. It's a peer-to-peer net= work that has undergone many significant protocol upgrades over the years w= hile maintaining compatibility. More recent clients have had the ability to= expose the capabilities of connected peers and modify behavior accordingly= , and overall it has worked very well.

Capability-based systems do work, and provide an excellent m= eans of trying out new algorithms, adding new features for upgraded clients= , and when necessary reverting protocol changes (by depreciating or removin= g extensions).

The problem with OpenGL was and continues to be that th= e two superpowers of that industry develop and maintain competing proposals= for similar functionality, which are thrust upon developers which must sup= port both if they want access to the latest and greatest features, until su= ch time that the ARB arbitrarily choses one to standardize upon (in the pro= cess creating yet another extension of the form ARB_* that may be different= and must be explicitly supported by developers).

I think the BitTorrent example shows that a loosely org= anized, open-source community *can* maintain a capability-based extension s= ystem without falling into capability-hell.

Mark
On Sun, Jun 17, 2012 at 9:30 AM, Amir= Taaki <zgenjix@yahoo.com> wrote:
As the only person to have created and maintaining a full reimplementation = of the Bitcoin protocol/standard, I do think Bitcoin is filled with arbitra= ry endianness and magic numbers. However it is a tiny and simple protocol.<= br>
The big problem is not implementing the Bitcoin protocol, but the fact that= once you have created a codebase, you want to perfect and fine-tune the de= sign. During the meantime, the Bitcoin protocol is being changed. Change to= the Bitcoin protocol is far more damaging to people that want to implement= the protocol than any issues with the current protocol.

That's not to say, I disagree with changes to the protocol. I think cha= nges should be a lot more conservative and have a longer time frame than th= ey do currently. Usually changes suddenly get added to the Satoshi client a= nd I notice them in the commit log or announcements. Then it's like &qu= ot;oh I have to add this" and I spend a week working to implement the = change without proper consideration or reflection which ends up with me hav= ing to compromise on design choices. That is to remain compatible with the = protocol.

However it is not my intent to slow down progress so I usually try to hedge= against that kind of feeling towards conservatism.



----- Original Message -----
From: Jeff Garzik <jgarzik@exmult= i.com>
To: Wladimir <laanwj@gmail.com&g= t;
Cc: bitcoin-de= velopment@lists.sourceforge.net
Sent: Sunday, June 17, 2012 5:19 PM
Subject: Re: [Bitcoin-development] Proposed new P2P command and response: g= etcmds, cmdlist

On Sat, Jun 16, 2012 at 4:42 AM, Wladimir <laanwj@gmail.com> wrote:
> Which is a perfectly reasonable requirement. However, one could simply=
> standardize what a 'thin client' and what a 'thick client&= #39; does and offers
> (at a certain version level), without having to explicitly enumerate > everything over the protocol.
>
> This also makes it easier to deprecate (lack of) certain features late= r on.
> You can simply drop support for protocol versions before a certain num= ber
> (which has happened before). With the extension system this is much ha= rder,
> which likely means you keep certain workarounds forever.
>
> Letting the node know of each others capabilities at connection time h= elps
> somewhat. It'd allow refusing clients that do not implement a cert= ain
> feature. Then again, to me it's unclear what this wins compared to=
> incremental protocol versions with clear requirements.
>
> I'm just afraid that the currently simple P2P protocol will turn i= nto a zoo
> of complicated (and potentially buggy/insecure) interactions.

What is missing here is some perspective on the current situation.=C2=A0 It=
is -very- easy to make a protocol change and bump PROTOCOL_VERSION in
the Satoshi client.

But for anyone maintaining a non-Satoshi codebase, the P2P protocol is
already filled with all sorts of magic numbers, arbitrarily versioned
binary data structures..=C2=A0 already an unfriendly zoo of complicated and=
potentially buggy interactions.=C2=A0 There is scant, incomplete
documentation on the wiki -- the Satoshi source code is really the
only true reference.

I see these problems personally, trying to keep ArtForz' half-a-node running on mainnet (distributed as 'blkmond' with pushpool).

In an era of HTTP and JSON, NFS and iSCSI, bitcoin's P2P protocol is woefully backwards, fragile, limited and inflexible when it comes to
parameter/extension exchange and negotiation.=C2=A0 Even iSCSI, that which<= br> is implemented on hard drive firmware, has the ability to exchange
key=3Dvalue=C2=A0 parameters between local and remote sides of the RPC
connection.

Calling the current P2P protocol "simple" belies all the
implementation details you absolutely -must- get right, to run on
mainnet today.=C2=A0 Satoshi client devs almost never see the fragility and=
complexity inherent in the current legacy codebase, built up over
time.

--
Jeff Garzik
exMULTI, Inc.
jgarzik@exmulti.com

---------------------------------------------------------------------------= ---
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122= 263/
_______________________________________________
Bitcoin-development mailing list
Bitcoin-develo= pment@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment


---------------------------------------------------------------------------= ---
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122= 263/
_______________________________________________
Bitcoin-development mailing list
Bitcoin-develo= pment@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-de= velopment

--e89a8ff1c7625b7f7904c2b51217--