Received: from sog-mx-3.v43.ch3.sourceforge.com ([172.29.43.193] helo=mx.sourceforge.net) by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1T10os-0000KP-5b for bitcoin-development@lists.sourceforge.net; Mon, 13 Aug 2012 20:00:58 +0000 X-ACL-Warn: Received: from wp303.webpack.hosteurope.de ([80.237.133.72]) by sog-mx-3.v43.ch3.sourceforge.com with esmtp (Exim 4.76) id 1T10op-0002Tp-9P for bitcoin-development@lists.sourceforge.net; Mon, 13 Aug 2012 20:00:58 +0000 Received: from 84-72-69-53.dclient.hispeed.ch ([84.72.69.53] helo=[192.168.0.21]); authenticated by wp303.webpack.hosteurope.de running ExIM with esmtpsa (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) id 1T10oc-0002VM-Uh; Mon, 13 Aug 2012 22:00:43 +0200 Message-ID: <50295CE4.5040606@justmoon.de> Date: Mon, 13 Aug 2012 22:00:36 +0200 From: Stefan Thomas User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:14.0) Gecko/20120713 Thunderbird/14.0 MIME-Version: 1.0 To: bitcoin-development@lists.sourceforge.net References: <5028AFBE.8070104@justmoon.de> In-Reply-To: X-Enigmail-Version: 1.4.3 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-bounce-key: webpack.hosteurope.de;moon@justmoon.de;1344888055;f69db562; X-Spam-Score: 0.0 (/) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. X-Headers-End: 1T10op-0002Tp-9P Subject: Re: [Bitcoin-development] BIP: Custom Services X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Aug 2012 20:00:58 -0000 > More elaboration of the use cases might therefore be useful. I'm implementing a DHT, rather than storing the whole blockchain index locally, a future version of BitcoinJS will store only a user defined percentage (anywhere from 0-100%.) Any misses will be resolved by querying the network. Thanks to ultraprune, we no longer need a full index for verification. And for all the other use cases like lightweight server queries and block explorer queries a bit of latency is acceptable in exchange for scalability and decentralization. This feature will give people to option to run anywhere from a large BitcoinJS node (100% index) to a small one (20% index) to a lightweight one (0% index.) All of them are equally queryable, so if you're just trying out BitcoinJS you won't have to download the block chain just to run the block explorer example. Only when your block explorer's traffic grows will you need to contribute some query services back to the network in order not to get rate-limited. > these capabilities are not advertised with > CAddress, so how does one usefully discover and make use of them? Pieter brought up this very point when he reviewed an earlier draft. This prompted the creation of the second BIP I mentioned: https://en.bitcoin.it/wiki/User:Justmoon/BIP_Draft:_Custom_Service_Discovery The basic principle is quite simple - prefix the standard addr message with a service-specific message to mark off service support. It's easy to implement and very efficient (without compression it's 1.288 bits per node, with compression it's quite possibly more efficient than the services field.) Because this stuff is a bit more complex and because it requires no changes to the Bitcoin protocol, Amir and I chose to split it off into a separate BIP and I want to work on it a little more. But if you are wondering how peer exchange might work for these custom services, please do take a look at it. > you may as well just form an entirely independent network When I designed the DHT, I did just that. Later I was working on a concept for a decentralized pool and I noticed I was solving the same problems again. And with all three services running a node might be maintaining three separate TCP connections to the exact same peer. So then I considered making the DHT network extensible, so that the decentralized pool protocol could live in there. But, well if I'm doing that, why not just make the regular Bitcoin protocol extensible and let both extensions live in there. For a custom service you DO need the following: - service-specific DNS seeds - service announcement (BIP: CS) - service-specific messages (BIP: CS) - service-specific peer exchange (BIP: CSD) And those four things cover a lot of what Bitcoin does. But once you're thinking about n custom services it starts to look easier to add semantics for "some nodes support some things" in one network rather than instantiating n networks. On the opposite end of the spectrum there are very simple services. Consider a WebSocket transport. Some nodes might offer Bitcoin-over-WebSocket, for example to implement an SPV client in the browser. But they don't connect to each other via WebSocket, since they prefer plain TCP. So in this case you need peer exchange, but not much else. You could create a new P2P network for the sole purpose of exchanging peers, but again it seems much easier if there were ways to do this on the Bitcoin network. One final point: A major focus of this BIP is to make it easy to canonicalize custom services if we choose to do so. The idea is that custom services get to prove themselves in the wild - those that work well may be added to the standard protocol. That's a good reason to 1) encourage custom services to live in-band and 2) recommend compatibility with Bitcoin's standard mechanisms (12 byte command names, 1 bit service announcement, 1 bit peer exchange etc.) On 8/13/2012 3:15 PM, Mike Hearn wrote: > I think it's pretty reasonable, although people will want to use node > flags to get into the addr broadcasts anyway. > > That said, I suspect (based on previous discussions) that there would > be quite some pushback against putting extra functionality into the > core Bitcoin network. Most likely people will re-use the code with > different peer discovery seeds and bootstrap similar but unrelated P2P > networks for doing new applications. > > For instance, what if we want to do the language translation app I've > talked about a few times before? You need a way to floodfill broadcast > invalid transactions to interested parties. The pubsub mechanism in > the Bitcoin protocol was an interesting way to do that, but I think it > got removed. To broadcast to interested nodes now, you'd have to find > them via addr broadcasts and then connect directly. And if you're > going to do that, you may as well just form an entirely independent > network. > > More elaboration of the use cases might therefore be useful. >