From bram at gawth.com Mon Jul 2 13:15:01 2001 From: bram at gawth.com (Bram Cohen) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] BitTorrent now works on all platforms Message-ID: A new release of BitTorrent is up, version 1.0.2 - http://bitconjurer.org/BitTorrent/download.html It now works on all platforms, copies comments from blobs_want to blobs_have, prints out when it finishes downloading a file, and catches socket.error on calls to recv(). Please let me know how it goes if you try it under Windows or OSX. The next release will kill phantom connections and (hopefully) do it's crypto in C. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From bram at gawth.com Tue Jul 3 16:45:01 2001 From: bram at gawth.com (Bram Cohen) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] BitTorrent 1.0.3 out Message-ID: I've put up BitTorrent release 1.0.3 - the cpu hosing problems should be gone now (apparently select() can tell you connections died in a couple of weird ways, BitTorrent is now calling it very defensively). http://bitconjurer.org/BitTorrent/ Coming up - killing phantom connections, being nice about swapping out, and the interminable promise of the crypto being done in C :). -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From bram at gawth.com Tue Jul 10 18:08:01 2001 From: bram at gawth.com (Bram Cohen) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] New, stable release of BitTorrent out! Web site updated! Message-ID: The BitTorrent web site has been update to be much clearer, and a stable release has been put up, you can get it here - http://bitconjurer.org/BitTorrent/ New in this release - select() calling bugs fixed - thanks to David C for help tracking this down runs on windows and macOSX - thanks to Andrew Loewenstern for helping with the port now gives a reasonable amount of output immediate connects on startup - no more waiting a few minutes for it to connect to anything miscellaneous bug fixes This is probably the first really stable release of BitTorrent. Unfortunately crypto in C isn't done yet, so it's still slow, but that's coming soon. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From dnm at pobox.com Wed Jul 11 11:39:01 2001 From: dnm at pobox.com (Dan Moniz) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] dLoo and BlueBox Message-ID: From the language-dev mailing list: >Date: Wed, 11 Jul 2001 10:16:48 -0600 >From: Nathan Torkington >To: perl6-language@perl.org >Cc: language-dev@netthink.co.uk >Subject: dLoo releases peer-to-peer programming language > > >From newsforge: > >nile writes, "Today, dLoo released the complete architecture of an >extensible peer-to-peer programming language. Unlike traditional languages, >this language is defined on the Internet. Its syntax and semantics can be >extended by posting additional pieces of the language. As developers add to >the language it scales in richness and functionality. Programs run in a >software browser called BlueBox which dynamically downloads and assembles >the parts of the language as needed. For more information and access to the >source visit http://www.dloo.org. BlueBox is a community driven project >released under the GPL." >http://www.dloo.org/ -- Dan Moniz [http://www.pobox.com/~dnm/] From zooko at zooko.com Sun Jul 15 15:25:02 2001 From: zooko at zooko.com (zooko@zooko.com) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] Andy Oram on micropayments Message-ID: I saw this linked from O'Reilly com front page, and in the interview Andy Oram references our "Mojo Nation Responses" re: micropayments. http://www.amazon.com/exec/obidos/tg/feature/-/167528/t/002-7927711-6516039 "Andy: I'll defer to experts on this question. A valuable exchange was held on the O'Reilly P2P site ( http://openp2p.com/) between journalist Clay Shirky (in his article " The Case Against Micropayments") and Jim McCoy of Mojo Nation (in his article " Mojo Nation Responds"). Putting these two point-counterpoint articles together (along with the chapters on accountability and reputation in the peer-to-peer book), I have drawn the following conclusion. Micropayments, once easy-to-use systems are designed, will prove useful for the underpinnings of communication. They'll be good for making sure people share computing resources fairly, like providing disk space to file-sharing systems. They'll help prevent denial-of-service attacks by requiring expensive computations on the computers that try to join a peer-to-peer system. They'll facilitate load balancing. In other words, micropayments will be hidden away and will do their job silently, like other complicated protocols on the Internet. And they'll be tit-for-tat systems: you provide your computing resources in exchange for someone else's. Micropayments have less of a future as true payment systems. Providing your computing resources in exchange for food and housing? Not. The translation from computer protocols to hard cash requires an enormous infrastructure (and a centralized one, it appears). Furthermore, people don't like to be nickled and dimed. They don't want to worry every second about how much they're spending. The Internet will be like a Club Med vacation: pay once and then forget about it." Regards, Zooko From hal at finney.org Sun Jul 15 16:36:01 2001 From: hal at finney.org (hal@finney.org) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] Andy Oram on micropayments Message-ID: <200107152336.QAA22336@finney.org> Zooko quotes Andy Oram on micropayments: > Micropayments, once easy-to-use systems are designed, will prove useful > for the underpinnings of communication. They'll be good for making sure > people share computing resources fairly, like providing disk space to > file-sharing systems. They'll help prevent denial-of-service attacks > by requiring expensive computations on the computers that try to join a > peer-to-peer system. They'll facilitate load balancing. In other words, > micropayments will be hidden away and will do their job silently, like > other complicated protocols on the Internet. And they'll be tit-for-tat > systems: you provide your computing resources in exchange for someone > else's. I don't see how this works. If the micropayments are so invisible that they require no thought, how do they end up having an impact on issues like load balancing, etc.? The way micropayments work is by affecting supply and demand. While this can be automated to a degree, ultimately a human being needs to be in the loop to notice when he's spending faster than he's earning, or vice versa. > Micropayments have less of a future as true payment systems. Providing > your computing resources in exchange for food and housing? Not. The > translation from computer protocols to hard cash requires an enormous > infrastructure (and a centralized one, it appears). Furthermore, > people don't like to be nickled and dimed. They don't want to worry > every second about how much they're spending. The Internet will be like > a Club Med vacation: pay once and then forget about it." It's true that the conversion issue is complex, but that doesn't mean that there will forever be a divide between micropayments and "true payment systems". If whatever is being paid in the micropayments is valuable (and buying access rights, disk space and other resources is certainly valuable) then it can be traded for other goods. The problem is partially terminology. There are two orthogonal axes here. New forms of money versus existing forms of money; very small payment amounts versus larger payment amounts. All four possibilities are relevant. Technically it appears difficult at the present time to do small payment amounts with existing forms of money. So we cross off one of the four possibilities for now. Large payments with existing forms of money are being done already with credit cards. So we cross off that box as being a solved problem. This leads two boxes regarding new forms of money. The word "micropayments" makes us focus on the box for small payments. But the tough problem here isn't handling small payments, it's getting a new form of money accepted. If you can achieve that, you can potentially use your new money for both large and small payments. I don't see any technical reason why a form of money that is acceptable for small payments would be unacceptable for large ones. (Well, I can imagine some specific implementations that would have that property, as for example fixed-size coins each worth $.0001. But most implementations of money would be more flexible than this.) It doesn't make sense economically for there to be a great divide between the form of money used for micropayments, and that used for larger payments. This is the equivalent of an economic vacuum, and it will be filled. And if people don't like to be nickel-and-dimed, that will apply to micropayments as well as regular ones. It is fundamentally contradictory to assume that your new money is so worthless that no one cares if they spend it (hence no nickel-and-diming worries), while it is so valuable that it can be used to manage resources. Hal From alk at pobox.com Sun Jul 15 17:26:01 2001 From: alk at pobox.com (Tony Kimball) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] Andy Oram on micropayments References: <200107152336.QAA22336@finney.org> Message-ID: <15186.13419.971013.488769@spanky.love.edu> Quoth hal@finney.org on Sunday, 15 July: : : Technically it appears difficult at the present time to do small payment : amounts with existing forms of money. The only technical difficulty is usability. It is trivial to automate the process of mailing a check, and in such a way that the cost of the transaction is not disproportionate to its value. : Large payments with existing forms of money are : being done already with credit cards. So we cross off that box as being : a solved problem. It can be expensive to use paypal or merchant accounts or express payments, so that while it is a solved problem, the quality of the solution will inevitably (thanks to the invisible hand) improve. : It is fundamentally contradictory : to assume that your new money is so worthless that no one cares if they : spend it (hence no nickel-and-diming worries), while it is so valuable : that it can be used to manage resources. Not if it's just a protocol token. Calling it money is then just a metaphor, an aid in describing the protocol, and taking that metaphor out of context is what leads to the absurdity. The usability factors which loom large in the context of fungible assets have no obviously necessary reason to be similarly problematic in the exchange of protocol tokens. From lucas at gonze.com Sun Jul 15 17:31:02 2001 From: lucas at gonze.com (Lucas Gonze) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] Andy Oram on micropayments In-Reply-To: <200107152336.QAA22336@finney.org> Message-ID: IMHO protocol tokens will be convertible once they're useful, but not before. If I can reliably get computing cycles with these tokens, they they're worth money to me. The tokens are basically backed by CPU at that point. So the time scale to reach that point is it will take to build widely adopted computing tools that rely on these tokens. From gotz at amiga.com Mon Jul 16 14:08:01 2001 From: gotz at amiga.com (g'o'tz ohnesorge) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] Andy Oram on micropayments References: Message-ID: <3B5352C2.FFB5CDFE@amiga.com> zooko@zooko.com wrote: > "Andy: I'll defer to experts on this question. A valuable exchange was held on the O'Reilly P2P > site ( http://openp2p.com/) between journalist Clay Shirky (in his article " The Case Against > Micropayments") and Jim McCoy of Mojo Nation (in his article " Mojo Nation Responds"). > Putting these two point-counterpoint articles together (along with the chapters on accountability > and reputation in the peer-to-peer book), I have drawn the following conclusion. I still hate all such schemes. I usually don't have time to build up reputation when I suddenly need a large amount of network service (storage, computation, whatever). And another problem,or rather a different wording of the same: It decreases the amount of service that is available to new users, so they may not even stay long enough to offer their own (which for Napster helped to degrade its desirability to users, recently, as just far less people offered their stuff). > Micropayments have less of a future as true payment systems. Providing your computing > resources in exchange for food and housing? Not. Try using a supercomputer, you can buy time on it, expensively, at commercial owners .. or at least when Cray was famous, years back. But for your lazy 1000$ or less PC, how much can you demand for 24 hours on it? 1$ or 2$, worth a tiny slice of pizza? If you owned 1000 leading edge PCs, you might again be in business selling time to bioresearch. > much they're spending. The Internet will be like a Club Med vacation: pay once and then forget > about it." That's been so for the most time till now, but some just again start to predict that's going away, real soon real big. From painlord at inwind.it Mon Jul 16 14:08:02 2001 From: painlord at inwind.it (Mirco Romanato) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] Andy Oram on micropayments References: <200107152336.QAA22336@finney.org> Message-ID: <002a01c10e0e$244c5d80$be75623e@painlordt91ikd> ----- Original Message ----- From: > Zooko quotes Andy Oram on micropayments: > > Micropayments, once easy-to-use systems are designed, will prove useful > > for the underpinnings of communication. They'll be good for making sure > > people share computing resources fairly, like providing disk space to > > file-sharing systems. They'll help prevent denial-of-service attacks > > by requiring expensive computations on the computers that try to join a > > peer-to-peer system. They'll facilitate load balancing. In other words, > > micropayments will be hidden away and will do their job silently, like > > other complicated protocols on the Internet. And they'll be tit-for-tat > > systems: you provide your computing resources in exchange for someone > > else's. > I don't see how this works. If the micropayments are so invisible that > they require no thought, how do they end up having an impact on issues > like load balancing, etc.? The way micropayments work is by affecting > supply and demand. While this can be automated to a degree, ultimately > a human being needs to be in the loop to notice when he's spending faster > than he's earning, or vice versa. The impact on load balancing will be that the human will notice slow speed when he publish/search/download files. I agree with you that this isn't a big incentive/disincentive for many users, so usually it will not work very well. But I think that if I will can spend 10 bucks or euros (I'm european) to accellerate the rate of downloads of 100 GB from 400K/s to 600K/s, I will think about the options (Is about 150 saved hours over an ADSL connections). Even I could use the tokens to pay other services over the net (if the other side accept). If the users cannot handle the mojos they cannot use them like money. If they can handle their mojos they can use them like money also. We only need a crude way to take the mojo from the stash and send it to a selected broker. Improvements of the protocols and the way they work will do transactions smoother and painless for the users. > > Micropayments have less of a future as true payment systems. Providing > > your computing resources in exchange for food and housing? Not. The Why not? Computing resources was payed from fourty years ago. I think many enterprise will be available to pay for aggregate computing resources, and the enterprise that will sell the aggregate resources could reward the user that will offer ther CPU time or HD space with tokens. Others could pay to have tokens to accellerate they downloads. Probably the amount will often be neglegible, but if the market will be large, demand and offer will grow because will be easy to access them. > If you can achieve that, you can potentially use your new money for both > large and small payments. I don't see any technical reason why a form > of money that is acceptable for small payments would be unacceptable for > large ones. (Well, I can imagine some specific implementations that would > have that property, as for example fixed-size coins each worth $.0001. > But most implementations of money would be more flexible than this.) I think will be a time-variable prices for resources. Like the Mojo pricing system. > And if people don't like to be nickel-and-dimed, that will apply to > micropayments as well as regular ones. It is fundamentally contradictory > to assume that your new money is so worthless that no one cares if they > spend it (hence no nickel-and-diming worries), while it is so valuable > that it can be used to manage resources. It isn't a contraddiction, because people exchange their resources with others resources. The limits are the limits of the resources exchanged (HD space, CPU cycles and bandwidth). But I agree that many will want exchange other resources (money) for HD, CPU and BW (and time). Mirco Romanato From bram at gawth.com Fri Jul 20 15:18:02 2001 From: bram at gawth.com (Bram Cohen) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] new *fast* version of BitTorrent out Message-ID: Version 1.0.5 of BitTorrent is out, you can get it here - http://bitconjurer.org/BitTorrent/ All the encryption is now being done in C, as a result of which all that old slowness has disappeared, resulting in a product which will happily utilize your whole net connection. Also in this release is a completely rewritten README.txt and miscellaneous improvements/cleanups. This version *should* work under Windows and MacOS X, but I don't have machines which run those. The next version will have platform-specific executable distributions which don't require you have the python development libraries installed to build them. I need volunteers for Windows and OSX build master! Coming up - hex instead of base-32 encoding, and miscellaneous under the hood improvements to prepare for the creation of torrent:// urls, a prospect I'm all excited about. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From sah at thalassocracy.org Fri Jul 20 16:02:01 2001 From: sah at thalassocracy.org (Steven Hazel) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] new *fast* version of BitTorrent out In-Reply-To: Bram Cohen's message of "Fri, 20 Jul 2001 15:18:21 -0700 (PDT)" References: Message-ID: <87u207dx37.fsf@azrael.dyn.cheapnet.net> Bram Cohen writes: > Coming up - hex instead of base-32 encoding Yay, sanity! -S From gojomo at usa.net Fri Jul 20 16:15:01 2001 From: gojomo at usa.net (Gordon Mohr) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] new *fast* version of BitTorrent out References: <87u207dx37.fsf@azrael.dyn.cheapnet.net> Message-ID: <01b001c11171$bf9b7ee0$6401a8c0@bitcollider.com> Steven Hazel writes: > Bram Cohen writes: > > > Coming up - hex instead of base-32 encoding > > Yay, sanity! What do you dislike about Base32? - Gojomo From sah at thalassocracy.org Fri Jul 20 16:39:01 2001 From: sah at thalassocracy.org (Steven Hazel) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] new *fast* version of BitTorrent out In-Reply-To: "Gordon Mohr"'s message of "Fri, 20 Jul 2001 16:14:32 -0700" References: <87u207dx37.fsf@azrael.dyn.cheapnet.net> <01b001c11171$bf9b7ee0$6401a8c0@bitcollider.com> Message-ID: <87puavdvcb.fsf@azrael.dyn.cheapnet.net> "Gordon Mohr" writes: > Steven Hazel writes: > > Bram Cohen writes: > > > > > Coming up - hex instead of base-32 encoding > > > > Yay, sanity! > > What do you dislike about Base32? Probably every programming language I have ever had to write a serious program in has a built-in function to convert a string of base-16 digits into an integer. Base32 doesn't even have a breakaway standard encoding. So how much benefit do you *really* get from halving the text length of your keys? Not enough. --Steven From gojomo at usa.net Fri Jul 20 17:25:01 2001 From: gojomo at usa.net (Gordon Mohr) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] new *fast* version of BitTorrent out References: <87u207dx37.fsf@azrael.dyn.cheapnet.net><01b001c11171$bf9b7ee0$6401a8c0@bitcollider.com> <87puavdvcb.fsf@azrael.dyn.cheapnet.net> Message-ID: <001701c1117b$91ffe200$6401a8c0@bitcollider.com> Steven Hazel writes: > "Gordon Mohr" writes: > > What do you dislike about Base32? > > Probably every programming language I have ever had to write a serious > program in has a built-in function to convert a string of base-16 > digits into an integer. Base32 doesn't even have a breakaway standard > encoding. So how much benefit do you *really* get from halving the > text length of your keys? Not enough. Actually, the savings in text-representation is only 20%. Even that savings could be important in a couple of contexts which binary data must become text -- such as achieving short URLs or squeezing into the DNS namespace. (Hence a number of IETF proposals feature Base32 to represent Unicode domain names in DNS packets.) Also, if keys are ever mixed in with human-readable text, saving just a few characters can help a lot in assisting presentation. The lack of a dominant encoding is a pain, but I expect the IETF's choice in the DNS work to quickly establish a favored alphabet, probably [a-k mn p-z 2-9]. Similarly, it's only a half-page of code in any language. Lots of free examples should be available as soon as any popular system needed Base32 expressions. I suppose I'm trying to say: we plan to use Base32, and I think we're sane! - Gojomo From alk at pobox.com Fri Jul 20 19:17:01 2001 From: alk at pobox.com (Tony Kimball) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] new *fast* version of BitTorrent out References: <87u207dx37.fsf@azrael.dyn.cheapnet.net> <01b001c11171$bf9b7ee0$6401a8c0@bitcollider.com> <87puavdvcb.fsf@azrael.dyn.cheapnet.net> Message-ID: <15192.58908.516314.975969@gargle.gargle.HOWL> : : Probably every programming language I have ever had to write a serious : program in has a built-in function to convert a string of base-16 : digits into an integer. Base32 doesn't even have a breakaway standard : encoding. So how much benefit do you *really* get from halving the : text length of your keys? Not enough. Not half. going from 5 bytes for 20 bits to 4 bytes for 20 bits is a 20% savings. Base32 is unreadable, but everyone's dog knows what base16 means. From bram at gawth.com Sun Jul 22 01:58:01 2001 From: bram at gawth.com (Bram Cohen) Date: Sat Dec 9 22:11:42 2006 Subject: [p2p-hackers] 1.0.6 out - should build on Windows and OS X now Message-ID: 1.0.6 is out. I put in a different rijndael library. This one doesn't abuse the C preprocessor, so it should build on all platforms just fine. It also fits cleanly in one file, compiles much faster and with no warnings, is much smaller, and has a much clearer intellectual property statement (it has no restrictions). I also moved around the web page, since people kept complaining that they didn't realize they should click on the 'what is it?' link to find out what it is - http://bitconjurer.org/BitTorrent/ -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From cyb at azrael.dyn.cheapnet.net Thu Jul 26 23:11:01 2001 From: cyb at azrael.dyn.cheapnet.net (Brandon K. Wiley) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Chord comments Message-ID: I have just read the longer Chord paper. I read the old one a while back, but the longer one has lots of interesting new stuff. Here are my thoughts on Chord and how it relates to a Freenet-like system. They are fresh, rough thoughts because I figured that it would be disadvantageous to continue pondering such things quietly when my understanding could be greatly enhanced by feedback. Please forward my comments to anyone that you think might want to read them. The main goal of my comments it to find a way to give some resistance to a Chord network of evil nodes being able to find the node responsible for an arbitrary key. This will give a Chord network a modicum of censorship-resistance. Currently it equally as vulnerable as Gnutella. The first and most obvious suggestion is to have requests pass back the file rather than the IP of the node which stores the file. Even after that is done, there are still various ways in the base Chord system by which an attacker can find the nodes for arbitrary keys. The basic strategy is that while the Chord paper simply has a function find_successor(key), which can be called arbitrarily, you in fact only call this method in certain contexts and with certain keys. Resisting network mapping attacks simply consists of checking and enforcing these contexts so that arbitrary keys can't be looked up. One use of find_successor() is to find your own successor. This can easily be verified by the node that you call it on as it knows your IP. Another use is to find the next best node when the best node you know about has disappeared. This can be checked by attempting to connect to the supposedly failed node. Creating a fake node IP for a non-existance node which is close to the key which you want to attack is a possibility. However, it would require breaking the SHA1 hash to generate such a fake IP. This is acceptable as the Chord system is fundamentally based on the assumption that breaking SHA1 is sufficiently hard to forget about. The final use of find_successor() is in stabalizing your routing table. You routinely ask all the nodes you know about if they have better matches for your routing table than you currently do. This can be made less attackable by instead using a method findBestMatch(slot). Instead of specifying a key which you want to find the closest node for, you specify which slot in your routing table you want to fill. The node you call this on uses your IP and the slot number to generate the key to match. This way you can't ask for arbitrary keys. However, this doesn't just magically solve the problem of network mapping. With a routing table size of 160, you can ask each node for 160 different IPs. If you continually harvest IPs and keep asking all of the IPs you have for the 160 IPs they have which best fit your slots, you can get a large number of IPs. It seems to me that the IPs you get will have a distribution where more of them are closer to your own keyspace. I don't think that you can map the whole network as there will be many redundant IPs and the overlap will increase as you gain IPs, and the rate at which you gain new IPs will drop. However, by running more nodes you can of course map more of the network. The question, then, is how much of the network can you map per node, and how long will it take. Another possibility to slow the mapping of the network is to have nodes periodically suggest new entries rather than periodically requesting new entries. This will cause the harvesting of IPs to happen at a rate decided by the network rather than the attacking node. But then of course the question is still the same, which is how long will it take to map the network and will it be sufficient to thwart an attacker. I would greatly appreciate any help anyone could provide in determining some numbers for the rate of mapping. And now a tangent about running virtual nodes (totally unrelated to the above): The paper states that load balancing can be improved greatly by having each node run log N virtual nodes. The obvious way to do this it to have several node instances running on the same machine on different ports. This is bad for reasons of resource usage and also because for various reasons it would be nice to be able to authenticate a node's keyspace, which becomes complicated because you can be fairly certain of a node's IP when it connects to it, but have to confirm its port. For simplicity I suggest that all nodes run a fixed number of virtual nodes, specifically 32 since that is the appropriate number of virtual nodes for a network in which every IP address has a node. I also propose that there be only one node per IP address. That makes verification very easy. Luckily, running a virtual node simply means keep 32 references tables for different keyspaces rather than just one. That should be easily managed. Then, of course, you must have the ability to attach a keyspace to the node. The previous method was hash(IP). With 32 virtual nodes per IP, you can simply use hash(IP), hash(IP+1), ..., hash(IP+31) to generate 32 keyspaces based on the node's IP. From arma at mit.edu Fri Jul 27 00:40:02 2001 From: arma at mit.edu (Roger Dingledine) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Chord comments In-Reply-To: ; from cyb@azrael.dyn.cheapnet.net on Fri, Jul 27, 2001 at 01:11:38AM -0500 References: Message-ID: <20010727033914.G7892@belegost.mit.edu> On Fri, Jul 27, 2001 at 01:11:38AM -0500, Brandon K. Wiley wrote: > Another use is to find the next best node when the best node you know > about has disappeared. This can be checked by attempting to connect to > the supposedly failed node. Creating a fake node IP for a non-existance > node which is close to the key which you want to attack is a ^^^^^^^^ > possibility. However, it would require breaking the SHA1 hash to > generate such a fake IP. This is acceptable as the Chord system is > fundamentally based on the assumption that breaking SHA1 is sufficiently > hard to forget about. No -- doing the above attack is very feasible. You cannot generate something which hashes directly to the key you want to attack, but getting "close" is just a matter of trying until you get one close enough. How close you want to be dictates how many tries it should take, but given the expected number of nodes in the system this is certainly easier than inverting SHA1. The authors attempt to mitigate this sort of problem by making node ids be the hash of the IP they're on -- so it's hard to make very many "tries", since each try needs a new IP. While this does in fact make it harder, there are still two attacks which I can see working nicely: a) Be on a class B network like me. You've got 65k IPs to try. If you're not on one, break into one. b) Determine which IPs would be useful to you, and break into machines on their subnets. > The final use of find_successor() is in stabalizing your routing table. > You routinely ask all the nodes you know about if they have better > matches for your routing table than you currently do. This can be made > less attackable by instead using a method findBestMatch(slot). Instead > of specifying a key which you want to find the closest node for, you > specify which slot in your routing table you want to fill. The node you > call this on uses your IP and the slot number to generate the key to > match. This way you can't ask for arbitrary keys. This brings up one of my biggest problems with Chord -- lack of verifiability of routing table updates. I'm willing to assume that when you join the ring you will join the correct ring (rather than the adversary's ring). But when a node is "fixing" the Chord network due to loss/addition of a node, how do you know that he's fixing it right? Nodes can flat-out give you a new routing table which is made up entirely of dead nodes, thus cutting you out of the ring. If you verify that new routes you get work, then the adversary can transition you onto his fake ring, and then drop you all at once (or just watch all your queries). This seems like a very serious problem. So far, the authors I've asked haven't had any good answers. One defense might be redundant routing information -- you get a routing table update from k people at once, so the adversary has to forge k lies at once rather than 1 in order to screw you. That certainly increases the challenge of taking advantage of this. I think it's quite a bit harder than this paragraph implies, though. > The paper states that load balancing can be improved greatly by > having each node run log N virtual nodes. The obvious way to do this it > to have several node instances running on the same machine on different > ports. This is bad for reasons of resource usage and also because for > various reasons it would be nice to be able to authenticate a node's > keyspace, which becomes complicated because you can be fairly certain of > a node's IP when it connects to it, but have to confirm its port. Agreed. > For simplicity I suggest that all nodes run a fixed number of > virtual nodes, specifically 32 since that is the appropriate number of > virtual nodes for a network in which every IP address has a node. I also > propose that there be only one node per IP address. That makes > verification very easy. > Luckily, running a virtual node simply means keep 32 references > tables for different keyspaces rather than just one. That should be > easily managed. > Then, of course, you must have the ability to attach a keyspace > to the node. The previous method was hash(IP). With 32 virtual nodes per > IP, you can simply use hash(IP), hash(IP+1), ..., hash(IP+31) to > generate 32 keyspaces based on the node's IP. I think this is the right approach, but a bit too simple. This would yield collisions for nodes with similar IPs, meaning there are multiple machines associated with a given key. That starts to get really ugly. Can you give the url of the paper you looked at, and I'll look at it in more detail? --Roger From sah at thalassocracy.org Fri Jul 27 01:05:01 2001 From: sah at thalassocracy.org (Steven Hazel) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Chord comments In-Reply-To: Roger Dingledine's message of "Fri, 27 Jul 2001 03:39:15 -0400" References: <20010727033914.G7892@belegost.mit.edu> Message-ID: <87y9pan6ft.fsf@azrael.dyn.cheapnet.net> Roger Dingledine writes: > > Then, of course, you must have the ability to attach a keyspace to > > the node. The previous method was hash(IP). With 32 virtual nodes > > per IP, you can simply use hash(IP), hash(IP+1), ..., hash(IP+31) > > to generate 32 keyspaces based on the node's IP. > > I think this is the right approach, but a bit too simple. This would > yield collisions for nodes with similar IPs, meaning there are > multiple machines associated with a given key. That starts to get > really ugly. The simple solution to that is to append the virtual node number rather than adding it: You hash the four IP bytes, and a fifth byte containing the virtual node number. Similar IPs won't collide, and it's not any more complicated than what Brandon said. --Steven From cyb at azrael.dyn.cheapnet.net Fri Jul 27 01:10:01 2001 From: cyb at azrael.dyn.cheapnet.net (Brandon K. Wiley) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Chord comments In-Reply-To: <87y9pan6ft.fsf@azrael.dyn.cheapnet.net> Message-ID: > The simple solution to that is to append the virtual node number > rather than adding it: You hash the four IP bytes, and a fifth byte > containing the virtual node number. Similar IPs won't collide, and > it's not any more complicated than what Brandon said. Sorry, that's what I meant. I've been in python for too long. :-) From gojomo at usa.net Fri Jul 27 10:27:01 2001 From: gojomo at usa.net (Gordon Mohr) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Chord comments References: Message-ID: <003401c116c1$7187a9e0$c600a8c0@gojovaio> Brandon Wiley writes: > The main goal of my comments it to find a way to give some resistance to > a Chord network of evil nodes being able to find the node responsible > for an arbitrary key. This will give a Chord network a modicum of > censorship-resistance. Currently it equally as vulnerable as Gnutella. I would take a step back; the essential quality of a Chord system is that it is easy to find the node(s) responsible for a key. So rather than changing that specifically, I think what you really want is for that ease not to make any index-nodes or content-provider susceptible to censorship. I see two possibilities, for use individually or in concert: Option #1: Have all the nodes that are cooperating for Chord-indexing, also cooperate as part of a big mix-net. (This option is mentioned in passing in Section 5 of the HOTOS Chord paper.) No actual file location would be advertised; just an entry-point into the mix-net that can route to the real location, through a number of non-collaborating nodes. (I suspect some sort of hash-chain might be useful for this purpose.) Option #2: Advertise *what you want*, rather than *what you're offering*. That is, the giant Chord-of-cooperating nodes becomes a list of "pending wishes" rather than material currently-being-offered. To provide files, search the net for wishes you can fulfill -- or perhaps arrange to be notified when wishes appear within certain ranges. Why might this second option help? First off, it precludes any passive way to collect information about file-sources. You actually have to post-a-wish, giving a delivery address, to probe for the availability of content. The wish-granter may then choose to obscure her location much more easily than in systems where your full share-collection is advertised at a reachable address. (For example, even if the Chord net itself doesn't provide mix-net capabilities, outside anonymizing proxies could be used, at the sender's discretion.) Even without providing "strong" untraceability, just by using a different single proxy each time (or per file) the provider can look like a series of "casual", one-time-providers, rather than a single "meganode" deserving priority attention of the censors. Even without using origin-obfuscation, a file-provider under such a system might be able to use an entrapment defense against prosecution in many jurisdictions. (e.g. "I didn't intend to violate your laws; I just wrote a bot that efficiently fulfilled peoples' content wishes. Many/most of the wish-fulfilliments were legal. Only because Agent-X posted a wish for illegal content did I get pulled into this. Arrest Agent-X, why don't you!") Of course, I am not a lawyer, so don't count on this. - Gojomo From cyb at azrael.dyn.cheapnet.net Mon Jul 30 13:59:02 2001 From: cyb at azrael.dyn.cheapnet.net (Brandon K. Wiley) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Chord comments In-Reply-To: <20010727033914.G7892@belegost.mit.edu> Message-ID: > a) Be on a class B network like me. You've got 65k IPs to try. If you're > not on one, break into one. So given a fixed size keyspace (all IP4 addresses), you can do a precomputed dictionary attack. If you have a lot of IPs (a.k.a. money) then you can very quickly determine which of your IPs is best (since in Chord "best" is absolute within a given set of IPs and a given key) and then set up a node there. So the question of how useful this attack is depends on the distribution of keyspaces over the network. My math is probably naive or just wrong, but my attempt at probability says that your probability of having the closest key is (evil / (good + evil)) ^ keys. So if you have 65k addresses and the network has 65k nodes, then your odds of being closest for a given key are 50%. Your odds for winning on two keys are 25%, etc.. So if you have the entire IP resources of a university, you can *almost* be assured of censoring one file, but if you want to, say, censor 65k files (one file per node), then you get .5 ^ 65k, which is really small. So I'm not sure if this is good or bad, but it doesn't seem *really* terrible, like, for instance, Gnutella. > b) Determine which IPs would be useful to you, and break into machines > on their subnets. Right now I'm just trying to defend against attackers that use legal measures such as scanning IPs. If you can take over arbitrary machines at will then I don't know of any currently existing system which will help you. > But when a node is "fixing" the Chord network due to > loss/addition of a node, how do you know that he's fixing it right? Nodes > can flat-out give you a new routing table which is made up entirely of > dead nodes, thus cutting you out of the ring. If you verify that new > routes you get work, then the adversary can transition you onto his fake > ring, and then drop you all at once (or just watch all your queries). An adversary cannot give you an entirely new routing table. Well, he can, but you will only use those parts of the new routing table which are actually better than what you already have. The adversary does not know what your routing table currently contains, so cannot know what keys it controls are better than what you already have. However, since the keys which you desire are relative to your own key, your key is based on your IP, and the adversary has your IP, the adversary can generate a routing table for you which contains the best keys under its control for the various slots in your routing table. The odds of the adversary taking over your entire routing table are, as before, (evil / (evil+good)) ^ keys. So given a 65k network and 65k addresses under the control of the adversary, the odds of taking over your entire routing table are .5 ^ 160 = 6.8e-49. If any good entries are left in your routing table then when the evil node drops you, you can rebuild your routing table from the good node. Chances are that the evil node isn't the best node for one of the slots in your routing table, so there is a possibility that it will be replaced by another node and then you won't get bad references anymore. > Can you give the url of the paper you looked at, and I'll look at it in > more detail? http://www.pdos.lcs.mit.edu/papers/chord:sigcomm01/ From bram at gawth.com Mon Jul 30 18:43:01 2001 From: bram at gawth.com (Bram Cohen) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] RE: i've got a design you guys would be interested in In-Reply-To: Message-ID: On Sun, 29 Jul 2001, Josh wrote: > To be honest, I've never heard of ocean store. Thanks, I will look it up > immediately. I found out about freenet via an attorney. I've been so focused > on the design that I haven't read up on all the others... yet.. As for the > presentation, you probably need the shockwave player, the flash player won't > do it. The link to it is at my site, www.mercuryfs.net/presentations. I poked around your site, and get the impression you're spinning your main advantage is downloading content from the local network, thus saving costs for the ISP. Others have this idea as well, but I think it's going after the wrong market. The customer of a p2p application is the end user. End users really, really don't care about costs to their ISP. They might care a bit about latency, although that's already toast for napster, gnutella, freenet, bittorrent, and just about everything else. They also might care about download time, but the bottleneck is going to be the bandwidth of their net connection. There are exceptions to this, such as students at very large universities and people in very large companies, who have much faster connections to each other than the net at large, but generally speaking, the main performance problems of p2p are finding someone else you want, getting them to offer it for upload, and using your whole net connection. I've basically chucked working on anything but those three problems for BitTorrent, and even that's proving to require quite the technical tour de force to get right. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes