From p2p at garethwestern.com Mon Mar 1 22:40:53 2004 From: p2p at garethwestern.com (Gareth Western) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Several Questions... Message-ID: <4043BBF5.6070309@garethwestern.com> Good evening everyone, I have a couple of questions to which I would greatly appreciate some answers/opinions.... 1) Can someone provide any links to information about chunking files for transfer in p2p networks (ie optimal chunk size in relation to filesize (if that matters??), error checking/recovery methods, etc)? 2) The Chord Project website (http://www.pdos.lcs.mit.edu/chord/) is a little out of date, but I am very interested in the CFS part of it. Is there anywhere else that reports the latest development regarding this project? 3) Following on from 2, is anyone aware of any other projects similar to CFS (not just regular file storage/retrieval, but a unix-like FS interface)? Thanks very much! ~Gareth From thiago at vetorial.net Tue Mar 2 20:46:37 2004 From: thiago at vetorial.net (Thiago Pinto Damas) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] NS and P2P In-Reply-To: <20040228061243.8410.qmail@web60207.mail.yahoo.com> References: <20040228061243.8410.qmail@web60207.mail.yahoo.com> Message-ID: <1078260397.4044f2ad97fa3@webmail.vetorial.net> I implemented two p2p search algorithms in NS-2, using the fulltcp code. The code was entirely written in OTcl, using the primitives described in the NS-2 manual. If someone is interessed, pvt-me !! ------------------ Thiago Pinto Damas Citando Rohit Bhalla : >> Hello, >> >> I found in the ns-users archives some messages that >> relate the implementation of P2P stuff in NS >> Already someone have implemented with success on of >> this types of agents (Narada,Yoid, gnutella)? If yes, >> i >> appreciate so much if someone can send the code to >> me... >> >> Thanks by your help! >> Rohit >> >> >> __________________________________ >> Do you Yahoo!? >> Get better spam protection with Yahoo! Mail. >> http://antispam.yahoo.com/tools >> _______________________________________________ >> p2p-hackers mailing list >> p2p-hackers@zgp.org >> http://zgp.org/mailman/listinfo/p2p-hackers >> _______________________________________________ >> Here is a web page listing P2P Conferences: >> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences >> From seth.johnson at RealMeasures.dyndns.org Sun Mar 7 06:27:04 2004 From: seth.johnson at RealMeasures.dyndns.org (Seth Johnson) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Call to Assembly: Internet Commons Congress 2004 References: <20031120084437.GQ10941@zork.net> <20031120232325.GC10941@zork.net> Message-ID: <404AC0B8.AEE57BC1@RealMeasures.dyndns.org> New Yorkers for Fair Use Call to General Assembly ------------------------------------------------- Internet Commons Congress 2004 March 24-25, 2004, Outside Washington, DC Scheduled Sessions/Participants: http://www.internationalunity.org/schedule.html http://www.internationalunity.org http://www.nyfairuse.org/icc Please forward this call to any other concerned parties you might know. Please visit the above links to register to attend and join in the fight to preserve the Internet commons. Today our commons is under attack. The attack is wide and pervasive. Even our right to own and use computers inside our homes and offices, is under attack. The time has come to assemble and declare our rights. We call upon advocates and organizers, authors and cow-orkers, readers and singers, politicians and students, grandmothers and children of all ages, and all who support the right of free human beings to the free dissemination and use of information rendered to the commons for the benefit of the public, to join us at the Internet Commons Congress outside Washington DC on March 24 and 25, 2004. We live in a time of vibrant prospects and shameful travesties, brought on as we confront the implications of a new and broader and greater empowerment in furtherance of our common wealth and in engagement in our common governance. Today we possess: - The Internet: the means to disseminate and make use of published information flexibly and powerfully, on a worldwide scale - Computers: tools to process, select, combine, analyze and synthesize information at the digital and logical level, and - Logical Freedom: the power to devise means of applying these tools through the free use and expression of logic in code But today we also confront: - attempts to create irrational and wildly artificial legal and regulatory trammels on new conventions, such as VoIP, in order to keep control of the world's communication channels in the hands of old oligopolies, monopolies, and tyrannical governments - an intransigent U.S. Federal Communications Commission, arrogating to itself an unprecedented authority to declare exclusive rights policy and to regulate the design of digital devices on that basis - consolidated mass media and entrenched communications monopolies that subvert principles of the public interest with the willing concurrence of complaisant regulators and legislators - elected representatives who have made plain their intention to enact a new exclusive right to factual information in databases - forceful attempts in Europe to subvert the law banning patents on software, by patent establishment professionals and the large companies they serve - specious arguments by public servants and privileged contractors for the supposed reliability of "new voting technology" - attempts by the Bio-Medical Cartel and others to seize the fruits of logical, biological, medical, and pharmaceutical researches carried out at publically financed institutions of science and learning - an already well advanced and well funded plan to impose a redesign of home computer hardware so that running software that you choose would be made impractical, and analyzing and processing information in the manner you choose would be made impossible; the new design, backed by laws such as the DMCA, would result in the emplacement of wiretap and remote control hardware and supporting software in every new low cost home computer sold in 2006 - massive ongoing and systematic violations of contract law and antitrust law and consumer protection law by Microsoft and its partners, by means of which most home users are left with no choice but to run Microsoft operating systems: most people are not offered any choice of operating system at point of sale of the hardware, and are therefore induced to employ systems that are difficult to use and easily parasitized, systems that are indeed so bad because Microsoft need not compete - a hundred million dollar campaign of barratry and red-baiting conducted by SCO, acting as agent for the convicted monopolist Microsoft, to induce businesses and individuals to steer away from exercising free control of their logic devices, away in particular from GNU/Linux operating systems; the assault led by SCO is only one of many of similar scale All these issues and more are part of a broad struggle by all the people, we who treasure our freedom and who wish to remain free to use our Net and our computers in all the ways that are both fit and just. We call all ready advocates and concerned constituencies to assemble at the Internet Commons Congress this March 24 and 25, 2004. Here we will forge a bond in our common cause of information freedom, detail our missions and callings and summon each other to join in common cause. Please click here for details regarding venue, schedule, logistics: http://www.nyfairuse.org/icc/ Registration for attendance is free: http://www.nyfairuse.org/icc/reg.xhtml Those in attendance will issue calls for action, as shall we. We call all free citizens to join the struggle against englobulation of our Commons and our computers by the loose association and alliance of cartels, oligopolies, monopolies, and parts of governments, that seek to keep or take control of all the communications systems of the world. At the moment New Yorkers for Fair Use knows of a few efforts which we will forward at the Congress: - Continued Actions for Refunds: We hope to prepare materials to move the FTC, Congress of the USA folk, the Federal antitrust team, and the judge in the Microsoft case to consider effective action on the basis of gross violations of both the 1994/1995 consent decree, and the recent conviction of Microsoft. This effort needs several score affidavits dealing with anti-competitive practices at point of sale of low cost computer hardware. - Education of Regulators and Legislators and Attorneys about Home Computer Hardware: We will explain and demonstrate the boot process today on untrammeled hardware and what the boot process would be like on Palladiated hardware, that is, hardware with hard DRM. - Procurement Policy Education and Action: We seek to collect and analyze the grossly inequitable policies and procedures by which vendors of source secret softwares keep their special privileged position in the machine rooms and desktops of government agencies. - Education of Regulators and Legislators and Judges about the Net: We will explain the fundamental principles which, for more than thirty years, have supported the psychic and moral and legal and engineering foundations of our Net. A popularly reported on issue directly connected with these principles is the "issue of Voice Over Internet Protocol". These four actions have been mentioned because organizations, tribes, and individuals from New York City have recently been working on these four efforts. We know that other efforts will also be carried forward at the Internet Commons Congress. Come and help! -- New Yorkers for Fair Use http://www.nyfairuse.org [CC] Counter-copyright: http://realmeasures.dyndns.org/cc I reserve no rights restricting copying, modification or distribution of this incidentally recorded communication. Original authorship should be attributed reasonably, but only so far as such an expectation might hold for usual practice in ordinary social discourse to which one holds no claim of exclusive rights. From martinla at comp.nus.edu.sg Sun Mar 7 11:18:45 2004 From: martinla at comp.nus.edu.sg (Martin LANDERS) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Backup Survey... Message-ID: Hi Folks, recently joined the list, and really like the in-depth discussions here. My name is Martin Landers, and I'm currently working on P2P Backup, the topic of my "Diplomarbeit" (think Master's thesis). As part of my work I'm trying to figure out in which "environment" a P2P application, especially P2P backup has to operate, in order to evaluate whether P2P backup is possible on a global scale (internet), or rather confined to scenarios like corporate LANs. To do so, I'm currently running a survey, trying to gather data from as many volunteer's machines as possible, in order to do statistics on things like: free hard disk space (total/used hard disk space), internet connection speed, distribution of file sizes, distribution of file modification times, etc. To make things easy for the people running the survey I've written a Java-application that collects all the neccessary data, and can upload it to a server. The survey is anonymous. People concerned about security can check the results out themselves before uploading. The survey application is available at: http://p2psurvey.ddns.comp.nus.edu.sg I figure the results of this survey might be interesting to people on this list as well, so I'm asking for your assistance in promoting the whole thing. Maybe you can help getting the word out, so I can collect more data, leading to more useful results. A try at getting the survey slashdotted failed so far - maybe you're more sucessful than I am ;) If you have any suggestions, questions, or whatever, don't hesitate to contact me, I'm interested in feedback of any kind, and would love to discuss the idea with people. Thanks, Martin Landers From markus-kern at gmx.net Sun Mar 7 12:47:38 2004 From: markus-kern at gmx.net (Markus Kern) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Backup Survey... In-Reply-To: References: Message-ID: <1926203734.20040307134738@gmx.net> On Sunday, March 7, 2004, 12:18:45 PM Martin LANDERS wrote: > Hi Folks, > recently joined the list, and really like the in-depth discussions here. > My name is Martin Landers, and I'm currently working on P2P Backup, the > topic of my "Diplomarbeit" (think Master's thesis). As part of my work > I'm trying to figure out in which "environment" a P2P application, > especially P2P backup has to operate, in order to evaluate whether P2P > backup is possible on a global scale (internet), or rather confined to > scenarios like corporate LANs. To do so, I'm currently running a survey, > trying to gather data from as many volunteer's machines as possible, > in order to do statistics on things like: free hard disk space (total/used > hard disk space), internet connection speed, distribution of file sizes, > distribution of file modification times, etc. Something I noticed when running the survey is that the file size distribution is skewed towards small files by all the source trees (I did not backup those) I have on the disk. You should probably account for this if the percentage of developers in the survey group differs significantly from the one in the user base of your intended application. - Markus From jdd at dixons.org Sun Mar 7 13:17:14 2004 From: jdd at dixons.org (Jim Dixon) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Backup Survey... In-Reply-To: Message-ID: <20040307131522.M21204-100000@localhost> On Sun, 7 Mar 2004, Martin LANDERS wrote: > To make things easy for the people running the survey I've written a > Java-application that collects all the neccessary data, and can upload it > to a server. The survey is anonymous. People concerned about security can > check the results out themselves before uploading. > > The survey application is available at: > > http://p2psurvey.ddns.comp.nus.edu.sg The application is supplied as a jar file containing classfiles and .dll files -- but no source. -- Jim Dixon jdd@dixons.org tel +44 117 982 0786 mobile +44 797 373 7881 http://jxcl.sourceforge.net Java unit test coverage http://xlattice.sourceforge.net p2p communications infrastructure From list-p2phack at ruffledpenguin.org Sun Mar 7 20:37:10 2004 From: list-p2phack at ruffledpenguin.org (Adam Lydick) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Backup Survey... In-Reply-To: <1926203734.20040307134738@gmx.net> References: <1926203734.20040307134738@gmx.net> Message-ID: <1078691828.773.8.camel@elessar> On Sun, 2004-03-07 at 04:47, Markus Kern wrote: > Something I noticed when running the survey is that the file size > distribution is skewed towards small files by all the source trees (I > did not backup those) I have on the disk. You should probably account > for this if the percentage of developers in the survey group differs > significantly from the one in the user base of your intended application. > > - Markus On a related note, Martin (and other curious parties) might want to check out the work done by MSR's Farsite team. The did a similar study for their distributed filesystem: http://research.microsoft.com/sn/Farsite/publications.htm "Feasibility of a Serverless Distributed File System Deployed on an Existing Set of Desktop PCs" - Adam From greg at electricrain.com Mon Mar 8 00:29:04 2004 From: greg at electricrain.com (Gregory P. Smith) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] References for using hashes as unique identifiers? In-Reply-To: References: <20040203160155.G78403-100000@localhost> <1077006022.26007.76.camel@monster.omnifarious.org> <4031F23F.6070503@gmx.de> Message-ID: <20040308002904.GI25897@zot.electricrain.com> On Tue, Feb 17, 2004 at 07:11:21AM -0500, Zooko O'Whielacronx wrote: > > Once upon a time there was a discussion about the risk of hash collision in Mojo > Nation on the Mojo Nation devel mailing list. Hal Finney said that such extreme > probabilities were beyond our ability to engineer. I agreed and said that there > was a higher chance that our hash-collision-handling code would be buggy than > that there would be a hash collision. > > Greg Smith wasn't satisfied with this and went ahead and implemented a simple > routine to detect and clean up after a hash collision. Shortly thereafter we > discovered that there was a sporadic bug in this routine which caused it to > trigger sometimes even in the absence of a collision. Rather than fixing the > bug, Greg removed the routine. hehe. i don't remember doing that so i'll have to take your word for it. :) From greg at electricrain.com Mon Mar 8 00:32:09 2004 From: greg at electricrain.com (Gregory P. Smith) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] References for using hashes as unique identifiers? In-Reply-To: <401FAC51.6040000@gmx.de> References: <401FAC51.6040000@gmx.de> Message-ID: <20040308003209.GJ25897@zot.electricrain.com> Your example is a good one. Talking to a friend at archive.org (the repository of the entire web over time): they use md5 hashes to store and address most things on their farm of >800 cheap terabyte servers. On Tue, Feb 03, 2004 at 04:12:33PM +0200, Benja Fallenstein wrote: > Hi, > > does anybody know references for using cryptographic hashes as unique > identifiers for files in very large repositories (think all of the Web)? > The references I've found (e.g. Handbook of Applied Cryptography) don't > talk explicitly about that, but only about applications in message > authentication, and attacks related to that; of course that's related, > but it would be nice to know whether there are references from > cryptology talking explicitly about hashes as unique identifiers in very > large collections of messages. > > Thanks, > - - Benja From markm at caplet.com Mon Mar 8 00:49:19 2004 From: markm at caplet.com (Mark S. Miller) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] References for using hashes as unique identifiers? In-Reply-To: <20040308003209.GJ25897@zot.electricrain.com> References: <401FAC51.6040000@gmx.de> <401FAC51.6040000@gmx.de> Message-ID: <5.2.1.1.2.20040307164136.00af3568@caplet.com> >On Tue, Feb 03, 2004 at 04:12:33PM +0200, Benja Fallenstein wrote: >> does anybody know references for using cryptographic hashes as unique >> identifiers for files in very large repositories (think all of the Web)? >> The references I've found (e.g. Handbook of Applied Cryptography) don't >> talk explicitly about that, but only about applications in message >> authentication, and attacks related to that; of course that's related, >> but it would be nice to know whether there are references from >> cryptology talking explicitly about hashes as unique identifiers in very >> large collections of messages. I believe the concept originated at Xanadu around 1990 or so, but was never implemented or published. AFAIK, it was implemented for the first time at Electric Communities as the "Repository" -- a component of EC Habitats. This was perhaps around 1995. Jim McCoy then took the idea to MojoNation, which is how the rest of the world became exposed to it. Of course, being an obvious idea, it may have been rediscovered many times, so it wouldn't surprise me if there were other independent historical sequences. If anyone knows any, I'd be curious to hear of them. Thanks. -- Text by me above is hereby placed in the public domain Cheers, --MarkM From bradneuberg at yahoo.com Mon Mar 8 08:02:52 2004 From: bradneuberg at yahoo.com (Brad Neuberg) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] References for using hashes as unique identifiers? In-Reply-To: <5.2.1.1.2.20040307164136.00af3568@caplet.com> Message-ID: <20040308080252.12560.qmail@web60702.mail.yahoo.com> > I believe the concept originated at Xanadu around > 1990 or so, but was never > implemented or published. That's an interesting reference to Xanadu. Can you talk more about how and why the Xanadu team came up with this concept in regards to their own vision of a hypertext web? Brad From markm at caplet.com Mon Mar 8 10:49:32 2004 From: markm at caplet.com (Mark S. Miller) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] References for using hashes as unique identifiers? In-Reply-To: <20040308080252.12560.qmail@web60702.mail.yahoo.com> References: <5.2.1.1.2.20040307164136.00af3568@caplet.com> Message-ID: <5.2.1.1.2.20040308014726.0283aae0@caplet.com> At 12:02 AM 3/8/2004 Monday, Brad Neuberg wrote: >> I believe the concept originated at Xanadu around >> 1990 or so, but was never >> implemented or published. > >That's an interesting reference to Xanadu. Can you >talk more about how and why the Xanadu team came up >with this concept in regards to their own vision of a >hypertext web? A link from document X to document Y must somehow designate document Y. It had never occurred to us that the designator should depend on specifying what machine Y is hosted on. That would obviously be insane. Documents must survive longer than their hosting machine and their hosting organization. Besides, popular documents would make for hotspots in the network -- an obviously unscalable architecture. The goals of the Xanadu project had long been to make a decentralized secure medium, robust against attack, including censorship attempts by governments, and able to scale to hold mankind's literature. We all felt that large scale electronic publishing was coming, expected and hoped that it would be hypertext, and realized that this transition would make mankind's literature either much more vulnerable or much less vulnerable to abuse, depending on the architecture. One of our slogans: "We want to build a medium Thomas Jefferson would be proud of." The architecture now known as Udanax Gold http://udanax.com/gold/index.html http://www.sunless-sea.net/forum http://www.caplet.com/papers/open-media.html has two concepts corresponding to what we mean by "document": The "work" and the "edition". A link can link to either kind of thing. The version of Udanax Gold that was actually built was not distributed, and so did not need a cryptographic basis for its integrity (or other security) properties. But it was designed to have a semantics that could be preserved by a decentralized implementation, where the decentralized implementation would necessarily depend on cryptographic controls to enforce the integrity specified by the semantics, in order to prevent abuse. An edition is an immutable snapshot of the contents of a document as of some moment. We planned to eventually designate an edition by the cryptohash of its contents. Nevermind that we made other decisions that would have made this impossible -- we didn't notice the conflicts at the time. Though we didn't yet know Zooko's phrase "self-authenticating designator", we had the concept, and we thought we were on a course to achieve it. A work is a mutable designator of a particular edition. It represents the notion of a living document, as in "the current rev of the biz plan". To read a work is to obtain access to its current edition. To revise a work is to set its current edition to an edition you specify. We planned to represent a work by a pair of a signing key and a signature verification key. The work would be designated by the signature verification key (or a fingerprint of this key). The right to revise a work would be based on knowledge of the signing key. To revise a work is to sign the hash of an edition (plus some meta info, perhaps identifying predecessor editions, I don't remember) as being an edition of this work, and to make the resulting signature available. Note that both editions and signed revision records can be replicated freely and are location independent. Within just these notions, one can't reliably read the *current* state of a work. Rather, the best you can do is to get the most recent of those you could find. We expected to have something like a netnews flooding algorithm, so that this would normally be recent enough for many purposes. In addition, we did have a location dependent notion of a work's current true home -- its hosting machine. To reliably get the work's *current* edition required a round trip to that machine, and the notion of current was still subject to abuse by the proprietors of that machine. We didn't like it, but the only way we knew to avoid this kind of per-work centralization was to give up the notion of "current", which we were unwilling to do. In any case, I'm proud to say that most of the notions stated above, including (I think) these same cryptographic integrity controls, are now part of OpenCM and used for decentralized high assurance configuration management. http://opencm.org/ This set of ideas actually works together much better than we had any right to expect ;). But the price of "current" remains location dependence. This constraint is even harder to escape than we'd feared. The only escape I know of is quorum systems of some kind (such as http://szabo.best.vwh.net/securetitle.html ), and, IMHO, these may still be too complex to be practical. -- Text by me above is hereby placed in the public domain Cheers, --MarkM From zooko at zooko.com Mon Mar 8 12:57:22 2004 From: zooko at zooko.com (Zooko O'Whielacronx) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] References for using hashes as unique identifiers? In-Reply-To: Message from "Mark S. Miller" of "Sun, 07 Mar 2004 16:49:19 PST." <5.2.1.1.2.20040307164136.00af3568@caplet.com> References: <401FAC51.6040000@gmx.de> <401FAC51.6040000@gmx.de> <5.2.1.1.2.20040307164136.00af3568@caplet.com> Message-ID: Mark Miller wrote: > > Jim McCoy then took the idea to MojoNation, which is how the rest of the > world became exposed to it. > > Of course, being an obvious idea, it may have been rediscovered many times, > so it wouldn't surprise me if there were other independent historical > sequences. If anyone knows any, I'd be curious to hear of them. Thanks. Mojo Nation wasn't the only vector, and probably not most important one. The Freenet white paper in 2000 [1] was probably a very important vector. citeseer listed that paper as #9 in "Most cited source documents published in 2000 in the CiteSeer database as of May 2003" [2]. Unfortunately that paper doesn't cite any prior work on the concept of what they call "Content Hash Keys", nor does it assert that the idea is novel in that paper. Eric Hughes filed patent #6,122,372 in 1997 that included the idea of using a hash of a message as an ID of that message. I worked with Eric at his company in the fall of 1999, and with Jim at Mojo Nation starting in 2000, but I recall talking to Jim in the summer of 1999 and hearing about his scheme for a global, decentralized, attack-resistant file system, which included the notion of using secure hashes as IDs. I think there are two related notions that we might be talking about here: cryptographic hashes as secure, independently-computed unique identifiers, and using independently-computed unique identifiers as locators. If we're just talking about using a hash value as a unique identifier, then that's pretty much what cryptographic hashes were invented for! The mathematical underpinings of the concept of a one-way hash function were first explicated by Merkle [3] in 1979 (according to [4], section 9.2), and keyed MACs was also discussed in the 70's (also according to [4]). The common use of a secure hash in crypto is to provide a smaller string that is used as a secure identifier of a larger string in order to compute public key crypto on the smaller string. ([4] says that this idea was published by Rabin in 1978.) The Chord File System paper (2001) [5] says "The use of content-hashes to securely link together different pieces of data is due to Merkle." -- citing Merkle's hash trees paper from 1987 [6]. If we're talking about using independently-computed unique identifiers as locators, then I guess that's what hashes were invented for! [4] tells me that Knuth, pp.506-549, [7] traces the origins of hash functions back to IBM 1953. So maybe what we're talking about is combining these two concepts, and more-over applying them to distributed or even decentralized systems. One milestone in my opinion was Karger et al. [8], who provided the formalization of the notion that if you use hashes to map to servers, and the set of servers changes, then a "consistent hash" is one that minimizes the number of mappings that have to be changed. Then there is the paper by Plaxton et al., 1999 [9], which is the grandfather of the crop of exciting "peer-to-peer" publications at the beginning of the new century. In [9], they simply say "Assign each object an n-bit id, chosen uniformly at random". So currently, if I were forced to choose a paper to cite as "the earliest publication of the concept of using a secure hash as a locator", I would choose the Freenet paper [1]. Regards, Zooko [1] "Freenet: A Distributed Anonymous Information Storage and Retrieval System" Clarke, Sandberg, Wiley, Hong 2000 http://freenetproject.org/freenet.pdf [2] http://citeseer.nj.nec.com/source-2000.html http://216.239.41.104/search?q=cache:aet9kSevkbEJ:citeseer.nj.nec.com/source-2000.html+freenet+most+cited+paper+citeseer&hl=en&ie=UTF-8 [3] "Secrecy Authentication and Public Key Systems" UMI Research Press, Merkle 1979 [4] "Handbook of Applied Cryptography" Menezes, van Oorschot, Vanstone 1997 http://www.cacr.math.uwaterloo.ca/hac/ [5] "Wide-area cooperative storage with CFS" Dabek, Kaashoek, Karger, Morris, Stoica ACM SOSP 2001 [6] "A digital signature based on a conventional encryption function" Merkle Crypto '87 1987 [7] "The Art of Computer Programming -- Sorting and Searching, vol. 3" Knuth 1973 [8] "Consistent Hashing and Random Trees" Karger, Lehman, Leighton, Levine, Lewin, Panigraphy 1997 [9] "Accessing nearby copies of replicated objects in a distributed environment" Plaxton, Rajaraman, Richa. Theory of Computing Systems, 32:241-280, 1999. From p2p-hackers at strufe.de Mon Mar 8 14:18:37 2004 From: p2p-hackers at strufe.de (Thorsten Strufe) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Backup Survey... In-Reply-To: <20040307131522.M21204-100000@localhost> References: <20040307131522.M21204-100000@localhost> Message-ID: <404C80BD.6080700@strufe.de> | The application is supplied as a jar file containing classfiles and | .dll files -- but no source. _and_ there are neither links to the research-project nor the university or the research-department...? Many people are unsure about executing unknown programs from the internet without having context-information... ;-) best, thorsten From zooko at zooko.com Mon Mar 8 14:31:27 2004 From: zooko at zooko.com (Zooko O'Whielacronx) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] References for using hashes as unique identifiers? In-Reply-To: Message from "Mark S. Miller" of "Mon, 08 Mar 2004 02:49:32 PST." <5.2.1.1.2.20040308014726.0283aae0@caplet.com> References: <5.2.1.1.2.20040307164136.00af3568@caplet.com> <5.2.1.1.2.20040308014726.0283aae0@caplet.com> Message-ID: MarkM wrote: > > Though we didn't > yet know Zooko's phrase "self-authenticating designator", we had the > concept, and we thought we were on a course to achieve it. *My* phrase? I thought I learned that idea -- and that phrase -- from you! Perhaps we came up with the phrase in conversation together. In particular, when I published (on my web site) in October 2001 the first version of "Names: Decentralized, Secure, Human-Meaningful: Choose Two" [1], I cited your "Pet Names Markup Language" [2], Mazieres and Kaashoek's "Self-Certifying File System" (1998) [3], and Freenet (2000) [4]. I also wrote in the acknowledgements: "Thanks to Mark Miller for teaching me about the important distinction between self-authenticating and non-self-authenticating key-value pairs and for helpful comments on an earlier draft of this article." Regards, Zooko [1] "Names: Decentralized, Secure, Human-Meaningful: Choose Two" O'Whielacronx http://zooko.com/distnames.html http://web.archive.org/web/20011020191610/http://zooko.com/distnames.html [2] "Lambda for Humans -- The Pet Name Markup Language" Miller http://www.erights.org/elib/capability/pnml.html [3] "Escaping the evils of centralized control with self-certifying pathnames" Mazieres, Kaashoek 8th ACM SIGOPS European Workshop 1998 [4] "Freenet: A Distributed Anonymous Information Storage and Retrieval System" Clarke, Sandberg, Wiley, Hong 2000 http://freenetproject.org/freenet.pdf From adam at cypherspace.org Mon Mar 8 15:01:39 2004 From: adam at cypherspace.org (Adam Back) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] References for using hashes as unique identifiers? In-Reply-To: <5.2.1.1.2.20040307164136.00af3568@caplet.com> References: <401FAC51.6040000@gmx.de> <401FAC51.6040000@gmx.de> <5.2.1.1.2.20040307164136.00af3568@caplet.com> Message-ID: <20040308150139.GA8961@bitchcake.off.net> Re rediscovery, eternity USENET [1] (1997) also used location independent URLs. The URLs where defined on non-existant TLD .eternity; the implementation used USENET as an anonymous broadcast (remailer -> USENET) system to distribute content. I Note the Xanadu folks also planned a USENET style flood ... same again, except my reason for using USENET was it's own censor resistance arising from it's distributed nature, plus existing anonymous broadcast support (remailers -> USENET). The actual _URL_ was hashed to arrive at a document tag (rather than the document being hashed). The original version of a document could also optionally contain a signing key, and subsequent versions could be updated using that key. The user could therefore prevent coerced revision by signing and then destroying the signing private key. Also had support for encrypted content so that only readers in a group could read that content. On top of that my thoughts (not implemented) for later features included some distributed / p2p system to allow eternity stores to talk to each other so that: - they could reach data that they did not see, - and so that the document space could scale beyond what single nodes could hold without manual partitioning -- by implementing random cache-replacement policy, and having nodes able to resolve data in each other's caches. Note that latter point I call over-cacheing, and is something I have not seen in p2p systems to date -- aim for each node to fill up it's storage space with a random stripe of data -- makes optimal use of storage which is cheap and plentiful, and trades off extra storage for faster, more local, less slow, multi-network hop DHT style lookups. I see Zooko / others also discussed using self-authenticating URLs, another system doing that was wax / eternal resource locator. Adam [1] http://www.cypherspace.org/eternity/ [2] http://www.cl.cam.ac.uk/ftp/users/rja14/wax.ps.gz [3] http://www.cl.cam.ac.uk/ftp/users/rja14/erl3.ps.gz [2,3] http://www.cl.cam.ac.uk/ftp/users/rja14/ On Sun, Mar 07, 2004 at 04:49:19PM -0800, Mark S. Miller wrote: > I believe the concept originated at Xanadu around 1990 or so, but was never > implemented or published. AFAIK, it was implemented for the first time at > Electric Communities as the "Repository" -- a component of EC Habitats. This > was perhaps around 1995. Jim McCoy then took the idea to MojoNation, which > is how the rest of the world became exposed to it. > > Of course, being an obvious idea, it may have been rediscovered many times, > so it wouldn't surprise me if there were other independent historical > sequences. If anyone knows any, I'd be curious to hear of them. Thanks. From cefn.hoile at bt.com Mon Mar 8 16:50:13 2004 From: cefn.hoile at bt.com (cefn.hoile@bt.com) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Placement available Message-ID: <21DA6754A9238B48B92F39637EF307FD0218076E@i2km41-ukdy.domain1.systemhost.net> Please feel free to forward to other individuals or relevant mailing lists. >>>>>>>>>>>>>>>> Vacancy in the BT Pervasive Systems Laboratory Closing date: 1st April 2004 Contact: Dr Robert Ghanea-Hercock, robert@cefn.com Student Placement: Agent Systems Research The Pervasive Systems Laboratory has an internship available as part of a research project in large-scale agent-based network systems. The project is focused on pervasive and secure agent-based systems. This position is part of a new and expanding project to develop cutting edge applications and novel solutions for BT. Among topics to be investigated are: * Peer-to-Peer and Grid type networks * Automated Service Discovery and Adaptive Behaviour Applicants should possess the following essential skill set: Required MSc or PhD in Computer Science, Agents, AI, or related discipline. Strong Java coding experience Preferred Software agent development experience or Web Server/Application development skills In the first instance a full CV and cover letter should be sent to: Dr Robert Ghanea-Hercock BTexact Intelligent Systems Laboratory Adastral Park, MLB1, pp12, Martlesham Heath, Ipswich, IP5 3RE, Suffolk, UK. Email: robert@cefn.com From b.fallenstein at gmx.de Mon Mar 8 16:55:01 2004 From: b.fallenstein at gmx.de (Benja Fallenstein) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] References for using hashes as unique identifiers? In-Reply-To: <5.2.1.1.2.20040308014726.0283aae0@caplet.com> References: <5.2.1.1.2.20040307164136.00af3568@caplet.com> <5.2.1.1.2.20040308014726.0283aae0@caplet.com> Message-ID: <404CA565.1080403@gmx.de> Mark S. Miller wrote: > At 12:02 AM 3/8/2004 Monday, Brad Neuberg wrote: >>That's an interesting reference to Xanadu. I agree. I was completely unaware that Xu ever used hashes as identifiers. (I do remember Ted talking about "Merkle hash enfilades" somewhere, which never made sense to me because WIDative enfilade operations are supposed to be horizontally associative, which hashing definitely is not?) >>Can you >>talk more about how and why the Xanadu team came up >>with this concept in regards to their own vision of a >>hypertext web? > > A link from document X to document Y must somehow designate document Y. It > had never occurred to us that the designator should depend on specifying > what machine Y is hosted on. That would obviously be insane. Documents must > survive longer than their hosting machine and their hosting organization. > Besides, popular documents would make for hotspots in the network -- an > obviously unscalable architecture. But the address exposed at least to the client in Udanax Green is the tumbler, which contains the hosting organization if not the machine -- how did you plan to securely map those to the location-independent cryptographic identifiers? > The architecture now known as Udanax Gold ... > has two concepts corresponding to what we mean by "document": The "work" and > the "edition". A link can link to either kind of thing. Weren't the links to spans of media, and by implication to all works and editions that transclude these spans? This is something I'd really like to understand -- did your system allow linking to a spanset in a *particular* work/edition? The above phrasing seems to imply that. > An edition is an immutable snapshot of the contents of a document as of some > moment. We planned to eventually designate an edition by the cryptohash of > its contents. Nevermind that we made other decisions that would have made this > impossible -- we didn't notice the conflicts at the time. Though we didn't > yet know Zooko's phrase "self-authenticating designator", we had the > concept, and we thought we were on a course to achieve it. Essentially, Freenet's CHK (i.e., hash-based key) and SSK (i.e., signature-based key). How were you going to find who in the network has a copy of the document identified by a particular hash? Flooding broadcast, or something like Freenet, or did you actually develop distributed hashtables back then, too? :-) It's great fun to me to find out about this, since we discovered these concepts independently one more time in 2001 -- and I actually compared them to Xu's work and edition back then, but didn't know that they were based on cryptographic hashes and signatures -- and we actually published a paper at Hypertext'02 called "Freenet-like GUIDs for implementing xanalogical hypertext" [1]. Guess that the fundamental idea wasn't exactly new, then. :-) (Although it's still my understanding that our approach is significantly simpler to implement than an enfilade- or ent-based system, without loss of user-level functionality, and I believe that the efficiency loss doesn't make the system unworkable.) [1] http://portal.acm.org/citation.cfm?id=513386&dl=ACM&coll=portal IIUC, Xu also had the notion of a work pointing to another work as its current edition, for example "Current issue of journal X" -> "January issue of journal X" -> "Edition n of January issue" (allowing for edits of the January issue). > We planned to represent a work by a pair of a signing key and a signature > verification key. The work would be designated by the signature verification > key (or a fingerprint of this key). The right to revise a work would be > based on knowledge of the signing key. To revise a work is to sign the hash > of an edition (plus some meta info, perhaps identifying predecessor > editions, I don't remember) as being an edition of this work, and to make > the resulting signature available. Note that both editions and signed > revision records can be replicated freely and are location independent. How did you intend to solve the problem of forgotten / compromised keys? > Within just these notions, one can't reliably read the *current* state of a > work. Rather, the best you can do is to get the most recent of those you > could find. We expected to have something like a netnews flooding algorithm, > so that this would normally be recent enough for many purposes. ... > This set of ideas actually works together much better than we had any right > to expect ;). But the price of "current" remains location dependence. This > constraint is even harder to escape than we'd feared. The only escape I know > of is quorum systems of some kind (such as > http://szabo.best.vwh.net/securetitle.html ), and, IMHO, these may still be > too complex to be practical. OceanStore does this, too. Our idea is to use a DHT to find the most current version available on the network (of course then there's the problem of data hiding attacks in DHTs). The idea is that if there's anybody (including the publisher) interested in users finding the newest version, they keep it alive in the DHT for clients to find. It may not always be perfect, but it seems good enough to us -- especially with the tradeoff against location-dependence. - Benja From gojomo at bitzi.com Mon Mar 8 17:09:18 2004 From: gojomo at bitzi.com (Gordon Mohr) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] References for using hashes as unique identifiers? In-Reply-To: <20040308150139.GA8961@bitchcake.off.net> References: <401FAC51.6040000@gmx.de> <401FAC51.6040000@gmx.de> <5.2.1.1.2.20040307164136.00af3568@caplet.com> <20040308150139.GA8961@bitchcake.off.net> Message-ID: <404CA8BE.7040005@bitzi.com> Search for "content-derived names" and you'll find a couple of papers in 97 and 98 which sing the praises of hash-based file-naming. MD5 is also listed as one of the possibilities for assigning URN names, an URN being "a name with global scope which does not imply a location," in 1994's RFC 1737, "Functional Requirements for Uniform Resource Names." Larry Masinter, one of the RFC 1737 coauthors, also casually refers to the technique in his 1995 article, "Document Management, Digital Libraries and the Web", and mentions a contemporaneous system ("LIFN") using hashes to create resource names. I've also heard anecdotally of use in configuration management, caching, or information-product distribution in the early 1990s or late 1980s. - Gordon From markm at caplet.com Mon Mar 8 17:32:37 2004 From: markm at caplet.com (Mark S. Miller) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] References for using hashes as unique identifiers? In-Reply-To: References: <5.2.1.1.2.20040307164136.00af3568@caplet.com> <401FAC51.6040000@gmx.de> <401FAC51.6040000@gmx.de> <5.2.1.1.2.20040307164136.00af3568@caplet.com> Message-ID: <5.2.1.1.2.20040308091202.026d6b58@caplet.com> At 04:57 AM 3/8/2004 Monday, Zooko O'Whielacronx wrote: >I think there are two related notions that we might be talking about here: >cryptographic hashes as secure, independently-computed unique identifiers, and >using independently-computed unique identifiers as locators. Our notion at Xanadu was to use the combined idea. I think the important point, though, is relating this to version control notions. By making each immutable edition of a document a first class designatable document, and my considering a commit to just be the setting of a mutable work to point at its current edition, we got to hash the contents of the editions themselves and to replicate them freely without synchronization issues. All the synchronization issues were localized to reading and writing the work. All non-trivial access control was also on the work -- it makes no sense to revoke access to an edition, since an edition is morally equivalent to revealed bits. >If we're just talking about using a hash value as a unique identifier, then >that's pretty much what cryptographic hashes were invented for! The >mathematical underpinings of the concept of a one-way hash function were first >explicated by Merkle [3] in 1979 (according to [4], section 9.2), and keyed >MACs was also discussed in the 70's (also according to [4]). > >The common use of a secure hash in crypto is to provide a smaller string that >is used as a secure identifier of a larger string in order to compute public >key crypto on the smaller string. ([4] says that this idea was published by >Rabin in 1978.) > >The Chord File System paper (2001) [5] says "The use of content-hashes to >securely link together different pieces of data is due to Merkle." -- citing >Merkle's hash trees paper from 1987 [6]. We were aware of Merkle's work. I'm sure it had some influence. >If we're talking about using independently-computed unique identifiers as >locators, then I guess that's what hashes were invented for! [4] tells me that >Knuth, pp.506-549, [7] traces the origins of hash functions back to IBM 1953. > > >So maybe what we're talking about is combining these two concepts, and >more-over applying them to distributed or even decentralized systems. One >milestone in my opinion was Karger et al. [8], who provided the formalization >of the notion that if you use hashes to map to servers, and the set of servers >changes, then a "consistent hash" is one that minimizes the number of mappings >that have to be changed. Then there is the paper by Plaxton et al., 1999 [9], >which is the grandfather of the crop of exciting "peer-to-peer" publications at >the beginning of the new century. In [9], they simply say "Assign each object >an n-bit id, chosen uniformly at random". > >So currently, if I were forced to choose a paper to cite as "the earliest >publication of the concept of using a secure hash as a locator", I would choose >the Freenet paper [1]. We did *not*, at that time, planning to use the DHT concept. Specifically, we were planning to use a computed hash as a unique self authenticating designator, but not to compute an index into a server-as-hash-bucket. The first invention of the DHT concept I'm aware of is in Robin Hanson's LinkText proposal: >Composite Places The above rule can be thought of as the managing of a >"global" place, which includes everything but has the same semantics as any >local place. The same technique can be used to manage a composite place >called "Seattle" composed of all the public places in Seattle. A hash >function could be used to assign a "gate" place within Seattle to every >work. If every work within Seattle is placed at its gate and the gate of >every work it references, then backfollowing will work within the boundary >of Seattle. The overhead for this can be substantial, but is small if most >works in Seattle have their home in Seattle. Places should be run by a >single responsible organization with a specific mail address, except that >the global place cannot be. Homes must be atomic (not composite) places. from "Toward Hypertext Publishing: Issues And Choices In Database Design", 1987. Online at http://hanson.gmu.edu/linktext.html . We were aware of Robin's ideas at Xanadu, but couldn't figure out how to make it usable with dynamic entry and exit of places from the set of hash buckets. Especially when the alleged places were mutually suspicious. -- Text by me above is hereby placed in the public domain Cheers, --MarkM From markm at caplet.com Mon Mar 8 17:34:38 2004 From: markm at caplet.com (Mark S. Miller) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] References for using hashes as unique identifiers? In-Reply-To: References: <5.2.1.1.2.20040308014726.0283aae0@caplet.com> <5.2.1.1.2.20040307164136.00af3568@caplet.com> <5.2.1.1.2.20040308014726.0283aae0@caplet.com> Message-ID: <5.2.1.1.2.20040308093312.0283aae0@caplet.com> At 06:31 AM 3/8/2004 Monday, Zooko O'Whielacronx wrote: > MarkM wrote: >> >> Though we didn't >> yet know Zooko's phrase "self-authenticating designator", we had the >> concept, and we thought we were on a course to achieve it. > >*My* phrase? I thought I learned that idea -- and that phrase -- from you! We did have the idea. I don't remember having had the phrase. >Perhaps we came up with the phrase in conversation together. Sounds likely. -- Text by me above is hereby placed in the public domain Cheers, --MarkM From markm at caplet.com Mon Mar 8 18:07:53 2004 From: markm at caplet.com (Mark S. Miller) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] References for using hashes as unique identifiers? In-Reply-To: <404CA565.1080403@gmx.de> References: <5.2.1.1.2.20040308014726.0283aae0@caplet.com> <5.2.1.1.2.20040307164136.00af3568@caplet.com> <5.2.1.1.2.20040308014726.0283aae0@caplet.com> Message-ID: <5.2.1.1.2.20040308094343.02d4a0d8@caplet.com> At 08:55 AM 3/8/2004 Monday, Benja Fallenstein wrote: >Mark S. Miller wrote: >>At 12:02 AM 3/8/2004 Monday, Brad Neuberg wrote: >>>That's an interesting reference to Xanadu. > >I agree. I was completely unaware that Xu ever used hashes as identifiers. I hope I was clear that we never actually implemented this. We simply constrained our semantics to be compatible with our future plans for cryptographic integrity controls. Or so we thought. (It's clear to me now that we hadn't succeeded at this.) >(I do remember Ted talking about "Merkle hash enfilades" somewhere, which >never made sense to me because WIDative enfilade operations are supposed to >be horizontally associative, which hashing definitely is not?) You are correct -- they don't make sense. However, I've recently heard about a paper (citation?) analyzing use of non-truncated addition as a secure associative-commutative hash-combining operator. Does anyone know of a secure associative-but-not-commutative hash combining operator? Such a thing would be more interesting for some uses. In any case, we had no *sensible* ideas along those lines at Xanadu. >>>Can you >>>talk more about how and why the Xanadu team came up >>>with this concept in regards to their own vision of a >>>hypertext web? >>A link from document X to document Y must somehow designate document Y. It had never occurred to us that the designator should depend on specifying what machine Y is hosted on. That would obviously be insane. Documents must survive longer than their hosting machine and their hosting organization. Besides, popular documents would make for hotspots in the network -- an obviously unscalable architecture. > >But the address exposed at least to the client in Udanax Green is the >tumbler, which contains the hosting organization if not the machine -- how >did you plan to securely map those to the location-independent cryptographic >identifiers? The best schemes we had for tumblers were no better than our current nightmare of CA hierarchies. That was among the reasons that Udanax Gold dispensed with the tumbler concept and went to opaque designators. Tumblers were well loved by some. The decision to kill them was quite controversial. >>The architecture now known as Udanax Gold >... >>has two concepts corresponding to what we mean by "document": The "work" and the "edition". A link can link to either kind of thing. > >Weren't the links to spans of media, and by implication to all works and >editions that transclude these spans? > >This is something I'd really like to understand -- did your system allow >linking to a spanset in a *particular* work/edition? The above phrasing >seems to imply that. Yes it did. However, from your phrasing, I can see that you're referring to Udanax Green concepts. In Udanax Green, the answer was also yes, but in a messier way. >>An edition is an immutable snapshot of the contents of a document as of some moment. We planned to eventually designate an edition by the cryptohash of its contents. Nevermind that we made other decisions that would have made this impossible -- we didn't notice the conflicts at the time. Though we didn't yet know Zooko's phrase "self-authenticating designator", we had the concept, and we thought we were on a course to achieve it. > >Essentially, Freenet's CHK (i.e., hash-based key) and SSK (i.e., >signature-based key). > >How were you going to find who in the network has a copy of the document >identified by a particular hash? Flooding broadcast, or something like >Freenet, or did you actually develop distributed hashtables back then, too? :-) We considered this a hard problem. We were aware of Robin Hanson's ideas towards DHTs, but couldn't figure out how to make it practical under dynamic entry and exit. We did not consider full flooding, as we did not imagine a world with enough disk space for that. Instead, we postponed that problem by first building a non-distributed server. Our vague sense was to use market pricing to adaptively move replicas towards their locations of greater demand. Indeed, a conversation with Phil Salin around 1980 or so, about use of markets to manage adaptive replication for Xanadu, was one of the crucial aha's that led to the agoric open systems papers. http://www.agorics.com/Library/agoricpapers.html Mariposa http://s2k-ftp.cs.berkeley.edu:8000/mariposa/ shows that the idea may have some merit. >It's great fun to me to find out about this, since we discovered these >concepts independently one more time in 2001 -- and I actually compared them >to Xu's work and edition back then, but didn't know that they were based on >cryptographic hashes and signatures -- and we actually published a paper at >Hypertext'02 called "Freenet-like GUIDs for implementing xanalogical >hypertext" [1]. Guess that the fundamental idea wasn't exactly new, then. >:-) (Although it's still my understanding that our approach is significantly >simpler to implement than an enfilade- or ent-based system, without loss of >user-level functionality, and I believe that the efficiency loss doesn't >make the system unworkable.) > >[1] http://portal.acm.org/citation.cfm?id=513386&dl=ACM&coll=portal Cool! We did indeed *WAY* overestimate the need for efficiency, and *WAY* underestimate the cost of designing fancy data structures. It was a mistake to try to engineer the ent on the way to 1.0. Better to get something inefficient out quickly, and then worry about efficiency later. But I didn't understand this at the time. >IIUC, Xu also had the notion of a work pointing to another work as its >current edition, for example "Current issue of journal X" -> "January issue >of journal X" -> "Edition n of January issue" (allowing for edits of the >January issue). I don't believe so. >>We planned to represent a work by a pair of a signing key and a signature verification key. The work would be designated by the signature verification key (or a fingerprint of this key). The right to revise a work would be based on knowledge of the signing key. To revise a work is to sign the hash of an edition (plus some meta info, perhaps identifying predecessor editions, I don't remember) as being an edition of this work, and to make the resulting signature available. Note that both editions and signed revision records can be replicated freely and are location independent. > >How did you intend to solve the problem of forgotten / compromised keys? I don't remember. I'm not sure we had a sensible idea for this at the time. >>Within just these notions, one can't reliably read the *current* state of a work. Rather, the best you can do is to get the most recent of those you could find. We expected to have something like a netnews flooding algorithm, so that this would normally be recent enough for many purposes. >... >>This set of ideas actually works together much better than we had any right to expect ;). But the price of "current" remains location dependence. This constraint is even harder to escape than we'd feared. The only escape I know of is quorum systems of some kind (such as http://szabo.best.vwh.net/securetitle.html ), and, IMHO, these may still be too complex to be practical. > >OceanStore does this, too. > >Our idea is to use a DHT to find the most current version available on the >network (of course then there's the problem of data hiding attacks in DHTs). >The idea is that if there's anybody (including the publisher) interested in >users finding the newest version, they keep it alive in the DHT for clients >to find. It may not always be perfect, but it seems good enough to us -- >especially with the tradeoff against location-dependence. Cool. -- Text by me above is hereby placed in the public domain Cheers, --MarkM From bkn3 at columbia.edu Tue Mar 9 03:31:56 2004 From: bkn3 at columbia.edu (Brad Neuberg) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] First use of FEC in P2P Systems? In-Reply-To: <404CA8BE.7040005@bitzi.com> References: <401FAC51.6040000@gmx.de> <401FAC51.6040000@gmx.de> <5.2.1.1.2.20040307164136.00af3568@caplet.com> <20040308150139.GA8961@bitchcake.off.net> <404CA8BE.7040005@bitzi.com> Message-ID: <6.0.1.1.2.20040308193057.01e9f908@pop.mail.yahoo.com> Who first had the idea of applying Forward Error Correction (FEC) to increase reliability in peer-to-peer systems? Brad From justin at chapweske.com Tue Mar 9 10:03:01 2004 From: justin at chapweske.com (Justin Chapweske) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] First use of FEC in P2P Systems? In-Reply-To: <6.0.1.1.2.20040308193057.01e9f908@pop.mail.yahoo.com> References: <401FAC51.6040000@gmx.de> <401FAC51.6040000@gmx.de> <5.2.1.1.2.20040307164136.00af3568@caplet.com> <20040308150139.GA8961@bitchcake.off.net> <404CA8BE.7040005@bitzi.com> <6.0.1.1.2.20040308193057.01e9f908@pop.mail.yahoo.com> Message-ID: <1078826579.2967.310.camel@bog> Well it depends on your definition of P2P, but the first usage of FEC in a modern "P2P" system that I am aware of was when I invented swarming downloads. FEC was used in Swarmcast to minimize data dissemination time over extremely unstable network topologies, though its primary focus was not on increasing reliability it is a side effect. (As an aside, I'll admit that our initial design was more concerned with theoretical worst-case scenarios rather than practical scenarios and was thus overly conservative in its design). The prior art that inspired my work was the use of FEC in reliable multicast and carousel distribution systems which was largely driven by Luigi Rizzo in 1997. In the sense that his work could be adapted to application-level or P2P multicast, the research would still be relevant, but it did not include the concept of a simultaneous many-to-one and one-to-many topology that is minimally required for swarming downloads. But, if you're just looking at increasing reliability and not Swarming, then I'd be very surprised if Shamir, with his secret sharing algorithm in '79, didn't at least contemplate such a use. Probably the best hint for an initial use would be when people started thinking about the Binary Erasure Channel as opposed to the Binary Symmetric Channel. I think Publius used Shamir's algorithm, anyone know when that was introduced? -Justin On Mon, 2004-03-08 at 21:31, Brad Neuberg wrote: > Who first had the idea of applying Forward Error Correction (FEC) to > increase reliability in peer-to-peer systems? > > Brad > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From tutschku at informatik.uni-wuerzburg.de Tue Mar 9 10:23:49 2004 From: tutschku at informatik.uni-wuerzburg.de (Kurt Tutschku) Date: Sat Dec 9 22:12:38 2006 Subject: AW: [p2p-hackers] First use of FEC in P2P Systems? In-Reply-To: <6.0.1.1.2.20040308193057.01e9f908@pop.mail.yahoo.com> Message-ID: <002201c405c0$9cc06250$70924cc0@musa> Hi Brad, you might have a look at: "OverQoS: An Overlay based Architecture for Enhancing Internet QoS" by L. Subramanian, I. Stoica, H. Balakrishnan, R. Katz They apply FEC for something similar: controlled loss virtual links. Cheers Kurt > -----Urspr?ngliche Nachricht----- > Von: p2p-hackers-bounces@zgp.org > [mailto:p2p-hackers-bounces@zgp.org] Im Auftrag von Brad Neuberg > Gesendet: Dienstag, 9. M?rz 2004 04:32 > An: Peer-to-peer development. > Betreff: [p2p-hackers] First use of FEC in P2P Systems? > > > Who first had the idea of applying Forward Error Correction (FEC) to > increase reliability in peer-to-peer systems? > > Brad > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/P> eerToPeerConferences > From martinla at comp.nus.edu.sg Tue Mar 9 15:23:54 2004 From: martinla at comp.nus.edu.sg (Martin LANDERS) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Backup Survey... In-Reply-To: <20040307131522.M21204-100000@localhost> References: <20040307131522.M21204-100000@localhost> Message-ID: Hi, > > The survey application is available at: > > > > http://p2psurvey.ddns.comp.nus.edu.sg > > The application is supplied as a jar file containing classfiles and > .dll files -- but no source. Good point. The source now is available at the home page (see Notes). I'm well aware of the distrust in binary-only applications you're supposed to run on your computer, and in fact I was amazed that you were the first person to "complain". The reason, for not publishing the source code right away is, that I'm not sure whether parts might be covered by patents, etc. Nontheless, the source code is online now, and I hope this removes the (legitimate) distrust in the survey application somewhat. Thanks for giving me the final push in the right direction, Martin From martinla at comp.nus.edu.sg Tue Mar 9 15:32:34 2004 From: martinla at comp.nus.edu.sg (Martin LANDERS) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Backup Survey... In-Reply-To: <404C80BD.6080700@strufe.de> References: <20040307131522.M21204-100000@localhost> <404C80BD.6080700@strufe.de> Message-ID: Hi Thorsten, > | The application is supplied as a jar file containing classfiles and > | .dll files -- but no source. > > _and_ there are neither links to the research-project nor the university > or the research-department...? Another valid concern. I've added a plethora of links to the universities and professors involved in the project, to give the whole thing a bit of "context". I must admit that the pages on the server are a bit "thrown together", and don't look much like a project page. Also, the research project itself doesn't have any kind of homepage, as it only is a "Diplomarbeit" (master's thesis), and I'd rather finish the work (deadline is the end of april), that write a homepage about it ;) But I admit that I'd also find such a "bare" server, telling me to download this *bells-and-whistles* great application and run it, more than a bit suspicious... I hope the links and available source code alleviate the concerns a bit ;) > Many people are unsure about executing unknown programs from the > internet without having context-information... ;-) Er, yes, and I'm one of them... Interesting how much one's perspective changes when one is on "the other side". I hope the page is better now. I you have further suggestions how I can convince people that I'm not a l33t hax0r kiddie trying to take over their computer, but rather a desperate student looking for nice results to include in my thesis ;) please let me know... Thanks, Martin From martinla at comp.nus.edu.sg Tue Mar 9 16:11:50 2004 From: martinla at comp.nus.edu.sg (Martin LANDERS) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Backup Survey... In-Reply-To: <1078691828.773.8.camel@elessar> References: <1926203734.20040307134738@gmx.net> <1078691828.773.8.camel@elessar> Message-ID: Hi, > http://research.microsoft.com/sn/Farsite/publications.htm > > "Feasibility of a Serverless Distributed File System Deployed on an > Existing Set of Desktop PCs" I'm aware of that study, and it's quite nice. They have a quite a few interesting results (like the average amount of free disk space on all the machines, data on uptime correlation, etc.). The "problem" with the study is, that all of that data has been collected from the computers of Microsoft Employees. This makes the data biased in at least two ways: a) it's a corporate scenario and b) it's (nearly) only Windows machines. Also, the study is more than 2 years old by now, which means downloading movies, etc. from the net was way less popular than it (supposedly) is now. In fact, this study is one of the reasons for my survey. Some of the numbers in there (avg. 50% hard disk space free) seem "too good to be true", which is why I'd like to find out if the results are still valid, when a) collecting data from arbitrary internet users and b) from a number of platforms, today. I guess that the figures will look a lot more grim under these circumstances. Sadly, so far my collected data won't help much, as most of it is highly biased. I figure that over 80% of all answers so far (73, ahem...) are from students here at NUS, making the study a study of Windows machines at a University. That's why I need your help in promoting the study to other users, outside of universities... So, if anyone of you has good ideas how to get other people to do the study (forums, newsgroups? which ones?), I'd love to hear about them. Thanks a lot, cu Martin From mccoy at mad-scientist.com Tue Mar 9 17:14:05 2004 From: mccoy at mad-scientist.com (Jim McCoy) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] First use of FEC in P2P Systems? In-Reply-To: <1078826579.2967.310.camel@bog> References: <401FAC51.6040000@gmx.de> <401FAC51.6040000@gmx.de> <5.2.1.1.2.20040307164136.00af3568@caplet.com> <20040308150139.GA8961@bitchcake.off.net> <404CA8BE.7040005@bitzi.com> <6.0.1.1.2.20040308193057.01e9f908@pop.mail.yahoo.com> <1078826579.2967.310.camel@bog> Message-ID: <2B2881EC-71ED-11D8-B88C-000393071F50@mad-scientist.com> On Mar 9, 2004, at 2:03 AM, Justin Chapweske wrote: > Well it depends on your definition of P2P, but the first usage of FEC > in > a modern "P2P" system that I am aware of was when I invented swarming > downloads. Actually, Digital Fountain and the work of the reliable multicast group predated you by quite a while. > But, if you're just looking at increasing reliability and not Swarming, > then I'd be very surprised if Shamir, with his secret sharing algorithm > in '79, didn't at least contemplate such a use. While Shamir may have contemplated such a use I am not aware of him publishing anything about it. The oldest precedents for the application of error-correction to increase data retrieval reliability probably go back to the RAID work initially done at Berkeley and if you expand the scope slightly to what most of us would consider "p2p" then the earliest such system is probably Zebra, although this was more of a "traditional" RAID network filesystem designed as a distributed set of storage agents. The earliest system that most of us would look at and call "p2p storage" I think would be the Archival Intermemory research from NEC Research in 97 (and Frangipani if you are thinking more of a replicated data layer that error-correction could be applied to...) > I think Publius used Shamir's algorithm, anyone know when that was > introduced? Early 2000, so even Mojo Nation predated this one... (we were using a variant of Shamir's information dispersal algorithm at the time, not FEC :) Jim From justin at chapweske.com Tue Mar 9 18:18:32 2004 From: justin at chapweske.com (Justin Chapweske) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] First use of FEC in P2P Systems? In-Reply-To: <2B2881EC-71ED-11D8-B88C-000393071F50@mad-scientist.com> References: <401FAC51.6040000@gmx.de> <401FAC51.6040000@gmx.de> <5.2.1.1.2.20040307164136.00af3568@caplet.com> <20040308150139.GA8961@bitchcake.off.net> <404CA8BE.7040005@bitzi.com> <6.0.1.1.2.20040308193057.01e9f908@pop.mail.yahoo.com> <1078826579.2967.310.camel@bog> <2B2881EC-71ED-11D8-B88C-000393071F50@mad-scientist.com> Message-ID: <1078856311.2967.357.camel@bog> On Tue, 2004-03-09 at 11:14, Jim McCoy wrote: > On Mar 9, 2004, at 2:03 AM, Justin Chapweske wrote: > > > Well it depends on your definition of P2P, but the first usage of FEC > > in > > a modern "P2P" system that I am aware of was when I invented swarming > > downloads. > > Actually, Digital Fountain and the work of the reliable multicast group > predated you by quite a while. > I never claimed otherwise. If you re-read my message I specifically refer to Luigi Rizzo's (reliable multicast) work as being the inspiration for Swarmcast, which is certainly evidenced by the fact that our FEC library is a direct port of his from C to Java. However, the reliable multicast systems are not typically thought of as P2P and certainly do not feature any kind of swarming. As I said, if one were to extend the reliable multicast work to include application layer/P2P multicast, then that would certainly be considered P2P. However, from what I've seen of the early application layer multicast systems such as Yoid, they did not at that time contemplate the use of FEC-style reliable multicast. > While Shamir may have contemplated such a use I am not aware of him > publishing anything about it. The oldest precedents for the > application of error-correction to increase data retrieval reliability > probably go back to the RAID work initially done at Berkeley and if you > expand the scope slightly to what most of us would consider "p2p" then > the earliest such system is probably Zebra, although this was more of a > "traditional" RAID network filesystem designed as a distributed set of > storage agents. The earliest system that most of us would look at and > call "p2p storage" I think would be the Archival Intermemory research > from NEC Research in 97 (and Frangipani if you are thinking more of a > replicated data layer that error-correction could be applied to...) I had forgotten about Archival Intermemory, and seeing as that was in '97 as well, I'm guessing the synchronicity of the early uses of software-based FEC was simply due to CPU power reaching a sweet spot to enable these applications. I'd guess that Luigi's paper, "On the Feasibility of Software FEC", also played a large role in raising of the awareness of researchers to the possibilities of software-based FEC. As an aside, Hakim Weatherspoon's paper on "Erasure Coding vs. Replication: A Quantitative Comparison" (http://oceanstore.cs.berkeley.edu/publications/papers/pdf/erasure_iptps.pdf) is a good read for anyone wanting to gain a greater appreciation for the performance trade offs when using FEC. Thanks, -Justin From aloeser at cs.tu-berlin.de Fri Mar 12 18:48:47 2004 From: aloeser at cs.tu-berlin.de (aloeser) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Storage Load Balancing in Chord Message-ID: <4052060F.E3B68A51@cs.tu-berlin.de> HI, distributing storage load uniformly among peers until the available storage is used optimally remains a major challenge for current structured P2P systems. Randomization seems to play an important role. What experiences did you made in the Chord network with strategies such as virtual server (One-to One, One-to Many, Many-to-Many)? Which storage load balancing strategies suitable for the CHORD network can you recommend and what experiences did you make? Alex -- ___________________________________________________________ M.Sc., Dipl. Wi.-Inf. Alexander L?ser Technische Universitaet Berlin Fakultaet IV - CIS bmb+f-Projekt: "New Economy, Neue Medien in der Bildung" hp: http://cis.cs.tu-berlin.de/~aloeser/ office: +49- 30-314-25551 fax : +49- 30-314-21601 ___________________________________________________________ From sam at neurogrid.com Fri Mar 12 20:23:22 2004 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Micropayments Message-ID: <40521C3A.5050508@neurogrid.com> Hi, Thought people might be interested in this paper on a micropayement system for P2P: http://www-db.stanford.edu/~byang/pubs/ppay.pdf Particularly x-mojo-nation people who wanted to let the authors know about MNet CHEERS> SAM From sam at neurogrid.com Fri Mar 12 20:29:52 2004 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Re: March issue of P2PJ In-Reply-To: <000501c40325$9655dfa0$927ba8c0@power> References: <000501c40325$9655dfa0$927ba8c0@power> Message-ID: <40521DC0.80905@neurogrid.com> Hi All, Apologies for multiple postings - just in case you hadn't seen the latest issue. CHEERS> SAM Raymond Gao wrote: > Hello everyone, > > > > March 2004 issue of P2PJ is now available online. The URL is > http://p2pjournal.com > > > > Regards, > > > > Ray Gao > Editor-in-Chief, P2P Journal > http://p2pjournal.com > From zooko at zooko.com Fri Mar 12 20:39:11 2004 From: zooko at zooko.com (Zooko O'Whielacronx) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Micropayments In-Reply-To: Message from Sam Joseph of "Sat, 13 Mar 2004 05:23:22 +0900." <40521C3A.5050508@neurogrid.com> References: <40521C3A.5050508@neurogrid.com> Message-ID: > http://www-db.stanford.edu/~byang/pubs/ppay.pdf I'm very interested in this research! It includes the "bilateral accounting scheme" that Doug Barnes and Jim McCoy invented for Mojo Nation (called "soft credit windows" in this paper), plus a very good idea of how to safely off-load the token-verification work from a central server to peers. > Particularly x-mojo-nation people who wanted to let the authors know > about MNet Actually, this has already happened! The authors generously mention Mojo Nation in that paper even though there is no extant documentation describing the economic aspects of Mojo Nation. Regards, Zooko From sam at neurogrid.com Fri Mar 12 20:46:15 2004 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Micropayments In-Reply-To: References: <40521C3A.5050508@neurogrid.com> Message-ID: <40522197.4080306@neurogrid.com> Hi Zooko, Zooko O'Whielacronx wrote: >>http://www-db.stanford.edu/~byang/pubs/ppay.pdf >> >> >I'm very interested in this research! It includes the "bilateral accounting >scheme" that Doug Barnes and Jim McCoy invented for Mojo Nation (called "soft >credit windows" in this paper), plus a very good idea of how to safely off-load >the token-verification work from a central server to peers. > Yes its interesting stuff - starts making me think if there couldn't even be simpler schemes that would allow a two-party interaction to not rely on a third party and avoid the layered coin inefficiencies. Some way of allowing a broker to issue a coin to user U, and then making it so that anyone could re-assign it to another user, but would invalidate their own coin in the process. Although maybe that's impossible. I'd like to spend more time thinking about this, but there's so much else to do! >>Particularly x-mojo-nation people who wanted to let the authors know >>about MNet >> >> >Actually, this has already happened! The authors generously mention Mojo >Nation in that paper even though there is no extant documentation describing >the economic aspects of Mojo Nation. > Right - I wondered if they had talked to you, but what surprised me was that they didn't mention MNet by name, since I was assuming there would be existing MNet documentation they could reference .... CHEERS> SAM From zooko at zooko.com Fri Mar 12 21:02:14 2004 From: zooko at zooko.com (Zooko O'Whielacronx) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Micropayments In-Reply-To: Message from Sam Joseph of "Sat, 13 Mar 2004 05:23:22 +0900." <40521C3A.5050508@neurogrid.com> References: <40521C3A.5050508@neurogrid.com> Message-ID: > Particularly x-mojo-nation people who wanted to let the authors know > about MNet Whoops -- in my previous reply I misread this and thought that you said "wanted to let the authors know about Mojo Nation". The current version of Mnet -- v0.6 [footnote 1] -- contains no economic mechanisms. The network depends upon resources altruistically contributed by the nodes. One particularly appealing use for Mnet v0.6, in my opinion, is "private Mnets", where all of the nodes are operated by your friends. The result will hopefully be sort of like a virtual filespace that you share with your friends. The Mnet FAQ tells how to set up such a network. I also think Mnet v0.6 will be useful in a public, open-access network, even though it lacks economic mechanisms. There is currently one such open Mnet network, which you access by default when you install Mnet. Mnet v0.7, currently under development, might have some economic mechanisms when it is released. One possibility that would be relatively easy for the Mnet hackers to implement would be persistent bilateral accounting. This would be similar to BitTorrent's "tit-for-tat" notion, but persistent. This would hopefully motivate people to contribute reliable nodes to the network, in order to get faster service from the network when they want to use it. We might also try out some more sophisticated economic ideas. Regards, Zooko [1] Due to be released Real Soon Now. Contact Arno Waschk , Captain of Mnet v0.6, for more information. See the issue tracker's "v0.6.2" group for current bugs: http://sourceforge.net/tracker/?group_id=43482&atid=436453 Try out the current release here: http://mnet.sourceforge.net/download.php From markm at caplet.com Sat Mar 13 01:33:38 2004 From: markm at caplet.com (Mark S. Miller) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Micropayments In-Reply-To: References: <40521C3A.5050508@neurogrid.com> <40521C3A.5050508@neurogrid.com> Message-ID: <5.2.1.1.2.20040312173145.00af3508@caplet.com> At 12:39 PM 3/12/2004 Friday, Zooko O'Whielacronx wrote: >> http://www-db.stanford.edu/~byang/pubs/ppay.pdf > >I'm very interested in this research! It includes the "bilateral accounting >scheme" that Doug Barnes and Jim McCoy invented for Mojo Nation (called "soft >credit windows" in this paper), plus a very good idea of how to safely off-load >the token-verification work from a central server to peers. > >[...] The authors generously mention Mojo >Nation in that paper even though there is no extant documentation describing >the economic aspects of Mojo Nation. What's the relationship to between these ideas and Digital Silk Road? http://www.agorics.com/Library/dsr.html -- Text by me above is hereby placed in the public domain Cheers, --MarkM From rah at shipwright.com Sat Mar 13 02:19:19 2004 From: rah at shipwright.com (R. A. Hettinga) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Micropayments In-Reply-To: <5.2.1.1.2.20040312173145.00af3508@caplet.com> References: <40521C3A.5050508@neurogrid.com> <40521C3A.5050508@neurogrid.com> <5.2.1.1.2.20040312173145.00af3508@caplet.com> Message-ID: At 5:33 PM -0800 3/12/04, Mark S. Miller wrote: >What's the relationship to between these ideas and Digital Silk Road? >http://www.agorics.com/Library/dsr.html That's everything. *Everything* is Digital Silk Road. :-) Cheers, RAH -- ----------------- R. A. Hettinga The Internet Bearer Underwriting Corporation 44 Farquhar Street, Boston, MA 02131 USA "... however it may deserve respect for its usefulness and antiquity, [predicting the end of the world] has not been found agreeable to experience." -- Edward Gibbon, 'Decline and Fall of the Roman Empire' From em at em.no-ip.com Sat Mar 13 11:03:17 2004 From: em at em.no-ip.com (Enzo Michelangeli) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Ideas for an opensource Skype lookalike References: <1515.1068322821@www49.gmx.net> <6059.1068460482@www30.gmx.net> Message-ID: <031001c408ea$cf238ec0$0200a8c0@em.noip.com> Hello everybody, I just joined this list after lurking for a while on its archive at http://zgp.org/pipermail/p2p-hackers/. I'd like to gather opinions about using P2P techniques to support a type of application that never managed to become really popular: a secure internet phone. I have recently begun to monitor the development of Speakfreely on Sourceforge (http://speak-freely.sourceforge.net/ ) after its creator John Walker decided that the future of Internet was an inhospitable environment for it and abandoned further development (http://www.fourmilab.ch/speakfree/ ). I think that John overlooked the possibilities offered by P2P architectures, in two critical areas: - Directories for location and presence. Nothing fancy here, already done before for P2P chat systems. - Working around NAT routers. John says of implementing third-party reflectors: "[...] no non-commercial site like mine could possibly afford the unlimited demands on bandwidth that would require. It's one thing to provide a central meeting point like a Look Who's Listening server, which handles a packet every five minutes or so from connected sites, but a server that's required to forward audio in real-time between potentially any number of simultaneously connected users is a bandwidth killer." However, what a centralized system can't do, is a piece of cake for a distributed system ("_One_ can't, perhaps," said Humpty Dumpty, "but two can.[...]"). The fact that something like Skype does exist, works, and may claim an average of more than 150,000 users online at any given time, looks like a proof of feasibility to me! Unfortunately, Skype is closed-source (which is a showstopper for a crypto application), and Windows-only to boot. However, nothing prevents borrowing some ideas at http://www.skype.com/skype_p2pexplained.html for an opensource alternative. Speakfreely might not represent the best starting point, but it usually works out of the box (which is more than can be said for most other Internet phones), it's multi-platform, and already contains an RTP stack and bulk encryption code. As an alternative to Speakfreely's code, one could assemble together an RTP stack such as oRTP (http://www.linphone.org/ortp/), a bulk encryption and authentication layer such as SRTP (http://srtp.sourceforge.net/srtp.html), a portable audio abstraction layer such as Portaudio (www.portaudio.com) and an unencumbered codec such as Speex (www.speex.org). It would be nice if all the components were or could be ported to WinCE, for use on wireless PDA's. What Speakfreely sorely lacks is a sensible session initiation protocol, and access to non-NATted reflectors to help NATted peers to find each other and exchange UDP traffic. That's where a P2P network (especially one supporting the concept of non-NATted "ultrapeers") can save the day. In my opinion, traditional server-based (i.e., non-P2P) session initiation protocols like SIP -not to mention H.323- represent a poor choice for a consumer-friendly application: they require an arsenal of infrastructural applications (directories, proxies, gatekeepers etc.) which make them attractive only to telcos and hardware vendors (hence Cisco's support for SIP, and the venom liberally spilled on Skype at http://www.voxilla.com/modules.php?op=modload&name=News&file=article&sid=18&mode=thread). Besides, as I wrote on speak-freely-devel@lists.sourceforge.net, "the mechanisms that SIP/SDP use for session key negotiation range from the pathetic (key sent in cleartext!!) to the impractical (S/MIME CMS, which is a monster built on the clay feet of a PKI that isn't quite there)". Skype claims to use RSA-based key exchange, which is good for multi-party conferencing but does not preserve forward secrecy. Maybe some variant of ephemeral D-H authenticated by RSA signatures, with transparent renegotiation every time someone joins the conference, could do the job better. But the thing I particularly would like to discuss here is if, and how, to leverage on existing P2P networks. One could always implement a brand new network, using Distributed Hash Table algorithms such as Chord or Kademlia, but it would be much easier to rely from the very beginning upon a large number of nodes (at least for directory and presence functionality, if not for the reflectors which require specific UDP code). That would somehow repeat the approach initially adopted by Vocaltec when, in 1995, they launched their Iphone making use of IRC servers to publish dynamic IP addresses. Incidentally, the IRC users community didn't particularly appreciate ;-), triggering the Great Iphone War, which quickly led Vocaltec to set up its own dedicated IRC servers. >From what I see, Gnutella is pretty hopeless for that purpose because searches are only based on flooding, and therefore full-network searches are nearly impossible; on the other hand, Overnet (which relies upon the Kademlia algorithm) could perhaps be used as a sort of distributed presence/location "server", and also key server (perhaps it would be wise to use an OpenPGP key format to enjoy WoT features from day one). The Overnet protocol is unpublished, but it's been reverse-engineered at least in part by the mldonkey team. Alternatively, Freenet or Entropy could perhaps provide similar services, but with a large code overhead (I'd like to keep the code small enough to be ported, one day, to a PDA) and perhaps slower propagation (?). Comments, as I said, are much welcome. Enzo From zooko at zooko.com Sat Mar 13 11:49:30 2004 From: zooko at zooko.com (Zooko O'Whielacronx) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2P Micropayments In-Reply-To: Message from "Mark S. Miller" of "Fri, 12 Mar 2004 17:33:38 PST." <5.2.1.1.2.20040312173145.00af3508@caplet.com> References: <40521C3A.5050508@neurogrid.com> <40521C3A.5050508@neurogrid.com> <5.2.1.1.2.20040312173145.00af3508@caplet.com> Message-ID: I, Zooko, wrote the lines prepended with "> > ". MarkM wrote the lines prepended with "> ". > >I'm very interested in this research! It includes the "bilateral accounting > >scheme" that Doug Barnes and Jim McCoy invented for Mojo Nation (called "soft > >credit windows" in this paper), plus a very good idea of how to safely off-load > >the token-verification work from a central server to peers. > > What's the relationship to between these ideas and Digital Silk Road? > http://www.agorics.com/Library/dsr.html I apologize for saying that Doug and Jim invented that. The basic idea is clearly stated in _DSR_, and Doug and Jim explicitly referenced _DSR_ when they were designing Mojo Nation (personal communication). Doug and Jim integrated the idea with agnostic Chaumian digital cash, integrated those ideas into a complete distributed storage system, and led the actual implementation and deployment of that system. And they (we) learned a great deal about the difficulties which the "soft credit windows" idea encounters in practice. It's a shame that this experience wasn't written down. In my opinion, the main contribution of "PPay" by Yang and Garcia-Molina is a scheme to off-load the token verification service from the O(1) central servers onto the O(n) peers in the network, thus making token verification scalable. Regards, Zooko From gbildson at limepeer.com Sat Mar 13 19:00:03 2004 From: gbildson at limepeer.com (Greg Bildson) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Ideas for an opensource Skype lookalike Message-ID: <200403131400.AA1377567030@mail.limepeer.com> Enzo, I like your analysis and I don't disagree with what you say. However, I do want to stop one popular fallacy from repeating ad infinitum. While I agree that a Gnutella based network would not be appropriate for your lookup, searches are no longer based on pure flooding in Gnutella. Newer popular clients (LimeWire and BearShare anyways - not Morpheus or Gnucleus) use dynamic querying, high outdegree and last hop routing, which vastly reduces query traffic. Queries are now more targeted and sent selectively on connections. All the "experts" that continue to refer to Gnutella as using flooding are in fact discussing a historical artifact. Your line of discussion is interesting even if we won't be allocating developer resources towards it in the short-term. Thanks -greg ---------- Original Message ---------------------------------- From: "Enzo Michelangeli" Reply-To: "Peer-to-peer development." Date: Sat, 13 Mar 2004 19:03:17 +0800 >Hello everybody, > >I just joined this list after lurking for a while on its archive at >http://zgp.org/pipermail/p2p-hackers/. > >I'd like to gather opinions about using P2P techniques to support a type >of application that never managed to become really popular: a secure >internet phone. I have recently begun to monitor the development of >Speakfreely on Sourceforge (http://speak-freely.sourceforge.net/ ) after >its creator John Walker decided that the future of Internet was an >inhospitable environment for it and abandoned further development >(http://www.fourmilab.ch/speakfree/ ). I think that John overlooked the >possibilities offered by P2P architectures, in two critical areas: > >- Directories for location and presence. Nothing fancy here, already done >before for P2P chat systems. >- Working around NAT routers. John says of implementing third-party >reflectors: > > "[...] no non-commercial site like mine could possibly > afford the unlimited demands on bandwidth that would > require. It's one thing to provide a central meeting > point like a Look Who's Listening server, which handles > a packet every five minutes or so from connected sites, > but a server that's required to forward audio in > real-time between potentially any number of > simultaneously connected users is a bandwidth killer." > >However, what a centralized system can't do, is a piece of cake for a >distributed system ("_One_ can't, perhaps," said Humpty Dumpty, "but two >can.[...]"). The fact that something like Skype does exist, works, and may >claim an average of more than 150,000 users online at any given time, >looks like a proof of feasibility to me! > >Unfortunately, Skype is closed-source (which is a showstopper for a crypto >application), and Windows-only to boot. However, nothing prevents >borrowing some ideas at http://www.skype.com/skype_p2pexplained.html for >an opensource alternative. > >Speakfreely might not represent the best starting point, but it usually >works out of the box (which is more than can be said for most other >Internet phones), it's multi-platform, and already contains an RTP stack >and bulk encryption code. As an alternative to Speakfreely's code, one >could assemble together an RTP stack such as oRTP >(http://www.linphone.org/ortp/), a bulk encryption and authentication >layer such as SRTP (http://srtp.sourceforge.net/srtp.html), a portable >audio abstraction layer such as Portaudio (www.portaudio.com) and an >unencumbered codec such as Speex (www.speex.org). It would be nice if all >the components were or could be ported to WinCE, for use on wireless >PDA's. > >What Speakfreely sorely lacks is a sensible session initiation protocol, >and access to non-NATted reflectors to help NATted peers to find each >other and exchange UDP traffic. That's where a P2P network (especially one >supporting the concept of non-NATted "ultrapeers") can save the day. > >In my opinion, traditional server-based (i.e., non-P2P) session initiation >protocols like SIP -not to mention H.323- represent a poor choice for a >consumer-friendly application: they require an arsenal of infrastructural >applications (directories, proxies, gatekeepers etc.) which make them >attractive only to telcos and hardware vendors (hence Cisco's support for >SIP, and the venom liberally spilled on Skype at >http://www.voxilla.com/modules.php?op=modload&name=News&file=article&sid=18&mode=thread). >Besides, as I wrote on speak-freely-devel@lists.sourceforge.net, "the >mechanisms that SIP/SDP use for session key negotiation range from the >pathetic (key sent in cleartext!!) to the impractical (S/MIME CMS, which >is a monster built on the clay feet of a PKI that isn't quite there)". >Skype claims to use RSA-based key exchange, which is good for multi-party >conferencing but does not preserve forward secrecy. Maybe some variant of >ephemeral D-H authenticated by RSA signatures, with transparent >renegotiation every time someone joins the conference, could do the job >better. > >But the thing I particularly would like to discuss here is if, and how, to >leverage on existing P2P networks. One could always implement a brand new >network, using Distributed Hash Table algorithms such as Chord or >Kademlia, but it would be much easier to rely from the very beginning upon >a large number of nodes (at least for directory and presence >functionality, if not for the reflectors which require specific UDP code). >That would somehow repeat the approach initially adopted by Vocaltec when, >in 1995, they launched their Iphone making use of IRC servers to publish >dynamic IP addresses. Incidentally, the IRC users community didn't >particularly appreciate ;-), triggering the Great Iphone War, which >quickly led Vocaltec to set up its own dedicated IRC servers. > >>From what I see, Gnutella is pretty hopeless for that purpose because >searches are only based on flooding, and therefore full-network searches >are nearly impossible; on the other hand, Overnet (which relies upon the >Kademlia algorithm) could perhaps be used as a sort of distributed >presence/location "server", and also key server (perhaps it would be wise >to use an OpenPGP key format to enjoy WoT features from day one). The >Overnet protocol is unpublished, but it's been reverse-engineered at least >in part by the mldonkey team. Alternatively, Freenet or Entropy could >perhaps provide similar services, but with a large code overhead (I'd like >to keep the code small enough to be ported, one day, to a PDA) and perhaps >slower propagation (?). > >Comments, as I said, are much welcome. > >Enzo > >_______________________________________________ >p2p-hackers mailing list >p2p-hackers@zgp.org >http://zgp.org/mailman/listinfo/p2p-hackers >_______________________________________________ >Here is a web page listing P2P Conferences: >http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > From gbildson at limepeer.com Sat Mar 13 19:41:02 2004 From: gbildson at limepeer.com (Greg Bildson) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Diversity in P2P world Message-ID: <200403131441.AA3609264406@mail.limepeer.com> Something that I find endlessly interesting is the diversity of P2P clients participating on the Gnutella network. This page shows an interesting report of this coexistence: http://www.limewire.com/english/content/uastats.shtml These statistics come out of an ultrapeer-centric crawl and as such are not complete at the leaf level. For some description of this data, you can refer here: http://www.slyck.com/news.php?story=411 Even with an evolving Gnutella protocol, an open protocol allows coexistence. I could go into the issues raised by protocol version incompatibilities but that is another story. Thanks -greg From bradneuberg at yahoo.com Sat Mar 13 23:03:18 2004 From: bradneuberg at yahoo.com (Brad Neuberg) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Ideas for an opensource Skype lookalike In-Reply-To: <200403131400.AA1377567030@mail.limepeer.com> Message-ID: <20040313230318.28537.qmail@web60710.mail.yahoo.com> The Skype page claims to have implemented something called a "Global Index". From their web page: "Global decentralized user directory: Most instant message or communication software requires some form of centralized directory for the purposes of establishing a connection between end users in order to associate a static username and identity with an IP number that is likely to change. This change can occur when a user relocates or reconnects to a network with a dynamic IP address. Most Internet-based communication tools track users with a central directory which logs each username and IP number and keeps track of whether users are online or not. Central directories are extremely costly when the user base scales into the millions. By decentralizing this resource-hungry infrastructure, Skype is able to focus all of our resources on developing cutting-edge functionality. P2P network technologies such as FastTrack (used by KaZaA) would be suitable for decentralizing this, if not for the fact that these networks are fragmented in nature – a search does not reach all nodes in the network. Clearly, in order to deliver high quality telephony with the lowest possible costs, a third generation of P2P technology (“3G P2P”), or Global Index (GI) was a necessary development and represents yet another paradigm shift in the notion of scaleable networks. The Global Index technology is a multi-tiered network where supernodes communicate in such a way that every node in the network has full knowledge of all available users and resources with minimal latency." Does anyone have any idea how they have done this securely, reliably, and with low latency? Also, what exactly is the "key" into this index (the phone number?) If they have truly achieved this then it is ground-breaking work on their part. Brad From gbildson at limepeer.com Sun Mar 14 00:10:54 2004 From: gbildson at limepeer.com (Greg Bildson) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Ideas for an opensource Skype lookalike Message-ID: <200403131910.AA3081371968@mail.limepeer.com> The username and full name appear to be searchable. The number of online users appears to be in the 100,000 to 200,000 range. There is an advanced search that allows all profile information to be searchable. I wouldn't be surprised if the main index is roughly divided up on keyword prefix ranges amongst the long uptime addressable hosts. Then again, I wouldn't be surprised if it is a more formal DHT. Actually, their ability to do full substring searches argues against both of these approaches. They appear to be doing a broadcast search within a fully distributed index. The broadcast search ends when a set number of matches are reached. Why does this sound ground breaking? Thanks -greg ---------- Original Message ---------------------------------- From: Brad Neuberg Reply-To: "Peer-to-peer development." Date: Sat, 13 Mar 2004 15:03:18 -0800 (PST) >The Skype page claims to have implemented something >called a "Global Index". From their web page: > >"Global decentralized user directory: > >Most instant message or communication software >requires some form of centralized directory for the >purposes of establishing a connection between end >users in order to associate a static username and >identity with an IP number that is likely to change. >This change can occur when a user relocates or >reconnects to a network with a dynamic IP address. >Most Internet-based communication tools track users >with a central directory which logs each username and >IP number and keeps track of whether users are online >or not. Central directories are extremely costly when >the user base scales into the millions. By >decentralizing this resource-hungry infrastructure, >Skype is able to focus all of our resources on >developing cutting-edge functionality. > >P2P network technologies such as FastTrack (used by >KaZaA) would be suitable for decentralizing this, if >not for the fact that these networks are fragmented in >nature ? a search does not reach all nodes in the >network. Clearly, in order to deliver high quality >telephony with the lowest possible costs, a third >generation of P2P technology (?3G P2P?), or Global >Index (GI) was a necessary development and represents >yet another paradigm shift in the notion of scaleable >networks. The Global Index technology is a >multi-tiered network where supernodes communicate in >such a way that every node in the network has full >knowledge of all available users and resources with >minimal latency." > >Does anyone have any idea how they have done this >securely, reliably, and with low latency? Also, what >exactly is the "key" into this index (the phone >number?) If they have truly achieved this then it is >ground-breaking work on their part. > >Brad From bkn3 at columbia.edu Sun Mar 14 06:05:33 2004 From: bkn3 at columbia.edu (Brad Neuberg) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Ideas for an opensource Skype lookalike In-Reply-To: <200403131910.AA3081371968@mail.limepeer.com> References: <200403131910.AA3081371968@mail.limepeer.com> Message-ID: <6.0.1.1.2.20040313220408.01ec2328@pop.mail.yahoo.com> Because they claim to have a scalable, secure, decentralized lookup mechanism that is easy to use. At 04:10 PM 3/13/2004, you wrote: >The username and full name appear to be searchable. The number of online >users appears to be in the 100,000 to 200,000 range. There is an advanced >search that allows all profile information to be searchable. > >I wouldn't be surprised if the main index is roughly divided up on keyword >prefix ranges amongst the long uptime addressable hosts. Then again, I >wouldn't be surprised if it is a more formal DHT. Actually, their ability >to do full substring searches argues against both of these >approaches. They appear to be doing a broadcast search within a fully >distributed index. The broadcast search ends when a set number of matches >are reached. > >Why does this sound ground breaking? > >Thanks >-greg > >---------- Original Message ---------------------------------- >From: Brad Neuberg >Reply-To: "Peer-to-peer development." >Date: Sat, 13 Mar 2004 15:03:18 -0800 (PST) > > >The Skype page claims to have implemented something > >called a "Global Index". From their web page: > > > >"Global decentralized user directory: > > > >Most instant message or communication software > >requires some form of centralized directory for the > >purposes of establishing a connection between end > >users in order to associate a static username and > >identity with an IP number that is likely to change. > >This change can occur when a user relocates or > >reconnects to a network with a dynamic IP address. > >Most Internet-based communication tools track users > >with a central directory which logs each username and > >IP number and keeps track of whether users are online > >or not. Central directories are extremely costly when > >the user base scales into the millions. By > >decentralizing this resource-hungry infrastructure, > >Skype is able to focus all of our resources on > >developing cutting-edge functionality. > > > >P2P network technologies such as FastTrack (used by > >KaZaA) would be suitable for decentralizing this, if > >not for the fact that these networks are fragmented in > >nature ? a search does not reach all nodes in the > >network. Clearly, in order to deliver high quality > >telephony with the lowest possible costs, a third > >generation of P2P technology (?3G P2P?), or Global > >Index (GI) was a necessary development and represents > >yet another paradigm shift in the notion of scaleable > >networks. The Global Index technology is a > >multi-tiered network where supernodes communicate in > >such a way that every node in the network has full > >knowledge of all available users and resources with > >minimal latency." > > > >Does anyone have any idea how they have done this > >securely, reliably, and with low latency? Also, what > >exactly is the "key" into this index (the phone > >number?) If they have truly achieved this then it is > >ground-breaking work on their part. > > > >Brad > >_______________________________________________ >p2p-hackers mailing list >p2p-hackers@zgp.org >http://zgp.org/mailman/listinfo/p2p-hackers >_______________________________________________ >Here is a web page listing P2P Conferences: >http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From em at em.no-ip.com Sun Mar 14 03:46:01 2004 From: em at em.no-ip.com (Enzo Michelangeli) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Ideas for an opensource Skype lookalike References: <200403131910.AA3081371968@mail.limepeer.com> Message-ID: <04dd01c40998$07e147e0$0200a8c0@em.noip.com> ----- Original Message ----- From: "Greg Bildson" To: "Peer-to-peer development." ; "Peer-to-peer development." Sent: Sunday, March 14, 2004 8:10 AM Subject: Re: [p2p-hackers] Ideas for an opensource Skype lookalike > The username and full name appear to be searchable. The number > of online users appears to be in the 100,000 to 200,000 range. > There is an advanced search that allows all profile information > to be searchable. > > I wouldn't be surprised if the main index is roughly divided > up on keyword prefix ranges amongst the long uptime addressable > hosts. Then again, I wouldn't be surprised if it is a more > formal DHT. Recently Morpheus added voice chat, and interestingly also a new DHT-based search protocol called NeoNet (http://www.morpheus.com/faq.html ). However I haven't tried it so far because I don't like adware, and I have no idea about the possible reliance of the former upon the latter. Enzo From gbildson at limepeer.com Sun Mar 14 18:46:38 2004 From: gbildson at limepeer.com (Greg Bildson) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Ideas for an opensource Skype lookalike Message-ID: <200403141346.AA1639776300@mail.limepeer.com> It sounds essentially like a rebuild of the FastTrack (Kazaa) infrastructure. If it can scale to 50 million then I will be really impressed. Thanks -greg ---------- Original Message ---------------------------------- From: Brad Neuberg Reply-To: "Peer-to-peer development." Date: Sat, 13 Mar 2004 22:05:33 -0800 >Because they claim to have a scalable, secure, decentralized lookup >mechanism that is easy to use. > >At 04:10 PM 3/13/2004, you wrote: > >>The username and full name appear to be searchable. The number of online >>users appears to be in the 100,000 to 200,000 range. There is an advanced >>search that allows all profile information to be searchable. >> >>I wouldn't be surprised if the main index is roughly divided up on keyword >>prefix ranges amongst the long uptime addressable hosts. Then again, I >>wouldn't be surprised if it is a more formal DHT. Actually, their ability >>to do full substring searches argues against both of these >>approaches. They appear to be doing a broadcast search within a fully >>distributed index. The broadcast search ends when a set number of matches >>are reached. >> >>Why does this sound ground breaking? >> >>Thanks >>-greg >> >>---------- Original Message ---------------------------------- >>From: Brad Neuberg >>Reply-To: "Peer-to-peer development." >>Date: Sat, 13 Mar 2004 15:03:18 -0800 (PST) >> >> >The Skype page claims to have implemented something >> >called a "Global Index". From their web page: >> > >> >"Global decentralized user directory: >> > >> >Most instant message or communication software >> >requires some form of centralized directory for the >> >purposes of establishing a connection between end >> >users in order to associate a static username and >> >identity with an IP number that is likely to change. >> >This change can occur when a user relocates or >> >reconnects to a network with a dynamic IP address. >> >Most Internet-based communication tools track users >> >with a central directory which logs each username and >> >IP number and keeps track of whether users are online >> >or not. Central directories are extremely costly when >> >the user base scales into the millions. By >> >decentralizing this resource-hungry infrastructure, >> >Skype is able to focus all of our resources on >> >developing cutting-edge functionality. >> > >> >P2P network technologies such as FastTrack (used by >> >KaZaA) would be suitable for decentralizing this, if >> >not for the fact that these networks are fragmented in >> >nature ? a search does not reach all nodes in the >> >network. Clearly, in order to deliver high quality >> >telephony with the lowest possible costs, a third >> >generation of P2P technology (?3G P2P?), or Global >> >Index (GI) was a necessary development and represents >> >yet another paradigm shift in the notion of scaleable >> >networks. The Global Index technology is a >> >multi-tiered network where supernodes communicate in >> >such a way that every node in the network has full >> >knowledge of all available users and resources with >> >minimal latency." >> > >> >Does anyone have any idea how they have done this >> >securely, reliably, and with low latency? Also, what >> >exactly is the "key" into this index (the phone >> >number?) If they have truly achieved this then it is >> >ground-breaking work on their part. >> > >> >Brad >> From bradneuberg at yahoo.com Sun Mar 14 21:21:24 2004 From: bradneuberg at yahoo.com (Brad Neuberg) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Ideas for an opensource Skype lookalike In-Reply-To: <200403141346.AA1639776300@mail.limepeer.com> Message-ID: <20040314212124.9900.qmail@web60705.mail.yahoo.com> Right, but they never claimed that when you search on the FastTrack infrastructure for "somesong.mp3" that the peer that said they have it is the real peer. For Skype they are claiming that when I search for a phone number or peer identifier that I am finding just the one peer that should have this in a decentralized and secure way. They are claiming much more security for Skype. Brad --- Greg Bildson wrote: > > It sounds essentially like a rebuild of the > FastTrack (Kazaa) infrastructure. If it can scale > to 50 million then I will be really impressed. > > Thanks > -greg > ---------- Original Message > ---------------------------------- > From: Brad Neuberg > Reply-To: "Peer-to-peer development." > > Date: Sat, 13 Mar 2004 22:05:33 -0800 > > >Because they claim to have a scalable, secure, > decentralized lookup > >mechanism that is easy to use. > > > >At 04:10 PM 3/13/2004, you wrote: > > > >>The username and full name appear to be > searchable. The number of online > >>users appears to be in the 100,000 to 200,000 > range. There is an advanced > >>search that allows all profile information to be > searchable. > >> > >>I wouldn't be surprised if the main index is > roughly divided up on keyword > >>prefix ranges amongst the long uptime addressable > hosts. Then again, I > >>wouldn't be surprised if it is a more formal DHT. > Actually, their ability > >>to do full substring searches argues against both > of these > >>approaches. They appear to be doing a broadcast > search within a fully > >>distributed index. The broadcast search ends when > a set number of matches > >>are reached. > >> > >>Why does this sound ground breaking? > >> > >>Thanks > >>-greg > >> > >>---------- Original Message > ---------------------------------- > >>From: Brad Neuberg > >>Reply-To: "Peer-to-peer development." > > >>Date: Sat, 13 Mar 2004 15:03:18 -0800 (PST) > >> > >> >The Skype page claims to have implemented > something > >> >called a "Global Index". From their web page: > >> > > >> >"Global decentralized user directory: > >> > > >> >Most instant message or communication software > >> >requires some form of centralized directory for > the > >> >purposes of establishing a connection between > end > >> >users in order to associate a static username > and > >> >identity with an IP number that is likely to > change. > >> >This change can occur when a user relocates or > >> >reconnects to a network with a dynamic IP > address. > >> >Most Internet-based communication tools track > users > >> >with a central directory which logs each > username and > >> >IP number and keeps track of whether users are > online > >> >or not. Central directories are extremely costly > when > >> >the user base scales into the millions. By > >> >decentralizing this resource-hungry > infrastructure, > >> >Skype is able to focus all of our resources on > >> >developing cutting-edge functionality. > >> > > >> >P2P network technologies such as FastTrack (used > by > >> >KaZaA) would be suitable for decentralizing > this, if > >> >not for the fact that these networks are > fragmented in > >> >nature ­ a search does not reach all nodes in > the > >> >network. Clearly, in order to deliver high > quality > >> >telephony with the lowest possible costs, a > third > >> >generation of P2P technology (“3G P2P”), or > Global > >> >Index (GI) was a necessary development and > represents > >> >yet another paradigm shift in the notion of > scaleable > >> >networks. The Global Index technology is a > >> >multi-tiered network where supernodes > communicate in > >> >such a way that every node in the network has > full > >> >knowledge of all available users and resources > with > >> >minimal latency." > >> > > >> >Does anyone have any idea how they have done > this > >> >securely, reliably, and with low latency? Also, > what > >> >exactly is the "key" into this index (the phone > >> >number?) If they have truly achieved this then > it is > >> >ground-breaking work on their part. > >> > > >> >Brad > >> > > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From aloeser at cs.tu-berlin.de Wed Mar 17 13:07:58 2004 From: aloeser at cs.tu-berlin.de (aloeser) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] Storage Load Balancing in Chord Message-ID: <40584DAE.6CF9A8D@cs.tu-berlin.de> HI, distributing storage load uniformly among peers until the available storage is used optimally remains a major challenge for current structured P2P systems. Randomization seems to play an important role. What experiences did you made in the Chord network with strategies such as virtual server (One-to One, One-to Many, Many-to-Many)? Which storage load balancing strategies suitable for the CHORD network can you recommend? Alex -- ___________________________________________________________ M.Sc., Dipl. Wi.-Inf. Alexander L?ser Technische Universitaet Berlin Fakultaet IV - CIS bmb+f-Projekt: "New Economy, Neue Medien in der Bildung" hp: http://cis.cs.tu-berlin.de/~aloeser/ office: +49- 30-314-25551 fax : +49- 30-314-21601 ___________________________________________________________ From Bernard.Traversat at Sun.COM Wed Mar 17 17:08:43 2004 From: Bernard.Traversat at Sun.COM (Bernard Traversat) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] [Fwd: [JXTA user] Announcing JXTA J2SE 2.2.1 "Churrasco" Release] Message-ID: <4058861B.1010302@sun.com> For people interested, JXTA J2SE 2.2.1 was just released. Cheers, B. -------- Original Message -------- Subject: [JXTA user] Announcing JXTA J2SE 2.2.1 "Churrasco" Release Date: Mon, 15 Mar 2004 17:52:01 -0800 From: Mike [bondolo] Duigou Reply-To: user@jxta.org To: dev@platform.jxta.org, announce@jxta.org CC: dev@jxta.org, discuss@jxta.org, user@jxta.org Followup-To: dev@platform.jxta.org Community Members, We are pleased to announce the availability of a new JXTA J2SE release, 2.2.1 "Churrasco". The 2.2.1 release is a maintenance release that enhances the previous 2.2 release with some important bug and performance fixes and adds a small number of new features. http://download.jxta.org/stablebuilds/index.html This release is designed to be API and protocol backwards compatible with other JXTA J2SE 2.x releases. The JXTA J2SE Release Team Acknowledgements ==================================================================== Special thanks to all community members who contributed ideas, reported problems, provided patches, and have helped us greatly to improve the quality, and robustness of this release. New Features ==================================================================== - JxtaMulticastSocket - Enhanced JXTA Class Loader - PeerGroupFactory.setConfigurator - Disabling PSE Auto-login JxtaMulticastSocket ------------------- The JxtaMulticastSocket class is useful for sending and receiving JXTA multicast packets. A JxtaMulticastSocket is a (UDP) DatagramSocket, with additional capabilities for joining "groups" of other multicast hosts on the internet. A multicast group is specified within the context of PeerGroup and a propagate pipe advertisement. When data is sent a message to a multicast group, all subscribing recipients to that peergroup and pipe receive the message (including themselves). When a socket subscribes to a multicast group/port, it receives datagrams sent by other hosts to the group/pipe, as do all other members of the group and pipe. A socket relinquishes membership in a group by the close() method. Multiple MulticastSocket's may subscribe to a multicast group and pipe concurrently, and they will all receive group datagrams. For more information and examples see: http://wiki.java.net/bin/view/Jxta/JxtaMulticastSocket Enhanced JXTA Class Loader -------------------------- The JXTA Class Loader has been extended to now extend URLClassLoader which provides additional class loading options and better integration with Java 2 Security features. You can also now load classes by ModuleSpecID. PeerGroupFactory.setConfigurator -------------------------------- Frequently JXTA application developers wish to completely disable the AWT based UI configuratior that JXTA will present at boot time. You can now provide your own Configurator or disable the builtin Configurator. For more information and instructions see: http://wiki.java.net/bin/view/Jxta/NullConfigurator Disabling PSE Auto-login ------------------------ The default behaviour of the JXTA J2SE platform since the introduction of the TLS Message Transport has been to ask the user for authentication information upon starting JXTA. In JXTA J2SE 2.2.1 it is now possible to disable the automatic request for authentication that occurs at JXTA boot using a newly defined java system property. This places authentication entirely under the control of the application developer. For more information and instructions see: http://wiki.java.net/bin/view/Jxta/NoAutoLogin Downloading and Installing ==================================================================== Prebuilt JXTA J2SE 2.2.1 builds can be installed from : You can also build from cvs : cvs -d:pserver:guest@cvs.jxta.org:/cvs co -r JXTA_2_2_1_00 \ platform shell security cms myjxta2 Each module has an Apache Ant build.xml file located in the 'binding/java/' directory. You can build each module with the following commands: cd security/binding/java ant dist These commands should be repeated for each module. Some of the modules depend upon other modules being built first. Build the modules in the following order: security, platform, shell, cms, myjxta2. Known incompatibilities between JXTA 2.2.1 and prior JXTA 2.x releases ==================================================================== A number of deprecated APIs have been removed in this release. See the "Impacts to Developers" section below for details. Impacts to Current Users : LOW ==================================================================== - JXTA J2SE 2.2.1 should a drop in replacement for any application currently using JXTA J2SE 2.2 Impacts to Developers : MEDIUM ==================================================================== - The following deprecated APIs were removed in JXTA 2.2.1: o Legacy accessor methods in Message o Legacy accessor methods in MessageElement o MessageElementEnumeration o StringEnumeration - Pay careful attention to the deprecation messages generated when compiling your application. JXTA J2SE 2.2.1 introduces several new deprecations. The next planned release (Jambalaya) will remove some interfaces which have been deprecated since JXTA J2SE 2.0 : o EndpointFilterListener -> MessageFilterListener o EndpointMessenger -> Messenger o JxtaTimer o net.jxta.JxtaUtilities o Endpoint.newMessage -> new Message() o Endpoint.newEndpointAddress -> new EndpointAddress() o PipeService.createMessage -> new Message() - Be very careful to avoid importing any "impl" packages (net.jxta.impl.*). The classes and methods in these packages may (and will) change from release to release and no effort will be made to retain compatibility of programs which depend upon "impl" packages. YOU HAVE BEEN WARNED! If there is no alternative available to importing from "impl", please file an issue against the platform so that an alternate mechanism can be developed. Impacts to Deployers / Rendezvous Managers : LOW ==================================================================== - This release includes improve performance and stability for the rendezvous and relay peers. JXTA J2SE 2.2.1 should be a drop in replacement for JXTA J2SE 2.2 releases. - Before restarting rendezvous/relays you should clear your cm directory. (It can also be argued that rdv/relays may benefit from doing this with every restart). Issued Closed During JXTA 2.2.1 ==================================================================== 793 DEFECT P3 Edge takes ~ 4min to resolve relay and an addtnl ~2min 836 DEFECT P2 hanging http client threads 884 DEFECT P3 Observed sluggish Relay failover 904 ENHANC P2 Slow to connect to RDV in a sub-group 923 ENHANC P2 Extensions to JxtaClassLoader 931 PATCH P2 Remove deprecated Message APIs 932 PATCH P3 Remove legacy JDK 1.3.X IPUtils Code 944 ENHANC P2 Implement expiration checking for PSE Credential 952 DEFECT P3 Relay Service Threads not marked daemon 962 PATCH P3 Remove obsolete impl.utils 966 DEFECT P3 API javadoc is incomplete 969 PATCH P2 Update Bouncy Castle provider to 1.21 974 PATCH P3 Discovery Service diagnostic improvements 979 ENHANC P3 Endpoint Service Diagnostic Improvements 980 ENHANC P3 Advertisement Tidying 981 DEFECT P2 Router does not pass hint during initial connection 993 ENHANC P2 Explicit svc dependence in startApp() for PipeService 1000 DEFECT P2 Need to improve reliability library throughput 1001 DEFECT P2 HttpClientMessenger can hang on exit 1005 DEFECT P2 default PC location should be ~/.jxta/PlatformConfig 1006 DEFECT P5 Enabled field, but checkbox not selected. 1008 DEFECT P2 ext:config port availability scan 1011 DEFECT P3 No mechanism to disable automatic PSE Login request 1015 DEFECT P2 port scan incomplete 1016 DEFECT P2 DOMXMLElement.getAttributes does not work 1018 DEFECT P2 ext:config uri path on win/* in util pkg 1021 DEFECT P3 Adaptive Flow Control does not count acks properly 1022 ENHANC P2 Completes PeerGroupFactory.setConfigurator support 1023 ENHANC P3 ext:config support for "incomplete" PlatformConfig 1024 DEFECT P3 Hierarchical groupID incomplete 1025 DEFECT P3 PeerInfoServiceaddRemoteMonitorListener problems 1033 DEFECT P1 Memory leak in http 1044 DEFECT P1 ModuleManager.createServiceAdvertisement() problem Known issues in JXTA 2.2.1 ==================================================================== (none) For a complete list of known issues follow the link: --------------------------------------------------------------------- To unsubscribe, e-mail: user-unsubscribe@jxta.org For additional commands, e-mail: user-help@jxta.org -- "As Java implies platform independence, and XML implies language independence, then JXTA implies network independence." From decapita at dti.unimi.it Wed Mar 17 20:42:51 2004 From: decapita at dti.unimi.it (Sabrina De Capitani di Vimercati) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] CFP: Workshop on Privacy in the Electronic Society Message-ID: [Apologies if you receive multiple copies of this message] CALL FOR PAPERS 3rd WORKSHOP ON PRIVACY IN THE ELECTRONIC SOCIETY Washington, DC, USA - October 28, 2004 Sponsored by ACM SIGSAC Held in association with 11th ACM CCS 2004 http://seclab.dti.unimi.it/wpes2004 ---------------------------------------------------------------------- Privacy issues have been the subject of public debates and the need for privacy-aware policies, regulations, and techniques has been widely recognized. Goal of this workshop is to discuss the problems of privacy in the global interconnected societies and possible solutions to it. The 2004 Workshop is the third in what we hope will be a yearly forum for papers on all the different aspects of privacy in today's electronic society. The first two workshops in the series were held in Washington, in conjunction with the 9th ACM CCS conference and with the 10th ACM CCS conference, respectively. The success of the first two editions of the workshop and the increased interest of the community in privacy issues, is the main reason for repeating the event. The workshop seeks submissions from academia and industry presenting novel research on all theoretical and practical aspects of electronic privacy, as well as experimental studies of fielded systems. We encourage submissions from other communities such as law and business that present these communities' perspectives on technological issues. Topics of interest include, but are not limited to: - anonymity, pseudonymity, unlinkability - business model with privacy requirements - data protection from correlation and leakage attacks - electronic communication privacy - information dissemination control - privacy-aware access control - privacy in the digital business - privacy enhancing technologies - privacy policies and human rights - privacy and anonymity in Web transactions - privacy threats - privacy and confidentiality management - privacy in the electronic records - privacy in health care and public administration - public records and personal privacy - privacy and virtual identity - personally identifiable information - privacy policy enforcement - privacy and data mining - relationships between privacy and security - user profiling - wireless privacy PAPER SUBMISSIONS Submitted papers must not substantially overlap papers that have been published or that are simultaneously submitted to a journal or a conference with proceedings. Papers should be at most 15 pages excluding the bibliography and well-marked appendices (using 11-point font and reasonable margins on letter-size paper), and at most 20 pages total. Committee members are not required to read the appendices, and so the paper should be intelligible without them. Papers should have a cover page with the title, authors, abstract and contact information. Authors are invited to submit their contributions electronically through the web site http://seclab.dti.unimi.it/wpes2004/submissions.html. Submission must be in the form of a ps (Postscript), or pdf (Adobe) file. Do NOT submit files formatted for word processing packages (e.g., Microsoft Word or WordPerfect files). Papers must be received by the deadline of June 11, 2004 in order to be considered. Notification of acceptance or rejection will be sent to authors by August 2, 2004. Authors of accepted papers must guarantee that their paper will be presented at the workshop. Accepted papers will be published by the ACM in a conference proceedings. GENERAL CHAIR Vijay Atluri Rutgers University, USA email: atluri@andromeda.rutgers.edu PROGRAM CHAIRS Sabrina De Capitani di Vimercati Paul Syverson University of Milan Naval Research Laboratory email: samarati@dti.unimi.it url: www.syverson.org IMPORTANT DATES Paper Submission due: June 11, 2004 Acceptance notification: August 2, 2004 Final papers due: August 30, 2004 PROGRAM COMMITTEE JC Cannon, Microsoft, USA Lorrie Cranor, Carnegie Mellon University, USA Ernesto Damiani, University of Milan, Italy George Danezis, University of Cambridge, UK Roger Dingledine, The Free Haven Project, USA Wenliang Du, Syracuse University, USA Philippe Golle, Stanford University, USA Mike Gurski, Information & Privacy Commission/Ontario, Canada Susan Landau, Sun Microsystems Laboratories, USA Andreas Pfitzmann, Dresden University of Technology, Germany Andrew Patrick, National Research Council, Ottawa, Canada Marc Rennhard, ETH Zurich, Switzerland Pierangela Samarati, University of Milan, Italy Matthias Schunter, IBM Zurich Research Laboratory, Switzerland Tomas Sander, Hewlet Packard, USA Marianne Winslett, U. of Illinois Urbana-Champaign, USA From sam at neurogrid.com Thu Mar 18 03:45:02 2004 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:12:38 2006 Subject: [p2p-hackers] P2PJ Call for Papers: Next Deadline 1st April Message-ID: <40591B3E.6080604@neurogrid.com> Apologies for if you receive multiple copies ------------------------------------------------------------------ CALL FOR PAPERS Peer-to-Peer Journal (http://p2pjournal.com) ------------------------------------------------------------------- The Peer-to-Peer Journal (P2PJ) is a bi-monthly journal that serves as a forum to individuals and companies interested in applying, developing, educating, & advertising in the fields of Peer-to-Peer (P2P) and parallel computing. The P2P Journal is currently accepting submissions of articles, whitepapers, product reviews, discussions, and letters or short communications. Topics of interest include, but are not limited to: Novel Peer-to-Peer applications and systems P2P simulation and network/traffic mapping/topology tools Instant Messaging (IM) Collaborative Computing FileSharing tools and protocols Content Distribution Networks (CDN) Parallel Computing Grid Networks Cluster Architectures For writer's guideline, see http://p2pjournal.com/main/p2p_writers_guideline.pdf Important Dates Submission Deadline for May Issue: 1st April 2004 Submission Deadline for July Issue: 1st July 2004 Please send submissions to editor@p2pjournal.com Best regards, Raymond F. Gao, Editor-in-Chief Daniel Brookshier, Editor Sam Joseph, Editor From mfreed at cs.nyu.edu Thu Mar 18 05:12:53 2004 From: mfreed at cs.nyu.edu (mfreed@cs.nyu.edu) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] Re: Hello Message-ID: An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20040318/a737dab6/attachment.html From sam at neurogrid.com Thu Mar 18 19:01:39 2004 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] Identity Crisis: Anonymity vs Reputation in P2P Systems Message-ID: <4059F213.60905@neurogrid.com> Hi All, So another interesting paper I found is "Identity Crisis: Anonymity vs Reputation in P2P Systems" Marti & Hector, 3rd P2P conference http://dbpubs.stanford.edu:8090/pub/2003-41 I think this is a solid paper, and its main thrust seems to be a comparison between central authority identity versus self managed identity in a gnutella-like file sharing system. Interestingly they use metadata to mediate their concept of authenticity, e.g. we might have a document with meta-data as follows Metadata Title: A Tale of Two Cities Author: Charles Dickens Publish Date: April 2002 Publisher: Barnes & Noble Books ... Content It was the best of times, it was the worst of times... "In general, a document is considered authentic if and only if its metadata fields are ?consistent? with each other and the content. If any information in the metadata does not ?agree? with the content or the rest of the metadata, then the document is considered to be inauthentic, or fake. In the example above, if the Author field were changed to Charles Darwin, this document would be considered inauthentic, since Barnes & Noble Books has never published a book titled A Tale of Two Cities written by Charles Darwin that begins ?It was the best of times..." In Marti and Hector's approach Authentication is something that is performed by a centralised authority, and one point of comparision for them between self-managed and centralised identity is how frequently is authentication required They describe this in terms of the verificaton ratio, the number of times authenticity has to be verified, divided by the number of successful queries. I would recommend reading this paper, but to summarise some of their results, they look at the effects of different strategies such as always selecting the most reputable source, versus a weighted selection, as well as considering different values of "default" reputation and trust thresholds. What they find is that in their simulations the efficiency is roughly as follows: LoginBest > LoginWeighted > SelfMgdBest > SelfMgdWeighted Where Best and Weighted refer to the download selection strategies, and Login and SelfMgd refer to centralised and decentralised identity managment respectively. We should note that this inequality above is generalising over some interesting variations. One other point of interest is that way in which login and selfmgd are implemented in the simulation. Login means nodes can't shake their identities, and selfmgd means that once a node cheats once it discards its identity and starts over. Anyways, I think the interplay between metadata and trust here was interesting .... CHEERS> SAM From mfreed at cs.nyu.edu Thu Mar 18 20:02:34 2004 From: mfreed at cs.nyu.edu (mfreed@cs.nyu.edu) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] Hidden message Message-ID: An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20040319/c6f2b867/attachment.htm From zooko at zooko.com Thu Mar 18 20:56:43 2004 From: zooko at zooko.com (Zooko O'Whielacronx) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] Identity Crisis: Anonymity vs Reputation in P2P Systems In-Reply-To: Message from Sam Joseph of "Fri, 19 Mar 2004 04:01:39 +0900." <4059F213.60905@neurogrid.com> References: <4059F213.60905@neurogrid.com> Message-ID: > "Identity Crisis: Anonymity vs Reputation in P2P Systems" > Marti & Hector, 3rd P2P conference > http://dbpubs.stanford.edu:8090/pub/2003-41 Your metadata is inauthentic; the last names of the authors are Marti and Garcia-Molina. Regards, O'Whielacronx From sam at neurogrid.com Thu Mar 18 21:09:12 2004 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] Re: Identity Crisis: Anonymity vs Reputation in P2P Systems In-Reply-To: References: <4059F213.60905@neurogrid.com> Message-ID: <405A0FF8.4060404@neurogrid.com> Thanks Zooko, Actually that raises an interesting point about the difference between inauthentic meta-data which has been intentionally changed in order to mislead and that which is just in error for non-malicious reasons .... Perhaps they should be treated differently ... or would all the spammers just cry "Honest mistake" all the time .... CHEERS> SAM Zooko O'Whielacronx wrote: >>"Identity Crisis: Anonymity vs Reputation in P2P Systems" >>Marti & Hector, 3rd P2P conference >>http://dbpubs.stanford.edu:8090/pub/2003-41 >> >> > >Your metadata is inauthentic; the last names of the authors are Marti and >Garcia-Molina. > >Regards, > >O'Whielacronx > >_______________________________________________ >p2p-hackers mailing list >p2p-hackers@zgp.org >http://zgp.org/mailman/listinfo/p2p-hackers >_______________________________________________ >Here is a web page listing P2P Conferences: >http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > > > > From wesley at felter.org Fri Mar 19 03:16:59 2004 From: wesley at felter.org (Wes Felter) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] Identity Crisis: Anonymity vs Reputation in P2P Systems In-Reply-To: <4059F213.60905@neurogrid.com> References: <4059F213.60905@neurogrid.com> Message-ID: On Mar 18, 2004, at 1:01 PM, Sam Joseph wrote: > Interestingly they use metadata to mediate their concept of > authenticity What is interesting to me is that the stuff about metadata and authenticity is pretty much irrelevant to the paper. If the receiver of the file used arbitrary criteria to decide whether the file is good or bad, the results would have been similar AFAICT. Wes Felter - wesley@felter.org - http://felter.org/wesley/ From sam at neurogrid.com Fri Mar 19 03:30:52 2004 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] Identity Crisis: Anonymity vs Reputation in P2P Systems In-Reply-To: References: <4059F213.60905@neurogrid.com> Message-ID: <405A696C.8050000@neurogrid.com> Hi Wes, Wes Felter wrote: > On Mar 18, 2004, at 1:01 PM, Sam Joseph wrote: > >> Interestingly they use metadata to mediate their concept of authenticity > > What is interesting to me is that the stuff about metadata and > authenticity is pretty much irrelevant to the paper. If the receiver > of the file used arbitrary criteria to decide whether the file is good > or bad, the results would have been similar AFAICT. I'm not quite sure what you mean by arbitrary criteria. Like deciding whether the file was good or bad depending on the file length? If they used arbitrary criteria then mailiciously provided files would be accepted by the user. i.e. they'd open what they thought was some helpful software, and find it was a worm or something. In order to actually implement the system you would have to have some sort of authentication server and some approach to handling the meta-data. I think the interest comes from what type of meta-data you are using. If your meta-data was a file hash, then you could check the integrity of the file without having to go to some central authenticator. You'd have a computational cost instead. I think you could argue that meta-data was irrelevant to the paper's results since authentication could be achieved by submitting a downloaded file along with your serach criteria to some authority (although I think the meta-data example makes it easier to understand the problem they are looking at). However authentication seems to be very strongly related to what the paper is about. Not the process of authentication necessarily, but the cost associated with it. And if we are seeing the results of the costs of authenticity in the paper, then this raises the question of what types of meta-data to use, e.g. if we say that in order for good performance we need a centrally managed login, then systems which wanted to be totally decentralised might restrict authentication to locally authenticable file-hashes. CHEERS> SAM From em at em.no-ip.com Sat Mar 20 09:05:56 2004 From: em at em.no-ip.com (Enzo Michelangeli) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] Questions about Overnet Message-ID: <10e101c40e5a$96cd50e0$0200a8c0@em.noip.com> I don't know if this is the right forum for my questions, but can anybody kindly enlighten me on this: 1. Are Overnet searches exhaustive, meaning that, if content is presently published, a search for it can be reasonably expected to return a hit with probability close to 1? 2. If a node publishes content on Overnet, after how long, on average, will that content it be "visible" to searches initiated from any other node? My reason for asking is that I'd like to understand if it is possible, and viable, to build a distributed directory and presence service on top of Overnet's publishing and searching facility. Enzo From Paul.Harrison at infotech.monash.edu.au Sun Mar 21 05:07:59 2004 From: Paul.Harrison at infotech.monash.edu.au (Paul Harrison) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] Questions about Overnet In-Reply-To: <10e101c40e5a$96cd50e0$0200a8c0@em.noip.com> Message-ID: On Sat, 20 Mar 2004, Enzo Michelangeli wrote: > I don't know if this is the right forum for my questions, but can anybody > kindly enlighten me on this: > > 1. Are Overnet searches exhaustive, meaning that, if content is presently > published, a search for it can be reasonably expected to return a hit with > probability close to 1? > > 2. If a node publishes content on Overnet, after how long, on average, > will that content it be "visible" to searches initiated from any other > node? > > My reason for asking is that I'd like to understand if it is possible, and > viable, to build a distributed directory and presence service on top of > Overnet's publishing and searching facility. > Overnet is based on Kademlia, a Distributed Hash-Table. So, assuming it's a robust implementation, searches will be exhaustive and content will be visible as soon as it is published. It's certainly possible to build presence services and directories on top of a DHT. For example, Circle (thecircle.org.au) has an instant messaging system built on top of a DHT (including the ability to search by name or cryptographic id, and to publish a request to be notified when a particular person comes online... basically everything a normal IM system does). cheers, Paul pfh@logarithmic.net | http://www.logarithmic.net/pfh end apartheid From em at em.no-ip.com Mon Mar 22 09:52:44 2004 From: em at em.no-ip.com (Enzo Michelangeli) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] Questions about Overnet References: Message-ID: <024001c40ff3$ac75fe20$0200a8c0@em.noip.com> ----- Original Message ----- From: "Paul Harrison" To: "Peer-to-peer development." Sent: Sunday, March 21, 2004 1:07 PM [...] > Overnet is based on Kademlia, a Distributed Hash-Table. So, assuming > it's a robust implementation, searches will be exhaustive and content > will be visible as soon as it is published. Right, but how soon is "soon"? 5 seconds, 5 minutes, half hour... And what happens when the node(s) handling the section of DHT that stores that piece of information go(es) offline? I'd like to pick the brain of some Overnet, eMule or mldonkey developer (or at least "power user" :-) ), as the protocol itself is very poorly documented - also because it was originally closed source / closed specs, and the "Kad" part of eMule 0.4x and the mldonkey implementation where based on reverse-engineering. BTW, what is the answer to my "publishing latency and persistence" questions in Circle's case? And how would you compare Chord to Kademlia? > It's certainly possible to build presence services and directories on > top of a DHT. For example, Circle (thecircle.org.au) has an instant > messaging system built on top of a DHT (including the ability to > search by name or cryptographic id, and to publish a request to be > notified when a particular person comes online... basically > everything a normal IM system does). I had found about Circle thanks to the infoanarchy.org wiki, and I like its architecture (including the separation between daemon and GUI, which would represent a must for my intended use, and the addition of audio and video communications through GnomeMeeting); but I've got the impression that it's still in a quite experimental phase, with specs subject to change, and efficiency temporarily sacrificed to rapid prototyping (BTW, to speed up the crypto part while preserving multiplatform capability you might consider using the OpenSSL Python wrappers at http://www.freenet.org.nz/python/SSLCrypto/ ). Another reason why I'd like to start with Overnet is its very large users base (which includes also eMule and mldonkey users). Besides, Overnet supports a "firewalled mode" (not yet implemented in eMule) for leaf nodes that can't directly participate in the Kamdelia exchange of UDP packets but, from what I understand, can still use "non-firewalled" nodes as a sort of proxy, using only outbound TCP connections. This might be ideal for mobile nodes based on GPRS or UMTS, even if not firewalled: per-packet billing and the blizzard of UDP packets exchanged by full-fledged Overnet nodes make a dangerous combination... Rather than writing code to handle Overnet's Kamdelia protocol directly, I was thinking of relying, at least initially, upon an existing Overnet-capable daemon (such as overnetclc or mlnet) controlled through the "GUI TCP connection". From what I can understand in the source code of the Circle (file: plugins/donkey.py), you guys do something similar for gaining access to the eDonkey/Overnet network. Anyway, if the "directory/presence over DHT" abstraction layer is clean enough, it should be possible to switch, at a later time, to a different DHT implementation with relatively little effort. Cheers -- Enzo From tiraboschi at cefriel.it Mon Mar 22 11:01:51 2004 From: tiraboschi at cefriel.it (Simone Tiraboschi) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] Questions about Overnet In-Reply-To: <024001c40ff3$ac75fe20$0200a8c0@em.noip.com> References: <024001c40ff3$ac75fe20$0200a8c0@em.noip.com> Message-ID: <405EC79F.7050105@cefriel.it> Ciao, > -----Original Message----- > From: p2p-hackers-bounces@zgp.org > [mailto:p2p-hackers-bounces@zgp.org]On > Behalf Of Enzo Michelangeli > Sent: Monday, March 22, 2004 10:53 > To: Peer-to-peer development. > Subject: Re: [p2p-hackers] Questions about Overnet > Right, but how soon is "soon"? 5 seconds, 5 minutes, half > hour... And what > happens when the node(s) handling the section of DHT that stores that > piece of information go(es) offline? I'd like to pick the > brain of some > Overnet, eMule or mldonkey developer (or at least "power > user" :-) ), as > the protocol itself is very poorly documented - also because it was > originally closed source / closed specs, and the "Kad" part > of eMule 0.4x > and the mldonkey implementation where based on reverse-engineering. In Kademlia each node have to republish its information every hour. The republish process acts as a search, it lasts a time comparable to a search, something like 5 seconds. Information is replied in k nodes, if they go down all together before a new republish the information is lost. Overnet is the first application using kademlia but is closed source, MLDonkey implementation is based on reverse engineering of overnet but is in OCAML and I don't know OCAML. The "kad" part of emule is a new implementation of Kademlia not compatible with overnet. > BTW, what is the answer to my "publishing latency and persistence" > questions in Circle's case? And how would you compare Chord > to Kademlia? In Chord if the system is "stable" you are sure to get the information you are looking for but is not so simple to maintain a system stable, Kademlia instead rely on probabilistic assurance based on k replies. The publish latency is comparable O(log(N)). > Another reason why I'd like to start with Overnet is its very > large users > base (which includes also eMule and mldonkey users). eMule Kad is not compatible with overnet. ReverseConnect also use Kademlia. Ciao Simone From jj at diacronic.org Tue Mar 23 17:08:12 2004 From: jj at diacronic.org (jj porrdige) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] rengineering Scribe ( Pastry multicast ) Message-ID: <1080061692.40606efc8a477@hosting-01.alicomitalia.it> Hi, I am studying Scribe, in FreePastry 1.3.2 distribution, to build a p2p audio/video muticast service. I would like to modify Scribe to obtain these reqirements : 1. a not member group can NOT be a fowarder for that group; 2. a peer can forward data only to a limited set of members, limited outdegree of a member; 3. to mask the transient state of a member ( when it change parents ), to avoid packet loss; 4. if a peer detect a packet loss from a parent, it change the parent; I would like to obtain this requirements maintaing the original Scribe efficient trees. Have you any suggestion ? jj ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From atuls at cs.rice.edu Tue Mar 23 17:21:03 2004 From: atuls at cs.rice.edu (Atul Singh) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] rengineering Scribe ( Pastry multicast ) In-Reply-To: <1080061692.40606efc8a477@hosting-01.alicomitalia.it> References: <1080061692.40606efc8a477@hosting-01.alicomitalia.it> Message-ID: Hi, You can have a look at SplitStream project, which has almost the same requirement as what you have. The design incorporates the bandwidth constraints that participating nodes might have. Morevoer to mask the parent failure, it stripes the data into multiple sub-streams and reconstructs the full stream using sub-streams from multiple parents. Using MDC (reference in paper), the full stream can be reconstructed with quality proportional to the number of sub-streams used in reconstruction, hence even if some parents are offline or leave, a node can still reconstruct the original stream with some degradation in quality. For full details, please have a look at the paper. SplitStream : A high bandwidth multicast system for cooperative environments . M. Castro, P. Druschel, A.M. Kermarrec, A. Nandi, A. Rowstron, A. Singh , SOSP 2003 FreePastry 1.3.2 distribution provides an implementation of SplitStream, though the documentation is not comprehensive. The Scribe implementation provides a configurable policy where you can tune the behavior of the participating nodes according to your desire, i.e. not to become a part of group where you are not interested etc. I am sure that this will give you a framework which provides most of the desired properties you want to have in your system. Let me know if you have any further questions, Thanks, Atul On Tue, 23 Mar 2004, jj porrdige wrote: > > Hi, > > > > I am studying Scribe, in FreePastry 1.3.2 distribution, to build a p2p > audio/video muticast service. > > I would like to modify Scribe to obtain these reqirements : > > > > 1. a not member group can NOT be a fowarder for that group; > > 2. a peer can forward data only to a limited set of members, limited > outdegree of a member; > > 3. to mask the transient state of a member ( when it change parents ), to > avoid packet loss; > > 4. if a peer detect a packet loss from a parent, it change the parent; > > > > I would like to obtain this requirements maintaing the original Scribe efficient > trees. > > > > Have you any suggestion ? > > > jj > > ---------------------------------------------------------------- > This message was sent using IMP, the Internet Messaging Program. > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ It is amazing what you can accomplish if you do not care who gets the credit. - Harry S Truman ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ From Yves.Roudier at eurecom.fr Tue Mar 23 19:35:33 2004 From: Yves.Roudier at eurecom.fr (Yves.Roudier@eurecom.fr) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] ESORICS 2004 - Final Call for Papers Message-ID: <200403231935.i2NJZXq4026057@zinnia.eurecom.fr> [Apologies for multiple copies of this announcement] CALL FOR PAPERS ESORICS 2004 9th European Symposium on Research in Computer Security Institut Eurécom, Sophia Antipolis, French Riviera, France September 13-15, 2004 http://esorics04.eurecom.fr ESORICS 2004 will be collocated with RAID 2004 Papers offering novel research contributions in any aspect of computer security are solicited for submission to the Ninth European Symposium on Research in Computer Security (ESORICS 2004). Organized in a series of European countries, ESORICS is confirmed as the European research event in computer security. The symposium started in 1990 and has been held on alternate years in different European countries and attracts an international audience from both the academic and industrial communities. >From 2002 it will be held yearly. The Symposium has established itself as one of the premiere, international gatherings on Information Assurance. Papers may present theory, technique, applications, or practical experience on topics including: access control accountability anonymity applied cryptography authentication covert channels cryptographic protocols cybercrime data and application security data integrity denial of service attacks dependability digital right management firewalls formal methods in security identity management inference control information dissemination control information flow control information warfare intellectual property protection intrusion tolerance language-based security network security non-interference peer-to-peer security privacy-enhancing technology pseudonymity secure electronic commerce security administration security as quality of service security evaluation security management security models security requirements engineering security verification smartcards steganography subliminal channels survivability system security transaction management trust models and trust trustworthy user devices management policies The primary focus is on high-quality original unpublished research, case studies and implementation experiences. We encourage submissions of papers discussing industrial research and development. Proceedings will be published by Springer-Verlag in the Lecture Notes in Computer Science series. PAPER SUBMISSIONS Submitted papers must not substantially overlap papers that have been published or that are simultaneously submitted to a journal or a conference with proceedings. Papers should be at most 15 pages excluding the bibliography and well-marked appendices (using 11-point font), and at most 20 pages total. Committee members are not required to read the appendices, and so the paper should be intelligible without them. To submit a paper, send to esorics04@dti.unimi.it a plain ASCII text email containing the title and abstract of your paper, the authors' names, email and postal addresses, phone and fax numbers, and identification of the contact author. To the same message, attach your submission (as a MIME attachment) in PDF or portable postscript format. Do NOT send files formatted for word processing packages (e.g., Microsoft Word or WordPerfect files). Submissions not meeting these guidelines risk rejection without consideration of their merits. Submissions must be received by March 26, 2004 in order to be considered. Notification of acceptance or rejection will be sent to authors by May 30, 2004. Authors of accepted papers must be prepared to sign a copyright statement and must guarantee that their paper will be presented at the conference. Authors of accepted papers must follow the Springer Information for Authors' guidelines for the preparation of the manuscript and use the templates provided there. ORGANIZING COMMITTEE General Chair Refik Molva Institut Eurécom email: Refik.Molva@eurecom.fr Program Chairs Peter Ryan Pierangela Samarati University of Newcastle upon Tyne University of Milan email: Peter.Ryan@newcastle.ac.uk email: samarati@dti.unimi.it Publication Chair Publicity Chair Dieter Gollmann Yves Roudier TU Hamburg-Harburg Institut Eurécom email: diego@tuhh.de email: roudier@eurecom.fr Sponsoring Chair Marc Dacier Institut Eurécom email: dacier@eurecom.fr PROGRAM COMMITTEE Vijay Atluri, Rutgers University, USA Joachim Biskup, Universitaet Dortmund, Germany Jan Camenisch, IBM Research, Switzerland David Chadwick, University of Salford, UK Ernesto Damiani, University of Milan, Italy Sabrina De Capitani di Vimercati, University of Milan, Italy Yves Deswarte, LAAS-CNRS, France Alberto Escudero-Pascual, Royal Institute of Technology, Sweden Simon Foley, University College Cork, Ireland Dieter Gollmann, TU Hamburg-Harburg, Germany Joshua D. Guttman, MITRE, USA Sushil Jajodia, George Mason University, USA Sokratis K. Katsikas, University of the Aegean, Greece Peng Liu, Pennsylvania State University, USA Javier Lopez, University of Malaga, Spain Roy Maxion, Carnegie Mellon University, USA Patrick McDaniel, AT&T Labs-Research, USA John McHugh, CERT/CC, USA Catherine A. Meadows, Naval Research Lab, USA Refik Molva, Institut Eurécom, France Peng Ning, NC State University, USA LouAnna Notargiacomo, The MITRE Corporation, USA Eiji Okamoto, University of Tsukuba, Japan Stefano Paraboschi, University of Bergamo, Italy Andreas Pfitzmann, TU Dresden, Germany Jean-Jacques Quisquater, Microelectronic laboratory, Belgium Steve Schneider, University of London, UK Christoph Schuba, Sun Microsystems, Inc., USA Michael Steiner, IBM T.J. Watson Research Laboratory, USA Paul Syverson, Naval Research Laboratory, USA Moti Yung, Columbia University, USA IMPORTANT DATES Paper Submission due: March 26, 2004 Acceptance notification: May 30, 2004 Final papers due: June 30, 2004 From Yves.Roudier at eurecom.fr Tue Mar 23 19:39:39 2004 From: Yves.Roudier at eurecom.fr (Yves.Roudier@eurecom.fr) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] RAID 2004 - Final Call for Papers Message-ID: <200403231939.i2NJddwd026159@zinnia.eurecom.fr> [Apologies for multiple copies of this announcement] CALL FOR PAPERS RAID 2004 "Intrusion Detection and Society" Seventh International Symposium on Recent Advances in Intrusion Detection Institut Eurécom, Sophia-Antipolis, French Riviera, France September 15-17, 2004 http://raid04.eurecom.fr This symposium, the seventh in an annual series, brings together leading researchers and practitioners from academia, government, and industry to discuss intrusion detection technologies and issues from the research and commercial perspectives. The RAID International Symposium series is intended to further advances in intrusion detection by promoting the exchange of ideas in a broad range of topics. For RAID 2004 there is a special theme: the interdependence between intrusion detection and society. Thus, we will also welcome papers that address issues that arise when studying intrusion detection, including information gathering and monitoring, as a part of a larger, not necessarily purely technical, perspective. For example, the implication of information gathering and detection technologies on enterprises, organisations and authorities, as well as legislative and governing bodies is within scope, but also the impact and restrictions from those bodies on the design and technology. This would include issues such as privacy, risk and emergency management, crisis management, security policies, standardisation and legal issues. An increasingly important dynamic is the strategic importance of protecting national information infrastructures, which is in some tension with the fact that much of this infrastructure is in the private sector. Related to this is the potential strategic impact of attacks at the intersection of information and physical infrastructure. The RAID 2004 program committee invites three types of submissions: - Full papers presenting mature research results. Papers accepted for presentation at the Symposium will be included in the RAID 2004 proceedings published by Springer Verlag in its Lecture Notes in Computer Science (LNCS) series. Full papers are limited to 20 pages when formatted according to the instructions provided by Springer Verlag. Papers must include an abstract and a list of keywords. - Practical experience reports describing a valuable experience or a case study, such as the design and deployment of a system or actual experience from intrusion detection or network monitoring. These reports are reviewed differently from full papers and do not necessarily include fundamental scientific contributions or new research ideas. Practical experience reports are limited to 12 pages when formatted according to the instructions provided by Springer Verlag. They must include an abstract and a list of keywords. - Panel proposals for presenting and discussing hot topics in the field of intrusion detection systems. The panel proposals should include both an outline of the format of the panel and a short rationale for the panel. Panels that include time for general discussion and questions/answers between the panelists and the Symposium attendees are preferred. All topics related to Intrusion Detection Systems and Technologies are within scope, including their design, use and maintenance, integration, correlation and self-protection, just to mention a few. With reference to this year's theme and extended scope we also invite papers on the following topics, as they bear on intrusion detection and the general problem of information security: Risk assessment and risk management Intrusion tolerance Deception systems and honeypots Privacy aspects Data mining techniques Visualization techniques Cognitive approaches Biological approaches Self-learning Case studies Legal issues Critical infrastucture protection (CIP) ORGANIZING COMMITTEE General Chair: Refik Molva Program Chairs: Erland Jonsson Alfonso Valdes Publication Chair: Magnus Almgren Publicity Chair: Yves Roudier Sponsor Chair: Marc Dacier PROGRAM COMMITTEE Tatsuya Baba (NTT Data, Japan) Lee Badger (DARPA, USA) Sungdeok Cha (KAIST, Korea) Steven Cheung (SRI International, USA) Herve Debar (France Telecom R&D, France) Simone Fischer-Hübner (Karlstad University, Sweden) Steven Furnell (University of Plymouth, UK) Bill Hutchinson (Edith Cowan University, Australia) Dogan Kesdogan (RWTH Aachen, Germany) Chris Kruegel (UCSB, USA) Håkan Kvarnström (TeliaSonera R&D, Sweden) Wenke Lee (Georgia Tech, USA) Douglas Maughan (DHS HSARPA, USA) Roy Maxion (Carnegie Mellon University, USA) John McHugh (CMU/SEI CERT, USA) Ludovic Me (Supélec, France) George Mohay (Queensland University of Technology, Australia) Vern Paxson (ICSI and LBNL, USA) Giovanni Vigna (UCSB, USA) Andreas Wespi (IBM Research, Switzerland) Felix Wu (UC Davis, USA) Diego Zamboni (IBM Research, Switzerland) IMPORTANT DATES Deadline for paper submission : March 31, 2004 Deadline for panel submission : April 30, 2004 Deadline for poster submission : August 20, 2004 Notification of acceptance or rejection : June 4, 2004 Final paper camera ready copy : July 2, 2004 RAID conference dates : September 15-17, 2004 SUBMISSIONS Submissions must not substantially duplicate work that any of the authors has published elsewhere or has submitted in parallel to any other conference or workshop with proceedings. The full papers must list all authors and their affiliations; in case of multiple authors, the contact author must be indicated (note that RAID does not require anonymized submissions). Authors are invited to submit their papers electronically. A detailed description of the electronic submission procedure is available at http://raid04.eurecom.fr/submit.html. Submissions must conform to this procedure and be received within the submission deadline in order to be considered. Each submission will be acknowledged by e-mail. If acknowledgment is not received within seven days, please contact RAID 2004's PC Chair . Poster submissions should be sent as a half-page abstract. For submission or practical details, please contact Marc Dacier . All submissions and presentations must be in English. CORPORATE SPONSORS We solicit interested organizations to serve as sponsors for RAID 2004, particularly in sponsorship of student travel and other expenses for RAID. Please contact the Sponsor Chair, Marc Dacier , for information regarding corporate sponsorship of RAID 2004. REGISTRATION Detailed registration information (including fees, suggested hotels, and travel directions) will be provided at the RAID 2004 web site (http://raid04.eurecom.fr). PROCEEDINGS Accepted papers will be published by Springer Verlag in its Lecture Notes in Computer Science (LNCS) series. Instructions for authors will be provided at the RAID 2004 web site (http://raid04.eurecom.fr). STEERING COMMITTEE Chair: Marc Dacier (Eurecom, France) Hervé Debar (France Telecom R&D, France) Deborah Frincke (University of Idaho, USA) Huang Ming-Yuh (The Boeing Company, USA) Wenke Lee (Georgia Institute of Technology, USA) Ludovic Mé (Supélec, France) S. Felix Wu (UC Davis, USA) Andreas Wespi (IBM Research, Switzerland) Giovanni Vigna (UCSB, USA) For further information, please contact the Program Chairs or the General Chair. ******************************************************************** From jj at diacronic.org Tue Mar 23 20:51:55 2004 From: jj at diacronic.org (jj porrdige) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] rengineering Scribe ( Pastry multicast ) In-Reply-To: <20040323200004.BD2713FD32@capsicum.zgp.org> References: <20040323200004.BD2713FD32@capsicum.zgp.org> Message-ID: <1080075115.4060a36b80837@hosting-01.alicomitalia.it> Hi Atul, SplitStream is a very interesting system but where can I find a software implementation of a MDC codec/encoder ? About FreePastry 1.3.2: 1) In your new Scribe implementation built on CommonAPI (package rice.p2p.scribe) there is not the old method create( NodeId topicId, Credentials cred) to make a new group. Which new method make a new group at the "root" ? And where must I put the "credentials" in the new API ? 2) Do you think that next FreePastry release incorporates techniques to achieve good performance and high dependability in realistic environments with high churn rates (like MSPastry or Bamboo-Dht) ? ("Performance and dependability of structured peer-to-peer overlays", Miguel Castro; MSPastry provides low delay routing with strong consistency guarantees and low overhead in realistic environments with high churn rates) Many thanks, JJ > > Message: 2 > Date: Tue, 23 Mar 2004 11:21:03 -0600 (CST) > From: Atul Singh > Subject: Re: [p2p-hackers] rengineering Scribe ( Pastry multicast ) > To: "Peer-to-peer development." > Message-ID: > Content-Type: TEXT/PLAIN; charset=US-ASCII > > Hi, > > You can have a look at SplitStream project, which has almost the same > requirement as what you have. The design incorporates the bandwidth > constraints that participating nodes might have. Morevoer to mask the > parent failure, it stripes the data into multiple sub-streams and > reconstructs the full stream using sub-streams from multiple parents. > Using MDC (reference in paper), the full stream can be reconstructed with > quality proportional to the number of sub-streams used in reconstruction, > hence even if some parents are offline or leave, a node can still > reconstruct the original stream with some degradation in quality. For full > details, please have a look at the paper. > > SplitStream : A high bandwidth multicast system for cooperative > environments . M. Castro, P. Druschel, A.M. Kermarrec, A. Nandi, A. > Rowstron, A. Singh , SOSP 2003 > > FreePastry 1.3.2 distribution provides an implementation of SplitStream, > though the documentation is not comprehensive. The Scribe implementation > provides a configurable policy where you can tune the behavior of the > participating nodes according to your desire, i.e. not to become a part of > group where you are not interested etc. I am sure that this will give you > a framework which provides most of the desired properties you want to have > in your system. > > Let me know if you have any further questions, > Thanks, > Atul > jj ---------------------------------------------------------------- This message was sent using IMP, the Internet Messaging Program. From Paul.Harrison at infotech.monash.edu.au Sat Mar 27 23:35:45 2004 From: Paul.Harrison at infotech.monash.edu.au (Paul Harrison) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] Questions about Overnet In-Reply-To: <024001c40ff3$ac75fe20$0200a8c0@em.noip.com> Message-ID: On Mon, 22 Mar 2004, Enzo Michelangeli wrote: > ----- Original Message ----- > From: "Paul Harrison" > To: "Peer-to-peer development." > Sent: Sunday, March 21, 2004 1:07 PM > > [...] > > Overnet is based on Kademlia, a Distributed Hash-Table. So, assuming > > it's a robust implementation, searches will be exhaustive and content > > will be visible as soon as it is published. > > Right, but how soon is "soon"? 5 seconds, 5 minutes, half hour... And what > happens when the node(s) handling the section of DHT that stores that > piece of information go(es) offline? I'd like to pick the brain of some > Overnet, eMule or mldonkey developer (or at least "power user" :-) ), as > the protocol itself is very poorly documented - also because it was > originally closed source / closed specs, and the "Kad" part of eMule 0.4x > and the mldonkey implementation where based on reverse-engineering. > > BTW, what is the answer to my "publishing latency and persistence" > questions in Circle's case? And how would you compare Chord to Kademlia? > In Circle, publishing is fairly fast, i expect usually well under five seconds. It persists until the publishing node logs off the network (the publishing node gets pinged every so often to see if it's still up). There's always the possibility of a slow node making publishing slow, or an unreliable node not handing over published keys when it drops out. The way to avoid this would be to publish to multiple points on the hashtable simultaneously (Circle does this for things that need to be high reliability). afaik, there's not much theoretical difference in performance or reliability between Chord and Kadmelia. I personally prefer Chord because it's easier to visualize. > > It's certainly possible to build presence services and directories on > > top of a DHT. For example, Circle (thecircle.org.au) has an instant > > messaging system built on top of a DHT (including the ability to > > search by name or cryptographic id, and to publish a request to be > > notified when a particular person comes online... basically > > everything a normal IM system does). > > I had found about Circle thanks to the infoanarchy.org wiki, and I like > its architecture (including the separation between daemon and GUI, which > would represent a must for my intended use, and the addition of audio and > video communications through GnomeMeeting); but I've got the impression > that it's still in a quite experimental phase, with specs subject to > change, and efficiency temporarily sacrificed to rapid prototyping (BTW, > to speed up the crypto part while preserving multiplatform capability you > might consider using the OpenSSL Python wrappers at > http://www.freenet.org.nz/python/SSLCrypto/ ). > Yeah... I'm no longer working on Circle, i'm not sure what the current developers are doing. It was an experimental project when i was working on it, but the design i was aiming at dotted all the i's and crossed all the t's, as it were, for a directory service such as you describe. > Another reason why I'd like to start with Overnet is its very large users > base (which includes also eMule and mldonkey users). Besides, Overnet > supports a "firewalled mode" (not yet implemented in eMule) for leaf nodes > that can't directly participate in the Kamdelia exchange of UDP packets > but, from what I understand, can still use "non-firewalled" nodes as a > sort of proxy, using only outbound TCP connections. This might be ideal > for mobile nodes based on GPRS or UMTS, even if not firewalled: per-packet > billing and the blizzard of UDP packets exchanged by full-fledged Overnet > nodes make a dangerous combination... > Ok. I put forward Circle as a proof that it's possible, not necessarily as the thing you should actually use. :-) Something that occurs to me: Overnet and Circle are designed for large numbers of low reliability keys (several per file, many files per node). It's possible to use a simpler and more reliable design if you don't have large numbers of keys: use a DHT without the hashtable :-). Have each node id represent a published key (each client would maintain several nodes). A node can achieve as high a reliability level as it wants by informing as many neighbours as it wants of its existence. cheers, Paul pfh@logarithmic.net | http://www.logarithmic.net/pfh end apartheid From ncbgroups at yahoo.com.br Mon Mar 29 16:55:13 2004 From: ncbgroups at yahoo.com.br (=?iso-8859-1?q?Nilton=20Braga?=) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] neighbors in gnutella In-Reply-To: Message-ID: <20040329165513.67318.qmail@web40006.mail.yahoo.com> Hi. Does anybody knows how many neighbors one node in a gnutella network has? More: and in other p2p networks, like kazaa, emule and soulseek? Thanks in advance. ______________________________________________________________________ Yahoo! Mail - O melhor e-mail do Brasil! Abra sua conta agora: http://br.yahoo.com/info/mail.html From tutschku at informatik.uni-wuerzburg.de Mon Mar 29 17:56:57 2004 From: tutschku at informatik.uni-wuerzburg.de (Kurt Tutschku) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] CfP: ETT Special Issue on P2P Networking and P2P Services Message-ID: <005101c415b7$3a266610$b46abb84@musa> Please, accept our apologies if multiple copies are received. *********************************************************************** Extension of deadline for paper submissions: April 15, 2004 *********************************************************************** European Transactions on Telecommunications (ETT) (http://www.interscience.wiley.com/) Special Issue on P2P Networking and P2P Services Due to multiple requests, the deadline for submitting papers has been extended to April 15, 2004. The call for paper (CFP) is available at: http://www-info3.informatik.uni-wuerzburg.de/staff/tutschku/ETTCFPP2P.pd f Electronic submission of pdf files is strongly encouraged. Please send your document to: tutschku@informatik.uni-wuerzburg.de The following new deadlines will apply: - Submission of manuscripts April 15, 2004 - Notification of acceptance June 15, 2004 - Submission of final manuscript July 15, 2004 - Publication 3rd quarter 2004 Kurt Tutschku (Coordinating guest editor) Depart. Distributed Systems, Institute of Computer Science, University of Wuerzburg, Germany. tutschku@informatik.uni-wuerzburg.de Hermann de Meer Faculty of Mathematics and Computer Science, University of Passau, Germany. demeer@fmi.uni-passau.de Frank-Uwe Andersen ICM - Future IP technologies - Siemens AG, Germany. Frank-Uwe.Andersen@siemens.com Konosuke Kawashima Dept. of Computer, Information and Communication Sciences Tokyo University of Agriculture & Technology, Japan. k-kawa@cc.tuat.ac.jp --- Dr. Kurt Tutschku University of Wuerzburg Department of Distributed Systems Institute of Computer Science Am Hubland 97074 Wuerzburg Germany Tel.: +49-931-8886641 FAX.: +49-931-8886632 mailto:tutschku@informatik.uni-wuerzburg.de or mailto:kurttutschku@hotmail.com http://www-info3.informatik.uni-wuerzburg.de/staff/tutschku From jcea at argo.es Wed Mar 31 13:04:10 2004 From: jcea at argo.es (Jesus Cea Avion) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] About DTH rings subversion attacks References: Message-ID: <406AC1CA.DC5488A5@argo.es> > afaik, there's not much theoretical difference in performance or > reliability between Chord and Kadmelia. I'm concerned about malicious nodes. That is, nodes subverting the correct ring behaviour, like searches and joins. Reading about Chord and Kademlia, it seems that kademlia could be more reliable because it uses several peers for each DTH bucket. If each kademlia node use several peers for ring operation, and the bootstrap nodes are reliable, it should keep working with malicious nodes. Any idea or experience?. -- Jesus Cea Avion _/_/ _/_/_/ _/_/_/ jcea@argo.es http://www.argo.es/~jcea/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ _/_/_/_/_/ PGP Key Available at KeyServ _/_/ _/_/ _/_/ _/_/ _/_/ "Things are not so easy" _/_/ _/_/ _/_/ _/_/ _/_/ _/_/ "My name is Dump, Core Dump" _/_/_/ _/_/_/ _/_/ _/_/ "El amor es poner tu felicidad en la felicidad de otro" - Leibniz From bryan.turner at pobox.com Wed Mar 31 17:38:16 2004 From: bryan.turner at pobox.com (Bryan Turner) Date: Sat Dec 9 22:12:39 2006 Subject: [p2p-hackers] About DTH rings subversion attacks In-Reply-To: <406AC1CA.DC5488A5@argo.es> Message-ID: Jesus, It may appear that Kademlia [1] is more reliable than Chord [2] on the surface, but in reality the problem is not as simple as that. It has been shown [3] that any non-authenticated protocol may be subverted if at least 1/3 of the participants are rogue entities. It has also been shown [4] that for a general P2P network, only a single rogue entitiy is needed to subvert it by creating 'faces' which appear to be other nodes, but are actually the same entity. Thus, a rogue need only join the network once, then allow other rogue entities (or faces) to join through it until the rogue entities make up 1/3 of the nodes in the network; at which time they can bring the entire network down. Chord attempts to alleviate this by binding the Node ID to the IP Address, but this is not a reliable strategy as any network admin will tell you. Kademlia, likewise, expects this problem to be solved outside the protocol. A more direct answer to your question would be that Kademlia is capable of surviving more individual rogue entities (each with very limited resources) than the basic Chord protocol. However, Chord may be fortified in the same manner as Kademlia, bringing that difference to naught. Additional fortification [5] brings a byzantine-agreement like protocol into the network during the lookup phase (called "spam-resistant lookup"). This allows you to survive any number of individual or collective rogue entities up to the byzantine limit. (or close to it, I don't recall a proof in the paper) To break the byzantine barrier you will need an entirely different security model with an identity authority. But this moves away from the pure-P2P model that most DHTs aim for. To my knowledge, there are no decentralized, secure identity authorities in the literature. (Please correct me if I'm wrong!) --Bryan bryan.turner@pobox.com References (available via http://citeseer.ist.psu.edu): 1. "Kademlia: A Peer-to-peer Information System Based on the XOR Metric" Petar Maymounkov, David Mazieres 2. "Chord: A Scalable Peer-to-peer Lookup Protocol for Internet Applications" Ion Stoica, et. al. 3. "The Byzantine Generals Problem" Leslie Lamport, et. al. 4. "The Sybil Attack" John Douceur 5. "A Simple Fault Tolerant Distributed Hash Table" Moni Naor, Udi Wieder -----Original Message----- From: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org]On Behalf Of Jesus Cea Avion Sent: Wednesday, March 31, 2004 8:04 AM To: Peer-to-peer development. Subject: [p2p-hackers] About DTH rings subversion attacks > afaik, there's not much theoretical difference in performance or > reliability between Chord and Kadmelia. I'm concerned about malicious nodes. That is, nodes subverting the correct ring behaviour, like searches and joins. Reading about Chord and Kademlia, it seems that kademlia could be more reliable because it uses several peers for each DTH bucket. If each kademlia node use several peers for ring operation, and the bootstrap nodes are reliable, it should keep working with malicious nodes. Any idea or experience?. -- Jesus Cea Avion