From hal at finney.org Thu Mar 1 09:08:05 2001 From: hal at finney.org (hal@finney.org) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] Oceanstore's routing (was Re: [freenet-devl] updating with access control lists) Message-ID: <200103011702.JAA19993@finney.org> Oskar writes: > What I meant was going backwards. So if your node had label 012345 (base 6 > now), you would find the node on the network that was closest, maybe > with label 012344 (it would be at the end of primary neighbor sequence > for your id). Then you check it's backwards neighbors in the 6th > level, ie those nodes that have it as a primary reference in the 6th > level, and you have all the nodes with the same id as you to 5th, from > which you can choose your primary references in the 6th level, and so on. I see how this would work. You are essentially enumerating all the nodes in the network by picking one as the root and walking the tree down. By itself this would be O(n), but you might be able to apply some heuristics based on assumed locality in the tree, since nodes especially in the lower branches are supposed to be physically close together. That's where you need pruning the most since it is where the number of nodes becomes large. Thanks for the clarification - Hal From wesley at felter.org Thu Mar 1 20:41:01 2001 From: wesley at felter.org (Wesley Felter) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] More on OceanStore routing Message-ID: It turns out that the OceanStore team is being pretty lazy about updating their publications page; the individual team members have various nuggets about the project on their home pages. In particular, here's a page on Tapestry (their variant of the Plaxton mesh) that might be more up-to-date than the overview paper: http://www.cs.berkeley.edu/~ravenben/tapestry/index.html Wesley Felter - wesley@felter.org - http://felter.org/wesley/ From sam at neurogrid.com Wed Mar 7 18:47:01 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] NeuroGrid Message-ID: <3AA6F261.3B5189A9@neurogrid.com> Hi All, Guess this might be the first mail huh? Actually it might now be the second as I just had to reconfigure my email address to post and this is my second attempt. Well, let's get down to it. I have a P2P system that I am trying to hack together ready for more general public release next Thursday (some of you may remember me from the O'Reilly conference, the British guy asking people to look at neurogrid.com in one month during the lightning talks). I say public release, I mean that I will tell more people about it. This is still a prototype and mainly for hackers like you and I (as opposed to investors, regular people etc.) So the thing is that I have made a NeuroGrid node which is a node of a p2p distributed search system. Which is great, but it's a bit difficult to show the amazing advantages of distributed search when you have only one node. I'll have more explanation and documents up on the web site next week, but the demo is basically ready - okay, okay it needs tidying up. You can check it out here http://www.neurogrid.net/servlet/com.neurogrid.prime.SurfingServlet Try typing in the keyword "search" and see what happens. Well maybe it doesn't work, or it sucks. It's a prototype okay. Well maybe you get the idea. The idea is that you can search for URLs and other search engines at the same time. At the moment this node only contains a few of my bookmarks and some search engines I entered. It has an adaptive mechanism so that as you search, the things you select gradually shift to the top of subsequent selections. The node is designed to be like for a hundred or so users, maybe, who can search purely their own stuff, or more generally within the node. I want to make a single-user download to your PC version in the future. The cool thing is that as other NeuroGrid nodes get set up as well, as recommending URLs to each other (like other search engines) they can recommend other nodes to each other. With enough Neurogrid nodes the user can hop from search engine to search engine through the web browser interface, with each recommendation based on the most popular search engines/nodes (for given query words) at each node. Hey, well, I think it's cool. What I'm really looking for now is some people who want to set up NeuroGrid nodes. At the moment you need a servlet engine (I use tomcat) and a database (I use mySQL), with some tweaking the system will run with other combinations, but I don't promise anything right now. What I was hoping was that somebody with some interest in the project, who might happen to have the resources to put together a NeuroGrid node would be interested in helping set up at least one more node before the end of next week. Naturally you would get full support from me in setting the system up, and then maybe we could test the communication between the nodes. Get in touch if you can help me out, or just want to comment on the system. CHEERS> SAM From sam at neurogrid.com Thu Mar 8 02:17:01 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] Scalability vs Network Stability Message-ID: <3AA75BD8.C12FF96E@neurogrid.com> Justin Chapweske wrote: > One thing that seems to be missing from these conversations on routing and > lookup is some quantifiable measure of network stability. A lot of these > routing systems being proposed seem nice from a pure scalabilty > perspective but I doubt that most of them will work in any sort of > unstable network environment. > Does anyone have any good quantifiable definition of network stability > that we can use to benchmark our systems against? I am unclear about what constitutes a stable network. Do you mean for example the amount of time in a given day that a particular node isconnected to the network? The amount of traffic that the average node can support over a given time period? The variance in the amount of traffic? I mean I'm no network systems expert (I'm agents and neural nets), but clearly one could come up with all sorts of measures. Can you explain a little further what you mean by stability? For what's it worth, to say that a network is scalable is also pretty much undefined. I guess we can say something like the average amount of traffic generated for a given number of nodes, but I guess it's more about people's perception of whether they are having their needs met my the system, as the number of participants continues to grow; if lot's of people can't contact each other, or find the things they need as the system grows then I guess the thing is failing to scale. But this is then also a function of what people actually expect the system to for them. If they expect to be in contact with their close circle of friends, they may not care that they can't contact people on the other side of the network, right? My apologies if this is all completely obvious, but I'm just pitching around for shared concepts so we can start thinking about this. CHEERS> SAM From md98-osa at nada.kth.se Thu Mar 8 04:49:02 2001 From: md98-osa at nada.kth.se (Oskar Sandberg) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] NeuroGrid In-Reply-To: <3AA6F261.3B5189A9@neurogrid.com>; from sam@neurogrid.com on Thu, Mar 08, 2001 at 11:45:53AM +0900 References: <3AA6F261.3B5189A9@neurogrid.com> Message-ID: <20010308135056.E800@hobbex.localdomain> On Thu, Mar 08, 2001 at 11:45:53AM +0900, Sam Joseph wrote: <> > The cool thing is that as other NeuroGrid nodes get set up as well, as > recommending URLs to each other (like other search engines) they can > recommend other nodes to each other. > > With enough Neurogrid nodes the user can hop from search engine to > search engine through the web browser interface, with each > recommendation based on the most popular search engines/nodes (for given > query words) at each node. Hey, well, I think it's cool. How do the nodes learn which other nodes are most popular for a given query words? -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg md98-osa@nada.kth.se From md98-osa at nada.kth.se Thu Mar 8 05:02:02 2001 From: md98-osa at nada.kth.se (Oskar Sandberg) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] Scalability vs Network Stability In-Reply-To: <3AA75BD8.C12FF96E@neurogrid.com>; from sam@neurogrid.com on Thu, Mar 08, 2001 at 07:15:53PM +0900 References: <3AA75BD8.C12FF96E@neurogrid.com> Message-ID: <20010308140409.F800@hobbex.localdomain> On Thu, Mar 08, 2001 at 07:15:53PM +0900, Sam Joseph wrote: <> > I am unclear about what constitutes a stable network. Do you mean for > example the amount of time in a given day that a particular node > isconnected to the network? The amount of traffic that the average node > can support over a given time period? The variance in the amount of > traffic? The big issue is the rate at which nodes come and go. I don't know what sort of statistics Napster has in this regard, but I have the feeling that the average Napster client is only logged in for a few hours at a time - making it almost impossible to set up any intelligent routing system between them (Napster doesn't have to, but a decentralized system has to). Perhaps we should try to milk the OpenNap admins of some statistics regarding this (I doubt that Napster would themselves would give it out). > I mean I'm no network systems expert (I'm agents and neural nets), but > clearly one could come up with all sorts of measures. Can you explain a > little further what you mean by stability? > > For what's it worth, to say that a network is scalable is also pretty > much undefined. I guess we can say something like the average amount of > traffic generated for a given number of nodes, but I guess it's more > about people's perception of whether they are having their needs met my > the system, as the number of participants continues to grow; if lot's of > people can't contact each other, or find the things they need as the > system grows then I guess the thing is failing to scale. But this is > then also a function of what people actually expect the system to for > them. If they expect to be in contact with their close circle of > friends, they may not care that they can't contact people on the other > side of the network, right? Your right of course, no network is going to be infinitely scalable (unless the time it takes to search it is constant with size - which I can't imagine), so it's really subjective what people define as scalable. However, peoples experience with Gnutella has proved quite clearly that worrying about how far a p2p system scales is not just an academic issue. Personally, my rule of thumb is that I consider a system to be scalable if the amount of work per action grows at a rate that is less than a rational expression of the size of the network. Of course, there could easily be systems with rational expression growth that still scale for most current uses - but I prefer to look at the long term. > My apologies if this is all completely obvious, but I'm just pitching > around for shared concepts so we can start thinking about this. > > CHEERS> SAM > > > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg md98-osa@nada.kth.se From sam at neurogrid.com Thu Mar 8 06:17:01 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] NeuroGrid References: <3AA6F261.3B5189A9@neurogrid.com> <20010308135056.E800@hobbex.localdomain> Message-ID: <3AA79431.36844988@neurogrid.com> Oskar Sandberg wrote: > On Thu, Mar 08, 2001 at 11:45:53AM +0900, Sam Joseph wrote: > > > With enough Neurogrid nodes the user can hop from search engine to > > search engine through the web browser interface, with each > > recommendation based on the most popular search engines/nodes (for given > > query words) at each node. Hey, well, I think it's cool. > > How do the nodes learn which other nodes are most popular for a given > query words? Whenever anybody does a search they get recommended a list of nodes. If a user clicks through on a node then that event is recorded, and each NeuroGrid node maintains a set of relations between search keywords and other nodes that it knows about, updating it as users perform searches. Of course that is only a very superficial measure of which nodes are related to which keywords. Beyond that, when the user clicks (or more importantly bookmarks) a url recommended by the recommended node, then the strength of association between the search keywords and the recommended node (as well as the recommended url) are increased further. There is more to it, but I'll have full (well, maybe more) documentation up on my site next week. Right now I was hoping to get someone to set up another node with me so that we could demonstrate the system, so perhaps I can give you a better answer after next Thursday, because there's a lot for me to do before then CHEERS> SAM From jim at at.org Thu Mar 8 14:51:01 2001 From: jim at at.org (Jim Carrico) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] potlatch protocol In-Reply-To: <3AA79431.36844988@neurogrid.com> References: <3AA6F261.3B5189A9@neurogrid.com> <20010308135056.E800@hobbex.localdomain> Message-ID: hi folks warning: spam alert I've been propagating a particular meme, and wish to bring it before your assembled brainpower. Some of you have been previously ambushed - either at the o'reilly p2p con, or on the decentralization list - apologies if this is redundant and/or uninteresting - if so please ignore. My point is that for artists and creators to feel the warm fuzzies when they think about p2p, rather than a litigious rage, we need to develop a way to *pay them*. One way to do this would be to charge a flat fee, subscription-style, for a p2p service, and divide the money up among the various stake-holders - this is the presumed 'riaa-approved napster' formula. Difficulties with this include the fact that all parties would have to agree on how this money is to be allocated, a sufficient explanation for the fact that napster has not yet offered such a service. (If and when the major labels agree on how to re-implement their distribution cartel online, one might have doubts about the place of artists and creators in such a scheme - but that's a kettle of fish i don't want to shoot into right now) What i'm proposing is a micropayment system which is flexible enough to accomodate direct transactions (ie. pay-per-download) as well as indirect transactions (ie. "tipping") in which file distribution is de-coupled from payment; and that is based on simple, open, non-proprietary interfaces with no lock-in and no self-appointed gate-keepers. draft 0.1 is here- http://potlatch.net/protocol.01.html The emphasis on the voluntary nature of these payments says nothing about the protocol itself, and everything about the current climate with respect to direct payment for digital products over the public networks - that is to say, if people don't want to pay, it will be very difficult to force them. From this perspective, even the subscription model outlined above will be essentially "voluntary" - a choice made by customers on the basis of convenience, trust, morality, or whatever incentives they may be offered. A key to this proposal is the notion of an open market for aggregation of these micro-payments (actually digitally signed promissory notes) - it seems to me that aggregation nullifies much of Clay Shirky's "case against micropayments" - ie. the "attention cost" of micropayments, while retaining the granularity of incentives that micropayments imply. The proposal also implies a reputation system - a thread was started on advogato.org on the subject of "moneyflow" the same day that i posted the above document to my site, and I became entangled in the discussion after finding it in my referrer logs - a very interesting coincidence given the degree to which the advogato folks have layed a foundation for this in their "trust metric" analysis. I'm working closely with independant musicians who are not afraid of file-sharing, who want to build a business model that assumes unlimited distribution of their work, and that appeals directly to fans to both support and promote them. Besides establishing 'substantial non-infringing use' protections for freenet, mojonation et al, this is an opportunity for the hacker community and the independant music community to join forces and solve some problems. Jim Carrico http://www.potlatch.net From sam at neurogrid.com Thu Mar 8 17:45:01 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:41 2006 Subject: [Fwd: Re: [p2p-hackers] Scalability vs Network Stability] Message-ID: <3AA83557.2DC09593@neurogrid.com> Oskar Sandberg wrote: > On Thu, Mar 08, 2001 at 07:15:53PM +0900, Sam Joseph wrote: > <> > > I am unclear about what constitutes a stable network. Do you mean for > > example the amount of time in a given day that a particular node > > isconnected to the network? The amount of traffic that the average node > > can support over a given time period? The variance in the amount of > > traffic? > > The big issue is the rate at which nodes come and go. I don't know what > sort of statistics Napster has in this regard, but I have the feeling that > the average Napster client is only logged in for a few hours at a time - > making it almost impossible to set up any intelligent routing system > between them (Napster doesn't have to, but a decentralized system has to). > > Perhaps we should try to milk the OpenNap admins of some statistics > regarding this (I doubt that Napster would themselves would give it out). So we might classify a stable network in terms of the average time that a node is connected over a day, as well as the variance. Well, in fact we could give each node a stability rating in terms of its own average connecitivity and variance, and the network stability would then be a function of the stability ratings of the nodes that participate ... > peoples experience with Gnutella has proved quite clearly that > worrying about how far a p2p system scales is not just an academic issue. > Personally, my rule of thumb is that I consider a system to be scalable if > the amount of work per action grows at a rate that is less than a rational > expression of the size of the network. Of course, there could easily be > systems with rational expression growth that still scale for most current > uses - but I prefer to look at the long term. Sure, but so we come to the idea that scalability is a function of the operation one is considering. The possible functions are Search - for items, other nodes (includes node discovery) Download/Upload - transfering an item from one location to another Chat - maintaining a rebroadcasting service for some subgroup of nodes Other things I'm overlooking? So one can imagine a network architecture that was scalable for Download/Upload but not for search, e.g. it had a data re-distribution method like swarmcast or FreeNet's dynamic caching, but searched by explosively distributing all search queries like Gnutella. Naturally each of these functions can be broken down further so that search can be subclassified as search where one attempts to query all the nodes in the system, search where one limits oneself to a subset of nodes. The "amount of work" you mention could be the amount of traffic generated to support an activity, the amount of time it takes, or other costs such as disk space or cpu cycles. CHEERS> SAM From orasis at acm.cs.umn.edu Thu Mar 8 18:18:01 2001 From: orasis at acm.cs.umn.edu (Justin Chapweske) Date: Sat Dec 9 22:11:41 2006 Subject: [Fwd: Re: [p2p-hackers] Scalability vs Network Stability] In-Reply-To: <3AA83557.2DC09593@neurogrid.com>; from sam@neurogrid.com on Fri, Mar 09, 2001 at 10:43:51AM +0900 References: <3AA83557.2DC09593@neurogrid.com> Message-ID: <20010308201828.A23900@sorry.cs.umn.edu> I think that average latency is also an important determination of network stability, since a system that requires global state to be communicated will need to be able to synchronize faster than the network falls apart. So if you were comparing a DivX trading network vs a GIF trading network (T3's vs modems) you'd find that with the same average uptime that the DivX network would perform better becuase it can propegate state changes faster. > > So we might classify a stable network in terms of the average time that > a node > is connected over a day, as well as the variance. Well, in fact we > > So one can imagine a network architecture that was scalable for > Download/Upload but not for search, e.g. it had a data re-distribution > method > like swarmcast or FreeNet's dynamic caching, but searched by explosively > distributing all search queries like Gnutella. > Yeah, Swarmcast would be insanely horrible for searching, but I think thats a good thing. I've always been of the opinion that you should optimize your topology for each specific function you are going to perform. So if you are doing both searching and file distribution then you should have seperate topologies optimized for each. -- Justin Chapweske, Lead Swarmcast Developer, OpenCola Inc. http://www.sourceforge.net/projects/swarmcast/ From orasis at acm.cs.umn.edu Thu Mar 8 18:25:02 2001 From: orasis at acm.cs.umn.edu (Justin Chapweske) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] NS topologies In-Reply-To: <20010308201828.A23900@sorry.cs.umn.edu>; from orasis@acm.cs.umn.edu on Thu, Mar 08, 2001 at 08:18:28PM -0600 References: <3AA83557.2DC09593@neurogrid.com> <20010308201828.A23900@sorry.cs.umn.edu> Message-ID: <20010308202455.B23900@sorry.cs.umn.edu> If anyone comes across (or creates) any ns topology profiles for current P2P networks I would appreciate a heads up. If you don't know what ns is: http://www.isi.edu/nsnam/ns/ -- Justin Chapweske, Lead Swarmcast Developer, OpenCola Inc. http://www.sourceforge.net/projects/swarmcast/ From brander at mnemonic.net Wed Mar 14 07:56:01 2001 From: brander at mnemonic.net (Brander "bay" Lien) Date: Sat Dec 9 22:11:41 2006 Subject: [Fwd: Re: [p2p-hackers] Scalability vs Network Stability] In-Reply-To: <20010308201828.A23900@sorry.cs.umn.edu> Message-ID: >Yeah, Swarmcast would be insanely horrible for searching, but I think >thats a good thing. I've always >been of the opinion that you should optimize your topology for each >specific function you are going to perform. So if you are doing both >searching and file distribution then you should have seperate topologies >optimized for each. > >-- >Justin Chapweske, Lead Swarmcast Developer, OpenCola Inc. >http://www.sourceforge.net/projects/swarmcast/ This is one thing I've been saying overall that the topologies of the networks should remain optimized and focused for the transport. One thing Gnutella is facing is "it can do anything" mainly cause it was left wide open for development purposes. People take this in the "it can do anything on the same network" which isn't an incorrect assumption, however pushing every packet across a generic gnutellaNET is not the scalable answer. Smaller more focused networks are the solution in keeping the networks functioning well (especially with the diveristy of node bandwidths in the gnutellaNET) while keeping the nodes on the network functioning. Justin: I met you at the SF p2p conf, I figure I'll write up a little explanation of that network I was talking to you about & throw it up on here in a bit. Cheers, Brander Lien , Code Ninja, Toadnode :) From gohnesorge at lh-computertechnik.de Wed Mar 14 15:31:01 2001 From: gohnesorge at lh-computertechnik.de (g'o'tz ohnesorge) Date: Sat Dec 9 22:11:41 2006 Subject: [Fwd: Re: [Fwd: Re: [p2p-hackers] Scalability vs Network Stability]] Message-ID: <3AAFFEE7.E89C2CED@lh-computertechnik.de> Brander \"bay\" Lien wrote: > >Yeah, Swarmcast would be insanely horrible for searching, but I think > >thats a good thing. I've always > >been of the opinion that you should optimize your topology for each > >specific function you are going to perform. So if you are doing both > >searching and file distribution then you should have seperate topologies > >optimized for each. > > > >-- > >Justin Chapweske, Lead Swarmcast Developer, OpenCola Inc. > >http://www.sourceforge.net/projects/swarmcast/ > > This is one thing I've been saying overall that the topologies of the > networks > should remain optimized and focused for the transport. One thing Gnutella > is facing is "it can do anything" mainly cause it was left wide open for > development purposes. People take this in the "it can do anything on the > same > network" which isn't an incorrect assumption, however pushing every packet > across a generic gnutellaNET is not the scalable answer. > > Smaller more focused networks are the solution in keeping the networks > functioning well (especially with the diveristy of node bandwidths in the > gnutellaNET) while keeping the nodes on the network functioning. Would it help if Gnutella would automatically generate some Napster-like search servers? Each time connecting to another node, a node would reveal whether it is a search server at that time. If a node wants to search, it will only ask the search servers around, not any other node. If a node doesn't know any search server around, it will itself become a search server (thus annoying The Enemy if their lawyers or other DoS'ers take one down). Search servers will poll a full list of any offered content of all neighbouring servers (and maybe updates should be pushed by nodes to known search servers), and thereupon reply their search results not only about their own, but also about the neighbouring servers' contents (returning results biggest files first). If a search server doesn't find anything, it will first ask all neighbouring search servers, and only thereafter try to search the old way, node to node. That should limit the search load by an order of magnitude. Other things that would be nice would be if completely downloaded files would automatically be offered to other nodes, and some way of automatically continuing a collapsed download from another node that has a file of the same name and length, probably with some bytes overlap to make it more attack-proof. Security plug-ins (SSL, VPNs, trying to look like :80 HTML traffic) would also be nice. From zooko at zooko.com Thu Mar 15 07:09:01 2001 From: zooko at zooko.com (zooko@zooko.com) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] news site recommendation: infoanarchy.org Message-ID: If you're not aware, infoanarchy.org[1] is the p2p news site with frequent updates, full coverage of the spectrum of p2p systems, and the informed perspective of site author Erik Moeller. Check out the "resources" page, which catalogs all known publically accessible p2p systems, each with a one-line summary by Erik[2]. Okay, so obviously I am biased because infoanarchy.org covers Mojo Nation more than competing p2p sites tend to. (For example, I think infoanarchy.org is the author of the world's only actual hands-on product review of Mojo Nation.[3]) But despite my bias, I think you'll agree that infoanarchy.org has a more technical and hands-on approach contrasted with other sites which seem to be somewhat marketing-oriented. Regards, Zooko [1] http://infoanarchy.org/ [2] http://www.infoanarchy.org/?op=special&page=resources [3] http://www.infoanarchy.org/?op=displaystory&sid=2001/3/4/145826/3831 From hal at finney.org Thu Mar 15 11:16:01 2001 From: hal at finney.org (hal@finney.org) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] news site recommendation: infoanarchy.org Message-ID: <200103151911.LAA01557@finney.org> Two other sites that I like are Peertal, www.peertal.com, although it has a little more of a commercial flavor, and the O'Reilly P2P news ticker, http://www.openp2p.com/p2p/news.csp. That one just has links to other stories but it gets about a dozen new entries a day so there's always something new to read. Hal From hopwood at zetnet.co.uk Thu Mar 15 14:17:01 2001 From: hopwood at zetnet.co.uk (David Hopwood) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] Scalability vs Network Stability References: <3AAFFEE7.E89C2CED@lh-computertechnik.de> Message-ID: <3AB13C5B.2A440734@zetnet.co.uk> -----BEGIN PGP SIGNED MESSAGE----- g'o'tz ohnesorge wrote: > Search servers will poll a full list of any offered content of all > neighbouring servers (and maybe updates should be pushed by nodes to > known search servers), and thereupon reply their search results not > only about their own, but also about the neighbouring servers' > contents (returning results biggest files first). Why that order? > If a search server doesn't find anything, it will first ask all > neighbouring search servers, and only thereafter try to search the > old way, node to node. > > That should limit the search load by an order of magnitude. Maybe, but it still doesn't scale in general, since a significant proportion of queries will not be to nearby nodes. [Can this list be set to Reply-To the list by default, BTW?] - -- David Hopwood Home page & PGP public key: http://www.users.zetnet.co.uk/hopwood/ RSA 2048-bit; fingerprint 71 8E A6 23 0E D3 4C E5 0F 69 8C D4 FA 66 15 01 Nothing in this message is intended to be legally binding. If I revoke a public key but refuse to specify why, it is because the private key has been seized under the Regulation of Investigatory Powers Act; see www.fipr.org/rip -----BEGIN PGP SIGNATURE----- Version: 2.6.3i Charset: noconv iQEVAwUBOrE8OTkCAxeYt5gVAQGM/gf/Q5lRmKO5p0/Sn2Jx/N83wqRyrFUSM0/N qmouFrSbJRFSt9x2KbGDoi6TJ9Fn+bw2AP2GbpyQITa/4CSyRBwHTYaexjjL/1ym rcmM4rKB6flv6LfASZVWgQdm8b5oeCEzGK447SRfpGaLKyOjcBYS7Cpc7hvl2keS eMUBTE5MPRvoJlkB9kNHiOWScyM2P86xK1GTXIPzbdv34oibmK5AId7QEOoCSjTm jPQGgPduw866oc4mbeRGhLZaiCYbNtjfCsG94XSd8ad7gsC9P6pOH8Op3CVsg8Gz MKAebBm+YTKGYQWF0QDlecM8YjWiI/9BB2MXzT7ICpHRDag37SCb4g== =2aUi -----END PGP SIGNATURE----- From zooko at zooko.com Thu Mar 15 14:47:02 2001 From: zooko at zooko.com (zooko@zooko.com) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] Scalability vs Network Stability In-Reply-To: Message from David Hopwood of "Thu, 15 Mar 2001 22:04:11 GMT." <3AB13C5B.2A440734@zetnet.co.uk> References: <3AAFFEE7.E89C2CED@lh-computertechnik.de> <3AB13C5B.2A440734@zetnet.co.uk> Message-ID: David Hopwood wrote: > > [Can this list be set to Reply-To the list by default, BTW?] Done. If anyone objects, let me know. Regards, Zooko From gohnesorge at lh-computertechnik.de Thu Mar 15 15:40:02 2001 From: gohnesorge at lh-computertechnik.de (g'o'tz ohnesorge) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] Scalability vs Network Stability References: <3AAFFEE7.E89C2CED@lh-computertechnik.de> <3AB13C5B.2A440734@zetnet.co.uk> Message-ID: <3AB15278.2B448A1B@lh-computertechnik.de> David Hopwood wrote: > -----BEGIN PGP SIGNED MESSAGE----- > > g'o'tz ohnesorge wrote: > > Search servers will poll a full list of any offered content of all > > neighbouring servers (and maybe updates should be pushed by nodes to > > known search servers), and thereupon reply their search results not > > only about their own, but also about the neighbouring servers' > > contents (returning results biggest files first). > > Why that order? Gnutella is pestered with incomplete files, so the bigger, the better the chance that it's complete. Moreso if all downloaded files are automatically re-offered to all others (which should again help to reduce and even out load). From zooko at zooko.com Thu Mar 15 15:56:01 2001 From: zooko at zooko.com (zooko@zooko.com) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] Re: p2p-hackers post from lucas@worldos.com requires approval In-Reply-To: Message from p2p-hackers-admin@zgp.org of "Thu, 15 Mar 2001 15:20:01 PST." <20010315232001.982D43FC0D@capsicum.zgp.org> References: <20010315232001.982D43FC0D@capsicum.zgp.org> Message-ID: Hey Lucas! Let me know if you want me to send this msg through. Regards, Zooko > As list administrator, your authorization is requested for the > following mailing list posting: > > List: > From: lucas@worldos.com > Subject: RE: [p2p-hackers] Scalability vs Network Stability > Reason: Post by non-member to a members-only list > > At your convenience, visit: > > http://zgp.org/mailman/admindb/p2p-hackers > > to approve or deny the request. > From lucas at worldos.com Thu Mar 15 16:57:01 2001 From: lucas at worldos.com (Lucas Gonze) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] Scalability vs Network Stability In-Reply-To: <3AB13C5B.2A440734@zetnet.co.uk> Message-ID: > > If a search server doesn't find anything, it will first ask all > > neighbouring search servers, and only thereafter try to search the > > old way, node to node. > > > > That should limit the search load by an order of magnitude. > > Maybe, but it still doesn't scale in general, since a significant > proportion of queries will not be to nearby nodes. Average cached data should match the average query, hence the majority of queries should be answerable with nearby resources. Stuff that is less popular but not totally obscure should still be more likely to be found before reaching the horizon. Even for totally obscure material, an adjacent caching scheme still cuts off the search one hop sooner. Since the outer ring on the search horizon is by far the largest (about three times the size of the next largest ring), this is a very large improvement in efficiency. Best case improvement is stopping the search at a single hop, worst case is eliminating the outer ring. Quite nice either way. What doesn't make sense to me is drawing a distinction between search servers and other nodes. - Lucas From sam at neurogrid.com Thu Mar 15 23:43:02 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] P2P Systems efficiency References: Message-ID: <3AB1C3AF.CDF2F1C9@neurogrid.com> Hi All, So, okay, I'm not addressing scalability as such because my simulations are all for a fixed number of nodes network, but I've done some simulations comparing message use efficiency of FreeNet, Gnutella and NeuroGrid as part of the NeuroGrid project. http://www.neurogrid.com/ng-simulation.html You can find the simulation results at the above link. The comparison is looking at message efficiency, which I define as the proportion of target documents a search finds divided by the number of messages generated during that search. As one might expect, Gnutella seems to be very low on efficiency. I have plenty more data not in the paper, but limited time to process it, make nice graphs etc, so I would be very interested to hear what kinds of stats people are interested in, and criticisms of my simulation model which is not necessarily a perfect model of either freenet or gnutella, in fact there are probably some serious flaws, but let's see. CHEERS> SAM From gohnesorge at lh-computertechnik.de Fri Mar 16 04:58:01 2001 From: gohnesorge at lh-computertechnik.de (g'o'tz ohnesorge) Date: Sat Dec 9 22:11:41 2006 Subject: [p2p-hackers] Re: p2p-hackers post from lucas@worldos.com requires approval References: <20010315232001.982D43FC0D@capsicum.zgp.org> Message-ID: <3AB20D9D.114D5A4A@lh-computertechnik.de> zooko@zooko.com wrote: > Hey Lucas! > > Let me know if you want me to send this msg through. > Wrong place - Lucas isn't subscribed (yet, or at least not under that same address), that's why the list software asks you for authorization of his post (usually to keep spam out). On the other hand, you just enabled "reply-to" (thanks), so your reply didn't get back to Lucas but instead to everyone else. > > Regards, > > Zooko > > > As list administrator, your authorization is requested for the > > following mailing list posting: > > > > List: > > From: lucas@worldos.com > > Subject: RE: [p2p-hackers] Scalability vs Network Stability > > Reason: Post by non-member to a members-only list > > > > At your convenience, visit: > > > > http://zgp.org/mailman/admindb/p2p-hackers > > > > to approve or deny the request. > > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers