From bram at gawth.com Thu Aug 9 18:53:01 2001 From: bram at gawth.com (Bram Cohen) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] BitTorrent 2.0 is out! Message-ID: I just put up version 2.0 of BitTorrent you can get it here - http://bitconjurer.org/BitTorrent/ I'm very excited about this release - it has a whole new UI, and marks a huge advance in the direction of clients using it simply by clicking on hyperlinks. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From dmarti at zgp.org Tue Aug 14 11:51:01 2001 From: dmarti at zgp.org (Don Marti) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] [dtype@dtype.org: [freenet-chat] p2p-legal mailing list] Message-ID: <20010814115008.E16119@zgp.org> ----- Forwarded message from "M. Drew Streib" ----- Date: Tue, 14 Aug 2001 17:20:20 +0000 From: "M. Drew Streib" Subject: [freenet-chat] p2p-legal mailing list To: devl@freenetproject.org, chat@freenetproject.org User-Agent: Mutt/1.3.18i X-Operating-System: Paper, pencils, and a couple million monkeys. Sender: chat-admin@freenetproject.org Reply-To: chat@freenetproject.org In the interest of some discussion between several of the p2p networking and network indexing groups, I've created a new mailing list, p2p-legal. http://dtype.org/mailman/listinfo/p2p-legal/ Just to qualify who I am... I am an opennap server admin and work on server implementation issues as well. I'm speaking at the Oreilly p2p conference in DC/September, and also working on a new type of indexing server to hopefully take care of some of the legal issues. This list will be for peer discussion of some of the legalities of both writing software and running servers or nodes in networks. While I expect to have a couple lawyers on the list listening in, this is not for 'official' legal advice, but rather best practice discussion, as well as potential ways to work on some of the legal issues. I apologize for the cross-post here. I'm not terribly familiar with the freenet mailing lists. This will be the only post regarding the new p2p list to the freenet lists. -drew -- M. Drew Streib | http://dtype.org/ FSG | Linux International freedb | SourceForge ----- End forwarded message ----- -- Don Marti What do we want? Free Dmitry! When do we want it? Now! http://zgp.org/~dmarti Free Dmitry: http://eff.org/ dmarti@zgp.org Free the web, burn all GIFs: http://burnallgifs.org/ From bram at gawth.com Thu Aug 23 20:22:01 2001 From: bram at gawth.com (Bram Cohen) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] BitTorrent 2.1 is out! Message-ID: Well, I've done the last really 'hardcore' piece of engineering work, you can get the new release here - http://bitconjurer.org/BitTorrent/download.html This one completely fixes a serious problem which was keeping it from using a whole net connection - it was getting tcp backoff, that was fixed by keeping multiple requests in the pipe at once. Downloader.py was completely rewritten, and a bunch of other stuff was tweaked. Interestingly, the new version is both shorter and easier to read despite being more sophisticated and working much better. -Bram Cohen "Markets can remain irrational longer than you can remain solvent" -- John Maynard Keynes From sam at neurogrid.com Tue Aug 28 04:26:01 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Semantic Routing BOF Message-ID: <3B8B7F69.684570DB@neurogrid.com> Hi All, I was just knocking around this idea for a potential O'Reilly BOF, and although most of you are probably on the decent list, I just thought I'd see if anybody here throught this sounded interesting, or stupid, or whatever. Semantic Routing in P2P networks Many P2P networks route queries on the basis of connectivity (Gnutella) or hash codes (Freenet). A further possibility exists in the form of semantic routing, where queries are routed on the basis of their meaning. Semantic routing assumes that each node is storing information (e.g. keywords) about the kinds of data stored in other nodes. Semantic routing is based on the idea of routing queries in the way that humans do. Each node stores data about what other nodes are storing. When faced with a query, the node checks the query against its knowledge of other nodes contents, and then forwards the query to a subset of those nodes. Imagine that you are asked if you know of a rock-climbing club by your friend. You don't know, but you think through your acquaintances, and pass your query on to the 2 or 3 people you think might be able to answer the question. Can semantic routing work in P2P networks? Would it be useful and/or efficient to deploy? Examples include NeuroGrid (http://www.neurogrid.net) and Prinkey's Content-based routing (http://aeolusres.homestead.com/files/index.html) If anyone wants to get involved (be part of a panel), has any suggestions for how other existing P2P systems (MojoNation, Morpheus, etc.) fit into this routing taxonomy, then feel free to mail me (sam@neurogrid.com) Thanks in advance. CHEERS> SAM From painlord at inwind.it Wed Aug 29 00:38:01 2001 From: painlord at inwind.it (Mirco Romanato) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Semantic Routing BOF Message-ID: <003c01c1305d$45b583e0$3156623e@painlordt91ikd> > ----- Original Message ----- > From: "Sam Joseph" > > Many P2P networks route queries on the basis of connectivity > (Gnutella) > > or hash codes (Freenet). A further possibility exists in the form > of > > semantic routing, where queries are routed on the basis of their > > meaning. > ............. > > Can semantic routing work in P2P > > networks? Would it be useful and/or efficient to deploy? > I think the idea is interesting, but crash hard with moundane > considerations. > A P2P with semantic routing appear very simple to censure. > > > > > Examples include NeuroGrid (http://www.neurogrid.net) and Prinkey's > > Content-based routing > (http://aeolusres.homestead.com/files/index.html) > > > If anyone wants to get involved (be part of a panel), has any > > suggestions for how other existing P2P systems (MojoNation, > Morpheus, > > etc.) fit into this routing taxonomy, then feel free to mail me > Mojo Nation go with Freenet (it would be transformed in a Freenet node > with little tweaking) > Morpheus(Music-City net) use an hash system to mark the files and a > supernode-gnutella like architecture to propagate the queries (in > Morpheus there is a limit to the number of files a user/node can share > but I don't know how the shared files are choosen). > > "It does me no injury for my neighbor to say there are twenty gods or > no God. > It neither picks my pocket nor breaks my leg." --- Thomas Jefferson > > > Mirco Romanato > Venezia - Italia > From sam at neurogrid.com Wed Aug 29 03:39:01 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) References: <003c01c1305d$45b583e0$3156623e@painlordt91ikd> Message-ID: <3B8CC5B7.4FB5315E@neurogrid.com> Do any of the Freenet or MojoNation people have an opinion on this? Mirco is suggesting that a MojoNation node could be turned into a Freenet node with a little tweaking. Does anybody know how much tweaking? Mirco Romanato wrote: > Mojo Nation go with Freenet (it would be transformed in a Freenet > node with little tweaking) > Morpheus(Music-City net) use an hash system to mark the files and a > supernode-gnutella like architecture to propagate the queries (in > Morpheus there is a limit to the number of files a user/node can > share but I don't know how the shared files are choosen). Also, are there any Morpheus developers here? Anybody know the details of query propagation between Morpheus supernodes? CHEERS> SAM From cyb at azrael.dyn.cheapnet.net Wed Aug 29 12:01:02 2001 From: cyb at azrael.dyn.cheapnet.net (Brandon K. Wiley) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: <3B8CC5B7.4FB5315E@neurogrid.com> Message-ID: > Do any of the Freenet or MojoNation people have an opinion on this? Mirco > is suggesting that a MojoNation node could be turned into a Freenet node > with a little tweaking. Does anybody know how much tweaking? Mojo Nation could not be turned into Freenet with a little tweaking. Turning Mojo Nation into Freenet would basically be equivalent to writing a Freenet node in Python from scratch. About the only code they could share is the code for verifying the hash of a file because MN is sane and uses SHA1. The routing is different, the protocol is different both syntactically and semantically, and Freenet uses its own non-standard encryption. From gojomo at usa.net Wed Aug 29 12:19:01 2001 From: gojomo at usa.net (Gordon Mohr) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) References: Message-ID: <009101c130bf$49122fe0$0ea7fea9@golden> Brandon Wiley writes: > About the only code they could share is the code for verifying the hash of > a file because MN is sane and uses SHA1. Sometimes the Freenet docs talk about a "Progressive CHK" that is not the full-file SHA1. Are some CHKs full-file SHA1 values, and others progressive (reverse-4k-blocks-in-a-maximally-lopsided-tree)? If so, how does one tell the difference between the two kinds? Just wondering, - Gojomo From oskar at freenetproject.org Wed Aug 29 12:23:01 2001 From: oskar at freenetproject.org (Oskar Sandberg) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: <009101c130bf$49122fe0$0ea7fea9@golden>; from gojomo@usa.net on Wed, Aug 29, 2001 at 12:17:46PM -0700 References: <009101c130bf$49122fe0$0ea7fea9@golden> Message-ID: <20010829212646.E383@sandbergs.org> On Wed, Aug 29, 2001 at 12:17:46PM -0700, Gordon Mohr wrote: > Brandon Wiley writes: > > About the only code they could share is the code for verifying the hash of > > a file because MN is sane and uses SHA1. > > Sometimes the Freenet docs talk about a "Progressive CHK" that > is not the full-file SHA1. > > Are some CHKs full-file SHA1 values, and others progressive > (reverse-4k-blocks-in-a-maximally-lopsided-tree)? > > If so, how does one tell the difference between the > two kinds? The Progressive CHK becomes a normal SHA1 hash if you only have one part. The size of the parts is always given with a piece of data. -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg oskar@freenetproject.org From gojomo at usa.net Wed Aug 29 12:33:01 2001 From: gojomo at usa.net (Gordon Mohr) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) References: <009101c130bf$49122fe0$0ea7fea9@golden> <20010829212646.E383@sandbergs.org> Message-ID: <00a001c130c1$49e2b460$0ea7fea9@golden> Oskar Sandberg writes: > The Progressive CHK becomes a normal SHA1 hash if you only have one > part. The size of the parts is always given with a piece of data. So, both total-file-length and progressive-part-length are encoded into every CHK? And thus when part-length >= file-length, the CHK is the full-file SHA1 hash? - Gojomo From sah at thalassocracy.org Wed Aug 29 12:37:02 2001 From: sah at thalassocracy.org (Steven Hazel) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: Gordon Mohr's message of "Wed, 29 Aug 2001 12:17:46 -0700" References: <009101c130bf$49122fe0$0ea7fea9@golden> Message-ID: <87bskyfyf1.fsf@azrael.dyn.cheapnet.net> Gordon Mohr writes: > Sometimes the Freenet docs talk about a "Progressive CHK" that is > not the full-file SHA1. > > Are some CHKs full-file SHA1 values, and others progressive > (reverse-4k-blocks-in-a-maximally-lopsided-tree)? Yes. (Although the 4k block size isn't fixed.) > If so, how does one tell the difference between the two kinds? One requests the file. The DataReply will have header information that indicates which kind it is (PartSize and DataLength). Every document larger than the part size will be progressively hashed. The purpose of this is to make it possible for nodes to periodically verify that they're not being fed bunk data, rather than forcing them to wait for the entire file before they can verify that it's what they requested. Incidentally, Brandon Wiley and I will be teaching a "Programming Freenet" tutorial at the upcoming O'Reilly P2P and Web Services conference, and I'll be covering this kind of thing in depth there: http://conferences.oreillynet.com/cs/p2pweb2001/view/e_sess/1830 If you're interested, please do sign up, as they're probably going to cancel the tutorial if we don't get more students soon. --Steven From gojomo at usa.net Wed Aug 29 13:16:02 2001 From: gojomo at usa.net (Gordon Mohr) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) References: <009101c130bf$49122fe0$0ea7fea9@golden> <87bskyfyf1.fsf@azrael.dyn.cheapnet.net> Message-ID: <00b301c130c7$61a960c0$0ea7fea9@golden> Steven Hazel writes: > > If so, how does one tell the difference between the two kinds? > > One requests the file. The DataReply will have header information > that indicates which kind it is (PartSize and DataLength). Every > document larger than the part size will be progressively hashed. OK. I presume the person who initially inserts the content gets to choose the PartSize. So, any one file, of byte length N, may have N different Freenet CHKs -- one for each choosable PartSize, 1 through N. > The purpose of this is to make it possible for nodes to periodically > verify that they're not being fed bunk data, rather than forcing them > to wait for the entire file before they can verify that it's what they > requested. Yes. I prefer a tree hash for that purpose, as it allows out-of-order subsegment verification, but the progressive hash works too. - Gojomo From sah at thalassocracy.org Wed Aug 29 13:44:01 2001 From: sah at thalassocracy.org (Steven Hazel) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: Gordon Mohr's message of "Wed, 29 Aug 2001 13:15:43 -0700" References: <009101c130bf$49122fe0$0ea7fea9@golden> <87bskyfyf1.fsf@azrael.dyn.cheapnet.net> <00b301c130c7$61a960c0$0ea7fea9@golden> Message-ID: <87vgj6egr1.fsf@azrael.dyn.cheapnet.net> Gordon Mohr writes: > I presume the person who initially inserts the content gets to > choose the PartSize. So, any one file, of byte length N, may have N > different Freenet CHKs -- one for each choosable PartSize, 1 through > N. Yes. > Yes. I prefer a tree hash for that purpose, as it allows > out-of-order subsegment verification, but the progressive hash works > too. Out-of-order subsegment verification isn't important in this case, but I'd have gone with a tree hash myself, just because it's marginally easier to implement and debug. --Steven From oskar at freenetproject.org Wed Aug 29 14:35:02 2001 From: oskar at freenetproject.org (Oskar Sandberg) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: <00a001c130c1$49e2b460$0ea7fea9@golden>; from gojomo@usa.net on Wed, Aug 29, 2001 at 12:32:06PM -0700 References: <009101c130bf$49122fe0$0ea7fea9@golden> <20010829212646.E383@sandbergs.org> <00a001c130c1$49e2b460$0ea7fea9@golden> Message-ID: <20010829233824.H383@sandbergs.org> On Wed, Aug 29, 2001 at 12:32:06PM -0700, Gordon Mohr wrote: > Oskar Sandberg writes: > > The Progressive CHK becomes a normal SHA1 hash if you only have one > > part. The size of the parts is always given with a piece of data. > > So, both total-file-length and progressive-part-length are > encoded into every CHK? No, only the progressive-part-length is encoded in the key itself, for security reasons (the idea with the progressive hash is that you can tell that the data was bad very soon, but somebody could claim the partsize was 1 gigabyte unless that requestor knows better). What is encoded in the key is only an upper bound (expressed as a 2 logarithm) - the exact part size is stored with the data (in metadata basically - though I don't want to call it that since it isn't related to user level metadata client Content-type.) > And thus when part-length >= file-length, the CHK is the > full-file SHA1 hash? Yes, but the file format is somewhat special (padded to 2 exponent length, including IV and metadata length, encrypted using the SHA1 hash of the plaintext, and then including the document information fields like the part-length) so it's not like you are going to see the same resulting value as other systems anyways. I considered if the way of calculating UID values from data might actually be an area where we should be trying to "interoperate" between the different networks, but I figure that even there the emphasis is too different for it be worth it. -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg oskar@freenetproject.org From oskar at freenetproject.org Wed Aug 29 14:41:02 2001 From: oskar at freenetproject.org (Oskar Sandberg) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: <87vgj6egr1.fsf@azrael.dyn.cheapnet.net>; from sah@thalassocracy.org on Wed, Aug 29, 2001 at 03:46:42PM -0500 References: <009101c130bf$49122fe0$0ea7fea9@golden> <87bskyfyf1.fsf@azrael.dyn.cheapnet.net> <00b301c130c7$61a960c0$0ea7fea9@golden> <87vgj6egr1.fsf@azrael.dyn.cheapnet.net> Message-ID: <20010829234448.I383@sandbergs.org> On Wed, Aug 29, 2001 at 03:46:42PM -0500, Steven Hazel wrote: <> > > Yes. I prefer a tree hash for that purpose, as it allows > > out-of-order subsegment verification, but the progressive hash works > > too. > > Out-of-order subsegment verification isn't important in this case, but > I'd have gone with a tree hash myself, just because it's marginally > easier to implement and debug. If what we are doing is subset of a tree hash, then how can it be easier to implement? -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg oskar@freenetproject.org From sah at thalassocracy.org Wed Aug 29 14:48:01 2001 From: sah at thalassocracy.org (Steven Hazel) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: Oskar Sandberg's message of "Wed, 29 Aug 2001 23:44:48 +0200" References: <009101c130bf$49122fe0$0ea7fea9@golden> <87bskyfyf1.fsf@azrael.dyn.cheapnet.net> <00b301c130c7$61a960c0$0ea7fea9@golden> <87vgj6egr1.fsf@azrael.dyn.cheapnet.net> <20010829234448.I383@sandbergs.org> Message-ID: <87n14iedrs.fsf@azrael.dyn.cheapnet.net> Oskar Sandberg writes: > On Wed, Aug 29, 2001 at 03:46:42PM -0500, Steven Hazel wrote: > > > Out-of-order subsegment verification isn't important in this case, > > but I'd have gone with a tree hash myself, just because it's > > marginally easier to implement and debug. > > If what we are doing is subset of a tree hash, then how can it be > easier to implement? I guess what I'm talking about is also a subset of a tree hash. What I meant is that I'd have made the CHK be the hash of all the part hashes. The "first part" would contain all of the part hashes, and the rest of the parts would just be data. --Steven From oskar at freenetproject.org Wed Aug 29 14:50:01 2001 From: oskar at freenetproject.org (Oskar Sandberg) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: <00b301c130c7$61a960c0$0ea7fea9@golden>; from gojomo@usa.net on Wed, Aug 29, 2001 at 01:15:43PM -0700 References: <009101c130bf$49122fe0$0ea7fea9@golden> <87bskyfyf1.fsf@azrael.dyn.cheapnet.net> <00b301c130c7$61a960c0$0ea7fea9@golden> Message-ID: <20010829235342.J383@sandbergs.org> On Wed, Aug 29, 2001 at 01:15:43PM -0700, Gordon Mohr wrote: > Steven Hazel writes: > > > If so, how does one tell the difference between the two kinds? > > > > One requests the file. The DataReply will have header information > > that indicates which kind it is (PartSize and DataLength). Every > > document larger than the part size will be progressively hashed. > > OK. > > I presume the person who initially inserts the content > gets to choose the PartSize. So, any one file, of byte > length N, may have N different Freenet CHKs -- one for > each choosable PartSize, 1 through N. There is a convention for how to choose the size. Since the use of the plaintext hash as the encryption key and IV is also a convention, giving 2^256 different possible CHKs for a file, it hardly matters. > > The purpose of this is to make it possible for nodes to periodically > > verify that they're not being fed bunk data, rather than forcing them > > to wait for the entire file before they can verify that it's what they > > requested. > > Yes. I prefer a tree hash for that purpose, as it allows > out-of-order subsegment verification, but the progressive > hash works too. There is obviously no need for out-of-order verfication on a stream that needs to be tunneled. -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg oskar@freenetproject.org From oskar at freenetproject.org Wed Aug 29 14:57:01 2001 From: oskar at freenetproject.org (Oskar Sandberg) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: <87n14iedrs.fsf@azrael.dyn.cheapnet.net>; from sah@thalassocracy.org on Wed, Aug 29, 2001 at 04:51:03PM -0500 References: <009101c130bf$49122fe0$0ea7fea9@golden> <87bskyfyf1.fsf@azrael.dyn.cheapnet.net> <00b301c130c7$61a960c0$0ea7fea9@golden> <87vgj6egr1.fsf@azrael.dyn.cheapnet.net> <20010829234448.I383@sandbergs.org> <87n14iedrs.fsf@azrael.dyn.cheapnet.net> Message-ID: <20010830000015.K383@sandbergs.org> On Wed, Aug 29, 2001 at 04:51:03PM -0500, Steven Hazel wrote: > Oskar Sandberg writes: > > > On Wed, Aug 29, 2001 at 03:46:42PM -0500, Steven Hazel wrote: > > > > > Out-of-order subsegment verification isn't important in this case, > > > but I'd have gone with a tree hash myself, just because it's > > > marginally easier to implement and debug. > > > > If what we are doing is subset of a tree hash, then how can it be > > easier to implement? > > I guess what I'm talking about is also a subset of a tree hash. > > What I meant is that I'd have made the CHK be the hash of all the part > hashes. The "first part" would contain all of the part hashes, and > the rest of the parts would just be data. Of course that is where we started (I even implemented that). We moved to the current method since it is better, and does not limit the number of parts that can be used. -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg oskar@freenetproject.org From sah at thalassocracy.org Wed Aug 29 15:00:02 2001 From: sah at thalassocracy.org (Steven Hazel) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: Oskar Sandberg's message of "Thu, 30 Aug 2001 00:00:15 +0200" References: <009101c130bf$49122fe0$0ea7fea9@golden> <87bskyfyf1.fsf@azrael.dyn.cheapnet.net> <00b301c130c7$61a960c0$0ea7fea9@golden> <87vgj6egr1.fsf@azrael.dyn.cheapnet.net> <20010829234448.I383@sandbergs.org> <87n14iedrs.fsf@azrael.dyn.cheapnet.net> <20010830000015.K383@sandbergs.org> Message-ID: <87heuqed7d.fsf@azrael.dyn.cheapnet.net> Oskar Sandberg writes: > Of course that is where we started (I even implemented that). We > moved to the current method since it is better, and does not limit > the number of parts that can be used. How does my method limit the number of parts that can be used? I don't think it does. --Steven From alk at pobox.com Wed Aug 29 17:44:02 2001 From: alk at pobox.com (Tony Kimball) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) References: <009101c130bf$49122fe0$0ea7fea9@golden> <87bskyfyf1.fsf@azrael.dyn.cheapnet.net> <00b301c130c7$61a960c0$0ea7fea9@golden> <20010829235342.J383@sandbergs.org> Message-ID: <15245.35901.233758.925334@gargle.gargle.HOWL> Quoth Oskar Sandberg on Wednesday, 29 August: : : There is obviously no need for out-of-order verfication on a stream that : needs to be tunneled. As long you specifically mean tunneled through a single route point which is willing and able (within operational requirements of the system as a whole) to buffer, then OK, ack. Otherwise, I would differ. From greg at electricrain.com Wed Aug 29 23:04:02 2001 From: greg at electricrain.com (Gregory P. Smith) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: ; from cyb@azrael.dyn.cheapnet.net on Wed, Aug 29, 2001 at 02:04:17PM -0500 References: <3B8CC5B7.4FB5315E@neurogrid.com> Message-ID: <20010829230334.A16864@zot.electricrain.com> On Wed, Aug 29, 2001 at 02:04:17PM -0500, Brandon K. Wiley wrote: > > > Do any of the Freenet or MojoNation people have an opinion on this? Mirco > > is suggesting that a MojoNation node could be turned into a Freenet node > > with a little tweaking. Does anybody know how much tweaking? > > Mojo Nation could not be turned into Freenet with a little tweaking. > Turning Mojo Nation into Freenet would basically be equivalent to writing > a Freenet node in Python from scratch. > > About the only code they could share is the code for verifying the hash of > a file because MN is sane and uses SHA1. The routing is different, the > protocol is different both syntactically and semantically, and Freenet > uses its own non-standard encryption. Brandon is correct. Very different protocols and purposes. A bridge of sorts could be constructed between the two using "generic IPC mechanism", probably after a couple of hacker days, such that a freenet node could use mojo nation for its data storage and a mojo nation broker could in turn use freenet for its data storage. However, I don't know of anybody actually working on such a thing. -- Gregory P. Smith gnupg/pgp: http://electricrain.com/greg/keys/ C379 1F92 3703 52C9 87C4 BE58 6CDA DB87 105D 9163 From oskar at freenetproject.org Wed Aug 29 23:28:01 2001 From: oskar at freenetproject.org (Oskar Sandberg) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: <87heuqed7d.fsf@azrael.dyn.cheapnet.net>; from sah@thalassocracy.org on Wed, Aug 29, 2001 at 05:03:18PM -0500 References: <009101c130bf$49122fe0$0ea7fea9@golden> <87bskyfyf1.fsf@azrael.dyn.cheapnet.net> <00b301c130c7$61a960c0$0ea7fea9@golden> <87vgj6egr1.fsf@azrael.dyn.cheapnet.net> <20010829234448.I383@sandbergs.org> <87n14iedrs.fsf@azrael.dyn.cheapnet.net> <20010830000015.K383@sandbergs.org> <87heuqed7d.fsf@azrael.dyn.cheapnet.net> Message-ID: <20010830083143.A328@sandbergs.org> On Wed, Aug 29, 2001 at 05:03:18PM -0500, Steven Hazel wrote: > Oskar Sandberg writes: > > > Of course that is where we started (I even implemented that). We > > moved to the current method since it is better, and does not limit > > the number of parts that can be used. > > How does my method limit the number of parts that can be used? I > don't think it does. After you have read the first part, you'll need to put all the part hashes in memory. -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg oskar@freenetproject.org From sam at neurogrid.com Wed Aug 29 23:43:01 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> Message-ID: <3B8DDFFA.CFA67BC9@neurogrid.com> "Gregory P. Smith" wrote: > On Wed, Aug 29, 2001 at 02:04:17PM -0500, Brandon K. Wiley wrote: > > > > Mojo Nation could not be turned into Freenet with a little tweaking. > > Turning Mojo Nation into Freenet would basically be equivalent to writing > > a Freenet node in Python from scratch. > > Brandon is correct. Very different protocols and purposes. > > A bridge of sorts could be constructed between the two using "generic IPC > mechanism", probably after a couple of hacker days, such that a freenet > node could use mojo nation for its data storage and a mojo nation broker > could in turn use freenet for its data storage. However, I don't know > of anybody actually working on such a thing. Okay so could anybody from MojoNation comment on what makes their routing methodology different from Freenet. I understand the Freenet does depth first greedy search when routing. MojoNation routes like Morpheus, or Gnutella perhaps? CHEERS> SAM From oskar at freenetproject.org Thu Aug 30 04:12:02 2001 From: oskar at freenetproject.org (Oskar Sandberg) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: <3B8DDFFA.CFA67BC9@neurogrid.com>; from sam@neurogrid.com on Thu, Aug 30, 2001 at 03:40:58PM +0900 References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8DDFFA.CFA67BC9@neurogrid.com> Message-ID: <20010830131553.A1801@sandbergs.org> On Thu, Aug 30, 2001 at 03:40:58PM +0900, Sam Joseph wrote: <> > Okay so could anybody from MojoNation comment on what makes their routing > methodology different from Freenet. I understand the Freenet does depth first > greedy search when routing. MojoNation routes like Morpheus, or Gnutella perhaps? Can a single step match of data UID against node UIDs really be called routing? -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg oskar@freenetproject.org From zooko at zooko.com Thu Aug 30 05:26:01 2001 From: zooko at zooko.com (zooko@zooko.com) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: Message from Sam Joseph of "Thu, 30 Aug 2001 15:40:58 +0900." <3B8DDFFA.CFA67BC9@neurogrid.com> References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8DDFFA.CFA67BC9@neurogrid.com> Message-ID: Sam Joseph wrote: > > "Gregory P. Smith" wrote: > > > A bridge of sorts could be constructed between the two using "generic IPC > > mechanism", probably after a couple of hacker days, such that a freenet > > node could use mojo nation for its data storage and a mojo nation broker > > could in turn use freenet for its data storage. However, I don't know > > of anybody actually working on such a thing. > > Okay so could anybody from MojoNation comment on what makes their routing > methodology different from Freenet. I understand the Freenet does depth first > greedy search when routing. MojoNation routes like Morpheus, or Gnutella perhaps? Err.. I'd be happy to, but what's the point, again? If the goal is to link the two networks together then do as Greg suggested (and as Brandon Wiley suggested in the "Peer To Peer" book) and make a gateway node which consists of one normal node from each network, linked with IPC. [1, 2] Sorry if I've missed part of this thread, but I'm curious why you're investigating the comparison between these networks. So anyway, Mojo Nation doesn't *do* routing (as Oskar notes in a followup to your message). I take "routing" to mean, in Freenet terms, on-demand recursive proxying of file operations. Mojo Nation does on-demand recursive proxying of messages (sufficient to bypass firewalls, and perhaps a good base on which to build an anonymizing message-mix), and it does introductions instead of proxying for extending your horizon. In addition, it does a batched recursive propagation of blocks (roughly once an hour to once a day depending on the configuration of the block server). Sam: are you going to be at p2pcon2 in Wash, D.C. in September? Oskar? Anyone else? Regards, Zooko [1] http://freenet.sourceforge.net/doc/xml-rpc.html [2] http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/~checkout~/mojonation/evil/hackerdocs/LJ_article.html?content-type=text/html From ian.soboroff at nist.gov Thu Aug 30 05:34:01 2001 From: ian.soboroff at nist.gov (Ian Soboroff) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: <20010829230334.A16864@zot.electricrain.com> References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> Message-ID: "Gregory P. Smith" writes: > A bridge of sorts could be constructed between the two using "generic IPC > mechanism", probably after a couple of hacker days, such that a freenet > node could use mojo nation for its data storage and a mojo nation broker > could in turn use freenet for its data storage. However, I don't know > of anybody actually working on such a thing. But I believe that the original start of this thread concerned _semantic_ routing. The idea here is to decide whether to send a query to a peer based on the contents of the documents on that server. What current p2p systems do is routing based on network topology and packet lifetime. In a semantic p2p system, peers would advertise content "summaries" (for example, highly relevant terms from titles of songs in a music system) to their neighbors in the network, and the routing algorithm could be tuned to send messages to only the top n neighbors or some such. Anyone have more thoughts on this? (Rather than the relative merits of hash implementations... ;-) If the community is still focused on routing queries for music, you might be interested in a collection of folk music at the New Zealand Digital Library (www.nzdl.org). Users hum their query, and the system can match it against songs. (This works because, while most people have terrible pitch, they do tend to remember the pattern of note changes and tempos rather well.) The question in a p2p environment is how to collect and spread the semantic summary information for routing cheaply. ian From sam at neurogrid.com Thu Aug 30 07:56:02 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8DDFFA.CFA67BC9@neurogrid.com> <20010830131553.A1801@sandbergs.org> Message-ID: <3B8E539F.FB02A024@neurogrid.com> Oskar Sandberg wrote: > On Thu, Aug 30, 2001 at 03:40:58PM +0900, Sam Joseph wrote: > <> > > Okay so could anybody from MojoNation comment on what makes their routing > > methodology different from Freenet. I understand the Freenet does depth first > > greedy search when routing. MojoNation routes like Morpheus, or Gnutella perhaps? > > Can a single step match of data UID against node UIDs really be called > routing? Well if you're forwarding a message as a result of the match, then I'd say yes. However the important point, for me, is what criteria is being used to forward messages. CHEERS> SAM From sam at neurogrid.com Thu Aug 30 08:02:01 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8DDFFA.CFA67BC9@neurogrid.com> Message-ID: <3B8E54E8.74229A89@neurogrid.com> zooko@zooko.com wrote: > Err.. I'd be happy to, but what's the point, again? If the goal is to link > the two networks together then do as Greg suggested (and as Brandon Wiley > suggested in the "Peer To Peer" book) and make a gateway node which consists of > one normal node from each network, linked with IPC. [1, 2] Actually I was hoping for a placement of MojoNation's "Message Forwarding" in my own terminology. I was thinking that Freenet forwards requests around on the basis of a hash lookup table, gnutella broadcasts messages within a certain horizon, Morpheus ? MojoNation has some hashing in it but ... This is all me trying to work out how similar or dis-similar the different techniques are to Semantic routing, when we pass on messages on the basis of their content. > So anyway, Mojo Nation doesn't *do* routing (as Oskar notes in a followup to > your message). I take "routing" to mean, in Freenet terms, on-demand recursive > proxying of file operations. Mojo Nation does on-demand recursive proxying of > messages (sufficient to bypass firewalls, and perhaps a good base on which to > build an anonymizing message-mix), and it does introductions instead of > proxying for extending your horizon. In addition, it does a batched recursive > propagation of blocks (roughly once an hour to once a day depending on the > configuration of the block server). So when I send a request in MojoNation, how does it flow through the system? Is it likely to visit more than one super-node (or whatever you call your more permanent nodes), does this question even make sense in the context of MojoNation. > Sam: are you going to be at p2pcon2 in Wash, D.C. in September? I hope so, and I hope to be running a little Semantic Routing BOF session if I can get enough people interested. CHEERS> SAM From sam at neurogrid.com Thu Aug 30 08:06:01 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> Message-ID: <3B8E55E7.50627812@neurogrid.com> Ian Soboroff wrote: > "Gregory P. Smith" writes: > > > A bridge of sorts could be constructed between the two using "generic IPC > > mechanism", probably after a couple of hacker days, such that a freenet > > node could use mojo nation for its data storage and a mojo nation broker > > could in turn use freenet for its data storage. However, I don't know > > of anybody actually working on such a thing. > > But I believe that the original start of this thread concerned > _semantic_ routing. The idea here is to decide whether to send a > query to a peer based on the contents of the documents on that > server. What current p2p systems do is routing based on network > topology and packet lifetime. In a semantic p2p system, peers would > advertise content "summaries" (for example, highly relevant terms from > titles of songs in a music system) to their neighbors in the network, > and the routing algorithm could be tuned to send messages to only the > top n neighbors or some such. > > Anyone have more thoughts on this? (Rather than the relative merits > of hash implementations... ;-) Well it's interesting you say advertise, because a publish and subscribe model is another way to go. I have this innate dislike of advertising, ever since I quit working for an advertising company, so I tend to think of semantic routing systems where each node is gradually learning about the contents of nearby nodes, so there is no advertising process. When a node returns a document, the other nodes that see it say "Ah, that node is storing data about XXXX" and they passively build up a picture of what is where in the network, and then forward queries on that basis. The best explanation of this is probably in: http://www.neurogrid.net/si-simulation02.html CHEERS> SAM From ian.soboroff at nist.gov Thu Aug 30 08:15:01 2001 From: ian.soboroff at nist.gov (Ian Soboroff) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: <3B8E55E7.50627812@neurogrid.com> References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8E55E7.50627812@neurogrid.com> Message-ID: Sam Joseph writes: > Well it's interesting you say advertise, because a publish and > subscribe model is another way to go. I have this innate dislike of > advertising, ever since I quit working for an advertising company, > so I tend to think of semantic routing systems where each node is > gradually learning about the contents of nearby nodes, so there is > no advertising process. When a node returns a document, the other > nodes that see it say "Ah, that node is storing data about XXXX" and > they passively build up a picture of what is where in the network, > and then forward queries on that basis. Ahh... I was using the term 'advertise' almost but not quite in the sense of 'publish'. In the intelligent agents field, 'advertise' means to make known that you have some kind of query you'll answer or information you'll provide. Kind of like, "I'm willing to answer queries for music by bands frequently played on such-and-such radio station." Agents decide to make subscriptions on the basis of advertisements, otherwise, how is the agent to know what to subscribe to? ian From zooko at zooko.com Thu Aug 30 08:17:01 2001 From: zooko at zooko.com (zooko@zooko.com) Date: Sat Dec 9 22:11:43 2006 Subject: proxying and introduction: the two fundamental operations of emergent networks (was: Re: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF)) In-Reply-To: Message from Sam Joseph of "Thu, 30 Aug 2001 23:54:24 +0900." <3B8E539F.FB02A024@neurogrid.com> References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8DDFFA.CFA67BC9@neurogrid.com> <20010830131553.A1801@sandbergs.org> <3B8E539F.FB02A024@neurogrid.com> Message-ID: Sam Joseph writes: > > Oskar Sandberg wrote: > > > On Thu, Aug 30, 2001 at 03:40:58PM +0900, Sam Joseph wrote: > > <> > > > Okay so could anybody from MojoNation comment on what makes their routing > > > methodology different from Freenet. I understand the Freenet does depth first > > > greedy search when routing. MojoNation routes like Morpheus, or Gnutella perhaps? > > > > Can a single step match of data UID against node UIDs really be called > > routing? > > Well if you're forwarding a message as a result of the match, then I'd say yes. However > the important point, for me, is what criteria is being used to forward messages. No, Mojo Nation does not use information from the higher-layer (file operations or metadata operations) to do routing at the message layer. We do not use proxying to extend the network horizon, we use introducing. Freenet's concept of "routing" is an implementation of proxying of file operations. I've wanted to write an essay for p2p-hackers but never had the time. In brief, there are two fundamental operations that can extend your network horizon: proxying and introducing. Imagine three nodes named Akira, Bakunin and Constantine. Akira has a link with Bakunin (all links are bi-directional), and Bakunin has a link with Constantine. So Akira's "network horizon" is of size 1: he only knows one node other than himself. Now there are two possible ways to make Constantine become part of Akira's network. The first, proxying, is where Akira sends a query to Bakunin, who forwards it to Constantine, receives the response from Constantine, and forwards the response back to Akira. Akira now *thinks* that his network horizon is only 1, but it is actually 2. The other technique is introduction: Bakunin tells Akira and Constantine how to talk to each other, and now each person on the network has two links. There are lots of interesting technical differences between these two techniques and how the resulting emergent network behaves, but that's the part that I don't have the time to write about. ;-) Oh, by the way, there are two ways that Mojo Nation *does* do proxying behavior, but neither is its primary mechanism of doing file operations. The first is message-relaying, which allows arbitrary higher-layer operations to bypass firewalls (currently "higher-layer operations" include file operations and metadata operations, but also any other kind of protocol could be written on the higher layer and it would transparently bypass firewalls). (Also message-relaying provides a nice starting point for someone who wants to implement Chaumian message mixes.) The second is what we call "block wholesaling", which is a batched operation in which each block server launches a recursive query (but *only* 1-level deep recursion) for blocks which many people have requested of it. That happens once a day, or once a week or something depending on how the user has configured their block server. Regards, Zooko P.S. Thanks to Oskar Sandberg, Adam Langley and Bram Cohen for arguing about "Mojo Nation's scalability" a few months ago on p2p-hackers, which got me thinking, and to Mark Miller for teaching me about the fundamental necessity of the Granovetter Operation (== introduction). From sam at neurogrid.com Thu Aug 30 08:32:01 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8E55E7.50627812@neurogrid.com> Message-ID: <3B8E5BEE.E4E76AA@neurogrid.com> Ian Soboroff wrote: > Sam Joseph writes: > > > Well it's interesting you say advertise, because a publish and > > subscribe model is another way to go. I have this innate dislike of > > advertising, ever since I quit working for an advertising company, > > so I tend to think of semantic routing systems where each node is > > gradually learning about the contents of nearby nodes, so there is > > no advertising process. When a node returns a document, the other > > nodes that see it say "Ah, that node is storing data about XXXX" and > > they passively build up a picture of what is where in the network, > > and then forward queries on that basis. > > Ahh... I was using the term 'advertise' almost but not quite in the > sense of 'publish'. In the intelligent agents field, 'advertise' > means to make known that you have some kind of query you'll answer or > information you'll provide. Kind of like, "I'm willing to answer > queries for music by bands frequently played on such-and-such radio > station." Agents decide to make subscriptions on the basis of > advertisements, otherwise, how is the agent to know what to subscribe > to? That's if you're talking about a publish and subscribe system, which I'm not. I'm suggesting that you don't have to subscribe to anybody, you can send out queries, initially at random, but as you learn they can be sent out selectively, and if every node in the network is doing the same you are routing messages semantically. CHEERS> SAM From oskar at freenetproject.org Thu Aug 30 08:38:01 2001 From: oskar at freenetproject.org (Oskar Sandberg) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: ; from ian.soboroff@nist.gov on Thu, Aug 30, 2001 at 08:33:16AM -0400 References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> Message-ID: <20010830174130.B1801@sandbergs.org> On Thu, Aug 30, 2001 at 08:33:16AM -0400, Ian Soboroff wrote: > "Gregory P. Smith" writes: > > > A bridge of sorts could be constructed between the two using "generic IPC > > mechanism", probably after a couple of hacker days, such that a freenet > > node could use mojo nation for its data storage and a mojo nation broker > > could in turn use freenet for its data storage. However, I don't know > > of anybody actually working on such a thing. > > But I believe that the original start of this thread concerned > _semantic_ routing. The idea here is to decide whether to send a > query to a peer based on the contents of the documents on that > server. What current p2p systems do is routing based on network > topology and packet lifetime. In a semantic p2p system, peers would > advertise content "summaries" (for example, highly relevant terms from > titles of songs in a music system) to their neighbors in the network, > and the routing algorithm could be tuned to send messages to only the > top n neighbors or some such. By that generalized definition of semantic routing Freenet, Chord, Plaxton, Oceanstore, PAST etc all use semantic routing. In fact, even Squid clusters (working with Bloom filters) would qualify. In all these systems though, the organisation is based on identifiers that are independent of the actual content of the data - which is what I assumed SJ was interested in since his Neurogrid system organizes based on subjective human reaction to the relevance of the results. > Anyone have more thoughts on this? (Rather than the relative merits > of hash implementations... ;-) A information storage system that wants to be scalable HAS to work by organizing the data internally in some manner. I am fond of the Freenet approach, because it is inherently "chaotic" (quoting to avoid mathematical definition), but it still requires a lot of work, and may yet prove to unworkable in pratice. Implementing Tapestry (Oceanstores Plaxton derivative) for our simulator has made me much more scepticle of the idea of trying to force fit a fixed mesh model (like all the hypercube derivative and Chord) into fickle networks like online p2p environments then even I was previously. > If the community is still focused on routing queries for music, you > might be interested in a collection of folk music at the New Zealand > Digital Library (www.nzdl.org). Users hum their query, and the system > can match it against songs. (This works because, while most people > have terrible pitch, they do tend to remember the pattern of note > changes and tempos rather well.) The question in a p2p environment is > how to collect and spread the semantic summary information for routing > cheaply. In "theory" (quoting because it's actually speculation) Freenet's routing should work equally well for any content identifier to which there is a non-trivial metric. -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg oskar@freenetproject.org From zooko at zooko.com Thu Aug 30 08:39:01 2001 From: zooko at zooko.com (zooko@zooko.com) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: Message from Sam Joseph of "Thu, 30 Aug 2001 23:59:52 +0900." <3B8E54E8.74229A89@neurogrid.com> References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8DDFFA.CFA67BC9@neurogrid.com> <3B8E54E8.74229A89@neurogrid.com> Message-ID: Sam Joseph writes: > > zooko@zooko.com wrote: > > > Err.. I'd be happy to, but what's the point, again? If the goal is to link > > the two networks together then do as Greg suggested (and as Brandon Wiley > > suggested in the "Peer To Peer" book) and make a gateway node which consists of > > one normal node from each network, linked with IPC. [1, 2] > > Actually I was hoping for a placement of MojoNation's "Message Forwarding" in my own terminology. I was thinking that Freenet > forwards requests around on the basis of a hash lookup table, gnutella broadcasts messages within a certain horizon, Morpheus ? > MojoNation has some hashing in it but ... Okay. Message relaying is done strictly on a sender-specified basis. I send a message to you and instruct you to forward it to him. This can be chained (although it is not, currently) so that I send a message to you, instructing you to forward it to him and instructing him to forward it to her. But there is no relayer-chosen routing (except, of course, that you as the relayer get to choose how to deliver the message to *him*, which might include another layer of relaying! (Although it never does on the current MNet.)). Now file operations are something else entirely. You just ask block servers that you have a direct link with, and if that isn't enough, then you get introduced to more, but you are always asking block servers with whom you have a direct link (although that "direct link" might be translated into one or more relay hops in practice, which is transparent to you). Okay, so I'm glad to be able to clarify this just so people understand Mojo Nation's design better, but for *your* purposes, concerning semantic routing, you are not interested in the messaging layer, nor in the file operations, you are interested in the metadata operations. So we do not do any proxying on metadata operations, but we do have implemented, but unused, a little trick to use semantic information to choose which of the Content Trackers that you know are most likely to have the information you seek. It was inspired by a comment of Raph Levien's, and it is similar to the "fuzzy searching" ideas that the Freenet people kicked around. It is simple enough that you can read the source code (even if you don't know Python, you could probably read this even if just by reading the comments): http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/~checkout~/mojonation/evil/common/ContentHandicappers.py?rev=HEAD&content-type=text/plain Also there is a Mojo Nation hacker who has implemented a technique that hashes each 3-letter substring of the query, or something like that. Regards, Zooko From sam at neurogrid.com Thu Aug 30 08:47:01 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:43 2006 Subject: proxying and introduction: the two fundamental operations of emergent networks (was: Re: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF)) References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8DDFFA.CFA67BC9@neurogrid.com> <20010830131553.A1801@sandbergs.org> <3B8E539F.FB02A024@neurogrid.com> Message-ID: <3B8E5F9C.F517F1CA@neurogrid.com> We're out of synch on mails here and I'm about to crash, but here goes my response to your response, before you sent your response to my second response. [I'll pick this up again in the morning] So okay, but if I want to send a message from A to B I can request that the message is forwarded to B by the intervening nodes. I think I must be missing something about MN's architecture. Like documents are hashed and then stored, but we don't have to search for them. So I went to the MN technical docs "File retrieval on Mojo Nation begins with a content search. At the search page, the user can select from a growing number of content types, and each of the content types presents its own array of type fields to delineate the user's search (that is, the user could search for a certain "bitrate" among the "audio" content types, but not others). After the user provides his search criteria and clicks "search," the Broker goes back to work. " So a broker is a not the MN node on my computer, but another stable node? "First the Broker locates every content tracker available on the system, then sorts them -- first by the price each asks to perform a lookup, and further by the tracker's reputations. " The broker performs this location through meta-trackers? This is a search right? So first I ask the nearest meta-tracker for a list of content trackers, and then that meta-tracker .... forwards the request to other meta-trackers ... the broker has to send directly to each meta-tracker ... how does the broker find new meta-trackers? I guess I'm talking about the introduction process but ... tomorrow I will run this through my cortex again. zooko@zooko.com wrote: > No, Mojo Nation does not use information from the higher-layer (file operations > or metadata operations) to do routing at the message layer. We do not use > proxying to extend the network horizon, we use introducing. Freenet's concept > of "routing" is an implementation of proxying of file operations. From ian.soboroff at nist.gov Thu Aug 30 09:04:01 2001 From: ian.soboroff at nist.gov (Ian Soboroff) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: <3B8E5BEE.E4E76AA@neurogrid.com> References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8E55E7.50627812@neurogrid.com> <3B8E5BEE.E4E76AA@neurogrid.com> Message-ID: Sam Joseph writes: > > Ahh... I was using the term 'advertise' almost but not quite in the > > sense of 'publish'. In the intelligent agents field, 'advertise' > > means to make known that you have some kind of query you'll answer or > > information you'll provide. Kind of like, "I'm willing to answer > > queries for music by bands frequently played on such-and-such radio > > station." Agents decide to make subscriptions on the basis of > > advertisements, otherwise, how is the agent to know what to subscribe > > to? > > That's if you're talking about a publish and subscribe system, which I'm > not. I'm suggesting that you don't have to subscribe to anybody, you can > send out queries, initially at random, but as you learn they can be sent > out selectively, and if every node in the network is doing the same you > are routing messages semantically. This is about learning where to route a query in the network, right? So essentially you're proposing to try to learn the contents of peers (so you'll know who's best to send later queries to) by looking at how they respond to queries. This is tricky... folks working on metasearch engines have tried something similar with mixed results. Furthermore, you're talking about systems that have highly dynamic collections, so what you've learned about a peer is going to be out of date at some point in the near future. Rather than publish-and-subscribe, you could honor a query for content-based metadata instead. That still presumes you have a list of peers to ask, which is how current systems work. ian From zooko at zooko.com Thu Aug 30 10:00:01 2001 From: zooko at zooko.com (zooko@zooko.com) Date: Sat Dec 9 22:11:43 2006 Subject: proxying and introduction: the two fundamental operations of emergent networks (was: Re: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF)) In-Reply-To: Message from Sam Joseph of "Fri, 31 Aug 2001 00:45:32 +0900." <3B8E5F9C.F517F1CA@neurogrid.com> References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8DDFFA.CFA67BC9@neurogrid.com> <20010830131553.A1801@sandbergs.org> <3B8E539F.FB02A024@neuro <3B8E5F9C.F517F1CA@neurogrid.com> Message-ID: Sam Joseph writes: > > I think I must be missing something about MN's architecture. Like documents are hashed and > then stored, but we don't have to search for them. > > So I went to the MN technical docs > > "File retrieval on Mojo Nation begins with a content search. At the search page, the user can > select from a growing number of content types, and each of the content types presents its own > array of type fields to delineate the user's search (that is, the user could search for a > certain "bitrate" among the "audio" content types, but not others). After the user provides > his search criteria and clicks "search," the Broker goes back to work. " > > So a broker is a not the MN node on my computer, but another stable node? No, all MN nodes are called "brokers". (Except for Meta Trackers, which serve a special purpose. A partial analogy for Meta Trackers is the "start page" that Freenet or Gnutella have that gives you a list of the IP addresses of nodes so you can connect to the network for the first time ever.) > "First the Broker locates every content tracker available on the system, then sorts them -- > first by the price each asks to perform a lookup, and further by the tracker's reputations. " > > The broker performs this location through meta-trackers? This is a search right? So first I > ask the nearest meta-tracker for a list of content trackers, and then that meta-tracker .... > forwards the request to other meta-trackers ... the broker has to send directly to each > meta-tracker ... how does the broker find new meta-trackers? > > I guess I'm talking about the introduction process but ... tomorrow I will run this through my > cortex again. Hrm. Yeah. Currently there are three MTs, which, in my limited analogy is rather like having three redundant web pages that serve up the same list of IP addresses of nodes. So after you have a list of nodes, some of which are Content Trackers, then you send to some of the CTs that you know a metadata query in XML. The CT sends back XML formatted metadata response, which contains the mojoid (the unique key useful for finding the actual file). This whole CT thing is related to the actual file distribution only by including the mojoid. If Bitzi or Google included mojoids in their metadata results then we would not need CTs at all. Regards, Zooko From oskar at freenetproject.org Thu Aug 30 10:15:01 2001 From: oskar at freenetproject.org (Oskar Sandberg) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: <3B8E54E8.74229A89@neurogrid.com>; from sam@neurogrid.com on Thu, Aug 30, 2001 at 11:59:52PM +0900 References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8DDFFA.CFA67BC9@neurogrid.com> <3B8E54E8.74229A89@neurogrid.com> Message-ID: <20010830191839.C1801@sandbergs.org> On Thu, Aug 30, 2001 at 11:59:52PM +0900, Sam Joseph wrote: > zooko@zooko.com wrote: > > > Err.. I'd be happy to, but what's the point, again? If the goal is to link > > the two networks together then do as Greg suggested (and as Brandon Wiley > > suggested in the "Peer To Peer" book) and make a gateway node which consists of > > one normal node from each network, linked with IPC. [1, 2] > > Actually I was hoping for a placement of MojoNation's "Message Forwarding" in my own terminology. I was thinking that Freenet > forwards requests around on the basis of a hash lookup table, gnutella broadcasts messages within a certain horizon, Morpheus ? Morpheus ("KaZaa") uses a broadcast among a subset of the nodes, chosen, I believe, by some measure of performance and connectivity. -- 'DeCSS would be fine. Where is it?' 'Here,' Montag touched his head. 'Ah,' Granger smiled and nodded. Oskar Sandberg oskar@freenetproject.org From sam at neurogrid.com Thu Aug 30 23:35:01 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8DDFFA.CFA67BC9@neurogrid.com> <3B8E54E8.74229A89@neurogrid.com> <20010830191839.C1801@sandbergs.org> Message-ID: <3B8F2F8E.C7DCF144@neurogrid.com> Oskar Sandberg wrote: > > Actually I was hoping for a placement of MojoNation's "Message Forwarding" in my own terminology. I was thinking that Freenet > > forwards requests around on the basis of a hash lookup table, gnutella broadcasts messages within a certain horizon, Morpheus ? > > Morpheus ("KaZaa") uses a broadcast among a subset of the nodes, chosen, > I believe, by some measure of performance and connectivity. Thanks Oskar. I don't suppose you know who I might talk to to get more details on that? CHEERS> SAM From sam at neurogrid.com Thu Aug 30 23:39:01 2001 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:11:43 2006 Subject: proxying and introduction: the two fundamental operations of emergent networks (was: Re: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF)) References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8DDFFA.CFA67BC9@neurogrid.com> <20010830131553.A1801@sandbergs.org> <3B8E539F.FB02A024@neuro <3B8E5F9C.F517F1CA@neurogrid.com> Message-ID: <3B8F3073.7E78FC23@neurogrid.com> zooko@zooko.com wrote: > Sam Joseph writes: > > "First the Broker locates every content tracker available on the system, then sorts them -- > > first by the price each asks to perform a lookup, and further by the tracker's reputations. " > > > > The broker performs this location through meta-trackers? This is a search right? So first I > > ask the nearest meta-tracker for a list of content trackers, and then that meta-tracker .... > > forwards the request to other meta-trackers ... the broker has to send directly to each > > meta-tracker ... how does the broker find new meta-trackers? > > > > I guess I'm talking about the introduction process but ... tomorrow I will run this through my > > cortex again. > > Hrm. Yeah. Currently there are three MTs, which, in my limited analogy is > rather like having three redundant web pages that serve up the same list of IP > addresses of nodes. > > So after you have a list of nodes, some of which are Content Trackers, then you > send to some of the CTs that you know a metadata query in XML. The CT sends > back XML formatted metadata response, which contains the mojoid (the unique key > useful for finding the actual file). Okay so I get hold of some set of content trackers and then I query them directly, and they may or not respond with anything useful. So presumably in order for me to find what I want I have to rely on the content trackers updating themselves in a timely fashion (which they do how?) and that the three MTs have a complete list of all the content trackers between them? So I guess my question becomes, how do the content trackers maintain fresh data about where content data is, and how do the meta-trackers maintain fresh data about where the content data is? CHEERS> SAM p.s. thanks for putting up with my loosely formed questions. From hal at finney.org Thu Aug 30 23:51:01 2001 From: hal at finney.org (hal@finney.org) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) Message-ID: <200108310650.XAA17174@finney.org> Regarding Morpheus, aka Kazaa, aka FastTrack, the protocol is proprietary and undocumented. However there is an open source effort to reverse engineer it and they have succeeded in creating a compatible client which can search and download. See http://sourceforge.net/projects/gift/ (formerly kazaatux). Hal From zooko at zooko.com Fri Aug 31 07:13:02 2001 From: zooko at zooko.com (zooko@zooko.com) Date: Sat Dec 9 22:11:43 2006 Subject: proxying and introduction: the two fundamental operations of emergent networks (was: Re: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF)) In-Reply-To: Message from Sam Joseph of "Fri, 31 Aug 2001 15:36:35 +0900." <3B8F3073.7E78FC23@neurogrid.com> References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8DDFFA.CFA67BC9@neurogrid.com> <20010830131553.A1801@sandbergs.org> <3B8E539F.FB02A024@neuro <3B8F3073.7E78FC23@neurogrid.com> Message-ID: Sam Joseph writes: > > Okay so I get hold of some set of content trackers and then I query them directly, and they may or > not respond with anything useful. So presumably in order for me to find what I want I have to rely > on the content trackers updating themselves in a timely fashion (which they do how?) and that the > three MTs have a complete list of all the content trackers between them? > > So I guess my question becomes, how do the content trackers maintain fresh data about where content > data is, and how do the meta-trackers maintain fresh data about where the content data is? Yes, this is how it works. CTs get updated (directly) by the agent that originally publishes a document. I don't understand the second part of your last question. One thing that makes this work is that all data is immutable and long-lived, so the CTs do not need to be updated all that frequently. But frankly CTs in Mojo Nation are a low-tech hack that was just "good enough", and which works for what it currently does. I would really rather that Bitzi, google, Neurogrid, or some other publically available service take over the job of CTing for MN. The innovative and cool parts of MN are the IDA, the block-fetching (you start with the blockid == SHA1(block) and you retrieve the block), the swarming delivery, the Mojo tokens, the "self-healing network" (transitive introductions with limited transitive reputation), and most of all the fact that it is all debugged and tuned enough that it works and you can run it today and enjoy the results. Regards, Zooko From painlord at inwind.it Fri Aug 31 07:24:01 2001 From: painlord at inwind.it (Mirco Romanato) Date: Sat Dec 9 22:11:43 2006 Subject: proxying and introduction: the two fundamental operations of emergent networks (was: Re: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF)) References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8DDFFA.CFA67BC9@neurogrid.com> <20010830131553.A1801@sandbergs.org> <3B8E539F.FB02A024@neuro <3B8E5F9C.F517F1CA@neurogrid.com> <3B8F3073.7E78FC23@neurogrid.com> Message-ID: <007001c13228$4a881f00$ce73623e@painlordt91ikd> ----- Original Message ----- From: "Sam Joseph" > zooko@zooko.com wrote: > > So after you have a list of nodes, some of which are Content Trackers, then you > > send to some of the CTs that you know a metadata query in XML. The CT sends > > back XML formatted metadata response, which contains the mojoid (the unique key > > useful for finding the actual file). > Okay so I get hold of some set of content trackers and then I query them directly, and they may or > not respond with anything useful. So presumably in order for me to find what I want I have to rely > on the content trackers updating themselves in a timely fashion (which they do how?) and that the > three MTs have a complete list of all the content trackers between them? > So I guess my question becomes, how do the content trackers maintain fresh data about where content > data is, and how do the meta-trackers maintain fresh data about where the content data is? The only thing MT know are that there are brokers and the services they offer (BS, CT, RS). They don't know where is the content, they only know where the BS are. As I Understand, after a brokers connect with the MT it receive the IP of many existent broker and their services. When need of other brokers raise, it (our broker) can ask to other brokers a few IP of other brokers and their services or ask again to the MTs. In the future, probably, will be a way to connect to MN without MT contact (like in Music City and Gnutella will be no need of a central server - not counting the Token Server) The broker that publish a resource will send to all (at time) the available CTs the data and the mojoid (and dinode) of the published resource. Also you can avoid this if you don't want the resource be available at a casual search. So only broker with the MojoId or the dinode will be able to download the resource. When the broker has the MojoId it will recreate the sharemap (the first series of blocks) that contain tha names of the blocks needed to download the resorce (also, if the resource is a small file, the sharemap will not exist and it's substituted with the blocks recreating the resource itseft) When the broker know the blocks it want, it will start to ask to the blockservers with a compatible mask; so the search is not deterministic, but probabilistic). "Rightful liberty is unobstructed action, according to our will, within limits drawn around us by the equal rights of others." -- Thomas Jefferson --------------------------------------------------------- Mirco Romanato Venezia - Italia From cyb at azrael.dyn.cheapnet.net Fri Aug 31 12:02:01 2001 From: cyb at azrael.dyn.cheapnet.net (Brandon K. Wiley) Date: Sat Dec 9 22:11:43 2006 Subject: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) In-Reply-To: <20010829230334.A16864@zot.electricrain.com> Message-ID: > A bridge of sorts could be constructed between the two using "generic IPC > mechanism", probably after a couple of hacker days, such that a freenet > node could use mojo nation for its data storage and a mojo nation broker > could in turn use freenet for its data storage. However, I don't know > of anybody actually working on such a thing. Indeed it is being worked on. I have defined a generic API for file inserts and requests. Freenet exposes this API via XML-RPC. The spec is at http://freenetproject.org/docs/xml-rpc.html. You could expose it via other RPC mechanisms as well. As far as I know the projects which are actively looking into integrating this are MojoNation and BitTorrent (well, because I offered to personally do the BitTorrent integration). I've talked to the Swarmcast developers about integrating in the next version of Swarmcast. We'll see how that pans out when the next version comes out. Also, there is one client, Snarfzilla, which uses this API to talk to the server. While it's technically a Freenet client, use of the XML-RPC bindings for the API means that you should be able to use it to talk to any of the systems which implement the API. From painlord at inwind.it Fri Aug 31 12:30:01 2001 From: painlord at inwind.it (Mirco Romanato) Date: Sat Dec 9 22:11:43 2006 Subject: Fw: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) Message-ID: <010201c13253$2b942cd0$0651623e@painlordt91ikd> ----- Original Message ----- From: "Mirco Romanato" To: Sent: Wednesday, August 29, 2001 2:55 PM Subject: Re: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF) > > Do any of the Freenet or MojoNation people have an opinion on this? > Mirco > > is suggesting that a MojoNation node could be turned into a Freenet > node > > with a little tweaking. Does anybody know how much tweaking? > This is my source: > > Sorceforge - CVS\MojoNation\evil\hackerdocs\interface_spec.html > say > "One advantage of adhering to the XML-RPC open standard is that Mojo > Nation's interface can imitate XML-RPC interfaces of similar systems > with little effort. The Mojo Nation XML-RPC interface has a "Freenet > Compatibility Mode" which, when enabled, makes a Broker appear just > like a Freenet node to any Freenet XML-RPC client" > > and > Sourceforge - CVS\MojoNation\evil\hackerdocs\LJ_article.html > say: > "By the way, Mojo Nation's XML-RPC API is identical to the Freenet > project's "simple XML-RPC API"." > > > > Also, are there any Morpheus developers here? Anybody know the > details of > > query propagation between Morpheus supernodes? > I only know what I read on a few news articles about Music City and > his network and the DCMA - RIAA problems. > Appear that the supernodes are hosted by user with Hi-Speed connection > (ADSL or more) and Hi-Speed computer (?). > The life of the node is variable and new supernodes will pop-up when > need arise. > The nodes will send request to the supernode and if supernode don't > find enough files it propagate the query to the other supernodes. > There is no need to connect to Music City, only to connect a existing > supernode (like gnutella) and this is a working possibility (they > tested it for defensive purpose). They say that they don't control the > network and don't control the indexes of available files, because the > indexes stay on supernodes (and supernodes have short life). > > > "It does me no injury for my neighbor to say there are twenty gods or > no God. > It neither picks my pocket nor breaks my leg." --- Thomas Jefferson > > > Mirco Romanato > Venezia - Italia > From painlord at inwind.it Fri Aug 31 12:34:01 2001 From: painlord at inwind.it (Mirco Romanato) Date: Sat Dec 9 22:11:43 2006 Subject: Fw: proxying and introduction: the two fundamental operations of emergent networks (was: Re: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF)) Message-ID: <010b01c13253$ac57e3c0$0651623e@painlordt91ikd> ----- Original Message ----- From: "Mirco Romanato" To: Sent: Friday, August 31, 2001 4:21 PM Subject: Re: proxying and introduction: the two fundamental operations of emergent networks (was: Re: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF)) > ----- Original Message ----- > From: "Sam Joseph" > > zooko@zooko.com wrote: > > > So after you have a list of nodes, some of which are Content > Trackers, then you > > > send to some of the CTs that you know a metadata query in XML. > The CT sends > > > back XML formatted metadata response, which contains the mojoid > (the unique key > > > useful for finding the actual file). > > Okay so I get hold of some set of content trackers and then I query > them directly, and they may or > > not respond with anything useful. So presumably in order for me to > find what I want I have to rely > > on the content trackers updating themselves in a timely fashion > (which they do how?) and that the > > three MTs have a complete list of all the content trackers between > them? > > > So I guess my question becomes, how do the content trackers maintain > fresh data about where content > > data is, and how do the meta-trackers maintain fresh data about > where the content data is? > The only thing MT know are that there are brokers and the services > they offer (BS, CT, RS). > They don't know where is the content, they only know where the BS are. > > As I Understand, after a brokers connect with the MT it receive the IP > of many existent broker and their services. > When need of other brokers raise, it (our broker) can ask to other > brokers a few IP of other brokers and their services or ask again to > the MTs. > > In the future, probably, will be a way to connect to MN without MT > contact (like in Music City and Gnutella will be no need of a central > server - not counting the Token Server) > > The broker that publish a resource will send to all (at time) the > available CTs the data and the mojoid (and dinode) of the published > resource. Also you can avoid this if you don't want the resource be > available at a casual search. So only broker with the MojoId or the > dinode will be able to download the resource. > > When the broker has the MojoId it will recreate the sharemap (the > first series of blocks) that contain tha names of the blocks needed to > download the resorce (also, if the resource is a small file, the > sharemap will not exist and it's substituted with the blocks > recreating the resource itseft) > > When the broker know the blocks it want, it will start to ask to the > blockservers with a compatible mask; so the search is not > deterministic, but probabilistic). > > > "Rightful liberty is unobstructed action, according to our > will, within limits drawn around us by the equal rights of > others." -- Thomas Jefferson > --------------------------------------------------------- > Mirco Romanato > Venezia - Italia > From greg at electricrain.com Fri Aug 31 22:13:01 2001 From: greg at electricrain.com (Gregory P. Smith) Date: Sat Dec 9 22:11:43 2006 Subject: proxying and introduction: the two fundamental operations of emergent networks (was: Re: [p2p-hackers] Morpheus, Freenet, MojoNation (was Semantic Routing BOF)) In-Reply-To: ; from zooko@zooko.com on Thu, Aug 30, 2001 at 08:08:50AM -0700 References: <3B8CC5B7.4FB5315E@neurogrid.com> <20010829230334.A16864@zot.electricrain.com> <3B8DDFFA.CFA67BC9@neurogrid.com> <20010830131553.A1801@sandbergs.org> <3B8E539F.FB02A024@neurogrid.com> Message-ID: <20010831221232.A766@zot.electricrain.com> > Oh, by the way, there are two ways that Mojo Nation *does* do proxying > behavior, but neither is its primary mechanism of doing file operations. > > The first is message-relaying, which allows arbitrary higher-layer operations ... > The second is what we call "block wholesaling", which is a batched operation in > which each block server launches a recursive query (but *only* 1-level deep > recursion) for blocks which many people have requested of it. That happens > once a day, or once a week or something depending on how the user has > configured their block server. ...there are two, no, Three ways that Mojo Nation behaves this way. The first method is (see above) and the second is (see above) our primary weapons are (above) and (above)... The third is that upon initial publication the broker a block of data is sent to carries with a flag saying "pass this on to someone else if you please." This isn't required for things to function, its just something I put in a while back that may or may not actually be bringing us any great benefit. -g