From izzy at lina.es.ncku.edu.tw Thu Oct 2 05:25:22 2003 From: izzy at lina.es.ncku.edu.tw (izzy) Date: Sat Dec 9 22:12:22 2006 Subject: [p2p-hackers] Partial Match Problem in DHT(Kademlia). Message-ID: <000d01c388a5$93c8af10$7327748c@izzy> As I know, partial match is a key disadvantage in DHT-based search algorithm(Chord,CAN,Pastry) . For the time being , I haven't seen any effective solution out there. Overnet and Emule Kademlia adopt Kademlia Algorithm to achieve fully decentralized self-organization. Is there anyone knowing how they make it possible on partial match problem ? Thanks a lot ! From mikei at kinitos.com Thu Oct 2 13:04:15 2003 From: mikei at kinitos.com (Michael Iles) Date: Sat Dec 9 22:12:22 2006 Subject: [p2p-hackers] Partial Match Problem in DHT(Kademlia). In-Reply-To: <000d01c388a5$93c8af10$7327748c@izzy> Message-ID: <022b01c388e5$af409740$66bba8c0@MIKEI> I've read about two approaches. To achieve fuzzy matching on a given identifier it can be broken into bigrams or trigrams and each trigram can be indexed separately. For example, 'Beethoven' would be indexed as 'Bee', 'eet', 'eth', 'tho', 'hov', 'ove', 'ven'. If a use submitted a search for 'Beetoven' the system would submit searches for the six trigrams and they would match all except 'eth', 'tho', 'hov' and so the result would come back as a partial match. This is written up in [1]. At a higher level, I've read briefly about a proposal to match a collection of tags identifying an item given a subset of those tags. For example, if a song were indexed by artist, album, title and the user searched only by title, the title search would return a list of queries that contained the search which could then be recursively queried to get to the specific result [2]. [1] Matthew Harren, Joseph M. Hellerstein, Ryan Huebsch, Boon T. Loo, Scott Shenker, and Ion Stoica. Complex queries in dht-based peer-to-peer networks. In Proceedings of the 1st International Workshop on Peer-to- Peer Systems (IPTPS), Cambridge, MA, March 2002. [2] P.A. Felber, E.W. Biersack, L. Garc?es-Erice, K.W. Ross, G. Urvoy-Keller. Data Indexing and Querying in DHT Peer-to-Peer Networks. Institut EURECOM, 06904 Sophia Antipolis, France. Mike. -- michael iles kinitos smart-client management mikei at kinitos dot com centralized control, distributed power -----Original Message----- From: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] On Behalf Of izzy Sent: October 2, 2003 1:25 AM To: p2p-hackers@zgp.org Subject: [p2p-hackers] Partial Match Problem in DHT(Kademlia). Importance: High As I know, partial match is a key disadvantage in DHT-based search algorithm(Chord,CAN,Pastry) . For the time being , I haven't seen any effective solution out there. Overnet and Emule Kademlia adopt Kademlia Algorithm to achieve fully decentralized self-organization. Is there anyone knowing how they make it possible on partial match problem ? Thanks a lot ! _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From sam at neurogrid.com Thu Oct 2 19:19:06 2003 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:12:22 2006 Subject: [p2p-hackers] Partial Match Problem in DHT(Kademlia). In-Reply-To: <000d01c388a5$93c8af10$7327748c@izzy> References: <000d01c388a5$93c8af10$7327748c@izzy> Message-ID: <3F7C7A2A.7000500@neurogrid.com> You could always try a search layer on top of your DHT: http://www.neurogrid.net/php/publications.php Top two papers give summaries of these kinds of approaches: @InProceedings{Joseph03b, author = {Sam Joseph}, title = {P2P MetaData Search Layers}, booktitle = {Second International Workshop on Agents and Peer-to-Peer Computing}, year = {2003} abstract = {Distributed Hashtables (DHTs) provide a scalable method of associating file-hashes with a particular location in a distributed network environment. Modifying DHTs directly to support meta-data is difficult, and meta-data search systems such as flooding tend to scale poorly. However, a number of more scalable distributed meta-data search systems have recently been developed that could be deployed in tandem with DHTs, and several are discussed here along with some novel simulation results that concern the scalability and resource limitations of a meta-data search layer that employs semantic routing. Semantic routing is a method of pruning a flooding search such that queries are preferentially forwarded to nodes that can answer those queries.}, URL= {http://www.neurogrid.net/P2PMetaDataSearchLayers-improved.pdf} } @article{Joseph03a, author = {Sam Joseph and Takashige Hoshiai}, title = {Decentralized Meta-Data Strategies: Effective Peer-to-Peer Search}, journal = {IEICE Transactions on Communications}, volume = {E86-B}, number = {6}, pages={1740-1753,} year = {2003}, abstract = {The release of the Gnutella file sharing system in Mach 2000 stirred worldwide interest in the Peer-to-Peer (P2P) computing model. Put simply, the P2P model does not depend on a centralised management broker, like those that play an important role in many internet business models. The P2P approach enables peers to discover other peers with desired content or similar interests, without having to go through an intermediary server. It seems clear that the wealth of content made available by peer-to-peer systems such as Gnutella and Freenet have spurred many authors into considering how meta-data might be used to support more effective search in such a distributed environment. This paper reviews a number of these systems and their decentralized meta-data strategies while attempting to identify some common themes.}, URL={http://search.ieice.org/2003/pdf/e86-b_6_1740.pdf} } CHEERS> SAM izzy wrote: >As I know, partial match is a key disadvantage in DHT-based search >algorithm(Chord,CAN,Pastry) . >For the time being , I haven't seen any effective solution out there. >Overnet and Emule Kademlia adopt Kademlia Algorithm to achieve fully >decentralized self-organization. >Is there anyone knowing how they make it possible on partial match problem ? >Thanks a lot ! > From izzy at lina.es.ncku.edu.tw Thu Oct 2 17:56:45 2003 From: izzy at lina.es.ncku.edu.tw (izzy) Date: Sat Dec 9 22:12:22 2006 Subject: [p2p-hackers] Partial Match Problem in DHT(Kademlia). Message-ID: <000b01c3890e$93c84930$7327748c@izzy> I have seen a similar approach in [1]. But considering the scenario of file-sharing application, most systems (Kazza,EDonkey) use file name as the keyword. In this situation, file name often are a sentence, like "Guns N' Roses Live Concert in Taiwan",etc. It would be indexed in huge numbers of ID. I don't think it's a feasible solution. The second approach mentioned above, though it achieves quite similar consequence. In the strictly speaking, it isn't really dealing with the partial match problem. Also it raise another issue, how to define your classification, which is also an open problem in Semantic Web. I guess Overnet and Emule Kademlia adopt Kademlia Algorithm only for discovering nodes and searching nodes. But they still use the flooding mechanism (Gnutella-like) to search files. That's just my guess. Anyway, I am happy to discuss with you. It's an interesting challenge in DHT-based system. [1] A Keyword Set Search System for Peer-to-Peer Networks. by Omprakash D Gnawali. Master's thesis, Massachusetts Institute of Technology, June 2002. From izzy at lina.es.ncku.edu.tw Thu Oct 2 18:15:15 2003 From: izzy at lina.es.ncku.edu.tw (izzy) Date: Sat Dec 9 22:12:22 2006 Subject: [p2p-hackers] Re: Partial Match Problem in DHT(Kademlia). Message-ID: <001001c38911$20be0800$7327748c@izzy> I have seen a similar approach in [1]. But considering the scenario of file-sharing application, most systems (Kazza,EDonkey) use file name as the keyword. In this situation, file name often are a sentence, like "Guns N' Roses Live Concert in Taiwan",etc. It would be indexed in huge numbers of ID. I don't think it's a feasible solution. The second approach mentioned above, though it achieves quite similar consequence. In the strictly speaking, it isn't really dealing with the partial match problem. Also it raise another issue, how to define your classification, which is also an open problem in Semantic Web. I guess Overnet and Emule Kademlia adopt Kademlia Algorithm only for discovering nodes and searching nodes. But they still use the flooding mechanism (Gnutella-like) to search files. That's just my guess. Anyway, I am happy to discuss with you. It's an interesting challenge in DHT-based system. [1] A Keyword Set Search System for Peer-to-Peer Networks. by Omprakash D Gnawali. Master's thesis, Massachusetts Institute of Technology, June 2002. From ganhawk1 at yahoo.com Fri Oct 3 05:29:53 2003 From: ganhawk1 at yahoo.com (GanHawk) Date: Sat Dec 9 22:12:22 2006 Subject: [p2p-hackers] P2P caching In-Reply-To: <3F7C7A2A.7000500@neurogrid.com> Message-ID: <20031003052953.39847.qmail@web20504.mail.yahoo.com> Hi all, I am new to this group and my greetings to everyone. I have been trying to solve the problem of Slashdot effect (Internet flash crowds) using P2P. I have also started a project called p2pbridge (http://p2pbridge.sourceforge.net/) caching system (more details on the web site). Does someone know of a similar system that can be used for caching web pages ? Right now its more of preference based, i.e users will cache only if they want to for other users. But I want to address the problem of leeching. I would also like to know if there are any systems that enforce a system of security tokens. Like a token will be given to a user and by accepting it he agrees to serve another user if the need arises. Thanks, Ganesh __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From gojomo at bitzi.com Fri Oct 3 06:35:20 2003 From: gojomo at bitzi.com (Gordon Mohr) Date: Sat Dec 9 22:12:22 2006 Subject: [p2p-hackers] P2P caching In-Reply-To: <20031003052953.39847.qmail@web20504.mail.yahoo.com> References: <20031003052953.39847.qmail@web20504.mail.yahoo.com> Message-ID: <3F7D18A8.5090300@bitzi.com> GanHawk wrote: > I am new to this group and my greetings to everyone. I > have been trying to solve the problem of Slashdot > effect (Internet flash crowds) using P2P. I have also > started a project called p2pbridge > (http://p2pbridge.sourceforge.net/) caching system > (more details on the web site). Does someone know of a > similar system that can be used for caching web pages > ? Peer-to-peer Caching Schemes to Address Flash Crowds (200) http://citeseer.nj.nec.com/stading02peertopeer.html This system, "Backslash", is based on a network of cooperating servers, rather than clients. A Lightweight, Robust P2P System to Handle Flash Crowds (2002) http://citeseer.nj.nec.com/stavrou02lightweight.html This system, "PROOFS", uses a dynamic network of cooperating clients. - Gordon From decapita at dti.unimi.it Sun Oct 5 15:53:42 2003 From: decapita at dti.unimi.it (Sabrina De Capitani di Vimercati) Date: Sat Dec 9 22:12:22 2006 Subject: [p2p-hackers] REMINDER: Call for Partecipation - ESORICS'03 Message-ID: [Apologies if you receive multiple copies of this message] CALL FOR PARTECIPATION 8TH EUROPEAN SYMPOSIUM ON RESEARCH IN COMPUTER SECURITY Gj?vik, Norway - October 13-15, 2003 Organized by Gj?vik University College Held in conjunction with Nordsec 2003 http://www.hig.no/esorics2003/ -------------------------------------------------------------------- AIMS AND SCOPE Organised in a series of European countries, ESORICS is confirmed as the European research event in computer security. The symposium started in 1990 and is now held every year in different European countries and attracts audience from both the academic and industrial communities. The Symposium has established itself as one of the premiere, international gatherings on Information Assurance. CONFERENCE VENUE Gj?vik is called "The White Town at Mj?sa" because of all the white houses up the hill from the lake. As you may understand, it's quite a small town. The number of inhabitants is 6000. Gj?vik town is the centre of a municipality, also called Gj?vik, and the number of inhabitants in the municipality is 26000. Gj?vik is lying on the shores of Mj?sa, the largest inland lake in Norway. Mj?sa is about 100 kilometres long, from south to north. On the north end, we find Lillehammer, where the 17th Winter Olympic Games took place in February 1994. We find Gj?vik midway between south and north end of the lake. FOR MORE INFORMATION For details: http://www.hig.no/esorics2003 On-site arrangements: esorics2003info@hig.no General program information: esorics2003prog@hig.no From decapita at dti.unimi.it Sun Oct 5 20:17:06 2003 From: decapita at dti.unimi.it (Sabrina De Capitani di Vimercati) Date: Sat Dec 9 22:12:22 2006 Subject: [p2p-hackers] IICIS'03 - Call for Partecipation Message-ID: [Apologies if you receive multiple copies of this message] CALL FOR PARTECIPATION SIXTH IFIP TC-11 WG 11.5 WORKING CONFERENCE ON INTEGRITY AND INTERNAL CONTROL IN INFORMATION SYSTEMS Lausanne, Switzerland - November 13-14, 2003 http://lbd.epfl.ch/e/conferences/IICIS03/ ----------------------------------------------------------------- AIMS AND SCOPE Confidentiality, integrity and availability are high-level objectives of IT security. The IFIP TC-11 Working Group 11.5 is in charge, within IFIP, with exploring the area of the integrity objective within IT security and the relationship between integrity in information systems and the overall internal control systems that are established in organizations to support corporate governance codes. This working conference series follows its tradition to establish the basis for an ongoing dialog between IT security specialists and internal control specialists so that both may work more effectively together to assist in creating effective business systems in the future. The goals for this conference are to find an answer to the following questions: * what is the status of research and development in the area of integrity and internal control? * where are the gaps between business needs on the one hand and research and development on the other and what needs to be done to bridge these gaps? * what precisely do business managers need to have confidence in the integrity of their information systems and their data? CONFERENCE VENUE Lausanne is an extremely pleasant location on the Lake Leman (also known as Lake of Geneva) at an altitude of app. 300 meters. Lausanne offers magnificent views over the Alps and the Lake. Lausanne offers plenty of cultural opportunities, in particular music and dance (Ballet Bejart). The Olympic Museum is the most visited spot. Amateurs of fine arts will have a unique chance to see extraordinary art by non-professional artists at the Collection de l'art brut. Excellent temporary art exhibitions are held at the Fondation L'Hermitage. Not far away is the Fondation Gianadda, in Martigny, which also houses permanent collections of Roman antiques, vintage cars and modern sculpture, and many other museums in Geneva. Montreux and the world-famous Chateau de Chillon are within short driving distance. PROGRAM Thursday November 13 -------------------- 08.45 - 09.00: OPENING REMARKS 09.00 - 10.00: KEYNOTE 1 - "From Security Culture to Effective E-Security Solutions" S. Ghernaouti-Helie 10.00 - 10.30: COFFEE BREAK 10.30 - 12.00: SESSION 1 - "Remote Integrity Checking" Y. Deswarte, J.J. Quisquater, A. Saidane - "Automated Checking of SAP Security Permissions" S. Hoehn, J. Jurjens - "A Formal Analysis of Digital Signature Architecture" D. Basin 12.00 - 13.30: LUNCH 13.30 - 15.00: SESSION 2 - "Using Parameterized UML to Specify and Compose Access Control Models" I. Ray, N. Li, D. Kim, R. France - "Enforcing Integrity in Multimedia Surveillance" N.B. Kodali, C. Farkas, D. Wijesekera - "A Learning-based Approach to Information Release Control" C. Bettini, S. Wang, S. Jajodia 15.00 - 15.30: COFFEE BREAK 15.30 - 17.00: PANEL/SESSION: Standards / Evaluation / Certification Moderator: L. Strous - "Information Security Governance using ISO 17799 and Cobit" E. Pretorius, B. von Solms - "The Role of System Certification in Meeting an Prganisation's Corporate Objective" (invited paper) W. List - Panel discussion Friday November 14 ------------------ 09.00 - 10.00: KEYNOTE 2 - "Consistent Query Answering: Recent Developments and Future Directions" J. Chomicki 10.00 - 10.30: COFFEE BREAK 10.30 - 12.00: SESSION 3 - "Tracing Attacks and Restoring Integrity with LASCAR" A. Aellig, P. Oechslin - "A Secure Multi-sited Version Control System for Distributed Collaborative Efforts" J. Zhang, I. Ray - "Integration of Integrity Constraints in Database Federations" H. Balsters, E. O. De Brock: 12.00 - 13.30: LUNCH 13.30 - 15.00: PANEL Moderator: B. Thuraisingham 15.00 - 15.30: COFFEE BREAK 15.30 - 17.00: SESSION 4 - "Reducing Disruption in Time-Tabled Condition Monitoring" B. Jin, S.M. Embury - "A Service Oriented System Based Information Flow Model For Damage Assessment" Y. Zuo, B. Panda - "An efficient OODB Model for Ensuring the Integrity of User-Defined Constraints" B. Zaqaibeh, H. Ibrahim, A. Mamat, Md. N. Sulaiman MORE INFORMATION Up-to-date information can be obtained from the program co-chairs or via the conference web site at: http://lbd.epfl.ch/e/conferences/IICIS03/index.html (also via the link on the TC-11 homepage http://www.ifip.tu-graz.ac.at/TC11) From Wolfgang.Mueller2 at uni-bayreuth.de Mon Oct 6 06:18:53 2003 From: Wolfgang.Mueller2 at uni-bayreuth.de (Wolfgang =?iso-8859-1?q?M=FCller?=) Date: Sat Dec 9 22:12:22 2006 Subject: [p2p-hackers] Partial Match Problem in DHT(Kademlia). In-Reply-To: <022b01c388e5$af409740$66bba8c0@MIKEI> References: <022b01c388e5$af409740$66bba8c0@MIKEI> Message-ID: <200310060818.53854.wolfgang.mueller2@uni-bayreuth.de> Hi, pSearch [1] is another useful link: They create distributed inverted files on top of an extended CAN (the system is called pVSM) and they use Latent Semantic Indexing on top of a CAN (pLSI). [2] looks into the feasibility of keyword search with P2P networks. Cheers, Wolfgang [1] C. Tang, Z. Xu, and M. Mahalingam. psearch: Information retrieval in structured overlays. In HotNets-I, October 2002. http://citeseer.nj.nec.com/context/2108602/545930 [2] On the Feasibility of Peer-to-Peer Web Indexing and Search. Jinyang Li Boon Thau Loo, Joseph M. Hellerstein, M. Frans Kaashoek, David R. Karger, Robert Morris. 2nd IPTPS, Berkeley, 2003 http://citeseer.nj.nec.com/560565.html From jdl at vinecorp.com Mon Oct 6 21:56:53 2003 From: jdl at vinecorp.com (jdl@vinecorp.com) Date: Sat Dec 9 22:12:22 2006 Subject: [p2p-hackers] [Silicon Valley] P2Punks meeting Tuesday October 14 -- 7:30 pm onwards Message-ID: <20031006175653.D14166@lynx.phpwebhosting.com> I've changed the day back to the second Tuesday of the month. See you there.. Where: Dana Street Roasting Company 744 Dana St., Mountain View Phone: (650) 390-9638 1/2 block off Castro St. When: 7:30pm onward Website: http://www.bitbin.org/p2punks From pfh at mail.csse.monash.edu.au Tue Oct 7 02:53:03 2003 From: pfh at mail.csse.monash.edu.au (Paul Harrison) Date: Sat Dec 9 22:12:22 2006 Subject: [p2p-hackers] Partial Match Problem in DHT(Kademlia). Message-ID: On Thu, 2 Oct 2003, izzy wrote: > As I know, partial match is a key disadvantage in DHT-based search > algorithm(Chord,CAN,Pastry) . > For the time being , I haven't seen any effective solution out there. > Overnet and Emule Kademlia adopt Kademlia Algorithm to achieve fully > decentralized self-organization. At the risk of repeating what someone else has already referenced... the Circle[1] approach is to break the file name and ID3 tags down into words, and publish each word. So "Guns 'n Roses.mp3" would get broken into "guns", "n", "roses", "mp3". You can then search for a set of words -- the way this is done is simply to look up the least common word (using the heuristic that longer words tend to be less common) then filter the resulting list through the remaining words. This isn't full partial matching, but is what you want most of the time. I think the best solution might just be to have a *lot* of (persistant) data replication. Hard disks are cheap :-), someone who downloads the occasional movie won't really mind a 10 gigabyte index file or two on their hard disk. Just set things up so that for every 100Mb downloaded, you also replicate, say, 1Mb of index data. regards, Paul [1] a Chord implementation Email: pfh@logarithmic.net Current cost to save one life: approx AU$300 (US$200) From clint at TheStaticVoid.net Wed Oct 8 08:34:22 2003 From: clint at TheStaticVoid.net (Clint Heyer) Date: Sat Dec 9 22:12:22 2006 Subject: [p2p-hackers] Partial Match Problem in DHT(Kademlia) In-Reply-To: <20031007190007.A140A3FD03@capsicum.zgp.org> Message-ID: <2003108103422.262594@platypus> Naanou takes a similar approach to Circle (Naanou is also based on Chord). It will either index on filename or available metadata (but not both). For a filename, it will split it up into word tokens, and throw away stop words[1]. Each word is indexed separately with the remaining words being stored alongside the token as contextual data. The context keys are then used to assist in relevancy calculation by the keystorer. A similar process occurs for files with metadata (AVI, MP3s, JPEGs and so forth), but each keyword is prefixed by the metadata type, (eg: 'artist:radiohead'). Filename data could also be additionally published with metadata keys to improve search quality. Prefixing with the metdata type allows searches to take place on specific fields (e.g., I only want songs by the artist 'blur', not just any filename with 'blur' in it). I was also thinking of using Porter stemming on words before they are published to increase their generality, but didn't have time to test. Perhaps a Soundex type algorithm could also be useful in this regard. cheers, .clint [1] If throwing away stop words results in the deletion if 70% of the index string/query, stop words are preserved. On Tue, 7 Oct 2003 12:00:07 -0700 (PDT), p2p-hackers-request@zgp.org wrote: > From: Paul Harrison > Subject: Re: [p2p-hackers] Partial Match Problem in DHT(Kademlia). > To: > Message-ID: > Content-Type: TEXT/PLAIN; charset=US-ASCII > > On Thu, 2 Oct 2003, izzy wrote: > > >> As I know, partial match is a key disadvantage in DHT-based >> search algorithm(Chord,CAN,Pastry) . >> For the time being , I haven't seen any effective solution out >> there. Overnet and Emule Kademlia adopt Kademlia Algorithm to >> achieve fully decentralized self-organization. >> > > At the risk of repeating what someone else has already > referenced... the Circle[1] approach is to break the file name and > ID3 tags down into words, and publish each word. So "Guns 'n > Roses.mp3" would get broken into "guns", "n", "roses", "mp3". You > can then search for a set of words -- the way this is done is > simply to look up the least common word (using the heuristic that > longer words tend to be less common) then filter the resulting list > through the remaining words. > > This isn't full partial matching, but is what you want most of the > time. > > > I think the best solution might just be to have a *lot* of > (persistant) data replication. Hard disks are cheap :-), someone > who downloads the occasional movie won't really mind a 10 gigabyte > index file or two on their hard disk. Just set things up so that > for every 100Mb downloaded, you also replicate, say, 1Mb of index > data. > > regards, > Paul > > > [1] a Chord implementation > > > Email: pfh@logarithmic.net > > > Current cost to save one life: approx AU$300 (US$200) ______________________________________ www: http://www.TheStaticVoid.net From anwitaman at hotmail.com Thu Oct 9 08:37:14 2003 From: anwitaman at hotmail.com (Anwitaman Datta) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] Partial Match Problem in DHT Message-ID: Partial matching is an interesting problem for DHTs! I have been thinking of what I call "pragmatic indexing". Initially nothing is indexed, and one has a gnutella (flooding network). Whenever something is queried, for example, Artist:Jackson or MovieName:Terminator ... we store it in the DHT, where the DHT assigns first "n" bits for the meta-information (Artist, SongTitle, MovieName), which is determined from a lookup table, while the appended hash bits is generated using standard hashing function. Also, any item in the index is purged after a timeout, and if it is queried again, then the timeout is reset. (why index what is not being searched in any case?) This strategy is also good for hot data .. we need to increase replicas based on the query frequency. In such a strategy, search is done in two steps - 1) search in the index. If found, reset time to live. Else, 2) gnutella/unstructured search. Insert in index along with TTL. There are ofcourse load-balancing and other issues (which too I am working on, in the context of P-Grid (www.p-grid.org)), but this mechanism enjoys the best of both worlds. Best, Anwitaman http://lsirpeople.epfl.ch/adatta ---------------------------------------------------------------------- Message: 1 Date: Wed, 8 Oct 2003 10:34:22 +0200 From: Clint Heyer Subject: Re: [p2p-hackers] Partial Match Problem in DHT(Kademlia) To: Message-ID: <2003108103422.262594@platypus> Content-Type: text/plain; charset="us-ascii" Naanou takes a similar approach to Circle (Naanou is also based on Chord). It will either index on filename or available metadata (but not both). For a filename, it will split it up into word tokens, and throw away stop words[1]. Each word is indexed separately with the remaining words being stored alongside the token as contextual data. The context keys are then used to assist in relevancy calculation by the keystorer. A similar process occurs for files with metadata (AVI, MP3s, JPEGs and so forth), but each keyword is prefixed by the metadata type, (eg: 'artist:radiohead'). Filename data could also be additionally published with metadata keys to improve search quality. Prefixing with the metdata type allows searches to take place on specific fields (e.g., I only want songs by the artist 'blur', not just any filename with 'blur' in it). I was also thinking of using Porter stemming on words before they are published to increase their generality, but didn't have time to test. Perhaps a Soundex type algorithm could also be useful in this regard. cheers, .clint [1] If throwing away stop words results in the deletion if 70% of the index string/query, stop words are preserved. On Tue, 7 Oct 2003 12:00:07 -0700 (PDT), p2p-hackers-request@zgp.org wrote: > From: Paul Harrison > Subject: Re: [p2p-hackers] Partial Match Problem in DHT(Kademlia). > To: > Message-ID: > Content-Type: TEXT/PLAIN; charset=US-ASCII > > On Thu, 2 Oct 2003, izzy wrote: > > >> As I know, partial match is a key disadvantage in DHT-based >> search algorithm(Chord,CAN,Pastry) . >> For the time being , I haven't seen any effective solution out >> there. Overnet and Emule Kademlia adopt Kademlia Algorithm to >> achieve fully decentralized self-organization. >> > > At the risk of repeating what someone else has already > referenced... the Circle[1] approach is to break the file name and > ID3 tags down into words, and publish each word. So "Guns 'n > Roses.mp3" would get broken into "guns", "n", "roses", "mp3". You > can then search for a set of words -- the way this is done is > simply to look up the least common word (using the heuristic that > longer words tend to be less common) then filter the resulting list > through the remaining words. > > This isn't full partial matching, but is what you want most of the > time. > > > I think the best solution might just be to have a *lot* of > (persistant) data replication. Hard disks are cheap :-), someone > who downloads the occasional movie won't really mind a 10 gigabyte > index file or two on their hard disk. Just set things up so that > for every 100Mb downloaded, you also replicate, say, 1Mb of index > data. > > regards, > Paul > > > [1] a Chord implementation > > > Email: pfh@logarithmic.net > > > Current cost to save one life: approx AU$300 (US$200) ______________________________________ www: http://www.TheStaticVoid.net ------------------------------ _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers End of p2p-hackers Digest, Vol 3, Issue 6 ***************************************** _________________________________________________________________ Contact brides & grooms FREE! http://www.shaadi.com/ptnr.php?ptnr=hmltag Only on www.shaadi.com. Register now! From izzy at lina.es.ncku.edu.tw Thu Oct 9 17:52:08 2003 From: izzy at lina.es.ncku.edu.tw (izzy) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] Simulation Question. Message-ID: <000f01c38e8e$118401f0$7327748c@izzy> Recently, I have to propose a simulation plan for my P2P lookup protocol. So far, the simulation plans I have surveyed all have simulators respectively. Some are written in JXTA, C..,etc. I am wondering why not using NS or NS-2 ? Is there any scaling problem on NS or NS-2 ? or other reasons ? Cheers ^^ Ian -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20031010/55b153c3/attachment.htm From adam.lydick at verizon.net Fri Oct 10 15:52:49 2003 From: adam.lydick at verizon.net (Adam Lydick) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] Partial Match Problem in DHT(Kademlia) In-Reply-To: <2003108103422.262594@platypus> References: <2003108103422.262594@platypus> Message-ID: <1065801169.1077.13.camel@elessar> I was pondering something similar a while back, but gave up on it because both porter and soundex are english language specific. :( (soundex might have less problems in that regard, but my secondhand info is that it shouldn't be used for other languages -- anyone know what modern multilingual word processors are doing?) Adam Lydick On Wed, 2003-10-08 at 01:34, Clint Heyer wrote: > with 'blur' in it). I was also thinking of using Porter stemming on > words before they are published to increase their generality, but > didn't have time to test. Perhaps a Soundex type algorithm could also > be useful in this regard. > > cheers, > .clint From decapita at dti.unimi.it Fri Oct 10 17:24:38 2003 From: decapita at dti.unimi.it (Sabrina De Capitani di Vimercati) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] Call for Partecipation: 2nd IEEE International Security in Storage Workshop Message-ID: [Apologies if you receive multiple copies of this message] CALL FOR PARTECIPATION 2ND IEEE INTERNATIONAL SECURITY IN STORAGE WORKSHOP Washington, DC, USA - October 31, 2003 Sponsored by the IEEE Computer Society Task Force on Information Assurance http://ieeeia.org/sisw2003/ ------------------------------------------------------------------------ AIMS AND SCOPE The ability to create large shared storage systems in a secure manner is an area that has received little formal research or results. A comprehensive, systems approach to storage security is required if storage consolidation is to succeed. This workshop serves as an open forum to discuss storage threats, technologies, methodologies and deployment. PROGRAM Friday October 31, 2003 ----------------------- 7:00-7:55 Registration and Continental Breakfast 7:55-8:00 Welcome 8:00-8:45 Decentralized Access Control Mechanism for Object-Based Storage Devices Vishal Kher and Yongdae Kim University of Minnesota 8:45-9:30 Fast and Secure Append-Only Storage with Infinite Capacity Yongge Wang and Yuliang Zheng University of North Carolina at Charlotte 9:30-10:00 Break 10:00-10:45 Key Management for Encrypted Data Storage in Distributed Systems Ludwig Seitz, Jean-Marc Pierson, and Lionel Brunie LIRIS, INSA de Lyon 10:45-11:30 Information Security on the Logistical Network: An End-to-End Approach Micah Beck, James S. Plank, Jeremy Millar, Scott Atchley, Stephen Soltesz, Alessandro Bassi, and Huadong Liu University of Tennessee 11:30- 1:00 Lunch (provided) 1:00-1:45 Implementation of a Storage Mechanism for Untrusted DBMSs Ernesto Damiani, Sabrina De Capitani di Vimercati, Stefano Paraboschi, and Pierangela Samarati Universita`?di Milano and Universita` di Bergamo 1:45-2:30 Cryptographic File Systems Performance: What You Don't Know Can Hurt You Charles P. Wright, Jay Dave, and Erez Zadok Stony Brook University 2:30-3:00 Break 3:00-3:45 Memories: a Survey of their Secure uses in Smart Cards Michael Neve, Eric Peeters, David Samyde and Jean-Jacques Quisquater Universit?? Catholique de Louvain 3:45-4:30 The Object Store Security Protocol Michael Factor, Dalit Naor, Noam Rinetzky, Julian Satran (IBM), David Nagle (Panasas), and Don Beaver, Erik Riedel (Seagate) 4:30-5:15 IEEE Standard for Encrypted Storage James Hughes Storage Technology Corporation MORE INFORMATION For further information, contact: James Hughes (James_Hughes@StorageTek.com) From thiago at vetorial.net Fri Oct 10 23:12:59 2003 From: thiago at vetorial.net (Thiago Pinto Damas) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 Message-ID: <3F873CFB.5020406@vetorial.net> Hi, anyone already tried to implement a p2p network in the ns-2 network simulator? Something simple like a flooding algorithm. Any more ideas about other simulators? thanks Thiago Damas From jdl at vinecorp.com Sat Oct 11 20:59:25 2003 From: jdl at vinecorp.com (jdl@vinecorp.com) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] [Silicon Valley] reminder: P2Punks meeting Tuesday October 14 -- 7:30 pm Message-ID: <20031011165925.E28433@lynx.phpwebhosting.com> Your weekend reminder... ----- I've changed the day back to the second Tuesday of the month. See you there.. Where: Dana Street Roasting Company 744 Dana St., Mountain View Phone: (650) 390-9638 1/2 block off Castro St. When: 7:30pm onward From eugen at leitl.org Mon Oct 13 10:34:29 2003 From: eugen at leitl.org (Eugen Leitl) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] [NEC] 2.10: File-sharing Goes Social (fwd from nec-admin@shirky.com) Message-ID: <20031013103428.GE7916@leitl.org> ----- Forwarded message from nec-admin@shirky.com ----- From: nec-admin@shirky.com Date: Sun, 12 Oct 2003 21:52:37 -0400 To: nec@shirky.com Subject: [NEC] 2.10: File-sharing Goes Social Reply-To: list-replies@shirky.com NEC @ Shirky.com, a mailing list about Networks, Economics, and Culture Published periodically / #2.10 / October 12, 2003 Subscribe at http://shirky.com/nec.html Archived at http://shirky.com Social Software weblog at http://corante.com/many/ In this issue: - Introduction - Essay: File-sharing Goes Social Also at http://www.shirky.com/writings/file-sharing_social.html - Worth Reading: Towsend on telecom and cities * Introduction ======================================================= This month's essay concerns the way the RIAA is creating envrionmental pressures that alter the design of file sharing networks, and how the current attacks on Kazaa et al are moving file sharing into socially bounded cells. -clay * Essay ============================================================== File-sharing Goes Social http://www.shirky.com/writings/file-sharing_social.html The RIAA has taken us on a tour of networking strategies in the last few years, by constantly changing the environment file-sharing systems operate in. In hostile environments, organisms often adapt to become less energetic but harder to kill, and so it is now. With the RIAA's waves of legal attacks driving experimentation with decentralized file-sharing tools, file-sharing networks have progressively traded efficiency for resistance to legal attack. The RIAA has slowly altered the environment so that relatively efficient systems like Napster were killed, opening up a niche for more decentralized systems like Gnutella and Kazaa. With their current campaign against Kazaa in full swing, we are about to see another shift in network design, one that will have file sharers adopting tools originally designed for secure collaboration in a corporate setting. Napster's problem, of course, was that although Napster nodes acted as both client and server, the central database still gave the RIAA a single target. Seeing this, Gnutella and Kazaa shifted to a mesh of nodes that could each act as client, server, and router. These networks are self-assembling and self-reconfiguring with a minimum of bootstrapping, and decentralize even addresses and pointers to files. The RIAA is now attacking these networks using a strategy that could be called Crush the Connectors. A number of recent books on networks, such as Gladwell's _The Tipping Point_ [isbn.nu/0316316962], Barabasi's _Linked_ [isbn.nu/0738206679], and Watts' _Six Degrees_ [isbn.nu/0393041425], have noted that large, loosely connected networks derive their effectiveness from a small number of highly connected nodes, a pattern called a Small World network. As a result, random attacks, even massive ones, typically leave the network only modestly damaged. The flipside is that attacks that specifically target the most connected nodes are disproportionately effective. The RIAA's Crush the Connectors strategy will work, not simply because highly publicized legal action will deter some users, but because the value of the system will decay badly if the RIAA succeeds in removing even a small number of the best-provisioned nodes. However, it will not work as well as the RIAA wants, even ignoring the public relations fallout, for two reasons. The first is that combining client, server, and router in one piece of software is not the last move available to network designers -- there is still the firewall. And the second is simply the math of popular music -- there are more people than songs. - Networks, Horizons, and Membranes Napster was the last file-sharing system that was boundary-less by design. There was, at least in theory, one Napster universe at any given moment, and it was globally searchable. Gnutella, Kazaa, and other similar systems set out to decentralize even the address and search functions. This made these systems more robust in the face of legal challenges, but added an internal limit -- the search horizon. Since such systems have no central database, they relay requests through the system from one node to the next. However, the "Ask two friends to ask two friends ad infinitum" search method can swamp the system. As a result, these systems usually limit the spread of search requests, creating an internal horizon. The tradeoff here is between the value of any given search (deeper searches are more effective) vs the load on the system as a whole (shallower searches reduce communications overhead.) In a world where the RIAA's attack mode was to go after central resources, this tradeoff worked well -- efficient enough, and resistant to Napster-style lawsuits. However, these systems are themselves vulnerable in two ways -- first, anything that reduces the number of songs inside any given user's search horizon reduces the value of the system, causing some users to defect, which weakens the system still further. Second, because search horizons are only perceptual borders, the activity of the whole network can be observed by a determined attacker running multiple nodes as observation points. The RIAA is relying on both weaknesses in its current attack. By working to remove those users who make a large number of files persistently available, the RIAA can limit the amount of accessible music and the trust the average user has in the system. Many of the early reports on the Crush the Connectors strategy suggest that users are not just angry with the RIAA, but with Kazaa as well, for failing to protect them. The very fact that Crush the Connectors is an attack on trustworthiness, however, points to one obvious reaction: move from a system with search horizons to one with real membranes, and making those membranes social as well as technological. - Trust as a Border There are several activities that are both illegal and popular, and these suffer from what economists call high transaction costs. Buying marijuana involves considerably more work than buying roses, in part because every transaction involves risk for both parties, and in part because neither party can rely on the courts for redress from unfair transactions. As a result, the market for marijuana today (or NYC tattoo artists in the 1980s, or gin in the 1920s, etc) involves trusted intermediaries who broker introductions. These intermediaries act as a kind of social Visa system; in the same way a credit card issuer has a relationship with both buyer and seller, and an incentive to see that transactions go well, an introducer in an illegal transaction has an incentive to make sure that neither side defects from the transaction. And all parties, of course, have an incentive to avoid detection. This is a different kind of border than a search horizon. Instead of being able to search for resources a certain topological distance from you, you search for resources a certain social distance from you. (This is also the guiding principle behind services like LinkedIn and Friendster, though in practice they represent their user's networks as being much larger than real-world social boundaries are.) Such a system would add a firewall of sorts to the client, server, and router functions of existing systems, and that firewall would serve two separate but related needs. It would make the shared space inaccessible to new users without some sort of invitation from existing users, and it would likewise make all activity inside the space unobservable to the outside world. Though the press is calling such systems "darknets" and intimating that they are the work of some sort of internet underground, those two requirements -- controlled membership and encrypted file transfer -- actually describe business needs better than consumer needs. There are many ways to move to such membrane-bounded systems, of course, including retrofitting existing networks to allow sub-groups with controlled membership (possibly using email white-list or IM buddy-list tools); adopting any of the current peer-to-peer tools designed for secure collaboration (e.g. Groove [groove.com], Shinkuro [shinkuro.com], WASTE [jabberwocky.de/waste/], etc); or even going to physical distribution. As Andrew Odlyzko has pointed out, sending disks through the mail can move enough bits in a 24 hour period to qualify as broadband [www.firstmonday.dk/issues/issue8_9/odlyzko/#o6], and there are now file-sharing networks whose members simply snail mail one another mountable drives of music. A critical factor here is the social fabric -- as designers of secure networks know, protecting the perimeter of a network only works if the people inside the perimeter are trustworthy. New entrants can only be let into such a system if they are somehow vetted or vouched for, and the existing members must have something at stake in the behavior of the new arrivals. The disadvantage of social sharing is simple -- limited membership means fewer files. The advantage is equally simple -- a socially bounded system is better than nothing, and safer than Kazaa. If Kazaa, Gnutella and others are severely damaged by the Crush the Connectors attack, users will either give up free file-sharing, or switch to less efficient social spaces. This might seem like an unalloyed win for the RIAA, but for one inconvenient fact: there are more people than are songs. - There Are More People Than Songs For the sake of round numbers, assume there are 500 million people using the internet today, and that much of the world's demand for popular music would be satisfied by the availability of something like 5 million individual songs (Apple's iTunes, by way of comparison, is a twentieth of that size.) Because people outnumber songs, if every user had one MP3 each, there would be a average of a hundred copies of every song somewhere online. A more realistic accounting would assume that at least 10% of the online population had at least 10 MP3 files each, numbers that are both underestimates, given the popularity of both ripping and sharing music. Worse for the RIAA, the popularity of songs is wildly unequal. Some songs -- The Real Slim Shady, Come Away With Me -- exist on millions of hard drives around the world. As we've moved from more efficient systems like Napster to less efficient ones like Kazaa, it has become considerably harder to find bluegrass, folk, or madrigals, but not that much harder to find songs by Britney, 50 Cent, or John Mayer. And as with the shift from Napster to Kazaa, the shift from Kazaa to socially-bounded systems will have the least significant effect on the most popular music. The worst news of all, though, is that songs are not randomly distributed. Instead, user clusters are a good predictor of shared taste. Make two lists, one of your favorite people and another of your favorite songs. What percentage of those songs could you copy from those people? Both of those lists are probably in the dozens at most, and if music were randomly distributed, getting even a few of your favorite songs from your nearest and dearest would be a rare occurrence. As it is, though, you could probably get a significant percentage of your favorite songs from your favorite people. Systems that rely on small groups of users known to one another, trading files among themselves, will be less efficient than Kazaa or Napster, but far more efficient than a random distribution of music would suggest. - What Happens Next? Small amounts of social file-sharing, by sending files as email attachments or uploading them to personal web servers, have always co-existed with the purpose-built file-sharing networks, but the two patterns may fuse as a result of the Crush the Connectors strategy. If that transition happens on a large scale, what might the future look like? Most file-sharing would go on in groups from a half dozen to a few dozen -- small enough that every member can know every other member by reputation. Most file-sharing would take place in the sorts of encrypted workspaces designed for business but adapted for this sort of social activity. Some users would be members of more than one space, thus linking several cells of users. The system would be far less densely interconnected than Kazaa or Gnutella are today, but would be more tightly connected than a simple set of social cells operating in isolation. It's not clear whether this would be good news or bad news for the RIAA. There are obviously several reasons to think it might be bad news: file-sharing would take place in spaces that would be much harder to inspect or penetrate; the lowered efficiency would also mean fewer high-yield targets for legal action; and the use of tools by groups that knew one another might make prosecution more difficult, because copyright law has often indemnified some types of non-commercial sharing among friends (e.g. the Audio Home Recording Act of 1992). There is also good news that could come from such social sharing systems, however. Reduced efficiency might send many users into online stores, and users seeking the hot new song might be willing to buy them online rather than wait for the files to arrive through social diffusion, which would effectively turn at least some of these groups into buyers clubs. The RIAA's reaction to such social sharing will be unpredictable. They have little incentive to seek solutions that don't try to make digital files behave like physical objects. They may therefore reason that they have little to lose by attacking social sharing systems with a vengeance. Whatever their reaction, however, it is clear that the current environment favors the development and adoption of social and collaborative tools, which will go on to have effects well outside the domain of file-sharing, because once a tool is adopted for one purpose, it often takes on a life of its own, as its users press such social toosl to new uses. -=- * Worth Reading ======================================================= Townsend on telecommunications and cities Anthony Townsend, (un)wirer of NYC parks, has published his doctoral thesis on the relationship between urban areas and telecom. It's interesting reading for anyone who cares about telecommunications regulation, or is interested in the effect of WiFi on cities. Townsend covers a lot of background, including the delightful insight that telephones made skyscrapers possible, because physical messaging between floors would never have scaled. He also notes that though telecom regulation is typically taken at the state and national level, it is really the municipalities where the critical choices take place. The background is mostly there to set the stage for Chapter 6, which outlines the potential of Wifi to alter urban space yet again, this time by further speeding up the "urban metabolism": In cities, it was also clear that the urban environment generated an enormous amount of information that needed to be anticipated, reacted to, and incorporated into everyday decision-making. Information about constantly changing traffic, weather, and economic conditions could be better transmitted through mobile phones and other wireless digital media. Traditionally, cities had functioned on a daily cycle of information flow with mass media like newspapers, third spaces like bars and cafes, and family conversations at the dinner table as the primary means of information exchange. With ubiquitous untethered communications, however, this old cycle was dramatically speeded up. As the information cycle sped up, there was a corresponding increase in the rate of urban metabolism -- the pace at which urban economic and social life consumed information and materiel. In effect, instead of the synchronous daily rhythm of the industrial city coordinated by standardized time, untethered communications were leading to a city coordinated on the fly in real-time. Worth a read. http://www.mit.edu/~amt/ * End ==================================================================== This work is licensed under the Creative Commons Attribution License. The licensor permits others to copy, distribute, display, and perform the work. In return, licensees must give the original author credit. To view a copy of this license, visit http://creativecommons.org/licenses/by/1.0 or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA. 2003, Clay Shirky _______________________________________________ NEC - Clay Shirky's distribution list on Networks, Economics & Culture NEC@shirky.com http://shirky.com/nec.html ----- End forwarded message ----- -- Eugen* Leitl leitl ______________________________________________________________ ICBM: 48.07078, 11.61144 http://www.leitl.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available Url : http://zgp.org/pipermail/p2p-hackers/attachments/20031013/034f8647/attachment.pgp From sam at neurogrid.com Mon Oct 13 20:17:02 2003 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 In-Reply-To: <3F873CFB.5020406@vetorial.net> References: <3F873CFB.5020406@vetorial.net> Message-ID: <3F8B083E.50200@neurogrid.com> Hi Ian and Thiago, Having looked at NS-2, but not used it, I came to the conclusion that it would be difficult to use it for large scale p2p simulations due to the fact that it simulates alot of low level network protocols in some details. If you are wanting to simulate a p2p network that will have many nodes, and perhaps lots of users with a variety of resources, may I recommend the NeuroGrid simulator. Despite the name it is actually a generic p2p simulator and supports Gnutella, Freenet and NeuroGrid simulations, and is more scalable because it simulates nodes and connections in the most simple form possible. Over the summer the simulator was extended to support a p2p DNS simulation and a p2p email simulation that includes pastry routing. I think you'll find this simulator highly extendible to support your own projects. Check out: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerSimulation The documentation process is ongoing, but anybody can edit the wiki and this has helped us document the new extensions. Please feel free to mail me with any questions: sam@neurogrid.com CHEERS> SAM Thiago Pinto Damas wrote: > anyone already tried to implement a p2p network in the ns-2 > network simulator? Something simple like a flooding algorithm. > Any more ideas about other simulators? Ian wrote: Recently, I have to propose a simulation plan for my P2P lookup protocol. So far, the simulation plans I have surveyed all have simulators respectively. Some are written in JXTA, C..,etc. I am wondering why not using NS or NS-2 ? Is there any scaling problem on NS or NS-2 ? or other reasons ? From thiago at vetorial.net Tue Oct 14 00:01:15 2003 From: thiago at vetorial.net (Thiago Pinto Damas) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 In-Reply-To: <3F8B083E.50200@neurogrid.com> References: <3F873CFB.5020406@vetorial.net> <3F8B083E.50200@neurogrid.com> Message-ID: <3F8B3CCB.4060408@vetorial.net> Hi, does the Neurogrid simulate latencies, losses, etc? Does the neurogrid support distributed simulations? Is it needed? Cheers Thiago Sam Joseph wrote: > Hi Ian and Thiago, > > Having looked at NS-2, but not used it, I came to the conclusion that it > would be difficult to use it for large scale p2p simulations due to the > fact that it simulates alot of low level network protocols in some details. > > If you are wanting to simulate a p2p network that will have many nodes, > and perhaps lots of users with a variety of resources, may I recommend > the NeuroGrid simulator. Despite the name it is actually a generic p2p > simulator and supports Gnutella, Freenet and NeuroGrid simulations, and > is more scalable because it simulates nodes and connections in the most > simple form possible. > > Over the summer the simulator was extended to support a p2p DNS > simulation and a p2p email simulation that includes pastry routing. I > think you'll find this simulator highly extendible to support your own > projects. > > Check out: > > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerSimulation > > The documentation process is ongoing, but anybody can edit the wiki and > this has helped us document the new extensions. > Please feel free to mail me with any questions: sam@neurogrid.com > > CHEERS> SAM > > > Thiago Pinto Damas wrote: > >> anyone already tried to implement a p2p network in the ns-2 >> network simulator? Something simple like a flooding algorithm. >> Any more ideas about other simulators? > > > Ian wrote: > > Recently, I have to propose a simulation plan for my P2P lookup protocol. > So far, the simulation plans I have surveyed all have simulators > respectively. > Some are written in JXTA, C..,etc. > I am wondering why not using NS or NS-2 ? > Is there any scaling problem on NS or NS-2 ? > or other reasons ? > > > From sam at neurogrid.com Tue Oct 14 00:23:39 2003 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 In-Reply-To: <3F8B3CCB.4060408@vetorial.net> References: <3F873CFB.5020406@vetorial.net> <3F8B083E.50200@neurogrid.com> <3F8B3CCB.4060408@vetorial.net> Message-ID: <3F8B420B.6040708@neurogrid.com> Hi Thiago, Now that we have switched the simulator over to an event based framework, where all actions can be set as events that will take place at some arbitrary point in the future, it is possible to simulate connections with different latencies. Nodes can be added and removed allowing node losses to be simulated. At the moment there is no explicit support for distributed simulations, however we are working on trying to make things run on systems with multiple cpus, i.e. get proper parcelling out of processes to threads and so forth, but I think that is an area with potentially a lot more work required before we can say it is supported. In the first instance I think you can simulate quite large networks on a single computer - say a million nodes or so, but to go larger we need 64bit memories, and then ideally multiple processors - I have been running some sims on a SunFire 15k with 72 cpus, but I am not confident that I am properly using all the processors. Depending on your resources splitting the simulator to run on multiple machines might be a good way to go ... The simulator is open source and designed to be extendible - so please help us implement any additional features you'd like to see. CHEERS> SAM Thiago Pinto Damas wrote: > Hi, > does the Neurogrid simulate latencies, losses, etc? > Does the neurogrid support distributed simulations? Is it needed? > > Cheers > Thiago > Sam Joseph wrote: > >> Hi Ian and Thiago, >> >> Having looked at NS-2, but not used it, I came to the conclusion that >> it would be difficult to use it for large scale p2p simulations due >> to the fact that it simulates alot of low level network protocols in >> some details. >> >> If you are wanting to simulate a p2p network that will have many >> nodes, and perhaps lots of users with a variety of resources, may I >> recommend the NeuroGrid simulator. Despite the name it is actually a >> generic p2p simulator and supports Gnutella, Freenet and NeuroGrid >> simulations, and is more scalable because it simulates nodes and >> connections in the most simple form possible. >> >> Over the summer the simulator was extended to support a p2p DNS >> simulation and a p2p email simulation that includes pastry routing. >> I think you'll find this simulator highly extendible to support your >> own projects. >> >> Check out: >> >> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerSimulation >> >> The documentation process is ongoing, but anybody can edit the wiki >> and this has helped us document the new extensions. >> Please feel free to mail me with any questions: sam@neurogrid.com >> >> CHEERS> SAM >> >> >> Thiago Pinto Damas wrote: >> >>> anyone already tried to implement a p2p network in the ns-2 >>> network simulator? Something simple like a flooding algorithm. >>> Any more ideas about other simulators? >> >> >> >> Ian wrote: >> >> Recently, I have to propose a simulation plan for my P2P lookup >> protocol. >> So far, the simulation plans I have surveyed all have simulators >> respectively. >> Some are written in JXTA, C..,etc. >> I am wondering why not using NS or NS-2 ? >> Is there any scaling problem on NS or NS-2 ? >> or other reasons ? >> >> >> > > > From thiago at vetorial.net Tue Oct 14 03:00:51 2003 From: thiago at vetorial.net (Thiago Pinto Damas) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 In-Reply-To: <3F8B420B.6040708@neurogrid.com> References: <3F873CFB.5020406@vetorial.net> <3F8B083E.50200@neurogrid.com> <3F8B3CCB.4060408@vetorial.net> <3F8B420B.6040708@neurogrid.com> Message-ID: <3F8B66E3.9040306@vetorial.net> Hi Sam, it sounds really cool! My plan it's use a simulator to view the characteristics of a p2p system, specially the PSearch[1] algorithm. Now, I'll study the various simulators available for this type of network, to implement and test the algorithm later. Of course, i'd like to help to add new features in the neurogrid simulator! How can I do this? cheers Thiago Sam Joseph wrote: > Hi Thiago, > > Now that we have switched the simulator over to an event based > framework, where all actions can be set as events that will take place > at some arbitrary point in the future, it is possible to simulate > connections with different latencies. Nodes can be added and removed > allowing node losses to be simulated. > > At the moment there is no explicit support for distributed simulations, > however we are working on trying to make things run on systems with > multiple cpus, i.e. get proper parcelling out of processes to threads > and so forth, but I think that is an area with potentially a lot more > work required before we can say it is supported. In the first instance > I think you can simulate quite large networks on a single computer - say > a million nodes or so, but to go larger we need 64bit memories, and then > ideally multiple processors - I have been running some sims on a SunFire > 15k with 72 cpus, but I am not confident that I am properly using all > the processors. > > Depending on your resources splitting the simulator to run on multiple > machines might be a good way to go ... > > The simulator is open source and designed to be extendible - so please > help us implement any additional features you'd like to see. > > CHEERS> SAM > > Thiago Pinto Damas wrote: > >> Hi, >> does the Neurogrid simulate latencies, losses, etc? >> Does the neurogrid support distributed simulations? Is it needed? >> >> Cheers >> Thiago >> Sam Joseph wrote: >> >>> Hi Ian and Thiago, >>> >>> Having looked at NS-2, but not used it, I came to the conclusion that >>> it would be difficult to use it for large scale p2p simulations due >>> to the fact that it simulates alot of low level network protocols in >>> some details. >>> >>> If you are wanting to simulate a p2p network that will have many >>> nodes, and perhaps lots of users with a variety of resources, may I >>> recommend the NeuroGrid simulator. Despite the name it is actually a >>> generic p2p simulator and supports Gnutella, Freenet and NeuroGrid >>> simulations, and is more scalable because it simulates nodes and >>> connections in the most simple form possible. >>> >>> Over the summer the simulator was extended to support a p2p DNS >>> simulation and a p2p email simulation that includes pastry routing. >>> I think you'll find this simulator highly extendible to support your >>> own projects. >>> >>> Check out: >>> >>> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerSimulation >>> >>> The documentation process is ongoing, but anybody can edit the wiki >>> and this has helped us document the new extensions. >>> Please feel free to mail me with any questions: sam@neurogrid.com >>> >>> CHEERS> SAM >>> >>> >>> Thiago Pinto Damas wrote: >>> >>>> anyone already tried to implement a p2p network in the ns-2 >>>> network simulator? Something simple like a flooding algorithm. >>>> Any more ideas about other simulators? >>> >>> >>> >>> >>> Ian wrote: >>> >>> Recently, I have to propose a simulation plan for my P2P lookup >>> protocol. >>> So far, the simulation plans I have surveyed all have simulators >>> respectively. >>> Some are written in JXTA, C..,etc. >>> I am wondering why not using NS or NS-2 ? >>> Is there any scaling problem on NS or NS-2 ? >>> or other reasons ? >>> >>> >>> >> >> >> > > > From justin at chapweske.com Tue Oct 14 03:34:21 2003 From: justin at chapweske.com (Justin Chapweske) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] Berkeley DB over DHT Message-ID: <3F8B6EBD.8010706@chapweske.com> Steve, Roger, Brandon, Bram, and I were toying with the idea of implementing a DHT underneath a DBM interface. I thought that this notion might be an interesting topic for discussion as it could possibly enable some really cool applications w/o modifying any their source code. For instance, someone said that mysql is implemented on top of bdb...Would bdb over a DHT give us a scalable SQL database?? What would need to be added to existing DHT's to make this happen? Who will be the first one to write an implementation? Talk amungst yaselves. -- Justin Chapweske, Onion Networks http://onionnetworks.com/ From sam at neurogrid.com Tue Oct 14 03:37:52 2003 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 In-Reply-To: <3F8B66E3.9040306@vetorial.net> References: <3F873CFB.5020406@vetorial.net> <3F8B083E.50200@neurogrid.com> <3F8B3CCB.4060408@vetorial.net> <3F8B420B.6040708@neurogrid.com> <3F8B66E3.9040306@vetorial.net> Message-ID: <3F8B6F90.7020106@neurogrid.com> Hi Thiago, Yes I know the PSearch algorithm - you might be interested in a review of different similar systems I wrote: The second paper in: http://www.neurogrid.net/php/publications.php In terms of helping to add new features to the NeuroGrid simulator we need to add you to the developers list on sourceforge - do you have a sourceforge id? CHEERS> SAM Thiago Pinto Damas wrote: > it sounds really cool! My plan it's use a simulator to view the > characteristics of a p2p system, specially the PSearch[1] algorithm. > Now, I'll study the various simulators available for this type of > network, to implement and test the algorithm later. > Of course, i'd like to help to add new features in the neurogrid > simulator! How can I do this? From coderman at charter.net Tue Oct 14 03:50:45 2003 From: coderman at charter.net (coderman) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] Berkeley DB over DHT In-Reply-To: <3F8B6EBD.8010706@chapweske.com> References: <3F8B6EBD.8010706@chapweske.com> Message-ID: <3F8B7295.4060207@charter.net> Justin Chapweske wrote: > Steve, Roger, Brandon, Bram, and I were toying with the idea of > implementing a DHT underneath a DBM interface. This would map well for the key->value lookup and storage, but iterators over an entire DHT space would be unrealistic. (I.e. listing all of the keys in a bdb, which some applications do rely on) This might be ok while small, but the lookup latency would start compounding greatly when dealing with 10,000 keys, 50,000 keys? > I thought that this notion might be an interesting topic for > discussion as it could possibly enable some really cool applications > w/o modifying any their source code. For instance, someone said that > mysql is implemented on top of bdb...Would bdb over a DHT give us a > scalable SQL database?? The hard part is the iterators. If you have a query that is doing a linear table scan, the DHT would probably not be a good solution? If you cache the key index promisciously, you then get to deal with cache coherence/consistency problems. If you place the index in the DHT as a whole, do hot spots matter? (i.e. the index node would get requested far more often that most other nodes) > What would need to be added to existing DHT's to make this happen? Iterators, lots of redundancy, concurrent lookup? This would be a fun implementation (can someone lend me some of that copious free time? :-) martin From justin at chapweske.com Tue Oct 14 04:35:33 2003 From: justin at chapweske.com (Justin Chapweske) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] Berkeley DB over DHT In-Reply-To: <3F8B7295.4060207@charter.net> References: <3F8B6EBD.8010706@chapweske.com> <3F8B7295.4060207@charter.net> Message-ID: <3F8B7D15.5080807@chapweske.com> > > > This would map well for the key->value lookup and storage, but iterators > over an entire DHT > space would be unrealistic. (I.e. listing all of the keys in a bdb, > which some applications do rely on) > This might be ok while small, but the lookup latency would start > compounding greatly when dealing > with 10,000 keys, 50,000 keys? > In systems where iteration is innefficient, the solution is often to explicitly store a record with a snapshot of the current keys. This may be an option here. Also, I think the notion of a read-only database implemented over a DHT would also be interesting. So if read/write proves too difficult, perhaps read-only would be a good starting point. As a next step, one could imagine a system that allows append-only additions to the key table. This might allow the database to be updated on a periodic basis... > > The hard part is the iterators. If you have a query that is doing a > linear table scan, the DHT would > probably not be a good solution? If you cache the key index > promisciously, you then get to deal with > cache coherence/consistency problems. If you place the index in the DHT > as a whole, do hot spots > matter? (i.e. the index node would get requested far more often that > most other nodes) > It seems to me that solving the hot spot problem for index replication would be useful for other applications as well. Is the solution to special-case the index and use some other replication mechanism, or is there a way to hack the DHT to provide higher availability for the index? >> What would need to be added to existing DHT's to make this happen? > > > Iterators, lots of redundancy, concurrent lookup? > > This would be a fun implementation (can someone lend me some of that > copious free time? :-) Okay, lets say that we don't go with the index approach. Are there other (sane) ways to do distributed iteration? -- Justin Chapweske, Onion Networks http://onionnetworks.com/ From dtsouma at umiacs.umd.edu Tue Oct 14 05:19:15 2003 From: dtsouma at umiacs.umd.edu (Dimitrios Tsoumakos) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] Berkeley DB over DHT In-Reply-To: <3F8B7D15.5080807@chapweske.com> Message-ID: See the PIER project in Berkeley (paper in VLDB 2003) Although the idea seems interesting, their proposal (not their fault of course) seems very inefficient...object = tuple...They try to do joins... --------------------------------------------------------- Dimitrios Tsoumakos Graduate Research Assistant Dept. of Computer Science 3220 AV Williams Building University of Maryland, College Park dtsouma@cs.umd.edu --------------------------------------------------------- From smitajoshi21 at hotmail.com Tue Oct 14 17:58:32 2003 From: smitajoshi21 at hotmail.com (Smita Thaker) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] Good 8086 disassembler tool Message-ID: I am looking for a good disassembler. Any tools which are really good. thanks smita -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20031014/27ec0138/attachment.htm From jdl at vinecorp.com Tue Oct 14 17:55:08 2003 From: jdl at vinecorp.com (jdl@vinecorp.com) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] [Silicon Valley] P2Punks meeting TONIGHT (Oct 14) 7:30pm onward at DSRC Message-ID: <20031014135508.B28860@lynx.phpwebhosting.com> My final reminder/pester. ----- I've changed the day back to the second Tuesday of the month. See you there.. Where: Dana Street Roasting Company 744 Dana St., Mountain View Phone: (650) 390-9638 1/2 block off Castro St. When: 7:30pm onward ----- End forwarded message ----- From badapple at netnitco.net Tue Oct 14 17:52:30 2003 From: badapple at netnitco.net (Fred Grott) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] Good 8086 disassembler tool In-Reply-To: References: Message-ID: <3F8C37DE.9080602@netnitco.net> Smita Thaker wrote: > I am looking for a good disassembler. Any tools which are really good. > > thanks > smita > >------------------------------------------------------------------------ > >_______________________________________________ >p2p-hackers mailing list >p2p-hackers@zgp.org >http://zgp.org/mailman/listinfo/p2p-hackers >_______________________________________________ >Here is a web page listing P2P Conferences: >http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > > which language? From if26 at cornell.edu Tue Oct 14 19:06:36 2003 From: if26 at cornell.edu (Ian Fung) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 In-Reply-To: <3F873CFB.5020406@vetorial.net> References: <3F873CFB.5020406@vetorial.net> Message-ID: <1066158396.704.4.camel@wynona> ya, i have. i implemented my own p2p protocol (a modified version of viceroy, a p2p butterfly network). anywho, there are other simulators, but my advisor for the project insisted that i use ns2. i have used it on more than one occasion and from my experience, i would not recommend using it. it does not scale that well if you're looking to do any sims more than 10,000 nodes. i think the code/interfaces (the C++ part) is very low quality and makes building your own protocol a pain. ian On Fri, 2003-10-10 at 18:12, Thiago Pinto Damas wrote: > Hi, > anyone already tried to implement a p2p network in the ns-2 network > simulator? Something simple like a flooding algorithm. > Any more ideas about other simulators? > > thanks > Thiago Damas > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://zgp.org/pipermail/p2p-hackers/attachments/20031014/7d3ab2e4/attachment.pgp From thiago at vetorial.net Tue Oct 14 20:16:33 2003 From: thiago at vetorial.net (Thiago Pinto Damas) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 In-Reply-To: <1066158396.704.4.camel@wynona> References: <3F873CFB.5020406@vetorial.net> <1066158396.704.4.camel@wynona> Message-ID: <1066162593.3f8c59a199d70@webmail.vetorial.net> Hi Ian so, which simulator do you recommend for implementing a p2p network? Thiago Citando Ian Fung : >> ya, i have. i implemented my own p2p protocol (a modified version of >> viceroy, a p2p butterfly network). anywho, there are other simulators, >> but my advisor for the project insisted that i use ns2. i have used it >> on more than one occasion and from my experience, i would not recommend >> using it. it does not scale that well if you're looking to do any sims >> more than 10,000 nodes. i think the code/interfaces (the C++ part) is >> very low quality and makes building your own protocol a pain. >> >> ian >> >> On Fri, 2003-10-10 at 18:12, Thiago Pinto Damas wrote: >> > Hi, >> > anyone already tried to implement a p2p network in the ns-2 network >> > simulator? Something simple like a flooding algorithm. >> > Any more ideas about other simulators? >> > >> > thanks >> > Thiago Damas From badapple at netnitco.net Tue Oct 14 20:56:27 2003 From: badapple at netnitco.net (Fred Grott) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 In-Reply-To: <1066162593.3f8c59a199d70@webmail.vetorial.net> References: <3F873CFB.5020406@vetorial.net> <1066158396.704.4.camel@wynona> <1066162593.3f8c59a199d70@webmail.vetorial.net> Message-ID: <3F8C62FB.50605@netnitco.net> Thiago Pinto Damas wrote: > Hi Ian > so, which simulator do you recommend for implementing a p2p network? > > Thiago > > >Citando Ian Fung : > > > >>>ya, i have. i implemented my own p2p protocol (a modified version of >>>viceroy, a p2p butterfly network). anywho, there are other simulators, >>>but my advisor for the project insisted that i use ns2. i have used it >>>on more than one occasion and from my experience, i would not recommend >>>using it. it does not scale that well if you're looking to do any sims >>>more than 10,000 nodes. i think the code/interfaces (the C++ part) is >>>very low quality and makes building your own protocol a pain. >>> >>>ian >>> >>>On Fri, 2003-10-10 at 18:12, Thiago Pinto Damas wrote: >>> >>> >>>> Hi, >>>> anyone already tried to implement a p2p network in the ns-2 network >>>>simulator? Something simple like a flooding algorithm. >>>> Any more ideas about other simulators? >>>> >>>> thanks >>>> Thiago Damas >>>> >>>> > >_______________________________________________ >p2p-hackers mailing list >p2p-hackers@zgp.org >http://zgp.org/mailman/listinfo/p2p-hackers >_______________________________________________ >Here is a web page listing P2P Conferences: >http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > > > Ian I would also be interested in what simulator you recommend From thiago at vetorial.net Tue Oct 14 21:47:24 2003 From: thiago at vetorial.net (Thiago Pinto Damas) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 In-Reply-To: <3F8C62FB.50605@netnitco.net> References: <3F873CFB.5020406@vetorial.net> <1066158396.704.4.camel@wynona> <1066162593.3f8c59a199d70@webmail.vetorial.net> <3F8C62FB.50605@netnitco.net> Message-ID: <3F8C6EEC.3010405@vetorial.net> Anyway, I'll try to use the neurogrid simulator, like Sam Joseph suggested me. But, I'd appreciate any help on those directions. []s Thiago Fred Grott wrote: > Thiago Pinto Damas wrote: > >> Hi Ian >> so, which simulator do you recommend for implementing a p2p network? >> >> Thiago >> >> >> Citando Ian Fung : >> >> >> >>>> ya, i have. i implemented my own p2p protocol (a modified version of >>>> viceroy, a p2p butterfly network). anywho, there are other simulators, >>>> but my advisor for the project insisted that i use ns2. i have used it >>>> on more than one occasion and from my experience, i would not recommend >>>> using it. it does not scale that well if you're looking to do any sims >>>> more than 10,000 nodes. i think the code/interfaces (the C++ part) is >>>> very low quality and makes building your own protocol a pain. >>>> >>>> ian >>>> >>>> On Fri, 2003-10-10 at 18:12, Thiago Pinto Damas wrote: >>>> >>>> >>>>> Hi, >>>>> anyone already tried to implement a p2p network in the ns-2 >>>>> network simulator? Something simple like a flooding algorithm. >>>>> Any more ideas about other simulators? >>>>> >>>>> thanks >>>>> Thiago Damas >>>>> >> >> >> _______________________________________________ >> p2p-hackers mailing list >> p2p-hackers@zgp.org >> http://zgp.org/mailman/listinfo/p2p-hackers >> _______________________________________________ >> Here is a web page listing P2P Conferences: >> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences >> >> >> > Ian I would also be interested in what simulator you recommend > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From if26 at cornell.edu Wed Oct 15 00:52:22 2003 From: if26 at cornell.edu (Ian Fung) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 In-Reply-To: <1066162593.3f8c59a199d70@webmail.vetorial.net> References: <3F873CFB.5020406@vetorial.net> <1066158396.704.4.camel@wynona> <1066162593.3f8c59a199d70@webmail.vetorial.net> Message-ID: <1066179142.747.16.camel@wynona> unfortunately, i have not used one that i would "recommend". admittedly, i havent used many others as thoroughly as i have ns2. i have been reading literature on neurogrid and must say it sounds good. on paper anyways =). sorry i can't be more helpful. ian On Tue, 2003-10-14 at 15:16, Thiago Pinto Damas wrote: > Hi Ian > so, which simulator do you recommend for implementing a p2p network? > > Thiago > > > Citando Ian Fung : > > >> ya, i have. i implemented my own p2p protocol (a modified version of > >> viceroy, a p2p butterfly network). anywho, there are other simulators, > >> but my advisor for the project insisted that i use ns2. i have used it > >> on more than one occasion and from my experience, i would not recommend > >> using it. it does not scale that well if you're looking to do any sims > >> more than 10,000 nodes. i think the code/interfaces (the C++ part) is > >> very low quality and makes building your own protocol a pain. > >> > >> ian > >> > >> On Fri, 2003-10-10 at 18:12, Thiago Pinto Damas wrote: > >> > Hi, > >> > anyone already tried to implement a p2p network in the ns-2 network > >> > simulator? Something simple like a flooding algorithm. > >> > Any more ideas about other simulators? > >> > > >> > thanks > >> > Thiago Damas > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://zgp.org/pipermail/p2p-hackers/attachments/20031014/31d100b0/attachment.pgp From muller at emse.fr Wed Oct 15 06:39:47 2003 From: muller at emse.fr (Guillaume MULLER) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] Good 8086 disassembler tool Message-ID: <3F8CEBB3.1080501@emse.fr> I wouldn't have asked which langage since when you dissassemble you dis-assemble (so, don't you ear "asm" ?) I would have asked which OS ! My answer is under win* : W32Dasm is good (recognize string calls or menu item calls or gives a list of exported/imported funcs and allows step-by-step debugging...) and it seems some crackers uses IDA (big : 3 Disks) under linux or other unices : gdb PS : yeah... I know some dissas know how to recognize some parts of C or other languages... -- Guillaume MULLER 29 rue Ponchardier 158 Cours Fauriel Phd Student SIMMO/SMA tel 33 1 (0)4 77 42 66 84 fax 33 1 (0)4 77 42 66 66 http://www.emse.fr/~gmuller From izzy at lina.es.ncku.edu.tw Thu Oct 16 06:41:31 2003 From: izzy at lina.es.ncku.edu.tw (izzy) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 Message-ID: <003601c393b0$896c7720$7327748c@izzy> Dear Thiago and Sam, MIT seems to be developing a new extensible simulator for P2P , including Chord, Kademlia,Koorde,pastry, tapestry routing simulation. It is not yet published. Take a look at CVS : http://pdos.lcs.mit.edu/cgi-bin/cvsweb.cgi/sfsnet/p2psim/ It might help~~ Ian Lee From sam at neurogrid.com Fri Oct 17 18:15:41 2003 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 In-Reply-To: <003601c393b0$896c7720$7327748c@izzy> References: <003601c393b0$896c7720$7327748c@izzy> Message-ID: <3F9031CD.1000104@neurogrid.com> Hi Izzy, That MIT thing looks interesting - I'd love to check it out so I can do an analysis of it offline - do you know if they have an anonymous CVS access? CHEERS> SAM izzy wrote: >Dear Thiago and Sam, > >MIT seems to be developing a new extensible simulator for P2P , including >Chord, Kademlia,Koorde,pastry, tapestry routing simulation. >It is not yet published. >Take a look at CVS : >http://pdos.lcs.mit.edu/cgi-bin/cvsweb.cgi/sfsnet/p2psim/ >It might help~~ > >Ian Lee > > >_______________________________________________ >p2p-hackers mailing list >p2p-hackers@zgp.org >http://zgp.org/mailman/listinfo/p2p-hackers >_______________________________________________ >Here is a web page listing P2P Conferences: >http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > > > > From jdd at dixons.org Fri Oct 17 18:55:03 2003 From: jdd at dixons.org (Jim Dixon) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 In-Reply-To: <3F9031CD.1000104@neurogrid.com> Message-ID: <20031017195342.O30630-100000@localhost> On Sat, 18 Oct 2003, Sam Joseph wrote: > Hi Izzy, > > That MIT thing looks interesting - I'd love to check it out so I can do > an analysis of it offline - do you know if they have an anonymous CVS > access? The crude solution is to download the relevant part of the Web site, using webcopy or something like it: http://www.inf.utfsm.cl/~vparada/webcopy.html -- Jim Dixon jdd@dixons.org tel +44 117 982 0786 mobile +44 797 373 7881 From ntarmos at ceid.upatras.gr Fri Oct 17 19:14:13 2003 From: ntarmos at ceid.upatras.gr (Nikos Ntarmos) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 In-Reply-To: <3F9031CD.1000104@neurogrid.com> References: <003601c393b0$896c7720$7327748c@izzy> <3F9031CD.1000104@neurogrid.com> Message-ID: <20031017191413.GA27300@Noth.ceid.upatras.gr> Hi there. On Sat, Oct 18, 2003 at 03:15:41AM +0900, Sam Joseph wrote: > That MIT thing looks interesting - I'd love to check it out so I can do > an analysis of it offline - do you know if they have an anonymous CVS > access? > > >http://pdos.lcs.mit.edu/cgi-bin/cvsweb.cgi/sfsnet/p2psim/ You can get a snapshot of the cvs tree from the above url by choosing file with tag HEAD and clicking on the "Download this directory in tarball or zip archive" links. \n\n -- Nikos "Noth" Ntarmos | < ntarmos at ceid dot upatras dot gr > NetCINS Lab. @ C.E.I.D. | [ http://{noth,p2p}.ceid.upatras.gr/ ] U. of Patras - Greece | ( 38.2594N, 21.7428E ) ( 1024D / CF95160A ) From sam at neurogrid.com Fri Oct 17 20:12:16 2003 From: sam at neurogrid.com (Sam Joseph) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 In-Reply-To: <20031017191413.GA27300@Noth.ceid.upatras.gr> References: <003601c393b0$896c7720$7327748c@izzy> <3F9031CD.1000104@neurogrid.com> <20031017191413.GA27300@Noth.ceid.upatras.gr> Message-ID: <3F904D20.8090906@neurogrid.com> Hi Nikos, Got it like that - many thanks. Thanks to others for suggestions too. CHEERS> SAM Nikos Ntarmos wrote: >Hi there. > >On Sat, Oct 18, 2003 at 03:15:41AM +0900, Sam Joseph wrote: > > >>That MIT thing looks interesting - I'd love to check it out so I can do >>an analysis of it offline - do you know if they have an anonymous CVS >>access? >> >> >> >>>http://pdos.lcs.mit.edu/cgi-bin/cvsweb.cgi/sfsnet/p2psim/ >>> >>> > >You can get a snapshot of the cvs tree from the above url by choosing >file with tag HEAD and clicking on the "Download this directory in >tarball or zip archive" links. > >\n\n > > > From izzy at lina.es.ncku.edu.tw Fri Oct 17 22:28:13 2003 From: izzy at lina.es.ncku.edu.tw (izzy) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] P2P in NS-2 Message-ID: <000e01c394fd$f41aa710$7327748c@izzy> Dear Sam, Sure, they have anonymous CVS: Anonymous CVS The anonymous option is open to anyone. Those using anonymous CVS are encouraged to submit patches to chord-dev at pdos.lcs.mit.edu. (Replace the at with an @.) To check out the sources via anonymous CVS run the following commands: % cvs -d :pserver:anoncvs@cvs.pdos.lcs.mit.edu:/cvs login Logging in to :pserver:anoncvs@cvs.pdos.lcs.mit.edu:2401/cvs CVS password: press return % cvs -d :pserver:anoncvs@cvs.pdos.lcs.mit.edu:/cvs co -P sfsnet Ian Lee >Hi Izzy, > >That MIT thing looks interesting - I'd love to check it out so I can do >an analysis of it offline - do you know if they have an anonymous CVS >access? > >CHEERS> SAM From eugen at leitl.org Mon Oct 20 16:50:03 2003 From: eugen at leitl.org (Eugen Leitl) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] [mnet-devel] Anonymity and Performance in Peer-to-Peer Systems (fwd from amichrisde@yahoo.de) Message-ID: <20031020165003.GJ15714@leitl.org> ----- Forwarded message from Some Guy ----- From: Some Guy Date: Mon, 20 Oct 2003 15:13:01 +0200 (CEST) To: mnet-devel@lists.sourceforge.net Cc: nikitab@cs.berkeley.edu Subject: [mnet-devel] Anonymity and Performance in Peer-to-Peer Systems Reply-To: mnet-devel@lists.sourceforge.net This post is supposed to get to Nikita Borisov and Jason Waddle regarding: http://www.cs.berkeley.edu/~nikitab/projects/p2p/proposal.html Part 1: "Research a formal metric for the anonymity of a system. We are currently performing a literature search; if we fail to find a suitable metric in past research, we will propose a new one." Here's a nice paper that analizes the annonymity of Crowds: Probabilistic Analysis of Anonymity Vitaly Shmatikov SRI International http://citeseer.nj.nec.com/shmatikov02probabilistic.html "We use discrete-time Markov chains to formally model the behavior of group members in Crowds [22], specify anonymity properties of the system as temporal logic formulas, and use a probabilistic model checker to verify them. This analysis method is applicable to other probabilistic systems such as Freenet." Part 2: DOS Resistance: I've got a basic handle now on DOS resistance vs. peformance and I can almost get a faint glimmer of what the a "perfect" DHT should look like. I've posted my ideas on it to the Freenet list and I'll post them to the MNet list today too. If anyone wants to discuss how to build a better Freenet, I'd LOVE to help. __________________________________________________________________ Gesendet von Yahoo! Mail - http://mail.yahoo.de Logos und Klingelt?ne f?rs Handy bei http://sms.yahoo.de ------------------------------------------------------- This SF.net email sponsored by: Enterprise Linux Forum Conference & Expo The Event For Linux Datacenter Solutions & Strategies in The Enterprise Linux in the Boardroom; in the Front Office; & in the Server Room http://www.enterpriselinuxforum.com _______________________________________________ mnet-devel mailing list mnet-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/mnet-devel ----- End forwarded message ----- -- Eugen* Leitl leitl ______________________________________________________________ ICBM: 48.07078, 11.61144 http://www.leitl.org 8B29F6BE: 099D 78BA 2FD3 B014 B08A 7779 75B0 2443 8B29 F6BE -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 198 bytes Desc: not available Url : http://zgp.org/pipermail/p2p-hackers/attachments/20031020/301fda04/attachment.pgp From shaifuljahari at yahoo.com Tue Oct 21 08:03:42 2003 From: shaifuljahari at yahoo.com (Shaiful) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] ns2 gnutella simulator Message-ID: <20031021080342.35372.qmail@web41612.mail.yahoo.com> Hi guys, This link might be useful: http://www.cc.gatech.edu/computing/compass/gnutella/install.html __________________________________ Do you Yahoo!? The New Yahoo! Shopping - with improved product search http://shopping.yahoo.com From paul at soniq.net Tue Oct 21 08:15:23 2003 From: paul at soniq.net (Paul Boehm) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] paper on p2p game theory Message-ID: <20031021081523.GB2903@soniq.net> http://arxiv.org/abs/cs/0310039 A Game Theoretic Framework for Incentives in P2P Systems Authors: Chiranjeeb Buragohain, Divyakant Agrawal, Subhash Suri Subj-class: Computer Science and Game Theory ACM-class: H.4, J.4 Journal-ref: Proc. of the Third International Conference on P2P Computing (P2P2003), Linkoping Sweden, 2003 Peer-To-Peer (P2P) networks are self-organizing, distributed systems, with no centralized authority or infrastructure. Because of the voluntary participation, the availability of resources in a P2P system can be highly variable and unpredictable. In this paper, we use ideas from Game Theory to study the interaction of strategic and rational peers, and propose a differential service-based incentive scheme to improve the system's performance. From seth.johnson at RealMeasures.dyndns.org Sat Oct 25 03:38:47 2003 From: seth.johnson at RealMeasures.dyndns.org (Seth Johnson) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] URGENT ALERT: Stop the "Broadcast Flag" at the FCC Message-ID: <3F99F047.8D5EFB3D@RealMeasures.dyndns.org> New Yorkers for Fair Use Action Alert: -------------------------------------- Please send a comment to the FCC AGAIN, opposing the "Broadcast Flag" Proposal Tell the FCC to Serve the Public, Not Hollywood! Okay, you folks understand this issue -- it's very important to send word to the FCC in the next few days, that you OPPOSE the Notice of Proposed Rulemaking #02-230. This rule would make it illegal for ordinary citizens to own fully functional digital television devices. We've made it easy; just follow the links below. 1) Please send in your comments to the FCC using the form provided below. Tell them that the movie industry should not have a special privilege to own fully-functional digital television devices. Read the alert below for details. 2) Please forward this alert to any other interested parties that you know of, who would understand and see the importance of this issue. 3) Volunteer to help us with this and other alerts related to your rights to flexible information technology in the future. Two roles you can take up are to become a Press Outreach Campaigner or a Commentator. Simply reply to this email to show your interest. New Yorkers for Fair Use Action Alert: -------------------------------------- Tell the FCC to Serve the Public, Not Hollywood! Send Public Comments to the FCC AGAIN to Stop the "Broadcast Flag" Please follow these links to let the FCC know that the public's rights are at stake: http://www.nyfairuse.org/action/fcc.flag/ What's Going On: The FCC is expected to decide this week that digital televisions will be required to work only according to the rules set by Hollywood, through the use of a "broadcast flag" assigned to digital TV broadcasts. As a result of the deliberations of a group called the Broadcast Protection Discussion Group, which has assiduously discounted the public's rights to use flexible information technology, Hollywood and leading technology players have devised a plan that would only allow "professionals" to have fully-functional devices for processing digital broadcast materials. Almost a year ago, you responded to our call to tell the FCC that they are to serve the public, not Hollywood. You delivered more than 4000 comments to the the FCC's public comments system in the space of the last week of their public comments period for the broadcast flag proposal. As a result of this, Congress took notice and called a hearing to question the FCC on the issue. When they asked the FCC's representative whether he believed they could make this copyright-related policy decision without stepping beyond their bounds and into Congress's jurisdiction, they answered in one word: "Yes." Now, their period of considering the proposal is drawing to an end, and they are expected to decide to mandate the broadcast flag in a matter of days, by the end of this month. It's time to demonstrate AGAIN that the public's interests take priority over the wishes of the MPAA. The idea of the broadcast flag is to implement universal content control and abolish the right of free citizens to own effective tools for employing digital content in useful ways. Hollywood and content producers must not be allowed to determine the rights of the public to use flexible information technology. The broadcast flag is theft. In the ongoing fight with old world content industries, the most essential rights and interests in a free society are those of the public. Free citizens are not mere consumers; they are not a separate group from so-called "professionals." The stakeholders in a truly just information policy in a free society are the public, not those who would reserve special rights to control public uses of information technology. Please let the FCC know that the public's rights are at stake: http://www.nyfairuse.org/action/fcc.flag/. Here is a page pulling together and summarizing the comments submitted after the last comments campaign: http://www.nyfairuse.org/bfpc/ Here is our Reply Comment: http://www.nyfairuse.org/bfpc/extdoc/NPRM%2002-231%20Reply%20Comments.pdf ---- The following link is the FCC's "Notice of Proposed Rulemaking" for the broadcast flag. http://hraunfoss.fcc.gov/edocs_public/attachmatch/FCC-02-231A1.pdf From decapita at dti.unimi.it Mon Oct 27 11:06:10 2003 From: decapita at dti.unimi.it (Sabrina De Capitani di Vimercati) Date: Sat Dec 9 22:12:23 2006 Subject: [p2p-hackers] Call for Papers: Workshop on Issues in the Theory of Security (WITS'04) Message-ID: [Apologies if you receive multiple copies of this message] CALL FOR PAPERS 2004 IFIP WG 1.7, ACM SIGPLAN and GI FoMSESS Workshop on Issues in the Theory of Security (WITS'04) April 3 - 4, 2004, Barcelona, Spain co-located with ETAPS'04 http://www.dsi.unive.it/IFIPWG1_7/wits2004.html ---------------------------------------------------------------------- OVERVIEW OF WITS WITS is the offical workshop organised by the IFIP WG 1.7 on "Theoretical Foundations of Security Analysis and Design", established to promote the investigation on the theoretical foundations of security, discovering and promoting new areas of application of theoretical techniques in computer security and supporting the systematic use of formal techniques in the development of security related applications. The members of WG hold their annual workshop as an open event to which all researchers working on the theory of computer security are invited. This is the fourth workshop of the series, and is organised in cooperation with ACM SIGPLAN and GI working group FoMSESS. Extended abstracts of work (accepted after selection and) presented at the Workshop are collected and distributed to the participants. There will be no formally published proceedings; however, selected papers will be invited for submission to a special issue of the Journal of Computer Security. Suggested submission topics include: * formal definition and verification of the various aspects of security: confidentiality, privacy, integrity, authentication and availability * new theoretically-based techniques for the formal analysis and design of cryptographic protocols and their manifold applications (e.g., electronic commerce) * information flow modelling and its application to the theory of confidentiality policies, composition of systems, and covert channel analysis * formal techniques for the analysis and verification of code security, including mobile code security * formal analysis and design for prevention of denial of service * security in real-time/probabilistic systems * language-based security IMPORTANT DATES Paper Submission: 15 December 2003 Author Notification: 25 January 2004 Final version due: 29 February 2004 Workshop: 3-4 April 2004 PROGRAM COMMITTEE David Basin, ETH Zurich Pierpaolo Degano, Universit? di Pisa Claudia Eckert, TU Darmstadt and Fraunhofer SIT Riccardo Focardi, Universit? di Venezia Dieter Gollmann, TU Hamburg-Harburg, Germany Roberto Gorrieri, Universit? di Bologna Joshua Guttman, MITRE Chris Hankin, Imperial College Jan J?rjens, Munich University of Technology Gavin Lowe, Oxford University Cathy Meadows, Naval Research Laboratory Jon Millen, SRI International Peter Ryan (chair), University of Newcastle Thomas Santen, Dresden University of Technology Steve Schneider, Royal Holloway, University of London Paul Syverson, Naval Research Laboratory SUBMISSION INSTRUCTIONS Authors are invited to submit an extended abstract, up to 12 pages long, in LNCS style or with 11pt or larger font and reasonable margins and line spacing. Submissions departing from the instructions above are rejected independently of their technical merit. Authors have to submit through the web. Alternatively, they may e_mail a .ps file. If necessary, they may mail a single hard copy of their paper to the program chair; in the last case, please allow ample time for delivery. Submissions should have the author's full name, address, fax number, and e-mail address. FURTHER INFORMATION The official web page of the conference is at the url http://www.dsi.unive.it/IFIPWG1_7/wits2004.html Contact person: Peter Ryan School of Computing Science, University of Newcastle Claremont Tower, Newcastle upon Tyne NE1 7RU, UK tel: +44 0191 222 8788, fax: +44 0191 222 8972 e-mail: peter.ryan@ncl.ac.uk http://www.csr.ncl.ac.uk/