From macambira at gmail.com Thu Sep 1 19:08:38 2005 From: macambira at gmail.com (Tiago Macambira) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Caches solutions to P2P file-sharing systems In-Reply-To: <1125284927.8471.29.camel@hades.no-ip.org> References: <1125284927.8471.29.camel@hades.no-ip.org> Message-ID: On 8/29/05, Fabr?cio Barros Cabral wrote: > Hello people! > > I'm searching about caches solutions to P2P file-sharing systems (KaZaA, > eDonkey, BitTorrent) and basically I found two commercial solutions: ... > > So, I'd like know if anyone has more information about these systems, > mainly the CacheLogic solution, because the web system is very poor > about it operation details. E ai, conseguiu alguma ? []s Tiago Alves Macambira From hatsmagee at gmail.com Sat Sep 3 05:17:04 2005 From: hatsmagee at gmail.com (tom jacobs) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Localhost - A new program that creates an Internet-wide decentralised file index hierarchy. Message-ID: <38b25b2c0509022217408a9b90@mail.gmail.com> Hi all, I've been working on a program, which I call Localhost. It creates a decentralised file index hierarchy, like a "file system", which you access through you web browser. There can be many different versions of each folder in the system, and the default version displayed is determined by popularity; the default version displayed is the one with the most users viewing that version. Hopefully it will evolve into a useful resource, perhaps an archive of Open Source software, or Creative Commons licenced content. The client is based off Azureus. See http://localhost.r8.org for more information. You can preview the system without downloading anything from that website. If it looks interesting, you can download the client and help build the "file system" by adding files. Tom. From joejoe99 at softhome.net Sat Sep 3 15:28:37 2005 From: joejoe99 at softhome.net (joejoe99@softhome.net) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Localhost - A new program that creates an Internet-wide decentralised file index hierarchy. In-Reply-To: <38b25b2c0509022217408a9b90@mail.gmail.com> References: <38b25b2c0509022217408a9b90@mail.gmail.com> Message-ID: Hi Tom, Looks like a really nice ware. Thanks! I didn't see a license on the site anywhere. Is this GPL? How remote is the possibility that it would support/allow a php/mysql/apache app to execute and run over this network? Thanks again for all of it. jj On Sat, 3 Sep 2005 15:17:04 +1000 tom jacobs wrote: > Hi all, > I've been working on a program, which I call Localhost. > > It creates a decentralised file index hierarchy, like a > "file system", which you access through you web browser. > There can be many different versions of each folder in the > system, and the default version displayed is determined by > popularity; the default version displayed is the one with > the most users viewing that version. Hopefully it will > evolve into a useful resource, perhaps an archive of Open > Source software, or Creative Commons licenced content. The > client is based off Azureus. > > See http://localhost.r8.org for more information. > > You can preview the system without downloading anything > from that website. If it looks interesting, you can > download the client and help build the "file system" by > adding files. > > Tom. > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From hatsmagee at gmail.com Sun Sep 4 04:47:29 2005 From: hatsmagee at gmail.com (tom jacobs) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: p2p-hackers Digest, Vol 26, Issue 2 In-Reply-To: <20050903190003.D65093FD06@capsicum.zgp.org> References: <20050903190003.D65093FD06@capsicum.zgp.org> Message-ID: <38b25b2c050903214755016018@mail.gmail.com> It is GPL. (The FAQ mentions this briefly.) The source code is on the download page. Yes, this system has lots of room for improvement and extension. I was thinking of having the tail ends of the folders, for example, /Company/SmallCompany to be a website, rather than just another folder. And then once that is done, we could have the software execute server-side code to serve pages. Its all a matter of "will this project be useful". Tom. On 9/4/05, p2p-hackers-request@zgp.org wrote: > Send p2p-hackers mailing list submissions to > p2p-hackers@zgp.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://zgp.org/mailman/listinfo/p2p-hackers > or, via email, send a message with subject or body 'help' to > p2p-hackers-request@zgp.org > > You can reach the person managing the list at > p2p-hackers-owner@zgp.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of p2p-hackers digest..." > > > Today's Topics: > > 1. Localhost - A new program that creates an Internet-wide > decentralised file index hierarchy. (tom jacobs) > 2. Re: Localhost - A new program that creates an Internet-wide > decentralised file index hierarchy. (joejoe99@softhome.net) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 3 Sep 2005 15:17:04 +1000 > From: tom jacobs > Subject: [p2p-hackers] Localhost - A new program that creates an > Internet-wide decentralised file index hierarchy. > To: p2p-hackers@zgp.org > Message-ID: <38b25b2c0509022217408a9b90@mail.gmail.com> > Content-Type: text/plain; charset=ISO-8859-1 > > Hi all, > I've been working on a program, which I call Localhost. > > It creates a decentralised file index hierarchy, like a "file system", > which you access through you web browser. There can be many different > versions of each folder in the system, and the default version > displayed is determined by popularity; the default version displayed > is the one with the most users viewing that version. Hopefully it will > evolve into a useful resource, perhaps an archive of Open Source > software, or Creative Commons licenced content. The client is based > off Azureus. > > See http://localhost.r8.org for more information. > > You can preview the system without downloading anything from that website. > If it looks interesting, you can download the client and help build > the "file system" by adding files. > > Tom. > > > ------------------------------ > > Message: 2 > Date: Sat, 3 Sep 2005 11:28:37 -0400 > From: joejoe99@softhome.net > Subject: Re: [p2p-hackers] Localhost - A new program that creates an > Internet-wide decentralised file index hierarchy. > To: hatsmagee@gmail.com, "Peer-to-peer development." > > Message-ID: > Content-Type: text/plain; charset=US-ASCII > > Hi Tom, > > Looks like a really nice ware. Thanks! > > I didn't see a license on the site anywhere. Is this GPL? > > How remote is the possibility that it would support/allow a > php/mysql/apache app to execute and run over this network? > > Thanks again for all of it. > > jj > > On Sat, 3 Sep 2005 15:17:04 +1000 > tom jacobs wrote: > > > Hi all, > > I've been working on a program, which I call Localhost. > > > > It creates a decentralised file index hierarchy, like a > > "file system", which you access through you web browser. > > There can be many different versions of each folder in the > > system, and the default version displayed is determined by > > popularity; the default version displayed is the one with > > the most users viewing that version. Hopefully it will > > evolve into a useful resource, perhaps an archive of Open > > Source software, or Creative Commons licenced content. The > > client is based off Azureus. > > > > See http://localhost.r8.org for more information. > > > > You can preview the system without downloading anything > > from that website. If it looks interesting, you can > > download the client and help build the "file system" by > > adding files. > > > > Tom. > > _______________________________________________ > > p2p-hackers mailing list > > p2p-hackers@zgp.org > > http://zgp.org/mailman/listinfo/p2p-hackers > > _______________________________________________ > > Here is a web page listing P2P Conferences: > > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > > > ------------------------------ > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > > > End of p2p-hackers Digest, Vol 26, Issue 2 > ****************************************** > From ali at sics.se Sun Sep 11 22:03:33 2005 From: ali at sics.se (Ali Ghodsi) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] JDHT: Java Distributed Hash Table Message-ID: <4324A9B5.5030008@sics.se> We have released a BETA version of JDHT (Java Distributed Hash Table) which aims at simplifying DHT usage by providing the same interface as the Java collections Map. For more information visit http://dks.sics.se/jdht/ JDHT, uses DKS under the hood. Visit http://dks.sics.se for papers, implementation, and docs on DKS. /**** Example: stand alone peer ****/ DHT node1DHT = new JDHT(); node1DHT.put("myKey", "Hello World!"); /**** Example: second peer connecting to the first ****/ JDHT node2DHT = new JDHT( node1DHT.getReference() ); String helloString = (String) node2DHT.get("myKey"); System.out.println(helloString); /******************************************/ Best regards, Ali Ghodsi From Bernard.Traversat at Sun.COM Mon Sep 12 19:15:54 2005 From: Bernard.Traversat at Sun.COM (Bernard Traversat) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] JXTA P2P Kitchen, Sept 26th & 27th! Message-ID: <4325D3EA.1000504@sun.com> You are invited to attend our upcoming JXTA Kitchen to be held at Sun's Santa Clara campus, Monday, Sept 26th and Tuesday, Sept 27, 2005. Come join us and... - Hear the newest updates from the JXTA P2P open-source technology (www.jxta.org) - Bring your code & meet with JXTA experts from around the world - Enjoy 1:1 meetings with a JXTA team member - Discuss and share exciting new ways to use JXTA SAVE THE DATE! WHEN: Monday, September 26, 2005 - All day group event Tuesday, September 27, 2005 - 1:1 meetings w/JXTA staff WHERE: Sun Offices, Santa Clara Campus (Building #21 and others) 4120 Network Circle, Santa Clara, CA Please contact Stephanie Kaul for registration or any related questions Stephanie Kaul JXTA Marketing Manager SUN MICROSYSTEMS, INC. Email: stephanie.kaul@sun.com Phone: +1 408 276-7898 / x17898 www.jxta.org ******************************** -- --http://weblogs.java.net/blog/tra "As Java implies platform independence, and XML implies language independence, JXTA implies network independence." From tutschku at informatik.uni-wuerzburg.de Tue Sep 13 12:30:38 2005 From: tutschku at informatik.uni-wuerzburg.de (Kurt Tutschku) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] MobileP2P 06: Deadline Extension Message-ID: <20050913123553.C040C6F58E@nero.informatik.uni-wuerzburg.de> --------------------------------------------------------------------- Extension: NEW DEADLINE Oct 6th, 2005 --------------------------------------------------------------------- Call-for-Paper 3rd IEEE International Workshop on Mobile Peer-to-Peer Computing (MP2P'06) http://www-info3.informatik.uni-wuerzburg.de/mp2p2006 Pisa, Italy, March 17, 2006 In conjunction with the 4th IEEE International Conference on Pervasive Computing and Communications (PerCom?06) http://www.percom.org/ ---------------------------------------------------------------------- Topics of particular interest include, but are not limited to: - Architecture and platforms for mobile P2P computing - Measurement studies on mobile P2P systems - Performance of mobile P2P services - Resource and service discovery in mobile P2P computing - Resource exchange mechanisms in mobile P2P application - Mobile P2P over different kinds of bearer services: 2.5/3G (GPRS/UMTS) / 802.11 (WLAN) - Mobile P2P & operator/provider requirements - Reliability and carrier-gradeness of mobile P2P services - Mobile P2P & fixed P2P system interworking - Issues of combining P2P services with mobility (mobile IP / MANET) - Peer access and control in mobile environment - Location dependent mobile P2P services - Data exchange and rendering techniques for mobile devices - Secure communication protocols for mobile P2P computing - Mobile P2P messaging systems - Peer-to-peer broadband wireless communications - Applications of mobile P2P - Nature-inspired algorithms for mobile P2P computing - Theoretical issues on mobile information diffusion - Mobile P2P video games - Stimulating cooperation in mobile computing Paper Submission: ----------------- Papers must be written in English and should not exceed 6 pages in IEEE proceedings style. Authors MUST submit their papers through the EDAS web site (http://www.edas.info/) in three steps: * Creation of a personal account on EDAS (if the author does not already have one) * Registration of the paper (requiring a short abstract of up to 150 words) * Upload of the paper. Only in pdf format Submission implies the willingness of at least one of the authors to register and present the paper. All submissions will be reviewed and selected based on their originality of the paper. Accepted papers must be presented at the workshop and will appear in a combined PerCom 2006 workshop proceedings published by IEEE Computer Society Press. Authors that present a system in their paper are highly encouraged to demonstrate also the system in the demo session of the workshop. Important Dates: ---------------- Papers due: 5pm EST, October 6, 2005 (NEW) Notification of acceptance November 15, 2005 Camera-ready papers due December, 2005 Organizing Committee, Program Co-chairs: ---------------------------------------- Kurt Tutschku, Department of Distributed Systems, University of W?rzburg, Germany. Frank-Uwe Andersen, Siemens Communications, Berlin, Germany. Li Li, Communications Research Center Canada, Canada. Maria Papadopouli, Department of Computer Science, University of North Carolina at Chapel Hill, USA Steering Board -------------- - Jiannong Cao, Department of Computing, Hong Kong Polytechnic University. - Y. Charlie Hu, School of Electrical and Computer Engineering, Purdue University - Cecilia Mascolo, Department of Computer Science, University College London - Maria Papadopouli, Department of Computer Science, University of North Carolina at Chapel Hill Publicity Chair --------------- Kurt Tutschku, Department of Distributed Systems, University of W?rzburg, Germany Program Committee Members: -------------------------- - Frank-Uwe Andersen, Siemens Communications, Germany. - Christian Bettstetter, DoCoMo Euro-Labs,Germany. - Jiannong Cao, Department of Computing, Hong Kong Polytechnic University. - Luca Caviglione, CNIT - University of Genoa Research Unit, University of Genova, Italy. - Geoff Coulson, Computing Department, University of Lancaster, UK. - Hermann de Meer, Department of Computer Networks & Computer Communications, University of Passau, Germany. - Babak Esfandiari, Carleton University, Canada. - Stephen Hailes, Department of Computer Science, University College of London, UK - Felix Hernandez-Campos, Department of Computer Science, University of North Carolina at Chapel Hill, USA - Y. Charlie Hu, School of Electrical and Computer Engineering, Purdue University - Norihiro Ishikawa, NTT Docomo, Japan. - Valerie Issarny, INRIA, France. - Wolfgang Kellerer, DoCoMo Communications Laboratories Europe, Germany. - Kazuhiro Kitagawa, Keio University, Japan. - Thomas Kunz, Dept. of Systems and Comp. Engineering, Carleton University. Canada. - Luciano Lenzini, University of Pisa, Pisa, Italy. - Li Li, Communications Research Center Canada, Canada. - Christoph Lindemann, FB Informatik IV, University of Dortmund , Germany. - Maria Papadopouli, Department of Computer Science, University of North Carolina at Chapel Hill - Rudolf Riedi, Department of Statistics, Rice University, USA - Pablo Rodriguez, Microsoft Research Ltd, Cambridge, UK. - George Roussos, School of Computer Science and Information Systems, Birkbeck College, University of London, UK - Shervin Shirmohammadi, School of Information Technology and Engineering (SITE), University of Ottawa, Canada. - Kurt Tutschku, Department of Distributed Systems, University of W?rzburg, Germany. From dcarboni at gmail.com Fri Sep 16 15:49:07 2005 From: dcarboni at gmail.com (Davide Carboni) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] traversing NAT and Java Message-ID: <71b79fa905091608497dcf5607@mail.gmail.com> Hi, I'm looking for a good and reusable Java implementation of NAT traversal solution like connection reversal, UDP/TCP hole punching etc. I'm aware that JXTA provides NAT traversal but I do not want to redesign my applications with JXTA. I'm also aware of P2PSockets which hides JXTA under the hood but I've tried it without success and the project seems a little "inactive". Bye Davide -- Io non l'ho votato! From ash at wiredreach.com Fri Sep 16 16:02:03 2005 From: ash at wiredreach.com (Ash Maurya) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] traversing NAT and Java In-Reply-To: <71b79fa905091608497dcf5607@mail.gmail.com> References: <71b79fa905091608497dcf5607@mail.gmail.com> Message-ID: <8860C1FB-E463-461A-813B-5F14C4616C10@wiredreach.com> Davide, JXTA also *fairly* recently introduced a JxtaSocket abstraction in the platform itself which works a lot like a regular java socket. This allows regular socket applications to be fairly easily JXTA- ized. For instance we use JxtaSocket with the Smack XMPP library to provide a XMPP over JXTA solution in the open source WiredReach platform. Regards, Ash --- Ashish Maurya Founder, WiredReach company: http://www.wiredreach.com weblog: http://www.wiredjournal.com On Sep 16, 2005, at 10:49 AM, Davide Carboni wrote: > Hi, I'm looking for a good and reusable Java implementation of NAT > traversal solution like connection reversal, UDP/TCP hole punching > etc. > I'm aware that JXTA provides NAT traversal but I do not want to > redesign my applications with JXTA. I'm also aware of P2PSockets which > hides JXTA under the hood but I've tried it without success and the > project seems a little "inactive". > > Bye > Davide > -- > Io non l'ho votato! > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > From codewarrior at cuseeme.de Fri Sep 16 16:09:24 2005 From: codewarrior at cuseeme.de (codewarrior@cuseeme.de) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] traversing NAT and Java In-Reply-To: <71b79fa905091608497dcf5607@mail.gmail.com> References: <71b79fa905091608497dcf5607@mail.gmail.com> Message-ID: On Sep 16, 2005, at 5:49 PM, Davide Carboni wrote: > Hi, I'm looking for a good and reusable Java implementation of NAT > traversal solution like connection reversal, UDP/TCP hole punching > etc. hi davide, this could be interesting for you. http://larytet.sourceforge.net/btRat.shtml cheers marc -- " The mind, once expanded to the dimensions of larger ideas, never returns to its original size." Les Enfants Terribles www.let.de From dbarrett at quinthar.com Sat Sep 17 11:17:18 2005 From: dbarrett at quinthar.com (David Barrett) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] TCP-like Congestion Control Message-ID: <432BFB3E.7040403@quinthar.com> Back to everyone's favorite topic, congestion control. Can you advise me how to improve my approach to give better performance and/or behave better with TCP? To review, TCP congestion control has three primary variables: - cwnd - # of bytes allowed on the network - pipe - # of bytes currently "in flight" on the network - ssthresh - Threshold between "slow start" and "congestion avoidance" I update these variables in the following primary events: onSend( numBytes ) - pipe += numBytes onACK( numBytes ) - if( cwnd < ssthresh ) cwnd += min( numBytes, MTU ) - else cwnd += MTU*MTU / 2 onDrop( ) - cwnd = max( MTU, cwnd/2 ) - ssthresh = cwnd onTimeout( ) - ssthresh = max( pipe/2, MTU*2 ) - cwnd = MTU - pipe = 0 My goal is to mimic TCP as closely as possible. Toward this end I've read RFCs 893 and 2581, but I find it rather ambiguous on some points. At the moment, the above algorithms seem to occilate more wildly than I'd hope. Can you recommend any improvements? Thanks! -david From lln at it.uu.se Sat Sep 17 11:34:29 2005 From: lln at it.uu.se (=?ISO-8859-1?Q?Lars-=C5ke_Larzon?=) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] TCP-like Congestion Control In-Reply-To: <432BFB3E.7040403@quinthar.com> References: <432BFB3E.7040403@quinthar.com> Message-ID: <3B44160C-10B9-4148-9376-F21DCB0999B3@it.uu.se> 17 sep 2005 kl. 13.17 skrev David Barrett: > > onACK( numBytes ) > - if( cwnd < ssthresh ) cwnd += min( numBytes, MTU ) > - else cwnd += MTU*MTU / 2 > Why do you divide by two? AFAIK, you should divide by cwnd if you're in the congestion avoidance phase. /Lars-?ke Larzon -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/pkcs7-signature Size: 2384 bytes Desc: not available Url : http://zgp.org/pipermail/p2p-hackers/attachments/20050917/2561d4e9/smime.bin From dbarrett at quinthar.com Sat Sep 17 19:06:52 2005 From: dbarrett at quinthar.com (David Barrett) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] TCP-like Congestion Control In-Reply-To: <3B44160C-10B9-4148-9376-F21DCB0999B3@it.uu.se> References: <432BFB3E.7040403@quinthar.com> <3B44160C-10B9-4148-9376-F21DCB0999B3@it.uu.se> Message-ID: <432C694C.1070908@quinthar.com> Lars-?ke Larzon wrote: > > 17 sep 2005 kl. 13.17 skrev David Barrett: >> >> onACK( numBytes ) >> - if( cwnd < ssthresh ) cwnd += min( numBytes, MTU ) >> - else cwnd += MTU*MTU / 2 > > Why do you divide by two? AFAIK, you should divide by cwnd if you're in > the congestion avoidance phase. Good catch. What about the others -- do they look right? onSend( numBytes ) - pipe += numBytes onACK( numBytes ) - if( cwnd < ssthresh ) cwnd += min( numBytes, MTU ) - else cwnd += MTU*MTU / cwnd onDrop( ) - cwnd = max( MTU, cwnd/2 ) - ssthresh = cwnd onTimeout( ) - ssthresh = max( pipe/2, MTU*2 ) - cwnd = MTU - pipe = 0 -david From afisk at speedymail.org Sat Sep 17 20:33:27 2005 From: afisk at speedymail.org (Adam Fisk) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] traversing NAT and Java In-Reply-To: <71b79fa905091608497dcf5607@mail.gmail.com> References: <71b79fa905091608497dcf5607@mail.gmail.com> Message-ID: <432C7D97.3070005@speedymail.org> You could also create a Socket subclass that delegates to the TFTP implementation in the Jakarta Commons Net project. This doesn't do the hole punching for you per se, but all you really have to do is send a packet in each direction just prior to starting the TFTP transfer, ideally using some for of session offer/answer protocol like SIP. There is also a reliable UDP socket in LimeWire that subclasses Socket that you could extract pretty easily, but the transfer is a little sluggish. JXTA doesn't do any hole punching, unfortunately. It just relays everything through non-firewalled relays. It works and the API is elegant, but it's not very efficient. As a larger point, though, the hole punching itself is much less of an issue than knowing which holes to punch. The ICE draft lays out really the ideal way to do it, but it's complicated. A simpler version using SIP and SDP may be the way to go, or you may have some home-grown solution that lets peers know the holes to punch. Also keep in mind all the cases you have to deal with, such as symmetric NAT's. Relaying is required in some instances, so you could look at TURN or use JXTA. Again, ICE handles this very robustly, but it's not trivial, especially if you're implementing it from scratch. -Adam Davide Carboni wrote: >Hi, I'm looking for a good and reusable Java implementation of NAT >traversal solution like connection reversal, UDP/TCP hole punching >etc. >I'm aware that JXTA provides NAT traversal but I do not want to >redesign my applications with JXTA. I'm also aware of P2PSockets which >hides JXTA under the hood but I've tried it without success and the >project seems a little "inactive". > >Bye >Davide > > From gbildson at limepeer.com Sat Sep 17 21:29:15 2005 From: gbildson at limepeer.com (gbildson@limepeer.com) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] traversing NAT and Java In-Reply-To: <432C7D97.3070005@speedymail.org> References: <71b79fa905091608497dcf5607@mail.gmail.com> <432C7D97.3070005@speedymail.org> Message-ID: <1126992555.432c8aabb549a@cyrus.limewire.com> One note about using TFTP is that it can be very slow with the lock step acks. It can be a good starting point although you may end up with a more TCP like protocol if you get fancier. It would be nice to have a standard ICE implementation in Java. I was considering doing that myself at some distant time ... Thanks -greg Quoting Adam Fisk : > You could also create a Socket subclass that delegates to the TFTP > implementation in the Jakarta Commons Net project. This doesn't do the > hole punching for you per se, but all you really have to do is send a > packet in each direction just prior to starting the TFTP transfer, > ideally using some for of session offer/answer protocol like SIP. There > is also a reliable UDP socket in LimeWire that subclasses Socket that > you could extract pretty easily, but the transfer is a little sluggish. > > JXTA doesn't do any hole punching, unfortunately. It just relays > everything through non-firewalled relays. It works and the API is > elegant, but it's not very efficient. > > As a larger point, though, the hole punching itself is much less of an > issue than knowing which holes to punch. The ICE draft lays out really > the ideal way to do it, but it's complicated. A simpler version using > SIP and SDP may be the way to go, or you may have some home-grown > solution that lets peers know the holes to punch. > > Also keep in mind all the cases you have to deal with, such as symmetric > NAT's. Relaying is required in some instances, so you could look at > TURN or use JXTA. Again, ICE handles this very robustly, but it's not > trivial, especially if you're implementing it from scratch. > > -Adam > > > Davide Carboni wrote: > > >Hi, I'm looking for a good and reusable Java implementation of NAT > >traversal solution like connection reversal, UDP/TCP hole punching > >etc. > >I'm aware that JXTA provides NAT traversal but I do not want to > >redesign my applications with JXTA. I'm also aware of P2PSockets which > >hides JXTA under the hood but I've tried it without success and the > >project seems a little "inactive". > > > >Bye > >Davide > > > > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > From paul at ref.nmedia.net Sun Sep 18 06:48:56 2005 From: paul at ref.nmedia.net (Paul Campbell) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Idea for new class of anonymous algorithms Message-ID: <20050918064856.GA1008@ref.nmedia.net> I've been mulling over an idea for a new class of anonymous communication algorithms. Currently, they fall into one of two classes: First, there's the Dining Cryptographer's version. This one makes information theoretical security proofs about security but it has a huge downside. The downside is that roughly O(N^2) bits have to be exchanged to transmit a single bit between a group of N nodes. There's a variation recently proposed (although I forgot where the paper is) that reduces this down to O(N) bits for each bit. The difference is that in the original protocol, each node is expected to respond consistently to all neighbors. In the new version, it may broadcast something differently to different selected neighbors. Also, there's a factor of "2" in most of these protocols...that the most information that can be transmitted is asymptotically 2(N^2). However, the protocol breaks down to being essentially a communication protocol across a channel that is subject to interference. There are plenty of tree-based collision resolution algorithms that will decrease this down to about 1/0.6. The second group of algorithms is typical of the "mix nets". Here, the idea is that the packet follows a chain of proxies who rebroadcast it until the path through the proxies is sufficiently trustworthy and convoluted enough to avoid detection. Here, I'm half-thinking of a third possibility. Network codes all one to broadcast using O(N) packets (which is the best that can be done). Network codes can also be used to prevent intermediate nodes from being able to read the packet stream (so-called "weakly secure" coding). Combining these two means that network codes allow for a dining cryptographer-like solution except that the upper limits on coding rates should be O(N) packets to broadcast a single packet, which matches the intuitive "theoretical" minimum. Anyways...I'm still happy with the results from the non-theoretical world of mix nets. The overhead can be as small as a few nodes. There's no O(n ln x ) this and O(n ln x) that. From david at pjort.com Mon Sep 19 06:49:00 2005 From: david at pjort.com (David =?iso-8859-1?Q?G=F6thberg?=) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] 22C3 Call for Papers / Lecture Submission Deadline Message-ID: <6.1.2.0.1.20050919084808.02796eb0@pop.spray.se> Hi everyone. Here is a call for papers that might interest some of you. I visited this conference last year and it was great fun! I highly recommend it. Note that the conference is open to anyone from anywhere, no need to belong to academia etc. Some of the talks are on a very high level and some on a less high level. ************************************************** Call for Papers 22C3: Private Investigations 22nd Chaos Communication Congress Berliner Congress Center Berlin, Germany, 27/28/29/30 December 2005 http://www.ccc.de/congress/2005/ Overview ======== The 22nd Chaos Communication Congress (22C3) is a four-day conference on technology, society and utopia. The Congress offers lectures and workshops on a multitude of topics including (but not limited to) information technology, IT-security, internet, cryptography and generally a critical-creative attitude towards technology and the discussion about the effects of technological advances on society. The Chaos Communication Congress is the annual congress of the Chaos Computer Club e.V. (CCC). The Congress has established itself as the "European Hacker Conference" assembling people from all over the world. Even more important, the Congress is a great party bringing together the brightest heads of a variety of cultures and interests strengthening the idea of cross-culture inspiration and borderless networking. 22C3 is fun. Topics ====== The 22C3 conference program is roughly divided into five separate categories. These categories should serve as guidelines for your submissions and as a help to our visitors. However, it is not mandatory for your talk to exactly match the descriptions below. Anything that is interesting for our audience will be strongly considered. Hacking ------- The 'Hacking' domain addresses topics dealing with technology, concentrating on current research with high technical merit. Traditionally, the majority of all talks held tend to be drawn from this domain. Topics in this domain include but are in no way limited to: network and system security, programming, hardware hacking, applied cryptography and creative use of technology in an unintended way in general. Science ------- The 'Science' domain deals with current or future objects of scientific research that have the potential to radically change our lives, be it basic research or conducted for the industry. We are looking for talks and papers on the current state of the art in this domain, covering subjects such as nano technology, quantum computing, high frequency physics, bio technology, brain-computer interfaces, automated analysis of surveillance cctv, etc. Society ------- This domain deals with the state of society itself. Anything about politics and sociology with a relevance for hackers would go in here. Possible topics could include wire tapping laws, surveillance practices, net politics, intellectual property and copyright issues, effects of technology on kids etc. Culture ------- Dealing with cultural mind expansion, this domain adresses to the fringe of the technoverse. Examples would be Diskordianism, geek entertainment, video game culture, computer music, pixel art, and literature for hackers. Community --------- The Chaos Communication Congress invites developers, projects and activists to present themselves and their topics at the Congress. Developer groups are also encouraged to ask for support to hold smaller on-site developer conferences and meetings in the course of the Congress. Publication =========== The Chaos Communication Congress Proceedings are published on paper and online on the Internet. Only talks and presentations that were actually accepted will be published. Audio and video recordings of each lecture will be published online as well in various formats. All material will be available under the Creative Commons Attribution-NonCommercial-NoDerivs 2.0 Germany license allowing free non-commercial redistribution of the material as long as the original credit to authors and publishers is retained. Licence URI: http://creativecommons.org/licenses/by-nc-nd/2.0/de/ If you'd like to publish your work under a more liberal license, you should state this with your submission. See below for submission guidelines. Further Information =================== Since the Chaos Communication Congress is a non-profit oriented event, we are unfortunately unable to pay any fees. Help on travel expenses and accomodation is generally possible but is to be agreed upon after acceptance of the lecture/workshop. You can find the preliminary agenda at http://www.ccc.de/congress/2005/ You will find also information on the registration procedure for participants here. For further information and questions please feel free to contact 22c3-content (at) cccv (dot) de. Submissions =========== Language -------- We are looking for speakers who can give lectures and/or workshops in either English or German. 22C3 is an international event, so we have a certain focus on having a large share of talks in English, for the benefit of our growing number of international guests. However, the quality of the presentation should come first. If you feel insecure, received criticism from your audience before, or if you just fear that the value and understandibility of your talk might suffer, please choose German. Lecture Requirements -------------------- Lectures should not exceed 45 minutes plus up to 10 minutes for Q&A. Longer timeslots are possible if we feel the topic demands it (please remark on submission). Workshops should include a talk on the basic principles and a practical hands-on section. Submitters are asked to submit their application by e-mail at 22c3-content (at) cccv (dot) de Your application should include the following information: Required speaker information ---------------------------- * Name: Full name of speaker * Public Name: Name you'd like to see published at 22C3 * Other Names: nicknames, titles etc. * Contact information ** Primary E-Mail address ** Phone number(s) ** Instant messaging address(es) * Language of your talk * A photo, square format, min. 128x128 pixels (optional) * Public home page, weblog and other speaker-related websites * Short Info: Compact description of speaker (one sentence) * Bio: Biography or other background information speaker * Postal address * Bank information (for money transfer) * Expected day of arrival and departure Rest assured that your contact information and other private data will be kept both secure and private: our dislike for spammers and obnoxious privacy invaders is hard to put into words to say the least. If you already submitted a lecture for 22C3 or were a registered speaker of 21C3 please tell us so on each lecture submission. Lecture information ------------------- * Title: Name of event or lecture (max 40 letters) * Subtitle: Additional title description (a couple of words, optional) (max. 100 letters) * Abstract: An abstract of the event's content (max. 250 letters) * Description: A longer and detailed description of the event's content (250 to 500 words) * Please state if you are going to submit a paper to be included in the 22C3 Proceedings * Please state if you are going to use slides in your talk and in which format you are going to provide a copy * Duration of your talk * Links to background information on the talk * Target Group: Beginners, Advanced Users, Pros * Resources you need for your talk (Internet, digital projector, overhead projector, pens, assitant personal, presentation computer) * Related talks at 22C3 you know of (including previously submitted talks by yourself) * A lecture logo, square format, min. 128x128 pixels (optional) You can provide papers and slides on submission or at the given deadline the latest. Papers ------ Accepted speakers can optionally hand in a paper which will be published with an ISBN in the 22C3 Proceedings. Papers will be accepted in Portable Document Format (PDF) only and should be around 5 pages. The PDF file must not contain passwords or other restrictions. Paper size should be DIN A4 in portrait orientation. All margins must be set to at least 2 cm (0.78 inches). Pictures should be greyscale and up to 300dpi. Apart from that, you are free to use any layout you want. Slides ------ Accepted speakers are asked to hand in slides used in their talks. Please use a well-known format for your slides. Dates and deadlines =================== The deadline for submission is October 1st, 2005. Notification of acceptance will be sent by e-mail on November 1st, 2005 the latest. However, you may very well get your notification earlier than that if needed. Final papers or slides are due by December 1st, 2005. 1 October 2005: Submission due 1 November 2005: Final notification of acceptance (or earlier) 1 December 2005: Final papers/presentations due 27 December - 30 December 2005: Conference takes place From bryan.turner at pobox.com Mon Sep 19 17:04:44 2005 From: bryan.turner at pobox.com (Bryan Turner) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Idea for new class of anonymous algorithms In-Reply-To: <20050918064856.GA1008@ref.nmedia.net> Message-ID: <200509191704.j8JH4iT6003735@rtp-core-1.cisco.com> Paul, Please include your bibliography, I've read most of the anonymous communication literature but your descriptions aren't matching up with what I recall. If you haven't seen [1,2] already, you should definitely take a look. They are k-anonymous message channels, with decent overhead (typically O(k) messages per round). The problem seems to be, as with Dining Cryptographers, the nodes who constantly broadcast junk, or misbehave in the protocol. This never reveals the information or participants, but it can stop progress of the protocol. Both papers solve cheaters with some ad-hoc, suboptimal designs. I'm not sure I follow your discussion on Network Codes, are you referring to the new rateless coding schemes [3], or a different use of the term? If you're referring to [3], the intermediate nodes do have enough information (cryptographically speaking) to reveal information about the communication patterns. You might be able to merge Network Codes with Secure Multiparty Sums to get something cryptographically secure.. --Bryan bryan.turner@pobox.com [1] k-Anonymous Message Transmission Louis von Ahn, Andrew Bortz, Nicholas J. Hopper http://citeseer.ifi.unizh.ch/690390.html [2] A New k-Anonymous Message Transmission Protocol Gang Yao, Gengguo Feng http://dasan.sejong.ac.kr/~wisa04/ppt/9A2.pdf [3] Network Coding http://tesla.csl.uiuc.edu/~koetter/NWC/ -----Original Message----- From: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] On Behalf Of Paul Campbell Sent: Sunday, September 18, 2005 2:49 AM To: p2p-hackers@zgp.org Subject: [p2p-hackers] Idea for new class of anonymous algorithms I've been mulling over an idea for a new class of anonymous communication algorithms. Currently, they fall into one of two classes: First, there's the Dining Cryptographer's version. This one makes information theoretical security proofs about security but it has a huge downside. The downside is that roughly O(N^2) bits have to be exchanged to transmit a single bit between a group of N nodes. There's a variation recently proposed (although I forgot where the paper is) that reduces this down to O(N) bits for each bit. The difference is that in the original protocol, each node is expected to respond consistently to all neighbors. In the new version, it may broadcast something differently to different selected neighbors. Also, there's a factor of "2" in most of these protocols...that the most information that can be transmitted is asymptotically 2(N^2). However, the protocol breaks down to being essentially a communication protocol across a channel that is subject to interference. There are plenty of tree-based collision resolution algorithms that will decrease this down to about 1/0.6. The second group of algorithms is typical of the "mix nets". Here, the idea is that the packet follows a chain of proxies who rebroadcast it until the path through the proxies is sufficiently trustworthy and convoluted enough to avoid detection. Here, I'm half-thinking of a third possibility. Network codes all one to broadcast using O(N) packets (which is the best that can be done). Network codes can also be used to prevent intermediate nodes from being able to read the packet stream (so-called "weakly secure" coding). Combining these two means that network codes allow for a dining cryptographer-like solution except that the upper limits on coding rates should be O(N) packets to broadcast a single packet, which matches the intuitive "theoretical" minimum. Anyways...I'm still happy with the results from the non-theoretical world of mix nets. The overhead can be as small as a few nodes. There's no O(n ln x ) this and O(n ln x) that. From Bernard.Traversat at Sun.COM Tue Sep 20 00:36:28 2005 From: Bernard.Traversat at Sun.COM (Bernard Traversat) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] traversing NAT and Java In-Reply-To: <432C7D97.3070005@speedymail.org> References: <71b79fa905091608497dcf5607@mail.gmail.com> <432C7D97.3070005@speedymail.org> Message-ID: <432F598C.2090503@sun.com> Adam Fisk wrote: > You could also create a Socket subclass that delegates to the TFTP > implementation in the Jakarta Commons Net project. This doesn't do the > hole punching for you per se, but all you really have to do is send a > packet in each direction just prior to starting the TFTP transfer, > ideally using some for of session offer/answer protocol like SIP. There > is also a reliable UDP socket in LimeWire that subclasses Socket that > you could extract pretty easily, but the transfer is a little sluggish. > JXTA doesn't do any hole punching, unfortunately. An update as things keep moving, we are close to add a UDP hole punching transport to JXTA. This will target NAT/Firewall home user configuration. The UDP transport will be able to leverage the already existing reliable and secure JXTAsocket service to create secure and reliable communication channels over UDP as we do today for TCP/IP and HTTP. > As a larger point, though, the hole punching itself is much less of an > issue than knowing which holes to punch. The ICE draft lays out really > the ideal way to do it, but it's complicated. Agree. JXTA endpoint service, peer advertisements with their endpoint addresses and JXTA Rendezvous service provide some of the core infrastructure to implement ICE. Anybody interested to get an ICE implementation for JXTA let me know. > > Also keep in mind all the cases you have to deal with, such as symmetric > NAT's. Relaying is required in some instances, so you could look at > TURN or use JXTA. This is true as we are seeing some commercial P2P requirements where multi-hop routes will be required for supporting multi DMZ traversal even with ICE. Cheers, B. > Again, ICE handles this very robustly, but it's not > trivial, especially if you're implementing it from scratch. > > -Adam > > > Davide Carboni wrote: > >> Hi, I'm looking for a good and reusable Java implementation of NAT >> traversal solution like connection reversal, UDP/TCP hole punching >> etc. >> I'm aware that JXTA provides NAT traversal but I do not want to >> redesign my applications with JXTA. I'm also aware of P2PSockets which >> hides JXTA under the hood but I've tried it without success and the >> project seems a little "inactive". >> >> Bye >> Davide >> >> > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences -- --http://weblogs.java.net/blog/tra "As Java implies platform independence, and XML implies language independence, JXTA implies network independence." From afisk at speedymail.org Tue Sep 20 22:24:55 2005 From: afisk at speedymail.org (Adam Fisk) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] traversing NAT and Java In-Reply-To: <432F598C.2090503@sun.com> References: <71b79fa905091608497dcf5607@mail.gmail.com> <432C7D97.3070005@speedymail.org> <432F598C.2090503@sun.com> Message-ID: <43308C37.6040106@speedymail.org> > An update as things keep moving, we are close to add a UDP hole > punching transport to JXTA. This will target NAT/Firewall home user > configuration. The UDP transport will be able to leverage the > already existing reliable and secure JXTAsocket service to create > secure and reliable communication channels over UDP as we > do today for TCP/IP and HTTP. That's fantastic news Bernard! > Agree. JXTA endpoint service, peer advertisements with their endpoint > addresses > and JXTA Rendezvous service provide some of the core infrastructure to > implement ICE. Anybody interested to get an ICE implementation for JXTA > let me know. I would actually think that a built-in ICE implementation would be the ideal way for JXTA to negotiate a JxtaSocket behind the scenes, where JXTA would do the offer/answer and would discover the best transport to use. The potential latency involved is annoying, but there should be ways to optimize it. Another interesting piece that I think would be a great addition to JXTA is the TCP-hole-punching techniques described in papers such as the following: http://pdos.csail.mit.edu/papers/p2pnat.pdf http://www.andrew.cmu.edu/user/ggw/natblaster.pdf All the best, -Adam From sg266 at cornell.edu Wed Sep 21 01:34:09 2005 From: sg266 at cornell.edu (Saikat Guha) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] traversing NAT and Java In-Reply-To: <43308C37.6040106@speedymail.org> References: <71b79fa905091608497dcf5607@mail.gmail.com> <432C7D97.3070005@speedymail.org> <432F598C.2090503@sun.com> <43308C37.6040106@speedymail.org> Message-ID: <1127266449.2388.146.camel@himalaya.cs.cornell.edu> On Tue, 2005-09-20 at 18:24 -0400, Adam Fisk wrote: > Another interesting piece that I think would be a > great addition to JXTA is the TCP-hole-punching techniques described in > papers such as the following: > > http://pdos.csail.mit.edu/papers/p2pnat.pdf > http://www.andrew.cmu.edu/user/ggw/natblaster.pdf You may also want to look at: http://nutss.gforge.cis.cornell.edu/pub/imc05-tcpnat.pdf for some additional techniques, evaluation and other TCP NAT Traversal concerns. -- Saikat From Bernard.Traversat at Sun.COM Wed Sep 21 00:59:36 2005 From: Bernard.Traversat at Sun.COM (Bernard Traversat) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] traversing NAT and Java In-Reply-To: <43308C37.6040106@speedymail.org> References: <71b79fa905091608497dcf5607@mail.gmail.com> <432C7D97.3070005@speedymail.org> <432F598C.2090503@sun.com> <43308C37.6040106@speedymail.org> Message-ID: <4330B078.4010404@sun.com> Adam Fisk wrote: > Another interesting piece that I think would be a > great addition to JXTA is the TCP-hole-punching techniques described in > papers such as the following: > > http://pdos.csail.mit.edu/papers/p2pnat.pdf > http://www.andrew.cmu.edu/user/ggw/natblaster.pdf Thanks for the pointer, we will look at it. B. > > All the best, > > -Adam > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences -- --http://weblogs.java.net/blog/tra "As Java implies platform independence, and XML implies language independence, JXTA implies network independence." From marco.ceci at gmail.com Wed Sep 21 09:20:02 2005 From: marco.ceci at gmail.com (Marco Ceci) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] p2p query method and implementation Message-ID: <72f8a45d05092102202ab9061e@mail.gmail.com> Hi, I need to inquiry every node of the network for the presence of information, obviusly make this in a full mesh scheme would be impossible. The problem is, imho, simpler that file retrieving becouse i only need to know if an information exist on nodes and count the reply. Do you know any existent (even efficient) java implementation to make this? Thanks for you help best regards -- Marco Ceci From marco.ceci at gmail.com Wed Sep 21 10:08:31 2005 From: marco.ceci at gmail.com (Marco Ceci) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: p2p query method and implementation In-Reply-To: <72f8a45d05092102202ab9061e@mail.gmail.com> References: <72f8a45d05092102202ab9061e@mail.gmail.com> Message-ID: <72f8a45d05092103081b57d2a3@mail.gmail.com> I think i've miss one important detail: every node update his information presence list very often, maybe some type of caching, hierarchical structure could be required. Thank you again On 9/21/05, Marco Ceci wrote: > Hi, I need to inquiry every node of the network for the presence of > ... -- Marco Ceci From afisk at speedymail.org Wed Sep 21 16:02:28 2005 From: afisk at speedymail.org (Adam Fisk) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] traversing NAT and Java In-Reply-To: <4330B078.4010404@sun.com> References: <71b79fa905091608497dcf5607@mail.gmail.com> <432C7D97.3070005@speedymail.org> <432F598C.2090503@sun.com> <43308C37.6040106@speedymail.org> <4330B078.4010404@sun.com> Message-ID: <43318414.2000209@speedymail.org> The nice thing about particularly the first TCP approach is that it should be a good bit easier than the UDP hole-punching in terms of working it into the JXTA API. I guess you guys already have all your reliability code in there, so maybe not, but it's nice that you end up with a TCP socket that's just like any other. -Adam Bernard Traversat wrote: > Adam Fisk wrote: > >> Another interesting piece that I think would be a great addition to >> JXTA is the TCP-hole-punching techniques described in papers such as >> the following: >> >> http://pdos.csail.mit.edu/papers/p2pnat.pdf >> http://www.andrew.cmu.edu/user/ggw/natblaster.pdf > > Thanks for the pointer, we will look at it. > > B. > >> >> All the best, >> >> -Adam >> >> _______________________________________________ >> p2p-hackers mailing list >> p2p-hackers@zgp.org >> http://zgp.org/mailman/listinfo/p2p-hackers >> _______________________________________________ >> Here is a web page listing P2P Conferences: >> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > > > From macambira at gmail.com Wed Sep 21 16:07:49 2005 From: macambira at gmail.com (Tiago Macambira) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] p2p query method and implementation In-Reply-To: <72f8a45d05092102202ab9061e@mail.gmail.com> References: <72f8a45d05092102202ab9061e@mail.gmail.com> Message-ID: On 9/21/05, Marco Ceci wrote: > Hi, I need to inquiry every node of the network for the presence of > information, obviusly make this in a full mesh scheme would be > impossible. The problem is, imho, simpler that file retrieving becouse > i only need to know if an information exist on nodes and count the > reply. Do you know any existent (even efficient) java implementation > to make this? Perhaps a DHT is what you are looking for. There are java implementation of various DHT algorithms. I would look for a chord implementation if the datasets and nodes are fairly stable or for kademlia if both nodes and datasets are very volatiles. If the amount of information every node stores/returns to queries is huge or depends on some calculation a hierarchical scheme would be better... Just my 2 cents... Cheers. Tiago Alves Macambira From gd at acm.org Wed Sep 21 18:11:22 2005 From: gd at acm.org (=?iso-8859-15?Q?G=FCnter_Dannh=E4user?=) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Implementing distributed quadtree in Java Message-ID: <19910129566.20050921201122@acm.org> Hi p2p-hackers, I?m looking into what could be summarized as "Geographic service discovery". The idea is to use a p2p distributed directory to register and look up services which provide information about spatial regions. For efficient search, spatial information is hierarchically organized in a virtual, distributed quadtree/octree. This was already done in [1][2][3] and implemented based on the Open Peer-to-peer Network (C-) library OPeN [4] using the Chord protocol. Since this is part of a bigger project using Java, I'm now looking for suitable Java-P2P-middleware to base the implementation on. Any suggestions or recommendations would be very welcome, especially opinions on using Bamboo or JXTA (Accord?, JDHT?) for this. Thanks, G?nter. [1] Aaron Harwood and Egemen Tanin, Hashing spatial content over peer-to-peer networks, in Proceedings of the 2003 Australian Telecommunications, Networks and Applications Conference, CD-ROM, Melbourne, Australia, 2003 http://www.cs.mu.oz.au/~aharwood/online/HarwoodTanin-2003d.pdf [2] Egemen Tanin, Aaron Harwood, Hanan Samet, Sarana Nutanong, and Minh Truong, A serverless 3D world, in Proceedings of the Symposium on Advances in Geographic Information Systems - ACM GIS, pages 157-165, Arlington, VA, 2004 http://www.cs.mu.oz.au/~egemen/gis04.pdf [3] Egemen Tanin, Aaron Harwood, and Hanan Samet, A distributed quadtree index for peer-to-peer settings, in Proceedings of the International Conference on Data Engineering - ICDE, pages 254-255, Tokyo, Japan, 2005 http://www.cs.mu.oz.au/~egemen/icde05.pdf [4] http://p2p.cs.mu.oz.au/software/OPeN/ From ap at hamachi.cc Wed Sep 21 19:37:51 2005 From: ap at hamachi.cc (Alex Pankratov) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] traversing NAT and Java In-Reply-To: <4330B078.4010404@sun.com> References: <71b79fa905091608497dcf5607@mail.gmail.com> <432C7D97.3070005@speedymail.org> <432F598C.2090503@sun.com> <43308C37.6040106@speedymail.org> <4330B078.4010404@sun.com> Message-ID: <4331B68F.3040401@hamachi.cc> Bernard Traversat wrote: > Adam Fisk wrote: > >> Another interesting piece that I think would be a great addition to >> JXTA is the TCP-hole-punching techniques described in papers such as >> the following: >> >> http://pdos.csail.mit.edu/papers/p2pnat.pdf This is a good paper, but it makes few assumptions that undermine its practical adoption. For example - >>> Most NATs will not forward ICMP TTL Exceeded messages >>> back to an internal host This might be true for the devices they tested with, but I would be very careful with saying 'Most NATs'. I can name at least two major firewall vendors (that also produce consumer grade devices) that handle _all_ ICMP messages affecting TCP session state. I am very actively involved with Hamachi, which is VPN system based on UDP hole punching, and looking at stats collected so far I must say that NAT devices out there sometimes do things that make no sense at all. Basically for every assumption one makes in their paper there is a NAT device will break it. Not a big percentage per-assumption, but it all adds up. It takes a lot of effort to take UDP hole-punching framework from a prototype with 4:1 success rate (canonical 80%) to a usable system yielding a rate of 20:1 or higher. And you can safely double the amount of effort needed for TCP/hp. Just a heads up :) Alex From srhea at cs.berkeley.edu Wed Sep 21 23:09:17 2005 From: srhea at cs.berkeley.edu (Sean Rhea) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Implementing distributed quadtree in Java In-Reply-To: <19910129566.20050921201122@acm.org> References: <19910129566.20050921201122@acm.org> Message-ID: <22D78221-362F-43F3-AE1F-662C40CC7EBC@cs.berkeley.edu> Also check out, "A Case Study in Building Layered DHT Applications" by Yatin Chawathe et al.: http://www.acm.org/sigs/sigcomm/sigcomm2005/paper-ChaRam.pdf The code for this is available from the authors, and it works well from what they tell me. The performance should be a lot better now, too, as OpenDHT is much faster since they ran their evaluation. Sean On Sep 21, 2005, at 2:11 PM, G?nter Dannh?user wrote: > Hi p2p-hackers, > > I?m looking into what could be summarized as "Geographic service > discovery". > The idea is to use a p2p distributed directory to register and look up > services which provide information about spatial regions. > For efficient search, spatial information is hierarchically organized > in a virtual, distributed quadtree/octree. > This was already done in [1][2][3] and implemented based on the Open > Peer-to-peer Network (C-) library OPeN [4] using the Chord protocol. > > Since this is part of a bigger project using Java, I'm now looking for > suitable Java-P2P-middleware to base the implementation on. > > Any suggestions or recommendations would be very welcome, especially > opinions on using Bamboo or JXTA (Accord?, JDHT?) for this. > > Thanks, > G?nter. > > [1] Aaron Harwood and Egemen Tanin, Hashing spatial content over > peer-to-peer networks, in Proceedings of the 2003 Australian > Telecommunications, Networks and Applications Conference, CD-ROM, > Melbourne, Australia, 2003 > http://www.cs.mu.oz.au/~aharwood/online/HarwoodTanin-2003d.pdf > > [2] Egemen Tanin, Aaron Harwood, Hanan Samet, Sarana Nutanong, and > Minh Truong, A serverless 3D world, in Proceedings of the Symposium on > Advances in Geographic Information Systems - ACM GIS, pages 157-165, > Arlington, VA, 2004 > http://www.cs.mu.oz.au/~egemen/gis04.pdf > > [3] Egemen Tanin, Aaron Harwood, and Hanan Samet, A distributed > quadtree index for peer-to-peer settings, in Proceedings of the > International Conference on Data Engineering - ICDE, pages 254-255, > Tokyo, Japan, 2005 > http://www.cs.mu.oz.au/~egemen/icde05.pdf > > [4] http://p2p.cs.mu.oz.au/software/OPeN/ > > > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences -- Everyone chooses his or her own instrument for rebellion. I don't know what my son's will be, but my only hope for him is this: That by sharing my passions with him, I have planted the seeds of defiance that will someday be turned against me. -- Soo Lee Young -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://zgp.org/pipermail/p2p-hackers/attachments/20050921/2545d951/PGP.pgp From dominic.heutelbeck at fernuni-hagen.de Thu Sep 22 12:39:13 2005 From: dominic.heutelbeck at fernuni-hagen.de (Dominic Heutelbeck) Date: Sat Dec 9 22:13:02 2006 Subject: Distributed Space Partitioning Trees (was: Re: [p2p-hackers] Implementing distributed quadtree in Java) In-Reply-To: <19910129566.20050921201122@acm.org> References: <19910129566.20050921201122@acm.org> Message-ID: <4332A5F1.40103@fernuni-hagen.de> Hello. G?nter Dannh?user schrieb: > I?m looking into what could be summarized as "Geographic service > discovery". > The idea is to use a p2p distributed directory to register and look up > services which provide information about spatial regions. This is pretty exactly the topic I recently addressed in my PhD thesis [1] :-) In my thesis I use something more resemling CAN than an quadtree (however, it is significantly different from CAN). However in my early work I tried to use a quadtree like structure [2][3], however I found it to be notwell suided to problems where the distribution of hot-spots is as dynamic as when managing objects with mobile spatial locations. In my thesis I describe a new general use P2P data structure similar to DHTs, the so-called distributed space partitioning trees (DSPTs). The difference to DHTs is basically that instead of publishing (key,value) tuples you publish (location, object) tuples. Where the location can be an (almost) arbitrary subset of the n-dimensional search space. You can also update the locations (e.g., modeling mobile entities). Instead of supporting exact matches or simple range queries, the DSPT supports geometrical queries, e.g., "return all objects whose location itersect area A", where A can again be an (almost) arbitrary subset of the search space. > For efficient search, spatial information is hierarchically organized > in a virtual, distributed quadtree/octree. > This was already done in [1][2][3] and implemented based on the Open > Peer-to-peer Network (C-) library OPeN [4] using the Chord protocol. Thanks for these references. I didn't know them so far. > Since this is part of a bigger project using Java, I'm now looking for > suitable Java-P2P-middleware to base the implementation on. While I do have a Java prototype of my protocols, I would not consider it sable enough for general release. > Any suggestions or recommendations would be very welcome, especially > opinions on using Bamboo or JXTA (Accord?, JDHT?) for this. > Anyway, I would like to use the opportunity to present my work here on the mailinglist. If anybody of you would like to discuss this topic or DSPTs please go ahead. :-) Regards, Dominic [1] Dominic Heutelbeck (2005): PhD thesis Distributed Space Partitioning Trees and their Application in Mobile Computing. - Informatik Bericht 327 der Fernuniversit?t Hagen, Hagen, Germany. http://www.informatik.fernuni-hagen.de/ia/de/staff/heutelbeck/heutelbeck05thesis.pdf [2] Dominic Heutelbeck, Reinhold R?th, Claus Unger (2003): Fault Tolerant Geographical Addressing. In: Proc. of Innovative Internet Community Systems 2003 (I2CS'03), Leipzig, Germany. http://www.informatik.fernuni-hagen.de/ia/de/staff/heutelbeck/heutelbeck03fault_tolerant.pdf [3] Dominic Heutelbeck (2002): Context Spaces - Self-Structuring Distributed Networks for Contextual Messaging and Resource Discovery. In: Proc. of Tenth International Conference on Cooperative Information Systems 2002 (CoopIS'02), University of California Irvine, USA. http://www.informatik.fernuni-hagen.de/ia/de/staff/heutelbeck/heutelbeck02context.pdf -- --------------------------------------------------------------------------- FernUniversit?t Hagen Phone: +49-2331-987-2923 Dominic Heutelbeck Fax: +49-2331-987-4487 Universit?tsstrasse 1 Email: Dominic.Heutelbeck@fernuni-hagen.de D-58097 Hagen, Germany Web: http://www.informatik.fernuni-hagen.de/ia/ --------------------------------------------------------------------------- From aharwood at cs.mu.OZ.AU Fri Sep 23 00:42:32 2005 From: aharwood at cs.mu.OZ.AU (Aaron Harwood) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] p2p query method and implementation In-Reply-To: References: <72f8a45d05092102202ab9061e@mail.gmail.com> Message-ID: <94ED4523-922A-485E-9C25-160849A6EE88@cs.mu.oz.au> We are using a publish/subscribe system built on efficient P2P range query techniques on a DHT. It depends a lot on what kind of information that you are querying for and what index is usable for that information. --Aaron On 22/09/2005, at 2:07 AM, Tiago Macambira wrote: > On 9/21/05, Marco Ceci wrote: > >> Hi, I need to inquiry every node of the network for the presence of >> information, obviusly make this in a full mesh scheme would be >> impossible. The problem is, imho, simpler that file retrieving >> becouse >> i only need to know if an information exist on nodes and count the >> reply. Do you know any existent (even efficient) java implementation >> to make this? >> > > Perhaps a DHT is what you are looking for. There are java > implementation of various DHT algorithms. I would look for a chord > implementation if the datasets and nodes are fairly stable or for > kademlia if both nodes and datasets are very volatiles. > > If the amount of information every node stores/returns to queries is > huge or depends on some calculation a hierarchical scheme would be > better... From aharwood at cs.mu.OZ.AU Fri Sep 23 04:03:10 2005 From: aharwood at cs.mu.OZ.AU (Aaron Harwood) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] JDHT: Java Distributed Hash Table In-Reply-To: <4324A9B5.5030008@sics.se> References: <4324A9B5.5030008@sics.se> Message-ID: Hi Ali and others, JDHT looks nice. We have a similar development called OPeN in the C++ world. Some of the nice things about the OPeN architecture is that the service is distinct from the protocol. Thus we can mix and match different services with different protocols. In the OPeN architecture, using C++, we have this kind of interface: char *key,*data; OpenManager open_manager; open_manager.OpenInitialize(argc, argv, env ); DataService *data_service = (DataService*) open_manager.GetService ("DataService", "DefaultDHT"); . . . data_service->Put(key, data) . . . cout << data_service->Get(key) . . . data_service->Delete(key); A second node connects to the first node when the service is initialized in the second node. Actually IP/port of existing peer is given in a configuration file / environment variable / command line / application UI, etc. --Aaron http://p2p.cs.mu.oz.au/ On 12/09/2005, at 8:03 AM, Ali Ghodsi wrote: > > We have released a BETA version of JDHT (Java Distributed Hash > Table) which aims at simplifying DHT usage by providing the same > interface as the Java collections Map. > > For more information visit http://dks.sics.se/jdht/ > JDHT, uses DKS under the hood. Visit http://dks.sics.se for papers, > implementation, and docs on DKS. > > > /**** Example: stand alone peer ****/ > > DHT node1DHT = new JDHT(); > node1DHT.put("myKey", "Hello World!"); > > /**** Example: second peer connecting to the first ****/ > > JDHT node2DHT = new JDHT( node1DHT.getReference() ); > String helloString = (String) node2DHT.get("myKey"); > System.out.println(helloString); > > /******************************************/ > > Best regards, > Ali Ghodsi > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > From arma at mit.edu Fri Sep 23 07:09:25 2005 From: arma at mit.edu (Roger Dingledine) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] WPES 2005 call for participation: Nov 7, Washington DC Message-ID: <20050923070925.GT21241@localhost.localdomain> The program for WPES 2005 is out: http://wpes05.dti.unimi.it/program.html Workshop on Privacy in the Electronic Society (WPES) is a one-day workshop attached to the ACM CCS conference. We have papers for much of the day with breaks to discuss about privacy, technology, society, and how they interact. Many of the attendees stick around for a few days afterwards to keep discussing (and to see the CCS talks). If you're in the area (or if you arrange to be), we look forward to seeing you there. Note that the early registration deadline is Sept 26 (this Monday). Feel free to forward this mail anywhere else it should go. Thanks, --Roger Here are the accepted papers: The Pynchon Gate: A Secure Method of Pseudonymous Mail Retrieval Len Sassaman (K.U. Leuven ESAT-COSIC, Belgium), Bram Cohen (BitTorrent, USA), Nick Mathewson (The Free Haven Project, USA) Maintaining Privacy on Derived Objects Nicola Zannone (University of Trento, Italy), Sushil Jajodia (George Mason University, USA), Fabio Massacci (University of Trento, Italy), Duminda Wijesekera (George Mason University, USA) Protecting Privacy in Tabular Healthcare Data: Explicit Uncertainty for Disclosure Control Brian Shand, Jem Rashbass (University of Cambridge, UK) Disabling RFID Tags with Visible Confirmation: Clipped Tags Are Silenced Guenter Karjoth (IBM Research, Switzerland), Paul Moskowitz (IBM Research, USA) Privacy for RFID Through Trusted Computing David Molnar, Andrea Soppera, David Wagner (University of California, Berkeley, USA) Specifying Electronic Voting Protocols in Typed MSR Theodoros Balopoulos, Stephanos Gritzalis, Sokratis K. Katsikas (University of Aegean, Greece) Anonymous yet Accountable Access Control Michael Backes, Jan Camenisch, Dieter Sommer (IBM Zurich Research Lab, Switzerland) Determining User Privacy Preferences by Asking the Right Questions: An Automated Approach Keith Irwin, Ting Yu (North Carolina State University, USA) Mining Rule Semantics to Understand Legislative Compliance Travis D. Breaux and Annie I. Ant?n (North Carolina State University, USA) Quantitative Evaluation of Unlinkable ID Matching Schemes Yasunobu Nohara, Sozo Inoue, Kensuke Baba, Hiroto Yasuura (Kyushu University, Japan) Coercion-Resistant Electronic Elections Ari Juels (RSA Laboratories, USA), Dario Catalano (CNRS-Ecole Normale Sup?rieure, France), Markus Jakobsson (Indiana University, USA) Information Revelation and Privacy Issues in Social Networking Sites Ralph Gross, Alessandro Acquisti (Carnegie Mellon University, USA) The Security of "Off-the-Record" Messaging Mario Di Raimondo (Universit? di Catania, Italy), Rosario Gennaro, Hugo Krawczyk (IBM T.J. Watson Research Center, USA) Peripheral Privacy Notifications for Wireless Networks Braden Kowitz, Lorrie Cranor (Carnegie Mellon University, USA) The Privacy Cost of the Second-Chance Offer Sumit Joshi, Yu-An Sun, Poorvi L. Vora (George Washington University, USA) From marco.ceci at gmail.com Fri Sep 23 07:52:24 2005 From: marco.ceci at gmail.com (Marco Ceci) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] p2p query method and implementation In-Reply-To: <94ED4523-922A-485E-9C25-160849A6EE88@cs.mu.oz.au> References: <72f8a45d05092102202ab9061e@mail.gmail.com> <94ED4523-922A-485E-9C25-160849A6EE88@cs.mu.oz.au> Message-ID: <72f8a45d0509230052119bd9f@mail.gmail.com> On 9/23/05, Aaron Harwood wrote: > We are using a publish/subscribe system built on efficient > P2P range query techniques on a DHT. It depends a lot on what kind > of information that you are querying for and what index is usable > for that information. I'm working on a way to weight the relevance of a alert generated by multiple intrusion detection system. The information consist in a pair (src_ip, alert_id) and i want to check how many times the global system have seen this. Every peer can put in the system that information and every other peer would read that. The frequence of put and check operation is supposed to be hight but is not important to have access to the latest information. I'm following the idea coming from the paper "Towards Collaborative Security and P2P Intrusion Detection" wich use bloom filters for storing information on peer ad a semi-random way to choose peer from the set for joining information. Any suggestion would be very appreciated. Thank you all. best regards -- Marco Ceci From aloeser at cs.tu-berlin.de Fri Sep 23 12:08:07 2005 From: aloeser at cs.tu-berlin.de (=?ISO-8859-15?Q?Alexander_L=F6ser?=) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] complexity theory for small world / shortcut based routing approaches Message-ID: <4333F027.7090003@cs.tu-berlin.de> Hi, DHT approaches provide a framework for deterministic query execution. E.g. most algorithm require O(logN) messages to route a query within a network of N peers. Nowadays a new group of query routing algorithms [1,2,3,4] is evolving. Based on an unstructured p2p network these algorithms cluster nodes with similar interests. In such algorithms each node recieving the message forwards it to one if its neighbors until the target is found or the maxium radius is reached. Based on the decision criteria they use in selecting a forwarding node, these algorithms can be categorized as follows: * Degree based: The decision is based on the degree structure of neighboring nodes. E.g. Nodes with a high in and out degree are chosen. * Simiraity based: The decision is based on how similar the neighboring nodes are to the target node in terms of attribute values. E.g., nodes having asked similar queries or providing similar content are chosen. Using such navigation the overlays often have small world properties, e.g. a small average path length, a high clustering coefficient. However, calculating the algorithm complexity for such overlays is still a hard problem. Does anybody know (academic or emperical) work, on how to estimate the algorithm complexity, e.g. O(N?), O(log N), O(N) .... ? Best's Alex [1] V. Cholvi, P. Felber, and E.W. Biersack. Efficient Search in Unstructured Peer-to-Peer Networks. [2] Adriana Iamnitchi, Matei Ripeanu and Ian Foster, Small World File Sharing Communities. [3] K. Sripanidkulchai. B. Maggs. H. Zhang. Efficient Content Location Using Interest Based Locality in Peer-to-Peer Systems [4] Searching dynamic communities with personal indexes. A. L?ser, C Tempich, B. Quilitz, W.T. Balke, S. Staab and W. Nejdl, ISWC '05 -- ___________________________________________________________ Alexander L?ser Technische Universit?t Berlin http://cis.cs.tu-berlin.de/~aloeser/ office : +49- 30-314-25551 fax : +49- 30-314-21601 ___________________________________________________________ From wolfgang.mueller at wiai.uni-bamberg.de Fri Sep 23 14:36:01 2005 From: wolfgang.mueller at wiai.uni-bamberg.de (Wolfgang Mueller) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] complexity theory for small world / shortcut based routing approaches In-Reply-To: <4333F027.7090003@cs.tu-berlin.de> References: <4333F027.7090003@cs.tu-berlin.de> Message-ID: <20050923143601.GA9182@portos.uni-bamberg.de> Dear Alex, I am not sure if you are aware of that work: http://cantor.ee.ucla.edu/~networks/papers.html This work assumes a power law degree distribution, however it assumes an otherwise random network. I am not aware of work especially geared towards power-law networks in which the assignment of edges is influenced by the content. Anyone aware of such work? Best regards, Wolfgang M?ller Dr. Wolfgang Mueller LS Medieninformatik Universitaet Bamberg From aharwood at cs.mu.oz.au Sat Sep 24 02:48:53 2005 From: aharwood at cs.mu.oz.au (Aaron Harwood) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Implementing distributed quadtree in Java In-Reply-To: <19910129566.20050921201122@acm.org> References: <19910129566.20050921201122@acm.org> Message-ID: <082BE573-781B-4347-A9E4-2C7ACED8FE28@cs.mu.oz.au> On 22/09/2005, at 4:11 AM, G?nter Dannh?user wrote: > I'm looking into what could be summarized as "Geographic service > discovery". > The idea is to use a p2p distributed directory to register and look up > services which provide information about spatial regions. > For efficient search, spatial information is hierarchically organized > in a virtual, distributed quadtree/octree. > This was already done in [1][2][3] and implemented based on the Open > Peer-to-peer Network (C-) library OPeN [4] using the Chord protocol. > > Since this is part of a bigger project using Java, I'm now looking for > suitable Java-P2P-middleware to base the implementation on. > In the past, we have implemented a Java application on top of our C++ peer service by using JNI at the service interface. Another thing to consider, though it doesn't really help you for a pure Java implementation, is that our OPeN library allows Chord to be replaced with other protocols, while a p2p distributed directory service implementation at the higher layer would remain unchanged. --Aaron From enzomich at gmail.com Sat Sep 24 23:49:15 2005 From: enzomich at gmail.com (Enzo Michelangeli) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] traversing NAT and Java References: <71b79fa905091608497dcf5607@mail.gmail.com> <432C7D97.3070005@speedymail.org><432F598C.2090503@sun.com> <43308C37.6040106@speedymail.org> Message-ID: <00d101c5c163$d1c68aa0$0200a8c0@em.noip.com> ----- Original Message ----- From: "Adam Fisk" To: "Peer-to-peer development." Sent: Wednesday, September 21, 2005 6:24 AM Subject: Re: [p2p-hackers] traversing NAT and Java > I would actually think that a built-in ICE implementation would be the > ideal way for JXTA to negotiate a JxtaSocket behind the scenes, where > JXTA would do the offer/answer and would discover the best transport to > use. Talking about which: are there free implementations of ICE? Enzo From sg266 at cornell.edu Sun Sep 25 14:29:02 2005 From: sg266 at cornell.edu (Saikat Guha) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Announcing: TCP NAT Traversal Library Message-ID: <1127658542.3061.59.camel@himalaya.cs.cornell.edu> Hi all, (apologies if you get multiple copies of this) I'd like to announce the availability of a open-source TCP NAT Traversal/Hole-Punching library based on our research published in [1]. [1] "Characterization and Measurement of TCP Traversal through NATs and Firewalls", S. Guha and P. Francis. IMC 2005. http://nutss.net/pub/imc05-tcpnat.pdf The key result of the paper is: TCP NAT Traversal can work 85%-90% of the time today (without any special assumptions on NATs), and 100% of the time between pairs of certain well-behaved NATs. See [1] for more details. An open-source Java library for TCP NAT Traversal is now available: binaries: http://nutss.net/stunt.jar source: http://nutss.net/jstunt_cvs.php documentation for library users: http://nutss.net/jstunt/doc documentation for library developers: http://nutss.net/jstunt/devdoc The above library has been tested for pair-wise connectivity across 11 brands of NATs from Windows and Linux hosts. NATs tested were Linksys, DLink, Netgear, Belkin, 3Com, Netopia, Allied Telesyn, SMC, Trendnet, USR. Out of the 121 possible pair-wise combinations, 113 connections are successful. The only ones that failed are when both the endpoints are behind the _same_ NAT that does not support TCP hairpin-behavior yet (see [1]). The java library is released under LGPL; contact me if this does not meet your needs. Feel free to extend it/port it etc. For P2P developers and users of the library -------------------------------------------- The JAR file above includes a sample echo-server/client program. The source code for the sample applications is at http://nutss.net/EchoServer.java and http://nutss.net/EchoClient.java respectively. To start the server, on a host behind a NAT, execute: java -cp stunt.jar EchoServer you@your.domain.com To connect to your echoserver from a client, execute: java -cp stunt.jar EchoClient you@your.domain.com (if you cannot run your own echo server, try pointing the client to the one I am running at echo@nutss.net) If everything goes well, you'll see something along the lines of: On Server: Accepted saikat930@ed.u.cs.cornell.edu On Client: Greetings saikat930@ed.u.cs.cornell.edu, this is the EchoServer at echo@nutss.net. Now you say something. On the client, you'll be able to type lines and have it be echoed by the server when you press enter. The library takes between 200ms to 1 second to connect, but slow DNS (sometimes due to the NAT) can increase the connection time by a bit. For people who want to develop the library further -------------------------------------------------- Anonymous CVS access is at: cvs -d :pserver:anonymous@gforge.cis.cornell.edu:/cvsroot/nutss checkout stunt_java The TCP hole-punching code and state machine are in the file STUNTCont.java. Extensive online documentation of all internal functions is provided. Things to do: - Wrap a java Sockets or Channels api around the existing API - Port to C, Python etc (may require writing a separate STUNT server) - Implement SIP backend for signaling messages (framework glue in place) - Implement some ICE-type negotiations for stacks of NATs - Integrate UDP hole-punching If you have any questions, comments, suggestions, or problems, do not hesitate to contact me. Cheers and have fun. -- Saikat From shudo at computer.org Sun Sep 25 13:36:21 2005 From: shudo at computer.org (shudo@computer.org) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Errata in the Chord paper ? Message-ID: <20050925.223621.343189907.shudo@aist.go.jp> Hi all, I have some doubt about a pseudocode in the Chord paper: I. Stoica, R. Morris, D. Karger, M. F. Kaashoek and H. Balakrishnan, Chord: A Scalable Peer-to-peer Lookup Protocol for Internet Applications. In Proc. ACM SIGCOMM 2001, pp.149-160, Aug. 2001. It is a pseudocode for a basic node join operation in the Fig. 6. The paper presents another join algorithm in the Fig. 7 and the latter one will be implemented usually because the basic algorithm in Fig. 6 is not tolerate concurrent joins (and failures). So, it's not a serious problem at all if there are errata in the basic algorithm. And, of course, my interpretation of the paper is not correct. Please give me a comment if you know the same possible errata have been pointed out somewhere before. (1) A finger table entry (finger[k].node) can be the node itself which have the finger table. But, the pseudocode in Fig. 6 does not allow it. A finger table is initialized by a procedure init_finger_table(). The init_finger_table() calls find_successor() on an existing node n'. When the init_finger_table() is called on a joining node, no existing node knows the joining node because existing nodes can notice the joining node only by update_others() called just after init_finger_table(). As a result, a joining node cannot have itself in its finger table, even if the joining node itself is the most appropriate node for a finger table entry. (2) A joining node calls find_predecessor(n - 2^(i-1)) in a procedure update_finger_others() to find out nodes which should have the joining node in their finger tables. The argument for the call is not correct and it should be find_predecessor(n - 2^(i-1) + 1). Let's consider the case of i = 1. Suppose n is the node ID of the joining node. In this case, the joining node should find out a node whose ID is n-1 (of course, mod 2^m) or prior to n-1. But find_predecessor(n - 2^(i-1)) results in n-2 or prior to n-2 because a procedure find_predecessor(id) in Fig. 4 returns id-1 or prior to id-1. I appreciate any comment. Cheers, Kazuyuki Shudo Grid Technology Research Center National Institute of Advanced Industrial Science and Technology (AIST) From gd at acm.org Sun Sep 25 17:01:35 2005 From: gd at acm.org (=?iso-8859-15?Q?G=FCnter_Dannh=E4user?=) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Implementing distributed quadtree in Java In-Reply-To: <082BE573-781B-4347-A9E4-2C7ACED8FE28@cs.mu.oz.au> References: <19910129566.20050921201122@acm.org> <22D78221-362F-43F3-AE1F-662C40CC7EBC@cs.berkeley.edu> <4332A5F1.40103@fernuni-hagen.de> <082BE573-781B-4347-A9E4-2C7ACED8FE28@cs.mu.oz.au> Message-ID: <1602071600.20050925190135@acm.org> Aaron Harwood wrote: > Another thing to consider, though it doesn't really help you for a > pure Java implementation, is that our OPeN library allows Chord to > be replaced with other protocols, while a p2p distributed directory > service implementation at the higher layer would remain unchanged. Sean Rhea wrote: > Also check out, "A Case Study in Building Layered DHT Applications" > by Yatin Chawathe et al.: > > http://www.acm.org/sigs/sigcomm/sigcomm2005/paper-ChaRam.pdf > > The code for this is available from the authors, and it works well > from what they tell me. The performance should be a lot better now, > too, as OpenDHT is much faster since they ran their evaluation. I didn't know that one, thanks for the reference. Though, contrary to their scenario, we're interested in point-queries on objects with spatial extent, the ideas of Prefix Hash Trees and space-filling-curves for indexing could certainly be adapted. Moreover, using an abstraction layer like the OPeN lib or completely decoupling deployment and management of the underlying DHT, using e.g. OpenDHT, is quite appealing - I'm now thinking of evaluating an iterative variant of the mentioned distributed quadtree algorithm using the OpenDHT service (someone mentioned improved performance? ;-) Dominic Heutelbeck wrote: > However in my early work I tried to use a quadtree like structure [2][3], > however I found it to be notwell suided to problems where the distribution of > hot-spots is as dynamic as when managing objects with mobile spatial locations. Service-offers and their locations are expected to be not very dynamic in our case. Load-balancing could become a more limiting problem with hot-spots, when limiting the depth of the tree becomes necessary, especially for an iterative solution. Thanks+cheers, G?nter. From afisk at speedymail.org Mon Sep 26 04:13:46 2005 From: afisk at speedymail.org (Adam Fisk) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] traversing NAT and Java In-Reply-To: <00d101c5c163$d1c68aa0$0200a8c0@em.noip.com> References: <71b79fa905091608497dcf5607@mail.gmail.com> <432C7D97.3070005@speedymail.org><432F598C.2090503@sun.com> <43308C37.6040106@speedymail.org> <00d101c5c163$d1c68aa0$0200a8c0@em.noip.com> Message-ID: <4337757A.4030501@speedymail.org> There likely are free implementations at places like SipFoundry.org, but I'm not sure, and it would take a little digging since it tends to be coupled with other SIP pieces. ICE is also still an evolving IETF draft, so it's not likely that any open source/free versions are yet compliant with the most recent update from late July, but someone might be particularly diligent. I will eventually be releasing an open source implementation, but the first iterations will likely not be fully compliant just because it's a bit of a pain in the *** to code... -Adam Enzo Michelangeli wrote: >----- Original Message ----- >From: "Adam Fisk" >To: "Peer-to-peer development." >Sent: Wednesday, September 21, 2005 6:24 AM >Subject: Re: [p2p-hackers] traversing NAT and Java > > > >>I would actually think that a built-in ICE implementation would be the >>ideal way for JXTA to negotiate a JxtaSocket behind the scenes, where >>JXTA would do the offer/answer and would discover the best transport to >>use. >> >> > >Talking about which: are there free implementations of ICE? > >Enzo > >_______________________________________________ >p2p-hackers mailing list >p2p-hackers@zgp.org >http://zgp.org/mailman/listinfo/p2p-hackers >_______________________________________________ >Here is a web page listing P2P Conferences: >http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > > > From bryan.turner at pobox.com Mon Sep 26 17:39:02 2005 From: bryan.turner at pobox.com (Bryan Turner) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Errata in the Chord paper ? In-Reply-To: <20050925.223621.343189907.shudo@aist.go.jp> Message-ID: <200509261739.j8QHd2T6008805@rtp-core-1.cisco.com> Kazuyuki, I have implemented a (slightly modified) version of Chord, here is how I solved the problems you mention: > A finger table entry (finger[k].node) can be the node itself which > have the finger table. But, the pseudocode in Fig. 6 does not allow it. I chose not to allow a node in its own finger table. Early testing was finding routing loops in the network, and this was exacerbating them, causing the TTL to eventually drop the packet. I restructured the code to always check the local cache before forwarding the packets. This fixed the routing loops, and improved some search paths, but added extra burden to the CPU load. The correct fix for this is to not use the Chord protocol as specified (see discussion below). My initialization scheme was much simpler than Chord, I contact the successor node, steal his routing table, and allow the stabilization protocol to fix it over time. > A joining node calls find_predecessor(n - 2^(i-1)) in a procedure > update_finger_others() to find out nodes which should have the joining > node in their finger tables. The argument for the call is not correct > and it should be find_predecessor(n - 2^(i-1) + 1). I believe you are correct. I did not implement this part of the update procedure (see discussion below). In practice, this would never be noticed. The chance of two nodes being immediately adjacent in the ring are exceptionally small (1 in 2^160 if using SHA-1). Also, nodes with adjacent IDs would cause one arc of the ring to be effectively useless (the predecessor would only be able to store one ID!!). It is more valuable to use their "virtual node" technique to select several points on the ring to balance the burden more evenly among the nodes. This also effectively minimizes the chance of two nodes being adjacent. Problems with Chord ------------------- The Chord protocol (as specified in the referenced paper) has a few issues which make it a poor choice for DHT implementations in the real world: - Too Rigid Chord specifies that you must maintain exact successor/predecessor and finger tables at all times. This requirement causes a significant amount of network overhead during network flapping events, and at times of the day when large swings in population occur. Often, this traffic is enough to destabilize the network and cause routing to fail. Chord is one of many algorithms to build a Small World network. It has been shown that perfectly aligned finger tables are not necessary (as some of their later papers mention). Instead, keep tabs on your local neighborhood and keep "some" long-range pointers for efficient routing. Specifically, your long range pointers should connect to well established, well connected, well trusted nodes at various distant locations on the ring. - Inefficient use of connections Most implementations in the real world will be using TCP. Chord does not make use of the two-way connections. This becomes a significant overhead on the operating system. Kademlia suggests (and I concur) that Chord should use both forward and backward fingers, routing using a greedy strategy over all incoming and outgoing connections. (see below for my rant on greedy routing) - "Unstable" New nodes are considered immediately more valuable in their local region than established, existing nodes. This quickly leads to destabilization in the network when a network flap or node migration occurs. Kademlia suggests (and again, I concur) that established nodes should be the preferred routing fabric, and only once you have reached a local neighborhood should the find-closest-predecessor method begin. One metric for "establishment" is up-time. This is as good as any, but you cannot rely on a node to give you this information, instead keep track of when you first notice a node in the network and use this as your up-time metric. Thus, you route to the closest node in that region with the highest up-time. - Route Scheduling It is a common mistake with (as far as I am aware) all existing DHT implementations to expect greedy routing to work most of the time. If I can offer one take home message it is this: Use probabilistic routing! Make a short list of good candidates, weigh them by their up time, number of neighbors, and/or some trust metric, then probabilistically choose one. Your network and your users will thank you. All major enterprise applications (Load balancing of web servers, DNS, Jini, Web Sphere, etc..) have probabilistic redirectors. This ensures your network continues functioning even during network migrations and flapping. Hope this helps, --Bryan bryan.turner@pobox.com -----Original Message----- From: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] On Behalf Of shudo@computer.org Sent: Sunday, September 25, 2005 9:36 AM To: p2p-hackers@zgp.org Subject: [p2p-hackers] Errata in the Chord paper ? Hi all, I have some doubt about a pseudocode in the Chord paper: I. Stoica, R. Morris, D. Karger, M. F. Kaashoek and H. Balakrishnan, Chord: A Scalable Peer-to-peer Lookup Protocol for Internet Applications. In Proc. ACM SIGCOMM 2001, pp.149-160, Aug. 2001. It is a pseudocode for a basic node join operation in the Fig. 6. The paper presents another join algorithm in the Fig. 7 and the latter one will be implemented usually because the basic algorithm in Fig. 6 is not tolerate concurrent joins (and failures). So, it's not a serious problem at all if there are errata in the basic algorithm. And, of course, my interpretation of the paper is not correct. Please give me a comment if you know the same possible errata have been pointed out somewhere before. (1) A finger table entry (finger[k].node) can be the node itself which have the finger table. But, the pseudocode in Fig. 6 does not allow it. A finger table is initialized by a procedure init_finger_table(). The init_finger_table() calls find_successor() on an existing node n'. When the init_finger_table() is called on a joining node, no existing node knows the joining node because existing nodes can notice the joining node only by update_others() called just after init_finger_table(). As a result, a joining node cannot have itself in its finger table, even if the joining node itself is the most appropriate node for a finger table entry. (2) A joining node calls find_predecessor(n - 2^(i-1)) in a procedure update_finger_others() to find out nodes which should have the joining node in their finger tables. The argument for the call is not correct and it should be find_predecessor(n - 2^(i-1) + 1). Let's consider the case of i = 1. Suppose n is the node ID of the joining node. In this case, the joining node should find out a node whose ID is n-1 (of course, mod 2^m) or prior to n-1. But find_predecessor(n - 2^(i-1)) results in n-2 or prior to n-2 because a procedure find_predecessor(id) in Fig. 4 returns id-1 or prior to id-1. I appreciate any comment. Cheers, Kazuyuki Shudo Grid Technology Research Center National Institute of Advanced Industrial Science and Technology (AIST) From mgp at ucla.edu Mon Sep 26 18:26:27 2005 From: mgp at ucla.edu (Michael Parker) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Errata in the Chord paper ? In-Reply-To: <200509261739.j8QHd2T6008805@rtp-core-1.cisco.com> References: <200509261739.j8QHd2T6008805@rtp-core-1.cisco.com> Message-ID: <20050926112627.jw6upqh8ys8g4k40@mail.ucla.edu> Just to chime in quickly on the single point below... In Chord, you can easily do away with the rigidity by allowing the first finger to point to any node in the range [2^159, 2^160), not just the first node encountered clockwise in this range. Likewise, the second finger can point to any node in the range [2^158, 2^159), and so forth. This affords you much greater flexibility for choosing neighbors (indeed, your farthest finger can be chosen from approximately half the nodes on the network), allowing you to exploit some sort of proximity metric such as Vivaldi in neighbor selection. The average number of hops when routing a message still remains (log N)/2. See the paper "The Impact of DHT Routing Geometry on Resilience and Proximity" by Gummadi et al for a more detailed discussion on this. Everything else Bryan says is good advice :) - Mike Parker Quoting Bryan Turner : > - Too Rigid > > Chord specifies that you must maintain exact successor/predecessor > and finger tables at all times. This requirement causes a significant > amount of network overhead during network flapping events, and at times of > the day when large swings in population occur. Often, this traffic is > enough to destabilize the network and cause routing to fail. > > Chord is one of many algorithms to build a Small World network. It > has been shown that perfectly aligned finger tables are not necessary (as > some of their later papers mention). Instead, keep tabs on your local > neighborhood and keep "some" long-range pointers for efficient routing. > > Specifically, your long range pointers should connect to well > established, well connected, well trusted nodes at various distant locations > on the ring. From bryan.turner at pobox.com Mon Sep 26 19:29:42 2005 From: bryan.turner at pobox.com (Bryan Turner) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Errata in the Chord paper ? In-Reply-To: <20050926112627.jw6upqh8ys8g4k40@mail.ucla.edu> Message-ID: <200509261929.j8QJTgT6012572@rtp-core-1.cisco.com> Mike, That's a very succinct way of describing it, thanks for the input! Bibliography: ------------- Vivaldi: A Decentralized Network Coordinate System Frank Dabek, Russ Cox, Frans Kaashoek, Robert Morris http://pdos.csail.mit.edu/~rtm/papers/p426-dabek.pdf This is an updated version of the original Vivaldi paper. If you read the first one, read this one too! Novel Architectures for P2P Applications: The Continuous Discrete Approach Moni Naor, Udi Wieder http://www.wisdom.weizmann.ac.il/~naor/PAPERS/dh.pdf My preferred DHT construction. Very flexible, stable, and simple. Ignore sections 2.2-2.4, 3, 4.1, 5.2 and 6. Pay particular attention to section 5.1. Chord is a restricted 1-dimensional instance of this construction. Also look up their other P2P papers, all very good ideas. Kademlia: A Peer-to-peer Information System Based on the XOR Metric Petar Maymounkov, David Mazieres http://citeseer.ist.psu.edu/529075.html An almost trivial extension to Chord that solves some of the problems I mentioned. Forego the XOR metric, it is actually irrelevant to the paper's results and causes some pedagogical head-scratching when trying to work out the path of queries. Symphony: Distributed Hashing in a Small World Gyrneet Singh Manku, Mayank Bawa, Prabhakar Raghavan http://citeseer.ist.psu.edu/manku03symphony.html I recalled incorrectly, it was this paper that had the suggestions for Chord, not the Kademlia paper. An overview of Small World network construction, with some useful results and applications to other networks. See comments for Chord in section 5.4(b). --Bryan bryan.turner@pobox.com -----Original Message----- From: p2p-hackers-bounces@zgp.org [mailto:p2p-hackers-bounces@zgp.org] On Behalf Of Michael Parker Sent: Monday, September 26, 2005 2:26 PM To: p2p-hackers@zgp.org Subject: RE: [p2p-hackers] Errata in the Chord paper ? Just to chime in quickly on the single point below... In Chord, you can easily do away with the rigidity by allowing the first finger to point to any node in the range [2^159, 2^160), not just the first node encountered clockwise in this range. Likewise, the second finger can point to any node in the range [2^158, 2^159), and so forth. This affords you much greater flexibility for choosing neighbors (indeed, your farthest finger can be chosen from approximately half the nodes on the network), allowing you to exploit some sort of proximity metric such as Vivaldi in neighbor selection. The average number of hops when routing a message still remains (log N)/2. See the paper "The Impact of DHT Routing Geometry on Resilience and Proximity" by Gummadi et al for a more detailed discussion on this. Everything else Bryan says is good advice :) - Mike Parker Quoting Bryan Turner : > - Too Rigid > > Chord specifies that you must maintain exact successor/predecessor > and finger tables at all times. This requirement causes a significant > amount of network overhead during network flapping events, and at > times of the day when large swings in population occur. Often, this > traffic is enough to destabilize the network and cause routing to fail. > > Chord is one of many algorithms to build a Small World network. It > has been shown that perfectly aligned finger tables are not necessary > (as some of their later papers mention). Instead, keep tabs on your > local neighborhood and keep "some" long-range pointers for efficient routing. > > Specifically, your long range pointers should connect to well > established, well connected, well trusted nodes at various distant > locations on the ring. _______________________________________________ p2p-hackers mailing list p2p-hackers@zgp.org http://zgp.org/mailman/listinfo/p2p-hackers _______________________________________________ Here is a web page listing P2P Conferences: http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From lgonze at panix.com Mon Sep 26 22:46:41 2005 From: lgonze at panix.com (Lucas Gonze) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <5691356f0509261451a3da329@mail.gmail.com> References: <433862ED.6050108@panix.com> <433866e0.78f3de53.5aa9.53bcSMTPIN_ADDED@mx.gmail.com> <5691356f0509261451a3da329@mail.gmail.com> Message-ID: <43387A51.8040105@panix.com> p2p-hackers, meet rest-discuss. rest-discuss, I'd like to introduce you to p2p-hackers. RESTafarians: there is a long-running conversation on p2p-hackers about friendnets, also known as darknets, small world networks, and F2F networks; also capabilities security, sometimes known as smart contracts. An example thread begins at http://zgp.org/pipermail/p2p-hackers/2005-August/002915.html p2p-hackers: Tyler Close' method for HTTP access control using nothing but unguessable (and secret) URIs came up on REST-discuss. That thread begins at http://groups.yahoo.com/group/rest-discuss/message/5228 In the context of friendnets, Tyler's scheme is a beautifully simple way of controlling access using nothing but low-tech means. Not only does it limit access to trusted parties, it also allows for transitive relationships. (Warning: his scheme is counterintuitive, since the dependence on secret URLs smells like security through obscurity). I don't mean to create a permathread out of out of this cc:list, rather to refer interested rest-discuss people to p2p-hackers and vice versa. If you reply to this, please choose one list or the other as the recipient. I've always been surprised that the intersection of REST and capabilities isn't a common theme, since both REST and capabilities revolve around precisely defined relationships. The success of defining HTTP methods in terms of privileges is due, I believe, to the way that it reflects the same underlying truth that capabilities formalize. From nlothian at educationau.edu.au Tue Sep 27 01:35:31 2005 From: nlothian at educationau.edu.au (Nick Lothian) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization Message-ID: > > p2p-hackers, meet rest-discuss. rest-discuss, I'd like to > introduce you to p2p-hackers. > > RESTafarians: there is a long-running conversation on > p2p-hackers about friendnets, also known as darknets, small > world networks, and F2F networks; also capabilities security, > sometimes known as smart contracts. An example thread begins > at http://zgp.org/pipermail/p2p-hackers/2005-August/002915.html > > p2p-hackers: Tyler Close' method for HTTP access control > using nothing but unguessable (and secret) URIs came up on > REST-discuss. That thread begins at > http://groups.yahoo.com/group/rest-discuss/message/5228 In > the context of friendnets, Tyler's scheme is a beautifully > simple way of controlling access using nothing but low-tech > means. Not only does it limit access to trusted parties, it > also allows for transitive relationships. (Warning: his > scheme is counterintuitive, since the dependence on secret > URLs smells like security through obscurity). > Interesting idea. It may not be security via obscurity, but it does appear to ignore a number of practical considerations. For instance, what about the secret URL being passed on in referrer headers to other pages? I think some browsers block it when you go from a secure page to a non-secure page on another site (although I'm unsure about that). The argument that users shouldn't put links to on a secured page is more surprising than the things it is trying to avoid (to me anyway). OTOH, all browsers block HTTP authenticaion credentials from being passed in the referrer header. Nick From tyler.close at gmail.com Tue Sep 27 02:33:42 2005 From: tyler.close at gmail.com (Tyler Close) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: References: Message-ID: <5691356f050926193344350482@mail.gmail.com> Hi Nick, On 9/26/05, Nick Lothian wrote: > Interesting idea. Thanks. > It may not be security via obscurity, but it does appear to ignore a > number of practical considerations. > > For instance, what about the secret URL being passed on in referrer > headers to other pages? I think some browsers block it when you go from > a secure page to a non-secure page on another site (although I'm unsure > about that). Fortunately, browsers don't send a Referer header field when the referrer is an https URL and the linked to site is from a different domain. I've been watching this implementation detail for a long time, and it has been reliable. > OTOH, all browsers block HTTP authenticaion credentials from being > passed in the referrer header. Unfortunately, browsers are very promiscuous with HTTP authentication credentials in other ways. For example, consider a resource at my bank which will perform a spend when sent a POST with the expected HTTP Auth credentials. This resource may be located at . Imagine I log into my bank account, check on some pending transactions and then go visit some interesting blogs. At the end of a blog post is a form submission button that claims to create a blog post comment. The form actually points to and does a spend when I click on it. I click on the button, and my browser does the form POST, happily responding to the bank's HTTP Auth challenge using it's cached copy of my password. The spend goes through as it was sent from my machine, using my password. Tyler -- The web-calculus is the union of REST and capability-based security: http://www.waterken.com/dev/Web/ Name your trusted sites to distinguish them from phishing sites. https://addons.mozilla.org/extensions/moreinfo.php?id=957 From nlothian at educationau.edu.au Tue Sep 27 03:37:37 2005 From: nlothian at educationau.edu.au (Nick Lothian) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization Message-ID: > > Hi Nick, > > On 9/26/05, Nick Lothian wrote: > > Interesting idea. > > Thanks. > > > It may not be security via obscurity, but it does appear to > ignore a > > number of practical considerations. > > > > For instance, what about the secret URL being passed on in referrer > > headers to other pages? I think some browsers block it when you go > > from a secure page to a non-secure page on another site > (although I'm > > unsure about that). > > Fortunately, browsers don't send a Referer header field when > the referrer is an https URL and the linked to site is from a > different domain. I've been watching this implementation > detail for a long time, and it has been reliable. > Fair enough. It does depend on the secured urls only being available under https, though, which is somewhat non-obvious. There is also increased vunerability to cross-site-scripting attacks. If the attacker can get the window.location passed to a third party site then they get access to your resource. HTTPS urls show up in a browser's history, too, which makes your history files a much more tempting target for spyware etc. I'm not convinced that the "unguessability" of the URL is something I'd want to bet on either. There have been plenty of vulnerabilities that allow attackers to guess/predict session id cookies from application servers. This appears to be the same problem, but made easier because the URLs don't have a time limit on how long they remain valid for (unlike session ids, where the attacker has a very limited time to try and find a valid session id before it expires) > > OTOH, all browsers block HTTP authenticaion credentials from being > > passed in the referrer header. > > Unfortunately, browsers are very promiscuous with HTTP > authentication credentials in other ways. For example, > consider a resource at my bank which will perform a spend > when sent a POST with the expected HTTP Auth credentials. > This resource may be located at . > Imagine I log into my bank account, check on some pending > transactions and then go visit some interesting blogs. > At the end of a blog post is a form submission button that > claims to create a blog post comment. The form actually > points to and does a spend when I > click on it. I click on the button, and my browser does the > form POST, happily responding to the bank's HTTP Auth > challenge using it's cached copy of my password. The spend > goes through as it was sent from my machine, using my password. > > Tyler > Yes, I read that example in the REST group archive. It is an interesting attack, but one the user can work around. For instance, in IE if you open your session to the bank in a separate instance of IE (instead of just a separate window) then the credentials won't be shared. I don't think Firefox lets to start multiple instances, though (on Windows anyway) The other thing is that both the bank and the user can limit the timeframe of potential secutiry breaches and increase security by either (a) enforcing short session time outs and/or (b) logging out of the application. Nick IMPORTANT: This e-mail, including any attachments, may contain private or confidential information. If you think you may not be the intended recipient, or if you have received this e-mail in error, please contact the sender immediately and delete all copies of this e-mail. If you are not the intended recipient, you must not reproduce any part of this e-mail or disclose its contents to any other party. This email represents the views of the individual sender, which do not necessarily reflect those of education.au limited except where the sender expressly states otherwise. It is your responsibility to scan this email and any files transmitted with it for viruses or any other defects. education.au limited will not be liable for any loss, damage or consequence caused directly or indirectly by this email. From tyler.close at gmail.com Tue Sep 27 06:29:18 2005 From: tyler.close at gmail.com (Tyler Close) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: References: Message-ID: <5691356f0509262329156e4cff@mail.gmail.com> Hi Nick, On 9/26/05, Nick Lothian wrote: > It does depend on the secured urls only being available under https, > though, which is somewhat non-obvious. Using https is a pretty standard thing to do in a secure web application. > There is also increased vunerability to cross-site-scripting attacks. If > the attacker can get the window.location passed to a third party site > then they get access to your resource. If the site has a XSS bug, the jig is up regardless of what authorization mechanism you are using. The attacker can completely control the presentation and functionality of the web page, including remote controlling it on the fly. > HTTPS urls show up in a browser's history, too, which makes your history > files a much more tempting target for spyware etc. There are no HTTPS history files. HTTPS history is only stored in RAM for the duration of the browser session. > I'm not convinced that the "unguessability" of the URL is something I'd > want to bet on either. There have been plenty of vulnerabilities that > allow attackers to guess/predict session id cookies from application > servers. Reading 128 bits from /dev/random is pretty easy to implement. App servers with these kinds of bugs are pathetic. > This appears to be the same problem, but made easier because > the URLs don't have a time limit on how long they remain valid for > (unlike session ids, where the attacker has a very limited time to try > and find a valid session id before it expires) I am skeptical of defenses based on expiration times, but if that's what you want, you can certainly time out your URLs, just as you do your session ids. > > Unfortunately, browsers are very promiscuous with HTTP > > authentication credentials in other ways. For example, > > consider a resource at my bank which will perform a spend > > when sent a POST with the expected HTTP Auth credentials. > > This resource may be located at . > > Imagine I log into my bank account, check on some pending > > transactions and then go visit some interesting blogs. > > At the end of a blog post is a form submission button that > > claims to create a blog post comment. The form actually > > points to and does a spend when I > > click on it. I click on the button, and my browser does the > > form POST, happily responding to the bank's HTTP Auth > > challenge using it's cached copy of my password. The spend > > goes through as it was sent from my machine, using my password. > > Yes, I read that example in the REST group archive. > > It is an interesting attack, but one the user can work around. For > instance, in IE if you open your session to the bank in a separate > instance of IE (instead of just a separate window) then the credentials > won't be shared. I don't think Firefox lets to start multiple instances, > though (on Windows anyway) I don't think this is an acceptable state of affairs, but beyond that, this example is just that, an example. This type of attack is present in many browser and non-browser scenarios. Really, it can be done anytime you are using ACL based authorization. The problem is that the user's authority gets paired with an identifier selected by the attacker. Whenever your software accepts an identifier from a potential attacker, you are vulnerable to this kind of attack. This is really serious if you think about what software does. Basically, all software does is pass around identifiers, and each transfer is an attack point when you are using ACL based authorization. A capability-based design is not vulnerable to this kind of attack. > The other thing is that both the bank and the user can limit the > timeframe of potential secutiry breaches and increase security by either > (a) enforcing short session time outs and/or (b) logging out of the > application. Software attacks are automated attacks. Human scale timeouts are futile. Thanks for your comments. It's great to have an excuse to talk about this stuff. Tyler -- The web-calculus is the union of REST and capability-based security: http://www.waterken.com/dev/Web/ Name your trusted sites to distinguish them from phishing sites. https://addons.mozilla.org/extensions/moreinfo.php?id=957 From coderman at gmail.com Tue Sep 27 12:57:16 2005 From: coderman at gmail.com (coderman) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <5691356f0509262329156e4cff@mail.gmail.com> References: <5691356f0509262329156e4cff@mail.gmail.com> Message-ID: <4ef5fec6050927055732142950@mail.gmail.com> On 9/27/05, Tyler Close wrote: > ... > Using https is a pretty standard thing to do in a secure web application. regardless of whether you use TLS or IPsec the main issue with this approach is that users are no longer uniquely authenticated, only resources are via the nonce or other random looking identifier assigned to them*. that is not to say the concept of nonce based locators exchanged between trusted parties is not a good idea or useful, just that it would be best to combine user authentication _and_ unguessable resource locators rather than picking one or the other. variations on this theme can further constrain the availability of the resource (like the expiring locators mentioned) i have been playing with this type of constrained distribution over wireless using unidirectional broadcast to deliver an encrypted payload. only those with the key (nonce, random locator, etc) are provided with access to the resource. if you use a self authenticating name space for file access (sfs) you can also achieve this kind of opaque resource identifier mechanism easily. note that in either case any caching or copying of the resource can occur by a devious or misguided peer outside of the network where it was obtained. (insert long tangent on the dual uses of DRM technology and hardened hardware) for example, user logs into https site using a trusted cert and maybe even presents client credentials during the TLS handshake as you are concerned someone might observe their keyboard interactive login while using the site. you then provide them the locator for the resource which they proceed to save plaintext on their spyware infested directly internet connected windows machine running eDonkey and fast track shares on C:\. (this is amusing precisely because it is so common) > > I'm not convinced that the "unguessability" of the URL is something I'd > > want to bet on either. There have been plenty of vulnerabilities that > > allow attackers to guess/predict session id cookies from application > > servers. > > Reading 128 bits from /dev/random is pretty easy to implement. App > servers with these kinds of bugs are pathetic. it's a deterrent that makes discovery of unknown but reachable content expensive. instead of cracking the holes in a poorly distributed PRNG (some processing time performed remotely) you now have to bribe the user's significant other for a copy or discern the identifier bits from the LED emmisions on the broadband router next to their computer**. a sufficiently large nonce would be very near impossible to guess so you can continue to worry about the other holes in your system that will leak the same information with a fraction of the effort. you could also utilize a variation of hash cash to make attacks against the namespace more difficult. "the resource is at http://peer/[compute 2^20 iterations of the SHA256 digest of 0xdecafbad....]/index.html" as a side note it's actually pretty easy to deplete /dev/random on a network server devoid of a monitor, keyboard, mouse and other common EGD sources. this is where true random number generators in hardware can be very useful. (VIA, Intel and AMD all provide hw entropy sources in some of their products.) best regards, * at least, that is my impression of ""Instead of having a password for each user, have a password for each resource."" as mentioned here: http://groups.yahoo.com/group/rest-discuss/message/5231 ** side channel attacks are great at rendering well designed (in theory) and strongly keyed crypto systems completely broken. :) From solipsis at pitrou.net Tue Sep 27 13:23:17 2005 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <5691356f050926193344350482@mail.gmail.com> References: <5691356f050926193344350482@mail.gmail.com> Message-ID: <1127827397.14438.16.camel@p-dvsi-418-1.rd.francetelecom.fr> Hi, > Unfortunately, browsers are very promiscuous with HTTP authentication > credentials in other ways. For example, consider a resource at my bank > which will perform a spend when sent a POST with the expected HTTP > Auth credentials. This resource may be located at > . Imagine I log into my bank account, check > on some pending transactions and then go visit some interesting blogs. > At the end of a blog post is a form submission button that claims to > create a blog post comment. The form actually points to > and does a spend when I click on it. I > click on the button, and my browser does the form POST, happily > responding to the bank's HTTP Auth challenge using it's cached copy of > my password. The bank's website is at fault if it does not check for fraudulent requests. The bank's website should generate (server-side) an unique ID each time the client asks for a spend. Then on return from the form, the server checks the unique ID is genuine (or any other similar type of check, for example involving a challenge/response with a client-side Javascript hash implementation - I know there are some for MD5, it's reasonable workable). This is not difficult to do. It involves a two-step interaction, which is not unexpected for dangerous actions. So I don't understand why you put the guilt on the HTTP protocol itself (or its client implementations). > Using https is a pretty standard thing to do in a secure web > application. Sorry, but I call bullshit on this ;) HTTPS is not available in a proper way for people using shared hosting services (i.e. 99.99% of the Web). Also, HTTPS takes a lot of resources. Thus it is not an option for most people. Regards Antoine. From justin at chapweske.com Tue Sep 27 13:25:21 2005 From: justin at chapweske.com (Justin Chapweske) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <43387A51.8040105@panix.com> References: <433862ED.6050108@panix.com> <433866e0.78f3de53.5aa9.53bcSMTPIN_ADDED@mx.gmail.com> <5691356f0509261451a3da329@mail.gmail.com> <43387A51.8040105@panix.com> Message-ID: <1127827522.3019.1356.camel@bog> Based on your description, such as a system wouldn't work very well due to leaks from referrers, logging, and other systems that don't mind communicating about visited URLs. Its for this same reason that people should be careful with private wikis linking to external sites. For instance, if I have a private wiki page called 'SwarmingPagerankPatent' for some fictitious new innovation that we've brewed up, and I were to click on an external URL link from that page, that information would now be available in someone's referrer logs. So, if you guys know any wiki authors, encourage them to automatically insert a referrer scrubber/redirect page when following any external URLs. -Justin -- Justin Chapweske, Founder and CEO - Swarmcast http://onionnetworks.com/ On Mon, 2005-09-26 at 12:46 -1000, Lucas Gonze wrote: > p2p-hackers, meet rest-discuss. rest-discuss, I'd like to introduce you > to p2p-hackers. > > RESTafarians: there is a long-running conversation on p2p-hackers about > friendnets, also known as darknets, small world networks, and F2F > networks; also capabilities security, sometimes known as smart > contracts. An example thread begins at > http://zgp.org/pipermail/p2p-hackers/2005-August/002915.html > > p2p-hackers: Tyler Close' method for HTTP access control using nothing > but unguessable (and secret) URIs came up on REST-discuss. That thread > begins at http://groups.yahoo.com/group/rest-discuss/message/5228 In > the context of friendnets, Tyler's scheme is a beautifully simple way of > controlling access using nothing but low-tech means. Not only does it > limit access to trusted parties, it also allows for transitive > relationships. (Warning: his scheme is counterintuitive, since the > dependence on secret URLs smells like security through obscurity). > > I don't mean to create a permathread out of out of this cc:list, rather > to refer interested rest-discuss people to p2p-hackers and vice versa. > If you reply to this, please choose one list or the other as the > recipient. > > I've always been surprised that the intersection of REST and > capabilities isn't a common theme, since both REST and capabilities > revolve around precisely defined relationships. The success of defining > HTTP methods in terms of privileges is due, I believe, to the way that > it reflects the same underlying truth that capabilities formalize. > > > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences From coderman at gmail.com Tue Sep 27 14:50:58 2005 From: coderman at gmail.com (coderman) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <1127827522.3019.1356.camel@bog> References: <433862ED.6050108@panix.com> <433866e0.78f3de53.5aa9.53bcSMTPIN_ADDED@mx.gmail.com> <5691356f0509261451a3da329@mail.gmail.com> <43387A51.8040105@panix.com> <1127827522.3019.1356.camel@bog> Message-ID: <4ef5fec6050927075060eec852@mail.gmail.com> On 9/27/05, Justin Chapweske wrote: > Based on your description, such as a system wouldn't work very well due > to leaks from referrers, logging, and other systems that don't mind > communicating about visited URLs. > ... > So, if you guys know any wiki authors, encourage them to automatically > insert a referrer scrubber/redirect page when following any external > URLs. hey, that's a resource discovery feature, not a bug! in all seriousness, sometimes this is useful and desired, like trackbacks in blogs, and sometimes it is a disastrous disclosure, exposing a bunch of really bad attitudes, for example. ;) what would be nice is a simple way to toggle your wiki/site software in one of two modes: [ ] tell remote hosts my referrer details [ ] do not disclose any information about me (referrer) to remote hosts right now i have to use complicated proxy filtering rules to accomplish this. (which are still useful, but needlessly cumbersome. usually...) From tyler.close at gmail.com Tue Sep 27 15:08:51 2005 From: tyler.close at gmail.com (Tyler Close) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <4ef5fec6050927075060eec852@mail.gmail.com> References: <433862ED.6050108@panix.com> <433866e0.78f3de53.5aa9.53bcSMTPIN_ADDED@mx.gmail.com> <5691356f0509261451a3da329@mail.gmail.com> <43387A51.8040105@panix.com> <1127827522.3019.1356.camel@bog> <4ef5fec6050927075060eec852@mail.gmail.com> Message-ID: <5691356f05092708081db45af2@mail.gmail.com> Hi Justin, coderman, On 9/27/05, coderman wrote: > On 9/27/05, Justin Chapweske wrote: > > Based on your description, such as a system wouldn't work very well due > > to leaks from referrers (snip) > what would be nice is a simple way to toggle your wiki/site software > in one of two modes: > [ ] tell remote hosts my referrer details > [ ] do not disclose any information about me (referrer) to remote hosts Which is exactly the formula implemented in web browsers for http vs https. Web browsers don't leek https URLs in the Referer header. Tyler -- The web-calculus is the union of REST and capability-based security: http://www.waterken.com/dev/Web/ Name your trusted sites to distinguish them from phishing sites. https://addons.mozilla.org/extensions/moreinfo.php?id=957 From tyler.close at gmail.com Tue Sep 27 15:15:39 2005 From: tyler.close at gmail.com (Tyler Close) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <1127827522.3019.1356.camel@bog> References: <433862ED.6050108@panix.com> <433866e0.78f3de53.5aa9.53bcSMTPIN_ADDED@mx.gmail.com> <5691356f0509261451a3da329@mail.gmail.com> <43387A51.8040105@panix.com> <1127827522.3019.1356.camel@bog> Message-ID: <5691356f050927081550c5c8e6@mail.gmail.com> Hi Justin, On 9/27/05, Justin Chapweske wrote: > Based on your description, such as a system wouldn't work very well due > to leaks from referrers, logging, and other systems that don't mind > communicating about visited URLs. These URLs are https URLs and so won't be leaked in HTTP Referer headers. Since they are communicated over an SSL connection, logging is only possible at the server and client machines. The processes at either end of the SSL connection are necessarily trusted with authority over the identified resource, since the resource is located at the server, and controlled by the user. The only remaining hurdle is that some endpoint processes are not coded to safely handle such authority. A web browser configured to relay every visited URL to a third party might be a problem, depending on the third party. Obviously such snoopy software is also a problem for sites that encode a session id in the URL. Tyler -- The web-calculus is the union of REST and capability-based security: http://www.waterken.com/dev/Web/ Name your trusted sites to distinguish them from phishing sites. https://addons.mozilla.org/extensions/moreinfo.php?id=957 From roberto.bayardo at gmail.com Tue Sep 27 15:37:18 2005 From: roberto.bayardo at gmail.com (Roberto Bayardo) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <5691356f05092708081db45af2@mail.gmail.com> References: <433862ED.6050108@panix.com> <433866e0.78f3de53.5aa9.53bcSMTPIN_ADDED@mx.gmail.com> <5691356f0509261451a3da329@mail.gmail.com> <43387A51.8040105@panix.com> <1127827522.3019.1356.camel@bog> <4ef5fec6050927075060eec852@mail.gmail.com> <5691356f05092708081db45af2@mail.gmail.com> Message-ID: <93a05d86050927083720bd6612@mail.gmail.com> > Web browsers don't leek https URLs in the Referer header. Not in my experience. For example, using both IE 6 and Mozilla 1.7.10, consider the following log snapshot: 9.49.221.110 - - [27/Sep/2005:15:24:27 +0000] "GET /images/unknown2.gif HTTP/1.1" 200 1006 " https://bayardo.userv.ibm.com/stuff/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.10) Gecko/20050716" 9.49.221.110 - - [27/Sep/2005:15:24:27 +0000] "GET /images/source_java.gif HTTP/1.1" 200 1031 " https://bayardo.userv.ibm.com/stuff/" "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.10) Gecko/20050716" .... 9.49.221.110 - - [27/Sep/2005:15:27:35 +0000] "GET /images/txt.gif HTTP/1.1" 200 1030 "https://bayardo.userv.ibm.com/stuff/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" 9.49.221.110 - - [27/Sep/2005:15:27:36 +0000] "GET /images/unknown2.gif HTTP/1.1" 200 1006 " https://bayardo.userv.ibm.com/stuff/" "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)" This is from the log of a webserver with domain "w3.userv.ibm.com" (an internal domain so it won't work for you :-) which hosts images referenced by pages on the server bayardo.userv.ibm.com. Both servers are accessed using HTTPS, both servers use different SSL certs issued by different CAs. They share the same top level domain, but I don't know if that's significant (if it is, it shouldn't be!). -------------- next part -------------- An HTML attachment was scrubbed... URL: http://zgp.org/pipermail/p2p-hackers/attachments/20050927/24a63e41/attachment.htm From coderman at gmail.com Tue Sep 27 16:02:57 2005 From: coderman at gmail.com (coderman) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <93a05d86050927083720bd6612@mail.gmail.com> References: <433862ED.6050108@panix.com> <433866e0.78f3de53.5aa9.53bcSMTPIN_ADDED@mx.gmail.com> <5691356f0509261451a3da329@mail.gmail.com> <43387A51.8040105@panix.com> <1127827522.3019.1356.camel@bog> <4ef5fec6050927075060eec852@mail.gmail.com> <5691356f05092708081db45af2@mail.gmail.com> <93a05d86050927083720bd6612@mail.gmail.com> Message-ID: <4ef5fec6050927090268e00715@mail.gmail.com> On 9/27/05, Roberto Bayardo wrote: >... They share the same top level domain, but I don't know if that's significant > (if it is, it shouldn't be!). this is the reason you see the referrer. if you link outside the ibm domain you will see the remote web logs are missing referrer info. in any case, i'd still like an easy solution that would allow https sites to provide referrer and http to not. i bet someone has a greasemonkey script for this 6 months ago... From tyler.close at gmail.com Tue Sep 27 16:34:39 2005 From: tyler.close at gmail.com (Tyler Close) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <1127827397.14438.16.camel@p-dvsi-418-1.rd.francetelecom.fr> References: <5691356f050926193344350482@mail.gmail.com> <1127827397.14438.16.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <5691356f05092709347fb65d0b@mail.gmail.com> Hi Antoine, On 9/27/05, Antoine Pitrou wrote: > > Unfortunately, browsers are very promiscuous with HTTP authentication > > credentials in other ways. For example, consider a resource at my bank > > which will perform a spend when sent a POST with the expected HTTP > > Auth credentials. This resource may be located at > > . Imagine I log into my bank account, check > > on some pending transactions and then go visit some interesting blogs. > > At the end of a blog post is a form submission button that claims to > > create a blog post comment. The form actually points to > > and does a spend when I click on it. I > > click on the button, and my browser does the form POST, happily > > responding to the bank's HTTP Auth challenge using it's cached copy of > > my password. > > The bank's website is at fault if it does not check for fraudulent > requests. > The bank's website should generate (server-side) an unique ID each time > the client asks for a spend. Then on return from the form, the server > checks the unique ID is genuine The first interesting thing to note is that your proposed solution implicitly acknowleges the fact that the username/password is not providing access control. In your proposed solution, the use of username/password is irrelevant. The unguessable ID is providing the access control. Note that the ID must be unguessable, not just unique. If the ID is predictable, your proposed solution is foiled by the same attack I describe above. Your proposed solution has the flavour of what I am proposing, using unguessable IDs for access control. Unfortunately, you have reified the authority to submit a spend as an unguessable ID, but you have not so reified the authority to request a spend. This is where your proposed solution fails. For example, the blog post page from the example above could do a GET on the URL that produces your FORM that contains a newly generated spend ID. The blog post page could then extract the spend ID from the returned FORM and use it to construct a new FORM, like the one it constructed in the example above. Now, when the user clicks on the generated submit button, the browser does the FORM POST, again happily responding to the bank's HTTP Auth challenge, as well as providing the expected spend ID. > So I don't understand why you put the guilt on the HTTP protocol itself > (or its client implementations). I don't put the guilt on the HTTP client, server or protocol. They are all faithful implementations of an ACL model. The problem is that the ACL model is broken. Your proposed solution is still vulnerable to attack, precisely because it is still depending upon an ACL check. Were you to reify the rest of the application's exported authorities as unguessable IDs, the application would no longer be vulnerable to this kind of attack, and you would have a design much like what I am proposing. Tyler -- The web-calculus is the union of REST and capability-based security: http://www.waterken.com/dev/Web/ Name your trusted sites to distinguish them from phishing sites. https://addons.mozilla.org/extensions/moreinfo.php?id=957 From solipsis at pitrou.net Tue Sep 27 16:56:08 2005 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <5691356f05092709347fb65d0b@mail.gmail.com> References: <5691356f050926193344350482@mail.gmail.com> <1127827397.14438.16.camel@p-dvsi-418-1.rd.francetelecom.fr> <5691356f05092709347fb65d0b@mail.gmail.com> Message-ID: <1127840168.15818.40.camel@p-dvsi-418-1.rd.francetelecom.fr> Hi, > The first interesting thing to note is that your proposed solution > implicitly acknowleges the fact that the username/password is not > providing access control. In your proposed solution, the use of > username/password is irrelevant. Well, this is the classical token-based approach. You login once, and then some temporary secret data allows you to enter again without to login. Note that you can also re-ask for the login/pass if the user tries to do something really dangerous (this is what is usually done when requesting a password change). (by the way, I'm not talking about HTTP auth, rather session cookie-based auth after a first authentication form or challenge) > For example, the blog post page from the > example above could do a GET on the URL that produces your FORM that > contains a newly generated spend ID. The blog post page could then > extract the spend ID from the returned FORM and use it to construct a > new FORM, like the one it constructed in the example above. Well, if Javascript allows the browser to fake a human being without the user being aware of it, I think there's nothing serious we can do against it. The only solution is to devise proper security inside Javascript, or to devise alerts in Web browsers when such a behaviour is detected. Regards Antoine. From tyler.close at gmail.com Tue Sep 27 17:24:12 2005 From: tyler.close at gmail.com (Tyler Close) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <4ef5fec6050927055732142950@mail.gmail.com> References: <5691356f0509262329156e4cff@mail.gmail.com> <4ef5fec6050927055732142950@mail.gmail.com> Message-ID: <5691356f050927102418b19d08@mail.gmail.com> Hi coderman, On 9/27/05, coderman wrote: > On 9/27/05, Tyler Close wrote: > > Using https is a pretty standard thing to do in a secure web application. > > regardless of whether you use TLS or IPsec the main issue with this > approach is that users are no longer uniquely authenticated, only > resources are via the nonce or other random looking identifier > assigned to them*. Yes, I am proposing that a resource request be honored solely based on presentation of the correct resource password. Specifically, I am proposing that username/password checks not be used to authorize resource access. Based on my study of the problem, I believe using user authentication to authorize resource access can only confuse the situation and cause vulnerabilities. > that is not to say the concept of nonce based locators exchanged > between trusted parties is not a good idea or useful, just that it > would be best to combine user authentication _and_ unguessable > resource locators rather than picking one or the other. variations on > this theme can further constrain the availability of the resource > (like the expiring locators mentioned) I think such a combination would only succeed in producing a baroque design with more vulnerabilities. Achieving security requires stripping away complexity, not adding it. > i have been playing with this type of constrained distribution over > wireless using unidirectional broadcast to deliver an encrypted > payload. only those with the key (nonce, random locator, etc) are > provided with access to the resource. if you use a self > authenticating name space for file access (sfs) you can also achieve > this kind of opaque resource identifier mechanism easily. This sounds like an interesting design. Note that nothing in the above description implies username/password based authorization. > for example, user logs into https site using a trusted cert and maybe > even presents client credentials during the TLS handshake as you are > concerned someone might observe their keyboard interactive login > while using the site. you then provide them the locator for the > resource which they proceed to save plaintext on their spyware > infested directly internet connected windows machine running eDonkey > and fast track shares on C:\. > (this is amusing precisely because it is so common) I agree that this scenario is far too common. Unfortunately, we are powerless to help the user with a machine infested with spyware. None of our designs are resilient to running in such an environment. Fortunately, it is my day job to help build machines that are less prone to being infected with spyware. See: http://hpl.hp.com/news/2005/apr-jun/virussafe.html Tyler -- The web-calculus is the union of REST and capability-based security: http://www.waterken.com/dev/Web/ Name your trusted sites to distinguish them from phishing sites. https://addons.mozilla.org/extensions/moreinfo.php?id=957 From tyler.close at gmail.com Tue Sep 27 17:33:14 2005 From: tyler.close at gmail.com (Tyler Close) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <1127840168.15818.40.camel@p-dvsi-418-1.rd.francetelecom.fr> References: <5691356f050926193344350482@mail.gmail.com> <1127827397.14438.16.camel@p-dvsi-418-1.rd.francetelecom.fr> <5691356f05092709347fb65d0b@mail.gmail.com> <1127840168.15818.40.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <5691356f05092710332a623de2@mail.gmail.com> Hi Antoine, On 9/27/05, Antoine Pitrou wrote: > Well, if Javascript allows the browser to fake a human being without the > user being aware of it, I think there's nothing serious we can do > against it. Quite the contrary, I am proposing a solution: capability URLs. > The only solution is to devise proper security inside > Javascript, or to devise alerts in Web browsers when such a behaviour is > detected. The blog page is just doing a GET on a well known URL. We don't really have a web if doing a GET is not safe operation. The problem is not the GET, nor the Javascript, nor the Web browser. The problem is the ACL based access control. The ACL model just doesn't work when there is communication between users. We don't need to further hamstring the browser, nor add more warning dialogs. We just need to use an access control model that actually controls access. Tyler -- The web-calculus is the union of REST and capability-based security: http://www.waterken.com/dev/Web/ Name your trusted sites to distinguish them from phishing sites. https://addons.mozilla.org/extensions/moreinfo.php?id=957 From dbarrett at quinthar.com Tue Sep 27 19:46:57 2005 From: dbarrett at quinthar.com (David Barrett) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <5691356f05092710332a623de2@mail.gmail.com> References: <5691356f05092710332a623de2@mail.gmail.com> Message-ID: <1127850423.3A956345@bd12.dngr.org> > On 9/27/05, Antoine Pitrou wrote: >> Well, if Javascript allows the browser to fake a human being without >> the >> user being aware of it, I think there's nothing serious we can do >> against it. Well, there are *some* limits to what a JavaScript page can do. For example, is it possible for a JavaScript page from "bad.com" to issue a request to "other.com" with a forged Referer header from "good.com"? It's my impression that Referer is a read-only property, and thus can't be spoofed with a typical request. And using an XML HTTP request is limited to the origin domain (in this case, "bad.com"). Am I missing another option? (Naturally native code using a simple socket can spoof anything, but I'm taking about what's possible in JavaScript without popping up security warnings.) -david From tyler.close at gmail.com Tue Sep 27 20:08:18 2005 From: tyler.close at gmail.com (Tyler Close) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <1127850423.3A956345@bd12.dngr.org> References: <5691356f05092710332a623de2@mail.gmail.com> <1127850423.3A956345@bd12.dngr.org> Message-ID: <5691356f050927130839a47622@mail.gmail.com> Hi David, On 9/27/05, David Barrett wrote: > > On 9/27/05, Antoine Pitrou wrote: > >> Well, if Javascript allows the browser to fake a human being without > >> the > >> user being aware of it, I think there's nothing serious we can do > >> against it. > > Well, there are *some* limits to what a JavaScript page can do. For > example, is it possible for a JavaScript page from "bad.com" to issue a > request to "other.com" with a forged Referer header from "good.com"? If the page from bad.com or good.com is an https page, other.com will not receive a Referer header from either and so cannot tell the difference between bad.com and good.com. Tyler -- The web-calculus is the union of REST and capability-based security: http://www.waterken.com/dev/Web/ Name your trusted sites to distinguish them from phishing sites. https://addons.mozilla.org/extensions/moreinfo.php?id=957 From coderman at gmail.com Tue Sep 27 20:46:27 2005 From: coderman at gmail.com (coderman) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <5691356f050927102418b19d08@mail.gmail.com> References: <5691356f0509262329156e4cff@mail.gmail.com> <4ef5fec6050927055732142950@mail.gmail.com> <5691356f050927102418b19d08@mail.gmail.com> Message-ID: <4ef5fec605092713465ec85a4f@mail.gmail.com> On 9/27/05, Tyler Close wrote: > ... > Yes, I am proposing that a resource request be honored solely based on > presentation of the correct resource password. Specifically, I am > proposing that username/password checks not be used to authorize > resource access. Based on my study of the problem, I believe using > user authentication to authorize resource access can only confuse the > situation and cause vulnerabilities. i would love to see the research behind this, as this is an intriguing approach. i did not mean to imply that username/password was required for distinct user authentication (passwords are broken anyway, right? :) but rather that some mechanism be used to ensure that access to resources can be constrained at a user level and audited at a user level. this is trivial to do if you ensure that no two users are given the same resource password; each use a distinct name to reference the same underlying resource. consider revocation of privilege: if there is no distinction between users than revocation of privilege to access a resource is performed on the resource as a whole and would affect all users (as they are all referring to the resource with the same password). if there is distinction between users then revocation of privilege can be applied to just that user which has violated a trust of some kind. note that in either case you are still using a password to access resources, and there is no need to authenticate users solely for the purpose of authenticating a user (username/password), but rather the passwords used to access resources imply a distinct user identity doing so. while this is not necessary in every case i think this is useful particularly in the context of privilege revocation / auditing. > I think such a combination would only succeed in producing a baroque > design with more vulnerabilities. Achieving security requires > stripping away complexity, not adding it. i agree with respect to an authentication step required for identifying a user and for no other purpose (username/password required before accessing resources via password). would the multiplicative effect of distinct resource passwords per user (when user distinction is desired) be as baroque and complex as the username/password scenario in your view? > I agree that this scenario is far too common. Unfortunately, we are > powerless to help the user with a machine infested with spyware. None > of our designs are resilient to running in such an environment. > Fortunately, it is my day job to help build machines that are less > prone to being infected with spyware. See: > > http://hpl.hp.com/news/2005/apr-jun/virussafe.html thanks for the link; this is indeed an interesting effort toward a simple and effective least privilege environment without the cumbersome complexity of explicit ACL maintenance and the accompanying error prone interface complexity and/or confused deputy issues. i am particularly interested in the use of virtual machines (xen) and profiling of file/device/network resource usage for anomaly detection. this requires some kind of distributed / collaborative effort to manage the privileges/authorities associated with specific tasks / contexts as the requisites may change from release to release or under new contexts. i'll try and clarify what i mean by collaborative least privilege maintenance and how profiling and anomaly detection can assist this process to make it as simple and self evident to interact with as an application/process. (i am not well versed in the theory here and an offhand attempt to describe would result in confusion more than anything else...) this too requires distinction between users at some level. best regards, From lgonze at panix.com Tue Sep 27 21:47:13 2005 From: lgonze at panix.com (Lucas Gonze) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <4ef5fec605092713465ec85a4f@mail.gmail.com> References: <5691356f0509262329156e4cff@mail.gmail.com> <4ef5fec6050927055732142950@mail.gmail.com> <5691356f050927102418b19d08@mail.gmail.com> <4ef5fec605092713465ec85a4f@mail.gmail.com> Message-ID: <4339BDE1.6030409@panix.com> coderman wrote: >... some mechanism be used to ensure that access to >resources can be constrained at a user level and audited at a user >level. > >this is trivial to do if you ensure that no two users are given the >same resource password; each use a distinct name to reference the same >underlying resource. > In this context the URL is the password, so the publisher can just give each distinct user a different URL. You can then revoke or expire resources as the need comes up. All these problems are amazingly trivial in the context of the kind of heavy-duty tech that privacy systems usually require. Concealing the HTTP referrer? What a great problem to have! There's nothing here that can't be done with a single CGI script. From solipsis at pitrou.net Wed Sep 28 09:50:49 2005 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <1127850423.3A956345@bd12.dngr.org> References: <5691356f05092710332a623de2@mail.gmail.com> <1127850423.3A956345@bd12.dngr.org> Message-ID: <1127901049.15818.47.camel@p-dvsi-418-1.rd.francetelecom.fr> > It's my impression that Referer is a read-only property, and thus can't > be spoofed with a typical request. And using an XML HTTP request is > limited to the origin domain (in this case, "bad.com"). Is this a design property of the ECMAscript standard or just an implementation detail? If the latter, you can't expect that it will always stay like this. Also, you can't rely on Referer to enforce security rules because Referer is not a mandatory header in HTTP, and it can be changed for various reasons (privacy "enhancers" for example). You can use it in heuristics to display various warning messages, but must not deny access based on it. I'm curious as to how "capability URLs" can't be stolen and re-used by a malicious piece of Javascript like other URLs can. I've tried to read the theoretical paper about "web calculus" but it's, well... theoretical ;) Is there a concrete example somewhere? (I'm not on rest-discuss so I may have missed some parts of the discussion, so sorry if I'm asking redundant questions here) > (Naturally native code using a simple socket can spoof anything, but I'm > taking about what's possible in JavaScript without popping up security > warnings.) Of course. regards Antoine. From seth.johnson at RealMeasures.dyndns.org Wed Sep 28 12:17:04 2005 From: seth.johnson at RealMeasures.dyndns.org (Seth Johnson) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] FFII, NoSWPats Win CNet Award for Outstanding Contribution to SW Development Message-ID: <433A89C0.7445C861@RealMeasures.dyndns.org> Please send congratulations to FFII and Nosoftwarepatents.com for their glorious work keeping software patents illegal in Europe! Seth > http://www.cnetnetworks.co.uk/awards/2005_winners.html#contri_software Outstanding Contribution to Software Development Winner: NoSoftwarePatents.com & The Foundation for a Free Information Infrastructure (FFII) NoSoftwarePatents.com and the FFII joined forces last year to campaign against the European software patent directive, which many feared would open the doors to an increasingly litigious marketplace in which small businesses would struggle to survive. In July this year, the European Parliament unexpectedly rejected the directive, a victory which can be attributed to the tireless work of this alliance. From tyler.close at gmail.com Wed Sep 28 14:55:43 2005 From: tyler.close at gmail.com (Tyler Close) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <1127901049.15818.47.camel@p-dvsi-418-1.rd.francetelecom.fr> References: <5691356f05092710332a623de2@mail.gmail.com> <1127850423.3A956345@bd12.dngr.org> <1127901049.15818.47.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <5691356f0509280755501a1c1d@mail.gmail.com> Hi Antoine, On 9/28/05, Antoine Pitrou wrote: > I'm curious as to how "capability URLs" can't be stolen and re-used by a > malicious piece of Javascript like other URLs can. Simply because a capability URL is unguessable. The blog page in our previous example was able to do a GET on the spend page because it knew the URL for the spend page. We block this attack by making all URLs that represent authority be unguessable. Now, the only way for a piece of code to use the authority is through someone who already has the URL voluntarily passing it to that piece of code. > I've tried to read > the theoretical paper about "web calculus" but it's, well... > theoretical ;) Is there a concrete example somewhere? Yes, I've built an access controlled wiki. It's online at: https://yurl.net/id/home That page explains the details of the application. > (I'm not on rest-discuss so I may have missed some parts of the > discussion, so sorry if I'm asking redundant questions here) There were some good questions asked on rest-discuss. You should read the archives that Lucas referred to in his cross post to p2p-hackers. Thanks for the questions. Tyler -- The web-calculus is the union of REST and capability-based security: http://www.waterken.com/dev/Web/ Name your trusted sites to distinguish them from phishing sites. https://addons.mozilla.org/extensions/moreinfo.php?id=957 From solipsis at pitrou.net Wed Sep 28 15:41:02 2005 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <5691356f0509280755501a1c1d@mail.gmail.com> References: <5691356f05092710332a623de2@mail.gmail.com> <1127850423.3A956345@bd12.dngr.org> <1127901049.15818.47.camel@p-dvsi-418-1.rd.francetelecom.fr> <5691356f0509280755501a1c1d@mail.gmail.com> Message-ID: <1127922062.15818.82.camel@p-dvsi-418-1.rd.francetelecom.fr> Hi again, > On 9/28/05, Antoine Pitrou wrote: > > I'm curious as to how "capability URLs" can't be stolen and re-used by a > > malicious piece of Javascript like other URLs can. > > Simply because a capability URL is unguessable. It is permanent too, so if it leaks once, then it's compromised forever. And you have to keep this URL somewhere... Given that it's full of random ascii garbage, you can't keep it in your head (contrast this with a properly chosen password), and you don't want to copy it by hand either. So it /will/ end up in electronic clickable form somewhere: for example in your bookmarks. >From your own explanation on the REST mailing-list : ? The user just *clicks on hyperlinks*, without ever needing to be aware of the resource password. ? Those hyperlinks have to be somewhere... (and of course, this totally mandates HTTPS, which is impossible for most Web sites for reasons I already explained) As a mix proposal, it would be more interesting if a new URL was generated everytime the user identifies (with login/password). More interesting again, it could be generated client side in Javascript using a formula like "HASH(HASH(password) + challenge)" where the challenge is a temporary value generated by the server for this very session (thus with an expiration time). Which means: - the URL is temporary (it expires with the challenge) - this URL does not need to be recorded anywhere on the client since it's generated at every new login - in plain non-encrypted HTTPS, the data which goes over the wire only gives temporary access to the resource Regards Antoine. From mgp at ucla.edu Wed Sep 28 19:39:43 2005 From: mgp at ucla.edu (Michael Parker) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] [Fwd: [Planetlab-users] Announcing: TCP NAT Traversal/Hole-Punching Library] Message-ID: <20050928123943.wxuudevkh6ogwcog@mail.ucla.edu> I just got this by way of the PlanetLab mailing list... Definitely thought it would be of interest! - Mike ---------------------------- Original Message ---------------------------- Subject: [Planetlab-users] Announcing: TCP NAT Traversal/Hole-Punching Library From: "Saikat Guha" Date: Wed, September 28, 2005 3:46 am -------------------------------------------------------------------------- Hi all, (apologies if you get multiple copies of this) I am pleased to announce the availability of our open-source TCP NAT Traversal/Hole-Punching library based on our research published in [1]. [1] "Characterization and Measurement of TCP Traversal through NATs and Firewalls", S. Guha and P. Francis. IMC 2005. http://nutss.net/pub/imc05-tcpnat.pdf The key result of the paper is: TCP NAT traversal can work 85%-90% of the time today (without any special assumptions about NATs), and 100% of the time between pairs of certain popular, well-behaved NATs. See [1] for more details. An open-source Java library for TCP NAT Traversal is now available: webpage: http://nutss.net/stunt.php faq: http://nutss.net/jstunt-faq.php library and example: http://nutss.net/jstunt-examples.php The above library has been tested for pair-wise connectivity across 11 brands of NATs from Windows and Linux hosts. NATs tested were Linksys, DLink, Netgear, Belkin, 3Com, Netopia, Allied Telesyn, SMC, Trendnet, USR, Buffalo Tech. Out of the 121 possible pair-wise combinations, 113 connections are successful. The only ones that failed are when both the endpoints are behind the _same_ NAT device that does not support TCP hairpin-behavior yet (see [1]). The java library is released under LGPL; contact me if this does not meet your needs. Feel free to extend it/port it etc. Q: I am a P2P developer/researcher. How does this help me? A: The library adds TCP NAT traversal out-of-the-box. This increases the connectivity in your P2P network since two users behind their NATs can now exchange data without having to go through an intermediary node. You can: - Use this library as is (for development of P2P software, research, small deployments, etc in java) - Study it to provide TCP NAT Traversal in your existing P2P applications in your language of choice. - etc. If you have any questions, comments, suggestions, or problems, do not hesitate to contact me. Cheers, -- Saikat _______________________________________________ Users mailing list: Users@lists.planet-lab.org https://lists.planet-lab.org/mailman/listinfo/users ----- End forwarded message ----- -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: not available Url : http://zgp.org/pipermail/p2p-hackers/attachments/20050928/974fc1b4/signature.pgp -------------- next part -------------- _______________________________________________ Gnucla-devel mailing list Gnucla-devel@starsky.ee.ucla.edu http://starsky.ee.ucla.edu/cgi-bin/mailman/listinfo/gnucla-devel From tyler.close at gmail.com Thu Sep 29 17:36:51 2005 From: tyler.close at gmail.com (Tyler Close) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <1127922062.15818.82.camel@p-dvsi-418-1.rd.francetelecom.fr> References: <5691356f05092710332a623de2@mail.gmail.com> <1127850423.3A956345@bd12.dngr.org> <1127901049.15818.47.camel@p-dvsi-418-1.rd.francetelecom.fr> <5691356f0509280755501a1c1d@mail.gmail.com> <1127922062.15818.82.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <5691356f050929103656e4f30f@mail.gmail.com> Hi Antoine, On 9/28/05, Antoine Pitrou wrote: > > On 9/28/05, Antoine Pitrou wrote: > > > I'm curious as to how "capability URLs" can't be stolen and re-used by a > > > malicious piece of Javascript like other URLs can. > > > > Simply because a capability URL is unguessable. > > It is permanent too, No, the lifetime of the URL is up to the application designer. Using capability URLs does not place any duration restrictions on how you define the lifetime of your resources. > And you have to keep this URL somewhere... Given that it's full of > random ascii garbage, you can't keep it in your head (contrast this with > a properly chosen password), and you don't want to copy it by hand > either. So it /will/ end up in electronic clickable form somewhere: for > example in your bookmarks. I think keeping capability URLs in your bookmarks is a perfectly sensible thing to do, providing you then protect your bookmarks. I run OS X, so my entire filesystem is encrypted, including my bookmarks file. > >From your own explanation on the REST mailing-list : ? The user just > *clicks on hyperlinks*, without ever needing to be aware of the resource > password. ? Those hyperlinks have to be somewhere... > > (and of course, this totally mandates HTTPS, which is impossible for > most Web sites for reasons I already explained) This argument is a little out of date. You can get affordable HTTPS hosting from providers like GoDaddy and 1and1. Even before the advent of shared hosting for HTTPS, colocation was already an affordable option and likely required anyways in order to get the performance characteristics that you want for your web application. For very small scale projects, running Apache on your home machine with a dyndns hostname and a 7.99 SSL certificate is also doable. > As a mix proposal, it would be more interesting if a new URL was > generated everytime the user identifies (with login/password). More > interesting again, it could be generated client side in Javascript using > a formula like "HASH(HASH(password) + challenge)" where the challenge is > a temporary value generated by the server for this very session (thus > with an expiration time). Which means: > - the URL is temporary (it expires with the challenge) > - this URL does not need to be recorded anywhere on the client since > it's generated at every new login > - in plain non-encrypted HTTPS, the data which goes over the wire only > gives temporary access to the resource When I attend DefCon, I am always amazed that people are surprised by the Wall of Sheep, people who know that network snooping is possible. I guess you just have to experience the efficiency of live network snooping in order to truly appreciate it. With the rise of ubiquitous WiFi, passing secrets, even temporary ones, over the network in the clear is asking for trouble. Your 15 minute session timeout is an eon on the timescale of a script watching the network for your protocol and exploiting it on the fly. SSL has finally come into a somewhat reasonable price range. We should go ahead and exploit it. We don't need to mess around with dodgy timeout based designs. Thanks again for the questions. Tyler -- The web-calculus is the union of REST and capability-based security: http://www.waterken.com/dev/Web/ Name your trusted sites to distinguish them from phishing sites. https://addons.mozilla.org/extensions/moreinfo.php?id=957 From coderman at gmail.com Thu Sep 29 20:50:24 2005 From: coderman at gmail.com (coderman) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <5691356f050929103656e4f30f@mail.gmail.com> References: <5691356f05092710332a623de2@mail.gmail.com> <1127850423.3A956345@bd12.dngr.org> <1127901049.15818.47.camel@p-dvsi-418-1.rd.francetelecom.fr> <5691356f0509280755501a1c1d@mail.gmail.com> <1127922062.15818.82.camel@p-dvsi-418-1.rd.francetelecom.fr> <5691356f050929103656e4f30f@mail.gmail.com> Message-ID: <4ef5fec605092913505674097b@mail.gmail.com> On 9/29/05, Tyler Close wrote: > ... > I think keeping capability URLs in your bookmarks is a perfectly > sensible thing to do, providing you then protect your bookmarks. I run > OS X, so my entire filesystem is encrypted, including my bookmarks > file. this is similar to a password manager which most browsers already support. in this case the entire file system is protected rather than just a password database. i really like this approach (full hdd encryption) as it alleviates concern about sensitive/private/secret information that may reside in the file system otherwise (either through application flaws, user error, or OS mechanisms like swap space and deleted but intact inodes) > This argument is a little out of date. You can get affordable HTTPS > hosting from providers like GoDaddy and 1and1. ... running Apache > on your home machine with a dyndns hostname and a 7.99 SSL > certificate is also doable. the virtual hosting problems are annoying; distinct IP's for each distinct certificate is a serious hurdle in some cases. i would love to know of ways to workaround this issue without hosting multiple sites under a single domain. > When I attend DefCon, I am always amazed that people are surprised by > the Wall of Sheep, people who know that network snooping is possible. > I guess you just have to experience the efficiency of live network > snooping in order to truly appreciate it. > > With the rise of ubiquitous WiFi, passing secrets, even temporary > ones, over the network in the clear is asking for trouble. Your 15 > minute session timeout is an eon on the timescale of a script watching > the network for your protocol and exploiting it on the fly. it's an eye opener for sure. we captured 94,805,303 packets / 10G data in one room alone spanning 2.5 days of activity (i wonder how much traffic the wall of sheep looked at). dense metropolitan areas are teaming with signals full of information that was never intended to be disclosed publicly. you should always assume your network traffic is monitored unless proven otherwise (where proof == strong encryption with secure key management). on a related note, this latest draft is informative: http://csrc.nist.gov/publications/drafts/800-21-Rev1_September2005.pdf "Guide for Cryptography In the Federal Government" best regards, From solipsis at pitrou.net Fri Sep 30 09:59:18 2005 From: solipsis at pitrou.net (Antoine Pitrou) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <4ef5fec605092913505674097b@mail.gmail.com> References: <5691356f05092710332a623de2@mail.gmail.com> <1127850423.3A956345@bd12.dngr.org> <1127901049.15818.47.camel@p-dvsi-418-1.rd.francetelecom.fr> <5691356f0509280755501a1c1d@mail.gmail.com> <1127922062.15818.82.camel@p-dvsi-418-1.rd.francetelecom.fr> <5691356f050929103656e4f30f@mail.gmail.com> <4ef5fec605092913505674097b@mail.gmail.com> Message-ID: <1128074358.25688.22.camel@p-dvsi-418-1.rd.francetelecom.fr> > > When I attend DefCon, I am always amazed that people are surprised by > > the Wall of Sheep, people who know that network snooping is possible. > > I guess you just have to experience the efficiency of live network > > snooping in order to truly appreciate it. But that's an argument against your scheme more than against mine. The only way your proposal can provide some kind of security is by using HTTPS. While most people factually only have access to bare HTTP. It may change in the future, but it's not the case right now. Regards Antoine. From afisk at speedymail.org Fri Sep 30 18:03:32 2005 From: afisk at speedymail.org (Adam Fisk) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] [Fwd: [Planetlab-users] Announcing: TCP NAT Traversal/Hole-Punching Library] In-Reply-To: <20050928123943.wxuudevkh6ogwcog@mail.ucla.edu> References: <20050928123943.wxuudevkh6ogwcog@mail.ucla.edu> Message-ID: <433D7DF4.2040005@speedymail.org> This is very much of interest -- thanks Mike! -Adam Michael Parker wrote: >I just got this by way of the PlanetLab mailing list... Definitely thought it >would be of interest! > >- Mike > > >---------------------------- Original Message ---------------------------- >Subject: [Planetlab-users] Announcing: TCP NAT Traversal/Hole-Punching >Library From: "Saikat Guha" >Date: Wed, September 28, 2005 3:46 am >-------------------------------------------------------------------------- > >Hi all, >(apologies if you get multiple copies of this) > >I am pleased to announce the availability of our open-source TCP NAT >Traversal/Hole-Punching library based on our research published in [1]. > >[1] "Characterization and Measurement of TCP Traversal through NATs > and Firewalls", S. Guha and P. Francis. IMC 2005. >http://nutss.net/pub/imc05-tcpnat.pdf > >The key result of the paper is: TCP NAT traversal can work 85%-90% of the >time today (without any special assumptions about NATs), and 100% of the >time between pairs of certain popular, well-behaved NATs. See [1] for more >details. > >An open-source Java library for TCP NAT Traversal is now available: > webpage: http://nutss.net/stunt.php > faq: http://nutss.net/jstunt-faq.php > library and example: http://nutss.net/jstunt-examples.php > >The above library has been tested for pair-wise connectivity across 11 >brands of NATs from Windows and Linux hosts. NATs tested were Linksys, >DLink, Netgear, Belkin, 3Com, Netopia, Allied Telesyn, SMC, Trendnet, USR, >Buffalo Tech. Out of the 121 possible pair-wise combinations, 113 >connections are successful. The only ones that failed are when both the >endpoints are behind the _same_ NAT device that does not support TCP >hairpin-behavior yet (see [1]). > >The java library is released under LGPL; contact me if this does not meet >your needs. Feel free to extend it/port it etc. > >Q: I am a P2P developer/researcher. How does this help me? >A: The library adds TCP NAT traversal out-of-the-box. This increases the >connectivity in your P2P network since two users behind their NATs can now >exchange data without having to go through an intermediary node. You can: >- Use this library as is (for development of P2P software, research, > small deployments, etc in java) >- Study it to provide TCP NAT Traversal in your existing P2P > applications in your language of choice. >- etc. > >If you have any questions, comments, suggestions, or problems, do not >hesitate to contact me. Cheers, > > >------------------------------------------------------------------------ > >_______________________________________________ >Gnucla-devel mailing list >Gnucla-devel@starsky.ee.ucla.edu >http://starsky.ee.ucla.edu/cgi-bin/mailman/listinfo/gnucla-devel > > >------------------------------------------------------------------------ > >_______________________________________________ >p2p-hackers mailing list >p2p-hackers@zgp.org >http://zgp.org/mailman/listinfo/p2p-hackers >_______________________________________________ >Here is a web page listing P2P Conferences: >http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > > From dbarrett at quinthar.com Fri Sep 30 18:18:29 2005 From: dbarrett at quinthar.com (David Barrett) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] [Fwd: [Planetlab-users] Announcing: TCP NAT Traversal/Hole-Punching Library] In-Reply-To: <433D7DF4.2040005@speedymail.org> References: <20050928123943.wxuudevkh6ogwcog@mail.ucla.edu> <433D7DF4.2040005@speedymail.org> Message-ID: <433D8175.80603@quinthar.com> Sheesh, and right when I get my TCP-like UDP congestion control perfected. Adam Fisk wrote: > This is very much of interest -- thanks Mike! > > -Adam > > > Michael Parker wrote: > >> I just got this by way of the PlanetLab mailing list... Definitely >> thought it >> would be of interest! >> >> - Mike >> >> >> ---------------------------- Original Message >> ---------------------------- >> Subject: [Planetlab-users] Announcing: TCP NAT Traversal/Hole-Punching >> Library From: "Saikat Guha" >> Date: Wed, September 28, 2005 3:46 am >> -------------------------------------------------------------------------- >> >> >> Hi all, >> (apologies if you get multiple copies of this) >> >> I am pleased to announce the availability of our open-source TCP NAT >> Traversal/Hole-Punching library based on our research published in [1]. >> >> [1] "Characterization and Measurement of TCP Traversal through NATs >> and Firewalls", S. Guha and P. Francis. IMC 2005. >> http://nutss.net/pub/imc05-tcpnat.pdf >> >> The key result of the paper is: TCP NAT traversal can work 85%-90% of the >> time today (without any special assumptions about NATs), and 100% of the >> time between pairs of certain popular, well-behaved NATs. See [1] for >> more >> details. >> >> An open-source Java library for TCP NAT Traversal is now available: >> webpage: http://nutss.net/stunt.php >> faq: http://nutss.net/jstunt-faq.php >> library and example: http://nutss.net/jstunt-examples.php >> >> The above library has been tested for pair-wise connectivity across 11 >> brands of NATs from Windows and Linux hosts. NATs tested were Linksys, >> DLink, Netgear, Belkin, 3Com, Netopia, Allied Telesyn, SMC, Trendnet, >> USR, >> Buffalo Tech. Out of the 121 possible pair-wise combinations, 113 >> connections are successful. The only ones that failed are when both the >> endpoints are behind the _same_ NAT device that does not support TCP >> hairpin-behavior yet (see [1]). >> >> The java library is released under LGPL; contact me if this does not meet >> your needs. Feel free to extend it/port it etc. >> >> Q: I am a P2P developer/researcher. How does this help me? >> A: The library adds TCP NAT traversal out-of-the-box. This increases the >> connectivity in your P2P network since two users behind their NATs can >> now >> exchange data without having to go through an intermediary node. You can: >> - Use this library as is (for development of P2P software, research, >> small deployments, etc in java) >> - Study it to provide TCP NAT Traversal in your existing P2P >> applications in your language of choice. >> - etc. >> >> If you have any questions, comments, suggestions, or problems, do not >> hesitate to contact me. Cheers, >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Gnucla-devel mailing list >> Gnucla-devel@starsky.ee.ucla.edu >> http://starsky.ee.ucla.edu/cgi-bin/mailman/listinfo/gnucla-devel >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> p2p-hackers mailing list >> p2p-hackers@zgp.org >> http://zgp.org/mailman/listinfo/p2p-hackers >> _______________________________________________ >> Here is a web page listing P2P Conferences: >> http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences >> >> > > _______________________________________________ > p2p-hackers mailing list > p2p-hackers@zgp.org > http://zgp.org/mailman/listinfo/p2p-hackers > _______________________________________________ > Here is a web page listing P2P Conferences: > http://www.neurogrid.net/twiki/bin/view/Main/PeerToPeerConferences > > From lgonze at panix.com Fri Sep 30 20:46:57 2005 From: lgonze at panix.com (Lucas Gonze) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <1128074358.25688.22.camel@p-dvsi-418-1.rd.francetelecom.fr> References: <5691356f05092710332a623de2@mail.gmail.com> <1127850423.3A956345@bd12.dngr.org> <1127901049.15818.47.camel@p-dvsi-418-1.rd.francetelecom.fr> <5691356f0509280755501a1c1d@mail.gmail.com> <1127922062.15818.82.camel@p-dvsi-418-1.rd.francetelecom.fr> <5691356f050929103656e4f30f@mail.gmail.com> <4ef5fec605092913505674097b@mail.gmail.com> <1128074358.25688.22.camel@p-dvsi-418-1.rd.francetelecom.fr> Message-ID: <433DA441.5000501@panix.com> Antoine Pitrou wrote: >The only way your proposal can provide some kind of security is by using >HTTPS. While most people factually only have access to bare HTTP. It may >change in the future, but it's not the case right now. > > Given that: The client has received at least one ID by secure means, and the client wants to fetch the corresponding representation. SSL/TLS is not available. The client has javascript. On the client: $salt = newRandomNumber(); $hashed = hash(concat($id,$salt)) $representation = GET /mapper?hashed=$hashed&salt=$salt On the server: $hashed = http_argument("hashed"); $salt = http_argument("salt"); if( alreadyUsed($salt)) returnError(); addToUsed($salt); while($id = nextSavedID()){ if( eq(hash(concat($id,$salt)), $hashed) ) return(objectForID($id)); } Work on the server is O(number of objects), but work for an attacker is the same as brute force. An attacker who can intercept the GET can perform a man in the middle attack. Work on the server can be reduced to O(1) if the arguments are passed from the client to the server according to this pseudocode: On the client: $pub = GET /mapper?publickey $salt = newRandomNumber(); $package = makeFormattedPackage($salt,$id); $encrypted = encrypt($package) $representation = GET /mapper?encrypted=$encrypted On the server: $decrypted = decrypt($privateKey,http_argument("encrypted")); $package = parseFormattedPackage($decrypted); $id = getPackageElement($package,"id"); return(objectForID($id)); From laurian at gmail.com Fri Sep 30 21:23:27 2005 From: laurian at gmail.com (Laurian Gridinoc) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <433DA441.5000501@panix.com> References: <5691356f05092710332a623de2@mail.gmail.com> <1127850423.3A956345@bd12.dngr.org> <1127901049.15818.47.camel@p-dvsi-418-1.rd.francetelecom.fr> <5691356f0509280755501a1c1d@mail.gmail.com> <1127922062.15818.82.camel@p-dvsi-418-1.rd.francetelecom.fr> <5691356f050929103656e4f30f@mail.gmail.com> <4ef5fec605092913505674097b@mail.gmail.com> <1128074358.25688.22.camel@p-dvsi-418-1.rd.francetelecom.fr> <433DA441.5000501@panix.com> Message-ID: <9782e3350509301423x2d32af2fn12fe5b0f8e452eb8@mail.gmail.com> Hello On 9/30/05, Lucas Gonze wrote: > Given that: > The client has received at least one ID by secure means, and the client > wants to fetch the corresponding representation. > SSL/TLS is not available. > The client has javascript. > On the client: > $salt = newRandomNumber(); > $hashed = hash(concat($id,$salt)) > $representation = GET /mapper?hashed=$hashed&salt=$salt > [...] The issue is how does the client know that the login page with the above javascript was not modified by the MITM? There is an ongoing discussion on webappsec@securityfocus.com about "Must we authenticate login forms (using SSL?)?" see below a nice issue about challenge-response and MITM: ---------- Forwarded message ---------- From: Rogan Dawes Date: Sep 30, 2005 11:54 AM Subject: Re: Must we authenticate login forms (using SSL?)? Cc: Web Application Security [...] The core problem with any security mechanism that is implemented using "stuff" (Javascript, Java applet, etc) that is downloaded from the server that you are visiting is that any man in the middle attack can simply remove that mechanism, and substitute it with their own. e.g. My bank logon script performs an MD5 hash of the username and password before sending it to the bank. The MITM tricks me to visiting their own site, and just "proxies" the comms to the real site. However, they strip out the MD5 hashing script,and replace it with an "identity" function (i.e. the output is the same as the input). When the MITM receives the form submission, it is trivial for them to extract the username and password from the form, replace it with the MD5 hash expected, and pass it on to the real bank. I had started implementing/experimenting with using Secure Remote Password (http://srp.stanford.edu/) for authentication to a secure web site, with the idea that the password is never transmitted in clear, or in a recoverable form. However, this stumbling point convinced me that I'd be wasting my time! One reason that SSL client certs offer real protection is that they are not controllable by page content, and do not rely on something that comes down from the site you are visiting. Regards, Rogan ---------- End Forwarded message ---------- From lgonze at panix.com Fri Sep 30 21:41:55 2005 From: lgonze at panix.com (Lucas Gonze) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] Re: [rest-discuss] Re: RESTful authorization In-Reply-To: <9782e3350509301423x2d32af2fn12fe5b0f8e452eb8@mail.gmail.com> References: <5691356f05092710332a623de2@mail.gmail.com> <1127850423.3A956345@bd12.dngr.org> <1127901049.15818.47.camel@p-dvsi-418-1.rd.francetelecom.fr> <5691356f0509280755501a1c1d@mail.gmail.com> <1127922062.15818.82.camel@p-dvsi-418-1.rd.francetelecom.fr> <5691356f050929103656e4f30f@mail.gmail.com> <4ef5fec605092913505674097b@mail.gmail.com> <1128074358.25688.22.camel@p-dvsi-418-1.rd.francetelecom.fr> <433DA441.5000501@panix.com> <9782e3350509301423x2d32af2fn12fe5b0f8e452eb8@mail.gmail.com> Message-ID: <433DB123.8050001@panix.com> Laurian Gridinoc wrote: >The issue is how does the client know that the login page with the >above javascript was not modified by the MITM? > > Or that the server ever really existed except as an attack vector, or any number of other problems with bootstrapping secure systems. The important thing is that the attacker has to perform their MITM attack on the bootstrap rather than any later stage. This gives the disadvantage to the attacker because: - it can't know in advance who it will want to attack later, giving it a large and diffuse set of targets. - the defenders have a small and focused point of vulnerability. For the purpose of friendnets, the advantage is overwhelmingly on the defenders' side. From srhea at cs.berkeley.edu Fri Sep 30 22:16:12 2005 From: srhea at cs.berkeley.edu (Sean Rhea) Date: Sat Dec 9 22:13:02 2006 Subject: [p2p-hackers] CFP: NetDB'06 Message-ID: <15738F3A-FBC8-4E42-A9AE-8672839AD126@cs.berkeley.edu> Apologies for duplicates. Sean -------------------------------------------------------------------- C A L L F O R P A P E R S 2nd IEEE International Workshop on Networking Meets Databases (NetDB'06) April 2, 2006 / Atlanta, GA, USA http://www.cs.brown.edu/research/db/netdb06/ in cooperation with 22nd IEEE Conference on Data Engineering (ICDE 2006) -------------------------------------------------------------------- The Second Workshop on Networking Meets Databases, NetDB'06, will bring together researchers in the networking and database communities. We are witnessing the blurring of the traditional boundaries between these two disciplines, most clearly in the emerging areas of peer-to-peer networks, sensor networks, and distributed information systems. The goal of the workshop is to promote discussion of ideas that will influence and foster continued research that draws heavily from both communities. The workshop will provide a venue for researchers to discuss key challenges and present new ideas that can significantly impact both communities, and perhaps give birth to a new community in the long term. We encourage submissions across the broad range of topics that lie at the intersection of networking and databases, specifically including peer-to-peer systems, sensor networks, widely-distributed query processing, data dissemination, and networked storage. Topics of interest in this context include, but are not limited to: * Architectures and applications for widely-distributed systems * Data analysis for network traffic estimation and security * Data mining and retrieval in widely-distributed systems * Data models, query models, and query languages for networking * Networked data placement and storage * Distributed data structures for data management * Distributed stream processing and dissemination * Dynamic schema integration in peer-to-peer and sensor networks * Indexing, caching, and replication techniques for wide-area storage * Query planning, execution, and optimization in networked systems * Transaction management for peer-to-peer networks The selection of NetDB papers will be based primarily on their originality and potential to influence future research. This influence can be exercised in many ways, exemplified by but not limited to the following: * Describing a novel approach to an old problem * Describing a new problem that requires our attention * Articulating a new perspective about networking and databases * Debunking an old perspective about networking or databases Submissions should be in the form of extended abstracts following the IEEE double-column format and no longer than 6 pages. Detailed submission information will be posted on the workshop web site (http://www.cs.brown.edu/research/db/netdb06/). The review process is NOT blind - each contributing author should be included on the first page. Only electronic submissions in PostScript or PDF will be accepted. Submissions must be written in English, render without error using standard tools (Ghostview or Acrobat Reader) and print on US Letter paper. Important Dates: Submissions due : Nov 7, 2005 Notification of acceptance: Dec 16, 2005 Camera-ready copy due : Jan 13, 2006 Program Co-Chairs: Ugur Cetintemel (Brown) John Jannotti (Brown) Steering Committee: Hari Balakrishnan (MIT) Michael Franklin (UC Berkeley) Ramesh Govindan (USC) Cyrus Shahabi (USC) Program Committee: Amr El Abbadi (UC Santa Barbara) Karl Aberer (EPFL, Lausanne) Magdalena Balazinska (MIT) Alex Buchmann (Darmstadt U. of Technology) John Byers (Boston U.) Amol Desphande (UMD, College Park) Minos Garofalakis (Intel Research) Frans Kaashoek (MIT) Peter Keleher (UMD, College Park) Dejan Kostic (EPFL, Lausanne) Alex Labrinidis (U. Pittsburgh) Wang Chien Lee (Pennsylvania State U.) Philip Levis (Stanford) Sam Madden (MIT) Peter Pietzuch (Harvard) Krithi Ramamritham (IIT, Bombay) Lakshmish Ramaswamy (U. Georgia) Sean Rhea (UC Berkeley) Scott Shenker (ICSI and UC Berkeley) Oliver Spatscheck (AT&T Labs) Ion Stoica (UC Berkeley) Ouri Wolfson (U. Illinois, Chicago) Jim Xu (Georgia Tech) -- But to say that the race is the metaphor for the life is to miss the point. The race is everything. It obliterates whatever isn't racing. Life is the metaphor for the race. -- Donald Antrim -------------- next part -------------- A non-text attachment was scrubbed... Name: PGP.sig Type: application/pgp-signature Size: 186 bytes Desc: This is a digitally signed message part Url : http://zgp.org/pipermail/p2p-hackers/attachments/20050930/b9a51677/PGP.pgp