Received: from sog-mx-2.v43.ch3.sourceforge.com ([172.29.43.192] helo=mx.sourceforge.net) by sfs-ml-3.v29.ch3.sourceforge.com with esmtp (Exim 4.76) (envelope-from ) id 1VS6wj-0007Wj-AI for bitcoin-development@lists.sourceforge.net; Fri, 04 Oct 2013 15:05:37 +0000 Received-SPF: pass (sog-mx-2.v43.ch3.sourceforge.com: domain of bitpay.com designates 74.125.82.181 as permitted sender) client-ip=74.125.82.181; envelope-from=jgarzik@bitpay.com; helo=mail-we0-f181.google.com; Received: from mail-we0-f181.google.com ([74.125.82.181]) by sog-mx-2.v43.ch3.sourceforge.com with esmtps (TLSv1:RC4-SHA:128) (Exim 4.76) id 1VS6wi-0003c6-FW for bitcoin-development@lists.sourceforge.net; Fri, 04 Oct 2013 15:05:37 +0000 Received: by mail-we0-f181.google.com with SMTP id p61so4617082wes.40 for ; Fri, 04 Oct 2013 08:05:30 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:mime-version:in-reply-to:references:date :message-id:subject:from:to:cc:content-type; bh=DWh3pBg9UjtKp4UQmolIFKac8tSHZ0Wc+6hNPA4g3P8=; b=WEN10ytMnnSEtLtic8ps3NCeswtwtQUTSDQQqW6lgjaUq0DFTjGc1Nmj7f/wuDy1tr XCyMesZn8B0EGV5CNuDBg1Z2GULQWvyXQELZEDI+CSN1PxOtnD5+p3Xq3jv8INU4Ogf0 zdypWPlyvl3ZhqDH6p5WIsTNQBN75+nppBW34gtCjV7bxx8UiwFZoUacdIZnrSem61MQ LOgfTMuLHC5slvsCpK82WQS7wyZpRtOcyscmS1F5VqTG4kdH2PQRZIg0WKfQBb15hZE1 xfdB8R9D3yaLZV8ddbdizKwP4durfir+H8Md+ci1YLsMNXy+fP7mnubdyQEPLM8lBKee SzlQ== X-Gm-Message-State: ALoCoQn4rqrlFIVtkKn/+5nX9X2QyXN4JZnuGYxMgcxvEI6xAanqA3rzaCe37M4j0nJa2V9kM0tq MIME-Version: 1.0 X-Received: by 10.180.182.228 with SMTP id eh4mr7664097wic.45.1380899130022; Fri, 04 Oct 2013 08:05:30 -0700 (PDT) Received: by 10.194.236.69 with HTTP; Fri, 4 Oct 2013 08:05:29 -0700 (PDT) In-Reply-To: References: Date: Fri, 4 Oct 2013 11:05:29 -0400 Message-ID: From: Jeff Garzik To: Gavin Andresen Content-Type: text/plain; charset=ISO-8859-1 X-Spam-Score: -1.6 (-) X-Spam-Report: Spam Filtering performed by mx.sourceforge.net. See http://spamassassin.org/tag/ for more details. 0.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: github.com] -1.5 SPF_CHECK_PASS SPF reports sender host as permitted sender for sender-domain -0.0 SPF_PASS SPF: sender matches SPF record -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature X-Headers-End: 1VS6wi-0003c6-FW Cc: "bitcoin-development@lists.sourceforge.net" Subject: Re: [Bitcoin-development] bitcoind stops responding X-BeenThere: bitcoin-development@lists.sourceforge.net X-Mailman-Version: 2.1.9 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Oct 2013 15:05:37 -0000 On Fri, Oct 4, 2013 at 2:22 AM, Gavin Andresen wrote: > RE: running into the maximum-of-4-keepalive-requests : simple workaround is > to run with -rpcthreads=11 (or however many keepalive connections you need > to support). I agree that the rpc code should be smarter; making the last > rpc thread ignore keepalive and always disconnecting should be a fairly > simple patch, and "patches welcome." It's all still working around a problem unchanged since Satoshi wrote it: the HTTP server code paths use blocking I/O. Amusingly, we do this through an async I/O library, which helps facilitate SSL, but all our connections and operations are blocking. That's why RPC was multi-threaded in part: to work around the ugly blocking nature of the code. At least with multiple threads, one thread will not stall another even if the network stalls (or a software bug triggers a stall etc.) Hopefully I can dive into the code and make is truly async I/O. It takes some work, and I'm happy if someone else steals the task, but that's what really needs to be done. I tried the multi-threaded approach, writing an entire boost::asio skeleton JSON-RPC HTTP server: https://github.com/jgarzik/rpcsrv This is working, tested code that uses boost::asio properly. It's also quite a bit of LOC, and a bit messy to boot (four LOC per boost async action and incomprehensible compiler errors in return for proper async+MT). A single thread with async I/O is likely sufficient for even heavy uses of RPC -- since today it all goes through a big lock anyway. -- Jeff Garzik Senior Software Engineer and open source evangelist BitPay, Inc. https://bitpay.com/