Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 03FF5B35 for ; Tue, 18 Apr 2017 10:34:07 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-yw0-f179.google.com (mail-yw0-f179.google.com [209.85.161.179]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id 339DF14F for ; Tue, 18 Apr 2017 10:34:06 +0000 (UTC) Received: by mail-yw0-f179.google.com with SMTP id k13so67386181ywk.1 for ; Tue, 18 Apr 2017 03:34:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc; bh=YGMmUSqHpoT+cXoY/grLm9dh3BuRFsOVyhdPOPvAZYM=; b=Ape3q4H7xZNwXQq7/eA7Z/fokreUBUUpMp4vypY98kIrg86ntklGewaFBxsO5FWvU7 BGFs5+SSC8DCXpHWA6dVOhZ6kBR6SXHkQHylLD4yftilbeFJv1l3p9OFeSwoP+AdK5ad 7slTBOviJ+0J9g6iiX+6PGgO75ADY7jtE+BKaRvFFkMohRNF9KT9J8fl2+IoMEZts1mS 3C/RXFiAocOu+vdabircsEapWAikHcZ6uFQxH90hUje/rlFxI/eGlfaQVKvAPo3jLp7o Udxw6eE9iu2JTrZmjRVfMLND7nPs9pWVAOqRWrpSzMYpaqciP5uwKLHRX0U7m77DrxSx Ap0w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:in-reply-to:references:from:date :message-id:subject:to:cc; bh=YGMmUSqHpoT+cXoY/grLm9dh3BuRFsOVyhdPOPvAZYM=; b=iwTqw7oYTPKRoF61cXuGIJr8W7smTjLiEXfXcUAPUDsrmqMosnhino3IEefYf26RRs wzdSLsbi0TDRwV7sXKLPqoM93k0NJJ1udaWcAOH4d699baSrvnBue6M6bhMMaM8L9X61 sTU69HzSAXlvZ1Wz17fbMcSmIe0Nu1MqN3e+CEuhSH1vAI02mgO6ubxx8Q5qOvTT56M+ KE8baJaf8L2BU37iXbg1MRiyGcpgJ3QwIuvhC4DsXYvI+Sv3EsfqedJfauNEPO9e5Eem /Z6YBiWtpayA/8N5lVk+LP22xHiowXoVBKJy0dwwW61AwsHf/Fo1GgrZF2CmHFeg67S2 ZubQ== X-Gm-Message-State: AN3rC/5hwmEQAw2Ds28C6EXdNWvTJG6FNBVOfE3shZaQ05FR7TimDZfU Num4GKrQW4rbD76FcsatgbeGi0IVbg== X-Received: by 10.13.221.208 with SMTP id g199mr18369430ywe.21.1492511645387; Tue, 18 Apr 2017 03:34:05 -0700 (PDT) MIME-Version: 1.0 Received: by 10.37.35.88 with HTTP; Tue, 18 Apr 2017 03:34:04 -0700 (PDT) Received: by 10.37.35.88 with HTTP; Tue, 18 Apr 2017 03:34:04 -0700 (PDT) In-Reply-To: References: From: Natanael Date: Tue, 18 Apr 2017 12:34:04 +0200 Message-ID: To: Erik Aronesty Content-Type: multipart/alternative; boundary=94eb2c06e3fa7251fe054d6e7289 X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, HTML_MESSAGE, RCVD_IN_DNSWL_NONE, RCVD_IN_SORBS_SPAM autolearn=no version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org Cc: Bitcoin Dev Subject: [bitcoin-dev] Properties of an ideal PoW algorithm & implementation X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Apr 2017 10:34:07 -0000 --94eb2c06e3fa7251fe054d6e7289 Content-Type: text/plain; charset=UTF-8 To expand on this below; Den 18 apr. 2017 00:34 skrev "Natanael" : IMHO the best option if we change PoW is an algorithm that's moderately processing heavy (we still need reasonably fast verification) and which resists partial state reuse (not fast or fully "linear" in processing like SHA256) just for the sake of invalidating asicboost style attacks, and it should also have an existing reference implementation for hardware that's provably close in performance to the theoretical ideal implementation of the algorithm (in other words, one where we know there's no hidden optimizations). [...] The competition would mostly be about packing similar gate designs closely and energy efficiency. (Now that I think about it, the proof MAY have to consider energy use too, as a larger and slower but more efficient chip still is competitive in mining...) What matters for miners in terms of cost is primarily (correctly computed) hashes per joule (watt-seconds). The most direct proxy for this in terms of algorithm execution is the number of transistor (gate) activations per computed hash (PoW unit). To prove that an implementation is near optimal, you would show there's a minimum number of necessary transistor activations per computed hash, and that your implementation is within a reasonable range of that number. We also need to show that for a practical implementation you can't reuse much internal state (easiest way is "whitening" the block header, pre-hashing or having a slow hash with an initial whitening step of its own). This is to kill any ASICBOOST type optimization. Performance should be constant, not linear relative to input size. The PoW step should always be the most expensive part of creating a complete block candidate! Otherwise it loses part of its meaning. It should however still also be reasonably easy to verify. Given that there's already PoW ASIC optimizations since years back that use deliberately lossy hash computations just because those circuits can run faster (X% of hashes are computed wrong, but you get Y% more computed hashes in return which exceeds the error rate), any proof of an implementation being near optimal (for mining) must also consider the possibility of implementations of a design that deliberately allows errors just to reduce the total count of transistor activations per N amount of computed hashes. Yes, that means the reference implementation is allowed to be lossy. So for a reasonably large N (number of computed hashes, to take batch processing into consideration), the proof would show that there's a specific ratio for a minimum number of average gate activations per correctly computed hash, a smallest ratio = X number of gate activations / (N * success rate) across all possible implementations of the algorithm. And you'd show your implementation is close to that ratio. It would also have to consider a reasonable range of time-memory tradeoffs including the potential of precomputation. Hopefully we could implement an algorithm that effectively makes such precomputation meaningless by making the potential gain insignificant for any reasonable ASIC chip size and amount of precomputation resources. A summary of important mining PoW algorithm properties; * Constant verification speed, reasonably fast even on slow hardware * As explained above, still slow / expensive enough to dominate the costs of block candidate creation * Difficulty must be easy to adjust (no problem for simple hash-style algorithms like today) * Cryptographic strength, something like preimage resistance (the algorithm can't allow forcing a particular output, the chance must not be better than random within any achievable computational bounds) * As explained above, no hidden shortcuts. Everybody has equal knowledge. * Predictable and close to constant PoW computation performance, and not linear in performance relative to input size the way SHA256 is (lossy implementations will always make it not-quite-constant) * As explained above, no significant reusable state or other reusable work (killing ASICBOOST) * As explained above, no meaningful precomputation possible. No unfair headstarts. * Should only rely on just transistors for implementation, shouldn't rely on memory or other components due to unknowable future engineering results and changes in cost * Reasonably compact implementation, measured in memory use, CPU load and similar metrics * Reasonably small inputs and outputs (in line with regular hashes) * All mining PoW should be "embarrassingly parallel" (highly parallellizable) with minimal or no gain from batch computation, performance scaling should be linear with increased chip size & cycle speed. What else is there? Did I miss anything important? --94eb2c06e3fa7251fe054d6e7289 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
To e= xpand on this below;

Den 18 apr. 2017 00:34 skrev "Natanael" <natanael.l@gmail.com>:
IMHO the best option if we change P= oW is an algorithm that's moderately processing heavy (we still need re= asonably fast verification) and which resists partial state reuse (not fast= or fully "linear" in processing like SHA256) just for the sake o= f invalidating asicboost style attacks, and it should also have an existing= reference implementation for hardware that's provably close in perform= ance to the theoretical ideal implementation of the algorithm (in other wor= ds, one where we know there's no hidden optimizations).=C2=A0

[...] The competition would mostl= y be about packing similar gate designs closely and energy efficiency. (Now= that I think about it, the proof MAY have to consider energy use too, as a= larger and slower but more efficient chip still is competitive in mining..= .)=C2=A0

What matters for miners in terms of cost is primarily (corr= ectly computed) hashes per joule (watt-seconds). The most direct proxy for = this in terms of algorithm execution is the number of transistor (gate) act= ivations per computed hash (PoW unit).=C2=A0

To prove that an implementation is near optimal, you w= ould show there's a minimum number of necessary transistor activations = per computed hash, and that your implementation is within a reasonable rang= e of that number.=C2=A0

= We also need to show that for a practical implementation you can't reus= e much internal state (easiest way is "whitening" the block heade= r, pre-hashing or having a slow hash with an initial whitening step of its = own). This is to kill any ASICBOOST type optimization. Performance should b= e constant, not linear relative to input size.=C2=A0

The PoW step should always be the most expensi= ve part of creating a complete block candidate! Otherwise it loses part of = its meaning. It should however still also be reasonably easy to verify.=C2= =A0

Given that there'= ;s already PoW ASIC optimizations since years back that use deliberately lo= ssy hash computations just because those circuits can run faster (X% of has= hes are computed wrong, but you get Y% more computed hashes in return which= exceeds the error rate), any proof of an implementation being near optimal= (for mining) must also consider the possibility of implementations of a de= sign that deliberately allows errors just to reduce the total count of tran= sistor activations per N amount of computed hashes. Yes, that means the ref= erence implementation is allowed to be lossy.=C2=A0
=
So for a reasonably large N (number of computed= hashes, to take batch processing into consideration), the proof would show= that there's a specific ratio for a minimum number of average gate act= ivations per correctly computed hash, a smallest ratio =3D X number of gate= activations / (N * success rate) across all possible implementations of th= e algorithm. And you'd show your implementation is close to that ratio.= =C2=A0

It would also hav= e to consider a reasonable range of time-memory tradeoffs including the pot= ential of precomputation. Hopefully we could implement an algorithm that ef= fectively makes such precomputation meaningless by making the potential gai= n insignificant for any reasonable ASIC chip size and amount of precomputat= ion resources.=C2=A0

A s= ummary of important mining PoW algorithm properties;

* Constant verification speed, reasonably fast= even on slow hardware=C2=A0

* As explained above, still slow / expensive enough to dominate the co= sts of block candidate creation

* Difficulty must be easy to adjust (no problem for simple hash-s= tyle algorithms like today)=C2=A0

* Cryptographic strength, something like preimage resistance (the= algorithm can't allow forcing a particular output, the chance must not= be better than random within any achievable computational bounds)=C2=A0

* As explained above, no h= idden shortcuts. Everybody has equal knowledge.=C2=A0

* Predictable and close to constant PoW compu= tation performance,=C2=A0and=C2=A0not linear in performance relat= ive to input size the way SHA256 is=C2=A0(lossy implementations will= always make it not-quite-constant)=C2=A0

=
* As explained above, no significant reusable state or ot= her reusable work (killing ASICBOOST)=C2=A0

* As explained above, no meaningful precomputation poss= ible. No unfair headstarts.=C2=A0

* Should only rely on just transistors for implementation, should= n't rely on memory or other components due to unknowable future enginee= ring results and changes in cost

* Reasonably compact implementation, measured in memory use, CPU = load and similar metrics=C2=A0

* Reasonably small inputs and outputs (in line with regular hashes)= =C2=A0

* All mining PoW = should be "embarrassingly parallel" (highly parallellizable) with= minimal or no gain from batch computation, performance scaling should be l= inear with increased chip size & cycle speed.=C2=A0

What else is there? Did I miss anything imp= ortant?
--94eb2c06e3fa7251fe054d6e7289--