Return-Path: Received: from smtp1.linuxfoundation.org (smtp1.linux-foundation.org [172.17.192.35]) by mail.linuxfoundation.org (Postfix) with ESMTPS id 5DAA784EB for ; Sun, 3 Feb 2019 20:15:32 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.7.6 Received: from mail-wr1-f43.google.com (mail-wr1-f43.google.com [209.85.221.43]) by smtp1.linuxfoundation.org (Postfix) with ESMTPS id C1CCF165 for ; Sun, 3 Feb 2019 20:15:31 +0000 (UTC) Received: by mail-wr1-f43.google.com with SMTP id z5so12303282wrt.11 for ; Sun, 03 Feb 2019 12:15:31 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:content-transfer-encoding:mime-version:subject:message-id:date :cc:to; bh=P+QfBuooBNNlyuoeOTvy/UOxq+DLAfyeS7G4WyouP2Q=; b=eXlAl6jV9M6GmRlfbmy2HWdvf+Ivkfti8Sv/jSqc3aBI7EX5B2Smk1KKNe2Y/GmK34 2dGRrJpaQDAP3hz6mgy4UCwi2hI0U/q5psEha3qZoIgWxxJS1PTe7clGr0bPa3c+ABcj aoXiQWgY8SZhU074XN3qlhCYag54bfrmY1jAHoWqe5aCZO7veIiRPRxaIA42yd6R/4hi jhufgKG2j1FXXXaAT2lzzmsBpo5M4BHQHxUxCrInG4X0XISXSk8FLFlpP+oGtKaeeFdi Yl1lGhBIr1UHi2UZgRB8hFM4a3e5od2HPkJeMnPNgFbt1oHJiHNjVQ8rRZ9SxXKHk0Hk EOWA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:content-transfer-encoding:mime-version :subject:message-id:date:cc:to; bh=P+QfBuooBNNlyuoeOTvy/UOxq+DLAfyeS7G4WyouP2Q=; b=IYaXBi6SDpKexn+9rillSGsZb050sNUfa+MJ+/UYPxEh9nqp+qJ74luDfYkskSNm6R zuGOvs1YFX8oZGaw9RqFApBjNzgUqKddcNVG7w509WrDvAss4Z+6efwaRlzqplpkGgj4 QFJbLX2ZXjL4KJ+V9qaOm6J54xzOSmMuahyDTNJHCo0VYbfzXMcJK33z7shkzk849e9N 44b5cu7A1wbNVRBn1njZuyPmCRu/w4NN/M5ROdeFT9eEEMY2+a0CscoBEVsLQ7L5q1SO 3qO5IUVptF+HScxUIUqSuackt+6qbggDRX762jrMdsQrJkb6phVfciIyeP7+NCse9be+ C24A== X-Gm-Message-State: AJcUukcuOlNhRimDaH4Ma8DPETblv1MYL36YjluFUzVB5Yig+j2og78a dgduu2pfRU/iqaL2jE9Vktrzx1OB X-Google-Smtp-Source: ALg8bN5aW+gDOSzhVWECIN27Dm9b9xHUQrRsOdJLH/XESO15LEQKUtuYl5ejYM2qOX50HmP+BN9dFA== X-Received: by 2002:adf:a78a:: with SMTP id j10mr47344054wrc.191.1549224929968; Sun, 03 Feb 2019 12:15:29 -0800 (PST) Received: from p200300dd672d1a81e0b322f99ad6a3db.dip0.t-ipconnect.de (p200300DD672D1A81E0B322F99AD6A3DB.dip0.t-ipconnect.de. [2003:dd:672d:1a81:e0b3:22f9:9ad6:a3db]) by smtp.gmail.com with ESMTPSA id k23sm7079369wmj.32.2019.02.03.12.15.28 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Feb 2019 12:15:29 -0800 (PST) From: Tamas Blummer Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Message-Id: <9509A1AC-287A-48E9-84E9-D507D6A2A4CD@gmail.com> Date: Sun, 3 Feb 2019 21:15:25 +0100 To: bitcoin-dev@lists.linuxfoundation.org X-Mailer: Apple Mail (2.3273) X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID, DKIM_VALID_AU, FREEMAIL_FROM, RCVD_IN_DNSWL_NONE autolearn=ham version=3.3.1 X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on smtp1.linux-foundation.org X-Mailman-Approved-At: Sun, 03 Feb 2019 20:50:04 +0000 Cc: jimpo@coinbase.com Subject: [bitcoin-dev] BIP157 server Murmel introduced, enchancement suggestion to BIP158 X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.12 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Feb 2019 20:15:32 -0000 TLDR: I suggest to add outpoint filter to BIP158 as it proved to be = useful while developing a filter server and allows further checks in = filter client. Murmel is my project within the rust-bitcoin community. = https://github.com/rust-bitcoin/murmel Its goal is to provide a lightweight, at least SPV security, settlement = layer for the Lightning Network implementation in Rust. = https://github.com/rust-bitcoin/rust-lightning Murmel relies on BIP157 (Client Side Block Filtering). Since Bitcoin = Core does not yet support this protocol extension, I also added filter = and block server functionality to Murmel and this might be useful for=20 development purposes any other BIP157 client project. You may compile and run a Murmel filter server to support your client = development. It bootstraps within a few hours. Follow the instructions = at: https://github.com/rust-bitcoin/murmel While implementing both client and server side I made an observation = that should be considered for BIP158: BIP158 specifies base filter containing all scripts spent or created by = a block (except those with OP_RETURN). I found it useful to also compute = a filter on spent and created outpoints. The Murmel filter server consults these outpoint filters to find the = transactions with the spent scripts while computing the base script = filter. Since outpoints usually getting spent shortly after created, = this approach works well enough to keep up with the blockchain, although far too slow to rely on it while = bootstrapping. An advantage of this approach of looking up UTXO is that = there is nothing to be recomputed at re-org, filters are consulted = following the path from current tip back to genesis. This fits well with = Murmel=E2=80=99s storage, that is my other project Hammersbald = https://github.com/rust-bitcoin/hammersbald, a highly efficient truly = append only blockchain store in Rust. Filter matching is also nicely parallelizable looking up subsets of = spent outputs in parallel.=20 A lightweight client can use outpoint filters to efficiently validate = spent coins or miner reward, which goes beyond SPV guarantees. This is = probabilistically possible now, and definitely once filters are = committed. For above reasons I suggest to also add outpoint filter to BIP158, so = filter servers may support it, as does Murmel. Murmel is moving quickly; = I tagged the version as of this mail with DEVLIST for later reference. Tamas Blummer=