1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
|
Return-Path: <gmaxwell@gmail.com>
Received: from smtp2.osuosl.org (smtp2.osuosl.org [IPv6:2605:bc80:3010::133])
by lists.linuxfoundation.org (Postfix) with ESMTP id 4AFC2C000B
for <bitcoin-dev@lists.linuxfoundation.org>;
Sat, 24 Apr 2021 19:17:57 +0000 (UTC)
Received: from localhost (localhost [127.0.0.1])
by smtp2.osuosl.org (Postfix) with ESMTP id 235E5400D2
for <bitcoin-dev@lists.linuxfoundation.org>;
Sat, 24 Apr 2021 19:17:57 +0000 (UTC)
X-Virus-Scanned: amavisd-new at osuosl.org
X-Spam-Flag: NO
X-Spam-Score: -2.1
X-Spam-Level:
X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5
tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
RCVD_IN_DNSWL_NONE=-0.0001, SPF_PASS=-0.001]
autolearn=ham autolearn_force=no
Authentication-Results: smtp2.osuosl.org (amavisd-new);
dkim=pass (2048-bit key) header.d=gmail.com
Received: from smtp2.osuosl.org ([127.0.0.1])
by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024)
with ESMTP id w7wKni6XXBr5
for <bitcoin-dev@lists.linuxfoundation.org>;
Sat, 24 Apr 2021 19:17:55 +0000 (UTC)
X-Greylist: whitelisted by SQLgrey-1.8.0
Received: from mail-ed1-x532.google.com (mail-ed1-x532.google.com
[IPv6:2a00:1450:4864:20::532])
by smtp2.osuosl.org (Postfix) with ESMTPS id 7A7944013E
for <bitcoin-dev@lists.linuxfoundation.org>;
Sat, 24 Apr 2021 19:17:55 +0000 (UTC)
Received: by mail-ed1-x532.google.com with SMTP id bx20so59980079edb.12
for <bitcoin-dev@lists.linuxfoundation.org>;
Sat, 24 Apr 2021 12:17:55 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=mime-version:references:in-reply-to:from:date:message-id:subject:to
:content-transfer-encoding;
bh=ThcwX9DV01qS59d1Gu+OX7SIO73krz+mold1zrWx0oE=;
b=Q+hUsChJf082EUP7M3rXLvoo44dLIpEYFdtJ+rjNReYQxgdK5Qmt/leffuk4UUJZHQ
UjNKaVye/5onZptdHTvckWTPsu/I/EwCQJs2p8nyOLXNXNLbORRXWSFx/BY28h/LfTru
FumsQDUz6uW6vfrHim/HLRnQLnq4Mb4JOE3GzjZXx/FixP7TFEvInIUAlC6Bjmf2aUTB
9zdspNU6fAWs+faIuDVqi2EFKSWcVnFpgoVEWoqUPVxopiWAdupRWn6p0o8+j6mpBJuB
+0sCX5qCxmA0NrSYakbIjqMZgKGys/OyVb0CZjTII8HfapSc0FaonTUYJTFOxSp0Yxj8
NYgA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to:content-transfer-encoding;
bh=ThcwX9DV01qS59d1Gu+OX7SIO73krz+mold1zrWx0oE=;
b=q6pm/GZ8dFo9xJQVZZqY/WJExTLyMi0eq4PuFZALavUODp8/o+gQte+eVLBvgDkd2q
hyzAOXP0u9spVlujqAPc2SJH4D8DiIGokn+KQEiBmkcr4+XUPnfd6uPaNmZKNnSUsyA+
GpZSdcDHN8tA0+ej8NHF9/MC5qb+w9IHcnh+Q8Q6H3YmtDxipv5wogidvp5btouszmnM
SEmYiETQpbedLWmYIUF+kW6H48L0dilPPimK+ycePwHc7ikA+xxvt6cFRmqWQ2z6FogO
pccNxXQo2PEdKMj4ihT8akCgP3lQvMqSmf1WNHixVbiryB0jU7hwU0mQfeCRDHIxFbB1
fmTw==
X-Gm-Message-State: AOAM533DmCuycRXNwXwLyFMMufdnk/yZZIjYHWhYOEIIOBk0V3EA8Kgd
d7jtNOThee99pXzC8dChuO1xk/UwAZNY0D7qHVI=
X-Google-Smtp-Source: ABdhPJzo6TI1xmpNY9fERs4J8N921FBQFexX4cd7lsSVIxpdK5nUKR7EdDznuXTaY/WuOUJ7Xu/YBpM8ZByrpL+2b3U=
X-Received: by 2002:a05:6402:46:: with SMTP id
f6mr11750377edu.252.1619291873701;
Sat, 24 Apr 2021 12:17:53 -0700 (PDT)
MIME-Version: 1.0
References: <trinity-e57863c6-6327-47b5-b345-623b3f985200-1619262151558@3c-app-mailcom-lxa11>
In-Reply-To: <trinity-e57863c6-6327-47b5-b345-623b3f985200-1619262151558@3c-app-mailcom-lxa11>
From: Greg Maxwell <gmaxwell@gmail.com>
Date: Sat, 24 Apr 2021 19:17:41 +0000
Message-ID: <CAAS2fgQvJv-r9JeL4LaH9bDBho6cgBStFtNC-mskhKcH9Y29AQ@mail.gmail.com>
To: NITSOPOULOS KONSTANTINOS <NITSOPOULOS@mail.com>,
Bitcoin Protocol Discussion <bitcoin-dev@lists.linuxfoundation.org>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Mailman-Approved-At: Sat, 24 Apr 2021 19:29:35 +0000
Subject: Re: [bitcoin-dev] 7z block compression, 18%
X-BeenThere: bitcoin-dev@lists.linuxfoundation.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Bitcoin Protocol Discussion <bitcoin-dev.lists.linuxfoundation.org>
List-Unsubscribe: <https://lists.linuxfoundation.org/mailman/options/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=unsubscribe>
List-Archive: <http://lists.linuxfoundation.org/pipermail/bitcoin-dev/>
List-Post: <mailto:bitcoin-dev@lists.linuxfoundation.org>
List-Help: <mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=help>
List-Subscribe: <https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev>,
<mailto:bitcoin-dev-request@lists.linuxfoundation.org?subject=subscribe>
X-List-Received-Date: Sat, 24 Apr 2021 19:17:57 -0000
https://bitcointalk.org/index.php?topic=3D5303978.msg55946632#msg55946632
Blockstream satellite codebase includes an alternative serialization
that works a single transaction at a time and saves ~20%. This avoids
violating layering, preserves single txn access, is compatible with
transaction relay. (and if you wanted could also be fed to a
multiple txn at a time compressor for additional savings). This
format makes intelligent use of the entirety of the transaction
format, not just amount encodings but even optimizing predictable
hashes (e.g. for p2sh embedded segwit) -- achieving savings that *no*
generic compressor could ever achieve.
Most of the savings from generic compressors comes from repeated
pubkeys and addresses, so it's highly variable, depends on address
reuse, and will likely be diminished by taproot (due to shrinking
multisig) and very much reduced by signature aggregation in the
future.
If you're thinking about use across networks generic compression
interfaces like zlib have had a pretty disappointing security track
record.
If you search the list archive and bitcoin-core github you'll find
many prior discussions of using generic compressors.
On Sat, Apr 24, 2021 at 6:12 PM NITSOPOULOS KONSTANTINOS via
bitcoin-dev <bitcoin-dev@lists.linuxfoundation.org> wrote:
>
> Hi, I can compress new blocks by up to 18% using the 7z file type. I use =
wxHexEditor to dump a block's raw hex onto my disk, and then compress that =
with 7-Zip. I'd like to see 7z built into Bitcoin Core, for block compressi=
on (storage & propagation). The entire blockchain could be under 290GB if i=
t's converted into a 7z archive-chain ("7z-chain"). I can compress some rec=
ent example blocks (hex -> less hex) if anyone wants. A future BIP could tr=
y increase the block Weight Units limit by up to 21% if 7z blocks are small=
er & quick.
>
> Programming compression seems too difficult for me, so I'm hoping somebod=
y else does it!
>
> Eventually I think transactions can be compressed by replacing addresses =
(& public keys) and coins, wherever possible, with just their own unique se=
quential index. TX amounts can also be compressed if they contain only a de=
cimal digit or two, e.g. 00000000.10000000 XBT. A TX of 0.1 XBT between two=
existing addresses could be somehow compressed to half or maybe a quarter =
of its current size!
>
> -Tinos
> _______________________________________________
> bitcoin-dev mailing list
> bitcoin-dev@lists.linuxfoundation.org
> https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev
|
