Return-Path: Received: from smtp3.osuosl.org (smtp3.osuosl.org [IPv6:2605:bc80:3010::136]) by lists.linuxfoundation.org (Postfix) with ESMTP id CC7E8C002D for ; Thu, 25 Aug 2022 08:59:29 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 9A85460DB7 for ; Thu, 25 Aug 2022 08:59:29 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org 9A85460DB7 Authentication-Results: smtp3.osuosl.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20210112 header.b=gnOGgKNB X-Virus-Scanned: amavisd-new at osuosl.org X-Spam-Flag: NO X-Spam-Score: -1.599 X-Spam-Level: X-Spam-Status: No, score=-1.599 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, PDS_BTC_ID=0.499, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001] autolearn=no autolearn_force=no Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id A3hb_yrR9zGc for ; Thu, 25 Aug 2022 08:59:28 +0000 (UTC) X-Greylist: whitelisted by SQLgrey-1.8.0 DKIM-Filter: OpenDKIM Filter v2.11.0 smtp3.osuosl.org A6D5B60B9F Received: from mail-lf1-x12b.google.com (mail-lf1-x12b.google.com [IPv6:2a00:1450:4864:20::12b]) by smtp3.osuosl.org (Postfix) with ESMTPS id A6D5B60B9F for ; Thu, 25 Aug 2022 08:59:27 +0000 (UTC) Received: by mail-lf1-x12b.google.com with SMTP id z6so27326824lfu.9 for ; Thu, 25 Aug 2022 01:59:27 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc; bh=1MFTozpTyBPInw/iE+Vvm2VoyRJ/BvmE/VXq0fbSBIs=; b=gnOGgKNBPyAZHc52VccfePcRLgA1fqg2Pf44NRYzqawUfkihhShdOpkGX1fPr325F5 m6zsq3TZGVb/GocbNzBOEnRGEbO87BYUlyIk71CECpF7X1yX5DQu+w8twB6ID9Ej0N5B 4SuFuZ5Mq9jkLCFQ1nlbn6JWjUm5A00UI1jmTRFRthaayRYisDV9qcZMhnr209qx+XwL VvZL5XOq8ZRKXK/HeEjd8YTLZhscrX9h/PaP8OZKbqnUxVkFjxccL13CukX2vyhwxG3n KXWxWRJkmPoIpox2DE/i0MgbisVYaCQUX8dwbIFrAvrLIILgo/qS+yQecsJlSkclf0l0 AJxQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc; bh=1MFTozpTyBPInw/iE+Vvm2VoyRJ/BvmE/VXq0fbSBIs=; b=rjH5V5e0d5+7RM8W91iMlZKp/b1CCis1ftJt5iN6OTFHBOYhsdzPmi2vY5ZzAcPKz6 ztMirjCWbz4Qy4VfB3ccZpJQuTBa78a6dnRB9/kuD9efz30HLiJv2Kvxum82TifSEzQo 8arH6yyyBwgLvSHHiXAJ71vGycQ0LXH2eotLa6fzifofmf/T6oY3YI+D/afuzFXG+vh5 /PybyfjUfFUlM2+Ch0HitczBE8SwGFr8J/camOTJ6BzLC35A4cVn2O7N1kkKNCspCBXh JX4jvSVwJhJL+JgmvE7fH4Uc1i3+xcYFU3v797Kr6mzSRCiqij6qhfYMtomN5S02AoAn T5JQ== X-Gm-Message-State: ACgBeo1Fpj0wZlX/rsvwvSwQ/pzOslsUK+B48B/DHHY/VeqY+zHlmpD4 Ze+nwl+ZZcdQHv8lmpznnNhfFOebeOIE0aKfpdOfBCMG1BQ= X-Google-Smtp-Source: AA6agR7/SOKnSc2Zfsc1JosZGlNai5NbUav98Gb/C/vrkvt+eVqZnoHDMbr7CRnNvW7v5JuKdqDp5Qirk/YTqgi2SMA= X-Received: by 2002:a05:6512:3a8e:b0:492:c760:b7ad with SMTP id q14-20020a0565123a8e00b00492c760b7admr933829lfu.473.1661417965392; Thu, 25 Aug 2022 01:59:25 -0700 (PDT) MIME-Version: 1.0 References: <1UV4d_Y74sQ_C8l5s6_gwZOOaFcB0hWnWYWl8TJ_PFs9bQ-fb_w_CYZjZOom2JJ0CSC6-w-Xi999ocafkWa7Mkz0MzsCs2Vg91M5to2fafA=@protonmail.com> In-Reply-To: <1UV4d_Y74sQ_C8l5s6_gwZOOaFcB0hWnWYWl8TJ_PFs9bQ-fb_w_CYZjZOom2JJ0CSC6-w-Xi999ocafkWa7Mkz0MzsCs2Vg91M5to2fafA=@protonmail.com> From: Craig Raw Date: Thu, 25 Aug 2022 10:59:13 +0200 Message-ID: To: rhavar@protonmail.com Content-Type: multipart/alternative; boundary="000000000000a6959405e70d0359" X-Mailman-Approved-At: Thu, 25 Aug 2022 09:45:49 +0000 Cc: Bitcoin Protocol Discussion Subject: Re: [bitcoin-dev] BIP Proposal: Wallet Labels Export Format X-BeenThere: bitcoin-dev@lists.linuxfoundation.org X-Mailman-Version: 2.1.15 Precedence: list List-Id: Bitcoin Protocol Discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Aug 2022 08:59:29 -0000 --000000000000a6959405e70d0359 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Thanks for your thoughts Ryan. Without reference to the quality feedback on this proposal, I was aware when submitting it for review that it provides an excellent opportunity for bike shedding. As developers, we have all experienced frustration with data formats. One thing that I did not perhaps make clear enough is that this format is not solely intended for developers, but general users who are probably not well represented on this list. While doing research for this proposal I spoke to several professional users of Sparrow Wallet (who are not developers). They all expressed a desire for the format to integrate with their business processes, which are driven by business tools such as Excel. Labelling provides an important function in UTXO and address management in these scenarios, and needs to be accessible and manageable outside of wallet software. If this is to be achieved, it immediately rules out JSON as a data format. Not only is JSON limited to editing only through specific software or text editors, but (in the latter case) it is fragile enough that a single missing character can cause an entire file to fail parsing. CSV is more forgiving in this regard. With respect to your comments on escaping, my expectation would be that developers will be using a mature CSV library rather than handling character escaping themselves. I would rather propose a format that is generally usable, even if occasionally a label is escaped incorrectly. Finally, I'll note that CSV files are already common and uncontroversial in Bitcoin wallet software. Bitcoin Core, Electrum, Sparrow (and no doubt many others) already export addresses and/or transactions with their labels as CSV files. This proposal simply attempts to create a standard for importing and exporting all the labels in a wallet. Craig On Wed, Aug 24, 2022 at 9:01 PM wrote: > I'd strongly suggest not using CSV. Especially for a standard. I've worke= d > with it as an interchange format many a times, and it's always been a > clusterfuck. > > Right off the bat, you have stuff like "The fields may be quoted, but thi= s > is unnecessary as the first comma in the line will always be the delimite= r" > which invariably leads to some implementations doing it, some > implementations not doing it, and others that are intolerant of the other > way. > > And you have also made the classic mistake of not strictly defining escap= e > rules. So everyone will pick their own (e.g. some will \, escape commas, > others will not cause it's quoted and escape quotes, and others will assu= me > no escaping is required since its the last column in a csv). > > Over time it morphs into its own mini-monster that introduces so much pai= n. > > On a similar note, allowing alternatives (like: txid>index vs txid:index) > provides no benefit, but creates additional work for implementations (who > quite likely only test formats they produce) and future incompatibilities= . > > I know everyone loves to hate on it, but really (line-separated?) json is > the way to go. > > { "tx": "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b= =E2=80=8E", > "label": "wow, such label" } > { "tx: "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b"= , > "txout": 4, "label": "omg this is so easy to parse" } > { "tx: "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b"= , > "txin": 0, "label": "wow this is going to be extensible as well" } > > > > > -Ryan > > ------- Original Message ------- > On Wednesday, August 24th, 2022 at 2:18 AM, Craig Raw via bitcoin-dev < > bitcoin-dev@lists.linuxfoundation.org> wrote: > > Hi all, > > I would like to propose a BIP that specifies a format for the export and > import of labels from a wallet. While transferring access to funds across > wallet applications has been made simple through standards such as BIP39, > wallet labels remain siloed and difficult to extract despite their value, > particularly in a privacy context. > > The proposed format is a simple two column CSV file, with the reference t= o > a transaction, address, input or output in the first column, and the labe= l > in the second column. CSV was chosen for its wide accessibility, especial= ly > to users without specific technical expertise. Similarly, the CSV file ma= y > be compressed using the ZIP format, and optionally encrypted using AES. > > The full text of the BIP can be found at > https://github.com/craigraw/bips/blob/master/bip-wallet-labels.mediawiki > and also copied below. > > Feedback is appreciated. > > Thanks, > Craig Raw > > --- > > 
> BIP: wallet-labels
> Layer: Applications
> Title: Wallet Labels Export Format
> Author: Craig Raw 
> Comments-Summary: No comments yet.
> Comments-URI:
> https://github.com/bitcoin/bips/wiki/Comments:BIP-wallet-labels
> Status: Draft
> Type: Informational
> Created: 2022-08-23
> License: BSD-2-Clause
>
> > =3D=3DAbstract=3D=3D > > This document specifies a format for the export of labels that may be > attached to the transactions, addresses, input and outputs in a wallet. > > =3D=3DCopyright=3D=3D > > This BIP is licensed under the BSD 2-clause license. > > =3D=3DMotivation=3D=3D > > The export and import of funds across different Bitcoin wallet > applications is well defined through standards such as BIP39, BIP32, BIP4= 4 > etc. > These standards are well supported and allow users to move easily between > different wallets. > There is, however, no defined standard to transfer any labels the user ma= y > have applied to the transactions, addresses, inputs or outputs in their > wallet. > The UTXO model that Bitcoin uses makes these labels particularly valuable > as they may indicate the source of funds, whether received externally or = as > a result of change from a prior transaction. > In both cases, care must be taken when spending to avoid undesirable leak= s > of private information. > Labels provide valuable guidance in this regard, and have even become > mandatory when spending in several Bitcoin wallets. > Allowing users to export their labels in a standardized way ensures that > they do not experience lock-in to a particular wallet application. > In addition, by using common formats, this BIP seeks to make manual or > bulk management of labels accessible to users without specific technical > expertise. > > =3D=3DSpecification=3D=3D > > In order to make the import and export of labels as widely accessible as > possible, this BIP uses the comma separated values (CSV) format, which is > widely supported by consumer, business, and scientific applications. > Although the technical specification of CSV in RFC4180 is not always > followed, the application of the format in this BIP is simple enough that > compatibility should not present a problem. > Moreover, the simplicity and forgiving nature of CSV (over for example > JSON) lends itself well to bulk label editing using spreadsheet and text > editing tools. > > A CSV export of labels from a wallet must be a UTF-8 encoded text file, > containing one record per line, with records containing two fields > delimited by a comma. > The fields may be quoted, but this is unnecessary, as the first comma in > the line will always be the delimiter. > The first line in the file is a header, and should be ignored on import. > Thereafter, each line represents a record that refers to a label applied > in the wallet. > The order in which these records appear is not defined. > > The first field in the record contains a reference to the transaction, > address, input or output in the wallet. > This is specified as one of the following: > * Transaction ID (txid) > * Address > * Input (rendered as txid) > * Output (rendered as txid>index or txid:index) > > The second field contains the label applied to the reference. > Exporting applications may omit records with no labels or labels of zero > length. > Files exported should use the .csv file extension. > > In order to reduce file size while retaining wide accessibility, the CSV > file may be compressed using the ZIP file format, using the .zip > file extension. > This .zip file may optionally be encrypted using either AES-128 > or AES-256 encryption, which is supported by numerous applications > including Winzip and 7-zip. > In order to ensure that weak encryption does not proliferate, importers > following this standard must refuse to import .zip files encrypt= ed > with the weaker Zip 2.0 standard. > The textual representation of the wallet's extended public key (as define= d > by BIP32, with an xpub header) should be used as the password. > > =3D=3DImporting=3D=3D > > When importing, a naive algorithm may simply match against any reference, > but it is possible to disambiguate between transactions, addresses, input= s > and outputs. > For example in the following pseudocode: > 
> if reference length < 64
> Set address label
> else if reference length =3D=3D 64
> Set transaction label
> else if reference contains '<'
> Set input label
> else
> Set output label
>
> > Importing applications may truncate labels if necessary. > > =3D=3DTest Vectors=3D=3D > > The following fragment represents a wallet label export: > 
> Reference,Label
>
> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b=E2=80=8E=
,Transaction
> 1A69TXnEM2ms9fMaY9UuiJ7415X7xZaUSg,Address
> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b=E2=80=8E=
<0,Input
> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b=E2=80=8E=
>0,Output
> c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b=E2=80=8E=
:0,Output
> (alternative)
>
> > =3D=3DReference Implementation=3D=3D > > TBD > > > > --000000000000a6959405e70d0359 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
Thanks for your thoughts Ryan.

Without = reference to the quality feedback on this proposal, I was aware when submit= ting it for review that it provides an excellent opportunity for bike shedd= ing. As developers, we have all experienced frustration with data formats. = One thing that I did not perhaps make clear enough is that this format is n= ot solely intended for developers, but general users who are probably not w= ell represented on this list.

While doing research= for this proposal I spoke to several professional users of Sparrow Wallet = (who are not developers). They all expressed a desire for the format to int= egrate with their business processes, which are driven by business tools su= ch as Excel. Labelling provides an important function in UTXO and address m= anagement in these scenarios, and needs to be accessible and manageable out= side of wallet software.

If this is to be achieved= , it immediately rules out JSON as a data format. Not only is JSON limited = to editing only through specific software or text editors, but (in the latt= er case) it is fragile enough that a single missing character can cause an = entire file to fail parsing. CSV is more forgiving in this regard. With res= pect to your comments on escaping, my expectation would be that developers = will be using a mature CSV library rather than handling character escaping = themselves. I would rather propose a format that is generally usable, even = if occasionally a label is escaped incorrectly.

Fi= nally, I'll note that CSV files are already common and uncontroversial = in Bitcoin wallet software. Bitcoin Core, Electrum, Sparrow (and no doubt m= any others) already export addresses and/or transactions with their labels = as CSV files. This proposal simply attempts to create a standard for import= ing and exporting all the labels in a wallet.

Crai= g

On Wed, Aug 24, 2022 at 9:01 PM <rhavar@protonmail.com> wrote:
I'd strongly suggest not using CSV. Especially for a standard. I'= ;ve worked with it as an interchange format many a times, and it's alwa= ys been a clusterfuck.=C2=A0

Right off the bat, you have stuf= f like "The fields may be quoted, but this is unnecessary as the= first comma in the line will always be the delimiter" which in= variably leads to some implementations doing it, some implementations=C2=A0= not doing it, and others that are intolerant of the other way.=C2=A0=

= And you have also made the classic mistake of not strictly defining escape = rules. So everyone will pick their own (e.g. some will \, escape commas, ot= hers will not cause it's quoted and escape quotes, and others will assu= me no escaping is required since its the last column in a csv).=C2=A0
Over time it morphs into its own mini-monster that introduces so much pai= n.

On a similar=C2=A0note, allowing alternatives (like:=C2=A0t= xid>index vs txid:index) provides no benefit, but creates additional wor= k for implementations (who quite likely only test formats they produce) and= future incompatibilities.=C2=A0

I know everyone loves to hate= on it, but really (line-separated?) json is the way to go.=C2=A0

{ &= quot;tx": "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fc= bb2ad088f767b37b=E2=80=8E", "label": "wow, such label&q= uot; }
{ "tx: "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f= 767b37b", "txout": 4, "label": "omg this is s= o easy to parse" }
{ "tx: "c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2ee= dd5fcbb2ad088f767b37b", "txin": 0, "label": "= wow this is going to be extensible as well" }
=




-Ryan
=20
=20

------- Original Message -------
On Wednesday, August 24th, 2022 at 2:18 AM, Craig Raw via bitcoin-d= ev <bitcoin-dev@lists.linuxfoundation.org> wrote:

Hi all,

I would like to pro= pose a BIP that specifies a format for the export and import of labels from= a wallet. While transferring access to funds across wallet applications ha= s been made simple through standards such as BIP39, wallet labels remain si= loed and difficult to extract despite their value, particularly in a privac= y context.

The proposed format is a simple two col= umn CSV file, with the reference to a transaction, address, input or output= in the first column, and the label in the second column. CSV was chosen fo= r its wide accessibility, especially to users without specific technical ex= pertise. Similarly, the CSV file may be compressed using the ZIP format, an= d optionally encrypted using AES.

The full text of= the BIP can be found at https://github.com/craigraw/bips/blob/master/bip-wallet-labe= ls.mediawiki and also copied below.

Feedback i= s appreciated.

Thanks,
Craig Raw

---

<pre>
BIP: wallet= -labels
Layer: Applications
Title: Wallet Labels Export Format Author: Craig Raw <craig@sparrowwallet.com= >
Comments-Summary: No comments yet.
Comments-URI: https://github.com/bitcoin/bip= s/wiki/Comments:BIP-wallet-labels
Status: Draft
Type: Informa= tional
Created: 2022-08-23
License: BSD-2-Clause
</pre><= br>
=3D=3DAbstract=3D=3D

This document specifies a format for the= export of labels that may be attached to the transactions, addresses, inpu= t and outputs in a wallet.

=3D=3DCopyright=3D=3D

This BIP is = licensed under the BSD 2-clause license.

=3D=3DMotivation=3D=3D
<= br>The export and import of funds across different Bitcoin wallet applicati= ons is well defined through standards such as BIP39, BIP32, BIP44 etc.
T= hese standards are well supported and allow users to move easily between di= fferent wallets.
There is, however, no defined standard to transfer any = labels the user may have applied to the transactions, addresses, inputs or = outputs in their wallet.
The UTXO model that Bitcoin uses makes these la= bels particularly valuable as they may indicate the source of funds, whethe= r received externally or as a result of change from a prior transaction.In both cases, care must be taken when spending to avoid undesirable leaks= of private information.
Labels provide valuable guidance in this regard= , and have even become mandatory when spending in several Bitcoin wallets.<= br>Allowing users to export their labels in a standardized way ensures that= they do not experience lock-in to a particular wallet application.
In a= ddition, by using common formats, this BIP seeks to make manual or bulk man= agement of labels accessible to users without specific technical expertise.=

=3D=3DSpecification=3D=3D

In order to make the import and ex= port of labels as widely accessible as possible, this BIP uses the comma se= parated values (CSV) format, which is widely supported by consumer, busines= s, and scientific applications.
Although the technical specification of = CSV in RFC4180 is not always followed, the application of the format in thi= s BIP is simple enough that compatibility should not present a problem.
= Moreover, the simplicity and forgiving nature of CSV (over for example JSON= ) lends itself well to bulk label editing using spreadsheet and text editin= g tools.

A CSV export of labels from a wallet must be a UTF-8 encod= ed text file, containing one record per line, with records containing two f= ields delimited by a comma.
The fields may be quoted, but this is unnece= ssary, as the first comma in the line will always be the delimiter.
The = first line in the file is a header, and should be ignored on import.
The= reafter, each line represents a record that refers to a label applied in th= e wallet.
The order in which these records appear is not defined.
The first field in the record contains a reference to the transaction, add= ress, input or output in the wallet.
This is specified as one of the fol= lowing:
* Transaction ID (<tt>txid</tt>)
* Address
* I= nput (rendered as <tt>txid<index</tt>)
* Output (rendered= as <tt>txid>index</tt> or <tt>txid:index</tt>)<= br>
The second field contains the label applied to the reference.
Ex= porting applications may omit records with no labels or labels of zero leng= th.
Files exported should use the <tt>.csv</tt> file extensi= on.

In order to reduce file size while retaining wide accessibility,= the CSV file may be compressed using the ZIP file format, using the <tt= >.zip</tt> file extension.
This <tt>.zip</tt> file = may optionally be encrypted using either AES-128 or AES-256 encryption, whi= ch is supported by numerous applications including Winzip and 7-zip.
In= order to ensure that weak encryption does not proliferate, importers follo= wing this standard must refuse to import <tt>.zip</tt> files en= crypted with the weaker Zip 2.0 standard.
The textual representation of = the wallet's extended public key (as defined by BIP32, with an <tt&g= t;xpub</tt> header) should be used as the password.

=3D=3DImpo= rting=3D=3D

When importing, a naive algorithm may simply match again= st any reference, but it is possible to disambiguate between transactions, = addresses, inputs and outputs.
For example in the following pseudocode:=
<pre>
if reference length < 64
Set address label else if reference length =3D=3D 64
Set transaction label
el= se if reference contains '<'
Set input label
else Set output label
</pre>

Importing applications may tru= ncate labels if necessary.

=3D=3DTest Vectors=3D=3D

The follo= wing fragment represents a wallet label export:
<pre>
Reference= ,Label
c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b= =E2=80=8E,Transaction
1A69TXnEM2ms9fMaY9UuiJ7415X7xZaUSg,Address
c3bd= ad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b=E2=80=8E<0,= Input
c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb2ad088f767b37b= =E2=80=8E>0,Output
c3bdad6e7dcd7997e16a5b7b7cf4d8f6079820ff2eedd5fcbb= 2ad088f767b37b=E2=80=8E:0,Output (alternative)
</pre>

=3D= =3DReference Implementation=3D=3D

TBD

<= br>

--000000000000a6959405e70d0359--