Name: Andreas Antonopoulos
Topic: Advanced Bitcoin Scripting
Location: SF Bitcoin Devs
Date: April 3rd 2017
Video: https://www.youtube.com/watch?v=yU3Sr07Qnxg
Mastering Bitcoin on GitHub: https://github.com/bitcoinbook/bitcoinbook
Intro
Today’s talk is different from some of the other talks I’ve done so bear with me. We are going to do this slowly in a very casual manner. This is not one of my usual talks where I am going to tell you about big picture vision and stuff like that. What I want to do is talk about some interesting features of Bitcoin Script and explain them. First question is how many of the people here are developers? Or have at least coded once or twice? How many people are completely unfamiliar with Bitcoin Script? I’m going to do a very quick introduction to some of the fundamentals of Bitcoin Script and then we are going to dive into some more advanced topics around conditionals and flow control and guard clauses, timelocks all the way up to if we get to it some of the Scripts you see in Lightning. I’m going to lose some of you on the way there but that is ok because hopefully you can or read the chapters and catch up. How many people did the homework? Part of what I am going to be talking about is some of the new content from the second edition of Mastering Bitcoin. As you probably know that book as with the first edition was written on GitHub in public. All the laundry out in public for everyone to see and criticize and notice all of the horrific mistakes I made and hopefully do pull requests before it goes into final production which is next week. Hint, hint. I will also be saying things tonight that are wrong. Extra points if you notice and you send me a message afterwards and help me fix them in the book. If I do say things that are wrong it is because I don’t know they are wrong which is why I need your help. If you do know they are wrong that is really what I could use. No promises that any of this is going to work or that it is correct but it will give you greater insights into how Bitcoin Script works.
Bitcoin Script
Let’s get started, I am going to use the whiteboard mostly. Bitcoin Script is a Forth-like stack based language that uses a reverse Polish notation. How many people did I lose? What that means is that it uses as its fundamental elements a stack and the stack evaluates functions or Script operands in the opposite order than you would expect and the parameters of functions or operators come before the operators themselves. An example of reverse Polish notation Script is… let’s say you want to do 1+1. Start with the easy stuff. It is 1 1 +. The reason it is like that is because this gets evaluated through a stack. A stack is a data structure where you push things into the stack or you pop things from the stack. Think of a stack of dishes. You can only access the top most dish. In order to get to the one below it you need the pop off the top one and then you get to the second one. If you want to evaluate this, I’ll use this notation which is the one that I used in my book. Let’s say this is the stack. The system will start by evaluating the first item in the Script. If it is just a number it gets pushed onto the stack. The 1 goes here, Script pointer moves to the next item, goes here, got another 1, push that onto the stack. Now we’ve got one item on the stack which is the plus operator. When you get to an operator, usually an operator takes a few parameters. The plus operator takes two parameters, the two things it is adding together. Where does it find those two parameters? It pops two items off the stack. A plus operator is defined as an operator that pops two items off the stack, adds them together and pushes the result back onto the stack. Pop, pop, add, push. Pop, pop, add them together and push the result onto the stack. And the program terminates. That is how you do 1+1=2. It is a bit weird right? In Bitcoin the scripting language works like this. We often use the prefix OP to denote operators. In Bitcoin Script this would look like OP_1 OP_1 OP_ADD. OP_1 just pushes 1 to the stack, OP_1 pushes 1 to the stack and OP_ADD pops two items, adds them together and pushes the result onto the stack. The end result is 2, same thing. For the rest of these examples I will not use the OP prefix. It is implied. The only reason the OP prefix exists is because in the actual code that implements Bitcoin, for example Bitcoin Core or bcoin or libbitcoin, the variables used to denote these operators are written with the prefix OP. Many programming languages don’t allow you to start a variable with a number. They require the first part of the variable name to be a letter. Hence prefix OP underscore whatever. It is redundant, everything has OP underscore in front of it so we just drop it from everything and we continue this conversation. So far so good. Who is with me? Very good.
P2PKH
Who is familiar with a pay-to-pubkey-hash Script? I’m going to run through an execution of that very quickly.
Q - When you say familiar what do you mean?
A - Has seen it run or understands how it is executed. Has heard of it. Is vaguely aware that some such thing exists.
When in Bitcoin you create a transaction that says “Alice pays Bob 1 Bitcoin for a cup of coffee” in the Bitcoin transaction what you get instead is a bunch of Alice’s inputs with some signatures to an output that identifies Bob’s address as the destination paid. Maybe some change as well but we’ll ignore that for now. How does it identify Bob’s address? With a Script format called pay-to-publickey-hash, P2PKH. The way that looks on the stack is:
DUP HASH160 <Bob PKH> EQUAL CHECKSIG
DUP which stands for duplicate. HASH160, Bob’s public key hash, EQUAL, CHECKSIG. It is not strictly to do all of this. There are reasons why it is done in this particular way. Some of the first transactions if you look at the coinbase for example of the first several hundred blocks, the coinbase is not paid like this. It is paid simply with a public key CHECKSIG. The problem with a public key CHECKSIG format if you just use the last part is that you have revealed Bob’s public key in the output before Bob has had a chance to redeem it. That’s dangerous. If there is ever a problem with the elliptic curve and you can take a public key and reverse it back to a private key say with quantum computing maybe you don’t want all the public keys to be known. This format preserves it behind a double hash which is a hell of a lot harder to break. You’ve got two layers of obfuscation there. There are other reasons why you would use a hash as well. We’ll see that in a bit. This is the Script. Every time you do a Bitcoin transaction, if we all pull out our Bitcoin wallets and our smartphones right now we start sending stuff to each other’s QR codes and sending transactions. You are going to be creating a bunch of Bitcoin transactions on the Bitcoin blockchain which will all have this Script. This is called P2PKH. They will all have this Script in the output.
Redeeming a P2PKH
How do you redeem this Script? You provide two parameters to this. Let’s say you are Bob and you want to spend this. You put BobSig and BobPubkey. When you put these two things they actually get executed in sequence before this is verified. Let’s say I’m Bob, I want to spend this, this is the signature I am going to provide. This goes in the scriptSig. I want to spend this, here is my signature. It will be two parts, signature and your public key. You put that in the transaction you want to spend. Then the system evaluates. Is Bob really authorized to spend this particular output? How do we know? Let’s run this through the scripting language first and then let’s run that through the scripting language. If the final outcome is true, done you get the Bitcoin. If it is false or anything else, invalid transaction. Everybody with me so far? What happens when you evaluate these two things through the scripting language? These things are numbers, they get pushed onto the stack. BobSig goes to the bottom and BobPubkey goes above it. This is the state of the stack when the locking Script is evaluated. Our locking Script has been pushed onto the stack, the locking Script is in the Script execution engine. Now we start running to see if this is a valid signer for this output. If Bob is in fact allowed to spend this. Who wants to try running this? Anyone want to try a P2PKH? The first Script execution item we have is DUP or OP_DUP, I’m taking out the OP. What does that do? It takes how many parameters off the stack? One, it pops one parameter off the stack? What does it do? It duplicates it and then it pushes two parameters back onto the stack. We run DUP. Bob’s pubkey gets popped off the stack and then gets pushed back onto the stack twice. I’d erase the bottom one and write both of them again but I’m lazy and you know what I’m talking about. Now you’ve got two copies of Bob’s public key on the stack. We move to the next item in the Script execution. Who wants to run the next one? The execution pointer is at HASH160. It takes one input and produces one output. The first input it takes is BobPubkey. What does it do to it? It pops it and does the HASH160 operation. Who wants to tell me what the HASH160 operation is? It is two hashes in one. It is a SHA256 wrapped in a RIPEMD160. It is a double hash. It is the same hashing operation that goes into constructing a Bitcoin address, also known as a public key hash. HASH160 took Bob’s public key, hashes it through SHA256, hashes it through RIPEMD160 and produces a 160 bit public key hash that gets pushed back onto the stack. Let’s call that BobPKH. At this point if all has gone well, if Bob’s public key was in fact the one that Alice was paying, these two should be the same. Let’s move to the next point in the stack. This is gone. This (
Q - Why is it called HASH160?
A - Because 160 is the RIPEMD160 which produces the 160 bits which is the final hash you apply. The output of HASH160 is the output of RIPEMD but it also internally does a SHA256.
I’ve demonstrated how the Script is executed on the stack by pushing things. You notice they go in reverse order. The first thing that gets pushed is the last thing that comes off. When you have a function in a Script it acts on the things that are closest to it because those gets pushed onto the stack last. Later things in the Script act on parts of the Script that are furthest from it because those are the first things that get pushed on the stack. First you move the parameters in, do something with them, create new parameters, do something with them, create new parameters, do something with them. Then there are maybe some leftovers at the bottom that get picked up at the end. That is a common thing. When you are looking at a Script it is not always easy when you look at it to figure out which part is acting on which other parts of data because you have to visualize the sequence of pushing things and popping things off the stack. It depends on how many parameters are being pushed and popped by each of the intermediate commands.
The VERIFY suffix
I’m not going to draw the stack again. What I am going to do instead is draw some Scripts. We talked briefly about the suffix VERIFY. Does anyone remember which of the opcodes have a suffix that can be VERIFY? EQUAL. EQUAL comes in two forms, EQUAL and EQUALVERIFY. CHECKSIG. You have CHECKSIG and CHECKSIGVERIFY. CHECKLOCKTIME? That only comes in one form, we’ll get to that in a bit. CHECKMULTISIG. CHECKMULTISIG and CHECKMULTISIGVERIFY. Has anyone figured out the difference between the two? Where would you use one and where would you use the other? Anything that has VERIFY will only continue execution of the Script if the outcome of the conditional operator is TRUE in which case it will not push that TRUE back to the stack, it will simply continue execution. Otherwise EQUAL, CHECKSIG and CHECKMULTISIG put the outcome, TRUE or FALSE on the stack and whether it is true or false you then have to further down decide what to do with that. Maybe afterwards you could do something to verify that it is true. That is not really useful because you would do it right there. You could use EQUAL and it could be cumbersome. Not much point of using these (EQUALVERIFY, CHECKSIGVERIFY, CHECKMULTISIGVERIFY) at the end of a Script because the end of the Script acts like a VERIFY. Meaning that when you reach the end of the Script if TRUE is on the stack you are good. The lock Script has been unlocked. If anything other than TRUE is on the stack your Script terminates with a FALSE, done. That locking Script has not been satisfied. So effectively the end of the Script acts as a VERIFY. Meaning that you can comfortably put EQUAL, CHECKSIG or CHECKMULTISIG at the end of a Script because the VERIFY is implied right afterwards. The reason you would use the EQUALVERIFY, CHECKSIGVERIFY or CHECKMULTISIGVERIFY is if you’re putting it earlier in the Script, you don’t want to leave anything on the stack and you want to terminate. This is a kill clause. You are not going to worry about whether this is true or false. If it is false you are done, you are not going to evaluate anything else. Does anybody remember in programming what we call clauses that decide whether the following part of the language is going to be executed or not? It is a very specific type of IF clause. It is called a guard clause. A guard clause is if the previous thing was true then do this. The this part only gets executed if you pass by the guard. It is a gatekeeper. It says “This code only gets executed under a specific condition.” Guard clauses are used extensively in programming for one simple reason. They are very easy to read and very clean. We use them in formal languages especially languages where bugs are very dangerous. The reason we use guard clauses is because if you wrap code in a simple guard clause that code will not run if the guard clause is not satisfied. You see that all the time. IF debug, print to log file. All you are doing is turning on or off that functionality when there is a debug. Otherwise the print to log file never happens. IF debug mode assert 0. That’s the guard clause that should have been in that line of Bitcoin Unlimited but wasn’t. Guard clauses are useful because they simply preclude the execution of the next thing. We will see how we use them in the next section. There is a specific type of command that only has a VERIFY version. That’s timelocks. There are two forms of Script based timelocks, CHECKLOCKTIMEVERIFY and CHECKSEQUENCEVERIFY. There is no CHECKLOCKTIME without VERIFY. There is no CHECKSEQUENCE without VERIFY. There is a very good reason for that. It actually makes it rather difficult to write Scripts that are valid now but not valid in the future. You can only write Scripts that are invalid until an event but from that point to infinity. This is deliberate. Timelocks allow you to activate a transaction but not expire it. Once something becomes valid it is valid forever. This is consistent throughout Bitcoin. There is no such thing as a Script that is valid today but is not valid tomorrow unless the consensus rules change. Transactions do not expire and Scripts do not expire. As a result CHECKLOCKTIME is written in such a way that you can’t say NOTIF and then run something afterwards to invert its meaning. CHECKLOCKTIMEVERIFY for example means this is valid only after block 1000. You can’t invert its meaning and say this is valid only before block 1000 and not after because the VERIFY doesn’t let you evaluate the outcome. It simply stops. Does that make sense? You can get around that but it is not easy. You have to use something called a revocation. Who is with me so far? Who is still having fun?
IF then ELSE
The VERIFY clause at the end of an operator is a form of a conditional. It is also a form of flow control. It is the same as writing IF TRUE do what comes after ELSE RETURN. The RETURN code in Bitcoin causes a Script to terminate and does not evaluate TRUE, it evaluates FALSE. Done. If you find a RETURN in your Script and run it. VERIFY is the same as saying “If TRUE do this ELSE RETURN”. It is a much more compact form and we will see how that works. The most useful form of conditional is a flow control statement. A flow control statement is an IF then ELSE statement. Everyone familiar with IF then ELSE? In Silicon Valley we call IF then ELSE “AI”. For many programmers AI is simply a hundred thousand IF then ELSEs concatenated together that evaluate all the possible ways something might work. I wrote an interpretative interactive psychiatrist called Eliza in college as a very weak AI that would interact and have a conversation with you. It consisted of a lot of IF then ELSE. Trying to figure out what the hell you were saying in English. IF then ELSE statements. This is Bitcoin of course. So take what you know about IF then ELSE sprinkle some weird on it and we get Bitcoin. IF A ELSE B ENDIF. This is what it looks like in Bitcoin. In the Bitcoin Script you have IF ELSE ENDIF. What is A? A condition? You would assume so. This is where the weird comes in. Second best guess? It is the code that gets executed if the condition is TRUE. This is the then part. You can assume that here is a “then” to make it easier to read. IF TRUE then A ELSE B. That is how it actually runs. You start reading Bitcoin Script, you see an IF, you assume the very next thing is the thing that you are testing. It is not. The very next thing is the thing you are executing if the thing you tested was TRUE. Then there is an ELSE statement. Or not, there might just be an ENDIF. Then there is the thing that runs if it was false. Then there is an ENDIF. Where is the condition? Before the IF. Actually the way this works is it looks like this. X IF A ELSE B ENDIF. The problem is the X is not in the Script. The X is not in the locking Script. It almost never is. It is not as a literal for sure. There might be other stuff before there. In which case whatever they put on the stack, maybe they put a TRUE or FALSE. If you had an EQUAL here, EQUAL gets evaluated, it dumps TRUE or FALSE on the stack. IF gets evaluated, it takes the TRUE or FALSE from the EQUAL. IF EQUAL then A, ELSE B. If this X was an EQUAL operator it took whatever two things were on the stack before, weighed them together. If they were equal this part runs, if they were different this part runs. You can actually take an EQUALVERIFY and write it as EQUAL IF what comes after ELSE RETURN ENDIF. It is a rather longwinded way of writing the same thing. Very often in Scripts there is no X and this gets very confusing. You read the Script and you think “It is just English right? Programming is just English. If A is true then I’m not quite sure but else B.” But this is not what this Script does. The first time you come across conditional flows in Bitcoin Script your natural inclination is to read it. You read for example “IF something CHECKLOCKTIMEVERIFY something CHECKSIG” and you think it is checking the CHECKLOCKTIME. It is not. The condition came before. But it is not on the Script so where the hell is it? And why is it not on the Script? You are doing Reverse Polish notation so first it has to get pushed which means it comes before the function that actually uses it. But why not put it before the IF in the locking Script. There’s a very good reason for that. Because if the condition is in the locking Script then you put it in place when the UTXO is created and it only ever has one value, the one you put there. Therefore the rest of the Script is pointless because if this was something that evaluates to TRUE and you put it in the locking Script then it will always run A. If it was false and you put it in the locking Script it will always run B. So why the hell do you have the IF there? Why don’t you just get rid of all that and just say A or B? The reason that you can’t put the condition in the Script is because the Script gets recorded on the blockchain and is immutable. It is the part that gets executed every time in exactly the same way. Whatever you put there will always evaluate to either TRUE or FALSE. It will always run A or B. If you are putting in the locking Script you are just wasting space. It is variable. So you need to put it in the part of the scripting language that is variable and that’s the unlocking Script. Which means that the person spending provides the condition. That’s weird. This comes to the basic understanding of what conditional flow does in Bitcoin. What that says is this is locked. This simple locking Script, if it was exactly like this, it says “This can be spent two ways and you choose. You show up with an unlocking Script that leaves TRUE on the stack we are going to try to spend this with method A. You show up with an unlocking Script that has FALSE on the stack we are going to try to spend this with Script B.” The unlocking Script chooses one of the two paths to execute. If statements in Bitcoin, conditional flow in Bitcoin, is very different from how we would use in procedural language because the condition you’re testing is user input. It comes from the unlocking Script. Essentially in Bitcoin Script you are saying there are two ways to spend this UTXO, A or B. You can choose equally between both of them by saying TRUE or FALSE in the spending Script. That seems weird because that doesn’t seem very secure. What do you mean the unlocking Script can choose? They can just choose A or B? Then we’ve got to see what happens here. Usually that means something sophisticated is happening here. Let’s take a look.
IF BobPubK CHECKSIG ELSE AlicePubK CHECKSIG ENDIF
Here’s a locking Script. You try reading it with English first “If Bob’s public key is CHECKSIG, no that’s wrong.” What this says is “If you give me a TRUE I’m going to try to use Bob’s public key to check this. If you give me a FALSE I’m going to try to use Alice’s public key to unlock this Script.” Which means that to spend this Bob passes as the unlocking Script BobSig TRUE. Alice can spend the same thing by saying AliceSig FALSE. That’s really weird. You probably don’t understand that at first glance. Let me explain how this works. Let’s say Bob presented this. How does this execute on the Script? Let’s put it in our mind and think about it. BobSig is the first thing, it is a number, it gets puts onto the stack. Then TRUE gets put on top of it in the stack. Then the IF clause is executed, it pops off the TRUE, drops everything else and runs this. Now PubK goes on top of BobSig, CHECKSIG runs, Sig, PubK, CHECKSIG, TRUE. Done. Bob spends it. Similarly here AliceSig goes on the stack, FALSE goes above it. IF executes, it is FALSE, all of this goes away, the ELSE clause runs, Alice’s PubK gets pushed on the stack, CHECKSIG pops AlicePubK and AliceSig. TRUE, done, Alice spends this output. I just wrote a really freaky weird 1-of-2 multisig between Bob and Alice. It is two keys, any one of them can spend it with a single signature. What I did was multisig but with an IF clause. All they have to do in order to choose the right clause for their particular execution path, the one that they want, the one that will achieve the result they want is put a TRUE or FALSE at the end. Everybody with me? Now you see why the condition is provided by the redeemer. Now that may be a problem. You may want to make sure that some parts of these things can only be executed by a very specific spender.
A More Complicated Script
Let’s say we wanted to do more here. Let’s say Bob had to produce a secret hash preimage and a signature. Then we could do:
IF BobPubK CHECKSIGVERIFY HASH160 <BobHash> EQUAL ELSE AlicePubK CHECKSIG ENDIF
Now Bob has to deliver three things to the stack, TRUE, a signature and some number that when hashed produces this hash, a preimage. Not quite. We need to turn this into a guard clause. How do we turn it into a guard clause? VERIFY. (It needs to be CHECKSIGVERIFY not CHECKSIG in first instance). What we are saying now is there are two execution paths available in this Script. The first execution path must first overcome this guard clause that requires a named public key to provide a signature and then they must also provide the preimage to a hash. Alice has a lesser burden. She only needs to produce a signature. We’ve piled two conditions. This is almost like a nested IF because it is a VERIFY. VERIFY is the same as doing IF ELSE RETURN. Everybody with me so far? This is getting complicated right?
Q - The reason for doing this would be you suspect someone has Bob’s private key?
A - No the reason for doing this is to allow a mode of redemption that uses a preimage instead of a public key. Lightning uses that kind of technique for hash timelocked contracts. You would actually probably put this the other way round so that the guard clause is “Give me the preimage first and then give me the signature.” They are equivalent. You could also have this without this (BobPubK CHECKSIGVERIFY). This means Alice can spend this or anyone with knowledge of the secret can spend this. Anyone at all. Once the secret is out the first person to get their transaction into the blockchain gets to redeem this.
An Even More Complicated Script
I think I have time for one more example which is the final example from Chapter 7. I want to get this right so I’m going to write it down.
IF IF 2 ELSE <+30 days> CHECKSEQUENCEVERIFY DROP <A PubKey> CHECKSIGVERIFY 1 ENDIF <B PubKey> <C PubKey> <D PubKey> 3 CHECKMULTISIG ELSE <+90 days> <CHECKSEQUENCEVERIFY> DROP <A PubKey> CHECKSIG ENDIF
At this point this should be crystal clear. It is the same thing with a couple of little twists. The same principles of how you push things on the stack and pop things off the stack and then evaluate them in sequence. What this is is a partnership company that has three partners B, C and D. These three partners operate a multisignature account to store the company’s money. Three lawyers working together. They have a lawyer who is not part of the company called A and that lawyer is responsible for assisting in the case where there is key loss. If one of the partners loses the keys, the lawyer who is under contract and has a fiduciary responsibility can step in and provide a backup key. Basically what this Script does is it says “Two out of three of the partners can spend this anytime.” The basic clause is down here which is a 2-of-3 multisig. It requires two of the three partners to sign, if they sign it works like a normal multisig. A small twist but mostly normal. Unless the output has not been spent within 30 days of being mined. If it hasn’t been spent 30 days after it is mined it can also be redeemed by a signature from the lawyer and one of the partners. Not two. Or if 90 days has elapsed from the moment this output has been mined the lawyer can spend it without any other partners’ signatures. There are three execution paths. At any moment in time all three of the execution paths can be selected. But not all of them can be run successfully because they have guard clauses. Those guard clauses are based on relative timelocks. CHECKSEQUENCEVERIFY is relative to when this was mined. That’s why I put the notation +30 days, +90 days. That would be in seconds relative to the time it was mined.
Q - Are the timelocks according to blocks?
A - They are according to median time past which is a specific consensus measurement of time introduced by BIP 68 which introduced CHECKSEQUENCEVERIFY. CHECKSEQUENCEVERIFY came also with BIP 113 when it was activated in November I think.
Q - What reference it is using?
A - It measures the median time of the last eleven blocks. It is permanently about an hour in the past but becomes it depends on the timestamps of eleven blocks no single miner can monkey with it. It makes it a useful consensus rule of time. You have to mentally adjust to the fact it will always be a hour behind because it is looking at the median time past. Eleven blocks is just short of two hours and the midpoint is exactly an hour ago. Not wall clock time, consensus time. Because each miner can make small changes and can drift any one block by a bit but they can’t drift all eleven past blocks by a lot therefore it is not in their hands to modify this and start sniping transactions that are not yet due under the timelock. That’s a consensus security feature that was introduced with BIP 113.
Q - Lawyers B, C and D would have to update this transaction approximately every day?
A - Bingo. In order for this to be effective you have to roll it over every 30 days, every 29 days to be precise to prevent one of the other partners from colluding with the external lawyer and taking the money. They would have to sign with 2 out of 3 a transaction every 30 days and roll it back into a Script like this. With the current fee structure in Bitcoin it is not likely they would do that. Maybe this isn’t suitable for now. It depends, maybe they are a very rich law firm. They are moving 10 million dollars out of this UTXO into another UTXO so a 5 dollar fee for a big Script isn’t a big deal.
Q - What does the DROP Script do?
Now let’s go into some of the nuances. You get the big picture, everybody gets the big picture. What this is trying to achieve. Multisig in the middle, very straightforward, two of three partners, a 30 day clause that allows one partner and the lawyer and a 90 day clause which is the they all died in a car crash together, I told them not to go in the same car. Now he is the last will executor for their last will and testament. He’s like “Dearly beloved we have gathered here to split ten million dollars.” That’s the final clause, that’s the backup clause. What we’ve done here is we’ve sequenced some events in time, this is really important, using timelocks. I want to point out some nuances. You remember how I said that timelocks can be activated but not expired because they only work one way. Once something is valid, the timelock has passed, it comes into play. But it doesn’t go out of play. Let me just point something out. You might at first glance think it is either, or, either, or. In fact it is this and this after 30 days and also this after 90 days. Meaning that 36 days after this is mined these guys can still do the 2-of-3. This clause doesn’t go away. This is always an option. Maybe it is A that got into a car crash and his key disappeared so none of the other stuff matters. This is always an option. 92 days after this is mined, all three clauses come into play. It can be redeemed in any of the three ways. The clauses come into play sequenced through time but they never expire. You can’t say “You can’t redeem anymore.” That is a problem or a solution. The way it is done in Lightning is using a revocation Script which you can find in Chapter 12. Here is an interesting twist. You may notice this. What the hell is that doing there? There is a number 2 up here. What is that doing up there? Let me give you some hints. CHECKMULTISIG takes at least three parameters: m which is the quorum, a number of keys minimum of 1 and n which is the how many keys in total. So CHECKMULTISIG, that 3 is the n. It says there are 3 keys total. Where is the m? There is the m, the 2, or one version of it. Because there is another m, the 1. This may start giving you a tiny bit of a glimpse of how flexible this is. I just took the CHECKMULTISIG and I spread it over two different conditional clauses, over two different flows, over two execution paths. One way of doing this is you go through this way and then it is 2, key, key, key 3, CHECKMULTISIG. Pretty much straightforward MULTISIG. Or you go through this path, guard clause 30 days and the signature from the attorney and then one of key, key, key, 3, CHECKMULTISIG. Then it is a 1-of-3. This one plays two different ways. It is a 2-of-3 or it is a 1-of-3. That is a really powerful feature. We just leave it dangling on the stack and then CHECKMULTISIG will pick it up when it is time.
Q - Technically you could say it is a 2-of-3 or a 2-of-4.
A - Very good point. It is a 2-of-3 or a 2-of-4 because we have inserted another key up here but it is difficult to write a 2-of-4 in this particular formulation. You could probably do it by splitting the CHECKMULTISIG at a different part. Very good point, it is a 2-of-4 because we introduced this.
Q - There are two different checks. One is the CHECKSIGVERIFY and it won’t even go to the CHECKMULTISIG. Either it is 1-of-3 or it is 2-of-3.
A - Actually it is not a 2-of-4. It is 1-of-1 and a 1-of-3. 2-of-4 would imply that two of these guys (B, C, D) could do it. But it has to be this plus one of the other three. Subtle nuances there. Also you start to realize that this needs some debugging. You really have to very carefully think about all the scenarios. How many keys come into play, when they come into play, what combinations exist, what these people might do to collude with each other. This isn’t even a Turing complete language. It is only 12 lines and you could fit an entire DAO mess right about here. One little bug, oops. What do you mean A ran away with all the money? Well you see if this happens before this we’re f****ed. Even with Bitcoin Script, even thought it is Turing incomplete, even though it is a very simple stack based language you can introduce quite a bit of complexity there. If you see some of the Lightning Network Scripts and think about all of the game theoretical aspects of who can operate on which timeframes and what can they do, it gets rather complicated to think about all the scenarios. There are some subtleties there. Who wants to try to tell me what you have to put on the stack as the unlocking Script in order to get to a 2-of-3. We have two IFs nested. Two signatures followed by two TRUEs. Why? First TRUE, IF we go here. Second TRUE, IF we go here. Then this (2) gets put on the stack. Everything from the ELSE to the ENDIF goes away. Then this (<B PubKey> <C PubKey> <D PubKey> 3 CHECKMULTISIG) gets put on the stack, then it gets evaluated with the two signatures. Done. We got here and here with TRUE TRUE. How do we get here (ELSE)? TRUE FALSE? No. Reverse Polish notation. FALSE TRUE. You have to put the thing that is going to be executed first last in your line. FALSE gets pushed on the stack, TRUE gets pushed on the stack, TRUE executes, FALSE executes, you go in here. And of the course the two signatures that are required to do that. Signature, signature, FALSE, TRUE. Final question of the day what the hell does that draw? OP_CHECKMULTISIG has a bug, it pops an extra parameter before the first signature. You also have to put an OP_0. In fact BIP 147 restricts what that one thing can be popped is down to a null dummy value. It is called the NULLDUMMY BIP. It requires that the value that gets popped by CHECKMULTISIG is a zero. The reason is to remove malleability. At the moment it pops whatever is there so if you create a Script that has a zero and I change it to a Script that has a one even though I don’t have any of the keys it still works because it will accept any value there. I think it is BIP (147) narrows that down to only zero accepted to overcome that bug. Esoteric but very useful. There is a chapter in the book on why bugs in Bitcoin become consensus rules forever. The final thing is the DROP. Anybody care to comment on why that bug is there? Every other form of VERIFY leaves nothing behind on the stack. CHECKSEQUENCEVERIFY and CHECKLOCKTIMEVERIFY leave this value (+30 days) on the stack. Why? Because you might want to do more with it. For example, let’s say after this we wanted to do plus another 5 days CHECKSEQUENCEVERIFY. So we could do 5 days OP_ADD OP_CHECKSEQUENCEVERIFY. It takes the value that was left on the stack, adds 5 days or the equivalent in seconds, runs OP_ADD, pops the two values, mushes them together, puts the sum and that’s a CHECKSEQUENCEVERIFY. You can do arithmetic with this later. One of the exceptions to the VERIFY rule. That’s all good, we’ve got a value that is going to stay on the stack but what if the very next thing is going to leave us some junk on the stack? We don’t want that junk passing on to the next thing so we DROP it. You will very often see after CHECKLOCKTIMEVERIFY and CHECKSEQUENCEVERIFY you will see a DROP occur immediately after. That is just to clean up the junk that is left over. You never get to the DROP unless the time is right.
Q - CHECKSEQUENCEVERIFY is a special case of this VERIFY?
A - CHECKSEQUENCEVERIFY is a command that was added by BIP 68 which is a relative locktime. VERIFY only exists as specific opcodes. You can’t just slap the word VERIFY onto something. It is a specific opcode whose name also includes VERIFY. It is not a compound operator, it is a single operator CHECKSEQUENCEVERIFY and it doesn’t exist in the other form.
Q - What happens after 29 days? You said you have to reboot the transaction?
A - If the partners don’t want to leave open the door to a pubkey being able to collude with one of the partners they had better spend this within the 30 day window and create a new UTXO whose time starts again. It is a dead man switch. You have to reset the clock before it runs out otherwise the dead man switch clause gets activated. Everybody understand what a dead man switch is?
Q - How can you change the meaning of an opcode with a BIP without breaking consensus?
A - You can do it but only in one direction. You can only tighten the meaning of an opcode. You can take something that was valid and make it valid in fewer cases but you can’t make it valid in more. If you make it valid in fewer cases new nodes will evaluate that and be more strict in their evaluations. Old nodes will accept it because it is still valid by the old rules. That is called a soft fork. CHECKSEQUENCEVERIFY was actually replacing OP_NOP3. What does NOP do? Nothing that’s why it is called NOP. It is a no op. If you take something that was a NOP and give it special meaning every other old system is going to read this and it is going to go “Number, NOP, DROP”. So far so good and we keep going. It won’t enforce the meaning of locktimes but it will still evaluate the Script as valid. That’s a soft fork. Old nodes don’t see the additional meaning. You can tighten consensus rules by redefining something more specifically. You cannot broaden consensus rules and make previously invalid things now valid because then the old nodes will reject it and fork themselves off the blockchain. That would be a hard fork. In the case of redefining the NULLDUMMY value the bug CHECKMULTISIG. Old nodes, any value will do including zero. New nodes, only zero. Therefore there is never going to be a conflict between them. If you put in 1 all the new nodes are going to reject it, they are going to enforce that consensus rule against you. The old nodes are going to go “It looks valid”. That’s a soft fork. All of these things have been soft forks. It appears now we can do anything with a soft fork. Someone has even suggested doing a change in the 21 million coins with a soft fork. Soft forks for the win! This is it. I’m not doing anymore. We’re going straight into the Q&A.
Q&A
Q - Does DROP take off more than one?
A - You can do 2DROP. 2DROP is another opcode that takes two off. It is a double drop. I don’t know if there is a 3DROP. (There isn’t.)
Q - This stuff reminds me of coding in assembly. Is there any movement towards making a compiler?
A - No because you can’t make a compiler into a higher level language for the simple reason that because this isn’t Turing complete, it is missing one fundamental construct which is there is no loop construct. There is no recursion construct either. Without a loop or recursion construct you have a language that is not Turing complete which means it cannot express all possible programs. If you take a higher level abstraction language that means that many of the programs you can write in that simply do not have an equivalence in this. With Turing completeness any symbolic language that is valid and consistent can be translated into any other symbolic language that is valid and consistent. That is the meaning of the Turing theory universal computing engine. This is not Turing complete. You can’t create a compiler. Probably a good thing because the compiler would hide details that would end up causing you bigger problems when you compile it down to this. You would have unanticipated consequences.
Q - I just wanted to make another esoteric point. I wrote an interpreter for all this. ELSE and ENDIF are not opcodes.
A - They are not opcodes. They are syntactic sugar that has no function in the language. They are like the curly brackets in C, they don’t actually do anything. They just give you the boundaries. They are scope identifiers. They define the scope of a function for syntactic purposes. Very good point, thank you.
Q - With the activation of SegWit it has clearly shown that Bitcoin is a lot more than money. Could you talk about that and specifically how entrepreneurs are dealing with outside factors?
A - SegWit isn’t activated. We’ll see. You may have noticed there is a bit of drama in Bitcoin. SegWit itself doesn’t really change the nature of Bitcoin being able to do a lot more than money. There are other technologies that have redefined it as more than money long before SegWit. Probably the two most influential were the original implementation of colored coins which allowed you to give an additional attribute or coloring to a specific value to mean “This satoshi is a share of IBM. This satoshi is a share of Microsoft and tradable as such.” It is like putting a stamp on a dollar bill and giving it a different meaning. Not in exclusion to its original meaning. It is still a satoshi that is spendable. You’d just be an idiot to spend it as a satoshi when it is worth so much more through its color. Colored coins came out in 2013 or 2012. Before that there was a specification which I think was even earlier than that. The second one is all the second layers that are enabled either with OP_RETURN or through hacks like Mastercoin now Omni, Counterparty and all of the other things that can trade assets and give other meanings.
Q - …
A - Bitcoin is a transactional state engine that among other things also transmits value. It is a non Turing complete transactional state engine as compared to say Ethereum which is a fully Turing complete transactional state engine. But it is a transactional state engine and you can do a lot with an abstract state machine.
Q - What is the best way you’ve found to write Scripts and test them, test their redemption conditions? Do you just submit them to testnet and then try to redeem them?
A - Yes you run them on testnet because unanticipated things will happen and you will lose money. One of the things you need to be aware of is all of the things that we talked about today go inside a pay-to-script-hash Script. That means they acquire a 3 address, an address that starts with 3 just like multisig. They are part of a pay-to-script-hash which means that the network doesn’t see this whole redemption Script until you try to redeem it. All you tell it is here is the fingerprint of what I am going to use later to test redemption of this. It has to match that fingerprint but you don’t say what it is. The beauty of that is you can give it a fingerprint to a completely s*** Script. The network will go “Fine I will lock up your coins with that.” Then you go back to redeem it, you say “Here’s my fancy redeem Script” and it goes “It’s invalid. Your coins are lost forever.” One of the things that you have to be aware of when you’re doing P2SH is that you are not validating the redeem Script. The chances of it being redeemable are kind of 50/50 or worse.
Q - I was wondering if there was any better way to test that?
A - There are five or six interpreters, Bitcoin Script interpreters. One is called WebBTC. It demonstrates the entire function of the stack. You can see things that get popped off step by step. What does the stack look like, what does the Script line look like? A similar notation to the one I used. That will do it for you. However it is a simulation. A lot of the Script interpreters are not faithful to the actual Core consensus rules. Which means if there is a bug like the CHECKMULTISIG bug you will read in the fine print at the bottom that it says “We don’t actually do the extra pop of the extra value per CHECKMULTISIG” which is great for troubleshooting your Scripts only they won’t work on Bitcoin. Bitcoin’s consensus rules include all the bugs. If you don’t faithfully replicate the bugs in your Script it won’t work. The interpreter can allow you to experiment for a bit and develop some familiarity. It is more of a learning tool. There is nothing that can test your Scripts other than testnet. Testnet is the full consensus rules, that’s the only way you can test it.
Q - Do you have examples of the different paths through this in the book making use of the stack notation you were using earlier?
A - I don’t have the full stack but for each one of these paths I show what the unlocking Scripts are and walk through the various things including dropping hints about and explaining why that was dropped there. It does a full analysis of this over about three or four pages. How do you do this one? How do you redeem that one? How do you redeem that one? Also why is the DROP there? What does the 2 do? How does it work out? It is a similar demonstration to what I did today although you can’t ask the book Q&A so a good thing you came today.
Q - Regarding cryptocurrencies, we seem to call everything cryptocurrencies and clearly it seems Bitcoin is more than a currency. What are your views on the space where there are things like Ethereum, a lot of these platforms which are seemingly trying to build these capabilities? Do you think Bitcoin is still going to be competitive?
A - That is the 20 billion dollar question. I think having worked in programming languages for a long while and having seen what a creative person can do with very little. I’ll give you an example. This is completely irrelevant to cryptocurrencies but it is a funny story. My first computer was a ZX80, a Sinclair computer in 1982. It only had 16 colors. Not 16 bit color, 16 total. It had the color perception of a three year old. No pink for you. How do you do video graphics for your games if your characters can only bright red, bright blue etc? I was dissatisfied by this state of affairs. I was offended so I made it do pink. The way I did it was I hacked the video code and invented interlacing. I was 11, I didn’t know that interlacing had even been invented. I figured that if I spent half of the time of the video refresh on the TV showing red and then during the time I am drawing the other half of the lines I show white and we do it really fast at 30 frames per second what is going to happen? Pink happened. I did it in assembly because that is the only way to do it at 30 frames per second at 11. The designers of that system had not anticipated pink and I did it for a very trivial reason. This was not a multimillion dollar project. This stuff is money. How much creativity can you get with adult experienced programmers who know what they are doing, motivated by really exciting applications and perhaps funded. Do not underestimate the creativity of the human mind and what it can create. Lightning Network is the perfect example. No one saw that coming. It is ingenious in its simplicity but also in its depth. It has got a lot of layers to it and once you start unwrapping you find more and more. There are an entire category of problems that Bitcoin cannot do because it is Turing incomplete. In fact based on the Turing theorem there are an infinite number of problems that Bitcoin cannot do. There are a finite number of problems that it can do. There is a very big number in the word finite. Sometimes when that number is big enough you can’t tell the difference. Here’s the interesting thing. We are trying to do Scripts. What are Scripts? Fancy people with VC money call them smart contracts. As you can see they are not that smart and not that contracty, they’re just Scripts. This is a smart contract for a governance program for a 3 person partnership with a recovery plan and a grandfathering plan and key rotation. All kinds of other features, pretty sophisticated. It’s a smart contract. It would be easier to write in Solidity but you can write it in Script. The interesting thing is what is the class of problems that you can solve in Script that don’t actually need the full blown Ethereum stack? I would say the answer is probably 80 percent of the most interesting financial instruments that correspond to trillions of dollars in value transacted everyday. That makes Bitcoin relevant for the long term. The fundamental reason is because this does it at a much higher level of security than Ethereum in my opinion because it is limited. Fewer things can go wrong with this surface. That doesn’t mean Ethereum isn’t useful, I’m a big fan. Just different problem classes.
Q - For everything else there is Rootstock right? You mentioned that soft forks can only make the rules narrower. You then also said soft forks can do anything. These seem like contradictory statements. Can you explain how that is?
A - Remember that part where I was talking about human ingenuity and surprising outcomes and never betting against the possibility of fitting a very large number in the word finite. I am still shocked at how SegWit was done. I just saw a new proposal for something else today that is a soft fork that was even more shocking. I’ve seen proposals for doing soft forks that do quite incredible things that I didn’t think were possible. I wouldn’t put it past them expanding the consensus rules in a soft fork manner. I don’t know, we’ll see. Maybe it is a contradiction, maybe it is not.
Q - Is this just because they are able to repurpose the existing opcodes that are in the base blocks and then add some of this data outside that new nodes will be able to interpret?
A - That is what SegWit does. The way SegWit works is rather ingenious. It creates a structure that simply pushes two numbers on the stack. What you are left with is a stack with two numbers on it. What does that stack evaluate to? For every node that is not looking at a Script it doesn’t have an OP_EQUAL, it doesn’t have a VERIFY, it doesn’t have anything. What is the default thing that happens at the end of a stack? The thing you don’t need to say at the end of a stack because the stack does it for you. VERIFY. It does a VERIFY for you effectively. The end of the Script execution you get to a state where it just takes what is on the stack and says “Is this a true value? If it is we’re good. If it’s not halt.” The definition of TRUE in Bitcoin is any number that is not zero. You push two numbers onto the stack and then you terminate. That’s TRUE. That’s a Script that anyone can spend. The concept “anyone can spend” that is defined in SegWit isn’t really a concept. It is just how about we push two numbers on the stack. Everybody who has implemented SegWit will know what those two numbers mean. They are a witness Script and a version. We will interpret them as a witness Script and a version. Anybody else will look at them and go “That looks like a TRUE.” We’re done here. That’s how you do a soft fork and that is pretty ingenious. I was surprised when I first saw it. I never thought of just push numbers on the stack and don’t put any ops, Scripts in there. Leave the interpretation of that to the nodes that come after.
Q - I think I just came up with a possibly correct way of explaining how creating restrictions can lead to less restrictions to John’s questions. Let me know if I’m making sense or not. We have an existing consensus in this room that 2+3=5 but a group of us can look at that statement and say “No 2+3=6”. We are organically growing our little consensus group. Eventually if we get big enough we have redefined what 3 means. The people who used to look at that same sequence of symbols and say “2+3=5” they can still compute. We can look at that and say we’ve redefined what 3 and 5 means so we get 2+3=5 but in our language 5 means 6.
A - You can do all kinds of hacks like that. There’s some interesting plays there in terms of what consensus means. Does it simply mean the more nodes that accept it? In Galileo’s time consensus briefly was that the earth is at the centre of the universe. Most modern history teaches us that’s what people believed at the time. No they didn’t. The ancient Greeks a thousand years before were like “The sun is in the middle and also the distance between us and the sun is so much.” We got it down to 0.3 percent accuracy by putting two sticks in the ground and measuring their shadows. A thousand years before Galileo. The Greeks didn’t think the earth was in the center of the universe. That came later. That was a hard fork from reality’s consensus. Short lived. We reconverged on the truth. For short periods of time you can have weird effects where a whole group of people are persuaded of something that is simply not true or is their truth and they can have their truth. Part of the difficulty we have here and you’ll notice this when we are talking about some of the bugs that are in Bitcoin is when you redefine things. Previously OP_X meant this but now it means you are going to see two more values that define another 150 opcodes so from now on don’t just interpret this but interpret the DROP after and the number after that as NOP DROP and then take the number and that’s a whole new set of opcodes that mean something else. You have basically jammed another language. In the space of one opcode you can jam an entire language. That’s how Unicode works. That’s called keying. You take one element of the language and you say “All of the other numbers mean what you think except for this one that means look at the next number that comes” and it gives you another 256 possibilities. Bingo we just upgraded everything. You do that, you keep doing it. What you are adding is layers of interpretation that overload the meaning of the language. That creates what is affectionately known in programming as crud. If you allow too much crud to accumulate you start having side effects because these things start conflicting with each other. You have old nodes and new nodes and they interpret the crud differently. You create layers. You keep doing that for 20 years, “We still want to be backwards compatible with those guys who are running that little thing back there” and you end up with Windows. You can run Lotus 123 from 1987. Fantastic. Yes but I can’t run anything anymore because in order to do that you’ve layered so much crud in the operating system that doing the simplest thing involves wading through the crud of four decades of crud laying. As a result everything is slow, inconsistent, buggy, full of errors and vulnerabilities. Every now and then you just go “How about we’re not able to run that old stuff anymore?” The problem with the consensus layer is that you can never do that. You can never do “How about we make sure that Satoshi can’t spend our coins anymore?” They have been sitting there for so long, they’ve been really quiet, let’s disenfranchise them out of a billion dollars whatever it is now of their coins. They had some weird rules back then. How about we just wipe the slate clean and say they are no longer consensus. The side effect will be that anyone who had coins in the first four thousand blocks can’t spend them anymore. Unacceptable in Bitcoin. That is the difference between hardware, software and trustware. Trustware says you have to carry these consensus rules forever and you have to make sure that the coins that were redeemable then are still redeemable now. That the blocks that were valid then can still be validated by a node now. Which means that you keep laying on the crud. You are going to have a natural accumulation of crud just to make simple advancements. If you then take that natural accommodation of crud and to avoid a political debate you add a whole layer of it voluntarily now you are just making problems for the future. This is the problem with developing trustware. Bugs become consensus code. Fixes to bugs sometimes introduce more crud than the bug itself so we don’t fix them. The cure is worse than the disease. Voluntary introductions of crud into the code will be a burden that we carry forever because once you put it in and someone writes one UTXO that is redeemable by that crud code, that crud code has to be carried forever so that that UTXO can be redeemed in the future. This is the problem with trustware. There is another approach and that is the approach that Ethereum is exploring. Didn’t work? Hard fork. There was that other… Hard fork. There’s a problem in the client now because we have 19 gigs of… Hard fork. That’s a different strategy. Very effective. I wouldn’t bury a UTXO in there for 2 years. I would be worried it wouldn’t be redeemable. There are different strategies to this. We are going to see things that move really fast, things that move really slow. While you can do tricks like that you have to consider the future cost.