SLP99 Andrew Chow – Hardware Wallets and Bitcoin Core with HWI and PSBT

Posted in: bitcoin Posted on

Andrew Chow, Bitcoin Core developer at Blockstream joins me in this episode to talk about his work on HWI – Hardware Wallet Interface and PSBT – Partially Signed Bitcoin Transactions. We discuss the benefits of PSBT and Hardware Wallet Interface in making Bitcoin’s user interface better for users who want to do multi sig or interact with hardware wallets using Bitcoin Core.

We talk:

  • Andrew’s background
  • Why PSBT (Partially Signed Bitcoin Transactions) is needed
  • HWI – Hardware Wallet Interface
  • Output descriptors
  • Derivation paths
  • Use of PSBT for CoinJoining and Multi-Sig
  • Where HWI is going from here

Andrew Chow Links and other links discussed in episode:

Sponsor links:

Stephan Livera links:

Podcast Transcript by GiveBitcoin.io:

Stephan Livera: Andrew, welcome to the show.

Andrew Chow: Thanks for having me.

Stephan Livera: Andrew, I know you’ve been doing a lot of work recently on things like HWI, PSBT and so on. But first, let’s hear a little bit about you. I know you’ve got a bit of a history with Bitcoin development. Tell us, how did you get into it and how did you get started with development?

Andrew Chow: I’m a Bitcoin Core contributor, and I’ve been working on this side project for Core called HWI, which is what we’re going to be talking about today. I got into developing Bitcoin like three or four years ago, just as something to do in my spare time while I was in high school, and got a lot more involved from there.

Stephan Livera: Now you’re working full-time as well on Bitcoin?

Andrew Chow: Yes. Right now, I’m at Blockstream, and I’m working full-time on Bitcoin Core and other things and other Blockstream things too.

Stephan Livera: Nice. That’s awesome. Let’s get into HWI. I’d love to hear a little bit of the story behind it. Why did you develop HWI?

Andrew Chow: In 2017, I was an intern at Blockstream, and at that time, I had also gotten a hardware wallet, and naturally, have a hardware wallet, want to use it. Well, I found out that I couldn’t use it with Core, and I actually couldn’t use it with any full node that I ran myself. I eventually did get ElectrumX running on a server, and then having Electrum connect to that, but I didn’t really like this thing, and your average person isn’t going to be able to do this.

Andrew Chow: I talked to my mentor, which was Pieter Wuille, and we were talking about what would be required to get hardware wallet support into Core. Basically, we came up with that we needed some driver, some program that had all the drivers, to talk to a hardware wallet that Core would call out to or connect to and send data. Then it would have to have some API or data format that Core could send to the drivers in order to communicate. If you where this is going, PSBT became that data format. That’s how we’re getting data from Core to the driver. And HWI became the driver.

Stephan Livera: Sorry, I want to take it back one step, just to make sure, just for the listeners who are not as advanced. I guess at a more basic level, why is it a good idea to connect your hardware wallet to your own full node, instead of trusting somebody else’s node, or giving up your privacy to somebody else’s server.

Andrew Chow: There’s the obvious privacy thing. You’re not sending all of your addresses to someone else, who can do whatever they want, like tracking or something with that information. When you use your own full node, it’s just coming from the local Blockchain data that your node has. There’s also a bit of security when you have a full node. You’re not trusting someone else to be providing you the correct transactions and the correct Blockchain and all that stuff. So it’s just better security and privacy-wise to be running your own full node. Hardware wallets are a better way to store your keys, and you kind of want to combine them.

Stephan Livera: Excellent. I guess before HWI, what were the main options available to people? I guess the option was either use say a Trezor or Ledger, the web interface, or you had the option of using Electrum public server, which is again calling out to all those public servers and giving off your own data. Then you mentioned ElectrumX, and I think the other ones are Electrum Personal Server by Chris Belcher, and also I think there’s Electrum Rust Server.

Stephan Livera: Do you want to just talk a little bit about some of those options?

Andrew Chow: At the time I started working on this, as far as I can tell, the only third party software that really had hardware wallet support, like decently, was Electrum. Electrum is an SPV wallet, it’s not a full node, so it calls out to these Electrum servers, which provide its data. It’s like kind of decentralized. The servers aren’t run by Electrum, they’re run by anyone. The Electrum wallet will connect to multiple of these servers. But it’s still SPV. They can attack you. They can be DDoS, like many have been.

Andrew Chow: If you want to have better privacy and security with Electrum, you should run your own Electrum server. The common software for that is ElectrumX. That’s what I was running when I did my original setup with Electrum and ElectrumX. At the time, Electrum Personal Server didn’t exist yet, or I don’t think it existed yet. Belcher hadn’t written that. The Electrum Rust Server, I actually didn’t hear about until like several months later, so I don’t know anything about that one. All I remember is Google setting up your own Electrum server, and it was just run ElectrumX, because the original Electrum server was too old and non-performant that it couldn’t actually catch up to the Blockchain.

Stephan Livera: Wow, I didn’t know that actually. So basically most of the ones that you are calling out to are ElectrumX servers, whether that’s your own personal one, or whether that’s Blockchain spy companies who are running these-

Andrew Chow: Yeah, most people run ElectrumX. It’s the most widely used and most supported, and it’s pretty easy to set up. That’s what I was using for the hardware wallet stuff. You have to run two different software, and you have to have a server and stuff. It’s kind of annoying.

Stephan Livera: Yeah, I see. Yeah, it’s a little bit finicky. So the idea then was HWI would be working by default. If it does eventually get merged into Bitcoin Core, then it would be a more simple process for the user to just double click, it’s all… and off you go. Is that the hope?

Andrew Chow: The idea is that HWI will be packaged with Bitcoin Core. When you do anything that requires a hardware wallet, Core will call out to HWI, and HWI will do something and return results back to core. It all happens in background, the user doesn’t notice. It would be the same as if we just implemented the drivers directly into Core itself, which we don’t want to do for security reasons. Hardware wallets are all over USB, and the USB driver stack is really big. It was just a major attack surface.

Andrew Chow: So we don’t want to have that in Core itself in case someone isn’t using hardware wallets, they don’t need it, but there’s this huge vulnerability somewhere in the USB stack, and now they’re vulnerable to it. We don’t want that. By having the separate program, we completely avoid this issue, because it’s only active when you’re actively using your hardware wallet.

Stephan Livera: That’s essentially what HWI enables. It enables people to use their Trezor or their Ledger or whatever hardware wallet or Coldcard directly with Bitcoin Core. I think we should unpack that a little bit, and start to talk about some of the pieces that make HWI possible. Can you give us a bit of an overview on that?

Andrew Chow: There’s three-ish components. There’s the actual drivers and talking to the device itself, and there’s the API wrapper around that, and then we have PSBT. The drivers part, for the most part that’s actually provided by the hardware wallet vendors. They all have Python libraries, and HWI’s written in Python, so I just pulled in their Python code, and now I can talk to all their devices without having to really, truly dive deep into everything that I have to do to talk to the hardware wallet, because if I had to do that, I probably wouldn’t have done it. It’s just way too much work. This also lets me support a bunch of different things. We can just add another driver in later for some other device.

Andrew Chow: Then the actual HWI part is the wrapper around that. It’s just calling the drivers when necessary, calling the correct APIs, and taking in the PSBT or other stuff from the command line and converting it into the right things that every device specific driver can use, because they’re all different. They all have their own APIs and stuff. HWI is just providing a single unified API to connect to all the devices.

Andrew Chow: Then we have-

Stephan Livera: One API to rule them all.

Andrew Chow: Yeah. The nice thing actually is because HWI will be a separate program, if HWI doesn’t support a hardware wallet, the vendor can actually just go and write their own version of HWI with support from their hardware wallet, have the same API, and now people can drop in, replace that and it’ll work with that new hardware wallet. Then I won’t have to do all the work.

Stephan Livera: Very nice.

Andrew Chow: The last layer is PSBT, which is just to hold all the transaction data itself, and be able to send it from Core to HWI, and even to other clients, because it’s designed to be fairly generic. It has all of the information, or it will have all of the information that you would need to sign and then create the final transaction that would be broadcast to the network.

Stephan Livera: Got it. Let’s unpack PSBT a little bit. For the listeners, that is Partially Signed Bitcoin Transactions. Andrew, can you give us some overview on that, and why was that necessary?

Andrew Chow: This is necessary because when you want to sign a transaction, you need a lot more than just the transaction that you’re signing. You need, for example, for a transaction that spends like a P2SH script, like multi sig, you need to know the script that you going to put in that final transaction, and if you don’t already know it, the transaction that you’re signing isn’t going to have it for you. For hardware wallets, they don’t know what this is. You have to tell it to them. Same with offline signers. They have to be told what the script that they’re going to be signing is.

Andrew Chow: Through PSBT, we can package this is the transaction we’re signing, and here’s a script that you need in order to sign it. Package them all together so it’s now one large data blob that we can send off, instead of having to have multiple pieces that need to be sent. If you look at Bitcoin Core’s sign raw transaction thing, it’s if Bitcoin Core doesn’t know the scripts, the API is awful to use. It’s JSON, and then you have to… JSON itself is annoying, to make sure that you get all the brackets and quotes correct. It’s really easy to screw up.

Andrew Chow: It’s also not super universal. The raw transaction format itself isn’t the same in Electrum or like Armory. All three of them have different transaction formats for stuff that aren’t signed yet, for transactions that haven’t been signed yet. And they’re all incompatible with each other, so if you want to say do a multi sig between Core and Electrum, because you’re paranoid, you’re going to have a hard time making sure that both Core and Electrum can sign the transaction, because they don’t share the same format.

Andrew Chow: The idea behind having a bit behind PSBT is that people will actually use it, and now we can have these different clients be able to sign transactions with each other.

Stephan Livera: Got it. Look, let me just try and break a little bit of that down, and make sure my understanding is correct as well. What we’re talking about here is hardware wallets are in some sense… they’re quote, unquote, dumb. They’re not connected to the network, so they don’t know some of what’s going on. Part of the complexity of what’s going on is when you spend a Bitcoin transaction, every transaction has certain inputs that go into the outputs. And in order to unlock, if you will, or be able to spend those inputs, you need to satisfy the encumbrance or the locking script placed upon that input.

Stephan Livera: That’s what you’re getting out there as I understand with the signature, and basically the device which holds the private keys to be able to sign that, it doesn’t know what script type. And that’s what you’re getting at there, that the PSBT is a format for different Bitcoin software to interact with the hardware wallet, and tell the hardware wallet, “Hey Trezor, I need you to sign X, Y and Z, and it is this type of script, and these are the addresses that I need you to spend into.” Is that more or less right?

Andrew Chow: Yeah. Hardware wallets actually need to know even more than just the scripts. The script is necessary in signing because it is actually signed over. That’s part of the message that is signed. But also, a hardware wallet needs to know what to sign with, because the hardware wallet isn’t going to store every single possible key it could generate. That would be something on an order of like billions or trillions of keys, and they just don’t have the memory for that. Instead, you need to tell it sign with the key at the derivation path M44-something-something. PSBT has another field that says this pub key is derived at this derivation path, so now HWI or something else can go tell the hardware wallet, “Hey, go sign with this derivation path.”

Andrew Chow: It has that too, which is… You don’t really need this in order to sign, but hardware wallets really do need this in order to sign. Other software, like Core actually just completely ignores this field, because it stores keys locally and it will just look them up if it can. It doesn’t derive keys as it signs.

Stephan Livera: Let’s try and break this down a little bit as well. This is getting into this concept of derivation paths. Now my understanding on this, and again correct me if I’m explaining any of this wrong… It’s a little technical so I might fluff this part. But my understanding here is you’ve got BIP32, that is known as hierarchical deterministic wallets. The idea is you take a seed, and that seed is used to generate what we call a master private key, like your xPriv. That master private key can in turn be used to generate child keys, and they are like a child private key sitting in that tree underneath your master private key. Then under that child private key, you have leafs, and those leafs are what can be used to make addresses, right?

Andrew Chow: Not quite. You have the master private key, and that can generate child private keys, but then the child private key is also another master private key, and so it can generate its own private keys, and this just goes ad infinitum. You just keep going down as far as you want. Each level is like a different step in the derivation path. So if you see like M/44, that means that master key M, and you derived the 44th child, and that’s the key at M/44, and you have like another /0. Now, under the 44th child key, you now derive its 0th child key, and you just keep going down like that. That’s how the derivation paths work.

Stephan Livera: Got it.

Andrew Chow: The indexes are a bit weird, and it’s just part of the derivation algorithm. It only makes sense if you’re reading it.

Stephan Livera: I see. Could you help break that down for us a little bit. You might have your master private key, or a master public key if you don’t want to give off the private key for it, and then you need the xPub and the derivation path to know… because you might have split accounts sitting underneath that master key. Is that where, like you said, M/44/0/0… Can you just help outline what are those numbers and what do they mean? One of them is the account number, and one of them is the-

Andrew Chow: That’s like part of BIP44, which I don’t actually totally remember. That’s kind of a… someone defined these things, but they’re not part of hierarchical… whatever. BIP32 itself. BIP32 is just how you go from a master key to a child key, and then just for its simplicity, instead of going directly, you can only have a single line of keys. It’s just a tree, and we stick an index somewhere in the derivation algorithm, and that’s what we mean by 0th or 1st, 2nd, 3rd, 44th key, that kind of thing. There’s just the term i, and that is the index, and you just replace that with whatever index you’re using.

Andrew Chow: BIP44 defines derivation paths in a specific pattern, and a lot of wallets use this, but it doesn’t totally make sense. I believe it goes like four or five levels. You have the first one, M/44. The 44 is the purpose, and that’s something that’s predefined. There’s a list on SatoshiLabs GitHub somewhere, that just lists out a ton of different purposes. And that’s what this first index, under the BIP44 scheme, should be.

Andrew Chow: Then there’s account, something else, maybe that’s backwards, which are just used for like… internally, you can change those, and get a different account, like a different derivation path tree, and you can get different addresses. Then at the end there is change. You set a zero if you want that derivation path to be for all your receiving addresses, and a one for your change addresses. That’s just to separate out, so you don’t have change and non-change keys mixed together. Then the last one’s the index for the child keys.

Andrew Chow: But you don’t have to follow BIP44. You can do something else. Bitcoin Core notably does not follow BIP 44. There is actually a security risk. There’s also a thing called hardened derivation and unhardened derivation. Hardened derivation means that you can only derive the keys if you have the master private key. If you don’t have the master private key, you’re not going to get any child keys. Unhardened derivation means that you can derive the child keys from the master public key. You can have the xPub, and then you can derive child keys from that. You don’t need to have the private key.

Andrew Chow: BIP44 does a mix of this. The first three, the purpose, account and that other thing, are all hardened derivation. Then the last one, the change and the index, are unhardened. This is good for hardware wallets. A hardware wallet gives you an xPub, and then you can load that into your software, and that’ll derive change and non-change and all the indexes from there, without needing to know the private key, because your software just knows the xPub. That’s great, but the caveat is that if you have a child key derived with unhardened derivation, and you have its private key, and you know the parent xPub, then you can also derive the parent private key.

Stephan Livera: Big security risk, right?

Andrew Chow: Which is a huge problem if you’re storing your private keys in a way that they could be exported. Bitcoin Core doesn’t do this at all, because we have dump priv key that just lets you export a private key. We don’t want people to export a private key, and then accidentally leak their parent private key, and now someone can steal all their coins. Bitcoin Core uses entirely hardened derivation, at least right now.

Stephan Livera: Let me just break that down again for the listeners. Listeners, check out the earlier episode with Michael Flaxman, where I think Michael explained a little bit in that episode around this risk that if somebody gets your child private key, and they’ve got your master public key, and as Andrew… as you’ve just explained, that allows them to now basically… they can back out the master private key, meaning they could just steal all your coins, whereas as Andrew, you’re pointing out, if you’re using hardened derivation, that’s not possible.

Stephan Livera: And that is the approach that Bitcoin Core as a group have taken as a way to be more conservative and more, I guess, security conscious, whereas perhaps other Bitcoin developers out there in the wild have used this method for usability reasons, I might say, to make it like easy for you to hook up your hardware wallet and someone, and generate addresses without having it connected in and that sort of thing.

Andrew Chow: Having an xPub that you can derive keys from is pretty useful. Just on a normal software wallet, if you encrypt your private keys, and you run out of the pre-generated key pool, on Bitcoin Core, when that happens, you have to unlock your wallet, let it generate more keys, before you can get new addresses. But if you’re using an xPub, you can just keep getting more and more addresses, even when your wallet’s fully encrypted.

Stephan Livera: Got it. Obviously a bit of research for this episode, I was looking on Bitcoin Stack Exchange, and I was looking at some of the answers you were giving, and some of the answers Pieter Wuille gave as well. He was talking about this idea of output descriptors. As I understand, part of it is there was a little bit of… I think owing to that idea you were talking about there, of not having a standard way of doing these things, people came up with things called… On top of the xPub, they had the yPub and the zPub, to generate say SegWit or native SegWit addresses.

Stephan Livera: But I think Pieter was getting at this idea that output descriptors are a better, less ambiguous way to describe those or distinguish those different script types. Can you touch on that a little bit?

Andrew Chow: Electrum and Trezor added to BIP32’s xPub the yPub and zPub, and they basically said that xPub means that you’re deriving that… any pub keys you generate from the xPub is going to be for a legacy non-SegWit address. Then any pub keys you derive from a yPub will be for P2SH wrapped I think, and zPub is Bech32. But this is kind of not what BIP32 was meant to do. xPub was never really supposed to say that… It was never supposed to define an address type. It’s just for an extended public key that you can derive public keys out of them, and use those public keys to do something else. Put them in scripts, multi sig, whatever.

Andrew Chow: Pieter has been working on mini script and output descriptors. What output descriptors do is that they actually define a script that you want to use in a human readable way. If you wanted to use public keys derived from an xPub in Bech32, you could put that inside an output descriptor, and then the descriptor would have WPKH at the front of it, and that would signal that everything inside this, any public key, should be used in a Bech32 output. That means any public key derived from this xPub should have a Bech32 address.

Andrew Chow: You can have different things in the front, and that just indicates that from a same xPub, you could have different kinds of addresses. Instead of having this thing where we’re just defining new magic numbers and saying that this magic numbers means public keys derived should be this address type, we now have some human readable way that says what address to use, and it’s also flexible in the future if we define some new address type. We can just add a new thing to the descriptors without changing how keys are derived.

Stephan Livera: Got it. Let me just quickly re-summarize that. Essentially it’s saying instead of using xPub, yPub, zPub… Like we’re going to run out of letters eventually, and it’s probably not the most unambiguous way, whereas a more specific way to talk about these is to have a specific format. And as I can see from Pieter’s answer on stack exchange, he lists out a couple of examples. It might say PKH, in brackets xPub, ../, and then that’s where the… 44/00, et cetera, which is the derivation path, and that might explain okay, this is a BIP44 address from a particular xPub, which is the Pay-to-Public-Key-Hash type. It might specify exactly what type. And so that way, when wallets and different devices are passing information to each other, they can say, “Hey, this is the output descriptor, this is the unambiguous way to derive the exact address,” let’s call it.

Andrew Chow: And it’s also human readable. I can look at that and know exactly what it’s saying. If I look at a yPub, I didn’t memorize what yPub is, so I actually don’t know what yPub does. But I can see SHWPKH, I know that’s script hash, with witness pub key hash inside of it, because these are common endings that we use in Bitcoin, like P2SH, P2WPKH. These are all things that we see working on wallets.

Stephan Livera: That’s just referring to the different script types, just for the more beginner level listeners.

Stephan Livera: Let’s bring it back to the PSBT, where it aligns back into PSBT. As I understand, there are different roles within PSBT. You’ve got creator, updater, signer, finalizer and extractor. Can you tell us a little bit about what each of those roles are?

Andrew Chow: Ideally, these are supposed to be self-explanatory. Creator creates the transaction. It decides what the inputs and the outputs of the transaction are going to be, like the TXIDs and the vout indexes that we want to spend, and which outputs and their scripts and the amounts that we want to create in this transaction. And it wraps that up in all the PSBT overhead basically, because when the creator makes a PSBT, it’s just going to be a raw transaction like you would see on the network, just without anything in the scriptSig, so there’s no signatures, no input scripts or any of that. That’s just going to be wrapped with the PSBT header and a couple of other bites that are needed for PSBTs.

Andrew Chow: A creator creates the transaction and sends it off to an updater. The updater will inspect the transaction the creator made, and add information about the inputs that are being spent. It’ll add like the UTXO. For SegWit inputs, it’ll take just the output from the previous transaction itself. It’ll just be the amount and the script. For non-SegWit it has to include the entire previous transaction. We need this because in order to sign we need to know what the script was, and the script of the output that we’re spending, and we sometimes need to know the amount. Like SegWit actually needs to know the amount for… I don’t remember reasons.

Andrew Chow: Then the updater will also add other information if has it. If it knows that the output that’s being spent was P2SH, it’ll add the redeem script for the P2SH. If it knows that it was a SegWit witness script hash, then it’ll add the witness script. If it knows any of the pub keys in the output, or in the redeem script or witness script, it’ll add the pub keys and their derivation paths if it knows them. Then there are future things that it could also add.

Andrew Chow: The updater adds all of this to the inputs. It also does it to the outputs, if it knows anything about the outputs. It’ll add the redeem script and the public keys to the outputs. All this is really only if the updater knows it. Once the updater’s done, it might send it off to another updater, who will add all the information that it knows, or it’ll send it off to a signer.

Andrew Chow: The signer will look at the transaction, and because the updater had added the UTXO and the scripts, the signer can be completely offline. It doesn’t need to know anything about the Blockchain, it doesn’t need to be connected to the internet. It could be a hardware wallet, or it could be a computer locked in the safe 50 feet underground. It can be completely air gapped and offline, and this was really what PSBT was meant to do.

Andrew Chow: The signer can now look at the transaction. If it needs to sign say input zero, it looks at the data that the updater added to input zero, and it sees there’s a UTXO. Now it knows the output script, which it needs to sign, for signing usually. If there’s a redeem script, it now knows the redeem script. If there’s a witness script, it also knows the witness script. Then the pub keys are also listed there, so it can go generate the private keys if it has to, or just pull up the private keys, and it knows everything that it needs to do to sign. It’ll check the transaction, makes sure that the user likes it, like it’ll look at the… Because we have all the UTXOs, we know what the transaction fee is going to be, which is useful. The user can be like, “Oh, that transaction fee’s too high, I’m not going to sign this transaction. I want a lower transaction fee,” or something like that. Or maybe, “The transaction fee’s too low, it’s never going to get accepted.”

Andrew Chow: The signer makes sure that the user likes the transaction, that the money’s going where it’s supposed to go, the user’s receiving their change if they are supposed to get change, and the fee is correct. Then it makes a signature, attaches it to the PSBT, and does that for every input that it can sign for. Once the signer’s done, it can send it to another signer if someone else is involved, or it can send it to a finalizer.

Andrew Chow: The finalizer, it does a conversion. The PSBT itself just has a bunch of records in it, but in a final transaction, you don’t have just a bunch of records. You have a script in the input. The finalizer produces that script. It takes the signatures the signer’s created, it takes the scripts that are already there, and it combines them all together in order to create the scriptSig or the script witness, and sticks that back into the PSBT, so that later these can be put together with the original unsigned transaction to be broadcast.

Andrew Chow: That last step is what the extractor does. It takes all the scriptSigs and script witnesses the finalizer created, sticks them into the unsigned transaction that’s at the top of the PSBT, and sends it off to the network.

Andrew Chow: There’s actually one more role, and that’s the combiner. In between updater, signer, finalizer, you can also send it off to a combiner, which will just take multiple PSBTs that have different data in the input and output, and just combine them all. It throws all the records together, as long as the global unsigned transaction is still the same.

Stephan Livera: It might be good to just talk through an example flow of a PSBT then. Let’s say just a single signature PSBT, and you’ve got an offline wallet, like a hardware wallet, and a watching only wallet. Could you just talk through what is the flow there? Would you create the transaction using Bitcoin Core? Is this like a CLI thing, or do we-

Andrew Chow: I’ll go through an example. This is how I use hardware wallets now with Core. Bitcoin Core acts as the creator. The hardware wallet is the signer, and Bitcoin Core is everything else. First I would use Bitcoin Core as the creator, and create a transaction. I choose my inputs, choose my outputs, and produce a PSBT. That’s what the creator does. Then Bitcoin will take that, and it will update it. It will pull from its Blockchain… from its access to the UTXO set and the Blockchain, and fill in all of the UTXO information for all the inputs that I’ve chosen, and then it’ll also add from the wallet the pub key information and the redeem script and witness script.

Andrew Chow: This is actually all done together as one command of ‘walletcreatefundedPSBT’. It just does everything at the same time. All you have to do as a user is say which inputs and which outputs. And maybe not even the inputs, because it will do coin selection for you if you don’t give enough money. It makes us a PSBT with all the updated information. I copy that, take that to HWI, and just give it the sign TX command, and the PSBT, click all the buttons on my hardware wallet that says like, “Are you sure you want to send to this address with this amount? Is the transaction fee correct?” Click through all the buttons if I like the transaction and nothing had changed it.

Andrew Chow: And HWI gives me another PSBT that has all the signatures in it now, and now I take this back to Bitcoin Core… what I do, finalize PSBT, and this is the finalizer step. It makes all the scriptSigs and the script witnesses. And it’s also the extractor. If Bitcoin Core finds that all the script sigs on script witnesses are there, and when it puts into the transaction, the transaction is valid, it gives me the final transaction that I can send to the network. So I can copy that and go to ‘send raw transaction’ and broadcast it.

Andrew Chow: Right now, this is actually how you use HWI with core, because we haven’t fully integrated it yet, and it’s just a command line thing. But it’s basically like four commands, I think. It’s ‘walletcreatefundedPSBT’, then ‘signTX’, then ‘finalizePSBT’, and ‘sendrawtransaction’.

Stephan Livera: Got it. Right now it is all CLI, character interface. Is there a hope or a plan that this would then become a GUI?

Andrew Chow: Yeah. Well, the hope is that it’ll all happen in the background. On the CLI, if I just did send to address, and I had a hardware wallet that was plugged in, the thing would show up on my hardware wallet, and I could click all the buttons there, and then it would magically all happen in the background with just one command. Then same thing for the GUI. When you click send, your hardware wallet’s plugged in, a thing will show up, and then it just magically all happens in the background. That’s what we’re working towards. Not quite there yet.

Stephan Livera: I suppose the other aspect to layer on there is multiple devices and also air gapping. Can we talk to some of that? Maybe let’s talk to the air gapping part of it first. Let’s say you wanted to export the unsigned transaction onto a micro SD card or ideally a QR code, could you talk to that process a little bit and what that might look like?

Andrew Chow: The transaction is just a base 64 string or the PSBT, and that contains all the PSBT information. Sometimes it’s also really big, but you just copy this string. You can put it like in a text file, or you could QR and code it, but QR encoders have a size limit, so you might run into problems there. But it’s just a string. Put it in a text file, something like that. Put it on a flash drive, micro SD, whatever. Take it out to your second computer or other device that has the private keys, and give it the string back. If that’s QR codes, yeah, it’ll just scan the QR codes, and the string will be there. Or you just copy it off your text file from the flash drive. Then you do the signing from the PSBT there.

Stephan Livera: Talking through that example, but in this case let’s talk about multi signature. Let’s say you’ve got a two of three hardware wallets set up or something like that, and basically we do that same PSBT process as you spoke about. Would the main change be that instead now, there would just be two signers instead of one? Is that the main change?

Andrew Chow: Yeah, pretty much.

Stephan Livera: Got it.

Andrew Chow: There could be two updaters also. Nevermind. Actually, that’s a bad example. That works for CoinJoins, not multi sig.

Stephan Livera: That’s another good example of PSBT as well.

Andrew Chow: In CoinJoins you would have multiple updaters, and everyone would update their own transaction, and it would all be combined before they all sign it again. But for multi sig, you’d probably have something where one person has their wallet set up to be watching for all the transactions, and they act as the creator and updater, and they send out the PSBT to the other signers. The other signers, without having to be watching that multi sig address, or even have that multi sig address in their wallet, they can just sign the PSBT and send it back to the creator.

Andrew Chow: I mean the flow is pretty much the same as the air gap one, except instead of giving it to your second computer, you’re just giving it to another person. That’s about it. CoinJoins are also pretty similar.

Stephan Livera: That’s interesting, because I guess… Obviously there’s so many different combinations, but I’m just trying to, in some sense, talk about some of the key ones that people might use or they might think about. And I guess the key scenarios here people would be thinking about are multi signature of their own coins, or maybe in a corporate context, they might want to use it as a control system, right? They want to, say, have a three of five, and they might want to make it so that the CFO and the head of finance have a key each or something like that, and they might want to split the controls up that way, and use the PSBT as a way to help pass around this uncompleted transaction until it’s ready to be signed and fully broadcast out to the network.

Andrew Chow: The thing with all these roles, and the way that PSBT is designed, is that for all these cases, it’s almost the same workflow. It’s just passing it from one person to the other. They’re doing almost the exact same thing as if it was just their own money and their own transaction, because the software is supposed to handle everything in the background.

Stephan Livera: Now let’s talk a little bit about some of the problems that we have seen in terms of making multi signature work nicely, and even for just hardware devices as well. One of the problems, or one attack that Michael Flaxman mentioned in my previous interview, which was this idea that at the time you are signing on your hardware device, and you’re doing say a multi signature transaction, you may not be aware of if your device is signing against your own other devices. A quick example might be in a two of three multi signature setup, you might be faced with the decision to hit accept or reject on your physical device, but you might not know that unbeknownst to you, that transaction might actually… The other cosigners might be your enemy, a hacker, rather than your own multi sig hardware wallets.

Stephan Livera: Can you talk a little bit about that problem?

Andrew Chow: PSBT has in the outputs… you can also include the scripts, the redeem script and the witness script. Ideally, you would also just look at that, and see that these are the public keys that you’re expecting to see and everything’s good. But there is a problem with xPubs when you’re doing a multi sig where every address is new because you’re deriving new addresses from xPubs that people have provided. This was the proposed extension that we added recently to PSBT, was that we would include an xPub at the top of the PSBT, and then when you see a public key, you can go and you can try to derive that public key from the xPub in the PSBT, and now you will know whether that xPub was used to derive that public key or not. You can see which xPubs are contributing to the outputs.

Andrew Chow: But the other problem is actually with change detection. Most users don’t actually really know about change, and they get scared when they see it, because they see their Bitcoins moving to some other address. If they see this change address being displayed to them by their hardware wallet, they probably won’t know what it is and might just reject the transaction even though it’s perfectly safe. The problem with change is that if we’re not showing the address to the user for them to verify themselves that this is correct, then the hardware wallet needs to do this check itself, and decide whether to hide it or to show it. The hardware has to decide if it’s safe for the user.

Andrew Chow: That was the primary motivation for having the xPub in the PSBT, so the hardware wallet can say, “I’ve signed this input that’s multi sig, and the pub keys in this input were derived from these xPubs.” Now, when I go to the outputs, and I see okay, this output has pub keys that are derived from those same xPubs, and it has the same threshold number. Now I can assume that this is our change, because we have the same people involved, and the same number of people need to sign it. So this is reasonably change. If one of the signers had replaced it, then this wouldn’t hold true, and we decide that this is not change, we’re going to tell the user that this is an output, instead of hiding it away as change.

Stephan Livera: Let me just summarize that again for the listeners, that might be a bit technical. I’m not sure I fully grasped it myself, but I’m going to try. When every transaction is crafted, think of it like your UTXO is being melted down and recast, and part of that is you’re receiving back some as change. What you’re talking about there is that idea that your hardware wallet has to make that call for the user, because many users don’t understand that concept themselves.

Stephan Livera: What you’re saying is that by seeing what are the xPubs going into this transaction, we could try to figure out if this is change or not, and then that’s when it would be flagged to the user, so that the user can know at the time that they’re pressing yes or no on their hardware wallet that, “Ah yes, this is either coming back to me as change because it’s my own money, or it’s actually a payment to somebody else. Somebody else is going to get the output.” Is that right?

Andrew Chow: Yes. This also happens to only really matter if the threshold, like the number of signers, the number of people that have to sign in this multi sig, is less than or equal to half of the total number of people in the transaction. If you have two of four, this would be a problem. But if you had a three of four, it actually isn’t really that big of a problem. And that’s because you can actually have a signer policy of if an output is change, if one of my keys is in it, I don’t really care about who else is signing it, as long as my key is there, and the threshold is… I don’t think the threshold actually matters that much, because as long as I’m there and I have control, I would be able to sign this in the future. Then from that standpoint, it’s safe.

Andrew Chow: If you have a three of four, and someone had replaced one of the keys with a second one of their own, and all three signers are following this same policy, the third guy… Let me think. Yeah, the third guy would be… The guy who’s key got replaced would not see his key in there, and so now he knows that this is not change. So he would be the one that refuses to sign, and now the transaction won’t go through, because that guy didn’t sign. But if it was a two of four instead, then I could sign, seeing my key there, and the attacker could sign, and he’s just replaced one of the other keys, but they’re not involved anymore. It’s just going to be me and him, and I’ve unknowingly sent away coins to an attacker. So this xPub thing really only matters when the threshold is less than or equal to half of the number of signers.

Stephan Livera: I get you, because basically it depends on how the attacker tries to go about that attack, and which keys they switch out, and what is the quorum required to make it happen. I suppose part of it though is most of the time when people talk about multi sig, they’re usually doing a setup where it’s majority. It’s two of three, it’s three of five, four of seven, et cetera.

Andrew Chow: Yeah. Usually, for most multi sigs, this probably isn’t even needed. You can just go by this selfish policy that I’ve described. When I was talking to Pieter about this, he was like, “Yeah, you can do this. If you just act really selfishly in this way, it doesn’t matter.”

Stephan Livera: The lesson is act selfishly when it comes to multi sig, guys.

Stephan Livera: I guess let’s talk a little bit about what the gold standard, or what we might call now the Bitcoin standard looks like, as far as hardware wallets and how they interact with… I guess the gold standard or Bitcoin standard might be the hardware wallets are all air gapped, they’re all doing something, ideally QR code, and a multi signature, and it’s all in a nice, easy to use GUI. Would that sound roughly right?

Andrew Chow: I guess. It sounds rather impossible.

Stephan Livera: Utopian, if you will.

Andrew Chow: Yeah.

Stephan Livera: But maybe something a little simpler can be done, like even if it’s three of five and say, you’re using Electrum and it’s got some kind of way to interact with HWI. Can we talk a little bit about that? How is the intended model going to be with HWI and some of these other pieces of software, so Electrum for example?

Andrew Chow: HWI was really written with the intended goal of being for Bitcoin Core and being used as an external binary that you can call. Any software that can run another program, which is actually all of them, because all languages have some way to do this, could use HWI to interact with hardware wallets. For example, Wasabi was actually the first project to… I think they’re also the only project, to actually integrate HWI into their software. Wasabi supports hardware wallets now, because they call HWI, which is great. Also found a billion bugs.

Andrew Chow: For something like Electrum, Electrum could actually do it differently, because HWI is also written in Python. Electrum is written in Python too. There is like a library that is provided with HWI that Electrum could use instead of executing it as a separate program. But I’m not sure that they’re going to do that. I highly doubt they would, because they have hardware wallets integrated in already, into their plugins thing, and they’re all pretty well built.

Stephan Livera: I see. But I suppose, if you can’t get that, could it at least get to a point where say you might interact using the PSBT? For example, I might have created that transaction using Bitcoin Core HWI, and created a PSBT, and then handed that to Electrum, because my Electrum might have one of the keys, and it might sign that PSBT.

Andrew Chow: Yeah. The utopian goal is that all clients and hardware wallets use PSBT. Electrum, I would like them to use PSBT, and I think they have an open issue or pull request to implement it, but they haven’t done that yet as far as I know. But it would be great if Electrum supported PSBT and Armory and, I don’t know, Blockchain.info, or like… What else is there? GreenAddress? It would be great if they all supported PSBT, because then we can just pass… If I have to work with someone, doing a multi sig or a CoinJoin with another wallet, we can use PSBT. Or if I’m doing my own multi sig, and using different wallets to avoid one having a critical vulnerability, it’d be great to just have this one thing I just can pass to each one, and not have to worry about incompatibilities.

Andrew Chow: Then if hardware wallets also [inaudible 00:56:57] give a PSBT to the hardware wallet instead of parsing the BSBT, breaking down what it needs and converting that into the special formats for each different hardware wallet. Right now HWI has to not only connect to the hardware wallet and send things over a USB, but it also has to take a PSBT, break it apart and convert it into the messages that the hardware wallets are expecting. If I can just have HWI pass along the PSBT as one data blob, that’d be great, and so much less work.

Andrew Chow: I like how the Coldcard did that, because Coldcard uses PSBT. The Coldcard implementation in HWI is like 20 lines. It’s just send over the thing we got over the command line.

Stephan Livera: Do you have any thoughts just generally on hardware wallets and how they are compatibility with HWI and PSBT? What are your thoughts there, out of the common hardware wallets?

Andrew Chow: In working on HWI, I found that everyone does everything slightly differently. It’s just so slightly different that they’re almost entirely incompatible with each other. It also makes the model that we’re trying to build with Core and HWI for hardware wallets… it actually makes that super kind of difficult to work with. For example, pin entry. On a Ledger, you can enter the pin on the device itself. If you have a Trezor T, you can do the same thing. But if you have a Trezor One, you can’t. You have to plug it into your computer, and then type in the pin on the computer, the scrambled pin, and send it to the Trezor.

Andrew Chow: But HWI isn’t really designed to do that. We want HWI to be just like I call HWI, it doesn’t have to save any state, it doesn’t have to save anything, remember anything, and just given the command line parameters, it knows which device to talk to and what to send to it. But if I have to now deal with oh, when I send this command to the Trezor, it’s going to send something back to me, and expect me to send something back to it, in the same session, which is really annoying to work with, because HWI is stateless. It doesn’t remember that the last time you called HWI, the Trezor wanted a pin.

Andrew Chow: I’ve figured out a work around for it, but it’s still like… I don’t like it. It would be nice if Trezor let you enter the pin on the device. I believe there is an open issue for that. And same thing with the password. All the devices, except the Trezor, have you enter the password on the device. Trezor One lets you do that, and Ledger, if you can find the setting for password, lets you have the password on the device. But Trezor One, same thing with the pin. It calls back to the computer, user has to enter their password, and it sends it to the device.

Andrew Chow: This is kind of annoying. Same thing with a pin. But at least with the password, I can prompt the user upfront and be like, “Give me your password in the command line,” and so when the Trezor asks for it, I can give it what you entered as your password in the command line. I can’t do that for the pin, because the pin is a scrambled thing that the user reads off their display. The Digital BitBox also has that issue with the password.

Andrew Chow: I just found that there’s just these different ways that the hardware wallets go about getting pins and passwords from the user that make trying to do them all in one thing with the same unified API kind of difficult.

Stephan Livera: Yeah, I see. I can understand there’s a lot of difficulty there, trying to get them all to work together. I suppose my next question would be if you’re a listener right now, and you’re thinking, “Okay, I want to get multi signature with different devices, and I want it to be easy in the future,” what devices would make sense for that? For example, as I understand you then, it makes more sense to have the Trezor model T device rather than the Trezor One, if you want that to be part of your multi sig stack, because in the future that might be easier to make it work with the HWI and the PSBT because you can enter the pin on the device.

Andrew Chow: Also, I’ve considered dropping the Trezor One from HWI, so maybe. I don’t know. Ledger actually supports multi sig really well, but it’s… If you consider the discussion we had earlier about change, it actually might not be safe there. You have to be careful. I believe Coldcard probably does it the best, with registering the multi sig with the device itself. In order to use multi sig on it, you have to register the xPubs that are going to be in the multi sig, and it’ll store those locally.

Andrew Chow: Then I guess Trezor T, but also I believe has the same issues with change, as the Ledger. But I haven’t-

Stephan Livera: Oh really? I didn’t know that.

Andrew Chow: None of them actually do the change thing, I think, because this was a new development. It was a new discussion. Now I can’t actually remember what the Trezor does for multi sig. There was one thing that I did for multi sig in HWI that is not the greatest thing and for supporting CoinJoins.

Stephan Livera: Summarizing that then, it’s kind of like if you look at Coldcard, Trezor model T, and potentially Ledger, but the change address problem is still there with Ledger, and potentially with model T?

Andrew Chow: Yeah. I mean, the change address problem I think is always going to be an issue with hardware wallets. Just change detection in general is hard. For some devices, I think you could lie to it, and it just wouldn’t check. You could say, “This address is change,” but maybe it’s not actually change, and it wouldn’t really know. But for the most part, they actually do handle that well, because they require you to send a derivation path, and it’ll try to derive that address from the derivation path and see if it matches. That’s how they detect change now. But multi sig change is very different.

Andrew Chow: Then it gets more complicated with complicated scripts. Going back to the output descriptors thing, what’s going to be expanding upon that is mini script, which lets you have arbitrary scripts. If you have an arbitrary script, it might be harder to determine what’s change and what’s not change.

Stephan Livera: Drawing it to a close then, where does HWI go from here, and what further development work is most needed for HWI?

Andrew Chow: Right now, it’s actually in a pretty done state. It’s mostly just bug fixes, integration problem, bugs that crop up, and some new features maybe. What I was working on earlier today was adding an update firmware command to it, which every device also has its own way of updating firmware, which was really annoying. I was adding this update firmware. That’ll be something new, but it isn’t really needed in signing or anything else. It’s just kind of nice to have. For the most part, HWI is just in pretty much bug fix mode. Wasabi has been reporting a bunch of bugs when the do integration. All right, I’ll fix those.

Andrew Chow: But there isn’t a lot of new functionality to add, nothing that really needs to be done with it. Everything else that we’re doing with HWI is actually on the software side, like the wallet side, with Bitcoin Core, and that’s what a lot of my focus has been, is getting Bitcoin Core ready to be able to interact with HWI seamlessly.

Stephan Livera: In terms of that then, making Bitcoin Core ready to interact with HWI, what’s required?

Andrew Chow: A lot. Recently, I’ve been working on this wallet refactor, and that’s because we need to refactor the wallet in order for HWI to work. Right now, the wallet is kind of a monolith. It has everything in it, like key generation, address generation, transaction tracking, watch-only things. Like anything wallet is inside a class called C wallet, and it’s gigantic. So we’re trying to separate this out, and move key generation into its own thing, and C wallet will now call this thing for key generation. The idea is by separating this, we can actually just define different classes of key generation types. Like we can have keys being generated from the old method of just randomly generating keys, or we have another class that says keys are generated from output descriptors, which is actually the main thing I was trying to get this key generation with output descriptors.

Andrew Chow: Or we can say if you have this class, keys are generated from a hardware wallet, and this class now has all the things to interact with a hardware wallet. So C wallet itself doesn’t need to know about the hardware wallet. It doesn’t need to store any information about hardware wallets. It just knows that if it calls this variable in memory, that thing will do some magic in the background and give it an address or a key. That thing in the background might be calling out to HWI and getting stuff from hardware wallet instead.

Andrew Chow: That’s kind of what I’ve been working on to prepare Bitcoin Core to do hardware wallet stuff. Then Sjors Provoost has been working on the GUI and the actual integration of HWI into Core. He’s been building on top of the PRs that I’ve created to add stuff to the GUI, add stuff to the command line that will do the ideal future of send to address and everything just happens in the background.

Stephan Livera: The promised land.

Andrew Chow: Yeah. It’s a still long way off, I think. The PR that I created to do the separation is like plus 4000 lines of code. People have to review this. It’s going to take a while to review. We probably won’t see hardware wallets for another two or three major versions of Bitcoin Core.

Stephan Livera: So that’s like six monthly releases, right?

Andrew Chow: Yeah. Let’s see. 0.19 will be out in like September, October, I think, maybe November, and then six months from that, March, April-ish… By this time next year, maybe we’ll have hardware wallets in Core.

Stephan Livera: I suppose in your own experience, have you done much multi signature work outside of Bitcoin Core, or have you mostly just used command line interface to do that sort of thing within Bitcoin Core?

Andrew Chow: I’ve pretty much only done command line within Bitcoin Core. I did attempt to do multi sig hardware wallet stuff, multi sig with multiple hardware wallets, and that failed miserably. It was just too difficult to use.

Stephan Livera: That was with Bitcoin Core, yeah?

Andrew Chow: Yeah, it was with Bitcoin Core and HWI, and then I also had a wrapper script around that. I guess it might be possible now. At the time I was doing it, Bitcoin Core did not support. You can’t have multi sig addresses from Bitcoin Core, it won’t give you them. With descriptor wallets, and the refactor I’m working on, it will actually be able to do this. It might be better soon, but when I was working on this side thing with multi sig, it didn’t work. I had to store addresses by myself, and then it had to remember what was used, what was not used, and it was just generally a pain to use, and it didn’t actually work.

Andrew Chow: That’s been the extent of my multi sig experimentation with hardware wallets.

Stephan Livera: It’s good to hear how you’re going with it, and potentially this will become easier over time. I think it will. It’s just a matter of the right pieces being put together. I suppose that’s all. Is there anything else you wanted to cover today, Andrew?

Andrew Chow: No, I don’t think so.

Stephan Livera: Well, that’s great. Look, make sure you let my listeners know where they can find you, where can they follow you online?

Andrew Chow: My handle literally everywhere is achow101, A-C-H-O-W-1-0-1. You can find me on Twitter, GitHub, Bitcoin Stack Exchange. What else? Bitcoin Talk. Occasionally I will stream coding actually. I don’t know how long I’ll keep that up. I started doing it last week. You can also find me on Twitch, achow101. But pretty much anywhere you see achow101, it’s probably me.

Stephan Livera: Fantastic. I’ll put the links in the description as well, so the listeners can find you there.

Stephan Livera: Well look, thank you very much, Andrew. It’s been a really educational discussion for me, and for my listeners, so thank you for the time today.

Andrew Chow: Thanks for having me.

Leave a Reply