Andrew Chow - Hardware Wallets and Bitcoin Core

Hi and welcome to the Stefan Livera podcast focused on Bitcoin and Austrian economics. This is episode ninety nine with Andrew Chow, we're carrying on with the hardware wallet interview series. But first, let me introduce the sponsors of the podcast. Over my years in Bitcoin, I've been impressed with the way Kraken operate. They have a very strong focus on security and acting Ethically in the space under Jesse Powell's leadership, they're one of the longest standing Bitcoin exchanges and they're consistently rated the best with a high quality platform offering the best liquidity in the industry. They've got high trading volume and low fees, with no minimum or hidden fees. Kraken have twenty four seven support and on the institutional and business solution side, they're very popular with institutions too. They're providing the best in class accounting, reconciliation and reporting services for cryptocurrency hedge funds, asset managers and fund administrators. Don't forget there's a Kraken OTC desk. Kraken offer five fiat currencies and they also offer margin and futures trading. To learn more and sign up, go to the Kraken link in the show notes. Next, look into Unchained Capital. These guys are doing Bitcoin financial services and they're also offering a two of three keys multi-signature vault product, so you can use Trezor or Ledger wallets and this helps you maintain control because you still hold two of the keys and you're reducing that single point of failure risk and at the same time, it's protecting You against things like the proverbial five dollar wrench attack, because your keys are distributed. It's really easy to set up as well, and if you do, you get three, three months of access to SafetyNet's Bitcoin Standard Research Bulletin. Don't forget, Unchained also offer Bitcoin collateralized loans, so you can get USD liquidity without selling your bitcoins. They're stored in what's called collaborative custody, so if you wanna learn more and sign up, go to the Unchained Capital link in the show notes. Alright, so carrying on with the hardware wallet interview series with Andrew Chow. Andrew is a Bitcoin Core developer working at Blockstream, and he's working on some really fascinating stuff, hardware wallet interface, H W I, and P S B T, partially signed Bitcoin transactions. So in this episode, we talk about some of the benefits of these technologies and how H W I can enable us to use our hardware wallets directly with Bitcoin Core, which might make it a bit easier, and also P S B T's will be important coming in the future because they will help us with things like coin joins and with more- multi-signature. So definitely try to pay attention. This episode is a little bit technical, so bear with me. I, I do my best to try and clarify for the newbie listeners, but just fair warning. Also, there was a slight audio problem just on Andrew's side, just for five or ten seconds, just warning you. I, I did what I could, but that's all there was. but otherwise, it's a very enjoyable discussion, and I'm sure you guys will learn a lot from it as well. So here's the interview. Andrew, welcome to the show. Thanks for having me. So, Andrew, I know, you've been doing a lot of work recently on things like H W I and P S B T and so on, but, first, let's, hear a little bit about you. I know you've, got a bit of a history with Bitcoin development. Tell us, how did you get into it and how did you get started with development?

Yeah, so, I'm a Bitcoin Core contributor, and I've been working on this, side project

What we're going to be talking about today. I got into developing Bitcoin like three or four years ago, just as something to do in my spare time while I was in high school, and, Got a lot more involved from there.

Yeah.

So now you're working full time as well on Bitcoin?

Yes. So right now I'm, at Blockstream and I'm working full time on Bitcoin Core and other things and other Blockstream things too.

Nice. That's awesome. Let's get into HWA. I'd love to hear a little bit of the story behind it. Why did you develop HWA? So in,

2017, I was an intern at Blockstream, and at that time I had also gotten a hardware wallet. And naturally, you know, have a hardware wallet, I wanna use it. well, I found out that I couldn't use it with Core, and I actually couldn't use it with any Full node that I ran myself. Like, like I eventually did get, Electrum X running on a server and then having Electrum connect to that, but I didn't really like this thing, and, you know, your average person isn't going to be able to do this. So, I talked to my mentor, which was Peter Walla, and we were talking about it and about what, would be required to get hardware wallet support into Core. And basically, we came up with, that we needed some driver, something, some program that had all the drivers to talk to a hardware wallet that Core would call out to or connect to, and send data. And then there, we would have to have some API or data format that Core could send to the drivers in order to communicate. And so, if you see where this is going, PSBT became that data format. That's how we're getting Data from Core to the driver, and HWI became the driver.

Got it. So let's just, sorry, I wanna take it back one step, just to make sure, just for the listeners who aren't as advanced. Let's just, I guess at a more basic level, why is it a good idea to connect your hardware wallet, to, you know, your own full node instead of trusting somebody else's node or giving up the data, giving up your privacy to somebody else's server?

Well, so there's the obvious privacy thing, you know, you're not sending all of your addresses to someone else who can do whatever they want, like tracking or something with that information. when you use your own full node, it's just coming from the local blockchain data that your node has. And there's also, you know, a bit of security when you have a full node, you're not trusting someone else to be providing you the correct transactions and the correct blockchain. Blockchain and all that stuff. So it's just better, security and privacy wise to be running your own full node and hardware wallets are a better way to store your keys and you kinda wanna combine them

Excellent. And so I guess before HWI, what were the main options available to people? I guess the option was either call, use, say, a Trezor or Ledger, the web interface, or you had the option of using Electrum public server, which is again calling out to all those public servers and giving off your own data. And then you mentioned there Electrum X, and I think the other ones are Electrum personal server by Chris Belcher, and also, I think there's Electrum Rust server. Do you wanna just talk a little bit about some of those? So at the

time I started working on this, the, as far as I can tell, the only third party software that really had hardware wallet support, like decently, was Electrum. And Electrum is a SPV wallet, it's not a full node, so it calls out to these Electrum servers which provide its data. And it's like kind of decentralized, the servers aren't run by Electrum, they're run by anyone, And the Electrum wallet will connect to multiple of these servers, but it's still SPV, they can attack you, they can be, DDoS like many have been, and So if you, if you wanna have better privacy and security with Electrum, you should run your own Electrum server. the common software for that is Electrum X. And so that's what I was running, when I did my original setup with Electrum and Electrum X. at the time, Electrum Personal Server didn't exist yet. I don't- I don't think it existed yet, Belcher hadn't written that. And the Rusts, the Electrum Rust server, I actually didn't hear about until like several months later. So I, I don't know anything about that one. But

all I remember is, is, you know, Google elect-setting up your own Electrum server, and it was just run Electrum X, 'cause the original Electrum server was too Old and non-performant that it couldn't actually catch up to the blockchain.

Oh, wow. I didn't know that actually. Yeah. So basically, most of the ones that you are calling out to are Electrum X servers, whether that's your own personal one or that's like, you know, blockchain spy companies who are running these, Yeah, most people

run Electrum X. It's like the, it's the most widely used and most supported, and it's pretty easy to set up. So that's what I was using for, for the hardware wallet stuff. And, you Different software, and you have to have a server and stuff. It's kind of annoying.

Yeah, I see. Yeah, it's a little bit finicky. And so the idea then was HWI would be this, by d-- on working by default, if it's, you know, if it's eventually, if it does eventually get merged into Bitcoin Core, then it would be a very more-- it would be a more simple process for the user to just, okay, double-click, install, and sort of off you go. Yeah. Is

that, is that, is that the hope? So When you do anything that requires a hardware wallet, Core will call out to HWI and HWI will do something and return results back to Core, and it all happens in background, the user doesn't notice. It's, it would be the same as if we ju-just implemented the drivers directly into Core itself, which we don't wanna do for security reasons. Hardware wallets are all over USB and the USB driver stack is really big and is just a major, attack surface. So we don't wanna have that in core itself in case, you know, someone isn't using Hardware wallets, they don't need it, but there's this huge vulnerability it somewhere in the USB stack, and now they're vulnerable to it. We don't want that. by having this separate program, we completely avoid this issue, 'cause it's only active When you're act-actively using your hardware wallet.

That's essentially what Hwi enables. It enables people to use their Trezor or their Ledger or whatever hardware wallet or cold card with, directly with Bitcoin Core. So I think we should unpack that a little bit and start to talk about some of the pieces that make Hwi possible. Can you give us a bit of an overview on that?

So there's three-ish components. There's the like actual drivers and talking to the device itself, then there's the API wrapper around that. And then we have PSBT. So the, the drivers part, that's all like, for the most part, that's actually provided by the hardware wallet vendors. They all have Python libraries, and HWI is written in Python, so I just pulled in their Python code and now I can talk to all their devices without having to really, truly dive deep into everything that I have to do to talk to the hardware wallet, 'cause if I had to do that, I probably wouldn't have done it. It's just way too much work. And this also lets me support a bunch of different things, you know, we can just add another driver in later for some other device. And so then the, the actual HWI part is the wrapper around that, it's just the calling the drivers when necessary, calling the correct APIs and taking in The PSBT or other stuff from the command line and converting it into the right things that, every device specific driver can use. 'Cause they're all different, they all have their own APIs and stuff. HWY is just providing a single unified API to connect to all the devices.

And then we have one API to rule them all.

Yeah. The, the nice thing actually is, because HWI will be a separate program, if someone doesn't, if HWI doesn't support a hardware wallet, the vendor can actually just go and write their own version of HWI with support for their hardware wallet, have the same API, and now people can drop in, replace that, and it'll, it'll work with that new hardware wallet. And, and then I won't have to do all the work. Very nice. Yeah. And so the last layer is PSPT, which is just to hold all the transaction data itself, and be able to send it from Core to Hwi and even to other clients, 'cause it's- Designed to be fairly generic. So it has all of the information, or it will have all of the information that you would need to sign and then create the final transaction that will be broadcast to the network.

Got it. So let's, let's unpack PSBT a little bit. So for the listeners, that is partially signed Bitcoin transactions. So Andrew, can you give us some overview on that and why was that necessary?

So this is necessary because when you want to sign a transaction, you need a lot more than just the transaction that you're signing. so you need, for example, for a transaction that spends like a P2WSIG, like multisig, you need to know the script that, that you're gonna put in that final transaction, and if you don't already know it, the transaction that you're signing isn't gonna have it for you. So for hardware wallets, they don't know what this is, you have to tell it to them. And like same with offline signers, they have to be told what the script that they're going to be signing is. So through PSBT we can package, you know, this is the transaction we're signing and here's a script that you need in order to sign it. Package them all together so it's now one large data blob that we can send off instead of having to have multiple pieces that need to be sent. So if you, if you look at, Bitcoin Core's SignRawTransaction thing, it's If Bitcoin Core doesn't know the scripts, the API is awful to use. It's JSON and then you have to like JSON itself is annoying, to make sure you get all the brackets and quotes correct, and it's really easy to screw up. And it's also not super universal. The raw transaction format itself isn't the same in Electrum or like Armory. All three of them have different Transaction formats for stuff that aren't signed yet, for transactions that haven't been signed yet, and they're all incompatible with each other. So if you want to also use, like, say, do a multisig between Core and Electrum because you're paranoid, you're gonna have a hard time making sure that both Core and Electrum can sign the transaction because they don't share the same format. And the idea behind, like- Having a bip behind PSBT is that people will actually use it, and now we can have these different clients be able to sign transactions with each other.

Got it. So look, let me just try and break a little bit of that down and make sure my understanding is correct as well. So What we're talking about here is hardware wallets are in some sense, you know, they're quote-unquote dumb. They don't, they're not connected to the network, so they don't know some of what's going on. And part of the complexity of what's going on is when you spend a Bitcoin transaction, every transaction has certain inputs that go into the outputs, and in order to unlock, if you will, or be able to spend those inputs, you need to satisfy the encumbrance or the locking script placed upon that input. And that's That's what you're getting out there, as I understand, with the signature and basically the device which holds the private keys to be able to sign the, to sign that, it doesn't know what script type, and that's what you're getting out there that this HWI, and sorry, the PSBT rather, is a way, is like a format for different Bitcoin software to interact with the hardware wallet and tell the hardware wallet, hey, I ne- hey, my Trezor, I need you to sign X, Y, and Z And it is this type of s-script, and I need this kind of, you know, these are the addresses that, you know, I need you to, spend into. Does that, is that, more or less right? Yeah.

Yeah. So the, and, hardware wallets actually need to know even more than just the scripts. like, so the script is actually, it's necessary in signing 'cause it's signed, it is actually s-signed over, it's part of the message that is signed. but also a hardware wallet needs So what to sign with? Because it doesn't, the hardware wallet isn't gonna store every single possible key it could generate. That would be something on an order of like billions or trillions of keys, and they just don't have the memory for that. instead You need to tell it, like, sign with the key at the derivation path M44 something something. and so PSBT has another field that says, you know, this pub key is derived at this derivation path. So now HWI or something else can go tell the hardware wallet, "Hey, go sign with this derivation path." So it has that too, which is, like, you know, you don't really need this in order to sign, but hardware wallets really do need this in order to sign. It's like other, other software, like Core actually just completely ignores this field, because it stores keys locally and it'll just look them up if it, if it can. It doesn't derive keys on, as it signs.

Okay, got it. So let's try and break this down a little bit as well. So this is getting into this concept of derivation paths. Now, my understanding on this, and again, correct me if I'm explaining any of this wrong, it's a little bit technical, so I might fluff this part, but my understanding here is You got BIP thirty-two, that is known as hierarchical deterministic wallets, and so the idea is you take a seed, and that seed is used to generate what we call a master private key, right? Like your xPub, and that master private key can in turn be used to generate child keys, and they are like a child private key to, underneath, sitting in that tree underneath your master private key, and then under that child private key, you have leaves. And those Leafs are what can be used to make addresses, right?

not quite. So you have the master private key, and that can generate child private keys. But then the child private key is also another master private key, and so it can generate its own child private keys, and this just goes ad infinitum. you just keep going down as far as you want, and so each level is like a different step in the derivation path. So if you see like M slash forty-four, that means that master key M, and you derive the forty-fourth child And that's the key at, at M slash forty-four, and you have like another slash zero. Now, under the forty-fourth child key, you now derive the zero- Like its zeroth child key and just keep going down like that, that's how the derivation paths work.

Got it, okay. The indexes are,

the indexes are a bit weird and this is just part of the derivation algorithm, it only makes sense if you're reading it.

Right, I see. So could you help break this down, that down for us a little bit? So you might have your master private key or a master public key, right? And if you don't wanna give off the private key for it, and then, so you kind of, you need the xPub. And the derivation path to know, you know, because you might have split accounts under, sitting underneath that master key. And then so is that where the, like you said, M slash forty-four slash zero slash zero? Can, can you just help outline what are those numbers and what do they mean? So it's like that's one of them is the account number and one of them is the--

Well, so that's like part of bip forty-four, which I don't actually totally remember. and that's kind of a, it's kind of a someone defined these things, but they don't Part of hierarchical, whatever, Bip thirty two itself.

Yeah, got it.

Bip thirty two is just how you go from a master- Key to a child key, and then, just for simplicity, instead of going directly, like, you can only have a, a single line of keys, it's a, it's just a tree, and we stick an index somewhere in the derivation algorithm, and that's what we mean by like zeroth or first, second, third, forty-fourth key, that kind of thing. So there's just the term I, and that is The index, and you just re-replace that with whatever index you're using. So BIP 44 defines derivation paths in a specific pattern, and a lot of wallets use this, but it doesn't- Totally makes sense. so at a, I believe it goes like four or five levels, So you have the first one, you M slash forty-four. So the forty-four is the purpose, and that's some-something that's predefined. there's a list on Satoshi Labs, GitHub somewhere that just lists out a ton of different purposes. and that's what this first index under the BIP forty-four scheme should be. And then there's Account, something else, maybe that's backwards, which are just used for like internally, you can like change those and get a different account, like a different derivation path tree, and get different addresses. And then the, at the end there is change, you set a zero if you want like that derivation path to be for all your receiving in-addresses and a one for your change-addresses. that's just to separate, separate out. So you don't have change and non-change keys mixed together. And then the last one's the index for the child keys. But you don't have to follow bit 44, you can do something else. Bitcoin Core notably doesn't follow bit 44. okay. There is actually a security risk. BIP 32 defines, there's also a thing called hardened derivation and unhardened derivation. So hardened derivation means that you can only derive the keys if you have the private key, the master private key. If you don't have the master private key, you're not gonna get any child keys. Unhardened derivation means that you can derive the child keys from the master public key. You can have the xPub and then you can derive child keys from that. You don't need to have the private key. So Bip 44 does a mix of this, the first three, the purpose, account, and that other thing, are all hardened derivation, and then the last one, the change and, and the index, are unhardened. So this lets, like, This is good for hardware wallets. So hardware wallet gives you an xPub, and then you can load that into your software and now derive change and non-change and all the indexes from there without needing to know the private keys. Your software just knows the xPub. That's great, but, The caveat is that if you have a child key derived with unhardened derivation and you have its private key and you know the parent xPub, then you can also derive the parent private key.

Yeah, big security risk, right? Which is a

huge problem. if like you're storing your private keys in a way that they could be exported. So Bitcoin Core doesn't do this at all because we have dump privkey that just lets you export a private key We don't want people to export a private key and then accidentally leak their parent private key, and now someone can steal all their coins. So, so Bitcoin Core uses entirely hardened derivation, at least right now.

I see. So let me just break that down again for the listeners. So, listeners, check out the earlier episode with Michael Flaxman where we explained a little bit of-- I think Michael explained a little bit in that episode around this risk that if somebody gets your child- Private key and they've got your master public key, and as Andrew, as you've just explained, that allows them to now basically re-- kind of back-- they can back out the master private key, meaning they could just steal all your coins. Whereas, as Andrew, you're pointing out, if the-- if you're using hardened derivation, that's not possible, and that is the approach that Bitcoin Core, as a group, have taken as a way to be more conservative and more, I guess, security conscious, whereas perhaps other Bitcoin developers- out there in the wild have used this method for usability reasons, I might say, to make it like easy for you to kind of hook up your hardware wallet and so on and generate addresses without having it connected in and that sort of thing.

Yeah, and I mean, having the, having an xPub that you can derive keys from is pretty useful if you- Have your, I mean, just like on, on a normal software wallet, if you encrypt your private keys and you run out of, the pre-generated keypool on Bitcoin Core, when that happens, you have to unlock your wallet, let it generate more keys before you can like get new addresses. But if you're using an xPub You can just keep getting more and more addresses even when your wallet's fully encrypted.

A quick word for a sponsor, Manning Publications. As you might know, they are the publisher of Grokking Bitcoin by Calle Rosenbaum, and we've got a special offer for my listeners: 40% off any book or video product by using the code Laverre. Part of being a Bitcoiner is about continual commitment to learning and improvement, and there are many titles such as Real World Cryptography, Linux in Action. Learn Git in a Month of Lunches. There's a title called Math for Programmers where you can learn 3D graphics, machine learning, and simulations with Python. It's got over two hundred exercises and mini projects with helpful graphics. The link for Manning is in the show notes or go to manning dot com and use code LIVERA for forty percent off. Back to the interview. And let's now talk a little bit. I've seen, some of the-- I was obviously a bit of research for this episode, I was looking on Bitcoin Stack Exchange and I was looking at some of Vella gave as well, and, he was talking about this idea of output descriptors, and so as I understand, part of it w-is there was a little bit of I think owing to that idea you were got talking about there of not having like a standard way of doing these things, people came up with things called the equ-- well, on top of the xPub, they had the yPub and the zPub to generate, say, SegWit or native SegWit, addresses. but I think Peter was getting at this idea that output descriptors are a better less ambiguous way to describe those or distinguish, those different script types. Can you touch on that a little bit? So

Electrum and Treasure, developed or added to bit thirty two, xPub, the yPub and zPub, and they basically said that xPub means that you're deriving that any pub keys you generate from the xPub is going to be for a legacy non-segwit address, and then anything you derive, any pub keys you derive from a yPub will be for P2SH wrapped, I think, and Zpub is B32. but this is kind of not what B32 was meant to do. Xpub was never really supposed to say that It was never supposed to define an address type, it's just for an extended public key that you can derive public keys out of them and use those public keys to do something else. You can put them in scripts, multisig, whatever. so Peter has been working on mini-script and output descriptors, and so what output descriptors do are, is that they Actually define a script that you want to use in a human-readable way. So if you wanted to use public key is derived from an xPub in batch 32. You could put that inside an output descriptor, and then the descriptor would have like, would have WPKH at the front of it, and that would signal that, you know, everything inside this, any public key, is should be used in a BIP32 output, and so that means any public key in derived from this xPub should have a BIP32 out, address. And you can have different things in the front, and that just indicates that, from, from the same xPub you could have different kinds of addresses. Instead of, so instead of having this thing where we're just defining new magic numbers and saying that this magic number means, public keys derived should be this address type, we now have some human readable way, and that's also, that says what address to use, and it's also flexible in the future if we define some new address type, we can just add a new thing to the descriptors without changing how keys. Are derived.

Got it. So let me just quickly re-summarize that. So essentially it's saying instead of using like xPub, yPub, zPub, like we're gonna run out of letters eventually, and it's probably not the best, it's not the most unambiguous way, whereas a more kind of specific way to, talk about these is to have a specific format. And as I can see from Peter's answer on Stack Exchange, he lists out a couple examples. So it might say P K H, in brackets, xPub dot dot,

Which is the derivation path, and that might explain, okay, this is a bit forty-four address from a particular xPub, which is the pay-to-public-key-hash type, all right? And it might specify exactly what type, and so that way, when wallets and different devices are passing information to each other, they can say, "Hey, this is the output descriptor, this is the unambiguous way to derive the exact address, let's call it." Yeah.

And it's also- Human readable. Like, I can look at that and know exactly what it's saying. If I look at a Ypub, I didn't memorize what Ypub is, so I actually don't know what Ypub does, but I can see, you know, s h w p k h, I know that's ScriptHash ins-with WitnessPubKeyHash inside of it, because these are common endings that we use in Bitcoin, like p2sh, p2wpkh. These are all things that we see working on wallets.

Right. And that- That's just referring to the different script types, just for the more beginner level listeners. so let's bring it back to, you know, the PSBT where it kind of aligns back into PSBT. As I understand, there are different roles within PSBT. You've got creator, updater, signer, finalizer, and extractor. Can you tell us a little bit about what each of those roles are?

Yeah, so ideally these are supposed to be, self-explanatory. So creator creates the transaction, it chooses what the in- it decides what the inputs and the outputs of the transaction are going to be, like, you know, the TXIDs and the V out indexes that we wanna spend and which outputs and their, you know, their scripts and the amounts that we want to create in this transaction. And it wraps that up in all the PSBT Overhead basically, 'cause, when the creator makes a pSPT, it's just gonna be, you know, a raw transaction like you would see on the network without, just without anything in the script sig, in the script sig, so there's no signatures, no, no input scripts or any of that. and that's just gonna be wrapped with the pSPT header and a couple other bytes that are needed for pSPTS. And so creator creates a transaction and sends it off to an updater. So the updater will inspect the transaction the creator made and add information about, the inputs that are being spent. So it all s- It'll add like the, the UTXO, so for SegWit transactions or SegWit inputs, it'll take the Just the output from the previous transaction itself, it'll just be the amount and the script. for non-segwit, it has to include the entire previous transaction. And we need this because in order to sign we need to know what the script was and the script of the output that we're spending, and we sometimes need to know the amount, like SegWit actually needs to know the amount. For, I don't remember reasons, but,

and then the updater will also add other information if it has it, if it knows that The output that's being spent was P2SH, it'll add the redeem script for the P2SH. If it knows that it was a SegWit witness script hash, then it'll add the witness script. If it knows any of the pub keys, in the output or in the redeem script or witness script, it'll add the pub keys and their derivation paths if it knows them. and then, you know, for future, there are future things that it could also add. So the updater adds all this to the inputs, it also does it to the outputs, if it knows anything about the outputs, it'll add the redeem script and the public keys to the outputs.

But on, and all this is really only if the updater knows it. And so once the updater's done, it might send it off to another updater who will add all the information that it knows, or it'll send it off to a signer. The signer will look at the transaction and because the updater had added the UTXO and the scripts, the signer can be completely offline. It doesn't, it doesn't need to know anything about the blockchain, it doesn't need to be connected to the internet. You know, it could be a hardware wallet or it could be a computer locked in the safe fifty feet underground.

It, it, it can be completely air-gapped and offline, and this is like really what PSBT was meant to do. So the signer can now look at the transaction, it can see, you know, if it needs to sign, say, input zero, it looks at the data that the updater added to input zero and it sees there's a UTXO, so now it knows The output script which it needs to sign, for signing usually. and there's a redeem script, and now there's a redeem script. There's a witness script, and also there's a witness script. And then the pub keys are also listed there, so it can go generate the private keys if it has to, or just pull up the private keys. And it knows everything that it needs to do to sign. So it'll, you know, check the, check the transaction, make sure that the user likes it, like it'll look at the, 'cause we have all the U Those, we can now, we know what the transaction fee is going to be, which is useful, like the user can be like, "Oh, that's a, that transaction fee's too high, I'm not gonna sign this transaction, I wanna lower transaction fee," or something like that. Or maybe, you know, transaction fee's too low, it's never gonna get accepted. So The signer, you know, makes sure that the user likes the transaction, that the money's going where it's supposed to go, the user's receiving their change if they are supposed to get change, and the fee's correct. And then it makes a signature, attaches it to the PSBT, and does that for every input that it can sign for. So once the signer is done, you know, it can send it to another signer if someone else is involved. Or it can send it to a finalizer, and the finalizer does, it does a conversion. So the, the PSBT itself just has a bunch of records in it, but In the final transaction, you don't have just a bunch of records, you have a script in the input. So the finalizer produces that script. It, you know, it takes the signatures the signers created, it takes the scripts that are already there, and it combines them all together in order to create, the script sig or the script witness and sticks that in back into the PSBT so that Later, these can be put together with the original unsigned transaction to be broadcast. And so that last step is what the extractor does. It takes all the script sigs and script witnesses, the finalizer. created, sticks them into the unsigned transaction that's at the top of the PSBT and sends it off to the network. There's actually one more role, and that's the combiner. so in- In between updater, signer, and finalizer, you can also send it off to a combiner, which will just take multiple PSBTs that have different data in the input and output and just combine them all. It throws all the records together as long as the global unsigned transaction is still the same.

okay, right. One thing, it might be good to just talk through an example flow of- A PSBT then. So let's say just a single signature PSBT, and you've got an offline wallet, like a hardware wallet, and, watching only wallet. Could you just talk through what is the flow there? Like, would you, you-- would you create the transaction using Bitcoin Core, or like, how, how would you-- Is this like a CLI thing or could we-- So actually,

I'll go through an example, like, like this is how, I use hardware wallets now with Core. So Bitcoin Core acts As the creator, so the hardware wallet is the signer, and Bitcoin Core is everything else. So first, I would use Bitcoin Core as the creator and create a transaction. I choose my inputs, choose my outputs, and produce a PSBT. That's what the creator does. Then Bitcoin Core will take that and will update it. It will pull from its blockchain From its access to the UTXO set and the blockchain, fill in all the,

UTXO information for all the inputs that I've chosen. and then it'll also add from the wallet, the pub key information and the redeem script and witness script. And this is actually all done together as one command, Wallet create funded PSPT. It just does everything in the same time, so all you have to do as a user is say which inputs and which outputs, and maybe not even the inputs, 'cause it will still coin selection for you if you don't give enough money,

So it makes this PSBT with all the updated information. I copy that, take that to HWI, and just give it the sign TX command and the PSBT. click all the buttons on my hardware wallet that says like, "Do you, are you sure you want to send to this address with this amount? Is the transaction fee correct?" You know, click through all the buttons if I like the transaction and nothing had changed it, and HWI gives me another PSBT that has all the signatures in it now. And now I take this back to Bitcoin Core, what I do, finalize PSBT, and this will-- this is the finalizer step. It makes all the script sigs and the script witnesses, and, it's also the extractor if Bitcoin Core finds that, all the scripts, all the script witnesses are there, and when it puts it into the transaction, the transaction is valid. It gives me the final transaction that I can send to the network, so I can copy that and go to send raw transaction and broadcast it. So right now, this is actually how you use HWI with Core because we haven't fully integrated it yet, and it's just a command line thing. But it's basically like, four commands, I think. Yeah. Says "wallet create funded p s p t, then sign t x, then finalize p s p t and Send raw transaction.

Got it. And I suppose, is there hope then, so right now it is all CLI, character interface, is there a hope that, or a plan that this would then become a GUI?

Yeah. well, the hope is that it'll all happen in the background. So if I, on the CLI, if I just did "send to address" and I had a hardware wallet that was plugged in, the thing would show up on my hardware wallet and I could click all the buttons there, and then it would magically all happen in the background with just one command. And then When you click send, your hardware wallet is plugged in, a thing will show up, and then it just magically all happens in the background. So that's what we're, that's what we're working towards, not quite there yet.

Yeah, right. And I suppose the other aspect to layer on there is multiple devices and also airgapping. So can we talk to some of that? So maybe, let's talk to the airgapping part of it first. So let's say you wanted to export the unsigned transaction onto a micro SD card or i-ideally a QR code Could you talk to that process a little bit and what that might look like?

yeah, so the, the transaction is, just a base sixty-four string or the PSBT, and that contains all the PSBT information. Sometimes it's also really big, but you just copy the string. You could put it like in a text file or, you could QR encode it, but QRs, QR encoders have a size limit. So you might run into problems there, but it's just a string, you know, put in a text file, something like that, put on a flash drive, micro SD, whatever, take it out to your second computer or other device that has the private keys and give it this string back, you know, if that's QR codes, yeah, it'll just scan the QR codes and the string will be there, or you just copied off your text file from the flash drive, and then you do, do the signing off the, from the PSBT there.

Great. And let's talk now about, so talking through that example, but in this case, let's talk about multi-signature. So let's say you've got a two of three hardware wallet set up or something like that, and, you know, basically we do that same PSBT process as you spoke about, would the main change be that instead now there would just be two signers instead of one? Is that the main change?

Yeah, pretty much.

Got it. Yeah.

You, there could be two updaters also? You could have a, you know, if-- Never mind, actually that's a bad example. That works for coinjoins, not multisig. Oh, okay,

yeah. That's another good example of PSBT as well. In, in

coinjoins, you would have multiple updaters, and everyone would update their own transaction, and it would all be combined before they all sign it again. But in- for multi-sig, you'd, you'd probably have like something where one wal-- one person has their wallet set up to be watching for all the transactions, and they act as the creator and updater, and they send out the PSBT to the other signers, and the other signers without having to Be watching that multisig, address, or even like, like they'll, or even have that multisig address in their wallet, they can just sign the PSBT and send it back to, the creator. and it's, I mean, the flow is pretty much the same as the air gap one, except instead of giving it to your second computer, you're just giving it to another person. That's about it. coin joins are also pretty similar.

Yeah, that's interesting, because I guess, obviously there's so many different combinations, but I'm just trying to, in some sense, talk about some of the key ones that people might use or they might think about, and I guess the key scenarios here people would be thinking about are multi-signature of their own coins or maybe They might want to use it as a control system, right? They might want to say, "Have a three or five," and, you know, they might want to make it so that the CFO and the head of finance have a key each or something like that, and, you know, they might wanna split the controls up that way and use the PSBT as a way to help pass around this un-incom- or uncompleted transaction until it's ready to be signed and, you know, fully broadcast out to the network.

Yeah, and, and the, the thing with all these roles and The way that PSBT is designed is that for all these cases, it's almost the same workflow. It's just passing it from one person to the other, and they're doing almost the exact same thing as if it was just their own money and their own transaction, because the software Is supposed to handle everything in the background.

Yeah, and now let's talk a little bit about some of the problems that we have seen in terms of ma-making multisig work nicely, and even for, you know, just hardware devices as well. So there is one of the problems or one attack that, Michael Flaxman mentioned in my previous interview, which was this idea that- At the time you are signing on your hardware device, you, and you're doing, say, a multi-signature transaction, you may not be aware of if your device is signing against your own other devices. So a quick example might be in a two-of-three multi-signature setup, you might be faced with the decision to hit accept or reject on your physical device, but you might not know that unbeknownst to you, that transaction might actually-- the other co-signers might be your enemy, right? A hacker rather than your own, multisig hardware wallet. Yeah. Can you talk a little bit about that problem?

So PSBT has, in the outputs, you can also include the scripts, the redeem script and the witness script. So Ideally, you would also just look at that and see that, you know, these are the public keys that you're expecting to see and everything's good. there is, but there is a problem with like xPub's, when you're doing a multi-sig where every address is new because you're deriving new addresses from xPubs that people have provided. And This was the proposed extension that we added recently to PSBT that was that we would include an xPub at the top of the PSBT, and then when you see a public key, you can go and you can try to derive That public key from the xpub in the PSBT, and now you will know whether,

the, it's the same that xpub is, was used to derive that public key or not, and you can see Who, which xpubs are contributing to the outputs? but the, the other problem is actually with change detection. so, you know, most users don't actually really know about change, and they get scared when they see it, they-- 'cause they see their bitcoins moving to some other address. and if they see this change address being displayed to them by their hardware wallet They probably won't know what it is or, it might just reject the transaction even though it's perfectly safe. So the, the problem with change is that if we're not showing The address to the user for them to verify themselves that this is correct, then the hardware wallet needs to do this check itself. And decide whether to hide it or to show it. The, the hardware has to decide if it's safe for the user. And so that's actually, that was the primary motive, primary motivation for Having the xPub in, the PSBT so the hardware wallet can say, I've signed this input that's multisig and the three pub, the pub keys in this input. were derived from these xpubs. Now, when I go to the outputs and I see, okay, this output has pub keys that are derived from those same xpubs. So, and it has the same threshold number. So now I can assume that this is the same, this is our change, because we have the same people involved and the same number of people need to sign it. So this is reasonably change, and that's how we can, if one of the Signers had replaced it, then this wouldn't hold true, and we decided that this isn't change, we're going to tell the user that this is an output instead of hiding it away as change.

Right. So let me just summarize that again for the listeners that might be a bit technical. I'm not sure I fully grasped it myself, but, but I'm gonna try. so The, when every tran- when every transaction is crafted, think of it like your UTXO is being kind of melted down and recast, and part of that is you're receiving back some as change. And what you're talking about there is that idea that your hardware wallet has to make that call for the user, because many users don't understand that concept themselves. And so what you're saying is that if by using, by seeing what are the xPubs going into this transaction, we could try to figure out if this is change or not. And then that's when it would be flagged to the user whether, so that the user can know at the time that they're pressing yes or no on their hardware wallet that, yes, this is actually-- this is either coming back to me as change 'cause it's my own money, or it's actually a payment to somebody else. Somebody else is gonna get the output. Is that, is that right?

Yes. And, this also happens to only really matter if the threshold, like the number of signers, the number of people that have to sign in this multisig, is less than is less than or equal to half of the total number of people in the transaction. So like, if I have, if you have a two of four, this would be a problem, but if you had a three of four, it actually isn't really that big of a problem. And that's because you can actually have a, like a, a signer policy of, you know, if a tran- an output is change, if my

if one of my keys is in it, I don't really care about who else is signing it, as long as my key is there. And the threshold is, I don't think the threshold actually matters that much, because as long as I'm there and I have control, like I would be able to sign this in the future. Then, from that standpoint, is it safe? If, so if you had like a three of four and someone had replaced one of the keys with a second one of their own and all three signers are following the same policy? The third guy, let me think.

Yeah, so the third guy would be, The guy whose key got replaced wouldn't see his key in there, and so now he knows that this isn't change, and so he would be the one that refuses to sign, and now the transaction won't go through because that guy didn't sign But if, this was, if it was a two of four instead, then I could sign, seeing my key there, and the attacker could sign, and he's just replaced one of the other keys, but they're not involved anymore. It's just gonna be me and him, and I've unknowingly sent away coins to an attacker. So this, this xPub thing really only matters when The, the threshold is less than or equal to half of the number of signers.

yeah, okay, I get you. Yeah. Because basically it, it depends on how the attacker tries to go about that attack and which keys they switch out and what is the quorum required to make it happen. I suppose part of it though is most of the time when people talk about multisig, they're usually doing a setup where it's majority, right? It's two of three, it's three of five, four of seven, et cetera.

Yeah. Yeah. So usually,

Probably isn't even needed. Like, you can just go by this selfish, policy that I've described. it's, yeah, from-- When I was talking to Peter about this, he was like, "Yeah, you can do this, like, if you just act really selfishly in this way, it doesn't matter."

So the listeners act selfishly when it comes to multi-seg guys.

so I guess let's talk a little bit about what the goal Standard or what we might call now the Bitcoin standard look, looks like as far as, kind of hardware wallets and how they interact with, you know, so the, I, I guess the gold, standard or Bitcoin standard might be The hardware wallets are all airgapped, they're all doing something ideally QR code and, and a multi-signature, and it's all in a nice, easy-to-use GUI. Is that kind of roughly what-- Would that sound roughly right?

I, I guess, sounds rather,

impossible.

Hahahahah, utopian, if you will. Yeah. But maybe something, a little simpler can be done, like even if it's three or five, and say, you know, you're using Electrum and it's got some kind of way to interact with Hwi. So can we talk a little bit about that? Like, how do-- how, how is the intended model gonna be with Hwi? Some of these other pieces of software, so HWI, so Electrum, for example.

Yeah. So HWI, was really written in, with the intended goal of being for Bitcoin Core and being used as, external binary that you can call. so any software that can run another program, which is actually all of them, because All languages have some way to do this. could use HWI to interact with hardware wallets. For example, Wasabi was actually the first project to, I think they're also the only project, to actually integrate HWI into their software. So Wasabi supports hardware wallets now because they call HWI, which is great, also found a billion bugs.

for something like Electrum, Electrum could actually do it differently because Hwi is also written in Python. Electrum is written in Python, too, so there is like a, a library that is provided with Hwi, that Electrum could use instead of executing it as

a separate program. but I'm not sure that they're gonna do that. I, I highly doubt they would because they already have all the hard-- they have hardware wallets integrated in already and into their plugins thing, and they're all pretty- Well built.

Yeah, I see. But I suppose if you can't get that, could it at least get to a point where, say, you might interact using the PSBT? So for example, I might have generated, I might have created that transaction using Bitcoin Core HWI and created a PSBT and then handed that to Electrum because Electrum have, might have, my Electrum might have one of the keys and it might sign that PSBT.

Yeah. So the- The, utopian goal is that all clients and hardware wallets use PSBT. So, so Electrum I would like them to use pSPV and I think they have an open issue or pull request to implement it, but they haven't done that yet as far as I know. But it'd be great if Electrum supported pSPV and Armory and, I don't know, Blockchain.info. Or, or like, what else is there? Green address. it'd be great if they all supported PSBT, because then we can just pass, if I have to work with someone doing a multisig or a coinjoin with another wallet, we can use PSBT, or if I'm doing like my own multisig and using different wallets to avoid one having a critical vulnerability. It'd be great to just have this one thing I can just pass to each one and not have to worry about incompatibilities. And then if hardware wallets also-

Just give a PSBT to the hardware wallet instead of parsing the PSBT, breaking down what it needs, and converting that into the special formats for hardware wallet, for each different hardware wallet. It's like, right now Hwi has to not only, connect to the hardware wallet and send things to it over USB, but it also has to take a PSBT Break it apart and convert it into the messages that the hardware wallets are expecting. If I can just have HWI pass along the PSBT as one data blob, that'd be great, so much less work. And I like how the Coldcard did that, 'cause Coldcard uses PSPT. So Coldcard, the Coldcard implementation in Hwi is like Twenty lines, let's just send over the thing we got over the command line.

Do you have any thoughts just generally on hardware wallets and how they are compatibility with H/WI and PSBT? What are your thoughts there out of the common hardware wallets?

In, in working on H/WI, I found that Everyone does everything slightly differently and it's just so slightly different that they're almost entirely incompatible with each other.

and it also makes this, the, the model that we're trying to build with Core and HWI for hardware wallets actually makes that super kind of difficult to, to work with. So, for example- PIN entry. On a ledger, you can enter the PIN on the device itself, right? And if you have a Trezor T, you can do the same thing. But if you have a Trezor One, you can't. You have to plug it into your computer and then type in the PIN on the computer, the scrambled PIN, and send it to the Trezor. But HWI isn't really designed to do that. we don't, we want HWI to be just like, I call HWI, it doesn't have to save any state, it doesn't have to save anything, remember anything, and just given the command line parameters, it knows which device to talk to and what to send to it. But If I have to now deal with, oh, when I send this command to the trezor, it's gonna send something back to me and expect me to send Something back to it in the same like session, which is really annoying to work with because HWI is stateless, it doesn't remember that the last time you called HWI, there was, the, the treasurer wanted a pen. And now, I, I've figured out a workaround for it, but it's still like, I don't like it. and it would be nice if, if Trezor let you enter the PIN on the device. I believe there is an open issue for that. And same thing with the pass-password. all the devices except the Trezor require, have you enter the password on the device. Trezor One lets you do that, and Ledger, if you can find the setting for password, lets you have the password on the device. But Trezor, same thing, Trezor One, same thing with the PIN, you have to-- it calls back to the computer, you're asked to enter their password, and it sends it to the device. And this is- Kind of annoying, you know, same thing with a PIN, but at least, with the password, I can prompt the user up front and be like Give me your password in the command line, and so when the treasurer asks for it, I can give it what you entered as your password in the command line. I can't do that for the PIN because the PIN is a scrambled thing that the user reads off their display. And, the digital bit box also has that. issue with the password. But I just found that this is that, there's just these different ways that the hardware wallets go about getting like pins and passwords from the user that make trying to do them all in one thing with the same unified API kind of difficult.

Yeah, I see, I can understand there's a lot of difficulty there trying to get them all to work together. I suppose my next question would be, if you're a listener right now and you're thinking, okay, I wanna get multi-signature with, you know, different devices and I want it to be easy in the future, what devices would sort of make sense for that? So for example, as I understand you, then it makes more sense to have the Trezor Model T device rather than the Trezor One if you want that to be part of your multi-sig stack Because you can enter the pin on the device.

Also, I've had, I've considered dropping Trezor One from HWY.

Right.

So maybe, I don't know. so I think Right now the, so Ledger is, is actually supports multisig really well, but it's, if you consider the discussion we had earlier about change, it actually might not be safe there. you have to be careful. I believe Coldcard probably does it the best with registering the multisig with the device itself. in order to use multisig on it, you have to register the xpubs that are gonna be in the multisig, and it'll store those locally. And then, I guess, yeah, Trezor T, it, it also, I believe, has the same issues with change, with, as the ledger. But I haven't- Oh really?

I didn't know that.

Because they, they, none of them, none of them actually do the change thing, I think. Because this is a new development, like, it was a new discussion. now I can't actually remember what the Trezor does for multisig. And there was one thing that I did for multisig in HWI that is Not the greatest thing. And, and for supporting coinjoins.

Summarizing that then, it's kind of like if you look at Coldcard, Trezor Model T, and potentially Ledger, but the change address problem is still there with Ledger and potentially with Model T.

The change address problem, I think, is Always gonna be an issue with hardware wallets, and just change detection in general is hard. For some, some devices, I think you could lie to it and it just wouldn't check. You could say like, "This address has changed," but, you know, maybe it's not actually changed, and it wouldn't really know. But, but for, for the most part, they actually do handle that well, because they require you to Send the derivation path, and then it'll try to derive that address from the derivation path and see if it matches. That's how they detect change now. But multisig changes very different. And then it gets more complicated with like complicated scripts. Going back to the output descriptors thing. What's going to be expanding upon that is many script, which lets you have arbitrary scripts. So if you have an arbitrary script, it might be harder to determine what's change, what's not change.

Drawing it to a close, then, where does H-W-I go from here? And what further development work is most needed for H-W-I?

So right now, it's actually in a pretty done state. it's mostly just Bug fixes, integration problem bugs that, that crop up, and like some new features maybe. so like what I was working on earlier today was adding An update firmware command, to it, which every device also has its own way of updating firmware, which was really annoying. so the, I was adding this update firmware, like that'll be something new, but it's, you know, it kind of isn't really needed in signing or anything else. It's just kind of like nice to have. So for the most part, Hwi is just in pretty much bug fix mode. Yeah. Wasabi's been reporting a bunch of bugs when they do integration, like, "Alright, I'll, we'll fix those." But there isn't A lot of new functionality to add, nothing that really needs to be done with it. Everything else that we're doing with HWI is actually on the software side, like the- the wallet side with Bitcoin Core, and that's what a lot of my focus has been, was, is getting Bitcoin Core ready to be able to interact with Hwi, seamlessly.

So in terms of that then, making Bitcoin Core ready to interact with Hwi, what's required?

a lot. So, recently I've been working on this, Wallet refactor, and that's because we need to refactor the wallet in order for HWI to work. Right now, the wallet is The wallet is kind of a monolith, it has everything in it, like key generation, address generation, transaction tracking, watch-only things. Like anything wallet is inside, a class called CWallet, and it's gigantic. So we're trying to separate this out and move key generation into its own thing. And, and C Wallet will now call this the thing for key generation, and the idea is by separating this, we can actually just define different classes of key generation types. Like, we can have keys being generated from the old method of just randomly generating keys, or we have another class that says keys are generated, from output descriptors, which is actually the main thing I was trying to get. This key generation with output descriptors. Or we can say, you know, if you have this class, keys are generated from a hardware wallet, and this class now has all the things to interact with a hardware wallet. So C wallet itself doesn't need to know about the hardware wallet, it doesn't need to store any information about hardware wallets. It just knows that if it calls this variable in memory, that thing will do some magic in the background and give it an address or a key, and that thing in the background might Might be calling out to HWI and getting stuff from hardware wallet instead. So that's, that's kind of what I've been working on to prepare Bitcoin Core to Do, hardware wallet stuff. And then Shore's, Shore's provost has been working on the GUI and the actual integration of HWI into Core. He's been building on top of the PRs that I've created, to add stuff to the GUI, add stuff to the command line that will do the, the ideal future of, you know, send to address and everything just happens in the background.

The promised land.

Yeah, it's, it's still a long way off, I think. Where, like the, the PR that I created to do the separation is- Like plus four thousand lines of code, and people have to review this, it's gonna take a while to review, so we probably won't see hardware wallets for Another two or three major versions of Bitcoin Core.

Right. So, that's like six monthly releases, right?

Yeah. So, let's see. Point 19 will be out in like September, October, I think, maybe November, and then, you know, six months from that, March, April-ish. So like By this time next year, maybe we'll have hardware wallets in Core.

Right. and I suppose in your own experience, have you done much multi-signature work outside of Bitcoin Core, or have you mostly just used character, command line interface to do that sort of thing within Bitcoin Core?

I've pretty much only done command line within Bitcoin Core. I did attempt to do multi-sig hardware wallet stuff. Multisig with multiple hardware wallets, and that failed miserably. It was just too difficult to use.

That was with Bitcoin Core, yeah?

Yeah, it was with Bitcoin Core and HWI, and then I also had a, a wrapper script around that, but I guess it, it might be possible now. At the time I was doing it, Bitcoin Core didn't support, you know, you can't have multisig addresses from Bitcoin Core, it won't give you them. with the Scripter wallets and the refactor I'm working on, it will actually be able to do this. So, it might be better Soon. But when I was working on this, side thing with multisig, it didn't work. I had to store, addresses, by myself, and then like I had to remember what was used, what wasn't used, and it was just generally, a pain to use, and it didn't actually work. So- That, that's been the extent of my multisig experimentation with hardware wallets.

Yeah, no, it's good to, good to hear how you're, going with it and potentially this will become easier over time. Well, I think it will, it's just a matter of, yeah, the, the right pieces being put together. I suppose that's all. Is there anything else you wanted to cover today, Andrew?

no, I don't think so.

Okay, well that's great. So look, make sure you let my listeners know where can they find you, where can they follow you online?

Yeah, so my, my handle literally everywhere is a chow 101, A C H O W 1 0 1. you can find me Twitter, GitHub, Bitcoin Stack Exchange. What else? Bitcoin Talk. Occasionally, I will stream coding, actually. I don't know how long I'll keep that up. I started doing it last week. so I'll be-- you can also find me on Twitch at HL101. Yeah, but pretty much anywhere you see HL101, it's probably me.

Fantastic. I'll put the links in the description as well, so the listeners can find you there. Well, look, thank you very much, Andrew. It's been a really educational discussion for me and for my listeners, so, thank you for the time today.

Thanks for having me.

So what did you think of that? Did you already know about derivation paths and hardware wallet interface and PSBTs? I thought it was pretty cool. Definitely a lot to learn there though, and it's quite technical material. So I've put some links in the show notes just for you guys to do a bit of research on your own as well, if you'd like to read a bit further into it. You can get the show notes on my website stephanlivera.com, you can subscribe there, and also you can share the episodes with your friends and help them learn also. Any feedback for me DM me on Twitter at stephan livera or email me stephan livera at p m dot me. That's it from me guys, and I will see you in the citadels.