Why does auditing Bitcoin matter? Why won’t the supply ever actually reach 21 million? Pierre Rochard of Kraken and Fabian Jahr (Bitcoin Core contributor) join me on the show to talk.
- Bitcoin supply
- Auditing
- Gettxoutset
- Bitcoin Core coin stats index
- More in depth bitcoin accounting
Links:
- Fabian Jahr github: @fjahr
- Fabian Jahr: @fjahr
- Article: https://fjahr.com/posts/where-are-the-coins/
- Pierre Rochard: @pierre_rochard
- Article: https://www.pierrerochard.com/auditing-bitcoin-supply/
Sponsors:
- Swan Bitcoin
- Unchained Capital (code LIVERA)
- Knox Custody
Stephan Livera links:
- Show notes and website
- Follow me on twitter @stephanlivera
- Subscribe to the podcast
- Patreon @stephanlivera
Podcast Transcript:
Stephan Livera:
Pierre and Fabian welcome to the show.
Pierre Rochard:
Hi, Stephan. How are ya?
Fabian Jahr:
Hey, Stephan really good to be here.
Stephan Livera:
That’s great. So look, Pierre, my listeners are very familiar with you, but Fabian, tell us a little bit about yourself.
Fabian Jahr:
Well I’ve been contributing to Bitcoin core for about a year now. I’ve been interested in Bitcoin, probably starting around 2015, 2016. I knew about it before, but that’s when I really kind of grasped its importance and that’s the reason why I got really interested in it. And even more when, when lightning came around, that was kind of another trigger for me to really spend a more significant part of my free time on it and really try to get involved, try to use it in projects, et cetera. And then the next big step up for me was participating in the Chaincode residency last year. That was really awesome. I learned a lot there just meeting all the Chaincode people there collaborating with other residents. I learned a lot and I’ve mostly been trying to just be helpful around Bitcoin development mostly in Bitcoin core since then.
Stephan Livera:
Today we’re going to get a little bit into this whole topic of where are all the Bitcoins now Fabian, I know you’ve got a very relevant blog post on this. It’s called, “Where are all the coins?” And Pierre as well. You’ve been making a lot of noise about this recently. So maybe Pierre, do you want to just start us off? Why is it important to verify these things?
Pierre Rochard:
Yeah, sure thing. So I’ve written up a post that kind of summarizes it and has some data points at pierrerochard.com. And basically the idea is that if you take any kind of economic entity typically let’s take a business for example. And you think about what its balance sheet looks like for, for all the items on the balance sheet, you’re going to be able to prove out exactly where they came from. Right. And if you can’t do that, then, then you’ve got a bad accountant or something. And basically when you apply that logic to a network like Bitcoin, the Bitcoin network is you don’t, you don’t really actually hold Bitcoin yourself, right, where you hold as a private key. And that private key is what’s gonna allow you to unlock Bitcoin held on the network.
Pierre Rochard:
And the place that they’re held in specifically in, Bitcoin is the UTXO set. So that’s the set of all the outputs that could possibly be spent at a given block height. And so in a way, the Bitcoin you know, the way I describe it is it’s actually issuing a liability, right? And so the Bitcoin network owes you the, I don’t know how to word it in, in a really good way, but owes you the ability to transfer Bitcoin to unlock Bitcoin. If you provide a valid input inside of a valid transaction you know, that has a high enough fee to get mined. And the network basically owes you that. Right. And so the way that you would approach it in terms of figuring out as an auditor, how you would prove this out first of all, you look at kind of what the expectations are, right. Of okay. People have this issuance function, right. And the issue. So, yeah, I guess I’m going off, I’m already going into it fully, but yeah, so I’ll take a breather here Stephan, so I don’t just biggest, I feel like I could filibuster for several hours on this topic.
Stephan Livera:
Of course, that’s fine. Fabian let’s hear a little bit from you. What was your motivation behind writing that initial? Where are all the coins blog post?
Fabian Jahr:
It was really just kind of summarizing what I had learned working on my project that is the coinstats index that we are going to get into, I think in a little bit. So in that project I was very much having to figure out, like, how does the body of coins, like what was just talking about that is also being called the UTXO set, the unspent transaction outputs that every node in some way keeps for the user in order to validate every transaction. How that actually gets created, like what the rules are that they get applied to every block in the blockchain that either allow an output to be included in the UTXO set or not be included in the UTXO set.
Fabian Jahr:
And even though I had already gone quite a bit down that rabbit hole, I was still learning quite a bit along that way. And yeah, I felt like just, just kind of summarizing that sort of for me, but also for everyone else, because at the time it also felt like pretty hard to find an answer to that question online. It turned out that there were some answers for a similar question, but it was just worded in different way, but I was kind of answering that question for myself summarizing what I had found there, in my research and is also called back to the time when I first started up running a node several years back. Because then I waited you remember that when I spun it up I was kinda following some tutorial and it said to run, gettxoutsetinfo.
Fabian Jahr:
And when you’re running it to yourself, it before, as somebody who doesn’t know the background of this there are two things that are usually surprising to you. The first thing is that it takes quite a long time to run. And the second time, the second thing is, is that it will give you an output that says total amount. And that is if you look at the help, that is supposed to be the total number of coins that are in circulation. And it’s a very crooked number. So then if you or you know, a little bit about Bitcoin and you probably do, because you’re running a node and then you know, that the block subsidies have been the last block subsidy has been 6.25. So there shouldn’t be any more crooked number that has anything else that has more than three digits after the comma or dot.
Fabian Jahr:
And so this was very surprising to me. And at the time back then, I was also Googling this and kind of trying to find the answer. And the answer I found was probably inaudible cause somebody on bitcoin talk or someone knowledgeable. But the question was the answer was pretty, it was kind of like, yeah, there’s some coins are lost and this is fine basically. And at the time I was kind of like, okay, I guess this is somebody who knows what’s going on. So I kind of accepted it and moved on. And then now several years later when I really dug into it and really understood fully how this number is really calculated I was kind of delighted that I just yeah, wanted to put it into writing. And if you look at my blog, I don’t write a lot. There’s really, a lot of things would have to happen that I, that I sit down and write a long blog post. But yeah, this was the reason that got me to do it.
Stephan Livera:
Excellent. So I think one point that I’d love to touch on before we go dive into further detail around exactly what you were doing. I think perhaps it’s a mindset thing now, Pierre, I know in the past you came, you were an auditor and funnily enough, I was also an auditor in a past life let’s say. And I’d love to hear a little bit from you Pierre on what are some of the things that are in an auditor’s mindset? Why should we try and approach things from that mindset rather than simply trusting?
Pierre Rochard:
Yeah, so I think that it really is about decentralization. So the moment that we have to trust you know, a third party that introduces a vulnerability in my mind into the Bitcoin system and into its immutability. So you know, that what Fabian just described of the you know the transaction, the UTXO set not adding up to a round number that we would expect given the current halving that in the audit world, you know, that that would be something that they would flag, right, as something that would be, that needs to be looked into. So , why does that matter? Why should I think, you know, to me, the real reason why this matters even if, and what we’re talking about is deflation, right?
Pierre Rochard:
So first of all you know, you could argue, it’s always in our favor, right? If it’s deflation. But I think that the knowing the exact numbers and where they come from gives us the knowledge that there isn’t a deflation bug that there isn’t outputs that unexpectedly are unspendable. And now they might be on spendable because that person lost their private key. We have no way of knowing that. Right. the only thing we can know is what is unspendable from the network’s point of view. And part of that is judgment on the behalf of the developers, right. That and then it’s kind of just like known deflation bugs that we can’t fix and won’t fix. And that’s totally fine. What we don’t want is someone to think that they have a spendable output and then for them to try to spend it only to find out that the network won’t let them spend it right.
Pierre Rochard:
That in fact they don’t have access to those coins, which would be a really bad situation. And so a deflation bug would be bad. The other thing that we want to look for well, so basically what we’re trying to reconcile right, is issuance to balances, right. To user balances and you know, in Bitcoin, there’s not really any concept of what the user is, right. It’s just an abstraction, but I’m just looking at the total current UTXO set and figuring out exactly where those differences are coming from, and also recalculating it at every block height, because what you might have in accounting is where you have maybe a deflation bug that reverses itself in a later block. So you would want to look for that as well. And really the only reason that we’re moving the goalpost right, from just counting up Bitcoin supply, is that we’re finding anomalies, we’re finding differences, and we have to reconcile those differences.
Pierre Rochard:
We can’t just decide, okay, well, it’s just going to be a black box of, you know, we don’t, as long as this number is less than total expected issuance, then we’re okay. You know, that’s not really a satisfactory answer from an audit perspective right. And really, it comes down to this approach of risk based auditing where you kind of look at you know, where are the risks in the system and looking into this, and we had a, a near miss or an unexploited inflation bug, there’s probably different ways of describing it, but that was fairly recent in Bitcoin. And so I think it reinforces the value of doing this exercise. And then we can get into what’s going on in Ethereum world. But yeah, I’ll stop it here. And really, so this is when I asked on Twitter, has someone done this for Bitcoin? And I got pointed to the coinstats index and that’s kind of where I was able to really bribe Fabian into updating coinstats index so that I could do the math I’m talking about here of adding total issuance to, you know, UTXO set.
Stephan Livera:
So Pierre actually, it’s probably a good point now to talk a little bit in contrast Bitcoin with Ethereum. So there was some recent, you know, back and forward on Twitter, around what is the supply of Ethereum. And, you know, some of the initial answers were things like, Oh, just trust the block explorer trust the online websites. And do you want to just tell us a little bit about that and whether that is a satisfactory answer or not?
Pierre Rochard:
Yeah, sure. So basically what happened was Eric Wall I think is his name. He’s a cryptocurrency researcher. He made the offhand remark on Twitter that Ethereum proponents are better than Bitcoin proponents. I forget what his exact wording was, but it was basically along those lines that they can better explain the value proposition for Ethereum and are more technically savvy than Bitcoin users. And so Michael Goldstein himself, Bitstein the memester replied asking how many Ethereum users know how to add up the total supply of ETH themselves, right, not trusting a third party. And that’s when people realized that there actually isn’t a script handy laying around on GitHub to even do that. And that all of the code that would allow one to add up total ETH supply at that point was actually proprietary.
Pierre Rochard:
And you know, run by data, providing companies, right. Trusted third parties. There’s no, and even though, you know, you could run an Ethereum node and in theory, you could write the code to query it, right. That’s what these data, third party data providers are doing, but nevertheless, that code just didn’t exist in the open source community, which you know, we can get into whether that’s, but anyway, so when, when this became apparent because of Michael Goldstein’s question that this open source script didn’t exist, you know, in the audit world, that would mean that there’s a very high risk, right? There’s a very high audit risk because if a client tells you that they don’t have the spreadsheet that does the reconciliation, then the numbers haven’t even been checked once, right. If there’s no trial balance that you could point at. And anyway so, so that was a huge, huge audit red flag that they just didn’t even have a supply script. You know, someone cobbled one together within a couple of days, and then someone else cobbled another one together. They didn’t agree. They didn’t agree with the block, explorers the proprietary websites either. And you know, they were, they weren’t off by huge amounts. Right but nevertheless as an accountant, you would expect that recalculating ledger balances at a specific block height would give you the same result. And so that wasn’t happening. And so they started a project to try to reconcile it and they’re able, what they’re able to do at this point finally is able to reconcile issuance. So in Bitcoin that’s you know, kind of adding up all the Coinbase transactions, all the Coinbase outputs it gives you, you know, historically, you know, the cumulative total issuance, creation of Bitcoin.
Pierre Rochard:
And so they were able to finally get that part of it together. Now getting account balances is you know, that’s pretty easy in Bitcoin, now we said that getting the, the summing up the UTXO set you know, it can take minutes on hardware, OK Hardware that, you know, that’s a, that’s an intensive thing for, for Bitcoin. Now, you know, that, I think the current UTXO set is in like a few gigabytes or something like that.
Fabian Jahr:
4 gigabytes.
Pierre Rochard:
Yeah, yeah. So it is, you know, that’s a tractable amount of data, but it is very large amount of data. And you’re just iterating over it and, you know, summing up in Ethereum it’s actually a lot more data than that because they don’t have an output. They don’t have an output model.
Pierre Rochard:
They have what they call an account balance model. And so adding up all those account balances is actually much more computationally demanding than it is in Bitcoin. And so they’re able to, do that at, at the block tip, but they can’t do it. They don’t have a coin stats index in Ethereum, and you know, that while they could in theory create one, right, there’s nothing you know, impossible about it computationally. I think that that’s what is prohibitive for them and that you would have to have a fairly powerful, you know, server to be able you know, with terabytes of SSD or even Ram and, you know, very powerful CPUs to be able to do that in a timely manner. Now, obviously somebody is doing it because these proprietary data websites are doing it, but in terms of for a normal person it’s really cost prohibitive to do that. And generally, I think that, you know, Ethereum users are gonna have to trust third parties with regards to Ethereum supply and their confidence in it .
Stephan Livera:
Fabian anything to add there in terms of contrasting Ethereum supply versus Bitcoin supply?
Fabian Jahr:
Yeah. I mean there’s not really that much they can say to that because I I still have so much to learn in terms of Bitcoin core. And so I kind of block out a lot of the noise about altcoins and Ethereum. And so my Twitter feed is curated well enough that usually I don’t even see that kind of stuff. But this ETH supply thing was something that made it to my feed and it actually was something that I noticed and that actually raised my eyebrows because yeah, it’s just amazing that this is even possible in Ethereum, that people don’t know this, that the people don’t have the script, when it’s like one of the most fundamental things of Bitcoin that I can think of.
Fabian Jahr:
And so that was just amazing to me and just just giving me, giving me a confirmation that I’m kind of on the right path, just focus on Bitcoin but that was kind of, sort of it and also of course at the thought that does this is kind of funny because I had already been working on the coinstats index for several months at that point that yeah, like that’s why I also had this strong feeling about it because I had been working on this very thing in Bitcoin. It was already existed in Bitcoin, of course, but I was putting a lot more work into basically usability and efficiency improvement of that very thing that seemingly wasn’t even possible in Ethereum. And then I only actually sprung into action when Pierre was then tweeting that he was interested in something similar in Bitcoin that was giving him the supply of Bitcoin and in more detail. And that was really just my cue. And I just said, Hey, I’ve been working on this for a long time. Mostly just need some more eyes for review. And that’s how we got together.
Stephan Livera:
That’s a really cool story. I think it, I mean, to the comments around Ethereum, I think, I don’t know, maybe Ethereum people, I mean, obviously the theater people might disagree, but to us as for me as a relative outsider, it seems to me that the Ethereum people don’t really know what they’re about. Whereas in Bitcoin, we know what we’re about. We want fixed supply that’s so, so important to us. So maybe if I were to steelman, the Ethereum people, that’s not their priority, I guess. I don’t know. But anyway, let’s focus..
Pierre Rochard:
Well hold on, I do want to respond to that, right. So, so that’s fair, right? That they say, okay, you know, Bitcoin is digital gold. So you’re interested in that supply. You know, Ethereum is like digital oil. It’s you know, it’s the utility value is what you’re interested in. And really ultimately, you know, their argument is that the value of your coin is going to be determined by a combination of supply and demand, right. And that you could have something that is not fixed supply. But if it has greater demand because there’s greater utility in it that, you know that coin is going to outperform, right. I think that’s as a core part of their thesis. And I think that where it all falls apart is that they’re advertising this as digital ledger technology. And it’s incomprehensible to me that you wouldn’t be able add up a ledger you know, fairly easily to get a reconciliation like we’re able to do with Bitcoin.
Pierre Rochard:
And you know so the bounty I paid out to Fabian for the coinstats index. And I also paid him a bonus bounty because he also wrote a pull request that I was able to use to export that coins stats index into a CSV file. And if there’s any accountants or auditors listening, obviously they know that CSV files are the best thing to get from an audit client. Right. And so I loaded that into Python. I recalculated the entire issuance myself in Python, and, you know, it came out to the same as the one in C ++. Great. And then I winded up with the data from coinstats index, and sure enough, there has been zero unexpected deflation and zero unexpected inflation in Bitcoin. And everything is going according to plan.
Stephan Livera:
That’s awesome. Let’s jump further into those numbers around Bitcoin. So I think one of the first things people learn about Bitcoin is, Hey, it’s got a supply of 21 million. But an interesting fact that came from your article Fabian is that it’s not actually going to be 21 million. Exactly. So can you tell us why that is?
Fabian Jahr:
Yeah. So maybe since you are asking about the, the 21 million, one thing that I also need to need to say is that I’m not writing about in my article is that even if none of the other things that we are talking about that have already sort of, let’s say that they’ve gone wrong, like something is if that’s wrong that something is excluded from you UTXO set then we would still never have ended up at 21 million. And that’s just one, one small thing to know is that the Block subsidy that gets gets smaller every happening eventually will end up dropping below one Satoshi and that is expected to be happening around 2140. And that is something that would have been very complicated to implement.
Fabian Jahr:
So the what actually happens is that the supply gets cut off at that point. And the total supply that kind of all block subsidies that we’re building towards is, I wrote this down, it’s 20,999,999.9796 Bitcoin. So that’s, that’s the first kind of smart fact to know whether it will never be 21 million. And then there are the coins that have already been excluded from the UTXO set at this point. And this is where my article’s kind of getting into I’m, I’m actually never talking about the 21 million in my article. I’m talking about the number that you see when you call gettxoutsetinfo. And this is a very crooked number that I mentioned before that it’s probably surprising people if they haven’t gotten that far down the rabbit hole yet.
Fabian Jahr:
And they see that crooked number and they just know what the coin subsidies are. And they expect a different number basically at this point. So the the coins that are missing in my article, I’m categorizing them in different in four different categories. And this was just a necessity for me also, as I was building the index to identify these from the consensus code. And then we implement them in the index as well, because I’m processing every block and I need to identify which outputs are not included in the UTXO set. So that the index is coming out to the same data that the UTXO set is coming out to. And the four categories that I ended up with is first the Genesis block the Genesis block had the block subsidy of 50 Bitcoin and these 50 Bitcoin are not spendable.
Fabian Jahr:
The reason for this is there’s just a very simple IF statement in the code. And that just checks, okay. If blockade is zero, then we just skip adding the coins of that block to the UTXO set. I think I didn’t do very deep research into it, but I remember reading that, I think nobody really questioned it ever. But yeah, just, I think nobody really knows why Satoshi did it that way and didn’t make it spendable. And I think we will then if that’s right then we’ll probably never get an answer to that. The second category is BIP30. BIP30 was bug in Bitcoin that was discovered pretty early this was happening, I think if I’m not wrong 2011 I have the you can look into the details of the Block Heights in my article.
Fabian Jahr:
But what was happening was that in the UTXO set we are saving the outputs by looking at the TXID of the transaction, and then also the index of the output of the transaction. And usually we can trust that this is unique because the TXIDs are unique because they depend on the TXID of the previous output. And so that way that the uniqueness is kind of cascading through these chains of transactions. But there’s one problem. One type of transaction that we have in the system doesn’t have an input and that’s the Coinbase transaction and the Coinbase transaction, it was not enforced that these are unique and what this meant was that we had a block with a TXID for a Coinbase transaction, and then another block occurred that was mined by the same person by the same miner.
Fabian Jahr:
And that has the same TXID for the Coinbase. That meant that it overwrote the output in the UTXO set of that was already there. This happened twice. And so this is how we lost another 100 Bitcoin, that miner lost 100 Bitcoin from that. And then there was hard-coded fixed for that in the code. I also have links to the specific lines in the consensus code and there is the the block heights and the block hashes of these very specific blocks are still hard-coded in the coinbase where to ensure that this is kind of an exception in the consensus code and for these two blocks, it is OK. That they are coinbases are extra different UTXO set, but this is never going to happen in the future because that bug has been fixed since then.
Stephan Livera:
You’ve got OP_RETURNs well, right?
Fabian Jahr:
Yeah, exactly. Then OP_RETURN very famously an opcode with which you can embed arbitrary data in the, not in the UTXO but not the blockchain. And these outputs are not spendable. And that means it’s also logical that they are not included in the UTXO set. And then the last one is maybe the trickiest one to wrap your head around. Or at least if you if you look at the code it seemed for me that way, because it is really describing something that is not there. So this is unclaimed miner rewards. So we have been talking about the block subsidy. The block subsidy is what we talk about when we mean these 50 Bitcoin and then 25 Bitcoin, and then 12.5 and now 6.25 that are given as a reward to the miner, but really the block subsidy is not the full reward.
Fabian Jahr:
And it’s not the full amount that the Coinbase transaction has because in the Coinbase transaction in the output are also included the fees. And this is also included in what we mean is, is the miner reward. And so what do you have to do? What, what kind of your obligation is as a miner when you mine a block and is that you look at the whole block and you look at the block subsidy that you, you will get at the current time. And you add all the fees that you get from all the transactions in a block together with the block subsidy. And that is the total reward that you will get from that block, if you happen to be lucky and mine it. And so this is not like, like now of course, everyone knows how this works and miners have functional software to do this.
Fabian Jahr:
But in the earlier days this was not the case and probably people were playing ground with some custom scripts to build blocks, maybe trying to make it more efficient and that led to the fact that yeah, there were of transactions created that had a smaller reward then they actually could have gotten from the old block. So there, there are some examples that I didn’t know these examples because I haven’t been around at the time. But Peter Wuille listed these in a in an article himself where miner rewards haven’t been claimed. And since these are just not there they, they will never be spendable because they are not included in the UTXO set because the amount of the output was too low. And the software, the consensus where doesn’t error on this still says, okay, the block is fine. Even if the reward is smaller the validation project just looks at that, the miner, what is at least, or is not higher than the miner reward that a miner can claim.
Stephan Livera:
Yeah, really interesting stuff. So it’s kind of like, through these different, mainly through those four categories, there are, it’s something on the order of 2.3 million sats that will never be created or less, it’s going to end up being less than the 21 million eventually once we get there and over a hundred years. And there’s some interesting trivia as all around some of those unclaimed miner rewards. So essentially, as you were saying, as you were explaining, Fabian miners can claim up to this amount. And some of them have made a mistake and accidentally claimed less. And I think in one case there was a miner, I think this is like an early Bitcoin user, midmagic who intentionally claimed less than the full amount. I mean, this was early days as well, but it was just a funny little story to read into.
Fabian Jahr:
Yeah, yeah. To be honest, I’m not sure if I got this right. When you said 2.3 million Sats did you mean that the total missing amount of where the coins are? Because 183 Bitcoin, I’m not sure which, what number you were referring to.
Stephan Livera:
Sorry, sorry. Yeah, you’re right. Actually. Yeah. It’s 183 in terms of the lost or not claimed coins. And then the 2.3 million sats is just the, like the high level that it’s not even 21 million.
Fabian Jahr:
Yeah, yeah, exactly. Yeah.
Stephan Livera:
Yeah. So it would be, I guess, really it would be the combo of those. Yeah. So it’d be like the got to do this, right. So the 183 that you’re saying have been lost or not claimed. And then plus that kind of 2.3 million Sats or whatever, that, it’s just, it was never going to be that 21 million anyway. Alright. So let’s get into Bitcoin core and checking the coins. So maybe Pierre, can you tell us a little bit about gettxoutsetinfo? What is that and why should we use that?
Pierre Rochard:
Yeah, so that’s going to add up the current total sum of UTXOs at the current chain height. So if that’s the number you’re looking for, then you it’s at the tip of your fingers, right. Now what is that useful for right? Because arguably, you know, people losing their private keys, eclipses this number. Well we have, no, we have no way of knowing that, right. Satoshi could come back tomorrow and spend his coins. So, you know, the argument there is that economic agents don’t really need to know what the current total money supply is at any point in time. Because really it’s it. Well, okay. So anyway, yeah. Let’s see. Where were we? Yeah, the, sorry, the UTXO set. So really what I wanted to find was historical UTXO set data and that’s hard to do without doing exactly what Fabian had been working on with coinstats index. And I think that adding more indices to node software is going to increase the number of users ultimately because there’s just more data available to them through the API.
Stephan Livera:
Maybe Fabian. And can you tell us a little bit about what you’re achieving with coin stats that goes above and beyond what gettxoutset is doing?
Fabian Jahr:
Yeah, so as Pierre said the gettxoutsetinfo call is all right. To give you the statistics on the current UTXO set and at the very least you can look at that number and you can maybe do small calculation on your calculator to see what the total block subsidies should be now. And you can see that the total amount that you see from gettxoutsetinfo is smaller than that. And so at least, you know, they haven’t been, there has been any inflation beyond what the current supply schedule gives you as the total number that should be out there. But there are a few things as we both have mentioned before from gettxoutsetinfo for that are sort of missing.
Fabian Jahr:
And that I am looking to add with the index. And the first thing is that it is quite slow depends on your hardware, of course on a recent laptop with good specs. You would probably be below two minutes still, but if you are running Raspberry Pi3, which a lot of people do have it running in the home. It’s going to be closer to something like seven or eight minutes. And that still sounds fine if you want to get a coffee yet at that time around the time or something like that. But it’s really not great if you want to do a little bit more and what was kind of an inspiration for going down this path and making these changes was Forkmonitor. And Forkmonitor that info is running some nodes and giving some stats on these nodes.
Fabian Jahr:
And one thing that they do very prominently, is they continually track the total supply of gettxoutsetinfo and in case that number increases in an amount that is not okay, then we know there’s no reflection back. And then this website will be one of the first to ring an alarm bell. I mean the fact that we would have an inflation bug means that the consensus code has been suppressed but the gettxoutsetinfo hopefully if that hasn’t been affected in some, some way as well which would be crazy that should then show that there are more coins that should be there and Forkmonitor is running this continuously as I said, and I don’t know what their hardware specs are.
Fabian Jahr:
But I mean, of course they don’t want to throw money out the window when they don’t need to. And what they told me was that when there are blocks coming in very fast like we’re aiming at 10 minutes, of course, on average, or when your block should come in, but then if you look at what actually happens is that sometimes we get three or four blocks of 10 minutes. Of course it’s going to even out in the end some way as well. But at that very moment, when we have these many blocks coming in the node will have a hard time catching up because the process is going to run on gettxoutsetinfo, and by the end of the time it has ended, there will be already been another block or maybe even two other blocks.
Fabian Jahr:
And that is not great. And this also made me realize that we will have a, we’ll have a problem in the future. If people were wanting to do this because right now we said the UTXO set it’s about four gigabytes. And when the, when we say, okay, I’m a user of a Raspberry Pi3, and I run this and it takes eight minutes. This number of eight minutes linearly increases with the size of the UTXO set. Now, let’s say in three, four years we’ve had two more bull markets. And also people are, are more of a people holding their own coins, controlling their own outputs. And so that means UTXO set probably will grow quite significantly. And that would also mean that the time to run gettxoutsetinfo will run significantly increase by the same amount.
Fabian Jahr:
So if it doubles then all the people that still follow the tutorials like I did back then, and gettxoutsetinfo, they will have to wait 15 minutes. And then we also know they will not be able to run this continual checking of gettxoutsetinfo and the total amount anymore, because 15 minutes, as long as the 10 minutes, and then this is just not possible anymore.
Stephan Livera:
Yeah. Fascinating.
Fabian Jahr:
Yeah. So what of course like this is I’m not really adding something new to Bitcoin because the data that I’m using and that I’m presenting is already there. It’s already in the blockchain anyway. What I’m essentially doing is usability and efficiency improvement. So I’m basically precalculating the data of gettxoutsetinfo for every block and saving it in an index.
Fabian Jahr:
So this means you can look at every past block if you’re interested in the data before. And is this something that, Pierre now has been using? He has been looking at every block and, and adding up the numbers for every block to ensure that there’s never anything been going wrong. And anecdotally also, maybe people want to want to just check numbers on a certain block height and don’t want to a website for that. And want to look at it on a node, and then what also happens then when we have these numbers, pre-calculated for every block we don’t have to iterate over the whole gettxoutsetinfo anymore to calculate the numbers for the next block. When a new block comes in. Now we can just look at, okay, how are the numbers changing in that new block? And then add those, or subtract those from the numbers that we have already said for the last block. And that is of course, much more efficient, much faster than iterating over the whole UTXO set with every block. And that means we can get the new numbers in less than a second after the new block has been validated.
Stephan Livera:
Pierre have you got anything to add there on the coin stats index and the review and the process around Bitcoin core there?
Pierre Rochard:
Yeah. I just realized something that relating to the fast blocks. There’s another form of an inflation bug, which is that the rapid increase in mining hash rate and the two week difficulty adjustment have made it such that we’re kind of living in the future or future Bitcoin issuance has been pulled forward. Right. Because we haven’t, we’ve been, you know, getting blocks more quickly than every 10 minutes. And so this is something else that we should account for, and we should look at what would issuance be today if it had actually been every 10 minutes. Now you could argue that that’s not really expected issuance because people expect the hash rate to go up. Maybe, I don’t know. That’s, I just realized this though. Thank you.
Stephan Livera:
We could call it Satoshi’s schedule.
Fabian Jahr:
It’s actually not sound money. Yeah.
Pierre Rochard:
This is an inflationary Fiat shitcoin. I can’t believe I’m so disappointed.
Stephan Livera:
Oh, well, pack it up, boys. It’s all over. Yeah. So this reminds me very much of a Saifedean’s comments around a plan B and Stock to flow, and he was saying, Oh, but what if you recut the numbers and do it on Satoshi’s schedule instead of the actual numbers. But certainly that’s an interesting point there. And so in terms of getting coin stats index into Bitcoin core, as I understand there are multiple components required to make this happen. So Fabian, can you just give us a bit of an update in terms of where this is at currently and what is still outstanding before this can be fully merged in?
Fabian Jahr:
Yeah. So this touches a bit of different topics that is very related, but also not, not really relevant to what we’re talking about, but I’ll try to just summarize it briefly. So the coin stats index that I set out to do several months ago and what caused me to even look at this? First of all, the biggest part of the work initially was to look at the hash function, because one part of gettxoutsetinfo that we haven’t talked about at all right now is a hash of the it’s called serialized hash, I think, in the output. And what does, does it just serializes and hashes every output in the UTXO set and then gives you one number and this can also be kind of a quick check to compare a UTXO set.
Fabian Jahr:
your UTXO set with somebody else’s UTXO set to ensure that you’re in consensus as well? It doesn’t really work that well also without the coin stats index, but yeah, this hash was one of the reasons why there was just not any other way to speed up to optimize the gettxoutsetinfo call because this hash function means that you cannot efficiently parallelize, the calculation of the hash of these UTXOs. So when a new block comes in and we just have the information that new block, and we have kind of the hash of the, UTXO set that existed before that block. And then we cannot reconcile that, we can, this hash has to just be calculated every time do you take UTXO set is changed.
Fabian Jahr:
It has to be calculated from scratch. And there was an idea from Peter Wuille to introduce a new type of hash which is cumulative. So this means that we can actually calculate the hash for block height 10,000. And then when block had 10,001 comes in, we can calculate the hash for the outputs of just the transactions in 10001. And then we just add or the way this works is it’s actually multiplied but add it to the hash that we already have from the previous height and this works and then we have a hash that we can compare and we can actually then save the intermittent hashes for every height also in the index and compare these as well. Something that, can also be potentially interesting to use for assumeUTXO, an upcoming interesting feature by James O’Beirne.
Fabian Jahr:
And so this is basically the one branch of this development. There’s two PRs have been merged. One is the Python implementation of this hashing algorithm. And one is alteration, refactoring basically of the gettxoutsetinfo where you can exclude the hash from gettxoutsetinfo, which is just a basic preparation, but also this is being used in what Pierre is using right now. And then following up to this hashing branch, basically what still needs to be merged is the C++ implementation of this hashing algorithm, which is called Muhash, by the way. And then what’s coming on top of that is adding Muhash actually to gettxoutsetinfo so that you can gettxoutsetinfo, with Muhash calculation instead of the serialized hash calculation.
Fabian Jahr:
And then on top of that, the Muhash will actually be added to the index. And so this is where basically the branches meet. So going back to where we’re at right now we have the index, the coinstats index that is ignoring the whole hashing stuff that just has no hash of the UTXO set, but just calculate all the other numbers most notably, the total supply of for every block. And that’s where in that branch we have now one big outstanding PR that Pierre has been working with that implements the index. And the way this works, if you want to check this out is you can check out this pull request and you can build it. You can run the node with coin stats index just works very similarly to TX index. So you will both see the progress in the blocks, but also you can now call an RPC, gettxindexinfo We’re just going to show you the progress of the index. And then you can call the, gettxoutsetinfo call with a specification that you don’t want and hash. And then this is automatically in the background going to use the index for you, and if you don’t provide any specific height parameter, or block hash parameter it is just going to look at the latest block height, but if you give it a specific parameter then it will look that up in the index. And then following up to that one is the last one that, Pierre has been using the most. And that is the one that looks at the index and exports the whole index as a CSV file.
Stephan Livera:
That’s really cool. So we’re going to get, hopefully we’re going to have these features and be able to run the numbers in additional ways. So I guess, Pierre, from your perspective, is there anything you would like to add? Is there, and also, is there anything you’d like to see in terms of future directions here?
Pierre Rochard:
Yeah. you know, I think that if we look at it from the audit perspective, what I’m seeing is the audit risk go down, right? Because we’ve a full and complete explanation of all the differences between the issuance and the current ledger balance. And on top of that, we’re adding internal controls within the system to make sure that, you know, we continue to have the right consensus, right. That we’re following the right consensus rules and that there aren’t any bugs in any implementations. So yeah, this is all very good and exciting for Bitcoin and continues to kind of prove that Bitcoin is unique in that it is provable scarcity, right. That you can run your own node and you can prove yourself without having to trust even, and really, I argue because of my reconciliation, I’m not even trusting the developers because I’m recalculating issuance myself. And then in pandas joining that with coin stats index. So even from that perspective, I feel like this is a pretty trust minimized audit of the ledger. So yeah, I think this is also probably the first time anyone’s ever done it. Like what we’re talking about you know, maybe outside of proprietary data providers.
Stephan Livera:
Yeah, for sure. Well look, I think this has been a very educational episode. I think it’s a good point to wrap it here, but of course, before we let you gentlemen go, where can we find out more about you? So let’s start with you Pierre.
Speaker 3:
Yeah. So go to pierrerochard.com and sign up for my email newsletter where I’ll sporadically send you things and go on Twitter, go subscribe or follow or whatever to @pierre_rochard.
Stephan Livera:
Excellent. And Fabian for any listeners who would like to support you, whether that’s github or Bitcoin donations, or they’d like to follow you, what’s the best place for that?
Fabian Jahr:
Yeah, I just set up get up donation donation site. So should hopefully be approved by the time this airs. So my, github user name is FJahr. You can also find me on Twitter, but I’m honestly not writing much. I think it’s better to maybe check out my github contributions and otherwise you can find me an IRC in the typical technical channels. And yeah. Just maybe you briefly mentioned it already, but my blog also doesn’t have a lot of action, but if you were interested in, what we talked about and you just want to kind of see it again in a little more detail, a little more overview you can go to my blog, which is also linked from my github.
Fabian Jahr:
And there is the article where are the coins and this yeah, I just, I put in a lot of work and I hope it helps people to kind of see that. And particularly I hope it’s helpful for people that are interested in getting more technical and diving that little bit deeper that I didn’t when I was at the beginning. So I have been linking to two parts of the code where specific parts are implemented and so that you can go in and kind of make that connection as well.
Stephan Livera:
Excellent. Well, thank you both for joining me on the show today.
Pierre Rochard:
Thanks for having us Stephan.
Fabian Jahr:
Thank you much.