In this episode, Stephan Livera discusses the latest developments in Bitcoin Core with The Charlatan, focusing on the significant updates in version 30, including the removal of the legacy wallet, preparations for the Great Consensus Cleanup, and the introduction of the Bitcoin Kernel project.
The conversation also covers the implications of alternative implementations, the ongoing data carrier size controversy, evolving fee rate policies, and improvements in initial block download times. Additionally, they touch on the future of the Bitcoin Core GUI and the importance of competition in the Bitcoin ecosystem.
Takeaways:
🔸Bitcoin Core version 30 introduces significant updates, including a new mining IPC interface.
🔸The legacy wallet is being removed to reduce technical debt and improve code maintenance.
🔸Preparations for the Great Consensus Cleanup are underway to address long-standing bugs.
🔸The Bitcoin Kernel project aims to separate consensus code for better modularity.
🔸Alternative implementations of Bitcoin are encouraged to foster competition and innovation.
🔸The data carrier size controversy highlights the balance between spam prevention and network performance.
🔸Fee rate policies are evolving in response to market dynamics and miner behavior.
🔸Improvements in initial block download times are ongoing, enhancing user experience.
🔸Address indexing features are being considered to improve wallet usability.
🔸The Bitcoin Core GUI is undergoing updates to better serve user needs.
Timestamps:
(00:00) – Intro
(01:30) – Major updates in Bitcoin Core version 30.0
(04:58) – Why is the legacy wallet being removed?
(09:07) – The preparation for the Great Consensus Cleanup
(13:25) – What is the libbitcoinkernel project?
(17:36) – Alternate Bitcoin node implementations
(23:46) – Sponsors
(25:09) – What is the future of competing Bitcoin clients?
(31:56) – Charlatan’s views on controversies surrounding Data carrier size
(37:50) – Is Bitcoin Core serving the interests of few?
(45:45) – What is the impact of Fee Rate Policies?
(53:00) – Dust limits & DOS concerns
(54:45) – What are the enhancements for Lightning Network transactions?
(57:16) – How bad could mining centralization get if left unchecked?; Libre relay and Transaction filtering
(1:03:00) – What are the concerns of illegal content in Bitcoin?
(1:05:45) – What is the most underappreciated aspect of Bitcoin development?
(1:11:10) – Reducing dependencies in Bitcoin Core
(1:14:00) – What are the possible future developments in Bitcoin Core?
(1:22:42) – What will be the role of competition in Bitcoin implementations?
(1:26:00) – Address indexing & Wallet improvements
(1:30:08) – Closing thoughts
Links:
Sponsor:
- CoinKite.com (code LIVERA)
Stephan Livera links:
- Follow me on X: @stephanlivera
- Subscribe to the podcast
- Subscribe to Substack
Transcript:
Stephan Livera (00:00)
Hi everyone, welcome back to Stephan Livera podcast. Today we’re going to be discussing a range of things on Bitcoin Core development. Of course, we’ll be handling some of the Core and Nuts stuff as well as what’s coming new that’s in Bitcoin Core version 30, as well as some of the discussions around the Bitcoin kernel. So joining me today is the Charlotten. The Charlotten, as I understand, I mean, maybe you can correct me, but as I understand, you sort of had a background in the Bitcoin industry, working in like hardware wallets and
Nowadays, obviously, you’re more focused on Bitcoin core development. first off, welcome to the show.
TheCharlatan (00:36)
Yeah, hi. And I think that sounded about right on how I got into Bitcoin development. Yeah.
Stephan Livera (00:46)
Fantastic. So look, let’s start with some of the big updates. ⁓ From your perspective, what are some of the big important or interesting updates on Bitcoin Core version 30?
TheCharlatan (00:55)
Yeah, so I guess outside of the controversy around the policy changes, there are a few pretty big updates in Bitcoin Core 30 that are a pretty major change to how we plan to ship features in the future and how we plan to structure the project in the future.
The biggest one among those is the new mining IPC interface. So this is a new ⁓ interface that users and other developers can interact with. And it is designed specifically to be useful for Stratton V2 mining clients.
And this has been developed now over the past year or so and then released a couple of weeks ago.
The Stratum V2 software can now interface with Bitcoin Core directly through this mining interface. And I’m currently even running my own little home miner through this new interface with Core V30.
Stephan Livera (02:20)
Great, and so that then means, I guess, the pools that support SV2. So I presume then mainly Demand Pool has full SV2 support, and I believe the Brains Pool also has it as well. So ⁓ I guess that’s the implication of this, right? What are the main implications that it’s making it easier for people to do stratum V2 mining with Bitcoin Core V30?
TheCharlatan (02:44)
Yeah, exactly. Or it’s even making it possible in like the same way.
Stephan Livera (02:52)
So can
you elaborate a bit on that? Like why would it have been harder without this?
TheCharlatan (02:56)
Yeah, so we had some limitations in the way that these proof of work jobs or block templates are retrieved through Bitcoin Core. And the way that Stratumv2 would have required it, you would have had to restart your Bitcoin Core node in order to reconfigure some of the template parameters every time.
This new interface which allows this kind of bi-directional communication allows you to set these settings like on the fly however you want to.
Stephan Livera (03:39)
Okay, and can you just explain for us a little bit how this interfaces with mining software that you may run? Like so the user is gonna run Bitcoin Core V30 plus some other kind of mining software, right?
TheCharlatan (03:51)
Yeah, exactly. That’s how it’s going to look like. I mean, in the end, it’s going to be similar to ⁓ running a, I’m not sure what exactly they call it, but the Datum job declarator. And I think in the future, it’s just going to be, you run Bitcoin Core and you run the extra mining software that you need. And you can just point that.
your either mining farm or home miner, whatever the setup is that you have.
Stephan Livera (04:25)
Great. ⁓ And then ⁓ also some other updates in the Core 30. I think there’s some discussion there about removing the legacy wallet. So can you just explain what was the legacy wallet? Why is it being removed?
TheCharlatan (04:38)
Yeah. So the legacy wallet is the original wallet that was shipped since the earliest version of Bitcoin Core. And the big downside of it was that it was basically just a bag of keys. So there wasn’t any hierarchy within it, no BIP32 devaration or anything like that.
And it was over time, a major pain to maintain. So because we adapted things like descriptors and bit 32, well, in the future probably also.
even better support for mini script descriptors and things like that. It’s been really difficult to port those features back to this original wallet architecture. And so it was decided like five, six years ago already to eventually remove this legacy wallet code.
and instead provides a way to upgrade these legacy wallets to the full modern descriptor wallet format.
And this upgrade has now finally been shipped in v30. And it should be pretty seamless for any users of the legacy wallet. So you can, with either tool or…
some commands in the GUI ⁓ upgrade from the legacy wallet to the new descriptor format.
Stephan Livera (06:26)
And so what would you say the main rationale here is, is it just to simplify the code base or reduce the technical debt from the developer’s standpoint? ⁓ Is it just seen as like, that’s a really old school wallet. We need everyone to upgrade to the new stuff. What’s the thinking there?
TheCharlatan (06:42)
Yeah, this is all about reducing the technical debt. So if you look at our issue tracker on GitHub and you search for the term legacy wallet, there’s just an endless stream of issues that affected it. And most of these issues were fixed in the modern wallet, but were really difficult to address properly inside of the legacy wallet architecture. So
Yeah, the rational decision was just to deprecate it and focus on the new architecture instead of just trying to continuously patch mistakes over and over again. Yeah.
Stephan Livera (07:26)
I And I guess a common discussion that I see, obviously I’m sure you see this all the time, is when people are discussing proposals for things, they’re sort of saying, no, that’s a bad idea because that idea might be confiscatory, right? That it might lock someone out from coins that they held from years ago or they had it in some specific use case. ⁓ And so I guess in this case, it’s not confiscatory because you’re providing this upgrade pathway for people who are still sitting on the legacy wallet. Is that, have I understood it correctly there?
TheCharlatan (07:54)
Yeah, exactly.
Yeah. And we spend a lot of time going through all the possible wallet topologies, let’s call them, that people might have had in the legacy wallets. And we ended up finding even more bugs in the legacy wallets through this process. So we’re fairly confident that
the migration will cover all past use cases of the legacy wallet.
Stephan Livera (08:29)
I see, and guess worst case people can still use old versions but then you’re running into the issue of maybe not having all the latest security updates and things.
TheCharlatan (08:32)
Yes.
Stephan Livera (08:36)
Okay, so ⁓ I noticed you’re also telling me about, just offline, about ⁓ preparation for the Great Consensus Cleanup. So what’s going on with that?
TheCharlatan (08:48)
Yeah, so we started with that in version 29 already. And the idea behind that is we adapt our software outside of the consensus rules already in a way that mimics as if the consensus cleanup would already be deployed. So one of the things ⁓
that was done in version 29 was we make it impossible to generate a block template that exploits the time warp bug through our software. And now in version 30, we added a policy rule that already enforces some stricter limits for the amount of operations that allowed during transaction.
validation and these director limits are part of the currently proposed consensus cleanup features.
And I think in terms of how important these pre cleanups are, I’d say they are both key to getting the great consensus cleanup deployed at some point in the future.
Stephan Livera (10:16)
Okay, and I guess just for listeners who are coming new and don’t really know what that is, I have an early episode with Antoine Ponceau, not the recent one, the one back before where we talked in more detail about the great consensus cleanup. The simple, I guess just for, know, simple people, just explained in simple terms for people who aren’t deep into it, is that there are these long-standing bugs. One is known as a time warp bug, and I think some others relate to kind of like DOSing the node, like DOS transactions or blocks that can like take it down.
And the idea is this great consensus cleanup is a soft fork idea that would patch some of these things. And it’s seen as like these are, I would say, as my understanding is these are relatively uncontroversial ⁓ in terms of developer circles. Most people accept that, it would be a good thing to do this to kind of patch these bugs. But it’s sort of a question of when will there be kind of, you know, all the work done ready to actually do it.
TheCharlatan (11:10)
Yeah, obviously there’s also always the timeline question in terms of getting the soft fork deployed.
Stephan Livera (11:17)
I see. And so then what you’re saying is some of this prep work is there to, I guess, make the nodes maybe a little bit more protected against some of the failure cases in the consensus cleanup. And as I understand, some of these details are, let’s say, private because you obviously don’t want to broadcast out, hey, here’s how you can like take down a bunch of nodes. ⁓ But at the same time, you have to still work in public, right?
TheCharlatan (11:46)
Yeah, exactly. So some of them are to protect nodes against some of the exploits that we’ve identified and tried to patch in the cleanup. And some of them are also to ensure that if miners apply the rules in the future, that they won’t end up generating invalid block templates for themselves.
Stephan Livera (12:11)
Gotcha. so with that also, I guess what you’re saying there is it also helps stop like an accidental chain fork because if you know, they don’t, or is it more just that they would generate just an invalid block?
TheCharlatan (12:25)
Yeah, mean…
I don’t think that the danger here would be a chain fork. That would probably require a bit more work on the bad side, but it would mean that ⁓ that mining operation would lose the subsidy for that embedded block.
Stephan Livera (12:43)
I see. Yeah. Yeah. And I think there’s like maybe one or two famous cases where I think a minor put out like they won the block but then it was invalid because of some other thing and they lost the reward basically. So, ⁓ okay. ⁓ So, yeah, let’s talk a little bit about ⁓ this ⁓ Bitcoin kernel project because I think that’s like a big as I understand you’re the leader for that or you’re the captain of that one.
⁓ Can you give us a bit of an overview there and I guess first just December? What’s the word disambiguate because there’s lib bitcoin which is a Like an alternate Bitcoin implementation by like Eric Voskow and stuff like that But this is a separate project is called lib bitcoin kernel. So can you just explain that for listeners?
TheCharlatan (13:32)
Yeah, so I’m, as you said, like the project captain for the Bitcoin kernel. And I try to call it the Bitcoin kernel now or the Bitcoin core kernel, just to disambiguate it a bit. There is no relation between the two projects. And I am also not responsible for the naming collision. So I’m not sure how that exactly happens.
Stephan Livera (14:02)
Yeah, okay, so can you just give us an idea? What is this? Like, my simple understanding is the idea is to sort of separate consensus code away from some of these other things so that those other things can be developed separately.
TheCharlatan (14:17)
Yeah. So I guess there’s two big ideas behind the project. One is that we internally isolate the consensus rules within Bitcoin Core from all the rest of the project. And this has been a long standing goal for the project since more than a decade now. So, Joshi’s code started as this big
massive blob of logic where it’s like not clear what exactly belongs to the wallet, what is consensus code, which code is used for powering the GUI. And over the past decade, we’ve done a lot of work to isolate that properly. And I think we’re at the point now where the consensus code itself is fairly isolated.
And we’ve now moved more towards what I would call stage two of the project, which is exposing the consensus logic as a reusable piece of code for other projects that want to do something with consensus, be it validate transactions or blocks, or even use it to
reads Bitcoin cores, ⁓ internal data structures.
Stephan Livera (15:46)
Okay, so can you help me understand what that would mean? Is that like a lightning implementation could call that to understand is this valid by bitcoins rules or like how would that work?
TheCharlatan (15:57)
Yeah, exactly. So lightning implementation, for example, can pass it a transaction and the outputs that that transaction consumes, pass it to a function in the library, let’s call it transaction verify or something, and then get the full confidence that the transaction is validated against the exact same rules that the rest of the network runs on.
Stephan Livera (16:23)
I see, maybe more controversial or whatever, but I’m sure you remember there was that famous incident, or maybe two incidents with, I believe, BTCD where there was like a lightning, it was like some of the lightning nodes were using BTCD and then someone had sent, I think Barack had sent like a 998 of 999 multi-seg or something ridiculous and like kind of falls, basically you get what I’m asking, like this kind of.
thing would help stop that example. Is that what you’re saying?
TheCharlatan (16:55)
Yeah, exactly. Yes.
Stephan Livera (16:58)
And so,
okay, so tell us a little bit more about what else, like so we said lightning is one example, what else, what other pieces of software might make use of this?
TheCharlatan (17:13)
Yeah, so the other big one is alternative node implementations. So you can reuse this kernel library now and write your own full node implementation with whatever features you want to. And so far based on the experimental interface that I’ve authored over the past year,
Stephan Livera (17:19)
Okay.
TheCharlatan (17:42)
I’ve developed my own Rust toy node against it. So I’m also a full node developer, I guess, next to a kernel library developer. But it’s also been tested out by Floresco, which is a UTREXO implementation.
Stephan Livera (18:01)
And aren’t they also based on BGCD?
TheCharlatan (18:03)
No, that’s another UTXO implementation. So there’s two big UTXO or fully fleshed out UTXO implementations. One is Floressta, which they plan to base on the kernel library. And the other one is, I think, UTXOD, they call it. And that’s going to be based on BNC.
Stephan Livera (18:10)
okay.
okay. Gotcha. Gotcha. Yeah.
Sorry. I confused that. Okay. Gotcha. And Flores does an interesting one as well. I know I’ve met the developer briefly. ⁓ Okay, so basically, the point is, you’re getting at this idea that it’s easier now for an alternate. ⁓ It’s easier now to create an alternate Bitcoin implementation. And then for that implementation to have, let’s say, quote, unquote, confidence that
TheCharlatan (18:28)
So yeah, that’s.
Stephan Livera (18:52)
I’m still in consensus with Bitcoin Core.
TheCharlatan (18:54)
Yeah, that’s the big goal here. Exactly.
Stephan Livera (18:59)
Okay, and so then it makes it more viable then to have what some people are asking for, which is multiple implementations in Bitcoin. Now, yes, there already are some, obviously, there’s Bitcoin Core, Bitcoin Nauts, Libre Relay, BTCD, maybe a few other ones. But ⁓ I guess the point is making it viable for more to exist.
TheCharlatan (19:18)
Yeah. And beyond just making it viable, one of the goals that I personally have is also removing the Bitcoin core specific idiosyncrasies out of the library. So you really get only the consensus rules and you can choose to have your own data model. Like you can run it with UTreeXO for example.
or you can do some crazy indexing logic similar to what the Bitcoin is trying to do. And this is another project that has recently been tested out on it. You come up with alternative ways to sync up a node faster. And this has been attempted in the Swift sync project. And they’ve also
developed a prototype node now based on the library.
Stephan Livera (20:24)
Okay. Now, I’m curious because from different people I’ve spoken to and just, I guess things I’ve heard, some people have said this kind of project, it might be really hard to do, right, to actually separate it out because ⁓ there might be very, very subtle ways in which Bitcoin Core effectively, you know, that there is no standard or there is no spec for Bitcoin per se. And that people that’s why so many people just run Bitcoin Core. So are you
kind of going against that in a way, like you’re trying to do something that a lot of people are saying is really hard to do, or how are you viewing it? Can you explain for us?
TheCharlatan (21:00)
Yeah, so that’s obviously something ⁓ that I really need to take into consideration when working on the library. And the way I’ve been dealing with that problem so far is that I’m trying to let any change that is required to make the library viable flow into the Bitcoin Core code base itself.
in order to ensure that whatever’s in the library doesn’t deviate from how Core does the same logic. And if we keep on doing that and really sticking to that approach, I think we can mitigate that concern as best as possible.
Stephan Livera (21:52)
to see. Now I guess the other side of that could just be, well now are you kind of expecting other implementations to run them plus Bitcoin Core at the same time? Is that effectively what it ends up being?
TheCharlatan (22:02)
I mean, I don’t expect anybody to do anything. Yeah. Right. This is just about.
We shouldn’t ⁓ keep this logic just to ourselves. We should allow our clients to build on it as far as possible. And it might even be useful, right? If you can integrate the kernel library alongside your own consensus re-implementation to test it and find bugs with your own implementation. And I think that’s an angle.
that can be sold to pretty much every other implementation out there, because it just strictly improves their robustness.
Stephan Livera (22:48)
I so I guess, again, I’m not a developer, I’m just an ADIQ podcaster, but maybe it’s like, it might also help them improve their own testing suite and their own kind of ways of making sure they’re still in, let’s say, meeting with Bitcoin Core in terms of how things work. And so, just to be clear, this is not, I think I saw in your blog post you mentioned you’re not doing P2P, so like,
This is just consensus, it’s not relating to how the nodes actually talk to each other.
TheCharlatan (23:20)
No, and at least in the interface of the library, it doesn’t include the mempool or policy either.
Stephan Livera (23:30)
Okay, yeah. And so I guess, yeah, like a lot of the kind of even a lot of the arguments about, you know, policy and consensus and so on. I guess it’s very focused on, you know, what is Bitcoin Core doing because Bitcoin Core is the default in a world where there are lots of different implementations and Bitcoin Core doesn’t have like such high dominance. How do you make sure all the P2P, all the different nodes?
like still talk to each other correctly? what is, is there a standard way for that to happen?
TheCharlatan (24:04)
I mean, I guess down the road we could do the same with our peer-to-peer codes and have a Bitcoin core peer-to-peer library that does something similar. But yeah, we’re still far away from that. I think we’ve done a decent job at isolating the consensus codes, but the peer-to-peer code is a different story still.
Stephan Livera (24:24)
Okay, so yeah.
Okay, yeah. And so where is this project at so far? like, what’s your kind of, do you have like an idea of like how long it’s going to take?
TheCharlatan (24:44)
Yeah, so we are trying to ship the interface or the library in the next version. So in v31, and it won’t include all the bells and whistles that you might expect from the fully featured consensus library. But the idea is to get something in and then incrementally improve it over time because
Just doing everything all at once is a recipe for failure. If you’re trying to get a project done in Bitcoin Core, the review cycle is brutal in that respect. You have to chunk things up and take them in small pieces, which is also good, right? Because we want the library to be mature, well thought through and yeah, well designed.
Stephan Livera (25:18)
Mm-hmm.
Okay, and so then as you’re saying the hope then is that I mean people say you want lots of eyes on the code you ideally want other people building on top of it you ideally want ⁓ as we spoke about like examples like maybe lightning implementations or I don’t know maybe arc or spark or some other L2 like maybe some of them would start actually using this to check on the Bitcoin side are there transactions you know aligned with Bitcoin Core is that how you’re seeing it?
TheCharlatan (26:10)
Yeah, that’s pretty much exactly it. ⁓ One other feature that I briefly wanted to touch on is if we manage to isolate the logic in a way where it doesn’t require any system specific resources, so stuff like database management, file management, ⁓ having to switch between
different CPU threads.
If we get it to that point where all that stuff is no longer required, then we can start doing some pretty interesting stuff. So at that point you could start validating blocks just purely in memory without any system requirements. So you could do it on embedded devices or very restricted environments like in
secure enclaves and stuff like Intel SGX environments. And the other interesting thing that I’m also looking at there is enabling its usage within proof generating virtual machines. So there are these techniques now to do zero knowledge proofs of arbitrary programs.
But these proofs obviously require that the program is barely restricted in itself. we needed to not use any system resources in order to get there. And the reason this is interesting is because you can then do proofs of the entire chain’s validity inside the Serenology Proof Generator.
and then publish a small proof that some specific chain state is valid up to a million or something because it’s going to be in the future.
And this might allow a different type of node where you wouldn’t have to be downloading the entire history anymore. You can just take this proof and apply it, validate it, and then get assurances that up to that height, the chain is valid.
Stephan Livera (28:48)
All right, this is like like zero sync with Robin Linus and that’s this kind of idea or related.
TheCharlatan (28:54)
Yeah, it’s kind of related to that.
Stephan Livera (28:56)
Okay, yeah, and so is it just you working on this or are there other people kind of also chipping away at this?
TheCharlatan (29:07)
Yeah, so I guess it’s been…
It’s been mostly me over the past two years or something, but that has changed a lot over the past few months. So there’s been a big influx of additional developers working on the project. And as far as I can tell, this happened pretty organically. Like I didn’t do a lot of publicity or anything. So it seems like other people think that this project is.
a good idea, though I’m basing their time into it.
Stephan Livera (29:42)
Yeah, and actually, I mean, we should probably talk about that as well. if we zoom out a bit, why is it important that this is done from your perspective?
TheCharlatan (29:51)
So I kind of like the future where we have different competing clients that run on this shared consensus code or this kernel. And I really do think that we should have competition between nodes, right? If a node implementation doesn’t serve your interests for how you want to use Bitcoin.
then you should be able to either code up a decent alternative or run another alternative that serves your interests better.
Stephan Livera (30:29)
Yeah, okay. I think that’s probably going to be a popular statement in today’s climate. ⁓ So I guess one other thing with like running with, yeah, I guess how I’m understanding it then is the work you’re doing helps make it so that someone else can create a competing implementation like in the future, like once this is done. ⁓ Now, ⁓ when it comes to, I guess, ⁓ the current
controversy around all this, you know, data carrier size and things. Well, maybe you want to just give us a bit of an overview there. Can you just give us like your sort of high level analysis or view of what’s going on with this data carrier size thing and the controversy that’s going on nowadays?
TheCharlatan (31:15)
Yeah, that’s going to be tough to do in a few minutes, I guess, because of all the history that has been discussed at length by a lot of people over the past few months.
To me it comes down to…
not wanting to give scammers a quarter in Bitcoin. And I think that’s obviously a desirable thing. We don’t scam projects using Bitcoin and tarnishing its reputation. But at the same time, policing this kind of activity aggressively does have
big impacts on both the performance of the network and its long-term viability. If you want to build the world’s largest money, you can’t require the entire network to update once a year to the numerous anti-spam filters. And I think those two goals are kind of at odds.
And reconciling both of these perspectives with each other is difficult.
Stephan Livera (32:37)
So.
Now, what about in the case, okay, so what would that look like if, you you were to just hypothetically to walk down that filter pathway of more very rapid, let’s say spam filtering. Do you think that there’s a deterrence effect of implementing more filters against spam?
TheCharlatan (33:02)
I think there might be a small deterrence, but if we look back at the history of the Bitcoin project, we can also see that these kinds of projects will just move to new ways of embedding their data if we prohibit them with some other mechanism. So when the opreturn ⁓
policy was initially loosened, well, from zero bytes to I think 40 bytes it was.
Stephan Livera (33:37)
back in like 2014 for 0.9 of Bitcoin.
TheCharlatan (33:39)
Yeah,
very long ago. ⁓
It was hoped that some of the meta protocols which were being discussed back then, so stuff like MasterCoin and counterparty would move to these operator outputs, but they weren’t permissive enough for the use cases that they wanted to use them for. So instead those projects launched their protocols on what
is commonly called a fake key output. And what that does is it embeds the data that they need in order to build their, let’s say, asset overlay into these fake outputs. And by that example, it seems clear to me that if
these kind of projects don’t have the option to embed their data in, let’s say, a harm minimizing way. They will do so in a harmful way instead, and it’s not going to deter them at all, or minimally deter them. So in 2015, people who were
around back then might remember, we had a brief, um, like counterparty bull run, I would call it or counterparty hype cycle is probably the correct term for it where similarly to the BRC 20 minutes, um, last year, was a big hype around launching new kinds of token projects and whatever on counterparty and
Yeah, those outputs are now around forever and you have to lock them around in our UTXO set forever. And we probably could have…
not have those outputs around forever if the operator carrier size was relaxed a bit earlier.
Stephan Livera (35:58)
And as I understand, Opriton has actually, I mean, you correct me if I’m wrong, but my understanding is Opriton has actually existed since like the early days of Bitcoin, but it wasn’t standard. And then so what happened in 2014 is that it got made standard and it was seen as like, okay, well, this is the way that we can have a kind of a garbage bin for data kind of in the harm minimizing context. But my understanding is it was consensus valid like from day one.
TheCharlatan (36:26)
Yes. Yeah.
Stephan Livera (36:27)
Okay, and then, so there’s been a lot of arguments in and out and back and forth around this issue. So I guess we should address or explore some of those. I guess walking back to Antoine’s email and rationale, was, you know, now the Boogeyman, which is Setreya.
Right? I think a lot of people in the community are seeing this as like, Bitcoin Core is just bending over for either spammers or is sort of serving the interest of this particular company or I think the company is called Janeway. Anyway, Citrae and the company behind it that Bitcoin Core is serving just only them. Can you explain your perspective on this? Whether it’s serving, you know, just one company.
or one particular niche case ⁓ and, quote unquote, opening the door for more spam.
TheCharlatan (37:26)
I don’t think anything about that is true in the remotest. So I have zero indication that Bitcoin Core changes policy to accommodate for the product of a certain startup or VC funded company. ⁓ So Dray is going to do their thing no matter what. I think the protocol that
I’m not even sure if they launched or not at the current moment. I think they had like a pre-launch thing, but the protocol that they’re rolling out is not even using operator, right? So they’re doing their thing no matter what, which yeah, to me echoes past behavior of these overlay protocols that want to embed data.
Stephan Livera (38:11)
Yes.
TheCharlatan (38:23)
They’re going to do it anyway. They’re going to embed data in these fakey outputs. It’s really hard to stop data in fakey outputs.
But then to roll back to ⁓ the data carrier size, let’s say proposal that Antoine made back in April. Trea was mentioned there because it was yet another example of a protocol embedding data in outputs. And that’s all the relation there was to Sotrea throughout this entire thing.
Stephan Livera (38:37)
Now, y’all going?
TheCharlatan (39:07)
So it’s just an example of another protocol or another party doing the thing that is, yeah, just more harmful than any kind of other data embedding that you could do.
Stephan Livera (39:20)
Yeah, and now my understanding in this case is Citreo as a I think it’s called the Clementine or Clementine I don’t know pronounce it. I think it’s a bit VM bridge and the idea is it’s a one-of-an-honesty trust assumption and my understanding Someone correct me if I’m getting it wrong But my understanding is they will be doing basically inscriptions to put some data into the chain and then what we’re talking about here is actually like they’re kind of a fraud challenge case or it’s kind of loosely analogous to a lightning justice transaction and then the idea is
they would, they have this theoretical challenge case that without the OP return ⁓ limit increase, they would have done fake pub keys ⁓ which go into UT, in the UTXO set and not relayed anyway. And hypothetically with the OP return change, if they do change their own protocol, they could use OP return instead. Now, I think one other criticism I’ve heard here is this idea that, well, actually the real specific thing that they’re getting is
the relay network of Bitcoin. And I think this is another area where people felt like, ⁓ it’s my node, I shouldn’t have to relay ⁓ other people’s stuff if I don’t want to. ⁓ And that was the concern that I heard that I’ve seen from some people. ⁓ And so do you want to answer that or speak to that?
TheCharlatan (40:40)
Yeah. So that’s why we have the data carrier size option in Bitcoin Core. You can set it to zero. ⁓ I set policy options on my nodes too. I set permit bare multi-seg equals zero because I don’t want to do that either. And I don’t want to give it any credence, but we also have to be realistic to ourselves that this is mostly a signal.
to the network, I guess, that some participants ⁓ inside the peer-to-peer network don’t like this type of usage. But it doesn’t really do much in terms of prohibiting the actual usage if that policy rule is not adapted by everyone participating.
Stephan Livera (41:32)
Yeah, I see. And so then there’s been a lot of talk about, I guess,
I think, I’m just trying to think of the right way to explain this. I think, at least my interpretation of this is that it’s kind of like, there’s been some changes kind of on the network, in the, the network conditions. And in some sense, it’s like just attempting to match ⁓ that so that individual users can still do the right, they can have their fee estimation, compact blocks. ⁓
you know, are still being able to have a high rate for compact block reconstruction. And the other argument I’ve heard is also around not encouraging direct to minor submission, like the Mara slipstreams of the world. ⁓ And so my interpretation, now I understand for some listeners that’s controversial or whatever, they see that as like you’re opening up for spam, whatever. ⁓ But I guess my interpretation, at least that’s my, you know,
my ADIQ podcaster reading of the situation. Can you explain, like if we’re zooming out, what’s going on? Do you think those other things, so I guess let me put the question to you this way. I think there are some of these social and economic and technical phenomena that have been occurring. So things like the ordinals phenomenon, ⁓ liberal relay, ⁓ bit VM, like the fact that bit VM bridging is a thing now. I think these are things that have shifted over time.
And I guess zooming out the way I’m interpreting it is Bitcoin Core is sort of trying to respond to some of those things. How are you seeing that? Would you agree or disagree or how are you viewing it?
TheCharlatan (43:09)
I wouldn’t say that Bitcoin Core is responding to, let’s say, BitVM exactly, or…
Yeah, similar projects doing similar things. But.
What we try to do in the project is ship a node that gives performance, ⁓ gives miners decent block templates out of the box, and gives Lightning Network nodes good fee estimations so that they can do either penalty transactions or they force close in a reasonable amount of time.
And both of these are really important things to keep Bitcoin and its L2 Lightning running in a stable way. And compromising on these two things to try to combat certain use of the chain is dangerous. And I think about President Tusset.
Like we don’t want to compromise the performance of protocols and users that are money depending on us to send ⁓ a message to other people transacting on the chain.
Stephan Livera (44:38)
Okay, and so I think there have been a lot of arguments back and forth on let’s say the the amount to which those things are true, right? So for example, they may say the impact on fee estimation, maybe it’s not that big a deal for them or ⁓ maybe there are disagreements on exactly how much mining centralization is being driven by this and is this specific operator and change really going to result in saving a lot of these ⁓
UTXOs that would have otherwise been created.
TheCharlatan (45:11)
Yeah, so we’re trying to mitigate the bad effects that come or the bad side effects that come from applying these policy rules. So we’re working on ways to get good fee estimation and get decent block relay, even if you apply aggressive filtering. And I think if
we can come up with neat ways to do that, then it might be time again to revisit this conversation again, because we don’t have these technical downsides that might jeopardize some of the operations of our users.
Stephan Livera (45:58)
I see. So you see it like it could actually make sense to lower the default ⁓ back down, let’s say hypothetically, if the ⁓ compact blocks, something could be done to improve that, that you, you, it might make sense to actually lower the data carrier size default that call comes out with.
TheCharlatan (46:18)
yeah, I’m not sure about the data carrier size itself, but it would definitely make it more feasible for people to run their own policy and have all the policy knobs that they want to and not get jeopardized by bad fees and bad relay performance.
Stephan Livera (46:22)
Right. You’re thinking more about the fee rate policies.
Okay. ⁓ And so let’s talk a little bit about the fee rate aspects of it as well, because I think there’s been some movement there on is it mean fee rate policies? Can you tell us a little bit about that? What it will first of all, just for listeners, I don’t know what is that and then explain a little bit about the change there.
TheCharlatan (46:57)
Yeah, so there is something called the minimum fee rates option. And this basically dictates what the minimum fee that a transaction has to pay to a miner in order for the transaction to be accepted into a Nozman pool. And this has been set to one Satoshi virtual byte for a long time.
And that has been lowered now in version 30 of Bitcoin Core in reaction to a majority of miners starting to include sub 1 Satoshi per Vbyte V-rate transactions. Yes.
Stephan Livera (47:45)
colloquially known as subsat summer, I
would argue is kind one of those social phenomena that has kind of happened. And I think maybe this is an example of reacting to what’s happening on the network.
TheCharlatan (47:58)
Yeah, yeah, it certainly is because the block relay performance of nodes degraded in a serious way because they weren’t relaying these subset V-rate transactions and had to then request them from peers when they arrived within a compact block.
and
Yeah, this has also been a controversial change. I don’t really follow along the reasoning for why this is controversial. I’m just happy that I stack even harder and not pay more fees for my Lightning Channel opens.
Stephan Livera (48:46)
Right, yeah, so I guess previously it was kind of, I guess loosely speaking, it was kind of understood, ⁓ just one sat per Vbyte is basically the minimum, and then all of a sudden, I think it kind of got memed into existence as a bit of a social phenomena. hey, what if you went below that? And think Mononaut might have something to do with this, but anyway, a few things went on there. And then, as you said, some of the mining pools lowered the rate at which they would accept…
Now I think some of them lowered it then brought it back up again, then brought it back down and maybe there was a little bit of trying to find the right level or the the sweet spot per se. ⁓ But now it seems like, yes, people are putting in transactions below one sap per V byte, ⁓ which I guess is a change. ⁓ Do you agree or disagree with that aspect of it? And maybe you could just touch on like how low should it go?
TheCharlatan (49:36)
So there definitely should be a flaw somewhere, right? We don’t want to relay transactions that don’t pay anything because then the network would just be, exactly, the network would just be trivially possible because anybody can just create a, ⁓ let’s say, 0.0001 ZappaBuy transaction. So we don’t want to go…
Stephan Livera (49:49)
Because then it’s like trivially docile, right?
TheCharlatan (50:04)
much lower from here I feel. The floor was put in place in the first point as a DOS protection mechanism but this as far as I remember was back when the price was around a thousand dollars. So yeah maybe if the price pumps.
Stephan Livera (50:27)
Yeah, here we are now as we speak it’s $108,000. So obviously,
you know, in Bitcoin terms, ⁓ it’s kind of, it’s come actually the fees people are paying now has come down a lot, but the fiat value has come up like 100x right from what that those days.
TheCharlatan (50:45)
Yes, exactly. So the costs in fiat terms is still higher now to submit a transaction at the very floor fee rate than was when this floor fee rate was put into place in the first place.
Stephan Livera (51:04)
I ⁓ And so I guess, yeah, there’s, I guess there’s some, I guess periodically this has to be updated just like as if we expect number go up, right? Bitcoin is going to keep going up, then ⁓ it’ll have to periodically be adjusted, right? Like to go down, I guess, theoretically.
TheCharlatan (51:20)
I think we’ll probably see a similar thing if the price, yeah, I don’t know if we’ll ever get to those kinds of price levels, but let’s say we see.
Stephan Livera (51:30)
Right, it would have to be
like a million dollars a coin or 10 million dollars a coin or something higher.
TheCharlatan (51:35)
Yeah, I would expect it to play out very similarly as it did now where like one miner just starts accepting them. A bunch of people start broadcasting transactions with even lower fees than a bunch of other miners. ⁓
are forced to do the same thing in order not to lose more fees because the blocks that they are producing will end up empty otherwise. So I think we would see a very similar dynamic play out again.
Stephan Livera (52:08)
Now, what we’ve been talking about so far is min fee rate policies. What about other things that are, let’s say, related? So things like dust limit or other things like kind of in the same area.
TheCharlatan (52:23)
Yeah, we’ve started discussing that over the past few weeks within Bitcoin call of, yeah, let’s say if miners suddenly start including even dustier transactions, what we should be doing. And there’s no consensus within the project at all yet on what to do there. Cause unlike
just lowering the fee rate, this has an actual impact on the long-term system resources because it’s going to make it easier for people to create these dusty outputs.
Stephan Livera (53:06)
And I guess that can also be seen as a DOSable thing, right? Like if you lower that too low, then again, there’s like a DOS concern, right?
TheCharlatan (53:13)
Yeah, exactly. Yes.
Stephan Livera (53:16)
So I guess it’s kind of a sweet spot, let’s say, based on the fiat price ⁓ over time, that maybe the dust limit also has to shift. ⁓ If Bitcoin goes to like a million dollars a coin or 10 million dollars a coin, then maybe the dust limits will change too.
TheCharlatan (53:32)
Yeah, I would expect it to change too eventually. I’m not sure if we’re at the price level to justify such a change yet. Yeah. But I’m honestly also not too keen to lower that because yeah, it might be dangerous.
Stephan Livera (53:41)
Right, it just doesn’t make sense yet. Yeah, okay.
Yeah, I mean the reason I’m asking is just out of curiosity and just because it’s kind of a related area. And then I guess there are some other things that have sort of shifted around like at a peer-to-peer level. I’m not sure how deep into this you are around things like ⁓ truck transactions and 1P1C and things like this. Can you talk to us a little bit about that and how the aim is to improve L2 performance?
TheCharlatan (54:18)
Yeah, so this is something that was already shipped in previous versions. Yeah, I think so. And for Lightning specifically, it allows you to attach a so-called anchor output to your transaction. And this anchor output has a zero amount attached to it. ⁓
Stephan Livera (54:24)
Was it 29? Yeah.
TheCharlatan (54:46)
doesn’t have ⁓ any coins on it. And the idea behind that is when you broadcast that ⁓
When you broadcast your transaction to the wider network, you can attach the fee that is associated with that transaction to that anchor or zero value anchor output. And this means that we don’t have to commit to the fee for that transaction in a Lightning Channel upfront. And we can do
fee calculation dynamically when we actually need it based on what the current state of the mempool and the network is.
Stephan Livera (55:38)
Yeah, I see. so then ⁓ the idea is that it allows it helps, I guess, L twos like lightning, because then people can just like not have to, I guess, estimate what the fee is going to be when they close the channel. And now they can put it in a really low fee and then amend later when they actually need to close the channel as an example is my understanding that is that right? Okay.
TheCharlatan (56:02)
Yeah, yeah.
It just makes everything much easier. Yes.
Stephan Livera (56:08)
I see.
And then as I understand there was also in recent versions, maybe not this version, but I think there was recent work done to help stop like some of the force closing in Lightning.
TheCharlatan (56:17)
I’m not familiar with that.
Stephan Livera (56:19)
Okay. Yeah. Okay. ⁓ so I guess, yeah, I think the other kind of back to the broader sort of not score kind of, ⁓ drama stuff. I’m curious to get your take on the mining centralization side of it. Like how, how bad do think it would be if, left unchecked? ⁓ if, ⁓ as an example, compact block, reconstruction rates were not, ⁓ fixed.
What exactly is the downside? What’s the bad outcome for us here?
TheCharlatan (56:53)
Yeah, so we do want to have an open relay network that has good performance for everybody to make it easier for new players to enter the market. We don’t want a cartel of miners to form that have their own relay rules with each other. So the idea is that we don’t want to entrench the monopoly that might be caused by this.
even further by degrading performance for new entrants into the market.
Stephan Livera (57:28)
Okay. And do you know if there’s any like kind of, I guess, yeah, there’s maybe it’s hard to quantify, but is it possible to quantify like, what is the impact of ⁓ slightly reduced block, compact block ⁓ reconstruction?
TheCharlatan (57:45)
It’s not clear. And I’m of the opinion that it is probably lower than most people would attribute to it. And the one thing I like to point to in that case is a study that one of the mempool.space guys that are in surf, he looked at the amount of orphan blocks that happens during
this initial phase of subset V-rate transaction rollout. And what you noticed was that even though the blocks containing these subset V-rate transactions relate lower across the network, there wasn’t a single orphan block that those miners, including the subset V-rate transactions produced.
So it didn’t have an impact on them at all, apparently. And just going by the math of it, we would have expected between one and two such orphans in the time frame we looked at. So it is, yeah, a small indication that the effect is probably lower than some people think it is. But there’s also a whole bunch, yeah.
Stephan Livera (59:10)
Okay.
TheCharlatan (59:13)
of other effects that we can’t really account for because the miners might preferentially peer with each other or they could even have their own network with each other just to relay the blocks. So yeah, it’s hard to estimate what the actual effect is going to be.
Stephan Livera (59:33)
I see. And I guess there are things like like Libre Relay, which of course, for some people is controversial for others, they see it as a censorship resistant thing. ⁓ Can you comment on that? Like, what do you what’s your view on Libre Relay? Do you think it should exist or should not exist? Or is it helping or hurting?
TheCharlatan (59:51)
⁓ I don’t think I really have an opinion on it. ⁓ yeah, I guess it’s just a thing that happens if you have an open adversarial peer to peer network where people can just do things. So, yeah.
Stephan Livera (1:00:09)
Any comment on this ⁓ so-called tolerance minority effect with ⁓ the filters in a policy context that a small number of nodes who are relaying a broader set of transactions, they can more easily get those transactions through to a minor somewhere somehow than ⁓ the ⁓ filter side of that equation, let’s say.
TheCharlatan (1:00:35)
Yeah, so this was observable over the past few months. So even though there weren’t, I think, any…
released or Bitcoin Core released nodes yet on the network that lowered the fee rates by default. It was already possible to just broadcast a transaction from my nodes and have it show up in a block. I think it was 20 minutes later, even or something. So it is enough for transactions to propagate to miners.
if we just have a few of the nodes running these lower phi filters.
Stephan Livera (1:01:22)
And as I understand, even during that time period, Libre Relay was not even going below one SAT per VBite. So it’s not that these people were all running Libre Relay to do it. So it’s like, it’s even more of an example about this tolerant minority case in a specific context, sure. ⁓ But people were getting sub one SAT transactions relayed in mind without using Libre Relay, just on the network.
TheCharlatan (1:01:43)
Yeah.
Yeah, exactly. So people had to go into their Bitcoin core configuration files, lower the minfiraid options and then run the nodes with it. Yeah.
Stephan Livera (1:01:54)
Yeah. And presumably enough people did that, like just went to manually set their fee rate or their setting lower that they would accept that and relay that. And it happened.
TheCharlatan (1:02:06)
Yes.
Stephan Livera (1:02:07)
Yeah, okay. ⁓ I guess, on the whole, I don’t want to get ⁓ banned off, you whatever, but let’s talk, let’s say illegal content. And I hope everyone understands, like, obviously, no one here is in favor of, you know, illegal content on the in Bitcoin. But what was your, I guess, do you have any comment on that about that raising the the up return relay limit? Is that
encouraging or discouraging or is it the same? Do you see it as increased risk of illegal content going on bitcoin or less or the same? How do see that?
TheCharlatan (1:02:45)
I think it’s the same because it’s already trivial. There are already websites that offer you to upload media into an inscription. yeah, OpReturn change doesn’t make that easier.
Stephan Livera (1:03:02)
Yeah. Although I think one counter argument would be that if you went to let’s say Mara slipstream that they might filter on their side, right? Like because they’ve got a website, obviously they’re a public listed company. They very likely run some kind of filter on their side to stop, you know, mining illegal content into the chain. So do you think that changes it? Or do you think of it’s more like, do you think there’s something else here?
TheCharlatan (1:03:25)
I mean, the same would apply to an operator and transaction with legal content inside. Right. But yeah, the, these websites offering these media embedding services already now, like you can just get the transaction from them broadcasted normally on the peer to peer network. then, yeah, it just takes one minor to include it anyway. yeah.
Stephan Livera (1:03:54)
Was there any discussion inside of Bitcoin Core that this would increase the risk about illegal content or was there a back and forth on this or was it more just everyone thought, you can already do this with inscriptions or this kind of thing?
Okay, so it’s kind of seen as like the it’s kind of it’s just the same risk.
Okay, yeah. Yeah, I think that’s probably one of the areas where there’s been a lot of, you know, lot of anger or emotion and engagement about this that it’s raising the risk. ⁓ Different lawyers in the space, I think six of seven polled have said they didn’t believe this increases the risk. ⁓ But I think that they said it’s likely an overblown risk. But I guess if people have that concern and
⁓ It’s worthwhile people at least discussing and talking about how they’re thinking about it.
TheCharlatan (1:04:44)
Yeah, I’m not sure what to comment on that still.
Stephan Livera (1:04:47)
Yeah.
TheCharlatan (1:04:47)
Yeah, I feel it’s completely blown out of proportion.
Stephan Livera (1:04:48)
Yep.
So in terms of where things are going from here, do you have any other comments on ⁓ things people should know about Bitcoin Core or ⁓ I guess, let me ask you this. What do you think is an unknown or underappreciated thing about Bitcoin development?
TheCharlatan (1:05:09)
Yeah, this is probably my favorite thing to talk about ⁓ Bitcoin Core specifically. One of the reasons I became fascinated with the project ⁓ from a very early stage on, and this is Bitcoin Core’s obsession with ensuring that the binaries that are published are as pristine as possible. So we not only audit
our own dependencies. We also ordered some of the tool chains that we use. And we use reproducible builds. And even a step further than just reproducible builds, we use a system called Geeks to produce our binaries. And this system ensures that the compiler that you
The compiler that you are using to compile the code is bootstrapped from a chain of source codes and not from some bootstrapping compiler. And this I think is really key. So
Basically, this goes into something called trust and trust where ⁓ one component in your tool chain eventually is just a binary that you need to trust. You can’t expect the source code and you have to trust that binary not to insert a backdoor into the rest of your bootstrapping process. So you might have reproducible builds and everything set up.
But somewhere down your bootstrap, you have this weird version of, let’s say, a C compiler that is backdoored. And I think for Bitcoin specifically, it is really important to get this right and to solve this problem because we want Bitcoin to be stable for decades. And an attacker can create a backdoor inside such a bootstrapping
chain now and sit on it for the next few decades. And we won’t learn about it until the attack is actually deployed.
And the system that we use in Bitcoin Core solves this by having this kind of bootstrap chain that goes all the way back to a couple hundred of bytes of machine code that can actually be read and understood by a software engineer.
Stephan Livera (1:07:56)
I And so I guess what you’re talking about there is this idea that having stronger confidence that people haven’t inserted bugs or ⁓ vulnerabilities into the code with us unknowing ⁓ because so it my understanding of it that there’s, you know, code is being, you know, people are making pull requests. It’s getting merged into the master. That code is getting, that branch is getting ⁓ compiled and create and because you have to create
I guess the actual binaries, is like, you know, whatever the dot exe or the dot tar gz or whatever these files that we the end users, right? Whether it’s windows, Mac and Linux, we actually run and install on our PCs. But I guess what you’re talking about is that process of how is it ⁓ built and made ready for the user to make sure that it actually is correctly what the code was written as is what is running on your machine. Have I got you there?
TheCharlatan (1:08:54)
Yeah, exactly. And the key beyond that is that you can also fully inspect all the tools used to generate that binary or that blob of machine code instructions. Yes.
Stephan Livera (1:09:12)
I see. Yeah, and so it is also possible, of course, that if you’re like maybe a power user or a developer or an advanced person, you can actually just like manually kind of pull the code together and build it, just compile it and build it yourself. ⁓ But like typical users might not do that. They might just download and install, or they might be using like the typical like Umbrol and Start9 and these node, my node and the node in a box packages.
who are in turn relying on the binaries that the Bitcoin Core team are creating.
TheCharlatan (1:09:46)
Yeah, but so the thing to understand here is even if you use your own tool chain as the developer and you compile it yourself, the tool chain that you’re using might still have a back door.
Stephan Livera (1:10:01)
Right, as in you could
yourself have been pwned somewhere along the line.
TheCharlatan (1:10:06)
No, not even you yourself, but the compiler that you are downloading at some point from the internet might be compromised in some way.
Stephan Livera (1:10:11)
Right, sorry that’s what I meant, yeah, go on, yeah. Yeah.
Okay, yeah, and then so that’s I guess this is kind of getting that’s like really I guess deep into the I the reproducible builds part of this now You also spoke a bit about dependencies and so as I understand the general idea is you want obviously you might have some dependencies on other software projects But the idea is you’re trying to remove unnecessary Dependencies over time and I understand this is something that Bitcoin core generally has been doing Do you want to just elaborate a bit on that and where that’s at nowadays?
TheCharlatan (1:10:46)
Yeah, so that was actually another reason to deprecate, ⁓ sorry, not deprecate, remove the legacy wallet because it used BerkeleyDB.
Stephan Livera (1:10:51)
are the Legacy Wallet.
Alright, this is the
famous 2013 fork, but yeah, go on.
TheCharlatan (1:11:01)
Yeah, that database has been unmaintained for a long time. yeah, writing code against it has been really painful. So a switch was made for the descriptor one, it’s to SQLite, I think four or five years ago now.
But we still had to keep BerkeleyDB as part of our dependency tree and still build it. So removing the legacy wallet allowed us to get rid of BerkeleyDB as a dependency. Yeah.
Stephan Livera (1:11:38)
I see. And yeah, the infamous 2013 ⁓ chain fork. I remember at that time, ⁓ you know, I was new to Bitcoin then and I remember there was like a big chain fork and it was a big deal. And I think Gavin sort of had to come out and I had to get, but I think that was caused by this Berkeley DB thing. And I think it was like, from then I believe Bitcoin moved to what’s called a level DB, right? Is it level DB?
TheCharlatan (1:12:04)
Yeah, exactly. This is also like one of the first Bitcoin incidents that I witnessed. It’s part of my memory too. Let’s put it up.
Stephan Livera (1:12:16)
Yeah.
And my, and I recall kind of even coming back to what we were saying at the start or earlier around consensus rules. I remember listening to a podcast, ⁓ SIPA, Peter Buller, ⁓ on, think it was Chaincode Labs, maybe three or four years ago. can’t remember now exactly, ⁓ where he was commenting that it was like the effect without knowing it. This Berkeley DB thing had effectively become part of the consensus rules of Bitcoin without us knowing it.
And so I guess that’s like another example of how difficult it can be to actually ⁓ separate consensus from the other elements of Bitcoin.
TheCharlatan (1:12:56)
Yeah, exactly. That’s like the one case we continuously remind ourselves of. That was just a big mistake and a mistake we try not to repeat for the kernel work. Yeah.
Stephan Livera (1:13:12)
Okay, so ⁓ looking out ahead, what are some of the things that people should look forward to seeing in Bitcoin Core? ⁓ Just generally.
TheCharlatan (1:13:23)
Yeah, so one of the things I didn’t get into with the mining interface for Stratum V2 is that we built it in a way that uses this new CapnProto interface. And that’s a new way to communicate with Bitcoin Core node.
but it is also a way for different parts of a Bitcoin Core node to be split into separate binaries and then communicate with each other through this CapnProto interface and ⁓ serialization format. So with this change, we now have the possibility of splitting
some of our components into separate binaries. And this has a bunch of downstream effects. having separate binaries that can communicate with each other. So let’s say you have a binary that runs the core node logic. So the peer-to-peer stuff, the mempool and the consensus rules.
And then you have another binary that has the wallets, another binary for the GUI, another one for databases or more indexes or whatever.
And this allows us to have much better isolation between components in our code. So it gives us more incentives to precise our architecture.
But it also has the effect that if you just want to change something in, let’s say the database, you can now just program against this interface and don’t have to change any logic in the node part. Or similarly for the wallet, if you just want to change something in the wallet, you can now just do it against that interface. And yeah, that’s important.
because we can probably relax our review standards a tiny bit for changes that just affect, yeah, a small indexing feature as opposed to a change that affects consensus or the peer-to-peer stack or whatever. And it also allows us to get out of each other’s way a bit.
So we can now focus on the stuff that we actually like to do, like to work on and are not required to, yeah, have to manage all the other changes going on in the rest of the code base. So.
Stephan Livera (1:16:28)
I see. So the
idea is when you are making, ⁓ you know, you’re trying to create a pull request to do some code, your concern might be, I’m working in this one particular area, but actually this code also touches some other area. And so what you’re saying is the idea is to try to separate them so that you can, each person can just focus on his own knitting per se.
TheCharlatan (1:16:46)
Yeah. So this has already gotten much better over the past few years. So these interfaces are already mostly in place where we’re not constantly interfering with each other anymore, but doing the split into separate binaries will entrench this and yeah, ensure that the architecture will hold for the future too. Yes. And
The last thing that this also means is that if there’s a bug in, say the indexing codes or the water codes, and it makes that part of the code crash, the crash doesn’t affect the consensus rules anymore. So you can still keep on going with your transaction validation or relaying blocks to the network, even though
some other part of the stack might be in a bad state.
Stephan Livera (1:17:51)
Yeah, actually, I’m curious if you are familiar because I know there’s some other aspects of call V 30. I don’t know how closely how closely you follow or you’re into those aspects of it. But I heard something about ⁓ like better n at traversal in Bitcoin call V 30. Is that true also?
TheCharlatan (1:18:07)
yeah, so.
⁓ One of the probably longest term contributors to Bitcoin Core and previous maintainer, LamWJ, wrote a replacement for mini-upnp and the mini-upnp library. So we’re back at reducing dependencies again, because this allowed us to remove
our dependence on this external UPnP library, which had produced past bugs for us. And some of them were really bad where you could remotely crash the node.
And this new library gave us some better natural features. And we now decided to switch these on by default in the newest core version.
Stephan Livera (1:19:07)
Yeah, so my understanding,
guess just quickly for listeners, NAT traversal, as I understand, you correct me if I’m wrong, I think it’s network address traversal. And the idea is when we’re in our home network, we’ve got our own like our own router and our own, you know, 192.168.1 dot whatever. And, but that’s different to the network from the outside world. And then my understanding then is unless you do port forwarding, your Bitcoin node may not be reachable. But now, as I understand then,
You told me if I’m getting this wrong, but my understanding then is this makes it easier for your node to just kind of default be able to be reachable to other Bitcoin nodes. Have I understood it correctly?
TheCharlatan (1:19:45)
Yeah, exactly. So you still need to configure it or switch it on on your router. I think most don’t switch it on by default, but you don’t have to.
Stephan Livera (1:19:57)
Okay, as in mini
UPnP is a setting that you have to do at your router level, but then once you have that, then your Bitcoin node is more easily connectable or reachable.
TheCharlatan (1:20:07)
So, well, we don’t support that particular protocol anymore. But many routers, when you switch on mini-upnp, they will also enable some other NAT traverser protocols. And these are what we currently support. So NAT PMP is the way we
intend to support the SUT mode.
Stephan Livera (1:20:38)
okay, so what was the name of it? missed it. NAT PMP, okay, gotcha. Sorry, I must have misheard you before. Okay, ⁓ and then ⁓ as I understand, there have also been some IBD, initial block download improvements in V30. Can you maybe elaborate on that if you are familiar?
TheCharlatan (1:20:40)
That’s BMP.
Yes, so these have been building on some prior improvements that have been done over the past four or five releases now. So I think since version 25, we managed to cut the IBD time in half again, which is pretty cool to still see happening all these releases later with continuous performance improvements. ⁓
In this release, there were a bunch of smaller improvements made, but the incremental improvement as far as I recollect isn’t that big over version 29. I think it’s around 10 or 20%. But I’m, yeah, I’m not sure anymore about those stats.
Stephan Livera (1:21:46)
Okay, gotcha. And I guess just zooming out a little on Bitcoin Core contrasted with other implementations or even earlier versions. Do you have any thought on what the world might look like if you know, there’s different implementations? And I guess I presume from what you were saying earlier, you’re comfortable with that, you’re okay with that idea that there’s different Bitcoin implementations and Bitcoin Core is one of multiple?
TheCharlatan (1:22:12)
Yeah, I think competition is definitely healthy here. ⁓ I really like Bitcoin Call and I really like the Bitcoin Call project. So I don’t intend to work on another implementation in the near future, at least. And I think it’s good if Bitcoin Call gets a bit of competition. Yeah, maybe that can help us come up with even better ways to
to optimize things, to be even more pedantic on security features and review and whatever. Yeah.
Stephan Livera (1:22:49)
Yeah. ⁓ I think one other concern I’ve seen from some people in the community or online, whatever we’re going to call it, Bitcoin, the online, screen first, people saying that, Bitcoin core is kind of controlling the default, right? Because it’s such a dominant implementation. Although of course the knots, ⁓ fans have been really, talking a lot about how the nuts, ⁓ percent has risen to, I don’t know, 20 something percent. ⁓ as we speak, ⁓
I guess what my point I’m asking here is, is it effectively changing Bitcoin if the default implementation changes certain, know, relay settings or things like that? because I understand on one side you could say, well, you don’t have to upgrade, right? Bitcoin Core has this long, no auto upgrade process, like that’s a long standing thing. But at the same time, there’s kind of security updates. And if you go more than, let’s say two or three releases out of date, now you’re kind of,
end of life. So is there some level of pressure there that it’s kind of like Bitcoin Core being the default can sort of push through certain things ⁓ because it is the dominant implementation?
TheCharlatan (1:23:57)
I think there is a bit of merit to it. Like many people do run it and I guess there is the linear effect to it. But at the same time, we’ve also seen that people are perfectly capable of changing the defaults themselves of publishing slightly tweaked versions of call or publishing heavily tweaked versions of call like knots. And as long as we don’t prohibit that then
Yeah, that’s just.
Stephan Livera (1:24:29)
Yeah, just so be it, okay?
TheCharlatan (1:24:30)
gives people the freedom to choose and run whatever they want to. Yeah.
Stephan Livera (1:24:35)
Okay, ⁓ and then I guess, so I guess soft fork wise, you’re mainly focused on the great consensus cleanup at this point. Is that the main one that’s kind of on your horizon?
TheCharlatan (1:24:47)
Yeah, I’m honestly not too interested in the soft forks trying to deploy new features for, yeah, lightning or some more crazy taproot transaction constructions. So I try to focus on making the core protocol just more robust. Yeah.
Stephan Livera (1:25:13)
Okay, I think those are the kind of the key, ⁓ sorry, one other one. I know this has been kind of discussed and batted back and forth a few times over the years, but I’m curious if there’s been thought about having like certain things built in with Core. So as an example, like an Electrum ⁓ address index kind of thing, right? Because what a lot of users are doing today is they’re basically running, you know, Bitcoin Core plus, you know,
some form of Electrum server, whether that’s Electris or ⁓ I forgot some of the other, Fulcrum and some of these other, ⁓ or Electrum, these different Electrum servers. And then they might be running, let’s say Sparrow or Nunchuck or Electrum as their coordinator software on top of that. Has there been any talk about doing an address index inside of Bitcoin Core so that people could just run Bitcoin Core and Sparrow or whatever?
TheCharlatan (1:26:09)
Yeah, I love this question because I’m in favor of us eventually having some kind of address indexing or maybe even a better protocol to retrieve address specific information from Bitcoin Core. I think we are kind of moving into that direction with some of the changes we’re currently trying to get through review in Bitcoin Core.
Stuff that just makes our indexing codes more robust, easier to code against. yeah, again, the whole approach with using IPC and CapnProto could really help there too, because once we have an interface that you can just connect any other program to it directly to do this indexing stuff, it might also be easier to
code up and then integrate into our project eventually.
Stephan Livera (1:27:11)
I see. yeah, sounds like ⁓ maybe, yeah, I guess, of course, Bitcoin Core is not a monolith. There’ll be some people in favor and some against, but it sounds like you’re in favor of that idea. ⁓ Because I think that’s maybe another criticism I’ve heard is some people think, Bitcoin Core is sort of not ⁓ focusing on what the hodlers want and some of the hodlers, they really want to be able to just kind of easily use their hardware wallet, ⁓ this kind of thing. And of course, there’s also been discussion about like the wallet GUI and things like that because
As I understand very few people directly use Bitcoin Core’s wallet itself, or least the GUI to be clear. ⁓ Many of them are just using some other wallet that’s connecting to Bitcoin Core or to an Electrum server. So do you have any comments on the wallet and the GUI?
TheCharlatan (1:27:54)
So there is a group of people working on a new Bitcoin core GUI. There’s some mock-ups and some first versions running off that. If people want to take a look at that, they’ll probably find it if they Google new Bitcoin core GUI or something. So yeah, I think that looks nice.
Stephan Livera (1:28:11)
Yeah, I think I’ve seen some of those floating around, but go on.
TheCharlatan (1:28:18)
For the feature specifically, think the wallet has been a bit underdeveloped the past few years, especially when it comes to features that people really want like better hardware wallet support. ⁓
maybe newer taproot features, it has taken a bit of time, but I’m also feeling some new momentum in that corner of the code at the moment. So.
We finally got the legacy wallet removed. ⁓ Music to support was merged, I think, last week, which is a pretty big step in allowing more efficient multi-seq setups.
So yeah, I think.
project obviously can’t do everything super well, but we should also have these kind of utilities for people to use in order to remain relevant and useful.
Stephan Livera (1:29:26)
Okay, so any closing thoughts, any last things you want to get out there before we close up?
TheCharlatan (1:29:32)
Cash on the internet, no order updates.
Stephan Livera (1:29:35)
Okay, yeah, this is like a saying I’ve seen some people say so all right cash on the internet no auto updates ⁓ Yeah, thank you for joining me ⁓ Charlton and I guess just before we let you go where can people find you or follow your work online?
TheCharlatan (1:29:49)
yeah, if people want to follow me on GitHub, they can do that. It’s just the charlatan.
Stephan Livera (1:29:54)
Great. OK, well, thank you.