Are you interested in building support for Bitcoin’s Lightning Network into your product or service? Check out this discussion of Greenlight with Blockstream’s Christian Decker. We talk:
- Improvements in lightning experience over the years
- No. of payments, reliability
- Greenlight experiences
- Mobile nodes
- Comparison with other providers
- Shared infrastructure
- Core Lightning features
- What features will help make lightning easy
Links:
Sponsors:
- Swan.com (code LIVERA)
- CoinKite.com (code LIVERA)
- Mempool.space
Stephan Livera links:
- Follow me on X @stephanlivera
- Subscribe to the podcast
Podcast Transcripts:
Stephan (00:00.627)
Christian, welcome back to the show.
cdecker (00:02.527)
Hey, Stephan. Thanks for having me back.
Stephan (00:04.883)
So lots of things going on. Obviously, we’re gonna talk a lot about lightning and green light and all kinds of things. I wanted to start with some of the recent developments in terms of what’s happening with the Lightning Network. So as I’m sure you probably saw, River recently put out a lightning report and I think it helped dispel some popular narratives that are out there. I’m curious if you had any, firstly, if you saw it and if you had any reactions to that. Obviously, your…
you know, working deeply at the protocol level and also at a product level with green light. So I’m sure it must have been good for you to see some of the statistics and points made in that report.
cdecker (00:45.078)
Oh yeah, it’s very much encouraging to be honest, to see how nicely the adoption is going on.
We’ve also seen a couple of numbers being shared from wallet of Satoshi, for example. And despite these only being sort of select a very small points of vantage that, that sort of see the traffic that is going through them. They still have very, very impressive numbers. And, and if we were to look at the network as a whole, it would probably look, look even more impressive. But we, we built in privacy from the get-go.
So that’s sadly in some sense not possible, but having these companies share their numbers is definitely awesome.
Stephan (01:32.635)
Gotcha, yeah. And just for context, what’s going on here is Sam Wouters from River, he actually went and asked some large lightning service providers and lightning companies, just asking them for their data. And some of them gave him that data and he was then able to basically calculate or estimate to be clear, these are estimates, how much lightning usage there is today. And so the, I think probably the most interesting number.
was the headline number of the lower bound. So the lower bound for the number of Lightning transactions per month recently is 6.6 million. And that’s incredible when you think about it, right? Because for so long it’s been a quote unquote, FUD line, oh, Bitcoin can only do seven transactions per second and so on. And if you just kind of divide 6.6 million by the number of seconds in a month, it’s, I can’t remember the exact number, but it’s like two to three transactions per second are happening today on Lightning, and that’s the lower bound. That’s not even counting
what lightning could do today.
cdecker (02:30.846)
Absolutely, yeah, and these despite being…
being estimates are very nice numbers that are very well researched and sort of always give you a range of lower and upper limit and I really like the approach of communicating the adoption this way. And yes, those are two to three million transactions that are not being sent to the Bitcoin blockchain that we have on top of what you can actually do on the blockchain itself. So that’s a force multiplier
system.
Stephan (03:06.415)
Yeah. And so there was also some internal statistics shared things like payment reliability. And I’m curious if you have anything to share on that or any experience you have, you know, comparing Lightning today, October 2023 versus the early days of Lightning.
cdecker (03:23.902)
Yeah, so the payment performance very much depends on how well you are managing your node, of course. The uptime has an impact, the availability, your connection to the Bitcoin network has an impact and…
While we’re not yet at 99.999% success rate, that is definitely something that we are looking to, where we are looking to close the gap by building more advanced payment systems and liquidity management systems. And those include the min-cost flow that René proposed a couple of years ago, and which is now implemented
lightning with the liquidity ads with the with the LSPs being building a specification. And all of these complementary system that actually help us get the performance to where it should be. It’s it’s also very nice to see those numbers creep up over time and getting better and better. And yeah.
Stephan (04:36.435)
So on that actually, just on the payment reliability question, do you have any, I guess just from the people you’re talking with and the work you’re doing, would you have any thesis on what were the main reasons that payment reliability wasn’t so great before? Is it mainly like nodes were going offline or maybe it’s like certain wallet types that people were using were just sort of not able to wake up?
on time to take the payment? Do you have any, you know, what would be sort of the main highlights there in terms of why payment reliability wasn’t at that grade in the earlier days and seems to have improved a lot now?
cdecker (05:11.31)
There’s probably a combination of factors that plays into this direction. First and foremost, I would say that node operators have become more and more professional.
the actual liquidity being provided in the network has been better allocated. People are starting to learn from experience what works and what doesn’t. And it’s this sort of collective learning that gets us a better success probability, better pay performance, better time to completion even. Right. We have seen a drop in Tor based.
notes because Tor has a tendency of adding quite a bit of latency. Latency then again brings into, brings with it the fact that if you end up using a channel that is very slow, you are actually cutting into your time budget and you may not be able to retry if, well, this payment attempt comes back after 30 seconds, you only have 30 seconds left to complete the payment due to some
timeouts on the recipient as well. And so it is an improvement in the network, both in terms of liquidity, but also in terms of pure communication primitives underneath it. And then of course we have
the improvements when it comes to providing liquidity on demand, such as LSPs. We do have quite a few LSPs nowadays that can actually open a channel on demand when there’s an incoming payment and the capacity isn’t sufficient. But also at the end points themselves, we are seeing more and more improvements with the recipient providing route hints, hinting at, hey, you might want to try this path because there is liquidity there.
cdecker (07:12.62)
also on the sender side, in particular, the sender side, where we are building better and better models about where is the liquidity in the network? What are our expectations? And how can we can we make use of that to perform our payments? And so I think it’s a multitude of factors. Yeah.
Stephan (07:28.039)
Gotcha. So it’s not really one answer. It’s a combination of all these things. And I remember in earlier days, you might try to route a payment and it would say, oh, route not found. And then it could be that, because in the earlier days we didn’t even have MPP, so the payments couldn’t even be split. So you needed that exact amount across every hop in the route for that recipient to be able to take that money. And nowadays, you know, payments can be split.
cdecker (07:45.46)
Yeah.
Stephan (07:54.835)
payments, you know, there’s all these, all of these aspects have come to make it better. There’s another interesting idea that I think Sam spoke about in the River Lightning report. He mentioned that there were a lot of payments that would not have been viable on chain, which is an interesting idea as well, right? Meaning, so just spelling that out for listeners, the on chain payment fee, if you were to do that as an on chain payment, it could have been $5, $10, especially in the recent fee spike with the ordinal inscriptions stuff. And
cdecker (08:10.104)
Mm-hmm.
Stephan (08:23.591)
these people were still able to transact using Lightning, whether that’s zapping on Nostra or it’s podcasting value for value, or whether it’s geyser.fund donations, or whether it’s payments to and fro the Lightning supporting exchanges. I think that’s another interesting area where Lightning is enabling more kinds of payments that were not viable before. Do you have any comment on that?
cdecker (08:45.614)
Absolutely.
That was one of the goals we had when we set out to build the Lightning Network. There was the real-time aspect that could open up new use cases that simply weren’t feasible on-chain. There was a scalability aspect where we could say, hey, all of a sudden we can do tiny payments. We can do machine-to-machine payments. We can do signal payments, right, where the payment isn’t the actual thing going back and forth.
actually attaching a message, very similar to how many of the early chat apps were working was essentially to piggyback a message on top of a tiny, tiny payment of maybe one Satoshi or one Milla Satoshi. And these use cases were simply not possible back on chain. And that is also an aspect that renders Lightning an excellent compliment to the on chain payments.
because they have different trade-offs and different use cases might need one or the other, depending on their own requirements. And so it’s less about on-chain and off-chain competing with each other, rather than essentially lightening, bringing on board many, many new use cases that are interesting to different users. And the overall system becomes a more welcoming and one more flexible system
that people can build applications on top.
Stephan (10:20.687)
Yeah, I think the key point you mentioned there is the instant payment aspect. I think, you know, people said a lot of things about lightning, whether it’s, you know, it’s meant to be cheaper and faster and all this stuff. But I think really that is the ultimate thing is the instant confirmation or not, not needing to wait for a confirmation. Whereas, you know, in years gone by, people would do things like, Oh, hey, here’s my Bitcoin address, Christian, make a payment to me. And people would do this sort of awkward dance of one, you know, someone having to wait for
significant for enough confirmations and it just kind of made it really awkward and people could overpay or underpay the amount whereas with lightning it’s you can have a set amount if we’re doing commerce I’m saying 70 50,000 Satoshi’s you can send me exactly that amount and you don’t we don’t get this kind of overpayment or underpayment factor we’re not waiting so there’s a lot of experience points that are improved in a lightning
user experience and in some cases you might be willing to pay more for that because you want the instant confirmation. So that’s also an interesting angle. So let’s get into Greenlight. I know you guys are recently launching that out and you’ve been going through a process so far with let’s say earlier testing. So tell us a little bit about the Greenlight launch and how things have gone there.
cdecker (11:35.582)
Yeah, so we announced Greenlight a couple of years ago. Back then it was still very much in close beta with a couple of handpicked partners with whom to develop. And these early learnings helped us a lot to…
to refine the APIs and the libraries we provide to talk to green light. And we now think it’s time for us to sort of open up this to the general public by essentially saying, hey, if you are somebody who wants to build on lightning or you want to run your own node for your own use case,
And you can essentially just sign up at greenlight.blogstream.com, get a certificate. The certificate allows you to create up to 1000 nodes for free and then you can start hacking and developing your application on top of Greenlight without having to pay a single dime.
The announcement we, and that is pretty much the announcement we did last week. And the important takeaway here is that we wanted to assure prospective developers and users that they won’t wake up to a surprise bill by essentially providing a very generous free tier and assuring them that if they start building on top of green light, that would be something that lasts and that remains available
for them for the foreseeable future. And if people go beyond those 1000 free notes, if you, for example, have a very successful wallet or a very successful service, you might want to reach out to us. We are looking to learn about the usage patterns and what it would cost to operate such a system. And therefore,
cdecker (13:49.712)
We’re very open to feedback when it comes to these things. And we’d be making, we’d be coming up with a custom pricing model for you in a way that allows you to build out your client without hitting limits. And for us to be able to assure you that we will provide the service as it is with the same performance and guarantees
you have access right now.
Stephan (14:22.003)
Got it. And so to be clear, that’s the free tier. What we’re talking about there is the 1000 free nodes. That’s the nodes operation, but that’s distinct from the LSP, right? Like to get channels opened and things like that. That’s another thing where either the, in this case, Greenlight’s customer is going to have to either provide their own LSP, be an LSP or pay an LSP. Or, I guess, are you also running an LSP in your background or is that something where they need to find an LSP to pair with Greenlight?
cdecker (14:29.261)
Mm-hmm.
cdecker (14:33.218)
Correct. Yeah.
cdecker (14:52.302)
So we’re not running an LSP ourselves and that’s for good reason because many of the attacks scenarios in the Lightning Network are only possible when you have essentially the node and the counterparty on the other side of the channel collaborating. And so we definitely do not want to have both endpoints under our control. Just like we don’t want to have custody of user funds, we don’t want to have that risk
essentially. And so what we currently do is we are building out the LSP specification for Core Lightning. And since Greenlight essentially runs just Core Lightning on our infrastructure, those features will be available eventually. But if you need an LSP right now, the team
cdecker (15:52.176)
Greenlight client and integrates with their LSP, essentially allowing you to even without any channel, just create an invoice and the channel will be established on demand as soon as somebody tries to pay that invoice. There’s a bit of magic going on there for payments for the LSP liquidity and stuff like that, but that’s all taken care of by Greenlight and the Breeze SDK.
So for end users, it becomes really easy to essentially just open the app, create an invoice, and somebody paying in the background will just happen automatically.
Stephan (16:34.379)
And so then the green light customer in this case, maybe a person who’s building a wallet or maybe a business who wants to be lightning enabled. And so that these are, these would be some of the main uses here, right? Like what, who were the main people who you’re envisioning as your customer for green light?
cdecker (16:54.558)
Yeah, so the goal…
here essentially is to make it really, really easy for developers to integrate with Lightning. And I don’t think I’m smart enough or creative enough to come up with the coolest and best use cases, but I am here to provide you the ability to integrate with Lightning if you already have an idea. And as such, I obviously will always mention wallet
and chat applications because those are the use cases that we’ve already seen. But there is a multitude of systems that might benefit from being lightning connected. Take for example,
cdecker (17:46.358)
That is probably more of a social media application, but it still benefits from the fact that you can just call an API on some server and be able to push over some stats to somebody that has posted something that is interesting to you and that you would like to tip.
And essentially by making it easy to integrate with Lightning, we are also broadening the scope of who can build applications, either on top of Lightning or integrating them with Lightning. And I think the future will give us quite a few interesting ones to see.
Stephan (18:28.495)
Yeah, interesting. And so, yeah, I think the obvious kind of the one that probably sticks out to most people is just like, if you’re trying to make a lightning wallet and you need, you know, you want to make it self-custodial and you don’t want to run the actual node infrastructure for those users, then green light is obviously a pretty obvious fit in that case. Uh, and then I guess you could sort of see it like, okay, up to 1000 users, end users, that is.
would be free. And then above that, that’s when you need to, I guess you need to figure out a business model for that wallet so that, you know, obviously you’ve got to, everyone’s got to get paid somehow to make this work. And so I guess, yeah, it’ll be interesting to see what kind of models come out there for monetization. Maybe in some cases it’s, you know, it’s more simple. Maybe it’s like a business just wants to onboard to Lightning and maybe they have different.
cdecker (19:17.522)
Absolutely. Yeah.
Stephan (19:19.875)
stores and each store has to have its own node. And that might be another example where that store needs to, wants to, let’s say have their own channels. I guess I’m curious, what do you think? Do you think that would be a plausible setup or do you think it might be, you know, maybe certain places would just sort of have one node that does all of their stores? I guess it’s all open at this point, isn’t it?
cdecker (19:43.114)
Absolutely, it very much depends on what you would like and…
Essentially, the flexibility that we provide is both an advantage for the end user as well as the developer itself, right? If you go with one of the classical models where you essentially bundle the node with the application itself, you all of a sudden have a node per application. And that means that if you want to try some new app, you just want to see, hey, this chat app works different.
bit better. With a bundled node you would have to tear down some other node in order to get the funds, move them over, recreate channels and essentially now start managing liquidity on yet another node. And the fact that with Greenlight you can actually share the node among as many applications as you want, it makes it really easy for end users to essentially just test a new app out.
And therefore also for developers to get testers because it’s less of a time and less of a money commitment now to actually try out a new application. And then there’s the other dimension, right? We might have a single app that has many Bitcoin backends. For example, if you run a retail store that has many different locations, you might want to give each of them a separate node. And so each retail location manager
have their own notes. And even then there’s the multi-client aspect where you could give multiple people access to the same note, but with different access permissions. So the point of sale, for example, could be allowed only to create invoices and check if an invoice was paid. And then there’s the back office that can send and receive. And then you maybe have the accountant
cdecker (21:42.576)
that it is seeing every movement of funds. And so there again, we have this, we have to structure where multiple clients are actually talking to the same Lightning node. And then we can, you can decide if you want to have a Lightning node per store location or a Lightning node for the entirety of Walmart, or even go even more granular
of the shop have their own Lightning Node. So essentially, there is no limit to the flexibility of how you can set this up. We just provide a Core Lightning Node instance running in the cloud, and that is reachable from wherever you want.
And if you want to take that node and something is preventing you from implementing your perfect scenario, then we give you the opportunity of exporting the node and running it on your own infrastructure, where you can then fully customize the node however you want.
Stephan (22:56.019)
Okay, and so yeah, so the idea is to have self-custodial nodes to let developers build in a way that’s more easy rather than trying to have to do everything themselves. So I guess in some ways Greenlight is competing with other lightning providers in the industry as well. So people like, I’m thinking Voltage or perhaps Light Spark kind of playing in a sort of similar space.
cdecker (23:22.081)
Mm-hmm.
Stephan (23:24.667)
Whereas let’s say somebody using like Open Node or Ebex Mercado is different because that’s more like a custodial provider of just like merchant services. Whereas this is at a more, let’s say generalized level that you’re helping run the node infrastructure for people. But in this case, you’re running the node, but not the LSP. So I guess that’s one other kind of distinction.
cdecker (23:44.082)
Exactly. Yeah. It’s just, it’s just a separation of concerns that we, that we, that we rather like and that we would like to maintain. It could be that, that we offer an LSP of last resort for
for people that don’t have an LSP offering out there, but it really is the LSP of last resort, so to speak. I think the comparison with voltage and light spark and open node are definitely valid.
Stephan (24:06.719)
Gotcha.
cdecker (24:17.218)
But there is, as always, there is a bit of a spectrum of custody, right? And for light spark and voltage, the keys are still on the hosted infrastructure. Whereas with us, we…
physically keep the device that is holding the keys is under the control of the user. And so I always say if any of the other offerings do get the non-custodial flag, we will definitely get it.
because we essentially provide an even stronger system where the operator can do even less. And if the operator can do less, an attacker can also do less.
Stephan (25:00.092)
Yeah.
Stephan (25:03.535)
Yeah, interesting. And that’s a fair point. I hadn’t really thought that through as deeply as you had. And so with the node spinning up and spinning down, as I understand, this is something you have built into the Greenlight system as well. So the idea is that the node could spin up when it needs to and then come back down. Can you explain a little bit how that’s working and how you make sure it is able to come up on time?
cdecker (25:28.446)
Yeah, so the core observation in all of the system is that essentially the keys being at the user means that if the user isn’t present, the node can’t actually do much.
And so we take that opportunity to essentially just shut the note down if it’s not being used at the moment. And that essentially just means that a couple of minutes after you last interacted with a node, we will take the note, shut it down gracefully, and release the resources for other users’ notes. And this gives us a…
I would say two to three orders of magnitude more use out of the resources that we are committing to Greenlight. And that in turn then becomes cost savings that we pass on to our end users. And
The way that we spin up on the other hand is much more interesting because that is very much the critical time, right? The user has just opened the app and wants to talk to their nodes. So we need to ensure that in the least amount of time possible the node is back fully functional, connected, has a full view of the network and can perform payments
scanning an invoice when opening the app. And so what we do there is essentially we have assigned each node a unique URL.
cdecker (27:10.234)
essentially glNodeID.nodes.gl.blogstream.com. And whenever you try to contact that endpoint, which is specific to every single node, what we will do is we will internally see an incoming request. We will check, hey, is that node running somewhere? If it’s not running somewhere, is there a slot where we can essentially start that node up? We will start the node up
and forward the connection to the now running node. And so from an end user point of view, it is absolutely seamless. You are connecting to your node, sometimes it takes a quarter of a second longer because we are actually spinning up the node and sometimes it’s immediate. But the slightly delay is the only thing that you should notice. And…
By doing that, we ensure that the node is available whenever you have the app open. And while testing with outside developers, we found that this fact that the node is actually spinning down often gets forgotten about.
The developers integrating with Lightning often have the feeling that the node is actually running 24-7, despite it not being running 24-7, simply because it is so easy to actually spin it up again.
Stephan (28:36.735)
Gotcha.
Stephan (28:40.659)
Yeah, and I presume, as you mentioned, each node has a unique ID. Is there some kind of a shared infrastructure? Like as an example, do the nodes share like gossip sync data? So as an example, lightning nodes, they have a view of the network and they wanna know, okay, what are the other nodes out there and what’s my view of the channels? And so in the green light context, are the nodes sharing that view?
cdecker (29:04.686)
Pretty much, yeah. So there is quite a bit of shared infrastructure. For example, the databases are being shared in part. We have watchtowers running that are also part of a shared infrastructure. We have a routing node with a routing server which maintains a coherent view of the Lightning network and then synchronizes that view down to the node and when it starts up.
And we have a whole, what I like to call the circulatory system in the form of our Bitcoin backend infrastructure where we can distribute Bitcoin information very quickly to all of the different parts of the system and of course also inject new transactions into the Bitcoin network in a very reliable and very fast way. And since you since you zoomed in,
on to the gossip, that is one of the most critical parts during startup, because an up-to-date view of the gossip is paramount for our chances of finding a successful payment.
And so it wouldn’t be much use if a node was offline for a couple of hours. Now it comes back and now it still uses an old view of the network. And therefore we, we went and built this routing server that aggregates information, aggregates views from, from all the different nodes running on a number of different networks, builds a cohesive view of, of the network, and then compresses it and streams it down to, to nodes.
when they start up. And so in a matter of seconds after startup, the node actually has a very, very up-to-date view of the network. And that serves to maximize the chances of the payment being successful. And so, yeah, that’s definitely one part that we have customized for Greenlight.
cdecker (31:12.098)
that in case you off board is less useful for you, but we’re considering to open up access to the rapid gossip sync implementation too, just because it turns out it’s really useful.
Stephan (31:28.443)
Yeah, and so as I understand that might also be useful in a range of contexts. So as you said, the payment reliability is also an important point. But even things like knowing if your channel has been closed or knowing if, you know, having the most up to date state of what’s happening even on the Bitcoin on chain level, because you just, you just want to make sure you’re sort of, you’re on stable ground here and you’re not sort of operating on this old view. And I think the other question
and maybe this is an important topic, I’m curious if you have any thoughts on this, is mobile nodes. Right now, in our recent episode, you, myself, and with Roy from Breeze, we were talking about how mobile nodes and so on, I think it’s, let me put it into an example. So as an example, I think with Breeze, at least as it is today, as an app, if you haven’t used Breeze, the app, for a little while,
cdecker (32:01.174)
Mm-hmm.
Stephan (32:24.115)
then it will then, and you open it, then it’ll have to do that sync up process because it is using like the neutrino style filters and it has to sort of understand the latest state of the chain. I guess in the green light context or even the breeze green light version, you won’t have to do that. And so it’ll just sort of be like you open it up and it’ll already be synced up per se. So I guess that’s gonna enable an easier mobile user experience. So can you maybe compare for us?
cdecker (32:44.234)
Absolutely, yeah.
Stephan (32:52.263)
the mobile nodes that try to do it all on the phone versus the mobile green light experience. So that might be an interesting point of comparison.
cdecker (32:59.986)
Oh, very different world rules there, but we’ll see which one ends up winning in the end. So one of the issues that we have with mobile nodes, as you correctly point out, is that, for example, they can’t run in the background and therefore can’t synchronize with the network.
If the node happened to be offline for a while, then, well, you’re starting to lag more and more and therefore have a longer startup time next time you wanna use it.
With Greenlight, on the other hand, we can actually start the node without the signer being present. And that means that the node can process blocks and identify changes in incoming funds or outgoing transactions being confirmed or channel closing or channels being funded and the funding confirming and stuff like that.
cdecker (34:02.508)
present means that we can ensure that your node is actually up as up to date as possible. We allow for up to six blocks of lag and then we end up essentially just starting your node quickly synchronizing the blocks and if nothing happens we will then put it to sleep again for you to essentially wake up when you need it next time and the most time you will have to wait to synchronize those six blocks.
Those six blocks are also being served out of our shared infrastructure and will be processed by the node running on our infrastructure too. So there is very little latency and we can process blocks relatively quickly. So a couple of seconds and you will be up to date no matter how long you kept your phone app closed.
Stephan (34:37.043)
the shared infrastructure we mentioned earlier.
cdecker (35:01.994)
The other thing that mobile phones can’t do and we can do because we have this very active monitoring system and we do start your node from time to time, is actually reach out to the signer and say, hey, there’s something happening there, please come see and help us resolve the situation. So for example, if one of our channels closes on chain,
cdecker (35:32.728)
based on that and therefore summoning the signer using a webhook system that we are currently implementing but will hopefully allow us to summon a signer in case we need it. And with the signer, we can then process the event, have the appropriate reactions if it, for example, is…
If it’s a cheat attempt, the watchtower will react, but also the node itself will essentially get a signal saying, hey, your attention is required, please help us resolve and get past this issue. And this, for example, wouldn’t be possible for a phone because the bundled node with the phone, because the phone would have to wake up the node at regular intervals and
to even identify that it is being needed. Whereas we have the server-side assistance where you can essentially track stuff on behalf of the user and wake it up if there’s something that needs attention. And then there’s a whole lot of trade-offs. We can probably spend an entire hour talking about trade-offs of mobile and bundled versus non-bundled applications.
having the ability of having multiple clients attached to the same node, not having to share resources between those nodes, whether that’s your own time managing liquidity or whether that’s actual funds being split up across a number of nodes. And let’s not forget the recovery story, right? I’m a passionate sailor and I’ve had my fair share of boating accidents. And if your node is essentially running on the phone,
Well, that boating accident, losing your phone, might actually cost you quite a bit of money.
cdecker (37:35.35)
because a seed is not sufficient to recover all of your funds, right? And so, with us, it essentially is just regaining access to your, to your node and having a signer be reestablished. And you can do both of these just starting from the seed that you use to register the node itself. And so the recovery and safety story for, for a hosted solution is much, much different from, from a bundled.
for example.
Stephan (38:07.151)
Okay. Yeah. So I guess in that context, it might be like the user has to have written down seed somewhere, but as you’re saying, the assistance of the server side infrastructure in this case with green light sort of helps them recover without losing stuff, basically.
cdecker (38:23.002)
Exactly, you don’t even have to close any channels like you would for the emergency recovery or have some real-time backup being streamed somewhere. It’s all being handled for you while you stay on the hosted solution and the same is true for when you off-board into your own infrastructure.
as long as it’s your phone going overboard and not your entire lightning infrastructure, which I think might be more difficult to throw an entire server overboard.
Stephan (38:56.735)
Right, yeah, that would be a next level boating accident. It’s like, oh, I lost my lightning node at sea. Oh no. Okay, and so I know Blockstream recently did a partnership or some kind of collaboration with VLS, validating lightning signer, and I know that’s, I think that’s gonna be part of the green light solution. So can you just spell out a little bit about VLS there?
cdecker (39:14.507)
Mm-hmm.
cdecker (39:22.186)
Yeah, so the VLS project by Def Random and Ken Sedwick, I know you had Ken on recently on the podcast. They’re excellent people to work with and we had a blast. The…
idea for the signer was essentially that we can’t have a signer that blindly signs off on just anything we send it, right? Because then having access to the node would be sufficient to do whatever you want, despite technically the keys being in the hands of the users. And so we needed to ensure that the signer is smart enough to enforce policies.
And the VLS team, we found out while doing our own research, was building exactly that and was already at a very advanced stage with their VLS signer based on LDK, but implemented very closely to CoreLighten.
And so it just happened that we found each other. They had been inspired by Core Lightning’s HSMD module.
to look into building a validating Lightning signer. And we were looking for a signer that could essentially perform these extra validations that we needed for the node operators or attackers to be sort of locked out of managing the funds for users.
cdecker (41:05.602)
And so what they have is they have re-implemented part of the Lightning logic, enough of the Lightning protocol to verify that whatever is going on matches the expectations that the policy set forth. And these policies could, for example, be, hey, you’re only allowed to spend $10 worth every day. We will never sign an old commitment transaction, which
could be used to cheat. And that if we perform a payment based on an invoice, we never exceed sort of the amount and all of these validations. On top of those, we then build what we call the end-to-end verification, where we say,
VLS is very good to verify policies that it has been configured with. But what we want instead is to have an additional layer that ensures that every single command that touches funds was actually initiated by an end user.
And so that’s where the end-to-end verification comes in. Each client essentially has an identity and is used to sign off on the user’s intent.
expressed as a command call and this command is then sent to the node. The node computes the state changes that are sort of needed to perform this command and then the command as well as any signature request that is needed for the signer to sign is getting passed down. So the signer has the entirety of the context who initiated the command, what was the command
cdecker (43:04.152)
this match the expectations that I have and is this user actually authorized to initiate this payment?
And so by essentially closing that loop, despite the node being in the middle, we can ensure that the signer has the contacts and can independently verify that whatever it is signing off on actually matches the user’s intent. And so that is the core contribution that we built on top of VLS, but VLS was an excellent basis to get started from.
cdecker (43:43.184)
brainstorming together about how we can perform certain things. And one of the things that we brought to the table there, for example, is the validating signer state server. So I mentioned before that you can have as many frontends as you want, but you can also have as many signers as you want. And the advantage of having multiple signers
node remains functional as long as one signer remains online. Now, signers aren’t stateless. They have to remember, oh, I processed 10 Satoshi’s worth of HTLCs and the invoice is asking for 10 more. So I will be allowed to essentially…
sign off on 10 additional Satoshis. And so having many signers, we all of a sudden have a distributed system where they need to somehow communicate about the state that they are collaboratively managing. And so with a validating signer state server, we can push state from one signer to the next through the use of an external server and ensure that we always, as signers, we always agree what the current state is
Stephan (44:58.79)
I see.
It can stay in sync. Right.
cdecker (45:05.784)
what we should sign off on and what we shouldn’t sign off on.
Stephan (45:08.895)
Gotcha, okay, yeah, interesting. So it helps you stay in sync then across different signing devices. And one other question I think people might be wondering is obviously Greenlight, you know, it’s Blockstream, it’s Core Lightning. So presumably then Greenlight, you know, inherits all the features that Core Lightning is, you know, taking on. So as an example, Rusty has been a big proponent of Bolt 12 and, you know, splicing liquidity ads, maybe even PeerSwap.
cdecker (45:12.726)
Yep.
cdecker (45:31.99)
Yep.
Stephan (45:36.379)
Are these features things that people could come to expect in green light?
cdecker (45:41.022)
Absolutely. My goal is to enable anything built on core lightning to also work on green light and vice versa.
So we have essentially taking all the learnings that we learned from the rollout of the new API back to Core Lightning. And for us, Greenlight is very much also a project that helps us validate some of the design choices that we did in Core Lightning and funnel back feedback from
cdecker (46:21.241)
to make the open source project better overall. And so it’s very much a two way street between the two projects. And…
Indeed, we should end up in a situation where anything that was built for Core Lightning can also be run on Greenlight. There are still some open questions. For example, what do we do with plugins? Plugins are very different in a hosted setup. For example, we need to ensure that plugins are actually safe to run, that they aren’t just some malicious payload.
that somebody else put out there. And at the same time, we need to ensure that all of the functionality that a user might want to is actually available as well. Also running plugins alongside with a node is just one option we have here. A plugin could, for example, all of the sudden be a web API that we call. And so…
We sort of reconnect the Lightning customization and Lightning extension world with the well-established world of web development and offerings web services. So I can, for example, see that somebody might want to use the plugin API that we have in Core Lightning to build an accounting software that automatically gets told about movements of funds.
that can present you with a very nice sort of view of what is happening with your node and pull that back into your existing accounting infrastructure for example.
Stephan (48:10.415)
Yeah, okay, gotcha. And so, yeah, so as you mentioned the plugin, so maybe it comes to a point where there’s certain plugins that you just say, okay, these aren’t allowed because security reasons, et cetera, but all the other ones are allowed or whatever, as long as you’ve vetted them and they’re fine to work, you know, technically. And then in terms of, I guess, technical features and things like that, like as an example, having Bolt 12 support, what kind of features, what kind of benefit do you see coming from that? Like, it means…
As I’m understanding it, it means the developer user of Greenlight can build in the function where his end users can have BOLT12 and they can just have a static QR code to say take donations as an example. Because Core Lightning has that feature.
cdecker (48:53.151)
Absolutely, yeah.
Absolutely, yeah. And I think we already have a vast majority of the Bolt 12 related methods exposed on the Greenlight API. So it’s available right now, is what I want to say. Ultimately, the expressiveness that you get for Greenlight should be at the same level as core
cdecker (49:26.448)
API methods available both on a local core lightning as well as on green light. And so building a version that is compatible with both should be absolutely a no-brainer after all.
it should open up many interesting use cases, like for example, we hope to eventually have access for browsers so that the browser itself can be an app talking directly with a lightning node and very much browser first experience should eventually be possible as well.
Stephan (50:07.439)
Yeah. When it comes to developers using Greenlight and maybe, I guess I’m thinking of like the parallel world maybe where they are using VMs and they need to kind of orchestrate or use lots of things at once or spin them down up and down. What does that look like from the developer’s perspective when they want to use Greenlight and spin nodes up and down?
cdecker (50:29.63)
So for developers, there’s very little to do. They can essentially always assume that there is a node in the cloud that.
node in the cloud is reachable. And it’s not up to them to register that node even. It’s actually the users themselves who are using the apps developed by developers are going and creating their nodes in our cloud. And so essentially the only thing that differs from assuming you have a
cdecker (51:09.484)
first have to call register on greenlight. And once you have registered with greenlight and so that greenlight knows that there is a no that should be spinning up, if a request comes in, once you’ve done that, you essentially just take the library and you tell it, hey, here is the private key, initialize the signer with it, and then it starts spinning up the entire rest
on you essentially just say hey node get info node pay this node create an invoice and all of these very simple RPC methods that we expose
Stephan (51:53.915)
Yeah, gotcha. One other question around offboarding. So is the idea then, so as you’ve mentioned, the idea with Greenlight is that these potentially end users could offboard that node to their own thing. Like maybe they’re running an Umbrol or a Raspberry Blitz or something like that. Is that the idea that you could have, maybe there’ll be like an app in these node in a box packages and then you can sort of offboard it to your own thing. Is that the idea or what does it look like?
cdecker (52:12.512)
Absolutely.
cdecker (52:20.478)
Absolutely. So what we run on our servers is pretty much a stock core lightning version with some minor changes. And so it is relatively easy for us to take a snapshot of your database, take your node out of rotation so we don’t start it anymore.
encrypt this backup and give you a link to that backup. And so you have this bundle essentially representing the state of your node, and then you can use a tool to decrypt it, and you can then take that backup and restore from it without incurring any sort of prolonged downtime.
And right now it is a bit of a manual process, right? You get a URL and then you download it and then you have to say, please decrypt this for me. And then you have to set up a database and load it in there. But I hope that eventually we will have a node in a box solution where you essentially just say, hey, export my node. Here’s my seed.
else happens in the background. And this is very much possible. And we actually have a couple of users who’ve already done it. The advantages of doing so are pretty clear, right? We don’t charge for nodes that are no longer running on our infrastructure. And you are not running your
cdecker (54:08.428)
access to some of your data for debugging for example. And so it is very, and we use this price and increase in security and increase in privacy to essentially nudge people to actually, once they seen.
what Lightning can do for them, learn, invest the time. And then once they have caught up with the state of the art of running Lightning to actually take their node and off board into their own infrastructure. That could be a pre-built Lightning node as a box, or it could be their own custom environment.
Stephan (54:50.199)
Yeah. Okay, so last question, just as we’re looking out into the future with Lightning, what kind of features do you think are going to really move the needle? Like, is it Bolt 12 or is it something else? Like, what kind of things do you see that will really bring a lot more people to use Lightning?
cdecker (55:07.87)
Ooh, probably a combination of many, many different changes, but let me just plug some of my favorite ones. So one of the most interesting ones that I’d like to see that from a user experience point of view is the absolute non plus ultra is async payments.
This green light being essentially a system for end user nodes that are fronted by an LSP are a perfect match for async payments, where the LSPs of the sender and the LSP of the receiver essentially negotiate, hey, are you online? Can I now forward this payment to you now that you are online? And essentially we defer the payment until the recipient is online.
line, making it appear as if a payment went through while you were offline. For us that for example, is a huge win. We are looking into a lot of smallish core lightning changes. For example, like I mentioned before, the gossip sync is one of the things that could be better.
cdecker (56:31.892)
to each node because there are private messages in there. And so we are trying to extract them into a separate store such that we can actually share the gossip among all of the nodes and then sort of add the private gossip back on top. Bolt 12 is definitely something that I’m very much looking forward to. It is, it…
allows us to without adding extra infrastructure like LN URL would, for example, allows us to negotiate payments going in either direction, have recurring payments, have payments denominated in different currencies, which again from a user experience point of view, it’s much easier for a common mortal to say, hey, I’d like I’d like 10.
10 US dollars instead of, oh, let me quickly take out the calculator and enter all of the zeros that I have to add before I get to Satoshi’s and then probably have a zero too many or too little and then your factor of 10 off. And so automating a lot of these conversions, for example, mentioned already recurring payments and
cdecker (57:52.812)
in the LN URL space already, we bring back to common mortals that do not know how to run a web server and do not want to trust somebody else to do it on their behalf.
Stephan (58:09.211)
Yeah, fascinating. And as you said, I think bringing back that early experience that a lot of people had from, let’s say earlier days of Bitcoin, where, you know, when people were making on-chain payments, they didn’t have to think about, oh, my Lightning Node being online or the other guy being available at this time. They used to, you know, and some people have an attachment to that level of user experience, which is fair enough for them really, because that’s what they used to. They would just put an address and take a payment to that whenever. And now with asynchronous payments, it can come back to that. Like you could have that with Lightning.
cdecker (58:37.695)
Yeah.
Stephan (58:38.707)
without being custodial and make it feel the same way where you just kind of give someone that invoice, they just pay it and without you knowing, you’ve kind of received this money offline, especially when it’s combined with things like Bolt 12. So I think that could be really cool to see and maybe that’s the next big leg up. But yeah.
cdecker (58:56.67)
Yeah, it very much goes back to this, to this fire and forget mentality that we had and still have for Bitcoin on chain and essentially every payment mechanism that we had so far, right? The moment I press send, I stopped thinking about a payment I just did. Whereas in Lightning, it can take a bit of time for the payment to complete, especially if the counterparty is currently offline.
cdecker (59:26.544)
the actual action being taken under the covers is something that again, abstracts away difficulties from the end user and makes it more approachable and should widen our user base as an end result.
Stephan (59:42.011)
Yeah, that’s a great point to finish on, I think, because obviously a lot of the end users are not really going to think about this stuff, but obviously listeners of this podcast are sort of more developers, builders, advocates, educators, and those people might be interested to know these things because that helps them out there when they’re building things, doing things, teaching people about Bitcoin and Lightning and so on. So we’ll finish up there. Christian will put all the links in the show notes and thank you for joining me again.
cdecker (01:00:06.942)
Hey, thank you so much for having me. It was a blast.
Stephan (01:00:11.493)
Okay.