Stratum v2 Bitcoin Mining Protocol

Hi and welcome to the Stefan Livera podcast focused on Bitcoin and Austrian economics. Today I've got Jan Čapek from Brains and Slushpool talking about the new Bitcoin mining protocol Stratum v2. But first, let me introduce the sponsors of the show. Firstly, look into Kraken, one of the world's leading Bitcoin exchanges. They're renowned for their focus on security. They provide A high quality platform, you can find high trading volume with low fees and no minimum or hidden fees. They also offer twenty four seven support and just recently they've got Kraken Pro mobile app. Kraken Pro delivers all the security and features you love about the Kraken exchange in a beautiful mobile first design for advanced Bitcoin trading on the go. You sign up and you can generate an API key from the Kraken website and sign in on your mobile app using that. Kraken also offer up to five times margin trading, long and short, and they also offer Kraken futures. up to fifty times leverage to benefit from price swings or to hedge your price risk. Kraken offer five fiat currencies, go and sign up at kraken dot com. Next is Unchained Capital. Unchained Capital, I really like these guys, they're doing Bitcoin financial services, they're empowering their customers with unprecedented financial freedom and control. They're using multi-signature, and I think they've got a really advanced approach with collaborative custody, giving the users control over their private keys as well as a financial partner and financial services. So Unchained offer two of three vaults, you can You can use Trezor or Ledger, they're a great option if you want to secure your Bitcoin for the long term, and if you ever need to access liquidity but without selling your bitcoins, Unchained offer collateralized loans, giving you a unique option. All Bitcoin is stored on chain, dedicated multi-signature addresses, it's never rehypothecated, and you can share in the security by holding one of three keys. I'm really impressed with Unchained, they offer excellent services, they're releasing valuable content and open source tools such as Caravan and Hermit, I think you'll Unchained Dash Capital dot com, so previously known as Bitcoin Outlet but now rebranding to twenty-one X dot io. Twenty-one X is delivering rare and extraordinary merchandise to warriors of Bitcoin. Outstanding design isn't just blindly slapping your logo on any object, every product they carry is a work of art with a thoughtful design and in keeping with the ethos of Bitcoin, all products created at twenty-one X are limited edition. Once that product sells out, it's gone. When you purchase something from twenty-one X, you'll be one of the only people in the world. To have it. So twenty one x is a sister company to Canada's Bull Bitcoin, both companies Bitcoin maximalist through and through. Twenty one x only supports Bitcoin. This core belief has led them to align with other unapologetically maximalist companies. So if you want to rock some merch from a designer with an actual moral compass and unwavering maximalist views, go to twenty one x dot io, grab yourself some merchandise, use coupon code livera for twenty one percent off anything in store. Check out givebitcoin dot io, the easiest and safest way to get your friends and family into Bitcoin. Bitcoin, you give, you time lock, and you educate. I've had trouble giving Bitcoin to people before because they lost it, they didn't know what they were receiving. That's why there's massive value in GiveBitcoin. You time lock that Bitcoin for one to five years, and every month for the first year, GiveBitcoin delivers a lesson from a world-class curriculum with input from many well-known Bitcoiners including Safdean, Matt O'Dell, Jan Pritzker, and others. I'm also an advisor with a small equity stake assisting with the curriculum. You can also get Bitcoin as a present

Put Bitcoin on your wish list at GiveBitcoin.io. Remember, it's not just about giving people Bitcoin, it's about advancing them to the point where they wanna become a hodler. So Jan Čapek is co-CEO of Brains and also working at Slushpool, so he joins me in this episode to talk about a new upgrade to a Bitcoin mining protocol, Stratum. And so Stratum v2 is the next generation protocol. We talk about how it can make data transfers more efficient, it's reducing physical infrastructure requirements and increasing security. We get pretty technical in this episode, so if If you're getting a bit lost, make sure you check out the transcript. This is episode one twenty-eight, and my website is stephanlivera dot com. Here's the interview. Jan, welcome to the show.

Hi, thanks for having me here.

So Jan, I had the pleasure of meeting you at, Riga at, for Baltic Honey Badger, and, I know you've been, doing a lot of work on Stratum, the second, version of the protocol. first, I wanted to get a bit of

So I come from the engineering background, and I started operating systems at the university, and back then, in about two thousand and five, that's where I met Pavel. in my first job, we're working for a mainframe company, so it was actually not, you know, the dream job that, a software engineer would wanna get, and then our ways pretty much parted because I went for, for the embedded systems area, 'cause I was really interested in operating systems and Linux. and Pavel was more into, you know, the information systems, banking and stuff like that. But then we got together again around two thousand nine, at a random project where I was building an infrastructure and he was designing an information system. And, around two thousand eleven, we got the idea to start the brains company, and our primary focus was Developing, custom firmware for embedded systems, so gensets and, you know, ECUs for, for cars and things like that. So, that was the primary focus, but then in 2013, we got involved with Bitcoin. Actually, I heard, about Bitcoin around 2011 or 12 from Slash, because, I met Slash, at some sailing trip. He was the best friend of Pavel from childhood. but I wasn't sure like if this is like the, you know, the thing that I should be, spending my time on. But then around two thousand and thirteen, I really realized that this is something that, is going to change the history a bit. And at that time, Slash was already running the pool for two years, and he was interested in moving over to a new project called, you know, Treasure of the Heart World. And he needed somebody responsible, who would take over the pool and get involved and scale it up because it was pretty much a small prototype running on one machine. So that's when Brain's company got involved, and we've been operating the pool since 2013. Up till now. another significant milestone for us was starting the, the BrainOS, the open source initiative for mining firmwares, because we do believe that, since Bitcoin is open source, we should have also, the mining side that's actually securing the blockchain to be open source as well. So this is how all these things, play together.

Yeah, that's a great explanation, thank you for that. And with the different components, it seems that there's this drive towards improving the decentralization and improving the control that the user has rather than trusting, let's say, the manufacturer by running your own open-source, software in that case with the Binance OS. and, so I'm interested also to understand a little bit around the background of how Stratum came together. So from my reading As I understand, there was some of the early days, there was this concept of get work, and then later there was get block template, and then there was the Stratum v1 protocol. Can you help us understand a little bit of the context around that? What are those different pieces and how do they fit together in the, in Bitcoin's mining history?

Okay. So originally, the, you know, the miners were connected directly to the Bitcoin core. The problem was that, you know, the solo mining wouldn't work because people needed a reward evenly spread out through time, and that's where Slash came up with the idea of, you know, concentrating the computing power. However, the existing protocols to distribute mining jobs weren't, sufficient. They wouldn't scale, sufficiently. So basically If a large enough miner would connect to your pool, you would have a big resource problem being able to, to ship mining jobs to, to the miner. So that's when, the idea of Stratum version one, it wasn't called version one at the time, but the idea of the, the original Stratum protocol, came, where, the pool would, ship only, the essential parts of the mining jobs, of the mining job to the miners, and he would be also controlling the difficulty of the jobs so basically the miners were, you know, connected to the pool, they were running on a long, long term connection on TCP/IP connection, and the pool would be supplying the jobs. And if, if, for some reason the miner, grows, it has more hash rate, it's, you know, running this hash rate through, through the single connection, the pool would, immediately see the change in, in the submission rate and would try to adjust the difficulty for it. So this was possible The Stratum protocol because it has this flow control called, difficulty. so that was the, the major invention, and basically with, with this, feature, you could scale, the amount of hash rate connected to your pool almost infinitely. at the same time, This optimization or this, this feature, works, if you try to, aggregate the hash rate on a single connection. the current, state of mining, it, it looks like some farms do want to have like separate connections for every single miner, but still the, the pool is able to control, the, the frequency of submits through the difficulty setting. so that was the Stratum v1, so it was a, a major shift allowing- operating the pools on large scale, and we have been, you know, using it, ever since then. But it had some major flaws, and one of the flaws was that the protocol itself was completely insecure. So it, it's pretty much text-based or JSON-based protocol. So any message sent to the server is clean, clear text, plain text, anybody can read it who's in the middle of the communication, and anybody can change it. So, this is a big risk for, for the farms. And there have been attacks, through, you know, rerouting traffic through BGP or even now today, there are malicious, routers In the infrastructure that actually are able to detect Stratum traffic, and they do some little changes, and basically they can steal any, any amount of hash rate that they like. And if it's some small number, you wouldn't even notice if you, if you're losing like one percent of your hash rate, you could say, "Oh, this is bad luck," or "There's some variance in my hash rate." so this is the first thing that we try to address with, the new protocol. Second part, we wanted to get rid of the, the text protocol completely because it's, not very, resource saving, it's consuming a lot of bandwidth. so we switched the, the protocol completely to binary with the Stratum v2. and, we have also looked into the construction of individual protocol messages, and we tried to design them so that they're more efficient. So basically, messages in, in the original- protocol that we're like combining two things at the same time. For example, you get, you as a miner get a notification about a new block and you're supposed to start mining on it right away. But, you can make this, a little bit more efficient if the pool tells you, "Oh, hey, here you have a new block template, but it's meant for the future, so you as a miner, Store it, the miner, the mining firmware or the proxy remembers it, and then the pool only sends a small notification saying, "Oh, there's a new block that has been found in the network, so please start mining on the, on the block template that I sent you some time ago." so that's the, you know, the, the different eff-efficiency improvements. another part of the protocol that we tried to address is that we wanted to have some Some, controlled and well organized way how to extend the protocol, and that's why we try to, Build the protocol around extensions. So basically any vendor, if they want to, they can design their own extension and they can pretty much run any protocol inside of it.

Third efficiency improvement I would see is something we call header-only mining. Maybe this is a little bit too technical, but, the current mining protocol, what it does, it sends you a lot of data so that you can build your own Merkle root. Basically, you have to build your own Merkle root. because there is a part, in the, in the block header, that, you know, you as a miner need to adjust so that you get more, you get bigger search space. This approach, is still supported in the new protocol because it allows some, you know, advanced features like proxying and, switching, you know, to different pools and stuff like that. however, we have introduced header-only mining where the pool is able to supply a full block template where you don't have to redo the full, Merkle root over and over again. And the benefit is that once you submit the result, the pool doesn't have to do the full Merkle root computation again. It's, so it doesn't have to build the Merkle root, you know, the tree of the transactions again, which is saving the CPU time, but more importantly, it's reducing the latency. So, once you submit your result, the pool is able to, to, evaluate your, your result or your submission, really fast, which also should, end up reducing Reducing, the reject rates. Reject rates, that's another, you know, parameter that miners do care about, as they don't want to see too many shares, you know, the, the results of the mining that they're submitting to pool being rejected because of being stale or for whatever reason.

Gotcha. So, so let, let's just clarify that there. So, you were mentioning, sorry, actually, do you mind if we just take a step back for one second? I just wanna, provide a little bit of context for the Of the software, if you will. So maybe we could just outline the difference and what are the different pieces. So you've got the firmware that's on the machines, you've got a management system, and you've got pool software. So could you just outline a little bit what are those pieces at a high level, and then how does the Stratum, protocol sort of work in with those?

Okay. Yeah, I will start from the pool. So pool is a piece of infrastructure and a piece of software whose responsibility is to distribute mining jobs. it has to do it on timely manner, so basically It periodically refreshes miners with the new mining assignments, jobs, mining jobs, so that the value of the block is, let's say, maximum, so the miners are collecting, as many transaction fees as possible. Also, if there's a new block found in the network, the pool has to notify the miners that they should stop mining on anything that they've been working on until now and that they have to start mining on a new block template because anything that they've been working on After a new block is found, it's completely invalid and will be rejected by the pool. the goal of the pool is to reduce, the, variance in miners' rewards, because all-- we know that, bitcoins are mined in blocks, and the block is worth twelve point five bitcoin now before the halving. you cannot mine One Bitcoin. and this is the reason why the miners do connect to the pool.

Okay, so that's, the pool part and distributing work from the pool to the miner. Can you tell us a little bit about the software that the miner uses in terms of their own management software and then the firmware that's actually on each individual mining device?

Okay, I will try to step, to the end. So let's speak about, about the mining devices first.

when Bitcoin mining started, we had a situation where you were able to mine on, basically on your laptop. The, the original idea was one CPU, one volt. but mining changed since then, and it evolved from mining on your machine And then your CPUs, to GPUs, then the first FPGAs came. FPGAs are field programmable gate arrays, but you can translate this, if you want to design a chip, A piece of silicon, you usually describe the chip in a hardware language and then you synthesize it, and you can test it on an FPGA. So it's kind of like a flexible development platform for, for designing, chips. so this was another speed up in, in mining, and then the ASICs came when, somebody had the idea, okay, we're, we're now good with, the FPGA, and we can try synthesizing a real silicon. And then we got into this- This, you know, performance efficiency race, since then. but why, why I'm explaining it is that originally you were running, a software called CG Miner, On your machine, on your, on your laptop, on your, on your server, whatever. and all the mining logic was, inside of the software. When, the GPUs came, the CG miner software was still being used, but parts of the CG miner were only drivers for the GPU. So there were some parts of code uploaded to your GPU card, and the CG miner was only controlling this piece of hardware. Same thing with was happening with, the FPGAs, and then when the ASICs came, even more logic has been moved out, of the CG miner and, CG miner itself, Is an open source software, using GPL licensing, but the problem was that people weren't like noticing that there was a, a shift in perception of what really open source is and We saw that the manufacturers were producing something that wasn't open source anymore. however, back to the, role of this, component in the whole mining stack. So the firmware itself in the mining device these days, is responsible for accepting the jobs, communicated through a mining protocol, and try to find the solution for such mining job. The, the solution is called a nonce. It isn't important to understand exactly, but this is a number that you can plug in to the Bitcoin block header, and when you do, SHA256 hashing of the, of the Bitcoin block header, you come to a result, and if this meets, a target currently set by the network, then you have a valid block.

these results are being submitted to the server. What the mining farm can also do is that if they want to save bandwidth, they can run a proxy, which is, A regular software running on a server that sits between their miners, their mining devices, and the pool. And these mining devices connect to the proxy, and the proxy aggregates, you know, the, the, the result submission to the pool, and at the same time, it looks, with the current protocol, it looks like One big miner. So this is for, for, for, for this part of the, of the mining stack.

Yeah. And so then, can we talk to the different, pieces within this puzzle? So you've got the devices, you've got the proxy, and then from the website, you've got hash rate consumer and job negotiator, and then they talk to Bitcoin D. So can you tell us what, what the role is of the hash rate consumer and the job negotiator?

With the new protocol, we wanted to provide a mechanism that would allow, miners to choose their own work, and the way we were thinking to do it, which was The, the main idea was inspired by BetterHash, because what, you know, Matt came with the idea that, it would be nice if miners would be able to select their work. The problem with BetterHash was that it wasn't safe for the pool. You as a pool can't simply accept, random work, because part of the work is also the reward for the miners. And the way the current mining works is that Pool sends you a mining job that contains, the miner reward that goes to the pool wallet, and the pool then decides How you participated in mining and since, you know, divides the reward from the block, according to your participation in, in each mining round. So it wouldn't be, safe for the pool to allow the miners to choose even this, part of the block template. And that's where the job negotiation protocol came in place, where, the miner Let's speak about, let's say, mining proxy. So the device, in a mining farm, can negotiate a custom job with a pool, with a, a transaction set that miner believes is the, the set that he wants to mine on, and if the pool approves such a transaction set He can start mining on it. So this is the part of, for, for the job negotiation, job negotiation protocol.

Okay, got it. Sorry, can, before we go further, can we just take one, a step back and just talk about that potential censorship angle that, Matt Corallo was trying to address with BetterHash? As I understand then, the idea would, would be that if- Say a few key big mining pools were compromised, then some transactions could be censored, or potentially those mining pools might be having their hash rate redirected towards a-- to performing a fifty-one percent attack. Could you just outline what was that censorship potential attack, and then how Stratum is trying to help change that with, having the miners select their own work?

Speaking to the, to the censorship attack, the feature in, in the mining protocol is there as a security measure to prevent something like this from happening? However, it is important to understand that even if you have this feature, it's still up to the pool if the pool wants to implement the negotiation part. So if we have an evil pool, doing censorship of transactions. Even if it implements, this protocol extension, it can always-- oh, it's not an ex-- sorry, it's not an extension, it's a protocol. It can always deny whatever templates you try to negotiate. So your chances are that you have to go to a different pool not doing this censorship. So it is a security measure in a way that... If this feature is needed, we can have pools that do support it. But at the same time, if, let's say, all the pools collude and don't support this, JUP negotiation protocol, then miners can't do anything about it. They would have to start their own pools, which, I mean, is also an option. I would see, the job negotiation feature or the benefits of this feature more on a, on the business level, where there could be cases where miners would be willing to pay premium to get their, their own Feature, you know, to, to, to get, to get their own, transaction set being mined, which they think is the, is the right one. Most of the miners are going to shoot for, for maximizing their, their mining rewards, so they want to get the most expensive transactions, you know, selected by Bitcoin Core. But at the same time, there could be some miners that do want to have a specific set.

Yeah, okay. Gotcha. And then, are there any other attacks or, sort of vectors on which, the- The current Stratum protocol is more vulnerable against?

we have covered the man-in-the-middle attack, but I can briefly repeat the, this part. If the current protocol was running over a secure channel, we wouldn't have this problem. But unfortunately, most cases, it's not supporting any, any, you know, encryption or any authentication. so what can happen with the current protocol? The, the only and major, major, problem is, stealing the hash rate by, by, you know, inserting man in the middle and like alternating the, the submissions. That's pretty much it, I would say.

Okay, gotcha. And so one other question, I guess, I'm just curious as well, is how do mining pools know that a miner isn't faking it or bluffing in terms of how much hash power they're contributing to the pool?

that's the whole point of the proof of work, so you, you can't, you can't fake, your computing power, because the, the mining puzzle, the proof of work, has certain difficulty, and the properties of, of this, you know, crypto game is that you have to, Use a certain amount of, energy in order to find the solution, because the, the crypto puzzle or the, the proof of work, is pretty much showing that you have, used a certain amount of energy in order to find the result, because the result is, is a random act, it's, it's a, it's a random event finding, finding a result, and the probability, of finding, the result is always the same. So It doesn't matter what you did in the history or what you-- it doesn't, it, you know, what you did in the history, whatever results you had, doesn't impact what you will find in the future. so it's, it's impossible for, for the miners to, fake their computing power because the, the result that they're submitting directly manifests the amount of computing power they have. And maybe an important feature to, to, explain is that the result is It's always associated with a difficulty that the pool assigned to the miner as a mining task, and this difficulty, directly, reflects, the amount of effort that, that the miner has used. But I can actually think of, there is a, there is an attack, on the mining, which is definitely difficult to prevent, and none of the current protocols are able to prevent it because it isn't, very simple and it would require Bitcoin to hard fork, and the attack is that miners Who actually find their, a block, if they intentionally or unintentionally, fail to submit such block? then it's called block withholding attack. then the pool has a little bit of a problem because they will not notice. It just looks like that the miner has a slightly lower hash rate, but, It's very difficult to detect it, you can detect it only on a, on a, you know, big amounts of data and on big miners. And if the attack is,

let's say, crafted in a very elaborate way where it's distributed through many miners, then it's, a real problem. But this isn't prevented by a better hash, this isn't prevented by Stratum v1 or Stratum v2. It, it can't be easily prevented unless we have, a- Change in Bitcoin, in Bitcoin, protocol so that the miner actually isn't able to evaluate if he or she found a block or not. So at the time of submission?

Yeah. Oh, I see, I see you. Yeah. So they would have to submit their work to the pool without knowing that they had correctly solved the block.

Yes, yes. And there are some proposals, on this, but, there's no solution, and the question is if you really need a solution, because Bitcoin is designed around incentives, so it isn't in best, you know, intentions or, there's no good incentive for miners to, be withholding blocks because they're, you know, damaging their own rewards as well. But it's an interesting attack on, specific pools, for example, PPS pools could easily bleed out because PPS pools are paying by share, so basically If you solve the puzzle for, for the pool on a difficulty set by the pool, which might, which, is much lower than the network difficulty, you get paid for, for every solution, and the, the payment is proportional to the current difficulty of the network. it can be easily computed. And this pool pays out these rewards completely independently from how many blocks it finds. So if, they have a big- Miner who is doing such an attack, which could be detected if, if the miner is big enough, using some statistical analysis Then they would have a problem because they would bleed out, they would be paying money, but, there would be no blocks found on their side. On our blocks, a smaller percentage of blocks, because the other miners are assumed not to do the block withholding.

Right. Like the miner in that case is kind of cheating or scamming the pool a bit out of money because they're just getting paid for shares, but that they're not actually providing, because they're withholding them, right?

Yeah, yeah, yeah.

Right?

Yeah, this is a realistic attack, but, we didn't hear about too many be-- being, you know, detected or, happened. Another case of such an attack is an unintentional block withholding attack, if, for example, you have, a buggy new firmware or if you're rolling your end Case, it should be detected on the pool in a way that, such shares are actually invalid. So, I mean, there are different cases, so it doesn't have to be always an intentional thing or because the code path in, in the miner itself, let's say, if you lo- if you can imagine that there's, there's a piece of code in your, in your mining software that's saying, okay, if I found a block, I just want to know about it, so I, I increment this counter, but this code path is not Being executed too often, right? Like how many times your miner finds a block. And if the software isn't properly tested, which for example, for the CG miner, there's no, there's no test suit available Then what if the software crashes in that exact moment where it's trying to increment that counter of found blocks, and since it's going through a code path that is never-- that is executed very rarely, there could be a bug in there that causes a crash? Some other problem, where the, the software actually doesn't get to submit the block because it's not running anymore. And this can happen.

Right. Yeah. ho-hopefully that sort of thing would be caught in a test, before the software rolls out, though.

That's why we're redoing the, the CG miner thing, called-- Our project is called BOS Miner. It's, it's an essential component for, for Binance OS that's coming out, hopefully for, for, December, or end of which is written, in a modern language and tries to address all these things and is, supporting Stratum v2 from day one. So, yeah, we do, we do realize that this could be a potential problem as well.

Gotcha. And, while we're on the topic of different, these different pieces of software, could you just give us an overview, as there are different, pieces of software out and about, out there in the system, out, out, out in the wild? So as I understand from, you know, from your Which is the operating system, and then you've got, BOS Miner, which you mentioned, and then you've got Stratum v2. but in terms of broader out there in the, in the Bitcoin mining world, I, I presume there are other pieces of software, as you mentioned, there's CG Miner, and there might be, there are other, protocols that might be in play. Could you just give us an overview of the Brain's stack compared to some of those others?

okay. When, we started the BrainSource initiative, we have looked at the current state of the mining, and like I tried explaining with the evolution of, the mining hardware, of the mining devices. Today, the miner is no longer just the mining software. You need, an operating system running on some embedded device, which is another essential component, which typically is closed source or is whatever the, the, the manufacturers publish is usually like incom- Complete. We started looking into this sometime around two thousand sixteen, two thousand seventeen, and we saw, oh, this is, not where it should be. I mean, if you wanna run a Bitcoin core, you just download the software, you compile it, and you're part of the network. If you want to build, firmware for your mining device that you bought for, for your money, you don't have, such great possibilities because, for example, Bitmain, they, they published the, the BM Miner, which is actually a fork of CG Miner. So Everything actually starts from, from CG Miner in this ecosystem.

but that's only the software, and it would be very difficult and challenging to find all the bits and pieces to run, the full firmware image on, on the mining device. And the question is, what else, could be broken, hidden, whatever, compromised inside of the miners if you don't have the full control of the stack? So We try to address it with BrainOS, where you have the full operating system, which is open source, it's based on, OpenWRT, a generic Linux distribution meant for routers and embedded devices, and then we used, a snapshot of BM Miner for, for the S9s, and we started developing from there. So for the time being, the current latest release, which is somewhere from June, still contains the BM miner. But in parallel to this activity, we also realized that we need to start writing a new software for mining called BOS miner, which would be a parallel thing to CG miner, bringing basically CG miner eventually to end of life. the reasons for, for doing this is that the current CG miner code base is Very cluttered and it isn't in very, very good shape throughout the years. Because what manufacturers did, they usually took the code base, they, they forked it, usually did some, you know, breaking changes in, in the source code, and usually never contributed it back to the original upstream CG miner project, which is also a violation of GPL, which I think is, kind of serious, where, you know, we're, we're dealing with money here and, People aren't willing to operate their mining devices, with closed source firmware that doesn't, you know, have proper audits, they, you know, you don't know what it's doing. We had affairs called end-bleed where there were backdoors from Bitmain allowing them to, shut down the devices even though the feature was, advertised as a, as a management feature, but nobody really knew about it. We had affairs like ASIC Boost, which is another manifestation of, the, you know, the firmware stack being closed, where Where, the S9s were able to, to, to do ASIC boost, but, but the way it was configured in, in the firmware from, from the manufacturer was that it was actually disabled, and if you enabled it, it was generating incorrect results, so you couldn't really use that feature, and that was actually preventing you from saving thirteen percent of energy. So these are all the reasons why, why we decided we want to build a full open source stack and make it a, a go-to place for Let's go for the community. I would, ideally, I would like this to be go to place for the industry, somewhere where the Linux kernel is. If you wanna build an embedded device, you go, you go to kernel dot org, you download the sources, you know, you add your drivers and hopefully contribute it back because you don't wanna maintain your own, fork of Linux kernel, but some manufacturers, also keep their forks. Yeah. So this is where the full Brainz OS and BOS Miner stack fits in, and I would like to say that currently, the Brainz OS is still running CG miner, but this component is going to be replaced, pretty soon. Great.

yeah, sorry to go back a little bit, but, you mentioned earlier rolling the end time, and, in my reading, I was looking, I was looking up this idea of version rolling bits. could you give us a bit of a- Context, what, what exactly is that, and is, is that related to header-only mining?

Yes. version rolling is, a feature that allows you touching, certain version fields of the Bitcoin block header that have been specified in BIP, in BIP three hundred, twenty, if I remember correctly. Yeah, I think so, yeah. Yeah. And these bits are can be used to extend the search space, because currently, the, the way, Bitcoin block header has been designed is that it only allows you to roll, a nonce field, which is four bytes. Which, a current miner is able to roll through in, in milliseconds, which is like no time to even like supply fresh jobs to it. And then you also have anytime, which are supposed to touch only every second, which is not fast enough. And then, you have, the version field, which, consists of, sixteen bits that can be freely used, and this extends the space to almost two to the forty-eight, if I'm correct. And that Two to the 48 doesn't really mean anything, but maybe some people do know. S-nines are roughly 16 terahash, and this space, is, enough to have, a miner that has roughly 280 terahash per second. But in that case, you would have to really supply jobs every second to the miner. Yeah, so, so this, this is it.

Gotcha. And so, as I understand then from reading, it looks like the Stratum v2 protocol has this natively, this version rolling feature, whereas in the past it was, it, this was like an extension to the original Stratum protocol.

Yeah, this, originally it was an extension really meant to allow machines that were supporting ASIC boost only to operate. So So the V1 has something called Mining Config, which is an extension that allows to negotiate, you know, protocol features, and one of the features was to specify the number of bets that you can, or that you are willing or need to roll as a mining device

Gotcha. Okay, and also you, sorry, we are jumping around a little bit, but there are just different areas that I wanted to jump into. So, the mining protocol, Stratum v2, defines three types of communication channels. So you've got standard, extended, and group channels. Can you help break that down for us?

Oh, sure. Standard channels are meant for, the header-only mining. So these are like the most, efficient way how to distribute work and present- The, the least, load on the server side when, when, the pool server are ver-is verifying jobs. Extended channels provide Much more flexibility so that you as a mining operation can, run a proxy that's actually distributing the work in a way that you like to have it distributed. so basically the extended jobs are, are somewhere close to the original Stratum v1, where the miner had to roll their extra nonce two field, and the extra nonce two was a field inside of the Coinbase transaction, which, was actually specified by the pool like how much space I give you. Typically, the pool gave you somewhere between four and eight bytes, which, extends the search space, for, the mining jobs, for the mining job pretty much infinitely. I, I mean, for the, for the validity of the job, the job typically, gets updated every twenty seconds, so it was more than enough to feed a, a big mining operation with, a job, and then the mining operation would be able to generate like sub-mining jobs to, to its, physical miners. so these were the, this is the extended job, and then The extended jobs, can be supplied to standard channels or extended channels, and this may be a little bit, confusing. But to explain it, if you're running a proxy that wants to connect through standard channels to the mining server, the mining server can send only one extended job to this, proxy, which is meant for all the standard channels downstream for the miners. The concept of group channels, then in this particular use case, denotes a set of channels that this extended job is meant for.

Okay. And help me understand here, i-is this the idea as well to help from a bandwidth saving perspective, or is it a computational power saving perspective, or what's the main, objective of these different, communication channels? This is,

yeah, yeah, this feature is- So meant for bandwidth saving and also for latency saving, because it's for, let's speak about a, a, a real scenario. Let's say you have a farm that has hundred thousand miners, and you have a proxy. it's more efficient if the pool supplies one extended job to this proxy, and the, I'm speaking still of, of header-only mining, so, the, the server is able to prepare, The Merkle roots for all these, for all these hundred thousand miners locally, but it doesn't have to send all these Merkle roots, so basically sending standard jobs hundred thousand times for every single miner, but it only sends one single extended job, and the proxy then distributes, these, jobs to the miners. So it's saving the bandwidth, and it's also s-saving, the, the latency, which, could eva-in the end effect could reduce, the reject rate. Because the, the miner would know about the job,

sooner. Yes. Okay. And, you mentioned the reject rate there, and as I understand, that's also related with the stale, as I understand, that might be where, let's say, I'm a miner, and then the mining pool that I'm contributing to, they've already found a block, but I didn't know that, and I contributed some work after the fact, and now that work is no longer useful. Is that correct? could, could you help us understand that? What,

Reject rate, if, if you send a result to the pool or if you submit a result to the pool, the pool has to do the validation and it can reject the, the share for various reasons, and one of the reasons is that, the share is no longer valid. I mean, it is valid technically, so it fulfills the difficulty that, that you've been assigned, but it is not valid because a new block has been found in the meantime and you're supposed to mine on, on a new block. and the new protocol tries to address This, this, reject rate issue by supplying you with a new block template as fast as possible, and actually it sends you the block template ahead. So you get a new block template with a flag, with a, with a future flag saying, "Oh, this is a block template that you're supposed to start mining when I tell you." And then when a new block is found, the pool only notifies you that you should start using this block template. And since, the block tem- Template, i-is a little bigger, message than, just this sm-small notification about, you being, supposed to mine on a new block. This also reduces the, the, you know, the chances that you, will be mining on, on the stale job, on the old job, for longer time than needed. So it doesn't eliminate the, the reject rate completely. there, there will always be some, you know, rejection because, you know, this is essentially a race condition in, in, in the mining, but it tries to minimize it to, to the very, to the low-

Gotcha. Yeah, thank you for that. And also with, sort of related, empty blocks. So, a-as I was reading, it looks like Stratum v1 is slower to send a full block than an empty block, whereas Stratum v2 has been designed to make it so that there's no extra delay to send a full block versus an empty block. So could you help us break that down? What's the-- Maybe we could just start with, what is the problem with empty blocks to start with?

I would say empty blocks overall aren't a problem. you can- We can look at it, from different perspectives. Currently, when, the Bitcoin network finds a new block, it isn't a simple task for the Bitcoin core to generate a new template. It literally takes Second, seconds, it, it really takes some time. So it's better for the miners to have something to work on than waiting and wasting the energy. I mean, and shutting down the miner isn't an option because, I mean, you can't shut down a hundred megawatt operation that would create glitches, you know, in the grid and- It's impossible. So the miners are sort of like race cars, they have to go like full throttle all the time. That's the most efficient, you know, approach how, how you have to run them. So you have to feed them with the jobs, otherwise you're wasting the energy. so with the current Stratum v1 protocol, it is better to, for, for the miner to have, an empty, you know, block template than, not work at all. At the same time With, the empty blocks, what we were thinking about with Stratum v2 is that if we can send, an empty block template in advance, we may be able to send a speculative block template that contains a complement of transactions that aren't in the current, in the current block template that's being mined, by the pool. And if the, if the transactions are shifted in the mempool by some offset, this could work very well. So chances are that if, a new block is found and the miner actually starts mining not on the, on the, on the empty template, but on a template that already contains some transactions that have been lower in the, in the mining pool, if somebody else finds a block, chances are that this template that you already supplied to the miners would still be valid, and then you're actually gaining time because you gave the miners already useful work that contains the transactions, and you, as a pool, can start negotiating with Bitcoin Core to generate a, a new block template for you, which is like the current valid one, and then you just, you know, refresh the miners with the new template, which is, not, a real-time process anymore. You don't have that, you know, that- That line where you really have to give somebody the, the, the new mining work as fast as possible, the new block template. Does it explain, the question?

I think so, yeah. And I, I guess maybe just a question to help improve my understanding here. So for example, typically miners, like if you're fee maximizing, you wanna take the highest fee first, right? but then is it a possibility then that as part of this, template getting sent in advance, there might be some lower fee transactions that are being sent as part of that, and then you might, if you're lucky, get some lower fee ones, confirmed as part of that?

That, that could be, the case too. I mean, the politics on choosing the, the alternative block templates can be different, and we didn't want to specify, a policy inside of the protocol. We would like to leave this up to the implementers. I could see that the pool actually sends, multiple block templates to, to the pool, not just two, not an empty one, and not a, a full one, but there could be like a couple more. But again, as I said, we would like to leave this policy up, up to the implementers.

Great. And so one other one I was curious about is, you've got here zero time backend switching. So as I understand then, this means that a miner can s-switch which pool more easily that they want to contribute hash power to. Could you outline what that is and what that process is?

When, I think I need to explain, what an extranonce one is, first. Sure. When a miner gets a job, the job is unique for, for that specific miner. And part of, with the v1, part of this job was something called extranonce one, which was, a value injected into the Coinbase, transaction. If you wanted to supply A job from a different backend, let's say from a different pool, let's say you were a proxy distributing such jobs, with the V1, you would have to, completely restart the mining session, or your miner would have to support an extension called Externons One Subscription, where the proxy or the pool was able to actually notify, "Oh, hey, here's a new Externons One, which is like the, the thing identifying, your mining session, and here's a new job, so please use it with this one." whereas, in the new protocol, we already do have, this feature, we call an Externons prefix, built in. to the protocol, so the, the pool or the service serving the job. I don't wanna say pool, because pool usually has no, incentive to distribute jobs from other pools, but in case of, switching to different algorithms, for example, sorry, to different coins, for example, you want to be able to supply jobs for Bitcoin, Bitcoin Cash, BT- Bitcoin SV or whatever, then, you don't want to harass the miners with say Oh, please, can you just connect somewhere else, and then there you're gonna get a new job. But instead of it, the, the service distributing such jobs can just notify the miner, hey, here's a new extranonce prefix which identifies your, your mining session, and here's a new job, and please use it. That's it. So, so we have, built this extranonce one subscription into the protocol.

Okay, great. and are there any barriers that you see to- To miners adopting Stratum v2, are there any obvious downsides or negatives that they might face?

I think the, the one barrier would be The firmware adaption. If you wanna take a full advantage of the protocol, you would, you need a firmware that supports the protocol, so it doesn't go to miners, but it really goes to the manufacturers. We try to address this fact by providing the BrainOS and the protocol stack also as reference implementation. In Rust language, so it should be fairly easily readable, very stable, and people can basically build their work on top of that. On the pool side, I see there are really good incentives for implementing the protocol because the pools would pretty much start saving bandwidth immediately, which would be manifested immediately in the quality of their service. They can support, you know Higher submission rates, so the more frequently the miner actually submits to the pool, because you're saving on the bandwidth, you can technically use the, the data, you know, to, to submit more results more often, and this, directly, It's reflected in your mining rewards, because the miner from pool perspective is sort of like a small Bitcoin network, because it's, it also has luck, it's trying to find a share with certain difficulty, and at sometimes it has, you know, more luck in some, in some periods of time, just lower luck, and the variance in the luck is, related to the difficulty that the pool assigns to you. So if you have a difficulty that means that you find a block every one second And so this one second is, because it follows the Poisson distribution, I think there's a sixty-three percent chance that you find a block within this one, sorry, the share within this one second, and there's like, ninety-five percent chance if that you find it in four seconds, whereas if you, are assigned a difficulty so that your submission rate is like five seconds, for example, then, the chance that you find the share within five seconds is still the same, sixty-three percent. so sixty-three percent of the shares are found at this time, and ninety-five percent of the shares will be found within twenty seconds. And, when you look in the statistics on the pool side, then you will see like a variation in your hash rate. And since the pool, when it divides the rewards, looks at your, at your hash rate, if I oversimplify the rewarding scheme, at the time the, the block found, the block has been found by the pool, then you may see a variation in your rewards. So this can be directly manifested. But Another incentive that I would see for pools to implement this feature is also the security part, which we didn't cover yet at all. I was just speaking about the man in the middle, but we try to, propose, an, basically an industry standard extension based on noise protocol framework, which is a well-proven, you know, cryptographic- Framework to build, handshake protocols, it's being used by Lightning Network and WhatsApp and, and many other, and it's essentially a platform that allows you to generate a handshake protocol that you like without making mistakes, and, security flaws. It's built, it's built on modern crypto, so all, all the, communication eventually after the handshake is done is, encrypted and it's using authenticated encryption by default. So basically, any message that has been tampered, can be easily detected and at the same time it's, it's encrypted. It's called authenticated encry-encryption with associated data. yeah. So I, I see, I mean, the, the incentives should be clear. what I find, a little bit challenging is that we need to, polish, certain things. For example, for the security part, we need to decide how we are going to standardize the way you- Distribute the, the public key of the pool, because, the whole point of encryption is that it is nice, but it would be worthless if you have no way to verify that the pool that you think you're talking to is the pool that you really wanted to connect to. so, you know, in, in the web world, we have HTTPS, where you have some X, X five o nine certificates, and here we should come up with, a little bit more loaded and, you know, more flexible scheme that would not, you know, have all these, you know, administration burdens like, like X5 or nine management has. so this is the, the only challenging part, which is, like a precondition to run the, the noise, protocol framework, protocol that we designed.

Great. And as this Stratum v2 has had input as well from, Matt Corallo, a very well-known Bitcoin developer, known for his contributions on, many, mining-related contributions as well. And also a security review by Peter Todd. Have you had any initial feedback from the community or from other industry players?

we're talking to big farms who are actually, commenting the docs pretty loudly. so far we didn't get any like major comment saying, "Oh, this is a complete no-go" or "There's like a huge flaw in the protocol." but we need to really polish, small things. So the, the feedback so far- That we got seems positive. The collaboration with Peter and, Matt was, I think beneficial to the whole project because, again, like I said, we do We use BetterHash as an inspiration for, for the decentralization part, and Matt does understand that it's important to have, an industry standard for the new mining protocol, and he's actually one of the co-authors, of the, of the new standard because we really wanted to get him on board. So all this, adds credit to, to the whole, initiative of, you know, fixing the protocol.

Great. And for any miners and mining pools who are using Stratum, the original protocol? Stratum v1, we might call it. Are there any complications around, let's call it backwards compatibility, or is there a layer or some way that, these, different, miners and pools can talk to each other?

We, the way we would like to proceed with rolling out the protocol once the standard is stable or considered final, is that, pools would be pretty much immediately able to provide the v2 service because all they would do is they would just- Put a simple V2 to V1 translating proxy in front of their service, so they wouldn't be risking any software issues or bugs or anything. And this would be like the immediate step, and this would bring, immediately the security part and the bandwidth saving part. It wouldn't be, it would still not be able to cover the, efficiency improvement on block template distribution because the, the old V1 protocol doesn't support it, and if you just use a simple proxy, it can't do more than that. But then once this is proven to work, they can, you know, start implementing it in-natively into their Stratum servers. And this is exactly what we started doing on Slashpool, where we are already running, v2 to v1 proxy so that miners can actually test, the service already. It's on v2.stratum.slashpool.com. And the next step would be, you know, once, once the standard is stable, to support, The protocol natively inside of the Stratum service with all the benefits.

And, I guess if we were to just turn to just Bitcoin mining and running a pool a little bit more broadly, can you offer us some insight into what are some of the scalability challenges just running a pool just generally?

well, generally the, the scalability challenge is that your pool is, distributed throughout the globe, but at the same time you, have to, you know, collect data from, from the- The whole pool in order to do all the accounting, and the challenge is that, the connectivity doesn't work out very well globally if you have operations in China and if you have operations in, in, the US and Europe, sometimes you have outages, so you have to find your ways around it with, redundant connections, IPsec links, you know, that allow you routing through, your own traffic, and so on. So these are the, I think these are the biggest challenges, even if- If you try to move over to cloud, we partially use cloud services as well, it's also not a hundred percent guarantee that you would be, a hundred percent time available, so you need to div-diversify, and then you have the problem back because you need to connect those, cloud data centers, so that they can talk to each other. so this, I think this is the biggest challenge basically, trying to, make sure your infrastructure, retains the connectivity globally.

great, yeah. a-and another comment I've seen, is this idea that amongst the Bitcoin mining industry, that there are some moves being made amongst pools to try to be more of a one-stop shop for their customers, so they might try and provide hardware needs and financial services needs at the same time rather than just being a traditional pool. Can you offer any of your thoughts on, what's your view on that?

I think this is definitely the way the industry is moving towards, and we are- We're also trying to participate in this movement, so we're trying to work on, on new services, that would cover also these areas.

Great. and, one other question just around, with Bitcoin mining pools and traditionally in the-- Well, coming from like a, even a financial services world or technology world, there is this whole idea of, SOC one and SOC two audits, right? Like technology controls audits. Is that something that you think Bitcoin mining pools- Would do as well to provide comfort to investors, or is that something that you don't really see as necessary?

I think it is an, interesting area to explore because, you know, investors, who are entering the mining industry are facing, a small reality check, for example, if you try buying, mining hardware, how does it work these days? You just pay a lot of money in advance. Is this an industry standard, in other- Other domains, I'm not sure about it. So, I think, those kind of audits, would be something that could actually be demanded and could be, a good, you know, product for, for some of the players in the industry, I would say.

Fantastic. Well, look, I think they're most of the questions I had, but, I guess just summarizing my understanding then, so Stratum v2 as a proposal and a protocol, I, I think the- The main benefits that I can see from our discussion is, well, I guess it helps decentralization of Bitcoin's mining and, helping stop that censorship as we mentioned. It, it helps, stop that man-in-the-middle attack, and there's also a bandwidth saving from moving from the JSON format into, the Stratum two format that's being used. but I suppose, in terms of next steps going forward, what are you mainly looking for? Are you looking for review comments on the protocol document? Are you looking for contribu- Contributions, what would you like to see from, the listeners?

Yeah, I would like to see commands. I think the most, important part is to finalize, also the security protocol. So basically decide on, on the cipher suit, which Currently, I think there's only choice of two, which the industry is going towards, is the AES in CCM mode and there's the ChaCha 2018-13-05 or whatever it's called. one is, better on the server side because you have hardware acceleration, the other one is better for the arms. So if you, if you wanted to have the, the encryption also on the firmware level, that the second might be a better choice. but more importantly, we need to, really- We look into what would be the industry standard for distributing the public keys, of the pools, sort of like a certificate, and this certificate, Would be, then used by the miners to verify that they're really, talking to the pool that they think they're talking to. this would be for the security thing. an important part, that still needs to get done is, changes in the Bitcoin core so that it's able to supply multiple block templates in a more efficient way. we didn't talk to, the topic of, template distribution protocol today, but this is, this is, what it is about, it's essentially a protocol between the Bitcoin Core and, and, and the, the provider of the jobs, so that the job provider is receiving a stream of, Bitcoin jobs that he can provide to the miners to negotiate. So this needs to be, this still needs to be, designed and implemented on the Bitcoin Core side. But the, the, the protocol itself has been outlined, so the challenges would be on the Bitcoin Core developer side. That's about it.

Okay, great. Well, look, I think that's, that's it for today, but, Jan, if you could, make sure you let the listeners know where they can find you and find the protocol as well.

if you wanna research the protocol, just go to stratumprotocol.org. if you're interested into our open source, initiative called Brains OS, go to brains-os dot org. if you wanna check out our pool, just go to slashpool dot com. And you can find me on Twitter under Jan Brains.

Fantastic. Well, that's, that's awesome, it's been very educational for me, so thank you for joining me today.

thanks for having me in your show. Hopefully, I wasn't being too technical, but, I mean, this deserves it a little bit.

I hope you found that useful, and if you did, make sure you leave me a review on your favorite podcast platform. Otherwise, find the website stephanelivera dot com. Thanks for listening, and I'll see you in the citadels.