{ "episodeId": "SLP775", "speakers": { "stephan": { "name": "Stephan Livera", "role": "host", "tag": "STEPHAN" }, "brandon_black": { "name": "Brandon Black", "role": "guest", "tag": "BRANDON" } }, "segments": [ { "speaker": "stephan", "time": "00:00", "start": 0.0, "text": "Hi everyone, welcome back to Stephan Livera Podcast. Today we're going to be talking about, well, we're kind of, I wanted to do a little mini series about non-Bitcoin Core implementations of Bitcoin and, my friend reardencode, aka Brandon Black, is doing one, it's called rbitcoin. So I wanted to welcome Rearden back to the show." }, { "speaker": "brandon_black", "time": "00:20", "start": 20.0, "text": "It's great to be here, Stephan. Great to talk with you and excited to share what I've been doing with rbitcoin." }, { "speaker": "stephan", "time": "00:24", "start": 24.0, "text": "So, obvious questions first, why do we even need non-Bitcoin Core implementations of Bitcoin? What, what, why from your perspective?" }, { "speaker": "brandon_black", "time": "00:34", "start": 34.0, "text": "So for me, there were, there were really two or maybe three reasons to make rbitcoin. One is that Bitcoin Core at this point is old C++ software, and as such, it has kind of some baggage that comes with being old C++ software, and I wanted to see in some ways, this is similar to why Roasbeef beef made btcd. You know, he didn't like C++. He wanted an implementation in Go that he could work on. So I wanted to see, you know, if we do it in Rust, which is kind of the more modern language that has a lot of the same, let's say, utility++ but with better, better protections, better type system, et cetera. What would that look like? Would it be performant? Would it be bad? Like, what, how would it go? So that was one reason. Another is that I think, you know, Bitcoin Core was developed before we knew how Bitcoin would grow over the years. And so the data structures used reflect almost a naivety of the future. And they have done an absolutely, and I mean this in absolute sincerity, incredible job over at Bitcoin Core of growing with the network. And as you know, Jameson, Lop does, does benchmarks and Core has mostly managed to keep sync time the same, even as the blockchain and the UTXOs grow on equivalent hardware. And that's just an exceptional achievement. But I was curious again to see what would be different if we used a different kind of data structure. Could we, could we compress the blockchain? Could we, could we have better performance that's more stable over time?" }, { "speaker": "brandon_black", "time": "02:01", "start": 121.0, "text": "You know, what would it look like? So different data structures. And then, and then I guess lastly, I've had my challenges with the Bitcoin Core organization structure as well over the years. Again, they do an exceptional job of a lot of what they do, but I don't feel like they're They're playing a leadership role in the Bitcoin ecosystem. And so I wonder also, and I was talking with John Attack on, how do you say his name, on X earlier today about this, you know, how could we look at different possible governance models for Bitcoin implementations? And so at this point we have now quite a few different Bitcoin implementations out there. There's rbitcoin, there's Lib Bitcoin, there's btcd, there's satd, there's Hornet, I think there's a few others I'm missing even in there. And as John said, each of these has kind of a strong leader as opposed to a decentralized leaderless group. And maybe there's a benefit to that. So all of this kind of comes together. Oh, oh, even a fourth reason actually, I am frustrated all the time that I have to run two demons to get my wallet that use Electrum or Esplora protocol to sync connected to my Bitcoin node. And so I wanted a single demon. Now, of course, there are other implementations that do that, so that's not on its own a single reason. I think both satd and libbitcoin also provide Esplora and Electrum in the same process. Yeah, and maybe even Fluoresa there too. Are you gone? Fluoresa, yeah. Anyway, so those are all of my reasons for building it. And I've always wanted to do something like this for years and years, like almost a decade basically." }, { "speaker": "brandon_black", "time": "03:31", "start": 211.0, "text": "But without AI couldn't do it, right? I'm not going to manage a massive project or get a budget to build if I have to hire developers, but now with AI could build it mostly single-handedly. Some, some contributors now that I, that I really appreciate," }, { "speaker": "stephan", "time": "03:45", "start": 225.0, "text": "But" }, { "speaker": "brandon_black", "time": "03:45", "start": 225.0, "text": "Yeah." }, { "speaker": "stephan", "time": "03:46", "start": 226.0, "text": "Yeah. So one other thing, before we get to the AI thing, I also just wanted to, maybe you can set the scene a little bit because this is the first episode in this little mini series, let's say. I'm going to, I'm going to try to get some other people on. Obviously, I'll try to get Laolu on from btcd, maybe Calvin as well. I'll try and get some other people on, but can you just give a sense of like, you know, rough numbers. Who, how many people are using Core? What percent is on Core? What percent is on using some of the others? Like, I think it's about 80. Last I checked on, actually, you know what, let's just get the numbers right now. Coin.Dance, of the 25,720 public nodes running on the Bitcoin network, 21,385 are Bitcoin Core. It says 4,300 are Bitcoin Knots, but I presume that's pre, obviously pre-110, you know. Then there are 17 BTC-D nodes. We have two libbitcoin nodes. And so it's basically it's Bitcoin Core. Like that's, can you just comment on that on the breakdown there?" }, { "speaker": "brandon_black", "time": "04:39", "start": 279.0, "text": "Yeah, I mean, I think as Toshi commented, there's risk of consensus divergence when you run an alternative node implementation because there's no spec for the Bitcoin protocol. And so I think that's why all the public nodes essentially, it is a rounding error other than that, run Bitcoin Core as their main consensus implementation. So I've been talking with some folks since I started rbitcoin on the topic of how can we change that? How can we make it safe to run multiple implementations? And there's a lot of different opinions about that. So Toby over at Hornet, he's trying to create a written spec for consensus so that we can move away from that idea that you have to run core to be in consensus. If we have it written down, we can actually check every node against those consensus rules. And then Epoch over at satd, he runs his own consensus rules in parallel with libbitcoin consensus against every Every transaction. So he's always checking with every transaction. Specifically, is that" }, { "speaker": "stephan", "time": "05:37", "start": 337.0, "text": "The libbitcoin kernel, or I think it's now been called Bitcoin Core kernel, the thing that satd is doing, that thing, or another thing? Yeah," }, { "speaker": "brandon_black", "time": "05:43", "start": 343.0, "text": "Yeah. And then, well, and then it's imported into Rust as libbitcoin consensus." }, { "speaker": "stephan", "time": "05:48", "start": 348.0, "text": "Yeah. Which is, which is confusing, but different to libbitcoin by Eric Voskuil, right? Correct." }, { "speaker": "brandon_black", "time": "05:53", "start": 353.0, "text": "Yes. We have to be clear on exactly what we're talking about. Anyways, he runs parallel consensus in satd, and then rbitcoin is taking a different approach where we started from a clean consensus implementation. And then we try to assert our compliance through testing. So, DeRogge did some fuzz testing based on the differential fuzz against Core. We then imported our own fuzz suite where we do differential and direct crash testing against rbitcoin compared to Core. And then we also run directly many of Core's own tests against rbitcoin to check our consensus. So rather than trying to either do it with a textual assertion or do it With a parallel execution, we try to do it through automated testing. And I think that's one of the big focuses of rbitcoin is that automated testing. So anyway, there's lots of questions there. I think" }, { "speaker": "stephan", "time": "06:42", "start": 402.0, "text": "We" }, { "speaker": "brandon_black", "time": "06:42", "start": 402.0, "text": "Will get to a future where, where consensus is not defined by core alone. That's my expectation for the future. It's just going to take some work to get there." }, { "speaker": "stephan", "time": "06:51", "start": 411.0, "text": "There are others who are skeptical that that could ever be a thing, right? As I'm sure you've heard that argument. Why do you disagree with them? Like, why do you think this is possible when there are others out there, some of them long-time Bitcoin developers or researchers who think it's not realistically possible?" }, { "speaker": "brandon_black", "time": "07:07", "start": 427.0, "text": "I think they're" }, { "speaker": "brandon_black", "time": "07:10", "start": 430.0, "text": "Discounting the power of automation in this. And again, AI has changed a" }, { "speaker": "stephan", "time": "07:16", "start": 436.0, "text": "Lot of this." }, { "speaker": "brandon_black", "time": "07:17", "start": 437.0, "text": "Well, AI has changed it in making it easier to build the automation, but the automation itself is not going to be AI driven directly, right? Fuzzing is a, is a, is a mechanical process, not AI, but AI can help us build the right fuzzing harnesses. And AI can help us build the right, we added a thing called mutants, mutation testing to rbitcoin recently, and AI has, has helped me be able to run the core functional tests against rbitcoin. So there's like a part that is AI driven, but, but the reality is that we can, we can build automated testing that gives us extremely high confidence that we are not going to have a surprise consensus divergence, because every time we think of a way consensus could possibly diverge, we can add an automated test that makes sure it's exactly the same as what core does. And that's something that is easier now because of AI, and then that automated test checks that every single build, every single day, and we know we're not diverging from core's consensus because of the automation we built." }, { "speaker": "stephan", "time": "08:08", "start": 488.0, "text": "Interesting. And so does that, I guess I'm trying to understand, is that like a test that lives outside of rbitcoin, or is it more like the user who is running rbitcoin is kind of having to run both effectively because now he's having like the double load of like checking it against cores rules and then checking it against rbitcoin rules." }, { "speaker": "brandon_black", "time": "08:29", "start": 509.0, "text": "So the last part is how satd does it. Well, they run cores consensus inside their process and check both. In rbitcoin, it's at build time. When we do, every time we make a commit to rbitcoin, we run a suite of automated tests, some of which do, and actually, sorry. And is this CI? Is this CI?" }, { "speaker": "brandon_black", "time": "08:50", "start": 530.0, "text": "Yeah, CI checks all the main automated rules, and then on a nightly basis, not on every PR because it's a little bit expensive, we do differential fuzz testing where we spin up both nodes and we hammer it with random shit to make sure they both give the same answers to random shit. And so as long as every, every night we do like an hour long run and every week we do I think a three or five hour run, we run millions and on the weeklies billions of permutations against both Core and rbitcoin to make sure they both give the same answers." }, { "speaker": "stephan", "time": "09:20", "start": 560.0, "text": "Interesting. And obviously for listeners, maybe if you're a little bit newer, obviously in Bitcoin, because it's decentralized, we want to make sure we all stay on the same chain. And the point you're getting, the point Rearden is getting at here is obviously if one of the implementations disagrees with the other, that could be kind of catastrophic depending on who you are, right? Because if like, and especially like right now, as we said, it's a, it's kind of like a rounding error of people who are non-Core implementations. But imagine a future where, you know, 10% of the network is running this and another 10% is running that. And if there were to be a consensus, you know, failure or, divergence, that's like 10% or 20% of the network forking off. And we can't, we can't afford to have that, right?" }, { "speaker": "brandon_black", "time": "10:05", "start": 605.0, "text": "Yeah, exactly. And so it's super important and that's why we have the automation running billions of permutations to make sure that we are in consensus. And I think over time, what will happen, and this is why it's kind of good, in my opinion, that rbitcoin started with this. Like we got that fuzzing in there within the first two months of its existence. Over time, the millions and billions of permutations run every single week will hit essentially every possible code path in consensus. And so we'll, we'll develop over time with this automation confidence that we don't diverge from core. And I think, again, with every implementation that's out there trying to compete with core has to have some story they tell that says we are in consensus. So satd does it one way, rbitcoin does it another way. Everyone does it in a different way, but we have to have a strong story that says, no, we are in consensus, so it is safe to run this node on the network with core nodes." }, { "speaker": "stephan", "time": "10:55", "start": 655.0, "text": "Yeah. Now, to my knowledge, there have been some examples where, you know, possible consensus splitting things were found. I think a little while back there were, you know, because of the btcd stuff, because it's often used for Lightning, there were maybe some examples where like, you know, certain individuals found kind of things that would have failed on one side but not on the other, and then it kind of, you know, that creates a risk there. So I guess it's all like kind of, maybe it's not just at rbitcoin, but it's like who is your, I guess, what's the word, your consumer or who is your client, right? Who is underlying using you in a way that might differ with Core or the rest of the network?" }, { "speaker": "brandon_black", "time": "11:36", "start": 696.0, "text": "Yeah, so the, those issues that happened with btcd were, were indeed consensus divergences where a certain block would pass core but would crash btcd. And so, so actually I got a, I got a set of findings from a security researcher today, which, which looks for that kind of class of crash bugs, limits checking type, type things in rbitcoin, and I'm going to get my clanker clanking away on resolving that batch, today, and then make sure that we, again, we have automated testing that makes sure we never have that class of issue again. So yes, absolutely something to be, to be mindful of. rbitcoin is targeted at kind of people running small wallet infrastructure, maybe lightning nodes as well. So it's in the same kind of risk category as btcd there." }, { "speaker": "brandon_black", "time": "12:17", "start": 737.0, "text": "And so we just have to, shouldn't use the word just, just along the Y. We have to be very, very serious about our automation and our robustness in how we're developing, because even within core, one of the things we have to be very careful of is We can't do this by reading the code and then implementing consensus, which is how rbitcoin started. I started by telling the clanker to go read core and make an equivalent consensus library, right? But you can't rely on that because core's code has certain intentions in it that may not be carried through because of underlying unknown. Hopefully, at this point, core is pretty clear on that, but that was the old hard fork where back in 2010, the level DB versus Berkeley DB changed. That was the 2013 example," }, { "speaker": "stephan", "time": "12:59", "start": 779.0, "text": "I think, yeah." }, { "speaker": "brandon_black", "time": "13:00", "start": 780.0, "text": "Yeah." }, { "speaker": "brandon_black", "time": "13:02", "start": 782.0, "text": "They, they didn't know there was a different consensus limit, right? And so we can't just look at the code. We have to actually differentially compare what passes core versus what passes rbitcoin versus what passes, say, libbitcoin. And only by directly comparing can we tell whether we're in consensus, and that's where the fuzzing comes in. Yeah, well," }, { "speaker": "stephan", "time": "13:20", "start": 800.0, "text": "It's a, so there's a lot there. Let's also talk a bit on the certain target user. Who is your, let's say, target user, at least in the early days, maybe you're starting on a certain thing and then it's going to expand out, but who do you see as the initial user base for rbitcoin?" }, { "speaker": "brandon_black", "time": "13:35", "start": 815.0, "text": "So, me was the first target user. I've been so busy developing, I keep meaning, oh yeah, I'm going to, I'm going to change my main node over to rbitcoin, but I'm, I'm so busy improving rbitcoin, I haven't had a chance to do that yet. But I'm, I'm the main, the first target user. So I'm someone who, who runs a publicly accessible Bitcoin node that is used for my wallets and some other friends and family's wallets. and I'm frustrated by both the performance of the combination of Core plus Electors plus the disk usage of those, plus just having to run two separate things. And so, rbitcoin is meant to improve that, using less disk, being more performant in those roles, and being only one thing I have to run, so it uses less RAM on my server. And that's, so that's the first target user. Someone running, you know, very small wallet infrastructure for a handful of people. I realized as I was building that, and I was doing some early benchmarking, that I think rbitcoin also performs very well in a larger wallet service role. And so I started pushing more in that direction in the development where I think someone running like, an example that comes to mind because I happen to use the wallet, the Cake Wallet public nodes, they could switch over to rbitcoin. And I think actually they would provide an improved experience for their users, switching over from their current customized electrs plus core to rbitcoin. Because I think rbitcoin overall performs better than that pairing. So basically anybody out there running core plus electors is our first target users. Because electors is already not made for like massive scale, Fulcrum and Esplora more hit those, those massive scale use cases." }, { "speaker": "brandon_black", "time": "15:05", "start": 905.0, "text": "But electors is targeted Small scale, and that's like right in our bread and butter. Now, again, as I've been developing it, depending on what kind of hardware you're looking at running on, I think rbitcoin may also have a chance of competing in the bigger wallet infrastructure as well. So it's really that's the place we're going. And then also matching Lightning nodes, Lightning nodes are the same class, right? So there's a couple of clients, one or two often Lightning nodes using one Bitcoin Core node, rbitcoin slots right in there as well." }, { "speaker": "stephan", "time": "15:29", "start": 929.0, "text": "I'm curious, do you think you need to have a lot of miners or mining pools who use you eventually, or does you don't think it really matters? Like Bitcoin miners can just use Bitcoin Core, it doesn't really matter, or they can use like slightly modified versions of Core, and of course everything's meant to be consensus compatible, but for the sake of I don't know, diversity or the sake of kind of getting a good user base, do you think it actually matters that miners should, it doesn't matter that miners should be able to use rbitcoin, is the question I'm trying to ask." }, { "speaker": "brandon_black", "time": "15:59", "start": 959.0, "text": "Yeah, yeah. I've gone back and forth on that a little bit. Honestly, I started out with making it an explicit non-goal to support miners at all. And then what honestly happened is that in order to do the level of automated testing that I wanted for rbitcoin, I was basically forced to build a fair amount of the mining functionality. If you think about it, if you want to automate testing, you have to mine blocks to do testing against, right? So I was like, crap, I'm building mining functionality. So I've still kept it fairly limited. Definitely are not functional as a full mining template generating node, I think, for public miners or anything yet. But it's no longer an explicit non-goal. I had to give up that non-goal as I'm trying to manage the product surface of rbitcoin. Is it important? I don't think so. I think miners already customize core, so in a certain sense, miners already risk being slightly out of consensus. Now, obviously, they are extremely careful that any changes they make are, are more restrictive and not less restrictive, so they don't go out of consensus. But they modify core anyway. You know, they're" }, { "speaker": "stephan", "time": "16:59", "start": 1019.0, "text": "I see. Yeah, fair enough. On the performance differences that you mentioned, can you talk to us a little bit about that? Like, why, where are their performance gains coming and, you know, talk to us about that." }, { "speaker": "brandon_black", "time": "17:11", "start": 1031.0, "text": "The biggest thing, honestly, and core, I mean, again, those guys do an incredible job on this stuff. They, they're, they're, the next release of core is going to be very nearly on par with rbitcoin, which drives me a little crazy because I was like, I'm going to have this big" }, { "speaker": "stephan", "time": "17:23", "start": 1043.0, "text": "Deal. Yeah. Yeah." }, { "speaker": "brandon_black", "time": "17:27", "start": 1047.0, "text": "So that kind of parallelization was baked into rbitcoin from day zero, and core is like getting ahead of me, just as I'm also doing it. So that's, you know, our, rbitcoin, a big part of it is that it's built with that fundamentally a parallel Confirmation. Of Ostros funny, because he says, says that rbitcoin does confirmation is linear, linearized, and there is some truth to that. We, we do everything in order, but it's very parallel in order. Core until recently was basically serial in order. They're moving towards parallel in order, more like rbitcoin at the same time as I'm developing rbitcoin. So that's one place. The other thing that rbitcoin is very intentional about is how, and this is, this is the niggle in my brain that made me want my own implementation was always this dbcache concept in Core. So, right, so Core has this thing where as it's going along verifying, verifying blocks, it's writing and changing the UTXO database. That's the fundamental structure of how Core validates transactions is that it has a UTXO database. It checks whether the coins being spent are in that database so that it can check against double spends. And then the problem is that that results in massive write amplification because every transaction deletes something from that database, which causes a write on your data, on your database file, and then creates some new ones that are most UTXOs are short-lived. So you're constantly churning the disk, creating and deleting UTXOs when you're using Core. To solve that, Core has the DB cache." }, { "speaker": "brandon_black", "time": "18:54", "start": 1134.0, "text": "What DB cache does is it holds UTXOs in memory so that most of them never actually go to the on-disk UTXO database, and it flushes this periodically. So this whole structure just drove me bonkers as a software engineer because that kind of churn is basically a smell in code, and it relates to how the thing was structured in the first place. I think it's the same reason in some ways that libbitcoin from Bosco exists, because he had one of the same frustrations." }, { "speaker": "brandon_black", "time": "19:20", "start": 1160.0, "text": "So we got away from that. Similar to how libbitcoin did, and instead of kind of churning through UTXOs, we just write all the transactions to disk, and then we have a bit set that marks them as confirmed, and we have a spent link that says, was this transaction spent? And we add those links over time. They can never go from spent to unspent, so you're just adding spent marks as you go. And so you're never having to go like delete stuff from a database. You're just going and adding over time. And that reduces the write amplification and helps with the kind of stable performance over time where there's no flushing of big coins database, coins DB to dish or anything like that. The last thing I'll say is that the other thing we did in rbitcoin, because I care about it because of how I use Bitcoin, is that we focused very hard on efficient use of disk space. And it's funny again, at the same time as I was crushing the size of the TX index for rbitcoin. My gosh, who was it? Was it Loring? Yeah, I think it was Loring on Core was crushing the size of the TX index on Core. I think I'm still a little smaller again, like I'm a little faster and a little smaller, but they're so close. Anyway, I think it's all fun, it's all great, but we're very efficient on disk usage. So you get a full Core plus Electrum server with rbitcoin in about 730 gigabytes, whereas, you know, my Bitcoin Core node is about 950 gigabytes to get Core plus Electors." }, { "speaker": "brandon_black", "time": "20:44", "start": 1244.0, "text": "Anyway, so it's a, it's a smaller archive to get the same functionality and I think even more efficient functionality, and I'm, I'm happy with that." }, { "speaker": "stephan", "time": "20:50", "start": 1250.0, "text": "You mentioned Electrum. The other interesting thing, I remember seeing a talk by Ghost43 from Electrum Wallet, and he was talking about, he was kind of giving a bit of an explanation of the story of Electrum as a protocol, right? So initially it was Electrum Wallet, but then what happened is it kind of became a protocol because there were other people making Electrum servers, and then the Electrum servers had to kind of coordinate certain things when they wanted to upgrade things, because as you said, there's electrs, there's Fulcrum, I think there used to be Electrum X, Esplorand, you know, maybe some others that I don't even know. I forget that new one by Craig Raw. So I guess my question to you is, how do you plan to stay up on any changes in the elect, because now you've got to not just stay up on what's happening at Bitcoin consensus level, you also have to make sure you're up to date on the Electrum server protocol." }, { "speaker": "brandon_black", "time": "21:39", "start": 1299.0, "text": "They did a great job. I was happy as I've been developing rbitcoin, making the Electrum protocol fairly additive. And so there's a negotiation of what's the highest level of Electrum you support between the wallet client and the server. And so I've been actually recently adding some additional features in there to match higher versions. Like I was 1.4, then 1.6, and now 1.7 Electrum support within rbitcoin. So because they've already kind of tackled that, it's not bad to kind of add in functionality within the Electrum kind of world. And then like Frigate and like the Cake electrs, rbitcoin supports two kind of bolt-ons, which is for silent payments tweaks in two different protocols, one for Frigate and one for Cake. Gotcha. In short, that's a fairly easy. So" }, { "speaker": "stephan", "time": "22:23", "start": 1343.0, "text": "Rbitcoin also supports silent payments. I knew you were a fan of silent payments to begin with, so I presume you wanted to obviously make it. If you're the target user." }, { "speaker": "brandon_black", "time": "22:31", "start": 1351.0, "text": "Yeah, yeah, exactly. So we support silent payments with both protocols that are currently kind of deployed out there. Again, like we're trying to make rbitcoin's first customer. The individual wallet user or like the small scale wallet provider where people are going to want those" }, { "speaker": "stephan", "time": "22:44", "start": 1364.0, "text": "Features, right? So it'd suit well with like the kind of the Umbrels and the RaspiBlitzes of the world, that kind of thing." }, { "speaker": "brandon_black", "time": "22:51", "start": 1371.0, "text": "Yeah, yeah. And then to that end, we just added, it's not released yet, but we added a Tor support and a CJDNS and kind of the other kind of home node privacy networks as well." }, { "speaker": "stephan", "time": "23:01", "start": 1381.0, "text": "I see. What else did I want to ask about? So another kind of interesting one is you've gone straight to, what's it called? BIP324, the P2P v2 protocol, as in transport layer encryption. And you've just gone only for that. Is that just mainly because you're starting fresh, you knew, you don't need to kind of, that's actually one benefit you have, that you don't have to care about old stuff. You can just say, hey, we're just going to go straight to the new stuff." }, { "speaker": "brandon_black", "time": "23:24", "start": 1404.0, "text": "Yeah, I mean, P2P v2 is fundamentally better. And so why would we, why would we implement V1? The, the V2 network is robust because core supports it for a couple of versions now. Yeah, just go there." }, { "speaker": "stephan", "time": "23:36", "start": 1416.0, "text": "Yeah. Any other, I guess, big kind of technical decisions that you have made with rbitcoin that you want to mention now?" }, { "speaker": "brandon_black", "time": "23:46", "start": 1426.0, "text": "So I think the biggest one, we talked about it very briefly early on, is that rbitcoin is similar to the entropy lab project that folks are working on. It is an AI coded project. Humans do not write code for rbitcoin. I have written zero lines of code in this process, and that's intentional. I have reviewed very little code. I do read code when I have to, but most of the AI, the AIs tell me about the code, and then I guide them through solving issues. I don't read the code myself. And I think some people have concerns about that method of coding, but as we've seen from the recent breaches and everything, there, the ability of AIs to hold and digest context exceeds what humans can do, and that ability means they actually write fewer high-risk bugs than, than humans do these days. So I think it's worth exploring, and it's part of, again, an rbitcoin Kind of intentional goal is we, humans don't write code for rbitcoin, AIs do." }, { "speaker": "stephan", "time": "24:41", "start": 1481.0, "text": "And I guess a corollary of that is, as you, as you're a noted Grok fan, as I am also, but we also have to remember that, you know, some of the, some of the other, you know, Grok won't catch all the bugs. So I presume you're having to do other runs or kind of passes over this with non-Grok AI, as well as the fuzzing and the other testing suites that you have." }, { "speaker": "brandon_black", "time": "25:04", "start": 1504.0, "text": "Yeah, I mean, I almost exclusively use Grok. I also use a little bit of Fable to do, reviews of my, of things that are kind of critical or challenging, but I've had external folks doing audits with other models that they use. Because, because I, there is a weird thing, I call it symbiosis between developers and models these days. I get how Grok works. When I'm working with Grok, I'm very comfortable with the responses it gives, and I know how to interpret what it's saying because I've been working with it intensively for a few months now. When I go use another model, It's uncomfortable. I don't, I don't have the same level of understanding of why did it say those words versus those words. and so I think there's going to be a thing where, where developers have one or two models they prefer because they have a relationship with those models, even though that's a strange thing to say. Right. It's a tool" }, { "speaker": "stephan", "time": "25:49", "start": 1549.0, "text": "You're used to using, kind of like that. Yeah. Yeah." }, { "speaker": "brandon_black", "time": "25:51", "start": 1551.0, "text": "So I'm a Grok user primarily. I use Fable for some audits and others use other model against it." }, { "speaker": "stephan", "time": "25:56", "start": 1556.0, "text": "But, I mean, obviously, with all the security stuff going on, there was a lot of talk about the Chinese open source, oh sorry, open weights models. Should say that, you know, Rob and Kelly and the red team guys were doing, they had their, apparently they have a custom harness and their, you know, so don't you have this sense that maybe you should have review from some of these other open weights models to catch things that Grok may not catch?" }, { "speaker": "brandon_black", "time": "26:19", "start": 1579.0, "text": "Oh yeah, and they're, they're, they're reviewing it. I've gotten some bindings from Rob directly and from others." }, { "speaker": "stephan", "time": "26:23", "start": 1583.0, "text": "So yeah. Okay. So you see that as part of the process, right? So even if you mainly build with Grok and a bit of Fable, you see it like a necessary part of this is that it actually gets reviewed by the open weights models because they will see things or catch things that Grok may not catch, right? Absolutely." }, { "speaker": "brandon_black", "time": "26:38", "start": 1598.0, "text": "Absolutely." }, { "speaker": "stephan", "time": "26:39", "start": 1599.0, "text": "Yeah. Okay." }, { "speaker": "brandon_black", "time": "26:40", "start": 1600.0, "text": "Well, yeah, and if the same as any software, right? Like, used to be that a human would write it and then a security researcher would review it. Like, of course, right? I'm the developer. I'm not a professional security researcher. I mean, I'm a, I'm a, I have limited security researcher chops, let's say. That's not my main thing. And so I do need other folks to review and help me with those things." }, { "speaker": "stephan", "time": "27:01", "start": 1621.0, "text": "Yeah. And so, yeah, sorry, I had another. Oh, yeah, DOS protections. So, talk to us a bit about that because I know this is like, you know, the last few years with all the kind of different spam arguments and things. But what are your thoughts on, like, DOS protections in, you know, for rbitcoin?" }, { "speaker": "brandon_black", "time": "27:19", "start": 1639.0, "text": "Yeah, I mean, I think we have very similar rules to Core in terms of protecting the node itself from DOS, from, you know, various types of transactions or what do they call them, DOS blocks, as Portland Hobble calls them. Again, very similar limits to Core there." }, { "speaker": "brandon_black", "time": "27:33", "start": 1653.0, "text": "Unlike Core, that's the main thing we block on relay. I think both rbitcoin and libbitcoin are again similar here, where we don't have, like a relay policy framework, which really we have no DOS protections, and that's what we have. We don't have other relay policy, really." }, { "speaker": "stephan", "time": "27:51", "start": 1671.0, "text": "Gotcha. And I know you were a fan of Peter Todd's LibreRelay and things like that. So does any of that philosophy come through into rbitcoin as well?" }, { "speaker": "brandon_black", "time": "28:01", "start": 1681.0, "text": "Yeah, we call it Libre, LibreRelay is kind of the module that we have that's like DOS" }, { "speaker": "stephan", "time": "28:05", "start": 1685.0, "text": "Protection only. Okay, gotcha. So it's kind of related. You see it as like, I guess the way I've heard, let's say Antoine Poinsot explained it is that there's, you know, DOS protection, upgrade hooks, and what's the other one? Basically, he saw those as like the legitimate reasons to have, you know, so-called filters or Policy, I guess, is the more technically, you know, core specific word, and you see it like, no, it should just be, it should just be DOS protections and maybe the upgrade hooks thing as well, right?" }, { "speaker": "brandon_black", "time": "28:34", "start": 1714.0, "text": "Yeah, I think upgrade hooks is kind of a weak argument. Upgrade hooks protect only if it's consensus compatible, and so, like, either it's consensus compatible, sorry, I meant, oh my gosh, incentive compatible is the word I'm looking for. If it's incentive compatible, upgrade hooks protection works. If it's not incentive compatible, people are going to break open those upgrade hooks. And so you don't really believe in... You go one step further," }, { "speaker": "stephan", "time": "28:54", "start": 1734.0, "text": "You literally don't even believe in having, let's say, standardness rules around upgrade hooks, basically." }, { "speaker": "brandon_black", "time": "29:00", "start": 1740.0, "text": "Oh, really? Because, again, if people had a legitimate use for them, they're going to use them no matter what the relay policy is. So it has to be that they're not incentivized, otherwise people are just going to use them. Incentives rule everything around me." }, { "speaker": "stephan", "time": "29:13", "start": 1753.0, "text": "Okay. so that'll be a clear difference, and I'm sure maybe some of the other implementations might have different views on some of those things. So, that'll be interesting to see what people say. And then pruned versus archival node and like that kind of aspects of it, I, is your node just an archival node, or do you also offer a non, you know, a pruned version or what?" }, { "speaker": "brandon_black", "time": "29:34", "start": 1774.0, "text": "So, because there's no separate UTXO database, rbitcoin can only work as an archival node in that it has to have every transaction output on disk. The current size of that is about 230 or 240 gigabytes, I think. That's, so if you strip out the input stuff, the signatures and the witnesses, you, you get down to under 300 gigabytes, let's just say. And so you can, as of commit I merged this morning, run rbitcoin without the input scripts, and you get down to that size. Now that's not nearly as small as a pruned core node. Now, on the flip side, though, if you're running in a pruned rbitcoin node, that can still actually serve as a complete historical wallet explorer at 300 gigabytes. The only thing it's missing is signatures and witness scripts. And it, so most, most explorers only show those things if you request like more details. And so you can actually run a 300 gigabyte rbitcoin node as a full explorer node versus an almost two terabyte fulcrum plus core node to get a full explorer service. So again, I'm, I'm trying to serve this smaller wallet situation or smaller explorer where you want to have limited resources, but still provide something very useful." }, { "speaker": "stephan", "time": "30:42", "start": 1842.0, "text": "Interesting. So you can actually, so when you say 300 gigabytes, you mean that's technically, that's an archival node, but minus some components. Like, I guess, archival node for the purposes of syncing a wallet. Is that" }, { "speaker": "brandon_black", "time": "30:56", "start": 1856.0, "text": "Fair to say or not? Not for IBD, because it can't for IBD without the scripts for the other nodes IBD from. So it's like semi-archival, archival for a wallet or explorer, but not archival for an IBD." }, { "speaker": "stephan", "time": "31:08", "start": 1868.0, "text": "Gotcha. And then are you going to have, obviously, I presume you'll have like a flag in your bitcoin.conf equivalent where you can set what you want, and that's kind of like whether you, what do you call those modes? Like, do you say full archival versus whatever that 300 gigabyte mode" }, { "speaker": "brandon_black", "time": "31:23", "start": 1883.0, "text": "Is? I don't have a better word for it, so it's, you can prune sequences, signatures, and signatures," }, { "speaker": "stephan", "time": "31:30", "start": 1890.0, "text": "Basically. Okay." }, { "speaker": "stephan", "time": "31:32", "start": 1892.0, "text": "Okay, yeah. And so, okay, so I guess, yeah, I'm thinking about the key things. I guess just kind of AI development more broadly, like you were touching on this obviously and as you, as you mentioned, some people have concerns about this idea." }, { "speaker": "stephan", "time": "31:47", "start": 1907.0, "text": "It seems to have moved really quickly, right? Because even like a year or two ago, it would have been seen like maybe you were crazy for letting AI write PRs, and now it's more like, oh, you're crazy if you don't have AI review your PRs. Where are we going with this? Like, is it eventually going to be like you become, you, you as a developer, do you become more, I don't know, are you more like a architect or are you more like an orchestrator of agents? Like, how does that, what does it look like?" }, { "speaker": "brandon_black", "time": "32:15", "start": 1935.0, "text": "I really view it a bit differently than some folks, I think. If you were to take the folks who were writing machine code directly on ENIAC back in the day, you know, coding by moving wires around, that kind of coding, and then tell them about a modern C++ or Java or other language compiler, the difference between what they were doing in programming by punch cards, wires, and hand coding machine code to a modern compiler is not bigger than modern compiler to AI. So what's changing is just like we've done before in software engineering, the layer at which we software developers work has gone up another layer. But it doesn't fundamentally change what it is that we're doing. We're still developing software through an interface. It went from an IDE to a, to an AI build harness, but the difference is smaller than others we've already done in the history of software engineering. I don't think that it fundamentally changes the role of software developers. I was saying to my wife, you know, one of the things that marks a good software engineer in the compiler era is someone who is willing when needed to go down and read the compiler code or willing to read the kernel code or willing to read even the byte code for the, for the, chip they're running on. The good developers will always go down to whatever layer is necessary to solve a problem, and that applies just as much if not more in the AI world. So the fact that my primary coding interface is through a build harness Doesn't change that when I have an issue, I'm going to go read the rust source code." }, { "speaker": "brandon_black", "time": "33:42", "start": 2022.0, "text": "I'm going to go read the standard library. I'm going to go read the kernel code to solve the problem. It's just that I moved up a layer in how I'm building. And I think that's, that's the best way for me to view it at least." }, { "speaker": "stephan", "time": "33:52", "start": 2032.0, "text": "Yeah. And as a Grok user, I presume you're using Grok build. How do you deal with like context limits on, you know, how many, however many thousand, like a hundred thousand lines of codes or whatever, like do you have that issue of like trying to fit things into the context window?" }, { "speaker": "brandon_black", "time": "34:11", "start": 2051.0, "text": "I do. That's definitely been a challenge in developing rbitcoin. I think it'll be an ongoing challenge, which is in a sense, again, it's similar to as humans, we have limited context. So when my project gets too complicated, and I'm working on it in a IDE, I have to start closing tabs and I lose context, right? So AIs are in some ways so weirdly human in that way, where when their context get full, they start to get stupid. And so I think building projects, you know, rbitcoin has, I think, close to a dozen separate crates in it. Having separation of concerns is still super important, even in the AI world. Documentation is still very important. So Grok and I work very hard on improving the documentation so that AIs know where to look for certain things so they don't blow up their context. All the techniques we use to protect humans from context bloat so that they could work on software efficiently still applies to LLMs. And so that's what I'm saying. It hasn't actually changed all that much. What we do is still kind of similar. It's just we do it faster." }, { "speaker": "stephan", "time": "35:05", "start": 2105.0, "text": "Right. So you're saying it's like conceptually similar, but you're just doing it in a slightly different way. And then As you mentioned, like crates, I was thinking about this idea of dependencies, right? Like this has been a big thing in Bitcoin Core, they've been trying to take dependencies away over time. How do you see that? Are you thinking of it similarly or no, or how do you see that?" }, { "speaker": "brandon_black", "time": "35:24", "start": 2124.0, "text": "Similarly, and probably even more so in the AI world, there's like a difficult fine line to walk with dependencies where if you re-implement everything, now you have to fix the bug in a million places, but if you don't re-implement everything, now you're taking on someone else's bugs whenever they upgrade the dependency. So rbitcoin is low dependency. We're not trying to go for zero or anything, but we're, we're definitely trying to keep the dependency tree small. And where there's a dependency that we can kind of build with just a few lines of code internally and where we need any customization, we will build it internally. An example of that is that early on we were using a Rust crate called Rayon. The name isn't important, but it was a multi-threading library that just made a pool of things to do work. And we kept fighting it. It kept doing not quite exactly what we wanted to, and it, the answer was to drop the dependency and build our own small version of the library. I think it only took 150 lines of code to do what we needed from it versus like a thousand plus line dependency. We needed 150 lines of it. And so there's, I think that's one of the keys to know if you, if you want to get rid of a dependency. If you're using a small piece of a bigger dependency, you probably don't want that dependency anymore. Just build it yourself." }, { "speaker": "stephan", "time": "36:30", "start": 2190.0, "text": "Interesting. Yeah. So it's just, yeah, I mean, it's, this has been a long going, ongoing theme I've seen from Bitcoin Core. People also of like trying to take our dependencies and it's just been a theme I've noticed over the years that they would try to look for ways to obviously minimize the dependencies for security reasons. And I think some of those, you know, really did work out well, like it was the right call. Obviously takes a lot of work though. Another question around keeping pace with core. So let's say core is putting out new features. So as an example, Cluster Mempool, right? V31, I believe. Anyway, the point is there'll be new features coming out. And some of those, you know, they're optional, you don't have to, but are you, are you, are you just going to do a pick and choose approach? Like you'll say, oh, Cluster Mempool, yes, I want that or this other thing, no, I don't want that." }, { "speaker": "brandon_black", "time": "37:16", "start": 2236.0, "text": "Yeah, yeah, exactly that. And we definitely Grok and I are going to have conversations probably about once every week so far over the past, whatever it is, 10 weeks we've been working on this, where we'll just look at what's changing in other Bitcoin node implementations and see if there's features we want that they're building, and then we also build them if there's things that are, that are important. One frustration that we're having, again, this is when I say we is me and Grok primarily, is that there's not an implementation of package relay in our, sorry, in rust-bitcoin. And since we use rust-bitcoin for our parsing, that means it's hard for rbitcoin to get package relay. I keep going back and forth with Grok as to whether it's worth building that or if we should wait for rust-bitcoin to do it. So there are, there are definitely challenges in trying to keep up with core because of dependencies in this case. So, you know, everything's trade-offs, put it that way." }, { "speaker": "stephan", "time": "38:06", "start": 2286.0, "text": "What's the relationship like with the rust-bitcoin developers then? I presume that are they, do they like what you're doing? Is there a possibility then that, I mean, let's say LDK is like a rust-based things, is that, is there like a natural synergy there or how do you see that?" }, { "speaker": "brandon_black", "time": "38:21", "start": 2301.0, "text": "I haven't talked to them very much. I'm, I'm very much an introvert, so I'm over here to do my own thing. They, they've pinged me a little bit, and we did, one of our docs called out places where rust-bitcoin has like very minor divergences from how Core does things, and therefore, rust-bitcoin specifically says don't use this for consensus, and then we went and used it for consensus. And as a result, we found some, some minor nits, let's say, in rust-bitcoin, and we have a doc that spells those out. When I get my head above water on rbitcoin thing, I do want to open PR against rust-bitcoin to fix a couple of those things. For now, we're working around them. But, so I think we will develop a relationship over time. You know, this is, it's a 10-week-old project. I'm, I'm cranking as fast as I can clank, and I, and as it is starting to stabilize. My, my wife was like,\"Oh, you exist again.\"" }, { "speaker": "stephan", "time": "39:07", "start": 2347.0, "text": "What" }, { "speaker": "brandon_black", "time": "39:07", "start": 2347.0, "text": "About, do" }, { "speaker": "stephan", "time": "39:08", "start": 2348.0, "text": "You think it would make sense for any wallet or app developers to build on top of rbitcoin?" }, { "speaker": "brandon_black", "time": "39:13", "start": 2353.0, "text": "So I think there's the biggest opportunity is just if you, if you want to build a wallet and you want to run some infrastructure, rbitcoin is something you can, you can plug in inexpensively on like a, on a, on a relatively affordable server to serve all your wallet clients on your own infrastructure rather than depending on other people's public infrastructure. So I wouldn't necessarily like, you're not going to use rbitcoin as the foundation for an app, but when you're building a service, you might use rbitcoin as part of the service. It's more" }, { "speaker": "stephan", "time": "39:37", "start": 2377.0, "text": "At the server level that you would apply it, okay, as opposed to like building the code directly onto it, okay. How do you see, like, do you have any ideas on, I guess, marketing, right? Who's going to be your, who's going to be your, your, your, your shill, you know, your, marketing guy? Or are you going to be the, you're going to, you're going to do it all. You're going to be the marketing guy as well. Who's going to be your mechanic?" }, { "speaker": "brandon_black", "time": "40:00", "start": 2400.0, "text": "Yeah, look, look, rbitcoin is still about 10% a joke, right? I launched rbitcoin as a production-ready™ Bitcoin node implementation as partially a joke. And then I, it's turned out that the ability of one person with AI to build good software has, has far exceeded my original expectations, and I've fallen down a deep rabbit hole of developing this software. So, so marketing was not part of the original plan, let's say. At this point, I've got, I've got Grok, a Grok bot that does shitposts about it, and I think another friend is gonna build me a better website, because Grok sucks at building websites, like really sucks. So we'll see what happens there. I'm at this point optimistic that at least a small community will form around rbitcoin. I'm trying to make it really useful software, and really useful software hopefully attracts a community, and then we'll get some marketing out of the community, because I am not a marketer. I am the opposite of a marketer. I tell people to wait. Too much truth to be a marketer, sorry." }, { "speaker": "stephan", "time": "41:01", "start": 2461.0, "text": "So, okay, so yeah, I guess maybe it just comes down to like finding a real, a user base who really love your product and they, and they become your natural advocates. But of course, as we said, there's a bunch of different implementations out there, right? Like as we just said, Bitcoin Core, libbitcoin, btcd, Floresta, Hornet, satd, rbitcoin." }, { "speaker": "stephan", "time": "41:23", "start": 2483.0, "text": "I" }, { "speaker": "brandon_black", "time": "41:23", "start": 2483.0, "text": "Think that's it. That's the, that's the one I can think of. There's like there's like the partial one in Java and then there's I think what called like Mako or something. There's a couple other like tiny ones out there that are less, less actively developed." }, { "speaker": "stephan", "time": "41:38", "start": 2498.0, "text": "Gotcha. Yeah. so I guess any thoughts on where this all goes? Like, do you, do you see a real future where lots of other implementations are flourishing? Do you see it more like core will be sort of the 80% and then in that other 20% there'll be a bunch of these other implementations? Like, where do you see it going?" }, { "speaker": "brandon_black", "time": "41:55", "start": 2515.0, "text": "So taking, I often take lessons from older other open source projects that have, have had kind of core communities that split or change over time. And the way I would see it is that having good alternativesNo matter what happens, even if rbitcoin is never more than 10 nodes on the public network or whatever, its existence is still an influence on how core develops. And the fact that there are people have alternatives, meaningful alternatives, affects the way an open source project moves. We kind of have two examples in history here where on one, the X386 project became stale, stopped taking contributions that would materially improve it because they were too risky or they Like them or whatever. And as a result, the XORD project was built. And XORD took like two years to completely subsume X3 86. And so I don't think that's going to happen to Core. And then on the other, on the flip side, there've been projects, even the Linux kernel is kind of an example where the Linux kernel has been running under one BDFL for forever, even though there are alternative kernel patch sets and out there because Linux has consistently brought in the best changes from everywhere in the ecosystem. And so I think that right now, Core is like dodging accepting ecosystem changes that affect consensus. And that's part of why it's important to have other Bitcoin node implementations out there." }, { "speaker": "brandon_black", "time": "43:23", "start": 2603.0, "text": "Now it's a challenge because Bitcoin, of course, is a consensus network, but you're seeing, let's call it little cracks of the edges because Core doesn't take a certain kind of changes. Without kind of weird bars that don't always make sense, they don't, they don't shepherd those changes through, like all these challenges around consensus changes. And I think that alternative implementations will help to keep court honest about that. Because if, if people have, feel free to switch to btcd or libbitcoin or rbitcoin when they're frustrated with core, that makes, helps keep core honest. On the flip side, of course, we have Bitcoin Knots was like, pretending that that was what Knots was like, oh, you can switch the knots from core, but it didn't have the kind of robustness needed to be that. And so I think John attacked, when we were talking earlier, and I pointed that out essentially about rbitcoin, that if rbitcoin's ever going to play that role, we have to provide a level of confidence that is extremely high. Because one of the most important things about core is the confidence you have in the reliability of the software. And so I'm going to be busting ass on that. That's basically the most of the rest of the work in rbitcoin to get to a 1.0 release is getting that level of robustness and reliability where people can really be confident when they use rbitcoin, that's going to match consensus, it's going to work on the network, and that makes it a real option for people, in contrast to some other implementations maybe out there." }, { "speaker": "stephan", "time": "44:50", "start": 2690.0, "text": "Interesting. Yeah. Well, I think that's a good spot to leave it there. And, the website is it, let me just get it right. rbitcoin.org, right?" }, { "speaker": "stephan", "time": "45:00", "start": 2700.0, "text": "Rbitcoin.org. So listeners, you can check it out there. Any, any last thoughts?" }, { "speaker": "brandon_black", "time": "45:03", "start": 2703.0, "text": "Contribute to rbitcoin. I would love to have more contributors try to review PRs pretty quickly. I've had, I've gotten PRs from three outside contributors so far via their clankers, and I would love your clanker to contribute as well." }, { "speaker": "stephan", "time": "45:15", "start": 2715.0, "text": "No human contributions allowed. I'm joking. I'm joking. All right. Yeah. All right. Well, thanks for joining me, Rearden." }, { "speaker": "brandon_black", "time": "45:23", "start": 2723.0, "text": "Thanks so much. Talk to you later." } ] }