The Bitcoin Transaction Capacity Problem — Why We Can’t Use Fancy Tricks to Avoid It

This article analyses some alternatives for increasing the transaction capacity of the Bitcoin network, in particular the Lightning Network. However, the arguments presented are general and apply to other proposed alternatives as well.

By Mike Hearn

First, some background.

Many years ago, Satoshi Nakamoto added a restriction to the Bitcoin source code. He artificially limited the transaction capacity of Bitcoin to prevent miners from maliciously sabotaging the process of making it very slow for clients to download the entire Bitcoin blockchain.

This limitation was originally planned to be removed. In fact, Satoshi planned to remove it as soon as an SPV wallet was developed.

I know this because I emailed him and asked him in late 2010. This is his response:

A higher block limit could be phased in, once we have actual usage close to that limit to ensure it works.

Eventually, when we have client-only wallets, the block size limit won’t matter, but until then, since all users still need to download the entire blockchain to use Bitcoin, we’ll probably keep it reasonable.

The client-only wallets we discussed were Android wallets, MultiBit, BreadWallet, etc. At that time, there were no SPV wallets available - this phrase was first used by me. (Translator's note: The author refers to the name SPV wallet, which was invented by him first.)

According to Satoshi's original intention, the block size limit should have been increased several years ago. But we left it to the last minute. According to my rough calculation, Bitcoin usage grows in the winter and stagnates in the summer. If the current trend continues, Bitcoin's capacity should be used up in early winter 2016, and it will probably be a few months earlier. Because upgrading takes time, we need to prepare for it now. So Gavin proposed a patch and started the discussion.

Some people prefer to believe that we can avoid changing this limit and rely on another way to scale Bitcoin. These people are typical unrealists. I don’t think it will work, and I will explain why it won’t work in this post.

Finally, a quick background on myself for those who don't know me. I've spent the last five years working on implementing Bitcoin wallets, and before that I worked at Google for almost eight years. Three of those years were on the Geography group (Maps/Earth), one of the busiest websites in the world, where I was a professional capacity planner. As it is now, doing capacity planning takes a long time. In the Google world, it was because we had to physically build the machines and physically build the data centers. In the Bitcoin world, it's because it takes time for people to upgrade. So I have experience with our current predicament.

Lightning Network

What is the Lightning Network? It’s the latest name for an old idea: using payment channels to send payments between various low-trust intermediaries. A payment channel is essentially a pot of bitcoins built on top of blockchain technology that can pass messages outside the Bitcoin blockchain network. But the final settlement still has to be settled on the Bitcoin chain. There are various Bitcoin protocol features that can be used to implement this low-trust way of paying.

The first and main use of this technology is for micropayments: payment information can be transmitted faster and cheaper than on the Bitcoin chain, because the payment is only between two people. Early observations show that this can establish low-trust fast payments between client/server and server/server. Effectively, a settlement network will run on top of this blockchain (translator's note: referring to the Lightning Network and the like), and the settlement layer settlement can be achieved at any time.

Such a settlement network should be better than other networks, because only in this way can the payment on this blockchain be guaranteed to be honest. In theory, it can handle more transactions than Bitcoin itself, because the Bitcoin blockchain only needs to see the total remittance amount.

It’s a neat and attractive idea. So attractive that, while the guys behind the Lightning Network are getting all the attention, their idea has already been implemented by a Swedish company called StrawPay.

StrawPay designed a protocol called Stroem (actually STRöM but many people don't know how to type and pronounce it). It defines wallets, hubs and traders, and sets up payment channels between them. They also implemented a GUI wallet that can actually be used across platforms to make it user-friendly.

The Stroem protocol is not the same as the Lightning Network - they differ in some technical details, notably, Stroem works on the basis of today's Bitcoin protocol and accepts caveats that the Lightning Network does not. Stroem specifies details that all other Lightning Network protocols ignore, such as how to actually make such payments in a backwards-compatible way (Bitcoin addresses cannot be used in these schemes).

I like the guys at StrawPay and I wish them all the best. I think Stroem has great potential to make micropayment routing faster and easier. There is a good chance it will find a place in the Bitcoin ecosystem. But its use should not be mandatory.

There are several reasons why this approach to scaling is not very effective.

First, the complexity of implementation will hurt decentralization.

Complexity kills the noobs

It’s human nature to notice problems and ignore the aspects of things that are going well - after all, there is only so much to praise. So Bitcoiners tend to emphasize mining, trading, payment processing, etc. We rarely take the time to reflect on our successes, and the wallet market is a huge success for decentralization.

There are probably hundreds of different wallet payment methods, using dozens of different basic algorithms for implementing payments. No one wallet payment method is clearly dominant. They compete for users in a dynamic market. This is very important: if there were only one or two wallet payment methods, then those wallet payment methods would become the controllers of the wallet payment system, and worse, they would change the rules of Bitcoin at a whim. The blockchain would be insignificant because people use the blockchain almost entirely through their wallet payment methods.

Setting up a wallet payment method is by no means a walk in the park. I should know - I've spent the last five years writing bitcoinj, a reusable "wallet engine" that sits at the heart of many Bitcoin applications and services. But it's still possible for someone to write bitcoinj in their spare time without a big team. Contrast that with HTML5, which is only available to big companies. The Bitcoin protocol isn't as simple as it looks, but it's simple enough. This simplicity is key to the diversity of wallet payment methods currently in use.

Unfortunately, payment channels are not simple. Especially once you introduce more advanced payment channels like the features proposed by the Lightning Network, the result is a larger, more complex block of code. Some people have made toy-like implementations of payment channels, but only bitcoinj has a real production-quality implementation. And toys don't have to worry about things like documentation, serializing state to disk, unit testing, UI integration, error management, etc. The difference between a toy and the real thing is whether it can be given to a non-nerd to use in an application.

Bitcoinj's payment channels were used to make PayFile, which allows paying per thousand words for downloading files, various Android prototypes for WiFi micropayments... and StrawPay itself. The entire StrawPay implementation is built on top of this, and bitcoinj and the GUI wallet are a fork of MultiBit HD. So we know the code works. Which means we also know how much work we put into it. Fortunately, much of this work was funded by companies.

As you can probably imagine, I'm happy about the idea of ​​everyone being forced to do payments in payment channels starting next summer. Many developers are assuming that there will inevitably be a bitcoinj/StrawPay replacement for their own code soon, and I'll get more users. But I don't want people to be forced to use my code in a hurry. That would cause me a huge developer support headache, and I don't want to see decentralization take a hit, even though bitcoinj is easy to fork. Plus developers don't want to be "required" to use my software, and I don't want that either.

Therefore, if we make it harder to design wallets, we believe there will be fewer wallet payment methods. StrawPay developers said:

We have a library written in java that the wallet should use, but it is a work in progress. No one has a wallet supporting iOS yet (they don't even have payment channels implemented AFAIK).

For wallet developers, there is a lot of GUI work and common issues to be done in the library before the wallet can work, even if the developer has mastered the library.

Regardless, the wallet market will likely consolidate around a handful of engines, just as the gaming industry has. But we don’t have to accelerate this trend. There’s no need to, because all this is solving a problem that doesn’t exist: Bitcoin’s core technology can scale to larger transaction capacity without any outside help.

Examples of rookies being eliminated

From the Lightning Network whitepaper:

If one person does not relay a transaction at the correct time, the counterparty could potentially steal funds. This problem can be mitigated by having a designated third party send the funds. Increase output fees to create an incentive for third parties to monitor the network.

This problem may seem academic, but leads to the common problem of "not delivering the message at the right time", perhaps due to your phone battery running out or roaming into an area with weak signal. Solved by adding more third parties that are not fully trusted... but then it raises the issue of complexity again!

For special cases like extremely fast micro-billing where there is no other way to achieve it, we just have to put up with this kind of thing. We should avoid it when we can.

Another example:

When one party loses data, the other party may steal funds.

another…

Although there may be ways to mitigate the threat [to hackers] to senders and receivers, the intermediary nodes must be online and likely to automatically process transactions. Therefore, the intermediary nodes may be at risk, so don't leave a large amount of money in this "hot wallet".

Because of lower fees, better-secured intermediaries will outperform other intermediaries and conduct greater volumes in the long run. Historically, one of the largest components of fees/benefits has been various forms of counterparty risk - the largest component of Bitcoin fees is likely to come from the security risk premium.

Hub-and-spoke payment channel networks require servers to be always online, which have sensitive signing keys. This is completely different from ordinary Bitcoin usage, where only the sender needs to sign, and it is common to do all signing offline. This increases its complexity and therefore its cost.

Bitcoin nodes are easier than hubs

A common complaint about Bitcoin scaling is that it will become harder to run a full node. People who make this complaint then point to payment channels for a solution, ignoring that the switch is just to make it easier to run a full node through a different node… one that is harder to build and run.

The main reason is that payment channel network hubs are stateful, so you want to make sure they have good uptime, reliability, and working backups. In contrast, Bitcoin nodes are almost completely stateless, and all software accessing the P2P network knows how to recover from a node that suddenly disappears. It happens all the time and is transparent to the user.

This makes running a full node easy and safe: if your cheap $10 VPS goes down and reboots, no one will notice or complain. If there is a large data loss, you can reinstall the node and it will recalculate all the data it needs from scratch. If it gets hacked, there is nothing to steal. Payment channel hubs can't really achieve these things.

High performance risk

StrawPay doesn’t use the design you might intuitively imagine, where the channel links are sent to the hub, hub to hub, and hub to recipient. The hub will then issue a promissory note to the recipient. The designer says:

I'm waiting for this question. This is probably the longest discussion we've ever had. Here's why:

1) Security: Keeping a payment channel server running is a complex matter. It must be secure. Most merchants don’t have this knowledge, so their wallets can be hacked. The acceptance note we designed can be used by anyone except the merchant, so there is no point in stealing it.

2) Performance: We are concerned that hub-to-hub small payments in a hub network are not progressing fast enough. It is better to tell merchants that the consumer has paid as quickly as possible, rather than having merchants hold acceptances.

This is a very difficult choice.

We already covered security, so this isn't surprising. But you might be unhappy when it comes to performance. Don't be unhappy - the payment channel protocol involves a lot of round trips of messages and creating signatures/verifications. Individually these steps aren't slow: they're measured in milliseconds. But if each step takes 200 milliseconds, and you have ten of them, all at once the payment will be a bit sluggish.

Passing Bitcoin transactions across the network isn't a fast thing either, but it's at least full of room for optimization. Local transaction verification is very fast: the problem of propagation delay is important. Payments are traced through a hub/spoke network: there's a big difference.

The correct statement about this is that the contactless EMV specification requires that the payment time should be less than 500 milliseconds. This time includes the time required to start the smart card, establish the wireless link, implement the ECDSA signature and authorize the payment, and the terminal verification. Each server request online authorization delay timeout is only 200 milliseconds. Some stations in the London underground network require people to go through the door in three seconds. Therefore, performance is very important.

Maybe the StrawPay guys are being too conservative - the point is, no one knows yet, because no one has tried to set up and deploy such a network. Therefore, many important questions remain unanswered.

there is no time

Let’s ignore all the above issues and assume that payment channel networks are the best way to pay for things right now. It doesn’t really matter because we’ll have a solution by the middle of next year at the latest, and the proposed network may not even exist yet.

Quite simply, we are running out of time. There is no credible technical solution that can gain widespread adoption in the next twelve months beyond simply increasing the payment capacity of existing systems, which everyone already understands and implements well enough.

I like the idea of ​​Stroem/Lightning for fast routing of small payments. I like the idea of ​​payment channels for sending/receiving small payments directly. Maybe one day we will all use wallets that support these systems. But talking about these things is pointless in solving our current problems.

Some people seem to think that if we keep Bitcoin's transaction capacity capped, transaction fees will just steadily rise and the free market will sort everything out. This is not true. In my next post, I'll write about what will actually happen if we don't deal with this problem in a timely manner.

to be continued …