Research report: Bitcoin software version has evolved over the past 7 years, where is the biggest change?

Source: BitMEX

Compiled by: First.vip

The BitMEX research team conducted 35 initial block downloads, using the initial block download time as a benchmark metric to test the performance of Bitcoin Core. Bitcoin software versions from 2012 to 2019 were used in the test.

Our testing of Bitcoin software versions from 2012 to 2019 shows that there have been considerable improvements in performance, but there is also a lot of variation. Even with the latest computer hardware, older versions of Bitcoin struggled to overcome the hurdles of increased transaction volume between 2015 and 2016. Therefore, we conclude that today's fast initial sync is almost impossible without software performance enhancements.

Figure 1 - Bitcoin initial block download time (days) - average time for 3 attempts

(Source: BitMEX Research, Note: Block height synced to 602,707)

Overview

To test the performance of Bitcoin Core during the initial synchronization process, we attempted 35 initial block downloads and recorded the time taken for each attempt. The results are shown in Figure 1, which shows that when Bitcoin Core 0.12.0 was launched in February 2016, the speed of Bitcoin was significantly improved due to the upgrade of signature verification from OpenSSL to libsecp256k1. (First Class Warehouse Note: After the upgrade of Bitcoin Core software version 0.12.0, the signature verification speed increased by about 7 times.)

Libsecp256k1 was built specifically for Bitcoin. Since then, the pace of speed improvements has slowed down, and due to the large differences in initial block download times, only multiple attempts to download can show significant improvements. However, after the release of Bitcoin Core 0.12.0, 0.13.0 to 0.19.0.1 were released, and each Bitcoin Core version had a small incremental performance improvement.

Of course, the initial block download time is just one metric, and there are other angles and conditions that can be used to evaluate the performance of Bitcoin Core. Although the initial block download time (IBD) is not the best metric for measuring software performance, it takes up a lot of resources and is therefore a good metric for benchmarking.

This report continues the two previous experiments:

In November 2018, Jameson Lopp attempted a similar study, but the analysis focused on independent implementations of older versions of Bitcoin Core (or simply “Bitcoin”, as some older software was named Bitcoin before “Bitcoin Core”).

Sjors Provoost also tried this experiment in July 2017, but Sjors had fewer syncs.

The complete test results and raw data are as follows

Figure 2-Bitcoin initial block download time (days)

(Source: BitMEX Research)

(Note: Block height is synchronized to 602,707)

System Specifications and Other Notes

Complete results table

(Source: BitMEX Research)

Results Analysis

As shown in Figure 2, even when attempting the initial block download using the same software and a computer with identical specifications, there is considerable variation in the times reported.

Figure 3 - Initial block download time vs. client release date (days) - average time for 3 attempts

(Source: BitMEX Research)

(Note: For Bitcoin 0.8.6 client, the above data is the average result of 2 attempts)

Figure 3 shows that, with the exception of Bitcoin Core 0.12.0, which has strong performance and has not changed, the performance of other software has gradually improved with each release. However, while there is a clear trend of performance improvement in Figure 3, there are huge differences in the initial block download time for each attempt, which may indicate that there is considerable uncertainty in the performance improvement. More sample data is needed to confirm the conclusion that performance has been improving since 2016. This difference may be due to connectivity issues with the Bitcoin P2P network or the Internet. Therefore, the best way to further study may be to re-scan the speed, that is, the time required to fully verify the blockchain once it has been downloaded.

Bitcoin Core 0.12.0 performs well in the above analysis. This may be because Bitcoin Core 0.12.0 enables libsecp256k but does not verify the transaction input signatures of the segregated witness. Therefore, Bitcoin Core 0.12.0 does not verify all signatures of the blockchain after August 2017, which gives it some kind of "unfair advantage".

However, Bitcoin Core 0.13.0 also has this advantage. Of course, all versions prior to 0.12.0 have the same "unfair" advantage, which pales in comparison to the disadvantages of using OpenSSL.

Sync the client to its release date

Figure 4 illustrates the time it takes to sync a client to the block height of its publishing date.

Figure 4 - Time required to synchronize the initial block download to the client release date (days)

(Source: BitMEX Research)

(Note: Node data running on Linux. Bitcoin Core 0.19.0.1 only synchronizes block heights to 602,707)

As can be seen from the figure, the change trend from Bitcoin Core 0.8.6 to Bitcoin Core 0.14.0 is relatively gentle. At this time, scalability has not followed the passage of time and the growth rate of block height, but it also shows an upward trend. The speed of software improvement has slowed down in recent years. It may be that those easy-to-implement improvements have been implemented, and the remaining ones are difficult to implement. The increase in transaction volume may also be one of the reasons. Improving scalability in the future may be more challenging. Even if the 4 million block height limit is maintained, the time for initial block download will continue to increase if the software is further upgraded and the software performance is improved.

Initial block download failed

We successfully compiled and ran versions of Bitcoin prior to 0.8.6, but in 2015-2016, the node synced slowly. Nodes prior to 0.8.6, such as 0.7.0, were able to successfully complete the 2013 hard fork by manually changing the lock limit, but in 2015, the node stopped processing blocks due to increased transaction volume, which was too challenging. We restarted the node, which did help, but it stopped again soon after.

We even ran Bitcoin Core 0.7.0 on our brand new local computer (with 64 GB of RAM and 8 Intel i9 processors), but the node still couldn’t make it past 2016. Since many of the scaling parameters involved are non-linear, we can’t simply throw more hardware at the problem.

When the node got stuck on a block we restarted, and after 4 failed restarts we gave up syncing. For 0.8.6 Bitcoin Core on a MacBook Pro, the leader stopped syncing in 2016. While a little disappointing, the remaining 35 attempts all synced without any restarts.

in conclusion

In addition to the fact that BitMEX research reports on MacBook Pros should be released with more caution, the data also shows that the scaling performance has greatly improved over the past 7 years. The libsecp256k migration is the most important improvement. The significant reduction in initial block download time and the inability of old nodes to fully sync show the importance of scalability for Bitcoin, otherwise Bitcoin is basically dead now even if users use the highest configuration hardware. In addition, the data also shows that technological innovation is unlikely to keep pace with the growth rate of the blockchain, and the initial block download time will continue to increase.