The principle of POW hollowing blocks and the solution to the problem of hollowing blocks

POW Mining Principle

Proof of Work (POW) mining is the process of generating new blocks in the blockchain. Including BTC, BCH, ETH, LTC, etc., POW mining is based on the following logic.

Mining is like this. The system will give a target value, which is a number, for example, with 18 zeros in front. Mining is to find a random number that is less than this target value, for example, with 19 zeros in front. Once you find it, you have mined a block. Keyword: random number. Key question: How is this random number calculated?

This random number is obtained by hashing the latest height of the blockchain by the mining pool. Key words: latest height. Key question: What is the latest height?

Now a mining pool has mined a block and broadcasted it. We assume that the block height is 50W. Then all mining pools will continue to mine after this 50W height. The specific action is that the mining pool first assembles a new block. This new block contains the "block header" and the "block body filled with transactions". There are two things in the block header that need special explanation. The first is the parent block hash value, which is the hash value of the 50W height here; the second is Nonce, which is a number, an adjustable number, starting from 0.

After assembling this new block, the mining machine hashes this new block (“50W+1” height) plus the previous block with a height of 50W. This new block is the “latest height” above.

The mining machine performs hashing once and obtains a hash value, which is a binary number. The mining pool will compare this value with the target value. If it is less than the target value, congratulations, you have successfully mined the mine. Otherwise, the Nonce will be increased by 1, hashed again, and compared again, ...

Obviously, if one mining machine is used to hash, it will be very slow to try to find the Nonce value that meets the target value. However, the mining pool can assign different Nonce values ​​to different mining machines for testing. For example, if there are 1,000 mining machines in the mining pool, the first machine will be given the value from 0 to 1000, and the second machine will be given the value from 10001 to 2000... This is the parallel mining.

The principle of POW mining block

We need to further elaborate on the 50W height and the latest height block above. Before the mining pool mines the latest height blockchain, it must verify the 50W height block. If the verification is successful, mining at the newly assembled block height of "50W+1" is legal. If the 50W height is an illegal block, all mining pools must return to the "50W" height to mine.

However, it takes time to verify a block, because the software and hardware of each mining pool are different. We assume an average time of 10 seconds to verify a 1M block. Before the verification is completed, the mining pool cannot confirm whether the block is legal, and cannot confirm whether the latest block with a height of "50W+1" should be added.

It also takes time to assemble a block with the latest height. Get a block template and start assembling it, fill in the parent block hash value and select transactions in the memory pool to fill into the block body. This all takes time. We assume it takes 0.1 seconds.

When verifying the "50W" height block and filling transactions into the "50W+1" block, there is a pre-emptive action: the filled transactions must not overlap with the transactions that have been packaged in the 50W height. Otherwise, your latest height block will be illegal.

It also takes time for the block with a height of "50W" to be mined and broadcast to all mining pools. Block broadcasting is divided into two steps. The first step is to broadcast the block header; the second step is to broadcast the block body. The block body is very small, only 80 bytes, which is smaller than a transaction. The broadcast is very fast, and all mining pools can get the block header very quickly. With the block header, you can get the "parent hash value" for assembling the new height block. The block body is the part that contains all transactions, which is generally very large. For example, 2,500 transactions are close to 1M. BTC can be as large as 1M, plus an additional 3M segregated witness block. BCH once packaged a 21M block body. It takes longer for all mining pools to receive the block body, which is called network latency. We assume that the network latency is usually 0.5 seconds.

The simplest way of mining in a mining pool is: a mining pool can only confirm which transactions are in the block body after getting the block body. And only after verifying the entire block can we know whether the block with a height of "50W" is legal. Then start assembling the latest height block, and then mine and calculate the hash value less than the target value. So how long does it take to get the block body, verify the entire block, and assemble the latest height block? 0.5+10+0.1=10.6 seconds.

If a mining pool mines according to the above process, it will not be able to mine during this 10.6 seconds, and the mining machine is ineffective. In addition, the mining machine is still turned on and consumes electricity.

If a mining pool works according to the following process: after getting the block header, it starts to assemble a new block of "50W+1" height. However, because it is impossible to confirm whether the transactions in this 50W height block and the entire block are legal, the mining pool does not fill the block body with additional transactions for this "50W+1" height block, but only fills it with a Coinbase reward transaction. This "50W+1" height block only has a block header and a Coinbase reward transaction - this is an empty block - and then starts mining directly. In this way, when a new block appears in the entire network, the mine only needs to wait for the time to get the block header and assemble the empty block before it can start mining. This time is very short, usually less than 0.5 seconds. This saves more than 10 seconds compared to 10.6 seconds.

According to the above mining process, the mining pool takes 10.6 seconds to receive the complete 50W height block and verify the 50W height block. During the above 0.5 to 10.6 seconds, if the mining machine calculates a random number less than the target value, the mining pool will successfully mine an empty block. If the mining machine does not calculate a random number less than the target value, the mining pool will give up the 10.1 seconds of mining time and instead assemble a latest height block of "50W+1" filled with transactions for mining.

The above is the entire logic of mining blocks.

Optimizing mining principles

In the above-mentioned principle of mining, confirming the transactions in the block with a height of "50W" is an important task. If these transactions can be known in advance, then when assembling the block with a height of "50W+1", can't we avoid these transactions and assemble other transactions? In this way, there is no need to create empty blocks.

Currently, BTC and BCH networks have two technologies that can optimize the propagation of this block body, which can also allow all mining pools to know which transactions are in the "50W" height more quickly. The first technology is called Bloom filter; the second is called compact block and thin block. (Compact block is developed by Core dev, and Xthin is developed by unlimited dev. Both have the same function.)

Bloom filter is used to mark which transactions are in the block with a height of "50W". It is just an identifier. The mining pool will know which transactions are in the block after getting this mark. Then the mining pool will search for these transactions in its own memory pool, list the transactions it does not have, and then ask other nodes for the transactions it does not have. After getting these transactions, it will reassemble a block with a height of "50W" locally instead of downloading a block with a height of "50W" from the network. This process is the principle of Compact block and Xthin block.

These two technologies can shorten the network delay time, but they still cannot shorten the time to verify the "50W" height block. The former is only 0.5 seconds, while the latter is 10 seconds.

With such a time gap, the mining pool still has to mine empty blocks, otherwise the entire mining machine will be wasting electricity. You must know that this waste of 10 seconds is an enormous amount of energy for the entire network.

What should I do then?

A complete solution to the problem of hollow blocks

The above analysis is about the principle of mining empty blocks and the principle of optimizing mining. The core reason for mining empty blocks is that the mining pool dare not fill the empty block "50W+1" with the transactions that already exist in the "50W" height.

If there is a way to ensure that the empty blocks at the height of "50W+1" assembled by the mining pool do not contain any transactions that have appeared at the height of "50W", wouldn't that solve the problem?

One way is for the mining pool to generate some transactions by itself, not broadcast them, and only save them in its own memory pool to fill the empty blocks with a height of "50W+1".

However, the mining pool cannot generate some meaningless transactions, otherwise it will be some junk transactions, which is a waste of the network. It is definitely meaningful to package the transactions sent by users, but the transactions sent by users are often broadcasted, and packaging will have the risk of conflicting with the transactions in the "50W" height block.

Be sure to look for transactions that have not been broadcast.

One method is for mining pools to cooperate with exchanges. When users initiate withdrawal transactions on exchanges, the exchanges do not broadcast them, but instead send them directly to the mining pools in an encrypted manner via IP to IP. These transactions will never be packaged in the "50W" height block. These transactions can be entered into the "50W+1" height with 100% no conflicts.

If the exchange and the mining pool are the same company, this method is very easy to implement. If they are not the same company, the mining pool is still afraid of being cheated by the exchange. If the exchange inserts a broadcasted transaction, the block reward of an empty block may be wasted.

However, this method will reduce the user experience of the exchange, because the probability of hitting an empty block is low. It will take a long time for these transactions to be packaged. Although the mining pool can package them in non-empty blocks, users have to wait for the mining pool to produce blocks before they can get confirmation, which is a long time after the entire network produces blocks.

Another method is to let the exchange submit confidential transactions to the mining pool not as user withdrawal transactions, but as transactions that combine scattered UTXOs into large UTXOs. This type of transaction greatly optimizes the entire UTXO volume, and these transactions do not have very strong time requirements. They are suitable for being packaged in empty blocks.

We only need to consider the economy because the POW mining mechanism is the optimal setting, and morality should not be involved. All the criticisms of mining pools for empty blocks are a lack of confidence in the POW mining mechanism.

Original author: Huang Shiliang

Original title: POW mining pool block mining principle and solution

Source: Lightning HSL