Talking about blockchain (22): In-depth understanding of soft and hard forks

The Bitcoin community is discussing the use of a soft fork to upgrade Segregated Witness (SegWit). There are currently many controversies. On the one hand, there are controversies surrounding Segregated Witness itself, and on the other hand, there are controversies caused by the soft fork (some people believe that Segregated Witness should be upgraded using a hard fork). Today, we will analyze it comprehensively from a technical perspective.

The issues involved in hard and soft forks are decentralized node software, protocols, and version upgrades. This is a very important issue that deserves discussion.

At the beginning, all nodes run the same software, follow the same consensus protocol, and maintain the same blockchain. At this time, some nodes upgrade to a new version of the software and have a new consensus protocol. At this time, the following four situations will occur:

1. The new node considers the blocks/transactions issued by the old node to be legitimate;

2. The new node believes that the blocks/transactions issued by the old node are illegal;

3. The old nodes believe that the blocks/transactions issued by the new nodes are legitimate (without noticing the changes);

4. The old node believes that the blocks/transactions issued by the new node are illegal (it has detected the change);

Note: Illegal transactions can be divided into illegal transactions of all and illegal transactions of some. To simplify the model, all transactions with some illegal transactions are classified as illegal transactions.

The actual situation is a combination of the above four situations. In addition, we need to add another dimension, which is the computing power ratio of new nodes to old nodes, which is divided into the following two types:

1. New node computing power>50%;

2. New node computing power <50%;

Please note that all the information about soft and hard forks that we have seen on the website so far is based on the discussion of the situation where the new node computing power is > 50%. Here we only discuss the situation where the new node computing power is > 50%. Please note that in another situation, that is, the new node computing power is < 50%, the situation will be completely different.

Soft fork is a term that came later. It did not exist before. Let’s first talk about what hard fork is.

What is a hard fork?

Hard fork means: when a new version of software (or protocol) appears in the system and is incompatible with the previous version of the software, the old nodes cannot accept all or part of the blocks mined by the new nodes (considering them illegal), resulting in two chains appearing at the same time. Although the new nodes have a larger computing power, for example, 99% of the computing power is new nodes, 1% of the old nodes will still maintain a different chain, because the old nodes cannot accept the blocks generated by the new nodes (even though they know that 99% of the nodes on the network have accepted them). This is called a hard fork.

Note: The above situation is based on the new node computing power > 50%.

When a hard fork is necessary, we need to require all nodes to upgrade their software at the same time, and nodes that have not been upgraded will not work properly. If many old nodes are unwilling to upgrade, they will work on a completely different chain (this is our ETC).

The picture below illustrates the reason for the hard fork: the new node requirements are much more relaxed than the old nodes.

What is a soft fork?

Soft fork means: when a new version of software (or protocol) appears in the system and is incompatible with the previous version of the software, the new node cannot accept all or part of the blocks mined by the old node (considering them illegal). Because the new node has a larger computing power, the blocks mined by the old node will not have a chance to be recognized. Both the new and old nodes work on the same chain from beginning to end. This is called a soft fork.

Note: The above situation is based on the new node computing power > 50%.

The picture below illustrates the reason for the soft fork: the new node requirements are much stricter than the old nodes.

Advantages and disadvantages of soft and hard forks

From the above definition, we can see that soft forks and hard forks have their own advantages and disadvantages. First of all, soft forks have the following advantages over hard forks:

1. A soft fork always has only one chain, and there is no risk of it splitting into two chains;

2. Soft forks do not require all nodes to upgrade at the same time, but allow for gradual upgrades, and do not affect system stability and effectiveness during the soft fork process;

But soft forks also have disadvantages:

1. The premise of soft fork is that old nodes can always accept blocks from new nodes, which requires the system to be designed to be forward compatible.

2. Soft forks are always based on deceiving old nodes, which prevents old nodes from noticing the changes that have actually taken place, which in some ways violates the principle of single-point complete verification.

Forward compatible

Forward compatibility is a very interesting term. It is different from the backward compatibility in software design. It requires a pre-judgment of new possible block forms and consensus mechanisms, and leaves room or backdoors in the old system. When new nodes are deployed, old nodes can accept the blocks generated by the new nodes.

So, how is forward compatibility achieved in Bitcoin?

The answer is: non-standard transaction .

The following rules are set in Bitcoin:

1. Classify all transactions into three types: standard transactions, non-standard transactions, and invalid transactions;

2. If an illegal transaction is found, it will be rejected. If the illegal transaction is in a block, the entire block will be rejected.

3. Under the standard configuration, all nodes take the following attitudes towards non-standard transactions:

1. Do not forward

2. Not accepted

3. No packaging

4. The following attitudes are taken for non-standard transactions that have entered the block:

1. Accept the block;

2. All transactions in the block, including the non-standard transaction, are verified according to the existing rules;

Below we use two examples to illustrate how Bitcoin uses non-standard transactions for soft fork upgrades.

1. P2SH soft fork upgrade

P2SH is included in BIP16, which upgrades the Bitcoin system through a soft fork, allowing Bitcoin to support a standard transaction type called P2SH on the basis of supporting P2PKH.

Prior to this, Bitcoin already supported the following script: "OP_HASH160 [20-byte-hash-value] OP_EQUAL". To spend such a script you only need to push the preimage of the hash value onto the stack.

P2SH is a stricter restriction on the above conditions. In P2SH mode (new node), you not only need to provide the original data of the hash value, but also ensure that:

1. The metadata is an executable script;

2. The script execution result returns: true;

3. The script cannot contain the "push data" operation;

4. And some other restrictions;

When a new node supporting P2SH is deployed, and the computing power of the new node is >50% (>=55% required for P2SH upgrade), the system will be in the following operating state:

• When a P2SH transaction is forwarded to an old node, it will be identified as a non-standard transaction (with a new or incomprehensible OP_CODE) by the old node, and the old node will not accept, package, or forward it;

• When the block containing the P2SH transaction is broadcast to the old node, the old node accepts the block and verifies the P2SH transaction according to the original rules. The result is passed because the old rules only need to verify whether the hash of the original data is equal (obviously they are equal);

• The old node created a script output with the original data: 123456 and spent the output. The old nodes were able to accept it. However, when broadcasting to the new node, it was not accepted according to the new rules (it must be a script...). The new node refused to accept it and deemed it illegal and would not package the transaction. Even if the transaction was packaged by the old node, it would be rejected by the new node. Because the new node controls the majority of computing power, such a transaction will never take effect;

• The system maintains one chain at a time.

P2SH upgrade details:

1. Require miners who support P2SH to include “/P2SH/” in their coinbase transactions;

2. On February 1, 2012, all blocks in the previous week (about 1,000) were checked, and if more than 550 contained it (about 55%), P2SH was activated.

2. SegWit soft fork upgrade

SegWit is mainly included in BIP141-144, which upgrades the Bitcoin system through soft forks to enable Bitcoin to support a series of SegWit function sets. For details about Segregated Witness, please refer to "Talking about Blockchain (21): Segregated Witness of Bitcoin".

Upgrading through a soft fork requires old nodes to recognize new transactions as non-standard transactions, but at the same time legal. How does segregated witness do this? The answer is: anyone-can-spend output transactions.

anyone-can-spend:

Here is an example of anyone-can-spend:

 scriptPubKey: (empty)

To spend such an output, you just need to provide a signature script like this:

 scriptSig: OP_TRUE

Segregated Witness perfectly combines script versioning with anyone-can-spend. The output of a standard P2WPKH is as follows:

 scriptPubKey: 0 <20-byte-key-hash>
                (0x0014{20-byte-key-hash})

The leading 0 will be the script version number in new nodes and an anyone-can-spend output in old nodes.

When a new node supporting SegWit is deployed, and the computing power of the new node is >50% (>=95% required for SegWit upgrade), the system will be in the following operating state:

• When SegWit transactions are forwarded to old nodes, they will be identified as non-standard transactions (anyone-can-spend) by the old nodes, and the old nodes will not accept, package, or forward them.

• When a block containing a SegWit transaction is broadcast to an old node, the old node accepts the block and verifies the SegWit transaction according to the original rules, and the result is passed (because it can be spent by anyone);

• The old nodes try to spend the SegWit transaction because it can be spent by anyone. The old nodes can accept it. However, when it is broadcast to the new nodes, it will not pass according to the new rules (the verification rules of the segregated witness). The new nodes refuse to accept it and regard it as illegal and will not package the transaction. Even if the transaction is packaged by the old nodes, it will be rejected by the new nodes. Because the new nodes control the majority of computing power, such transactions will never take effect;

• The system maintains one chain at a time.

SegWit upgrade details:

1. Upgrade to core 0.13.1 and above to support SegWit;

2. In November 2017, if 95% of the computing power supports it, SegWit will be activated.

Problems with the 1MB limit:

Blocks larger than 1MB are not non-standard transactions for old nodes, but illegal transactions. To solve this problem, SegWit has taken the trick of deceiving old nodes to the extreme. It puts the entire Witness part outside the original block. For old nodes, they don’t know that there is a witness structure behind the entire block they received.

Conclusion

Decentralized and distributed system upgrades will be one of the main challenges in the future. Both soft and hard forks will play an important role. Soft fork solutions are smarter and can effectively prevent blockchain forks. However, they rely on software forward compatibility, on deceiving old nodes, and on means or backdoors reserved in old systems.

Sometimes, in order to implement a soft fork, a function that could have been designed simply must be designed to be more complicated, increasing the risk of bugs, which is not worth the cost!

Despite this, soft forks still have huge advantages over hard forks, allowing for smoother upgrades and reducing the risk of community splits!

As for the debate on whether SegWit should be a soft fork or a hard fork, I believe you already have your own opinion. Welcome to reply to the author on WeChat official account to exchange views!