Blockchain Basics: Demystifying Misconceptions About Mining and Consensus

As a "traditional software" developer, I feel that the learning curve is still relatively steep when learning blockchain development. For example, I often misunderstand some seemingly simple concepts. I suggest that beginners read less articles like this one. I cannot guarantee that my current understanding is completely accurate. The purpose of this article is to take notes and share it with everyone for discussion.

Beginners should try to read some relevant classic materials first: such as white papers or official documents, relevant protocol documents, bip documents, or classics such as "Mastering Bitcoin". Those who are capable are recommended to read source code and release notes, and even read some email records or chat logs of developers, such as bitcoincore dev log;

With these basics, you can then look at other people's interpretations and related papers with questions in mind, which can reduce the probability of being misled by wrong views. In the world of blockchain, misunderstanding basic concepts can be fatal because it is all real money. Too many accidents are caused by the introduction of fatal vulnerabilities in a few lines of code, or by developing products based on codebases or frameworks that are not fully understood.

Blockchain technology has introduced many new terms, such as POW POS, distributed consensus, etc. This article will first talk about the more frequently used POW/POS and consensus algorithms, and first quote two passages:

1> What is consensus?

Consensus is a form of automated governance that populates and publishes ledger entries every x period. The “blockchain” is proof that this governance structure actually works as designed, providing near ABSOLUTE assurance over the integrity of the ledger data. -- https://blog.goodaudience.com/1-assurances-in-crypto-14c55a1fd616

2> Emin Gün Sirer

POS/POW itself is not a consensus:

1. Guessing game - mining mechanism solve the puzzle

POW and POS are different mining mechanisms. In essence, they are different guessing games with similar rules. They both aim to construct a block whose hash is less than a target value, which is the so-called dynamically adjusted difficulty value.

POW is to guess a nonce random value when constructing a block to meet the above conditions. When all nonces are invalid, you can adjust the transactions contained in the block and fine-tune the timestamp. The more computing power, the greater the chance. This is proof of work.

;

POS is to find a UTXO to construct a coinstake transaction, satisfying the kernel hash is less than the weighted difficulty value corresponding to the UTXO.

The more UTXOs there are, or more precisely, the larger the stake, the greater the chance of finding it. This is proof of stake.

It can be seen that the so-called POW POS is just a different requirement for producing blocks. The guessing game is played in rounds, and a block is generated in each round. After each round of block production, it is broadcast and published on the blockchain network. All nodes will select the winning block of this round based on the consensus algorithm and connect to the tip, that is, connect to the current blockchain. So I call POW POS a ticket to consensus or a ticket to become a candidate.

#POW applet example: https://github.com/bitcoinbook/bitcoinbook/blob/df1828b7205a5950a16a3182cf9b15421ee70658/ch10.asciidoc#pow_example_outputs

#POSv3 detailed explanation:

http://earlz.net/view/2017/07/27/1904/the-missing-explanation-of-proof-of-stake-version

2. Voting procedure - trustless consensus

POW POS is just a ticket to become a candidate. The final winner of this round is decided by the majority of voters. The rules of their voting are the consensus algorithm itself. Voters are nodes in the network. Of course, I am talking about full nodes here. The work of voting includes (the detailed process is not expanded here, it is a high-level overview):

1) Validate and relay transaction: The process of verifying and relaying the received transaction can be regarded as one of the verification steps of voting. If a candidate block contains an illegal transaction, then the candidate block loses its legitimacy.

2) Full validate block and activate best chain: Perform more detailed validation on the received block (block level and transaction level validation) and find the longest chain. Of course, the actual process is dynamic, and the longest chain at the moment may not be the final longest chain.

There will be continuous soft forks, orphan blocks, and even reorgs. Bitcoin generally requires six confirmations (generally speaking, as long as the world's network does not experience large-scale and long-term paralysis, so-called deep reorgs are unlikely to occur, unless it is a malicious long range attack);

In the blockchain network, mainly referring to the public chain represented by Bitcoin, each client, whether it is an ordinary mobile wallet or a mining node, has its own role in this network. For ordinary mobile wallets (hot wallets), their main task is to broadcast and relay transaction information, while the purpose of mining nodes is to earn coinbase rewards and handling fees, so miner nodes strive to generate candidate blocks. These miner nodes may run different official versions, some are modified unofficial versions, and some are even malicious hacker nodes, so the consensus algorithm is the basis for these good and bad nodes to collaborate without trusting each other, that is, to reach the so-called trustless consensus;

The essence of consensus is to solve the Byzantine Generals' Problem. The consensus algorithm is the Byzantine Fault Tolerance algorithm (this is a general term, not specifically BFT or PBFT). The purpose of POW POS is to increase the cost of bad nodes or traitor nodes to do evil. In fact, the cost of doing evil for POW is really high, especially for the Bitcoin network. However, there are many low-cost ways to do evil for POS, which will be mentioned in the following 3.

Common consensus algorithms include Nakamoto consensus, which is a Nakamoto consensus algorithm composed of a series of consensus rules based on the longest chain principle. For details, see: https://github.com/bitcoinbook/bitcoinbook/blob/develop/ch10.asciidoc#decentralized-consensus

Other consensus algorithms include PBFT (used by Ripple, Stellar, and Hyperledger), see detailed explanation

In simple terms, when a client sends a transaction request, the primary node (analogous to the voter representative) sends a pre-prepare message to other nodes, and the nodes that receive the message respond to the prepare message. When each node receives the ack from other nodes, all nodes send a commit message. When enough commit messages are received, they reply to the client.

Then here we have to mention the so-called DPOS algorithm

The DPOS algorithm is divided into two parts: electing a group of block producers and scheduling production. The election process makes sure that stakeholders are ultimately in control because stakeholders lose the most when the network does not operate smoothly.

--

https://steemit.com/dpos/@dantheman/dpos-consensus-algorithm-this-missing-white-paper

As mentioned above, the DPOS algorithm has two parts, ‘D’: selecting block producers, ‘POS’: generating blocks, so DPOS itself is not a consensus, because consensus occurs between these block producers after the block is generated;

In my opinion, the main difference between DPOS and POS is that POS is a ticket for the majority to participate in the consensus, while DPOS is a ticket for a small number of selected people to participate in the consensus. Therefore, for EOS, consensus must still be reached based on the longest chain principle. The potential problem here is that some rich nodes with a large number of coins may do whatever they want, so EOS proposes to introduce the BFT consensus model. If you are interested, you can read this article "DPOS BFT— Pipelined Byzantine Fault Tolerance"

If you want to ask what will happen if some nodes do not comply with the consensus or modify the consensus, the answer is that a fork will occur. However, if there are not enough nodes to support and join your new consensus, your forked chain will be meaningless. Therefore, consensus is the consensus of the majority. The consensus of the minority is only meaningful on public chains based on DPOS or some consortium chains.

3. Attack vector attack vector

Understanding these will give you a clearer understanding of some complete problems or safety incidents, such as

1) Some attack methods are economically feasible for many POS chains, but not economically feasible for POW chains such as Bitcoin

For example, long range attack and noting at stake attack. The main reason is that the cost of producing a block on POW is very high, not to mention deliberately choosing the wrong branch, the cost is even greater. POS does not have this problem. There is no cost for producing a block. As long as there is a certain amount of stake, it can be done. Sometimes you can even buy stake, then construct an attack transaction, and then sell the stake. The cost is just the handling fee. Don’t misunderstand nothing at stake attack as no stake is required. It means that the miner or forger will mine on multiple branches at the same time. Because the cost of producing blocks is low, multiple branches can increase the chance of obtaining rewards. Nothing at stake will eventually lead to double-spending attacks.

The solution is to improve the consensus algorithm, such as setting checkpoints (which has also been adopted by Bitcoin), penalizing equivocation nodes,

Specifically, there is the Slasher algorithm. If you are interested, you can take a closer look at Ethereum's Casper.

2) With the same codebase and the same header-first node synchronization, POS will suffer from resource exhaustion attacks (i.e., attacks that exhaust memory and disk space), while POW can avoid this problem.

This attack method simply takes advantage of the fact that when a node is synchronizing a forked branch block, if no reorg occurs, that is, the branch node is not the longest node, full validation will not be performed (full validation of double spending and signatures, etc.), thereby consuming memory (sending headers) or disk (sending blocks).

-- Resource Exhaustion Attacks on PoS

You can see Qtum’s response to this research:

Re: “Fake Stake” attacks on chain-based Proof-of-Stake cryptocurrencies

From the above two examples, we can see that any small changes or vulnerabilities in the consensus algorithm may introduce multiple attack vectors. We can learn a lot of ways of thinking from them. Here are the links to the vulnerabilities that have occurred in Bitcoin history:

https://en.bitcoin.it/wiki/Common_Vulnerabilities_and_Exposures

https://en.bitcoin.it/wiki/Weaknesses

You can learn the following analysis report: two vulnerabilities caused by one line of code: DOS and double spend

BitcoinCore CVE-2018-17144 Vulnerability Research and Analysis

4. More thoughts

Although I am just a blockchain technology enthusiast and have no chance to participate in public chain development, my in-depth understanding is helpful to my development ability. It is quite challenging to cultivate a decentralized global thinking. My peers who are familiar with distributed development or multi-threaded development may find it easier to get started than me. Of course, in addition to technical thinking, many places also need to combine economic incentive mechanisms. For example, this paper raises the verification dilemma of miner nodes:

If a full verification (including script execution verification) is performed, it will take a certain amount of time. The advantage is that illegal transactions can be excluded. If the verification is skipped, more time can be gained than other miners, but the danger is that illegal transactions may be included, thereby wasting computing power and paying the price. This paper is quite interesting. It feels a bit like showing the contradictions between the mining mechanism and the consensus algorithm mentioned above. Driven by the interests of mining, should consensus rules be fully followed or selectively followed? If you are interested, you can learn about the solution proposed by the author, "THE VERIFIER'S DILEMMA".