How Ethereum Works (Part 3) Final Chapter

Preface: This article mainly explains the specific operating principles of Ethereum at present, which helps us understand the various concepts and operations behind Ethereum and is suitable for beginners to read. ETH2.0 will undergo very big changes in the future. For details, please refer to the previous articles "ETH2.0: What will it be? (I)" and "ETH2.0: What will it be? (II)".

Continuing from the previous article "How Ethereum Works (Part 1)" and "How Ethereum Works (Part 2)".

Contract Creation

To recap, there are two types of accounts in Ethereum: contract accounts and external accounts. When we say a transaction is "creating a contract", we mean that the purpose of the transaction is to create a new contract account.

To create a new contract account, we first declare the new account address using a special formula, and then we initialize the new account as follows:

• Set the nonce (random number) to zero.

• If the sender sends a certain amount of Ether as value with the transaction, set that value as the account balance.

• The value sent to the new account is deducted from the sender's balance.

• Set the storage to empty.

• Set the contract's codeHash to the hash of the empty string.

Once we initialize the account, the init code sent with the transaction actually creates the account (for more information about the init code, see the "Transactions and Messages" section). Various changes occur when executing this initialization code. Depending on the constructor of the contract, it may upgrade the account's storage, create other contract accounts, make other message calls, etc.

When executing the code that initializes the contract, it uses gas. Transactions are not allowed to use more gas than the remaining gas. If this happens, the execution encounters an out of gas exception and exits. If the transaction exits due to an out of gas exception, the state is immediately restored to the point before the transaction. The sender does not receive a refund for gas that was used up in the previous execution.

However, if the sender sent any Ether value with the transaction, the Ether value will be returned even if the contract creation fails.

If the initialization code is executed successfully, the cost of the final successful contract creation will be paid. This is the storage cost, and the payment fee is proportional to the size of the contract code created. There is no free lunch here. If there is not enough remaining gas to pay the final cost, the transaction will be aborted due to insufficient gas again.

If all goes well and we have made it this far without anomalies, the remaining unused gas is refunded to the original sender of the transaction, and the altered state is allowed to persist.

Message call

The execution of a message call is similar to the execution of a contract creation, but there are some differences.

The message call execution does not include any init code, as no new accounts are created, however, it can contain input data if the data is provided by the transaction sender. Once executed, the message call has additional components, which contain output data, which will be used if the subsequent execution requires this data.

As with contract creation, if a message call exits due to insufficient gas or an invalid transaction (such as a stack overflow, invalid jump target, or invalid instruction), the gas used is not refunded to the original caller. Instead, all remaining unused gas is consumed and the state is restored to the point before the balance transfer.

Until Ethereum’s latest upgrade, there was no way to stop or revert the execution of a transaction without a system consuming all the gas you provided. For example, suppose a contract you authorized throws an error when the caller is not authorized to perform certain transactions. In previous versions of Ethereum, the remaining gas would still be consumed, and none of the gas would be refunded to the sender. But in the Byzantium upgrade, it includes new “recovery” code that allows contracts to stop execution and revert state changes instead of consuming the remaining gas, while also returning the reason why the transaction failed. If a transaction is aborted due to a reversion, the unused gas is refunded to the sender.

Execution Mode

So far, we have seen the series of steps a transaction must go through from start to finish. Now, let's look at how transactions are actually executed on the VM.

The part of the protocol that actually handles the transaction flow is Ethereum's own virtual machine, the so-called EVM. The EVM is a Turing-complete virtual machine. Compared to other Turing-complete machines, the only limitation of the EVM is that it is intrinsically bound to gas. In other words, its computational power is intrinsically bound by the amount of gas.

Source: CMU

In addition, EVM has a stack-based architecture. Stack machines use a last-in-first-out stack to store temporary values. The size of each stack item in EVM is 256 bits, and the maximum stack size is 1024.

The EVM has memory where items are stored as word-addressable byte arrays. The memory is volatile, meaning it is not persistent.

The EVM also has storage. Unlike memory, storage is stable and is maintained as part of the system state. The EVM stores program code separately in a virtual ROM that can only be accessed through special instructions. In this way, the EVM differs from the classic von Neumann architecture, where program code is stored in memory or storage.

EVM also has its own language: "EVM Byte Code". When a programmer writes an Ethereum smart contract, generally speaking, they use a high-level language such as Solidity. Afterwards, we can compile it into EVM Byte Code that the EVM can understand.

So, let’s start executing now.

Before performing a particular calculation, the processor ensures that the following information is available and valid:

• System Status

• The remaining gas for calculation

• The address of the account that owns the code being executed

• The address of the sender of the transaction that initiated the execution

• The address of the account that caused the code to be executed (may be different from the original sender)

• The gas price of the transaction that generated this execution

• Enter the data for this execution

• As part of the current execution, send value (Wei) to this account

• Machine code to execute

• The block header of the current block

• The depth of the current message call or contract creation stack

• When execution begins, memory and the stack are empty, and the program counter is zero.

PC: 0 STACK: [] MEM: [], STORAGE: {}

After that, the EVM recursively executes the transaction, calculating the system state as well as the machine state for each loop. The system state is just the global state of Ethereum. The machine state includes:

• Available gas

• Program Counter

• Memory content

• Number of active words in memory

• Stack contents

Add or remove stack items from the leftmost portion of the series.

In each cycle, the appropriate amount of gas is reduced from the remaining gas and the program counter is incremented.

At the end of each loop, there are three possibilities:

1. The machine reaches an abnormal state (e.g. insufficient gas, invalid instruction, insufficient stack items, stack items overflowed due to exceeding 1024, invalid JUMP/JUMPI destination address, etc.), so it must be paused and any changes will be discarded.

2. The sequence continues to process and enters the next loop

3. The machine reaches a controlled stop (the execution process ends)

Assuming the execution does not reach an abnormal state, and does not reach a "controlled" or normal stop, the machine generates the resulting state, the remaining gas after the execution, the accumulated sub-states, and the resulting output.

We have learned about one of the most complex parts of Ethereum. Even if you don’t understand it all, that’s okay. You don’t need to understand the details unless you want to work at a very deep level.

How blocks are finalized

Finally, let’s look at how blocks containing many transactions are finalized.

When we say "finalization," it can mean two different things, depending on whether the block is new or existing. If it's a new block, we're referring to the process required to mine the block. If it's an existing block, then, we're referring to the process of validating the block. In either case, there are four requirements to achieve block "finality."

1) Verify (or, in the case of mining, confirm) ommers

Each ommer block in the block header must be a valid block header and within the sixth generation of the current block.

2) Verify (or, in the case of mining, confirm) the transaction

The amount of used gas on the block must equal the cumulative gas used by the transactions listed in the block. (Recall that when executing transactions, we keep track of the block's gas counter, which keeps track of the total gas used by all transactions in the block).

3) Apply for rewards (mining only)

The beneficiary address is awarded 5 Ether for block mining. (According to Ethereum EIP-649 proposal, the 5ETH reward will soon be reduced to 3ETH). In addition, for each ommer, the beneficiary of the current block will receive an additional 1/32 of the current block reward. Finally, the beneficiary of the ommer block will also receive a specific amount of rewards (it is calculated according to a special formula, which is not detailed here.)

4) Verify (or, if mining, calculate valid) state and nonce

Ensure that all transactions and resulting state changes are applied, then define the new block as the state after all block rewards have been applied to the final transaction result state. Verify this by checking this final state against the state trie stored in the block header.

PoW Mining

The "Blocks" section briefly introduces the concept of block difficulty. The algorithm that gives meaning to block difficulty is called PoW. Ethereum's PoW algorithm is called "Ethash" (formerly known as Dagger-Hashimoto).

The algorithm is formally defined as follows:

Where m is mixHash, n is nonce (random number), Hn is the new block header (excluding the nonce and mixHash components, which must be calculated), Hn is the nonce of the block header, and d is the DAG, which is a large data set.

In the "Block" section, we mentioned the various items in the block header. Two of these components are called mixHash and nonce. You may remember:

• The mixHash is a hash value that, when combined with the nonce, proves that enough computation has been performed on the block.

• The nonce is a hash value that, when combined with the mixHash, proves that enough computation has been performed on the block.

The PoW function is used to evaluate the above two items.

Calculating the mixHash and nonce using the PoW function is a bit complicated and we could dive into that in a separate article. But at a high level, this is how it works:

“A “seed” is calculated for each block. This seed is different for each “epoch”, where each epoch is 30,000 blocks long. For the first epoch, the seed is the hash of a series of 32 bytes of zeros. For subsequent epochs, it is the hash of the previous seed hash. In this way, using this seed, the node can calculate a pseudo-random “cache”.”

This cache is very useful because it enables the concept of "light nodes". The purpose of light nodes is to allow specific nodes to have the ability to verify transactions, but at the same time do not need to store the entire blockchain data set. A light node can verify the validity of a transaction, it is only based on this cache, because the cache can regenerate the specific block it is verifying.

Using the cache, nodes are able to generate a DAG "dataset" where each item in the dataset depends on a small set of pseudo-randomly selected items from the cache. In order to become a miner, you must generate this full dataset; all full-data clients and miners store this dataset, and this dataset grows linearly over time.

Miners can take random data set slices and hash them together using a mathematical function, turning them into a "mixHash". Miners will repeatedly generate the mixHash until the output is below the required target nonce. When the output meets the requirement, the nonce is considered valid and the block is added to the chain.

Mining is a security mechanism

In general, the purpose of PoW is to prove in a cryptographically secure way that a certain amount of computation has been expended to generate a certain output (i.e., a nonce). This is because there is no better way to find a random number that is below the required threshold than to enumerate all possibilities. The output of repeatedly applying a hash function has a uniform distribution, so we can be sure that, on average, the time required to find such a random number depends on the difficulty threshold. The higher the difficulty, the longer it takes to find the nonce. In this way, the PoW algorithm gives meaning to the concept of difficulty, which is used to ensure the security of the blockchain.

What does it mean to make a blockchain secure? It's simple: we want to create a blockchain that everyone can trust. As we mentioned before, if more than one chain exists, users will lose trust because they can't reasonably determine which chain is the "valid" blockchain. In order for a group of users to accept the underlying state stored on the blockchain, we need a single canonical blockchain that everyone trusts.

This is what the PoW algorithm does: it ensures that a particular blockchain will always remain canonical, making it very difficult for an attacker to create new blocks that rewrite the history of transactions (i.e., erase transactions or create forked transactions) or maintain forks. In order for their blocks to be validated first, the attacker needs to continuously calculate random numbers faster than everyone else in the network, so that the network believes that their chain is the longest chain (based on the principles of the GHOST protocol). This will be impossible unless the attacker has more than half of the computing power, which is the so-called 51% attack scenario.

Mining as a Wealth Distribution Mechanism

In addition to providing a secure blockchain, PoW also provides a way to distribute wealth to those who expend computing power to provide security. To recap, miners receive rewards from mining, including:

• The winning block will receive a static block reward of 5 Ether (soon to be 3 Ether)

• Gas cost of transactions in a block

• Additional reward for including ommers as part of a block

The PoW algorithm is a method of security and wealth distribution. To ensure it is sustainable in the long term, Ethereum strives to instill these two properties:

Make it accessible to as many people as possible. In other words, people don't need special or uncommon hardware to run the algorithm. The goal is to make the wealth distribution model as open as possible so that anyone can get rewarded with Ether by providing a certain amount of computing power.

Reduce the possibility of any single node gaining disproportionate benefits. If any node can gain disproportionate benefits, it also means that the node will have a huge influence on determining the norms of the blockchain. This is troublesome because it reduces the security of the network.

In the Bitcoin blockchain network, one problem related to the above two properties is that the PoW algorithm is the SHA256 hash function. The weakness of this type of function is that it is more efficient using specific hardware, known as ASICs.

To alleviate this problem, Ethereum's chosen PoW algorithm is Ethhash, whose algorithm makes sequence memory difficult. This means that Ethereum's algorithm design requires a lot of memory and bandwidth to calculate random numbers. For computers, large memory requirements make it difficult to use memory for parallel calculations to find multiple random numbers at the same time, and higher bandwidth requirements make it difficult to find multiple random numbers at the same time, even for super-fast computers. This reduces the risk of centralization and creates a more fair competitive environment for verification nodes.

One thing to note is that Ethereum is moving from a PoW consensus mechanism to what is known as “PoS” proof. This is a rich topic and perhaps could be explored in a separate article in the future.

Conclusion

This is a lot to digest. If you need to read it a few times before you understand it, that’s ok, I hope the above explanations are helpful.