Practical information丨Detailed explanation of how Ethereum 2.0 will merge with 1.0

Note: The original author is Danny Ryan (djrtwo), the coordinator of Ethereum 2.0. In this article, he details how Ethereum 1.0 will merge with Ethereum 2.0. According to his introduction, in the eth1+eth2 combined client, the eth2 client can handle the complexity of PoS and sharding consensus, while the attached eth1 client can become an eth1 engine, which can handle the complexity of things such as status, transactions, virtual machines, etc.

(Photo from: tuchong.com)

The relationship between Ethereum 1.0 and Ethereum 2.0 clients

Since Vitalik proposed an early eth1 <-> eth2 merger alternative in December 2019, researchers have been actively discussing possible forms of this merger from a software perspective, and expectations for prototype design have become stronger. Our vision is to create a hybrid where the core consensus work is managed by the Ethereum 2.0 client (hereafter referred to as the eth2 client), and the state/blocks are managed by an Ethereum 1.0 engine (hereafter referred to as the eth1 engine), and together they constitute the eth1+eth2 combined client.

This article aims to more clearly distinguish the responsibilities between the eth2 client and the attached eth1 engine, in order to provide a better basis for conversation, specification writing and prototyping. Note that this article does not define the specific details of the protocol (such as the precise method in which the eth1 client calls the eth2 engine), and any examples included in the article are only used to help describe and subsequent discussions.

To understand the content of this article, the prerequisite is that you need to be basically familiar with the concepts of Ethereum 2.0 and stateless Ethereum.

Clear division of labor

The purpose of the eth1+eth2 merger is to leverage the existing Ethereum 1.0 state, ecosystem, and software in the upgraded Ethereum 2.0 consensus environment.

In a nutshell, what we think of as eth2 clients today handles core PoS and shard consensus. In essence, the eth2 protocol and eth2 clients are designed to be very good at generating and reaching consensus on a bunch of "stuff", which are a lot of shard chains full of data and (final) states. Compared to the PoW consensus layer of eth1 today, the "consensus layer" of eth2 is much more advanced and complex.

Today, the eth1 client has a relatively simple and thin consensus layer, it has only one chain, and PoW can handle most of the complexity in hardware outside the protocol. Most of the complexity and optimization of the eth1 client are located in the user layer (including state storage/management, state synchronization, virtual machine execution, transaction processing, transaction pool, etc.).

This separation of concerns allows for a nice pairing when eth1 is incorporated as a shard into eth2, where the eth2 client can handle the complexity of PoS and sharded consensus, while the attached eth1 client can become the eth1 engine, which handles the complexity of state, transactions, virtual machines, and things closer to the user layer.

Minimal changes to achieve local communication

There are many possible approaches to combining eth1 and eth2 client software (full merger, importing eth1 as a library, communication protocol between the two, etc.), but in this post we will focus on the most minimally invasive and modular approach - a native communication protocol between eth2 clients and a simplified eth1 engine.

Given the diversity of eth1 and eth2 client implementations, this approach prevents client software lock-in on either side, allowing client teams to remain independent and focus on their own R&D efforts, keeping software projects largely stable for rapid prototyping.

So what will it look like?

Roughly speaking, an eth1+eth2 combined client would look like this:

The eth2 engine runs together with the eth1 engine, communicating locally via RPC driven by the eth2 client.

Both will maintain their own p2p interface, connect to peers and handle the network protocols associated with each specific domain.

Ethereum 2.0 Client

1. The beacon chain and beacon state (the core consensus objects on which the rest of the system is built);

2. Shard chains (1. eth1 shard chain, 2. many data-only shard chains);

3. Mempool operations [not shown] (attestation, deposit, exit, etc.)

4. P2P interface (1. consensus layer information, 2. including eth1 shard block gossip);

5. RPC to eth1 engine (all calls are driven by eth2 client);

Ethereum 1.0 Engine

1. EVM virtual machine (execution and verification of eth1 shard blocks);

2. eth1 state (the state of user-layer eth1 in Ethereum today);

3. Transaction storage pool Mempool (user transaction mempool, preparing for block production);

4. P2P interface (1. transaction gossip on Ethereum today, 2. state synchronization, 3. no eth1 shard block gossip);

5. RPC from eth2-client (all calls are driven by eth2-client);

consensus

From the perspective of core consensus, the eth2 client is responsible for and drives the construction of the beacon chain, data shard chain, and eth1 shard chain. The eth2 client directly provides any knowledge about the eth1 shard chain and core consensus (beacon chain/state) to the eth1 engine through RPC.

Specifically, the attached eth1-engine must be able to access the eth2-client because it cannot maintain its own consensus. In Ethereum's PoW today, the eth1-client checks the proof of work, forms a tree structure, and runs the fork selection rule to find the tip of the chain. In eth2, these mechanisms are very different, which requires a deep understanding of eth2's core consensus. The eth2 client provides the latest information about the head of the eth1 shard chain so that the eth1 engine can maintain an accurate view of the eth1 state.

Since the eth1 engine completely relies on the eth2 client to promote consensus, we propose that the communication between the eth2 client and the eth1 engine is all methods on the eth1 engine called by the eth2 client (such as addBlock, getBlockProposal, etc.). This will enforce a leader/follower relationship to reduce the complexity of system reasoning and limit the business logic required by the eth1 engine.

From the perspective of the eth2 client and core consensus, eth1 shard chains are processed almost exactly like all other shard chains (fork choice, crosslinks, block structure, signatures, etc.). The main difference is that shard block content can be executed against the eth1 engine, so the format of eth1 shard block data must be relevant to eth1 and additional validation must be done for this to succeed.

state

eth2 has a state related to the core consensus, which is called the "beacon state". The beacon state data is small (about 10-40MB, depending on the size of the validator set), and it contains all the information needed to understand the core consensus and how to process the shard chain. In fact, to process the consensus-related parts of the shard chain, the client must be able to access the beacon state (for example, the latest crosslink to run the shard chain fork selection, the current validation set to verify the shard chain signature, or the shuffling random allocation).

The state of eth2 does not always interact with the state of the user layer. The most interactive part is the availability of the shard chain data. The actual user layer data root is located in the shard chain data. For the eth1 shard chain, it is the current Ethereum user state root.

The following discusses different situations in which eth1 states are relevant to the eth2 client:

1. eth2 client without eth1 engine

The core eth2 protocol can run without the additional eth1 engine. A separate eth2 client can follow the beacon chain and shard chains (including eth1 shards). Without the eth1 engine, clients will not be able to execute stateless eth1 shard blocks, and therefore cannot fully verify them or obtain any useful user information from them. However, based on the assumptions about the eth2 core consensus and validators, the head of the eth1 shard chain can still be found safely.

2. eth2 client with stateless eth1 engine

To run a validator node, you must run the eth2 client with the eth1 engine attached. This can be done in a stateless manner (i.e. without storing the entire eth1 state locally), so the eth1 shard blocks have the witness available for execution. The beacon committee can check the availability of shard block data and the validity of data about eth1 by making stateless calls to the eth1 engine.

In addition to validators, many user/application nodes may also run using a stateless or semi-stateful eth1 engine. Use a thin eth2 client to follow the head of the eth1 shard chain and interact with it in a stateless or semi-stateless manner.

3. eth2 client with stateful eth1 engine

To run a validator that can produce eth1 shard blocks, the eth2 protocol must be run with an attached eth1 engine and the full eth1 state (developers are exploring stateless block production methods, but for simplicity we will not discuss them). The local state and transaction storage pool (TX mempool) can then be used to form new valid blocks on demand (more on this below).

In addition to validators, many user/application nodes may also run using the fully stateful eth1 engine, such as block explorers, archive nodes, state providers, etc.

network

For simplicity, eth2 and eth1 will initially maintain their own separate network stacks and protocols. In response to the transfer of responsibilities (such as eth1 shard block gossip), developers have deprecated the use of some existing eth1 protocols (such as eth1 shard block gossip) and replaced them with eth2 protocols. After the initial prototyping phase, or at a further stage, it may be necessary to migrate the eth1 protocol to libp2p to unify the network stack, but this is not required.

eth2-client and eth1-engine have access to the same discv5 DHT, but independently find appropriately capable peers and maintain connections independently.

ENR

Combined eth1+eth2 clients will use one ENR because the node sits behind a logical network identity with multiple capabilities.

eth1 functionality (state, transactions, etc.) is represented by the existing eth (or new eth1) key in ENR.

eth2 functionality (core consensus) is represented in ENR by the eth2 key.

The existence of each protocol means that the node is able and willing to identify the category of the underlying network protocol.

Wire Protocol

1. eth2 protocol

1. eth2 request/response (1. status, 2. beacon block synchronization, 3. shard block synchronization); 2. Core consensus gossip (1. Beacon block, 2. proof, 3. shard block, including eth1 shard, 4. other validator operations);

2.eth1 protocol

1. A subset of the eth1 wire protocol (1. transaction gossip, 2. synchronization methods such as getnodedata or new methods, 3. getting receipts)

2. NOT (messages related to block hash, block header or body);

3. Why does the eth2 client process eth1 block gossip?

eth2 is specifically designed to handle the production, gossip, and validation of shard blocks. Our goal is to make eth1 shards the standard shard and as consistent as possible with the rest of the shards. Regarding core consensus, the main difference of eth1 blocks compared to other shards is the ability to execute/validate block content against the eth1 engine,

When the validator is forking the eth1 shard block to the beacon chain, the eth2 client will again call the eth1 engine to execute and verify the block.

When a stateful eth1+eth2 combined node receives a new eth1 shard block, the eth2 client will call the eth1 engine again to validate the block and update the local state storage.

Transaction gossip and mempool

The eth1 engine will maintain user transaction gossip and eth1 transaction storage pool in almost the same way as the current Ethereum. The same network protocol and local mechanism can be used to maintain gossip and storage pool to prepare for block production.

The main differences are in how knowledge of spent transactions is determined, and how the storage pool is used for block production, but these are arguably in a layer external to the storage pool.

The eth1 shard blocks are provided to the eth1 engine from attached eth2 clients. The transactions contained in these blocks should be cleared from the storage pool in a similar manner to current Ethereum mainnet PoW blocks.

The eth1 shard blocks are generated based on the contents of the mempool from the attached eth2 client. This RPC method and underlying functionality is similar to getWork, but will return the full block contents instead of just a hash.

Block production

In the eth2 protocol, all blocks (beacon blocks, shard blocks, eth1 shard blocks) must be produced and signed by PoS validators according to the core consensus. To this end, the eth2 client is ultimately responsible for the production of all blocks.

For beacon blocks and non-eth1 shard blocks, the eth2 client has everything it needs to produce a valid block.

For eth1 shard blocks, eth2 clients immediately/readily access eth1 states, transactions, and other underlying eth1 structures to generate valid blocks. Conversely, when a designated validator generates an eth1 block, the eth2 client requests a feasible eth1 block data (TX, state root, etc.) from the eth1 engine. The eth2 client then packages this eth1 block data into a complete shard block (adding slot, positer_index, positer_signature, etc.) and broadcasts the block to the network.

The eth1 engine is able to generate valid/viable eth1 block data because it manages the eth1 transaction storage pool in the same way that Ethereum mainnet does today, and it maintains the latest information about the eth1 header state through updates from the eth2 client.

What’s the next step?

If the overall design is accepted by everyone, the next steps include:

1. Ensure that assumptions about eth2-client driving the eth1-engine are consistent with, and do not introduce unexpected burdens on, existing eth1 software;

2. More clearly defined communication protocols used to drive the eth1 engine, e.g. new_head(block), validate_block_transition(block), get_proposal(parent_root), etc.

3. Define network components, such as which subset of the eth1 protocol is needed and how to use ENR specifically;

4. Scaling Ethereum 2.0 Phase 1 Specification

5. prototype!

This article is translated with permission from the author Danny Ryan.