Ethereum: Understanding Sharding in the Easiest Way

Introduction

As we all know, Ethereum's success has surpassed its original model, especially its scalability and its use of proof of work as a recognition of trust. One of the dreams of the Internet was to build an infrastructure that was distributed around the world and capable of processing and services, but we ended up with a centralized approach where data was concentrated in centralized servers. In Ethereum's worldview, data is distributed around the world and stored in many places that have no centralized control over the current state of the Internet. In a perfect Internet, we need to have a local version of the blockchain and communicate with it, and we know that this is the same state that someone else on the other side of the earth is using.

We are now seeing sharding being rolled out to the Ethereum infrastructure, which will significantly increase the number of transactions that can be processed at a given time. With sharding, Ethereum nodes only need to store a portion of the distributed registry and forward that information to other nodes in the network.

Sharding

Let’s take a closer look at how sharding works.

The main problem we are solving is that in order to create a consensus, every node must perform all the calculations and verify every transaction. When the number of transactions increases, it slows down the network and Ethereum hits a bottleneck here.

For all transactions on the Ethereum network, we have a certain state - the global state. Everyone should be able to check this state and see all transactions. But now we can split the state using a Merkle tree, with the root defining the global state and breaking the state down into shard nodes. Below, we have three shards that make up the global root, and each shard collects the transactions they are responsible for:

So we split the overall state of the infrastructure into multiple shards, and each unique account is in one shard, and these accounts can only trade with other accounts in this shard. Vitalik Buterin thinks that you can think of the blockchain as being divided into thousands of islands, each of which can do its own thing. On each island, people on the island trade and trade with each other without requiring people from other islands to get involved. As long as we have a reliable agent on the island recording the transaction, it can be entered into the global transaction ledger. If people on the island contact other islands, we can set up a mechanism to support this and link the transactions.

The way the sharding system works is by creating a transaction group that has a transaction group header and a transaction group body (and stores all transactions related to that shard). In the example below, we take Shard ID 22 as an example. In the transaction group header we have:

• Shard ID: This identifies the shard that the transaction is associated with.

• Pre-state root: It defines the state Shard 22 root before the new transaction.

• Post state root: It defines the state Shard 22 root after the new transaction.

• Receipt Root: This is the receipt for Shard 22 used for the transaction.

We can also see the signature in the transaction group header. These nodes are chosen to validate transactions in the shard, and this is chosen randomly. Therefore, each shard identifies itself with an ID and then maintains its own state with its own transactions and mapping back to the previous state. This is similar to running a separate blockchain for a shard, where transactions within the shard do not interact with other transactions outside the shard.

But what happens when an account in one shard wants to transact with another shard? We record a transaction and send a receipt proof to Shard B. That shard then sends back the receipt for the spend.

The main challenge of sharding is whether we can ensure that the shards work on trusted nodes and where the shards can trust each other.

Danksharding

In the process of achieving a fully sharded infrastructure, we need to migrate existing methods to a fully sharded infrastructure, and one of the methods being implemented is Danksharding (named after Ethereum developer Dankrad Feist). Danksharding aims to increase speed, reduce gas fees, and large-capacity data storage. It is basically part of the move towards a fully sharded infrastructure for Ethereum. In this regard, we say goodbye to miners and now have applicants, builders, and validators.

In summary, we select the data and transactions that go into a slot through trusted block builders, who then make requests to applicants. Builders offer transaction fees to applicants, who then choose the highest bidder. If a builder is selected for a given slot in a block, they are responsible for adding the data and transactions to that slot. Validators then verify that the slot was built correctly. The core advantage is that builders propose a merge-free scheme from transactions, so the gas fees for transactions are significantly reduced because there are no gas fees associated with each transaction. As long as there are not too many bidders, the total cost of filling a slot can be quite low.

Miners in our existing Ethereum infrastructure are basically just block applicants, but once the new infrastructure is implemented, they will become validators, and new software agents called block builders will play a key role in collecting and proposing transactions for data and slots. If a slot is verified as invalid, an erasure code can be proposed, so the data there can eventually be removed from the slot.

EIP-4844 and Blob

The rollout of any new infrastructure needs to be done with caution, as a single vulnerability could lead to a large-scale compromise. Therefore, EIP-4844 was proposed as a prototype specification for danksharding:

The full name of the proposal is related to Binary Large Objects (blobs). While data blobs exist in many other application areas, the main difference of the new proposal is that they can be associated with smart contracts. For blobs, the proposers suggest that there should be a data packet associated with the transaction. They can have a header and a data body and be digitally signed by the applicant. The maximum size of this data blob is proposed to be 1MB. This will allow transactions to access shared data.

in conclusion

If Bitcoin is the Ford T, then Ethereum is the Tesla. Along the way, the Ford T got an upgrade, and it wasn’t a radical upgrade. So let’s see how one of the largest computing engines ever built will handle this new upgrade.