Filecoin Features: Distributed Randomness and Leader Election

This is the latest in our series highlighting the unique features of the Filecoin network. Filecoin will be the first high-profile user of the drand network in production. This post dives deep into what drand is and how it works.

This month, a coalition of organizations dedicated to providing a publicly verifiable, unbiased source of randomness as a foundational Internet service for organizations will upgrade to a production-grade service that will be used as an off-chain randomness source for Filecoin. In the future, Drand can provide a reliable and unbiased source of randomness for many blockchain systems and applications. This post explores the history and value of drand, and what it means for the future of the Internet.

A random system produces outcomes that have a uniform probability distribution and are stochastically independent of each other. We encounter randomness every time we roll a dice and toss a coin: when you toss a fair coin, the probability of heads and tails is equal or uniform (uniform distribution), and you cannot predict the outcome of any single coin toss based on past outcomes (stochastic independence).

But randomness is used for more than just determining the statistics of coin tosses and dice rolls. In fact, we rely on randomness for everything from encryption and cybersecurity to election auditing and online gambling.

That’s because randomness can make certain digital processes resistant to manipulation. To take just one example, any cryptographically secure system is only as resistant to fraud and hacking as the cryptographic keys it uses are implausible. And the keys are only difficult to guess if the source values ​​used to generate them are suitably random. In other words, the foundation of secure encryption is randomness.

However, not all randomness is created equal. For example, when generating a random integer between 1 and 10, you should have a 1 in 10 chance of guessing it correctly. However, if your randomness generator exhibits biases or patterns, the chance of guessing the integer correctly could be more than 1 in 10. In other words, a key problem is using a "predictable" generator. Predictable randomness, again, opens the door to fraud and hacking. We need more than just randomness; we need good randomness. And good randomness is harder than you think.

The quality of randomness is measured by its unpredictability, a property of randomness called “entropy.” Since computers are deterministic systems, it is algorithmically impossible to create truly high-entropy randomness using computer algorithms. However, even physical randomness, when generated from a single source, can be easily biased. If you can control, manipulate, or otherwise influence the source of randomness, then you can bias apparently random results in your favor, as a recent high-profile lottery fraud case clearly demonstrated. For many use cases for randomness, such as election auditing and encryption, you also need the randomness to be publicly verifiable. A randomness generator is only as good as how it is communicated to the user. If you cannot verify that the phone numbers you receive are actually random, then your system can be attacked and manipulated.

Good randomness, that is, high entropy, bias-resistant, and publicly verifiable, is hard to find. For years, no service could generate this kind of randomness at scale. But a new project has emerged that is up to the task. It’s called drand (short for distributed randomness), and it’s what Filecoin uses as its source of randomness.

Drand is what is known as a randomness beacon, a service that provides publicly accessible randomness for use by third parties. While randomness beacons have been around for a while, drand is unique in that it generates randomness by combining contributions from a network of independently operated servers . This means that drand incorporates the strength, security, and resilience of a distributed network into the randomness generation process. In many ways, this makes drand an industry first for randomness-as-a-service.

Drand's open source software runs on a global alliance (the Entropy Alliance), which is comprised of the Protocol Organization, C4DT, ChainSafe, cLabs, Cloudflare, Emerald Onion, EPFL DEDIS, the Ethereum Foundation, IC3, Kudelski Security, PTisp, University of Chile, Tierion, and UCL. Because the drand network is distributed across multiple server nodes, there is no central point of failure, which means that drand cannot be biased and stopped by the compromise of a single insider or a single network partner. And, as an open source project, drand is directly supported by its users, ensuring that the protocol remains a vibrant, evolving project that adapts to the needs of its community.

The primary function of drand is to generate publicly available randomness that can be used by everyone. To perform this function reliably, all nodes in the drand network work together to generate randomness at regular intervals (called "rounds"). In each round of randomness generation, drand provides three different properties to the randomness: unpredictability, unbiasability, and verifiability.

This means that for each round of random number generation:

(1) They cannot be predicted by any party involved in the random number generation process;

(2) Uniformly randomized and unbiased;

(3) It can be verified as truly random by verifying a single signature.

Drand provides the most reliable, high-entropy public randomness to date. This is why we use drand-generated randomness in the Filecoin network.

Many cryptocurrencies and blockchain-based networks require good randomness to operate securely and efficiently. Filecoin is no exception. Filecoin relies on good randomness values ​​to maintain “liveliness,” or consensus among all clients on the correct history of the Filecoin blockchain. That’s because Filecoin uses randomness to perform leader elections to determine which miner will publish new blocks to the blockchain.

Leader election is how the Filecoin blockchain grows. Blocks in the Filecoin blockchain are arranged in epochs, and each epoch has multiple potential elected leaders. In a leader election, at least one miner is chosen from a group of miners weighted by the storage power these miners have contributed to the Filecoin network. The elected miner is chosen to start a new epoch by generating the next block on the blockchain and receiving the reward for that block.

Leader elections record the history of the Filecoin blockchain, so it is important that the process of electing leaders is tamper-proof, fair, unpredictable, and impartial. To ensure that this election is fair and just, Filecoin needs good randomness. At the same time, Filecoin needs the randomness it uses to be publicly verifiable, because elected leaders need some way to prove to other miners that they were actually elected.

Drand provides precisely the high-entropy, unbiased, and publicly verifiable randomness values ​​that Filecoin needs to guarantee the fairness and security of its leader elections. Its unique features ensure that Filecoin's leader elections are as fault-tolerant as drand and are immune to bias and adversarial manipulation. Using drand as a source of randomness, the Filecoin network can guarantee activity.

Drand is a rapidly evolving project with plenty of room to grow. As drand develops, we may see the network scale in new ways to meet growing industry needs and adapt to new technologies, such as advances in cryptography and quantum computing. But drand is already a cutting-edge service that can make important contributions to many digital projects, including the Filecoin network.

To learn more about drand, join us on Thursday, August 13th at the Randomness Summit, hosted by Protocol Labs and ETHGlobal. The Randomness Summit is a one-day virtual conference for distributed systems and critical infrastructure enthusiasts to discuss the state of research and deployment of randomness beacons. The full schedule can be viewed here.