Filecoin Features: Distributed Randomness & Leader Election

This is the latest in our series highlighting the unique features of the Filecoin network. Filecoin will be the first known user of the Drand network in production. This post dives deep into what drand is and how it works.

This month, a coalition of organizations dedicated to providing a publicly verifiable, unbiased source of randomness as a foundational internet service for organizations is upgrading to a production-grade service that will be used as an off-chain source of randomness for Filecoin. Drand can provide a reliable and unbiased source of randomness for many blockchain systems and applications in the future. This post explores the history and value of Drand, and what it means for the future of the internet.

A random system produces outcomes that have a uniform probability distribution and are stochastically independent of each other. We encounter randomness every time we roll a dice and toss a coin: when you toss a fair coin, the probability of heads and tails is the same or uniform (uniform distribution), and you cannot predict the outcome of any single coin toss based on past outcomes (stochastic independence).

But randomness can be used to determine more than just the statistics of coin tosses and dice rolls. In fact, we rely on randomness for everything from encryption and cybersecurity to election auditing and online gambling.

That’s because randomness is what makes certain digital processes resistant to manipulation. To take just one example, any cryptographically secure system is only as resistant to fraud and hacking as the cryptographic keys it uses are implausible. And the keys are only as hard to guess as the source values ​​used to generate them are suitably random. In other words, the foundation of secure encryption is randomness.

However, not all randomness is created equal. For example, when generating a random integer between 1 and 10, you should have a 1 in 10 chance of guessing it correctly. However, if your randomness generator exhibits biases or patterns, the chance of guessing the integer correctly may be greater than 1 in 10. In other words, a key problem is using a "predictable" generator. Predictable randomness once again opens the door to fraud and hacking. We need not only randomness, but good randomness. And good randomness is harder to come by than you think.

The quality of randomness is measured by its unpredictability, a property of randomness called "entropy." Since computers are deterministic systems, it is algorithmically impossible to create true high-entropy randomness using computer algorithms. However, even physical randomness, if generated by a single source, can easily be biased.

If you can control, manipulate, or otherwise influence the source of randomness, then you can bias apparently random results in your favor, as recent high-profile lottery fraud revealed clearly. For many use cases for randomness, such as election auditing and encryption, you also need the randomness to be publicly verifiable. Random number generators only have the means to communicate to users. If you cannot verify that the numbers you receive are actually random, then your system can be attacked and manipulated.

Good randomness, that is, high entropy, bias-resistant, and publicly verifiable, is hard to find. For years, no service could generate this kind of randomness at scale. But a new project has emerged that is up to the task. It’s called Drand (short for distributed randomness), and it’s what Filecoin uses as its source of randomness.

Drand is a so-called randomness beacon, a service that provides publicly accessible randomness for use by third parties. While randomness beacons have been around for some time, drand is unique in that it generates randomness by combining contributions from a network of independently operated servers. This means that drand incorporates the strength, security, and resilience of a distributed network into the randomness generation process. In many ways, this makes Drand an industry first for randomness-as-a-service.

Drand is open source software running on a global network and entropy alliance including Protocol Labs, C4DT, ChainSafe, cLabs, CloudFlare, EPFL DEDIS, IC3, Kudelski Security, PTisp, University of Chile, Tierion and UCL. Because the Drand network is distributed across multiple server nodes, there is no central point of failure, which means Drand is not susceptible to bias and cannot be stopped by a single insider or a single network partner's sabotage. And, as an open source project, Drand is directly supported by its users, ensuring that the protocol remains an evolving project that meets the needs of its community.

Drand’s primary function is to generate publicly available randomness that is available to everyone. To perform this function reliably, all nodes in the drand network work together to generate randomness at regular intervals (called “rounds”). In each round of randomness generation, drand provides three different properties to the randomness: unpredictability, unbiasability, and verifiability.

This means that each round randomly generates:

(1) They cannot be predicted by any party involved in the random generation process.

(2) It is uniformly random and unbiased.

(3) It can be verified as truly random by verifying a single signature.

Drand provides the most reliable, high-entropy public randomness to date. This is why we use drand-generated randomness in the Filecoin network.

Filecoin relies on good randomness values ​​to maintain “liveliness,” or keep all clients in consensus on the correct history of the Filecoin blockchain. That’s because Filecoin uses randomness to perform leader elections to determine which miner will publish new blocks to the blockchain.

Leader election is how the Filecoin blockchain grows. Blocks in the Filecoin blockchain are arranged in epochs, and each epoch has the potential to elect multiple leaders. In a leader election, at least one miner is chosen from a group of miners that are weighted based on the storage power these miners have contributed to the Filecoin network. The miner that is chosen starts a new epoch by generating the next block on the blockchain and receiving the reward for that block.

Leader elections record the history of the Filecoin blockchain, so it is important that the process of electing leaders is immutable, fair, unpredictable, and equitable. To ensure this election is fair and equitable, Filecoin needs good randomness. At the same time, Filecoin needs the randomness it uses to be publicly verifiable, because elected leaders need some way to prove to other miners that they were actually elected.

Drand provides precisely the high-entropy, unbiased, and publicly verifiable randomness values ​​that Filecoin needs for the fairness and security of its leader elections. Its unique features ensure that Filecoin's leader elections are as fault-tolerant as drand and are immune to bias and adversarial manipulation. Using drand as a source of randomness values, the Filecoin network can guarantee liveness.

Drand is a rapidly evolving project with plenty of room to grow. As drand develops, we may see the network scale in new ways to meet growing industry needs and adapt to new technologies, such as advances in cryptography and quantum computing. But drand is already a cutting-edge service that can make important contributions to many digital projects, including the Filecoin network.

To learn more about drand, join us on Thursday, August 13th at the Randomness Summit, hosted by Protocol Labs and ETHGlobal. The Randomness Summit is a one-day virtual conference for distributed systems and critical infrastructure enthusiasts to explore the state of research and deployment of randomness beacons.