【Filecoin Official】What makes Filecoin different?

Original author: Filecoin Community

This article is compiled by the IPFS China Community

Original link:

https://filecoin.io/blog/filecoin-proof-system/
Like other large technological innovations, blockchain is a combination of several mature technologies that we have used and trusted for decades. “Consensus mechanisms” have been studied since the 1970s and developed in the 1990s as a tool to fight spam, which allows users in a distributed system to reach an agreement without a central arbitrator.

Filecoin is built on a variation of proof of space. It is also related to proof of stake, because the stake relationship is not just the token as a stake relationship, but exists in the amount of proven storage, which determines the probability of miners producing blocks. When building a distributed storage network, we set out to build a proof structure. In this structure, consensus is achieved by generating data storage. With the release of testnet, we will launch a new set of storage-based proof systems to achieve decentralized consensus.

When we announced Filecoin in 2017, we set out to create a decentralized storage network built on a strong decentralized market. To foster this market, decentralize market functions, and encourage early miner participation, we created a cryptographic token as a byproduct of Filecoin consensus. This token is generated on the basis of useful work (i.e., useful proof of replication and proof of spacetime).

These stories of proof

Juan Benet recently explored the history of Filecoin’s proof structure in an interview on the Zero Knowledge podcast. Here are excerpts from the interview:

“Filecoin advances blockchain in many different ways.

Proof of Replication is a proof system used to verify that storage miners actually own what they are storing and are not cheating . But how do you prove to the network that you are actually storing something and not lying?

Filecoin also attempts to solve some other interesting problems, including higher throughput consistency and interoperable, content-addressable linked data structures.

At the end of the day, it’s all about taking advantage of all the unused storage space on the planet, organizing it with incentives, building the most powerful computational storage network, and driving down the price of that storage .

Filecoin's proof of replication is both a proof of storage and a proof of space, which are slightly different. In Filecoin, data units are stored in so-called sectors. You seal specific data in a sector on disk in a slow encoding process and submit the verification to the blockchain. Sealing is a very labor-intensive task. To forge such a proof, you must use the original data stored by the client on Filecoin to complete a specific task.

A proof system is a cryptographic protocol where there is a prover and a verifier - the prover will prove something to the verifier . For example, in POW, the prover needs to have done some computation, or spent some computation cycles. Their typical proof is a hash.

Proofs of Storage is a simple proof system that proves that I own certain data. For example: I can prove to you that I have data X, without showing data X, or that the data is several GB in size, but in a more concise way.

It’s Proof of Retrievability. Not only do I have to prove that I have X, but also that these proofs can be used to recover X in case I have malicious intent and want to hide X.

Proofs of Space are another type - I can guarantee you that I spent a certain amount of storage space . If I stored 1gb, and then generated a random GB, then I can prove that I stored this random GB and not something else. This allows miners to use storage space as proof of work.

The fun part is combining proof of space with normal proofs of data possession — I hope X is useful and not just a random string. The hard part is creating a proof of space that is also used to store useful data. This is what proof of replication is in the cryptographic protocol of the Filecoin network — as a fundamental primitive.

Other proofs of storage were created to create more trustworthy clouds because they can prove to you that they are backing up your data. But they are completely unused in a normal centralized cloud environment because trust is contractual. Now they are used in the whole decentralized space because this is where we use incentive structures to guarantee things instead of contractual agreements.

We also use SNARKs to prove some actual proofs of replication that generate a lot of output. We want to do a lot of challenges on these proofs of replication, aggregate them so that they can be chained together in a very small, compact way. SNARKs are a great way to do that, it gives you a way to prove your correctness, and then you can put this SNARK proof on the chain. Then, parties can now verify some very small inputs themselves, and the actual SNARK proof, and know that the proof has been generated correctly.

In the proof of replication process, we take, say, 32GB, and apply a very slow encoding that produces a grid-like graph in a layer where a node might be a 32-byte segment. Generating a graph requires a sequential process and sequentially hashing each node. Because of the hash function, it has to be done one after another.

One type of generated graph is a DRG (depth-radius-graph) which is connected to these extended graphs to form a complex lattice structure. Finally, we encode the original data into what we call a replica and commit it as a value. You can take the same source data and encode it multiple times and you will end up with multiple different, uniquely encoded replicas.

Now that we've done that, to prove that we've encoded it correctly, we can just sample a few challenges to prove that we've stored this. Let's say we randomly sample 1,000 challenges throughout the proof, and then we do the computation in the SNARK. We take the source encoded data, and we decode it, and then show that it goes all the way back to the root that we committed to. That's what we want in a succinct proof. Otherwise it's a 32-byte "leaf," and the entire Merkle chain all the way back to the root is going to be a pretty big amount of data, and then multiply that by 1,000. If it takes 100 KB or MB to generate a proof, we can compress it with a SNARK, I'd say it's around 200B.

A great story about all of this work is what we call the proof roller coaster. Over time, you end up creating a ton of different structures with all these different parameters serving all these different use cases.

This choice of parameters, the choice that was demonstrated in Filecoin is the biggest reason why it took us so long to release all of this stuff. Because you choose a structure that has a certain shape, produces artifacts of a certain size, and maybe that’s good, and then you tweak some parameters like, “Hey, maybe we want sectors to be a little bit bigger.” That makes some other parameters have to change.

Very quickly you get into a large parameter space with a lot of different variables, and once you adjust one thing here, there are a lot of other things that have to change as well. Because a lot of the algorithms are optimizing, doing complexity management, it's very difficult. Because a lot of these structures, slow encodings, you want to be slow enough to prove it, but fast enough that it's not expensive . It's a very difficult challenge to dial it in to get it just right, and then nail the special SNARK structures to make sure you can do this efficiently and concisely.

All of these parameter optimizations can be so intense and difficult that we actually had to write software to handle them . We have a constraint solver that handles the constrained optimization problem of choosing the proof structure and parameters in Filecoin. This was an unexpected result that other groups can now use to make their lives easier, but we had to write this.

We use a tool called Orient, which is on Github and everything is open source (see Filecoin's parameters in Orient and Ubercalc). It has a special language where you can define specific algorithms and the artifacts they generate, and then combine them into larger algorithms with all these variables and parameters. Then you can do experimental results, like how long a certain hash function takes, plug data into some parameters and calculate some other parameters. For example, based on this hash function and how long it takes inside or outside the SNARK, this is a special construction that you want to use because it can minimize the time or minimize the on-chain footprint, and all of these things are calculated by this solver.

Making blockchain technology, because its structure is so complex—both the individual primitives and how they are woven into the chain—so we need this software to help us write software. Just like chip manufacturing, chip manufacturing was going well until it reached a certain density, and then they couldn't just produce chips by hand. They had to start using software to lay out the chips. I think we've reached that point in blockchain, and some of the structures we're doing, we need software to help us design them.

I think there is no other network using Proof of Replication, and that is an advantage we have, we created this space. It is a differentiator .

We are also the only company that has this liquid market structure , which is meant to optimize around a ask and bid structure where miners and clients are able to reason about price together and then make deals based on that.

I think we are also the only one that has consensus backed by valid storage . For other networks, it might be a consensus backed by proof of space, but in our case, it's useful. Those are the three biggest differentiators of Filecoin.

And then there’s the tight integration with IPFS through libp2p, and some other things that are already heavily used on IPFS. It’s going to be easy to back up all of this data directly into Filecoin. It’s also worth mentioning that IPFS is an open network, and we’re already seeing other networks starting to add support for it, which is also cool. It’s meant to be a separation layer for that reason.”