Filecoin: Integration of IPFS and Filecoin

This year, Filecoin is mainly focused on two major events. One is FVM, which has attracted much attention, and the other is the construction of the search market. Protocol Labs has invested more engineers in the development and construction of the search market than FVM, because the search market is not only about the Filecoin network, but also needs to be combined with IPFS.

Since the launch of Filecoin, the community has been paying attention to the search market. The reason is simple, because only with the search market can the data stored in Filecoin have real use value and play the role of infrastructure in the new generation of the Internet. Of course, FVM itself is also for this purpose. The implementation of FVM allows Web3 applications and other Web3 infrastructure to interact with Filecoin and use Filecoin to store and process data. Of course, the search market truly makes this data accessible and reflects commercial value.

The construction of the search market must first consider several issues:

1. How to implement fast retrieval service?

2. How to utilize the resources of the existing IPFS network?

3. How to motivate search service providers?

These are some very difficult problems. After more than a year of exploration, we now have preliminary solutions and the network is being developed and built.

Efficiency and trust issues

Everyone is very familiar with Filecoin storage. Its innovative proof of replication and proof of space-time solve the trust problem of storage in the decentralized world. But at the same time, since zero-knowledge proof is needed to establish trust, its storage speed will naturally not be high. This is not a big problem for storage, but it is fatal for retrieval. The goal of the Filecoin retrieval market is to provide services that are no less than Web2 retrieval performance .

It is extremely difficult to realize the retrieval market solely relying on the Filecoin chain. The first problem is the proof of retrieval, which is to prove that a service provider provides real retrieval services. This is easier to achieve in a network without incentives, as long as both parties trade and pay the network fee. But for a blockchain network, if incentives are added, there needs to be a problem of preventing attacks. The main attacks here are similar to storage, including witch attacks and outsourcing attacks.

When there is no breakthrough in cryptographic solutions in these areas, a feasible way is to establish trust through governance through solutions similar to the current Filecoin+. This can be done through the notary governance model, and another common way is to govern through the DAO model.

Even if governance can be achieved through DAO and notaries, how can the efficiency of retrieval be achieved? An obvious conclusion is that it is currently impossible for Filecoin storage service providers to unseal data and provide retrieval through Unseal. In fact, Filecoin took this into consideration when it was first designed, so there is a Fast Retrieval mode, which means that Filecoin storage service providers can retain a copy of the original data, and when providing retrieval services, they do not need to use Unseal to seal the data, but can directly provide the original data.

Then another question arises, in what form can the original data be stored and provided with fast service? There is no unified solution at present. For Protocol Labs, a natural idea is to use IPFS. There are many advantages: 1) IPFS is a natural CDN network; 2) The IPFS network has a wide coverage. Filecoin has only a few thousand storage service nodes, while IPFS has hundreds of thousands of nodes, which will greatly expand the scope of Filecoin's influence; 3) IPFS has a fast retrieval speed, and using Filecoin as a transaction token can promote the use of Filecoin on the one hand, and promote IPFS retrieval service providers to improve service quality on the other.

Integration of IPFS and Filecoin

Filecoin's retrieval market will be a market provided by the Filecoin+IPFS network. Once this market is built, the scope of Filecoin's nodes (including storage nodes and retrieval nodes) will be expanded hundreds of times. At the same time, the use of Filecoin as a token will be greatly expanded. The 15% of tokens reserved by Filecoin may also be used as an incentive to promote the construction of this market at the appropriate time.

The first thing to consider when integrating IPFS and Filecoin is the trust issue. How to obtain the data stored in the Filecoin network from IPFS and ensure that it is correct? This is not difficult because IPFS is a content-addressed network. After the content is obtained, it can be directly verified and consistent with the transaction data promised by the storage service provider on Filecoin. There is no problem. Of course, IPFS retrieval nodes can also retrieve data from Filecoin storage nodes and cache data to serve other customers. In this way, retrieval nodes and storage nodes are separated. In the Filecoin+IPFS retrieval network, we will see many more retrieval nodes than storage nodes, so that higher quality and faster retrieval services can be provided.

The integration of IPFS+Filecoin still needs to solve some technical problems.

First, how do users know where to search? The current method is to provide some nodes similar to DNS in the network, that is, content-addressable nodes. These nodes are called Indexer nodes. Users can first visit these nodes to obtain which service providers can provide the content they need. Indexer will provide a group of high-efficiency nodes close to the user according to the user's request, and then the user can directly access those search nodes to provide services. This is similar to the process of accessing websites in the current Web2 network.

Secondly, how do users pay? There is a compatibility issue here, that is, in this retrieval market, we hope that the user experience is consistent. Whether the user obtains data from the Filecoin network or the IPFS network, there is no difference in the experience, the interface is consistent, and the payment method is consistent, that is, FIL is used to pay. However, we know that IPFS is free, so we need to add this function to the IPFS node that can be paid for by FIL. This also involves the fact that the data transmission method of IPFS is different from that of Filecoin, one is Bitswap and the other is GraphSync. This requires some modifications.

Finally, how to motivate? Of course, if we just build a market without incentives, it is relatively simple, but in this case, the initial construction of the network depends entirely on the Protocol Labs itself. This is obviously backward and inappropriate in the blockchain world. The storage market similar to Filecoin has become the fastest growing storage network through token incentives. If there is a suitable incentive method, the retrieval market of Filecoin will grow faster than the storage network. Incentives must of course be consistent with network contributions, and cannot be defrauded or attacked, and the cost of implementation cannot be too high. This is very difficult. The research team currently has some solutions for this and hopes to make breakthroughs in the next few months and a year. Of course, a non-completely decentralized solution can also be adopted, which will be less difficult. If some compromises are made, the difficulty and cost will be reduced, which is also an option for the establishment of an ecosystem.

Storage Market Serving Web3 + Web2

The goal of Protocol Labs is to combine IPFS and Filecoin, and use IPLD, LibP2P and other projects as an overall solution to build a distributed storage system as the infrastructure of Web3. The development of Web3 is of course inseparable from storage. Currently, many projects are using Filecoin or IPFS as their storage base. When the Filecoin+IPFS storage market is established, the business model of the entire market will be standardized. Any application and public chain can freely use this storage network on this basis.

From this perspective, the decentralized storage built by Filecoin+IPFS belongs to Layer 0 (the most basic part) in the entire Web3 ecosystem, which is used to support other Layer 1 chains, including Ethereum, Solana, Cardano, Near, Flow, AVAX, etc. Layer 0 is indispensable for all chains and applications that require content storage. This can also allow Web3 to expand from DeFi to other wider applications, such as Game, Social, Video, metaVerse, etc. It also truly makes it possible for the Internet to move from Web2 to Web3.

The storage and retrieval market of Filecoin+IPFS, with the help of the wide distribution and CDN features of IPFS, actually builds a CDN (content acceleration network) while building storage and retrieval. The goal of the Filecoin retrieval network is to provide a retrieval speed that is more efficient than the current Web2.0. In this way, it can not only serve Web3, but also Web2, and provide a path for Web2 applications to migrate to Web3.