Three factors that restrict the development of Filecoin: user experience, information security, and storage technology solutions

Original author: Frank Li, Vice President of Investment at ParallelVC

What is this Web 3.0 that everyone is talking about?

The development of the Internet has gone through two stages: Web 1.0 and Web 2.0. Most websites in the Web 1.0 stage are static websites. There is no interaction between network users and network information. Users can only obtain information displayed on the network. The effectiveness and efficiency of information acquisition are relatively low.

As the speed of the Internet increases and the bandwidth increases, people gradually begin to interact with the Internet. In 2003, Dale Doherty, vice president of O'Reilly Media, proposed the concept of Web 2.0.

Web2.0 is called a read-write network. In the early days of Web2.0, compared to Web1.0, all Internet users could create their own content and upload it to the Internet instead of just getting information from the Internet, which greatly increased the richness of Internet information. With the further development of technologies such as AI and big data in recent years, human-computer interaction has been elevated to a new stage. The online behavior data such as browsing information, clicks, and searches generated by users on the Internet are captured and recorded. The technical background can analyze a more accurate user portrait through the user's real-time data combined with his past information, and recommend corresponding products or information to the user based on the user portrait. In this way, the purchase conversion efficiency of the merchant is improved, and the user body can find the goods they may want to buy more quickly, making the user experience better.

However, while information centralization is convenient, it also has a major drawback, which is that all user data is collected and used by the platform without the user's knowledge, and even the ownership of the data is unclear.

In the early days of Web1.0 and Web2.0, because the amount of user data was small and the data dimensions were relatively few, the user's personal data could not generate much value. However, as people's use of the Internet has increased in recent years, the value of online personal information has become important. In the past two years, there have been cases of Internet companies infringing on personal privacy data and user data being stolen in various countries. In the future, with the development of artificial intelligence Internet of Things (AIoT) and 5G networks, the dimensions of personal network data will be more comprehensive and more valuable, making data security and data privacy even more important.

The emergence of Web3.0 is to solve the problems currently faced by Web2.0. The blockchain network has the functions and properties of de-trust, immutability and confirmation of ownership, which well meets the needs of Web3.0 as the underlying technology. At the same time, due to the changes in network architecture, data is no longer a simple number but a commodity with valuable attributes, which also makes our existing data network gradually transform into a value network.

Web3.0 needs decentralized storage

As mentioned above, if you want to build a decentralized network to ensure the security and privacy of data, you need to ensure that the data can only be controlled by its owner (data generator), which includes the right to obtain or authorize others to use it. Unfortunately, the current storage solutions on the Internet are centralized storage, and the main storage providers are central institutions such as Amazon, Alibaba Cloud, and Google Cloud. At the same time, the personal data of users generated online are now owned by various platforms and APPs. Even if the ownership of personal data can be returned to the user in the future, in the case of centralized storage, users cannot ensure that their data will not be used or modified by the APP platform or even the storage provider.

In addition, 5G and AIoT cutting-edge technologies are developing rapidly. The dimension of personal data in the future network will further increase, and the value of data will also grow rapidly. Only by using decentralized storage can the security and privacy of data be further guaranteed.

Without decentralized storage technology as the underlying technical support for decentralized networks, even if decentralized information transmission and decentralized computing are achieved, the decentralization of data cannot be truly guaranteed. Therefore, decentralized storage will be an essential technical component of the future Web3.0 ecosystem.

Current status of traditional cloud storage

At present, the traditional cloud storage field can be divided into three forms: public cloud, private cloud and hybrid cloud. Public cloud is the most common cloud service form. Public cloud is owned and managed by technology providers and provides services to multiple customers. At the same time, public cloud can be divided into three technical forms: SaaS, PaaS and IaaS.

SaaS is the abbreviation of "Software as a service". This type of service provides applications to users as services. Its main customers are individual home users and small and medium-sized enterprise users. The main service providers in China are Baidu Netdisk and Tencent WeCloud, while the main service providers abroad are Dropbox, iCloud, etc.

PaaS is the abbreviation of "Platform as a service". This type of service provides the development platform as a service to users, and its main customers are small and medium-sized enterprises and individual developers. The leading PaaS platforms in China include Baidu Cloud, while the main PaaS platforms abroad are Google App Engine and Openshift developed by Red Hat.

IaaS is the abbreviation of "Infrastructure as a service", which mainly provides cloud computing resources such as virtual machines and cloud storage components as a service network to users. The leading domestic companies include Alibaba Cloud and Huawei Cloud, while Amazon EC2 is the main provider of such services abroad.

Unlike public clouds, private clouds and hybrid clouds mostly serve large enterprise users. Private clouds are used and maintained by a single enterprise or organization, and users have more control over personalization, while hybrid clouds are a solution that mixes and matches public and private clouds to achieve a relatively high cost-effective solution.

First, we can analyze the data on domestic individual users’ use of cloud storage: According to Aurora’s 2019 monthly MAU share data, the top four are Baidu Netdisk, Tencent WeCloud, Caiyun, and Tianyi Cloud Disk.

Among them, Baidu Netdisk's active user share reached 82.9%. Continuing to observe Baidu Netdisk's user interest TGI, it can be concluded that the data users mainly store is content related to movies, animation and sports.

At the company level, data shows that 39% of enterprises are currently using cloud storage-related services (such as AWS, Azure, Google Cloud, etc.). According to forecasts, by 2022, the number of enterprises using cloud storage-related services will reach about 60%. Among the three major cloud storage companies in the United States, Microsoft Azure has a market share of 44%, AWS has a market share of about 32%, and Google Cloud has a market share of only 19%. In the Spiceworks 2019 Public Cloud Report, Azure was also rated as the most competitive cloud service provider. At the same time, through the survey, it was learned that 79% of business owners mainly use Windows systems in their businesses, and Microsoft's Azure cloud storage service has a better product experience when paired with Windows systems.

The current cloud storage service architecture is relatively mature, and users can choose to use public cloud, private cloud or hybrid cloud services according to their needs. In public cloud services, users can choose different storage methods based on the frequency of data calls, saving costs to the greatest extent without affecting usage.

Compared with cloud storage, nearly 81% of enterprises are still using traditional storage matrices, that is, hardware storage. Dell EMC currently has the highest market share of 42%. According to the survey, 80% of enterprises choose traditional storage matrices because of their higher reliability. They also believe that compared with cloud storage, traditional hardware storage has higher privacy and security.

From a technical perspective, traditional cloud storage has gone through three stages of iteration. The first stage is a storage architecture solution based on NAS (Network Attached storage) and SAN (Storage Area Network). This architecture initially formed a cloud storage solution, but it was difficult to flexibly call different storage clusters when the server was located at a high geographical location, so there were major bottlenecks in capacity and performance expansion.

The beginning of the second phase was marked by the emergence of the EMC VPLEX architecture. VPLEX implemented virtual storage and heterogeneous storage functions based on the original storage architecture, freeing storage hardware from geographical restrictions and solving cross-cluster operation problems. Through intelligent distributed caching, cross-host, cross-cluster, and cross-data center access and data sharing are achieved, greatly improving cloud storage in terms of capacity and scalability. The services it provides are similar to those provided by current IAAS.

The third stage is the current stage of cloud storage. The current underlying architecture of cloud storage is a cloud-like structure. Multiple storage devices are interconnected, and hardware devices are intelligently switched according to different needs. At the same time, multiple copies of consistency, disaster recovery, elastic expansion and other aspects have been greatly improved on the original basis. The current cloud storage SaaS and PaaS platforms can provide users with simple and easy-to-use operation and storage services.

The development of decentralized storage

In the current cloud storage infrastructure, distributed architecture has become the standard because of its advantages such as high performance, multi-distribution consistency, and support for hierarchical storage. Decentralized storage can actually be classified as a type of distributed storage.

Compared with other distributed storage solutions, the data center in the decentralized storage architecture will not be completely owned by the cloud provider, but will be provided by multiple nodes in the network, and the data will be stored in an encrypted manner. In this way, the privacy and security issues of the data are better solved, and even the data center or storage hardware provider cannot obtain the stored data.

Currently, the most representative projects in decentralized storage are IPFS and Filecoin. IPFS stands for "Inter Planetary File System" and its Chinese name is "Star File System". IPFS is an underlying network transmission protocol equivalent to the HTTP (Hypertext) protocol in the current Internet world. HTTP is a relatively simple request-response protocol used for interaction between users and servers.

The function of IPFS is similar to HTTP, but it incorporates the architectural features of the p2p network. Compared with the HTTP protocol, the IPFS protocol is more efficient. HTTP is a single-threaded communication, and can only perform one task on a server at a time, while IPFS uses p2p for multi-threaded downloads, which can save more than 50% of bandwidth costs. At the same time, due to the centralized nature of current Internet servers, information in the network can be completely controlled and difficult to save, but if decentralized protocols such as IPFS are used, as long as the information is owned by any user in the network, the entire network can obtain this information.

In the decentralized storage ecosystem, IPFS mainly plays the role of the underlying technical protocol, and other specific business solutions will be implemented by the layer2 project. Currently, the most likely first implementation based on the IPFS protocol is the incentive layer Filecoin developed by the IPFS team itself. This project has already launched a test network and will launch the main network as early as this year.

In addition to IPFS, there are many projects trying to implement decentralized storage, but there has been no actual progress, so I will not introduce them in detail.

Understanding Filecoin

Filecoin is a decentralized storage project based on the IPFS protocol. It provides users with decentralized storage services by building a bridge between users and storage service providers through the protocol.

Participants in the Filecoin network can be divided into three parts: storage miners, retrieval miners, and users. Storage miners provide storage space for the decentralized storage network and pledge a portion of tokens in the network to prevent miners from doing evil. Storage miners need to prove to the network that they have stored the user's target data within a specific period of time. If they fail to do so, a certain percentage of the pledged tokens will be deducted as a penalty.

Retrieval miners are mainly responsible for providing data retrieval services to users in the network. When users initiate a demand, retrieval miners need to find the corresponding information in the network and send it to users. At the same time, the network has no requirements for the degree of participation of miners. Miners can be both storage miners and retrieval miners at the same time, or they can only participate in one of them.

Users also have greater flexibility when using Filecoin. On the one hand, users can use the Filecoin network as a network disk. Users only need to indicate the data they want to store and the price, and the system will match the user's price with the miner's price. When the price is unified, the system will send the user's data to the storage miner.

Storage miners put the data into different sectors (storage units) according to the size of the data, then encapsulate and slice the sectors, and the transaction information will be stored in the hash table in the network. During the order process, the node will continue to challenge the miner to check whether the miner has saved the file. This completes the entire storage process.

In terms of data privacy, when creating a storage subscription, users can choose whether the data they store is only for personal viewing or open to the entire network. If it is open to the entire network, everyone in the network can query this data through the search miner. Users store data in several different nodes.

In terms of consensus algorithms, Filecoin uses three different consensus algorithms: Expected Consensus, Proof-of-replication, and Proof-of-Spacetime.

Expected Consensus is a consensus algorithm used in Filecoin to specify block rules. It is a probability-based Byzantine fault-tolerant algorithm. According to Expected Consensus, a leader will be elected to generate blocks when each new block is generated. Similar to BTC mining, the probability of being selected is proportional to the proportion of computing power to the total network computing power, but in the Filecoin network, it is calculated based on the effective storage space provided by the miner rather than the computing power in the BTC network. Before each block is generated, the network will generate a newticket based on the ticket value of the previous block prove through the linear process of VRF and VDF, and then confirm whether it is qualified to generate blocks by comparing the value of the newticket and the ratio of the effective computing power to the total computing power of the entire network. If the newticket is less than its effective computing power/total network computing power, this miner is elected as the block miner.

Unlike the BTC network, Filecoin's election method may result in no one producing a block or multiple people producing blocks as leaders. When no one produces a block, the network displays an empty block at that height. Because it is possible for multiple people to produce blocks at the same time, the Filecoin team designed tipset, which is a collection of blocks. Like the BTC network, due to network latency issues, two tipsets may exist at the same time, but in the end the network will elect a unique tipset based on weight and develop a chain mechanism downward. The calculation formula for the weight of each block is as follows:

Weight = ParentWeight + ECV + ECPrM * ratio

Currently, ECV is set to 10, ECPrM is set to 100, and Ratio is the storage efficiency of the current node (node ​​storage capacity/storage capacity of all nodes). In other words, under the current algorithm, the higher the node ratio, the higher the weight. When the weight is the same, the tipset with a smaller ticket value will be selected. Because of this technical design, the underlying architecture of filecoin is more inclined to the DAG structure.

The Proof-of-Replication consensus algorithm is used to prove that miners have indeed completed the specified work. In the Filecoin network, the main job of miners is to store user data. Miners need to seal the user's data in a sector. During the sealing process, the system will generate a DRG (Depth-Robust-Graph), also known as a replica value. Only the storage space corresponding to the generated replica value can be recognized as valid storage space.

The Prove-of-spacement consensus is responsible for the final verification step. When generating each new block, the network will randomly challenge the miners, and the miners need to provide replica values ​​for verification, which is verified through zero-knowledge proof. If the result is correct, it means that the miner has stored the encapsulated data normally, and its effective storage share remains unchanged. If there is a replica value that cannot be verified correctly, the storage space corresponding to this replica value will be removed from the effective storage space.

What is the development prospect of Filecoin?

At present, Filecoin's technical solutions still have a large gap compared with traditional mature cloud storage solutions. The following will briefly analyze Filecoin's current problems.

1. Information security issues

Compared with traditional centralized storage, the biggest advantage of decentralized storage is that it can better protect the privacy of data security. However, the current design of Filecoin has caused people to question the security and privacy of data.

Let's take a closer look at the user storage process: after the order is successfully matched, the user will send the original data directly to the miner, and the user's data is not encrypted or split. Miners can directly access the complete user data, and it cannot be detected by the network or the user. According to this technical solution, the privacy and security of user personal data in Filecoin will be greatly threatened, and its security and privacy are even weaker than traditional cloud storage solutions. At the same time, there are loopholes that cause direct data leakage in the early stage of the storage process, so the encryption, splitting and other methods of ensuring data privacy in the subsequent process become meaningless.

On the other hand, because all the data of a user is stored in a single hard disk of a miner, if the miner gives up mining for some reason, the user's data stored there will be completely lost from the network, and there is no mechanism set up in the network for users to retrieve their data before the miner is out. Such unpredictable events also greatly reduce the security and reliability of data stored in the Filecoin network.

At the same time, according to the information mentioned in the previous AMA of this project, Filecoin is just a protocol to coordinate the needs of storage providers and users, and cannot require storage providers to perform specific operations. Although this reduces the risk of decentralization and the occurrence of platform manipulation of user data, it reduces the cost of storage providers to do evil. Even if the storage applicable party requires the storage provider to delete or destroy the data it stores, the storage provider can deceive the user to privately copy the user data or even forcibly not execute the user's request. At present, the technical equipment or service providers of traditional cloud storage are all large companies. After the problem occurs, the responsible party can be immediately identified and the problem can be solved. However, miners in decentralized networks such as Filecoin are relatively scattered and unknown, which also creates a great obstacle to the subsequent solution after the problem occurs, making its comprehensive security weaker than centralized storage.

2. User experience issues

According to the overall description in the white paper, the user experience of Filecoin should be poor. First of all, in addition to the simplest storage function, the Filecoin project does not provide any additional technical services to users, such as disaster recovery solutions. Users can only deal with the problem of data inaccessibility or even data loss caused by storage unit disconnection, damage, etc. by storing their files in several different nodes. For users who do not understand technology and disaster recovery, it may cause permanent loss of user data.

In addition, due to the information security issues mentioned above, if users want to further ensure the security of their data, they need to encrypt the files before storing them. This requires higher operation from the user, making the user experience worse.

3. Incomplete storage technology solutions

At present, Filecoin's technical solution is not yet complete, which is an important reason that affects users' use of its storage network, such as the disaster recovery problem mentioned above. According to Filecoin's current design, the data stored by users each time will be stored independently by a miner, and other miners will not actively store backups of this file. The requirements for disaster recovery technology are greater than those for traditional cloud storage, because traditional cloud storage is mainly due to technical reasons of the equipment itself, but Filecoin needs to prevent storage miners from stopping providing services due to various major reasons while preventing technical failures of the equipment itself. The factors that cause this kind of situation will be far more than those that cause simple technical failures, and the solutions will be much more complicated.

Currently, Filecoin's solution to the disaster recovery problem is to allow users to store data in multiple miners for active backup. However, in the early days of the network, due to the instability of the miners themselves, it cannot be ruled out that multiple backup data may be lost or even all backups may be lost. This will be a huge blow to users and network operations.

At the same time, the redundancy problem of the storage network cannot be solved because of the simple active backup. In traditional storage, because the storage is centralized, the system can analyze, process and optimize all the data in the background, remove duplicate data and improve the efficiency of the storage network. Although such processing cannot be performed on non-public data stored by users due to the decentralized nature, for the public data stored by users, how to combine the characteristics of the IPFS network to optimize the data storage structure is a direction that the team needs to study carefully.

The three problems mentioned above restrict the future development of Filecoin at different levels. From the analysis of the traditional storage market at the beginning of the article, it can be seen that the current storage market is basically divided into two categories: the first category is mainly for personal and small business users, mainly in the form of SaaS platform technology. The user experience and product diversity of the SaaS platform have become relatively mature after years of development. At the same time, cloud disk applications such as Baidu and Google Drive have their own characteristics and advantages in file transfer, file search or collaborative office. Filecoin itself has no advantage in its competition. If there are still loopholes in data security, then filecoin will not survive in this competition.

Even if the data security problem is solved, from the perspective of real storage demand, the current development of Filecoin is not easy to be optimistic. From the storage data of Chinese online disk users provided at the beginning of the article, it can be seen that the top three TGIs of cloud disk users are movies, sports and animation. And these types of video or picture data focus more on dissemination rather than privacy. Compared with decentralized storage, traditional online disks that can be shared or queried by users will have more advantages.

The data that really needs decentralized storage can be roughly divided into two categories: First, data such as personal photos and private files have a greater demand for decentralized storage, but such data is relatively small in the current stock market. Second, personal data generated by future personal IoT devices, but because leading IoT device companies such as Xiaomi now regard user data as an important asset and can generate huge value through it, leading companies will not return user data to customers without specific regulations. If leading companies are unwilling to return personal data and decentralized storage is not applicable, then even if individual users are willing to use it, it is meaningless because it does not achieve its fundamental purpose.

The second type of users are large and medium-sized enterprises. Large and medium-sized enterprises use cloud storage mainly to store company-related data, so they have high requirements for the reliability, security and privacy of storage solutions. At present, traditional solutions include public cloud, private cloud, hybrid cloud and traditional storage matrix. The combination of private cloud and traditional storage hardware solves the concerns of corporate customers about the data privacy of public cloud, and can also meet customers' needs for the reliability and security of storage solutions. At present, Filecoin lags behind traditional cloud storage solutions in all three aspects, so it cannot compete with them. Compared with traditional storage matrix, the benefits brought by decentralized storage are not what traditional storage matrix users pursue, so it is difficult to convert among these users. For decentralized storage, in the future, it will be possible to start developing ToB related businesses only when data reliability, security and privacy can be guaranteed and the price can be lower than the current traditional cloud storage solutions.

Apart from technical factors, there are also many uncertainties in the Filecoin project.

First of all, it is a team factor. The two mainnet launch promises in 2018 were not fulfilled. Mining was changed from CPU mining to GPU mining, and the mining rules were changed during the testnet stage. Although the subsequent changes were to prevent miners from maliciously brushing garbage data, such frequent changes reduced the trust of users and network participants.

In addition, to date, the mechanism and distribution rules of Filecoin mining rewards have not been determined, which further increases the uncertainty of the project.

Finally, Filecoin’s mining logic is different from other POW mining types. The project requires long-term participation of miners and they cannot withdraw in the middle, because the withdrawal of miners may cause the data to be lost forever. However, the specific details of Filecoin mining are too complicated, and the revenue situation is difficult to estimate, which also increases the uncontrollable factors. Whether there will be enough miners to participate and stay smoothly is a problem worthy of attention. At the same time, the impact of the departure of miners in the middle of the network can only be judged after the main network is launched.

In terms of price, because Filecoin has the various technical and product issues mentioned above, it needs a price that is extremely attractive compared to traditional cloud storage to reduce users' usage costs.

From the above analysis, we can know that at the current stage, Filecoin's main competitors will be SaaS applications in the traditional cloud storage system. So the following will sort out the mainstream toB and toC SaaS service charging plans in the market for readers' reference:

Dropbox:

The personal version provides 2G of storage space for free, the Plus version provides 2T of storage space and also offers text search, mobile offline storage and other services for RMB 78 per month, and the professional version provides 3T of storage space, as well as AutoOCR, file collaborative editing, file locking and other services based on PLUS, for RMB 130 per month.

The toB version, plus, is 81 yuan per month, providing 5TB of storage space and 2GB of file transfer capacity, with a minimum purchase of 3 people. The Professional version is 130 yuan per month, providing wireless data space and 300GB of file transfer capacity, also with a minimum purchase of 3 people.

Box:

The personal version provides 10G storage space for free, and the upload size limit of a single file is 250M. The paid versions are divided into 5 USD/month, 15 USD/month, 25 USD/month, and 35 USD/month. The 5 USD plan provides 100G storage space, and the upload size limit of a single file is 2GB; the 15 USD plan provides wireless storage space, and the upload size limit of a single file is 5GB. At the same time, the box can be connected to an app, and the app data can be directly stored in the box. The 25 USD plan allows the connection of 3 apps, and the 35 USD plan allows the connection of unlimited apps.

Baidu Cloud Disk:

The payment plan of Baidu Personal Cloud Disk is shown in the figure above. The monthly fee for super members is 18 yuan, and the monthly fee for ordinary members is 8 yuan. Non-members have 15G of storage space before completing the task, and will get 2T of storage space after completing a series of sharing tasks.

The above picture shows the introduction and price of Baidu Enterprise Cloud Disk.

Compared with the three centralized storage providers listed above, Filecoin still has a hard time competing with them in terms of enterprise-level services. In terms of personal services, Filecoin can only compete with them by customizing more flexible payment plans and lower prices. At the same time, Filecoin and other distributed storage projects also need to try their best to enrich their additional functions to improve their competitiveness.

Prospects for the development of decentralized storage

Compared with the centralized cloud storage industry, which is relatively mature in terms of technical architecture and operation mode, the concept of decentralized storage has only been proposed in the past two years. There is still a lot of room for improvement in both technical design and operation mode. The development of decentralized storage should fully learn from the currently mature cloud storage solutions and improve them according to their own characteristics.

It is too hasty to completely deny all previous centralized storage technical solutions, and it is difficult to gain market recognition to try to directly use new imperfect solutions to replace the original solutions. Only by finding a good balance between innovation and tradition can we create products that meet market demand.

At present, re-establishing the underlying network protocol like IPFS is a very good attempt, but its usability is relatively poor and it is difficult to meet the fundamental needs of storage users. Relatively speaking, if a layer can be built on the current public cloud using blockchain technology, it may be a better option, because this solution can not only comprehensively dispatch high-performance storage resources, but also ensure the privacy of user information, while also fully enjoying the complete storage technology system of traditional cloud storage. Such a landing plan can relatively quickly realize the needs of decentralization to ensure data security and privacy, while giving decentralized storage technology sufficient development time. Not only does it give users time and opportunities to better understand centralized storage, it is also conducive to the large-scale implementation of decentralized technology in the future. However, it is undeniable that only through continuous innovation and experimentation can decentralized ecological storage open up a path that suits itself.

BlockBeats reminds that according to the document "Risk Warning on Preventing Illegal Fund Raising in the Name of "Virtual Currency" and "Blockchain"" issued by the China Banking and Insurance Regulatory Commission and other five departments in August 2018, the general public is requested to look at blockchain rationally, not blindly believe in the exaggerated promises, establish correct monetary concepts and investment ideas, and effectively enhance risk awareness; any clues of illegal and criminal activities discovered can be actively reported to the relevant departments.