A brief history of Filecoin mining hardware changes

Give Filecoin mining machines, rather than giving Filecoin to mining machines.

Original title: "Filecoin Hardware Chronicle 1"
Written by: lb

Give civilization to the years, not years to civilization.

Give Filecoin mining machines, rather than giving Filecoin to mining machines.

The title is "Chronicle 1" because this change is not over and may continue to change even after the mainnet is launched.

Prehistoric Times: 2017 - February 13, 2019

This is an era where most people cannot tell the difference between IPFS and Filecoin, an era where people believe that Filecoin mining is just assembling storage servers, and an era where people believe that hardware advantages are core advantages.

At this time, the so-called IPFS server or Filecoin mining machine, although many products are equipped with server outer shell, but they have PC internal organs. At first glance, the appearance is a 2U/3U/4U rack-mounted server, but the motherboard inside is plugged with a desktop CPU and a memory stick without ECC error correction function. What's more, low-end machines using the J1900 Celeron CPU once occupied the vast majority of such machines, and even became the facade model of some scams. These machines have memory ranging from 1GiB, 2GiB to 8GiB, and rarely have a configuration of more than 16GiB.

The reason for such configuration is very simple. As for storage, the key point is that a large number of hard disks can store more data, unlike Bitcoin, which does a lot of meaningless HASH calculations. This is also true. In fact, a large number of NAS devices, even enterprise-level NAS devices (except for high-performance distributed storage), do not really have high requirements for CPU and memory.

Every manufacturer in this era claims to have mastered the core technology and is reliable. What everyone is more concerned about is the "price/performance ratio" - whose machine is cheaper, and whose per-TB cost is the lowest.

However, many manufacturers have already run away without any results even in prehistoric times. Of course, there are also more manufacturers who directly transform their machines to mine other currencies. No matter what kind of machine configuration, there is always something to mine - if you really can't find a project, you can also create one.

At this stage, Protocol Labs warns everyone remotely: Don’t buy the so-called mining machine.

Spring and Autumn Period: February 14, 2019 - end of November 2019

Filecoin's code has finally been released. Although there are many bugs, as a great project, everyone can understand and guess that its launch process is long and tortuous. In fact, compared with other blockchain projects, we can see its code and document updates every day, and we can "feel" its algorithm from various angles and understand its activity.

But at this time, many people discovered a surprising fact. Something is wrong! How come the time spent on computing is more than the time spent on storage? With an ordinary machine, it only takes a few seconds to actually write 1GiB of data to the hard disk (it may be faster if it is RAID), but the Seal operation for replication proof takes tens of minutes? Isn't this a storage project?

This brings up a very serious problem: the machines with very low configurations in the past, such as J1900, seem to be no longer viable. At least they are not viable for Seal, which is the most time-consuming step in the entire Filecoin mining process. Of course, it is not impossible to use old machines, just treat them as storage cabinets. A slightly better configuration can still make use of the residual heat. But the configuration of the newly added machines must be at least more powerful. Simply add more server CPUs and a slightly larger memory. Gradually, second-hand servers entered this market, and entry-level Xeon E3, long-obsolete Xeon E5645 and other CPUs, and E5 2600 series v1 and v2 CPUs all came in droves. The memory size gradually became mainly 16GiB, and 32GiB began to appear.

Many people already think that the hardware configuration is too high. It has 32GiB memory. If it is higher, it will force people to rebel! Some people even clamored, is this still a storage project? How did it deviate from the direction? However, the direction of development of things cannot be changed by human will. Just like we think that the stock has fallen low enough and cannot fall any further, so we are reluctant to sell it, but the next day we find that it has fallen to a new bottom. If you think that the hardware configuration is high, you will think that you have made an empirical mistake when you look back after half a year.

However, some people still speculate that the subsequent hardware requirements are getting higher and higher through technical sensitivity, and better hardware configuration will not suffer in terms of cost performance. For example, at least the E5 v3 series server CPU, which supports features such as the avx2 instruction set, has better performance than v2 and a higher cost performance.

In this era, people gradually realize that capacity is not the only thing, or even not a problem. The speed of accumulating computing power is the key. Otherwise, no matter how many hard drives you have, you will only see those who run faster sharing the money.

Official hashrate rankings began to appear. Various vendors also upgraded their marketing methods, from simply promoting their own strong technology to claiming that they could be on the list. As a result, many "firsts" appeared. However, due to many bugs in the code itself, this ranking did not take shape and not many people paid attention to it.

Moreover, everyone realized a problem. With a machine, it is not easy to run Filecoin, or even simply compile and run it. If you want to run better and more stably, you need to modify the code yourself. All of this requires a lot of R&D resources and technical accumulation. So, people realized that software, operation and maintenance, and continuous iteration are more important than hardware. The official code changes every day, so why don’t you work hard?

At this stage, Protocol Labs still tells everyone: It is best not to buy the so-called mining machine.

Transition from the Spring and Autumn Period to the Warring States Period: End of November 2019 - Before the testnet launch on December 12, 2019

Protocol Labs suddenly announced that mining requires GPUs. This caused a stir. In fact, people have been studying the acceleration of GPUs for proof of replication and proof of space-time. With GPUs, algorithms such as FFT and Multiexp elliptic curves will indeed be much faster. However, in order to make the network more secure, Protocol Labs introduced two major changes, and the hardware configuration requirements also increased dramatically. GPUs have almost become standard from optional:

1. When producing blocks, the election proof (Election PoSt) is used. In order to complete it within a block height (preferably within half a block), Protocol Labs recommends the use of GPU. In other words, if you don't use GPU, even if you have computing power, it is difficult to get FIL! Isn't it a waste of mining?

2. Because small-capacity sectors are not safe, only 32GiB sectors are supported when the mainnet is launched. Although the testnet currently supports 1GiB sectors, the hardware standard must at least be aligned with 32GiB sectors.

Thus, the new hardware standard is out. At least 128GiB memory, preferably a multi-core CPU (relatively high-end Intel E5 v3 and above series, or powerful AMD Threadripper series), and an NVIDIA 2080 Ti graphics card... Although the CPU and graphics card can run with a slightly lower configuration, the efficiency will be compromised. If the memory is less, then sorry, it is easy to be out of memory.

Warring States Era: Testnet launched on December 12, 2019 - Now

After the testnet was launched, the Filecoin industry landscape has changed. At least everyone is paying attention to the official computing power rankings, block efficiency, FIL output and other parameters. In fact, these are just numbers and do not represent anything. However, to obtain these numbers, it does require a lot of investment. Investment in R&D, and investment in equipment.

Although the Filecoin market may be huge and many manufacturers can share this pie together, the competition between manufacturers is also intensifying in this era. There are all kinds of competition for the top spot, all kinds of articles praising and slandering, and new works of creating gods. Some say that it is meaningless to brush the list, and some say that not being on the list at least shows that you don’t have the technology; some say that your cost is high, and some say that my technology is awesome... These are all normal. A senior once said that market promotion is needed. The main network is online, and the results will naturally be revealed. I won’t say more here.

A major change led by the testnet is that the cluster mining model has become the mainstream. There are different opinions on which is better, stand-alone mining or cluster mining. Multiple machines with different accounts seem to have stronger resistance to penalties, and the total FIL mined may not be less; clusters require a lot of extra overhead for internal coordination, and intranet data transmission may also cause bottlenecks, but the computing power accumulates quickly, at least it can quickly reach the official block computing power threshold. But there is a very important reason why everyone has to use clusters: a larger total computing power makes it easier to get on the list, and it has the advantage of ranking high in terms of computing power, block rate, and number of FIL obtained.

As for cluster mining, each manufacturer has different implementations. But at least it reduces the dependence on GPU to a great extent. The machine that produces blocks and does time-space proof needs GPU, and other machines can be equipped with or not. The cost has increased so much, plus the ever-increasing price of 2080Ti, it is already crazy. Everyone also realizes that the non-storage factors of Filecoin hardware equipment account for an increasing proportion of the cost.

Of course, some manufacturers continue to increase the computing power growth rate of a single device and use more advanced hardware devices. For example, multiple GPUs are used at the same time, and better CPUs and larger memory are used. With more computing power, the cost has also increased. Is it more cost-effective? For a specific version of the program, it is indeed true, provided that there is a large amount of code-level optimization to fully utilize the parallelism of the GPU. If you think this is not enough? Then use FPGA, which can be optimized for specific operations; if FPGA is done well, it can even be made into ASIC chips. It feels like a review of the growth path of BTC mining machines. These things are indeed a bit technical and not so easy to make. But it is not only the hardware cost that needs to be paid, but also more R&D costs. No, wait, is it off track? Isn't Filecoin a storage project? How come the hardware configuration is almost the same as the computing cluster for deep learning?

Protocol Labs is still patiently reminding everyone: Please be cautious before the mainnet hardware configuration standards are announced.

The Age of Isolation: After the 2020 COVID-19 Pandemic

While everyone is discussing whether the coronavirus caused the delay of the second phase of the Filecoin testnet and the mainnet, the team at Protocol Labs is frequently updating the code day and night. Quietly, a lot of changes have taken place.

One of the biggest changes is that in the precommit phase of proof of replication, the winSDR algorithm was changed to SDR. The previous winSDR divided the data into windows of 128 MiB, which could speed up the calculation through parallel computing. However, this algorithm is unsafe. For example, using devices such as GPUs, a greater degree of parallelism can be achieved, and it can be done in a few seconds. This brings concerns to the security of the chain. After changing to SDR, the negative effects of over-optimization can be eliminated, and related calculations can only be performed in serial. Although it takes longer, the security risks are greatly reduced.

The improvement in security comes at the cost of more computing costs. What if a single task can only use 1 CPU Core? Aren't a lot of resources idle? Although the power consumption on the meter has decreased, it takes a very, very, very long time to complete a Seal. The miners' hearts are bleeding! Then run multiple tasks at the same time? However, the calculation of 32GiB sectors consumes memory on the machine. The more concurrent tasks there are, the more memory is required. Even the memory of the fat node for genetic computing is not that large! Let's not discuss whether the motherboard has so many memory sticks. After the outbreak of the epidemic, the sharp rise in memory prices has been worrying!

Can we use less memory and support more tasks? Naturally, they thought of this, so they made the following attempt.

But this code has not been merged. I guess we are too busy to solve many specific problems in the process. The idea is easy, but there are still many details to implement. But no matter what, the speed of hard disk is much slower than that of memory. Even NVMe SSD is the same. What's more, adding CPU, memory and GPU is already driving people crazy, and now adding NVMe?

In order to make everyone feel more at ease to accept SDR, the people in Protocol Labs can only work hard. Wild lilies also have their spring, and the long Seal can also be accelerated a little bit. So, I thought of how to squeeze out the performance of the CPU. In the SDR process, the SHA256 hash calculation (the hash algorithm of Bitcoin) is used to calculate each node. Can you think of a trick to speed it up without relying on new hardware? Well, I have it. Use the SHA extension, and through the newly added instructions, the calculation of SHA256 is greatly accelerated. Cool! Wait, something is wrong. This SHA extension seems to be proposed by Intel, but Intel's desktop CPU and server CPU do not support it, but AMD's CPU supports it very well. However, this is not a problem, because the work machine used by the developers of Protocol Labs is AMD's Threadripper. Naturally, your own machine should be optimized first. Although the CPU is super expensive, it is super cool to use.

AMD's CPU market share in the server market has always been low, and a large number of server CPUs are still Intel. Taking care of AMD for its own use, let's also take care of Intel. So, we work on the SIMD instruction set and see how to use these instruction sets for optimization:

The above sse, sse2, sse4.1, sse4.2, avx, avx2, etc. are all commonly used SIMD instructions. SIMD means single instruction multiple data, which means that one instruction can perform multiple groups of calculations, which is naturally faster. After relevant optimization during compilation, it can show better performance improvement. However, some older CPUs suffer, for example, E5 v2 does not support avx2, so the default compiled version cannot run.

But the sha option makes AMD's CPU efficiency more obvious. I even wonder if AMD has sent people to embed in the protocol lab to increase its CPU market share in the storage market in this reasonable way? Yes, in theory, it is possible. However, NVIDIA is also suspected. In addition, the second phase of the test network has not been launched yet, and the algorithm may change at any time. Intel still has a lot of opportunities...

The higher the road, the higher the devil. Are there still some black technologies that can break through these bottlenecks? Maybe. However, it doesn't matter. The algorithm is dead, but people are alive. The algorithm can be changed again. What about the main network online? The pace of updates will not stop, and it can be forked at any time. This... won't there be a lot of dissatisfaction? For example, accusations such as cost, performance, and energy consumption.

These problems obviously exist. Protocol Labs is also aware of this. They also want to optimize and adapt to more hardware. For example, the following discussion, which has 66 floors, is about the argument that many hardware do not support the default compilation configuration and cannot run:

But in any case, the security of the chain is the most important. The road ahead is itself a controversial one. But if the chain is not secure, for example, one miner occupies the majority of the computing power, then the chain is doomed, and all the efforts of everyone will be reduced to zero.

Therefore, no matter how much we complain, we can only make way for safety.

Give Filecoin mining machines, rather than giving Filecoin to mining machines.

Off topic

For the same price, the more nodes there are, the higher the cost performance?

No, if there are large differences in CPU, GPU, memory, etc., the computing power accumulation speed of the same node may differ by dozens of times.

Why are the prices different for the same hardware configuration?

Because software optimization is the core factor that determines FIL output. The same hardware, running different codes, may have very different hardware optimizations and different stability.

The price of memory has already increased a lot now. Once the mainnet hardware standards are set, will the prices of related hardware increase significantly?

Risk and benefit are closely related. Although it may be cheaper to determine the hardware in advance, there is a risk that the algorithm cannot be optimized for the new version of the code (especially with the recent frequent changes). Everyone has a different estimate of risk, and this can only be decided by everyone.

My device has low configuration, can I still participate in Filecoin mining?

It depends on how you define mining. If you outsource the proof part, it is often feasible, it just depends on the cost-effectiveness.

Will hardware configuration continue to increase in the future?

Maybe. But it is also possible that the chain will be considered secure enough and the barriers to entry will start to decrease. But that is in the future.

Source link: mp.weixin.qq.com