Storj mining, IPFS mining exercise

Why Storj

Among the distributed storage virtual currencies that have been deployed, Storj is the one that is relatively stable and has the mining model closest to IPFS. It can be simply understood as the blockchain version of Baidu Netdisk. Of course, there are broader application prospects in the future. It has reached cooperation with many network companies including blockchain, such as the famous open source FTP software FileZilla. Since July 2017, you can use FileZilla's client software to directly upload files to Storj's network, which is very convenient. The recent cooperative blockchains include SONM, DOC.AI, BitClave, etc. Our goal is to verify whether our hardware and network are capable of the upcoming IPFS mining, among which network optimization is the key. Storj mining does not require collateral and has no penalty mechanism. It can be mined at any time and abandoned at any time, which is very convenient to get started.

Mining income

What is most criticized now is that its payment system and storage system are separated, which means that the settlement of mining is still manual. At the beginning of each month, a Google online form is published in the official chat channel for everyone to check the previous month's income ( calculated in US dollars ), and then the money is transferred in the middle of the month. The income is converted into Storj tokens based on the market price before the mid-month transfer. This compromise is bound to cause dissatisfaction and doubts. The official promised that an algorithm will be used to achieve automatic settlement later.

The income situation is different every month, which is related to the data you store and download that month. If you simply look at the amount of data stored, the income data will vary greatly. Take myself as an example. In November, my average storage was about 37T of data. The table shows that the income was $670, an average of $18 per T per month. In December, my average storage was 40T, but the salary table showed that my income was $260. The storage data increased slightly from the previous month, but the income was less than 40% of the previous month. What is the reason for this? Let's first look at the official income calculation formula:

 paymentModelFunction = function(gbHours, downloadedBytes) { HOURS_IN_MONTH = (24 * 365) / 12 ## Average number of hours in a month STORJ_USD_RATE = 2.20 ## This is determined by value reported on https://coinmarketcap.com/ at time ## of payout calculation. gbHoursScaled = sapply((gbHours - median(gbHours)) / sd(gbHours), function(x) ifelse(x < 0, 0, x)) downloadedBytesScaled = sapply((downloadedBytes - median(downloadedBytes)) / sd(downloadedBytes), function(x) ifelse(x < 0, 0, x)) ## Both gbHoursScaled and downloadedBytesScaled can not be less than 0 to ## ensure everyone gets at least the base payout amount. downloadedBytesFlag = as.numeric(downloadedBytes > 0) gbHoursFlag = as.numeric(gbHours >= 730) isQualifiedFlag = sapply(gbHoursFlag + downloadedBytesFlag, function(x) ifelse(x > 0, 1, 0)) ## At least one of the above criteria must be met to qualify for a payment. basePayout = (1.50 / STORJ_USD_RATE) * isQualifiedFlag ## The current base payout is set to $1.50 USD. ghHourPayout = 4.3536 * gbHoursScaled * isQualifiedFlag downloadedBytesPayout = 3.0060 * downloadedBytesScaled * isQualifiedFlag payoutAmountSTORJ = ghHourPayout + downloadedBytesPayout + basePayout payoutAmountUsd = payoutAmountSTORJ * STORJ_USD_RATE  cbind(payoutAmountSTORJ, payoutAmountUsd)}

Attached is the official salary form for November and December for download

Don’t understand? No problem, I don’t understand either. You just need to know some key points:

• The minimum payment amount is $1.5

• At least 1G of data needs to be stored and run 24 hours a day for a whole month, which is about 730 hours.

• GigabyteHours is the average value of the data you store and calculated based on an average of 730 hours per month.

• DownloadedBytes is the total amount of data that the storage user has downloaded from you

By comparing the tables, we found that the difference comes from DownloadedBytes. In November, I provided users with a total of 1.4T downloads, while in December I only provided 5G. Therefore, only when users download more data can I get more benefits. Assuming that we are storage users, we will not always keep downloading files from the network disk. Storj has plans to open public downloads. At that time, files can be shared, and miners can get more traffic benefits. Click to visit PC byte calculation converter

Preparation

• A host can be a Raspberry Pi or your gaming PC host. The configuration of CPU and memory size depends on how many nodes you plan to run and how much storage space you configure. The storj windows interface version sets each CPU thread to create a node, and the maximum storage of a single node is limited to 8T. The more nodes you have, the more mining tasks you get, and the corresponding memory consumption and network burden will also increase. Because storj is not well-known in China and there are few storage users, it is necessary to build more nodes to increase task acquisition. Now there are more monks and less porridge. There is a lot of free time, and 0.5G memory per TB is enough.

• A question I often see about hard drives is whether it is necessary to use a professional hard drive of red disk or above. I think it is not necessary for storj, as it does not have long-term periodic reading and writing of data. As for IPFS, we will have to wait for further announcement of mining details to know.

• Here, let's focus on the network . Now, 100M bandwidth is quite popular in China. It is safer to use the bandwidth of the three major operators (China Telecom, China Mobile, and China Unicom), because the biggest problem of other small operators is the complex NAT environment, which may be nested with multiple layers of NAT and cannot allocate external IP, which is very troublesome. We know that this type of storage mining is essentially the same as our P2P ancestor BT download, both involving uploading and downloading. The two most commonly used protocols for P2P now are DHT (distributed hash table) based on Kademila, and UPNP (universal plug and play, click to check the encyclopedia). People born in the 1980s may know that BT was very popular before the emergence of blood-sucking Thunder. If you want to increase the download and upload speed, you must do port mapping. Let's take the delivery of express delivery as an example. Before port mapping, you are in a building in the community. The courier has to get the exact location of your building through the doorman when he comes for the first time, find it and deliver it to you. After port mapping, the courier does not need to ask the location and goes straight to your house to pick up the goods. Obviously, this is efficient. I have verified that storj has a certain NAT penetration ability (limited), but when facing a small operator with a complex NAT environment, storj's bridge may not be able to connect to you, resulting in the inability to mine. Although the effect of NAT penetration can be achieved with the help of third-party software and services, this is absolutely inefficient and requires additional costs, so it is not recommended here. If the three major operators do not assign you an external network IP, you can usually solve it by calling to complain. I will explain the port mapping and UPNP settings in detail in the mining optimization section later.

• Wallet address Set up an ETH wallet address to receive Storj mining income every month. The official recommendation is to use Parity, Mist, and MyEtherWallet generated by online wallets

Mining Settings

How to use the Windows interface version

Download Storj interface version mining program 64 bit 7.3.1

Download Storj interface version mining program 32 bit 7.3.1

Download the time synchronization tool NetTime3.14

• Install and set up NetTime. Time synchronization in Windows systems has always been a problem. Most of the time, the difference is only a few seconds to tens of seconds, which we usually can't feel. In order to keep the timestamps of each node consistent, Storj requires that the error should not exceed 500 milliseconds, otherwise the mining connection will fail. Therefore, it must be installed here. After completing the installation by default, find NetTime in the tray, right-click and find the property settings
  

• According to the figure below, set the following servers in China, change the synchronization time to 15 minutes, click Update and then turn it off. The program will automatically run the update in the background

 hk.pool.ntp.orgjp.pool.ntp.orgkr.pool.ntp.org0.asia.pool.ntp.org

• The default connection mode for installing the mining program is NAT (UPNP). UPNP can be understood as automatic port mapping. As long as there is an external IP and the software, system, and network device all support and enable the UPNP function, it can be used normally ( if UPNP fails, the backup plan reverse http tunnel will be enabled for NAT connection ). Open the installation program and check Add firewall rules

After the installation is complete, click Start Setup. The first step is to fill in the wallet address.

Step 2: Set up your mining data storage folder

The third step is to set the size of the folder, and the unit can be changed later.

Step 4: If Reachable is not checked by default, it will be connected via UPNP

After completion, we can see that the bridge has been connected. Delta is displayed as a green delay number, indicating that it is normal. The random orange tunnel on the port also indicates that upnp is working properly.

Finally, we need to use the official API to confirm whether our node has entered the mining queue. Copy a string of the NodeID list and put it in the following address to query https://api.storj.io/contacts/yourNodeID For example, my demonstration https : //api.storj.io/contacts/c004bfed2e16884b83c63148b5aac9367f767cae

If you see "responseTime":10000," this parameter means everything is normal. If not, delete the node and add it again.

How to use the Windows command line version

For players with high performance requirements, this method is recommended because there is no interface display and no graphics rendering is required. In this way, even if you use a graphics card for mining, it will not affect the graphics card speed. Secondly, it is not limited by the CPU thread of the GUI version, and you can establish nodes unlimitedly, but each node requires at least 1G memory support , which is slightly more complicated than the interface version.

Download Git 2.15.1 64-bit

Download NodeJS 8.9.4 64-bit

• *Installation environment support* Both programs are installed in NEXT mode. Click the Start menu and enter cmd to open the command prompt. Enter

 node --versionnpm --version

If the version number is returned, everything is normal

• Install the command line client. Enter powershell in the Start menu, right-click, and select Open as Administrator.

In the command line that opens, enter npm install -- global windows - build - tools to install the Visual Studio Build toolkit. When you see two green items after the installation, it means the installation is complete.

Time synchronization is installed and set up in the same way as the interface version above

Close PowerShell, reopen cmd, and enter npm install - g storjshare - daemon to install the command line mining program

After the installation is complete, enter storjshare -- version If the version number is returned, the installation is successful.

Create a directory on your computer. For example, my example here is the folder linkcoin.net1 on the C drive. Then enter storjshare - create -- storj 0xfeb5c46006fca94f5022c7d693b3d22bb7295ee7 -- storage C : \linkcoin . net1 -- size 3GB -- outfile C : \linkcoin . net1\config . json -- logdir C : \linkcoin . net1 This creates a node. The directory of each node must be set separately.

Enter storjshare daemon to start the process

Enter storjshare start -- config C : \linkcoin . net1\config . json to run the mining node. If successful, the Node.exe window will pop up. Remember to allow the firewall to pass.

Type storjshare status to get the current status.

Finally, remember to query the Node running status of the official api according to the URL of the windows interface version above

How to use Linux

Here we take Debian/Ubuntu as an example. Follow the steps below to execute the command. + storjshare - daemon@5 . 3.0 displayed at the end, the installation is successful.

 wget -qO- https://raw.githubusercontent.com/creationix/nvm/v0.33.3/install.sh | bashExit the command lineOpen a new temporary command linenvm install --ltsRestart the command linesudo apt-get update && sudo apt-get dist-upgradesudo apt-get install git python build-essential -ynpm config set user 0npm config set unsafe-perm truenpm install --global storjshare-daemon

Create a new folder in the root directory, such as linkcoin.net. Because I am using the vultr server for demonstration, it is an external network and does not require UPNP and forwarding ports, so directly enter the direct connection command (please replace your own wallet address, IP address and directory location):

storjshare - create -- storj 0xfeb5c46006fca94f5022c7d693b3d22bb7295ee7 -- storage / linkcoin . net -- size 1GB -- outfile / linkcoin . net / config . json -- rpcaddress 198.13 . 54.178 -- rpcport 5000 -- manualforwarding true -- logdir / linkcoin . net

After pressing Enter, a configuration file editing box pops up, enter : wq to save

Enter storjshare daemon to run the main program, and enter storjshare start -- config / linkcoin . net / config . json to start the mining process

The Linux system does not need to synchronize time. Enter storjshare status to display the mining status.

Finally, follow the URL given by the Windows interface version to query the Node running status of the official API. If there is no responseTime parameter, you need to delete and re-establish the Node, otherwise you will find that there is no data after 10 days of downtime.

Advanced Optimization

Download Peanut Shell official client

• Port forwarding and dynamic domain names

We can see that the port running in the default upnp mode shows a yellow Tunnel, while the direct connection shows a green TCP. Just from the color, we know that green is definitely the best state. This is true. Upnp will take up extra resources to keep the connection stable. Direct connection is the best choice for performance. IPFS will also encounter this situation. After reading this article, you will know how to deal with it. The following assumes that you have a dynamic IP address on the external network (home broadband is basically dynamic, not fixed IP address). You can do the following to achieve direct connection.

If you use a modem to dial directly, you only need to apply for a free dynamic domain name to bind your dynamic IP. There are many businesses in China that provide free dynamic domain names. I will use Peanut Shell for demonstration.

After installation, open the client and click Register to apply for a free account. After successful registration, you can get a second-level domain name without purchasing a paid domain name.

After registration, log in to the client. No other operations are required. At this time, when you ping the secondary domain name provided by Peanut Shell, your public IP address will be returned.

For friends who use router dial-up, we need to set up port mapping in the router background. First, we need to know our intranet address, open the command prompt, enter ipconfig to obtain the network card settings, and find your intranet IP address and gateway address.

Open the network card settings. If your network card obtains IP automatically, you need to change it to manual according to the sequence in the figure below to fix the internal network IP.

Then open the router backend. If you don't know the router backend address, you can look at the label on the back of the router. It will usually be written there. Because there are so many types of routers and the systems are very different, I can't show you all of them. I can only show you the configuration of my own router. Generally, there is a port forwarding function. You need to look carefully. Add the local IP as my intranet IP address in the forwarding list, and set the port at will, as long as it is the same as the one set in storj. I will show you the setting of 4000 here. You can choose BOTH as the protocol. Click Apply after adding.

Now let's go to the final storj settings. Let's take the first node of the interface version as an example. In the settings panel, click Edit to edit. If you have not installed any third-party text software, you can open it with Notepad for easy typesetting.

Fill in your Peanut Shell secondary domain name in rpcAddress , set the port to the same as the router forwarding list, set doNotTraverseNat to true , save and close

Right click on the tray to exit the storj mining program and reopen it. Now you can see that the green TCP is displayed at the port. If you are not sure whether the port mapping is successful, you can open this port mapping website for testing. If the result is open, it means that the port mapping and dynamic domain name are both normal.

Finally, I attached a comparison chart of NAT and TCP direct connection that I tested in a virtual machine for 14 hours. Of course, this difference can only be used as a reference, the time is not long enough.

• Configuration file analysis The following is an example from the official description. Except for the Chinese annotations, you can ignore the others and keep the default. If you need to switch between direct connection/UPNP (NAT), you can edit the . config After the modification, you need to save and restart storj.

 { // Set the ETH wallet address "paymentAddress": "0x4646fBd7323521929DC06b3Aff9DEb338a966ade", // Subscribes to the given contract topics // See https://storj.github.io/core/tutorial-contract-topics.html "opcodeSubscriptions": [> "0f01020202", "0f02020202", ], // An array of bridges to connect and accept contracts, // send exchange reports and discover network seeds. "bridges": [ "url": "https://api.storj.io", "extendedKey": "xpub6AHweYHAxk1EhJSBctQD1nLWPog6Sy2eTpKQLExR1hfzTyyZQWvU4EYNXv1NJN7GpLYXnDLt4PzN874g6zSjAQdFCHZN7U7nbYKYVDUzD42" { } ], // Known preferred seeds in form of a storj URI // Example: "storj://[ip.or.hostname]:[port]/[nodeid]" "seedList": [], // Direct connection needs to be set here. For static external IP, just write the IP. For dynamic external IP, write the dynamic domain name. For UPNP(NAT) mode, write 127.0.0.1 "rpcAddress": "89.180.103.39", // Port that needs to be set for direct connection. UPNP does not need to worry about it "rpcPort": 4000, // "tunnelGatewayRange": { "min": 0, "max": 0 }, // Number of times to retry joining the network and the wait time between "joinRetry": { "times": 3, "interval": 10000 // Temporarily stop sending OFFER messages if more than this number of shard // transfers are active }, "offerBackoffLimit": 4, // ECDSA private key for your network identity, your Node ID is derived from // this and it is used to sign and verify messages "networkPrivateKey": { "networkPrivateKey": 1, // Number of times to retry joining the network and the wait time between "joinRetry": { "times": 3, "interval": 10000 // Temporarily stop sending OFFER messages if more than this number of shard // transfers are active }, "offerBackoffLimit": 4, // ECDSA private key for your network identity, your Node ID is derived from // this and it is used to sign and verify messages "b9cd6834fc097ef00c3a3ddf46928cf10f60f7de8c4365d737041b03b932f532", // Log record details setting, you can set 0 to close the log // 4 - DEBUG | 3 - INFO | 2 - WARN | 1 - ERROR | 0 - SILENT "loggerVerbosity": 3, // Log save path setting, if not written, the default location is: // $HOME/.config/storjshare/logs/[nodeid].log "loggerOutputFile": "C:\\Users\\user\\.config\\storjshare\\logs", // Storage location of this node "storagePath": "C:\\Users\\storj", // The hard disk space prepared for this node, the unit supports B, KB, MB, GB, TB "storageAllocation": "7TB"}

In addition to the above public configuration, there is also a hidden setting for the size of the received fragments. If a node crashes, the number in the Restarts column increases. You can adjust this parameter maxShardSize , which is written below the space size setting. The official recommendation is to start with 50-100M, and then reduce it by a few MB each time to find the best setting for your hardware.

 "storageAllocation": "2GB", // Max size of shards that will be accepted and stored // Use this and make this lower if you don't have a strong internet connection "maxShardSize": "100MB",

• Command parsing命令命令信息start启动一个挖矿节点stop停止一个挖矿节点restart重启一个挖矿节点status检查节点状态logs最近的节点日志create创建新的配置文件destroy结束挖矿进程killall结束所有挖矿进程并停止挖矿守护程序Daemon启动挖矿守护程序Help [ CMD ]显示帮助[ cmd ] command

 Example of executing commands Information of executing commandsstorjshare daemon Start the daemon processstorjshare start --config path/to/config.json Start the mining node configuration program in the specified directorystorjshare start --config --unsafe Run the node in unsafe mode and skip resource monitoringstorjshare stop --nodeid Stop the specified nodestorjshare restart --nodeid Restart the specified nodestorjshare status Check the node statusstorjshare logs --nodeid Node's latest logsstorjshare create --storj … Create a new nodestorjshare destroy --nodeid End the specified nodestorjshare killall End all node processes and stop the daemon storehouse help logs Show the help command of the logsstorjshare save Save the running node snapshotstorjshare load Load all nodes from the snapshot

storjshare save and storjshare load are very useful. After creating all nodes, run save. In this way, you don’t need to run config one by one the next time you start. Just run load snapshot to automatically load and run all nodes. Before loading, you must first use storjshare daemon to start the daemon program.

troubleshooting

• The firewall port is not open, as indicated by the port number being displayed in black.

Solution: Enter the system firewall settings to add release, enter the firewall in the start menu to open, enter the advanced settings, and add the storj process in the inbound rules

Final Thoughts

Due to the small number of network nodes in China and even fewer uploaders using Storj, the mining income in China is not considerable. Even most of my friends who have tested it told me that it is still not enough to beat the electricity bill. I suggest that you use it to test the mining environment and practice. It is not recommended to mine this coin with a large investment. If you have any operational problems, you can leave a message in my community https://bbs.linkcoin.net/d/3