Emergency Response on Linux (Basic Knowledge Record)

View CPU usage#

The purpose of viewing CPU usage is to determine which process is consuming high CPU usage (for mining).

top -c -o %CPU
htop -t

# View the top five processes consuming CPU usage
ps -eo pid,ppid,%mem,%cpu,cmd --sort=-%cpu | head -n 5

Memory usage#

top -c -o %MEM
htop -t

# View the top five processes consuming memory
ps -eo pid,ppid,%mem,%cpu,cmd --sort=-%mem | head -n 5

Network usage#

iftop # Requires installation and root permission to run, does not display process ID
nethogs # Requires installation and root permission to run, displays process ID
ss -ntu | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -nr # View the number of local network card IP connections (source IP)
ss -ntu | awk '{print $6}' | cut -d ":" -f1 | sort | uniq -c | sort -nr # View the number of destination IP connections (destination IP)

External IP#

# Search by destination IP
netstat -pantu | grep 1.1.1.1
netstat -pantu | grep 3389
lsof -i:3389 # Requires root permission

# Search by local IP
netstat -pantu | grep 3389
lsof -i:3389

Finding malicious samples#

Get PID - Find the path of the malicious file
Found the malicious file - PID

Get PID based on process name or partial string

pidof "name"
ps -aux | grep "name"
ps -ef | grep "name" | grep -v grep | awk '{print $2}'
pgrep -f "name"

Get detailed information of the process based on PID

lsof -p PID # Requires root permission
pwdx PID # Requires root permission, get the directory where the PID process was started, which is the path where the malicious file was started
systemctl status PID # Get the status information of the process
cat /proc/PID/maps # Output the memory mapping information of the specified process
ls -al /proc/1505945/exe # Output the absolute path of the program executed by the specified process
cat /proc/$$/mountinfo # View the file system information mounted by the current process, where $$ represents the process ID (PID) of the current process.

View threads

ps H -T -p PID # PID is the process ID, spid is the thread ID, CMD represents the command line of the process/thread
ps -Lf PID # Display information of each thread in the specified process
pstree -agplU # Display all process relationships in the system ✅

Get PID based on file

lsof | grep FILENAME
lsof FILENAME
fuser FILENAME # Command used to find the process that uses a specific file or socket

Determine program running time#

ps -eo pid,lstart,etime,cmd | grep  PID

# Compare the creation time of the malicious file
stat FILENAME
ls -al FILENAME

Handling abnormal processes#

Download samples from the server
Online virus analysis
Process killing

1. Check if there are child processes
ps ajfx ✅
systemctl status

2. If there are no child processes
kill -9 PID

3. If there are child processes
kill -9 -PID

Delete malicious files#

Check if the process is occupying the file

lsof FILENAME

If the file cannot be deleted due to the a and i attributes

chattr -a
chattr -i

If the file cannot be deleted due to strange file names

Delete the file using the inode number

ls -li FILENAME # View the inode

Delete the file

find ./* -inum INODE -delete
find ./ -inum INODE -exec rm {} \;
find ./* -inum INODE -exec rm -i {} \; # Prompt for confirmation before deleting
find ./* -inum INODE -exec rm -f {} \; # Force delete
find ./* -inum INODE | xargs rm -rf
rm `find ./* -inum INODE`

These are the commands needed for most emergency incidents. There may be some differences for different events:

Extension 1: TCP connection status in netstat#

Status	Type	Description
LISTEN	TCP listening port	Listening state. Indicates that the port is waiting for a connection from the other end for communication.
SYN_SENT	TCP state	Connection request has been sent. Indicates that the TCP connection has been initiated but no acknowledgment has been received yet.
SYN_RECV	TCP state	Connection request is being received. Indicates that the TCP connection has been received and is waiting for acknowledgment. Usually only appears on the server side, indicating that a request has been received from the client.
ESTABLISHED	TCP state	Indicates that the TCP connection has been established and is in communication.
FIN_WAIT1	TCP state	Indicates that the TCP connection has been closed and is waiting for the other end to close the connection.
FIN_WAIT2	TCP state	Indicates that the TCP connection has been closed and is waiting for the other end to close the connection or is receiving final acknowledgment.
TIME_WAIT	TCP state	Indicates that the TCP connection has been closed and all data has been transmitted, waiting for a period of time to ensure that all packets have been processed.
CLOSE_WAIT	TCP state	Indicates that the TCP connection has been closed but the local application has not closed the connection.
LAST_ACK	TCP state	Close request has been sent and waiting for the other end's close request.
CLOSING	TCP state	Indicates that the TCP connection is in the process of closing.