Suricata+Arkime搭建网络流量分析系统-526互联

操作系统：openEuler 22.03 (LTS-SP2)

Suricata版本：7.0.2

Arkime版本：4.6.0-1.el9

ElasticSearch版本：elasticsearch-oss-7.10.2

服务器配置：8vCPU，16G内存，1T硬盘（ens16镜像口，ens18管理口） [可通过lshw -c network -businfo查看网卡信息]

一、操作系统基础配置

1.更新操作系统软件

echo 'proxy=http://proxy.test.work:3128' >> /etc/yum.conf

# dnf -y update

# reboot

3.安装工具软件和时间同步软件

＃ dnf -y install net-tools chrony lsof tar lrzsz make

4.配置时间同步

vi /etc/chrony.conf
将第三、四行修改为时间同步服务器地址
pool 192.168.xxx.1 iburst
pool 192.168.xxx.2 iburst

# systemctl start chronyd
# systemctl enable chronyd
# systemctl status chronyd

5.启动时自动开启ens16网卡混杂模式

cat << 'EOF' > /usr/lib/systemd/system/set-ens16promisc-mode.service
[Unit]
Description=Set ens16 to promiscuous mode

[Service]
Type=oneshot
ExecStart=/sbin/ip link set dev ens16 promisc on

[Install]
WantedBy=multi-user.target
EOF

# systemctl daemon-reload
# systemctl start set-ens16promisc-mode.service
# systemctl enable set-ens16promisc-mode.service

二、安装Suricata

1.下载suricata安装包

https://www.openinfosecfoundation.org/download/suricata-7.0.2.tar.gz

2.安装依赖包

# dnf -y install gcc pcre2-devel libyaml-devel jansson-devel libpcap-devel python3-pip file-devel lua-devel libmaxminddb-devel zlib-devel rustc cargo

# pip install pyyaml

3.编译安装suricata

# ./configure --enable-nfqueue --enable-lua --enable-geoip --prefix=/usr --sysconfdir=/etc --localstatedir=/var
# make && make install && make install-full
# cp -d suricata-7.0.2/libhtp/htp/.libs/libhtp.so* /lib64
# ldd /usr/bin/suricat

4.更新suricata规则

＃ /etc/suricata/suricata-update -o /etc/suricata/rules/

5.配置suricata自启动文件

# cat << 'EOF' > /etc/sysconfig/suricata 
OPTIONS="--af-pack -i ens16"
EOF

# cat << 'EOF' > /usr/lib/systemd/system/suricata.service
[Unit]
Description=Suricata IDS/IPS
After=network.target

[Service]
# Environment file to pick up $OPTIONS. On openEuler/EL this would be
# /etc/sysconfig/suricata, or on openEuler, /etc/suricata.
EnvironmentFile=-/etc/sysconfig/suricata
ExecStartPre=/usr/bin/rm -f /var/run/suricata/suricata.pid
ExecStart=/usr/bin/suricata -c /etc/suricata/suricata.yaml --pidfile /var/run/suricata/suricata.pid $OPTIONS
ExecReload=/bin/kill -USR2 $MAINPID

[Install]
WantedBy=multi-user.target
EOF

# systemctl daemon-reload
# systemctl enable suricata

6.编辑suricata配置文件

# cd /etc/suricata/
# cp suricata.yaml  suricata.yaml-default
# vi suricata.yaml
修改18行为本机网段
＃HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]"
HOME_NET: "[192.168.xxx.0/124]"

修改24和25行为
＃EXTERNAL_NET: "!$HOME_NET"
EXTERNAL_NET: "any"

修改2146~2149行改为suricata规则目录
default-rule-path: /var/lib/suricata/rule
 
rule-files:
- "*.rules"      #此处可以匹配多条规则

修改617和808行为镜像抓包端口
af-packet:
    - interface: ens16
pcap:
    - interface: ens16

7.建立一個警告转换成丢弃封包的调整设置文件

# cat << 'EOF' > /etc/suricata/modify.conf
re:. ^alert drop
EOF

＃ chmod 600 /etc/suricata/modify.con

更新规则库
# suricata-update --modify-conf /etc/suricata/modify.conf --no-merge 
# rm /var/lib/suricata/rules/tor.rules

测试Suricata Rules文件配置
suricata -c /etc/suricata/suricata.yaml -s /etc/suricata/rules/suricata.rules -T

测试suricata是否可以正常运行
# suricata --suricata-conf /etc/suricata/suricata.yaml -D
# suricata -c /etc/suricata/suricata.yaml -i ens16
# tail -20f /var/log/suricata/fast.log
# systemctl start suricata
# systemctl status suricata

三、安装Arkime(下载elasticsearch前，先访问arkime网址查看arkime和elastic兼容版本)

1.下载arkime和elasticsearch安装包

https://github.com/arkime/arkime/releases/download/v4.6.0/arkime-4.6.0-1.el9.x86_64.rpm

https://www.docker.elastic.co/r/elasticsearch/elasticsearch-oss:7.10.2-arm64

2.安装依赖包

dnf -y install perl-JSON perl-LWP-Protocol-https perl-libwww-perl

3.安装elasticsearch和arkime

# rpm -ivh elasticsearch-oss-7.10.2-x86_64.rpm
# systemctl daemon-reload
# systemctl enable elasticsearch
# systemctl start elasticsearch
# systemctl status elasticsearch

# rpm -ivh arkime-4.6.0-1.el9.x86_64.rpm
# /opt/arkime/bin/Configure

4.下载GEO文件

# mkdir /opt/arkime/geoip
# cd /opt/arkime/geoip
# wget https://github.com/boundary/wireshark/blob/master/manuf
# wget https://www.iana.org/assignments/ipv4-address-space/ipv4-address-space.csv
# wget https://github.com/P3TERX/GeoLite.mmdb/releases/download/2023.12.28/GeoLite2-ASN.mmdb
# wget https://github.com/P3TERX/GeoLite.mmdb/releases/download/2023.12.28/GeoLite2-Country.mmdb
# mv manuf.txt oui.txt
# chmod a+r *.*

5.提取arkimecapture需要的so文件

下载openssl二进制安装包

http://ftp.openssl.org/source/openssl-3.2.0.tar.gz

编译安装openssl
# tar -zxvf openssl-3.2.0.tar.gz
# cd openssl-3.2.0
# ./Configure
# make
# cp libssl.so.3 /usr/lib64/ && cp libcrypto.so.3 /usr/lib64/
# ldconfig
# /opt/arkime/bin/capture ＃查看所有so文件是否都已具备

6.初始化elasticsearch数据库

/opt/arkime/db/db.pl http://localhost:9200 init

7.添加管理员账号口令用于Arkime控制台登录

/opt/arkime/bin/arkime_add_user.sh admin "Admin User" admin --admin

8.在Arkime配置中添加Suricata插件

# vi /opt/arkime/etc/config.ini
修改114行
＃viewPort=8005
viewPort=80

修改122和123行内容为
geoLite2Country=/opt/arkime/geoip/GeoLite2-Country.mmdb
geoLite2ASN=/opt/arkime/geoip/GeoLite2-ASN.mmdb

修改129为
irFile=/opt/arkime/geoip/ipv4-address-space.csv

修改133行为
ouiFile=/opt/arkime/geoip/oui.txt

修改209行为
plugins=suricata.so
并在209行后插入如下内容
# suricataAlertFile should be the full path to your alert.jsaon or eve.json file
suricataAlertFile=/var/log/suricata/eve.json
suricataExpireMinutes=60

修改300~306行内容，优化arkime性能
magicMode=basic
pcapReadMethod=tpacketv3
tpacketv3NumThreads=2
pcapWriteMethod=simple
pcapWriteSize=2560000
packetThreads=5
maxPacketsInQueue=200000

9.修改Arkime和Suricata联动权限
chmod o+r /var/log/suricata/eve.json
sed -i 's/dropUser=nobody/dropUser=root/g' /opt/arkime/etc/config.ini

10.启动arkime服务

systemctl start arkimeviewer.service #arkime控制台服务
systemctl enable arkimeviewer.service
systemctl status arkimeviewer.service
systemctl start arkimecapture.service #arkime抓包服务
systemctl enable arkimecapture.service
systemctl status arkimecapture.service

11.Arkime控制台查阅Suricata标志

http://{arkime地址}

admin / admin #用添加arkime管理员账号登录

在搜索栏输入：suricata.signature == EXISTS! ＃查看suricata标志

四、编写内存释放和磁盘清理文件

# vi /home/free_mem.sh 
#!/bin/bash
memory_threshold_mb=1000

while true; do
    free_memory=$(free -m | awk 'NR==2{print $4}')

    if [[ $free_memory -lt $memory_threshold_mb ]]; then
        sync; echo 1 > /proc/sys/vm/drop_caches
    fi

    sleep 1h
done

# vi /home/clear_disk_space.sh 
#!/bin/bash
while true; do
  disk_usage=$(df -h / | tail -n 1 | awk '{print $5}' | tr -d '%')
  if [ $disk_usage -gt 80 ]; then
    rm -f /opt/arkime/raw/* &&
    echo '' > /var/log/suricata/eve.json
  fi
  
  sleep 1800
done

# chmod +x /home/free_mem.sh && chmod +x /home/clear_disk_space.sh
# bash /home/free_mem.sh &
# bash /home/clear_disk_space.sh &

五、清理Elasticsearch中arkime的session索引内容

1.查看索引磁盘占用量

curl 'localhost:9200/_cat/indices?v'

2.清理指定所有的全部内容

curl -X POST "http://localhost:9200/索引名称/_delete_by_query" -H 'Content-Type: application/json' -d'
{
  "query": {
    "match_all": {}
  }
}
'

3.删除指定索引

curl -X DELETE 'http://localhost:9200/索引名称'