【实战】Splunk Indexer Cluster部署

【实战】Splunk Indexer Cluster部署过程

1 部署准备工作

1.1 检查防火墙

1.1.1 查看防火墙状态

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
[root@splunk_master opt]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: inactive (dead)
Docs: man:firewalld(1)
[root@splunk_master opt]# systemctl start firewalld //启动防火墙
[root@splunk_master opt]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
Loaded: loaded (/usr/lib/systemd/system/firewalld.service; disabled; vendor preset: enabled)
Active: active (running) since Wed 2022-05-04 13:11:27 EDT; 2s ago
Docs: man:firewalld(1)
Main PID: 1599 (firewalld)
CGroup: /system.slice/firewalld.service
└─1599 /usr/bin/python2 -Es /usr/sbin/firewalld --nofork --nopid
May 04 13:11:27 splunk_master systemd[1]: Starting firewalld - dynamic firewall daemon...
May 04 13:11:27 splunk_master systemd[1]: Started firewalld - dynamic firewall daemon.
May 04 13:11:27 splunk_master firewalld[1599]: WARNING: AllowZoneDrifting is enabled. This is considered an insecure configuration option. It will be removed in a future release. Please consider disabling it now.
[root@splunk_master opt]#

1.1.2 添加防火墙Splunk策略

节点角色 端口 描述
Search Head 8000 SplunkWeb服务端口
Search Head 8089 Splunk管理端口
Search Head 34567 搜索头集群数据同步端口
Cluster Manages 8000 SplunkWeb服务端口
Cluster Manages 8089 Splunk管理端口
Indexers 9997 Splunk数据接收端口
Indexers 8089 Splunk管理端口
Indexers 9887 Splunk索引器集群数据同步端口
1
2
3
4
5
6
7
8
9
# Cluster Manages
firewall-cmd --zone=public --add-port=8000/tcp --add-port=8089/tcp --permanent
# Indexers
firewall-cmd --zone=public --add-port=9997/tcp --add-port=8089/tcp --add-port=9887/tcp --permanent
# Search Head
firewall-cmd --zone=public --add-port=8000/tcp --add-port=8089/tcp --add-port=34567/tcp --permanent

firewall-cmd --reload # 防火墙配置重新加载
firewall-cmd --list-port # 显示启用的端口

1.2 创建splunk用户

1
2
$ groupadd splunk  # 创建splunk用户组
$ useradd -m -g splunk splunk # 创建splunk用户,并指定用户组为splunk

1.3 Splunk Indexers 磁盘挂载(方法一)

注意事项:此步骤只需要在Indexers服务器上操作,SearchHead/Heavy Forward/Master服务器上不需要操作!!!

说明:硬盘分为SSD和HDD两种

SSD硬盘:建议用于保存热、温数据数据模型数据。

HDD硬盘:建议用于保存冷数据桶冻结数据

1.3.1 创建/splunk_db和/splunk_colddb目录

/splunk_db > 保存热温数据

/splunk_colddb > 保存冷数据

1
2
$ mkdir /splunk_db  # 创建热温数据桶保存目录
$ mkdir /splunk_colddb # 创建冷数据桶保存目录

1.3.2 挂载vdc磁盘(ssd)到/splunk_db

1.3.2.1 查看vdc磁盘(ssd)信息
1
2
3
4
5
6
7
8
9
10
11
12
13
[root@splunk-index01 ~]# fdisk –l
......
Disk /dev/vdc: 536.9 GB, 536870912000 bytes, 1048576000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/vdd: 536.9 GB, 536870912000 bytes, 1048576000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
[root@splunk-index01 ~]#

vdc磁盘为SSD磁盘

vdd为hdd磁盘

1.3.2.2 给vdc磁盘创建vdc1分区
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
[root@splunk-index01 ~]# fdisk /dev/vdc  # 开始给/dev/vdc设置分区 
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.
Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x4d8b4ba7.
Command (m for help): n # 新建磁盘分区
Partition type: p
primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p # 添加主分区
Partition number (1-4, default 1): 1 # 设置分区号
First sector (2048-1048575999, default 2048): # 设置分区起始大小,默认一直回车
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-1048575999, default 1048575999): # 设置分区结束大小,默认一直回车

Using default value 1048575999
Partition 1 of type Linux and of size 500 GiB is set

Command (m for help): p # 显示分区信息

Disk /dev/vdc: 536.9 GB, 536870912000 bytes, 1048576000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x4d8b4ba7

Device Boot Start End Blocks Id System

/dev/vdc1 2048 1048575999 524286976 83 Linux

Command (m for help): w # 保存分区信息并退出
The partition table has been altered!
Calling ioctl() to re-read partition table.
Syncing disks.
[root@splunk-index01 ~]#

查看新区,会生产一个/dev/sdb1的新分区

1
fdisk -l 

对新建磁盘分区进行格式化,分区类型为“ext4”

1
mkfs -t ext4 /dev/vdc1 
1.3.2.3 挂载vdc磁盘,并设置开机自动挂载
1
2
[root@splunk-index01 ~]# mount /dev/vdc1 /splunk_db
# 将/dev/vdc1分区挂载到/splunk_colddb

编辑/etc/fstab文件添加以下内容:

/dev/vdc1 /splunk_db ext4 defaults 0 0

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[root@splunk-index01 ~]# vim /etc/fstab
[root@splunk-index01 ~]# cat /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Mar 24 08:18:05 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/centos-root / xfs defaults 0 0
UUID=36e55c09-7c6e-4a00-b168-5ffb298638d9 /boot xfs defaults 0 0
/dev/mapper/centos-swap swap swap defaults 0 0
/dev/vdc1 /splunk_db ext4 defaults 0 0
/dev/vdd1 /splunk_colddb ext4 defaults 0 0
[root@splunk-index01 ~]#
1.3.2.4 查看磁盘挂载情况
1
2
3
4
5
6
7
8
9
10
11
12
[root@splunk-index01 ~]# df - h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 7.8G 0 7.8G 0% /dev
tmpfs 7.8G 0 7.8G 0% /dev/shm
tmpfs 7.8G 26M 7.8G 1% /run
tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup
/dev/mapper/centos-root 8.0G 2.2G 5.9G 27% /
/dev/vda1 1014M 181M 834M 18% /boot
tmpfs 1.6G 0 1.6G 0% /run/user/0
/dev/vdc1 493G 73M 467G 1% /splunk_db
/dev/vdd1 493G 73M 467G 1% /splunk_colddb

1.3.3 挂载vdd磁盘(hdd)到/splunk_colddb

1.3.3.1 查看vdd磁盘(hdd)信息
1
2
3
4
5
6
7
8
9
10
11
12
13
[root@splunk-index01 ~]# fdisk -l
......
Disk /dev/vdc: 536.9 GB, 536870912000 bytes, 1048576000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/vdd: 536.9 GB, 536870912000 bytes, 1048576000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
[root@splunk-index01 ~]#

vdc磁盘为SSD磁盘,vdd为hdd磁盘。

1.3.3.2 给vdd磁盘创建vdd1分区
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
[root@splunk-index01 ~]# fdisk /dev/vdd # 设置/dev/vdd磁盘
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

Device does not contain a recognized partition table
Building a new DOS disklabel with disk identifier 0x4d8b4ba7.

Command (m for help): n # 新建磁盘分区
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p # 添加主分区
Partition number (1-4, default 1): 1 # 设置分区号
First sector (2048-1048575999, default 2048): # 设置分区起始大小,默认一直回车
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-1048575999, default 1048575999): # 设置分区结束大小,默认一直回车
Using default value 1048575999
Partition 1 of type Linux and of size 500 GiB is set

Command (m for help): p # 显示分区信息

Disk /dev/vdd: 536.9 GB, 536870912000 bytes, 1048576000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x4d8b4ba7

Device Boot Start End Blocks Id System

/dev/vdd1 2048 1048575999 524286976 83 Linux

Command (m for help): w # 保存分区信息并退出

The partition table has been altered!

Calling ioctl() to re-read partition table.
Syncing disks.
[root@splunk-index01 ~]#

查看新区,会生产一个/dev/sdd1的新分区

1
fdisk -l 

对新建磁盘分区进行格式化,分区类型为“ext4”

1
$ mkfs -t ext4 /dev/vdd1
1.3.3.3 挂载vdd磁盘,并设置开机自动挂载
1
2
[root@splunk-index01 ~]# mount /dev/vdd1 /splunk_colddb
# 将/dev/vdd1分区挂载到/splunk_colddb

编辑/etc/fstab文件添加以下内容:

/dev/vdd1 /splunk_colddb ext4 defaults 0 0

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
[root@splunk-index01 ~]# vim /etc/fstab
[root@splunk-index01 ~]# cat /etc/fstab
#
# /etc/fstab
# Created by anaconda on Tue Mar 24 08:18:05 2020
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/centos-root / xfs defaults 0 0
UUID=36e55c09-7c6e-4a00-b168-5ffb298638d9 /boot xfs defaults 0 0
/dev/mapper/centos-swap swap swap defaults 0 0
/dev/vdc1 /splunk_db ext4 defaults 0 0
/dev/vdd1 /splunk_colddb ext4 defaults 0 0
#
1.3.3.4 查看磁盘挂载情况
1
2
3
4
5
6
7
8
9
10
11
[root@splunk-index01 ~]# df - h
Filesystem Size Used Avail Use% Mounted on
devtmpfs 7.8G 0 7.8G 0% /dev
tmpfs 7.8G 0 7.8G 0% /dev/shm
tmpfs 7.8G 26M 7.8G 1% /run
tmpfs 7.8G 0 7.8G 0% /sys/fs/cgroup
/dev/mapper/centos-root 8.0G 2.2G 5.9G 27% /
/dev/vda1 1014M 181M 834M 18% /boot
tmpfs 1.6G 0 1.6G 0% /run/user/0
/dev/vdc1 493G 73M 467G 1% /splunk_db
/dev/vdd1 493G 73M 467G 1% /splunk_colddb

1.3.4 将/splunk_db、/splunk_colddb属主,属组设置成splunk

1
chown -R splunk:splunk /splunk* 

1.4 Splunk Indexers 磁盘挂载(方法二)

1.4.1 创建/data目录

1
2
3
[root@splunk_master opt]# mkdir /data
[root@splunk_master opt]# ll /|grep data
drwxr-xr-x. 2 root root 6 May 4 13:23 data

1.4.2 挂载硬盘到/data目录

1
2
3
4
$ fdisk /dev/vdd
$ mkfs -t ext4 /dev/vdd1
$ echo "/dev/vdd1 /data ext4 defaults 0 0" >> /etc/fstab
$ mount /dev/vdd /data

1.4.3 将/data目录属主和属组设置成splunk

1
2
3
4
[root@splunk_master opt]# chown -R splunk:splunk /data/
[root@splunk_master opt]# ll /|grep data
drwxr-xr-x. 2 splunk splunk 6 May 4 13:23 data
[root@splunk_master opt]#

1.5 调整系统配置

1.5.1 禁用THP<透明大页>

编辑/etc/rc.d/rc.local配置文件,在文件底部添加一下配置

redhat系的操作系统transparent_hugepage目录可能为redhat_transparent_hugepage

1
2
3
4
5
6
if test -f /sys/kernel/mm/transparent_hugepage/enabled; then
echo never > /sys/kernel/mm/transparent_hugepage/enabled
fi
if test -f /sys/kernel/mm/transparent_hugepage/defrag; then
echo never > /sys/kernel/mm/transparent_hugepage/defrag
fi

给/etc/rc.d/rc.local配置文件添加执行权限并执行

1
2
3
4
[root@splunk-search01 opt]# chmod +x /etc/rc.d/rc.local 
[root@splunk-search01 opt]# ll /etc/rc.d/rc.local
-rwxr-xr-x 1 root root 716 Mar 31 16:02 /etc/rc.d/rc.local
[root@splunk-search01 opt]# /etc/rc.d/rc.local

1.5.2 调整操作系统的资源限制

查看操作系统资源限制

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
$ ulimit -a 
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 7902
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 7902
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

修改/etc/security/limits.conf添加以下内容

1
2
3
4
5
* hard nofile 64000
* soft nofile 64000
* hard nproc 10240
* soft nproc 10240
* hard fsize -1
1
2
3
4
5
echo "* hard nofile 64000
* soft nofile 64000
* hard nproc 10240
* soft nproc 10240
* hard fsize -1" >> /etc/security/limits.conf

1.6 检查服务器网络联通性

1
$ ping xxx.xxx.xxx.xxx

2 部署Splunk Indexer Cluster

Splunk Enterprise Version:8.2.6

2.1 下载Splunk Enterprise安装包

1
$ wget -o /opt/splunk-8.2.6-a6fe1ee8894b-Linux-x86_64.tgz https://download.splunk.com/products/splunk/releases/8.2.6/linux/splunk-8.2.6-a6fe1ee8894b-Linux-x86_64.tgz

2.2 安装Splunk

切换至Splunk用户,在各节点服务器上安装Splunk

解压Splunk安装包

1
2
3
4
5
6
$ tar -zxvf splunk-8.2.6-a6fe1ee8894b-Linux-x86_64.tgz -C /opt
$ chown -R splunk:splunk splunk
$ ll
total 539596
drwxr-xr-x. 9 splunk splunk 237 Mar 23 21:36 splunk
-rw-r--r--. 1 root root 552543435 May 4 13:05 splunk-8.2.6-a6fe1ee8894b-Linux-x86_64.tgz

启动Splunk服务

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
$ su splunk  # 切换到splunk用户
$ /opt/splunk/bin/splunk start --accept-license # 启动splunk服务,并且同意license

This appears to be your first time running this version of Splunk.

Splunk software must create an administrator account during startup. Otherwise, you cannot log in.
Create credentials for the administrator account.
Characters do not appear on the screen when you type in credentials.

Please enter an administrator username: admin
Password must contain at least:
* 8 total printable ASCII character(s).
# 设置密码
Please enter a new password:
Please confirm new password:
---- 省略 ----
All preliminary checks passed.

Starting splunk server daemon (splunkd)...
Generating a RSA private key
....+++++
..........................+++++
writing new private key to 'privKeySecure.pem'
-----
Signature ok
subject=/CN=splunk_idx01/O=SplunkUser
Getting CA Private Key
writing RSA key
Done
[ OK ]

Waiting for web server at http://127.0.0.1:8000 to be available.......... Done


If you get stuck, we're here to help.
Look for answers here: http://docs.splunk.com

The Splunk web interface is at http://splunk_idx01:8000

设置开机启动

1
2
3
$ sudo /opt/splunk/bin/splunk enable boot-start --user splunk
Init script installed at /etc/init.d/splunk.
Init script is configured to run at boot.

修改.bash_profile

编辑~/.bash_profile文件,添加splunk路径配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
$ vim ~/.bash_profile
$ source ~/.bash_profile # 刷新.bash_profile
$ cat ~/.bash_profile
# .bash_profile

# Get the aliases and functions
if [ -f ~/.bashrc ]; then
. ~/.bashrc
fi

# User specific environment and startup programs

export SPLUNK_HOME="/opt/splunk"
export PATH=$PATH:$HOME/.local/bin:$HOME/bin:$SPLUNK_HOME/bin

2.3 修改Splunk主机名/服务器名

1
2
/opt/splunk/bin/splunk set servername splunk-manages
/opt/splunk/bin/splunk set default-hostname splunk-manages

2.4 Cluster Manages 初始化

在 Cluster Manages 节点上执行下列命令,完成 Cluster Manages节点初始化。

1
2
$SPLUNK_HOME/bin/splunk edit cluster-config -mode master -replication_factor 3 -search_factor 2 -secret splunk3du -cluster_label eccom_splunk
$SPLUNK_HOME/bin/splunk restart

2.5 添加 Indexers 节点

在每台 Indexers 节点上执行下列命令,完成 Indexers 添加到集群中。

1
2
$SPLUNK_HOME/bin/splunk edit cluster-config -mode slave -master_uri https://[CM-ip]:8089 -replication_port 9887 -secret splunk3du
$SPLUNK_HOME/bin/splunk restart

2.7 部署Search Head Cluster

2.7.1 设置Search Head Cluster Deployer角色

编辑Deployer节点上的$SPLUNK_HOME/etc/local/system/server.conf配置文件,添加下述配置内容:

1
2
3
[shclustering]
pass4SymmKey = shc # Search Head集群密钥
shcluster_label = shc # SearchHead集群标签

重启Deployer节点Splunk服务

1
$SPLUNK_HOME/bin/splunk restart

2.7.2 搭建 Search Head Cluster Cluster

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# 初始化集群成员 SH01
/opt/splunk/bin/splunk init shcluster-config -auth admin:admin@123 -mgmt_uri https://[SH01]:8089 -replication_port 8292 -replication_factor 3 -conf_deploy_fetch_url https://[Deployer]:8089 -secret shc -shcluster_label shc

# 初始化集群成员 SH02
/opt/splunk/bin/splunk init shcluster-config -auth admin:admin@123 -mgmt_uri https://[SH02]:8089 -replication_port 8292 -replication_factor 3 -conf_deploy_fetch_url https://[Deployer]:8089 -secret shc -shcluster_label shc

# 初始化集群成员 SH03
/opt/splunk/bin/splunk init shcluster-config -auth admin:admin@123 -mgmt_uri https://[SH03]:8089 -replication_port 8292 -replication_factor 3 -conf_deploy_fetch_url https://[Deployer]:8089 -secret shc -shcluster_label shc

# 重启搜索头集群SH01-SH03成员Splunk服务
/opt/splunk/bin/splunk restart

# 设置搜索头集群队长
/opt/splunk/bin/splunk bootstrap shcluster-captain -servers_list "https://[SH01]:8089,https://[SH02]:8089,https://[SH03]:8089" -auth admin:admin@123

2.7.3 Deployer节点推送给Search Head Cluster Apps

1
2
# Deployer 节点角色下推Apps
/opt/splunk/bin/splunk apply shcluster-bundle --answer-yes -target https://[SH01]:8089 -auth admin:admin@123

2.8 将Search Head Cluster 与 Indexers Cluster集成

将每个搜索头群集成员配置为索引器群集上的搜索头。使用 CLI splunk edit cluster-config 命令在每台 Search Head 节点上执行下列命令,完成 Search Head 添加到集群中。

1
2
$SPLUNK_HOME/bin/splunk edit cluster-config -mode searchhead -master_uri https://[CM-ip]:8089 -secret splunk3du
$SPLUNK_HOME/bin/splunk restart

2.9 初始化集群配置

2.9.1 关闭idx web和打开9997数据接收端口

对象节点:Cluster Manages

进入$SPLUNK_HOME/etc/manager-apps目录,创建org_all_indexer_base目录和创建其他的配置文件。

1
2
3
4
$ cd $SPLUNK_HOME/etc/manager-apps
$ mkdir org_all_indexer_base
$ mkdir -p org_all_indexer_base/local
$ mkdir -p org_all_indexer_base/metadata

local/app.conf设置Apps相关属性

1
2
3
4
5
6
7
8
9
[install]
state = enabled

[package]
check_for_updates = false

[ui]
is_visible = false
is_manageable = false

local/inputs.conf开启9997端口

1
[splunktcp://9997]

local/web.conf禁用web页面

1
2
[settings]
startwebserver = 0

2.9.2 设置license

对象节点:Cluster Manages

进入$SPLUNK_HOME/etc/manager-apps目录,创建org_all_license_server目录和创建其他的配置文件。

1
2
3
4
$ cd $SPLUNK_HOME/etc/manager-apps
$ mkdir org_all_license_server
$ mkdir -p org_all_license_server/local
$ mkdir -p org_all_license_server/metadata

local/app.conf设置Apps相关属性

1
2
3
4
5
6
7
8
9
[install]
state = enabled

[package]
check_for_updates = false

[ui]
is_visible = false
is_manageable = false

local/server.conf设置license

1
2
[license]
master_uri = https://[CM]:8089

2.9.3 标注化 indexes.conf 添加volume

对象节点:Cluster Manages

进入$SPLUNK_HOME/etc/manager-apps目录,创建org_all_indexes目录和创建其他的配置文件。

1
2
3
4
$ cd $SPLUNK_HOME/etc/manager-apps
$ mkdir org_all_indexes
$ mkdir -p org_all_indexes/local
$ mkdir -p org_all_indexes/metadata

local/indexes.conf添加volume配置

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
# [default]
# [volume:primary]
# path = /opt/splunk/var/lib/splunk
# maxVolumeDataSizeMB = 500000

# Two volumes for a "tiered storage" solution--fast and slow disk.
[volume:home]
path = /data/splunk_db
# 250GB
maxVolumeDataSizeMB = 256000

# Longer term storage on slower disk.
[volume:cold]
path = /data/splunk_colddb
# 5TB with some headroom leftover (data summaries, etc)
maxVolumeDataSizeMB = 4600000

[main]
homePath = volume:home/defaultdb/db
coldPath = volume:cold/defaultdb/colddb
thawedPath = $SPLUNK_DB/defaultdb/thaweddb

[history]
homePath = volume:home/historydb/db
coldPath = volume:cold/historydb/colddb
thawedPath = $SPLUNK_DB/historydb/thaweddb

[summary]
homePath = volume:home/summarydb/db
coldPath = volume:cold/summarydb/colddb
thawedPath = $SPLUNK_DB/summarydb/thaweddb

[_internal]
homePath = volume:home/_internaldb/db
coldPath = volume:cold/_internaldb/colddb
thawedPath = $SPLUNK_DB/_internaldb/thaweddb

# For version 6.1 and higher
[_introspection]
homePath = volume:home/_introspection/db
coldPath = volume:cold/_introspection/colddb
thawedPath = $SPLUNK_DB/_introspection/thaweddb

# For version 6.5 and higher
[_telemetry]
homePath = volume:home/_telemetry/db
coldPath = volume:cold/_telemetry/colddb
thawedPath = $SPLUNK_DB/_telemetry/thaweddb

[_audit]
homePath = volume:home/audit/db
coldPath = volume:cold/audit/colddb
thawedPath = $SPLUNK_DB/audit/thaweddb

[_thefishbucket]
homePath = volume:home/fishbucket/db
coldPath = volume:cold/fishbucket/colddb
thawedPath = $SPLUNK_DB/fishbucket/thaweddb

2.9.4 设置 indexers discovery

对象节点:Cluster Manages

开启Splunk 索引器集群发现功能。编辑$SPLUNK_HOME/etc/system/local/server.conf配置文件,添加以下配置内容:

1
2
[indexer_discovery] 
pass4SymmKey = idxdiscovery # 索引器集群发现密钥

2.9.5 将最新的配置下发至 Indexers 集群

1
2
3
$SPLUNK_HOME/bin/splunk validate cluster-bundle --check-restart
$SPLUNK_HOME/bin/splunk apply cluster-bundle --answer-yes
$SPLUNK_HOME/bin/splunk show cluster-bundle-status

3 配置MC(Monitor Console)

如果需要在Cluster Manages上配置Monitor Console功能,需要将Search Head配置为搜索节点。通过下列命令进行添加:

1
$SPLUNK_HOME/bin/splunk add search-server https://[SH-ip]:8089 -auth admin:splunk3du -remoteUsername admin -remotePassword splunk3du

在Cluster Manages节点上打开分布式监控进行配置

附录:集群的维护参考相关命令

  • Use the CLI to validate the bundle and check restart

https://docs.splunk.com/Documentation/Splunk/latest/Indexer/Updatepeerconfigurations

1
2
3
$SPLUNK_HOME/bin/splunk validate cluster-bundle
$SPLUNK_HOME/bin/splunk validate cluster-bundle --check-restart
$SPLUNK_HOME/bin/splunk show cluster-bundle-status
  • Use the CLI to apply the bundle
1
2
$SPLUNK_HOME/bin/splunk apply cluster-bundle
$SPLUNK_HOME/bin/splunk apply cluster-bundle --answer-yes
  • Rollback the configuration bundle using the CLI
1
$SPLUNK_HOME/bin/splunk rollback cluster-bundle

【实战】Splunk Indexer Cluster部署
https://hesc.info/864c9ef9fa35/
作者
需要哈气的纸飞机
发布于
2022年5月26日
许可协议