背景

TiDB集群安装部署很简便，依靠tiup命令，可以实现部署、扩容、缩容、修改参数等；但是集群部署所需要的deploy.yaml文件需要手工编写。对于大规模的集群来说，编写deploy.yaml文件耗费时间还是很多的，尤其是几十台甚至上百台机器，编写配置文件所需要的工作量会成倍增加。

基于大规模集群编写配置文件的痛点，若有快速生成集群配置文件的工具，会给部署集群带来很大的便利，让部署TiDB集群变得更easy。

由于部署集群前，负责硬件的同事会提供一个机器IP的基本信息表，利用该表格的信息，在添加一些部署集群所需的信息，自动生成deploy.yaml文件。我把我的想法跟马学涛（github：dxhmxt）老师沟通了一下，他觉得这个想法不错，就利用自己休息时间，写了一个快速部署工具tidbdeploy-v1.0.4.tar。

部署工具介绍

TiDB部署助手工具用于TiDB for OP离线部署，基于tiup和linux expect在部署节点（Tiup节点）执行文本菜单数字选项完成集群部署工作。TiDB部署助手工具对设置root用户和创建实例用户及其互信设置、初始化系统参数、挂载数据盘、编辑部署yaml文件、部署集群、设置数据库root密码、初始化数据库参数部署工作逻辑进行了封装。优化了重复执行命令、编辑yaml文件工作，可帮助大家提升部署效率，尤其是大规模、多套集群部署场景。

TiDB部署助手工具实现功能有：

1）快速生成集群部署所需的配置文件；

2）根据集群拓扑规划文件，配置安装用户之间的互信、并进行互信结果检查；

3）读取集群部署初始配置文件和集群拓扑规划文件快速完成整个集群部署系统参数初始化；

4）读取集群拓扑规划文件对不同类型节点（tidb节点和非tidb节点）进行数据盘初始化和挂载；

5）读取集群部署初始配置文件和集群集群拓扑规划文件，结合linux expect快速完成用户创建和互信设置及设置互信后结果检查；

6）读取集群拓扑规划文件，实现两个功能：a.快速生成host和role对应yaml格式文件、遍历采集服务器CPU和MEM配置匹配集群拓扑规划文件生成动态参数值，最后生成部署yaml文件；b.快速生成相同架构规划yaml文件；（尤其同一个系统部署多套相关配置和规划的环境）；

7）读取集群部署初始配置文件和yaml文件，解压安装介质包进行部署集群；

8）安装mysql客户端命令包（mariadb）；

9）编辑和读取数据库变量参数文件，对变量参数进行设置和对设置结果进行检查。

TiDB部署助手工具目录结构

工具下载链接：https://github.com/dxhmxt/TiDB-deploy-tool

部署工具使用手册

部署工具安装

在tiup节点root用户/root/目录下，执行以下部署命令：

tar zxf tidbdeploy-v1.x.x.tar sh/root/tidbdeploy/scripts/00_tidb_deploy_text_menu.sh

输入菜单数字，进入编辑集群部署初始配置文件菜单

____________________________________________________ | TiDB部署工具 | |===================部署相关步骤=======================| | 1.编辑集群部署相关配置文件 | | 2.设置系统root用户互信 | | 3.初始化操作系统参数 | | 4.挂载数据盘 | | 5.创建集群实例用户和设置互信 | | 6.生成yaml配置文件 | | 7.部署集群 | | 8.设置数据库root用户密码 | | 9.初始化数据库参数 | | 10.TiDB部署工具介绍和部署说明 | |————————————————————————————————————————————————————| | 99.退出 | | 请输入选项： | |____________________________________________________| 1 |—————————————————————————————————————————————————| | 1.编辑集群部署相关配置文件 | |—————————————————————————————————————————————————| | 1.1 编辑集群部署初始配置文件 | | 1.2 集群拓扑规划模板1-最小配置 | | 1.3 集群拓扑规划模板2-简单配置 | | 1.4 集群拓扑规划模板3-详细配置 | | 1.5 编辑集群拓扑规划文件 | | 1.6 快速生成相同架构集群拓扑规划文件 | | 1.7 编辑集群ip列表 | |—————————————————————————————————————————————————| | 99.退出 | | 请输入选项： | |_________________________________________________|

编辑集群配置文件

接上面菜单，选 1，把集群的基本信息输入进去：

编辑集群部署初始配置文件 [y/n]y 你选择了 是 编辑集群初始配置文件 Vim: 警告: 输出不是到终端(屏幕) #实例用户 tidbInstanceUser=tidb tidbInstanceUserPassword=123456 dirHome=/data1/tidb #ssh端口，支持非默认端口，如36000 sshPort=22 #集群规划信息，arch为amd64或者arm64 version=v6.5.4 arch=amd64 clusterName=tidb-test #部署并发，默认的是5，根据集群规模可调整 concurrencyint=10 #介质名称 serverPackage=tidb-community-server-v6.5.4-linux-amd64.tar.gztoolkitPackage=tidb-community-toolkit-v6.5.4-linux-amd64.tar.gz #变量communityOrEnterpriseVersion表示安装包是community还是enterprise版本 communityOrEnterpriseVersion=community #tidb database 用户 tidbDatabaseUserRoot=root tidbDatabaseUserRootPassword=passwd #grafana user passwd设置 grafanaUserPassword=passwd ~

快速生成部署文件

这里拿简单配置做介绍：

|—————————————————————————————————————————————————| | 1.编辑集群部署相关配置文件 | |—————————————————————————————————————————————————| | 1.1 编辑集群部署初始配置文件 | | 1.2 集群拓扑规划模板1-最小配置 | | 1.3 集群拓扑规划模板2-简单配置 | | 1.4 集群拓扑规划模板3-详细配置 | | 1.5 编辑集群拓扑规划文件 | | 1.6 快速生成相同架构集群拓扑规划文件 | | 1.7 编辑集群ip列表 | |—————————————————————————————————————————————————| | 99.退出 | | 请输入选项： | |_________________________________________________| 输入 1.3 集群拓扑规划模板2 [y/n] y 输出如下： #模板2 #注意每个role的Port1、Port2、Data Dir、Deploy Dir四个要素必须填完整；例如，部分role端口和目录不涉及的用"-"填充;tiflash因为默认端口多，采用一个Host部署一个节点，采用"-"填充。 #Host Role Port1 Port2 Data Dir Deploy Dir 192.168.1.77 pd2379 2380 /data1/tidb-data/pd-2379 /data1/tidb-deploy/pd-2379 192.168.1.88 pd 2379 2380 /data1/tidb-data/pd-2379 /data1/tidb-deploy/pd-2379 192.168.1.99 pd 2379 2380 /data1/tidb-data/pd-2379 /data1/tidb-deploy/pd-2379 192.168.1.11 tikv 20160 20180 /data1/tidb-data/tikv-20160 /data1/tidb-deploy/tikv-20160 192.168.1.11 tikv 20161 20181 /data1/tidb-data/tikv-20161 /data1/tidb-deploy/tikv-20161 192.168.1.22 tikv 20160 20180 /data1/tidb-data/tikv-20160 /data1/tidb-deploy/tikv-20160 192.168.1.22 tikv 20161 20181 /data1/tidb-data/tikv-20161 /data1/tidb-deploy/tikv-20161 192.168.1.12 tidb 4000 10080- /data1/tidb-deploy/tidb-4000192.168.1.12 tidb 4001 10081 - /data1/tidb-deploy/tidb-4001 192.168.1.14 drainer 8249 - /data1/tidb-data/drainer-8250 /data1/tidb-deploy/drainer-8250 192.168.1.15 pump 8250 - /data1/tidb-data/pump-8250 /data1/tidb-deploy/pump-8250 192.168.1.16 cdc 8300- /data1/tidb-data/cdc-8300 /data1/tidb-deploy/cdc-8300192.168.1.19 prometheus 9090 12020 /data1/tidb-data/prometheus-9090 /data1/tidb-data/prometheus-9090 192.168.1.19 grafana 3000 - - /data1/tidb-deploy/grafana-3000 192.168.1.17 tispark_masters 7077 8080 - /data1/tidb-deploy/tispark_masters-7077 192.168.1.17 tispark_workers 7078 8081- /data1/tidb-deploy/tispark_masters-7078192.168.1.66 tiflash - - /data1/tidb-data/tiflash-9000 /data1/tidb-deploy/tiflash-9000 192.168.1.19 alertmanager 9093 9394 /data1/tidb-data/alertmanager-9093 /data1/tidb-deploy/alertmanager-9093 #模板2 转换成yaml格式结果 pd_servers: - host: 192.168.1.77 client_port: 2379peer_port:2380 deploy_dir: /data1/tidb-deploy/pd-2379 data_dir: /data1/tidb-data/pd-2379 - host: 192.168.1.88 client_port: 2379 peer_port: 2380 deploy_dir: /data1/tidb-deploy/pd-2379 data_dir: /data1/tidb-data/pd-2379 - host: 192.168.1.99 client_port: 2379 peer_port: 2380 deploy_dir: /data1/tidb-deploy/pd-2379 data_dir: /data1/tidb-data/pd-2379 tikv_servers: - host: 192.168.1.11 port: 20160 status_port: 20180deploy_dir: /data1/tidb-deploy/tikv-20160 data_dir: /data1/tidb-data/tikv-20160 - host:192.168.1.11 port: 20161 status_port: 20181 deploy_dir: /data1/tidb-deploy/tikv-20161 data_dir: /data1/tidb-data/tikv-20161 - host: 192.168.1.22 port: 20160 status_port: 20180 deploy_dir: /data1/tidb-deploy/tikv-20160 data_dir: /data1/tidb-data/tikv-20160 - host: 192.168.1.22 port: 20161 status_port: 20181deploy_dir: /data1/tidb-deploy/tikv-20161 data_dir: /data1/tidb-data/tikv-20161 tidb_servers: - host:192.168.1.12 port: 4000 status_port: 10080 deploy_dir: /data1/tidb-deploy/tidb-4000 - host: 192.168.1.12 port: 4001 status_port: 10081 deploy_dir: /data1/tidb-deploy/tidb-4001 drainer_servers: - host: 192.168.1.14 port: 8249 deploy_dir: /data1/tidb-deploy/drainer-8250 data_dir: /data1/tidb-data/drainer-8250 config: syncer.db-type: file syncer.to.retention-time: 2syncer.to.read-timeout: 24h pump_servers: - host:192.168.1.15 port: 8250 deploy_dir: /data1/tidb-deploy/pump-8250 data_dir: /data1/tidb-data/pump-8250 cdc_servers: - host: 192.168.1.16 port: 8300deploy_dir: /data1/tidb-deploy/cdc-8300 data_dir: /data1/tidb-data/cdc-8300 monitoring_servers: - host:192.168.1.19 port: 9090 deploy_dir: /data1/tidb-data/prometheus-9090 data_dir: /data1/tidb-data/prometheus-9090 grafana_servers: - host: 192.168.1.19 port:3000 deploy_dir: /data1/tidb-deploy/grafana-3000 username: admin password: admin tispark_masters: - host: 192.168.1.17 port: 7077 web_port: 8080deploy_dir: /data1/tidb-deploy/tispark_masters-7077 tispark_workers: - host:192.168.1.17 port: 7078 web_port: 8081 deploy_dir: /data1/tidb-deploy/tispark_masters-7078 tiflash_servers: - host: 192.168.1.66 deploy_dir: /data1/tidb-deploy/tiflash-9000 data_dir: /data1/tidb-data/tiflash-9000 alertmanager_servers: - host:192.168.1.19 web_port: 9093 cluster_port: 9394 deploy_dir: /data1/tidb-deploy/alertmanager-9093 data_dir: /data1/tidb-data/alertmanager-9093 输入任意键继续

修改操作系统配置文件

修改sysctl.conf、limits.conf等配置文件

____________________________________________________ | TiDB部署工具 | |===================部署相关步骤=======================| | 1.编辑集群部署相关配置文件 | | 2.设置系统root用户互信 | | 3.初始化操作系统参数 | | 4.挂载数据盘 | | 5.创建集群实例用户和设置互信 | | 6.生成yaml配置文件 | | 7.部署集群 | | 8.设置数据库root用户密码 | | 9.初始化数据库参数 | | 10.TiDB部署工具介绍和部署说明 | |————————————————————————————————————————————————————| | 99.退出 | | 请输入选项： | |____________________________________________________| 输入 3 ##cluster instance user is tidb. ###set 172.16.4.126 optimal parameters.### vm.swappiness need to delete,and new add vm.swappiness = 0. delete vm.swappiness through sedmatching. delete vm.swappiness done. startingset disableswap. disableswap set done. selinuxstate check and disable set. Disabled selinux state is disabled. starting set kernelset. kernelset setdone. tidb limitsset is set. startingset limitsset. limitsset set done. starting set tidbInstanceUsersudoset. tidb sudo grant is set. tidbInstanceUsersudoset setdone. this is not kylin-release system. will be cpup and thp set. backup /etc/tuned/balanced-tidb-optimal/tuned.conf to /etc/tuned/balanced-tidb-optimal/tuned.conf.bak. machinetype check. machine type is virtual machine. virtual machine thp setdone,TIPS restart the system setting takes effect.#############start check os optimal parameters set############ ##check swappinessdisableset. vm.swappiness = 0 ##check seliunxdisableset. Disabled ##check kernelset. fs.file-max = 1000000 net.core.somaxconn = 32768 net.ipv4.tcp_tw_recycle = 0 net.ipv4.tcp_syncookies = 0 vm.overcommit_memory = 1 ##check limitsset.tidb soft nofile1000000 tidb hard nofile 1000000 tidb soft stack 32768 tidb hard stack 32768 ##check tidbInstanceUsersudoset. tidb ALL=(ALL) NOPASSWD: ALL ##check cpuper_and_thp_set. [main] include=balanced [vm] transparent_hugepage=never ###set 172.16.4.163 optimal parameters.### vm.swappiness need to delete,and new add vm.swappiness = 0. delete vm.swappiness through sed matching. delete vm.swappiness done. starting set disableswap. disableswap setdone. selinuxstate check and disable set. Disabled selinux state is disabled. startingset kernelset. kernelset set done. tidb limitsset is set. starting set limitsset. limitsset set done. starting set tidbInstanceUsersudoset. tidb sudogrant is set. tidbInstanceUsersudosetset done. this is not kylin-release system. will be cpup and thp set. backup /etc/tuned/balanced-tidb-optimal/tuned.conf to /etc/tuned/balanced-tidb-optimal/tuned.conf.bak. machine type check. machine typeis virtual machine. virtual machine thpset done,TIPS restart the system setting takes effect. #############start check os optimal parameters set############ ##check swappinessdisableset. vm.swappiness = 0 ##check seliunxdisableset. Disabled ##check kernelset. fs.file-max = 1000000 net.core.somaxconn = 32768 net.ipv4.tcp_tw_recycle = 0 net.ipv4.tcp_syncookies = 0 vm.overcommit_memory = 1 ##check limitsset. tidb soft nofile 1000000 tidb hard nofile 1000000 tidb soft stack 32768 tidb hard stack 32768 ##check tidbInstanceUsersudoset. tidb ALL=(ALL) NOPASSWD: ALL ##check cpuper_and_thp_set. [main] include=balanced [vm] transparent_hugepage=never ###set 172.16.4.169 optimal parameters.### vm.swappiness need to delete,and new add vm.swappiness = 0. delete vm.swappiness through sed matching. delete vm.swappiness done. starting set disableswap. disableswap setdone. selinuxstate check and disable set. Disabled selinux state is disabled. startingset kernelset. kernelset set done. tidb limitsset is set. starting set limitsset. limitsset setdone. startingset tidbInstanceUsersudoset. tidb sudo grant is set. tidbInstanceUsersudoset setdone. this is not kylin-release system. will be cpup and thp set. backup /etc/tuned/balanced-tidb-optimal/tuned.conf to /etc/tuned/balanced-tidb-optimal/tuned.conf.bak. machinetype check. machine type is virtual machine. virtual machine thp set done,TIPS restart the system setting takes effect. #############start check os optimal parameters set############ ##check swappinessdisableset. vm.swappiness = 0 ##check seliunxdisableset. Disabled ##check kernelset. fs.file-max = 1000000 net.core.somaxconn = 32768 net.ipv4.tcp_tw_recycle = 0 net.ipv4.tcp_syncookies = 0 vm.overcommit_memory = 1 ##check limitsset. tidb soft nofile 1000000 tidb hard nofile 1000000 tidb soft stack 32768 tidb hard stack 32768 ##check tidbInstanceUsersudoset. tidb ALL=(ALL) NOPASSWD: ALL ##check cpuper_and_thp_set. [main] include=balanced [vm] transparent_hugepage=never ###set 172.16.4.184 optimal parameters.### vm.swappiness need to delete,and new add vm.swappiness = 0. delete vm.swappiness through sedmatching. delete vm.swappiness done. startingset disableswap. disableswap set done. selinuxstate check and disable set. Disabled selinux state is disabled. starting set kernelset. kernelset setdone. delete unnecessary tidbInstanceUser tidb limitsset info startingset limitsset. limitsset set done. starting set tidbInstanceUsersudoset. tidb sudo grant is set. tidbInstanceUsersudoset setdone. this is not kylin-release system. will be cpup and thp set. backup /etc/tuned/balanced-tidb-optimal/tuned.conf to /etc/tuned/balanced-tidb-optimal/tuned.conf.bak. machinetype check. machine type is virtual machine. virtual machine thp setdone,TIPS restart the system setting takes effect.#############start check os optimal parameters set############ ##check swappinessdisableset. vm.swappiness = 0 ##check seliunxdisableset. Disabled ##check kernelset. fs.file-max = 1000000 net.core.somaxconn = 32768 net.ipv4.tcp_tw_recycle = 0 net.ipv4.tcp_syncookies = 0 vm.overcommit_memory = 1 ##check limitsset. tidb soft nofile 1000000 tidb hard nofile 1000000 tidb soft stack 32768 tidb hard stack 32768 ##check tidbInstanceUsersudoset. tidb ALL=(ALL) NOPASSWD: ALL ##check cpuper_and_thp_set. [main] include=balanced [vm] transparent_hugepage=never ###set 172.16.4.207 optimal parameters.###vm.swappiness need to delete,and newadd vm.swappiness = 0. delete vm.swappiness through sed matching. delete vm.swappiness done. starting set disableswap. disableswap setdone. selinuxstate check and disable set. Disabled selinux state is disabled. startingset kernelset. kernelset set done. tidb limitsset is set. starting set limitsset. limitsset set done. starting set tidbInstanceUsersudoset. tidb sudogrant is set. tidbInstanceUsersudosetset done. this is not kylin-release system. will be cpup and thp set. backup /etc/tuned/balanced-tidb-optimal/tuned.conf to /etc/tuned/balanced-tidb-optimal/tuned.conf.bak. machine typecheck. machinetype is virtual machine. virtual machine thp set done,TIPS restart the system setting takes effect. #############start check os optimal parameters set############ ##check swappinessdisableset. vm.swappiness = 0 ##check seliunxdisableset. Disabled ##check kernelset. fs.file-max = 1000000net.core.somaxconn= 32768 net.ipv4.tcp_tw_recycle = 0 net.ipv4.tcp_syncookies = 0 vm.overcommit_memory = 1 ##check limitsset. tidb soft nofile 1000000 tidb hard nofile 1000000 tidb soft stack 32768 tidb hard stack 32768 ##check tidbInstanceUsersudoset. tidb ALL=(ALL) NOPASSWD: ALL ##check cpuper_and_thp_set. [main] include=balanced [vm] transparent_hugepage=never ###set 172.16.4.212 optimal parameters.### vm.swappiness need to delete,and new add vm.swappiness = 0. delete vm.swappiness through sed matching. delete vm.swappiness done. starting set disableswap. disableswap setdone. selinuxstate check and disable set. Disabled selinux state is disabled. startingset kernelset. kernelset set done. tidb limitsset is set. starting set limitsset. limitsset setdone. startingset tidbInstanceUsersudoset. tidb sudo grant is set. tidbInstanceUsersudoset setdone. this is not kylin-release system. will be cpup and thp set. backup /etc/tuned/balanced-tidb-optimal/tuned.conf to /etc/tuned/balanced-tidb-optimal/tuned.conf.bak. machinetype check. machine type is virtual machine. virtual machine thp set done,TIPS restart the system setting takes effect. #############start check os optimal parameters set############ ##check swappinessdisableset.vm.swappiness= 0 ##check seliunxdisableset. Disabled ##check kernelset. fs.file-max = 1000000 net.core.somaxconn = 32768 net.ipv4.tcp_tw_recycle = 0 net.ipv4.tcp_syncookies = 0 vm.overcommit_memory = 1 ##check limitsset. tidb soft nofile 1000000 tidb hard nofile 1000000 tidb soft stack 32768 tidb hard stack 32768 ##check tidbInstanceUsersudoset. tidb ALL=(ALL) NOPASSWD: ALL ##check cpuper_and_thp_set. [main] include=balanced [vm] transparent_hugepage=never ###set 172.16.4.214 optimal parameters.### vm.swappiness need to delete,and new add vm.swappiness = 0. delete vm.swappiness through sedmatching. delete vm.swappiness done. startingset disableswap. disableswap set done. selinuxstate check and disable set. Disabled selinux state is disabled. starting set kernelset. kernelset setdone. tidb limitsset is set. startingset limitsset. limitsset set done. starting set tidbInstanceUsersudoset. tidb sudo grant is set. tidbInstanceUsersudoset setdone. this is not kylin-release system. will be cpup and thp set. backup /etc/tuned/balanced-tidb-optimal/tuned.conf to /etc/tuned/balanced-tidb-optimal/tuned.conf.bak. machinetype check. machine type is virtual machine. virtual machine thp set done,TIPS restart the system setting takes effect. #