Raid 配置的规划

安装 TiDB 集群的物理机配置如下：

组件配置描述CPU2 * Intel Xeon Gold 5218R(2.1GHz, 20Core)内存384GB系统盘2 * 480GB SATA *** RAID1数据盘14 * 1.92TB SATA *** RAID10

用 14 * 1.92TB SATA *** 组成 RAID10 后，减去 RAID 的元数据和 1024 的转换损失，实际共 12819716 MB 可用。因为所有的节点都配置了 *** 磁盘，且数据规模较大，所以需要配置成 TiDB 和 TiKV 的混合部署模式。

目前我们共配置 80 VCPU 的主机，按 16 个 VCPU 一个组件，大概可以配置 5 个组件，每台物理主机共部署 4 个 TiKV 和 1 个 TiDB。按一个 TiKV 约 2 TB 的容量规划，配置一个 3 TB 的空间，保证空间占用在 80% 以下。所以 12819716 MB 的容量空间分成 4 个 3 TB 的空间，挂接在 /data{0123}，和 1 个 500 GB 的空间，挂接在 /deploy。

混合部署的 CPU 和内存配置，可以参考文档 https://docs.pingcap.com/zh/tidb/dev/hybrid-deployment-topology 。

在操作系统中可以使用 RAID 卡的命令行管理工具，实现在线的 RAID 重配置，免去了重启和控制台访问等问题。

目前 x86 服务器中，主要使用 Microchip 的 SmartROC 系列（收购 adaptec 公司后发布的产品）和 Broadcom 的 Megaraid 系列（收购 avago 公司后发布的产品)，以下以这两款 RAID 卡配置为例。

Microchip Adaptec RAID arcconf 

RAID 卡背景信息

使用操作系统命令查看 RAID 卡型号是 ZTE ***服务器 R5300 G4 的 RAID 卡 SDPSA_L-18i

[host bin]$ sudo dmidecode |grep -i "product name" Product Name: R5300 G4 Product Name: R5300G4 [host bin]$ sudo dmidecode |grep -i RAID [host bin]$ sudo lspci |grep -i RAID [host bin]$ sudo dmesg |grep -i RAID [ 6.360527] smartpqi 0000:3c:00.0: added 14:1:0:0 4000000000000000 Direct-Access ZTE LOGICAL VOLUME ***SmartPathCap- En- RAID-1(1+0) [ 6.386016] smartpqi 0000:3c:00.0: added 14:1:0:1 4000000100000000 Direct-Access ZTE LOGICAL VOLUME ***SmartPathCap- En- RAID-1(1+0) [ 6.396947] smartpqi 0000:3c:00.0: added 14:2:0:0 0000000000000000 RAID ZTE SDPSA_L-18i 2G [ 6.398251] scsi 14:2:0:0: RAID ZTE SDPSA_L-18i 2G 5.00 PQ: 0 ANSI: 5

通过搜索可以查到 SDPSA_L-18i 实际上是 Microchip 的 SmartROC3100 RAID 卡。

https://www.vmware.com/resources/compatibility/detail.php?deviceCategory=vsanio&productid=50320&vcl=true

核对产品资料确认使用 arcconf 命令行工具可以配置 RAID。

https://ww1.microchip.com/downloads/en/DeviceDoc/00003022A.pdf

软件下载位置

https://storage.microsemi.com/en-us/speed/raid/storage_manager/arcconf_B25611_zip.php

参考产品手册进行配置

https://download.adaptec.com/pdfs/user_guides/adaptec_cli_smarthba_smartraid_11_22_ug.pdf

查看 RAID 卡和数据盘信息

主机配置有两个 vd，vd1 是 14 块盘配置的 RAID 10，需要删除 vd1。

[root@host /tmp/arcconf/linux_x64]# ./arcconf list 1 Controllers found: 1 ---------------------------------------------------------------------- Controller information ---------------------------------------------------------------------- Controller ID : Status, Slot, Mode, Name, SerialNumber, WWN ---------------------------------------------------------------------- Controller 1: : Optimal, Slot 0, Mixed, ZTE SmartROC3100 SDPSA_L-18i 2G, 746001400375, 50015EBE07AA5A3F ---------------------------------------------------------------------- Array Information ---------------------------------------------------------------------- Array ID : Status (Interface, TotalSize MB, FreeSpace MB) ---------------------------------------------------------------------- Array 0 : Ok (SATA ***, 915725 MB, 0 MB) Array 1 : Ok (SATA ***, 25639884 MB, 1 MB) ---------------------------------------------------------------------- Logical device information ---------------------------------------------------------------------- Logical ID : Status (RAID, Interface, Size MB) Name ---------------------------------------------------------------------- Logical 0 : Optimal (1, Data, 457830 MB) os Logical 1 : Optimal (10, Data, 12819716 MB) vd ---------------------------------------------------------------------- maxCache information ---------------------------------------------------------------------- Array ID : Status (Interface, TotalSize MB, FreeSpace MB) ---------------------------------------------------------------------- No maxCache device created ---------------------------------------------------------------------- Physical Device information ---------------------------------------------------------------------- Physical ID : State (Interface, BlockSize, SizeMB, Vendor, Model, Type) WWN, [Location] ---------------------------------------------------------------------- Physical 0,0 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A41, [Direct Attached, Slot 0(Connector 0:CN0)] Physical 0,1 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A42, [Direct Attached, Slot 1(Connector 0:CN0)] Physical 0,2 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A40, [Direct Attached, Slot 2(Connector 0:CN0)] Physical 0,3 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A3F, [Direct Attached, Slot 3(Connector 0:CN0)] Physical 0,4 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A45, [Direct Attached, Slot 4(Connector 1:CN1)] Physical 0,5 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A46, [Direct Attached, Slot 5(Connector 1:CN1)] Physical 0,6 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A44, [Direct Attached, Slot 6(Connector 1:CN1)] Physical 0,7 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A43, [Direct Attached, Slot 7(Connector 1:CN1)] Physical 0,8 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A49, [Direct Attached, Slot 8(Connector 2:CN2)] Physical 0,9 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4A, [Direct Attached, Slot 9(Connector 2:CN2)] Physical 0,10 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A48, [Direct Attached, Slot 10(Connector 2:CN2)] Physical 0,11 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A47, [Direct Attached, Slot 11(Connector 2:CN2)] Physical 0,12 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4D, [Direct Attached, Slot 12(Connector 3:CN3)] Physical 0,13 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4E, [Direct Attached, Slot 13(Connector 3:CN3)] Physical 0,16 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A4F, [Direct Attached, Slot 16(Connector 4:CN4)] Physical 0,17 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A50, [Direct Attached, Slot 17(Connector 4:CN4)] Physical 2,0 : Ready (SES2, Not Applicable, Not Applicable, ZTE , Virtual SGPIO, Enclosure Services Device) 50015EBE07AA5A51, [Not Applicable] Command completed successfully. [root@host /tmp/arcconf/linux_x64]# ./arcconf getconfig 1 ld 1 Controllers found: 1 -------------------------------------------------------- Logical device information -------------------------------------------------------- Logical Device number 1 Logical Device name : vd Disk Name : /dev/sdb Block Size of member drives : 512 Bytes Array : 1 RAID level : 10 Status of Logical Device : Optimal Size : 12819716 MB Stripe-unit size : 256 KB Full Stripe Size : 1792 KB Interface Type : SATA *** Device Type : Data Boot Type : None Heads : 255 Sectors Per Track : 32 Cylinders : 65535 Caching : Enabled Mount Points : Not Mounted LD Acceleration Method : Controller Cache Logical Device On Boot Connector : No Volume Unique Identifier : 600508B1001CD9EDCFD7F2DC2F11B617 -------------------------------------------------------- Logical Device segment information -------------------------------------------------------- Group 0, Segment 0 : Present (1831420MB, SATA, ***, Connector:0, Device:0) 222638F89768 中间输出略 Group 6, Segment 1 : Present (1831420MB, SATA, ***, Connector:3, Device:13) 22253910CA2E Command completed successfully. [root@host /tmp/arcconf/linux_x64]#

删除 VD1

删除 vd1，14 块盘是 ready 状态。

[root@host /tmp/arcconf/linux_x64]# ./arcconf delete 1 logicaldrive 1 Controllers found: 1 WARNING: Deleting this logical device will automatically delete array 1 because it is the only logical device present on that array. All data in logical device 1 will be lost. Delete the logical device? Press y, then ENTER to continue or press ENTER to abort: y Deleting: logical device 1 ("vd") Command completed successfully. [root@host /tmp/arcconf/linux_x64]# ./arcconf list 1 Controllers found: 1 ---------------------------------------------------------------------- Controller information ---------------------------------------------------------------------- Controller ID : Status, Slot, Mode, Name, SerialNumber, WWN ---------------------------------------------------------------------- Controller 1: : Optimal, Slot 0, Mixed, ZTE SmartROC3100 SDPSA_L-18i 2G, 746001400375, 50015EBE07AA5A3F ---------------------------------------------------------------------- Array Information ---------------------------------------------------------------------- Array ID : Status (Interface, TotalSize MB, FreeSpace MB) ---------------------------------------------------------------------- Array 0 : Ok (SATA ***, 915725 MB, 0 MB) ---------------------------------------------------------------------- Logical device information ---------------------------------------------------------------------- Logical ID : Status (RAID, Interface, Size MB) Name ---------------------------------------------------------------------- Logical 0 : Optimal (1, Data, 457830 MB) os ---------------------------------------------------------------------- maxCache information ---------------------------------------------------------------------- Array ID : Status (Interface, TotalSize MB, FreeSpace MB) ---------------------------------------------------------------------- No maxCache device created ---------------------------------------------------------------------- Physical Device information ---------------------------------------------------------------------- Physical ID : State (Interface, BlockSize, SizeMB, Vendor, Model, Type) WWN, [Location] ---------------------------------------------------------------------- Physical 0,0 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A41, [Direct Attached, Slot 0(Connector 0:CN0)] Physical 0,1 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A42, [Direct Attached, Slot 1(Connector 0:CN0)] Physical 0,2 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A40, [Direct Attached, Slot 2(Connector 0:CN0)] Physical 0,3 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A3F, [Direct Attached, Slot 3(Connector 0:CN0)] Physical 0,4 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A45, [Direct Attached, Slot 4(Connector 1:CN1)] Physical 0,5 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A46, [Direct Attached, Slot 5(Connector 1:CN1)] Physical 0,6 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A44, [Direct Attached, Slot 6(Connector 1:CN1)] Physical 0,7 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A43, [Direct Attached, Slot 7(Connector 1:CN1)] Physical 0,8 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A49, [Direct Attached, Slot 8(Connector 2:CN2)] Physical 0,9 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4A, [Direct Attached, Slot 9(Connector 2:CN2)] Physical 0,10 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A48, [Direct Attached, Slot 10(Connector 2:CN2)] Physical 0,11 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A47, [Direct Attached, Slot 11(Connector 2:CN2)] Physical 0,12 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4D, [Direct Attached, Slot 12(Connector 3:CN3)] Physical 0,13 : Ready (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4E, [Direct Attached, Slot 13(Connector 3:CN3)] Physical 0,16 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A4F, [Direct Attached, Slot 16(Connector 4:CN4)] Physical 0,17 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A50, [Direct Attached, Slot 17(Connector 4:CN4)] Physical 2,0 : Ready (SES2, Not Applicable, Not Applicable, ZTE , Virtual SGPIO, Enclosure Services Device) 50015EBE07AA5A51, [Not Applicable] Command completed successfully.

创建第一个 VD

使用 14 块创建 RAID 10 ,并划分出第一个 3070000 MB 的 LUN。

命令参数

值

arcconf create 1 logicaldrive

在 控制器 1 上创建 LD

name / VD 名字

data0

LDcache / 缓存启用

LON

VD 容量

3070000

RAID 级别

10

磁盘从 0 到 13

0 0,0 1,0 2,0 3,0 4,0 5,0 6,0 7,0 8,0 9,0 10,0 11,0 12,0 13

[root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data0 LDcache LON 3070000 10 0 0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 0 10 0 11 0 12 0 13 Controllers found: 1 Do you want to add a logical device to the configuration? Press y, then ENTER to continue or press ENTER to abort: y Creating logical device: data0 Command completed successfully. [root@host /tmp/arcconf/linux_x64]# ./arcconf list 1 Controllers found: 1 ---------------------------------------------------------------------- Controller information ---------------------------------------------------------------------- Controller ID : Status, Slot, Mode, Name, SerialNumber, WWN ---------------------------------------------------------------------- Controller 1: : Optimal, Slot 0, Mixed, ZTE SmartROC3100 SDPSA_L-18i 2G, 746001400375, 50015EBE07AA5A3F ---------------------------------------------------------------------- Array Information ---------------------------------------------------------------------- Array ID : Status (Interface, TotalSize MB, FreeSpace MB) ---------------------------------------------------------------------- Array 0 : Ok (SATA ***, 915725 MB, 0 MB) Array 1 : Ok (SATA ***, 25639884 MB, 19499435 MB) ---------------------------------------------------------------------- Logical device information ---------------------------------------------------------------------- Logical ID : Status (RAID, Interface, Size MB) Name ---------------------------------------------------------------------- Logical 0 : Optimal (1, Data, 457830 MB) os Logical 1 : Optimal (10, Data, 3070000 MB) data0 ---------------------------------------------------------------------- maxCache information ---------------------------------------------------------------------- Array ID : Status (Interface, TotalSize MB, FreeSpace MB) ---------------------------------------------------------------------- No maxCache device created ---------------------------------------------------------------------- Physical Device information ---------------------------------------------------------------------- Physical ID : State (Interface, BlockSize, SizeMB, Vendor, Model, Type) WWN, [Location] ---------------------------------------------------------------------- Physical 0,0 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A41, [Direct Attached, Slot 0(Connector 0:CN0)] Physical 0,1 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A42, [Direct Attached, Slot 1(Connector 0:CN0)] Physical 0,2 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A40, [Direct Attached, Slot 2(Connector 0:CN0)] Physical 0,3 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A3F, [Direct Attached, Slot 3(Connector 0:CN0)] Physical 0,4 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A45, [Direct Attached, Slot 4(Connector 1:CN1)] Physical 0,5 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A46, [Direct Attached, Slot 5(Connector 1:CN1)] Physical 0,6 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A44, [Direct Attached, Slot 6(Connector 1:CN1)] Physical 0,7 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A43, [Direct Attached, Slot 7(Connector 1:CN1)] Physical 0,8 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A49, [Direct Attached, Slot 8(Connector 2:CN2)] Physical 0,9 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4A, [Direct Attached, Slot 9(Connector 2:CN2)] Physical 0,10 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A48, [Direct Attached, Slot 10(Connector 2:CN2)] Physical 0,11 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A47, [Direct Attached, Slot 11(Connector 2:CN2)] Physical 0,12 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4D, [Direct Attached, Slot 12(Connector 3:CN3)] Physical 0,13 : Online (SATA, 512 Bytes, 1831420MB, ATA, Micron_5300_MTFDDAK1T9TDS, Solid State Drive) 30015EBE07AA5A4E, [Direct Attached, Slot 13(Connector 3:CN3)] Physical 0,16 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A4F, [Direct Attached, Slot 16(Connector 4:CN4)] Physical 0,17 : Online (SATA, 512 Bytes, 457862MB, ATA, SAMSUNG MZ7LH480HAHQ-00005, Solid State Drive) 30015EBE07AA5A50, [Direct Attached, Slot 17(Connector 4:CN4)] Physical 2,0 : Ready (SES2, Not Applicable, Not Applicable, ZTE , Virtual SGPIO, Enclosure Services Device) 50015EBE07AA5A51, [Not Applicable] Command completed successfully. [root@host /tmp/arcconf/linux_x64]# fdisk -l Disk /dev/sda: 480.1 GB, 480070426624 bytes, 937637552 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 262144 bytes Disk label type: dos Disk identifier: 0x000a9444 Device Boot Start End Blocks Id System /dev/sda1 * 2048 937506479 468752216 83 Linux /dev/sda2 937506480 937637551 65536 c W95 FAT32 (LBA) Partition 2 does not start on physical sector boundary. Disk /dev/sdb: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 1835008 bytes [root@host /tmp/arcconf/linux_x64]# parted -s -a optimal /dev/sdb mklabel gpt -- mkpart primary ext4 1 3200GB Warning: The resulting partition is not properly aligned for best performance.

附： “Warning: The resulting partition is not properly aligned for best performance.” 处理方法

http://rainbow.chard.org/2013/01/30/how-to-align-partitions-for-best-performance-using-parted/

[root@host /tmp/arcconf/linux_x64]# cat /sys/block/sdb/queue/optimal_io_size 1835008 [root@host /tmp/arcconf/linux_x64]# cat /sys/block/sdb/alignment_offset 0 [root@host /tmp/arcconf/linux_x64]# parted -s -a optimal /dev/sdb mklabel gpt -- mkpart primary ext4 3584s 100% [root@host /tmp/arcconf/linux_x64]# mkfs.ext4 /dev/sdb1 [root@host /tmp/arcconf/linux_x64]#

创建后续 VD

在 14 块创建 RAID 10 的 ARRAY 磁盘组继续划分出其他 LUN。

命令

注释

arcconf create 1 logicaldrive

在 控制器 1 上创建 LD

name /VD 名字

data1~data4

LDcache / 缓存启用

LON

VD 容量

3070000 * 3 + 499000 * 1

RAID 级别

10

ARRAY / 现有 RAID 组

1（第 1 个 LD 创建时自动定义的 ARRAY）

noprompt不交互，建议删除操作不使用。 [root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data1 LDcache LON 3070000 10 ARRAY 1 Controllers found: 1 Do you want to add a logical device to the configuration? Press y, then ENTER to continue or press ENTER to abort: y Creating logical device: data1 Command completed successfully. [root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data2 LDcache LON 3070000 10 ARRAY 1 noprompt [root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data3 LDcache LON 3070000 10 ARRAY 1 noprompt [root@host /tmp/arcconf/linux_x64]# ./arcconf create 1 logicaldrive name data4 LDcache LON 499000 10 ARRAY 1 noprompt [root@host /tmp/arcconf/linux_x64]# fdisk -l Disk /dev/sda: 480.1 GB, 480070426624 bytes, 937637552 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 262144 bytes Disk label type: dos Disk identifier: 0x000a9444 Device Boot Start End Blocks Id System /dev/sda1 * 2048 937506479 468752216 83 Linux /dev/sda2 937506480 937637551 65536 c W95 FAT32 (LBA) Partition 2 does not start on physical sector boundary. WARNING: fdisk GPT support is currently new, and therefore in an experimental phase. Use at your own discretion. Disk /dev/sdb: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 1835008 bytes Disk label type: gpt Disk identifier: 08A14123-EF5C-4775-A0C0-2DE80827BD40 # Start End Size Type Name 1 3584 6287357439 3T Microsoft basic primary Disk /dev/sdc: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 1835008 bytes Disk /dev/sdd: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 1835008 bytes Disk /dev/sde: 3219.1 GB, 3219128320000 bytes, 6287360000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 1835008 bytes Disk /dev/sdf: 523.2 GB, 523239424000 bytes, 1021952000 sectors Units = sectors of 1 * 512 = 512 bytes Sector size (logical/physical): 512 bytes / 4096 bytes I/O size (minimum/optimal): 262144 bytes / 1835008 bytes 后续分区格式化挂接略

Broadcom avago MegaRAID storcli 

RAID 卡背景信息

使用操作系统命令查看 RAID 卡型号是超聚变（原*** x86）服务器 2288H V5 的 OEM Megaraid RAID 卡

# dmidecode |grep -Ei "manufacturer|product name" Manufacturer: XFUSION Product Name: 2288H V5 # dmesg|grep -i megaraid ... [ 7.846978] scsi host0: Avago SAS based MegaRAID driver ...

下载和安装 storcli 管理工具

登录BROADCOM网站上RAID控制卡的页面。

在“DOWNLOADS”页签中单击“Management Software and Tools”。

打开管理软件和工具列表。

下载最新版本的“StorCLI”工具。

解压后得到适用于不同操作系统的工具包，安装 linux 的 rpm 包。

使用 storcli 管理工具

storcli 管理工具的使用教程较多，简单列举如下：

查看 RAID 卡配置和当前磁盘 RAID 配置

通常 v1 是数据磁盘。

./storcli64 /c0 show ./storcli64 /c0/v1 show all

删除 v1

./storcli64 /c0/v1 del

按需添加 v1 等多个 LUN

./storcli64 /c0 add vd r10 size=2000GB drives=252:0-13