某君more

Appearance

raid 创建 ​

使用mdadm创建raid，简单用法如下:

-l 用于指定创建的raid方式

参数简写完整写法说明
-C--create从未使用的设备创建新阵列
-a--auto=yes决定建立后面接的软体磁盘阵列设备，亦即/ dev/md0的，/ dev/md1上
-n--raid-devices=N使用几个磁盘 (partition) 作为磁盘阵列的设备
-l--level=[0,1,4,5,6,10]设置磁盘阵列的等级，常用 0,1,4,5,6,10

1. 添加两块硬盘 ​

新增两块硬盘，使用fdisk -l查看新增的硬盘，新增了 /dev/sdb 、/dev/sdc

sh
[root@moujun ~]# fdisk -l

Disk /dev/sda: 107.4 GB, 107374182400 bytes, 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x000a9b3d

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     2099199     1048576   83  Linux
/dev/sda2         2099200   209715199   103808000   8e  Linux LVM

Disk /dev/mapper/centos-root: 53.7 GB, 53687091200 bytes, 104857600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/centos-swap: 4160 MB, 4160749568 bytes, 8126464 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/centos-home: 48.4 GB, 48444211200 bytes, 94617600 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdb: 21.5 GB, 21474836480 bytes, 41943040 sectors 
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/sdc: 21.5 GB, 21474836480 bytes, 41943040 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

[root@moujun ~]#

2. 创建raid1 ​

2.1 resync非常慢 ​

提前说明这个，防止出现问题无法解决,查看以下两个文件，得知默认的最大速度：200000，最小速度：1000

cat /proc/sys/dev/raid/speed_limit_max

cat /proc/sys/dev/raid/speed_limit_min

将最小速度设置大点，可以有效解决同步慢的问题 ，将其设置到了150MB/s

echo 150000 > /proc/sys/dev/raid/speed_limit_min

2.2 创建raid1 ​

创建不同类型的raid，仅需 修改 -l raid方式，-n参数设置磁盘数量，以及最后添加具体硬盘

sh
[root@moujun ~]#  mdadm -C /dev/md1 -ayes -l1 -n2 /dev/sd[bc]
mdadm: Note: this array has metadata at the start and
    may not be suitable as a boot device.  If you plan to
    store '/boot' on this device please ensure that
    your boot-loader understands md/v1.x metadata, or use
    --metadata=0.90
Continue creating array? y
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md1 started.
[root@moujun ~]#

查看创建进度cat /proc/mdstat，等待resync完成

sh
[root@moujun ~]#  cat /proc/mdstat
Personalities : [raid1] 
md1 : active raid1 sdc[1] sdb[0]
      20954112 blocks super 1.2 [2/2] [UU]
      [==>..................]  resync = 13.3% (2800128/20954112) finish=1.5min speed=200009K/sec
      
unused devices: <none>
[root@moujun ~]#

输入 mdadm -D /dev/md*命令查看相关信息

sh
[root@moujun ~]# mdadm -D /dev/md1
/dev/md1:
           Version : 1.2
     Creation Time : Sun Mar  3 19:41:31 2024
        Raid Level : raid1
        Array Size : 20954112 (19.98 GiB 21.46 GB)
     Used Dev Size : 20954112 (19.98 GiB 21.46 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

       Update Time : Sun Mar  3 19:43:16 2024
             State : clean 
    Active Devices : 2
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : resync

              Name : moujun.top:1  (local to host moujun.top)
              UUID : 67c00e23:ab70e9d7:99efe3df:48cc3421
            Events : 17

    Number   Major   Minor   RaidDevice State
       0       8       16        0      active sync   /dev/sdb
       1       8       32        1      active sync   /dev/sdc
[root@moujun ~]#

或者 mdadm --detail --scan 简写 mdadm -Ds

sh
[root@moujun ~]# mdadm --detail --scan
ARRAY /dev/md1 metadata=1.2 name=moujun.top:1 UUID=67c00e23:ab70e9d7:99efe3df:48cc3421

创建包含 RAID 信息的配置文件

根据输出新增/etc/mdadm.conf文件内容：

ARRAY /dev/md/* level=raid1 num-devices=2 UUID={uuid}

go
# mdadm.conf written out by anaconda
MAILADDR root
AUTO +imsm +1.x -all
ARRAY /dev/md/1 level=raid1 num-devices=2 UUID=9f8ad06c:da23d88f:2a209233:9fce51f9

3. 格式化raid阵列 ​

需要对磁盘进行格式化，根据系统要求进行格式化：ext4、 xfs

3.1 ext4格式化 ​

命令格式：mkfs.ext4 -T largefile /dev/md1 , -T largefile 是快速格式化

sh
[root@moujun ~]# mkfs.ext4 -T largefile /dev/md1 
mke2fs 1.42.9 (28-Dec-2013)
Filesystem label=
OS type: Linux
Block size=4096 (log=2)m'k
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
20480 inodes, 5238528 blocks
261926 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=2153775104
160 block groups
32768 blocks per group, 32768 fragments per group
128 inodes per group
Superblock backups stored on blocks: 
	32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
	4096000

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done   

[root@moujun ~]#

3.2 xfs格式化 ​

命令格式：mkfs.xfs -f /dev/md1

-f :磁盘已被格式化，强制覆盖原有数据进行格式（丢失数据）

sh
[root@moujun ~]# mkfs.xfs -f /dev/md1 
meta-data=/dev/md1               isize=512    agcount=4, agsize=1309632 blks
         =                       sectsz=512   attr=2, projid32bit=1
         =                       crc=1        finobt=0, sparse=0
data     =                       bsize=4096   blocks=5238528, imaxpct=25
         =                       sunit=0      swidth=0 blks
naming   =version 2              bsize=4096   ascii-ci=0 ftype=1
log      =internal log           bsize=4096   blocks=2560, version=2
         =                       sectsz=512   sunit=0 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0
[root@moujun ~]#

4. 挂载磁盘【可选】 ​

注意

如果当前raid是需要lvm的，请跳过此操作

4.1 临时挂载 ​

sh
mount /dev/md1  /mnt/raid/

4.2 永久挂载 ​

参数说明：

md
 设备              挂载点         文件系统类型        挂载参数       是否备份     是否检测
 /dev/sdc         /data           ext4               defaults         0            0

在/etc/fstab文件中添加挂载信息

go
# /etc/fstab
# Created by anaconda on Mon Feb 19 02:16:44 2024
#
# Accessible filesystems, by reference, are maintained under '/dev/disk'
# See man pages fstab(5), findfs(8), mount(8) and/or blkid(8) for more info
#
/dev/mapper/centos-root /                       xfs     defaults        0 0
UUID=36410410-ab0e-42ad-a762-c9be30da02fa /boot                   xfs     defaults        0 0
/dev/mapper/centos-home /home                   xfs     defaults        0 0
/dev/mapper/centos-swap swap                    swap    defaults        0 0
/dev/md1 /mnt/raid                    xfs     defaults        0 0

5. 硬盘故障替换 ​

仅举例raid1场景，其他场景请参考从场景做法

  1. 模拟raid1 中某块磁盘故障

    sh
    [root@moujun ~]# mdadm -f /dev/md1 /dev/sdc
mdadm: set /dev/sdc faulty in /dev/md1

    查看阵列详情

    sh
    [root@moujun ~]# mdadm -D /dev/md1
/dev/md1:
           Version : 1.2
     Creation Time : Mon Mar  4 09:06:29 2024
        Raid Level : raid1
        Array Size : 20954112 (19.98 GiB 21.46 GB)
     Used Dev Size : 20954112 (19.98 GiB 21.46 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

       Update Time : Mon Mar  4 09:58:41 2024
             State : clean, degraded 
    Active Devices : 1
   Working Devices : 1
    Failed Devices : 1
     Spare Devices : 0

Consistency Policy : resync

              Name : moujun.top:1  (local to host moujun.top)
              UUID : d1c6cbe0:1cb73de2:293c4671:a8269ef6
            Events : 49

    Number   Major   Minor   RaidDevice State
       1       8       16        1      active sync   /dev/sdb
       2       8       32        -      faulty   /dev/sdc
[root@moujun ~]#

  2. 移除故障的硬盘,仅剩一块盘

    sh
    [root@moujun ~]# mdadm -r /dev/md1 /dev/sdc
mdadm: hot removed /dev/sdc from /dev/md1
您在 /var/spool/mail/root 中有新邮件
[root@moujun ~]# mdadm -D /dev/md1
/dev/md1:
           Version : 1.2
     Creation Time : Mon Mar  4 09:06:29 2024
        Raid Level : raid1
        Array Size : 20954112 (19.98 GiB 21.46 GB)
     Used Dev Size : 20954112 (19.98 GiB 21.46 GB)
      Raid Devices : 2
     Total Devices : 1
       Persistence : Superblock is persistent

       Update Time : Mon Mar  4 10:01:29 2024
             State : clean, degraded 
    Active Devices : 1
   Working Devices : 1
    Failed Devices : 0
     Spare Devices : 0

Consistency Policy : resync

              Name : moujun.top:1  (local to host moujun.top)
              UUID : d1c6cbe0:1cb73de2:293c4671:a8269ef6
            Events : 50

    Number   Major   Minor   RaidDevice State
       1       8       16        1      active sync   /dev/sdb
[root@moujun ~]#

  3. 添加新的硬盘

    sh
    [root@moujun ~]# mdadm -a /dev/md1 /dev/sdc
mdadm: added /dev/sdc
[root@moujun ~]#

    查看阵列详情，发现新增的盘显示 spare rebuildin,表示正在重建，等待重建完成显示active sync表示恢复正常

    sh
    [root@moujun ~]# mdadm -D /dev/md1
/dev/md1:
           Version : 1.2
     Creation Time : Mon Mar  4 09:06:29 2024
        Raid Level : raid1
        Array Size : 20954112 (19.98 GiB 21.46 GB)
     Used Dev Size : 20954112 (19.98 GiB 21.46 GB)
      Raid Devices : 2
     Total Devices : 2
       Persistence : Superblock is persistent

       Update Time : Mon Mar  4 10:03:08 2024
             State : clean, degraded, recovering 
    Active Devices : 1
   Working Devices : 2
    Failed Devices : 0
     Spare Devices : 1

Consistency Policy : resync

    Rebuild Status : 8% complete

              Name : moujun.top:1  (local to host moujun.top)
              UUID : d1c6cbe0:1cb73de2:293c4671:a8269ef6
            Events : 53

    Number   Major   Minor   RaidDevice State
       2       8       32        0      spare rebuilding   /dev/sdc
       1       8       16        1      active sync   /dev/sdb
[root@moujun ~]#