20091114 系统宕机处理记录

故障描述：今天上午客户来电说虚机的D盘找不到了，值班小伙重启物理主机后系统出现：Kernel panic - not syncing: Attempting to kill init后一直重启！

 

初步判断 可能是硬盘分区错误、添加、删除逻辑导致错误，进入linux rescue模式

 

#pvdisplay

  Couldn't find device with uuid 'PlU1AI-zrdA-1vuS-ljg9-x9wK-2bGB-ljeEXJ'.
  Couldn't find all physical volumes for volume group VolGroup00.
  Couldn't find device with uuid 'PlU1AI-zrdA-1vuS-ljg9-x9wK-2bGB-ljeEXJ'.
  Couldn't find device with uuid 'PlU1AI-zrdA-1vuS-ljg9-x9wK-2bGB-ljeEXJ'.

#vgdisplay

  Couldn't find device with uuid 'PlU1AI-zrdA-1vuS-ljg9-x9wK-2bGB-ljeEXJ'.
  Couldn't find all physical volumes for volume group VolGroup00.
  Couldn't find device with uuid 'PlU1AI-zrdA-1vuS-ljg9-x9wK-2bGB-ljeEXJ'.
  Couldn't find device with uuid 'PlU1AI-zrdA-1vuS-ljg9-x9wK-2bGB-ljeEXJ'.

 

bash-3.1#ls /dev/VolGroup00/
aotian  D-virtualhostforasp  hongjingda  LogVol00  LogVol01  virtualhostforasp

bash-3.1# cat /etc/fstab

/dev/VolGroup00/LogVol00 /                       ext3    defaults        1 1
LABEL=/boot             /boot                   ext3    defaults        1 2
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0
/dev/VolGroup00/LogVol01 swap                    swap    defaults        0 0

 

网上查资料后

 

bash-3.1# vgreduce --removemissing VolGroup00
  /dev/hdc: open failed: 只读文件系统
  /dev/hdc: open failed: 只读文件系统
  Attempt to close device '/dev/hdc' which is not open.
  Couldn't find device with uuid 'PlU1AI-zrdA-1vuS-ljg9-x9wK-2bGB-ljeEXJ'.
  Couldn't find all physical volumes for volume group VolGroup00.
  Couldn't find device with uuid 'PlU1AI-zrdA-1vuS-ljg9-x9wK-2bGB-ljeEXJ'.
  Couldn't find all physical volumes for volume group VolGroup00.
  Couldn't find device with uuid 'PlU1AI-zrdA-1vuS-ljg9-x9wK-2bGB-ljeEXJ'.
  Couldn't find device with uuid 'PlU1AI-zrdA-1vuS-ljg9-x9wK-2bGB-ljeEXJ'.
  Wrote out consistent volume group VolGroup00

 

bash-3.1#pvdisplay
  --- Physical volume ---
  PV Name               /dev/sda2
  VG Name               VolGroup00
  PV Size               279.18 GB / not usable 27.26 MB
  Allocatable           yes
  PE Size (KByte)       32768
  Total PE              8933
  Free PE               4142
  Allocated PE          4791
  PV UUID               WEu3RP-t8Xu-DvBK-NrEZ-2xcl-NYc7-NjvA2x
 bash-3.1# vgdisplay
  --- Volume group ---
  VG Name               VolGroup00
  System ID            
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  84
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                6
  Open LV               5
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               279.16 GB
  PE Size               32.00 MB
  Total PE              8933
  Alloc PE / Size       4791 / 149.72 GB
  Free  PE / Size       4142 / 129.44 GB
  VG UUID               YdpRjw-jmYV-zLj2-gPqh-9GK7-H6E5-xPZl0b

bash-3.1# vgdisplay
  --- Volume group ---
  VG Name               VolGroup00
  System ID            
  Format                lvm2
  Metadata Areas        1
  Metadata Sequence No  84
  VG Access             read/write
  VG Status             resizable
  MAX LV                0
  Cur LV                6
  Open LV               5
  Max PV                0
  Cur PV                1
  Act PV                1
  VG Size               279.16 GB
  PE Size               32.00 MB
  Total PE              8933
  Alloc PE / Size       4791 / 149.72 GB
  Free  PE / Size       4142 / 129.44 GB
  VG UUID               YdpRjw-jmYV-zLj2-gPqh-9GK7-H6E5-xPZl0b
   
bash-3.1#lvdisplay
  --- Logical volume ---
  LV Name                /dev/VolGroup00/LogVol00
  VG Name                VolGroup00
  LV UUID                X8puvY-24ym-xgCE-7iJs-vLA8-V67n-AoCWWR
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                7.81 GB
  Current LE             250
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
  Block device           253:0
  
  --- Logical volume ---
  LV Name                /dev/VolGroup00/LogVol01
  VG Name                VolGroup00
  LV UUID                K14sMW-rzpq-TArA-j97H-ViXJ-Hqce-4F6qQi
  LV Write Access        read/write
  LV Status              available
  # open                 1
  LV Size                3.91 GB
  Current LE             125
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
  Block device           253:1
  
  --- Logical volume ---
  LV Name                /dev/VolGroup00/virtualhostforasp
  VG Name                VolGroup00
  LV UUID                w91DPt-PFq4-yuWO-BsGs-uPCa-k84j-2VzlIZ
  LV Write Access        read/write
  LV Status              available
  # open                 2
  LV Size                8.00 GB
  Current LE             256
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
  Block device           253:2
  
  --- Logical volume ---
  LV Name                /dev/VolGroup00/D-virtualhostforasp
  VG Name                VolGroup00
  LV UUID                XhSheE-zJPL-SLiI-dnBp-lVMw-dzcF-qYf9C5
  LV Write Access        read/write
  LV Status              available
  # open                 2
  LV Size                50.00 GB
  Current LE             1600
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
  Block device           253:3
  
  --- Logical volume ---
  LV Name                /dev/VolGroup00/hongjingda
  VG Name                VolGroup00
  LV UUID                wc0xWB-9YJy-Il7y-gKtH-DOsG-zPAT-I8LMJw
  LV Write Access        read/write
  LV Status              available
  # open                 2
  LV Size                20.00 GB
  Current LE             640
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
  Block device           253:4
  
  --- Logical volume ---
  LV Name                /dev/VolGroup00/aotian
  VG Name                VolGroup00
  LV UUID                AaOWu2-MU0t-lcCO-eKUe-2kRI-Awsk-OG1XrM
  LV Write Access        read/write
  LV Status              available
  # open                 0
  LV Size                60.00 GB
  Current LE             1920
  Segments               1
  Allocation             inherit
  Read ahead sectors     0
  Block device           253:5

重启系统后，开机正常.....但客户的D盘（逻辑卷/dev/VolGroup00/D_hongjingda）莫名其妙的丢失掉这个问题还没有解决，数据是否可以恢复，还要等待小黄那边的消息了.....但愿一切正常