Netapp: snapmirror resync 的一些探讨

1、测试snapmirror 关系在源端snap restore之后重建同步关系：
环境：fas960-hz:/vol/testvol → fas3070-hz:/vol/testvol

 

测试卷大小：testvol，10GB，有效数据240MB

fas3070-hz*>  df -h
Filesystem               total       used      avail capacity  Mounted on
/vol/vol0/                32GB      171MB       31GB       1%  /vol/vol0/
/vol/vol0/.snapshot     8192MB       12MB     8179MB       0%  /vol/vol0/.snapshot
/vol/cn_db_30/          4800GB     2115GB     2684GB      44%  /vol/cn_db_30/
/vol/cn_db_30/.snapshot     1200GB       22MB     1199GB       0%  /vol/cn_db_30/.snapshot
/vol/testvol/           8192MB      240MB     7951MB       3%  /vol/testvol/
/vol/testvol/.snapshot     2048MB       48MB     1999MB       2%  /vol/testvol/.snapshot

 

在960上snap restore hourly.0 ：
FAS960-hz*> snap list testvol
Volume testvol
working...

 

  %/used       %/total  date          name
----------  ----------  ------------  --------
  0% ( 0%)    0% ( 0%)  Aug 01 08:39  fas3070-hz(0118064597)_testvol.1 (snapmirror)
  0% ( 0%)    0% ( 0%)  Aug 01 00:00  hourly.0      
  0% ( 0%)    0% ( 0%)  Jul 31 18:00  hourly.1      
  0% ( 0%)    0% ( 0%)  Jul 31 17:00  hourly.2      
  0% ( 0%)    0% ( 0%)  Jul 31 16:00  hourly.3      
 16% (16%)    0% ( 0%)  Jul 31 15:00  hourly.4      
 16% ( 0%)    0% ( 0%)  Jul 31 14:00  hourly.5      
 16% ( 0%)    0% ( 0%)  Jul 31 13:00  hourly.6      
 17% ( 1%)    0% ( 0%)  Jul 31 12:00  hourly.7      
 17% ( 0%)    0% ( 0%)  Jul 31 11:00  hourly.8      

 

FAS960-hz*> snap restore -s hourly.0 testvol

 

WARNING! This will revert the volume to a previous snapshot.
All modifications to the volume after the snapshot will be
irrevocably lost.

 

Volume testvol will be made restricted briefly before coming back online.

 

Are you sure you want to do this? y

 

You have selected volume testvol, snapshot hourly.0

 

Proceed with revert? y
Fri Aug  1 08:43:41 CST [wafl.snaprestore.revert:notice]: Reverting volume testvol to a previous snapshot.
Volume testvol: revert successful.

 

FAS960-hz*>
FAS960-hz*> snap list testvol
Volume testvol
working...

 

  %/used       %/total  date          name
----------  ----------  ------------  --------
  0% ( 0%)    0% ( 0%)  Aug 01 00:00  hourly.0      
  0% ( 0%)    0% ( 0%)  Jul 31 18:00  hourly.1      
  0% ( 0%)    0% ( 0%)  Jul 31 17:00  hourly.2      
  0% ( 0%)    0% ( 0%)  Jul 31 16:00  hourly.3      
 16% (16%)    0% ( 0%)  Jul 31 15:00  hourly.4      
 16% ( 0%)    0% ( 0%)  Jul 31 14:00  hourly.5      
 16% ( 0%)    0% ( 0%)  Jul 31 13:00  hourly.6      
 17% ( 1%)    0% ( 0%)  Jul 31 12:00  hourly.7      
 17% ( 0%)    0% ( 0%)  Jul 31 11:00  hourly.8      

 

snapmirror 自动产生的快照已经消失，此时在3070上update snapmirror：
fas3070-hz*> snapmirror update fas3070-hz:testvol
Fri Aug  1 08:45:04 CST [snapmirror.dst.err:error]: SnapMirror destination transfer from fas960-hz:testvol to testvol : incremental update not possible; a resync or initialize is necessary.
Transfer aborted: incremental update not possible; a resync or initialize is necessary.

 

在3070上用snapmirror resync -S这个命令：
fas3070-hz*>  snapmirror resync -S fas960-hz:testvol
Destination path specification required.
usage:
snapmirror resync [-n][-f][-k ][-s ][-c ][-S [:]][-w] [:]
 in which and are
  or
        - resynchronizes a broken mirror with a source
fas3070-hz*>  snapmirror resync -S fas960-hz:testvol fas3070-hz:testvol 
Fri Aug  1 08:48:21 CST [snapmirror.dst.resync.failed:error]: SnapMirror resync of testvol to fas960-hz:testvol : The destination is already a replica, but its newest snapshot cannot be used as the resync base because it does not exist on the source.  To resync, you must first break the mirror and then retry the resync..
Snapmirror resynchronization of testvol to fas960-hz:testvol : The destination is already a replica, but its newest snapshot cannot be used as the resync base because it does not exist on the source.  To resync, you must first break the mirror and then retry the resync.
Aborting resync.

 

听话，先break这个mirror关系：
fas3070-hz*> snapmirror break fas3070-hz:testvol
snapmirror break: Destination testvol is now writable.
Volume size is being retained for potential snapmirror resync.  If you would like to grow the volume and do not expect to resync, set vol option fs_size_fixed to off.

 

再执行上次的命令：
fas3070-hz*>  snapmirror resync -S fas960-hz:testvol fas3070-hz:testvol 
The resync base snapshot will be: hourly.0
These newer snapshots will be deleted from the destination:
    fas3070-hz(0118064597)_testvol.1
Are you sure you want to resync the volume? y
Fri Aug  1 08:51:30 CST [snapmirror.dst.resync.info:notice]: SnapMirror resync of testvol to fas960-hz:testvol is using hourly.0 as the base snapshot.
Volume testvol will be briefly unavailable before coming back online.
Fri Aug  1 08:51:49 CST [wafl.snaprestore.revert:notice]: Reverting volume testvol to a previous snapshot.
Fri Aug  1 08:51:49 CST [wafl.vol.guarantee.replica:info]: Space for replica volume 'testvol' is not guaranteed.
Revert to resync base snapshot was successful.
Fri Aug  1 08:51:56 CST [snapmirror.dst.resync.success:notice]: SnapMirror resync of testvol to fas960-hz:testvol successful.
Transfer started.
Monitor progress with 'snapmirror status' or the snapmirror log.
fas3070-hz*> snapmirror status
Snapmirror is on.
Source                         Destination                     State          Lag        Status
fas960-hz:testvol              fas3070-hz:testvol              Snapmirrored   00:05:44   Idle

 

snapmirror log中显示传输的数据量：

cmd Fri Aug  1 08:51:30 CST fas960-hz:testvol fas3070-hz:testvol Resync_command (hourly.0)
dst Fri Aug  1 08:51:56 CST fas960-hz:testvol fas3070-hz:testvol Request (Resync)
dst Fri Aug  1 08:52:02 CST fas960-hz:testvol fas3070-hz:testvol Start
dst Fri Aug  1 08:52:27 CST fas960-hz:testvol fas3070-hz:testvol End (436 KB)

GUI界面中该次resync传输的数据量显示：

传输量为436KB，明显不是初始化的传输量。

 

再次进行update：

fas3070-hz*>   snapmirror update fas3070-hz:testvol
Transfer started.
Monitor progress with 'snapmirror status' or the snapmirror log.
fas3070-hz*> snapmirror status                     
Snapmirror is on.
Source                         Destination                     State          Lag        Status
fas960-hz:testvol              fas3070-hz:testvol              Snapmirrored   00:07:32   Transferring
fas3070-hz*> snapmirror status
Snapmirror is on.
Source                         Destination                     State          Lag        Status
fas960-hz:testvol              fas3070-hz:testvol              Snapmirrored   00:00:28   Idle

 

结论：在snapmirror关系的源Filer端进行了snap restore之后，snapmirror创建的基准快照被删除的情况下，系统可以以自动创建的最近一个时间的快照为基准进行增量同步，而不需要再进行一次初始化同步。

 

 

2、测试snapmirror 关系断开后反向增量同步

环境：原方向fas960-hz:/vol/testvol → fas3070-hz:/vol/testvol，断开后改为fas3070-hz:/vol/testvol→fas960-hz:/vol/testvol

 

测试卷大小：testvol，10GB，有效数据240MB

同步关系下：

fas3070-hz*>    df -h testvol
Filesystem               total       used      avail capacity  Mounted on
/vol/testvol/           8192MB      240MB     7951MB       3%  /vol/testvol/
/vol/testvol/.snapshot     2048MB       48MB     1999MB       2%  /vol/testvol/.snapshot

FAS960-hz*> df -h testvol         
Filesystem               total       used      avail capacity  Mounted on
/vol/testvol/           8192MB      240MB     7951MB       3%  /vol/testvol/
/vol/testvol/.snapshot     2048MB       48MB     1999MB       2%  /vol/testvol/.snapshot

 

断开snapmirror同步关系：

fas3070-hz*> snapmirror status
Snapmirror is on.
Source                         Destination                     State          Lag        Status
fas960-hz:testvol              fas3070-hz:testvol              Snapmirrored   00:15:41   Idle
fas3070-hz*> snapmirror quiesce fas3070-hz:testvol
snapmirror quiesce: in progress
 This can be a long-running operation. Use Control - C (^C) to interrupt.
snapmirror quiesce: testvol :  Successfully quiesced
fas3070-hz*> snapmirror break fas3070-hz:testvol 
snapmirror break: Destination testvol is now writable.
Volume size is being retained for potential snapmirror resync.  If you would like to grow the volume and do not expect to resync, set vol option fs_size_fixed to off.
fas3070-hz*>

 

目的端新增120MB数据量：

fas3070-hz*> df -h testvol
Filesystem               total       used      avail capacity  Mounted on
/vol/testvol/           8192MB      360MB     7831MB       4%  /vol/testvol/
/vol/testvol/.snapshot     2048MB       48MB     1999MB       2%  /vol/testvol/.snapshot

 

目的端数据情况：

#ll -h /mnt/222/
total 218M
-rw-r--r--  1 root root    0 Jul 31 17:31 17.hour
-rw-r--r--  1 root root    0 Jul 31 18:07 18.hour
drwxr-xr-x  2 root root 4.0K May  1  2006 anacron
-rw-r--r--  1 root root  49K Feb 25  2006 config-2.6.9-34.EL
-rw-r--r--  1 root root  49K Feb 25  2006 config-2.6.9-34.ELsmp
drwxr-xr-x  2 root root 4.0K Jul 11 10:15 grub
drwxr-xr-x  4 root root 4.0K Jan  2  2008 litao
drwx------  2 root root 4.0K Apr  8  2006 lost+found
-rw-r--r--  1 root root  23K Aug  4  2005 message
-rw-r--r--  1 root root  21K Aug  4  2005 message.ja
-rw-r--r--  1 root root 121M Aug  1 09:17 moredata.tar
-rw-r--r--  1 root root    0 Aug  1 08:56 pdcserver
-rw-r--r--  1 root root  14M Jul 31 12:10 pic1.tar
-rw-r--r--  1 root root  82M Jul 31 12:10 pic.tar
-rw-r--r--  1 root root 1.5M Feb 25  2006 vmlinuz-2.6.9-34.EL
-rw-r--r--  1 root root 1.4M Feb 25  2006 vmlinuz-2.6.9-34.ELsmp
drwxr-xr-x  2 root root 4.0K Oct 25  2006 xinetd.d

源端数据情况（无moredata.tar）：

#ll -h /mnt/111
total 98M
-rw-r--r--  1 root root    0 Jul 31 17:31 17.hour
-rw-r--r--  1 root root    0 Jul 31 18:07 18.hour
drwxr-xr-x  2 root root 4.0K May  1  2006 anacron
-rw-r--r--  1 root root  49K Feb 25  2006 config-2.6.9-34.EL
-rw-r--r--  1 root root  49K Feb 25  2006 config-2.6.9-34.ELsmp
drwxr-xr-x  2 root root 4.0K Jul 11 10:15 grub
drwxr-xr-x  4 root root 4.0K Jan  2  2008 litao
drwx------  2 root root 4.0K Apr  8  2006 lost+found
-rw-r--r--  1 root root  23K Aug  4  2005 message
-rw-r--r--  1 root root  21K Aug  4  2005 message.ja
-rw-r--r--  1 root root    0 Aug  1 09:18 pdcserver
-rw-r--r--  1 root root  14M Jul 31 12:10 pic1.tar
-rw-r--r--  1 root root  82M Jul 31 12:10 pic.tar
-rw-r--r--  1 root root 1.5M Feb 25  2006 vmlinuz-2.6.9-34.EL
-rw-r--r--  1 root root 1.4M Feb 25  2006 vmlinuz-2.6.9-34.ELsmp
drwxr-xr-x  2 root root 4.0K Oct 25  2006 xinetd.d

 

检查作为目的地的fas960-hz上的卷：

FAS960-hz*> snap list testvol
Volume testvol
working...

 

  %/used       %/total  date          name
----------  ----------  ------------  --------
  0% ( 0%)    0% ( 0%)  Aug 01 09:00  hourly.0      
  0% ( 0%)    0% ( 0%)  Aug 01 08:59  fas3070-hz(0118064597)_testvol.2 (snapmirror)
  0% ( 0%)    0% ( 0%)  Aug 01 00:00  hourly.1      
  0% ( 0%)    0% ( 0%)  Jul 31 18:00  hourly.2      
  0% ( 0%)    0% ( 0%)  Jul 31 17:00  hourly.3      
  0% ( 0%)    0% ( 0%)  Jul 31 16:00  hourly.4      
 16% (16%)    0% ( 0%)  Jul 31 15:00  hourly.5      
 16% ( 0%)    0% ( 0%)  Jul 31 14:00  hourly.6      
 16% ( 0%)    0% ( 0%)  Jul 31 13:00  hourly.7      
 17% ( 1%)    0% ( 0%)  Jul 31 12:00  hourly.8      
 17% ( 0%)    0% ( 0%)  Jul 31 11:00  hourly.9      
FAS960-hz*>

FAS960-hz*> snapmirror status
Snapmirror is on.
Source                             Destination                        State          Lag        Status

FAS960-hz:testvol                  fas3070-hz:testvol                 Source         00:24:05   Idle

 

开始反向重新同步：

FAS960-hz*>   snapmirror resync -S fas3070-hz:testvol FAS960-hz:testvol
The resync base snapshot will be: fas3070-hz(0118064597)_testvol.2
These newer snapshots will be deleted from the destination:
    hourly.0
Are you sure you want to resync the volume? y
Fri Aug  1 09:22:53 CST [snapmirror.dst.resync.info:notice]: SnapMirror resync of testvol to fas3070-hz:testvol is using fas3070-hz(0118064597)_testvol.2 as the base snapshot.
Volume testvol will be briefly unavailable before coming back online.
Fri Aug  1 09:22:56 CST [wafl.snaprestore.revert:notice]: Reverting volume testvol to a previous snapshot.
Fri Aug  1 09:22:56 CST [wafl.vol.guarantee.replica:info]: Space for replica volume 'testvol' is not guaranteed.
Revert to resync base snapshot was successful.
Fri Aug  1 09:22:57 CST [snapmirror.dst.resync.success:notice]: SnapMirror resync of testvol to fas3070-hz:testvol successful.
Transfer started.
Monitor progress with 'snapmirror status' or the snapmirror log.
FAS960-hz*>
FAS960-hz*> snapmirror status
Snapmirror is on.
Source                             Destination                        State          Lag        Status
fas3070-hz:testvol                 FAS960-hz:testvol                  Snapmirrored   00:23:48   Transferring  (58 MB done)

FAS960-hz:testvol                  fas3070-hz:testvol                 Source         00:24:05   Idle

 

snapmirror的log显示：

cmd Fri Aug  1 09:22:53 CST fas3070-hz:testvol FAS960-hz:testvol Resync_command (fas3070-hz(0118064597)_testvol.2)
dst Fri Aug  1 09:22:57 CST fas3070-hz:testvol FAS960-hz:testvol Request (Resync)
dst Fri Aug  1 09:23:06 CST fas3070-hz:testvol FAS960-hz:testvol Start
dst Fri Aug  1 09:23:21 CST fas3070-hz:testvol FAS960-hz:testvol End (123740 KB)

传输量123740KB，与moredata.tar文件大小相符。检查源端nfs文件系统：

#ll -h /mnt/111
total 218M
-rw-r--r--  1 root root    0 Jul 31 17:31 17.hour
-rw-r--r--  1 root root    0 Jul 31 18:07 18.hour
drwxr-xr-x  2 root root 4.0K May  1  2006 anacron
-rw-r--r--  1 root root  49K Feb 25  2006 config-2.6.9-34.EL
-rw-r--r--  1 root root  49K Feb 25  2006 config-2.6.9-34.ELsmp
drwxr-xr-x  2 root root 4.0K Jul 11 10:15 grub
drwxr-xr-x  4 root root 4.0K Jan  2  2008 litao
drwx------  2 root root 4.0K Apr  8  2006 lost+found
-rw-r--r--  1 root root  23K Aug  4  2005 message
-rw-r--r--  1 root root  21K Aug  4  2005 message.ja
-rw-r--r--  1 root root 121M Aug  1 09:17 moredata.tar
-rw-r--r--  1 root root    0 Aug  1 09:18 pdcserver
-rw-r--r--  1 root root  14M Jul 31 12:10 pic1.tar
-rw-r--r--  1 root root  82M Jul 31 12:10 pic.tar
-rw-r--r--  1 root root 1.5M Feb 25  2006 vmlinuz-2.6.9-34.EL
-rw-r--r--  1 root root 1.4M Feb 25  2006 vmlinuz-2.6.9-34.ELsmp
drwxr-xr-x  2 root root 4.0K Oct 25  2006 xinetd.d

 

FAS960-hz*> df testvol
Filesystem              kbytes       used      avail capacity  Mounted on
/vol/testvol/          8388608     369448    8019160       4%  /vol/testvol/
/vol/testvol/.snapshot    2097152      50172    2046980       2%  /vol/testvol/.snapshot

 

fas3070-hz*> df testvol
Filesystem              kbytes       used      avail capacity  Mounted on
/vol/testvol/          8388608     369448    8019160       4%  /vol/testvol/
/vol/testvol/.snapshot    2097152      50232    2046920       2%  /vol/testvol/.snapshot

 

检查snapmirror log：

cmd Fri Aug  1 09:22:53 CST fas3070-hz:testvol FAS960-hz:testvol Resync_command (fas3070-hz(0118064597)_testvol.2)
dst Fri Aug  1 09:22:57 CST fas3070-hz:testvol FAS960-hz:testvol Request (Resync)
dst Fri Aug  1 09:23:06 CST fas3070-hz:testvol FAS960-hz:testvol Start
dst Fri Aug  1 09:23:21 CST fas3070-hz:testvol FAS960-hz:testvol End (123740 KB)
传输数据量123740KB，与数据变化量差不多。

 

结论：在某种情况下，例如灾难切换，snapmirror关系断开之后，由灾备站点提供服务，此时发生的数据变化在原站点恢复功能之后，可以增量反向同步到原站点Filer，而不需要重新初始化数据。

 

 

3、测试snapmirror 关系断开后删除基准快照再反向增量同步

环境：原方向fas960-hz:/vol/testvol → fas3070-hz:/vol/testvol，断开后改为fas3070-hz:/vol/testvol→fas960-hz:/vol/testvol

 

测试卷大小：testvol，10GB，有效数据360MB

将上一步测试的同步关系改回原方向 fas960-hz:/vol/testvol → fas3070-hz:/vol/testvol：

FAS960-hz*> snapmirror quiesce FAS960-hz:testvol  
snapmirror quiesce: in progress
 This can be a long-running operation. Use Control - C (^C) to interrupt.
snapmirror quiesce: testvol :  Successfully quiesced
FAS960-hz*> snapmirror break FAS960-hz:testvol    
snapmirror break: Destination testvol is now writable.
Volume size is being retained for potential snapmirror resync.  If you would like to grow the volume and do not expect to resync, set vol option fs_size_fixed to off.

FAS960-hz*> snap list testvol
Volume testvol
working...

 

  %/used       %/total  date          name
----------  ----------  ------------  --------
  0% ( 0%)    0% ( 0%)  Aug 01 09:23  FAS960-hz(0050418338)_testvol.1
  0% ( 0%)    0% ( 0%)  Aug 01 08:59  fas3070-hz(0118064597)_testvol.2 (snapmirror)
  0% ( 0%)    0% ( 0%)  Aug 01 08:51  fas3070-hz(0118064597)_testvol.1
  0% ( 0%)    0% ( 0%)  Aug 01 00:00  hourly.0      
  0% ( 0%)    0% ( 0%)  Jul 31 18:00  hourly.1      
  0% ( 0%)    0% ( 0%)  Jul 31 17:00  hourly.2      
  0% ( 0%)    0% ( 0%)  Jul 31 16:00  hourly.3      
 11% (11%)    0% ( 0%)  Jul 31 15:00  hourly.4      
 11% ( 0%)    0% ( 0%)  Jul 31 14:00  hourly.5      
 12% ( 0%)    0% ( 0%)  Jul 31 13:00  hourly.6      
 12% ( 0%)    0% ( 0%)  Jul 31 12:00  hourly.7      
 12% ( 0%)    0% ( 0%)  Jul 31 11:00  hourly.8      

 

FAS960-hz*> snap delete testvol FAS960-hz(0050418338)_testvol.1
Fri Aug  1 09:33:28 CST [wafl.snap.delete:info]: Snapshot copy FAS960-hz(0050418338)_testvol.1 on volume testvol NetApp was deleted by the Data ONTAP function snapcmd_delete. The unique ID for this Snapshot copy is (23, 336).

 

FAS960-hz*> snapmirror status                                  
Snapmirror is on.
Source                             Destination                        State          Lag        Status
FAS960-hz:testvol                  fas3070-hz:testvol                 Source         00:34:13   Idle

 

 

fas3070-hz*>  snapmirror status
Snapmirror is on.
Source                         Destination                     State          Lag        Status
fas960-hz:testvol              fas3070-hz:testvol              Broken-off     00:35:02   Idle
fas3070-hz:testvol             FAS960-hz:testvol               Source         00:11:24   Idle
fas3070-hz*>  snapmirror resync -S fas960-hz:testvol              fas3070-hz:testvol
The resync base snapshot will be: fas3070-hz(0118064597)_testvol.2
These newer snapshots will be deleted from the destination:
    FAS960-hz(0050418338)_testvol.1
Are you sure you want to resync the volume? y
Fri Aug  1 09:37:16 CST [snapmirror.dst.resync.info:notice]: SnapMirror resync of testvol to fas960-hz:testvol is using fas3070-hz(0118064597)_testvol.2 as the base snapshot.
Volume testvol will be briefly unavailable before coming back online.

Fri Aug  1 09:37:33 CST [wafl.snaprestore.revert:notice]: Reverting volume testvol to a previous snapshot.
Fri Aug  1 09:37:34 CST [wafl.vol.guarantee.replica:info]: Space for replica volume 'testvol' is not guaranteed.
Revert to resync base snapshot was successful.
Fri Aug  1 09:37:40 CST [snapmirror.dst.resync.success:notice]: SnapMirror resync of testvol to fas960-hz:testvol successful.
Transfer started.
Monitor progress with 'snapmirror status' or the snapmirror log.

fas3070-hz*> snapmirror status
Snapmirror is on.
Source                         Destination                     State          Lag        Status
fas960-hz:testvol              fas3070-hz:testvol              Snapmirrored   00:00:28   Idle

 

断开同步关系：

fas3070-hz*>  snapmirror quiesce fas3070-hz:testvol  
snapmirror quiesce: in progress
 This can be a long-running operation. Use Control - C (^C) to interrupt.
snapmirror quiesce: testvol :  Successfully quiesced
fas3070-hz*>  snapmirror break fas3070-hz:testvol    
snapmirror break: Destination testvol is now writable.
Volume size is being retained for potential snapmirror resync.  If you would like to grow the volume and do not expect to resync, set vol option fs_size_fixed to off.
fas3070-hz*>

 

删除snapmirror相关的基准快照：

fas3070-hz*> snaplist testvol
snaplist not found.  Type '?' for a list of commands
fas3070-hz*> snap list testvol
Volume testvol
working...

 

  %/used       %/total  date          name
----------  ----------  ------------  --------
  0% ( 0%)    0% ( 0%)  Aug 01 09:42  fas3070-hz(0118064597)_testvol.4
  0% ( 0%)    0% ( 0%)  Aug 01 09:37  fas3070-hz(0118064597)_testvol.3
  0% ( 0%)    0% ( 0%)  Aug 01 08:59  fas3070-hz(0118064597)_testvol.2 (snapmirror)
  0% ( 0%)    0% ( 0%)  Aug 01 00:00  hourly.0      
  0% ( 0%)    0% ( 0%)  Jul 31 18:00  hourly.1      
  0% ( 0%)    0% ( 0%)  Jul 31 17:00  hourly.2      
  0% ( 0%)    0% ( 0%)  Jul 31 16:00  hourly.3      
 11% (11%)    0% ( 0%)  Jul 31 15:00  hourly.4      
 11% ( 0%)    0% ( 0%)  Jul 31 14:00  hourly.5      
 12% ( 0%)    0% ( 0%)  Jul 31 13:00  hourly.6      
 12% ( 0%)    0% ( 0%)  Jul 31 12:00  hourly.7      
 12% ( 0%)    0% ( 0%)  Jul 31 11:00  hourly.8      

fas3070-hz*> snap delete testvol fas3070-hz(0118064597)_testvol.4
Fri Aug  1 09:45:55 CST [wafl.snap.delete:info]: Snapshot copy fas3070-hz(0118064597)_testvol.4 on volume testvol NetApp was deleted by the Data ONTAP function snapcmd_delete. The unique ID for this Snapshot copy is (27, 350).
fas3070-hz*> snap delete testvol fas3070-hz(0118064597)_testvol.3
Fri Aug  1 09:46:05 CST [wafl.snap.delete:info]: Snapshot copy fas3070-hz(0118064597)_testvol.3 on volume testvol NetApp was deleted by the Data ONTAP function snapcmd_delete. The unique ID for this Snapshot copy is (25, 344).
fas3070-hz*> snap delete testvol fas3070-hz(0118064597)_testvol.2
Fri Aug  1 09:46:13 CST [wafl.snap.delete:info]: Snapshot copy fas3070-hz(0118064597)_testvol.2 on volume testvol NetApp was deleted by the Data ONTAP function snapcmd_delete. The unique ID for this Snapshot copy is (22, 325).
fas3070-hz*> snap list testvol
Volume testvol
working...

 

  %/used       %/total  date          name
----------  ----------  ------------  --------
  0% ( 0%)    0% ( 0%)  Aug 01 00:00  hourly.0      
  0% ( 0%)    0% ( 0%)  Jul 31 18:00  hourly.1      
  0% ( 0%)    0% ( 0%)  Jul 31 17:00  hourly.2      
  0% ( 0%)    0% ( 0%)  Jul 31 16:00  hourly.3      
 11% (11%)    0% ( 0%)  Jul 31 15:00  hourly.4      
 11% ( 0%)    0% ( 0%)  Jul 31 14:00  hourly.5      
 12% ( 0%)    0% ( 0%)  Jul 31 13:00  hourly.6      
 12% ( 0%)    0% ( 0%)  Jul 31 12:00  hourly.7      
 12% ( 0%)    0% ( 0%)  Jul 31 11:00  hourly.8      
fas3070-hz*>

 

FAS960-hz*>  snap list testvol
Volume testvol
working...

 

  %/used       %/total  date          name
----------  ----------  ------------  --------
  0% ( 0%)    0% ( 0%)  Aug 01 09:42  fas3070-hz(0118064597)_testvol.4 (snapmirror)
  0% ( 0%)    0% ( 0%)  Aug 01 08:59  fas3070-hz(0118064597)_testvol.2 (snapmirror)
  0% ( 0%)    0% ( 0%)  Aug 01 00:00  hourly.0      
  0% ( 0%)    0% ( 0%)  Jul 31 18:00  hourly.1      
  0% ( 0%)    0% ( 0%)  Jul 31 17:00  hourly.2      
  0% ( 0%)    0% ( 0%)  Jul 31 16:00  hourly.3      
 11% (11%)    0% ( 0%)  Jul 31 15:00  hourly.4      
 11% ( 0%)    0% ( 0%)  Jul 31 14:00  hourly.5      
 12% ( 0%)    0% ( 0%)  Jul 31 13:00  hourly.6      
 12% ( 0%)    0% ( 0%)  Jul 31 12:00  hourly.7      
 12% ( 0%)    0% ( 0%)  Jul 31 11:00  hourly.8      
FAS960-hz*> snap delete testvol fas3070-hz(0118064597)_testvol.4
Fri Aug  1 09:47:39 CST [wafl.snap.delete:info]: Snapshot copy fas3070-hz(0118064597)_testvol.4 on volume testvol NetApp was deleted by the Data ONTAP function snapcmd_delete. The unique ID for this Snapshot copy is (27, 350).
FAS960-hz*> snap delete testvol fas3070-hz(0118064597)_testvol.2
Fri Aug  1 09:47:47 CST [wafl.snap.delete:info]: Snapshot copy fas3070-hz(0118064597)_testvol.2 on volume testvol NetApp was deleted by the Data ONTAP function snapcmd_delete. The unique ID for this Snapshot copy is (22, 325).
FAS960-hz*>
FAS960-hz*>  snap list testvol                                  
Volume testvol
working...

 

  %/used       %/total  date          name
----------  ----------  ------------  --------
  0% ( 0%)    0% ( 0%)  Aug 01 00:00  hourly.0      
  0% ( 0%)    0% ( 0%)  Jul 31 18:00  hourly.1      
  0% ( 0%)    0% ( 0%)  Jul 31 17:00  hourly.2      
  0% ( 0%)    0% ( 0%)  Jul 31 16:00  hourly.3      
 11% (11%)    0% ( 0%)  Jul 31 15:00  hourly.4      
 11% ( 0%)    0% ( 0%)  Jul 31 14:00  hourly.5      
 12% ( 0%)    0% ( 0%)  Jul 31 13:00  hourly.6      
 12% ( 0%)    0% ( 0%)  Jul 31 12:00  hourly.7      
 12% ( 0%)    0% ( 0%)  Jul 31 11:00  hourly.8      
FAS960-hz*>

 

检查两边文件系统状况：

FAS960-hz*> df testvol
Filesystem              kbytes       used      avail capacity  Mounted on
/vol/testvol/          8388608     370388    8018220       4%  /vol/testvol/
/vol/testvol/.snapshot    2097152      49688    2047464       2%  /vol/testvol/.snapshot

fas3070-hz*>   df testvol
Filesystem              kbytes       used      avail capacity  Mounted on
/vol/testvol/          8388608     371316    8017292       4%  /vol/testvol/
/vol/testvol/.snapshot    2097152      49692    2047460       2%  /vol/testvol/.snapshot

 

在fas960上进行反向同步：

FAS960-hz*> snapmirror resync -S fas3070-hz:testvol  FAS960-hz:testvol
The resync base snapshot will be: hourly.0
Are you sure you want to resync the volume? y
Fri Aug  1 09:51:00 CST [snapmirror.dst.resync.info:notice]: SnapMirror resync of testvol to fas3070-hz:testvol is using hourly.0 as the base snapshot.
Volume testvol will be briefly unavailable before coming back online.
Fri Aug  1 09:51:03 CST [wafl.snaprestore.revert:notice]: Reverting volume testvol to a previous snapshot.
Fri Aug  1 09:51:03 CST [wafl.vol.guarantee.replica:info]: Space for replica volume 'testvol' is not guaranteed.
Revert to resync base snapshot was successful.
Fri Aug  1 09:51:04 CST [snapmirror.dst.resync.success:notice]: SnapMirror resync of testvol to fas3070-hz:testvol successful.
Transfer started.
Monitor progress with 'snapmirror status' or the snapmirror log.
FAS960-hz*>
FAS960-hz*> snapmirror status
Snapmirror is on.
Source                             Destination                        State          Lag        Status
fas3070-hz:testvol                 FAS960-hz:testvol                  Snapmirrored   09:51:06   Transferring  (36 MB done)
FAS960-hz*> snapmirror status
Snapmirror is on.
Source                             Destination                        State          Lag        Status
fas3070-hz:testvol                 FAS960-hz:testvol                  Snapmirrored   09:51:09   Transferring
FAS960-hz*> snapmirror status
Snapmirror is on.
Source                             Destination                        State          Lag        Status
fas3070-hz:testvol                 FAS960-hz:testvol                  Snapmirrored   09:51:15   Transferring
FAS960-hz*> snapmirror status
Snapmirror is on.
Source                             Destination                        State          Lag        Status
fas3070-hz:testvol                 FAS960-hz:testvol                  Snapmirrored   00:00:19   Idle

 

对比源和目的的文件系统大小：

FAS960-hz*> df testvol                                                
Filesystem              kbytes       used      avail capacity  Mounted on
/vol/testvol/          8388608     371316    8017292       4%  /vol/testvol/
/vol/testvol/.snapshot    2097152      49692    2047460       2%  /vol/testvol/.snapshot

fas3070-hz*>   df testvol
Filesystem              kbytes       used      avail capacity  Mounted on
/vol/testvol/          8388608     371316    8017292       4%  /vol/testvol/
/vol/testvol/.snapshot    2097152      49748    2047404       2%  /vol/testvol/.snapshot

 

查看snapmirror log：

cmd Fri Aug  1 09:51:00 CST fas3070-hz:testvol FAS960-hz:testvol Resync_command (hourly.0)
dst Fri Aug  1 09:51:04 CST fas3070-hz:testvol FAS960-hz:testvol Request (Resync)
dst Fri Aug  1 09:51:09 CST fas3070-hz:testvol FAS960-hz:testvol Start
slk Fri Aug  1 09:51:24 CST state.softlock.testvol.00000145.022.fas3070-hz:testvol->FAS960-hz:testvol->fas3070-hz:testvol Softlock_delete (Clean_Softlocks)
slk Fri Aug  1 09:51:24 CST state.softlock.testvol.0000015e.027.fas3070-hz:testvol Softlock_delete (Clean_Softlocks)
dst Fri Aug  1 09:51:24 CST fas3070-hz:testvol FAS960-hz:testvol End (125644 KB)
传输量125644KB，并不是初始化的数据量，也大大超出了数据变化量，可能是丢失基准快照后对文件系统数据块map对比传输了很多数据。（有待厂商确证）

 

结论：丢失了基准快照后进行反向再同步也是没有问题的，只是传输的数据量会大于数据变化量。

 

总体结论：snapmirror很好很强大。