Managing the rootdg diskgroup

Veritas volume manager requires that each disk be part of a diskgroup. Furthermore, there is a special diskgroup called "rootdg" which must exist for VxVM to function. The administrator can create additional diskgroups, but the first diskgroup must be named rootdg.

Veritas recommends that the rootdg diskgroup contain the encapsulated operating system disk and its mirror. In order to preserve the data on the operating system disk, the vxinstall script encapsulates the root disk. This preserves the operating system while at the same time creating the private and public disk regions that Veritas requires. However, the process of encapsulation fundamentally alters the structure of the root disk.

Furthermore, the root disk is typically the only disk on the system that is encapsulated. Typically, every other disk is initialized by VxVM, and then user data is placed onto it. The administrator soon comes to understand the nature of initialized disks, but does not get sufficient exposure to an encapsulated disk. This lack of exposure makes it difficult to maintain the critical root disk in the event of a disk failure.

We will demonstrate alternative approaches to managing the rootdg diskgroup:

• In the first case, we will mirror the operating system with Solstice DiskSuite and then create a single-slice rootdg. With this approach, the private and public region of the required rootdg diskgroup is constrained to a single disk slice of a few cylinders.
• The second example will demonstrate what is commonly referred to as the Howard/Trantham method described in the August 2000 Sun Blueprint article .

With this approach, both Solstice DiskSuite and Veritas volume manager are installed on a single host. DiskSuite is used to mirror the operating system, and Veritas volume manager is used to manage all the application data. This approach uses the strengths of each product, and requires that the administrative staff understands both products.

Note that the use of a single-slice rootdg is considered an unsupported configuration by both Veritas and Sun. The "upgrade-start" and "vxunroot" scripts will not work if you use the single-slice rootdg. However, the fact that the operating system is not under control of VxVM obviates the need for "upgrade-start" and "vxunroot" scripts.

So long as you understand what you are doing, the single-slice rootdg solution solves more problems than it creates.

We recommend that you install the operating system so that there are three unused slices of roughly 10MB in size. Two of these slices will be used for DiskSuite state database replicas, and the final slice will be used to the single-slice rootdg diskgroup.

1. First, install DiskSuite and mirror the operating system.
2. Next, install the Veritas volume manager packages. Do not run vxinstall.
3. Finally, make use of the unused slice seven on disks c0t0d0 and c0t1d0 to create a rootdg diskgroup on those slices.

    # vxconfigd -m disable
    # vxdctl init
    # vxdg init rootdg
    # vxdctl add disk c0t0d0s7 type=simple
    # vxdisk -f init c0t0d0s7 type=simple
    # vxdg adddisk c0t0d0s7
    # vxdctl add disk c0t1d0s7 type=simple
    # vxdisk -f init c0t1d0s7 type=simple
    # vxdg adddisk c0t1d0s7
    # vxdctl enable
    # rm /etc/vx/reconfig.d/state.d/install-db
   

In the preceding sequence, VxVM was told to treat slice seven of each disk as a "simple disk" (as opposed to a sliced disk). Consequently, VxVM placed both the private and public region on slice seven. We then added both those "disks" (c0t0d0s7 and c0t1d0s7) into the rootdg diskgroup. This should be clearer from the output of "vxprint -ht":

# vxprint -ht                                                          
Disk group: rootdg                                                              
                                                                                
DG NAME         NCONFIG      NLOG     MINORS   GROUP-ID                         
DM NAME         DEVICE       TYPE     PRIVLEN  PUBLEN   STATE                   
RV NAME         RLINK_CNT    KSTATE   STATE    PRIMARY  DATAVOLS  SRL           
RL NAME         RVG          KSTATE   STATE    REM_HOST REM_DG    REM_RLNK      
V  NAME         RVG          KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
PL NAME         VOLUME       KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE 
SD NAME         PLEX         DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE 
SV NAME         PLEX         VOLNAME  NVOLLAYR LENGTH   [COL/]OFF AM/NM    MODE 
DC NAME         PARENTVOL    LOGVOL                                             
SP NAME         SNAPVOL      DCO                                                
                                                                                
dg rootdg       default      default  0        1036007852.1025.pegasus          
                                                                                
dm c0t0d0s7     c0t0d0s7     simple   1024     22495    -                       
dm c0t1d0s7     c0t1d0s7     simple   1024     22495    -

Originally presented in the August 2000 Sun Blueprint article , the following procedure results in a root disk and mirror that have been initialized rather than encapsulated. This configuration is easier to maintain in the event of a disk failure.

1. Install the Veritas volume manager packages.
2. Install any required Veritas licenses.

3. Use vxinstall to encapsulate the root disk c0t0d0. Do not initialize any other disks. Do not mirror the operating system. Once the root disk c0t0d0 has been encapsulated and the system reboots, the output of "vxprint -ht" is as follows:

   # vxprint -ht
   Disk group: rootdg
   
   DG NAME         NCONFIG      NLOG     MINORS   GROUP-ID
   DM NAME         DEVICE       TYPE     PRIVLEN  PUBLEN   STATE
   RV NAME         RLINK_CNT    KSTATE   STATE    PRIMARY  DATAVOLS  SRL
   RL NAME         RVG          KSTATE   STATE    REM_HOST REM_DG    REM_RLNK
   V  NAME         RVG          KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
   PL NAME         VOLUME       KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE
   SD NAME         PLEX         DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
   SV NAME         PLEX         VOLNAME  NVOLLAYR LENGTH   [COL/]OFF AM/NM    MODE
   DC NAME         PARENTVOL    LOGVOL
   SP NAME         SNAPVOL      DCO
   
   dg rootdg       default      default  0        1036078650.1025.pegasus
   
   dm rootdisk     c0t0d0s2     sliced   3359     17690400 -
   
   v  rootvol      -            ENABLED  ACTIVE   13423200 ROUND     -        root
   pl rootvol-01   rootvol      ENABLED  ACTIVE   13423200 CONCAT    -        RW
   sd rootdisk-B0  rootvol-01   rootdisk 17690399 1        0         c0t0d0   ENA
   sd rootdisk-02  rootvol-01   rootdisk 0        13423199 1         c0t0d0   ENA
   
   v  swapvol      -            ENABLED  ACTIVE   2100000  ROUND     -        swap
   pl swapvol-01   swapvol      ENABLED  ACTIVE   2100000  CONCAT    -        RW
   sd rootdisk-01  swapvol-01   rootdisk 13423199 2100000  0         c0t0d0   ENA
   
   v  var          -            ENABLED  ACTIVE   2100000  ROUND     -        fsgen
   pl var-01       var          ENABLED  ACTIVE   2100000  CONCAT    -        RW
   sd rootdisk-03  var-01       rootdisk 15523199 2100000  0         c0t0d0   ENA
   

4. Initialize the root mirror disk c0t1d0:

   # /usr/lib/vxvm/bin/vxdisksetup -i c0t1d0
# vxdg adddisk rootmirror=c0t1d0

5. Manually attach the mirror plexes to the root volumes. The "vxrootmir" command mirrors the rootvol volume. We then manually mirror the swapvol and var volumes in the order that we desire:

   # /usr/lib/vxvm/bin/vxrootmir rootmirror
# vxassist mirror swapvol alloc=rootmiror
# vxassist mirror var alloc=rootmiror

   The mirror process may take a while to complete. One can monitor its progress via the "vxtask list" command.

6. Now that the root disk is mirrored, disassociate the original plexes on c0t0d0, remove them, and remove the encapsulated rootdisk from the rootdg diskgroup:

   # vxplex dis rootvol-01 swapvol-01 var-01
# vxedit -r rm rootvol-01 swapvol-01 var-01
# vxdg rmdisk rootdisk

   At this point, the operating system is based just on the initialized disk c0t1d0:

   # vxprint -ht
   Disk group: rootdg
   
   DG NAME         NCONFIG      NLOG     MINORS   GROUP-ID
   DM NAME         DEVICE       TYPE     PRIVLEN  PUBLEN   STATE
   RV NAME         RLINK_CNT    KSTATE   STATE    PRIMARY  DATAVOLS  SRL
   RL NAME         RVG          KSTATE   STATE    REM_HOST REM_DG    REM_RLNK
   V  NAME         RVG          KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
   PL NAME         VOLUME       KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE
   SD NAME         PLEX         DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
   SV NAME         PLEX         VOLNAME  NVOLLAYR LENGTH   [COL/]OFF AM/NM    MODE
   DC NAME         PARENTVOL    LOGVOL
   SP NAME         SNAPVOL      DCO
   
   dg rootdg       default      default  0        1036078650.1025.pegasus
   
   dm rootmirror   c0t1d0s2     sliced   6463     17683680 -
   
   v  rootvol      -            ENABLED  ACTIVE   13423200 ROUND     -        root
   pl rootvol-02   rootvol      ENABLED  ACTIVE   13423200 CONCAT    -        RW
   sd rootmirror-01 rootvol-02  rootmirror 0      13423200 0         c0t1d0   ENA
   
   v  swapvol      -            ENABLED  ACTIVE   2100000  ROUND     -        swap
   pl swapvol-02   swapvol      ENABLED  ACTIVE   2100000  CONCAT    -        RW
   sd rootmirror-02 swapvol-02  rootmirror 13423200 2100000 0        c0t1d0   ENA
   
   v  var          -            ENABLED  ACTIVE   2100000  ROUND     -        fsgen
   pl var-02       var          ENABLED  ACTIVE   2100000  CONCAT    -        RW
   sd rootmirror-03 var-02      rootmirror 15523200 2100000 0        c0t1d0   ENA
   

7. Initialize (as opposed to encapsulate) c0t0d0:

   # /usr/lib/vxvm/bin/vxdisksetup -i c0t0d0
# vxdg adddisk rootdisk=c0t0d0

8. Now, mirror the root volumes from c0t1d0 to the freshly initialized rootdisk c0t0d0:

   # /usr/lib/vxvm/bin/vxrootmir rootdisk
# vxassist mirror swapvol alloc=rootdisk
# vxassist mirror var alloc=rootdisk

   Once the mirror process is complete, the output of "vxprint -ht" is as follows. Note that there is no longer a "rootdisk-BO" subdisk, which is used by VxVM to protect the boot block:

   # vxprint -ht
   Disk group: rootdg
   
   DG NAME         NCONFIG      NLOG     MINORS   GROUP-ID
   DM NAME         DEVICE       TYPE     PRIVLEN  PUBLEN   STATE
   RV NAME         RLINK_CNT    KSTATE   STATE    PRIMARY  DATAVOLS  SRL
   RL NAME         RVG          KSTATE   STATE    REM_HOST REM_DG    REM_RLNK
   V  NAME         RVG          KSTATE   STATE    LENGTH   READPOL   PREFPLEX UTYPE
   PL NAME         VOLUME       KSTATE   STATE    LENGTH   LAYOUT    NCOL/WID MODE
   SD NAME         PLEX         DISK     DISKOFFS LENGTH   [COL/]OFF DEVICE   MODE
   SV NAME         PLEX         VOLNAME  NVOLLAYR LENGTH   [COL/]OFF AM/NM    MODE
   DC NAME         PARENTVOL    LOGVOL
   SP NAME         SNAPVOL      DCO
   
   dg rootdg       default      default  0        1036078650.1025.pegasus
   
   dm rootdisk     c0t0d0s2     sliced   6463     17683680 -
   dm rootmirror   c0t1d0s2     sliced   6463     17683680 -
   
   v  rootvol      -            ENABLED  ACTIVE   13423200 ROUND     -        root
   pl rootvol-01   rootvol      ENABLED  ACTIVE   13423200 CONCAT    -        RW
   sd rootdisk-01  rootvol-01   rootdisk 0        13423200 0         c0t0d0   ENA
   pl rootvol-02   rootvol      ENABLED  ACTIVE   13423200 CONCAT    -        RW
   sd rootmirror-01 rootvol-02  rootmirror 0      13423200 0         c0t1d0   ENA
   
   v  swapvol      -            ENABLED  ACTIVE   2100000  ROUND     -        swap
   pl swapvol-01   swapvol      ENABLED  ACTIVE   2100000  CONCAT    -        RW
   sd rootdisk-02  swapvol-01   rootdisk 13423200 2100000  0         c0t0d0   ENA
   pl swapvol-02   swapvol      ENABLED  ACTIVE   2100000  CONCAT    -        RW
   sd rootmirror-02 swapvol-02  rootmirror 13423200 2100000 0        c0t1d0   ENA
   
   v  var          -            ENABLED  ACTIVE   2100000  ROUND     -        fsgen
   pl var-01       var          ENABLED  ACTIVE   2100000  CONCAT    -        RW
   sd rootdisk-03  var-01       rootdisk 15523200 2100000  0         c0t0d0   ENA
   pl var-02       var          ENABLED  ACTIVE   2100000  CONCAT    -        RW
   sd rootmirror-03 var-02      rootmirror 15523200 2100000 0        c0t1d0   ENA
   

9. Create disk slices corresponding to the Veritas volume manager subdisks. The syntax of the vxmksdpart command is:

   vxmksdpart [-f] [-g diskgroup] subdisk sliceno [tag flags]

   According to the fmthard manual page, tag and flag numbers are defined as follows:

   tag	description
   0x00	UNASSIGNED
   0x01	BOOT
   0x02	ROOT
   0x03	SWAP
   0x04	USR
   0x05	BACKUP
   0x06	STAND
   0x07	VAR
   0x08	HOME

   flag	description
   0x00	MOUNTABLE
   0x01	UNMOUNTABLE
   0x10	READ-ONLY

   Since the vxrootmir command creates the slices for the root (/) partition, there is no need to run vxmksdpart for this slice. However, one needs to create slices for swap, var, and any other slices on the rootdisk and rootmirror disks.

   First, create the slices for the swap slice on both disks:

   # /usr/lib/vxvm/bin/vxmksdpart rootdisk-02 1 0x03 0x01
# /usr/lib/vxvm/bin/vxmksdpart rootmirror-02 1 0x03 0x01

   Create the slices for the var partition on both disks. Since slices three and four are normally used by Veritas for private and public regions, we will use slice five for the var partition:

   # /usr/lib/vxvm/bin/vxmksdpart rootdisk-03 5 0x07 0x00
# /usr/lib/vxvm/bin/vxmksdpart rootmirror-03 5 0x07 0x00

   To assist in recovery, update the /etc/vfstab and /etc/vfstab.prevm files to reflect the slice selections made with vxmksdpart. Furthermore, capture the current disk partition information for both disks:

   # /usr/lib/vxvm/bin/vxprtvtoc -f /tmp/v1 /dev/rdsk/c0t0d0s2
# /usr/lib/vxvm/bin/vxprtvtoc -f /tmp/v2 /dev/rdsk/c0t1d0s2
# cat /tmp/v1 /tmp/v2 > /etc/vtoc

10. Specify the primary swap partition as the system dump device:

    # dumpadm -d /dev/dsk/c0t0d0s1
          Dump content: kernel pages
           Dump device: /dev/dsk/c0t0d0s1 (swap)
    Savecore directory: /var/crash/pegasus
      Savecore enabled: yes
    

11. If the primary boot disk should fail, make it easy to boot from the mirror. Some sites choose to alter the OBP "boot-device" variable; in this case, we choose to simply define the device aliases "vx-rootdisk" and "vx-rootmirror". In the event that the primary boot device ("disk" or "vx-rootdisk") should fail, the administrator simply needs to type "boot vx-rootmirror" at the OBP prompt.

    Determine the device path to the boot devices for both the primary and mirror:

    # ls -l /dev/dsk/c0t0d0s0 /dev/dsk/c0t1d0s0
    lrwxrwxrwx   1 root     root          41 Oct 17 11:48 /dev/dsk/c0t0d0s0 -> ../..
    /devices/pci@1f,4000/scsi@3/sd@0,0:a
    lrwxrwxrwx   1 root     root          41 Oct 17 11:48 /dev/dsk/c0t1d0s0 -> ../..
    /devices/pci@1f,4000/scsi@3/sd@1,0:a
    

    Use the device paths to define the vx-rootdisk and vx-rootmirror device aliases (note that we use the label "disk" instead of "sd" in the device alias path):

    # eeprom "nvramrc=devalias vx-rootdisk /pci@1f,4000/scsi@3/disk@0,0
    devalias vx-rootmirror /pci@1f,4000/scsi@3/disk@1,0"
    # eeprom "use-nvramrc?=true"
    

    Test the process of booting from either vx-rootdisk or vx-rootmirror.