=:The OpenVMS Frequently Asked Questions(FAQ)C

The OpenVMS Frequently Asked Questions(FAQ)



 r \ ^  
PreviousContentsIndex

N

15.6.2 Cluster System Parameter Settings?



FThe following sections contain details of configuring cluster-related system parameters.d

15.6.2.1 What is the correct value for EXPECTED_VOTES in a VMScluster?



GThe VMScluster connection manager uses the concept of votes and quorum Hto prevent disk and memory data corruptions---when sufficient votes are @present for quorum, then access to resources is permitted. When Esufficient votes are not present, user activity will be blocked. The Gact of blocking user activity is called a "quorum hang", and is better Cthought of as a "user data integrity interlock". This mechanism is Hdesigned to prevent a partitioned VMScluster, and the resultant massive disk data corruptions.

DOn each OpenVMS node in a VMScluster, one sets two values in SYSGEN:AVOTES, and EXPECTED_VOTES. The former is how many votes the node Gcontributes to the VMScluster. The latter is the total number of votes 2expected when the full VMScluster is bootstrapped.

HSome sites erroneously attempt to set EXPECTED_VOTES too low, believing Gthat this will allow when only a subset of voting nodes are present in <a VMScluster. It does not. Further, an erroneous setting in FEXPECTED_VOTES is automatically corrected once VMScluster connections ?to other nodes are established; user data is at risk of severe Ccorruptions during the earliest and most vulnerable portion of the ?system bootstrap, before the connections have been established.

GOne can operate a VMScluster with one, two, or many voting nodes. With Bany but the two-node configuration, keeping a subset of the nodes Hactive when some nodes fail can be easily configured. With the two-node Econfiguration, one must use a primary-secondary configuration (where Hthe primary has all the votes), a peer configuration (where when either Enode is down, the other hangs), or (preferable) a shared quorum disk.

GUse of a quorum disk does slow down VMScluster transitions somewhat -- Fthe addition of a third voting node that contributes the vote(s) that Hwould be assigned to the quorum disk makes for faster transitions---but Bthe use of a quorum disk does mean that either node in a two-node AVMScluster configuration can operate when the other node is down.

=If you choose to use a quoum disk, a QUORUM.DAT file will be automaticallyHcreated when OpenVMS first boots and when a quorum disk is specified -- Awell, the QUORUM.DAT file will be created when OpenVMS is booted 4without also needing the votes from the quorum disk.

GIn a two-node VMScluster with a shared storage interconnect, typically ?each node has one vote, and the quorum disk also has one vote. EXPECTED_VOTES is set to three.

FUsing a quorum disk on a non-shared interconnect is unnecessary---the Huse of a quorum disk does not provide any value, and the votes assigned Bto the quorum disk should be assigned to the OpenVMS host serving access to the disk.

DFor information on quorum hangs, see the OpenVMS documentation. For Finformation on changing the EXPECTED_VOTES value on a running system, Fsee the SET CLUSTER/EXPECTED_VOTES command, and see the documentation Hfor the AMDS and Availability Manager tools. Also of potential interest Gis the OpenVMS system console documentation for the processor-specific Dconsole commands used to trigger the IPC (Interrrupt Priority Level F%x0C; IPL C) handler. AMDS, Availability Manager, and the IPC handler Fcan each be used to clear a quorum hang. Use of AMDS and Availability HManager is generally recommended over IPC, particularly because IPC can Fcause CLUEXIT bugchecks if the system should remain halted beyond the cluster sanity timer limits.

DThe quorum scheme is a set of "blade guards" deliberately Fimplemented by OpenVMS Engineering to provide data integrity---remove Hthese blade guards at your peril. OpenVMS Engineering did not implement Cthe quorum mechanism to make your life more difficult---quorum was 5implemented to keep your data from getting scrambled.X

15.6.2.2 Explain disk (or tape) allocation class settings?



FThe allocation class mechanism provides the system manager with a way Dto configure and resolve served and direct paths to storage devices Hwithin a cluster. Any served device that provides multiple paths should Hbe configured using a non-zero allocation class, either at the MSCP (or GTMSCP) storage controllers, at the port (for port allocation classes), Eor at the OpenVMS MSCP (or TMSCP) server. All controllers or servers Dproviding a path to the same device should have the same allocation 1class (at the port, controller, or server level).

FEach disk (or tape) unit number used within a non-zero disk (or tape) Eallocation class must be unique, regardless of the particular device Fprefix. For the purposes of multi-path device path determination, any Fdisk (or tape) device with the same unit number and the same disk (or Ftape) allocation class configuration is assumed to be the same device.

GIf you are reconfiguring disk device allocation classes, you will want Eto avoid the use of allocation class one ($1$) until/unless you have FFibre Channel storage configured. (Fibre Channel storage specifically 8requires the use of allocation class $1$. eg: $1$DGA0:.)a

15.6.2.2.1 How to configure allocation classes and Multi-Path SCSI?



FThe HSZ allocation class is applied to devices, starting with OpenVMS EV7.2. It is considered a port allocation class (PAC), and all device Hnames with a PAC have their controller letter forced to "A". (You might ?infer from the the text in the "Guidelines for OpenVMS Cluster FConfigurations" that this is something you have to do, though OpenVMS 0will thoughtfully handle this renaming for you.)

>You can force the device names back to DKB by setting the HSZ Gallocation class to zero, and setting the PKB PAC to -1. This will use Fthe host allocation class, and will leave the controller letter alone E(that is, the DK controller letter will be the same as the SCSI port H(PK) controller). Note that this won't work if the HSZ is configured in Gmultibus failover mode. In this case, OpenVMS requires that you use an allocation class for the HSZ.

CWhen your configuration gets even moderately complex, you must pay Bcareful attention to how you assign the three kinds of allocation Cclass: node, port and HSZ/HSJ, as otherwise you could wind up with 7device naming conflicts that can be painful to resolve.

FThe display-able path information is for SCSI multi-path, and permits Fthe multi-path software to distinguish between different paths to the Gsame device. If you have two paths to $1$DKA100, for example by having Htwo KZPBA controllers and two SCSI buses to the HSZ, you would have two >UCBs in a multi-path set. The path information is used by the :multi-path software to distinguish between these two UCBs.

GThe displayable path information describes the path; in this case, the DSCSI port. If port is PKB, that's the path name you get. The device Hname is no longer completely tied to the port name; the device name now Ddepends on the various allocation class settings of the controller, SCSI port or node.

EThe reason the device name's controller letter is forced to "A" when @you use PACs is because a shared SCSI bus may be configured via Hdifferent ports on the various nodes connected to the bus. The port may Ebe PKB on one node, and PKC on the other. Rather obviously, you will Hwant to have the shared devices use the same device names on all nodes. BTo establish this, you will assign the same PAC on each node, and FOpenVMS will force the controller letter to be the same on each node. GSimply choosing "A" was easier and more deterministic than negotiating @the controller letter between the nodes, and also parallels the Gsolution used for this situation when DSSI or SDI/STI storage was used.

ETo enable port allocation classes, see the SYSBOOT command SET/BOOT, +and see the DEVICE_NAMING system parameter.

>This information is also described in the Cluster Systems and 6Guidelines for OpenVMS Cluster Configurations manuals.P

15.6.3 Tell me about SET HOST/DUP and SET HOST/HSC



CThe OpenVMS DCL commands SET HOST/DUP and SET HOST/HSC are used to Hconnect to storage controllers via the Diagnostics and Utility Protocol A(DUP). These commands require that the FYDRIVER device driver be Cconnected. This device driver connection is typically performed by @adding the following command(s) into the system startup command procedure:

On OpenVMS Alpha:

 

"
$ RUN SYS$SYSTEM:SYSMAN 9SYSMAN> IO CONNECT FYA0/NOADAPTER/DRIVER=SYS$FYDRIVER 




On OpenVMS VAX:

 

"
$ RUN SYS$SYSTEM:SYSGEN "SYSGEN> CONNECT FYA0/NOADAPTER 




EAlternatives to the DCL SET HOST/DUP command include the console SET FHOST command available on various mid- to recent-vintage VAX consoles:

4Access to Parameters on an Embedded DSSI controller:

 

"
6SET HOST/DUP/DSSI[/BUS:{0:1}] dssi_node_number PARAMS 




<Access to Directory of tools on an Embedded DSSI controller:

 

"
6SET HOST/DUP/DSSI[/BUS:{0:1}] dssi_node_number DIRECT 




0Access to Parameters on a KFQSA DSSI controller:

 

"
2SHOW UQSSP ! to get port_controller_number PARAMS 1SET HOST/DUP/UQSSP port_controller_number PARAMS 




EThese console commands are available on most MicroVAX and VAXstation B3xxx series systems, and most (all?) VAX 4xxx series systems. For Dfurther information, see the system documentation and---on most VAX $systems---see the console HELP text.

FEK-410AB-MG, _DSSI VAXcluster Installation and Troubleshooting_, is a Egood resource for setting up a DSSI VMScluster on OpenVMS VAX nodes. H(This manual predates coverage of OpenVMS Alpha systems, but gives good >coverage to all hardware and software aspects of setting up a FDSSI-based VMScluster---and most of the concepts covered are directly Eapplicable to OpenVMS Alpha systems. This manual specifically covers Ethe hardware, which is something not covered by the standard OpenVMS VMScluster documentation.)

kAlso see Section 15.3.3, and for the SCS name of the OpenVMS host see 0Section 5.6.K

15.6.4 How do I rename a DSSI disk (or tape?)



AIf you want to renumber or rename DSSI disks or DSSI tapes, it's ,easy---if you know the secret incantation...

From OpenVMS:

 

"
$ RUN SYS$SYSTEM:SYSGEN "SYSGEN> CONNECT FYA0/NOADAPTER SYSGEN> ^Z @$ SET HOST/DUP/SERV=MSCP$DUP/TASK=PARAMS <DSSI-NODE-NAME> ... PARAMS> STAT CONF F<The software version is normally near the top of the display.> PARAMS> EXIT ... 




EFrom the console on most 3000- and 4000-class VAX system consoles... <(Obviously, the system must be halted for these commands...)

Integrated DSSI:

 

"
6SET HOST/DUP/DSSI[/BUS:[0:1]] dssi_node_number PARAMS 




KFQSA:

 

"
1SET HOST/DUP/UQSSP port_controller_number PARAMS 




FFor information on how to get out into the PARAMS subsystem, also see Hthe HELP at the console prompt for the SET HOST syntax, or see the HELP @on SET HOST /DUP (once you've connected FYDRIVER under OpenVMS).

EOnce you are out into the PARAMS subsystem, you can use the FORCEUNI Coption to force the use of the UNITNUM value and then set a unique DUNITNUM inside each DSSI ISE---this causes each DSSI ISE to use the Cspecfied unit number and not use the DSSI node as the unit number. FOther parameters of interest are NODENAME and ALLCLASS, the node name 0and the (disk or tape) cluster allocation class.

FEnsure that all disk unit numbers used within an OpenVMS Cluster disk Fallocation class are unique, and all tape unit numbers used within an FOpenVMS Cluster tape allocation class are also unique. For details on jthe SCS name of the OpenVMS host, see Section 5.6. For details of SET BHOST/DUP, see Section 15.6.3.]

15.6.5 Where can I get Fibre Channel Storage (SAN) information?

K

15.6.6 How can I split up an OpenVMS Cluster?



?Review the VMScluster documentation, and the System Management Hdocumentation. The following are the key points, but are likely not the $only things you will need to change.

BOpenVMS Cluster support is directly integrated into the operating Csystem, and there is no way to remove it. You can, however, remote @site-specific tailoring that was added for a particular cluster configuration.

EFirst: Create restorable image BACKUPs of each of the current system Edisks. If something gets messed up, you want a way to recover, right?

FCreate standalone BACKUP kits for the OpenVMS VAX systems, and create >or acquire bootable BACKUP kits for the OpenVMS Alpha systems.

FUse CLUSTER_CONFIG or CLUSTER_CONFIG_LAN to remove the various system <roots and to shut off boot services and VMScluster settings.

CCreate as many architecture-specific copies of the system disks as Brequired. Realize that the new systems will all likely be booting Gthrough root SYS0---if you have any system-specific files in any other roots, save them.

HRelocate the copies of the VMScluster common files onto each of the new system disks.

HReset the console parameters and boot flags on each system for use on a standalone node.

GReset the VAXCLUSTER and NISCS_LOAD_PEA0 parameters to 0 in SYSGEN and in MODPARAMS.DAT.

:Clobber the VMScluster group ID and password using SYSMAN.

7Reboot the systems seperately, and run AUTOGEN on each.

@Shut off MOP services via NCP or LANCP on the boot server nodes.

HPermanent seperation also requires the duplication of shared files. For Aa list of the files commonly shared, please see the most current Gversion of SYS$STARTUP:SYLOGICALS.TEMPLATE, and specifically a version Efrom OpenVMS V7.2 or later. The following files are typically shared within a cluster:

 

"
K  Filename:              default directory (in common root) and file type: 0    SYSUAF                      SYS$SYSTEM:.DAT 0    SYSUAFALT                   SYS$SYSTEM:.DAT 0    SYSALF                      SYS$SYSTEM:.DAT 0    RIGHTSLIST                  SYS$SYSTEM:.DAT 0    NETPROXY                    SYS$SYSTEM:.DAT 0    NET$PROXY                   SYS$SYSTEM:.DAT 0    NETOBJECT                   SYS$SYSTEM:.DAT 0    NETNODE_REMOTE              SYS$SYSTEM:.DAT M    QMAN$MASTER                 SYS$SYSTEM: (this is a set of related files) 0    LMF$LICENSE                 SYS$SYSTEM:.LDB 1    VMSMAIL_PROFILE             SYS$SYSTEM:.DATA 0    VMS$OBJECTS                 SYS$SYSTEM:.DAT 1    VMS$AUDIT_SERVER            SYS$MANAGER:.DAT 1    VMS$PASSWORD_HISTORY        SYS$SYSTEM:.DATA 1    NETNODE_UPDATE              SYS$MANAGER:.COM 1    VMS$PASSWORD_POLICY         SYS$LIBRARY:.EXE A    LAN$NODE_DATABASE           SYS$SYSTEM:LAN$NODE_DATABASE.DAT 




BAlso see the topics on "cluster divorce" in the Ask The Wizard area.



WFor additional information, please see Section 3.9.

bInformation on changing node names is included in Section 5.6.B

15.6.7 Details on Volume Shadowing?



EThis section contains information on host-based volume shadowing; on 9the disk mirroring capabilities available within OpenVMS.c

15.6.7.1 Does volume shadowing require a non-zero allocation classes?



BYes, use of host-based Volume Shadowing requires that the disk(s) 6involved be configured in a non-zero allocation class.

<Edit SYS$SYSTEM:MODPARAMS.DAT to include a declaration of anHnon-zero allocation class, such as setting the host allocation class to the value 7:

 

"
ALLOCLASS = 7 




$Then AUTOGEN the system, and reboot.

HYou should now be able to form the shadow set via a command such as the following:

 

"
=$ MOUNT dsa1007: /SHADOW=($7$dkb300:,$7$dkb500:) volumelabel 




CWhen operating in an OpenVMS Cluster, this sequence will typically @change the disk names from the SCSNODE prefix (scsnode$dkann) toI the allocation-class prefix ($7$dkannn). This may provide you with the H opportunity to move to a device-independent scheme using logical name H constructs such as the DISK$volumelabel logical names in your startup D and application environments; an opportunity to weed out physical  device references.

FAllocation class one is used by Fibre Channel devices; it can be best Fto use another non-zero allocation class even if Fibre Channel is not /currently configured and not currently planned.N

15.6.7.2 Volume Shadowing MiniCopy vs MiniMerge?



AMiniMerge support has been available for many years with OpenVMS Fhost-based volume shadowing, so long as you had MSCP controllers (eg: EHSC, HSJ, or HSD) which supported the Volume Shadowing Assist called ""Write History Logging".

HIf you want minimerges on HSG80 (Fibre Channel) controllers, please see ?the "Fibre Channel in a Disaster-Tolerant OpenVMS Cluster System" whitepaper at:



FMinimerge support on HSG80 is expected to require ACS 8.7 and OpenVMS >Alpha V7.3-1, assuming the development goes according to plan.

EThe following sections describe both MiniCopy and MiniMerge, and can provide a basis for discussions.5

15.6.7.2.1 MiniCopy?



@A Shadowing Full Copy occurs when you add a disk to an existing Eshadowset using a MOUNT command; the entire contents of the disk are Ceffectively copied to the new member (using an algorithm that goes Hthrough in 127-block increments and reads one member, compares with the Dtarget disk, and if the data differs, writes the data to the target Gdisk and loops back to the read step, until the data is equal for that D127-block section). (This is one of the reasons why the traditional Brecommendation for adding new volumes to a shadowset was to use a EBACKUP/PHYSICAL copy of an existing shadowset volume, simply because Fthe reads then usually matched and thus shadowing usually avoided the need for the writes.)

AIf you warn OpenVMS ahead of time (at dismount time) that you're Hplanning to remove a disk from a shadowset but re-add it later, OpenVMS Fwill keep a bitmap tracking what areas of the disk have been modified Fwhile the disk was out of the shadowset, and when you re-add it later Awith a MOUNT command OpenVMS only has to update the areas of the Freturned disk that the bit-map indicates are now out-of-date. OpenVMS Edoes this with a read source / write target algorithm, which is much Ffaster than the shenanigans the Full Copy does, so even if all of the 8disk has changed, a MiniCopy is faster than a Full Copy.6

15.6.7.2.2 MiniMerge?



CA Shadowing Merge is initiated when an OpenVMS node in the cluster <(which had a shadowset mounted) crashes or otherwise leaves Eunexpectedly, without dismounting the shadowset first. In this case, @OpenVMS must ensure that the data is identical, since Shadowing Hguarantees that the data on the disks in a shadowset will be identical. EIn a regular Merge operation, Shadowing uses an algorithm similar to Athe Full Copy algorithm (except that it can choose either of the Hmembers' contents as the source data, since both are considered equally Gvalid), and scans the entire disk. Also, to make things worse, for any Eread operations in the area ahead of what has been merged, Shadowing Hwill first merge the area containing the read data, then allow the read to occur.

CA Merge can be very time-consuming and very I/O intensive, so some =controllers have Shadowing Assists to make it faster. If the Fcontrollers support Write History Logging, the controllers record the Eareas (LBNs) that are the subject of Shadowing writes, and if a node Hcrashes, the surviving nodes can query the controllers to find out what Eexact areas of the disk the departed node was writing to just before Hthe crash, and thus Shadowing only needs to merge just those few areas, Fso this tends to take seconds as opposed to hours for a regular Merge.




 ^ \  
IndexContents