Labels

Saturday, January 17, 2009

Virtual 2003 MSCS Cluster in ESX VI3

This post shares a method I've used to create test-lab instances of standard 2003 file and print Microsoft Cluster Services (MSCS) clusters in a VMware ESX VI3 virtual environment. The resultant solution is not supported and definitely not production-ready, but if you want a real multi-node MSCS cluster in an ESX lab environment, this process might be helpful with a minimum set of requirements.

With my usual theme of repeatable command-line execution, most of these operations can be completed via the command-line, either in the ESX service console or a command-prompt from the virtual MSCS nodes.

I followed bits and pieces of the VMware supported method - which is very specific and quite restrictive. Note that I’m a little dubious that this cluster would be particularly stable – the SCSI reservations MSCS uses to lock disks are in no way supported when using a shared VMDK through a shared SCSI adapter (I think RDM is the only supported method), but it does work and at least provided me with a test environment.

The shared nothing model of 2003 MSCS clustering dictates that only one node accesses the partition at any one time, but the disk still needs to be visible to both nodes. A limitation of this solution is that both MSCS nodes need to be hosted on one ESX server – a requirement you could satisfy with a DRS rule to keep the two nodes together. However, if DRS decided to migrate both VMs, the cluster would almost certainly break during the failover (and possibly after).

If you follow the steps below, you should end up with two virtual x64 2003 enterprise servers, both members of a single MSCS cluster. In the cluster there will three shared disks (VMDKs), one for the quorum and one each for file and print – with a virtual server and relevant cluster resources. A test file share is created, along with drivers and a test printer. You'll need to modify the commands that reference the public adapter and IP addresses

Steps involved:

  1. Create an area for storage of the shared disk on your datastore:
    1. mkdir /vmfs/volumes/%datastore%/cluster01
  2. Create a 5GB quorum disk:
    1. vmkfstools -d thick -a lsilogic -c 5G /vmfs/volumes/%datastore%/cluster01/MSCS-Quorum.vmdk
  3. Create a 5GB disk for shared data:
    1. vmkfstools -d thick -a lsilogic -c 5G /vmfs/volumes/%datastore%/cluster01/MSCS-disk01.vmdk
  4. Create two 2003 x64 enterprise virtual machines, either through cloning, deployment with templates or whatever your standard build process may be
  5. If cloning was used, run sysprep on both nodes to give a unique SID and join your lab domain
  6. Shutdown the first node and add the shared disk
    1. Add the quorum disk, mounted under scsi 1:0 (which adds a new SCSI adapter)
    2. Set the newly created SCSI Adapter to SCSI bus sharing virtual
    3. Add disk01, attached as scsi 1:1
  7. In the first VM, use disk administrator (or diskpart) to initialise the quorum and disk01 disks, partitioned with basic. Record the signature of the disk and the drive letter used (although this is the disk volume when the disk is owned by the OS, not the cluster).
  8. Add a service account for the cluster service:
    1. dsadd user "CN=clustersvc,CN=Users,DC=test,DC=local" -pwdneverexpires yes -pwd password -disabled no -desc "MSCS VM cluster service account"
    2. Ensure the service account is an administrator of each virtual 2003 node
  9. Use Cluster Administrator to install the cluster on the first node, with your chosen cluster name, using the created quorum disk and service account
  10. Verify correct operation of the single-node cluster, and then add the second VM node to the cluster.
  11. Create a new port group to allow a second private adapter on each ESX server:
    1. esxcfg-vswitch -A MSCS-Private Private
    2. Add a second interface to each VM cluster node, allocated separate address space
    3. Verify connectivity (ping) and configuration following cluster best practices (no gateway, no DNS etc)
    4. Mark as a private heartbeat connection for the cluster, prioritised above the LAN connection.
  12. Create a virtual resource group, creating IP, network name and disk resources in the group, the following commands will create a group called v01, in the lab01 cluster. For these steps, you’ll need the drive letter to use (M: below), the disk signature, the public network name, IP Address and subnet mask of the virtual server being created:
    1. cluster /cluster:lab01 group "v01" /create
    2. cluster /cluster:lab01 res "v01 Disk01" /create /group:"v01" /type:"physical disk"
    3. cluster /cluster:lab01 res "v01 Disk01" /priv Drive="M:"
    4. cluster /cluster:lab01 res "v01 Disk01" /priv signature=0x%disksignature%
    5. cluster /cluster:lab01 res "v01 Disk01" /prop Description="M: disk01"
    6. cluster /cluster:lab01 res "v01 Disk01" /On
    7. cluster /cluster:lab01 res "v01 IP" /create /group:"v01" /type:"IP Address"
    8. cluster /cluster:lab01 res "v01 IP" /priv Network="%publicNetwork%"
    9. cluster /cluster:lab01 res "v01 IP" /priv Address=192.168.10.10
    10. cluster /cluster:lab01 res "v01 IP" /priv SubnetMask=255.255.255.0
    11. cluster /cluster:lab01 res "v01 IP" /priv EnableNetBIOS=1
    12. cluster /cluster:lab01 res "v01 IP" /priv OverrideAddressMatch=0
    13. cluster /cluster:lab01 res "v01 IP" /AddDep:"v01 Disk01"
    14. cluster /cluster:lab01 res "v01 IP" /On
    15. cluster /cluster:lab01 res "v01" /create /group:"v01" /type:"Network Name"
    16. cluster /cluster:lab01 res "v01" /priv RequireKerberos=1
    17. cluster /cluster:lab01 res "v01" /AddDep:"v01 IP"
    18. cluster /cluster:lab01 res "v01" /priv Name="v01"
    19. cluster /cluster:lab01 res "v01" /On
  13. Install ABEUIamd64.msi on each node if Access Based Enumeration is required
  14. To create a test directory, share and ABE resource on the new virtual server on the cluster (v01):
    1. md \\v01\m$\Dir01
    2. cluster /cluster:lab01 res "v01 Dir01 Share" /create /group:"v01" /type:"File Share"
    3. cluster /cluster:lab01 res "v01 Dir01 Share" /priv path="M:\Dir01"
    4. cluster /cluster:lab01 res "v01 Dir01 Share" /priv Sharename=Dir01
    5. cluster /cluster:lab01 res "v01 Dir01 Share" /priv Remark="Dir01 File Share"
    6. cluster /cluster:lab01 res "v01 Dir01 Share" /prop Description="Dir01 File Share"
    7. cluster /cluster:lab01 res "v01 Dir01 Share" /priv security=Everyone,grant,F:security
    8. cluster /cluster:lab01 res "v01 Dir01 Share" /AddDep:"v01"
    9. cluster /cluster:lab01 res "v01 Dir01 Share" /AddDep:"v01 Disk01"
    10. cluster /cluster:lab01 res "v01 Dir01 Share" /On
    11. cluster /cluster:lab01 res "v01 Dir01 ABE" /create /group:"v01" /type:"Generic Application"
    12. cluster /cluster:lab01 res "v01 Dir01 ABE" /priv CommandLine="cmd.exe /k abecmd.exe /enable Dir01"
    13. cluster /cluster:lab01 res "v01 Dir01 ABE" /priv CurrentDirectory="%SystemRoot%"
    14. cluster /cluster:lab01 res "v01 Dir01 ABE" /priv InteractWithDesktop=0
    15. cluster /cluster:lab01 res "v01 Dir01 ABE" /priv UseNetworkName=0
    16. cluster /cluster:lab01 res "v01 Dir01 ABE" /prop SeparateMonitor=1
    17. cluster /cluster:lab01 res "v01 Dir01 ABE" /prop Description="Access Based Enumeration for Dir01 File Share"
    18. cluster /cluster:lab01 res "v01 Dir01 ABE" /AddDep:"v01"
    19. cluster /cluster:lab01 res "v01 Dir01 ABE" /AddDep:"v01 Disk01"
    20. cluster /cluster:lab01 res "v01 Dir01 ABE" /AddDep:"v01 Dir01 Share"
    21. cluster /cluster:lab01 res "v01 Dir01 ABE" /On
  15. Additional shared cluster disks can be created as required, eg:
    1. vmkfstools -d thick -a lsilogic -c 5G /vmfs/volumes/%datastore%/cluster01/MSCS-disk02.vmdk
    2. Add the disks to one node, (scsi 1:2 in this example). Initialise and allocate in the cluster (as in step 7 above)
  16. To create a virtual print server (assuming you’ve mounted disk02 from step 15 for use in the cluster):
    1. cluster /cluster:lab01 group "v02" /create
    2. cluster /cluster:lab01 res "v02 Disk02" /create /group:"v02" /type:"physical disk"
    3. cluster /cluster:lab01 res "v02 Disk02" /priv Drive="P:"
    4. cluster /cluster:lab01 res "v02 Disk02" /priv signature=0x%disksignature%
    5. cluster /cluster:lab01 res "v02 Disk02" /prop Description="P: print01"
    6. cluster /cluster:lab01 res "v02 Disk02" /On
    7. cluster /cluster:lab01 res "v02 IP" /create /group:"v02" /type:"IP Address"
    8. cluster /cluster:lab01 res "v01 IP" /priv Network="%publicNetwork%"
    9. cluster /cluster:lab01 res "v01 IP" /priv Address=192.168.10.11
    10. cluster /cluster:lab01 res "v01 IP" /priv SubnetMask=255.255.255.0
    11. cluster /cluster:lab01 res "v02 IP" /priv EnableNetBIOS=1
    12. cluster /cluster:lab01 res "v02 IP" /priv OverrideAddressMatch=0
    13. cluster /cluster:lab01 res "v02 IP" /AddDep:"v02 Disk02"
    14. cluster /cluster:lab01 res "v02 IP" /On
    15. cluster /cluster:lab01 res "v02" /create /group:"v02" /type:"Network Name"
    16. cluster /cluster:lab01 res "v02" /priv RequireKerberos=1
    17. cluster /cluster:lab01 res "v02" /AddDep:"v02 IP"
    18. cluster /cluster:lab01 res "v02" /priv Name="v02"
    19. cluster /cluster:lab01 res "v02" /On
  17. Create v02 print spooler:
    1. cluster /cluster:lab01 res "v02 Spooler" /create /group:"v02" /type:"print spooler"
    2. cluster /cluster:lab01 res "v02 Spooler" /priv DefaultSpoolDirectory="P:\Spool"
    3. cluster /cluster:lab01 res "v02 Spooler" /prop Description="v02 Print Spooler"
    4. cluster /cluster:lab01 res "v02 Spooler" /AddDep:"v02 Disk02"
    5. cluster /cluster:lab01 res "v02 Spooler" /AddDep:"v02"
    6. cluster /cluster:lab01 res "v02 Spooler" /On
  18. On v02, add a standard Laserjet 4000 retail driver for x64 and x86, run from a cluster node:
    1. rundll32 printui.dll,PrintUIEntry /ia /c \\v02 /m "HP LaserJet 4000 Series PCL6" /h "x64" /v "Windows XP and Windows Server 2003"
    2. rundll32 printui.dll,PrintUIEntry /ia /c \\v02 /m "HP LaserJet 4000 Series PCL6" /h "x86" /v "Windows 2000, Windows XP and Windows Server 2003"
  19. Create a test printer on v02 called printer01 using the LJ 4000 driver, with a record in DNS, published in AD, set to duplex by default, with customised permissions using the standard winprint processor:
    1. dnscmd %DNSserver% /recordadd %zone% printer01 A 192.168.10.100
    2. cscript //nologo portmgr.vbs -a -c \\v02 -p printer01 -h 192.168.10.100 -t LPR -q printer01
    3. cscript //nologo prnmgr.vbs -a -c \\v02 -b printer01 -m "HP LaserJet 4000 Series PCL6" -r printer01
    4. cscript //nologo prncfg.vbs -s -b \\v02\printer01 -h printer01 -l "%Location%" +published
    5. setprinter.exe \\v02\printer01 8 "pDevMode=dmDuplex=2,dmCollate=1,dmFields=duplex collate"
    6. subinacl /printer \\v02\printer01 /grant=%domain%\%group%=F
    7. setprinter \\v02\printer01 2 pPrintProcessor="WinPrint"

References

VMware Support method of running MSCS clusters:
http://www.vmware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_mscs.pdf

Implementing an MSCS 2003 server cluster Cluster
http://waynes-world-it.blogspot.com/2008/03/implementing-mscs-2003-server-cluster.html

subinacl 5.2.3790.1180:
http://www.microsoft.com/downloads/details.aspx?FamilyID=E8BA3E56-D8FE-4A91-93CF-ED6985E3927B

Windows Server 2003 Resource Kit Tools:
http://www.microsoft.com/downloads/details.aspx?FamilyID=9d467a69-57ff-4ae7-96ee-b18c4790cffd&DisplayLang=en


Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.

Posted by Wayne Martin at 9:41 AM ♦ DiggIt! ♦ Add to del.icio.us ♦
Labels: ,

No comments:

Post a Comment


All Posts

printQueue AD objects for 2003 ClusterVirtualCenter Physical to VirtualVirtual 2003 MSCS Cluster in ESX VI3
Finding duplicate DNS recordsCommand-line automation – Echo and macrosCommand-line automation – set
Command-line automation - errorlevels and ifCommand-line automation - find and findstrBuilding blocks of command-line automation - FOR
Useful PowerShell command-line operationsMSCS 2003 Cluster Virtual Server ComponentsServer-side process for simple file access
OpsMgr 2007 performance script - VMware datastores...Enumerating URLs in Internet ExplorerNTLM Trusts between 2003 and NT4
2003 Servers with Hibernation enabledReading Shortcuts with PowerShell and VBSModifying DLL Resources
Automatically mapping printersSimple string encryption with PowerShellUseful NTFS and security command-line operations
Useful Windows Printer command-line operationsUseful Windows MSCS Cluster command-line operation...Useful VMware ESX and VC command-line operations
Useful general command-line operationsUseful DNS, DHCP and WINS command-line operationsUseful Active Directory command-line operations
Useful command-linesCreating secedit templates with PowerShellFixing Permissions with NTFS intra-volume moves
Converting filetime with vbs and PowerShellDifference between bat and cmdReplica Domain for Authentication
Troubleshooting Windows PrintingRenaming a user account in ADOpsMgr 2007 Reports - Sorting, Filtering, Charting...
WMIC XSL CSV output formattingEnumerating File Server ResourcesWMIC Custom Alias and Format
AD site discoveryPassing Parameters between OpsMgr and SSRSAnalyzing Windows Kernel Dumps
Process list with command-line argumentsOpsMgr 2007 Customized Reporting - SQL QueriesPreventing accidental NTFS data moves
FSRM and NTFS Quotas in 2003 R2PowerShell Deleting NTFS Alternate Data StreamsNTFS links - reparse, symbolic, hard, junction
IE Warnings when files are executedPowerShell Low-level keyboard hookCross-forest authentication and GP processing
Deleting Invalid SMS 2003 Distribution PointsCross-forest authentication and site synchronizati...Determining AD attribute replication
AD Security vs Distribution GroupsTroubleshooting cross-forest trust secure channels...RIS cross-domain access
Large SMS Web Reports return Error 500Troubleshooting SMS 2003 MP and SLPRemotely determine physical memory
VMware SDK with PowershellSpinning Excel Pie ChartPoke-Info PowerShell script
Reading web content with PowerShellAutomated Cluster File Security and PurgingManaging printers at the command-line
File System Filters and minifiltersOpsMgr 2007 SSRS Reports using SQL 2005 XMLAccess Based Enumeration in 2003 and MSCS
Find VM snapshots in ESX/VCComparing MSCS/VMware/DFS File & PrintModifying Exchange mailbox permissions
Nested 'for /f' catch-allPowerShell FindFirstFileW bypassing MAX_PATHRunning PowerSell Scripts from ASP.Net
Binary <-> Hex String files with PowershellOpsMgr 2007 Current Performance InstancesImpersonating a user without passwords
Running a process in the secure winlogon desktopShadow an XP Terminal Services sessionFind where a user is logged on from
Active Directory _msdcs DNS zonesUnlocking XP/2003 without passwords2003 Cluster-enabled scheduled tasks
Purging aged files from the filesystemFinding customised ADM templates in ADDomain local security groups for cross-forest secu...
Account Management eventlog auditingVMware cluster/Virtual Center StatisticsRunning scheduled tasks as a non-administrator
Audit Windows 2003 print server usageActive Directory DiagnosticsViewing NTFS information with nfi and diskedit
Performance Tuning for 2003 File ServersChecking ESX/VC VMs for snapshotsShowing non-persistent devices in device manager
Implementing an MSCS 2003 server clusterFinding users on a subnetWMI filter for subnet filtered Group Policy
Testing DNS records for scavengingRefreshing Computer Account AD Group MembershipTesting Network Ports from Windows
Using Recovery Console with RISPAE Boot.ini Switch for DEP or 4GB+ memoryUsing 32-bit COM objects on x64 platforms
Active Directory Organizational Unit (OU) DesignTroubleshooting computer accounts in an Active Dir...260+ character MAX_PATH limitations in filenames
Create or modify a security template for NTFS perm...Find where a user is connecting from through WMISDDL syntax in secedit security templates

About Me

I’ve worked in IT for over 20 years, and I know just about enough to realise that I don’t know very much.