Labels

Tuesday, February 24, 2009

Security with 2003 R2 FSRM quotas

This post provides information on several aspects of security with Windows Server 2003 R2 FSRM quotas, which you may come across if you use the previous two posts to implement FSRM quotas on a standalone server or MSCS clustered node.

LocalService Command Notification Task

If your quota has a notification task that runs a command, you may need to set the security of the executable or areas accessed by the command. By default, FSRM runs commands as the LocalService account - an account with restricted local permissions.

In this example, the command executed is the logentry.bat batch file from the previous post, which simply writes a log entry to a file. To allow the batch file to run as the localservice account – which is a member of the 'Authenticated Users' group, I set the following permissions:

  • Add authenticated users:R to the root (not OICI - object inherit, container inherit) using SDDL with cacls
  • Add authenticated users:R to c:\scripts to execute the logentry.bat file
  • Add authenticated users:C to c:\logs to allow writing the log entry
  • Ensure that cmd.exe can be executed (default permissions should suffice)

This was done with the following commands:

  • cacls c:\ /S:"D:PAI(A;OICI;FA;;;BA)(A;;0x1200a9;;;AU)(A;OICI;FA;;;SY)"
  • cacls c:\scripts /e /g "authenticated users":R
  • cacls c:\logs /e /g "authenticated users":C

Note that while testing this, I was using localsystem (unrestricted local permissions) to verify that security was the issue, but in an MSCS cluster when failing over the virtual server from one cluster node to the other, the quota template reset itself to localservice instead of localsystem.

Before setting permissions to allow LocalService to run the script, this left the notification task not executing successfully. Depending upon what your command does you may require elevated local or remote privilege, for which you may want to use localsystem or networkservice.

Target Permissions preventing execution

When specifying a command to run as a notification task, 2003 R2 FSRM will not allow running a command which standard users have change/full control to, directly or through inheritance.

To me this seems like an unusual approach for an out of the box Microsoft product and isn't very intuitive, as in my lab inherited permissions from a parent directory allowing Users:F were causing the task to fail.


References

Implementing 2003 FSRM quotas Command-line
http://waynes-world-it.blogspot.com/2009/02/implementing-2003-fsrm-quotas.html

2003 FSRM and NTFS Quotas compared
http://waynes-world-it.blogspot.com/2009/02/2003-fsrm-and-ntfs-quotas-compared.html

LocalService Account
http://msdn.microsoft.com/en-us/library/ms684188(VS.85).aspx

Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.


Read more!

Tuesday, February 10, 2009

Implementing 2003 FSRM quotas

This post provides information on implementing Windows Server 2003 FSRM quotas using the command-line dirquota.exe utility, creating a template and then applying that template to the filesystem using an autoquota. Example control files are provided to create the template, and a simple batch file to create a daily log file with quota notifications.

The first command below creates the soft 100MB quota template with three thresholds (85/99/150), and four notifications for those thresholds, two for email and one each for and command execution and event log entry. The second command creates an autoquota using the template on the specified remote server against the specified path. Note that in this example, the commands were run on a 64-bit physical server running MSCS, and the target server was a virtual cluster server (v01).
  • dirquota template add /Remote:v01 /Template:Test_Quota /Limit:100mb /type:Soft /label:"Test Default Quota" /add-threshold:85 /add-notification:85,M,c:\admin\control\Test_Quota_Email.txt /add-threshold:99 /add-notification:99,M,c:\admin\control\Test_Quota_Email.txt /add-notification:99,C,c:\admin\control\Test_Quota_Command.txt /add-notification:99,E,c:\admin\control\Test_Quota_Event.txt /add-threshold:150 /add-notification:150,M,c:\admin\control\Test_Quota_Email.txt
  • dirquota autoquota add /Remote:v01 /path:q:\folder1 /sourcetemplate:"Test_Quota"
Notes:

  1. The use of 99% instead of 100% was intentional, as 100% is not a notification; it’s a limit, so the pre-defined variables are different. This is obvious in the subject of the email, where the ‘[Quota Threshold]’ variable isn’t resolved when it’s a 100% 'notification'.
  2. The quota information is logged to \\p01\c$\logs\QuotaUsage_YYYYMMDD.log for 99% of quota usage, as well as sending an email to DiskUsageMonitor and logging an event on the cluster, and all quota information is available through the FSRM MMC snap-in.
  3. Within a single email notification, an email can be sent to one or more administrators, and/or the person who took the limit over the threshold – the owner of the file. In the template below, emails will be sent to both users for all three thresholds, and administrators for the 150% notification. E-mail notifications will be limited to one per day for the same notification.
  4. Any command can be run; in the example here a simple batch file is run that appends a log entry to a daily log file, providing an easy method to see quota alerts for each day.
  5. The notifications aren’t triggered until first create, so if you apply quotas to existing data, the notifications won’t start appearing until new data is written.
  6. A quirk with the 64-bit OS - creating the quota template only works with the 32-bit version of the dirquota.exe utility. If the physical server (p01 in the example above) were a 64-bit server, you would have to run the 32-bit dirquota.exe utility to create the template and autoquota (2003 enterprise R2 x64 SP1).
Configuration Files

Templates, quotas and autoquotas can all be created via command-line utilities. When creating templates with notifications, the information is supplied via control files. Example control files for running a command, logging an event, and sending an email are shown below.

Note the following global properties can be specified in each notification:
  • Notification – m | e | c | r
  • m - an e-mail notification
  • e - an event log notification
  • c - a command or script execution
  • r - a report generation
  • RunLimitInterval – The number of minutes to wait between sending notifications to save repeated unnecessary notifications. A setting of 0 indicates a notification will be sent on each trigger.


Test_Quota_Command.txt

Notification=c
Command=c:\windows\system32\cmd.exe
Arguments=/c c:\admin\scripts\logentry.bat "%Date%,%Time%,[Source Io Owner],[Quota Path],[Server],[Quota Limit MB] MB,[Quota Used MB] MB,[Quota Used Percent]"
MonitorCommand=Disable
Account=LocalService
LogResult=Enable
RunLimitInterval=0



Test_Quota_Event.txt

Notification=e
RunLimitInterval=1440
EventType=Warning
Message=Excessive usage by [Source Io Owner] on [Quota Path], shared on [Server]. Limit of [Quota Limit MB] MB, [Quota Used MB] MB in use ([Quota Used Percent]% of limit).



Test_Quota_Email.txt

Notification=m
RunLimitInterval=1440
To=[Source Io Owner Email]
From=FSRM@server.domain.com
ReplyTo=FSRM-DoNotReply@server.domain.com
Cc=DiskUsageMonitor@domain.com
Subject=[Quota Threshold]% quota threshold exceeded
Message=A file written by [Source Io Owner] has exceeded the [Quota Threshold]% quota threshold for the quota on [Quota Path] on server [Server]. \
\
The quota limit is [Quota Limit MB] MB, and [Quota Used MB] MB currently is in use ([Quota Used Percent]% of limit).\
\
\
\


Logging to File
I'm not sure why a method to append to a log file wasn't included in the GUI, but this batch file appends a one-line entry to a rolling log file:



:: LogEntry.bat

:: Write a log for quota alerts

::%Date%,%Time%,[Source Io Owner],[Quota Path],[Server],[Quota Limit MB] MB,[Quota Used MB] MB,[Quota Used Percent]%

Set AdminLog=C:\Logs

for /f "tokens=1-8 delims=/:. " %%i in ('echo %date%') do Set DateFlat=%%l%%k%%j
Set LogFile=%AdminLog%\QuotaUsage_%DateFlat%.log

Echo %~1 >> %LogFile%



References

2003 FSRM and NTFS Quotas compared
http://waynes-world-it.blogspot.com/2009/02/2003-fsrm-and-ntfs-quotas-compared.html

FSRM and NTFS Quotas in 2003 R2
http://waynes-world-it.blogspot.com/2008/06/fsrm-and-ntfs-quotas-in-2003-r2.html

Configuration files for notifications in File Server Resource Manager
http://technet2.microsoft.com/windowsserver2008/en/library/a4426339-5345-44d5-81b7-a35a703daaac1033.mspx?mfr=true

How to use File Server Resource Manager (FSRM) to configure the notification feature for File Screening Management in Windows Server 2003 R2
http://support.microsoft.com/kb/926092

File Server Resource Manager Protocol Specification
http://download.microsoft.com/download/9/5/E/95EF66AF-9026-4BB0-A41D-A4F81802D92C/%5BMS-FSRM%5D.pdf

Dirquota admin options
http://technet2.microsoft.com/windowsserver2008/en/library/14c2a340-54cf-46fa-8b0d-beed6c220c671033.mspx?mfr=true

LocalService Account
http://msdn.microsoft.com/en-us/library/ms684188(VS.85).aspx

Configuration files for notifications in File Server Resource Manager
http://technet2.microsoft.com/windowsserver2008/en/library/a4426339-5345-44d5-81b7-a35a703daaac1033.mspx?mfr=true

Create an auto quota
http://technet2.microsoft.com/WindowsServer/en/library/0de5535e-ef25-4ffa-a724-155573044ddc1033.mspx

Dirquota autoquota
http://technet2.microsoft.com/WindowsServer2008/en/library/2809c575-8d93-47cb-8bfc-a427da83cc2a1033.mspx


File Services
http://technet2.microsoft.com/windowsserver2008/en/library/6e5bb377-db25-4603-b1ff-ecc4f6c29b691033.mspx?mfr=true

Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.


Read more!

Monday, February 9, 2009

2003 FSRM and NTFS Quotas compared

This post provides a quick comparison of 2003 FSRM and NTFS quotas, which I find useful when explaining how quotas in 2003 R2 work, as opposed to (or combined with) NTFS quotas. Also included is information on where the quota data is stored, and some methods to view that data.

Quota metafile information has been part of the NT filesystem since Windows NT 3.5, but has not been supported by the Operating System since the implementation of NTFS 5.0 – available in Windows 2000 and later.

When enabled, NTFS quotas track information as part of each write operation to the filesystem, providing a per-volume mapping between user SID and logical disk usage based on file ownership. While all the necessary information is stored in the NT filesystem, managing NTFS quotas is time-consuming and challenging for administrators.

Windows Server 2003 R2 File System Resource Manager (FSRM) provides a filesystem minifilter to control quotas, and a much improved interface to manage and report on quotas from a per-folder perspective.

The main differences between the two distinct quota methods available in 2003 R2 are that:

  1. FSRM provides per-folder quotas, as opposed to per-user/volume NTFS quotas. Regardless of file ownership, files in a folder will count towards the FSRM-set limits.
  2. SMB calls to return the free disk space are based on hard quotas at the root of the share or volume, not the quota applied to a folder - regardless of the share access point. NTFS hard quotas are volume-wide, and disk space is presented based on used-hard quota total, regardless of the share root or access method (remote SMB or local).
  3. FSRM quotas count only the size on disk of files, as opposed to NTFS quotas which count the logical uncompressed size. This is primarily considered for NTFS compressed files, but is presumably the same for offline files.
  4. FSRM quotas are controlled by a file system mini-filter storing quota data in \System Volume Information\SRM\quota.md and quota.xml, as opposed to NTFS quotas which are stored as part of the filesystem in \$Extend\$Quota file in $INDEX_ROOT NTFS attributes
  5. FSRM allows autoquota's, a concept of setting a quota at a top-level directory and each direct child subdirectory automatically inherits a copy of that quota. This provides an easy method of exception-based quotas. Managing NTFS quotas is GUI-based unless the WMI automation interface is used and an NTFS quota entry is automatically created for each new user SID.
  6. FSRM provides much improved reporting and alerting for quotas, whereas NTFS quotas only provide rudimentary reporting and eventlog entry alerting.
  7. FSRM has no supported automation interface to manage quotas, whereas NTFS quotas can be managed by WMI. However, the .Net assembly srmlib.dll provides an undocumented framework for managing FSRM quotas, which could be scripted through PowerShell if required.
  8. FSRM provides very strong support for command-line administration with dirquota.exe, with NTFS quotas having limited support available through fsutil
  9. In a MSCS cluster scenario, FSRM stores settings in the registry, located in HKLM\Cluster\SRM\Settings\SrmGlobalSettings\Data. NTFS quotas have all information stored on the filesystem, making both methods functional in a MSCS server cluster with shared storage.
  10. FSRM quotas provide improved notification - including in-built email, event logging, running a command or triggering a report.
  11. FSRM quotas allow for templates to be created, separating the creation of a standard set of quotas from the application of those quotas. This allows scalability and much improved process control.

How FSRM quota information is stored

FSRM quotas are stored in the "?:\System Volume Information\SRM\quota.xml" and "?:\System Volume Information\SRM\quota.sd" files, with the XML containing the configuration, and the SD file containing the actual quota information.

To see the configuration of FSRM quotas for a particular volume:

• psexec /s /i /d cmd.exe
• xcopy /h "?:\System Volume Information\SRM\quota.xml" %temp%
• attrib -r -s -h "%temp%\quota.xml"

The SD file is secured so only system can access, is marked as system/hidden and is locked by the mini-filter. One method to view the SD:

• psexec /s /i /d cmd.exe
• nfi "h:\System Volume Information\SRM\quota.md"
• diskedit Read Sectors (as returned by nfi)

How NTFS quota information is stored

NTFS stores quota information in a metafile on each volume called \$Extend\$Quota, with the information contained in the INDEX_ROOT $O and $Q NTFS attributes. Nfi.exe and diskedit.exe can be used to identify the file, and view the data contained in the logical sectors.

nfi q:

File 24
\$Extend\$Quota
$STANDARD_INFORMATION (resident)
$FILE_NAME (resident)
$INDEX_ROOT $O (resident)
$INDEX_ROOT $Q (resident)
$INDEX_ALLOCATION $Q (nonresident)
logical sectors 1036140-1036147 (0xfcf6c-0xfcf73)
$BITMAP $Q (resident)

Quota Minifilter driver

FSRM quotas use a minifilter driver to function – quota.sys – mounted by default in the I/O stack with an altitude of 125000 as part of the ‘FSFilter Physical Quota Management’ group. While this altitude can be changed by modifying a registry value, this is not recommended.

Both the R2 file screen filter (260800) and the cluster file system (200000-209999) are loaded higher in the stack then the quota minifilter.


fltmc filters & fltmc instances

Filter Name Num Instances Frame
------------------------------ ------------- -----
DfsDriver
Datascrn 0 0
Quota 1 0

Filter Volume Name Altitude Instance Name
----------------------------- -----------------------------
Quota Q: 125000 Quota


To detach the filter from a volume, the following command can be run:
• fltmc detach [volume:]

Note that doing so leaves the SRM directory in the ‘System Volume Information’ on the volume, and during testing when fltmc was used to reattach the quota filter to the volume, the previous quotas were seen as invalid and returned errors.


References

FSRM and NTFS Quotas in 2003 R2
http://waynes-world-it.blogspot.com/2008/06/fsrm-and-ntfs-quotas-in-2003-r2.html

Inside Win2K NTFS, Part 1
http://msdn.microsoft.com/en-us/library/ms995846.aspx

You cannot create quotas on File Server Resource Manager (FSRM) in Windows Server 2003 R2
http://support.microsoft.com/kb/555941

FSRM quota information does not appear in the NTFS file system Quota Entries window, and NTFS file system disk quota information does not appear in FSRM in Windows Server 2003 R2
http://support.microsoft.com/kb/915042

Limited Group Policy management for NTFS quotas.
http://technet2.microsoft.com/windowsserver/en/library/2d82decb-6726-4c5c-b872-1658b0fc3e3e1033.mspx?mfr=true

Disk Quotas Tools and Settings
http://technet2.microsoft.com/windowsserver/en/library/3b5b242b-7bb2-48e4-8e5f-224a08b36b271033.mspx

HOW TO: Configure Disk Quotas for a Shared Disk in a Cluster
http://support.microsoft.com/kb/278365

Disk Quotas Tools and Settings
http://technet2.microsoft.com/windowsserver/en/library/3b5b242b-7bb2-48e4-8e5f-224a08b36b271033.mspx

Managing Disk Quotas in Windows Server 2003 and Windows XP
http://www.microsoft.com/technet/scriptcenter/topics/win2003/quotas.mspx

Designing a Disk Quota Strategy
http://technet2.microsoft.com/windowsserver/en/library/1EE8754E-48D6-4472-9B53-29E8D1DE09F81033.mspx

Quotas in a cluster:
http://support.microsoft.com/kb/278365

How Disk Quotas Work
http://technet2.microsoft.com/windowsserver/en/library/5becbcd6-8da3-4c3b-bc0e-258acd3ec1811033.mspx?mfr=true

Disk Quotas and Free Space
http://www.microsoft.com/technet/prodtechnol/windows2000serv/reskit/core/fncd_str_ctkj.mspx?mfr=true

Quota Minifilter Driver
http://technet2.microsoft.com/windowsserver2008/en/library/7c5a0b98-d963-4a1d-a499-316322746a8e1033.mspx?mfr=true

MUP Changes in Microsoft Windows Vista
http://msdn.microsoft.com/en-us/library/aa488427.aspx

File System Minifilter Load Order Groups and Altitude Ranges
http://www.microsoft.com/whdc/driver/filterdrv/alt-range.mspx


Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.


Read more!

Friday, January 30, 2009

printQueue AD objects for 2003 Cluster

Print queue objects in AD provide a useful facility when users are trying to find printers, but with a 2003 MSCS clustered virtual print spooler, occasionally the information in AD does not reflect the current state of printers. This post describes some problems I've come across with duplicate/incorrect information and some ideas of how to automatically combat the problem.

Print Queue Objects in AD

Print queue objects in 2003 clustering are named with the virtual print server name, but they are children off a physical computer account. Which computer account the printers are children of is determined by the physical node that owned the cluster spooler resource when the printer was originally published in AD. As a virtual print server fails between nodes, the printer objects in the directory are not re-published (I assume unless the object is not found in the directory).

It's intuitive that print queue objects would be republished on failover to the node that currently owns the spooler, but that could potentially be hundreds or thousands of printer objects being created/deleted with each failover so it's practical not to. It appears the printer object is confirmed using the virtual print server name, and no change is made if the object is found - regardless of which physical node the print queue object is a child of.

In the scenario of a stand-alone printer server, when a printer is deleted, the spoolsv service also removes the directory object. In a clustered virtual print server this also occurs, however, it appears that in a 2003 cluster the object is not automatically removed from the directory if the node that owns the object when deleted is different than the publishing node.

None of this really matters if everything is working perfectly, but in a 2003 MSCS I have seen the following situations:

  1. Print queues that no longer exist still being visible through a search in AD
  2. Duplicate print queue objects, published against each physical none in the cluster that has hosted the virtual print spooler.

The first was a bigger problem, and I believe the following scenario will result in stale print queue objects persisting:

  1. You have a two node cluster, CL01 and CL02. CL01 owns a virtual print spooler and other cluster groups, under which you create all the print queues.
  2. At a later time you decide that the load could be better split, and move the virtual print spooler to CL02
  3. You then clean up your print queues from the virtual server, also expecting that they will be automatically removed from AD.

In the scenario above, the print queue objects would not be removed from AD, as the physical node that owns the spooler (CL02) does not own the original print queue objects - as they were created when CL01 owned the resources. In this state, the invalid print queue objects will not be purged. Note that this is assuming you aren't using AD printer pruning - by disabling the spooler service on your DCs or using Group Policy to control pruning.

I'm unsure of the exact scenario that caused the duplicate print queue objects, presumably there was some problem finding the existing record, so at some point it was created off the other node as well - resulting in duplicate results in a search (both of which would work, but still).

Some low maintenance ideas to correct this problem:

  1. Use AD printer pruning, which will ensure print queue objects in AD are managed. Note that this sounds like the obvious solution, but does have caveats and may not suit all environments.
  2. Periodically remove published records from all but the designated primary node, toggle the published attribute on those printers no longer having a record in AD, causing the printers to be republished against the primary node. This could easily be scripted and scheduled
  3. Modify printer creation change control processes to ensure that new printers are only created and deleted when the preferred owner is hosting the virtual print server

In an ideal world, three above followed by one make the most sense, but if you needed option two you could do something like this:

  1. dsrm CN=%virtual_server%-%QueueName%,CN=%physical_server%,DC=domainRoot
  2. cscript prncfg.vbs -s -b \\%virtual_server%\%QueueName% -published
  3. cscript prncfg.vbs -s -b \\%virtual_server%\%QueueName% +published
  4. dsquery * -limit 0 -filter "(&(objectClass=printQueue)(objectCategory=printQueue))" -attr cn printerName distinguishedname find /i "%QueueName%"

This removes the AD object against the 'incorrect' node, toggles the published flag (using prncfg from the Resource Kit Tools - see 'Network Printing Tools and Settings' reference below), and then queries AD to verify the printQueue object has been created.

Printer Pruning in AD

Pruning of printer objects in Active Directory is controlled either by the server that deletes the printer from its local spooler, or Domain Controllers through periodic printer pruning. Printer pruning is a domain/site-wide activity which processes all printQueue objects.

In a clustered solution, I believe when a Domain Controller looks up the printqueue objects, it will connect to the virtual print spooler node to verify the printers still exist. So regardless of which physical is publishing the printer, as long as the printer is contactable through the virtual server it shouldn’t be pruned.

As long as the spooler service is enabled on at least one Domain Controller, it will prune printers (at the default of 3x8 hour checks). There are risks of doing this, primarily that if the print server is down for longer than 24 hours (or if the DC can’t contact the server), all printers will be pruned from the directory. This logs an Event 50 for each pruned printer in the system event log of the DC that pruned the object - at least it’s easy to trace.

Printer Commands

Query and compare the printers published from each server to determine duplicates:

  • dsquery * "CN=%physical_server%,DC=domainRoot" -limit 0 -filter "(&(objectClass=printQueue)(objectCategory=printQueue))" -attr cn printerName driverName printCollate printColor printLanguage printSpooling driverVersion printStaplingSupported printMemory printRate printRateUnit printMediaReady printDuplexSupported > CL1.txt
  • dsquery * "CN=%physical_server%,DC=domainRoot" -limit 0 -filter "(&(objectClass=printQueue)(objectCategory=printQueue))" -attr cn printerName driverName printCollate printColor printLanguage printSpooling driverVersion printStaplingSupported printMemory printRate printRateUnit printMediaReady printDuplexSupported > CL2.txt
  • for /f "skip=1" %i in (CL1.txt) do @find /i "%i" CL2.txt

The following two commands help identify mismatches in printers published in AD versus those shared through the virtual print server.

Count the number of printers published in AD:

  • find /i /c "%virtual_server%" CL?.txt

The number of printers shared against a node:

  • rmtshare \\%physical_server% find /i "\\%virtual_Server%" /c

Query printers published against a physical server:

  • dsquery * "CN=%physical_server%,DC=domainRoot" -limit 0 -filter "(&(objectClass=printQueue)(objectCategory=printQueue))" -attr cn printerName driverName printCollate printColor printLanguage printSpooling driverVersion printStaplingSupported printMemory printRate printRateUnit printMediaReady printDuplexSupported

References:

Network Printing Tools and Settings
http://technet.microsoft.com/en-us/library/cc778201.aspx

Printer Pruner May Prune All the Print Queue Objects on Its Site
http://support.microsoft.com/kb/246906

Printer Pruner May Not Remove Printer Queue Objects from Active Directory
http://support.microsoft.com/kb/246174/

A server does not prune printers on a Microsoft Windows Server 2003-based server cluster
http://support.microsoft.com/kb/908128

Useful Windows Printer command-line operations:
http://waynes-world-it.blogspot.com/2008/09/useful-windows-printer-command-line.html

Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.


Read more!

Monday, January 19, 2009

VirtualCenter Physical to Virtual

This post describes a process I executed to replace a physical VMware VirtualCenter box with a virtual equivalent running in its own cluster. There were no VM outages of machines running on the cluster – each ESX kept merrily running the virtuals until everything was sorted out with the VC layer.

Note that this was done in a lab instance of ESX 3.5 and VirtualCenter 2.5, had this been production I probably would have taken a little more care.

I don’t think there is any compelling reason why you wouldn’t run a virtual VC box, it would be hypocritical of VMware to suggest virtualizing your application servers, except VirtualCenter which should be physical. Having said that, this process could also be used in reverse, taking a VM VirtualCenter instance physical should the need arise. This would also be useful for a VirtualCenter disaster recovery scenario.

The configuration before completing these steps

  1. Physical VirtualCenter 2.5 running on Windows Server 2003, called vc01, static IP
  2. Virtual Windows Server 2003 computer running in the cluster, called vc02, dynamic IP \
  3. SQL 2005 database for VC, running on a separate SQL server.
  4. ESX 3.5 VC 2.5, single network, iSCSI shared storage

Prerequisites

  1. The IP address of the VirtualCenter server
  2. The ESX host of the VM becoming the new VC server
  3. The path to your VMware FlexLM license file (assuming you’re using a license server)
  4. The logon details for the SQL connection between vpxd and the database

Steps Taken

On the physical VirtualCenter box that is going to be decommissioned:

  1. Take note of the physical ESX host running the VM becoming the new VC server
  2. Stop the vpxd service and change the startup to manual (sc stop %service%, sc config %service% start= demand)
  3. Stop the vmountVpx, vmware-ufad-vci, vmware-converter and webAccess services change the startup to manual
  4. Stop the flexlm instance and set the startup to manual
  5. Take a backup of your .lic license file
  6. Change the IP address to dynamic (assuming it is static)
  7. Power off the physical machine
  8. Delete the computer account for vc01 from the domain
  9. On the SQL server hosting the VirtualCenter database, backup the VC database and log file with the following command executed through management studio:

    BACKUP DATABASE [VMVC]
    TO DISK = N'c:\temp\VC_PreMoveToVM.bak'
    WITH
    DESCRIPTION = N'VirtualCenter Pre-move to VM backup'
    , INIT
    , NAME =
    N'VC Pre move to VM'
    GO

    RESTORE VERIFYONLY
    FROM DISK =
    N'c:\temp\VC_PreMoveToVM.bak '
    WITH FILE = 1
    GO
  10. In the service console of one of the ESX hosts, backup the current certificates (just in case):
    1. mkdir /tmp/cert_backup
    2. cp /etc/vmware/ssl/* /tmp/cert_backup
  11. On the VM becoming the new VirtualCenter server running on a host in the cluster: Rename the server from vc02 to vc01, with the same static IP as the previous vc01
  12. Restart the virtual machine
  13. Install the SQL Native Client - required for VC 2.5 SQL connectivity on a non-SQL server
  14. Install VirtualCenter including FlexLM, connecting to the existing database and using the license file copied off the physical server
  15. In VirtualCenter, disconnect the first ESX host in the cluster - maintenance mode is not possible at this stage – login errors occur because of the incorrect certificate
  16. Copy the new VC certificates - I use pscp here, but whatever you normally use to copy files to ESX would be fine:
    1. cd "C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\ssl"
    2. pscp rui.crt vcadmin@esx01:/etc/vmware/ssl/
    3. pscp rui.key vcadmin@esx01:/etc/vmware/ssl/"
  17. In the service console of the ESX node you’ve just updated the certificates on, restart the management interface, the ESX host did not seem to pick up the new certificate dynamically (maybe it would on a schedule without a restart):
    1. service mgmt-vmware restart
  18. Connect the ESX host to VC through the VI Client interface
  19. Repeat steps 15-18 for each other ESX host in the cluster
  20. In VirtualCenter, run 'Reconfigure for HA' on each ESX node

    Testing

  21. Ensure the vpxd.log file reports no problems with host connectivity or certificates (also check the VC/ESX logs)
  22. Ensure each ESX host is receiving licenses from the 'new' license server, either through the VI client or the FlexLM admin tool.
  23. Ensure you can perform simple tasks such as powering on a virtual machine
  24. Ensure VMotion/HA/DRS is working

The (untested) rollback plan if something goes wrong:

  1. Shutdown the new VirtualCenter VM
  2. Restore the database from the backup taken
  3. Power on the physical VC box, change the IP to the static IP
  4. Restart the vpxd and VMware License Server services
  5. For each ESX host in the cluster, disconnect the host, restore the old certificates, restart the management service and connect the ESX host to the old VirtualCenter instance

Additional notes:

  • Even though user login errors were returned when the vpxd service tried to form the cluster - which points to the vpxuser account used by VC to manage ESX hosts - this was misleading as this username and password is stored in the VC database – which had not been modified (in the vpx_host table). The next logical step was certificates, which lead to certificate update process used above.
  • Manually copying the certificates may not be strictly required, as when I went part-way to reconnecting a host without updating the certificates, I was prompted that another VC instance was managing these servers, would I like to continue. Presumably it would have automatically updated the certificates as required.


Read more!

Saturday, January 17, 2009

Virtual 2003 MSCS Cluster in ESX VI3

This post shares a method I've used to create test-lab instances of standard 2003 file and print Microsoft Cluster Services (MSCS) clusters in a VMware ESX VI3 virtual environment. The resultant solution is not supported and definitely not production-ready, but if you want a real multi-node MSCS cluster in an ESX lab environment, this process might be helpful with a minimum set of requirements.

With my usual theme of repeatable command-line execution, most of these operations can be completed via the command-line, either in the ESX service console or a command-prompt from the virtual MSCS nodes.

I followed bits and pieces of the VMware supported method - which is very specific and quite restrictive. Note that I’m a little dubious that this cluster would be particularly stable – the SCSI reservations MSCS uses to lock disks are in no way supported when using a shared VMDK through a shared SCSI adapter (I think RDM is the only supported method), but it does work and at least provided me with a test environment.

The shared nothing model of 2003 MSCS clustering dictates that only one node accesses the partition at any one time, but the disk still needs to be visible to both nodes. A limitation of this solution is that both MSCS nodes need to be hosted on one ESX server – a requirement you could satisfy with a DRS rule to keep the two nodes together. However, if DRS decided to migrate both VMs, the cluster would almost certainly break during the failover (and possibly after).

If you follow the steps below, you should end up with two virtual x64 2003 enterprise servers, both members of a single MSCS cluster. In the cluster there will three shared disks (VMDKs), one for the quorum and one each for file and print – with a virtual server and relevant cluster resources. A test file share is created, along with drivers and a test printer. You'll need to modify the commands that reference the public adapter and IP addresses

Steps involved:

  1. Create an area for storage of the shared disk on your datastore:
    1. mkdir /vmfs/volumes/%datastore%/cluster01
  2. Create a 5GB quorum disk:
    1. vmkfstools -d thick -a lsilogic -c 5G /vmfs/volumes/%datastore%/cluster01/MSCS-Quorum.vmdk
  3. Create a 5GB disk for shared data:
    1. vmkfstools -d thick -a lsilogic -c 5G /vmfs/volumes/%datastore%/cluster01/MSCS-disk01.vmdk
  4. Create two 2003 x64 enterprise virtual machines, either through cloning, deployment with templates or whatever your standard build process may be
  5. If cloning was used, run sysprep on both nodes to give a unique SID and join your lab domain
  6. Shutdown the first node and add the shared disk
    1. Add the quorum disk, mounted under scsi 1:0 (which adds a new SCSI adapter)
    2. Set the newly created SCSI Adapter to SCSI bus sharing virtual
    3. Add disk01, attached as scsi 1:1
  7. In the first VM, use disk administrator (or diskpart) to initialise the quorum and disk01 disks, partitioned with basic. Record the signature of the disk and the drive letter used (although this is the disk volume when the disk is owned by the OS, not the cluster).
  8. Add a service account for the cluster service:
    1. dsadd user "CN=clustersvc,CN=Users,DC=test,DC=local" -pwdneverexpires yes -pwd password -disabled no -desc "MSCS VM cluster service account"
    2. Ensure the service account is an administrator of each virtual 2003 node
  9. Use Cluster Administrator to install the cluster on the first node, with your chosen cluster name, using the created quorum disk and service account
  10. Verify correct operation of the single-node cluster, and then add the second VM node to the cluster.
  11. Create a new port group to allow a second private adapter on each ESX server:
    1. esxcfg-vswitch -A MSCS-Private Private
    2. Add a second interface to each VM cluster node, allocated separate address space
    3. Verify connectivity (ping) and configuration following cluster best practices (no gateway, no DNS etc)
    4. Mark as a private heartbeat connection for the cluster, prioritised above the LAN connection.
  12. Create a virtual resource group, creating IP, network name and disk resources in the group, the following commands will create a group called v01, in the lab01 cluster. For these steps, you’ll need the drive letter to use (M: below), the disk signature, the public network name, IP Address and subnet mask of the virtual server being created:
    1. cluster /cluster:lab01 group "v01" /create
    2. cluster /cluster:lab01 res "v01 Disk01" /create /group:"v01" /type:"physical disk"
    3. cluster /cluster:lab01 res "v01 Disk01" /priv Drive="M:"
    4. cluster /cluster:lab01 res "v01 Disk01" /priv signature=0x%disksignature%
    5. cluster /cluster:lab01 res "v01 Disk01" /prop Description="M: disk01"
    6. cluster /cluster:lab01 res "v01 Disk01" /On
    7. cluster /cluster:lab01 res "v01 IP" /create /group:"v01" /type:"IP Address"
    8. cluster /cluster:lab01 res "v01 IP" /priv Network="%publicNetwork%"
    9. cluster /cluster:lab01 res "v01 IP" /priv Address=192.168.10.10
    10. cluster /cluster:lab01 res "v01 IP" /priv SubnetMask=255.255.255.0
    11. cluster /cluster:lab01 res "v01 IP" /priv EnableNetBIOS=1
    12. cluster /cluster:lab01 res "v01 IP" /priv OverrideAddressMatch=0
    13. cluster /cluster:lab01 res "v01 IP" /AddDep:"v01 Disk01"
    14. cluster /cluster:lab01 res "v01 IP" /On
    15. cluster /cluster:lab01 res "v01" /create /group:"v01" /type:"Network Name"
    16. cluster /cluster:lab01 res "v01" /priv RequireKerberos=1
    17. cluster /cluster:lab01 res "v01" /AddDep:"v01 IP"
    18. cluster /cluster:lab01 res "v01" /priv Name="v01"
    19. cluster /cluster:lab01 res "v01" /On
  13. Install ABEUIamd64.msi on each node if Access Based Enumeration is required
  14. To create a test directory, share and ABE resource on the new virtual server on the cluster (v01):
    1. md \\v01\m$\Dir01
    2. cluster /cluster:lab01 res "v01 Dir01 Share" /create /group:"v01" /type:"File Share"
    3. cluster /cluster:lab01 res "v01 Dir01 Share" /priv path="M:\Dir01"
    4. cluster /cluster:lab01 res "v01 Dir01 Share" /priv Sharename=Dir01
    5. cluster /cluster:lab01 res "v01 Dir01 Share" /priv Remark="Dir01 File Share"
    6. cluster /cluster:lab01 res "v01 Dir01 Share" /prop Description="Dir01 File Share"
    7. cluster /cluster:lab01 res "v01 Dir01 Share" /priv security=Everyone,grant,F:security
    8. cluster /cluster:lab01 res "v01 Dir01 Share" /AddDep:"v01"
    9. cluster /cluster:lab01 res "v01 Dir01 Share" /AddDep:"v01 Disk01"
    10. cluster /cluster:lab01 res "v01 Dir01 Share" /On
    11. cluster /cluster:lab01 res "v01 Dir01 ABE" /create /group:"v01" /type:"Generic Application"
    12. cluster /cluster:lab01 res "v01 Dir01 ABE" /priv CommandLine="cmd.exe /k abecmd.exe /enable Dir01"
    13. cluster /cluster:lab01 res "v01 Dir01 ABE" /priv CurrentDirectory="%SystemRoot%"
    14. cluster /cluster:lab01 res "v01 Dir01 ABE" /priv InteractWithDesktop=0
    15. cluster /cluster:lab01 res "v01 Dir01 ABE" /priv UseNetworkName=0
    16. cluster /cluster:lab01 res "v01 Dir01 ABE" /prop SeparateMonitor=1
    17. cluster /cluster:lab01 res "v01 Dir01 ABE" /prop Description="Access Based Enumeration for Dir01 File Share"
    18. cluster /cluster:lab01 res "v01 Dir01 ABE" /AddDep:"v01"
    19. cluster /cluster:lab01 res "v01 Dir01 ABE" /AddDep:"v01 Disk01"
    20. cluster /cluster:lab01 res "v01 Dir01 ABE" /AddDep:"v01 Dir01 Share"
    21. cluster /cluster:lab01 res "v01 Dir01 ABE" /On
  15. Additional shared cluster disks can be created as required, eg:
    1. vmkfstools -d thick -a lsilogic -c 5G /vmfs/volumes/%datastore%/cluster01/MSCS-disk02.vmdk
    2. Add the disks to one node, (scsi 1:2 in this example). Initialise and allocate in the cluster (as in step 7 above)
  16. To create a virtual print server (assuming you’ve mounted disk02 from step 15 for use in the cluster):
    1. cluster /cluster:lab01 group "v02" /create
    2. cluster /cluster:lab01 res "v02 Disk02" /create /group:"v02" /type:"physical disk"
    3. cluster /cluster:lab01 res "v02 Disk02" /priv Drive="P:"
    4. cluster /cluster:lab01 res "v02 Disk02" /priv signature=0x%disksignature%
    5. cluster /cluster:lab01 res "v02 Disk02" /prop Description="P: print01"
    6. cluster /cluster:lab01 res "v02 Disk02" /On
    7. cluster /cluster:lab01 res "v02 IP" /create /group:"v02" /type:"IP Address"
    8. cluster /cluster:lab01 res "v01 IP" /priv Network="%publicNetwork%"
    9. cluster /cluster:lab01 res "v01 IP" /priv Address=192.168.10.11
    10. cluster /cluster:lab01 res "v01 IP" /priv SubnetMask=255.255.255.0
    11. cluster /cluster:lab01 res "v02 IP" /priv EnableNetBIOS=1
    12. cluster /cluster:lab01 res "v02 IP" /priv OverrideAddressMatch=0
    13. cluster /cluster:lab01 res "v02 IP" /AddDep:"v02 Disk02"
    14. cluster /cluster:lab01 res "v02 IP" /On
    15. cluster /cluster:lab01 res "v02" /create /group:"v02" /type:"Network Name"
    16. cluster /cluster:lab01 res "v02" /priv RequireKerberos=1
    17. cluster /cluster:lab01 res "v02" /AddDep:"v02 IP"
    18. cluster /cluster:lab01 res "v02" /priv Name="v02"
    19. cluster /cluster:lab01 res "v02" /On
  17. Create v02 print spooler:
    1. cluster /cluster:lab01 res "v02 Spooler" /create /group:"v02" /type:"print spooler"
    2. cluster /cluster:lab01 res "v02 Spooler" /priv DefaultSpoolDirectory="P:\Spool"
    3. cluster /cluster:lab01 res "v02 Spooler" /prop Description="v02 Print Spooler"
    4. cluster /cluster:lab01 res "v02 Spooler" /AddDep:"v02 Disk02"
    5. cluster /cluster:lab01 res "v02 Spooler" /AddDep:"v02"
    6. cluster /cluster:lab01 res "v02 Spooler" /On
  18. On v02, add a standard Laserjet 4000 retail driver for x64 and x86, run from a cluster node:
    1. rundll32 printui.dll,PrintUIEntry /ia /c \\v02 /m "HP LaserJet 4000 Series PCL6" /h "x64" /v "Windows XP and Windows Server 2003"
    2. rundll32 printui.dll,PrintUIEntry /ia /c \\v02 /m "HP LaserJet 4000 Series PCL6" /h "x86" /v "Windows 2000, Windows XP and Windows Server 2003"
  19. Create a test printer on v02 called printer01 using the LJ 4000 driver, with a record in DNS, published in AD, set to duplex by default, with customised permissions using the standard winprint processor:
    1. dnscmd %DNSserver% /recordadd %zone% printer01 A 192.168.10.100
    2. cscript //nologo portmgr.vbs -a -c \\v02 -p printer01 -h 192.168.10.100 -t LPR -q printer01
    3. cscript //nologo prnmgr.vbs -a -c \\v02 -b printer01 -m "HP LaserJet 4000 Series PCL6" -r printer01
    4. cscript //nologo prncfg.vbs -s -b \\v02\printer01 -h printer01 -l "%Location%" +published
    5. setprinter.exe \\v02\printer01 8 "pDevMode=dmDuplex=2,dmCollate=1,dmFields=duplex collate"
    6. subinacl /printer \\v02\printer01 /grant=%domain%\%group%=F
    7. setprinter \\v02\printer01 2 pPrintProcessor="WinPrint"

References

VMware Support method of running MSCS clusters:
http://www.vmware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_mscs.pdf

Implementing an MSCS 2003 server cluster Cluster
http://waynes-world-it.blogspot.com/2008/03/implementing-mscs-2003-server-cluster.html

subinacl 5.2.3790.1180:
http://www.microsoft.com/downloads/details.aspx?FamilyID=E8BA3E56-D8FE-4A91-93CF-ED6985E3927B

Windows Server 2003 Resource Kit Tools:
http://www.microsoft.com/downloads/details.aspx?FamilyID=9d467a69-57ff-4ae7-96ee-b18c4790cffd&DisplayLang=en


Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.


Read more!

Friday, January 9, 2009

Finding duplicate DNS records

Internal DNS in many organisations is usually a little messy and if tasked with cleaning it, my first step would be to identify duplicate records in DNS - either an IP address that is recorded against multiple host names or a host name with multiple host (A) records. Assuming you have a file containing one record per line, this is fairly simple to report on.

The following examples use dnscmd.exe, which assumes Windows 2000/2003 DNS, but you could use any input, as long as it contains a name and an IP.

The two commands:

  1. dnscmd %dnsServer% /enumrecords test.local @ /additional /continue | find /i /v "Aging:" | find /i "192.168" > DNSRecords.txt
  2. echo. > DuplicateIPs.txt & (for /f "tokens=1,4" %i in (DNSRecords.txt) do @if "%j" NEQ "" @find /i "%j" DuplicateIPs.txt >nul & if errorlevel 1 for /f "tokens=1" %m in ('"findstr /i "%j$" DNSRecords.txt | find /i /c "%j""') do @if %m GTR 1 (@echo %j,%m: & findstr /i "%j$" DNSRecords.txt & echo.) >> DuplicateIPs.txt) & type DuplicateIPs.txt


The first command uses dnscmd to enumerate the records from the root of the test.local zone and exclude aging (dynamic DNS) records. The last find command further filters the output by IP, which can be useful when targeting specific subnets/sites. You might also want to check aging dynamic records instead of static to see how well scavenging is working – if you do, just change the tokens to 1,5 instead of 1,4 (as the aging data is another token separating the name and IP).

The second command:
  1. Creates a new file called DuplicateIPs.txt in the current working directory
  2. Iterates through each line in the DNS record dump, extracting the first and fourth token (Name and IP)
  3. The commands in the first FOR loop check there is a value, then check the value hasn't already been processed in the duplicate list (otherwise you'd have duplicates of each duplicate) and then counts the duplicates and appends them to the file.
  4. The final command types the duplicate IP file created by the for loop iteration.

For example, supposing your DNS export contained the following records:

printer1 3600 A 192.168.10.100
printer2 3600 A 192.168.10.101
printer3 3600 A 192.168.10.100
printer4 3600 A 192.168.10.102
printer5 3600 A 192.168.10.103
printer6 3600 A 192.168.10.100
printer7 3600 A 192.168.10.102


After running the second command above, a file called DuplicateIPs.txt would be created and then typed to the prompt:

192.168.10.100,3:
printer1 3600 A 192.168.10.100
printer3 3600 A 192.168.10.100
printer6 3600 A 192.168.10.100

192.168.10.102,2:
printer4 3600 A 192.168.10.102
printer7 3600 A 192.168.10.102


The command below is modified to report duplicate names instead of IP addresses. This was done by using the first token (%i) instead of the second (%j), and modifying the findstr command to use a literal string search ending with a space rather than the regular expression EOL:

echo. > DuplicateIPs.txt & (for /f "tokens=1,4" %i in (DNSRecords.txt) do @if "%i" NEQ "" @find /i "%i" DuplicateIPs.txt >nul & if errorlevel 1 for /f "tokens=1" %m in ('"findstr /i /c:"%i " DNSRecords.txt | find /i /c "%i""') do @if %m GTR 1 (@echo %i,%m: & findstr /i /c:"%i " DNSRecords.txt & echo.) >> DuplicateIPs.txt) & type DuplicateIPs.txt


If you wanted a summary rather than the detail of each duplicate, you could also run the following command:

echo. > DuplicateIPSummary.txt & (for /f "tokens=1,4" %i in (DNSRecords.txt) do @if "%j" NEQ "" @find /i "%j" DuplicateIPSummary.txt >nul & if errorlevel 1 for /f "tokens=1" %m in ('"findstr /i "%j$" DNSRecords.txt | find /i /c "%j""') do @if %m GTR 1 (@echo %j,%m) >> DuplicateIPSummary.txt) & type DuplicateIPSummary.txt


In the example above, this would produce the following report:

192.168.10.100,3
192.168.10.102,2


I use this sort of command to generate reports on duplicates, in this case from DNS, but it could also be useful in DHCP, WINS, or any number of Active Directory objects/attributes. People (myself included) are often wary of automated processes that make changes, but this is an excellent example of how powerful read-only automated commands can be – you can take thousands of objects and produce a report in seconds to quickly identify inconsistencies in an environment.

Dnscmd Overview
http://technet.microsoft.com/en-us/library/cc778513.aspx


Read more!

Monday, December 29, 2008

Command-line automation – Echo and macros

This is the fifth and last in a series of posts containing information on what I consider the building blocks to automate repetitive tasks at the Windows command-line. These components are the for, find, findstr, set, if and echo commands, control files used to control data input, combined with errorlevels, command concatenation, nested loops and if/then/else constructs.

Described in this post is the echo command and how to execute your commands using macros with doskey.

Echo

Echo is one of the most important commands to use when constructing repeatable commands – you should never run a command against multiple objects without first using echo. Some commands can be disastrous without appropriate input checking.

For example, using a wildcard incorrectly in a file spec, instead of putting *.txt, below I’ve put ‘* .txt’ – the wildcard selecting everything in the current directory (with the unfortunate del command for each of those files).
for %i in (* .txt) do echo del %i

Or more commonly – an incorrectly formed control file. The following command is one I’ve used to fix invalidly set UPNs on user accounts in the local directory. This command is quite powerful, and can be used to fix dozens or hundreds of accounts in seconds – but if the input file is not well formed, you could end up breaking each account:
for /f "skip=1 tokens=1,2-3" %i in (NoUPN.txt) do echo dsmod user "%j %k" -upn %i@%upnsuffix%

Incidentally, the command used to generate the ‘NoUPN.txt’ file was:
dsquery * %OU% -filter "&(objectclass=user)(objectcategory=person)(!(userprincipalname=*))" -s %server% -scope onelevel -attr name distinguishedname > NoUPN.txt

If you are untrusting of using the ‘for /f’ to execute operations against multiple objects, you can also use echo to construct a batch file that you can run at a later time – good practice rather than writing and executing the command in a single attempt. This may also useful when you’re modifying hundreds or thousands of objects and you don’t want to execute the commands immediately – for example if the modifications were in Active Directory and you wanted to prevent excessive replication during the day.

Running the command above with an appended redirect (>>) to a file, you could then pass this file to someone else for sanity checking before actual execution of the batch job:
for /f "skip=1 tokens=1,2-3" %i in (NoUPN.txt) do echo dsmod user "%j %k" -upn %i@%upnsuffix% >> %temp%\SetUPN.bat

Doskey macros

In the previous post there was a command to calculate a subnet at the command-line. If you had to type it every time - or even copy and paste - it’s not something you would use (okay, you’d probably never use it anyway), but doskey macros provide a simple method of allowing command re-use.

Creating a macro file

A text file can be created containing your macros, choose a shorthand name for the command, followed by the command you wish to execute. One slightly odd aspect with a macro file is that variables passed at the command-line for the macro use the $ prefix instead of %.

To make macros more useful, I often add two options – one without any parameters with a defined default, and another accepting a parameter to customise the command. For example, the following macro - called ‘AD2003’ - would by default query the local directory for all 2003 computer accounts. If a parameter is passed ($1), the dsquery command passes the parameter as the start node for the search (which could be a DN to limited the containers queried, or a different domain than domainroot).

AD2003=if _$1==_ (dsquery * domainroot -filter "(&(objectCategory=Computer)(objectClass=Computer)(operatingSystem=Windows Server 2003))" -limit 0) else (dsquery * $1 -filter "(&(objectCategory=Computer)(objectClass=Computer)(operatingSystem=Windows Server 2003))" -limit 0)

The find subnet command from the previous post could be shortened to execute the command with something as simple as ‘fs 192.168.11.40 255.255.254.0’, by adding the following to a macro file (and executing on a command processor with delayed expansion enabled):

FS=for /f "tokens=1-8 delims=.- " %i in ('echo $1 $2') do @set /a Octet1="%i & %m" >nul & @set /a Octet2="%j & %n" >nul & @set /a Octet3="%k & %o" >nul & @set /a Octet4="%l & %p" >nul & Echo !Octet1!.!Octet2!.!Octet3!.!Octet4!

Automatically enabling the macro file

It would also be unwieldy if you had to enable the doskey macros each time you ran a command shell. However, you can easily automate this with the autorun key of the shell, stored in either

HKCU or HKLM:
Key - HKEY_LOCAL_MACHINE\software\microsoft\Command Processor
Value - AutoRun
Value Type - REG_SZ
Value Data - doskey /macrofile=c:\util\macros.txt (or wherever)


This concludes what hopefully was a useful description of how I use built-in shell commands to automate repetitive tasks and make my work life easier and more efficient. Some of the resultant commands are not particularly easy to follow, but each can be broken down into the component parts and digested separately.

For more real-world examples of how I use the information in the last five posts, see my command-line operations:
http://waynes-world-it.blogspot.com/2008/09/useful-ntfs-and-security-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-printer-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-mscs-cluster-command.html
http://waynes-world-it.blogspot.com/2008/09/useful-vmware-esx-and-vc-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-general-command-line-operations.html
http://waynes-world-it.blogspot.com/2008/09/useful-dns-dhcp-and-wins-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-active-directory-command-line.html


Wayne's World of IT (WWoIT), Copyright 2008 Wayne Martin.


Read more!

Saturday, December 27, 2008

Command-line automation – set

This is the fourth in a series of posts containing information on what I consider the building blocks to automate repetitive tasks at the Windows command-line. These components are the for, find, findstr, set, if and echo commands, control files used to control data input, combined with errorlevels, command concatenation, nested loops and if/then/else constructs.

Described in this post is the ‘set’ command, which I find very useful when automating repetitive tasks with ‘for’. I use set in four ways – string manipulation, simple arithmetic operations with ‘set /a’, gathering input with ‘set /p’ or concatenating output in a single for loop with delayed variable expansion.

String Manipulation

My most frequent use of the set command is simple string manipulation at the command prompt. I use this to replace one character with another or extract a substring.

Replace

Replacing one string with another is a core function of string manipulation. Even though this is included under help for ‘set’, this functionality works for other commands, including the echo command.

For example, say you have the fully qualified user DNS name – my.domain.com – and this conveniently is also the distinguished name of your AD domain, the following command replaces periods with ‘,DC=’ and prefixes an additional string to provide the distinguished name:
echo DC=%userdnsdomain:.=,DC=%

This would turn ‘my.domain.com’ into:
DC=my,DC=domain,DC=com

Wildcards can also be used, for example, to remove the first element, you could replace up to the first dot with nothing:
echo %userdnsdomain:*.=%

You could also remove spaces and print the output, for example, if the ‘test’ variable contained ‘A Sentence With Spaces’, running:
echo %test: =%

Would result in:
ASentenceWithSpaces

Substrings

You can extract from existing variables (or inline if using delayed expansion), providing Left/Right/Mid functionality. This can be useful in many ways, but for most of the input I deal with, splitting with ‘for’ and delimiters (such as comma or space) makes for simpler processing with dynamic input. For examples sake, if you had a defined length of padded text, and you wanted the 10 characters starting at the 20th character (0-based), you could run:
echo %input:~19,10%

You could also extract the last 10 characters:
echo %input:~-10%

Or all but the last 10 characters:
echo %input:~0,-10%

Arithmetic operations

‘set /a’ supports a relatively simple but still powerful set of arithmetic, bitwise and shift operators, and while I’ve never found much use for them, upon occasion they have come in handy.

For example, this command will tell you what next year will be and set the NextYear environment variable, based on the current year + 1:
for /f "tokens=3 delims=/" %i in ('echo %date%') do set /a NextYear=%i+1

Similarly, find the current month, increment by one, unless the current month is 12, then wrap around and start at 1 again:
for /f "tokens=2 delims=/" %i in ('echo %date%') do if %i==12 (Set NextMonth=1 & echo !NextMonth!) else (Set /a NextMonth=%i+1)

Note that the example above uses delayed environment variable expansion, see below for a description.

Incidentally, if you were going to use this output for something and if the month were a single digit you would lose the padding, which would then be incorrectly sorted (as compared to other padded output). The example below pads the month, then extracts the last two characters, effectively padding single-digit months (see string manipulation above):
set NextMonth=0%NextMonth%
echo %nextmonth:~-2%


An actual use I have found for the arithmetic operators is to calculate subnets from the command-line. I have made use of this when parsing netsh dhcp output (there were thousands of leases across several DHCP servers serving dozens of subnets, and I wanted to know from the leases which subnets were in use).

This can be done with a single command-line, but it’s a little complicated, the batch file below may be easier to demonstrate the ‘set /a’ functionality. Running this batch file with an IP and mask, eg ‘findsubnet.bat 192.168.11.40 255.255.254.0’ will return the subnet (192.168.10.0 in this example) using bitwise AND of the two sets of octets:



@Echo Off
Set Input=%1
Set Mask=%2
If /i "%Input%"=="" Goto End

for /f "tokens=1-8 delims=.- " %%i in ('echo %Input% %Mask%') do Call :FindOctet %%i %%j %%k %%l %%m %%n %%o %%p
Goto End

:FindOctet
::Echo %1 %2 %3 %4 %5 %6 %7 %8

set /a Octet1="%1 & %5" >nul
set /a Octet2="%2 & %6" >nul
set /a Octet3="%3 & %7" >nul
set /a Octet4="%4 & %8" >nul
Echo %1.%2.%3.%4,%Octet1%.%Octet2%.%Octet3%.%Octet4%,%5.%6.%7.%8
Set Octet1=
Set Octet2=
Set Octet3=
Set Octet4=
Goto End

:InvalidArgs
Echo Invalid arguments, please provide an IP and longhand mask, eg 192.168.11.40 255.255.254.0
Goto End

:End



Input

Set can be used to gather input, making your commands or batch files interactive. For example, the following command assigns the input to the ‘test’ environment variable, after asking the question:
set /p test=Tell me something?

I don’t often use this, as it somewhat defeats the purpose of automation – I prefer to have all the required input in a control file, which separates the intended action from the actual operator.

Set with Delayed expansion

Delayed environment variable expansion provides a method of executing commands and storing the output for use by other commands within a loop – either in different iterations or nested commands. This is required when looping through ‘for’ commands or nesting commands, otherwise state between each command is not maintained.

For example, without delayed expansion, the following would not work:
set test=test variable & echo %test%

However, using delayed environment variable expansion, this problem can be easily overcome, using the !variable! syntax, instead of the normal %variable%. Note that you may need to enable delayed environment variable expansion, which can be done using ‘cmd /v:on’ when starting the command prompt, or running ‘setlocal ENABLEDELAYEDEXPANSION’ within an existing shell.

With delayed expansion, the first command would set the variable and the second echo the newly set variable:
set test=test variable & echo !test!

Or to re-use an example above, calculate next year and then do something with that information (echo in this case):
for /f "tokens=3 delims=/" %i in ('echo %date%') do set /a NextYear=%i+1 1>nul & echo Next year is !NextYear!

Unless there is already a variable of the same name, running the commands above using the percent signed variables will result in the variable name being printed, as the environment hasn’t yet been changed by the first command.

Note that while testing, it seems the Vista command shell has been updated and as long as delayed expansion is enabled, this command would work:
set test=test variable & echo %test%

The following command is a one-line version of the batch above, using delayed variable expansion to calculate the variables as each command is executed:
for /f "tokens=1-8 delims=.- " %i in ('echo 192.168.11.40 255.255.254.0') do set /a Octet1="%i & %m" >nul & set /a Octet2="%j & %n" >nul & set /a Octet3="%k & %o" >nul & set /a Octet4="%l & %p" >nul & Echo %i.%j.%k.%l,!Octet1!.!Octet2!.!Octet3!.!Octet4!,%m.%n.%o.%p

An easier example to follow, set the environment variable called ‘number’ to a random number (using %random% - very useful), echo the newly set variable, and then use ‘set /a’ to increment the number by one:
set number=%random% & echo !number! & set /a number=!number!+1

If you are using nested ‘for’ commands, you can just reference variables from an earlier command, but in some circumstances delayed expansion is the only possibility. I have had to use this syntax when using ‘set /a’ and using the replace/substring functionality, as this requires a full variable (eg. %path%, not a temporary %i type variable.

This was a basic overview of ‘set’ and some of the ways in which I use this command when automating tasks at the command-line.

For more real-world examples of how I use ‘set’, see my command-line operations:
http://waynes-world-it.blogspot.com/2008/09/useful-ntfs-and-security-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-printer-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-mscs-cluster-command.html
http://waynes-world-it.blogspot.com/2008/09/useful-vmware-esx-and-vc-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-general-command-line-operations.html
http://waynes-world-it.blogspot.com/2008/09/useful-dns-dhcp-and-wins-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-active-directory-command-line.html


Wayne's World of IT (WWoIT), Copyright 2008 Wayne Martin.


Read more!

Thursday, December 18, 2008

Command-line automation – errorlevels and if

This is the third in a series of posts containing information on what I consider the building blocks to automate repetitive tasks at the Windows command-line. These components are the for, find, findstr, set, if and echo commands, control files used to control data input, combined with errorlevels, command concatenation, nested loops and if/then/else constructs.

Described in this post are errorlevels, and if/then/else statements, useful for branching in a single command-line to execute based on the error level set or some other condition.

Using errorlevels, concatenated commands and if/then/else statements provides enhanced functionality, and most complex operations would not be possible on a single command-line without these features.

errorlevels

In the context of command-line automation, I find error levels most useful when trying to cater for both sets of outcomes when executing a command – either what you wanted happened, or it didn’t.

In the previous examples, I’ve used a control file to ping one or more machines and then echoed the IP address of those that were contactable. This is only part of the picture – what about those machines in the list that didn’t respond to a ping? They are simply dropped from the output. This may be what you are after, but normally I would prefer to know the positive and the negative.

To work around this problem, the following command could be used:
for /f "tokens=3 delims=: " %i in ('"ping -n 1 computer | find /i "reply" & if errorlevel 1 echo 0:1:Not-Found"') do echo %i

This will ping ‘computer’, echo the IP Address if found, otherwise echo NotFound. This occurs because the find command sets the errorlevel according to whether the string was found, and then we are concatenating the command with another to check if an errorlevel occurred indicating the string was not found, and echoing an alternate output to be picked up by the body of the for loop.

A simpler way of looking at this may be without the ‘for’ loop:
ping -n 1 computer | find /i "reply" >nul & if errorlevel 1 (echo No Reply) else (echo Replied)

The command is similar to the example above, it will ping ‘computer’, and if the word reply was found in the output, it will print ‘replied’, otherwise it will print ‘no reply’.

Command concatenation and if/then/else

Commands can be concatenated – when running natively or inside a ‘for’ loop – providing a simple method to group commands. For example, looping through a control file, you might want to nslookup and then ping each device:
for /f %i in (c:\temp\control.txt) do nslookup %i & ping %i

This is a simple chain of commands, execute one and then the other. However, provided each command returns a relevant errorlevel, the if/then/else syntax can also be used to execute the second only if the first succeeded.

Unfortunately nslookup does not provide an errorlevel, but the find command does:
ping -n 1 computer | find /i "reply" >nul & if errorlevel 1 (echo No Reply) else (echo Replied)
ping -n 1 computer | find /i "reply" >nul && echo Replied || echo No Reply

The above commands return the same result – they each print a statement depending on whether ‘reply’ was found in the ping output, the second using the shorthand ‘if’ (&&) and ‘else’ (||) syntax.

‘If’ can be used to check the errorlevel (as in the above example), check whether one string equals another, and check whether a file/directory exists.

If ErrorLevel checking

The following examples send an invalid service a stop control, and perform various checks on the errorlevel returned. Note that sc.exe on XP does not return error codes, this was Vista which does at least return 1060 when the specified service does not exist.

Check whether errorlevel (a special variable that can either be enclosed in % or not) was exactly 1060:
sc stop invalidservice >nul & if errorlevel==1060 echo Service not found

Check whether errorlevel was exactly 1060, if so echo ‘service not found’, otherwise echo the errorlevel returned:
sc stop invalidservice >nul & if errorlevel==1060 (echo Service not found) else (echo Error code %errorlevel% returned)

Check whether the errorlevel was 1060 or greater, and not 1061 (therefore only 1060).
sc stop invalidservice >nul & if errorlevel 1060 if not errorlevel 1061 echo Service not found

Check if anything above errorsuccess (0) was returned, echoes an error occurred and then uses ‘net helpmsg’ to return the error description for the specified error code:
sc stop invalidservice >nul & if errorlevel 1 echo An error occurred: %errorlevel% & net helpmsg %errorlevel%

If string checking

If can be used to conditionally check two strings to determine the action taken. Note that if the string could be blank, you’ll need to either append/prepend another character or enclose the string in quotes.

Using the above example of pinging a computer, check the response and branch accordingly (different echoes in this case):
for /f "tokens=3 delims=: " %i in ('"ping -n 1 computer | find /i "reply" & if errorlevel 1 echo 0:1:Not-Found"') do if "%i"=="reply" (echo Ping success) else (echo Ping failed)

Note that ‘if /i’ can be used for a case insensitive comparison.

Additionally, if command extensions are enabled, and the strings in comparison are numeric, both are converted to numbers and additional operators are available for the comparison, for example:
If 9 equ 9 echo Equals
If 5 lss 14 echo Less than

If exist

To determine whether a file or directory exists, the ‘if exist’ syntax can be used. This can be useful when checking flag files or control files exist to avoid encountering unexpected errors.

If exist %windir% echo %windir% exists
If not exist c:\temp\control1.txt (echo Control File does not exist, terminating & break) else (echo continuing)

Some other examples of concatenating commands which help to demonstrate the concepts:

Use ‘sc.exe’ to send a service a stop control, if the service was running (exists, security allows etc) and was sent a stop, sc will return errorsuccess (0), pause for 5 seconds and then restart the service, otherwise echo that the service was not running in the first place:
sc stop "service" && (echo error %errorlevel% & sleep 5 & sc start "service") || echo Service was not running, error %errorlevel%

nslookup www.microsoft.com, find the second address (the first in nslookup is the DNS server address, and echo that address:
for /f "skip=1 tokens=2" %i in ('"nslookup www.microsoft.com 2>nul | find /i "address""') do echo %i

nslookup www.microsoft.com, skip the first DNS server address (the second line of the nslookup output – find /n), return the other address if found, otherwise return ‘Not-Found’ in the %i variable – the second token of the string:
for /f "tokens=2" %i in ('"nslookup www.microsoft.com 2>nul | find /i /n "address" | find /i /v "[2]" & if errorlevel 1 echo Address Not-Found"') do echo %i

Create a file, if the file is found type the contents, and then delete the file if the type succeeded, otherwise report that the ‘type’ command failed:
echo test > %temp%\test.txt & if exist %temp%\test.txt type %temp%\test.txt && del %temp%\test.txt || Echo could not type %temp%\test.txt

This was a basic overview of the errorlevels and if/then/else branching, future posts will build on this with other useful commands and nested ‘for’ loops.


For more real-world examples of how I use errorlevels and if/then/else, see my command-line operations:
http://waynes-world-it.blogspot.com/2008/09/useful-ntfs-and-security-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-printer-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-mscs-cluster-command.html
http://waynes-world-it.blogspot.com/2008/09/useful-vmware-esx-and-vc-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-general-command-line-operations.html
http://waynes-world-it.blogspot.com/2008/09/useful-dns-dhcp-and-wins-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-active-directory-command-line.html


Wayne's World of IT (WWoIT), Copyright 2008 Wayne Martin.


Read more!

All Posts

printQueue AD objects for 2003 ClusterVirtualCenter Physical to VirtualVirtual 2003 MSCS Cluster in ESX VI3
Finding duplicate DNS recordsCommand-line automation – Echo and macrosCommand-line automation – set
Command-line automation - errorlevels and ifCommand-line automation - find and findstrBuilding blocks of command-line automation - FOR
Useful PowerShell command-line operationsMSCS 2003 Cluster Virtual Server ComponentsServer-side process for simple file access
OpsMgr 2007 performance script - VMware datastores...Enumerating URLs in Internet ExplorerNTLM Trusts between 2003 and NT4
2003 Servers with Hibernation enabledReading Shortcuts with PowerShell and VBSModifying DLL Resources
Automatically mapping printersSimple string encryption with PowerShellUseful NTFS and security command-line operations
Useful Windows Printer command-line operationsUseful Windows MSCS Cluster command-line operation...Useful VMware ESX and VC command-line operations
Useful general command-line operationsUseful DNS, DHCP and WINS command-line operationsUseful Active Directory command-line operations
Useful command-linesCreating secedit templates with PowerShellFixing Permissions with NTFS intra-volume moves
Converting filetime with vbs and PowerShellDifference between bat and cmdReplica Domain for Authentication
Troubleshooting Windows PrintingRenaming a user account in ADOpsMgr 2007 Reports - Sorting, Filtering, Charting...
WMIC XSL CSV output formattingEnumerating File Server ResourcesWMIC Custom Alias and Format
AD site discoveryPassing Parameters between OpsMgr and SSRSAnalyzing Windows Kernel Dumps
Process list with command-line argumentsOpsMgr 2007 Customized Reporting - SQL QueriesPreventing accidental NTFS data moves
FSRM and NTFS Quotas in 2003 R2PowerShell Deleting NTFS Alternate Data StreamsNTFS links - reparse, symbolic, hard, junction
IE Warnings when files are executedPowerShell Low-level keyboard hookCross-forest authentication and GP processing
Deleting Invalid SMS 2003 Distribution PointsCross-forest authentication and site synchronizati...Determining AD attribute replication
AD Security vs Distribution GroupsTroubleshooting cross-forest trust secure channels...RIS cross-domain access
Large SMS Web Reports return Error 500Troubleshooting SMS 2003 MP and SLPRemotely determine physical memory
VMware SDK with PowershellSpinning Excel Pie ChartPoke-Info PowerShell script
Reading web content with PowerShellAutomated Cluster File Security and PurgingManaging printers at the command-line
File System Filters and minifiltersOpsMgr 2007 SSRS Reports using SQL 2005 XMLAccess Based Enumeration in 2003 and MSCS
Find VM snapshots in ESX/VCComparing MSCS/VMware/DFS File & PrintModifying Exchange mailbox permissions
Nested 'for /f' catch-allPowerShell FindFirstFileW bypassing MAX_PATHRunning PowerSell Scripts from ASP.Net
Binary <-> Hex String files with PowershellOpsMgr 2007 Current Performance InstancesImpersonating a user without passwords
Running a process in the secure winlogon desktopShadow an XP Terminal Services sessionFind where a user is logged on from
Active Directory _msdcs DNS zonesUnlocking XP/2003 without passwords2003 Cluster-enabled scheduled tasks
Purging aged files from the filesystemFinding customised ADM templates in ADDomain local security groups for cross-forest secu...
Account Management eventlog auditingVMware cluster/Virtual Center StatisticsRunning scheduled tasks as a non-administrator
Audit Windows 2003 print server usageActive Directory DiagnosticsViewing NTFS information with nfi and diskedit
Performance Tuning for 2003 File ServersChecking ESX/VC VMs for snapshotsShowing non-persistent devices in device manager
Implementing an MSCS 2003 server clusterFinding users on a subnetWMI filter for subnet filtered Group Policy
Testing DNS records for scavengingRefreshing Computer Account AD Group MembershipTesting Network Ports from Windows
Using Recovery Console with RISPAE Boot.ini Switch for DEP or 4GB+ memoryUsing 32-bit COM objects on x64 platforms
Active Directory Organizational Unit (OU) DesignTroubleshooting computer accounts in an Active Dir...260+ character MAX_PATH limitations in filenames
Create or modify a security template for NTFS perm...Find where a user is connecting from through WMISDDL syntax in secedit security templates

About Me

I’ve worked in IT for over 20 years, and I know just about enough to realise that I don’t know very much.