WWoIT - Wayne's World of IT

vSphere Storage vMotion traffic

2010-08-15T21:11:00.000+10:00

This post provides information on storage vMotion in VMware vSphere 4 with ESXi 4.0 U2 hosts. While using storage vMotion to move data between unconnected storage, I conducted some simple testing to identify the networks and ports used for vMotion.

I’ve never seen an explanation of how storage vMotion works, and while it seems kind of obvious in retrospect, I thought some people might find this interesting.

Summary

The scope of a storage vMotion operation is datacenter wide, not limited to clusters or which host can see which storage.
In the testing conducted, storage vMotion relies upon the management network to transfer the disk data between the two ESXi hosts
If this method was used to import a large number of VMs from foreign storage then the limitation would typically be the 1GB vmnic used for the management network.
To allow the migration between datastores not visible by a single host, the VM would also be migrated between hosts.

Scenario

Two hosts managed by single vCenter instance, vc01:

esx01 part of the CLUS1 cluster
esx02 not in a cluster
esx01, connected only to the iSCSI SAN 1
esx02, connected only to the iSCSI SAN 2

Management network on vmk0 using two GbE interfaces on both hosts. Vmk0 on the management network with load balancing based on the virtual port ID.

Test

The test conducted was using storage vmotion to move a virtual machine through VirtualCenter on vc01 from a datastore visible only to esx02 to a datastore visible only to esx01.

The source datastore was LUN02 (visible only to esx02) and the destination datastore was LUN01 (visible only to esx01).
vMotion was used to automatically migrate the VM from esx02 to esx01 (as the VM had moved storage)

Traffic and network usage observed

On esx02 unsupported console, vmk0 source 192.168.0.12.60504 (esx02), destination 192.168.0.11.902 (esx01).
Command: /bin/tcpdump-uw -I vmk0 -nn

esx02

MbRx 190Mb/sec on software iSCSI NIC – Reading the disk information from iSCSI SAN 2 for the VM
MbTx 136Mb/sec – Transmit the information across the management network to esx01.
Command: resxtop --server esx02 (‘n’ for network information)

esx01

MbRx 120Mb/sec – receiving the disk data from esx02 across the management network
MbTx 200Mb/sec on software iSCSI NIC – writing the data to the SAN across the iSCSI network
Command: resxtop --server esx01 (‘n’ for network information)

The following diagram shows the traffic flow of storage vMotion on an iSCSI network:

Resizing a VM's Windows system OS with Set-HardDisk

2010-08-14T20:17:00.000+10:00

With vSphere PowerCLI Update 1 there is now an automated method of increasing the system disk of a virtual machine. This essentially performs the same thing as I’ve been doing manually for a while now - the disk is resized and mounted to a ‘helper vm’ and diskpart extends the disk, the helper VM is then shutdown and the risk removed, and when the resized VM starts up the OS prompts for a final reboot after detecting the disk change.

Note that both the target and helper VM must be powered off, and credentials must be passed or the helpervm argument is ignored. The guest credentials are used on the Windows helper VM to authenticate using the VMtools mechanism to invoke a command inside a VM.


$guetsusername = "domain\username"
$guestpass = Read-Host "Guest password (administrator of the server)?" -assecurestring
$hostusername = "root"
$hostpass = Read-Host "Host Password (VI SDK access to the ESX host)?" -assecurestring
$hostcredential = new-object System.Management.Automation.PSCredential($hostusername, $hostpass)
$guestcredential = new-object System.Management.Automation.PSCredential($guetsusername, $guestpass)
$sizeGB = 30
$sizeKB = $sizeGB *1024*1024
$harddisk = Get-HardDisk -vm vm01
Set-HardDisk -HardDisk $harddisk[0] -capacityKB $sizeKB -hostcredential $hostcredential -guestcredential $guestcredential -helpervm $vm

An error occurred when trying to use domain credentials to pass to the vCenter SDK – host credentials are required, not credentials to vCenter (which seems odd).


Set-HardDisk -HardDisk $harddisk[0] -capacityKB 31457280 -hostcredential $hostcredential -guestcredential $guestcredential -helpervm helpervm
Set-HardDisk : 4/02/2010 1:25:04 PM    Set-HardDisk    919835D1-AD13-41DF-ABDE-D630E96FE722    While performing operation 'Connect to host service 'https://esx01/sdk' at port 902' the following error occured: 'Insufficient permissions in host operating system'
At line:1 char:13
+ Set-HardDisk  <<<< -HardDisk $harddisk[0] -capacityKB 31457280 -hostcredential $hostcredential -guestcredential $guestcredential -helpervm helpervm


Another error occurred when trying to resize a VM, the script used failed.  Note that I tried this manually and it also failed (must have been something particular to that VM).  It was interesting to see that from this it looks like the cmdlet uses the invoke-vmscript to run a remote script through the VMtools interface that calls the diskpart commands.
Set-HardDisk : 4/02/2010 11:22:11 AM    Set-HardDisk    919835D1-AD13-41DF-ABDE-D630E96FE722    Execution of the  script in guest OS on VM 'helpervm' failed: '
Microsoft DiskPart version 5.2.3790.3959
Copyright (C) 1999-2001 Microsoft Corporation.
On computer: HELPERVM
Disk 1 is now the selected disk.
Volume 2 is the selected volume.
The volume you have selected may not be extended.
Please select another volume and try again.
'
At line:1 char:13
+ Set-HardDisk  <<<< -HardDisk $harddisk[0] -capacityKB 31457280 -hostcredential $hostcredential -guestcredential $guestcredential -helpervm $vm

Keeping Citrix sessions alive with sendkeys

2010-05-09T17:40:00.001+10:00

I use the following script as a way of keeping my Citrix session alive to thwart security guys and their over-aggressive timeouts.

Normally I don't like to use sendkeys, but this seemed like the perfect way to trick ica into thinking keys were being pressed...

PowerShell and VBScript versions below.



#
# Find the wfica.exe process and poke keystrokes at it, preventing a Citrix session from timing out
# Be careful what you have open and what keystrokes you poke

[System.Object[]]$processes = get-process | where {$_.ProcessName -eq "wfica32.exe"}
if ($processes.count -ge 1) { $process = $processes[0] }

if ($process -is [System.Diagnostics.Process])
{
  $processes | format-list -property *

  for (;;)
  {
    [void] [System.Reflection.Assembly]::LoadWithPartialName("'Microsoft.VisualBasic")

    [Microsoft.VisualBasic.Interaction]::AppActivate($process.Id)
    start-sleep -seconds 1

    [void] [System.Reflection.Assembly]::LoadWithPartialName("'System.Windows.Forms")
    [System.Windows.Forms.SendKeys]::SendWait("1")
    start-sleep -seconds 240
  }

}



Set objWshShell = CreateObject("Wscript.Shell")

strProcessName = "wfica32.exe"

intProcessID = 0
Call FindProcessID(strProcessName, intProcessID)

if (intProcessID > 0) Then
 wscript.echo "Found process ID: " & intProcessID

 Do 
  objWshShell.AppActivate(intProcessID)
  WScript.Sleep 1000
  objWshShell.SendKeys "1"
  WScript.Sleep 240000
 Loop

End If


Function FindProcessID(ByRef strProcessName, ByRef intProcessID)

 strQuery = "Select * from Win32_Process Where Name = '" & strProcessName & "'" ' WQL query string looking for the specified process

 Set objNameSpace = GetObject("winmgmts://./root/cimv2")    ' The cimv2 namespace of the local machine

 FindProcessID = vbFalse
 Set objProcessSet = objNameSpace.ExecQuery(strQuery)     ' Execute the query
 For Each objProcess in objProcessSet      ' For each process in the set, ordered by oldest to newest
  intProcessID = objProcess.ProcessID
  WScript.Echo "Found process, ID: " & intProcessID
  WScript.Echo objProcess.Name & ", " & intProcessID & ", " & objProcess.CreationDate & ", " & objProcess.CommandLine & ", " & objProcess.Priority & ", " & objProcess.WorkingSetSize & ", " & objProcess.PageFileUsage
  FindProcessID = vbTrue
 Next

 Set objNameSpace = Nothing : Set objProcessSet = Nothing : Set objProcess = Nothing
End Function

More useful command-lines

2010-03-27T21:33:00.002+10:00

This is the third edition of useful command lines, adding another 95 commands that I've found useful. Note that many of the command-line calls may require Microsoft utilities (such as dsquery, wmic, dnscmd).

Most of the commands are for the windows-based command interpreter, with a few PowerShell and ESX service console commands creeping in. They range from diagnostics, troubleshooting and simply automating recurring tasks.

Each command-line can be copied and pasted at the command prompt, if you use a batch file you'll need to reference variables with double-percent (%%).

See the previous two posts for more command-line operations:
http://waynes-world-it.blogspot.com/2008/09/useful-command-lines.html
http://waynes-world-it.blogspot.com/2009/09/more-useful-command-lines.html



Given an IP and mask, return the subet the IP belongs to
for /f "tokens=1-8 delims=.- " %i in ('echo 192.168.5.200 255.255.255.0') do set /a Octet1="%i & %m" >nul & set /a Octet2="%j & %n" >nul & set /a Octet3="%k & %o" >nul & set /a Octet4="%l & %p" >nul & Echo %i.%j.%k.%l,!Octet1!.!Octet2!.!Octet3!.!Octet4!,%m.%n.%o.%p

Disable the internal speaker beep
sc config beep start= demand & net stop beep

List the Volume Shadow Copy Sets
vssadmin list shadows

List the volume Shadow Copy storage associations and sizes
vssadmin list shadowstorage

Create an FSRM storage report to check duplicate files
Storrept Reports Add /Report:DuplicateFiles /Task:FSRM-DuplicateFiles /Scope:"E:\Data" /Name:"Duplicate Files" /Format:CSV /MailTo:user@somewhere.com

Create a scheduled task prior to creating a 2003 FSRM report 
schtasks /create /sc MONTHLY /D 1 /tn FSRM-DuplicateFiles /tr "storrept reports generate /scheduled /task:FSRM-DuplicateFiles" /st 19:00:00 /ru system

Find the number of mailboxes on the specified msExchHomeServerName
dsquery * -filter "(&(objectClass=User)(objectCategory=Person)(mailnickname=*)(msExchHomeServerName=*servername*))" -limit 0 | find /i /c "cn"

Join text copied from the command-line to a single line when pasting
Hold down the shift key when copying the text

Query the current control set, last known good and failed hives 
reg query HKLM\system\select

Make persistent changes to service control dependencies
sc \\%server% config snmp depend= EventLog/

Assign a drive letter to a recently created partition/volume
diskpart assign letter=e

Convert a group from type distribution to type security
dsmod group %groupDN% -secgrp Yes  

Query for global security or distribution groups
dsquery * -filter "(&(groupType:1.2.840.113556.1.4.803:=2))"

Query for global security groups
dsquery * -filter "(&(groupType:1.2.840.113556.1.4.803:=-2147483646))"

Given a list of group DNs, find if those groups are a member of other groups 
for /f "tokens=*" %i in (groups.txt) do @for /f "tokens=*" %m in ('"dsget group -memberof %i | find /i "cn" & if errorlevel 1 Echo Not a member of any groups"') do @echo %i,%m

Find the domain controller GUID of the specified DC 
dsquery * "CN=dcname,OU=Domain Controllers,DC=domainRoot" -scope base -attr objectguid

Find the domain GUID 
dsquery * "domainRoot" -scope base -attr objectGUID

Display the local WLBS Microsoft Network Load Balancing configuration
nlb display

Perform an LDAP ping against rootDSE to get the DC NETLOGON_SAM_LOGON_RESPONSE
adfind -rootDSE -f "(&(DnsDomain=domainFQDN)(Host=%computer%)(User=%computer%$)(DomainGuid=\30\FA\03\19\36\4F\47\1D\35\8F\C4\96\72\74\69\B3)(NtVer=\06\00\00\00))"  netLogon

Find the master browser for the subnet from NetBIOS browser
browstat getmaster \Device\NetBT_Tcpip_{3393FDD1-0D42-4DA4-8E53-36D9E00195F5} NETBIOS_DOMAIN_NAME

List the databases on an MSSQL 2005 server
sqlcmd -S %server% -d master -W -s "," -Q "SELECT NAME FROM sysdatabases Order By Name"

List the Service Control Manager security descriptor
sc sdshow SCMANAGER

Update the Service Control Manager SD on 2003 to allow non-admin access
sc sdset SCMANAGER D:(A;;CCLCRPRC;;;AU)(A;;CCLCRPRC;;;IU)(A;;CCLCRPRC;;;SU)(A;;CCLCRPWPRC;;;SY)(A;;KA;;;BA)S:(AU;FA;KA;;;WD)(AU;OIIOFA;GA;;;WD)

Update the DACL for a service to allow a user to query and interrogate the svc
subinacl /service \\%server%\SysmonLog /grant=domain\user=QSI

Find DFS roots from Active Directory
dsquery * "CN=Dfs-Configuration,CN=System,domainRoot" -filter "(objectClass=fTDfs)"

List the DFS trust domain SPC cache
dfsutil /spcinfo

Find the SQL Server version
sqlcmd -S %server% -d master -W -s "," -Q "SELECT  SERVERPROPERTY('productversion'), SERVERPROPERTY ('productlevel'), SERVERPROPERTY ('edition')"

Find the SQL Server 2000 affinity mask 
sqlcmd -S %server% -d master -W -s "," -Q "select c.value from master..sysconfigures c, master..spt_values v, master..syscurconfigs r where v.type = 'C' and v.number = c.config and v.number >= 0 and v.number = r.config and v.name ='affinity mask'"

Find the SQL Server 2005 affinity mask
sqlcmd -S %server% -d master -W -s "," -Q "SELECT Name, Value, Minimum, Maximum, Value_in_use, is_dynamic, is_advanced from SYS.Configurations Where Name = 'affinity mask'"

Find SQL Server 2005 configuration options
sqlcmd -S %server% -d master -W -s "," -Q "SELECT Name, Value, Minimum, Maximum, Value_in_use, is_dynamic, is_advanced from SYS.Configurations Order By Name"

Find SQL Server 2000 license information, Mode 0x0 per-seat, 0x2 per-proc
reg query "\\%server%\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Microsoft SQL Server\80\MSSQLLicenseInfo\MSSQL8.00

Find SQL Server 2000/2005 instances
reg query "\\%server%\hklm\software\microsoft\Microsoft SQL Server" /v InstalledInstances

Set environment variables for YYYYMMDD and YYYYMMDDHHMMSS
for /f "tokens=1-8 delims=/:. " %i in ('echo %date% %time%') do Set DateTime=%l%k%j&Set DateTimeLong=%l%k%j%m%n%o

Microsoft iSCSI, list sessions
iscsicli SessionList

Microsoft iSCSI, Report iSCSI and MPIO version information
iscsicli VersionInfo

Show disk usage by username (Windows Resource Kit)
diskuse c:\

Check whether external or forest trusts have SID filter qurantining enabled
nltest /domain_trusts /v

From a list of computers, run a command if the machine responds to a ping
for /f %i in (computers.txt) do @ping -n 1 %i >nul & if errorlevel 0 if not errorlevel 1 echo %i

Extract performance counters from a perfmon binary file and save as csv 
relog.exe -f csv Perfmon.blg -o perfmon.csv

Make a bootable CD using the specified boot sector file and source files
oscdimg /m /n /h -bc:\temp\bootsector.bin "c:\temp\source" install.iso

Find the Exchange storage groups known to AD
dsquery * "CN=Configuration,domainRoot" -filter "(&(objectClass=msExchStorageGroup))" -attr name msExchESEParamCircularLog msExchESEParamEventSource msExchESEParamLogFilePath msExchESEParamLogFileSize msExchESEParamSystemPath msExchESEParamZeroDatabaseDuringBackup msExchRecovery msExchESEParamEnableOnlineDefrag msExchESEParamEnableIndexChecking msExchESEParamBaseName msExchESEParamCommitDefault msExchESEParamDbExtensionSize msExchESEParamPageTempDBMin msExchESEParamPageFragment msExchESEParamCheckpointDepthMax

Check if an executable is large address aware (confirms support)
dumpbin /headers sqlservr.exe

Find all file/registry/thread/event details for a remote machine
psexec \\%server% -c handle.exe -a > c:\temp\handle.txt

Dump configuration with netsh that can be used to import
netsh interface ip dump > IPConfig.txt

Reapply settings saved with netsh export
netsh -f IPConfig.txt

List the SCSI devices for a group of machines
for /f %i in (computers.txt) do @for /f "tokens=1,2,*" %m in ('"devcon -m:\\%i listclass SCSIAdapter | find /i "VEN""') do @echo %i,%o

Adjust the priority of a process given its PID 
setprio /P %PID% /L 2

Find the paged and non-paged pool memory allocations
poolmon

Create a local tag file for paged and non-paged pool memory allocations
poolmon /c

Add a trigger to an existing scheduled task
jt.exe /sac "\\%server%\c$\windows\Tasks\taskname.job" /ctj StartTime=15:10 StartDate=02/02/2010 type=DAILY typearguments=1

Find AdminSDHolder / SDPROP groups in the directory
dsquery * -filter "(&(objectClass=Group)(objectCategory=Group)(adminCount=1))"   

Query for all objects that are protected (that are members of the groups above)
dsquery * -filter "(adminCount=1)"

Count how many user accounts are protected by the SDPROP process
dsquery * -filter "(&(objectClass=User)(objectCategory=Person)(adminCount=1))" | find /i /c "CN="

Find disk and partition relationships
wmic path Win32_LogicalDiskToPartition

LDAP string to search for mail-enabled user and contact objects
"(&(|((objectClass=Contact)(objectClass=User)))(objectCategory=Person)(mailnickname=*))"

Find SCPs (Service Connection Points) for objects other than print queues
dsquery * -filter "&((objectClass=ConnectionPoint)(!(objectClass=PrintQueue)))"  -limit 0

List VSS Shadowstorage associations
wmic /node:%server% path win32_shadowstorage

List VSS Max storage for disabled volumes
reg query HKLM\System\CurrentControlSet\Services\VSS\Volumes\Associations\{00b810fd-8ab6-11de-9d34-806e6f6e6963}\{00b810fd-8ab6-11de-9d34-806e6f6e6963}\MaxDiffSpace

Force a STOP error and generate a memory dump (Stop 0x2E) after enabling
Right Ctrl+ScrollLock+ScrollLock

Enable the option to generate a STOP error on demand
reg add "HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\i8042prt\Parameters /v CrashOnCtrlScroll /t reg_dword /d 0x1

Logoff a remote terminal sessions session (after using query.exe)
logoff rdp-tcp#42 /v /server:%server%

List the DLL function exports of the specified DLL
dumpbin c:\temp\dllname.dll /exports

Terminate a disconnected terminal services session
rwinsta 0 /server:%server%

Find memory information from a list of machines
for /f %i in (computers.txt) do for /f "tokens=1-5" %m in ('"wmic /node:%i path Win32_PerfFormattedData_PerfOS_Memory get PoolNonpagedBytes,PoolPagedBytes,PoolPagedResidentBytes,FreeSystemPageTableEntries,AvailableMBytes | find /i /v "bytes""') do echo %i,%m,%n,%o,%p,%q

Set the percentage at which the paged pool working set is trimmed
reg add "\\%server%\HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\Memory Management" /v PoolUsageMaximum /t reg_dword /d 60

Calculate a replication summary for the domain
repadmin /replsum /bysrc /bydest /sort:delta

Check a 2008 R2 schema forest prep has run successfully (rvevision=5)
dsquery * "CN=ActiveDirectoryUpdate,CN=ForestUpdates,CN=Configuration,domainRoot" -attr * -scope base

Check a 2008 R2 schema domain prep has run successfully (revision=5)
dsquery * "CN=ActiveDirectoryUpdate,CN=DomainUpdates,CN=System,domainRoot" -attr * -scope base

Find the infrastructure role holder for the DNS domain application partition
dsquery * "CN=Infrastructure,DC=DomainDNSZones,domainRoot" -attr fSMORoleOwner   

Find the application partitions from the forest root configuration NC
dsquery * "CN=Partitions,CN=Configuration,domainRoot" -filter "(&(objectClass=crossRef)(msDS-SDReferenceDomain=*))" -attr ncName dnsRoot

Monitor the page faults of an application (hard and soft)
pfmon.exe c:\temp\application.exe

Monitor the page faults of a process (hard and soft)
psmon /p %PID%

Reset the SystemPages memory management for available PTE's to server 2003 def.
reg add "\\%server%\HKLM\SYSTEM\CurrentControlSet\Control\Session Manager\Memory Management" /v SystemPages /t reg_dword /d 0x0x

Find Exchange mailbox limits for user accounts
csvde -f limits.csv -l samaccountname,cn,mDBStorageQuota,mDBOverQuotaLimit,mDBUseDefaults,mDBOverHardQuotaLimit,homeMDB -r "(&(objectClass=User)(objectCategory=Person)(mailnickname=*))"

Find computer accounts not server, XP or Vista
dsquery * domainroot -filter "(&(objectCategory=Computer)(objectClass=Computer)(&(!(operatingSystem=*prof*)(!(operatingSystem=*server*)(!(operatingSystem=*vista*))))))" -limit 0 -attr CN operatingSystem operatingSystemVersion

Find SPNs from the directory
dsquery * -filter "(servicePrincipalName=*)" -attr cn servicePrincipalName -limit 0

Query a list of groups and determine the group type (global/local/universal)
dsquery group -name *groupfilter* | dsget group -scope -secgrp -dn

Perform a NetBIOS name lookup
nblookup %NETBIOS_NAME%

Extract DNS scavenging events from the DNS eventlog
wmic /node:"%server%" path Win32_NTLogEvent WHERE "SourceName='DNS' AND LogFile='DNS Server' AND EventCode=2501"

Dump GPO information on priority and inherited links for an OU
cscript DumpSOMInfo.wsf "OUName" /ShowInheritedLinks

Copy a GPO 
cscript copygpo.wsf "Default Domain Policy" "Test Domain Policy" domain.com domain.com %SourceDC% %TargetDC%

Backup a number of GPOs from a control file input
for /f "tokens=*" %i in (gpos.txt) do cscript backupgpo.wsf "%i" c:\temp\gpo

Create a number of blank GPOs from a control file input         
for /f "tokens=*" %i in (c:\temp\gpo.txt) do cscript CreateGPO.wsf "%i" /domain:domain.com

Add GPO permissions for a number of GPOs
for /f "tokens=*" %i in (c:\temp\gpo.txt) do cscript SetGPOPermissions.wsf "%i" domain\user /permission:fulledit

List the available features for a 2008 R2 online installation 
dism /online /get-features

Allow remote desktop connections on a 2008 R2 server (requires reboot)
reg add "HKLM\SYSTEM\CurrentControlSet\Control\Terminal Server" /v fDenyTSConnections /t reg_dword /d 0x0

Allow remote desktop connections on a 2008 R2 server core  (requires reboot)
cscript C:\Windows\System32\Scregedit.wsf /ar 0

Set the boot timeout for a 2008 server
bcdedit /timeout 5

Modify startup and recovery settings prevent automatic restart on 2008 server 
reg add HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\CrashControl /v "AutoReboot" /t reg_dword /d 0x0

Modify the performance options to adjust for best performance
reg add HKU\.default\Software\Microsoft\Windows\CurrentVersion\Explorer\VisualEffects /v VisualFXSetting /t reg_dword /d 0x2

Disable hibernation on 2008 server
powercfg -hibernate off

Change the default user profile to adjust performance options for best perf
reg load c:\users\Default\ntuser.dat & reg add HKLM\a\Software\Microsoft\Windows\CurrentVersion\Explorer\VisualEffects /v VisualFXSetting /t reg_dword /d 0x2 & reg unload hkml\a

Enable boot logging on 2008 server
bcdedit /set {default} bootlog yes  

Disable Windows Error Reporting on 2008 server
serverWerOptin /disable

Adjust windows server 2008 event log sizes to 128MB
wevtutil sl Security /ms:135266304

Install Windows Server 2008 R2 optional components (features and roles)
cmd /c start /w ocsetup RSAT-NIS;SNMP;WMISnmpProvider;WindowsServerBackup;WindowsServerBackupCommandlet;NetFx3;MicrosoftWindowsPowerShellISE /quiet /norestart /log:c:\temp\install.log

Check Microsoft iSCSI initiator MaxRequestHoldTime 
reg query "\\%server%\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Class\{4D36E97B-E325-11CE-BFC1-08002BE10318}\0002\Parameters" /v MaxRequestHoldTime

PowerCLI: Copy-VMGuestFile

2010-01-14T22:33:00.004+10:00

In VMware PowerCLI Update 1, a new cmdlet has been added to copy files to/from guest OS’s using the VMTools. Under normal circumstances this wouldn’t be all that useful, but if a machine is not accessible on the network – either a DMZ or a test-lab environment for example – there is now a method to easily copy files to and from the guest machines.

For example:

copy-vmguestfile -source c:\windows\system32\imageres.dll -destination c:\temp\ -vm vm01 -guesttolocal -hostuser root -hostpassword * -guestuser administrator -guestpassword *

Note that you need to specify the passwords, or not specify credentials at all for a prompt, or you can use a pscredential object.

Unfortunately wildcards are currently unsupported, but it would be easy to loop through local files within PowerShell to upload based on wildcards.

I ran this with the measure-command cmdlet to see how fast it was – about 0.5MB/sec– not too quick, but a lot better than nothing.

measure-command -expression {copy-vmguestfile -source c:\windows\system32\imageres.dll -destination c:\temp\ -vm vm01 -guesttolocal -hostuser root -hostpassword * -guestuser administrator -guestpassword *}

Seconds : 26
Milliseconds : 539
Ticks : 265390633
TotalDays : 0.000307165084490741
TotalHours : 0.00737196202777778
TotalMinutes : 0.442317721666667
TotalSeconds : 26.5390633
TotalMilliseconds : 26539.0633

vCenter file copy with virtual floppy disk

2009-11-29T16:07:00.000+10:00

n an ESXi or vCenter environment, without network connectivity to a virtual machine from your administrative workstation it's quite hard to bi-directionally transfer files.

This post discusses a simple method I use for small quick file transfer from a secure virtual machine to my administrative workstation, using virtual floppy drives managed from the command-line.

This process uses vfd.exe, but presumably any virtual floppy software on your workstation could be used. The virtual machine also needs to have a floppy drive...

To bi-directionally transfer files from a non-network connected VM to my administrative workstation:

Create a virtual floppy disk on my workstation

Mount that floppy disk file as a virtual floppy drive on my workstation as A:

Through VI client, connect the VM to the A: drive

On the VM, copy files to/from A: drive and disconnect from the floppy

On my workstation, copy files to/from the virtual A:

Close the virtual floppy

Using vfd.exe:

vfd install

vfd open c:\temp\new.flp

Use VI Client to connect to A:

Copy files, then disconnect the VM virtual device

copy a:\*.*

vfd close

The obvious limitation of this is that the size is limited to 1.44MB. I tried with a 2.44MB floppy and the virtual machine didn't recognise this disk.

This doesn’t work with a virtual CD-ROM, as it’s mounted as read-only in the VM, so bi-directional copies aren’t available. You can still use a similar process for creating an ISO to copy files into a VM though. For this I use oscdimg.exe (Microsoft utility) to create my ISO from the command-line.

VMware PowerCLI commands

2009-10-30T23:14:00.003+10:00

The VMware PowerCLI PowerShell interface provided for managing vSphere systems is a fantastic tool that should be useful for all VMware admins.

I've gathered these commands while implementing and managing ESXi 4.0 clusters, use with caution on any production system.

This is an extension of a previous post:
VMware vSphere PowerCLI commands



Join a cluster by moving an ESX host from one location to the cluster
Move-Inventory -Item (Get-VMHost -Name esxHost) -Destination (Get-Cluster -Name clusterName)

Get the VMware.Vim.ClusterComputeResource MO from the PowerCLI cluster object
$clusterview = get-view $cluster.Id

Reconfigure a host for VMware HA (high availability)
$vmhost = get-vmhost -name esxHost; $hostMO = Get-View -ID $vmhost.ID; $hostMO.ReconfigureHostForDAS()

Find migration events for the last day
$events = Get-VIEvent -Start (Get-Date).AddDays(-1) | where {$_.fullFormattedMessage -match "Migrating.*"}

Find events other than CPU Alarms or user login/logout for the last day
$events = Get-VIEvent -Start (Get-Date).AddDays(-1) | where {$_.fullFormattedMessage -notmatch "Alarm.*CPU.*|User.*logged.*"}

Find events for degraded MPIO path redundancy 
$events = Get-VIEvent -Start (Get-Date).AddDays(-1) | where {$_.fullFormattedMessage -match "Path redundancy to storage.*degraded"}

Report the date, host and description for MPIO path redundancy errors
foreach ($event in $events) {write-output ($event.createdTime.ToString() + "," + $event.host.get_name() + "," + $event.fullFormattedMessage)}

List a table of VI events with only the date and message
$events | format-table -wrap -autosize -property createdTime,fullFormattedMessage

List the physical networks adapters and the current link speed (ESX 4.0)
$hostSystem = get-view -ViewType HostSystem; $hostConfigManager = $hostSystem.get_ConfigManager(); $hostNetworkSystem = $hostConfigManager.get_NetworkSystem(); $netSystem = Get-View $hostNetworkSystem; $netSystem.NetworkConfig.pnic; foreach ($pnic in  $netSystem.NetworkConfig.pnic) {Write-Output ($pnic.Device + "," + $pnic.spec.linkspeed.SpeedMB)}

List the vSwitches and the uplinks currently attached
$hostSystem = get-view -ViewType HostSystem; $hostConfigManager = $hostSystem.get_ConfigManager(); $hostNetworkSystem = $hostConfigManager.get_NetworkSystem(); $netSystem = Get-View $hostNetworkSystem; foreach ($vswitch in  $netSystem.NetworkConfig.vSwitch) {Write-Output ($vSwitch.Name + "," + $vswitch.spec.policy.NicTeaming.NicOrder.ActiveNic)}

Remove snapshots from a group of machines
$VMs = Get-VM -Location (get-folder -name "vmFolder"); foreach ($vm in $vms) {remove-snapshot -snapshot (Get-Snapshot -vm $vm) -confirm:$false}

Take snapshots of a group of machines
$VMs = Get-VM -Location (get-folder -name "vmFolder"); foreach ($vm in $VMs) {New-Snapshot -Name "snapshot 01" -description "Snapshot description" -vm $vm -Quiesce:$false}

Find VM name, description and primary disk datastore
$VMs = get-vm; foreach ($vm in $VMs) {write-output ($vm.Name + ",""" + $vm.Description + """," + $vm.harddisks[0].FileName.Replace(" ", ",")) | out-file -append -filepath c:\temp\VM_Datastores.txt}

Bring a host out of maintenance most
Set-VMHost -VMHost esxHost -State Connected

Generate diagnostic support bundles for all hosts
get-log -vmhost (get-vmhost) -bundle -destinationpath c:\temp\bundles

Find the network adapter type for each VM
$vms = get-vm ; foreach ($vm in $vms) {write-host $vm.Name "-"  $vm.networkadapters[0].type}

Find physical NICs and whether they're set to autonegotiate or hardcoded
foreach ($pnic in $hostNetwork.pnic) {if($pnic.linkSpeed -eq $null) {$ls = "Auto"} else {$ls= $pnic.linkSpeed.speedMB.toString() + ":" + $pnic.linkSpeed.duplex} ;write-output ($pnic.Device + "," + $ls)}

Find host sytem build information
$hostSystems = get-view -ViewType HostSystem; foreach ($hostSystem in $hostSystems) {Write-Output ($hostSystem.Name + "," + $hostSystem.config.product.Fullname)}

Find VMs and whether the VMtools is configured to synchronising time 
$vmSet = Get-VM ; foreach ($vm in $vmSet) { $view = get-view $vm.ID ;$config = $view.config; $tools = $config.tools; Write-Output ($vm.Name + "," + $tools.SyncTimeWithHost) }

Revert to a snapshot
set-vm -vm vmName -snapshot (get-snapshot -vm vmName) -confirm:$false

Remove a virtual machine from inventory and delete from disk
remove-vm -DeleteFromDisk:$true -RunAsync:$true -vm vmName

Shutdown one or more Virtual Machine guests
shutdown-vmguest -vm $vms -confirm:$false

Start one or more Virtual Machine guests
start-vm -vm $vms -confirm:$false

Forcefully power off one or more Virtual Machines
stop-vm $vms -confirm:$false

Get a virtual switch from the specified ESX host
get-virtualswitch -name vSwitch1 -vmhost esxHost

Create a new port group on the specified vSwitch
New-VirtualPortGroup -Name "pgName" -VirtualSwitch $vs

Find ESX memory balloon averages for the last five days
get-stat -entity $hosts -start (Get-Date).AddDays(-5) -finish (Get-Date) -stat mem.vmmemctl.average

Export a list of VMs
$vms | select-object -prop Name | out-file -filepath c:\temp\vms.txt

Export a list of VM guest hostnames 
$vms = get-vm; foreach ($vm in $vms) { write-output $vm.guest.get_HostName()}

Service Control Manager Security for non-admins

2009-10-13T22:38:00.000+10:00

Allowing non-administrators rights for service control sounds deceptively simple, but unfortunately it’s not. After Windows Server 2003 SP1, the default rights are very focused on administrator-only access for service control.

This post describes how to view and modify the security descriptor for the Service Control Manager (SCM) and individual services as required.

Show the descriptor for SCM:
sc sdshow SCMANAGER

D:(A;;CC;;;AU)(A;;CCLCRPRC;;;IU)(A;;CCLCRPRC;;;SU)(A;;CCLCRPWPRC;;;SY)(A;;KA;;;BA)S:(AU;FA;KA;;;WD)(AU;OIIOFA;GA;;;WD)

In this case, the output shows that by default, Authenticated Users (AU) only have connect, but not enumerate (LC) for SCM.

If you want to allow a non-administrator to connect to the Service Control Manager and enumerate the services, you can modify the security descriptor by using something like the following command to add enumerate, read control and query lock status for Authenticated Users with SCM:
sc sdset SCMANAGER

D:(A;;CCLCRPRC;;;AU)(A;;CCLCRPRC;;;IU)(A;;CCLCRPRC;;;SU)(A;;CCLCRPWPRC;;;SY)(A;;KA;;;BA)S:(AU;FA;KA;;;WD)(AU;OIIOFA;GA;;;WD)

This allows connecting to the SCM and enumerating services. However, if the DACL on the individual services only allows administrators access to the services, then they still won’t be accessible. You’ll need to run specific 'sc sdset' commands against particular services, or use subinacl to change all services with one command.

Note that the sc.exe version with XP does not support this syntax – use the sc.exe on 2003 server.

For individual services, you could then allow query and interrogate with the following command:

subinacl /service \\server\* /grant=domain\user=QSI

Note that to map the ACE flags to the meaning with regards to service control, I went through the following process:

Find the access rights from the flag, eg CC = SDDL_CREATE_CHILD = ADS_RIGHT_DS_CREATE_CHILD (ACE Strings link below)
Find the constant matching this value, eg. 0x1 (ADS_RIGHTS_ENUM Enumeration link below)
Match this to the SCM access right for the hexadecimal value, eg. 0x1 = SC_MANAGER_CONNECT (Service Security and Access Rights link below)

Map between sdshow output, right, hex value and SC/service meaning:


"CC"  ADS_RIGHT_DS_CREATE_CHILD          = 0x1,    SC_MANAGER_CONNECT, SERVICE_QUERY_CONFIG
"DC"  ADS_RIGHT_DS_DELETE_CHILD          = 0x2,    SC_MANAGER_CREATE_SERVICE, SERVICE_CHANGE_CONFIG
"LC"  ADS_RIGHT_ACTRL_DS_LIST            = 0x4,    SC_MANAGER_ENUMERATE_SERVICE, SERVICE_QUERY_STATUS
"SW"  ADS_RIGHT_DS_SELF                  = 0x8,    SC_MANAGER_LOCK, SERVICE_ENUMERATE_DEPENDENTS
"RP"  ADS_RIGHT_DS_READ_PROP             = 0x10,   SC_MANAGER_QUERY_LOCK_STATUS, SERVICE_START, 
"WP"  ADS_RIGHT_DS_WRITE_PROP            = 0x20,   SC_MANAGER_MODIFY_BOOT_CONFIG, SERVICE_STOP
"DT"  ADS_RIGHT_DS_DELETE_TREE           = 0x40,   SERVICE_PAUSE_CONTINUE
"LO"  ADS_RIGHT_DS_LIST_OBJECT           = 0x80,   SERVICE_INTERROGATE
"CR"  ADS_RIGHT_DS_CONTROL_ACCESS        = 0x100   SERVICE_USER_DEFINED_CONTROL
"RC"  READ_CONTROL                       = 0x20000 READ_CONTROL

Access right Description for services and SCM:


SERVICE_QUERY_CONFIG (0x0001) Required to call the QueryServiceConfig and QueryServiceConfig2 functions to query the service configuration. 
SERVICE_CHANGE_CONFIG (0x0002) Required to call the ChangeServiceConfig or ChangeServiceConfig2 function to change the service configuration. Because this grants the caller the right to change the executable file that the system runs, it should be granted only to administrators.  
SERVICE_QUERY_STATUS (0x0004) Required to call the QueryServiceStatusEx function to ask the service control manager about the status of the service. 
SERVICE_ENUMERATE_DEPENDENTS (0x0008) Required to call the EnumDependentServices function to enumerate all the services dependent on the service. 
SERVICE_START (0x0010) Required to call the StartService function to start the service. 
SERVICE_STOP (0x0020) Required to call the ControlService function to stop the service. 
SERVICE_PAUSE_CONTINUE (0x0040) Required to call the ControlService function to pause or continue the service. 
SERVICE_INTERROGATE (0x0080) Required to call the ControlService function to ask the service to report its status immediately. 
SERVICE_USER_DEFINED_CONTROL(0x0100) Required to call the ControlService function to specify a user-defined control code. 
SERVICE_ALL_ACCESS (0xF01FF) Includes STANDARD_RIGHTS_REQUIRED in addition to all access rights in this table. 
READ_CONTROL Required to call the QueryServiceObjectSecurity function to query the security descriptor of the service object. 

SC_MANAGER_CONNECT (0x0001) Required to connect to the service control manager. 
SC_MANAGER_CREATE_SERVICE (0x0002) Required to call the CreateService function to create a service object and add it to the database. 
SC_MANAGER_ENUMERATE_SERVICE (0x0004) Required to call the EnumServicesStatusEx function to list the services that are in the database. 
SC_MANAGER_LOCK (0x0008) Required to call the LockServiceDatabase function to acquire a lock on the database. 
SC_MANAGER_QUERY_LOCK_STATUS (0x0010) 
SC_MANAGER_MODIFY_BOOT_CONFIG (0x0020) Required to call the NotifyBootConfigStatus function. 
SC_MANAGER_ALL_ACCESS (0xF003F) Includes STANDARD_RIGHTS_REQUIRED, in addition to all access rights in this table.

Directory service object access rights


"RC"  SDDL_READ_CONTROL  READ_CONTROL 
"RP"  SDDL_READ_PROPERTY  ADS_RIGHT_DS_READ_PROP  
"WP"  SDDL_WRITE_PROPERTY  ADS_RIGHT_DS_WRITE_PROP  
"CC"  SDDL_CREATE_CHILD  ADS_RIGHT_DS_CREATE_CHILD  
"DC"  SDDL_DELETE_CHILD  ADS_RIGHT_DS_DELETE_CHILD  
"LC"  SDDL_LIST_CHILDREN  ADS_RIGHT_ACTRL_DS_LIST  
"SW"  SDDL_SELF_WRITE  ADS_RIGHT_DS_SELF  
"LO"  SDDL_LIST_OBJECT  ADS_RIGHT_DS_LIST_OBJECT  
"DT"  SDDL_DELETE_TREE  ADS_RIGHT_DS_DELETE_TREE  
"CR"  SDDL_CONTROL_ACCESS  ADS_RIGHT_DS_CONTROL_ACCESS

ADS enum:


typedef enum  {
  ADS_RIGHT_DS_CREATE_CHILD          = 0x1,
  ADS_RIGHT_DS_DELETE_CHILD          = 0x2,
  ADS_RIGHT_ACTRL_DS_LIST            = 0x4,
  ADS_RIGHT_DS_SELF                  = 0x8,
  ADS_RIGHT_DS_READ_PROP             = 0x10,
  ADS_RIGHT_DS_WRITE_PROP            = 0x20,
  ADS_RIGHT_DS_DELETE_TREE           = 0x40,
  ADS_RIGHT_DS_LIST_OBJECT           = 0x80,
  ADS_RIGHT_DS_CONTROL_ACCESS        = 0x100 

} ADS_RIGHTS_ENUM;

READ_CONTROL = 0x20000;

References:

Applying Security Descriptors on the Device Object
http://msdn.microsoft.com/en-us/library/ms793368.aspx

Non-administrators cannot remotely access the Service Control Manager after you install Windows Server 2003 Service Pack 1
http://support.microsoft.com/default.aspx?scid=kb;EN-US;907460

Securing a Remote WMI Connection
http://msdn.microsoft.com/en-us/library/aa393266(VS.85).aspx

Configuring a Report Server for Remote Administration
http://msdn.microsoft.com/en-us/library/ms365170(SQL.90).aspx

Service Security and Access Rights
http://msdn.microsoft.com/en-us/library/ms685981(VS.85).aspx

How to grant users rights to manage services in Windows 2000
http://support.microsoft.com/kb/288129

How to troubleshoot WMI-related issues in Windows XP SP2
http://support.microsoft.com/kb/875605

ACE Strings
http://msdn.microsoft.com/en-us/library/aa374928(VS.85).aspx

ADS_RIGHTS_ENUM Enumeration
http://msdn.microsoft.com/en-us/library/aa772285(VS.85).aspx

Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.

VMware Command-Line Interface commands

2009-09-30T23:03:00.002+10:00

The Windows command-line interface provided for managing ESX/ESXi systems is an invaluable tool for managing ESX infrastructure at the command-line. When using ESXi without a service console the CLI becomes even more useful.

I've gathered these commands while implementing and managing ESXi 4.0 clusters, use with caution on any production system.

ESXi 4.0 RCLI:


List the NTP servers used by the host
vicfg-ntp.pl --server esx01 --list

Add a Software iSCSI NIC named vmk2
esxcli --server=esx01 swiscsi nic add -n vmk2 -d vmhba33

List the Software iSCSI NICs
esxcli --server=esx01 swiscsi nic list -d vmhba33

List the software iSCSI status on a host
vicfg-iscsi.pl --server esx01 --swiscsi --list

Enable software iSCSI on an ESX host
vicfg-iscsi.pl --server esx01 --swiscsi --enable

List the adapters bound to software iSCSI
esxcli --server=esx01 swiscsi nic list -d vmhba33

List the VMKernel NICs
vicfg-vmknic.pl --server esx01 --list

List the software iSCSI adapters
vicfg-iscsi.pl --server esx01 --adapter --list

Set the iSCSI alias for the specified adapter
vicfg-iscsi.pl --server esx01 --iscsiname --alias esx01 vmhba33

Bind a VMK software iSCSI NIC for MPIO PSA
esxcli --server=esx01 swiscsi nic add -n vmk2 -d vmhba33

Rescan a storage adapter bus
vicfg-rescan.pl --server esx01 vmhba33

Add a dynamic iSCSI discovery target
vicfg-iscsi.pl --server esx01 --discovery --add --ip 10.1.1.10:3260 vmhba33

Add CHAP authentication to an iSCSI discovery target
vicfg-iscsi.pl --server esx01 --authentication --level chapRequired --method CHAP --auth_username esxchap --auth_password chapacc3ss80 --ip 10.2.128.33:3260 vmhba33

Find the current ScratchConfig scratch location
vicfg-advcfg.pl --server esx01 -g ScratchConfig.ConfiguredScratchLocation

Set the scratch location for ESXi
vicfg-advcfg.pl --server esx01 -s "/vmfs/volumes/esxds01/Scratch/esx01" ScratchConfig.ConfiguredScratchLocation

Check if CIM OEM providers are enabled (such as Dell OM)
vicfg-advcfg.pl --server esx01 -g UserVars.CIMOEMProvidersEnabled

Enable CIM OEM Providers (such as Dell OM)
vicfg-advcfg.pl --server esx01 -s "1" UserVars.CIMOEMProvidersEnabled

Query the patches/updates/bulletins/VIBs installed on ESXi
vihostupdate.pl --server esx01 -q

Set the SNMP community for the ESXi host
vicfg-snmp.pl --server esx01 -c public

Enable the SNMP agent on an ESXi host
vicfg-snmp.pl --server esx01 -E

List the iSCSI node name
vicfg-iscsi.pl --server esx01 --list --iscsiname --adapter vmhba36

List the preferred nativte multipathing (NMP) path for a device
 esxcli --server esx01 nmp fixed getpreferred -d naa.6090332880cfdc44fda634b1ca2457b8

Check whether round robin path selection is used for a device
esxcli --server esx01 nmp roundrobin getconfig -d naa.6090a02833cfdc7ffd4434b1ca5457b8 

List the disk NAA/UUIDs known to a host
esxcli --server esx01 nmp device list

List the MPIO path to device mapping
esxcfg-mpath.pl --server esx01 -m

List the SCSI devices known to a host
vicfg-scsidevs.pl --server esx01 --list   

List the available datastores on a host
vifs.pl --server esx01 --listds

List the contents of a datastore
 vifs.pl --server esx01 --dir [datastore]

Upload a local file to a datastore through vifs
vifs.pl --server esx01 --put c:\temp\file.txt dir/file.txt?dsName=datastoreName

List virtual switches, port groups, uplinks and MTU
vicfg-vswitch.pl -l --server esx01

Browse the datastores or local host through ssl
https://esx01/folder or https://esx01/host

Find the vmnic configuration, including driver, current speed
vicfg-nics.pl --server esx01 -l 

Set a vNIC to auto-negotiate
vicfg-nics.pl --server esx01 --vihost esx01 -a vmnic0

List the host-based files on an ESXi client
vifs.pl --server esx01 --dir /host

A few other useful VMware tips:


Recreate the rui.pfx file for VirtualCenter
openssl pkcs12 -export -in rui.crt -inkey rui.key -name rui -passout pass:testpassword -out rui.pfx

Forcefully power off a suspended VM
Delete the vmss file, and then power on the VM (state is lost)

ESXi 4.0, access unsupported ssh console through the dcui
Press Alt+F1, type unsupported, then the root password

More useful command-lines

2009-09-20T23:57:00.000+10:00

This is the second edition of useful command lines, adding another 132 commands that I've found useful. Note that many of the command-line calls may require Microsoft utilities (such as dsquery, wmic, dnscmd).

Most of the commands are for the windows-based command interpreter, with a few PowerShell and ESX service console commands creeping in. They range from diagnostics, troubleshooting and simply automating recurring tasks.

Each command-line can be copied and pasted at the command prompt, if you use a batch file you'll need to reference variables with double-percent (%%).

See the original post with another 425 commands http://waynes-world-it.blogspot.com/2008/09/useful-command-lines.html



Given an IP and mask, return the subet the IP belongs to
for /f "tokens=1-8 delims=.- " %i in ('echo 192.168.5.200 255.255.255.0') do set /a Octet1="%i & %m" >nul & set /a Octet2="%j & %n" >nul & set /a Octet3="%k & %o" >nul & set /a Octet4="%l & %p" >nul & Echo %i.%j.%k.%l,!Octet1!.!Octet2!.!Octet3!.!Octet4!,%m.%n.%o.%p

Display the contents of the client DNS resolver cache
ipconfig /displaydns

Find the package source path of a program from SMS
wmic /namespace:\\root\sms\site_%sitecode% /node:"server" path SMS_Package Where "Name like '%programname%'" get Name,ShareName,PkgSourcePath

Find the session associated with a process
wmic path win32_process get name,sessionid

List the local winstation windows objects
objdir \Windows\Windowstations\Winsta0

Query the configuration container for Exchange mailbox stores
dsquery * ",CN=Configuration,DC=domainroot" -filter "(&(objectClass=msExchPrivateMDB)(objectCategory=msExchPrivateMDB))"

Query a Virtual Centre/VC 2.5 database for Virtual Machine details
sqlcmd -S server -d database -W -s "," -Q "select ENT.Name as 'Name', Lower(DNS_Name) as 'DNS Name', Guest_OS as 'OS', Mem_Size_MB as 'Mem', Num_VCPU as 'CPU', Num_NIC as 'NIC', IP_Address as 'IP', NET.MAC_Address as 'MAC Address', VM.FILE_Name as 'VMX location' from vpx_vm VM inner join VPX_NIC NET on VM.ID = NET.ENTITY_ID inner join VPX_ENTITY ENT on VM.ID = ENT.ID Order By ENT.Name"

Query a Virtual Centre/VC 2.5 database for Virtual Machine snapshots (GMT+10)
sqlcmd -S server -d database -W -s "," -Q "select ENT.Name as 'Name', Lower(DNS_Name) as 'DNS Name', Guest_OS as 'OS', Mem_Size_MB as 'Mem', IP_Address as 'IP', VM.FILE_Name as 'VMX location', VM.Suspend_Time as 'Suspend Time', VM.Suspend_Interval as 'Suspend Interval', VMS.Snapshot_Name as 'Snapshot Name', VMS.Snapshot_Desc 'Snapshot Description', DateAdd(Hour, 10, VMS.Create_Time) as 'Snapshot Time', VMS.Is_Current_Snapshot 'Current Snapshot' from vpx_vm VM inner join VPX_NIC NET on VM.ID = NET.ENTITY_ID inner join VPX_ENTITY ENT on VM.ID = ENT.ID inner join VPX_SNAPSHOT VMS on VM.ID = VMS.VM_ID"

Test the password for a domain account (assumes no existing IPC connection)
net use \\server\ipc$ /user:%domain%\%testuser% *

View the last-access, modified, created and MFT entry modified timestampes
timestomp "%fullpathtoFile%" -v

Create a scheduled task escaped with a command containing double-quotes (2003)
schtasks /create /SC Daily /TN "Task" /ST 12:00 /TR "cmd /c echo \"Test\"" /RU System

Create a scheduled task running two commands
schtasks /create /SC Daily /TN "Task" /ST 12:00 /TR "cmd /c echo Test1 & cmd /c echo Test2" /RU System

Check a number of computers to see if hibernation is enabled
for /f %i in (%controlfile%.txt) do @if exist \\%~i\c$\hiberfil.sys (echo %~i,Enabled) else (echo %~i,Disabled)

For each path in a control file, list the 8.3 short equivalent
for /f "tokens=*" %i in (test.txt) do echo %~si

Use if exist and disabled path parsing to bypass max_path
for /f "tokens=*" %i in (test.txt) do if exist "\\?\UNC\%~pnxi" echo File exists

Enumerate a cluster through WMI
wmic /node:"%node%" /namespace:\\root\mscluster path MSCluster_Cluster

Given a path exceeding MAX_PATH, return the 8.3 equivalent of the directories
for /f "tokens=*" %i in (longfiles.txt) do for /d %m in ("\\%~pi") do echo %~sm%~nxi

Given a path you know contains deeper than 260, batch to return the 8.3 subdirs
(3 lines) @for /f "tokens=*" %%i in (c:\temp\longdir1.txt) do @for /d %%m in ("\\%%~pi") do @Call :Process "%%~si" || :process || @if "%~1"=="" (goto :EOF) else (@for /d %%i in ("%~1\*.*") do @echo %%~si & Call :Process "%%~si")

Set a Domain Controller to be a Global Catalog server
dsmod server "%DC_DN%" -isgc yes

Check which network connections (drive mappings) a computer has
wmic /node:"%computer%" path win32_logicaldisk where "DriveType=4" get DeviceID,ProviderName 

Query the current site of a remote computer using nltest
nltest /dsgetsite /server:%computer%

Query the current site of a remote computer using the registry
reg query \\%computer%\hklm\system\currentcontrolset\services\netlogon\parameters /v DynamicSiteName

Check the schema version on a Domain Controller (R2=31)
reg query \\%dc%\hklm\system\currentcontrolset\services\NTDS\parameters /v "Schema Version"

Query the revision of 2003 Update (R2=9)
dsquery * CN=Windows2003Update,CN=ForestUpdates,CN=Configuration,%forestRoot% -attr revision

Check the schema version on a Domain Controller (R2=31)
dsquery * "CN=Schema,CN=Configuration,%forestRoot%" -attr objectVersion -scope base   

Find the disk signature of a disk through diskpart
echo select disk 0 > %temp%\diskpart.txt & echo detail disk >> %temp%\diskpart.txt & diskpart /s %temp%\diskpart.txt | find /i "Disk ID:"

Search a dnscmd export for duplicate IP address references
for /f "tokens=1,5" %i in (DNSExport.txt) do @if "%j" NEQ "" @for /f "tokens=1" %m in ('"findstr /i "%j$" DNSExport.txt find /i /c "%j""') do @if %m GTR 1 @echo %i,%j,%m

Search and report duplicate IPs from a dnscmd export
for /f "tokens=1,4" %i in (DNSExport.txt) do @if "%j" NEQ "" @for /f "tokens=1" %m in ('"findstr /i "%j$" DNSExport.txt find /i /c "%j""') do @if %m GTR 1 (@echo %j,%m: & findstr /i "%j$" DNSExport.txt & echo.)

Dump the dfsr config from the active directory
dfsrdiag dumpadcfg

Remove Outlook 2003 prevention of PST usage
reg add HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Office\11.0\Outlook /v DisablePST /t reg_dword /d 0x0 

Set the command prompt to include the time of the last command
prompt $t $p$g

Create a zero byte file
echo file 2>zero.txt

Given a list of files, echo those that are zero bytes in size
for %i in (%source%\*) do @if %~zi == 0 @echo %i

From a dnscmd export, find duplicate IP addresses
echo. > DuplicateIPs.txt & (for /f "tokens=1,4" %i in (DNSRecords.txt) do @if "%j" NEQ "" @find /i "%j" DuplicateIPs.txt >nul & if errorlevel 1 for /f "tokens=1" %m in ('"findstr /i "%j$" DNSRecords.txt | find /i /c "%j""') do @if %m GTR 1 (@echo %j,%m: & findstr /i "%j$" DNSRecords.txt & echo.) >> DuplicateIPs.txt) & type DuplicateIPs.txt

Start xperf performance tracing using the 'Diag' group
xperf -start -on Diag -f %temp%\tracing.etl 

Open an xperf trace, exporting context switching for threads and processes
xperf -i %temp%\tracing.etl -a cswitch -thread -process

Check whether VMware VMFS partitions are block aligned to 128
/sbin/fdisk -lu

Query a Virtual Centre/VC 2.5 database for Consolidation performance stats
sqlcmd -S server -d virtualcenter -W -s "," -Q "Select Top 6 ip_address as 'IP', cpu_mhz_avg/1000 as 'CPU', mem_mb_avg/1000 as 'RAM' , disk_percent_avg/1000 as 'Disk' from vpx_csl_system_perf vPERF inner join vpx_csl_system_ip_address vIP on vPERF.System_ID = vIP.system_ID order by sample_time desc"

Identify Virtual Machines that are currently powered on
/usr/sbin/vcbVmName -h %server% -u username -s powerstate:on

Find OCS 2007 classes/attributes in AD
dsquery * "CN=Schema,CN=Configuration,DC=forestRoot" -filter "(&((cn=*rtc*)(|(objectCategory=classSchema)(objectCategory=attributeSchema))))"

Find OCS 2007 server from DNS service records
nslookup -type=srv _SipInternalTLS._tcp.{FQDN}

Find OCS 2007 Pools published in the current directory
dsquery * -filter "(objectClass=msRTCSIP-Pools)"

Find OCS 2007 SCPs from the local domain
dsquery * "CN=Pools,CN=RTC Service,CN=Microsoft,CN=System,DC=domainRoot" -attr *

Export config from OCS 2007 from a remote server
lcscmd /config /action:export /level:machine /configfile:config.xml /fqdn:%server%

IIS Authentication and Access Control Diagnostics
authdiag.exe

Find the number of VMs per datastore from the VC database
sqlcmd -S server -d virtualcenter -W -s "," -Q "select DS.name, Count(VMDS.VM_ID) as 'VMs' from vpxv_vm_datastore VMDS inner join vpx_datastore DS on VMDS.DS_ID = DS.ID group by DS.name" 

Find detail on the VMs per datastore from the VC database
sqlcmd -S server -d virtualcenter -W -s "," -Q "select DS.name, VMS.Name from vpxv_vm_datastore VMDS inner join vpx_datastore DS on VMDS.DS_ID = DS.ID inner join vpxv_vms VMS on VMDS.VM_ID = VMS.VMID order by DS.Name"

Unattended install of IIS (assuming INF created with relevant [components])
Sysocmgr.exe /i:%windir%\inf\sysoc.inf /u:%iisComponents%.inf

Find the Exchange 2003 organization from AD
dsquery * forestroot -filter "(&(objectCategory=msExchOrganizationContainer))"

Mount a virtual floppy
vfd install & vfd start & vfd open

Send an SMTP mail using blat
blat -f smtprelay@relay.local -to user@domain.com -subject Test -body "Test body" -server smtprelay

Create MAPI profiles with an Exchange connection on a server without Outlook
profman2.exe

Find mailboxes that are excluded from Recipient Update Policies
dsquery * -filter "(&(objectClass=User)(objectCategory=Person)(msExchPoliciesExcluded=*))" -attr cn msExchPoliciesExcluded | find /i "{26491CFC-9E50-4857-861B-0CB8DF22B5D7}"

Export a connector space from MIIS/IIFP to XML
csexport %maName% maExport.xml

IIFP permissions, write proxyAddresses to user objects, inherited to subobjects
dsacls "OU=%targetOU%,%domainRoot%" /I:S /G %DOMAIN%\%GROUP%:WP;proxyAddresses;user

IIFP permissions, create and delete contact objects, inherited to subobjects
dsacls "OU=%targetOU%,%domainRoot%" /I:S /G %DOMAIN%\%GROUP%:CCDC;contact

IIFP permissions, read/write all properties, inherited to subobjects
dsacls "OU=%targetOU%,%domainRoot%" /I:S /G %DOMAIN%\%GROUP%:RPWP;;contact

Find extended rights in the directory that apply to schema classes
dsquery * "CN=Extended-Rights,CN=Configuration,dc=forestRoot" -attr displayName CN

Trigger the SD propagator adminsdholder process in a domain
admod -rootdse "FixUpInheritance::1"

Set interrupt processor affinity for PnP drivers
intfiltr.exe

Set interrupt processor affinity for processes persistent across reboots
imagecfg.exe -a 0xF calc.exe (the mask to use the first four logical processors)

Start an executable with the specified processor affinity
start /affinity f calc.exe (the mask to use the first four logical processors)

Modify a server to use only one processor
boot.ini, add /onecpu switch

Query DC/DNS servers and find unconditional non-ds forwarders
for /f %i in ('dsquery server -domain %userdnsdomain% -o rdn') do @for /f "tokens=1,3" %m in ('"dnscmd %i /info > DNS_%i.txt & tail -5 DNS_%i.txt | find /i "addr[" | find /i "addr""') do @echo %~ni,%m,%n

Create an Active Directory integrated DNS conditional forwarder (5.2.3790.0)
dnscmd /ZoneAdd %targetDomain% /DsForwarder %targetDomainNSIP%

Find DNS forwarder zones
dnscmd %server% /enumzones /forwarder

Find DNS forwarder targets
for /f %i in ('"dnscmd %server% /enumzones /forwarder | find /i "forwarder""') do dnscmd %server% /zoneinfo %i | find /i "master"

Find AdminSDHolder groups with GROUP_TYPE_SECURITY_ENABLED
dsquery * domainroot -filter "(&(objectCategory=Group)(objectClass=Group)(groupType:1.2.840.113556.1.4.803:=2147483648))"

Query an MIIS/IIFP database to find the management agent AD configuration
select ma_name, private_configuration_xml from mms_management_agent

Check the MIIS/IIFP GALSync.xml file to find the management agent AD config
Extensions\GALSync.xml

Query an MIIS/IIFP database to find the management agent AD containers to sync
select filter_xml from mms_partition MMSP inner join mms_management_agent MMSA on MMSP.ma_id = MMSA.ma_id where ma_name = 'MA-NAME' and partition_name = 'DC=domainRoot'

Set the IP address of a machine using netsh
netsh interface ip set address name="Local Area Connection" source=static addr=192.168.0.10 mask=255.255.255.0 gateway=192.168.0.1 1

Set local DNS client primary using netsh
netsh interface ip add dns name="Local Area Connection" addr=192.168.0.10 index=1

Set local DNS client secondary using netsh
netsh interface ip add dns name="Local Area Connection" addr=192.168.0.11 index=2

Set local WINS client primary using netsh
netsh interface ip add wins name="Local Area Connection" addr="192.168.0.10" index=1

Set local WINS client secondary using netsh
netsh interface ip add wins name="Local Area Connection" addr="192.168.0.11" index=2

Modify service DACLs to allow service start stop (assumes query already exists)
subinacl /service schedule /grant=builtin\users=TO

User cmdkey to add a stored credential when connecting to a remote server
cmdkey /add:remote.domain.com /user:domain\user /pass:*

Find the holders of the specified NT right / privilege
showpriv SeProfileSingleProcessPrivilege

Query the privileges the current user holds
whoami /priv

Delete the policy restriction to run adsiedit.msc
reg delete "HKCU\Software\Policies\Microsoft\MMC\{1C5DACFA-16BA-11D2-81D0-0000F87A7AA3}"

Stop and then restart the ESX software iSCSI initiator
/usr/sbin/esxcfg-swiscsi -d | /usr/sbin/esxcfg-swiscsi -e

Reset a computer account secure channel
nltest /sc_reset:%domain%[\%dc%]

Reset the password for a computer account
nltest /sc_change_pwd:%domain%

CSV directory export of one or more subcontainers of a container
for /f %i in ('"dsquery ou OU=People,DC=domainRoot -scope onelevel -o rdn"') do csvde -f UserExport-%~i.csv -l givenName,sn,displayname,mail,targetAddress,proxyAddresses,mailnickname -d "OU=%~i,OU=People,DC=domainRoot" -r "(&(objectClass=Contact)(objectCategory=Person))"

Query VMFS volume information from the service console
/usr/sbin/vmkfstools -P /vmfs/volumes/%GUID%

Change the volume label of a disk
label %drive%: %newlabel%

Find VMware CDP info from the service console
esxcfg-info | grep -C 18 '\==+CDP Summary'

Add a non expiring enabled user account to the Active Directory
dsadd user "CN=user,OU=Users,DC=test,DC=com" -pwd "password" -pwdneverExpires yes -disabled no -desc "Description"

Clear local DNS client settings using netsh
netsh interface ip delete dns name="Local Area Connection" addr=ALL

From an ESX service console, scan for updates from a depot for UpdateManager
/usr/sbin/esxupdate --HA --flushcache -d http://esx01/vci/hostupdates/hostupdate/esx/esx-3.5.0 scan

Check the VI35 Legato AAM HA agent
cat /var/log/vmware/aam/aam_config_util_addnode.log

VMware VI35 HA Legato AAM, list the cluster manager
/opt/vmware/aam/bin/ftcli -domain vmware -timeout 60 -cmd "listrules"

VMware VI35 HA Legato AAM, list the cluster nodes
/opt/vmware/aam/bin/ftcli -domain vmware -connect esx01 -port 8042 -timeout 60 -cmd "listnodes"

VMware ESX VI35 List the software iSCSI targets
/usr/sbin/vmkiscsi-tool -L -l vmhba32

Mount a local volume inside a local folder
mountvol c:\temp\mount1 \\?\Volume{f856ff87-70ae-11dc-8b8d-806d6172696f}\

Remove a mount point
mountvol C:\temp\mount1\ /d

List junctions or mount points
junction -s c:\temp

Find the boot device for an ESX installation
esxcfg-info -s | grep -A10 "Diagnostic Partition"

Find the boot device for an ESX installation
esxcfg-info -s | egrep -A4 "Parallel SCSI Interface|Block SCSI Interface"

Use vSphere RCLI to list an ESXi host filesystem
vifs.pl --server %server% --username %username% --password %password% -D /host

Use vSphere RCLI to backup an ESXi host (esxcfg-cfgbackup.pl)
vicfg-cfgbackup.pl --server %server% --username %username% --password %password% -s server.tgz

ESX VI35 list the virtual machines and their disks for performance analysis
/usr/lib/vmware/bin/vscsiStats -l

ESX VI35 gather disk statistics and display the latency histogram in CSV
/usr/lib/vmware/bin/vscsiStats -s; /usr/lib/vmware/bin/vscsiStats -x; /usr/lib/vmware/bin/vscsiStats -p latency -c;

Get Windows Remote Management config on the local machine
winrm get winrm/config

Windows Remote Management quick configuration to create a listener
winrm quickconfig

Test Windows Remote Management listener on the local host
winrm id

Create a Windows Remote Management https listener on the local host
winrm quickconfig -transport:https

Create a self-signed certificate
makecert" -r -pe -n -r 30/12/2039 -eku 1.3.6.1.5.5.7.3.1 -ss my-sr localMachine -sky Exchange -sp "Microsoft RSA SChannel Cryptographic Provider" -sy 12 c:\temp\test.cer

Query the SNTP servers a comptuer is using for time synchronisation
net time \\server /querysntp   

Set the SNTP servers used for w32time synchronistaion
net time \\server /setsntp:"192.168.0.10 192.168.0.11"

Convert time from 100 nanosecond intervals since epoch 01/01/1601
w32tm /ntte 127076450620627215

Install the w32time Windows Time service
w32tm /register

Enable NTP Server for a w32time service
reg add \\server\HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\W32Time\TimeProviders\NtpServer /v Enabled /t reg_dword /d 0x1 /f

Find the error description given a win32 error number
net helpmsg 2

Delete the policy value controlling whether recently run programs are recorded
reg delete HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Policies\Explorer /v NoRecentDocsHistory

Set registry permissions (subinacl 5.2.3790.1180 or later)
subinacl /keyreg "\\server\HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows NT\CurrentVersion" /grant=domain\group=F

Set service DACLs (Q : Query SC, S: status, I: interrogate, T: Start, O: Stop)
subinacl /service \\server\schedule /grant=domain\group=TOQSI

Rename a LAN interface name (ncpa.cpl)
netsh interface set int name="Local Area Connection 2" newname="Local Area Connection"

Find all network devices
devcon findall =net

Find all network adapters for the Net class
devcon listclass net

Remove an old VMware VI3 PCNET Flexible/VLance/VMXNET adapter instance (@)
devcon remove "@PCI\VEN_1022&DEV_2000&SUBSYS_20001022&REV_10\3&61AAA01&0&88"

Find where a file is in the path
for %i in (calc.exe) do echo %~$PATH:i

Find the physical disk sector size
wmic path win32_diskdrive get BytesPerSector

Find the current amount of memory used by the file system virtual cache
wmic path Win32_PerfFormattedData_PerfOS_Memory get SystemCacheResidentBytes

Use robocopy in backup mode to take a copy of folder-level permissions
Robocopy \\server\source c:\temp\copy zxcvsadfqwer /E /B /COPYALL /R:1 /W:1

Find remote shares and paths using WMI
wmic /node:%server% path win32_share get Name,Path,Description

Find total memory, free memory and used paging file
wmic /node:%server% path Win32_OperatingSystem Get FreePhysicalMemory,FreeSpaceInPagingFiles,TotalVirtualMemorySize,TotalVisibleMemorySize

Search a remote computer's registry for a string
regfind -m \\%server% -y -b -n search_string

Set a computer to use a specified number of available processors
modify boot.ini, use the /NUMPROC=x switch or /ONECPU switch

Check the Exchange ESE buffer cache size
dsquery * "CN=InformationStore,CN=exchserver01,CN=Servers,CN=AdminGroup01,CN=Administrative Groups,CN=organisation,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=domainRoot" -attr msExchESEParamCacheSizeMax -scope base

Query the start of authority record for a DNS zone
dnscmd %server% /enumrecords %fqdn% @ /type SOA

VMware PowerCLI - Reconfigure cluster options

2009-08-11T22:24:00.000+10:00

The following script contains a simple example of how to modify the configuration of a cluster – in this case to add two advanced options to the HA settings. This uses the new ClusterConfigSpecEx object, which can be used with the more generic function to reconfigure a resource – ReconfigureComputeResource().

The options set in this example are the das.allowNetworkX options, configuring HA to set specific management networks/service consoles that are used for HA communication and heartbeats.

Consider the following scenario for a reason why you might want to use these options:

Software iSCSI with ESX 3.x – using the service console for authentication, meaning you need a service console on your iSCSI network (or routes between your corporate network and iSCSI). The standard result is that HA could be used across this interface.
You are progressing with an upgrade to ESXi 4.0, still using software iSCSI, which is improved to remove the need for a management interface on your iSCSI network
You put an ESX 3.5 and ESXi 4.0 host in the same cluster, HA no longer works because your iSCSI service console on ESX 3.5 can’t talk to your ESXi 4.0 host.
Adding the two options above (with names changed appropriately) would force HA to use the service console on ESX 3.5 and the equivalent management network on ESXi 4.0, ignoring the iSCSI service console on 3.5 - resolving HA issues.



# http://www.vmware.com/support/developer/vc-sdk/visdk400pubs/ReferenceGuide/vim.ComputeResource.html#reconfigureEx
# http://www.vmware.com/support/developer/vc-sdk/visdk400pubs/ReferenceGuide/vim.cluster.DasConfigInfo.html#field_detail

param (
    $vcServerName = ""
)

$viServer = Connect-VIServer -server $vcServerName

$optionValue = New-Object Vmware.Vim.OptionValue
$optionValue.Key = "das.allowNetwork0"
$optionValue.Value = "Service Console"

$optionValue2 = New-Object Vmware.Vim.OptionValue
$optionValue2.Key = "das.allowNetwork1"
$optionValue2.Value = "Management Network"

[Vmware.Vim.OptionValue[]]$optionValues = $optionvalue, $optionValue2  # Create the array of the two new option values

$cluster = get-cluster -Name 'Cluster01'     # Get the cluster
$clusterview = get-view $cluster.Id      # Get the SDK object from the PowerCli object MO

$spec = New-Object Vmware.Vim.ClusterConfigSpecEx
$spec.dasConfig = New-Object Vmware.Vim.ClusterDasConfigInfo   # New VMware HA config

$spec.dasConfig.option = $optionValues      # Add the array of optionValues
$clusterview.ReconfigureComputeResource($spec, $true)    # Modify the configuration. When configuring clusters, can be a ClusterConfigSpecEx object


# Check the settings were saved

$cluster = get-cluster -Name 'Cluster01'     # Get the cluster
$clusterview = get-view $cluster.Id      # Get the SDK object from the PowerCli object MO

$clusterview.Configuration.dasConfig.Option | format-list   # Retrieve the advanced options

VMware PowerCLI - Backup ESXi 4.0 firmware

2009-08-11T22:06:00.000+10:00

The following script contains a simple function to use the VMware vSphere PowerCLI to connect to a vCenter and/or ESXi 4.0 host and backup the 'firmware' - the configuration of the ESX host. I don’t think this functionality is exposed directly through the PowerCLI, so a call to the very useful Get-View function is used to get the VI SDK object to call the appropriate method.

From what I can tell all this command does is generate a config dump on the server and return the HTTP URL to access the file download, which I’m then using the .Net web client object to download the file and store as the filename returned with a unique date suffix.

This certainly isn’t original, all I did was look at the vicfg-backup.pl perl RCLI script and (badly) translate to PowerCLI from there.


#
# Description:
#  Backup the firmware configuration on an ESXi 4.0 host
#
# Limitations:
#  -
#
# Assumptions, this script works on the assumption that:
#  The caller provides credentials with permissions to connect to the specified host
#
# Arguments:
#  esxServer, the ESX host to connect to, eg. esx01
#  vcServerName, The vCenter Server to connect to, eg vc01
#  outputDir, The directory to write the backup file to, defaults to %temp%
#  username, The username to use when connecting to the vCenter server (if specified), defaults to %username%
#  password, The password to use for the connection to vCenter, secure string prompt by default
#
# Usage:
#  PowerShell . .\BackupFirmware.ps1 -esxServer 'esx01'
#  PowerShell . .\BackupFirmware.ps1 -esxServer 'esx01' -vcServer vc01 -u domain\username
#
# References:
#  http://www.vmware.com/support/developer/vc-sdk/visdk400pubs/ReferenceGuide/vim.host.FirmwareSystem.html
#
# Changes:
#  04/07/2009, Wayne Martin, initial version

#$ErrorActionPreference = "Continue"


param (
    $esxServer = "",
    $vcServerName = "",

    $outputDir = $env:Temp,

    $username = $env:username,
    $password = ""
)

if ($password -eq "" -and !($pass)) {
    write-output "No password specified from the command-line"
    $pass = Read-Host "Password?" -assecurestring
    $credential = new-object System.Management.Automation.PSCredential($username,$pass)    
}

if ($esxServer -eq "" -OR ($vcServerName -eq "" -and $esxServer -eq "")) {
    Write-Output "Please specify either a standalone host, or a host and a cluster"
    Exit 2
}

$viServer = $null


function BackupConfiguration([string] $vcServerName, [string] $esxServer, [string] $outputDir){
    $hostSystem = get-view -ViewType HostSystem -Filter @{"Name" = $esxServer}    # Find the .Net view of the specified host

    $hostConfigManager = $hostSystem.get_ConfigManager()      # Get the config manager
    $hostfirmwareSystem = $hostConfigManager.get_firmwareSystem()     # Find the MOR of the host firmware system

    $hostfirmware = Get-View $hostfirmwareSystem        # Get the VMware.Vim.HostFirmwareSystem object from the MOR

    $backupDownload = $hostfirmware.BackupFirmwareConfiguration()     # Call the backup method to generate the config bundle

    $backupDownload = $backupDownload.Replace("*", $esxServer)      # Replace '*' with the server name

    Write-Output "Backup saved to $backupDownload on the ESX host"
    $fileName = $backupDownload.SubString($backupDownload.LastIndexOf("/")+1)    # Extract the filename to reuse
    $fileType = $fileName.SubString($fileName.LastIndexOf("."))      # Find the extension (.tgz in this case)

    $Now = [DateTime]::Now.ToString("yyyyMMddTHHmmss")          # Unique identifier for the filename
    $file = $fileName.SubString(0, $fileName.Length - $fileType.Length)     # File name without extension
    $outputFile = $outputDir + "\" + $file + "_" + $Now + $fileType     # Construct the full filename path\bundle_date.tgz

    $wc = new-object system.net.WebClient        # use the .Net web client
    $wc.DownloadFile($backupDownload, $outputFile)       # Download the file from the URL returned

    if (test-path -path $outputFile) {         # Does the output file exist?
        Write-Output "$outputFile downloaded"
    } else {
        Write-Output "Error: $outputFile was not downloaded from $backupDownload"
    }

}

if ($vcServerName -ne "") {
    $viServer = Connect-VIServer -server $vcServerName -Credential $credential
} elseif ($esxServer -ne "") {
    $esxServer = Connect-VIServer -server $esxServer       # connect to VC
}


$results = ""
$results = BackupConfiguration $vcServerName $esxServer $outputDir

$results

if ($vcServerName -ne "") {
    Disconnect-VIServer -confirm:$false
}

VMware vSphere PowerCLI commands

2009-08-06T19:56:00.000+10:00

The commands below are PowerCLI commands used to automate VMware ESX and VirtualCenter at the command prompt. Most of these commands were built on 3.5 with vSphere PowerCLI, and the majority have been tested against ESXi 4.0 and vCenter 4.0 infrastructure.

Each command-line can be copied and pasted at a PowerCLI command prompt, and most commands assume you already have a connection to the target, be it vCenter or ESX.

Note that most of these commands use OBN - Object By Name - instead of using the get* command to get the object. This is supported with all but a few of the commands I've come across.



List the vSphere PowerCLI commands
Get-VICommand

Connect to a ESX or VirtualCenter instance
connect-viserver -server %server%

List the currently available datastores
Get-Datastore | sort

List the currently available datastores filtered and sorted
Get-Datastore | where {$_.Name -like '*pr*'} | sort

Find the VMs attached to one or more datastores
foreach ($prodDatastore in $prodDatastores) { write-output $prodDatastore.Name; get-vm -datastore $proddatastore; write-output ''}

Get a Virtual Machine
$vm = get-vm -name '%vm%'

Get the virtual harddisk for the specified VMs
Get-HardDisk -vm $vm

Move a virtual machine to another container
Move-VM -Destination $prodApps -VM $vm

Update the VM description for a list of CSV entries
foreach ($virtualServer in $virtualservers) {$arr = $virtualServer.split(","); $desc = $arr[1]; $vmName = $arr[0]; write-output $vmName; $desc; $vm = get-vm -name $vmName; Set-VM -VM $vm -description $desc}

Query for a list of VMs and output in ANSI format
get-vm | sort-object | format-table -property Name | out-file -encoding ASCII -filepath c:\temp\vms_20090625.txt

Find VMware machine performance statistics
 get-stat -entity $vm -disk -start 01/01/2009 -finish ([DateTime]::Now.ToString("dd/MM/yyyy"))

For a group of VMs, report performance statistics and save to file
foreach ($vm in $devVMs) {get-stat -entity $vm -disk -start 01/01/2009 -finish ([DateTime]::Now.ToString("dd/MM/yyyy")) | out-file -filepath ("c:\temp\" + $vm.Name + "DiskPerformance.txt")}

Find VM datastore disk usage
$devVMs = get-vm -name '*dv*'; foreach ($vm in $devvms) {$vm.harddisks}

Find VM datastore disk usage
$testVMs = Get-VM -Location (get-folder -name "Test") ;foreach ($vm in $testVMs) {$vm.harddisks | format-table -hideTableHeaders -wrap -autosize | findstr /i /c:per}

Find SCSI devices attached to an ESX server
get-scsilun -vmhost (Get-VMHost -Location "cluster")[0]

Rescan HBAs on an ESX server
get-VMHostStorage -VMHost (Get-VMHost -Location "cluster")[0] -RescanAllHba

Storage vMotion a virtual machine to a new datastore
Move-VM -vm "vmName" -datastore "NewDatastore"

Storage vMotion a group of machines from a CSV input file
$servers = get-content -path inputfile.txt; foreach ($server in $servers) {move-vm -vm $server.split(",")[0] -datastore $server.split(",")[1]}

Remove a snapshot and child snapshots, reporting how long the operation took
measure-command -expression {remove-snapshot -snapshot $snapshots[0] -removechildren}

Find datastore space, usage and number of VMs per datastore
$datastores = get-datastore | sort-object; write-output "Name,Size,Used,Free,% Used,#VMs"; foreach ($datastore in $datastores) { write-output ($datastore.Name + "," + [math]::round($datastore.CapacityMB/1024) + "," + [math]::round(($datastore.CapacityMB/1024)-($datastore.FreeSpaceMB/1024)) + "," + [math]::round($datastore.FreeSpaceMB/1024) + "," + [math]::round(((($datastore.CapacityMB/1024) - ($datastore.FreeSpaceMB/1024)) / ($datastore.CapacityMB/1024)) * 100) + "," + (get-vm -datastore $datastore).count)}

From a set of VMs, find which have snapshots
foreach ($testvm in $testvms) {if (get-snapshot -vm $testvm){write-output $testvm.Name}}

Find the size of the first hard disk in each VM
foreach ($vm in $vms) {$vm.harddisks[0] | format-table -hideTableHeaders -wrap -autosize | findstr /i /c:per }

Find disk information for VMs in the specified datastore
 $VMs = Get-VM ;foreach ($vm in $VMs) {$vm.harddisks | where {$_.FileName -like '*clusterpr*'} | format-table -hideTableHeaders -wrap -autosize | findstr /i /c:per}

Find VMs in the specified datastore
$VMs = Get-VM | where {$_.harddisks[0].FileName -like '*clusterpr*'}

Get VM guest information, including virtual OS
get-vm | get-vmguest | format-table -wrap -autosize

Find virtual machines and their description/notes
$vms = get-vm ; $vms | format-table -wrap -autosize -property Name,Description

Create an associative array containing VM names and descriptions
$vmdesc = @{}; foreach ($vm in $vms) {$vmdesc.add($vm.Name, $vm.Description)}

Migrate a virtual machine to another host in a VMware ESX cluster
move-vm -vm %vmName% -destination %hostname%

Find the host a VM is currently located on
get-vmhost -vm %vnName%

Add a new harddisk to a virtual machine
New-HardDisk -vm %vmName% -CapacityKB 20971520

Retrieve details on the resource pools from the currently connected datacenter
Get-ResourcePool | format-table -wrap -autosize -property Name,Id,CpuExpandableReservation,CpuLimitMHz,CpuReservationMHz,CpuSharesLevel,CustomFields,MemExpandableReservation,MemLimitMB,MemReservationMB,MemSharesLevel,Name,NumCpuShares,NumMemShares

Find virtual machines and if they have a CD-ROM
get-vm | format-table -wrap -autosize -property Name,CDDrives

Find the last 100 events that aren't alarm related
$events = Get-VIEvent -MaxSamples 100 | where {$_.fullFormattedMessage -notmatch "Alarm*"}

Find all events for machine deployments from templates
$events = Get-VIEvent | where {$_.fullFormattedMessage -match "Deploying (.*) on host (.*) in (.*) from template (.*)"}

Create a resource pool with high CPU and memory shares
New-ResourcePool -location (get-cluster -name 'cluster') -Name ResPool1 -CpuSharesLevel [VMware.VimAutomation.Types.SharesLevel]::High -MemSharesLevel [VMware.VimAutomation.Types.SharesLevel]::High

Create a folder from the root of the tree
New-Folder -Name Workstations -location (get-folder -name 'vm')

Move one or more VMs to a resource pool (or other destination)
$vms = get-vm -name vmNames*; move-vm -vm $vms -destination (Get-ResourcePool -name 'ResPool1')

Get an OS customization specification, and list the properties in wide format
Get-OSCustomizationSpec -name "SpecName" | format-list

Take a snapshot of a virtual machine
New-Snapshot -Name "Snapshot 01" -description "Snapshot description" -vm vmName -Quiesce:$true

Convert a virtual machine to a template
$vmView = get-vm -name vm01 | Get-View; $vmView.MarkAsTemplate()

Find Datastore usage (custom written function)
get-datastoreusage

Get an ESX log bundle using PowerCLI
Get-Log -VMHost esxhostname -Bundle -DestinationPath c:\temp

Query for snapshots
Get-VM | Get-Snapshot | export-csv -path c:\temp\VMsnapshots.csv

Query for snapshot information
Get-VM | Get-Snapshot | foreach-object {$out=  $_.VM.Name + "," + $_.Name + "," + $_.Description + "," + $_.PowerState; $out}

Resetting Computer Account Passwords

2009-08-06T19:38:00.002+10:00

I was trying to tell from a workstation point of view when the computer account password was last set. I'm sure this information is stored locally somewhere, but in the end it was easier to query the AD and find when the password for the computer account was last set.

This is used for Virtual Machine templates - we use nltest to reset the computer account password, such that we can maintain a single template image and turn it on periodically for updates without having to rejoin to the domain because of mismatched computer accounts.

Forcefully reset the computer account password:



nltest /SC_CHANGE_PWD:%domain%

Query the workstation in the domain and find when the password was last set - returns the number of 100 nanosecond intervals since 01/01/1601.



dsquery computer -name ws01
dsquery * "CN=ws01,OU=Computers,DC=domain,DC=com" -attr pwdlastset
pwdlastset
128934012123005000

Use PowerShell to convert the number to a human readable date format:



powershell [datetime]::FromFileTime(128934012123005000)

Thursday, 30 July 2009 2:20:12 PM

Use w32tm to convert the number to a human readable date format:



w32tm /ntte 128934012123005000

149229 04:20:12.3005000 - 30/07/2009 2:20:12 PM

Use VBScript to convert the number to a human readable date format:



cscript ConvertFileTime.vbs 128934012123005000

30/07/2009 2:20:12 PM


' ConvertFileTime.vbs
' VBScript doesn't support 64-bit integers, so it can't handle the number of 100 nanosecond intervals since 01/01/1601
' http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnclinic/html/scripting09102002.asp

' Either use ADSI provider and the IADs/IADsLargeInteger object
' LargeIntValue = objLargeInt.HighPart * 2^32 + objLargeInt.LowPart

' http://msdn.microsoft.com/library/default.asp?url=/library/en-us/adsi/adsi/iadslargeinteger.asp'
' Or WMI, which handles the conversion between 64-bit datetime structure / UTC / and VB var datetime

If Wscript.Arguments.UnNamed.Count > 0 Then 
        strDateTime = Wscript.Arguments.UnNamed(0)
        Set objDateTime = CreateObject("WbemScripting.SWbemDateTime")
        If IsDate(strDateTime) Then
                Call objDateTime.SetVarDate(strDateTime, False)
                wscript.echo objDateTime.GetFileTime
        Else
                Call objDateTime.SetFileTime(strDateTime, False)
                wscript.echo objDateTime.GetVarDate
        End If
        intReturn = 0
Else
        WScript.Echo "Specify a filetime or a date to convert, eg 127076450620627215, or ""11/04/2006 11:17:10 AM"""
        intReturn = 2
End If
WScript.Quit(intReturn)

PowerCli Relocate VM storage with VI35 / VI4

2009-07-27T19:00:00.000+10:00

This post provides a script to help with storage vMotion – migrating a VMware virtual machine from one VMFS datastore to another with no downtime. It’s a great tool, but in VI35 at least it’s not exposed through the GUI or VC scheduled tasks, so I’ve written a script to perform a few checks and allow for very simplistic 'scheduling' (delay).

I’ve only tested this with VI3.5 and PowerCLI 1.0, but I have no reason to think this wouldn’t with with vSphere vCenter and VI 4.0 hosts.

There are a few caveats with storage vMotion (in VC 2.5/ESX 3.5) at least, you can’t:
- Move the storage of a VM that has a snapshot if the VM is powered on
- Move the storage of a VM that has a snapshot (in any power state) if the VM has disks in a different location than the config file.

Based on these caveats, if instructed the script will suspend a VM with snapshots in order to move the storage, then power the VM back on. Use this with caution, as this may cause an outage of your VMs.


#
# Description:
#   Relocate a virtual machine from one datastore to another.
#
# Limitations:
#  -
#
# Assumptions, this script works on the assumption that:
#   The caller provides credentials with permissions to perform the operations
#
# Arguments:
#  vmName, lowecase short name for the virtual machine, eg pibutepr03
#  dataStoreName, the new datastore to migrate the VM to
#  suspend, Whether or not to suspend a VM and move if the VM has snapshots (which prevent live storage vMotion vi VI35)
#  username, The username to connect with, default to the current username environment variable
#  password, The Password to use for the connection, not specifying a password will result in a prompt to enter a secure string
#  delay, The optional number of seconds to delay before starting the operation
#
#
# Usage:
#   PowerShell . .\RelocateVM.ps1 -vmName "vm01" -datastore 'ds02'
#   PowerShell . .\RelocateVM.ps1 -vmName "vm01" -datastore 'ds02' -suspend yes -username domain\user
#
# Changes:
#  07/04/2009, Wayne Martin, Initial version
#

param (

    $vmName = "",
    $dataStoreName = "",
    $suspendIfRequired = $false,
    $username = $env:username,
    $password = "",
    $delay = 0
)


$ErrorActionPreference = "Continue"

 $invalidArgs = $false
if ($dataStoreName -eq "") { write-output "Please specify the datastore target for the VM, eg. ds02"; $invalidArgs = $true}
if ($vmName -eq "") { write-output "Please specify the virtual machine to move, eg vm01"; $invalidArgs = $true}
if ($account -eq "") { write-output "Please specify a user account to connect with, eg domain\user"; $invalidArgs = $true}

if ($invalidArgs) { write-output "Invalid Arguments, terminating"; exit}


Write-Output "Moving VM '$vmName' to the '$dataStoreName' datastore"

if ($delay -gt 0) {
    $hours = $delay / 60 /60
    Write-Output "Delaying $delay seconds before beginning ($hours hours)"
    Sleep -seconds $delay
}


if ($suspendIfRequired) {
    Write-Output "The virtual machine will be suspended if snapshots are preventing storage vMotion"
} else {
    Write-Output "If the virtual machine has snapshots and is powered on the storage vMotion will not work"
}

if ($password -eq "" -and !($pass)) {
    write-output "No password specified from the command-line"
    $pass = Read-Host "Password?" -assecurestring
}
$credential = new-object System.Management.Automation.PSCredential($username,$pass)    


$viServer = $null
$suspend = $false
$poweredOn = ""
$snapshot = $null
$hasSnapshot = $null

$viServer = Connect-VIServer -server $vcServerName -Credential $credential


if ($viServer) {
    Write-Output (Get-Date -format "dd/MM/yyyy HH:mm:ss")
    write-output ("Connected to server " + $viServer.Name + " on port " + $viServer.Port)

    $vm = get-vm -name $vmName
    if ($vm -and $vm -isnot [object[]]) {
        Write-Output ("Found " + $vm.Name)

        $hardDisks = $vm.hardDisks
        $vmSize = 0
        $vmSizeMB = 0
        foreach ($harddisk in $vm.hardDisks) {
            $vmSize += $hardDisk.CapacityKB
        }
        $vmSizeMB = $vmSize /1024

        $datastore = get-datastore -name $dataStoreName
        if ($datastore) {
            $freeSpace = $datastore.FreeSpaceMB
        
            if ($freeSpace -gt $vmSizeMB)
            {
                Write-Output "The datastore $datastoreName has $freeSpace MB available, the VM has disks totalling $vmSizeMB MB"
            
                switch ($vm.PowerState)
                {
                    ([VMware.VimAutomation.Types.PowerState]::PoweredOn)
                    {
                        Write-Output "The virtual machine is currently powered on"
                        $poweredOn = $true
                    }

                    ([VMware.VimAutomation.Types.PowerState]::PoweredOff)
                    {
                        Write-Output "The virtual machine is currently powered off"
                    }

                    ([VMware.VimAutomation.Types.PowerState]::Suspended)
                    {
                        Write-Output "The virtual machine is currently suspended"
                    }
            
                    default
                    {
                        write-output "Virtual machine power state unknown"
                    }
                }

                [object[]]$snapshot = get-snapshot -vm $vm
                if ($snapshot) 
                {
                    $hasSnapshot = $true
                    $numSnapshots = $snapshot.Count
                    Write-Output "$vmName currently has $numSnapshots snapshot(s)"
                    if ($poweredOn) {
                        if ($suspendIfRequired) {
                            Write-Output "$vmName is powered on and has a snapshot, and will be suspended during the move"
                            $suspend = $true
                        } else {
                            Write-Output "Error: $vmName is powered on and has a snapshot, but will not be suspended, process aborted"
                            $suspend = $false
                            exit 2
                        }
                    }
                } else {
                    $suspend = $false
                    $hasSnapshot = False
                    Write-Output "No snapshots currently found for $vmName"

                }

                if ($suspend) {
                    Write-Output "Suspending $vmName"
                    Suspend-VM -vm $vm -confirm:$false
                }

                Write-Output "Moving $vmName to $datastoreName"
                Move-VM -vm $vmName -datastore $datastoreName

                if ($suspend) {
                    Write-Output "Bringing $vmName out of suspension"
                    Start-VM -vm $vm -confirm:$false
                }
                Write-Output "$vmName migrated to $datastoreName"

            } else {
                Write-Output "The datastore only has $freeSpace MB available, but the VM has disks totalling $vmSizeMB MB"
            }
        } else {
            Write-Output "Error: datastore $datastoreName not found"
        }
    } else {
        if ($vm -is [object[]])
        {
            Write-Output "Multiple objects returned for $vmname, please specify a single VM"
        } else {
            write-output "VM Not found - $vmName"
        }
    }
} else {
    write-output "ERROR: VI server not found - $viServer"
}

Write-Output (Get-Date -format "dd/MM/yyyy HH:mm:ss")

Disconnect-VIServer -confirm:$false
exit 0

PowerShell - vSphere VMKernel NIC MTU

2009-07-01T22:51:00.000+10:00

I came across a problem with VI4 (vSphere) on ESXi, where I was unable to create a vmkernel NIC for iSCSI with the vicfg-vmknic.pl script to set a custom MTU of 9000. This post provides a method using the GUI/perl script, and a PowerShell vSphere PowerCLI script to modify an existing VMKernel NIC.

It seems there is a bug in the perl script doesn't allow you to create a VMKernel NIC, however it does work if you create one through the GUI, and then use the script to delete and re-create the virtual NIC.

This was using a standalone ESXi server with a normal vSwitch - it wasn't managed by VC and it wasn't a DVS.

The GUI/PL script method:

Create a VMKernel NIC in an existing vSwitch through the VI Client, with a port group called iSCSI1 in this example.
Using the CLI, run the following commands, the first to delete the existing NIC, and then another to re-create the same NIC, but with an increased MTU:
vicfg-vmknic.pl -d iSCSI1
vicfg-vmknic.pl -a -i x.x.x.x -n x.x.x.x -m 9000 iSCSI1
Run 'vicfg-vmknic.pl –l' to list the VMK NICs, which should show the adjusted MTU

I wasn't particularly happy with this method, so I looked at using the vSphere PowerCLI to set the MTU. For whatever reason, VMware chose not to expose the MTU property, which leaves only the SDK.

Luckily the PowerCLI makes it much easier to use the .Net SDK than previously possible with the Get-View cmdlet, so the script below is a combination of standard PowerCLI with a hook into the vNIC collection and the UpdateVirtualNic method to change a virtual NIC spec.


param (

    $MTU = 9000, 
    $nicName = vmk1, 
    $vcServer = "vcServer", 
    $vmServerName = "esx01"
)

#
#
# Description:
#  Update the MTU of a vmkernel NIC used for iSCSI
#
# Limitations:
#  DVS untested
#
# Assumptions, this script works on the assumption that:
#  The caller provides credentials with permissions to change the specified NIC
#
# Arguments:
#  MTU, The new MTU size, eg. 9000
#  nicName, The vmkernel NIC, eg. vmk1.  Note that this differs from the port group (eg. iSCSI1, with a NIC name of vmk1)
#  vcServer, the VC instance to connect to, eg. vcServer
#  vmServerName, The server name controlled by the VC instance (or direct), eg. esx01
#
# Usage:
#  PowerShell . .\UpdateVMKMTU.ps1 -MTU 9000 -nicName 'vmk1' -vcServer 'vcServer' -vmServerName 'esx01'
#
# References:
#  http://www.vmware.com/support/developer/vc-sdk/visdk400pubs/ReferenceGuide/vim.host.VirtualNic.Specification.html
#
# Changes:
#  26/06/2009, Wayne Martin, initial version

#$ErrorActionPreference = "Continue"


Connect-VIServer -server $vcServer        # connect to VC
$hostSystem = get-view -ViewType HostSystem -Filter @{"Name" = $vmServerName}   # Find the .Net view of the specified host
$hostConfigManager = $hostSystem.get_ConfigManager()      # Get the config manager
$hostNetworkSystem = $hostConfigManager.get_NetworkSystem()     # Find the MOR of the host network system
$netSystem = Get-View $hostNetworkSystem       # Get the object from the reference for the update method later

$hostconfig = $hostSystem.Config        # Get the current host config
$hostNetwork = $hostconfig.Network        # Get the current network host config
$hostvNIC = $hostNetwork.vNic         # Get the virtual NICs

$nicBeingUpdated = $null
foreach ($hostVirtualNIC in $hostvNIC) {       # For each virtual NIC
    if ($hostVirtualNIC.Device -eq $nicName) {        # Is this the device specified?
        $nicBeingUpdated = $hostVirtualNIC       # Yes, copy the object
    }
}

if ($nicBeingUpdated) {          # Was something found?
    $nicSpec = $nicBeingUpdated.Spec        # Yes, get the current spec
    $currentMTU = $nicSpec.MTU         # Get the current MTU from the spec

    if ($currentMTU -ne $MTU) {         # Is the MTU different?
        $nicSpec.set_Mtu($MTU)         # Yes, update the MTU on the copy of the spec of the existing NIC
        $netSystem.UpdateVirtualNic($nicName, $nicSpec)             # Call the update method from the netsystem, to update the NIC with the modified device spec

        write-output "MTU for $nicName updated from $currentMTU to $MTU"
        (get-vmhostnetwork).VirtualNic        # Output updated information
    } else {
        write-output "MTU for $nicName already set to $MTU, no changes made"
    }

} else {

    write-output "NIC $nicName not found, no changes made"

}

Processor affinity on Windows server 2003/2008

2009-06-28T15:16:00.003+10:00

This post describes a few methods of setting the processor affinity mask for a service, process or executable on a Windows Server 2003/2008 machine. There are several reasons to do this; increasingly as software becomes licensed per socket or even per core, but also separating or tuning the performance of applications on multi-core and hyper-threaded processors, i.e. to force usage of shared cache.

Of course this can be done through the task manager GUI (taskmgr.exe), but following the theme of command-line automation and scripting, I’ve found a few other methods that may be more efficient. The options below have different advantages/disadvantages and really depend on what you’re trying to achieve and how the process gets started.

In no particular order:

WSRM

Use Windows System Resource Manager, which among other things can be used for persistent application to processor mappings. This requires 2003 enterprise, 2003 datacenter or server 2008. WSRM is useful when trying to run multiple applications on a single physical box, allowing granular control through a GUI (the configuration of which is persistent across reboots). WSRM also has a command-line interface, and can be used for accounting and to set per-processor utilisation, not just processor affinity.

Note that when I was testing WSRM, it detected calc.exe and matched it against a rule to limit 50% CPU usage, but using factorial of a large number still resulted in 100% CPU. WSRM uses soft caps apparently, only constraining usage when there is contention (I didn’t do any more testing to confirm). There is also a note about WSRM only supporting management of non-OS processes (I’m unsure of whether it would detect calc.exe as such).

PowerShell

Use PowerShell to set the processor affinity for one or more running processes. There’s an example script below, setting the processor mask of calc.exe to the first 4 processors. I like this method because the script is simple, it would be easy to schedule, works on x86 and x64, supports multiple processes of the same name and at least partly because it highlights just how easy administration with PowerShell is.

Note that if you use factorial of a large number with calc.exe (n!) you’ll generate100% CPU which can be useful for testing. The mask below is 0xf = 1111 – a mask allowing use of only the first four processors:

$calcSet = Get-Process -ProcessName "calc"
foreach ($calc in $calcSet) {$calc.ProcessorAffinity=0xF}

Start or psexec

Use a wrapper to launch the process, including either ‘start /affinity’ or ‘psexec -a’ to launch the application, setting affinity as required. Note that ‘start /affinity’ is available on 2003 server, Vista and 2008 server, but not XP.

psexec /a 0,1 calc.exe
start /affinity 1 calc.exe

If you’re trying to control a service, you could use instsrv/srvany to create a service that wraps the start or psexec command around the real service binary. For example, the commands below create another version of the spooler service that will only run on the first processor.

instsrv Test c:\util\srvany.exe
reg add hklm\system\currentcontrolset\services\test\Parameters
reg add hklm\system\currentcontrolset\services\test\Parameters /v Application /t reg_sz /d cmd.exe
reg add hklm\system\currentcontrolset\services\test\Parameters /v AppParameters /t reg_sz /d "/c start /affinity 1 C:\WINDOWS\system32\spoolsv.exe"

ImageCfg

Use imagecfg.exe - a Windows Server 2000 Resource Kit Supplement utility - to modify the binary to set the processor affinity. This actually modifies the .exe – changing the initialisation section of the executable, and will ensure the affinity whenever the executable is run. Note that in my testing, modifying an x64 binary with this utility did not set the affinity correctly, only x86 binaries were successful (running on x64 or x86 OS). If you’re using calc.exe to test this – use a copy in another folder, otherwise windows file protection will replace the updated binary, undoing any change you make.

The mask below is 0xf = 1111 – a mask allowing use of only the first four processors:

imagecfg -a 0x0F c:\temp\calc.exe

ProcAff.exe

Use procaff.exe (a third-party utility) to set the processor affinity of a running process. This could be run as startup scheduled task, setting the mask of any process started as a service (you might need to build in some logic to wait for the processes to start). Note that this utility worked on both x86 and x64 systems while testing, but it’s limited in that if there are multiple processes with the same name, it requires a PID. This is easy enough to script using a ‘for’ loop and pslist to identify the PIDs and then call procaff.

procaff /set 1 calc.exe

Boot.ini and HAL options

Some rather more severe options – reducing the OS to only one CPU:

The /NUMPROC switch in boot.ini on the server. This will set the number of processes Windows will see when booted. I assume this will just apply a mask at startup, but in a multi-CPU multi-core server I haven't tested to confirm the cores are paired or whether the first core from each CPU is presented.
The /ONECPU switch in boot.ini on the server. This will tell Windows to use only one CPU, limiting any application running on the OS. On a multi-core server, the OS would use only one core of one package. I successfully tested this on 2003 enterprise OS.
Downgrade the OS from a multiprocessor HAL to a uniprocessor HAL, removing support for multiple CPUs from the OS. I didn’t actually test this method, but it’s something I used to do on NT4, and I don’t see why it wouldn’t work on 2003/2008.

Plug-and-Play device interrupts

Use Intfilter.exe - allowing you to perform similar functionality to drivers – binding interrupts to a particular processor, useful to control plug-and-play driver-level CPU binding.

References

Procaff
http://www.stefan-kuhr.de/cms/index.php?option=com_content&view=article&id=60&Itemid=77

Boot INI Options Reference
http://technet.microsoft.com/en-us/sysinternals/bb963892.aspx

HAL options after Windows XP or Windows Server 2003 Setup
http://support.microsoft.com/kb/309283

Available Switch Options for Windows NT Boot.ini File
http://support.microsoft.com/kb/170756

How to Manually Add Support for a Second Processor
http://support.microsoft.com/kb/156358

Windows System Resource Manager: Frequently Asked Questions
http://www.microsoft.com/windowsserver2003/techinfo/overview/wsrmfaq.mspx

Windows System Resource Manager
http://www.microsoft.com/downloads/details.aspx?FamilyID=848306EF-F57E-4B3F-984D-50E9BCA44383&displaylang=en

Best Practices for Managing Applications with Process Control
http://technet.microsoft.com/en-us/library/bb742469.aspx

The Context Switch Action (xperf)
http://msdn.microsoft.com/en-us/library/cc305227.aspx

Monitoring Context Switches and Threads
http://technet.microsoft.com/en-us/library/cc938639.aspx

Analyzing Processor Activity
http://technet.microsoft.com/en-us/library/cc958310.aspx

Monitoring Activity on Multiprocessor Systems
http://technet.microsoft.com/en-us/library/cc938649.aspx

Windows Performance Toolkit x86 and x64 v4.1.1
http://download.microsoft.com/download/e/2/7/e2700369-d072-4fdc-a451-c3355eab0613/xperf_x86.msi

http://download.microsoft.com/download/e/2/7/e2700369-d072-4fdc-a451-c3355eab0613/xperf_x64.msi

Windows Performance Toolkit
http://msdn.microsoft.com/en-us/library/cc305187.aspx

Intfilter to manage drivers:
http://support.microsoft.com/default.aspx?scid=KB;en-us;252867

Calling CPUID from PowerShell for Intel VT

2009-06-01T20:19:00.000+10:00

I was trying to work out whether the Intel VT (VMX) extensions had been enabled on a server and whether the server support the SSE4.1 and SSE4.2 processor instructions - without having to reboot the server to check the BIOS. I ended up 'writing' a PowerShell script that compiles some C# to call a DLL written in assembler which has run the CPUID instruction and returned the results. Note that I didn't really write any of this, the PowerShell script compiling and running the C# is from the reference below, as is the actual C$ and asm dll.

Does this seem like a lot of effort? Sure does, but unfortunately the simple method - using Win32_Processor.ProcessorID was useless in this example, as for some reason MS chose to only return the EAX and EDX registers, whereas the bits I was after were returned in ECX. There may also be other ways to achieve this through PowerShell/.Net framework, rather than compiling c# to call unmanaged code.

Below is the PowerShell script, which contains the inline C# that is compiled and executed, which in turn requires the dll, using the EFLAGS register to confirm support and the CPUID instruction with eax=1 (if further supported) to query feature information from the CPU.

The original c# and asm dll: (see below for the hex text stream)
http://devpinoy.org/blogs/cvega/archive/2006/04/07/2658.aspx

The compile-csharp function: (adding the -unsafe parameter to allow pointer use)
http://monadblog.blogspot.com/2005/12/calling-win32-api-functions-through.html


#
# Description:
#  Call the CPUID function from an external DLL through C# inline code.
#  Report whether Intel VT (VMX) is supported, bit 5 of the ECX register returned from CPUID eax=1 call.
#  Note that WMI Win32_Processor:ProcessID contains the results of a call to CPUID - except that it only contains EAX and EDX, Intel VT (what I was after) is in ECX
#
# Author:
#  Wayne Martin, note that I didn't write any of this code, the inline C# was from the monad reference below, and the CPUID DLL and C# was from the first reference

#References:
# cpuid.dll and the namespace from: http://devpinoy.org/blogs/cvega/archive/2006/04/07/2658.aspx
# Intel cpuid: http://www.intel.com/Assets/PDF/appnote/241618.pdf
# http://monadblog.blogspot.com/2005/12/calling-win32-api-functions-through.html
# http://msdn.microsoft.com/en-us/library/aa394373.aspx


function Compile-Csharp ([string] $code, $FrameworkVersion="v2.0.50727",
[Array]$References)
{
    $cp = new-object Microsoft.CSharp.CSharpCodeProvider
    $cpar = New-Object System.CodeDom.Compiler.CompilerParameters
    $cpar.CompilerOptions = "-unsafe"      # unsafe to compile the code which uses pointers in the DLL call
    $cpar.GenerateInMemory = $true
    $cpar.GenerateExecutable = $false
    $cpar.OutputAssembly = "custom"
    # $cpar.ReferencedAssemblies.AddRange($refs)
    $cr = $cp.CompileAssemblyFromSource($cpar, $code)

    if ( $cr.Errors.Count)
    {
        $codeLines = $code.Split("`n");
        foreach ($ce in $cr.Errors)
        {
            write-host "Error: $($codeLines[$($ce.Line - 1)])"
            write-host $ce
           # $ce out-default
        }
        Throw "INVALID DATA: Errors encountered while compiling code"
    }
}



$code = @'
namespace CPUID
{
 using System;
 using System.Runtime.InteropServices;
 using System.Text;

 public class cpuid
 {
  private cpuid()
  {
  }

  [DllImport("cpuid.dll")]
     public static extern bool CPUIDIsSupported();

  [DllImport("cpuid.dll")]
     private unsafe static extern bool __cpuid
            (uint function, 
             int* eax, 
             int* ebx, 
             int* ecx, 
             int* edx);

  // Invoke __cpuid function
  public unsafe static bool Invoke
      (uint level, 
       out int eax, 
       out int ebx, 
       out int ecx, 
       out int edx)
  {
   int __eax = 0;
   int __ebx = 0;
   int __ecx = 0;
   int __edx = 0;

   if (__cpuid(level, &__eax, &__ebx, &__ecx, &__edx))
   {
    eax = __eax;
    ebx = __ebx;
    ecx = __ecx;
    edx = __edx;

    return true;
   }
   else
   {
    eax = 0;
    ebx = 0;
    ecx = 0;
    edx = 0;

    return false;
   }
  }
 }
}


'@


compile-CSharp $code
$eax = ""
$ebx = ""
$ecx = ""
$edx = ""
[CPUID.CPUID]::Invoke(1, [ref]$eax, [ref]$ebx, [ref]$ecx, [ref]$edx) 

$INTEL_CPUID_VT_FLAG = 0x0020                                                           # bit 5 of ECX returned from CPUID EAX=1
$INTEL_CPUID_SSE41 = 0x80000                                                            # bit 19 of ECX returned from CPUID EAX=1
$INTEL_CPUID_SSE42 = 0x100000                                                           # bit 20 of ECX returned from CPUID EAX=1
$INTEL_CPUID_XD = 0x00100000

write-output ("Intel VT: " + ($ecx -band $INTEL_CPUID_VT_FLAG) )                        # Does this processor support Intel VT / VMX?
write-output ("SSE 4.1: " + ($ecx -band $INTEL_CPUID_SSE41) )                           # Does this processor support SSE 4.1?
write-output ("SSE 4.2: " + ($ecx -band $INTEL_CPUID_SSE42) )                           # Does this processor support SSE 4.2?
write-output ("eXecute Disable: " + ($edx -band $INTEL_CPUID_XD) )                      # Does this processor support XD?

Note that in a previous post, I have written two powershell scripts, one to convert binary to a hex string, and vice versa. The following stream is the cpuid.dll file, if you write it to cpuid.txt and run the following command it will create the dll:



Powershell . .\HexStringToBinary.ps1 cpuid.txt cpuid.dll

4d5a90000300000004000000ffff0000b800000000000000400000000000000000000000000000000000000000000000000000000000000000000000c00000000e1fba0e00b409cd21b8014ccd21546869732070726f6772616d2063616e6e6f742062652072756e20696e20444f53206d6f64652e0d0d0a240000000000000071d4f7db35b5998835b5998835b59988c9958b8834b59988bbaa8a8834b599885269636835b59988000000000000000000000000000000000000000000000000504500004c0103006a793a440000000000000000e0000e210b01050c00020000000400000000000000100000001000000020000000000010001000000002000004000000040000000400000000000000004000000004000070fa000002000000000010000010000000001000001000000000000010000000002000005f0000000000000000000000000000000000000000000000000000000000000000000000003000000800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002e746578740000006c000000001000000002000000040000000000000000000000000000200000602e726461746100005f000000002000000002000000060000000000000000000000000000400000402e72656c6f6300000c000000003000000002000000080000000000000000000000000000400000420000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000558becb801000000c9c20c005352b8000000009c588bc83500002000509d9c5b33c325000020007507b801000000eb0233c0519d5a5bc3558bec5357e8cbffffff48740433c0eb1e8b45080fa28b7d0c89078b7d14890f8b7d1889178b7d10891fb8010000005f5bc9c214000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006a793a44000000003c2000000100000002000000020000002820000030200000382000000c1000003710000046200000572000000000010063707569642e646c6c0043505549444973537570706f72746564005f5f6370756964000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

Extending a server 2003 VM system/boot disk

2009-05-30T10:45:00.003+10:00

Whether using basic or dynamic disks, Windows Server 2003 doesn’t provide a method to extend the size of the system or boot disk. In VI35, I have used the following process to extend the system/boot disk of 2003 server virtual machines. Note that this is for the system/boot disk only, data volumes can be grown and extended live.

This process requires the VM to be shutdown, then a small outage while the disk is extended, then a reboot once the OS is first started.

The process I used in VI 3.5 U4, with a 2003 SP2 virtual machine:

Turn the VM off - VM1 in this example
Use VC to increase the size of the boot/system disk of VM1 by xGB (6 in my test)
Use VC to attach the disk inside another running VM that can be shutdown, eg. VM2
On VM2, load diskmgmt.msc, rescan disks (or run diskpart rescan)
On VM2, start diskpart:
1. List vol (get a list of volumes)
2. Select vol x (where x is the number of the newly added disk)
3. Extend
Shutdown VM2 and detach the disk (don’t delete it!)
Start VM1
On my test, after logon, a setupapi message ‘Windows has finished installing new devices. Do you want to restart your computer now?’.
Said yes and the server restarted, diskmgmt.msc shows correctly sized disk.
Ran chkdsk c: to verify no corruption

Step 8 has a matching event ID 271 from PlugPlayManager and setupapi.log entry:

The Plug and Play operation cannot be completed because a device driver is
preventing the device from stopping. The name of the device driver is listed as
the vetoing service name below.
Vetoed device:
STORAGE\VOLUME\1&30A96598&0&SIGNATURE6AF3AEFAOFFSET7E00LENGTH53FD06C00
Vetoing device:
STORAGE\Volume\1&30a96598&0&Signature6AF3AEFAOffset7E00Length53FD06C00
Vetoing service name: FileSystem\Ntfs
Veto type 6: PNP_VetoDevice
When Windows attempts to install, upgrade, remove, or reconfigure a device,
it queries the driver responsible for that device to confirm that the operation
can be performed. If any of these drivers denies permission (query-removal
veto), then the computer must be restarted in order to complete the operation.
User Action
Restart your computer.

Setupapi.log driver logging showing the first vetoed attempt at installation and then the shadow copy volume snapshot install:
[2009/05/06 08:45:46 404.3 Driver Install]
#-019 Searching for hardware ID(s): storage\volume
#-198 Command line processed: C:\WINDOWS\system32\services.exe
#I393 Modified INF cache "C:\WINDOWS\inf\INFCACHE.1".
#W383 "volume.PNF" migrate: PNF Language = 0409, Thread = 0c09.
#I022 Found "STORAGE\Volume" in C:\WINDOWS\inf\volume.inf; Device: "Generic volume"; Driver: "Generic volume"; Provider: "Microsoft"; Mfg: "Microsoft"; Section name: "volume_install".
#I023 Actual install section: [volume_install.NTx86]. Rank: 0x00000000. Driver date: 10/01/2002. Version: 5.2.3790.3959.
#-166 Device install function: DIF_SELECTBESTCOMPATDRV.
#I063 Selected driver installs from section [volume_install] in "c:\windows\inf\volume.inf".
#I320 Class GUID of device remains: {71A27CDD-812A-11D0-BEC7-08002BE2092F}.
#I060 Set selected driver.
#I058 Selected best compatible driver.
#-166 Device install function: DIF_INSTALLDEVICEFILES.
#I124 Doing copy-only install of "STORAGE\VOLUME\1&30A96598&0&SIGNATURE6AF3AEFAOFFSET7E00LENGTH53FD06C00".
#-166 Device install function: DIF_REGISTER_COINSTALLERS.
#I056 Coinstallers registered.
#-166 Device install function: DIF_INSTALLINTERFACES.
#-011 Installing section [volume_install.NTx86.Interfaces] from "c:\windows\inf\volume.inf".
#I054 Interfaces installed.
#-166 Device install function: DIF_INSTALLDEVICE.
#I123 Doing full install of "STORAGE\VOLUME\1&30A96598&0&SIGNATURE6AF3AEFAOFFSET7E00LENGTH53FD06C00".
#W100 Query-removal during install of "STORAGE\VOLUME\1&30A96598&0&SIGNATURE6AF3AEFAOFFSET7E00LENGTH53FD06C00" was vetoed by "STORAGE\Volume\1&30a96598&0&Signature6AF3AEFAOffset7E00Length53FD06C00" (veto type 6: PNP_VetoDevice).
#W104 Device "STORAGE\VOLUME\1&30A96598&0&SIGNATURE6AF3AEFAOFFSET7E00LENGTH53FD06C00" required reboot: Query remove failed (install) CfgMgr32 returned: 0x17: CR_REMOVE_VETOED.
#I121 Device install of "STORAGE\VOLUME\1&30A96598&0&SIGNATURE6AF3AEFAOFFSET7E00LENGTH53FD06C00" finished successfully.
[2009/05/06 08:55:18 400.3 Driver Install]
#-019 Searching for hardware ID(s): storage\volumesnapshot
#-198 Command line processed: C:\WINDOWS\system32\services.exe
#W383 "volsnap.PNF" migrate: PNF Language = 0409, Thread = 0c09.
#I022 Found "STORAGE\VolumeSnapshot" in C:\WINDOWS\inf\volsnap.inf; Device: "Generic volume shadow copy"; Driver: "Generic volume shadow copy"; Provider: "Microsoft"; Mfg: "Microsoft"; Section name: "volume_snapshot_install".
#I023 Actual install section: [volume_snapshot_install.NTx86]. Rank: 0x00000000. Driver date: 10/01/2002. Version: 5.2.3790.3959.
#-166 Device install function: DIF_SELECTBESTCOMPATDRV.
#I063 Selected driver installs from section [volume_snapshot_install] in "c:\windows\inf\volsnap.inf".
#I320 Class GUID of device remains: {533C5B84-EC70-11D2-9505-00C04F79DEAF}.
#I060 Set selected driver.
#I058 Selected best compatible driver.
#-166 Device install function: DIF_INSTALLDEVICEFILES.
#I124 Doing copy-only install of "STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOT1".
#-166 Device install function: DIF_REGISTER_COINSTALLERS.
#I056 Coinstallers registered.
#-166 Device install function: DIF_INSTALLINTERFACES.
#-011 Installing section [volume_snapshot_install.NTx86.Interfaces] from "c:\windows\inf\volsnap.inf".
#I054 Interfaces installed.
#-166 Device install function: DIF_INSTALLDEVICE.
#I123 Doing full install of "STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOT1".
#I121 Device install of "STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOT1" finished successfully.

References

How to extend a data volume in Windows Server 2003, in Windows XP, in Windows 2000, and in Windows Server 2008
http://support.microsoft.com/kb/325590

Generating 100% CPU with calc or PowerShell

2009-05-16T17:26:00.000+10:00

Occasionally when testing something I want to generate 100% CPU load on a Windows computer. There are several utilities out there to do this, but that implies you have the utility on hand and are comfortable running it on the server. A colleague of mine (thanks Mark S.) showed me this nifty trick of using calc.exe to generate 100% CPU. The best thing about this is that every standard Windows OS installation has calc.exe.

This post provides two methods of generating 100% CPU load, the original calc.exe method, and a simple one line PowerShell command to do the same thing from the command-line (locally or on a remote server with PS v1 and psexec). Note that there may be a better method with PowerShell, I simply scripted the same operation calc was performing.

On dual-CPU/core computers this uses 100% of one CPU/core. To use more than that, calc or the PowerShell command can be run more than once, and Windows by default will run the new process on another less-busy CPU/core.

One practical application of this is to load-test a VMware VI3 cluster, generating 100% CPU on one or more VMs to see how ESX and DRS/VC handles the load. I have also used this in the past when testing multi-threaded applications and processor affinity to see how Windows allocates a processor.

calc.exe

Use calc to calculate the factorial of a number - the product of all integers from 1 up to and including the number specified, eg 5! = 1x2x3x4x5

Run calc.exe and switch to scientific mode
Type a large number (eg. 12345678901234567890), press the 'n!' button.
Calc will ask to confirm after warning this will take a very long time
100% CPU utilisation will now occur (essentially forever)

PowerShell

Using the largest int32 positive integer, calculate the factorial to generate 100% CPU utilisation
$result = 1; foreach ($number in 1..2147483647) {$result = $result * $number};

Depending on how fast the CPU is, this could finish, so a loop to run the command above 2 billion times:
foreach ($loopnumber in 1..2147483647) {$result=1;foreach ($number in 1..2147483647) {$result = $result * $number};$result}

If you want to see how long the command takes to run:
Measure-Command {$result = 1; foreach ($number in 1..2147483647) {$result = $result * $number}}

Using the command-line then provides the ability to run the command remotely. To use psexec to remotely execute powershell v1 factorial to generate 100% CPU:
psexec \\%computername% /s cmd /c "echo. | powershell $result = 1; foreach ($number in 1..2147483647) {$result = $result * $number}"

Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.

Converting VHD to VMDK SCSI for ESX

2009-05-04T07:14:00.000+10:00

I had problems converting a 2003 server VHD to a vmdk that I could import into a VM running on ESX. I used WinImage to convert eh VHD->VMDK, but it seems WinImage creates the VMDK as an IDE device, which is unsupported by ESX. I'm sure there are better ways to do this, such as satisfying whatever the pre-requisites are to getting VMware converter to automatically inject the drivers, but it was interesting doing it manually.

Below is information on the process that I thought would have worked automatically, followed by the manual steps I took to make it work.

To convert the vmdk, the following process was first tried to get VMware converter to convert the IDE disk to something ESX would recognise:

1. Use WinImage to create the vmdk from the vhd
2. Use a VMware workstation VMX, including the disk as an IDE device. (a modified vmx is fine you don’t actually need VMware workstation)
3. Use VMware converter to import the workstation VMX into VC

I thought this would have been enough, but the VMware Converter process failed at 95% saying that it couldn’t find symmpi.sys. Symmpi.sys is the LSI Logic SCSI driver for the virtual SCSI adapter. I’m guessing that running VMware converter should automatically inject the drivers into the vmdk but couldn’t in this scenario (maybe because my local PC didn’t have the driver, or maybe because the driver cache cab files containing symmpi.sys weren’t on the target vmdk).

Powering on the machine resulted in a stop 0x7b inaccessible disk error. To manually fix the problem, I then:

Added the disk to another VM. When starting the VM it warned that the new disk was created for LSI not buslogic. I said yes to convert to buslogic (which this VM was using as opposed to LSI).

The drive was then accessible through the VM, and I added the drivers (file and registry):

Copied the driver file to the drivers directory: copy "\\%workingVM%\c$\WINDOWS\system32\drivers\symmpi.sys" "%mountedDrive%:\WINDOWS\system32\drivers"
Copied the driver cache files to the machine from a working 2003: copy "\\%workingVM%\c$\WINDOWS\Driver Cache\i386\*.*" "%mountedDrive%:\WINDOWS\Driver Cache\i386"
Exported the HKLM\SYSTEM\CurrentControlSet\Services\symmpiregistry and HKLM\SYSTEM\CurrentControlSet\Control\CriticalDeviceDatabase\pci#ven_1000&dev_0030 entries from a working server as regedit4 files (not Unicode).
Loaded the %mountedDrive%:\windows\system32\config\system registry hive on the disk to HKLM\VM: reg load HKLM\VM %mountedDrive%:\windows\system32\config\system
Modified the reg files to match the path the hive was loaded to (eg HKLM\VM\controlset001 instead of HKLM\system\currentcontrolset). The modified reg files are included below.
Imported the registry files which modified the loaded system hive on the disk
Disconnected the hive, shutdown the VM and disconnected the disk from the VM and reattached to the server created during the VMware Converter process
Turned on the server and was prompted to convert the disk type back to LSI (which I did).
The server started normally

Note that the conversion between buslogic and LSI logic in both directions were only required because the virtual machine that I mounted the disk on had a different adapter type.

---



REGEDIT4
[HKEY_LOCAL_MACHINE\VM\ControlSet001\Services\symmpi]
"ErrorControl"=dword:00000001
"Group"="SCSI miniport"
"Start"=dword:00000000
"Type"=dword:00000001
"ImagePath"=hex(2):73,79,73,74,65,6d,33,32,5c,44,52,49,56,45,52,53,5c,73,79,6d,\
6d,70,69,2e,73,79,73,00
"Tag"=dword:00000021

[HKEY_LOCAL_MACHINE\VM\ControlSet001\Services\symmpi\Parameters]
"BusType"=dword:00000001

[HKEY_LOCAL_MACHINE\VM\ControlSet001\Services\symmpi\Parameters\PnpInterface]
"5"=dword:00000001

[HKEY_LOCAL_MACHINE\VM\ControlSet001\Services\symmpi\Enum]
"0"="PCI\\VEN_1000&DEV_0030&SUBSYS_00000000&REV_01\\3&61aaa01&0&80"
"Count"=dword:00000001
"NextInstance"=dword:00000001



REGEDIT4
[HKEY_LOCAL_MACHINE\VM\ControlSet001\Control\CriticalDeviceDatabase\pci#ven_1000&dev_0030]
"Service"="symmpi"
"ClassGUID"="{4D36E97B-E325-11CE-BFC1-08002BE10318}"

References:
Injecting SCSI controller device drivers into Windows
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1005208

Troubleshooting a virtual machine that fails to boot with STOP 0x7B error
http://kb.vmware.com/selfservice/viewContent.do?externalId=1006295&sliceId=1

Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.

VMware VI3 iSCSI with multiple non-HBA NICs

2009-05-02T21:37:00.003+10:00

While researching iSCSI on VI3, I came across some interesting information when using the ESX iSCSI software initiator that would be applicable to many installations, highlighting a potential bottleneck.

The short version is that if you’re using the iSCSI software initiator connecting to a single iSCSI target, multiple uplinks in an ESX network team for the VMKernel iSCSI port would not be used for load balancing.

This can be easily proven by connecting to the service console and running esxtop (n) to view the traffic for individual network adapters. Assuming your storage is in use, one or more physical uplinks for the vSwitch handling iSCSI should be showing traffic. You can also use resxtop through the RCLI on ESXi.

Why this happens

My understanding is that current ESX software initiated iSCSI connections have a 1:1 relationship between NIC and iSCSI targets. An iSCSI target in this sense is a connection to the IP-based SAN storage, not LUN targets. This limitation applies when the SAN presents a single IP address for connectivity.

VI3 software initiated iSCSI doesn’t support multipathing, which within ESX leaves only load balancing the physical uplinks in a team. Unfortunately, that leaves load balancing up to the vSwitch load balancing policy exceptions. I don’t believe any of the three choices fit most scenarios when connectivity to the iSCSI is through a single MAC/IP:

Route based on the originating virtual switch port ID, based on virtual port ID, of which there is only one VMKernel iSCSI port
Route based on source MAC hash, based on source MAC, of which there is only one
Route based on IP hash, based on layer 3 source-destination IP pair, of which there is only one (VMKernel -> iSCSI virtual address). I don’t think this is a generally recommended load balancing approach anyway

Link aggregation

The VI3 SAN Deploy guide does state that one connection is established to each target. This seems to indicate one connection per LUN target, but the paragraph starts with software iSCSI and switches half way through to discuss iSCSI HBA’s.

I’m still unsure of whether software iSCSI has multiple TCP sessions, one per target (I don’t believe this is the case). The blog referenced below also talks about 802.3 link aggregation which states the ESX 3.x software initiator does not support multiple TCP sessions.

However, if multiple TCP sessions were being established for the iSCSI software initiator to a single target IP address, this opens the possibility of link aggregation at the physical switch. When using 802.3ad LACP in this IP-IP scenario, the switches would have to distribute connections based on the hash of TCP source/destination ports, rather than just IP/MAC.

The following excerpt from the SAN deploy guide:

Software iSCSI initiators establish only one connection to each target.
Therefore, storage systems with a single target that contains multiple LUNs have all LUN traffic routed through that one connection. In a system that has two targets, with one LUN each, two connections are established between the ESX host and the two available volumes. For example, when aggregating storage traffic from multiple connections on an ESX host equipped with multiple iSCSI HBAs, traffic for one target can be set to a specific HBA, while traffic for another target uses a different HBA. For more information, see the “Multipathing” section of the iSCSI SAN Configuration Guide. Currently, VMware ESX provides active/passive multipath capability. NIC teaming paths do not appear as multiple paths to storage in ESX host configuration displays, however. NIC teaming is handled entirely by the network layer and must be configured and monitored separately from ESX SCSI storage multipath configuration.

VI4/vSphere

Excerpts from the following blog, indicate that changes in vSphere for software iSCSI to support multiple iSCSI sessions, allowing multipathing or link aggregation, which would allow separate iSCSI TCP sessions to be spread across more than one NICs (depending on how many iSCSI sessions).

http://virtualgeek.typepad.com/virtual_geek/2009/01/a-multivendor-post-to-help-our-mutual-iscsi-customers-using-vmware.html

The current experience discussed above (all traffic across one NIC per ESX host):

VMware can’t be accused of being unclear about this. Directly in the iSCSI SAN Configuration Guide: ESX Server‐based iSCSI initiators establish only one connection to each target. This means storage systems with a single target containing multiple LUNs have all LUN traffic on that one connection, but in general, in my experience, this is relatively unknown.
This usually means that customers find that for a single iSCSI target (and however many LUNs that may be behind that target – 1 or more), they can’t drive more than 120-160MBps. This shouldn’t make anyone conclude that iSCSI is not a good choice or that 160MBps is a show-stopper. For perspective I was with a VERY big customer recently (more than 4000 VMs on Thursday and Friday two weeks ago) and their comment was that for their case (admittedly light I/O use from each VM) this was working well. Requirements differ for every customer.

The changes in vSphere:

Now, this behavior will be changing in the next major VMware release. Among other improvements, the iSCSI initiator will be able to use multiple iSCSI sessions (hence multiple TCP connections). Looking at our diagram, this corresponds with “multiple purple pipes”for a single target. It won’t support MC/S or “multiple orange pipes per each purple pipe” – but in general this is not a big deal (large scale use of MC/S has shown a marginal higher efficiency than MPIO at very high end 10GbE configurations) .
Multiple iSCSI sessions will mean multiple “on-ramps” for MPIO (and multiple “conversations” for Link Aggregation). The next version also brings core multipathing improvements in the vStorage initiative (improving all block storage): NMP round robin, ALUA support, and EMC PowerPath for VMware which integrates into the MPIO framework and further improves multipathing. In the spirit of this post, EMC is working to make PowerPath for VMware as heterogeneous as we can.
Together – multiple iSCSI sessions per iSCSI target and improved multipathing means aggregate throughput for a single iSCSI target above that 160MBps mark in the next VMware release, as people are playing with now. Obviously we’ll do a follow up post.

Vista Sidebar Gadget for SiteMeter

2009-03-05T22:08:00.002+10:00

This post provides a Vista Sidebar gadget to report on sitemeter information for a web site being monitored. In a previous post I’ve provided a command-line PowerShell method to retrieve the same information, but I thought I’d try this GUI thing people keep talking about.

This is a very simple Vista sidebar gadget, it doesn’t have settings, or fly-outs, or links, or any other bits of cleverness, it was really just my first look into how Vista gadgets work. All I did was take the Microsoft hello-world example gadget and add a bit of javascript to (badly) scrape the sitemeter web page.

The gadget:

Has a transparent PNG background image and writes the text in white on two lines, with a gadget size of 128x64
Uses the MSXML DOM to issue a HTTP GET and for the sitemeter URL, using an asynchronous callback
Parses the HTTP response, looking for the first index of the word ‘Today’ and extracts the number
Updates the second field in the gadget with the current date/time to tell when the last successful get occurred
Uses the SetTimeout method to sleep for an hour before calling the getData() function again

What the gadget doesn’t do that could make this better:

Have the URL and timeouts stored in a settings file to save having to modify the JS when scraping a different URL or changing the timeout.
Have a flyout or something which shows the other summary information on the sitemeter page
Use a separate CSS and init() function rather than the in-line code from the example

To create this:

Download the gadget samples from http://www.microsoft.com/downloads/details.aspx?FamilyID=b1e14e4f-3108-4c57-8b78-1157ca40dcc2
Copy SDK_HelloWorld.gadget to *.zip
Unzip the contents of the gadget zip file to a working directory and remove the read-only attribute from the files
Create a transparent PNG background 64x64 in size (or use the one provided in this post), and overwrite Background.png
Update images\aerologo.PNG with the png from this post
Update HelloWorld.html with the contents below:
Update the height and width to 128x64
- Add the script tag:
- Update the in-place init() funciton to call getData()
- Create SiteMeter.js with the contents below
Create a new directory and copy all of the gadget files to "%Systemdrive%\users\%username%\appdata\local\microsoft\Windows Sidebar\gadgets\SiteMeterCounter.gadget"
Update the URL in SiteMeter.js with the page to scrape
Add the gadget

HelloWorld.html


<!--
 *************************************************************************
 *
 * Name: SiteMeter.html
 *
 * Description: 
 * Simple SiteMeter counter
 * Displays the current count of today's visitors from the provided SiteMeter URL
 * 
 *
 * Modified from the 'Hello World' Microsoft example 
 ************************************************************************
-->
<html xmlns="http://www.w3.org/1999/xhtml">
    <head>
     <title>SiteMeter Count</title>
     <style type="text/css">
      body
      {
          width: 128px;
          height: 64px;
                        font-family: calibri;
                        color: white; 
      }
      #gadgetContent
      {
          width: 128px;
                        top: 3px;
          text-align: center;
                        overflow: hidden;
                        font-weight: bold;
                        font-size: 14px;
      }
      #lastUpdate
      {
          width: 128px;
                        top: 20px;
          text-align: center;
                        overflow: hidden;

                        font-size: 9px;
      }
     </style>
     <script type="text/javascript" src="SiteMeter.js"></script>
     <script type="text/jscript" language="jscript">


        // --------------------------------------------------------------------
        // Initialize the gadget.
        // --------------------------------------------------------------------
     function init()
     {
         var oBackground = document.getElementById("imgBackground");
         oBackground.src = "url(images/background.png)";
                getData();
        }
     </script>
    </head>
 
<body onload="init()">
    <g:background id="imgBackground">
 <span id="gadgetContent">-</span>
 <span id="lastUpdate">-</span>
 </g:background>
</body>
</html>

SiteMeter.js


var globalURL = "http://www.sitemeter.com/default.asp?a=stats&s=s451qaz2wsx";              // URL for lookup
var globalTimeoutId;
var globalError;
var XMLHttp;

function getData()
{
    try 
        { XMLHttp = new ActiveXObject("Msxml2.XMLHTTP"); 
    } 
    catch (e) 
        { XMLHttp = new ActiveXObject("Microsoft.XMLHTTP"); 
    }

    // Timeout, if call not come back in 30 seconds, abort with the default error message.
    globalTimeoutId = setTimeout(function() {
                                         XMLHttp.abort();
                                         document.getElementById("gadgetContent").innerHTML = "Timeout";
                                         }, 30000); 
        
    // Asynchronous get
    XMLHttp.open("GET", globalURL, true);

    XMLHttp.setRequestHeader("Content-Type","application/x-www-form-urlencoded");
    XMLHttp.setRequestHeader("Connection", "close");

    // Callback for when the page has loaded
    XMLHttp.onreadystatechange = parseData

    // Send the request
    XMLHttp.send(null);

}

function parseData()
{
    var Results;
    var td;

    gadgetContent.innerHTML = XMLHttp.status;

    if (XMLHttp.readyState == 4 && XMLHttp.status == 200)
    {

        var date = new Date();
        lastUpdate.innerHTML = date.getDate() + "/" + (date.getMonth() +1)  + "/" + date.getYear() + " " + date.getHours() + ":" + date.getMinutes() + ":" + date.getSeconds();

        globalError = 0;

        Results = XMLHttp.responsetext;
        if (Results.indexOf("Today") >= 0)
        {

            td = Results.substring(Results.indexOf("Today")+20, Results.indexOf("Today")+80);
            td = td.substring(td.indexOf("<font"), td.length);
            td = td.substring(0, td.indexOf("</font>")+7);

            gadgetContent.innerHTML = td;
        }
        else
        {   
            globalError = 1;
            document.getElementById("gadgetContent").innerHTML = "Error";
        }

        // Set a callback for the getData function in one hour
        clearTimeout(globalTimeoutId);
        globalTimeoutId = setTimeout(getData, 3600000);
    }

}

Security with 2003 R2 FSRM quotas

2009-02-24T22:28:00.002+10:00

This post provides information on several aspects of security with Windows Server 2003 R2 FSRM quotas, which you may come across if you use the previous two posts to implement FSRM quotas on a standalone server or MSCS clustered node.

LocalService Command Notification Task

If your quota has a notification task that runs a command, you may need to set the security of the executable or areas accessed by the command. By default, FSRM runs commands as the LocalService account - an account with restricted local permissions.

In this example, the command executed is the logentry.bat batch file from the previous post, which simply writes a log entry to a file. To allow the batch file to run as the localservice account – which is a member of the 'Authenticated Users' group, I set the following permissions:

Add authenticated users:R to the root (not OICI - object inherit, container inherit) using SDDL with cacls
Add authenticated users:R to c:\scripts to execute the logentry.bat file
Add authenticated users:C to c:\logs to allow writing the log entry
Ensure that cmd.exe can be executed (default permissions should suffice)

This was done with the following commands:

cacls c:\ /S:"D:PAI(A;OICI;FA;;;BA)(A;;0x1200a9;;;AU)(A;OICI;FA;;;SY)"
cacls c:\scripts /e /g "authenticated users":R
cacls c:\logs /e /g "authenticated users":C

Note that while testing this, I was using localsystem (unrestricted local permissions) to verify that security was the issue, but in an MSCS cluster when failing over the virtual server from one cluster node to the other, the quota template reset itself to localservice instead of localsystem.

Before setting permissions to allow LocalService to run the script, this left the notification task not executing successfully. Depending upon what your command does you may require elevated local or remote privilege, for which you may want to use localsystem or networkservice.

Target Permissions preventing execution

When specifying a command to run as a notification task, 2003 R2 FSRM will not allow running a command which standard users have change/full control to, directly or through inheritance.

To me this seems like an unusual approach for an out of the box Microsoft product and isn't very intuitive, as in my lab inherited permissions from a parent directory allowing Users:F were causing the task to fail.

References

Implementing 2003 FSRM quotas Command-line
http://waynes-world-it.blogspot.com/2009/02/implementing-2003-fsrm-quotas.html

2003 FSRM and NTFS Quotas compared
http://waynes-world-it.blogspot.com/2009/02/2003-fsrm-and-ntfs-quotas-compared.html

LocalService Account
http://msdn.microsoft.com/en-us/library/ms684188(VS.85).aspx

Implementing 2003 FSRM quotas

2009-02-10T13:43:00.023+10:00

This post provides information on implementing Windows Server 2003 FSRM quotas using the command-line dirquota.exe utility, creating a template and then applying that template to the filesystem using an autoquota. Example control files are provided to create the template, and a simple batch file to create a daily log file with quota notifications.

The first command below creates the soft 100MB quota template with three thresholds (85/99/150), and four notifications for those thresholds, two for email and one each for and command execution and event log entry. The second command creates an autoquota using the template on the specified remote server against the specified path. Note that in this example, the commands were run on a 64-bit physical server running MSCS, and the target server was a virtual cluster server (v01).

dirquota template add /Remote:v01 /Template:Test_Quota /Limit:100mb /type:Soft /label:"Test Default Quota" /add-threshold:85 /add-notification:85,M,c:\admin\control\Test_Quota_Email.txt /add-threshold:99 /add-notification:99,M,c:\admin\control\Test_Quota_Email.txt /add-notification:99,C,c:\admin\control\Test_Quota_Command.txt /add-notification:99,E,c:\admin\control\Test_Quota_Event.txt /add-threshold:150 /add-notification:150,M,c:\admin\control\Test_Quota_Email.txt
dirquota autoquota add /Remote:v01 /path:q:\folder1 /sourcetemplate:"Test_Quota"

Notes:

The use of 99% instead of 100% was intentional, as 100% is not a notification; it’s a limit, so the pre-defined variables are different. This is obvious in the subject of the email, where the ‘[Quota Threshold]’ variable isn’t resolved when it’s a 100% 'notification'.
The quota information is logged to \\p01\c$\logs\QuotaUsage_YYYYMMDD.log for 99% of quota usage, as well as sending an email to DiskUsageMonitor and logging an event on the cluster, and all quota information is available through the FSRM MMC snap-in.
Within a single email notification, an email can be sent to one or more administrators, and/or the person who took the limit over the threshold – the owner of the file. In the template below, emails will be sent to both users for all three thresholds, and administrators for the 150% notification. E-mail notifications will be limited to one per day for the same notification.
Any command can be run; in the example here a simple batch file is run that appends a log entry to a daily log file, providing an easy method to see quota alerts for each day.
The notifications aren’t triggered until first create, so if you apply quotas to existing data, the notifications won’t start appearing until new data is written.
A quirk with the 64-bit OS - creating the quota template only works with the 32-bit version of the dirquota.exe utility. If the physical server (p01 in the example above) were a 64-bit server, you would have to run the 32-bit dirquota.exe utility to create the template and autoquota (2003 enterprise R2 x64 SP1).

Configuration Files

Templates, quotas and autoquotas can all be created via command-line utilities. When creating templates with notifications, the information is supplied via control files. Example control files for running a command, logging an event, and sending an email are shown below.

Note the following global properties can be specified in each notification:

Notification – m | e | c | r
m - an e-mail notification
e - an event log notification
c - a command or script execution
r - a report generation
RunLimitInterval – The number of minutes to wait between sending notifications to save repeated unnecessary notifications. A setting of 0 indicates a notification will be sent on each trigger.



Test_Quota_Command.txt

Notification=c
Command=c:\windows\system32\cmd.exe
Arguments=/c c:\admin\scripts\logentry.bat "%Date%,%Time%,[Source Io Owner],[Quota Path],[Server],[Quota Limit MB] MB,[Quota Used MB] MB,[Quota Used Percent]"
MonitorCommand=Disable
Account=LocalService
LogResult=Enable
RunLimitInterval=0



Test_Quota_Event.txt

Notification=e
RunLimitInterval=1440
EventType=Warning
Message=Excessive usage by [Source Io Owner] on [Quota Path], shared on [Server]. Limit of [Quota Limit MB] MB, [Quota Used MB] MB in use ([Quota Used Percent]% of limit).



Test_Quota_Email.txt

Notification=m
RunLimitInterval=1440
To=[Source Io Owner Email]
From=FSRM@server.domain.com
ReplyTo=FSRM-DoNotReply@server.domain.com
Cc=DiskUsageMonitor@domain.com
Subject=[Quota Threshold]% quota threshold exceeded
Message=A file written by [Source Io Owner] has exceeded the [Quota Threshold]% quota threshold for the quota on [Quota Path] on server [Server]. \
\
The quota limit is [Quota Limit MB] MB, and [Quota Used MB] MB currently is in use ([Quota Used Percent]% of limit).\
\
\
\

Logging to File

I'm not sure why a method to append to a log file wasn't included in the GUI, but this batch file appends a one-line entry to a rolling log file:


:: LogEntry.bat

:: Write a log for quota alerts

::%Date%,%Time%,[Source Io Owner],[Quota Path],[Server],[Quota Limit MB] MB,[Quota Used MB] MB,[Quota Used Percent]%

Set AdminLog=C:\Logs

for /f "tokens=1-8 delims=/:. " %%i in ('echo %date%') do Set DateFlat=%%l%%k%%j
Set LogFile=%AdminLog%\QuotaUsage_%DateFlat%.log

Echo %~1 >> %LogFile%

References

2003 FSRM and NTFS Quotas compared
http://waynes-world-it.blogspot.com/2009/02/2003-fsrm-and-ntfs-quotas-compared.html

FSRM and NTFS Quotas in 2003 R2
http://waynes-world-it.blogspot.com/2008/06/fsrm-and-ntfs-quotas-in-2003-r2.html

Configuration files for notifications in File Server Resource Manager
http://technet2.microsoft.com/windowsserver2008/en/library/a4426339-5345-44d5-81b7-a35a703daaac1033.mspx?mfr=true

How to use File Server Resource Manager (FSRM) to configure the notification feature for File Screening Management in Windows Server 2003 R2
http://support.microsoft.com/kb/926092

File Server Resource Manager Protocol Specification
http://download.microsoft.com/download/9/5/E/95EF66AF-9026-4BB0-A41D-A4F81802D92C/%5BMS-FSRM%5D.pdf

Dirquota admin options
http://technet2.microsoft.com/windowsserver2008/en/library/14c2a340-54cf-46fa-8b0d-beed6c220c671033.mspx?mfr=true

LocalService Account
http://msdn.microsoft.com/en-us/library/ms684188(VS.85).aspx

Configuration files for notifications in File Server Resource Manager
http://technet2.microsoft.com/windowsserver2008/en/library/a4426339-5345-44d5-81b7-a35a703daaac1033.mspx?mfr=true

Create an auto quota
http://technet2.microsoft.com/WindowsServer/en/library/0de5535e-ef25-4ffa-a724-155573044ddc1033.mspx

Dirquota autoquota
http://technet2.microsoft.com/WindowsServer2008/en/library/2809c575-8d93-47cb-8bfc-a427da83cc2a1033.mspx

File Services

http://technet2.microsoft.com/windowsserver2008/en/library/6e5bb377-db25-4603-b1ff-ecc4f6c29b691033.mspx?mfr=true

Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.

2003 FSRM and NTFS Quotas compared

2009-02-09T13:48:00.003+10:00

This post provides a quick comparison of 2003 FSRM and NTFS quotas, which I find useful when explaining how quotas in 2003 R2 work, as opposed to (or combined with) NTFS quotas. Also included is information on where the quota data is stored, and some methods to view that data.

Quota metafile information has been part of the NT filesystem since Windows NT 3.5, but has not been supported by the Operating System since the implementation of NTFS 5.0 – available in Windows 2000 and later.

When enabled, NTFS quotas track information as part of each write operation to the filesystem, providing a per-volume mapping between user SID and logical disk usage based on file ownership. While all the necessary information is stored in the NT filesystem, managing NTFS quotas is time-consuming and challenging for administrators.

Windows Server 2003 R2 File System Resource Manager (FSRM) provides a filesystem minifilter to control quotas, and a much improved interface to manage and report on quotas from a per-folder perspective.

The main differences between the two distinct quota methods available in 2003 R2 are that:

FSRM provides per-folder quotas, as opposed to per-user/volume NTFS quotas. Regardless of file ownership, files in a folder will count towards the FSRM-set limits.
SMB calls to return the free disk space are based on hard quotas at the root of the share or volume, not the quota applied to a folder - regardless of the share access point. NTFS hard quotas are volume-wide, and disk space is presented based on used-hard quota total, regardless of the share root or access method (remote SMB or local).
FSRM quotas count only the size on disk of files, as opposed to NTFS quotas which count the logical uncompressed size. This is primarily considered for NTFS compressed files, but is presumably the same for offline files.
FSRM quotas are controlled by a file system mini-filter storing quota data in \System Volume Information\SRM\quota.md and quota.xml, as opposed to NTFS quotas which are stored as part of the filesystem in \$Extend\$Quota file in $INDEX_ROOT NTFS attributes
FSRM allows autoquota's, a concept of setting a quota at a top-level directory and each direct child subdirectory automatically inherits a copy of that quota. This provides an easy method of exception-based quotas. Managing NTFS quotas is GUI-based unless the WMI automation interface is used and an NTFS quota entry is automatically created for each new user SID.
FSRM provides much improved reporting and alerting for quotas, whereas NTFS quotas only provide rudimentary reporting and eventlog entry alerting.
FSRM has no supported automation interface to manage quotas, whereas NTFS quotas can be managed by WMI. However, the .Net assembly srmlib.dll provides an undocumented framework for managing FSRM quotas, which could be scripted through PowerShell if required.
FSRM provides very strong support for command-line administration with dirquota.exe, with NTFS quotas having limited support available through fsutil
In a MSCS cluster scenario, FSRM stores settings in the registry, located in HKLM\Cluster\SRM\Settings\SrmGlobalSettings\Data. NTFS quotas have all information stored on the filesystem, making both methods functional in a MSCS server cluster with shared storage.
FSRM quotas provide improved notification - including in-built email, event logging, running a command or triggering a report.
FSRM quotas allow for templates to be created, separating the creation of a standard set of quotas from the application of those quotas. This allows scalability and much improved process control.

How FSRM quota information is stored

FSRM quotas are stored in the "?:\System Volume Information\SRM\quota.xml" and "?:\System Volume Information\SRM\quota.sd" files, with the XML containing the configuration, and the SD file containing the actual quota information.

To see the configuration of FSRM quotas for a particular volume:

• psexec /s /i /d cmd.exe
• xcopy /h "?:\System Volume Information\SRM\quota.xml" %temp%
• attrib -r -s -h "%temp%\quota.xml"

The SD file is secured so only system can access, is marked as system/hidden and is locked by the mini-filter. One method to view the SD:

• psexec /s /i /d cmd.exe
• nfi "h:\System Volume Information\SRM\quota.md"
• diskedit Read Sectors (as returned by nfi)

How NTFS quota information is stored

NTFS stores quota information in a metafile on each volume called \$Extend\$Quota, with the information contained in the INDEX_ROOT $O and $Q NTFS attributes. Nfi.exe and diskedit.exe can be used to identify the file, and view the data contained in the logical sectors.

nfi q:

File 24
\$Extend\$Quota
$STANDARD_INFORMATION (resident)
$FILE_NAME (resident)
$INDEX_ROOT $O (resident)
$INDEX_ROOT $Q (resident)
$INDEX_ALLOCATION $Q (nonresident)
logical sectors 1036140-1036147 (0xfcf6c-0xfcf73)
$BITMAP $Q (resident)

Quota Minifilter driver

FSRM quotas use a minifilter driver to function – quota.sys – mounted by default in the I/O stack with an altitude of 125000 as part of the ‘FSFilter Physical Quota Management’ group. While this altitude can be changed by modifying a registry value, this is not recommended.

Both the R2 file screen filter (260800) and the cluster file system (200000-209999) are loaded higher in the stack then the quota minifilter.

fltmc filters & fltmc instances

Filter Name Num Instances Frame
------------------------------ ------------- -----
DfsDriver
Datascrn 0 0
Quota 1 0

Filter Volume Name Altitude Instance Name
----------------------------- -----------------------------
Quota Q: 125000 Quota

To detach the filter from a volume, the following command can be run:
• fltmc detach [volume:]

Note that doing so leaves the SRM directory in the ‘System Volume Information’ on the volume, and during testing when fltmc was used to reattach the quota filter to the volume, the previous quotas were seen as invalid and returned errors.

References

FSRM and NTFS Quotas in 2003 R2
http://waynes-world-it.blogspot.com/2008/06/fsrm-and-ntfs-quotas-in-2003-r2.html

Inside Win2K NTFS, Part 1
http://msdn.microsoft.com/en-us/library/ms995846.aspx

You cannot create quotas on File Server Resource Manager (FSRM) in Windows Server 2003 R2
http://support.microsoft.com/kb/555941

FSRM quota information does not appear in the NTFS file system Quota Entries window, and NTFS file system disk quota information does not appear in FSRM in Windows Server 2003 R2
http://support.microsoft.com/kb/915042

Limited Group Policy management for NTFS quotas.
http://technet2.microsoft.com/windowsserver/en/library/2d82decb-6726-4c5c-b872-1658b0fc3e3e1033.mspx?mfr=true

Disk Quotas Tools and Settings
http://technet2.microsoft.com/windowsserver/en/library/3b5b242b-7bb2-48e4-8e5f-224a08b36b271033.mspx

HOW TO: Configure Disk Quotas for a Shared Disk in a Cluster
http://support.microsoft.com/kb/278365

Disk Quotas Tools and Settings
http://technet2.microsoft.com/windowsserver/en/library/3b5b242b-7bb2-48e4-8e5f-224a08b36b271033.mspx

Managing Disk Quotas in Windows Server 2003 and Windows XP
http://www.microsoft.com/technet/scriptcenter/topics/win2003/quotas.mspx

Designing a Disk Quota Strategy
http://technet2.microsoft.com/windowsserver/en/library/1EE8754E-48D6-4472-9B53-29E8D1DE09F81033.mspx

Quotas in a cluster:
http://support.microsoft.com/kb/278365

How Disk Quotas Work
http://technet2.microsoft.com/windowsserver/en/library/5becbcd6-8da3-4c3b-bc0e-258acd3ec1811033.mspx?mfr=true

Disk Quotas and Free Space
http://www.microsoft.com/technet/prodtechnol/windows2000serv/reskit/core/fncd_str_ctkj.mspx?mfr=true

Quota Minifilter Driver
http://technet2.microsoft.com/windowsserver2008/en/library/7c5a0b98-d963-4a1d-a499-316322746a8e1033.mspx?mfr=true

MUP Changes in Microsoft Windows Vista
http://msdn.microsoft.com/en-us/library/aa488427.aspx

File System Minifilter Load Order Groups and Altitude Ranges
http://www.microsoft.com/whdc/driver/filterdrv/alt-range.mspx

printQueue AD objects for 2003 Cluster

2009-01-30T11:35:00.003+10:00

Print queue objects in AD provide a useful facility when users are trying to find printers, but with a 2003 MSCS clustered virtual print spooler, occasionally the information in AD does not reflect the current state of printers. This post describes some problems I've come across with duplicate/incorrect information and some ideas of how to automatically combat the problem.

Print Queue Objects in AD

Print queue objects in 2003 clustering are named with the virtual print server name, but they are children off a physical computer account. Which computer account the printers are children of is determined by the physical node that owned the cluster spooler resource when the printer was originally published in AD. As a virtual print server fails between nodes, the printer objects in the directory are not re-published (I assume unless the object is not found in the directory).

It's intuitive that print queue objects would be republished on failover to the node that currently owns the spooler, but that could potentially be hundreds or thousands of printer objects being created/deleted with each failover so it's practical not to. It appears the printer object is confirmed using the virtual print server name, and no change is made if the object is found - regardless of which physical node the print queue object is a child of.

In the scenario of a stand-alone printer server, when a printer is deleted, the spoolsv service also removes the directory object. In a clustered virtual print server this also occurs, however, it appears that in a 2003 cluster the object is not automatically removed from the directory if the node that owns the object when deleted is different than the publishing node.

None of this really matters if everything is working perfectly, but in a 2003 MSCS I have seen the following situations:

Print queues that no longer exist still being visible through a search in AD
Duplicate print queue objects, published against each physical none in the cluster that has hosted the virtual print spooler.

The first was a bigger problem, and I believe the following scenario will result in stale print queue objects persisting:

You have a two node cluster, CL01 and CL02. CL01 owns a virtual print spooler and other cluster groups, under which you create all the print queues.
At a later time you decide that the load could be better split, and move the virtual print spooler to CL02
You then clean up your print queues from the virtual server, also expecting that they will be automatically removed from AD.

In the scenario above, the print queue objects would not be removed from AD, as the physical node that owns the spooler (CL02) does not own the original print queue objects - as they were created when CL01 owned the resources. In this state, the invalid print queue objects will not be purged. Note that this is assuming you aren't using AD printer pruning - by disabling the spooler service on your DCs or using Group Policy to control pruning.

I'm unsure of the exact scenario that caused the duplicate print queue objects, presumably there was some problem finding the existing record, so at some point it was created off the other node as well - resulting in duplicate results in a search (both of which would work, but still).

Some low maintenance ideas to correct this problem:

Use AD printer pruning, which will ensure print queue objects in AD are managed. Note that this sounds like the obvious solution, but does have caveats and may not suit all environments.
Periodically remove published records from all but the designated primary node, toggle the published attribute on those printers no longer having a record in AD, causing the printers to be republished against the primary node. This could easily be scripted and scheduled
Modify printer creation change control processes to ensure that new printers are only created and deleted when the preferred owner is hosting the virtual print server

In an ideal world, three above followed by one make the most sense, but if you needed option two you could do something like this:

dsrm CN=%virtual_server%-%QueueName%,CN=%physical_server%,DC=domainRoot
cscript prncfg.vbs -s -b \\%virtual_server%\%QueueName% -published
cscript prncfg.vbs -s -b \\%virtual_server%\%QueueName% +published
dsquery * -limit 0 -filter "(&(objectClass=printQueue)(objectCategory=printQueue))" -attr cn printerName distinguishedname find /i "%QueueName%"

This removes the AD object against the 'incorrect' node, toggles the published flag (using prncfg from the Resource Kit Tools - see 'Network Printing Tools and Settings' reference below), and then queries AD to verify the printQueue object has been created.

Printer Pruning in AD

Pruning of printer objects in Active Directory is controlled either by the server that deletes the printer from its local spooler, or Domain Controllers through periodic printer pruning. Printer pruning is a domain/site-wide activity which processes all printQueue objects.

In a clustered solution, I believe when a Domain Controller looks up the printqueue objects, it will connect to the virtual print spooler node to verify the printers still exist. So regardless of which physical is publishing the printer, as long as the printer is contactable through the virtual server it shouldn’t be pruned.

As long as the spooler service is enabled on at least one Domain Controller, it will prune printers (at the default of 3x8 hour checks). There are risks of doing this, primarily that if the print server is down for longer than 24 hours (or if the DC can’t contact the server), all printers will be pruned from the directory. This logs an Event 50 for each pruned printer in the system event log of the DC that pruned the object - at least it’s easy to trace.

Printer Commands

Query and compare the printers published from each server to determine duplicates:

dsquery * "CN=%physical_server%,DC=domainRoot" -limit 0 -filter "(&(objectClass=printQueue)(objectCategory=printQueue))" -attr cn printerName driverName printCollate printColor printLanguage printSpooling driverVersion printStaplingSupported printMemory printRate printRateUnit printMediaReady printDuplexSupported > CL1.txt
dsquery * "CN=%physical_server%,DC=domainRoot" -limit 0 -filter "(&(objectClass=printQueue)(objectCategory=printQueue))" -attr cn printerName driverName printCollate printColor printLanguage printSpooling driverVersion printStaplingSupported printMemory printRate printRateUnit printMediaReady printDuplexSupported > CL2.txt
for /f "skip=1" %i in (CL1.txt) do @find /i "%i" CL2.txt

The following two commands help identify mismatches in printers published in AD versus those shared through the virtual print server.

Count the number of printers published in AD:

find /i /c "%virtual_server%" CL?.txt

The number of printers shared against a node:

rmtshare \\%physical_server% find /i "\\%virtual_Server%" /c

Query printers published against a physical server:

dsquery * "CN=%physical_server%,DC=domainRoot" -limit 0 -filter "(&(objectClass=printQueue)(objectCategory=printQueue))" -attr cn printerName driverName printCollate printColor printLanguage printSpooling driverVersion printStaplingSupported printMemory printRate printRateUnit printMediaReady printDuplexSupported

References:

Network Printing Tools and Settings
http://technet.microsoft.com/en-us/library/cc778201.aspx

Printer Pruner May Prune All the Print Queue Objects on Its Site
http://support.microsoft.com/kb/246906

Printer Pruner May Not Remove Printer Queue Objects from Active Directory
http://support.microsoft.com/kb/246174/

A server does not prune printers on a Microsoft Windows Server 2003-based server cluster
http://support.microsoft.com/kb/908128

Useful Windows Printer command-line operations:
http://waynes-world-it.blogspot.com/2008/09/useful-windows-printer-command-line.html

VirtualCenter Physical to Virtual

2009-01-19T21:06:00.003+10:00

This post describes a process I executed to replace a physical VMware VirtualCenter box with a virtual equivalent running in its own cluster. There were no VM outages of machines running on the cluster – each ESX kept merrily running the virtuals until everything was sorted out with the VC layer.

Note that this was done in a lab instance of ESX 3.5 and VirtualCenter 2.5, had this been production I probably would have taken a little more care.

I don’t think there is any compelling reason why you wouldn’t run a virtual VC box, it would be hypocritical of VMware to suggest virtualizing your application servers, except VirtualCenter which should be physical. Having said that, this process could also be used in reverse, taking a VM VirtualCenter instance physical should the need arise. This would also be useful for a VirtualCenter disaster recovery scenario.

The configuration before completing these steps

Physical VirtualCenter 2.5 running on Windows Server 2003, called vc01, static IP
Virtual Windows Server 2003 computer running in the cluster, called vc02, dynamic IP \
SQL 2005 database for VC, running on a separate SQL server.
ESX 3.5 VC 2.5, single network, iSCSI shared storage

Prerequisites

The IP address of the VirtualCenter server
The ESX host of the VM becoming the new VC server
The path to your VMware FlexLM license file (assuming you’re using a license server)
The logon details for the SQL connection between vpxd and the database

Steps Taken

On the physical VirtualCenter box that is going to be decommissioned:

Take note of the physical ESX host running the VM becoming the new VC server
Stop the vpxd service and change the startup to manual (sc stop %service%, sc config %service% start= demand)
Stop the vmountVpx, vmware-ufad-vci, vmware-converter and webAccess services change the startup to manual
Stop the flexlm instance and set the startup to manual
Take a backup of your .lic license file
Change the IP address to dynamic (assuming it is static)
Power off the physical machine
Delete the computer account for vc01 from the domain
On the SQL server hosting the VirtualCenter database, backup the VC database and log file with the following command executed through management studio:

BACKUP DATABASE [VMVC]
TO DISK = N'c:\temp\VC_PreMoveToVM.bak'
WITH
DESCRIPTION = N'VirtualCenter Pre-move to VM backup'
, INIT
, NAME =
N'VC Pre move to VM'
GO

RESTORE VERIFYONLY
FROM DISK =
N'c:\temp\VC_PreMoveToVM.bak '
WITH FILE = 1
GO
In the service console of one of the ESX hosts, backup the current certificates (just in case):

mkdir /tmp/cert_backup
cp /etc/vmware/ssl/* /tmp/cert_backup

On the VM becoming the new VirtualCenter server running on a host in the cluster: Rename the server from vc02 to vc01, with the same static IP as the previous vc01
Restart the virtual machine
Install the SQL Native Client - required for VC 2.5 SQL connectivity on a non-SQL server
Install VirtualCenter including FlexLM, connecting to the existing database and using the license file copied off the physical server
In VirtualCenter, disconnect the first ESX host in the cluster - maintenance mode is not possible at this stage – login errors occur because of the incorrect certificate
Copy the new VC certificates - I use pscp here, but whatever you normally use to copy files to ESX would be fine:
1. cd "C:\Documents and Settings\All Users\Application Data\VMware\VMware VirtualCenter\ssl"
2. pscp rui.crt vcadmin@esx01:/etc/vmware/ssl/
3. pscp rui.key vcadmin@esx01:/etc/vmware/ssl/"
In the service console of the ESX node you’ve just updated the certificates on, restart the management interface, the ESX host did not seem to pick up the new certificate dynamically (maybe it would on a schedule without a restart):
1. service mgmt-vmware restart
Connect the ESX host to VC through the VI Client interface
Repeat steps 15-18 for each other ESX host in the cluster
In VirtualCenter, run 'Reconfigure for HA' on each ESX node

Testing
Ensure the vpxd.log file reports no problems with host connectivity or certificates (also check the VC/ESX logs)
Ensure each ESX host is receiving licenses from the 'new' license server, either through the VI client or the FlexLM admin tool.
Ensure you can perform simple tasks such as powering on a virtual machine
Ensure VMotion/HA/DRS is working

The (untested) rollback plan if something goes wrong:

Shutdown the new VirtualCenter VM
Restore the database from the backup taken
Power on the physical VC box, change the IP to the static IP
Restart the vpxd and VMware License Server services
For each ESX host in the cluster, disconnect the host, restore the old certificates, restart the management service and connect the ESX host to the old VirtualCenter instance

Additional notes:

Even though user login errors were returned when the vpxd service tried to form the cluster - which points to the vpxuser account used by VC to manage ESX hosts - this was misleading as this username and password is stored in the VC database – which had not been modified (in the vpx_host table). The next logical step was certificates, which lead to certificate update process used above.
Manually copying the certificates may not be strictly required, as when I went part-way to reconnecting a host without updating the certificates, I was prompted that another VC instance was managing these servers, would I like to continue. Presumably it would have automatically updated the certificates as required.

Virtual 2003 MSCS Cluster in ESX VI3

2009-01-17T09:41:00.002+10:00

This post shares a method I've used to create test-lab instances of standard 2003 file and print Microsoft Cluster Services (MSCS) clusters in a VMware ESX VI3 virtual environment. The resultant solution is not supported and definitely not production-ready, but if you want a real multi-node MSCS cluster in an ESX lab environment, this process might be helpful with a minimum set of requirements.

With my usual theme of repeatable command-line execution, most of these operations can be completed via the command-line, either in the ESX service console or a command-prompt from the virtual MSCS nodes.

I followed bits and pieces of the VMware supported method - which is very specific and quite restrictive. Note that I’m a little dubious that this cluster would be particularly stable – the SCSI reservations MSCS uses to lock disks are in no way supported when using a shared VMDK through a shared SCSI adapter (I think RDM is the only supported method), but it does work and at least provided me with a test environment.

The shared nothing model of 2003 MSCS clustering dictates that only one node accesses the partition at any one time, but the disk still needs to be visible to both nodes. A limitation of this solution is that both MSCS nodes need to be hosted on one ESX server – a requirement you could satisfy with a DRS rule to keep the two nodes together. However, if DRS decided to migrate both VMs, the cluster would almost certainly break during the failover (and possibly after).

If you follow the steps below, you should end up with two virtual x64 2003 enterprise servers, both members of a single MSCS cluster. In the cluster there will three shared disks (VMDKs), one for the quorum and one each for file and print – with a virtual server and relevant cluster resources. A test file share is created, along with drivers and a test printer. You'll need to modify the commands that reference the public adapter and IP addresses

Steps involved:

Create an area for storage of the shared disk on your datastore:
1. mkdir /vmfs/volumes/%datastore%/cluster01
Create a 5GB quorum disk:
1. vmkfstools -d thick -a lsilogic -c 5G /vmfs/volumes/%datastore%/cluster01/MSCS-Quorum.vmdk
Create a 5GB disk for shared data:

vmkfstools -d thick -a lsilogic -c 5G /vmfs/volumes/%datastore%/cluster01/MSCS-disk01.vmdk

Create two 2003 x64 enterprise virtual machines, either through cloning, deployment with templates or whatever your standard build process may be
If cloning was used, run sysprep on both nodes to give a unique SID and join your lab domain
Shutdown the first node and add the shared disk

Add the quorum disk, mounted under scsi 1:0 (which adds a new SCSI adapter)
Set the newly created SCSI Adapter to SCSI bus sharing virtual
Add disk01, attached as scsi 1:1

In the first VM, use disk administrator (or diskpart) to initialise the quorum and disk01 disks, partitioned with basic. Record the signature of the disk and the drive letter used (although this is the disk volume when the disk is owned by the OS, not the cluster).
Add a service account for the cluster service:

dsadd user "CN=clustersvc,CN=Users,DC=test,DC=local" -pwdneverexpires yes -pwd password -disabled no -desc "MSCS VM cluster service account"
Ensure the service account is an administrator of each virtual 2003 node

Use Cluster Administrator to install the cluster on the first node, with your chosen cluster name, using the created quorum disk and service account
Verify correct operation of the single-node cluster, and then add the second VM node to the cluster.
Create a new port group to allow a second private adapter on each ESX server:

esxcfg-vswitch -A MSCS-Private Private
Add a second interface to each VM cluster node, allocated separate address space
Verify connectivity (ping) and configuration following cluster best practices (no gateway, no DNS etc)
Mark as a private heartbeat connection for the cluster, prioritised above the LAN connection.

Create a virtual resource group, creating IP, network name and disk resources in the group, the following commands will create a group called v01, in the lab01 cluster. For these steps, you’ll need the drive letter to use (M: below), the disk signature, the public network name, IP Address and subnet mask of the virtual server being created:

cluster /cluster:lab01 group "v01" /create
cluster /cluster:lab01 res "v01 Disk01" /create /group:"v01" /type:"physical disk"
cluster /cluster:lab01 res "v01 Disk01" /priv Drive="M:"
cluster /cluster:lab01 res "v01 Disk01" /priv signature=0x%disksignature%
cluster /cluster:lab01 res "v01 Disk01" /prop Description="M: disk01"
cluster /cluster:lab01 res "v01 Disk01" /On
cluster /cluster:lab01 res "v01 IP" /create /group:"v01" /type:"IP Address"
cluster /cluster:lab01 res "v01 IP" /priv Network="%publicNetwork%"
cluster /cluster:lab01 res "v01 IP" /priv Address=192.168.10.10
cluster /cluster:lab01 res "v01 IP" /priv SubnetMask=255.255.255.0
cluster /cluster:lab01 res "v01 IP" /priv EnableNetBIOS=1
cluster /cluster:lab01 res "v01 IP" /priv OverrideAddressMatch=0
cluster /cluster:lab01 res "v01 IP" /AddDep:"v01 Disk01"
cluster /cluster:lab01 res "v01 IP" /On
cluster /cluster:lab01 res "v01" /create /group:"v01" /type:"Network Name"
cluster /cluster:lab01 res "v01" /priv RequireKerberos=1
cluster /cluster:lab01 res "v01" /AddDep:"v01 IP"
cluster /cluster:lab01 res "v01" /priv Name="v01"
cluster /cluster:lab01 res "v01" /On

Install ABEUIamd64.msi on each node if Access Based Enumeration is required
To create a test directory, share and ABE resource on the new virtual server on the cluster (v01):

md \\v01\m$\Dir01
cluster /cluster:lab01 res "v01 Dir01 Share" /create /group:"v01" /type:"File Share"
cluster /cluster:lab01 res "v01 Dir01 Share" /priv path="M:\Dir01"
cluster /cluster:lab01 res "v01 Dir01 Share" /priv Sharename=Dir01
cluster /cluster:lab01 res "v01 Dir01 Share" /priv Remark="Dir01 File Share"
cluster /cluster:lab01 res "v01 Dir01 Share" /prop Description="Dir01 File Share"
cluster /cluster:lab01 res "v01 Dir01 Share" /priv security=Everyone,grant,F:security
cluster /cluster:lab01 res "v01 Dir01 Share" /AddDep:"v01"
cluster /cluster:lab01 res "v01 Dir01 Share" /AddDep:"v01 Disk01"
cluster /cluster:lab01 res "v01 Dir01 Share" /On
cluster /cluster:lab01 res "v01 Dir01 ABE" /create /group:"v01" /type:"Generic Application"
cluster /cluster:lab01 res "v01 Dir01 ABE" /priv CommandLine="cmd.exe /k abecmd.exe /enable Dir01"
cluster /cluster:lab01 res "v01 Dir01 ABE" /priv CurrentDirectory="%SystemRoot%"
cluster /cluster:lab01 res "v01 Dir01 ABE" /priv InteractWithDesktop=0
cluster /cluster:lab01 res "v01 Dir01 ABE" /priv UseNetworkName=0
cluster /cluster:lab01 res "v01 Dir01 ABE" /prop SeparateMonitor=1
cluster /cluster:lab01 res "v01 Dir01 ABE" /prop Description="Access Based Enumeration for Dir01 File Share"
cluster /cluster:lab01 res "v01 Dir01 ABE" /AddDep:"v01"
cluster /cluster:lab01 res "v01 Dir01 ABE" /AddDep:"v01 Disk01"
cluster /cluster:lab01 res "v01 Dir01 ABE" /AddDep:"v01 Dir01 Share"
cluster /cluster:lab01 res "v01 Dir01 ABE" /On

Additional shared cluster disks can be created as required, eg:

vmkfstools -d thick -a lsilogic -c 5G /vmfs/volumes/%datastore%/cluster01/MSCS-disk02.vmdk
Add the disks to one node, (scsi 1:2 in this example). Initialise and allocate in the cluster (as in step 7 above)

To create a virtual print server (assuming you’ve mounted disk02 from step 15 for use in the cluster):

cluster /cluster:lab01 group "v02" /create
cluster /cluster:lab01 res "v02 Disk02" /create /group:"v02" /type:"physical disk"
cluster /cluster:lab01 res "v02 Disk02" /priv Drive="P:"
cluster /cluster:lab01 res "v02 Disk02" /priv signature=0x%disksignature%
cluster /cluster:lab01 res "v02 Disk02" /prop Description="P: print01"
cluster /cluster:lab01 res "v02 Disk02" /On
cluster /cluster:lab01 res "v02 IP" /create /group:"v02" /type:"IP Address"
cluster /cluster:lab01 res "v01 IP" /priv Network="%publicNetwork%"
cluster /cluster:lab01 res "v01 IP" /priv Address=192.168.10.11
cluster /cluster:lab01 res "v01 IP" /priv SubnetMask=255.255.255.0
cluster /cluster:lab01 res "v02 IP" /priv EnableNetBIOS=1
cluster /cluster:lab01 res "v02 IP" /priv OverrideAddressMatch=0
cluster /cluster:lab01 res "v02 IP" /AddDep:"v02 Disk02"
cluster /cluster:lab01 res "v02 IP" /On
cluster /cluster:lab01 res "v02" /create /group:"v02" /type:"Network Name"
cluster /cluster:lab01 res "v02" /priv RequireKerberos=1
cluster /cluster:lab01 res "v02" /AddDep:"v02 IP"
cluster /cluster:lab01 res "v02" /priv Name="v02"
cluster /cluster:lab01 res "v02" /On

Create v02 print spooler:

cluster /cluster:lab01 res "v02 Spooler" /create /group:"v02" /type:"print spooler"
cluster /cluster:lab01 res "v02 Spooler" /priv DefaultSpoolDirectory="P:\Spool"
cluster /cluster:lab01 res "v02 Spooler" /prop Description="v02 Print Spooler"
cluster /cluster:lab01 res "v02 Spooler" /AddDep:"v02 Disk02"
cluster /cluster:lab01 res "v02 Spooler" /AddDep:"v02"
cluster /cluster:lab01 res "v02 Spooler" /On

On v02, add a standard Laserjet 4000 retail driver for x64 and x86, run from a cluster node:

rundll32 printui.dll,PrintUIEntry /ia /c \\v02 /m "HP LaserJet 4000 Series PCL6" /h "x64" /v "Windows XP and Windows Server 2003"
rundll32 printui.dll,PrintUIEntry /ia /c \\v02 /m "HP LaserJet 4000 Series PCL6" /h "x86" /v "Windows 2000, Windows XP and Windows Server 2003"

Create a test printer on v02 called printer01 using the LJ 4000 driver, with a record in DNS, published in AD, set to duplex by default, with customised permissions using the standard winprint processor:

dnscmd %DNSserver% /recordadd %zone% printer01 A 192.168.10.100
cscript //nologo portmgr.vbs -a -c \\v02 -p printer01 -h 192.168.10.100 -t LPR -q printer01
cscript //nologo prnmgr.vbs -a -c \\v02 -b printer01 -m "HP LaserJet 4000 Series PCL6" -r printer01
cscript //nologo prncfg.vbs -s -b \\v02\printer01 -h printer01 -l "%Location%" +published
setprinter.exe \\v02\printer01 8 "pDevMode=dmDuplex=2,dmCollate=1,dmFields=duplex collate"
subinacl /printer \\v02\printer01 /grant=%domain%\%group%=F
setprinter \\v02\printer01 2 pPrintProcessor="WinPrint"

References

VMware Support method of running MSCS clusters:
http://www.vmware.com/pdf/vi3_35/esx_3/r35u2/vi3_35_25_u2_mscs.pdf

Implementing an MSCS 2003 server cluster Cluster
http://waynes-world-it.blogspot.com/2008/03/implementing-mscs-2003-server-cluster.html

subinacl 5.2.3790.1180:
http://www.microsoft.com/downloads/details.aspx?FamilyID=E8BA3E56-D8FE-4A91-93CF-ED6985E3927B

Windows Server 2003 Resource Kit Tools:
http://www.microsoft.com/downloads/details.aspx?FamilyID=9d467a69-57ff-4ae7-96ee-b18c4790cffd&DisplayLang=en

Finding duplicate DNS records

2009-01-09T22:21:00.001+10:00

Internal DNS in many organisations is usually a little messy and if tasked with cleaning it, my first step would be to identify duplicate records in DNS - either an IP address that is recorded against multiple host names or a host name with multiple host (A) records. Assuming you have a file containing one record per line, this is fairly simple to report on.

The following examples use dnscmd.exe, which assumes Windows 2000/2003 DNS, but you could use any input, as long as it contains a name and an IP.

The two commands:

dnscmd %dnsServer% /enumrecords test.local @ /additional /continue | find /i /v "Aging:" | find /i "192.168" > DNSRecords.txt
echo. > DuplicateIPs.txt & (for /f "tokens=1,4" %i in (DNSRecords.txt) do @if "%j" NEQ "" @find /i "%j" DuplicateIPs.txt >nul & if errorlevel 1 for /f "tokens=1" %m in ('"findstr /i "%j$" DNSRecords.txt | find /i /c "%j""') do @if %m GTR 1 (@echo %j,%m: & findstr /i "%j$" DNSRecords.txt & echo.) >> DuplicateIPs.txt) & type DuplicateIPs.txt

The first command uses dnscmd to enumerate the records from the root of the test.local zone and exclude aging (dynamic DNS) records. The last find command further filters the output by IP, which can be useful when targeting specific subnets/sites. You might also want to check aging dynamic records instead of static to see how well scavenging is working – if you do, just change the tokens to 1,5 instead of 1,4 (as the aging data is another token separating the name and IP).

The second command:

Creates a new file called DuplicateIPs.txt in the current working directory
Iterates through each line in the DNS record dump, extracting the first and fourth token (Name and IP)
The commands in the first FOR loop check there is a value, then check the value hasn't already been processed in the duplicate list (otherwise you'd have duplicates of each duplicate) and then counts the duplicates and appends them to the file.
The final command types the duplicate IP file created by the for loop iteration.

For example, supposing your DNS export contained the following records:


printer1 3600 A 192.168.10.100
printer2 3600 A 192.168.10.101
printer3 3600 A 192.168.10.100
printer4 3600 A 192.168.10.102
printer5 3600 A 192.168.10.103
printer6 3600 A 192.168.10.100
printer7 3600 A 192.168.10.102

After running the second command above, a file called DuplicateIPs.txt would be created and then typed to the prompt:


192.168.10.100,3:
printer1 3600 A 192.168.10.100
printer3 3600 A 192.168.10.100
printer6 3600 A 192.168.10.100

192.168.10.102,2:
printer4 3600 A 192.168.10.102
printer7 3600 A 192.168.10.102

The command below is modified to report duplicate names instead of IP addresses. This was done by using the first token (%i) instead of the second (%j), and modifying the findstr command to use a literal string search ending with a space rather than the regular expression EOL:

echo. > DuplicateIPs.txt & (for /f "tokens=1,4" %i in (DNSRecords.txt) do @if "%i" NEQ "" @find /i "%i" DuplicateIPs.txt >nul & if errorlevel 1 for /f "tokens=1" %m in ('"findstr /i /c:"%i " DNSRecords.txt | find /i /c "%i""') do @if %m GTR 1 (@echo %i,%m: & findstr /i /c:"%i " DNSRecords.txt & echo.) >> DuplicateIPs.txt) & type DuplicateIPs.txt

If you wanted a summary rather than the detail of each duplicate, you could also run the following command:

echo. > DuplicateIPSummary.txt & (for /f "tokens=1,4" %i in (DNSRecords.txt) do @if "%j" NEQ "" @find /i "%j" DuplicateIPSummary.txt >nul & if errorlevel 1 for /f "tokens=1" %m in ('"findstr /i "%j$" DNSRecords.txt | find /i /c "%j""') do @if %m GTR 1 (@echo %j,%m) >> DuplicateIPSummary.txt) & type DuplicateIPSummary.txt

In the example above, this would produce the following report:


192.168.10.100,3
192.168.10.102,2

I use this sort of command to generate reports on duplicates, in this case from DNS, but it could also be useful in DHCP, WINS, or any number of Active Directory objects/attributes. People (myself included) are often wary of automated processes that make changes, but this is an excellent example of how powerful read-only automated commands can be – you can take thousands of objects and produce a report in seconds to quickly identify inconsistencies in an environment.

Dnscmd Overview
http://technet.microsoft.com/en-us/library/cc778513.aspx

Command-line automation – Echo and macros

2008-12-29T23:35:00.003+10:00

This is the fifth and last in a series of posts containing information on what I consider the building blocks to automate repetitive tasks at the Windows command-line. These components are the for, find, findstr, set, if and echo commands, control files used to control data input, combined with errorlevels, command concatenation, nested loops and if/then/else constructs.

Described in this post is the echo command and how to execute your commands using macros with doskey.

Echo

Echo is one of the most important commands to use when constructing repeatable commands – you should never run a command against multiple objects without first using echo. Some commands can be disastrous without appropriate input checking.

For example, using a wildcard incorrectly in a file spec, instead of putting *.txt, below I’ve put ‘* .txt’ – the wildcard selecting everything in the current directory (with the unfortunate del command for each of those files).
for %i in (* .txt) do echo del %i

Or more commonly – an incorrectly formed control file. The following command is one I’ve used to fix invalidly set UPNs on user accounts in the local directory. This command is quite powerful, and can be used to fix dozens or hundreds of accounts in seconds – but if the input file is not well formed, you could end up breaking each account:
for /f "skip=1 tokens=1,2-3" %i in (NoUPN.txt) do echo dsmod user "%j %k" -upn %i@%upnsuffix%

Incidentally, the command used to generate the ‘NoUPN.txt’ file was:
dsquery * %OU% -filter "&(objectclass=user)(objectcategory=person)(!(userprincipalname=*))" -s %server% -scope onelevel -attr name distinguishedname > NoUPN.txt

If you are untrusting of using the ‘for /f’ to execute operations against multiple objects, you can also use echo to construct a batch file that you can run at a later time – good practice rather than writing and executing the command in a single attempt. This may also useful when you’re modifying hundreds or thousands of objects and you don’t want to execute the commands immediately – for example if the modifications were in Active Directory and you wanted to prevent excessive replication during the day.

Running the command above with an appended redirect (>>) to a file, you could then pass this file to someone else for sanity checking before actual execution of the batch job:
for /f "skip=1 tokens=1,2-3" %i in (NoUPN.txt) do echo dsmod user "%j %k" -upn %i@%upnsuffix% >> %temp%\SetUPN.bat

Doskey macros

In the previous post there was a command to calculate a subnet at the command-line. If you had to type it every time - or even copy and paste - it’s not something you would use (okay, you’d probably never use it anyway), but doskey macros provide a simple method of allowing command re-use.

Creating a macro file

A text file can be created containing your macros, choose a shorthand name for the command, followed by the command you wish to execute. One slightly odd aspect with a macro file is that variables passed at the command-line for the macro use the $ prefix instead of %.

To make macros more useful, I often add two options – one without any parameters with a defined default, and another accepting a parameter to customise the command. For example, the following macro - called ‘AD2003’ - would by default query the local directory for all 2003 computer accounts. If a parameter is passed ($1), the dsquery command passes the parameter as the start node for the search (which could be a DN to limited the containers queried, or a different domain than domainroot).

AD2003=if _$1==_ (dsquery * domainroot -filter "(&(objectCategory=Computer)(objectClass=Computer)(operatingSystem=Windows Server 2003))" -limit 0) else (dsquery * $1 -filter "(&(objectCategory=Computer)(objectClass=Computer)(operatingSystem=Windows Server 2003))" -limit 0)

The find subnet command from the previous post could be shortened to execute the command with something as simple as ‘fs 192.168.11.40 255.255.254.0’, by adding the following to a macro file (and executing on a command processor with delayed expansion enabled):

FS=for /f "tokens=1-8 delims=.- " %i in ('echo $1 $2') do @set /a Octet1="%i & %m" >nul & @set /a Octet2="%j & %n" >nul & @set /a Octet3="%k & %o" >nul & @set /a Octet4="%l & %p" >nul & Echo !Octet1!.!Octet2!.!Octet3!.!Octet4!

Automatically enabling the macro file

It would also be unwieldy if you had to enable the doskey macros each time you ran a command shell. However, you can easily automate this with the autorun key of the shell, stored in either

HKCU or HKLM:
Key - HKEY_LOCAL_MACHINE\software\microsoft\Command Processor
Value - AutoRun
Value Type - REG_SZ
Value Data - doskey /macrofile=c:\util\macros.txt (or wherever)

This concludes what hopefully was a useful description of how I use built-in shell commands to automate repetitive tasks and make my work life easier and more efficient. Some of the resultant commands are not particularly easy to follow, but each can be broken down into the component parts and digested separately.

For more real-world examples of how I use the information in the last five posts, see my command-line operations:
http://waynes-world-it.blogspot.com/2008/09/useful-ntfs-and-security-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-printer-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-mscs-cluster-command.html
http://waynes-world-it.blogspot.com/2008/09/useful-vmware-esx-and-vc-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-general-command-line-operations.html
http://waynes-world-it.blogspot.com/2008/09/useful-dns-dhcp-and-wins-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-active-directory-command-line.html

Command-line automation – set

2008-12-27T22:48:00.004+10:00

This is the fourth in a series of posts containing information on what I consider the building blocks to automate repetitive tasks at the Windows command-line. These components are the for, find, findstr, set, if and echo commands, control files used to control data input, combined with errorlevels, command concatenation, nested loops and if/then/else constructs.

Described in this post is the ‘set’ command, which I find very useful when automating repetitive tasks with ‘for’. I use set in four ways – string manipulation, simple arithmetic operations with ‘set /a’, gathering input with ‘set /p’ or concatenating output in a single for loop with delayed variable expansion.

String Manipulation

My most frequent use of the set command is simple string manipulation at the command prompt. I use this to replace one character with another or extract a substring.

Replace

Replacing one string with another is a core function of string manipulation. Even though this is included under help for ‘set’, this functionality works for other commands, including the echo command.

For example, say you have the fully qualified user DNS name – my.domain.com – and this conveniently is also the distinguished name of your AD domain, the following command replaces periods with ‘,DC=’ and prefixes an additional string to provide the distinguished name:
echo DC=%userdnsdomain:.=,DC=%

This would turn ‘my.domain.com’ into:
DC=my,DC=domain,DC=com

Wildcards can also be used, for example, to remove the first element, you could replace up to the first dot with nothing:
echo %userdnsdomain:*.=%

You could also remove spaces and print the output, for example, if the ‘test’ variable contained ‘A Sentence With Spaces’, running:
echo %test: =%

Would result in:
ASentenceWithSpaces

Substrings

You can extract from existing variables (or inline if using delayed expansion), providing Left/Right/Mid functionality. This can be useful in many ways, but for most of the input I deal with, splitting with ‘for’ and delimiters (such as comma or space) makes for simpler processing with dynamic input. For examples sake, if you had a defined length of padded text, and you wanted the 10 characters starting at the 20th character (0-based), you could run:
echo %input:~19,10%

You could also extract the last 10 characters:
echo %input:~-10%

Or all but the last 10 characters:
echo %input:~0,-10%

Arithmetic operations

‘set /a’ supports a relatively simple but still powerful set of arithmetic, bitwise and shift operators, and while I’ve never found much use for them, upon occasion they have come in handy.

For example, this command will tell you what next year will be and set the NextYear environment variable, based on the current year + 1:
for /f "tokens=3 delims=/" %i in ('echo %date%') do set /a NextYear=%i+1

Similarly, find the current month, increment by one, unless the current month is 12, then wrap around and start at 1 again:
for /f "tokens=2 delims=/" %i in ('echo %date%') do if %i==12 (Set NextMonth=1 & echo !NextMonth!) else (Set /a NextMonth=%i+1)

Note that the example above uses delayed environment variable expansion, see below for a description.

Incidentally, if you were going to use this output for something and if the month were a single digit you would lose the padding, which would then be incorrectly sorted (as compared to other padded output). The example below pads the month, then extracts the last two characters, effectively padding single-digit months (see string manipulation above):
set NextMonth=0%NextMonth%
echo %nextmonth:~-2%

An actual use I have found for the arithmetic operators is to calculate subnets from the command-line. I have made use of this when parsing netsh dhcp output (there were thousands of leases across several DHCP servers serving dozens of subnets, and I wanted to know from the leases which subnets were in use).

This can be done with a single command-line, but it’s a little complicated, the batch file below may be easier to demonstrate the ‘set /a’ functionality. Running this batch file with an IP and mask, eg ‘findsubnet.bat 192.168.11.40 255.255.254.0’ will return the subnet (192.168.10.0 in this example) using bitwise AND of the two sets of octets:



@Echo Off
Set Input=%1
Set Mask=%2
If /i "%Input%"=="" Goto End

for /f "tokens=1-8 delims=.- " %%i in ('echo %Input% %Mask%') do Call :FindOctet %%i %%j %%k %%l %%m %%n %%o %%p
Goto End

:FindOctet
::Echo %1 %2 %3 %4 %5 %6 %7 %8

set /a Octet1="%1 & %5" >nul
set /a Octet2="%2 & %6" >nul
set /a Octet3="%3 & %7" >nul
set /a Octet4="%4 & %8" >nul
Echo %1.%2.%3.%4,%Octet1%.%Octet2%.%Octet3%.%Octet4%,%5.%6.%7.%8
Set Octet1=
Set Octet2=
Set Octet3=
Set Octet4=
Goto End

:InvalidArgs
Echo Invalid arguments, please provide an IP and longhand mask, eg 192.168.11.40 255.255.254.0
Goto End

:End

Input

Set can be used to gather input, making your commands or batch files interactive. For example, the following command assigns the input to the ‘test’ environment variable, after asking the question:
set /p test=Tell me something?

I don’t often use this, as it somewhat defeats the purpose of automation – I prefer to have all the required input in a control file, which separates the intended action from the actual operator.

Set with Delayed expansion

Delayed environment variable expansion provides a method of executing commands and storing the output for use by other commands within a loop – either in different iterations or nested commands. This is required when looping through ‘for’ commands or nesting commands, otherwise state between each command is not maintained.

For example, without delayed expansion, the following would not work:
set test=test variable & echo %test%

However, using delayed environment variable expansion, this problem can be easily overcome, using the !variable! syntax, instead of the normal %variable%. Note that you may need to enable delayed environment variable expansion, which can be done using ‘cmd /v:on’ when starting the command prompt, or running ‘setlocal ENABLEDELAYEDEXPANSION’ within an existing shell.

With delayed expansion, the first command would set the variable and the second echo the newly set variable:
set test=test variable & echo !test!

Or to re-use an example above, calculate next year and then do something with that information (echo in this case):
for /f "tokens=3 delims=/" %i in ('echo %date%') do set /a NextYear=%i+1 1>nul & echo Next year is !NextYear!

Unless there is already a variable of the same name, running the commands above using the percent signed variables will result in the variable name being printed, as the environment hasn’t yet been changed by the first command.

Note that while testing, it seems the Vista command shell has been updated and as long as delayed expansion is enabled, this command would work:
set test=test variable & echo %test%

The following command is a one-line version of the batch above, using delayed variable expansion to calculate the variables as each command is executed:
for /f "tokens=1-8 delims=.- " %i in ('echo 192.168.11.40 255.255.254.0') do set /a Octet1="%i & %m" >nul & set /a Octet2="%j & %n" >nul & set /a Octet3="%k & %o" >nul & set /a Octet4="%l & %p" >nul & Echo %i.%j.%k.%l,!Octet1!.!Octet2!.!Octet3!.!Octet4!,%m.%n.%o.%p

An easier example to follow, set the environment variable called ‘number’ to a random number (using %random% - very useful), echo the newly set variable, and then use ‘set /a’ to increment the number by one:
set number=%random% & echo !number! & set /a number=!number!+1

If you are using nested ‘for’ commands, you can just reference variables from an earlier command, but in some circumstances delayed expansion is the only possibility. I have had to use this syntax when using ‘set /a’ and using the replace/substring functionality, as this requires a full variable (eg. %path%, not a temporary %i type variable.

This was a basic overview of ‘set’ and some of the ways in which I use this command when automating tasks at the command-line.

For more real-world examples of how I use ‘set’, see my command-line operations:
http://waynes-world-it.blogspot.com/2008/09/useful-ntfs-and-security-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-printer-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-mscs-cluster-command.html
http://waynes-world-it.blogspot.com/2008/09/useful-vmware-esx-and-vc-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-general-command-line-operations.html
http://waynes-world-it.blogspot.com/2008/09/useful-dns-dhcp-and-wins-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-active-directory-command-line.html

Command-line automation – errorlevels and if

2008-12-18T15:23:00.004+10:00

This is the third in a series of posts containing information on what I consider the building blocks to automate repetitive tasks at the Windows command-line. These components are the for, find, findstr, set, if and echo commands, control files used to control data input, combined with errorlevels, command concatenation, nested loops and if/then/else constructs.

Described in this post are errorlevels, and if/then/else statements, useful for branching in a single command-line to execute based on the error level set or some other condition.

Using errorlevels, concatenated commands and if/then/else statements provides enhanced functionality, and most complex operations would not be possible on a single command-line without these features.

errorlevels

In the context of command-line automation, I find error levels most useful when trying to cater for both sets of outcomes when executing a command – either what you wanted happened, or it didn’t.

In the previous examples, I’ve used a control file to ping one or more machines and then echoed the IP address of those that were contactable. This is only part of the picture – what about those machines in the list that didn’t respond to a ping? They are simply dropped from the output. This may be what you are after, but normally I would prefer to know the positive and the negative.

To work around this problem, the following command could be used:
for /f "tokens=3 delims=: " %i in ('"ping -n 1 computer | find /i "reply" & if errorlevel 1 echo 0:1:Not-Found"') do echo %i

This will ping ‘computer’, echo the IP Address if found, otherwise echo NotFound. This occurs because the find command sets the errorlevel according to whether the string was found, and then we are concatenating the command with another to check if an errorlevel occurred indicating the string was not found, and echoing an alternate output to be picked up by the body of the for loop.

A simpler way of looking at this may be without the ‘for’ loop:
ping -n 1 computer | find /i "reply" >nul & if errorlevel 1 (echo No Reply) else (echo Replied)

The command is similar to the example above, it will ping ‘computer’, and if the word reply was found in the output, it will print ‘replied’, otherwise it will print ‘no reply’.

Command concatenation and if/then/else

Commands can be concatenated – when running natively or inside a ‘for’ loop – providing a simple method to group commands. For example, looping through a control file, you might want to nslookup and then ping each device:
for /f %i in (c:\temp\control.txt) do nslookup %i & ping %i

This is a simple chain of commands, execute one and then the other. However, provided each command returns a relevant errorlevel, the if/then/else syntax can also be used to execute the second only if the first succeeded.

Unfortunately nslookup does not provide an errorlevel, but the find command does:
ping -n 1 computer | find /i "reply" >nul & if errorlevel 1 (echo No Reply) else (echo Replied)
ping -n 1 computer | find /i "reply" >nul && echo Replied || echo No Reply

The above commands return the same result – they each print a statement depending on whether ‘reply’ was found in the ping output, the second using the shorthand ‘if’ (&&) and ‘else’ (||) syntax.

‘If’ can be used to check the errorlevel (as in the above example), check whether one string equals another, and check whether a file/directory exists.

If ErrorLevel checking

The following examples send an invalid service a stop control, and perform various checks on the errorlevel returned. Note that sc.exe on XP does not return error codes, this was Vista which does at least return 1060 when the specified service does not exist.

Check whether errorlevel (a special variable that can either be enclosed in % or not) was exactly 1060:
sc stop invalidservice >nul & if errorlevel==1060 echo Service not found

Check whether errorlevel was exactly 1060, if so echo ‘service not found’, otherwise echo the errorlevel returned:
sc stop invalidservice >nul & if errorlevel==1060 (echo Service not found) else (echo Error code %errorlevel% returned)

Check whether the errorlevel was 1060 or greater, and not 1061 (therefore only 1060).
sc stop invalidservice >nul & if errorlevel 1060 if not errorlevel 1061 echo Service not found

Check if anything above errorsuccess (0) was returned, echoes an error occurred and then uses ‘net helpmsg’ to return the error description for the specified error code:
sc stop invalidservice >nul & if errorlevel 1 echo An error occurred: %errorlevel% & net helpmsg %errorlevel%

If string checking

If can be used to conditionally check two strings to determine the action taken. Note that if the string could be blank, you’ll need to either append/prepend another character or enclose the string in quotes.

Using the above example of pinging a computer, check the response and branch accordingly (different echoes in this case):
for /f "tokens=3 delims=: " %i in ('"ping -n 1 computer | find /i "reply" & if errorlevel 1 echo 0:1:Not-Found"') do if "%i"=="reply" (echo Ping success) else (echo Ping failed)

Note that ‘if /i’ can be used for a case insensitive comparison.

Additionally, if command extensions are enabled, and the strings in comparison are numeric, both are converted to numbers and additional operators are available for the comparison, for example:
If 9 equ 9 echo Equals
If 5 lss 14 echo Less than

If exist

To determine whether a file or directory exists, the ‘if exist’ syntax can be used. This can be useful when checking flag files or control files exist to avoid encountering unexpected errors.

If exist %windir% echo %windir% exists
If not exist c:\temp\control1.txt (echo Control File does not exist, terminating & break) else (echo continuing)

Some other examples of concatenating commands which help to demonstrate the concepts:

Use ‘sc.exe’ to send a service a stop control, if the service was running (exists, security allows etc) and was sent a stop, sc will return errorsuccess (0), pause for 5 seconds and then restart the service, otherwise echo that the service was not running in the first place:
sc stop "service" && (echo error %errorlevel% & sleep 5 & sc start "service") || echo Service was not running, error %errorlevel%

nslookup www.microsoft.com, find the second address (the first in nslookup is the DNS server address, and echo that address:
for /f "skip=1 tokens=2" %i in ('"nslookup www.microsoft.com 2>nul | find /i "address""') do echo %i

nslookup www.microsoft.com, skip the first DNS server address (the second line of the nslookup output – find /n), return the other address if found, otherwise return ‘Not-Found’ in the %i variable – the second token of the string:
for /f "tokens=2" %i in ('"nslookup www.microsoft.com 2>nul | find /i /n "address" | find /i /v "[2]" & if errorlevel 1 echo Address Not-Found"') do echo %i

Create a file, if the file is found type the contents, and then delete the file if the type succeeded, otherwise report that the ‘type’ command failed:
echo test > %temp%\test.txt & if exist %temp%\test.txt type %temp%\test.txt && del %temp%\test.txt || Echo could not type %temp%\test.txt

This was a basic overview of the errorlevels and if/then/else branching, future posts will build on this with other useful commands and nested ‘for’ loops.

For more real-world examples of how I use errorlevels and if/then/else, see my command-line operations:
http://waynes-world-it.blogspot.com/2008/09/useful-ntfs-and-security-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-printer-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-mscs-cluster-command.html
http://waynes-world-it.blogspot.com/2008/09/useful-vmware-esx-and-vc-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-general-command-line-operations.html
http://waynes-world-it.blogspot.com/2008/09/useful-dns-dhcp-and-wins-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-active-directory-command-line.html

Command-line automation – find and findstr

2008-12-17T12:46:00.004+10:00

This is the second in a series of posts containing information on what I consider the building blocks to automate repetitive tasks at the Windows command-line. These components are the for, find, findstr, set, if and echo commands, control files used to control data input, combined with errorlevels, command concatenation, nested loops and if/then/else constructs.

Described in this post are the ‘find’ and ‘findstr’ commands - unless you want to spend all your time manually creating or filtering control files you will often use these commands when constructing repeatable command-lines.

These two commands are crucial in efficient ‘for’ loop processing, invaluable when filtering a control file or the results of an in-line command. Help on each is available from the command prompt with the ‘/?’ argument.

Find

I usually use find because more often than not I want to filter a string literal – a specific word or combination of characters – from an input string. As often as not, I also use ‘find /v’, returning the lines that don’t match the specified string.

For example, if you have a control file listing servers in several geographic locations, but you are only interested in processing servers in ‘bne’ (case insensitive with ‘/i’), you could run:
find /i "bne" c:\temp\control.txt

Combined with a ‘for’ command, this could be:
for /f "skip=2" %i in ('find /i "bne" c:\temp\control.txt') do echo %i

Note that this line uses the following syntax explained in the previous post on ‘for’:

‘Skip=2’ tells the loop to skip the first two lines – the result header of the ‘find’ command
Inside the brackets I’ve specified to run a command and parse the results, wrapping the command in single-quotes.

The results are that for each server containing the word ‘bne’, the body of the ‘for’ statement will be executed – in this case simply to echo the first token.

You might also want to run the commands against all servers except those in bne, which is the above command repeated with the addition of the ‘/v’ switch to tell ‘find’ to return those lines not containing a match:
for /f "skip=1" %i in ('find /i /v "bne" c:\temp\control.txt') do echo %i

Filtering in-line output with find

Filtering output of a command executed within a ‘for’ loop is syntactically a little more complicated, but makes sense once you get the hang of it. For example, suppose you want to filter the output of a ping command, returning the IP address of those hosts that replied to a ping, you could run:
for /f "tokens=3" %i in ('"ping -n 1 computer find /i "reply""') do @echo %i

This will:

Execute the ping command, pinging the host named computer, filtering the output to find the word ‘reply’ (indicating a successful reply). Note that to use the pipe inside the brackets, the whole string needs to be enclosed in double-quotes – inside the single-quotes.
Using spaces/tabs, choose the third token to return in the first variable - %i. This will be the IP address in the standard ping output returned by XP and Vista.

Unfortunately the third token of the ping output includes a irrelevant colon, so to strip that, we could specify colons and spaces as delimiters in the for loop:
for /f "tokens=3 delims=: " %i in ('"ping -n 1 computer find /i "reply""') do echo %i

This will be better explained in a future post, but you can also nest two ‘for’ commands and for each host in the control file (in bne in this example), ping and return the IP address if there is a reply:
for /f "skip=2" %i in ('find /i "bne" c:\temp\control.txt') do @for /f "tokens=3 delims=: " %m in ('"ping -n 1 %i find /i "reply""') do @echo %i,%m

The output would give you a comma-separated list of server names and IP address of those from the original control file that currently respond to a ping. Theoretically not all that useful with servers, as they’d usually be known IPs and always on, but if you’re wanting to check something on hundreds or thousands of workstations this would give you a point-in-time control file of online machines (assuming name resolution is accurate).

It’s also worth noting that ‘find’ commands can be cascaded – piping the output of one find command to another if you need to further refine the output. For example to find servers in ‘bne’ and then servers in ‘bne’ that also contain ‘vm’:
find /i /c "bne" c:\temp\control.txt find /i "vm"

The possibilities are nearly endless, I use ‘find’ to:

Filter in and out items that I want, ‘find /i’ and ‘find /i /v’ from generic control files that I use for multiple operations, either in-line in a for command, or to create a stand-alone filtered control file
Run commands and then filter out useless or useful information, often headers/footers or other extraneous detail, making it possible to list data, filter that data and run a command against each filtered item in a single command.
Use errorlevels to determine whether the find command was successful or not (explained in a future post). Often the only way of knowing whether a command returned what you were after is the presence or absence of a string – and ‘find’ will tell you this based on the errorlevel (an advantage over findstr which in XP at least does not set the errorlevel).

Note that find also has several other features that are useful in some scenarios:

By default, offline files will be skipped – if you use a file archiving product that sets the offline (‘O’) attribute, these files will not be searched unless you specify the /offline parameter
The ‘find /n’ option will return the line number the match was found, occasionally useful when parsing large or sequenced control files and you want to know where in a file the string was found.
The ‘find /c’ option will return a count of the matches found, instead of the actual matching lines. This can often be useful when determining how many records would be processed. For example, in the example above you could first run the following command to determine how many records would be processed with the for loop:
find /i /c "bne" c:\temp\control.txt

Findstr

These same principles can be used with findstr to filter output – findstr provides advanced searching, with the following benefits in my use of the commands, you can:

Search for more than one string in a single command. This simplifies searches, in the example above, you could search for ‘bne’ and ‘syd’ in a single findstr command.
Use basic regular expressions and/or string literals

In the example above with a control file containing a list of servers with geographic names (or descriptions somewhere on the line), you could run the following command to return servers in both syd and bne and then run the subsequent command against each server:
for /f %i in ('findstr /i /c:bne /c:syd c:\temp\control.txt') do echo %i

Regular expressions are used to group similar items, without expressly providing each possible combination. For example, parsing robocopy log output, to search for new directories that have been created with a regular expression, you could run:
findstr /i /n "^.*New.Dir" Robocopy.log"

You could just use ‘find’ and search for the string ‘new dir’, but this could easily return false positives if a file or directory has the string ‘new dir’ in it. Using the regular expression looking for the start of the line followed by whitespace followed by the ‘New Dir’ is much more precise.

This was a basic overview of the ‘find’ and ‘findstr’ commands, future posts will build on this foundation with other useful commands, error levels, if/then/else statements and nested ‘for’ loops.

For more real-world examples of how I use the ‘find’ and ‘findstr’ commands, see my command-line operations:
http://waynes-world-it.blogspot.com/2008/09/useful-ntfs-and-security-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-printer-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-mscs-cluster-command.html
http://waynes-world-it.blogspot.com/2008/09/useful-vmware-esx-and-vc-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-general-command-line-operations.html
http://waynes-world-it.blogspot.com/2008/09/useful-dns-dhcp-and-wins-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-active-directory-command-line.html

Building blocks of command-line automation – FOR

2008-12-16T15:10:00.004+10:00

This is the first in a series of posts containing information on what I consider the building blocks to automate repetitive tasks at the Windows command-line. These components are the for, find, findstr, set, if and echo commands, control files used for data input, combined with errorlevels, command concatenation, nested loops and if/then/else constructs.

Described in this post is the ‘for’ command, the most important component in command-line automation. This command provides several methods of looping through a list, and running a command against each element in that list. Use the ‘for /?’ to get Microsoft help on this command.

Using the ‘for’ command with one of the syntaxes below provides many benefits, including:

Repeatability – If you save the command you run, it can be re-run several or hundreds of times
Change control and testing – It’s easy to record what you are planning and simple to run the same commands in a test-lab environment. The output of commands can also be redirected to file, making accountability much easier. Using control files also provides a straightforward method of recording the targets of various actions, knowing that you have one master list, and do not risk accidentally missing an entry.
Documentation – Implementation using a series of commands very easily translates to an as-built document – with the added benefit of providing a quicker DR process.
Efficiency – Even though designing the command for the first run may not be as quick as using the GUI, every time after that will usually be much quicker, and previous commands can often be quickly adapted to new tasks.

You can use the ‘for’ command to:

Process a set of filenames or string literals

The filenames or literals can either be directly named in a space-separated set, or you can use wildcards to process more than one file. For example:
for %i in (*.txt) do echo %i
for %i in (test1.txt test2.txt) do echo %i

For example, I would use this syntax if I’m trying to:

Quickly execute something against a group of machines, eg ping each machine:
for %i in (server1 server2 server3 server4) do ping -n 1 %i

Process a series of data files that I have created from another process, eg this uses the regview utility to export the registry entries modified by *.pol files into *.txt:
for %i in (*.pol) do regview %i > %i.txt

Quickly execute the same command with a different variable, eg use the setprinter utility to view all levels of configuration for the specified printer (you could also use for /l in this example):
for %i in (0 1 2 3 4 5 6 7 8 9) do setprinter -show \\server\printer %i

Process a set of directories

The directory names can either be directly named in a space-separated set, or you can use wildcards to process more than one directory. For example:
for /d %i in (%windir%\*) do echo %i
for /d %i in (c:\temp c:\windows) do echo %i

I would use this syntax if I’m trying to do something with each top-level directory, for example:

Report or set ACLs:
for /d %i in (%rootDir%\*) do icacls %i /save %~ni.txt

Rename all the top-level directories to start with a new prefix:
for /d %i in (%rootDir%\*) do ren %i New-%~ni

Process the contents of a text file, line by line

The contents of a file – which I usually refer to as a control file – can be read line-by-line and your command would be run once for each line, substituting tokens from the control file. This provides unlimited capability – construct a control file through any means available and you can then process the entries one-by-one and run a command against that entry.

Note that in Vista at least, just a LF is enough to separate the lines, rather than the Windows standard CR+LF.

For example, assuming you have a control file with a list of servers or workstations, you could:

Lookup the IP address of each machine:
for /f %i in (test.txt) do nslookup %i

Ping each machine:
for /f %i in (test.txt) do ping %i

Remote dir of each machine:
for /f %i in (test.txt) do dir \\%i\c$

I use this constantly to run a command against multiple AD objects, machines, printers, or other network devices, whether the command queries or checks something, or makes a change to each device.

Process the results of a command, line by line

The results of almost any command can be used as the looping mechanism for a ‘for /f’ command, providing an in-memory control file. For example, you could:

Find the local hostname and then nslookup the computer (you could also use %computername% for this):
for /f %i in ('hostname') do nslookup %i

Query the local Active Directory for a list of DCs (server records) and lookup the IP of each DC:
for /f %i in ('dsquery server -o rdn') do nslookup %i

Recursively enumerate a path

It’s possible to recursively enumerate files or directories from a specified starting location, passing each to the body of the for loop. This provides a rudimentary search and response facility, allowing you to traverse a tree looking for objects of a particular type – and then execute something for each found.

For example, you could search from the root of C: for *.txt files, and then report the filename and size (you would just use dir /s if all you wanted to do was echo)
for /r c:\ %i in (*.txt) do echo %i %~zi

Step through a sequence of numbers and execute for each

The ‘for /l’ option allows stepping through a sequence of numbers, passing the number as a parameter to the body of the ‘for’ loop.

I don’t use this method very often, but it would be another method to the setprinter command above:
for /l %i in (1,1,9) do setprinter -show \\server\printer %i

Variable Expansion

When using ‘for’, ‘for /f’ and ‘for /d’ variable references can also be modified to return substrings or additional information. Note that when using ‘for /f’, most of these only make sense if you are processing lists of files or directories, but if you did have a control file with files/paths variable expansion does work as expected.

This substitution can be very useful, particularly when constructing parameters to pass to the command in the body of the ‘for’ loop. For Example:

If you had a number of control files that you wanted to process, outputting the results to a same-named log file:
for %i in (c:\temp\*.txt) do echo Do something with %i > %~ni.log

If the output of a previous command wrapped the results in quotes, but you need to append/prepend something else you can easily remove surrounding quotes:
for /f %i in ('echo "c:\windows"') do echo %~i\temp

Given a list of files, echo those that are zero bytes in size:
for %i in (c:\temp\*.txt) do @if %~zi == 0 @echo %i

Given a full path, split into drive, path and filename:
for %i in (c:\windows\temp\test.txt) do echo %~di %~pi %~nxi

Tokens, delimiters and skipping lines

The simple functionality of the ‘for /f’ command can be extended very easily with three options:

Tokens – By default only the first token is returned in the variable specified. You can change this behaviour to return one or more tokens, eg tokens=2,3,5 or tokens=1-3 would populate %i, %j and %k with the respective tokens
Delimiters – Instead of the normal space and tab delimiters, one or more alternate characters can be specified. For example, you can specify a comma as a delimiter to process as CSV file
Skipping lines – the skip command can be used to skip one or more lines from the start of a control file, useful when trying to skip a header line, or bypass logo information in a command result.

These options can be used individually or as a combination, for example:

Skip the first line of the control file:
for /f "skip=1" %i in (test.txt) do echo %i

Skip the first line, and use comma’s as the delimiter:
for /f "skip=1 delims=," %i in (test.txt) do echo %i

Skip the first two lines, use the second token, separated by comma and space:
for /f "skip=2 tokens=2 delims=, " %i in (test.txt) do echo %i

This was a basic overview of the ‘for’ command, future posts will build on this foundation with multiple commands, error levels, if/then/else statements and nested ‘for’ loops.

For more real-world examples of how I use the ‘for’ command, see my command-line operations:
http://waynes-world-it.blogspot.com/2008/09/useful-ntfs-and-security-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-printer-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-mscs-cluster-command.html
http://waynes-world-it.blogspot.com/2008/09/useful-vmware-esx-and-vc-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-general-command-line-operations.html
http://waynes-world-it.blogspot.com/2008/09/useful-dns-dhcp-and-wins-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-active-directory-command-line.html

Useful PowerShell command-line operations

2008-12-11T22:40:00.002+10:00

The commands below are a small but growing list of powershell one-liners I've used (and when I remember to take note). Note that this may not be the best and almost certainly isn't the only way in most cases.

Each command-line can be copied and pasted at a PowerShell command prompt, or you can use the commands as part of a PS1 script file if you prefer.

Show help on commands such as 'if', regular expressions etc
help about*

Show help about about_Automatic_variables such as $_ $DebugPreference $OFS
help about_Automatic_variables

Run a command and pipe the results to a variable
cscript.exe //nologo script.wsf | Set-Variable -name scriptResults

Run a command and set a variable with the output
$scriptResults = cscript.exe //nologo script.wsf

Run a command and set a variable with the output using the call operator
$scriptResults = & cscript.exe //nologo script.wsf

Filter an object based on an array of strings
$array = "a", "b"; write-output a b c d | select-string -pattern $array -simpleMatch

Reformat today's date to YYYYMMDD
$today = [DateTime]::Now.ToString("yyyyMMdd")

Find local disks except C: using powershell and output to CSV
Get-WmiObject -Namespace root\cimv2 -ComputerName %server% -Query "SELECT * from Win32_LogicalDisk WHERE FileSystem='NTFS' AND Description = 'Local Fixed Disk' AND Name != 'C:'" | export-csv c:\disk.csv

Use the VI Toolkit Powershell snap-in to query for snapshots
Get-VM | Get-Snapshot | export-csv -path c:\temp\VMsnapshots.csv

Use the VI Toolkit Powershell snap-in to query for snapshot information
Get-VM | Get-Snapshot | foreach-object {$out= $_.VM.Name + "," + $_.Name + "," + $_.Description + "," + $_.PowerState; $out}

Start a remote process using Powershell/WMI
$computer = "."; ([WMICLASS]"\\$computer\root\CIMv2:win32_process").Create("notepad.exe")

Find remote processes and the command-line parameters with PowerShell
Get-WmiObject win32_process | Format-Table ExecutablePath,Caption,CommandLine,CreationDate,WorkingSetSize,ProcessId

Generate a password 10 characters in length, at least 5 punctuation marks
[void] [System.Reflection.Assembly]::LoadWithPartialName("System.Web") ; [System.Web.Security.Membership]::GeneratePassword(10,5)

Format a date to XML XSD:dateTime
$Now = [DateTime]::Now.ToString("yyyy-MM-ddTHH:mm:ss")

Check if the given variable is an object
if ($variable -is [object]) {}

Check which network connections (drive mappings) a 2003 computer has
get-WMIObject -computerName "$env:computername" -class win32_logicaldisk -property ProviderName, DeviceID -filter "DriveType=4" | Format-List -property ProviderName, DeviceID

See whether an IP can be resolved to a host name
[System.net.Dns]::GetHostEntry($IP)

See whether a host name can be resolve to an IP
$ip = & { trap {continue}; [System.net.Dns]::GetHostAddresses($computer) }

View the datastore information for the available storage
get-vc -server $server ; Get-Datastore

Check if a file or path exists or not
test-path -path $path ; if (!(test-path -path $path)) {write-output "$path does not exist"}

Use psexec to run a powershell command remotely and report to stdout (eg query 'remote' event logs in this example)
psexec \\machine /s cmd /c "echo. | powershell . get-eventlog -newest 5 -logname application | select message"

Wayne's World of IT (WWoIT), Copyright 2008 Wayne Martin.

MSCS 2003 Cluster Virtual Server Components

2008-12-11T22:07:00.004+10:00

This post provides my interpretation of a simple MSCS 2003 virtual server with a file share, including how the cluster interacts with the OS and network services to provide access to the share. This follows on from the last post on low-level detail of file access in an attempt to provide a clearer picture of these often taken-for-granted components.

Note that this is only my opinion, based on less-than complete knowledge and more than likely contains semantic errors if nothing else.

File & Print Cluster Native x64 (EM64T/AMD64)

Cluster Service. Includes Checkpoint Manager, Database Manager, Event Log Replication Manager, Failover Manager, Global Update Manager, Log Manager, Membership Manager, Node Manager.

Operating System Interaction with the LanManServer Service which advertised shares.
NetBIOS registration of the virtual server name through existing network services
DNS registration of the virtual server name through existing network services
Kerberos SPNs registered against an AD computer account through Active Directory

Resource Monitor. Spawned child process of the cluster service, separate resource monitors can exist for resource DLLs
ClusRes.dll Physical Disk <-> IsAlive/LooksAlive SCSI reservation and directory access. LooksAlive issues a SCSI reservation every 3 seconds through ClusDisk.sys against all managed disks. IsAlive performs a ‘dir’ equivalent
ClusRes.dll Network Name <-> IsAlive/LooksAlive check on NetBT/DNS registration. LooksAlive relies on MSCS NIC failure detection. IsAlive queries local TCP/IP stack for virtual IP and the NetBT driver if NetBIOS is enabled
ClusRes.dll IP Address <-> IsAlive/LooksAlive check on cluster NIC. LooksAlive queries NetBT driver and every 24 hours issues a dynamic DNS host record registration. If ‘DNS is required’ resource will fail if DNS registration fails. Same test for IsAlive
ClusRes.dll Resource DLL File Shares <-> IsAlive/LooksAlive check on file share visibility. LooksAlive queries lanmanserver service for the share name. IsAlive does the same, and if unlimited users, the first file on the share is copied
32-bit Resource Monitor WOW64, Eg. Enterprise Vault Cluster application. Third-party cluster resources, eg Enterprise Vault, which in this case notifies the local FSA placeholder service on each physical node of virtual server changes
ABE enabled by generic cluster resource. Access based enumeration with a generic cluster application running abecmd.exe during virtual server/share creation. Uses 32-bit cluster resource monitor with WOW64, setting SHI1005_FLAGS_ACCESS_BASED_DIRECTORY_ENUM (0x0800) flag set on the otherwise standard share.

Pretty picture view:

References

Server side processes for simple file access
http://waynes-world-it.blogspot.com/2008/11/server-side-process-for-simple-file.html

Access Based Enumeration
http://technet.microsoft.com/en-us/library/cc784710.aspx

SHARE_INFO_1005 Structure
http://msdn.microsoft.com/en-us/library/bb525404.aspx

Wayne's World of IT (WWoIT), Copyright 2008 Wayne Martin.

Server-side process for simple file access

2008-11-07T17:06:00.005+10:00

This post provides my interpretation of what happens at a low level when a user on a workstation tries to access a file on a server - in this case a Windows server 2003 x64 MSCS cluster. I was trying to demonstrate the complexity of what seems like such a simple action, and in particular trying to incorporate the cluster network/disk elements and highlighting the WOW64 side of things when you are running x64 with x86 third-party software (such as archiving, quotas etc).

Note that I'm reasonably confident this isn't correct because my understanding if lacking, combined with lack of time/information, but even if it were accurate in content, the step-by-step / flowchart views aren't the best way to represent these multi-layered processes. But anyway, somebody else might also think it interesting (nerds!)

Workstation redirector file access to a virtual node in the cluster. Includes DNS/NetBIOS calls to determine the cluster virtual server IP address
LanManServer service. User-mode LAN Manager 2.x server service, providing file and print sharing, and with 2003 SP1, Access Based Enumeration
NDIS layer. Network Driver Interface Specification, Hardware interrupts passing frames to the NIC driver, then passed to the bound transport driver. send and receive raw packets, includes LLC
TDI Layer. Single interface for upper-level clients to access transport providers, such as TCP/IP
ClusNet.sys Driver. Cluster specific driver interpreting and routing intracluster traffic and determining communication failure
Srv.sys Server Service. SMB file server driver, kernel-mode companion to the LanManServer service
I/O Manager. I/O support routines, I/O Request Packets (IRPs) and Fast I/O interfaces
FSD Manager. File System Driver Manager, loads FSDs and legacy file system filters, interacts with the file system cache manager.
Filter Manager. A file system filter that provides a standard interface for minifilters, managing altitudes and connectivity to the file system stack. Interacts with the file system cache manager, and in the case of x86 filters, an instance of WOW64 controls the runspace for the filters (on an x64 platform)
File System Cache Manager. A file system filter, working with kernel memory-mapped and paging routines to provide file level caching
ntfs.sys File System FSD. Windows NT File System driver, creates a Control Device Object (CDO) and Volume Device Object (VDO)
ClusDisk.sys upper-level storage filter driver. Cluster specific storage filter driver maintaining exclusive access to cluster disks using SCSI reserve commands
Volume Snapshot Volsnap.sys. Manages software snapshots through a standard storage filter
Volume Manager ftDisk.sys. Presents volumes and manages I/O for basic and dynamic disk configurations
Partition Manager Partmgr.sys. Filter driver that sits on top of the disk driver, creates partition devices, notifies volume manager and exposes IOCTLs
Class Driver disk.sys. Presents a standard disk interface to the storage stack, creating a Function Device Object (FDO)
Storage Port Driver - Storport. Assists with PnP and power functionality, providing a Physical Device Object (PDO) for the device->bus connection
Miniport Driver. Interface to the storage adapter’s hardware, combining with the storport driver to create the storage stack
SMB response to the client. SMB response to the redirector on the workstation requesting the file

Pretty picture view:

OpsMgr 2007 performance script - VMware datastores

2008-11-02T17:04:00.006+10:00

This post provides a method of collecting VirtualCenter datastore information for ESX hosts in one or more clusters as performance data in Operations Manager 2007. You can then trend datastore usage for longer term analysis, and alert on the data for proactive problem management. This is my first attempt at using script to gather performance data in OpsMgr 2007, and my lack of OpsMgr knowledge combined with the limited documentation/examples made this difficult and possibly harder than it needs to be.

Background

In OpsMgr 2007, I thought it would be useful to report on LUN disk space on the ESX side of things in a similar fashion to agent-based Windows disk performance counters, and while this sounds quite easy, I couldn’t find a simple method because:

ESX doesn’t seem to support an SNMP trap to report on VMFS volume capacity/free space. You could write a shell script and add a cron job on the Linux ESX service console, but this would be targeted at each ESX server, and the results would then be duplicated (and wouldn't work with ESX 3i)
VirtualCenter doesn’t appear to support a custom alert based on storage usage, only storage transfer rates.
The ESX servers are only in OpsMgr as network devices, not true agents, so we can’t use WBEM or any other method to query directly. If the upcoming OpsMgr 2007 cross platform extensions work on ESX, this may be easier, although as with option 1 this would still return duplicate datastore information for each ESX server in a cluster and wouldn't work with ESX 3i)

This left a few options to gather the data in OpsMgr:

Query the VMFS storage through the VI snap-in and a PowerShell Script.
Query the VC database directly for this information from a VBS or PowerShell script.
Create a DTS job on the VC SQL server to extract the data to a suitable format.

Options two and three above would not be supported by VMware and may change as the database structure changes between VC versions.

Solution

Querying the VMFS storage from PowerShell and collect the information with VBScript, which includes:

The PowerShell script using the VI snap-in to extract the datastore information
The VBScript to gather the information as performance data in Operations Manager 2007
A trigger to execute the powershell script and put the data in a file to be gathered.
Rules in OpsMgr to gather the data using the type 'Script (Performance)'.

Steps taken:

Install the VMware VI PowerShell snap-in to your Virtual Centre box (you may need to install PowerShell first). VMware-Vim4PS-1.0.0-113525.exe is the binary I used.
Create an AD group and user for rights to VC. Add the user to the group.
Set rights in VirtualCenter to allow the group read-only rights at the root. This should be more granular if possible.
Add a recurring trigger on your VC box to run the PowerShell script to extract the information. Eg, this could be a scheduled task that runs a batch file or directly executes:
1. powershell . ".\PerfGatherVMwareDSSpace.ps1"
Create two rules in Operations Manager, Script (Performance) to gather the daily log file and store in the database as performance data. One rule is for the free space, another is for the free space as a percentage of the capacity. See the performance mapper notes below.
1. The rules were created under ‘Windows Server 2003 Computer’, not enabled by default and overridden for the VC server (where the powershell script runs). You may want to target the rule differently, depending on your environment.
Create a performance view, showing performance data collected by the two rules above.
Create a monitor, using the LogicalDisk ‘% Disk Free’ object and counter to monitor on a threshold of 10%. Similarly, the monitor could be created under ‘Windows Server 2003 Computer’, not enabled by default and overridden for the VC server. The alert contains the following information:

Object: $Data/Context/ObjectName$, Counter: $Data/Context/CounterName$, Instance: $Data/Context/InstanceName$, Value: $Data/Context/Value$

Note that when using the script performance data provider, the ‘Performance Mapper’ tab is used to map the data returned by the script to the database. To make this generic, both the instance name and the value are used from each element:

Object: LogicalDisk
Counter: Free Megabytes
Instance: $Data/Property/@Name$
Value: $Data/Property[@Name]$

Note that the VBScript creates three typed ‘property bags’ and returns all. This results in a single XML that contains three dataitem elements, one for each instance. Creating a single property bag and adding the three instances results in only the first being processed.

This should now gather free space in MB and as a percentage of the size for all VMFS datastores your VC server knows about, visible through the performance view, and alerted on when free space is less than 10% of the capacity for each volume.

SQL Queries

Some SQL queries against the OperationsManagerDW database I used along the way to query from the datawarehouse that the data was being gathered. Note that where I've specified 'datastore%' as a filter, you'll need to change this to the prefix of your datastores, eg ds01, ds02 would be ds%.

/*Return the raw performance data collected for the datastore* instances: */

select DateAdd(Hour, 10, DateTime) as DateTime, InstanceName, SampleValue from perf.vperfRaw
inner join vPerformanceRuleInstance on perf.vperfRaw.PerformanceRuleInstanceRowID = vPerformanceRuleInstance.PerformanceRuleInstanceRowID
where InstanceName like 'datastore%'
order by datetime desc

/* Find new rules: */
select top 10 * from dbo.vPerformanceRuleInstance
where instancename like 'datastore%'
order by performanceruleinstancerowid desc

/* Find new raw performance data: */

select top 100 DateTime, SampleValue, ObjectName, CounterName, FullName, Path, Name, DisplayName, ManagedEntityDefaultName from perf.vperfraw
inner join vPerformanceRule on perf.vperfraw.PerformanceRuleInstanceRowID = vPerformanceRule.RulerowID
inner join vManagedEntity on perf.vperfraw.ManagedEntityRowID = vManagedEntity.ManagedEntityRowID
order by datetime desc

/* Find new rule instances that have been created: */

select top 10 * from dbo.vPerformanceRuleInstance
order by performanceruleinstancerowid desc
--


#PowerShell Script - PerfGatherVMwareDSSpace.ps1
# Note that this includes a context to connect to VC, with hardcoded username and password.  You could avoid this by running the scheduled task under the security context you created with inherent rights to VC, and then remove the explicit -credential argument to Connect-VIServer.

$ErrorActionPreference = "SilentlyContinue"
add-pssnapin VMware.VimAutomation.Core

$ADMINLOG = "c:\admin\logs"
$outputFile = ""
$today = [DateTime]::Now.ToString("yyyyMMdd")

$scriptName = $MyInvocation.MyCommand.Name
$scriptName = $scriptname.substring(0, $scriptname.LastIndexOf("."))

if ($env:adminlog -ne $null) {        # Was there an adminlog environment variable?
    $outputFile = $env:adminlog
} else {
   $outputFile = $ADMINLOG        # No, use the constant default  
}

$outputFile += "\" + $scriptname + "_" + $today + ".csv"    # Construct the full path to the output file 

$server = "vc01"          # VC instance
$username = "domain\user"        # Hardcoding is bad, but at least this is a RO account.
$password = "password"

$pwd = convertto-securestring $password -asplaintext -force

$cred = New-Object Management.Automation.PSCredential ($username, $pwd)   # Create the credentials to use with the VC connection

$vmserver = & { trap {continue}; Connect-VIServer -server $server -Credential $cred } # Connect to VC, trapping the error

if ($vmserver -is [System.Object]) {       # Do we have a connection?
    Get-Datastore -server $vmserver | export-csv -noTypeInformation -path $outputFile # Yes, get the datastore details and store as CSV
    if (test-path -path $outputFile) {
        write-output "Datastore details exported to $outputFile"
    } else {
        write-output "Error: Datastore details were not exported to $outputFile"
    }
    $vmserver = $null
} else {
    write-output "Could not connect to $server"
}


#------



' VBScript


' Parse a CSV file containing the VMware volume information, and return the specified field from the data file or the calculated percent free

Const TOKEN_DATE = "%date%"
Dim SOURCE_FILE : SOURCE_FILE = "c:\admin\logs\PerfGatherVMwareDSSpace_" & TOKEN_DATE & ".csv"  ' Today's log file exported from the PowerShell VI-snapin script

Const DELIMITER = ","           ' CSV data file
Const ForReading = 1
Const DISKTYPE_VMFS = "VMFS"          ' We're interested only in lines with VMFS volumes

Const QUERY_SIZE = "Size"
Const QUERY_FREE = "Free"
QUERY_PERCENT = "Percent"

Dim QUERY_DEFAULT : QUERY_DEFAULT = QUERY_FREE 

Const FIELD_SIZE = 1           ' The field in the CSV that contains the capacity of the VMFS volume
Const FIELD_FREE = 0           ' The field in the CSV that contains the free disk space on the VMFS volume
Const FIELD_PERCENT = 2

Const PerfDataType  = 2

Set oFSO = CreateObject("Scripting.FileSystemObject")

Main()

wscript.quit(0)

Sub Main()

 If WScript.Arguments.UnNamed.Count >= 1 Then
  strQueryType = WScript.Arguments.UnNamed(0)
  wscript.echo "Command-line argument passed, querying for " & strQueryType
 Else
  strQueryType = QUERY_DEFAULT
  wscript.echo "No command-line argument passed, querying for the default of " & strQueryType
 End if

 Select Case strQueryType
  Case QUERY_SIZE  strField = FIELD_SIZE 
  Case QUERY_FREE  strField = FIELD_FREE
  Case QUERY_PERCENT strField = QUERY_PERCENT
  Case Else  strField = FIELD_FREE
 End Select

 dtmDate = Now 
 strToday = DatePart("yyyy", dtmDate)         ' YYYY
 strToday = strToday & String(2 - Len(DatePart("m", dtmDate)),"0") & DatePart("m", dtmDate) ' MM
 strToday = strToday & String(2 - Len(DatePart("d", dtmDate)),"0") & DatePart("d", dtmDate) ' DD

 strDataFile = Replace(SOURCE_FILE, TOKEN_DATE, strToday)     ' Build the path to the data file
 wscript.echo "Looking for " & strDataFile

 Dim oAPI, oBag
 Set oAPI = CreateObject("MOM.ScriptAPI")

 If oFSO.FileExists(strDataFile) Then        ' Does the data file exist for today?
  WScript.Echo "Log file found, continuing"
  Call ReadFile(strDataFile, strBuffer)       ' Yes, read the contents of the file into the buffer

  For Each strLine in Split(strBuffer, vbCRLF)      ' For each line
   arrEntry = Split(strLine, DELIMITER)      ' Split the line into an array on comma
   If UBound(arrEntry) = 5 Then       ' Did this line have a valid number of fields?
    If strComp(arrEntry(3), DISKTYPE_VMFS) = 0 Then    ' Yes, is it a VMFS volume? (excludes the header)
    wscript.echo "Processing " & strLine
     If IsNumeric(arrEntry(FIELD_FREE)) AND IsNumeric(arrEntry(FIELD_SIZE)) Then     ' Yes, is the field we're after a numeric?
      Set oBag = oAPI.CreateTypedPropertyBag(PerfDataType)  Create a typed name/value pair property bag

      If (strField = FIELD_SIZE) OR (strField = FIELD_FREE ) Then
       Call oBag.AddValue(arrEntry(5),CLng(arrEntry(strField))) ' Yes, convert to a long and add to the bag
      ElseIf strField = QUERY_PERCENT Then
       dblFreePercent = CDbl(arrEntry(FIELD_FREE) / arrEntry(FIELD_SIZE)*100)
       Call oBag.AddValue(arrEntry(5), Round(dblFreePercent, 2))
      End If
      oAPI.AddItem(oBag)     ' Add the item to the bag (doing it here results in three datatime elements)

     End If  
    End If
   End If
  Next

 Else
  WScript.Echo "Error: Daily data file not found - " & strDataFile 

 End If

 Call oAPI.ReturnItems 

End Sub    


'********************************************************
' Purpose: Read a file and store the contents in the buffer
' Assumptions: oFSO exists
'              strLogFile contains the path/filename of the file to operate on
' Effects: strBuffer is by reference
' Inputs: strFileName, Path and filename of the source file
'   strBuffer, the buffer used to store the contents of the file
' Returns: None
'
'********************************************************
Sub ReadFile (ByVal strFileName, ByRef strBuffer)   
 On Error Resume Next
 Dim objTextStream

 If Not oFSO.FileExists(strFileName) Then
  WScript.Echo "Error: " & strFileName & " file not found."
  Exit Sub
 End If
    Set objTextStream = oFSO.OpenTextFile(strFileName, ForReading)
 strBuffer = objTextStream.ReadAll
End Sub

'------

References:

How to Create a Probe-Based Performance Collection Rule in Operations Manager 2007
http://technet.microsoft.com/en-us/library/bb381406.aspx

MOMScriptAPI.CreatePropertyBag Method
http://msdn.microsoft.com/en-us/library/bb437556.aspx

XPath Examples:
http://msdn.microsoft.com/en-us/library/ms256086(VS.85).aspx

http://blogs.msdn.com/mariussutara/archive/2008/01/24/momscriptapi-createtypedpropertybag-method.aspx

http://blogs.technet.com/kevinholman/archive/2008/07/02/collecting-and-monitoring-information-from-wmi-as-performance-data.aspx

http://www.afinn.net/2008/07/collecting-performance-data-in-operations-manager-2007-and-publishing-to-sharepoint-part-1/

http://blogs.msdn.com/mariussutara/archive/2007/11/13/alert-description-and-parameter-replacement.aspx

Wayne's World of IT (WWoIT), Copyright 2008 Wayne Martin.

Enumerating URLs in Internet Explorer

2008-11-01T22:46:00.003+10:00

Have you ever wanted to get a list of the URLs you are currently browsing in Internet Explorer? Probably not, but occasionally I've got a large number of IE windows open and I realise I need to close them all, but I would like to know the pages I'm in the middle of browsing.

The PowerShell and VBScripts below do just that - enumerate the current iexplore.exe windows, and report the URL and name of any identified iexplore.exe windows.

This is also good for gathering references, if you've got many pages open and you'd like to record the URLs for later reference, I find this script easier than repeated copy/paste.



# PowerShell

$shell = new-object –com Shell.Application

$windows = $shell.Windows()

write-output ($windows.count.ToString() + " windows found")
foreach ($window in $windows) {
  if ($window.FullName -like "*iexplore*") {
    write-output ($window.LocationURL + ", " + $window.LocationName)
  }
}

$shell = $null



' VBScript
' Find the URLs of the currently running Internext Explorer Windows

' References:
'  http://windowssdk.msdn.microsoft.com/en-us/library/ms630310.aspx
'  http://msdn.microsoft.com/library/default.asp?url=/workshop/browser/webbrowser/reference/objects/internetexplorer.asp

Const IE_EXE = "iexplore.exe"

Call FindCurrentURLs(strURLSet)
WScript.Echo strURLSet

wscript.quit(0)

Function FindCurrentURLs(ByRef strURLSet)
 Dim objShell, objWindowSet, objWindow
 Dim strwindowName, strURL, strFullName

 Set objShell = CreateObject("Shell.Application")    ' Create a Windows shell automation object
 Set objWindowSet = objShell.Windows      ' Get the collection of open windows belonging to the shell

 Wscript.Echo "Processing " & objWindowSet.Count & " windows"   ' Report how many instances were found

 For Each objWindow in objWindowSet      ' For each InternetExplorer object in the ShellWindows set
  strFullName = objWindow.FullName     ' Get the full path and executable of this window
  If InStr(1, strFullName, IE_EXE, 1) <> 0 Then    ' Is this an IE shell object?
   strURL = objWindow.LocationURL     ' Get the URL

   If strURL <> "" Then 
    strURLSet = strURLSet & vbCRLF & strURL   ' Append to the set of URLs
   End If
  Else         ' No, probably explorer.exe skip
   WScript.Echo "Skipped " & strFullName & " - not IE"
  End If
 Next

 If Len(strURLSet) >= Len(vbCRLF) Then strURLSet = Right(strURLSet, Len(strURLSet) - Len(vbCRLF)) ' Strip the leading vbCRLF
 
 Set objShell = Nothing
 Set objWindowSet = Nothing : Set objWindow = Nothing
End Function

NTLM Trusts between 2003 and NT4

2008-10-26T10:00:00.003+10:00

This post includes a few notes and example on an NTLM trust between two domains in a scenario I was working on - creating a trust between a 2003 user domin and an NT4 resource domain.

I realise NT4 was a long time ago, but NT 4.0 was one end of a trust I set up earlier this year (2008), so other people might be in a similar situation. Note that to have a forest trust - even if it's only a one way trust - both the domain and the forest have to be in Windows 2003 functional mode, so whenever you're talking about trusts between external domains/forests, if one side is NT4 or 2000 then it's an NTLM trust.

The types of access through the trust that will need to be considered:

Cross-domain resource access, including pass-through authentication, Trusted Client->Trusting Server->Trusting DC->Trusted DC
Authenticating access requests for a cross-domain object, Trusting DC -> Trusted DC
Using the object picker and retrieving a list of trusted domains and trusted accounts also results in PDC->PDC communication between trusting and trusted.
Establishing the trust, the PDC of the trusting NT4 domain will try and resolve and then query the PDC of the trusted domain. TCP and NetBIOS sessions will be established, with named pipes, SMB and RPC. A combination of LSA RPC and Netlogon RPC
Sometimes there may be a need to control the DC Locator process on the NT4 DCs to ensure that only the single DC is contacted to authenticate cross-domain access requests. This will be a local lmhosts #PRE #DOM: entry, or a static WINS record, but this would only work when controlling authentication, some other operations require PDC emulator connectivity.

How pass-through authentication across domains between 2003 and NT4 typically works:

Client from 2003 sends a request to the resource server in the NT4 domain
NT4 resource domain server passes the request to a NT4 DC in the local trusting domain
The NT4 DC checks the request is from a trusted domain, then establishes NetBIOS/SMB sessions with a DC in the trusted domain
Netr netlogon RPCs to create a secure channel and authenticate the samlogon request

If you need network security between the trusts, I recommend this be completed in a lab first. Otherwise you will probably end up with the connection between here and there looking like swiss cheese, or at the very least having larger holes than really required. VMware is great for creating a test-lab for this type of exercise, you could reproduce your existing production configuration and monitor the traffic going between various locations - this also requires something of a test network though.

Note that this information can also be used in concert with the 'Replica Domain for Authentication' post to create a trust with a replica of your production domain, reducing the risk of exposing your domain.

Both the RPC and SMB will be over NetBIOS when coming from NT4:

nbname 137/UDP
nbname 137/TCP
nbdatagram
138/UDP
nbsession 139/TCP

In the scenario I was working on, the applications in the resource domain were either accessed via windows authenticated HTTP or direct SQL connectivity. Two examples below show the conversation in authenticating over the trust, between an XP member of the 2003 domain to a member server in the NT4 domain.

Assuming there’s a 2003 member server running IIS using windows authentication in the NT 4.0 resource domain, and XP clients from the corporate forest are accessing the application directly:

Internet Explorer from the workstation initiates a HTTP session with the web server
HTTP 401 is returned by the web server, indicating that authentication is required, with the WWW-Authenticate response headers indicating Negotiate and NTLM are the available schemes.
NTLM Negotiate (0x00000001) is then sent from the client to the server indicating supported NTLM options
The web server responds with a challenge message (0x00000002) for the client to prove their identity
The workstation responds with an authenticate message (0x00000003) – an encrypted challenge response based on the logged on users’ password hash
A Netlogon RPC call is initiated from the web server to the NT 4.0 DC the server has its secure channel with to initiate the samlogon request, providing the username, domain name, challenge, and challenge-response.
The NT 4.0 DC checks the information and determines that it has a trust with the named domain.
The NT 4.0 DC establishes a secure channel with the trusted replica domain (or uses the existing channel)
A second Netlogon RPC call is then passed through to the replica 2003 domain using the same information as the first call (wrapped in NetBIOS in this case)
The replica DC retrieves the hash of the specified user’s password, the original challenge is then encrypted using this hash and compared to the challenge-response provided with the request.
Assuming passwords are synchronised, the hashes match and the trusted netlogon call is successful
The Netlogon call from the web server is also returned as successful
The web page is served to the XP workstation

Connecting across domains to SQL in NT4 from an XP workstation in 2003:

The user starts the local MDE (which has an embedded connection to remote SQL)
The workstation initiates a connection over TCP 1433 to the SQL server
The workstation tries to verify the name/IP of the SQL server, with a reverse lookup DNS query and a NetBIOS Adapter Status Request to the IP over UDP 137 NBNS.
The local Active Directory KDS is queried to see if there is a matching SPN for the requested SQL service. This is unsuccessful and the authentication protocol falls back to NTLM.
A TDS NTLMSSP challenge/response occurs between the workstation and the cross-domain SQL server, including:
a. The negotiate response from the workstation contains the client name and the database (0x1)
b. The NT4 challenge from the SQL server (0x2)
c. The authenticate response messages, containing response to the challenge using the currently logged in credentials (of the XP workstation)
A Netlogon RPC call is initiated from the SQL server to the NT4 DC the server has its secure channel with to initiate the samlogon request, providing the username, challenge, and challenge-response.
The NT4 DC checks the information and determines that it has a trust with the named domain (the 2003 user domain)
The NT4 DC establishes a secure channel with the trusted domain
A second Netlogon RPC call is then passed through to a trusted DC using the same information as the first call
The NT4 DC retrieves the hash of the specified user’s password, the original challenge is then encrypted using this hash and compared to the hash provided with the request.
Assuming the password is correct, the hashes match and the trusted netlogon call is successful across the trust.
The Netlogon call between workstation SQL server is also returned as successful

References

Replica Domain for Authentication
http://waynes-world-it.blogspot.com/2008/08/replica-domain-for-authentication.html

How Domain and Forest Trusts Work
http://technet2.microsoft.com/windowsserver/en/library/f5c70774-25cd-4481-8b7a-3d65c86e69b11033.mspx?mfr=true

Trust Technologies
http://technet2.microsoft.com/windowsserver/en/library/9d688a18-15c7-4d4e-9d34-7a763baa50a11033.mspx?mfr=true

Establishing Trust Relationships
http://www.microsoft.com/technet/archive/winntas/plan/capacityplanning/a03_trst.mspx?mfr=true

NTLM user authentication in Windows
http://support.microsoft.com/kb/102716

Direct hosting of SMB over TCP/IP
http://support.microsoft.com/KB/204279

How to configure a firewall for domains and trusts
http://support.microsoft.com/kb/179442

Windows NT 4.0 tries to resolve manually-typed names by contacting the PDC for the remote user's domain (UDP 138). If that communication fails, a Windows NT 4.0-based computer contacts its own PDC, and then asks for resolution of the name.

Describes some of the network traffic between 2003 trusts, including methods of setting the LSA and Network RPC ports used for various operations.
http://technet2.microsoft.com/windowsserver/en/library/108124dd-31b1-4c2c-9421-6adbc1ebceca1033.mspx?mfr=true

I've also used this document in the past, touching on some topics such as DNS, netlogon optimisation/site synchronisation
http://www.microsoft.com/technet/prodtechnol/windowsserver2003/technologies/directory/activedirectory/mtfstwp.mspx

Forest trust information – the flow and types of traffic for cross-forest authentication, which shows the workstation from the account domain (trusted), contacting a KDC in the resource domain (trusting) for kerberos authentication, often overlooked - this means traffic goes directly from all workstation endpoints to DCs in the resource domain, as opposed to an NTLM trust where DCs normally talk to each other)
http://technet2.microsoft.com/windowsserver/en/library/517b4fa4-5266-419c-9791-6fb56fabb85e1033.mspx?mfr=true

2003 Servers with Hibernation enabled

2008-10-21T23:19:00.003+10:00

How many of your 2003 servers have hibernation enabled and you never realised? If you're like me, possibly a few, and I can't think of a scenario where I would want to hibernate a server.

This post provides a simple method of detecting whether hibernation is enabled, and then a command to remotely disable hibernation on each server. It works based on the premise that if hibernation is enabled, you'll have a file called hiberfil.sys on the root of your system drive.

Create a text file called servers.txt listing one server per-line and then run the following command
for /f %i in (servers.txt) do @if exist \\%~i\c$\hiberfil.sys (echo %~i,Enabled) else (echo %~i,Disabled)
Save the output filterd by Disabled (find /i "Disabled" results.txt) and run the following command to disable on each (assuming you have psexec)
for /f %i in (servers_hibernate.txt) do psexec \\%i powercfg /HIBERNATE off

This should automatically delete the hibernation file, reclaiming the hard disk space equal to the RAM of the machine.

Note that this assumes your systemdrive is C: and that you have administrative shares enabled (C$). To accurately distinguish between servers that don't have C:/C$. you could also run a nested if command (which assumes an installation directory of Windows):
for /f %i in (servers.txt) do @if exist \\%~i\c$\windows (if exist \\%~i\c$\hiberfil.sys (echo %~i,Enabled) else (echo %~i,Disabled)) else echo %~i,Not Found

You can also manually run the command to disable hibernation against one server:
psexec \\%server% powercfg /HIBERNATE off

Reading Shortcuts with PowerShell and VBS

2008-09-28T14:59:00.004+10:00

This post provides a simple method of enumerating shortcuts and retrieving their properties, either using PowerShell or VBScript. There is nothing particularly clever here, but I had a need a while back to use the enumeration, and I wrote the simple powershell equivalent to see how easy it would be. Note that I couldn't easily see a managed code .Net method to process shortcuts, so I fell back on the wscript COM object. More than likely there is a better method (get it? :) than this.

Run the scripts:


powershell . .\ReadShortcut.ps1 -p \\%server%\%share%\folder\path
powershell . .\ReadShortcut.ps1 -f \\%server%\%share%\shortcut.lnk
cscript //nologo "ReadShortcut.vbs" \\%server%\%share%\folder\path
cscript //nologo "ReadShortcut.vbs" \\%server%\%share%\shortcut.lnk

#
param(
   [string] $path = "",
   [string] $file = ""
   )

$WshShell = new-object -comobject "WScript.Shell"       # Instantiate the wscript.shell COM object

if ($path -ne "") {
    $shortcuts = get-childitem -path $path -filter "*.lnk" -rec    # Find all .lnk files, recursing in to subdirectories
    $shortcuts | foreach-object {$WshShell.CreateShortcut($_.FullName) }  # For each file, pass the fullname to the COM object to open the shortcut and enumerate the properties
} elseif ($file -ne "") {
    $shortcut = get-item -path $file       # Get the single file
    if ($shortcut -ne $null) { $WshShell.CreateShortcut($shortcut) }   # If exists, read the shortcut properties
} else {
    write-output "No arguments specified, please use either -p to specify a path or -f for a specific .lnk file"
}

#-------#


'ReadShortcut.vbs
' Read a shortcut or a top-level directory of shortcuts and write the properties to stdout
If WScript.Arguments.UnNamed.Count = 1 Then
 strShortcut = WScript.Arguments.UnNamed(0)
Else
 WScript.Echo "Please supply the name of an lnk file or directory to read, eg c:\test.lnk or c:\shortcuts"
 WScript.Quit(1)
End If

Set objFSO = CreateObject("Scripting.FileSystemObject")

If objFSO.FolderExists(strShortCut) Then      ' Was a directory specified?
 Set objFolder = objFSO.getFolder(strShortcut)     ' Get the folder
 
 For Each objfile in objFolder.Files      ' For each file in the top-level directory
  If objfile.type = "Shortcut" Then     ' Is this file a shortcut?
   Call Readshortcut(objFile.Path, strProperties)   ' yes, read the properties
   dtmCreationDate = objFile.DateCreated
   WScript.Echo dtmCreationDate & "," & strProperties  ' output the results
  End If 
 Next
ElseIf objFSO.FileExists(strShortCut) Then      ' Was an individual file specified?
 Call Readshortcut(strShortcut, strProperties)     ' read the properties of the file
 WScript.Echo strProperties       ' output the results
Else           ' file-not-found
 WScript.Echo "Error: Could not read '" & strShortcut & "'"
 WScript.Quit(2)
End If
Set objFSO = Nothing

Function Readshortcut(ByRef strShortcut, ByRef strProperties)

 set objWshShell = WScript.CreateObject("WScript.Shell")    ' Create the shell object
 set objShellLink = objWshShell.CreateShortcut(strShortcut)   ' Execute the createshortcut method, which also retrieves an existing shortcut
 strProperties = strShortCut & "," & objShellLink.TargetPath & "," & objShellLink.WindowStyle & "," & objShellLink.Hotkey & "," & objShellLink.IconLocation & "," & objShellLink.Description & "," & objShellLink.Arguments & "," & objShellLink.FullName & "," & objShellLink.WorkingDirectory & """"

 ' This propertly can be set, but not read? - objShellLink.RelativePath
 Set objShellLink = Nothing
 Set objWshshell = Nothing
End Function

Modifying DLL Resources

2008-09-27T21:17:00.002+10:00

While it wouldn't be supported by Microsoft, there are a few methods you can use to modify the resources contained in DLLs - usually useful if you would like to change how a window or dialog box is displayed. The easiest way I've seen is with a program called reshacker.exe.

A practical example is when you might be trying to display a large legal notice text using the standard MS gina on XP - and you can't actually scroll down to view the whole message due to the lack of scroll bars in the legal notice text dialog box.

To add a dialog box to this display in msgina.dll in a XP SP2 machine, you could:

Use reshacker to open msgina.dll, and modify the dialog number 2500 - with 1033 language in my case (en-au)
Edit the control, allowing the WS_VSCROLL option
Recompile the script and save the new DLL.
You'll then either have to modify WFP behaviour, disable it, or to test this I overwrote the cached copy to fool Windows File Protection in system32\dllcache

Any new logon through the interactive console or TS to a workstation with a legal notice text should display it in a dialog box with a scroll bar.

Resource Hacker homepage:
http://angusj.com/resourcehacker/

Automatically mapping printers

2008-09-21T17:25:00.004+10:00

A while back I came across one of the new 'features' in server 2003 R2, a method of centrally managing printers through the ‘Print Management’ snap-in, and then deploying those printers to workstations or users through a GPO.

This sounds clever, but I believe it's really just an automated method of creating a computer startup/user logon script, which uses a control file to determine which printers are to be installed and then runs a commandline utility to install those printers (pushprinterconnections.exe).

This made me reconsider the approach to managing printers for workstations, and similar or better functionality could be provided by one or more of the methods below:

Use policy to deploy all printers for a building/floor/subnet to workstations. This would be the least management, but probably the least effective as there could be dozens or hundreds of printers per building/floor/subnet.
Have a startup script that automatically adds a default printer when a user first logs on. The default would have to be either based on AD subnets and a mapping file between these subnets and default printers, or possibly based on user information such as location and department, which would be used to query AD/printer names to map appropriate printers, eg a user location of 'Building 1/Floor 2' and department of 'Finance' for a user could easliy be translated to a printer called FINPRINT1 with an AD location of 'Building 1/Floor 2'.
Have a process that exports MAC/PCs connected per floor switch, which would be used as a control file to automatically map printers based on floor. This would typically require assistance from your friendly data communications team.
Use 'DHCP snooping' to further identify where clients are coming from. For example, DHCP relay agent option 82 can be used to insert the switch MAC address/subscriber VLAN/port, SNMP information to the DHCP request packet (when using layer 2 DHCP relay agents as opposed to layer 3 ip helpers). This information could potentially be parsed by the DHCP server to identify the switch/port clients are coming from by parsing the circuit-id/remote-id. The output could be used as a control file to automatically map printers based on floor.
Use the information from 4 and information on floor patching to construct a virtual map of your floors/buildings, which would place one or more workstations close to one or more printers, and map them automatically.
When you build each workstation (this wouldn't work for roaming devices), create a control file that indicates physically where this workstation is located using the Microsoft format for subnet locations, eg Building/Floor 1/West Corner/Pod 4. Populate printer locations to this degree, and then your logon script could search the AD for matching printers and map those that are closest. You could also use this information to create a shortcut called 'Find printers' that uses 'rundll32 dsquery,OpenSavedDsQuery %Config%.qds' to open a saved query file that you've populated with this location. You would then need processes in place to update this control file as the workstation moves, otherwise it becomes incorrect - worse than nothing at all.
Do nothing clever and let users add their own printers

Of course the 'nothing clever' option normally wins out, but in a large organisation one of the options above could add real value, and if done correctly would not be high maintenance.

References:

Manage Printers with New Group Policy Features in Windows Server 2003 R2
http://www.microsoft.com/technet/technetmag/issues/2006/08/ManagePrinters/

RFC 3046 - DHCP Relay Agent Information Option
http://www.faqs.org/rfcs/rfc3046.html

Configuring the DHCP Option 82 for Subscriber Identification
http://www.cisco.com/en/US/docs/switches/lan/catalyst3550/software/release/12.1_13_ea1/configuration/guide/swdhcp82.html

Simple string encryption with PowerShell

2008-09-13T20:27:00.003+10:00

While looking for a method to obfuscate passwords in script files, I started with securestring input combined with the convertfrom-securestring and convertto-securestring functions. Unfortunately (or fortunately from a security perspective) the securestring seems relevant only for per session/user/process/computer (or a combination thereof).

From documentation these functions use Rijndael symmetric encryption, so I modified another example for a very simple, key-less passphrase-less encryption/decryption of a string. This isn’t even very good obfuscation, let alone encryption, but if you encrypted a password and then reproduce the last 10 lines to decrypt the encrypted string, it’s slightly better than storing passwords in very visible paintext (sort of). I guess this could be obfuscated further by reading the encrypted string from a secured file or registry key.

This could be made as complex as you like with keys and initialisation vectors, but unless you make people enter the key (and then why not just make them enter the password?), there would still be something in plaintext, so I didn’t think there was much point.

I'm still undecided on whether there is any benefit with such simple obfuscation, but I thought I'd post the script nonetheless.



$string = "LongStringToEncryptAsATest"
$string

$r = new-Object System.Security.Cryptography.RijndaelManaged  # use Rijndael symmetric key encryption
$c = $r.CreateEncryptor((1..16), (1..16))    # Set the key and initialisation vector to 128-bytes each of (1..16)
$ms = new-Object IO.MemoryStream
$cs = new-Object Security.Cryptography.CryptoStream $ms,$c,"Write" # Target data stream, transformation, and mode
$sw = new-Object IO.StreamWriter $cs
$sw.Write($String)       # Write the string through the crypto stream into the memory stream
$sw.Close()
$cs.Close()
$ms.Close()
$r.Clear()
[byte[]]$result = $ms.ToArray()      # Byte array from the encrypted memory stream
$encstring = [Convert]::ToBase64String($result)    # Convert to base64 for transport

$encstring         # The encrypted base64 string representation


$Encrypted = [Convert]::FromBase64String($encstring)   # Convert the encrypted string to a byte array
$r = new-Object System.Security.Cryptography.RijndaelManaged  # use Rijndael symmetric key encryption
$d = $r.CreateDecryptor((1..16), (1..16))    # Set the key and initialisation vector to 128-bytes each of (1..16)

$ms = new-Object IO.MemoryStream @(,$Encrypted)    # Create a memorystream from a single-element name/value pair hash table of the byte array
$cs = new-Object Security.Cryptography.CryptoStream $ms,$d,"Read" # Target data stream, transformation, and mode
$sr = new-Object IO.StreamReader $cs     # Read the string through the crypto stream from the encrypted memory stream
write-output $sr.ReadToEnd()      # Write the unencrypted string
$sr.Close()
$cs.Close()
$ms.Close()
$r.Clear()

Useful NTFS and security command-line operations

2008-09-08T21:42:00.002+10:00

The commands below are a subset of the complete command list found in Useful command-lines, and provide different methods to query, modify and report on aspects of NTFS, including security, MFT, ABE, alternate data streams, security templates and SDDL.

Each command-line can be copied and pasted at the command prompt, if you use a batch file you'll need to reference variables with double-percent (%%).

Disable path parsing to access a file using UNC
type "\\?\UNC\server\share\path\file.txt"

Dump file and security attributes on an NTFS filesystem
filestat %file%

Show the ACL in SDDL format using cacls (2003)
cacls c:\ /s

Add authenticated users:R to the root (not OICI) using SDDL with cacls
cacls c:\ /S:"D:PAI(A;OICI;FA;;;BA)(A;;0x1200a9;;;AU)(A;OICI;FA;;;SY)"

Query Storage Report Management global settings (file screening and quotas)
reg query HKEY_LOCAL_MACHINE\Cluster\SRM\Settings\SrmGlobalSettings /v Data

Show the current mounted volumes on the local system
mountvol OR mountvol c:\ /l

Create a hardlink directory entry linking one file to another
fsutil hardlink create %NewFileLink% %ExistingFile%

create symbolic dir, file links, hard links and dir junctions Vista/2008
mklink

Add a string to an alternate stream stored as a $DATA attribute
echo this content will be stored in the file sub stream > test.txt:stream1

Add the contents of a file to an alternate data strem
type file.txt > test.txt:stream1

Create a new file in an alternate stream as part of a directory
md test & echo alternate stream > test:stream1

Display the stream
more <> test:test.exe

Run the executable in the alternate stream
cmd /c start .\test:test.exe

Retrieve the binary file from an alternate data stream (port of Unix Cat)
cat test:test.exe > note.exe

Remove an NTFS alternate data stream (does not maintain timestamps)
type test.txt>test1.txt

Delete an alternate data stream
streams -d test.txt

Dump NTFS info from the MFT for a File Record Segment (unix port)
ntfsinfo -d /dev/hda1 -i 36

Find cluster disk size and free space
echo clusnode1 > clusternodes.txt & echo clusnode2 >> clusternodes.txt & echo clusnode3 >> clusternodes.txt & echo clusnode4 >> clusternodes.txt & wmic /node:@clusternodes.txt path Win32_LogicalDisk WHERE "FileSystem='NTFS' AND Name != 'C:' AND Name != 'D:'" GET SystemName,Name,Size,FreeSpace,VolumeName

Create a security template from the given directory for SDDL secedit processing
for /d %i in (%source%\*.*) do @for /f "tokens=2" %j in ('cacls.exe %i /S') do @echo Processing %~ni & echo "%i",2,%j >> Security.txt

Make test directories from a security template
for /f "tokens=1,2,3,* delims=\," %i in ('"find /i "dir1\dir2" template.inf"') do md "c:\temp\sectest\%j\%k

Reformat the target of a security template for offline testing
for /f "tokens=1,2,3,* delims=\," %i in ('"find /i "dir1\dir2" template.inf"') do @echo "c:\temp\sectest\%j\%k,%l

Remotely configure filesystem NTFS security using a secedit template
psexec \\%server% secedit /configure /db c:\windows\temp\%random%.sdb /cfg c:\windows\security\templates\%template%.inf /log c:\windows\temp\Configure.log

Disable automatic mounting of volumes added to a 2003 system
automount disable
Remove automatic mount points for volumes that no longer exist
automount scrub

Report the NTFS permissions for restore from a SDDL template
setacl -on c:\windows -ot file -actn list -rec cont_obj -lst f:sddl -bckp test.txt

List the file system filters and instances installed on the system
fltmc filters & fltmc instances

Run diskpart to list NTFS volumes remotely
psexec \\%server% cmd /c "echo list volume diskpart find /i "NTFS""

Find local disks other than C: and D: that are NTFS
wmic /node:"%server%" path Win32_LogicalDisk WHERE "FileSystem='NTFS' AND Name != 'C:' AND Name != 'D:'" GET SystemName,Name

Find local disks using powershell and output to CSV
Get-WmiObject -Namespace root\cimv2 -ComputerName %server% -Query "SELECT * from Win32_LogicalDisk WHERE FileSystem='NTFS' AND Description = 'Local Fixed Disk' AND Name != 'C:' AND Name != 'D:'" export-csv c:\disk.csv

Read directories from a security template, checking if they exist
for /f "skip=2 tokens=1 delims=," %i in ('"find /i "OICI" \\%server%\%Template%"') do @if not exist \\%server%\%drive%$%~pi echo \\%server%\%drive%%~pi

Query the NTFS attributes of a file
nfi "C:\windows\system32\notepad.exe"

Report and save the security on one or more NTFS Filesystem objects
ICACLS %object% /save %outputfile%

Reset the inherited DACL flag on an object to not protected, inherit below
setacl -on %Directory% -ot file -actn setprot -op dacl:np

Reset inherited permissions for specified NTFS objects
setacl -on %Directory%\*.* -ot file -actn rstchldrn -rst DACL

Check user home drive permissions from hmDir (top-level)
for /f %i in (%users.txt%) do @for /f "tokens=*" %m in ('"dsquery user -name %i dsget user -hmdir find /i "%i""') do cacls "%m"

Check Access Based Enumeration is enabled on a top-level share
psexec \\%server% abecmd %share%

Check permissions are being inherited
for /f %i in (%users.txt%) do @for /f "tokens=1" %m in ('"dsquery user -name %i dsget user -hmdir find /i "%i""') do @for /d %p in (%m\*.*) do @echo cacls "%p"

Check ownership of files
for /f %i in (%users.txt%) do @for /f "tokens=1" %m in ('"dsquery user -name %i dsget user -hmdir find /i "%i""') do @dir %m /q/s

Process robocopy /l output to find files/data that would be copied
for /f "tokens=2,5 delims=_:" %i in (output.txt) do @If "!User!" NEQ "%i" (@Set User=%i&@Set Line=%i,%j) else (@Set Line=!Line!,%j & @Echo !Line!)

List ACLs in SDDL format
setacl -on %filepath% -ot file -actn list -lst f:sddl

Change the ownership of all subdirectories and files in a folder
subinacl /subdirectorie %Path%\*.* /setowner=%Domain%\%User%

Change the ownership of a folder
subinacl /file %Path% /setowner=%Domain%\%User%

Logical Disk Information
wmic /node:%server% path Win32_LogicalDisk

Physical Disk Information
wmic /node:%server% path Win32_DiskPartition

Wayne's World of IT (WWoIT), Copyright 2008 Wayne Martin.

Useful Windows Printer command-line operations

2008-09-08T20:30:00.003+10:00

The commands below are a subset of the complete command list found in Useful command-lines, and perform various operations against printers on a Windows 200x server.

The commands are useful for querying information from one or more printers, as well as making changes to configuration items for printers, including queues, shares, AD information and drivers. Generally Microsoft Windows printing is relatively simple - but add MSCS clustering, the plethora of PCL unidrv versions out there and badly written third-party drivers - troubleshooting can be quite difficult.

Each command-line can be copied and pasted at the command prompt, if you use a batch file you'll need to reference variables with double-percent (%%).

Use a custom WMIC alias to report printer info from a cluster node in CSV
wmic /node:"%server%" spoolerjobs list /format:table

Use a custom WMIC alias to report sorted HTML printer info from a cluster node
wmic /node:"%server%" spoolerjobs list /format:htable:"datatype=number":"sortby=TotalJobsPrinted"

Use a custom WMIC alias to report a brief summary of printer statistics
wmic /node:"%server%" spoolerjobs list brief /format:htable:"datatype=number":"sortby=TotalJobsPrinted"

Use a custom WMIC alias and xsl to sort print jobs output descending HTML table
wmic /node:"%server%" spoolerjobs list brief /format:"htabledesc-sortby.xsl":"datatype=number":"orderby=descending":"sortby=TotalJobsPrinted" > test.html

Reset inheritance starting at a folder and recursing
icacls \\UNC\Path /reset /t /c

Open a saved printer query window
rundll32 dsquery,OpenSavedDsQuery %Config%.qds

Query HP printers for the page count using SNMP
for /f %i in (Printers.txt) do @echo %i & @snmputil walk %i public printmib.prtMarker.prtMarkerTable.prtMarkerEntry.prtMarkerLifeCount | find /i "value"

Query printers for job counts >= 1
wmic /node:"%server%" path Win32_PerfFormattedData_Spooler_PrintQueue Where "Jobs>=1" Get Name,Jobs,MaxJobsSpooling,JobErrors,OutOfPaperErrors

Query printer information from AD (speed, memory etc)
dsquery * %serverDN% -limit 0 -filter "(&(objectClass=printQueue)(objectCategory=printQueue))" -attr cn printerName driverName printCollate printColor printLanguage printSpooling driverVersion printStaplingSupported printMemory printRate printRateUnit printMediaReady

Export printer information from AD (speed, memory etc)
csvde -f Printers.csv -d %serverDN% -r "(&(objectClass=printQueue)(objectCategory=printQueue))" -l cn,printerName,location,driverName,printCollate,printColor,printLanguage,printSpooling,driverVersion,printStaplingSupported,printMemory,printRate,printRateUnit,printMediaReady

Modify a printer or all printers on a print server to use the winprint processor
setprinter \\%Server%[\%Printer%] 2 pPrintProcessor="WinPrint"

Query the unidrv DLL versions from a list of pingable computers in an OU
for /f %i in ('"dsquery computer %OU_DN% -limit 0 -o rdn"') do @for /f "tokens=3 delims=: " %m in ('"ping -n 1 %i find /i "reply from""') do @for /f "tokens=1-9" %q in ('"filever \\%~i\C$\windows\system32\spool\drivers\w32x86\3\uni*.dll find /i /v "\\%~i""') do @echo %i,%q,%r,%s,%t,%u,%v,%w,%x,%y

check printers are being published in the directory through setprinter
setprinter -show \\%server% 7

Run the 32-bit version of regsvr32 on an x64 platform to register the prnadmin DLL
c:\WINDOWS\SysWOW64\regsvr32 c:\admin\util\prnadmin.dll

Run the 32-bit version of cscript that accesses a 32-bit DLL on x64
c:\WINDOWS\SysWOW64\cscript portmgr.vbs

Use SetPrinter to capture all settings for a printer
for %i in (0 1 2 3 4 5 6 7 8 9) do setprinter -show \\%server%\%printer% %i

Report print queue current length and various performance statistics
wmic /node:"%server%" path Win32_PerfFormattedData_Spooler_PrintQueue Get Name,Jobs,TotalJobsPrinted,TotalPagesPrinted,MaxJobsSpooling,JobErrors,OutOfPaperErrors

List printer drivers on a physical or cluster Windows 2003 server
cscript drvmgr.vbs -l -c \\%server%

Print a test page to one or more printers
for /f "tokens=1-4 delims=," %i in (%Printer.txt%) do cscript prnctrl.vbs -t -b \\%PrintServer%\%i

Given a list of printers, create an LPR port for each
for /f "tokens=1-4 delims=," %i in (%Printer.txt%) do @cscript portmgr.vbs -a -c \\%PrintServer% -p %i -h %k -t LPR -q %i

Given a list of printers, create a printer using the specified driver/port
for /f "tokens=1-4 delims=," %i in (%Printer.txt%) do @cscript prnmgr.vbs -a -c \\%PrintServer% -b %i -m "%j" -r %i

Given a list of printers, set the location, share name and publish in AD
for /f "tokens=1-4 delims=," %i in (%Printer.txt%) do @cscript prncfg.vbs -s -b \\%PrintServer%\%i -h %i -l "%l" +published

Given a list of printers, set glocal default printer properties to duplex
for /f "tokens=1-4 delims=," %i in (%Printer.txt%) do @setprinter \\%PrintServer%\%i 8 "pDevMode=dmDuplex=2,dmCollate=1,dmFields=duplex collate"

Given a list of printers, set printer permissions to allow for queue management
for /f "tokens=1-4 delims=," %i in (%Printer.txt%) do @c:\util\subinacl /printer \\%PrintServer%\%i /grant=%Domain%\%Group%=M

Find printer event log entries for the last 15 minutes
for /f %i in ('cscript //nologo c:\data\scripts\updated\time\finddatetime.vbs') do wmic /node:"%server%" path Win32_NTLogEvent WHERE "SourceName='Print' AND LogFile='System' AND EventCode=10 AND TimeGenerated > '%i'"

Find printer event log entries for the specified user
wmic /node:"%server" path Win32_NTLogEvent WHERE "SourceName='Print' AND LogFile='System' AND EventCode=10 AND User='%Domain%\\%Username%'" Get User,TimeGenerated,InsertionStrings

Automated Printer configuration
rundll32 printui.dll,PrintUIEntry /?

Create a port using prnadmin.dll on a cluster node
cscript portmgr.vbs -a -c \\%PrintServer% -p %Printer% -h %PrinterIP% -t raw -n 9100

Create a printer using prnadmin.dll on a cluster node
cscript prnmgr.vbs -a -c \\%PrintServer% -b %Printer% -m "HP Universal Printing PCL 6" -r %Printer%

Set printer configuration using prnadmin.dll on a cluster node
cscript prncfg.vbs -s -b \\%PrintServer%\%Printer% -h %Printer% -l "Building/Level 1" +published

Dump printer log entries
dumpel -s \\%PrintServer% -l System -e 10 -m Print -d 1

Add an x64 print driver to a cluster node
rundll32 printui.dll,PrintUIEntry /ia /c \\%PrintServer% /m "HP Color LaserJet 8550 PCL 5c" /h "x64" /v "Windows XP and Windows Server 2003" /f hpc8550t.inf

Add an x86 print driver to a cluster node
rundll32 printui.dll,PrintUIEntry /ia /c \\%PrintServer% /m "HP Color LaserJet 8550 PCL 5c" /h "x86" /v "Windows 2000, Windows XP and Windows Server 2003" /f hpc8550b.inf

Add to the permissions on a printer
subinacl /printer \\%PrintServer%\%Printer% /grant=%Domain%\%Group%=M

List the printers on a server
cscript "c:\Program Files\Windows Resource Kits\Tools\prnmgr.vbs" -l -c \\%PrintServer% find /i "printername"

Set the glocal default printer properties to allow duplexing
setprinter \\%PrintServer%\%i 8 "pDevMode=dmDuplex=2,dmCollate=1,dmFields=duplex collate"

Query the printers published against a specific computer account
dsquery * %ComputerDN%" -attr printShareName

Query all printers published in the Active Directory
dsquery * domainroot -filter "(objectCategory=printQueue)" -limit 0

Show the current default settings send down to clients from a printer
setprinter -show \\%printServer%\%Printer% 8

Find the print processor used by all printers on a print server
setprinter -show \\%server% 2 findstr /c:"pPrinterName" /c:"pPrintProcessor"

Check through SNMP whether a printer supports a duplexing media path
snmputil walk %printer% public printmib.prtMediaPath.prtMediaPathTable.prtMediaPathEntry.prtMediaPathDescription

Check through SNMP the printer device model
snmputil getnext %printer% public host.hrDevice.hrDeviceTable.hrDeviceEntry.hrDeviceDescr

View/Modify the ACL security on spooler pipe
pipesec \\%computer%\pipe\spoolss

Query printer tray configuration of printers on a cluster (HP UPD)
for /f %i in (Printers.txt) do @for /f "tokens=1,2,*" %m in ('"reg query \\%PrintServer%\HKEY_LOCAL_MACHINE\Cluster\Resources\%ClusterResourceGUID%\Parameters\Printers\%i\PrinterDriverData /v InputSlot 2>nul find /i "InputSlot" & if errorlevel 1 echo 1 2 NotFound"') do @echo %i,%o>>PrinterTrays.txt

Use SNMP to get the printer media path, indicating simplex/duplex capabilities
for /f %i in (Printers.txt) do @for /f "tokens=3,*" %m in ('"snmputil walk %i public printmib.prtMediaPath.prtMediaPathTable.prtMediaPathEntry.prtMediaPathDescription 2>nul find /i "value" & if errorlevel 1 echo 1 2 3 NotFound"') do @echo %i,%n

Wayne's World of IT (WWoIT), Copyright 2008 Wayne Martin.

Useful Windows MSCS Cluster command-line operations

2008-09-08T20:27:00.003+10:00

The commands below are a subset of the complete command list found in Useful command-lines, and are command-line operations for Microsoft Windows MSCS server clusters. Most commands are based around the Microsoft cluster.exe utility, with some using WMI, defrag and diruse to provide information on cluster disk resources.

Each command-line can be copied and pasted at the command prompt, if you use a batch file you'll need to reference variables with double-percent (%%).

Find cluster disk size and free space in CSV format
wmic /node:"%server%","%server%","%server%","%server%" path Win32_LogicalDisk WHERE "FileSystem='NTFS' AND Name != 'C:' AND Name != 'D:'" GET SystemName,Name,Size,FreeSpace,VolumeName /format:csv

Find cluster disk size and free space in modified CSV format with thousand sep.
wmic /node:"%server%","%server%","%server%","%server%" path Win32_LogicalDisk WHERE "FileSystem='NTFS' AND Name != 'C:' AND Name != 'D:'" GET Name,Size,FreeSpace,VolumeName /format:csv2

Report the windows MSCS cluster virtual groups
cluster /cluster:%cluster% group /prop | find /i "description" | find /i /v "pbx" | find /i /v "cluster"

Report folders being archived from Enterprise Vault EV FSA
sqlcmd -S sqlServer%\%instance% -o ArchivedFolders.txt -d %enterprisevaultdirectory% -W -s "," -Q "select FSVP.UncName, FSVP.VolumeName, FSFE.FolderPath, FSVP.UncName + '\' + FSVP.VolumeName + '\' + FSFE.FolderPath as 'Path' from dbo.FileServerFolderEntry FSFE inner join dbo.vw_FileServer_Volume_Policy FSVP on FSFE.VolumeEntryID = FSVP.VolumeEntryID"

Report folders from the one or more servers not being archived compared to FSA export
for %i in (\\%server%\share% \\%server%\share% ) do @for /f "tokens=1-4,*" %m in ('"dir %i\* /ad /tc | find "DIR" | find "-""') do @find /i "%q" ArchivedFolders.txt >nul & @If errorlevel 1 (echo %q,%i,%m %n %o) >> NotArchived.csv

Delete a cluster resource type
cluster restype "%resource_name%" /delete /type

Find cluster disk size and free space
echo clusnode1 > clusternodes.txt & echo clusnode2 >> clusternodes.txt & echo clusnode3 >> clusternodes.txt & echo clusnode4 >> clusternodes.txt & wmic /node:@clusternodes.txt path Win32_LogicalDisk WHERE "FileSystem='NTFS' AND Name != 'C:' AND Name != 'D:'" GET SystemName,Name,Size,FreeSpace,VolumeName

show the MSCS cluster multicast address properties
cluster /cluster:%Cluster% network "%PublicNetwork%" /priv

Find the MSCS cluster resources
cluster /cluster:%Cluster% res /prop find /i "sr"

Find the disks currently owned by each cluster node
for %i in (%server1% %server2%) do @wmic /node:"%i" path Win32_LogicalDisk WHERE "FileSystem='NTFS' AND Name != 'C:' AND Name != 'D:'" GET SystemName,Name find /i "%server_prefix%"

In a 2003 cluster, find each disk volume and analyse file fragmentation
for /f "tokens=2,5,6,8" %i in ('"cluster /cluster:%cluster% resource /prop find /i "disk" find /i "description" find /i "%CommonTag%""') do echo \\%i\%k %j %l>> Defrag_%i_%j.txt && psexec \\%i defrag %k -a -v >> Defrag_%i_%j.txt

From cluster defrag analysis, print out details for each cluster volume
for /f "tokens=1,* delims=:" %i in ('"findstr /i /c:%server% /c:"Total files" /c:"Volume size" /c:"Used space" /c:"Percent free space" /c:"Total fragmented files" defrag*"') do @echo %j

Create a cluster file share:
cluster /cluster:%cluster% res "%share_res_name%" /create /group:"%group%" /type:"File Share"
cluster /cluster:%cluster% res "%share_res_name%" /priv path="%path%"
cluster /cluster:%cluster% res "%share_res_name%" /priv Sharename=%share_name%
cluster /cluster:%cluster% res "%share_res_name%" /priv Remark="File Share Remark"
cluster /cluster:%cluster% res "%share_res_name%" /prop Description="File Share Description"
cluster /cluster:%cluster% res "%share_res_name%" /priv security=Everyone,grant,F:security
cluster /cluster:%cluster% res "%share_res_name%" /AddDep:"%networkname_res%"
cluster /cluster:%cluster% res "%share_res_name%" /AddDep:"%disk_res%"
cluster /cluster:%cluster% res "%share_res_name%" /On

Create an ABE resource for the file share
cluster /cluster:%cluster% res "%shareabe_res_name%" /create /group:"%group%" /type:"Generic Application"
cluster /cluster:%cluster% res "%shareabe_res_name%" /priv CommandLine="cmd.exe /k abecmd.exe /enable %share_name%"
cluster /cluster:%cluster% res "%shareabe_res_name%" /priv CurrentDirectory="%SystemRoot%"
cluster /cluster:%cluster% res "%shareabe_res_name%" /priv InteractWithDesktop=0
cluster /cluster:%cluster% res "%shareabe_res_name%" /priv UseNetworkName=0
cluster /cluster:%cluster% res "%shareabe_res_name%" /prop SeparateMonitor=1
cluster /cluster:%cluster% res "%shareabe_res_name%" /prop Description="Access Based Enumeration for %share_name% File Share"
cluster /cluster:%cluster% res "%shareabe_res_name%" /AddDep:"%networkname_res%"
cluster /cluster:%cluster% res "%shareabe_res_name%" /AddDep:"%disk_res%"
cluster /cluster:%cluster% res "%shareabe_res_name%" /AddDep:"%share_res_name%"
cluster /cluster:%cluster% res "%shareabe_res_name%" /On

Wayne's World of IT (WWoIT), Copyright 2008 Wayne Martin.

Useful VMware ESX and VC command-line operations

2008-09-08T20:26:00.003+10:00

The commands below are a subset of the complete command list found in Useful command-lines, and are a combination of ESX service console commands, PowerShell VI Toolkit or SQL commands (reading from a VirtualCenter database). Note that these have been tested on VI3 - ESX 3.0.x, VC 2.0.x, and some have also been used on VI3.5 - ESX 3.5 / VC 2.5.

Each command-line can be copied and pasted at the command prompt, if you use a batch file you'll need to reference variables with double-percent (%%).

ESX service console

Report the hosts controlled by a VirtualCentre instance
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select [Name] from vpxv_hosts order by [Name]"

Report the hosts controlled by a VirtualCentre instance
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select DNS_Name, IP_Address, PRODUCT_FULLNAME from vpx_host Order by DNS_Name"

Query the Virtual Centre/VC database for Virtual Machine LUN mapping and size
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select VMS.Name, TDS.Name, Cast(Round(Sum(Cast(VDISK.CAPACITY as numeric))/1024/1024/1024, 1) as int) as 'Total Disk', Cast(Round(Sum(Cast(VDISK.FREE_SPACE as numeric))/1024/1024/1024, 1) as int) as 'Free Disk' from vpxv_vms VMS inner join vpxv_vm_datastore VVMDS on VMS.VMID = VVMDS.VM_ID inner join vpx_datastore TDS on VVMDS.DS_ID = TDS.ID inner join VPX_GUEST_DISK VDISK on VMS.VMID = VDISK.VM_ID group by VMS.Name, TDS.Name Order by TDS.Name, VMS.Name"

Allow Dell OpenManage Server Administrator traffic through ESX
esxcfg-firewall -o 1311,tcp,in,OpenManageRequest

Dump the VMware IP config
cat /proc/vmware/net/tcpip/ifconfig

View network traffic in the vswif service console interface
/usr/sbin/tcpdump -i vswif0 grep -v ssh

Check the Legato AAM HA agent
cat /opt/LGTOaam512/log/aam_config_util_addnode.log

VMware HA Legato AAM, list the cluster manager
/opt/LGTOaam512/bin/ftcli -domain vmware -timeout 60 -cmd "listrules"
(run 'FT_DIR=/opt/LGTOaam512' and then 'export FT_DIR' first)

VMware HA Legato AAM, list the cluster nodes
/opt/LGTOaam512/bin/ftcli -domain vmware -connect %node% -port 8042 -timeout 60 -cmd "listnodes"
(run 'FT_DIR=/opt/LGTOaam512' and then 'export FT_DIR' first)

Query ESX for SNMPD service firewall configuration
esxcfg-firewall -q snmpd

Query vmfs volumes for snapshot files from the console
ls -Ral /vmfs/volumes/*.* grep .vmsn

Change security on a file in the ESX service console
chmod u+rwx,g+rx,o+rx somefile

Register a Virtual Machine in ESX
vmware-cmd -s register "/path/to/vmx/or/vmtx"

unregister a Virtual Machine in ESX
vmware-cmd -s unregister "/path/to/vmx/or/vmtx"

Enable SSH client out on an ESX host firewall, permitting SCP as well
esxcfg-firewall -e sshClient

Query the ESX firewall to determine whether SSH client is allowed out
esxcfg-firewall -q grep sshClient

Query the VirtualCenter Database

Query VMs and their MAC addresses from the VC database
sqlcmd -S %sqlServer% -d %Database% -W -s "," -q "select ENT.Name, NET.MAC_Address from VPX_GUEST_NET_ADAPTER NET inner join VPX_ENTITY ENT on NET.VM_ID = ENT.ID"

Query the Virtual Centre/VC database for Virtual Machine Disk Totals
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select VM.Name, Cast(Round(Sum(Cast(VDISK.CAPACITY as numeric))/1024/1024/1024, 1) as int) as 'Total Disk', Cast(Round(Sum(Cast(VDISK.FREE_SPACE as numeric))/1024/1024/1024, 1) as int) as 'Free Disk' from dbo.VPX_GUEST_DISK VDISK inner join VPXV_VMS VM on VDISK.VM_ID = VM.VMID group by VM.Name compute sum(Cast(Round(Sum(Cast(VDISK.CAPACITY as numeric))/1024/1024/1024, 1) as int))"

Query the Virtual Centre/VC database for Virtual Machine Disk information
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select VM.Name, VDISK.* from dbo.VPX_GUEST_DISK VDISK inner join VPXV_VMS VM on VDISK.VM_ID = VM.VMID"

Query the Virtual Centre/VC database for Virtual Machine Disk Total
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select Cast(Round(Sum(Cast(VDISK.CAPACITY as numeric))/1024/1024/1024, 1) as int) from dbo.VPX_GUEST_DISK VDISK"

Query VirtualCentre for hot migration events
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select Create_Time as 'Relocation Start', UserName, VM_Name, Host_Name as 'Host Source', (select Host_Name from dbo.VPX_EVENT Where Chain_ID = EVTDEST.Chain_ID and event_type = 'vim.event.VmMigratedEvent') as 'Host Destination', ComputeResource_Name, DataCenter_Name from dbo.VPX_EVENT EVTDEST where event_type = 'vim.event.VmBeingHotMigratedEvent'"

Query VirtualCentre for migration events
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select Create_Time as 'Relocation Start', UserName, VM_Name, Host_Name as 'Host Source', (select Host_Name from dbo.VPX_EVENT Where Chain_ID = EVTDEST.Chain_ID and event_type = 'vim.event.VmRelocatedEvent') as 'Host Destination', ComputeResource_Name, DataCenter_Name from dbo.VPX_EVENT EVTDEST where event_type = 'vim.event.VmBeingRelocatedEvent'"

Query VirtualCentre for DRS migration events
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select Create_Time as 'Relocation Finished', VM_Name, Host_Name as 'Host Destination', (select Host_Name from dbo.VPX_EVENT Where Chain_ID = EVTDEST.Chain_ID and event_type = 'vim.event.VMBeingHotMigratedEvent') as 'Host Source', ComputeResource_Name, DataCenter_Name from dbo.VPX_EVENT EVTDEST where event_type = 'vim.event.DrsVmMigratedEvent'"

Query the Virtual Centre/VC database for Virtual Machine details
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select ENT.Name as 'Name', DNS_Name as 'DNS Name', Guest_OS as 'OS', Mem_Size_MB as 'Mem', Num_VCPU as 'CPU', Num_NIC as 'NIC', IP_Address as 'IP', NET.MAC_Address as 'MAC Address', VM.FILE_Name as 'VMX location' from vpx_vm VM inner join VPX_GUEST_NET_ADAPTER NET on VM.ID = NET.VM_ID inner join VPX_ENTITY ENT on VM.ID = ENT.ID where dns_name like '%'"

Query the Virtual Centre/VC database for Virtual Machine snapshots (GMT+10)
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select ENT.Name as 'Name', DNS_Name as 'DNS Name', Guest_OS as 'OS', Mem_Size_MB as 'Mem', IP_Address as 'IP', VM.FILE_Name as 'VMX location', VM.Suspend_Time as 'Suspend Time', VM.Suspend_Interval as 'Suspend Interval', VMS.Snapshot_Name as 'Snapshot Name', VMS.Snapshot_Desc 'Snapshot Description', DateAdd(Hour, 10, VMS.Create_Time) as 'Snapshot Time', VMS.Is_Current_Snapshot 'Current Snapshot' from vpx_vm VM inner join VPX_GUEST_NET_ADAPTER NET on VM.ID = NET.VM_ID inner join VPX_ENTITY ENT on VM.ID = ENT.ID inner join VPX_SNAPSHOT VMS on VM.ID = VMS.VM_ID

Query the Virtual Centre/VC database for Virtual Machine LUN attachments
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select VMS.Name, TDS.Name from vpxv_vms VMS inner join vpxv_vm_datastore VVMDS on VMS.VMID = VVMDS.VM_ID inner join vpx_datastore TDS on VVMDS.DS_ID = TDS.ID Order by VMS.Name"

Query the Virtual Centre/VC database for Virtual Machine config LUN
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select VMS.Name as 'VM Name', TDS.Name as 'Config Datastore', SubString(ConfigFileName, CharIndex('/', ConfigFileName, Len('sanfs://vmfs'))+1, 255) as 'Config Path' from vpxv_vms VMS inner join vpx_datastore TDS on TDS.Storage_URL = SubString(VMS.ConfigFileName, 1, CharIndex('/', VMS.ConfigFileName, Len('sanfs://vmfs'))) Order By 'Config Datastore'"

Query the Virtual Centre/VC database for volume total and free disk space
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select Name, Cast(Round(Cast(Capacity as numeric)/1024/1024/1024, -1) as int) as 'Total Space', Cast(Round(Cast(Free_Space as numeric)/1024/1024/1024, -1) as int) as 'Free Space', Type from vpx_Datastore"

Find VM distribution on VMware cluster physical nodes
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "Select VH.DNS_NAME as 'Host', count(VM.HOST_ID) from vpx_vm VM inner join vpx_host VH on VM.HOST_ID = VH.ID group by VM.HOST_ID, VH.DNS_NAME order by 'Host'"

Query the Virtual Centre/VC database for VM CPU alarms
sqlcmd -S %sqlServer% -d %Database% -W -s "," -Q "select ALR.Created_Time, ENT.Name from vpx_alarm_runtime ALR inner join vpx_alarm AL on ALR.Alarm_ID = AL.Alarm_ID inner join vpx_entity ENT on ALR.Entity_ID = ENT.ID where AL.Name = 'Virtual Machine CPU Usage' order by created_time"

PowerShell VI Toolkit

Use the VI Toolkit Powershell snap-in to query for snapshots
Get-VM Get-Snapshot export-csv -path c:\temp\VMsnapshots.csv

Use the VI Toolkit Powershell snap-in to query for snapshot information
Get-VM Get-Snapshot foreach-object {$out= $_.VM.Name + "," + $_.Name + "," + $_.Description + "," + $_.PowerState; $out}

ESX Server Advanced Configuration Options

Hide LUNs with advanced ESX options
disk.MaskLUNs: vmhba0:0:4-5;vmhba1:0:4-5

Other

SNMP check against ESX
Snmputil.exe get %host% %community% .iso.org.dod.internet.private.enterprises.6876.1.1.0

Wayne's World of IT (WWoIT), Copyright 2008 Wayne Martin.