Labels

Sunday, June 28, 2009

Processor affinity on Windows server 2003/2008

This post describes a few methods of setting the processor affinity mask for a service, process or executable on a Windows Server 2003/2008 machine. There are several reasons to do this; increasingly as software becomes licensed per socket or even per core, but also separating or tuning the performance of applications on multi-core and hyper-threaded processors, i.e. to force usage of shared cache.

Of course this can be done through the task manager GUI (taskmgr.exe), but following the theme of command-line automation and scripting, I’ve found a few other methods that may be more efficient. The options below have different advantages/disadvantages and really depend on what you’re trying to achieve and how the process gets started.

In no particular order:

WSRM

Use Windows System Resource Manager, which among other things can be used for persistent application to processor mappings. This requires 2003 enterprise, 2003 datacenter or server 2008. WSRM is useful when trying to run multiple applications on a single physical box, allowing granular control through a GUI (the configuration of which is persistent across reboots). WSRM also has a command-line interface, and can be used for accounting and to set per-processor utilisation, not just processor affinity.

Note that when I was testing WSRM, it detected calc.exe and matched it against a rule to limit 50% CPU usage, but using factorial of a large number still resulted in 100% CPU. WSRM uses soft caps apparently, only constraining usage when there is contention (I didn’t do any more testing to confirm). There is also a note about WSRM only supporting management of non-OS processes (I’m unsure of whether it would detect calc.exe as such).

PowerShell

Use PowerShell to set the processor affinity for one or more running processes. There’s an example script below, setting the processor mask of calc.exe to the first 4 processors. I like this method because the script is simple, it would be easy to schedule, works on x86 and x64, supports multiple processes of the same name and at least partly because it highlights just how easy administration with PowerShell is.

Note that if you use factorial of a large number with calc.exe (n!) you’ll generate100% CPU which can be useful for testing. The mask below is 0xf = 1111 – a mask allowing use of only the first four processors:

$calcSet = Get-Process -ProcessName "calc"
foreach ($calc in $calcSet) {$calc.ProcessorAffinity=0xF}

Start or psexec

Use a wrapper to launch the process, including either ‘start /affinity’ or ‘psexec -a’ to launch the application, setting affinity as required. Note that ‘start /affinity’ is available on 2003 server, Vista and 2008 server, but not XP.

psexec /a 0,1 calc.exe
start /affinity 1 calc.exe


If you’re trying to control a service, you could use instsrv/srvany to create a service that wraps the start or psexec command around the real service binary. For example, the commands below create another version of the spooler service that will only run on the first processor.

instsrv Test c:\util\srvany.exe
reg add hklm\system\currentcontrolset\services\test\Parameters
reg add hklm\system\currentcontrolset\services\test\Parameters /v Application /t reg_sz /d cmd.exe
reg add hklm\system\currentcontrolset\services\test\Parameters /v AppParameters /t reg_sz /d "/c start /affinity 1 C:\WINDOWS\system32\spoolsv.exe"

ImageCfg

Use imagecfg.exe - a Windows Server 2000 Resource Kit Supplement utility - to modify the binary to set the processor affinity. This actually modifies the .exe – changing the initialisation section of the executable, and will ensure the affinity whenever the executable is run. Note that in my testing, modifying an x64 binary with this utility did not set the affinity correctly, only x86 binaries were successful (running on x64 or x86 OS). If you’re using calc.exe to test this – use a copy in another folder, otherwise windows file protection will replace the updated binary, undoing any change you make.

The mask below is 0xf = 1111 – a mask allowing use of only the first four processors:

imagecfg -a 0x0F c:\temp\calc.exe

ProcAff.exe

Use procaff.exe (a third-party utility) to set the processor affinity of a running process. This could be run as startup scheduled task, setting the mask of any process started as a service (you might need to build in some logic to wait for the processes to start). Note that this utility worked on both x86 and x64 systems while testing, but it’s limited in that if there are multiple processes with the same name, it requires a PID. This is easy enough to script using a ‘for’ loop and pslist to identify the PIDs and then call procaff.

procaff /set 1 calc.exe

Boot.ini and HAL options

Some rather more severe options – reducing the OS to only one CPU:

  1. The /NUMPROC switch in boot.ini on the server. This will set the number of processes Windows will see when booted. I assume this will just apply a mask at startup, but in a multi-CPU multi-core server I haven't tested to confirm the cores are paired or whether the first core from each CPU is presented.
  2. The /ONECPU switch in boot.ini on the server. This will tell Windows to use only one CPU, limiting any application running on the OS. On a multi-core server, the OS would use only one core of one package. I successfully tested this on 2003 enterprise OS.
  3. Downgrade the OS from a multiprocessor HAL to a uniprocessor HAL, removing support for multiple CPUs from the OS. I didn’t actually test this method, but it’s something I used to do on NT4, and I don’t see why it wouldn’t work on 2003/2008.



Plug-and-Play device interrupts

Use Intfilter.exe - allowing you to perform similar functionality to drivers – binding interrupts to a particular processor, useful to control plug-and-play driver-level CPU binding.

References

Procaff
http://www.stefan-kuhr.de/cms/index.php?option=com_content&view=article&id=60&Itemid=77

Boot INI Options Reference
http://technet.microsoft.com/en-us/sysinternals/bb963892.aspx

HAL options after Windows XP or Windows Server 2003 Setup
http://support.microsoft.com/kb/309283

Available Switch Options for Windows NT Boot.ini File
http://support.microsoft.com/kb/170756

How to Manually Add Support for a Second Processor
http://support.microsoft.com/kb/156358

Windows System Resource Manager: Frequently Asked Questions
http://www.microsoft.com/windowsserver2003/techinfo/overview/wsrmfaq.mspx

Windows System Resource Manager
http://www.microsoft.com/downloads/details.aspx?FamilyID=848306EF-F57E-4B3F-984D-50E9BCA44383&displaylang=en

Best Practices for Managing Applications with Process Control
http://technet.microsoft.com/en-us/library/bb742469.aspx

The Context Switch Action (xperf)
http://msdn.microsoft.com/en-us/library/cc305227.aspx

Monitoring Context Switches and Threads
http://technet.microsoft.com/en-us/library/cc938639.aspx

Analyzing Processor Activity
http://technet.microsoft.com/en-us/library/cc958310.aspx

Monitoring Activity on Multiprocessor Systems
http://technet.microsoft.com/en-us/library/cc938649.aspx

Windows Performance Toolkit x86 and x64 v4.1.1
http://download.microsoft.com/download/e/2/7/e2700369-d072-4fdc-a451-c3355eab0613/xperf_x86.msi

http://download.microsoft.com/download/e/2/7/e2700369-d072-4fdc-a451-c3355eab0613/xperf_x64.msi

Windows Performance Toolkit
http://msdn.microsoft.com/en-us/library/cc305187.aspx

Intfilter to manage drivers:
http://support.microsoft.com/default.aspx?scid=KB;en-us;252867


Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin. 


Read more!

Monday, June 1, 2009

Calling CPUID from PowerShell for Intel VT

I was trying to work out whether the Intel VT (VMX) extensions had been enabled on a server and whether the server support the SSE4.1 and SSE4.2 processor instructions - without having to reboot the server to check the BIOS. I ended up 'writing' a PowerShell script that compiles some C# to call a DLL written in assembler which has run the CPUID instruction and returned the results. Note that I didn't really write any of this, the PowerShell script compiling and running the C# is from the reference below, as is the actual C$ and asm dll.

Does this seem like a lot of effort? Sure does, but unfortunately the simple method - using Win32_Processor.ProcessorID was useless in this example, as for some reason MS chose to only return the EAX and EDX registers, whereas the bits I was after were returned in ECX. There may also be other ways to achieve this through PowerShell/.Net framework, rather than compiling c# to call unmanaged code.

Below is the PowerShell script, which contains the inline C# that is compiled and executed, which in turn requires the dll, using the EFLAGS register to confirm support and the CPUID instruction with eax=1 (if further supported) to query feature information from the CPU.

The original c# and asm dll: (see below for the hex text stream)
http://devpinoy.org/blogs/cvega/archive/2006/04/07/2658.aspx

The compile-csharp function: (adding the -unsafe parameter to allow pointer use)
http://monadblog.blogspot.com/2005/12/calling-win32-api-functions-through.html



#
# Description:
#  Call the CPUID function from an external DLL through C# inline code.
#  Report whether Intel VT (VMX) is supported, bit 5 of the ECX register returned from CPUID eax=1 call.
#  Note that WMI Win32_Processor:ProcessID contains the results of a call to CPUID - except that it only contains EAX and EDX, Intel VT (what I was after) is in ECX
#
# Author:
#  Wayne Martin, note that I didn't write any of this code, the inline C# was from the monad reference below, and the CPUID DLL and C# was from the first reference

#References:
# cpuid.dll and the namespace from: http://devpinoy.org/blogs/cvega/archive/2006/04/07/2658.aspx
# Intel cpuid: http://www.intel.com/Assets/PDF/appnote/241618.pdf
# http://monadblog.blogspot.com/2005/12/calling-win32-api-functions-through.html
# http://msdn.microsoft.com/en-us/library/aa394373.aspx


function Compile-Csharp ([string] $code, $FrameworkVersion="v2.0.50727",
[Array]$References)
{
    $cp = new-object Microsoft.CSharp.CSharpCodeProvider
    $cpar = New-Object System.CodeDom.Compiler.CompilerParameters
    $cpar.CompilerOptions = "-unsafe"      # unsafe to compile the code which uses pointers in the DLL call
    $cpar.GenerateInMemory = $true
    $cpar.GenerateExecutable = $false
    $cpar.OutputAssembly = "custom"
    # $cpar.ReferencedAssemblies.AddRange($refs)
    $cr = $cp.CompileAssemblyFromSource($cpar, $code)

    if ( $cr.Errors.Count)
    {
        $codeLines = $code.Split("`n");
        foreach ($ce in $cr.Errors)
        {
            write-host "Error: $($codeLines[$($ce.Line - 1)])"
            write-host $ce
           # $ce out-default
        }
        Throw "INVALID DATA: Errors encountered while compiling code"
    }
}



$code = @'
namespace CPUID
{
 using System;
 using System.Runtime.InteropServices;
 using System.Text;

 public class cpuid
 {
  private cpuid()
  {
  }

  [DllImport("cpuid.dll")]
     public static extern bool CPUIDIsSupported();

  [DllImport("cpuid.dll")]
     private unsafe static extern bool __cpuid
            (uint function, 
             int* eax, 
             int* ebx, 
             int* ecx, 
             int* edx);

  // Invoke __cpuid function
  public unsafe static bool Invoke
      (uint level, 
       out int eax, 
       out int ebx, 
       out int ecx, 
       out int edx)
  {
   int __eax = 0;
   int __ebx = 0;
   int __ecx = 0;
   int __edx = 0;

   if (__cpuid(level, &__eax, &__ebx, &__ecx, &__edx))
   {
    eax = __eax;
    ebx = __ebx;
    ecx = __ecx;
    edx = __edx;

    return true;
   }
   else
   {
    eax = 0;
    ebx = 0;
    ecx = 0;
    edx = 0;

    return false;
   }
  }
 }
}


'@


compile-CSharp $code
$eax = ""
$ebx = ""
$ecx = ""
$edx = ""
[CPUID.CPUID]::Invoke(1, [ref]$eax, [ref]$ebx, [ref]$ecx, [ref]$edx) 

$INTEL_CPUID_VT_FLAG = 0x0020                                                           # bit 5 of ECX returned from CPUID EAX=1
$INTEL_CPUID_SSE41 = 0x80000                                                            # bit 19 of ECX returned from CPUID EAX=1
$INTEL_CPUID_SSE42 = 0x100000                                                           # bit 20 of ECX returned from CPUID EAX=1
$INTEL_CPUID_XD = 0x00100000

write-output ("Intel VT: " + ($ecx -band $INTEL_CPUID_VT_FLAG) )                        # Does this processor support Intel VT / VMX?
write-output ("SSE 4.1: " + ($ecx -band $INTEL_CPUID_SSE41) )                           # Does this processor support SSE 4.1?
write-output ("SSE 4.2: " + ($ecx -band $INTEL_CPUID_SSE42) )                           # Does this processor support SSE 4.2?
write-output ("eXecute Disable: " + ($edx -band $INTEL_CPUID_XD) )                      # Does this processor support XD?




Note that in a previous post, I have written two powershell scripts, one to convert binary to a hex string, and vice versa. The following stream is the cpuid.dll file, if you write it to cpuid.txt and run the following command it will create the dll:



Powershell . .\HexStringToBinary.ps1 cpuid.txt cpuid.dll

4d5a90000300000004000000ffff0000b800000000000000400000000000000000000000000000000000000000000000000000000000000000000000c00000000e1fba0e00b409cd21b8014ccd21546869732070726f6772616d2063616e6e6f742062652072756e20696e20444f53206d6f64652e0d0d0a240000000000000071d4f7db35b5998835b5998835b59988c9958b8834b59988bbaa8a8834b599885269636835b59988000000000000000000000000000000000000000000000000504500004c0103006a793a440000000000000000e0000e210b01050c00020000000400000000000000100000001000000020000000000010001000000002000004000000040000000400000000000000004000000004000070fa000002000000000010000010000000001000001000000000000010000000002000005f0000000000000000000000000000000000000000000000000000000000000000000000003000000800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002e746578740000006c000000001000000002000000040000000000000000000000000000200000602e726461746100005f000000002000000002000000060000000000000000000000000000400000402e72656c6f6300000c000000003000000002000000080000000000000000000000000000400000420000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000558becb801000000c9c20c005352b8000000009c588bc83500002000509d9c5b33c325000020007507b801000000eb0233c0519d5a5bc3558bec5357e8cbffffff48740433c0eb1e8b45080fa28b7d0c89078b7d14890f8b7d1889178b7d10891fb8010000005f5bc9c214000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006a793a44000000003c2000000100000002000000020000002820000030200000382000000c1000003710000046200000572000000000010063707569642e646c6c0043505549444973537570706f72746564005f5f6370756964000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000


Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin. 


Read more!

Saturday, May 30, 2009

Extending a server 2003 VM system/boot disk

Whether using basic or dynamic disks, Windows Server 2003 doesn’t provide a method to extend the size of the system or boot disk. In VI35, I have used the following process to extend the system/boot disk of 2003 server virtual machines. Note that this is for the system/boot disk only, data volumes can be grown and extended live.

This process requires the VM to be shutdown, then a small outage while the disk is extended, then a reboot once the OS is first started.

The process I used in VI 3.5 U4, with a 2003 SP2 virtual machine:

  1. Turn the VM off - VM1 in this example
  2. Use VC to increase the size of the boot/system disk of VM1 by xGB (6 in my test)
  3. Use VC to attach the disk inside another running VM that can be shutdown, eg. VM2
  4. On VM2, load diskmgmt.msc, rescan disks (or run diskpart rescan)
  5. On VM2, start diskpart:
    1. List vol (get a list of volumes)
    2. Select vol x (where x is the number of the newly added disk)
    3. Extend
  6. Shutdown VM2 and detach the disk (don’t delete it!)
  7. Start VM1
  8. On my test, after logon, a setupapi message ‘Windows has finished installing new devices. Do you want to restart your computer now?’.
  9. Said yes and the server restarted, diskmgmt.msc shows correctly sized disk.
  10. Ran chkdsk c: to verify no corruption

Step 8 has a matching event ID 271 from PlugPlayManager and setupapi.log entry:

The Plug and Play operation cannot be completed because a device driver is
preventing the device from stopping. The name of the device driver is listed as
the vetoing service name below.
Vetoed device:
STORAGE\VOLUME\1&30A96598&0&SIGNATURE6AF3AEFAOFFSET7E00LENGTH53FD06C00
Vetoing device:
STORAGE\Volume\1&30a96598&0&Signature6AF3AEFAOffset7E00Length53FD06C00
Vetoing service name: FileSystem\Ntfs
Veto type 6: PNP_VetoDevice
When Windows attempts to install, upgrade, remove, or reconfigure a device,
it queries the driver responsible for that device to confirm that the operation
can be performed. If any of these drivers denies permission (query-removal
veto), then the computer must be restarted in order to complete the operation.
User Action
Restart your computer.


Setupapi.log driver logging showing the first vetoed attempt at installation and then the shadow copy volume snapshot install:
[2009/05/06 08:45:46 404.3 Driver Install]
#-019 Searching for hardware ID(s): storage\volume
#-198 Command line processed: C:\WINDOWS\system32\services.exe
#I393 Modified INF cache "C:\WINDOWS\inf\INFCACHE.1".
#W383 "volume.PNF" migrate: PNF Language = 0409, Thread = 0c09.
#I022 Found "STORAGE\Volume" in C:\WINDOWS\inf\volume.inf; Device: "Generic volume"; Driver: "Generic volume"; Provider: "Microsoft"; Mfg: "Microsoft"; Section name: "volume_install".
#I023 Actual install section: [volume_install.NTx86]. Rank: 0x00000000. Driver date: 10/01/2002. Version: 5.2.3790.3959.
#-166 Device install function: DIF_SELECTBESTCOMPATDRV.
#I063 Selected driver installs from section [volume_install] in "c:\windows\inf\volume.inf".
#I320 Class GUID of device remains: {71A27CDD-812A-11D0-BEC7-08002BE2092F}.
#I060 Set selected driver.
#I058 Selected best compatible driver.
#-166 Device install function: DIF_INSTALLDEVICEFILES.
#I124 Doing copy-only install of "STORAGE\VOLUME\1&30A96598&0&SIGNATURE6AF3AEFAOFFSET7E00LENGTH53FD06C00".
#-166 Device install function: DIF_REGISTER_COINSTALLERS.
#I056 Coinstallers registered.
#-166 Device install function: DIF_INSTALLINTERFACES.
#-011 Installing section [volume_install.NTx86.Interfaces] from "c:\windows\inf\volume.inf".
#I054 Interfaces installed.
#-166 Device install function: DIF_INSTALLDEVICE.
#I123 Doing full install of "STORAGE\VOLUME\1&30A96598&0&SIGNATURE6AF3AEFAOFFSET7E00LENGTH53FD06C00".
#W100 Query-removal during install of "STORAGE\VOLUME\1&30A96598&0&SIGNATURE6AF3AEFAOFFSET7E00LENGTH53FD06C00" was vetoed by "STORAGE\Volume\1&30a96598&0&Signature6AF3AEFAOffset7E00Length53FD06C00" (veto type 6: PNP_VetoDevice).
#W104 Device "STORAGE\VOLUME\1&30A96598&0&SIGNATURE6AF3AEFAOFFSET7E00LENGTH53FD06C00" required reboot: Query remove failed (install) CfgMgr32 returned: 0x17: CR_REMOVE_VETOED.
#I121 Device install of "STORAGE\VOLUME\1&30A96598&0&SIGNATURE6AF3AEFAOFFSET7E00LENGTH53FD06C00" finished successfully.
[2009/05/06 08:55:18 400.3 Driver Install]
#-019 Searching for hardware ID(s): storage\volumesnapshot
#-198 Command line processed: C:\WINDOWS\system32\services.exe
#W383 "volsnap.PNF" migrate: PNF Language = 0409, Thread = 0c09.
#I022 Found "STORAGE\VolumeSnapshot" in C:\WINDOWS\inf\volsnap.inf; Device: "Generic volume shadow copy"; Driver: "Generic volume shadow copy"; Provider: "Microsoft"; Mfg: "Microsoft"; Section name: "volume_snapshot_install".
#I023 Actual install section: [volume_snapshot_install.NTx86]. Rank: 0x00000000. Driver date: 10/01/2002. Version: 5.2.3790.3959.
#-166 Device install function: DIF_SELECTBESTCOMPATDRV.
#I063 Selected driver installs from section [volume_snapshot_install] in "c:\windows\inf\volsnap.inf".
#I320 Class GUID of device remains: {533C5B84-EC70-11D2-9505-00C04F79DEAF}.
#I060 Set selected driver.
#I058 Selected best compatible driver.
#-166 Device install function: DIF_INSTALLDEVICEFILES.
#I124 Doing copy-only install of "STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOT1".
#-166 Device install function: DIF_REGISTER_COINSTALLERS.
#I056 Coinstallers registered.
#-166 Device install function: DIF_INSTALLINTERFACES.
#-011 Installing section [volume_snapshot_install.NTx86.Interfaces] from "c:\windows\inf\volsnap.inf".
#I054 Interfaces installed.
#-166 Device install function: DIF_INSTALLDEVICE.
#I123 Doing full install of "STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOT1".
#I121 Device install of "STORAGE\VOLUMESNAPSHOT\HARDDISKVOLUMESNAPSHOT1" finished successfully.

References

How to extend a data volume in Windows Server 2003, in Windows XP, in Windows 2000, and in Windows Server 2008
http://support.microsoft.com/kb/325590


Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.


Read more!

Saturday, May 16, 2009

Generating 100% CPU with calc or PowerShell

Occasionally when testing something I want to generate 100% CPU load on a Windows computer. There are several utilities out there to do this, but that implies you have the utility on hand and are comfortable running it on the server. A colleague of mine (thanks Mark S.) showed me this nifty trick of using calc.exe to generate 100% CPU. The best thing about this is that every standard Windows OS installation has calc.exe.

This post provides two methods of generating 100% CPU load, the original calc.exe method, and a simple one line PowerShell command to do the same thing from the command-line (locally or on a remote server with PS v1 and psexec). Note that there may be a better method with PowerShell, I simply scripted the same operation calc was performing.

On dual-CPU/core computers this uses 100% of one CPU/core. To use more than that, calc or the PowerShell command can be run more than once, and Windows by default will run the new process on another less-busy CPU/core.

One practical application of this is to load-test a VMware VI3 cluster, generating 100% CPU on one or more VMs to see how ESX and DRS/VC handles the load. I have also used this in the past when testing multi-threaded applications and processor affinity to see how Windows allocates a processor.

calc.exe

Use calc to calculate the factorial of a number - the product of all integers from 1 up to and including the number specified, eg 5! = 1x2x3x4x5

  1. Run calc.exe and switch to scientific mode
  2. Type a large number (eg. 12345678901234567890), press the 'n!' button.
  3. Calc will ask to confirm after warning this will take a very long time
  4. 100% CPU utilisation will now occur (essentially forever)
PowerShell
 
Using the largest int32 positive integer, calculate the factorial to generate 100% CPU utilisation
$result = 1; foreach ($number in 1..2147483647) {$result = $result * $number};

Depending on how fast the CPU is, this could finish, so a loop to run the command above 2 billion times:
foreach ($loopnumber in 1..2147483647) {$result=1;foreach ($number in 1..2147483647) {$result = $result * $number};$result}

If you want to see how long the command takes to run:
Measure-Command {$result = 1; foreach ($number in 1..2147483647) {$result = $result * $number}}

Using the command-line then provides the ability to run the command remotely. To use psexec to remotely execute powershell v1 factorial to generate 100% CPU:
psexec \\%computername% /s cmd /c "echo. | powershell $result = 1; foreach ($number in 1..2147483647) {$result = $result * $number}"

Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin. 


Read more!

Monday, May 4, 2009

Converting VHD to VMDK SCSI for ESX

I had problems converting a 2003 server VHD to a vmdk that I could import into a VM running on ESX. I used WinImage to convert eh VHD->VMDK, but it seems WinImage creates the VMDK as an IDE device, which is unsupported by ESX. I'm sure there are better ways to do this, such as satisfying whatever the pre-requisites are to getting VMware converter to automatically inject the drivers, but it was interesting doing it manually.

Below is information on the process that I thought would have worked automatically, followed by the manual steps I took to make it work.

To convert the vmdk, the following process was first tried to get VMware converter to convert the IDE disk to something ESX would recognise:

  1. 1. Use WinImage to create the vmdk from the vhd
  2. 2. Use a VMware workstation VMX, including the disk as an IDE device. (a modified vmx is fine you don’t actually need VMware workstation)
  3. 3. Use VMware converter to import the workstation VMX into VC

I thought this would have been enough, but the VMware Converter process failed at 95% saying that it couldn’t find symmpi.sys. Symmpi.sys is the LSI Logic SCSI driver for the virtual SCSI adapter. I’m guessing that running VMware converter should automatically inject the drivers into the vmdk but couldn’t in this scenario (maybe because my local PC didn’t have the driver, or maybe because the driver cache cab files containing symmpi.sys weren’t on the target vmdk).

Powering on the machine resulted in a stop 0x7b inaccessible disk error. To manually fix the problem, I then:

  1. Added the disk to another VM. When starting the VM it warned that the new disk was created for LSI not buslogic. I said yes to convert to buslogic (which this VM was using as opposed to LSI).

The drive was then accessible through the VM, and I added the drivers (file and registry):

  1. Copied the driver file to the drivers directory: copy "\\%workingVM%\c$\WINDOWS\system32\drivers\symmpi.sys" "%mountedDrive%:\WINDOWS\system32\drivers"
  2. Copied the driver cache files to the machine from a working 2003: copy "\\%workingVM%\c$\WINDOWS\Driver Cache\i386\*.*" "%mountedDrive%:\WINDOWS\Driver Cache\i386"
  3. Exported the HKLM\SYSTEM\CurrentControlSet\Services\symmpiregistry and HKLM\SYSTEM\CurrentControlSet\Control\CriticalDeviceDatabase\pci#ven_1000&dev_0030 entries from a working server as regedit4 files (not Unicode).
  4. Loaded the %mountedDrive%:\windows\system32\config\system registry hive on the disk to HKLM\VM: reg load HKLM\VM %mountedDrive%:\windows\system32\config\system
  5. Modified the reg files to match the path the hive was loaded to (eg HKLM\VM\controlset001 instead of HKLM\system\currentcontrolset). The modified reg files are included below.
  6. Imported the registry files which modified the loaded system hive on the disk
  7. Disconnected the hive, shutdown the VM and disconnected the disk from the VM and reattached to the server created during the VMware Converter process
  8. Turned on the server and was prompted to convert the disk type back to LSI (which I did).
  9. The server started normally

Note that the conversion between buslogic and LSI logic in both directions were only required because the virtual machine that I mounted the disk on had a different adapter type.
---



REGEDIT4
[HKEY_LOCAL_MACHINE\VM\ControlSet001\Services\symmpi]
"ErrorControl"=dword:00000001
"Group"="SCSI miniport"
"Start"=dword:00000000
"Type"=dword:00000001
"ImagePath"=hex(2):73,79,73,74,65,6d,33,32,5c,44,52,49,56,45,52,53,5c,73,79,6d,\
6d,70,69,2e,73,79,73,00
"Tag"=dword:00000021

[HKEY_LOCAL_MACHINE\VM\ControlSet001\Services\symmpi\Parameters]
"BusType"=dword:00000001

[HKEY_LOCAL_MACHINE\VM\ControlSet001\Services\symmpi\Parameters\PnpInterface]
"5"=dword:00000001

[HKEY_LOCAL_MACHINE\VM\ControlSet001\Services\symmpi\Enum]
"0"="PCI\\VEN_1000&DEV_0030&SUBSYS_00000000&REV_01\\3&61aaa01&0&80"
"Count"=dword:00000001
"NextInstance"=dword:00000001



REGEDIT4
[HKEY_LOCAL_MACHINE\VM\ControlSet001\Control\CriticalDeviceDatabase\pci#ven_1000&dev_0030]
"Service"="symmpi"
"ClassGUID"="{4D36E97B-E325-11CE-BFC1-08002BE10318}"

References:
Injecting SCSI controller device drivers into Windows
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=1005208

Troubleshooting a virtual machine that fails to boot with STOP 0x7B error
http://kb.vmware.com/selfservice/viewContent.do?externalId=1006295&sliceId=1

Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.


Read more!

Saturday, May 2, 2009

VMware VI3 iSCSI with multiple non-HBA NICs

While researching iSCSI on VI3, I came across some interesting information when using the ESX iSCSI software initiator that would be applicable to many installations, highlighting a potential bottleneck.

The short version is that if you’re using the iSCSI software initiator connecting to a single iSCSI target, multiple uplinks in an ESX network team for the VMKernel iSCSI port would not be used for load balancing.

This can be easily proven by connecting to the service console and running esxtop (n) to view the traffic for individual network adapters. Assuming your storage is in use, one or more physical uplinks for the vSwitch handling iSCSI should be showing traffic. You can also use resxtop through the RCLI on ESXi.

Why this happens

My understanding is that current ESX software initiated iSCSI connections have a 1:1 relationship between NIC and iSCSI targets. An iSCSI target in this sense is a connection to the IP-based SAN storage, not LUN targets. This limitation applies when the SAN presents a single IP address for connectivity.

VI3 software initiated iSCSI doesn’t support multipathing, which within ESX leaves only load balancing the physical uplinks in a team. Unfortunately, that leaves load balancing up to the vSwitch load balancing policy exceptions. I don’t believe any of the three choices fit most scenarios when connectivity to the iSCSI is through a single MAC/IP:

  • Route based on the originating virtual switch port ID, based on virtual port ID, of which there is only one VMKernel iSCSI port
  • Route based on source MAC hash, based on source MAC, of which there is only one
  • Route based on IP hash, based on layer 3 source-destination IP pair, of which there is only one (VMKernel -> iSCSI virtual address). I don’t think this is a generally recommended load balancing approach anyway

Link aggregation

The VI3 SAN Deploy guide does state that one connection is established to each target. This seems to indicate one connection per LUN target, but the paragraph starts with software iSCSI and switches half way through to discuss iSCSI HBA’s.

I’m still unsure of whether software iSCSI has multiple TCP sessions, one per target (I don’t believe this is the case). The blog referenced below also talks about 802.3 link aggregation which states the ESX 3.x software initiator does not support multiple TCP sessions.

However, if multiple TCP sessions were being established for the iSCSI software initiator to a single target IP address, this opens the possibility of link aggregation at the physical switch. When using 802.3ad LACP in this IP-IP scenario, the switches would have to distribute connections based on the hash of TCP source/destination ports, rather than just IP/MAC.

The following excerpt from the SAN deploy guide:

Software iSCSI initiators establish only one connection to each target.

Therefore, storage systems with a single target that contains multiple LUNs have all LUN traffic routed through that one connection. In a system that has two targets, with one LUN each, two connections are established between the ESX host and the two available volumes. For example, when aggregating storage traffic from multiple connections on an ESX host equipped with multiple iSCSI HBAs, traffic for one target can be set to a specific HBA, while traffic for another target uses a different HBA. For more information, see the “Multipathing” section of the iSCSI SAN Configuration Guide. Currently, VMware ESX provides active/passive multipath capability. NIC teaming paths do not appear as multiple paths to storage in ESX host configuration displays, however. NIC teaming is handled entirely by the network layer and must be configured and monitored separately from ESX SCSI storage multipath configuration.



VI4/vSphere

Excerpts from the following blog, indicate that changes in vSphere for software iSCSI to support multiple iSCSI sessions, allowing multipathing or link aggregation, which would allow separate iSCSI TCP sessions to be spread across more than one NICs (depending on how many iSCSI sessions).

http://virtualgeek.typepad.com/virtual_geek/2009/01/a-multivendor-post-to-help-our-mutual-iscsi-customers-using-vmware.html

The current experience discussed above (all traffic across one NIC per ESX host):

VMware can’t be accused of being unclear about this. Directly in the iSCSI SAN Configuration Guide: ESX Server‐based iSCSI initiators establish only one connection to each target. This means storage systems with a single target containing multiple LUNs have all LUN traffic on that one connection, but in general, in my experience, this is relatively unknown.

This usually means that customers find that for a single iSCSI target (and however many LUNs that may be behind that target – 1 or more), they can’t drive more than 120-160MBps. This shouldn’t make anyone conclude that iSCSI is not a good choice or that 160MBps is a show-stopper. For perspective I was with a VERY big customer recently (more than 4000 VMs on Thursday and Friday two weeks ago) and their comment was that for their case (admittedly light I/O use from each VM) this was working well. Requirements differ for every customer.


The changes in vSphere:

Now, this behavior will be changing in the next major VMware release. Among other improvements, the iSCSI initiator will be able to use multiple iSCSI sessions (hence multiple TCP connections). Looking at our diagram, this corresponds with “multiple purple pipes”for a single target. It won’t support MC/S or “multiple orange pipes per each purple pipe” – but in general this is not a big deal (large scale use of MC/S has shown a marginal higher efficiency than MPIO at very high end 10GbE configurations) .

Multiple iSCSI sessions will mean multiple “on-ramps” for MPIO (and multiple “conversations” for Link Aggregation). The next version also brings core multipathing improvements in the vStorage initiative (improving all block storage): NMP round robin, ALUA support, and EMC PowerPath for VMware which integrates into the MPIO framework and further improves multipathing. In the spirit of this post, EMC is working to make PowerPath for VMware as heterogeneous as we can.

Together – multiple iSCSI sessions per iSCSI target and improved multipathing means aggregate throughput for a single iSCSI target above that 160MBps mark in the next VMware release, as people are playing with now. Obviously we’ll do a follow up post.


Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.


Read more!

Thursday, March 5, 2009

Vista Sidebar Gadget for SiteMeter

This post provides a Vista Sidebar gadget to report on sitemeter information for a web site being monitored. In a previous post I’ve provided a command-line PowerShell method to retrieve the same information, but I thought I’d try this GUI thing people keep talking about.

This is a very simple Vista sidebar gadget, it doesn’t have settings, or fly-outs, or links, or any other bits of cleverness, it was really just my first look into how Vista gadgets work. All I did was take the Microsoft hello-world example gadget and add a bit of javascript to (badly) scrape the sitemeter web page.

The gadget:
  1. Has a transparent PNG background image and writes the text in white on two lines, with a gadget size of 128x64
  2. Uses the MSXML DOM to issue a HTTP GET and for the sitemeter URL, using an asynchronous callback
  3. Parses the HTTP response, looking for the first index of the word ‘Today’ and extracts the number
  4. Updates the second field in the gadget with the current date/time to tell when the last successful get occurred
  5. Uses the SetTimeout method to sleep for an hour before calling the getData() function again
What the gadget doesn’t do that could make this better:
  1. Have the URL and timeouts stored in a settings file to save having to modify the JS when scraping a different URL or changing the timeout.
  2. Have a flyout or something which shows the other summary information on the sitemeter page
  3. Use a separate CSS and init() function rather than the in-line code from the example
To create this:
  1. Download the gadget samples from http://www.microsoft.com/downloads/details.aspx?FamilyID=b1e14e4f-3108-4c57-8b78-1157ca40dcc2
  2. Copy SDK_HelloWorld.gadget to *.zip
  3. Unzip the contents of the gadget zip file to a working directory and remove the read-only attribute from the files
  4. Create a transparent PNG background 64x64 in size (or use the one provided in this post), and overwrite Background.png
  5. Update images\aerologo.PNG with the png from this post
  6. Update HelloWorld.html with the contents below:
  7. Update the height and width to 128x64
    - Add the script tag:
    - Update the in-place init() funciton to call getData()
    - Create SiteMeter.js with the contents below
  8. Create a new directory and copy all of the gadget files to "%Systemdrive%\users\%username%\appdata\local\microsoft\Windows Sidebar\gadgets\SiteMeterCounter.gadget"
  9. Update the URL in SiteMeter.js with the page to scrape
  10. Add the gadget

HelloWorld.html

<!--
 *************************************************************************
 *
 * Name: SiteMeter.html
 *
 * Description: 
 * Simple SiteMeter counter
 * Displays the current count of today's visitors from the provided SiteMeter URL
 * 
 *
 * Modified from the 'Hello World' Microsoft example 
 ************************************************************************
-->
<html xmlns="http://www.w3.org/1999/xhtml">
    <head>
     <title>SiteMeter Count</title>
     <style type="text/css">
      body
      {
          width: 128px;
          height: 64px;
                        font-family: calibri;
                        color: white; 
      }
      #gadgetContent
      {
          width: 128px;
                        top: 3px;
          text-align: center;
                        overflow: hidden;
                        font-weight: bold;
                        font-size: 14px;
      }
      #lastUpdate
      {
          width: 128px;
                        top: 20px;
          text-align: center;
                        overflow: hidden;

                        font-size: 9px;
      }
     </style>
     <script type="text/javascript" src="SiteMeter.js"></script>
     <script type="text/jscript" language="jscript">


        // --------------------------------------------------------------------
        // Initialize the gadget.
        // --------------------------------------------------------------------
     function init()
     {
         var oBackground = document.getElementById("imgBackground");
         oBackground.src = "url(images/background.png)";
                getData();
        }
     </script>
    </head>
 
<body onload="init()">
    <g:background id="imgBackground">
 <span id="gadgetContent">-</span>
 <span id="lastUpdate">-</span>
 </g:background>
</body>
</html>



SiteMeter.js

var globalURL = "http://www.sitemeter.com/default.asp?a=stats&s=s451qaz2wsx";              // URL for lookup
var globalTimeoutId;
var globalError;
var XMLHttp;

function getData()
{
    try 
        { XMLHttp = new ActiveXObject("Msxml2.XMLHTTP"); 
    } 
    catch (e) 
        { XMLHttp = new ActiveXObject("Microsoft.XMLHTTP"); 
    }

    // Timeout, if call not come back in 30 seconds, abort with the default error message.
    globalTimeoutId = setTimeout(function() {
                                         XMLHttp.abort();
                                         document.getElementById("gadgetContent").innerHTML = "Timeout";
                                         }, 30000); 
        
    // Asynchronous get
    XMLHttp.open("GET", globalURL, true);

    XMLHttp.setRequestHeader("Content-Type","application/x-www-form-urlencoded");
    XMLHttp.setRequestHeader("Connection", "close");

    // Callback for when the page has loaded
    XMLHttp.onreadystatechange = parseData

    // Send the request
    XMLHttp.send(null);

}

function parseData()
{
    var Results;
    var td;

    gadgetContent.innerHTML = XMLHttp.status;

    if (XMLHttp.readyState == 4 && XMLHttp.status == 200)
    {

        var date = new Date();
        lastUpdate.innerHTML = date.getDate() + "/" + (date.getMonth() +1)  + "/" + date.getYear() + " " + date.getHours() + ":" + date.getMinutes() + ":" + date.getSeconds();

        globalError = 0;

        Results = XMLHttp.responsetext;
        if (Results.indexOf("Today") >= 0)
        {

            td = Results.substring(Results.indexOf("Today")+20, Results.indexOf("Today")+80);
            td = td.substring(td.indexOf("<font"), td.length);
            td = td.substring(0, td.indexOf("</font>")+7);

            gadgetContent.innerHTML = td;
        }
        else
        {   
            globalError = 1;
            document.getElementById("gadgetContent").innerHTML = "Error";
        }

        // Set a callback for the getData function in one hour
        clearTimeout(globalTimeoutId);
        globalTimeoutId = setTimeout(getData, 3600000);
    }

} 


Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.


Read more!

Tuesday, February 24, 2009

Security with 2003 R2 FSRM quotas

This post provides information on several aspects of security with Windows Server 2003 R2 FSRM quotas, which you may come across if you use the previous two posts to implement FSRM quotas on a standalone server or MSCS clustered node.

LocalService Command Notification Task

If your quota has a notification task that runs a command, you may need to set the security of the executable or areas accessed by the command. By default, FSRM runs commands as the LocalService account - an account with restricted local permissions.

In this example, the command executed is the logentry.bat batch file from the previous post, which simply writes a log entry to a file. To allow the batch file to run as the localservice account – which is a member of the 'Authenticated Users' group, I set the following permissions:

  • Add authenticated users:R to the root (not OICI - object inherit, container inherit) using SDDL with cacls
  • Add authenticated users:R to c:\scripts to execute the logentry.bat file
  • Add authenticated users:C to c:\logs to allow writing the log entry
  • Ensure that cmd.exe can be executed (default permissions should suffice)

This was done with the following commands:

  • cacls c:\ /S:"D:PAI(A;OICI;FA;;;BA)(A;;0x1200a9;;;AU)(A;OICI;FA;;;SY)"
  • cacls c:\scripts /e /g "authenticated users":R
  • cacls c:\logs /e /g "authenticated users":C

Note that while testing this, I was using localsystem (unrestricted local permissions) to verify that security was the issue, but in an MSCS cluster when failing over the virtual server from one cluster node to the other, the quota template reset itself to localservice instead of localsystem.

Before setting permissions to allow LocalService to run the script, this left the notification task not executing successfully. Depending upon what your command does you may require elevated local or remote privilege, for which you may want to use localsystem or networkservice.

Target Permissions preventing execution

When specifying a command to run as a notification task, 2003 R2 FSRM will not allow running a command which standard users have change/full control to, directly or through inheritance.

To me this seems like an unusual approach for an out of the box Microsoft product and isn't very intuitive, as in my lab inherited permissions from a parent directory allowing Users:F were causing the task to fail.


References

Implementing 2003 FSRM quotas Command-line
http://waynes-world-it.blogspot.com/2009/02/implementing-2003-fsrm-quotas.html

2003 FSRM and NTFS Quotas compared
http://waynes-world-it.blogspot.com/2009/02/2003-fsrm-and-ntfs-quotas-compared.html

LocalService Account
http://msdn.microsoft.com/en-us/library/ms684188(VS.85).aspx

Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.


Read more!

Tuesday, February 10, 2009

Implementing 2003 FSRM quotas

This post provides information on implementing Windows Server 2003 FSRM quotas using the command-line dirquota.exe utility, creating a template and then applying that template to the filesystem using an autoquota. Example control files are provided to create the template, and a simple batch file to create a daily log file with quota notifications.

The first command below creates the soft 100MB quota template with three thresholds (85/99/150), and four notifications for those thresholds, two for email and one each for and command execution and event log entry. The second command creates an autoquota using the template on the specified remote server against the specified path. Note that in this example, the commands were run on a 64-bit physical server running MSCS, and the target server was a virtual cluster server (v01).
  • dirquota template add /Remote:v01 /Template:Test_Quota /Limit:100mb /type:Soft /label:"Test Default Quota" /add-threshold:85 /add-notification:85,M,c:\admin\control\Test_Quota_Email.txt /add-threshold:99 /add-notification:99,M,c:\admin\control\Test_Quota_Email.txt /add-notification:99,C,c:\admin\control\Test_Quota_Command.txt /add-notification:99,E,c:\admin\control\Test_Quota_Event.txt /add-threshold:150 /add-notification:150,M,c:\admin\control\Test_Quota_Email.txt
  • dirquota autoquota add /Remote:v01 /path:q:\folder1 /sourcetemplate:"Test_Quota"
Notes:

  1. The use of 99% instead of 100% was intentional, as 100% is not a notification; it’s a limit, so the pre-defined variables are different. This is obvious in the subject of the email, where the ‘[Quota Threshold]’ variable isn’t resolved when it’s a 100% 'notification'.
  2. The quota information is logged to \\p01\c$\logs\QuotaUsage_YYYYMMDD.log for 99% of quota usage, as well as sending an email to DiskUsageMonitor and logging an event on the cluster, and all quota information is available through the FSRM MMC snap-in.
  3. Within a single email notification, an email can be sent to one or more administrators, and/or the person who took the limit over the threshold – the owner of the file. In the template below, emails will be sent to both users for all three thresholds, and administrators for the 150% notification. E-mail notifications will be limited to one per day for the same notification.
  4. Any command can be run; in the example here a simple batch file is run that appends a log entry to a daily log file, providing an easy method to see quota alerts for each day.
  5. The notifications aren’t triggered until first create, so if you apply quotas to existing data, the notifications won’t start appearing until new data is written.
  6. A quirk with the 64-bit OS - creating the quota template only works with the 32-bit version of the dirquota.exe utility. If the physical server (p01 in the example above) were a 64-bit server, you would have to run the 32-bit dirquota.exe utility to create the template and autoquota (2003 enterprise R2 x64 SP1).
Configuration Files

Templates, quotas and autoquotas can all be created via command-line utilities. When creating templates with notifications, the information is supplied via control files. Example control files for running a command, logging an event, and sending an email are shown below.

Note the following global properties can be specified in each notification:
  • Notification – m | e | c | r
  • m - an e-mail notification
  • e - an event log notification
  • c - a command or script execution
  • r - a report generation
  • RunLimitInterval – The number of minutes to wait between sending notifications to save repeated unnecessary notifications. A setting of 0 indicates a notification will be sent on each trigger.


Test_Quota_Command.txt

Notification=c
Command=c:\windows\system32\cmd.exe
Arguments=/c c:\admin\scripts\logentry.bat "%Date%,%Time%,[Source Io Owner],[Quota Path],[Server],[Quota Limit MB] MB,[Quota Used MB] MB,[Quota Used Percent]"
MonitorCommand=Disable
Account=LocalService
LogResult=Enable
RunLimitInterval=0



Test_Quota_Event.txt

Notification=e
RunLimitInterval=1440
EventType=Warning
Message=Excessive usage by [Source Io Owner] on [Quota Path], shared on [Server]. Limit of [Quota Limit MB] MB, [Quota Used MB] MB in use ([Quota Used Percent]% of limit).



Test_Quota_Email.txt

Notification=m
RunLimitInterval=1440
To=[Source Io Owner Email]
From=FSRM@server.domain.com
ReplyTo=FSRM-DoNotReply@server.domain.com
Cc=DiskUsageMonitor@domain.com
Subject=[Quota Threshold]% quota threshold exceeded
Message=A file written by [Source Io Owner] has exceeded the [Quota Threshold]% quota threshold for the quota on [Quota Path] on server [Server]. \
\
The quota limit is [Quota Limit MB] MB, and [Quota Used MB] MB currently is in use ([Quota Used Percent]% of limit).\
\
\
\


Logging to File
I'm not sure why a method to append to a log file wasn't included in the GUI, but this batch file appends a one-line entry to a rolling log file:



:: LogEntry.bat

:: Write a log for quota alerts

::%Date%,%Time%,[Source Io Owner],[Quota Path],[Server],[Quota Limit MB] MB,[Quota Used MB] MB,[Quota Used Percent]%

Set AdminLog=C:\Logs

for /f "tokens=1-8 delims=/:. " %%i in ('echo %date%') do Set DateFlat=%%l%%k%%j
Set LogFile=%AdminLog%\QuotaUsage_%DateFlat%.log

Echo %~1 >> %LogFile%



References

2003 FSRM and NTFS Quotas compared
http://waynes-world-it.blogspot.com/2009/02/2003-fsrm-and-ntfs-quotas-compared.html

FSRM and NTFS Quotas in 2003 R2
http://waynes-world-it.blogspot.com/2008/06/fsrm-and-ntfs-quotas-in-2003-r2.html

Configuration files for notifications in File Server Resource Manager
http://technet2.microsoft.com/windowsserver2008/en/library/a4426339-5345-44d5-81b7-a35a703daaac1033.mspx?mfr=true

How to use File Server Resource Manager (FSRM) to configure the notification feature for File Screening Management in Windows Server 2003 R2
http://support.microsoft.com/kb/926092

File Server Resource Manager Protocol Specification
http://download.microsoft.com/download/9/5/E/95EF66AF-9026-4BB0-A41D-A4F81802D92C/%5BMS-FSRM%5D.pdf

Dirquota admin options
http://technet2.microsoft.com/windowsserver2008/en/library/14c2a340-54cf-46fa-8b0d-beed6c220c671033.mspx?mfr=true

LocalService Account
http://msdn.microsoft.com/en-us/library/ms684188(VS.85).aspx

Configuration files for notifications in File Server Resource Manager
http://technet2.microsoft.com/windowsserver2008/en/library/a4426339-5345-44d5-81b7-a35a703daaac1033.mspx?mfr=true

Create an auto quota
http://technet2.microsoft.com/WindowsServer/en/library/0de5535e-ef25-4ffa-a724-155573044ddc1033.mspx

Dirquota autoquota
http://technet2.microsoft.com/WindowsServer2008/en/library/2809c575-8d93-47cb-8bfc-a427da83cc2a1033.mspx


File Services
http://technet2.microsoft.com/windowsserver2008/en/library/6e5bb377-db25-4603-b1ff-ecc4f6c29b691033.mspx?mfr=true

Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.


Read more!

Monday, February 9, 2009

2003 FSRM and NTFS Quotas compared

This post provides a quick comparison of 2003 FSRM and NTFS quotas, which I find useful when explaining how quotas in 2003 R2 work, as opposed to (or combined with) NTFS quotas. Also included is information on where the quota data is stored, and some methods to view that data.

Quota metafile information has been part of the NT filesystem since Windows NT 3.5, but has not been supported by the Operating System since the implementation of NTFS 5.0 – available in Windows 2000 and later.

When enabled, NTFS quotas track information as part of each write operation to the filesystem, providing a per-volume mapping between user SID and logical disk usage based on file ownership. While all the necessary information is stored in the NT filesystem, managing NTFS quotas is time-consuming and challenging for administrators.

Windows Server 2003 R2 File System Resource Manager (FSRM) provides a filesystem minifilter to control quotas, and a much improved interface to manage and report on quotas from a per-folder perspective.

The main differences between the two distinct quota methods available in 2003 R2 are that:

  1. FSRM provides per-folder quotas, as opposed to per-user/volume NTFS quotas. Regardless of file ownership, files in a folder will count towards the FSRM-set limits.
  2. SMB calls to return the free disk space are based on hard quotas at the root of the share or volume, not the quota applied to a folder - regardless of the share access point. NTFS hard quotas are volume-wide, and disk space is presented based on used-hard quota total, regardless of the share root or access method (remote SMB or local).
  3. FSRM quotas count only the size on disk of files, as opposed to NTFS quotas which count the logical uncompressed size. This is primarily considered for NTFS compressed files, but is presumably the same for offline files.
  4. FSRM quotas are controlled by a file system mini-filter storing quota data in \System Volume Information\SRM\quota.md and quota.xml, as opposed to NTFS quotas which are stored as part of the filesystem in \$Extend\$Quota file in $INDEX_ROOT NTFS attributes
  5. FSRM allows autoquota's, a concept of setting a quota at a top-level directory and each direct child subdirectory automatically inherits a copy of that quota. This provides an easy method of exception-based quotas. Managing NTFS quotas is GUI-based unless the WMI automation interface is used and an NTFS quota entry is automatically created for each new user SID.
  6. FSRM provides much improved reporting and alerting for quotas, whereas NTFS quotas only provide rudimentary reporting and eventlog entry alerting.
  7. FSRM has no supported automation interface to manage quotas, whereas NTFS quotas can be managed by WMI. However, the .Net assembly srmlib.dll provides an undocumented framework for managing FSRM quotas, which could be scripted through PowerShell if required.
  8. FSRM provides very strong support for command-line administration with dirquota.exe, with NTFS quotas having limited support available through fsutil
  9. In a MSCS cluster scenario, FSRM stores settings in the registry, located in HKLM\Cluster\SRM\Settings\SrmGlobalSettings\Data. NTFS quotas have all information stored on the filesystem, making both methods functional in a MSCS server cluster with shared storage.
  10. FSRM quotas provide improved notification - including in-built email, event logging, running a command or triggering a report.
  11. FSRM quotas allow for templates to be created, separating the creation of a standard set of quotas from the application of those quotas. This allows scalability and much improved process control.

How FSRM quota information is stored

FSRM quotas are stored in the "?:\System Volume Information\SRM\quota.xml" and "?:\System Volume Information\SRM\quota.sd" files, with the XML containing the configuration, and the SD file containing the actual quota information.

To see the configuration of FSRM quotas for a particular volume:

• psexec /s /i /d cmd.exe
• xcopy /h "?:\System Volume Information\SRM\quota.xml" %temp%
• attrib -r -s -h "%temp%\quota.xml"

The SD file is secured so only system can access, is marked as system/hidden and is locked by the mini-filter. One method to view the SD:

• psexec /s /i /d cmd.exe
• nfi "h:\System Volume Information\SRM\quota.md"
• diskedit Read Sectors (as returned by nfi)

How NTFS quota information is stored

NTFS stores quota information in a metafile on each volume called \$Extend\$Quota, with the information contained in the INDEX_ROOT $O and $Q NTFS attributes. Nfi.exe and diskedit.exe can be used to identify the file, and view the data contained in the logical sectors.

nfi q:

File 24
\$Extend\$Quota
$STANDARD_INFORMATION (resident)
$FILE_NAME (resident)
$INDEX_ROOT $O (resident)
$INDEX_ROOT $Q (resident)
$INDEX_ALLOCATION $Q (nonresident)
logical sectors 1036140-1036147 (0xfcf6c-0xfcf73)
$BITMAP $Q (resident)

Quota Minifilter driver

FSRM quotas use a minifilter driver to function – quota.sys – mounted by default in the I/O stack with an altitude of 125000 as part of the ‘FSFilter Physical Quota Management’ group. While this altitude can be changed by modifying a registry value, this is not recommended.

Both the R2 file screen filter (260800) and the cluster file system (200000-209999) are loaded higher in the stack then the quota minifilter.


fltmc filters & fltmc instances

Filter Name Num Instances Frame
------------------------------ ------------- -----
DfsDriver
Datascrn 0 0
Quota 1 0

Filter Volume Name Altitude Instance Name
----------------------------- -----------------------------
Quota Q: 125000 Quota


To detach the filter from a volume, the following command can be run:
• fltmc detach [volume:]

Note that doing so leaves the SRM directory in the ‘System Volume Information’ on the volume, and during testing when fltmc was used to reattach the quota filter to the volume, the previous quotas were seen as invalid and returned errors.


References

FSRM and NTFS Quotas in 2003 R2
http://waynes-world-it.blogspot.com/2008/06/fsrm-and-ntfs-quotas-in-2003-r2.html

Inside Win2K NTFS, Part 1
http://msdn.microsoft.com/en-us/library/ms995846.aspx

You cannot create quotas on File Server Resource Manager (FSRM) in Windows Server 2003 R2
http://support.microsoft.com/kb/555941

FSRM quota information does not appear in the NTFS file system Quota Entries window, and NTFS file system disk quota information does not appear in FSRM in Windows Server 2003 R2
http://support.microsoft.com/kb/915042

Limited Group Policy management for NTFS quotas.
http://technet2.microsoft.com/windowsserver/en/library/2d82decb-6726-4c5c-b872-1658b0fc3e3e1033.mspx?mfr=true

Disk Quotas Tools and Settings
http://technet2.microsoft.com/windowsserver/en/library/3b5b242b-7bb2-48e4-8e5f-224a08b36b271033.mspx

HOW TO: Configure Disk Quotas for a Shared Disk in a Cluster
http://support.microsoft.com/kb/278365

Disk Quotas Tools and Settings
http://technet2.microsoft.com/windowsserver/en/library/3b5b242b-7bb2-48e4-8e5f-224a08b36b271033.mspx

Managing Disk Quotas in Windows Server 2003 and Windows XP
http://www.microsoft.com/technet/scriptcenter/topics/win2003/quotas.mspx

Designing a Disk Quota Strategy
http://technet2.microsoft.com/windowsserver/en/library/1EE8754E-48D6-4472-9B53-29E8D1DE09F81033.mspx

Quotas in a cluster:
http://support.microsoft.com/kb/278365

How Disk Quotas Work
http://technet2.microsoft.com/windowsserver/en/library/5becbcd6-8da3-4c3b-bc0e-258acd3ec1811033.mspx?mfr=true

Disk Quotas and Free Space
http://www.microsoft.com/technet/prodtechnol/windows2000serv/reskit/core/fncd_str_ctkj.mspx?mfr=true

Quota Minifilter Driver
http://technet2.microsoft.com/windowsserver2008/en/library/7c5a0b98-d963-4a1d-a499-316322746a8e1033.mspx?mfr=true

MUP Changes in Microsoft Windows Vista
http://msdn.microsoft.com/en-us/library/aa488427.aspx

File System Minifilter Load Order Groups and Altitude Ranges
http://www.microsoft.com/whdc/driver/filterdrv/alt-range.mspx


Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin.


Read more!

All Posts

printQueue AD objects for 2003 ClusterVirtualCenter Physical to VirtualVirtual 2003 MSCS Cluster in ESX VI3
Finding duplicate DNS recordsCommand-line automation – Echo and macrosCommand-line automation – set
Command-line automation - errorlevels and ifCommand-line automation - find and findstrBuilding blocks of command-line automation - FOR
Useful PowerShell command-line operationsMSCS 2003 Cluster Virtual Server ComponentsServer-side process for simple file access
OpsMgr 2007 performance script - VMware datastores...Enumerating URLs in Internet ExplorerNTLM Trusts between 2003 and NT4
2003 Servers with Hibernation enabledReading Shortcuts with PowerShell and VBSModifying DLL Resources
Automatically mapping printersSimple string encryption with PowerShellUseful NTFS and security command-line operations
Useful Windows Printer command-line operationsUseful Windows MSCS Cluster command-line operation...Useful VMware ESX and VC command-line operations
Useful general command-line operationsUseful DNS, DHCP and WINS command-line operationsUseful Active Directory command-line operations
Useful command-linesCreating secedit templates with PowerShellFixing Permissions with NTFS intra-volume moves
Converting filetime with vbs and PowerShellDifference between bat and cmdReplica Domain for Authentication
Troubleshooting Windows PrintingRenaming a user account in ADOpsMgr 2007 Reports - Sorting, Filtering, Charting...
WMIC XSL CSV output formattingEnumerating File Server ResourcesWMIC Custom Alias and Format
AD site discoveryPassing Parameters between OpsMgr and SSRSAnalyzing Windows Kernel Dumps
Process list with command-line argumentsOpsMgr 2007 Customized Reporting - SQL QueriesPreventing accidental NTFS data moves
FSRM and NTFS Quotas in 2003 R2PowerShell Deleting NTFS Alternate Data StreamsNTFS links - reparse, symbolic, hard, junction
IE Warnings when files are executedPowerShell Low-level keyboard hookCross-forest authentication and GP processing
Deleting Invalid SMS 2003 Distribution PointsCross-forest authentication and site synchronizati...Determining AD attribute replication
AD Security vs Distribution GroupsTroubleshooting cross-forest trust secure channels...RIS cross-domain access
Large SMS Web Reports return Error 500Troubleshooting SMS 2003 MP and SLPRemotely determine physical memory
VMware SDK with PowershellSpinning Excel Pie ChartPoke-Info PowerShell script
Reading web content with PowerShellAutomated Cluster File Security and PurgingManaging printers at the command-line
File System Filters and minifiltersOpsMgr 2007 SSRS Reports using SQL 2005 XMLAccess Based Enumeration in 2003 and MSCS
Find VM snapshots in ESX/VCComparing MSCS/VMware/DFS File & PrintModifying Exchange mailbox permissions
Nested 'for /f' catch-allPowerShell FindFirstFileW bypassing MAX_PATHRunning PowerSell Scripts from ASP.Net
Binary <-> Hex String files with PowershellOpsMgr 2007 Current Performance InstancesImpersonating a user without passwords
Running a process in the secure winlogon desktopShadow an XP Terminal Services sessionFind where a user is logged on from
Active Directory _msdcs DNS zonesUnlocking XP/2003 without passwords2003 Cluster-enabled scheduled tasks
Purging aged files from the filesystemFinding customised ADM templates in ADDomain local security groups for cross-forest secu...
Account Management eventlog auditingVMware cluster/Virtual Center StatisticsRunning scheduled tasks as a non-administrator
Audit Windows 2003 print server usageActive Directory DiagnosticsViewing NTFS information with nfi and diskedit
Performance Tuning for 2003 File ServersChecking ESX/VC VMs for snapshotsShowing non-persistent devices in device manager
Implementing an MSCS 2003 server clusterFinding users on a subnetWMI filter for subnet filtered Group Policy
Testing DNS records for scavengingRefreshing Computer Account AD Group MembershipTesting Network Ports from Windows
Using Recovery Console with RISPAE Boot.ini Switch for DEP or 4GB+ memoryUsing 32-bit COM objects on x64 platforms
Active Directory Organizational Unit (OU) DesignTroubleshooting computer accounts in an Active Dir...260+ character MAX_PATH limitations in filenames
Create or modify a security template for NTFS perm...Find where a user is connecting from through WMISDDL syntax in secedit security templates

About Me

I’ve worked in IT for over 20 years, and I know just about enough to realise that I don’t know very much.