Labels

Sunday, June 28, 2009

Processor affinity on Windows server 2003/2008

This post describes a few methods of setting the processor affinity mask for a service, process or executable on a Windows Server 2003/2008 machine. There are several reasons to do this; increasingly as software becomes licensed per socket or even per core, but also separating or tuning the performance of applications on multi-core and hyper-threaded processors, i.e. to force usage of shared cache.

Of course this can be done through the task manager GUI (taskmgr.exe), but following the theme of command-line automation and scripting, I’ve found a few other methods that may be more efficient. The options below have different advantages/disadvantages and really depend on what you’re trying to achieve and how the process gets started.

In no particular order:

WSRM

Use Windows System Resource Manager, which among other things can be used for persistent application to processor mappings. This requires 2003 enterprise, 2003 datacenter or server 2008. WSRM is useful when trying to run multiple applications on a single physical box, allowing granular control through a GUI (the configuration of which is persistent across reboots). WSRM also has a command-line interface, and can be used for accounting and to set per-processor utilisation, not just processor affinity.

Note that when I was testing WSRM, it detected calc.exe and matched it against a rule to limit 50% CPU usage, but using factorial of a large number still resulted in 100% CPU. WSRM uses soft caps apparently, only constraining usage when there is contention (I didn’t do any more testing to confirm). There is also a note about WSRM only supporting management of non-OS processes (I’m unsure of whether it would detect calc.exe as such).

PowerShell

Use PowerShell to set the processor affinity for one or more running processes. There’s an example script below, setting the processor mask of calc.exe to the first 4 processors. I like this method because the script is simple, it would be easy to schedule, works on x86 and x64, supports multiple processes of the same name and at least partly because it highlights just how easy administration with PowerShell is.

Note that if you use factorial of a large number with calc.exe (n!) you’ll generate100% CPU which can be useful for testing. The mask below is 0xf = 1111 – a mask allowing use of only the first four processors:

$calcSet = Get-Process -ProcessName "calc"
foreach ($calc in $calcSet) {$calc.ProcessorAffinity=0xF}

Start or psexec

Use a wrapper to launch the process, including either ‘start /affinity’ or ‘psexec -a’ to launch the application, setting affinity as required. Note that ‘start /affinity’ is available on 2003 server, Vista and 2008 server, but not XP.

psexec /a 0,1 calc.exe
start /affinity 1 calc.exe


If you’re trying to control a service, you could use instsrv/srvany to create a service that wraps the start or psexec command around the real service binary. For example, the commands below create another version of the spooler service that will only run on the first processor.

instsrv Test c:\util\srvany.exe
reg add hklm\system\currentcontrolset\services\test\Parameters
reg add hklm\system\currentcontrolset\services\test\Parameters /v Application /t reg_sz /d cmd.exe
reg add hklm\system\currentcontrolset\services\test\Parameters /v AppParameters /t reg_sz /d "/c start /affinity 1 C:\WINDOWS\system32\spoolsv.exe"

ImageCfg

Use imagecfg.exe - a Windows Server 2000 Resource Kit Supplement utility - to modify the binary to set the processor affinity. This actually modifies the .exe – changing the initialisation section of the executable, and will ensure the affinity whenever the executable is run. Note that in my testing, modifying an x64 binary with this utility did not set the affinity correctly, only x86 binaries were successful (running on x64 or x86 OS). If you’re using calc.exe to test this – use a copy in another folder, otherwise windows file protection will replace the updated binary, undoing any change you make.

The mask below is 0xf = 1111 – a mask allowing use of only the first four processors:

imagecfg -a 0x0F c:\temp\calc.exe

ProcAff.exe

Use procaff.exe (a third-party utility) to set the processor affinity of a running process. This could be run as startup scheduled task, setting the mask of any process started as a service (you might need to build in some logic to wait for the processes to start). Note that this utility worked on both x86 and x64 systems while testing, but it’s limited in that if there are multiple processes with the same name, it requires a PID. This is easy enough to script using a ‘for’ loop and pslist to identify the PIDs and then call procaff.

procaff /set 1 calc.exe

Boot.ini and HAL options

Some rather more severe options – reducing the OS to only one CPU:

  1. The /NUMPROC switch in boot.ini on the server. This will set the number of processes Windows will see when booted. I assume this will just apply a mask at startup, but in a multi-CPU multi-core server I haven't tested to confirm the cores are paired or whether the first core from each CPU is presented.
  2. The /ONECPU switch in boot.ini on the server. This will tell Windows to use only one CPU, limiting any application running on the OS. On a multi-core server, the OS would use only one core of one package. I successfully tested this on 2003 enterprise OS.
  3. Downgrade the OS from a multiprocessor HAL to a uniprocessor HAL, removing support for multiple CPUs from the OS. I didn’t actually test this method, but it’s something I used to do on NT4, and I don’t see why it wouldn’t work on 2003/2008.



Plug-and-Play device interrupts

Use Intfilter.exe - allowing you to perform similar functionality to drivers – binding interrupts to a particular processor, useful to control plug-and-play driver-level CPU binding.

References

Procaff
http://www.stefan-kuhr.de/cms/index.php?option=com_content&view=article&id=60&Itemid=77

Boot INI Options Reference
http://technet.microsoft.com/en-us/sysinternals/bb963892.aspx

HAL options after Windows XP or Windows Server 2003 Setup
http://support.microsoft.com/kb/309283

Available Switch Options for Windows NT Boot.ini File
http://support.microsoft.com/kb/170756

How to Manually Add Support for a Second Processor
http://support.microsoft.com/kb/156358

Windows System Resource Manager: Frequently Asked Questions
http://www.microsoft.com/windowsserver2003/techinfo/overview/wsrmfaq.mspx

Windows System Resource Manager
http://www.microsoft.com/downloads/details.aspx?FamilyID=848306EF-F57E-4B3F-984D-50E9BCA44383&displaylang=en

Best Practices for Managing Applications with Process Control
http://technet.microsoft.com/en-us/library/bb742469.aspx

The Context Switch Action (xperf)
http://msdn.microsoft.com/en-us/library/cc305227.aspx

Monitoring Context Switches and Threads
http://technet.microsoft.com/en-us/library/cc938639.aspx

Analyzing Processor Activity
http://technet.microsoft.com/en-us/library/cc958310.aspx

Monitoring Activity on Multiprocessor Systems
http://technet.microsoft.com/en-us/library/cc938649.aspx

Windows Performance Toolkit x86 and x64 v4.1.1
http://download.microsoft.com/download/e/2/7/e2700369-d072-4fdc-a451-c3355eab0613/xperf_x86.msi

http://download.microsoft.com/download/e/2/7/e2700369-d072-4fdc-a451-c3355eab0613/xperf_x64.msi

Windows Performance Toolkit
http://msdn.microsoft.com/en-us/library/cc305187.aspx

Intfilter to manage drivers:
http://support.microsoft.com/default.aspx?scid=KB;en-us;252867


Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin. 


Read more!

Monday, June 1, 2009

Calling CPUID from PowerShell for Intel VT

I was trying to work out whether the Intel VT (VMX) extensions had been enabled on a server and whether the server support the SSE4.1 and SSE4.2 processor instructions - without having to reboot the server to check the BIOS. I ended up 'writing' a PowerShell script that compiles some C# to call a DLL written in assembler which has run the CPUID instruction and returned the results. Note that I didn't really write any of this, the PowerShell script compiling and running the C# is from the reference below, as is the actual C$ and asm dll.

Does this seem like a lot of effort? Sure does, but unfortunately the simple method - using Win32_Processor.ProcessorID was useless in this example, as for some reason MS chose to only return the EAX and EDX registers, whereas the bits I was after were returned in ECX. There may also be other ways to achieve this through PowerShell/.Net framework, rather than compiling c# to call unmanaged code.

Below is the PowerShell script, which contains the inline C# that is compiled and executed, which in turn requires the dll, using the EFLAGS register to confirm support and the CPUID instruction with eax=1 (if further supported) to query feature information from the CPU.

The original c# and asm dll: (see below for the hex text stream)
http://devpinoy.org/blogs/cvega/archive/2006/04/07/2658.aspx

The compile-csharp function: (adding the -unsafe parameter to allow pointer use)
http://monadblog.blogspot.com/2005/12/calling-win32-api-functions-through.html



#
# Description:
#  Call the CPUID function from an external DLL through C# inline code.
#  Report whether Intel VT (VMX) is supported, bit 5 of the ECX register returned from CPUID eax=1 call.
#  Note that WMI Win32_Processor:ProcessID contains the results of a call to CPUID - except that it only contains EAX and EDX, Intel VT (what I was after) is in ECX
#
# Author:
#  Wayne Martin, note that I didn't write any of this code, the inline C# was from the monad reference below, and the CPUID DLL and C# was from the first reference

#References:
# cpuid.dll and the namespace from: http://devpinoy.org/blogs/cvega/archive/2006/04/07/2658.aspx
# Intel cpuid: http://www.intel.com/Assets/PDF/appnote/241618.pdf
# http://monadblog.blogspot.com/2005/12/calling-win32-api-functions-through.html
# http://msdn.microsoft.com/en-us/library/aa394373.aspx


function Compile-Csharp ([string] $code, $FrameworkVersion="v2.0.50727",
[Array]$References)
{
    $cp = new-object Microsoft.CSharp.CSharpCodeProvider
    $cpar = New-Object System.CodeDom.Compiler.CompilerParameters
    $cpar.CompilerOptions = "-unsafe"      # unsafe to compile the code which uses pointers in the DLL call
    $cpar.GenerateInMemory = $true
    $cpar.GenerateExecutable = $false
    $cpar.OutputAssembly = "custom"
    # $cpar.ReferencedAssemblies.AddRange($refs)
    $cr = $cp.CompileAssemblyFromSource($cpar, $code)

    if ( $cr.Errors.Count)
    {
        $codeLines = $code.Split("`n");
        foreach ($ce in $cr.Errors)
        {
            write-host "Error: $($codeLines[$($ce.Line - 1)])"
            write-host $ce
           # $ce out-default
        }
        Throw "INVALID DATA: Errors encountered while compiling code"
    }
}



$code = @'
namespace CPUID
{
 using System;
 using System.Runtime.InteropServices;
 using System.Text;

 public class cpuid
 {
  private cpuid()
  {
  }

  [DllImport("cpuid.dll")]
     public static extern bool CPUIDIsSupported();

  [DllImport("cpuid.dll")]
     private unsafe static extern bool __cpuid
            (uint function, 
             int* eax, 
             int* ebx, 
             int* ecx, 
             int* edx);

  // Invoke __cpuid function
  public unsafe static bool Invoke
      (uint level, 
       out int eax, 
       out int ebx, 
       out int ecx, 
       out int edx)
  {
   int __eax = 0;
   int __ebx = 0;
   int __ecx = 0;
   int __edx = 0;

   if (__cpuid(level, &__eax, &__ebx, &__ecx, &__edx))
   {
    eax = __eax;
    ebx = __ebx;
    ecx = __ecx;
    edx = __edx;

    return true;
   }
   else
   {
    eax = 0;
    ebx = 0;
    ecx = 0;
    edx = 0;

    return false;
   }
  }
 }
}


'@


compile-CSharp $code
$eax = ""
$ebx = ""
$ecx = ""
$edx = ""
[CPUID.CPUID]::Invoke(1, [ref]$eax, [ref]$ebx, [ref]$ecx, [ref]$edx) 

$INTEL_CPUID_VT_FLAG = 0x0020                                                           # bit 5 of ECX returned from CPUID EAX=1
$INTEL_CPUID_SSE41 = 0x80000                                                            # bit 19 of ECX returned from CPUID EAX=1
$INTEL_CPUID_SSE42 = 0x100000                                                           # bit 20 of ECX returned from CPUID EAX=1
$INTEL_CPUID_XD = 0x00100000

write-output ("Intel VT: " + ($ecx -band $INTEL_CPUID_VT_FLAG) )                        # Does this processor support Intel VT / VMX?
write-output ("SSE 4.1: " + ($ecx -band $INTEL_CPUID_SSE41) )                           # Does this processor support SSE 4.1?
write-output ("SSE 4.2: " + ($ecx -band $INTEL_CPUID_SSE42) )                           # Does this processor support SSE 4.2?
write-output ("eXecute Disable: " + ($edx -band $INTEL_CPUID_XD) )                      # Does this processor support XD?




Note that in a previous post, I have written two powershell scripts, one to convert binary to a hex string, and vice versa. The following stream is the cpuid.dll file, if you write it to cpuid.txt and run the following command it will create the dll:



Powershell . .\HexStringToBinary.ps1 cpuid.txt cpuid.dll

4d5a90000300000004000000ffff0000b800000000000000400000000000000000000000000000000000000000000000000000000000000000000000c00000000e1fba0e00b409cd21b8014ccd21546869732070726f6772616d2063616e6e6f742062652072756e20696e20444f53206d6f64652e0d0d0a240000000000000071d4f7db35b5998835b5998835b59988c9958b8834b59988bbaa8a8834b599885269636835b59988000000000000000000000000000000000000000000000000504500004c0103006a793a440000000000000000e0000e210b01050c00020000000400000000000000100000001000000020000000000010001000000002000004000000040000000400000000000000004000000004000070fa000002000000000010000010000000001000001000000000000010000000002000005f0000000000000000000000000000000000000000000000000000000000000000000000003000000800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000002e746578740000006c000000001000000002000000040000000000000000000000000000200000602e726461746100005f000000002000000002000000060000000000000000000000000000400000402e72656c6f6300000c000000003000000002000000080000000000000000000000000000400000420000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000558becb801000000c9c20c005352b8000000009c588bc83500002000509d9c5b33c325000020007507b801000000eb0233c0519d5a5bc3558bec5357e8cbffffff48740433c0eb1e8b45080fa28b7d0c89078b7d14890f8b7d1889178b7d10891fb8010000005f5bc9c214000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000006a793a44000000003c2000000100000002000000020000002820000030200000382000000c1000003710000046200000572000000000010063707569642e646c6c0043505549444973537570706f72746564005f5f6370756964000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000008000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000


Wayne's World of IT (WWoIT), Copyright 2009 Wayne Martin. 


Read more!

All Posts

printQueue AD objects for 2003 ClusterVirtualCenter Physical to VirtualVirtual 2003 MSCS Cluster in ESX VI3
Finding duplicate DNS recordsCommand-line automation – Echo and macrosCommand-line automation – set
Command-line automation - errorlevels and ifCommand-line automation - find and findstrBuilding blocks of command-line automation - FOR
Useful PowerShell command-line operationsMSCS 2003 Cluster Virtual Server ComponentsServer-side process for simple file access
OpsMgr 2007 performance script - VMware datastores...Enumerating URLs in Internet ExplorerNTLM Trusts between 2003 and NT4
2003 Servers with Hibernation enabledReading Shortcuts with PowerShell and VBSModifying DLL Resources
Automatically mapping printersSimple string encryption with PowerShellUseful NTFS and security command-line operations
Useful Windows Printer command-line operationsUseful Windows MSCS Cluster command-line operation...Useful VMware ESX and VC command-line operations
Useful general command-line operationsUseful DNS, DHCP and WINS command-line operationsUseful Active Directory command-line operations
Useful command-linesCreating secedit templates with PowerShellFixing Permissions with NTFS intra-volume moves
Converting filetime with vbs and PowerShellDifference between bat and cmdReplica Domain for Authentication
Troubleshooting Windows PrintingRenaming a user account in ADOpsMgr 2007 Reports - Sorting, Filtering, Charting...
WMIC XSL CSV output formattingEnumerating File Server ResourcesWMIC Custom Alias and Format
AD site discoveryPassing Parameters between OpsMgr and SSRSAnalyzing Windows Kernel Dumps
Process list with command-line argumentsOpsMgr 2007 Customized Reporting - SQL QueriesPreventing accidental NTFS data moves
FSRM and NTFS Quotas in 2003 R2PowerShell Deleting NTFS Alternate Data StreamsNTFS links - reparse, symbolic, hard, junction
IE Warnings when files are executedPowerShell Low-level keyboard hookCross-forest authentication and GP processing
Deleting Invalid SMS 2003 Distribution PointsCross-forest authentication and site synchronizati...Determining AD attribute replication
AD Security vs Distribution GroupsTroubleshooting cross-forest trust secure channels...RIS cross-domain access
Large SMS Web Reports return Error 500Troubleshooting SMS 2003 MP and SLPRemotely determine physical memory
VMware SDK with PowershellSpinning Excel Pie ChartPoke-Info PowerShell script
Reading web content with PowerShellAutomated Cluster File Security and PurgingManaging printers at the command-line
File System Filters and minifiltersOpsMgr 2007 SSRS Reports using SQL 2005 XMLAccess Based Enumeration in 2003 and MSCS
Find VM snapshots in ESX/VCComparing MSCS/VMware/DFS File & PrintModifying Exchange mailbox permissions
Nested 'for /f' catch-allPowerShell FindFirstFileW bypassing MAX_PATHRunning PowerSell Scripts from ASP.Net
Binary <-> Hex String files with PowershellOpsMgr 2007 Current Performance InstancesImpersonating a user without passwords
Running a process in the secure winlogon desktopShadow an XP Terminal Services sessionFind where a user is logged on from
Active Directory _msdcs DNS zonesUnlocking XP/2003 without passwords2003 Cluster-enabled scheduled tasks
Purging aged files from the filesystemFinding customised ADM templates in ADDomain local security groups for cross-forest secu...
Account Management eventlog auditingVMware cluster/Virtual Center StatisticsRunning scheduled tasks as a non-administrator
Audit Windows 2003 print server usageActive Directory DiagnosticsViewing NTFS information with nfi and diskedit
Performance Tuning for 2003 File ServersChecking ESX/VC VMs for snapshotsShowing non-persistent devices in device manager
Implementing an MSCS 2003 server clusterFinding users on a subnetWMI filter for subnet filtered Group Policy
Testing DNS records for scavengingRefreshing Computer Account AD Group MembershipTesting Network Ports from Windows
Using Recovery Console with RISPAE Boot.ini Switch for DEP or 4GB+ memoryUsing 32-bit COM objects on x64 platforms
Active Directory Organizational Unit (OU) DesignTroubleshooting computer accounts in an Active Dir...260+ character MAX_PATH limitations in filenames
Create or modify a security template for NTFS perm...Find where a user is connecting from through WMISDDL syntax in secedit security templates

About Me

I’ve worked in IT for over 20 years, and I know just about enough to realise that I don’t know very much.