Sunday, August 15, 2010

vSphere Storage vMotion traffic

This post provides information on storage vMotion in VMware vSphere 4 with ESXi 4.0 U2 hosts. While using storage vMotion to move data between unconnected storage, I conducted some simple testing to identify the networks and ports used for vMotion.

I’ve never seen an explanation of how storage vMotion works, and while it seems kind of obvious in retrospect, I thought some people might find this interesting.


  1. The scope of a storage vMotion operation is datacenter wide, not limited to clusters or which host can see which storage.
  2. In the testing conducted, storage vMotion relies upon the management network to transfer the disk data between the two ESXi hosts
  3. If this method was used to import a large number of VMs from foreign storage then the limitation would typically be the 1GB vmnic used for the management network.
  4. To allow the migration between datastores not visible by a single host, the VM would also be migrated between hosts.


Two hosts managed by single vCenter instance, vc01:
  1. esx01 part of the CLUS1 cluster
  2. esx02 not in a cluster
  3. esx01, connected only to the iSCSI SAN 1
  4. esx02, connected only to the iSCSI SAN 2

Management network on vmk0 using two GbE interfaces on both hosts. Vmk0 on the management network with load balancing based on the virtual port ID.


The test conducted was using storage vmotion to move a virtual machine through VirtualCenter on vc01 from a datastore visible only to esx02 to a datastore visible only to esx01.
  1. The source datastore was LUN02 (visible only to esx02) and the destination datastore was LUN01 (visible only to esx01).
  2. vMotion was used to automatically migrate the VM from esx02 to esx01 (as the VM had moved storage)

Traffic and network usage observed

  1. On esx02 unsupported console, vmk0 source (esx02), destination (esx01).
  2. Command: /bin/tcpdump-uw -I vmk0 -nn

  1. MbRx 190Mb/sec on software iSCSI NIC – Reading the disk information from iSCSI SAN 2 for the VM
  2. MbTx 136Mb/sec – Transmit the information across the management network to esx01.
  3. Command: resxtop --server esx02 (‘n’ for network information)

  1. MbRx 120Mb/sec – receiving the disk data from esx02 across the management network
  2. MbTx 200Mb/sec on software iSCSI NIC – writing the data to the SAN across the iSCSI network
  3. Command: resxtop --server esx01 (‘n’ for network information)

The following diagram shows the traffic flow of storage vMotion on an iSCSI network:

Read more!

Saturday, August 14, 2010

Resizing a VM's Windows system OS with Set-HardDisk

With vSphere PowerCLI Update 1 there is now an automated method of increasing the system disk of a virtual machine. This essentially performs the same thing as I’ve been doing manually for a while now - the disk is resized and mounted to a ‘helper vm’ and diskpart extends the disk, the helper VM is then shutdown and the risk removed, and when the resized VM starts up the OS prompts for a final reboot after detecting the disk change.

Note that both the target and helper VM must be powered off, and credentials must be passed or the helpervm argument is ignored. The guest credentials are used on the Windows helper VM to authenticate using the VMtools mechanism to invoke a command inside a VM.

$guetsusername = "domain\username"
$guestpass = Read-Host "Guest password (administrator of the server)?" -assecurestring
$hostusername = "root"
$hostpass = Read-Host "Host Password (VI SDK access to the ESX host)?" -assecurestring
$hostcredential = new-object System.Management.Automation.PSCredential($hostusername, $hostpass)
$guestcredential = new-object System.Management.Automation.PSCredential($guetsusername, $guestpass)
$sizeGB = 30
$sizeKB = $sizeGB *1024*1024
$harddisk = Get-HardDisk -vm vm01
Set-HardDisk -HardDisk $harddisk[0] -capacityKB $sizeKB -hostcredential $hostcredential -guestcredential $guestcredential -helpervm $vm

An error occurred when trying to use domain credentials to pass to the vCenter SDK – host credentials are required, not credentials to vCenter (which seems odd).

Set-HardDisk -HardDisk $harddisk[0] -capacityKB 31457280 -hostcredential $hostcredential -guestcredential $guestcredential -helpervm helpervm
Set-HardDisk : 4/02/2010 1:25:04 PM    Set-HardDisk    919835D1-AD13-41DF-ABDE-D630E96FE722    While performing operation 'Connect to host service 'https://esx01/sdk' at port 902' the following error occured: 'Insufficient permissions in host operating system'
At line:1 char:13
+ Set-HardDisk  <<<< -HardDisk $harddisk[0] -capacityKB 31457280 -hostcredential $hostcredential -guestcredential $guestcredential -helpervm helpervm

Another error occurred when trying to resize a VM, the script used failed.  Note that I tried this manually and it also failed (must have been something particular to that VM).  It was interesting to see that from this it looks like the cmdlet uses the invoke-vmscript to run a remote script through the VMtools interface that calls the diskpart commands.
Set-HardDisk : 4/02/2010 11:22:11 AM    Set-HardDisk    919835D1-AD13-41DF-ABDE-D630E96FE722    Execution of the  script in guest OS on VM 'helpervm' failed: '
Microsoft DiskPart version 5.2.3790.3959
Copyright (C) 1999-2001 Microsoft Corporation.
On computer: HELPERVM
Disk 1 is now the selected disk.
Volume 2 is the selected volume.
The volume you have selected may not be extended.
Please select another volume and try again.
At line:1 char:13
+ Set-HardDisk  <<<< -HardDisk $harddisk[0] -capacityKB 31457280 -hostcredential $hostcredential -guestcredential $guestcredential -helpervm $vm

Wayne's World of IT (WWoIT), Copyright 2010 Wayne Martin. 

Read more!

All Posts

printQueue AD objects for 2003 ClusterVirtualCenter Physical to VirtualVirtual 2003 MSCS Cluster in ESX VI3
Finding duplicate DNS recordsCommand-line automation – Echo and macrosCommand-line automation – set
Command-line automation - errorlevels and ifCommand-line automation - find and findstrBuilding blocks of command-line automation - FOR
Useful PowerShell command-line operationsMSCS 2003 Cluster Virtual Server ComponentsServer-side process for simple file access
OpsMgr 2007 performance script - VMware datastores...Enumerating URLs in Internet ExplorerNTLM Trusts between 2003 and NT4
2003 Servers with Hibernation enabledReading Shortcuts with PowerShell and VBSModifying DLL Resources
Automatically mapping printersSimple string encryption with PowerShellUseful NTFS and security command-line operations
Useful Windows Printer command-line operationsUseful Windows MSCS Cluster command-line operation...Useful VMware ESX and VC command-line operations
Useful general command-line operationsUseful DNS, DHCP and WINS command-line operationsUseful Active Directory command-line operations
Useful command-linesCreating secedit templates with PowerShellFixing Permissions with NTFS intra-volume moves
Converting filetime with vbs and PowerShellDifference between bat and cmdReplica Domain for Authentication
Troubleshooting Windows PrintingRenaming a user account in ADOpsMgr 2007 Reports - Sorting, Filtering, Charting...
WMIC XSL CSV output formattingEnumerating File Server ResourcesWMIC Custom Alias and Format
AD site discoveryPassing Parameters between OpsMgr and SSRSAnalyzing Windows Kernel Dumps
Process list with command-line argumentsOpsMgr 2007 Customized Reporting - SQL QueriesPreventing accidental NTFS data moves
FSRM and NTFS Quotas in 2003 R2PowerShell Deleting NTFS Alternate Data StreamsNTFS links - reparse, symbolic, hard, junction
IE Warnings when files are executedPowerShell Low-level keyboard hookCross-forest authentication and GP processing
Deleting Invalid SMS 2003 Distribution PointsCross-forest authentication and site synchronizati...Determining AD attribute replication
AD Security vs Distribution GroupsTroubleshooting cross-forest trust secure channels...RIS cross-domain access
Large SMS Web Reports return Error 500Troubleshooting SMS 2003 MP and SLPRemotely determine physical memory
VMware SDK with PowershellSpinning Excel Pie ChartPoke-Info PowerShell script
Reading web content with PowerShellAutomated Cluster File Security and PurgingManaging printers at the command-line
File System Filters and minifiltersOpsMgr 2007 SSRS Reports using SQL 2005 XMLAccess Based Enumeration in 2003 and MSCS
Find VM snapshots in ESX/VCComparing MSCS/VMware/DFS File & PrintModifying Exchange mailbox permissions
Nested 'for /f' catch-allPowerShell FindFirstFileW bypassing MAX_PATHRunning PowerSell Scripts from ASP.Net
Binary <-> Hex String files with PowershellOpsMgr 2007 Current Performance InstancesImpersonating a user without passwords
Running a process in the secure winlogon desktopShadow an XP Terminal Services sessionFind where a user is logged on from
Active Directory _msdcs DNS zonesUnlocking XP/2003 without passwords2003 Cluster-enabled scheduled tasks
Purging aged files from the filesystemFinding customised ADM templates in ADDomain local security groups for cross-forest secu...
Account Management eventlog auditingVMware cluster/Virtual Center StatisticsRunning scheduled tasks as a non-administrator
Audit Windows 2003 print server usageActive Directory DiagnosticsViewing NTFS information with nfi and diskedit
Performance Tuning for 2003 File ServersChecking ESX/VC VMs for snapshotsShowing non-persistent devices in device manager
Implementing an MSCS 2003 server clusterFinding users on a subnetWMI filter for subnet filtered Group Policy
Testing DNS records for scavengingRefreshing Computer Account AD Group MembershipTesting Network Ports from Windows
Using Recovery Console with RISPAE Boot.ini Switch for DEP or 4GB+ memoryUsing 32-bit COM objects on x64 platforms
Active Directory Organizational Unit (OU) DesignTroubleshooting computer accounts in an Active Dir...260+ character MAX_PATH limitations in filenames
Create or modify a security template for NTFS perm...Find where a user is connecting from through WMISDDL syntax in secedit security templates

About Me

I’ve worked in IT for over 20 years, and I know just about enough to realise that I don’t know very much.