Labels

Wednesday, December 17, 2008

Command-line automation – find and findstr

This is the second in a series of posts containing information on what I consider the building blocks to automate repetitive tasks at the Windows command-line. These components are the for, find, findstr, set, if and echo commands, control files used to control data input, combined with errorlevels, command concatenation, nested loops and if/then/else constructs.

Described in this post are the ‘find’ and ‘findstr’ commands - unless you want to spend all your time manually creating or filtering control files you will often use these commands when constructing repeatable command-lines.

These two commands are crucial in efficient ‘for’ loop processing, invaluable when filtering a control file or the results of an in-line command. Help on each is available from the command prompt with the ‘/?’ argument.

Find

I usually use find because more often than not I want to filter a string literal – a specific word or combination of characters – from an input string. As often as not, I also use ‘find /v’, returning the lines that don’t match the specified string.

For example, if you have a control file listing servers in several geographic locations, but you are only interested in processing servers in ‘bne’ (case insensitive with ‘/i’), you could run:
find /i "bne" c:\temp\control.txt

Combined with a ‘for’ command, this could be:
for /f "skip=2" %i in ('find /i "bne" c:\temp\control.txt') do echo %i

Note that this line uses the following syntax explained in the previous post on ‘for’:

  1. ‘Skip=2’ tells the loop to skip the first two lines – the result header of the ‘find’ command
  2. Inside the brackets I’ve specified to run a command and parse the results, wrapping the command in single-quotes.

The results are that for each server containing the word ‘bne’, the body of the ‘for’ statement will be executed – in this case simply to echo the first token.

You might also want to run the commands against all servers except those in bne, which is the above command repeated with the addition of the ‘/v’ switch to tell ‘find’ to return those lines not containing a match:
for /f "skip=1" %i in ('find /i /v "bne" c:\temp\control.txt') do echo %i

Filtering in-line output with find

Filtering output of a command executed within a ‘for’ loop is syntactically a little more complicated, but makes sense once you get the hang of it. For example, suppose you want to filter the output of a ping command, returning the IP address of those hosts that replied to a ping, you could run:
for /f "tokens=3" %i in ('"ping -n 1 computer find /i "reply""') do @echo %i

This will:

  • Execute the ping command, pinging the host named computer, filtering the output to find the word ‘reply’ (indicating a successful reply). Note that to use the pipe inside the brackets, the whole string needs to be enclosed in double-quotes – inside the single-quotes.
  • Using spaces/tabs, choose the third token to return in the first variable - %i. This will be the IP address in the standard ping output returned by XP and Vista.

Unfortunately the third token of the ping output includes a irrelevant colon, so to strip that, we could specify colons and spaces as delimiters in the for loop:
for /f "tokens=3 delims=: " %i in ('"ping -n 1 computer find /i "reply""') do echo %i

This will be better explained in a future post, but you can also nest two ‘for’ commands and for each host in the control file (in bne in this example), ping and return the IP address if there is a reply:
for /f "skip=2" %i in ('find /i "bne" c:\temp\control.txt') do @for /f "tokens=3 delims=: " %m in ('"ping -n 1 %i find /i "reply""') do @echo %i,%m

The output would give you a comma-separated list of server names and IP address of those from the original control file that currently respond to a ping. Theoretically not all that useful with servers, as they’d usually be known IPs and always on, but if you’re wanting to check something on hundreds or thousands of workstations this would give you a point-in-time control file of online machines (assuming name resolution is accurate).

It’s also worth noting that ‘find’ commands can be cascaded – piping the output of one find command to another if you need to further refine the output. For example to find servers in ‘bne’ and then servers in ‘bne’ that also contain ‘vm’:
find /i /c "bne" c:\temp\control.txt find /i "vm"

The possibilities are nearly endless, I use ‘find’ to:

  • Filter in and out items that I want, ‘find /i’ and ‘find /i /v’ from generic control files that I use for multiple operations, either in-line in a for command, or to create a stand-alone filtered control file
  • Run commands and then filter out useless or useful information, often headers/footers or other extraneous detail, making it possible to list data, filter that data and run a command against each filtered item in a single command.
  • Use errorlevels to determine whether the find command was successful or not (explained in a future post). Often the only way of knowing whether a command returned what you were after is the presence or absence of a string – and ‘find’ will tell you this based on the errorlevel (an advantage over findstr which in XP at least does not set the errorlevel).

Note that find also has several other features that are useful in some scenarios:

  • By default, offline files will be skipped – if you use a file archiving product that sets the offline (‘O’) attribute, these files will not be searched unless you specify the /offline parameter
  • The ‘find /n’ option will return the line number the match was found, occasionally useful when parsing large or sequenced control files and you want to know where in a file the string was found.
  • The ‘find /c’ option will return a count of the matches found, instead of the actual matching lines. This can often be useful when determining how many records would be processed. For example, in the example above you could first run the following command to determine how many records would be processed with the for loop:
    find /i /c "bne" c:\temp\control.txt

Findstr

These same principles can be used with findstr to filter output – findstr provides advanced searching, with the following benefits in my use of the commands, you can:

  • Search for more than one string in a single command. This simplifies searches, in the example above, you could search for ‘bne’ and ‘syd’ in a single findstr command.
  • Use basic regular expressions and/or string literals

In the example above with a control file containing a list of servers with geographic names (or descriptions somewhere on the line), you could run the following command to return servers in both syd and bne and then run the subsequent command against each server:
for /f %i in ('findstr /i /c:bne /c:syd c:\temp\control.txt') do echo %i

Regular expressions are used to group similar items, without expressly providing each possible combination. For example, parsing robocopy log output, to search for new directories that have been created with a regular expression, you could run:
findstr /i /n "^.*New.Dir" Robocopy.log"

You could just use ‘find’ and search for the string ‘new dir’, but this could easily return false positives if a file or directory has the string ‘new dir’ in it. Using the regular expression looking for the start of the line followed by whitespace followed by the ‘New Dir’ is much more precise.

This was a basic overview of the ‘find’ and ‘findstr’ commands, future posts will build on this foundation with other useful commands, error levels, if/then/else statements and nested ‘for’ loops.

For more real-world examples of how I use the ‘find’ and ‘findstr’ commands, see my command-line operations:
http://waynes-world-it.blogspot.com/2008/09/useful-ntfs-and-security-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-printer-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-windows-mscs-cluster-command.html
http://waynes-world-it.blogspot.com/2008/09/useful-vmware-esx-and-vc-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-general-command-line-operations.html
http://waynes-world-it.blogspot.com/2008/09/useful-dns-dhcp-and-wins-command-line.html
http://waynes-world-it.blogspot.com/2008/09/useful-active-directory-command-line.html


Wayne's World of IT (WWoIT), Copyright 2008 Wayne Martin.

No comments:


All Posts

printQueue AD objects for 2003 ClusterVirtualCenter Physical to VirtualVirtual 2003 MSCS Cluster in ESX VI3
Finding duplicate DNS recordsCommand-line automation – Echo and macrosCommand-line automation – set
Command-line automation - errorlevels and ifCommand-line automation - find and findstrBuilding blocks of command-line automation - FOR
Useful PowerShell command-line operationsMSCS 2003 Cluster Virtual Server ComponentsServer-side process for simple file access
OpsMgr 2007 performance script - VMware datastores...Enumerating URLs in Internet ExplorerNTLM Trusts between 2003 and NT4
2003 Servers with Hibernation enabledReading Shortcuts with PowerShell and VBSModifying DLL Resources
Automatically mapping printersSimple string encryption with PowerShellUseful NTFS and security command-line operations
Useful Windows Printer command-line operationsUseful Windows MSCS Cluster command-line operation...Useful VMware ESX and VC command-line operations
Useful general command-line operationsUseful DNS, DHCP and WINS command-line operationsUseful Active Directory command-line operations
Useful command-linesCreating secedit templates with PowerShellFixing Permissions with NTFS intra-volume moves
Converting filetime with vbs and PowerShellDifference between bat and cmdReplica Domain for Authentication
Troubleshooting Windows PrintingRenaming a user account in ADOpsMgr 2007 Reports - Sorting, Filtering, Charting...
WMIC XSL CSV output formattingEnumerating File Server ResourcesWMIC Custom Alias and Format
AD site discoveryPassing Parameters between OpsMgr and SSRSAnalyzing Windows Kernel Dumps
Process list with command-line argumentsOpsMgr 2007 Customized Reporting - SQL QueriesPreventing accidental NTFS data moves
FSRM and NTFS Quotas in 2003 R2PowerShell Deleting NTFS Alternate Data StreamsNTFS links - reparse, symbolic, hard, junction
IE Warnings when files are executedPowerShell Low-level keyboard hookCross-forest authentication and GP processing
Deleting Invalid SMS 2003 Distribution PointsCross-forest authentication and site synchronizati...Determining AD attribute replication
AD Security vs Distribution GroupsTroubleshooting cross-forest trust secure channels...RIS cross-domain access
Large SMS Web Reports return Error 500Troubleshooting SMS 2003 MP and SLPRemotely determine physical memory
VMware SDK with PowershellSpinning Excel Pie ChartPoke-Info PowerShell script
Reading web content with PowerShellAutomated Cluster File Security and PurgingManaging printers at the command-line
File System Filters and minifiltersOpsMgr 2007 SSRS Reports using SQL 2005 XMLAccess Based Enumeration in 2003 and MSCS
Find VM snapshots in ESX/VCComparing MSCS/VMware/DFS File & PrintModifying Exchange mailbox permissions
Nested 'for /f' catch-allPowerShell FindFirstFileW bypassing MAX_PATHRunning PowerSell Scripts from ASP.Net
Binary <-> Hex String files with PowershellOpsMgr 2007 Current Performance InstancesImpersonating a user without passwords
Running a process in the secure winlogon desktopShadow an XP Terminal Services sessionFind where a user is logged on from
Active Directory _msdcs DNS zonesUnlocking XP/2003 without passwords2003 Cluster-enabled scheduled tasks
Purging aged files from the filesystemFinding customised ADM templates in ADDomain local security groups for cross-forest secu...
Account Management eventlog auditingVMware cluster/Virtual Center StatisticsRunning scheduled tasks as a non-administrator
Audit Windows 2003 print server usageActive Directory DiagnosticsViewing NTFS information with nfi and diskedit
Performance Tuning for 2003 File ServersChecking ESX/VC VMs for snapshotsShowing non-persistent devices in device manager
Implementing an MSCS 2003 server clusterFinding users on a subnetWMI filter for subnet filtered Group Policy
Testing DNS records for scavengingRefreshing Computer Account AD Group MembershipTesting Network Ports from Windows
Using Recovery Console with RISPAE Boot.ini Switch for DEP or 4GB+ memoryUsing 32-bit COM objects on x64 platforms
Active Directory Organizational Unit (OU) DesignTroubleshooting computer accounts in an Active Dir...260+ character MAX_PATH limitations in filenames
Create or modify a security template for NTFS perm...Find where a user is connecting from through WMISDDL syntax in secedit security templates

About Me

I’ve worked in IT for over 20 years, and I know just about enough to realise that I don’t know very much.