In my previous post about PowerShell and BITS, I stumbled on a weird quirk in the ForEach-Object function, which had me scratching my head for a bit.

In most programming languages, where you have a for, loop, while, foreach, or other such loop, there is often a set of control functions that go with it. Continue and Break. These functions alter the way a loop is behaving at that time. A call to Continue will stop the current loop iteration at that point in the code, and jump onto the next. A simple example would look like this:

This code simply takes the variable $i, sets it to 0, and whilst $i is less than 5, it loops through the code, each iteration increasing the value of $i by 1. The if statement executes special code when $i gets to the number 2. Now if we run this code, we get the following:

What happened was the counter hit 2, and the code said that it was done with this iteration, and continue processing the outer loop. A Break on the other hand just tells the loop it’s done processing, and to stop looping, and get out. Using the same example as above, but changing Continue to Break we get the output that looks like this:

This is because PowerShell has been told to stop processing any further.

Now for the quirk. I discovered that ForEach-Object isn’t actually a programmatic keyword, it’s a cmdlet. How does this affect the usage of the control functions? Lets look at my code from my BITS article, and compare it to what I had.

As you can see, I’m using the continue function inside the array of strings for the images. If the string value is empty, I skip onto the next one. This is what the original code looked like:

Whilst the code seems pretty similar, there is one distinct difference. The first block of code is inside a loop, the second set is not. Even though you think that ForEach-Object seems like a loop, it’s a cmdlet that does not behave the same way that a loop does. How does this affect the behavior of the code? Badly! Because there is no loop, the control function looks for the next operation it can operate on. It turns out that the next function it can operate on is the entire script. It essentially kills the script right where the line is. This means no processing of other objects in the ‘loop’, no continuing with the code further in the script, no nice handling of anything else in the script. It’s done, finished, over. My hint something wasn’t behaving right was the fact I should have had several thousand images, and yet I only had about 150. It took 4 or 5 attempts, as well as various debug statements throughout the code to figure out why this was happening. After I realized where it was going wrong, a Google search dropped me over to James Manning’s post PowerShell gotcha – foreach keyword vs. foreach-object cmdlet.

So if you ever find yourself in need of an object loop, remember that ForEach-Object isn’t really a keyword, it’s a cmdlet, and does not behave the same.

See any mistakes? Want to add your feedback? Leave me a note in the comments, I love to hear from you.

Ever had to download a bunch of files from a website, and didn’t want to have to write an HTTP handler for it? This comes up on a regular basis here. We get requests from customers that have transferred from one vendor to another, and want to import all their photos to us. Here is how I solve it.

What is BITS?

BITS, or Background Intelligent Transfer Service, is a service bundled with Windows that is used to transfer files from websites. It was originally introduced in Windows XP RTM back in October 2001. You may not know it, but your computer uses it all the time if you have Windows Automatic Updates enabled. It is an intelligent service that asynchronously transfers files, automatically adjusting bandwidth usage, throttling downloads, as bandwidth becomes available. It can even be used to download files across computer starts.

How is it useful in PowerShell?

BITS is really easy to use, a few commands which I’ll show shortly, and you can download a bunch of files quickly, without having to write HTTP handler. In my case above, I receive text files, with a list of image URLs, that need to be imported.

Here is how to do a very simple, single file, transfer.

Relatively simple, a few lines of code for a single file. However, BITS really shines when you’re working with multiple files to download.

As the files I receive are quite often comma-separated values (CSV) files, we can use the rather handy Import-CSV function, and we build up from there.

This is a little more complicated, but is still pretty simple. It introduces Add-BitsFile which allows you to add files to an existing job, and a simple ForEach loop to go through an array of URLs.

I’ve made a few assumptions here, and thrown out all kinds of error handling in favor of a quick code turn around. For example, the job state of a BITS job can be one of nine options (which you can see here), but I assume only two. I’m also not handling cases where I exceed the number of jobs, but that never happens because I only ever have one job running at a time. I’ll probably work on tidying it up a little for other people’s use, but for now, it’s just me using it where I work, so the impact of an error is very minimal.

Give it a shot, download some images off of Flickr, or go a little further, and test it against downloads from Microsoft.

Got any questions? Hints? Tips? Leave them in the comments, I’d love to hear from you.

Whilst sitting here on a product release call, I was reading over my twitter feed, specifically looking at #powershell, and I saw a post by @tonjoh. Feeling I had a few minutes on my hands whilst QA bashed away at the servers, I decided to take a look…

I’m not sure what WMI counter is being used as I cannot find one called free-diskspace-query, but I do know there is a Win32_LogicalDisk, which has enough information in it to tell us free space. So here we go… Lets just get the information we want first, and build up from there…

This is pretty simple, and can probably be found as an example all over the place. For the curious, the output looks like this:

So we can see the size (~111GB), and the amount of free space (~45GB). Now onto the fun stuff, using expressions.

Expressions allow us to do calculations on the resultset that came back from the pre-piped data. This can be something like calculating a percentage, or throwing in miscellaneous data (like requested, a timestamp).

In the above code, I’ve used the back-tick (`) to wrap the lines, and make it easier to read. PowerShell understands this, and will treat the 4 lines as one. I’ve created 2 extra, named, columns, one PercFree, and the other ReportDate. The expression in the first is a calculation based on the values from the Win32_LogicalDisk output, whilst the expression for the second is just extra data in the form of a formatted date. With the above code, you get the following…

And there we have it, two extra columns, which were calculated on the fly.

Part of our application at work sends email notifications to various uses when certain criteria are met. This is great, except testing is difficult as you really don’t want to send out mails to valid customers, so we disable relaying outbound, and the local mail server just chomps on it a bit, then decides to not deliver it and drops it to the local file system. This is great, except in a the matter of days, we have 100k message files. This becomes entirely unmanageable with Windows, so PowerShell is here to help…

There are plenty of articles on file management with PowerShell, pretty much all of them revolving around the Get-ChildItem function. This usually works pretty well, but is terrible on memory when it comes to handling this many files, and working with filters (only selecting files that are X days old).

This is when my brain clicked to a PowerShell.com tip a few weeks back about Windows Management Instrumentation (WMI) based filters. I won’t go into the dark recesses of my brain, and try an explain why I have a weird ability to remember some weird function that I used 5 years ago, but I’ll save you some work and introduce you to the WMI class CIM_DataFile. The cool thing about using WMI to do this, you can use the server side filters to do most of the dirty work for you. So lets see the example:

Okay, that was pretty easy… right? That just gets all the files in the folder C:\inetpub\mailroot\badmail on the server yourremoteserver. Now we want to limit it to just keeping the mail from the last 3 days. Before I did into the code that does this, the date format on CIM_DataFile through PowerShell is a little ‘odd’. Don’t believe me? Try this:

You will probably end up with something that looks like this:

Okay, not too odd, it’s really just the date in the format YYYYmmddhhmmss.nnnnnn-tzo. n is milliseconds, and tzo is the timezone offset in minutes. In the above example, 300 would be 5 hours, 360 is 6 hours. This means, for the query, we have to modify slightly.

You might be wondering what that weird stuff on line 3 is, it’s the PowerShell equivalent to C#’s string.Format. This takes the date we had, and reformats it nicely for WMI to use. This is much nicer, gets WMI to do all the grunt work, and is much more friendly on the server (memory/CPU). Next, the delete…

As I mentioned at the beginning, it’s possible to use the Get-ChildItem function to do the same thing (except it’s not remote), so I figured I’d show how too…

The problem you can see here, the filtering is done after all the items are fetched, rather than restricting the fetched list to begin with. This cannot be executed remotely either (unless using network shares), so running this from a central job server is a little more tricky.

I’m sticking with the WMI method for now, it’s fast, memory and CPU friendly, and does the job just as nicely. How do you guys manage remote files with PowerShell? Any hints and tips?

Edit: Thanks to Steven Klassen, I edited some of the code. Apparently the CodeColorer plugin was escaping my escape and only showing a single backslash on the end of some of the commands.

In a follow up to @ScriptingGuys post about restarting services using VBScript, here is the same using PowerShell. I’ll probably be dragging the same onto some server monitoring stuff to get a service back up and running we’ve been having a weird issue with.

This takes a file “computers.txt”, loops through the data, and uses WMI to connect to the remote machines to get the wuauserv service (or in plain English Windows Automatic Update). Easy as pie.

This is a quick and handy one for server monitoring, and tracking down that process that is using all your CPU. It makes use of WMI counters.

Okay, this one is a mouthful of pipes, and formatting to make it easier to see. But the breakdown is pretty easy…

• Get the process information
• Exclude _Total and Idle
• Sort the output by descending processor usage
• Only get the first 5 rows
• Format the results into a table, with the columns Name, Process ID, and processor usage.

Output looks something like this:

Another handy hint here, the back-tick is PowerShell’s way of allowing you to line-wrap. This makes it a little easier to read, especially when giving it as samples like above.

I’ll be bundling parts of this for Nagios to return what processes are using the most processor when I get CPU alerts, allowing me to make quicker judgments on what I need to do to correct the issue.

Curious about how long your script took to execute? How about just that cmdlet? Powershell has a built in function for you.

• Open Powershell command prompt
• Use the command Measure-Command like so:

Enjoy the output broken down by days, hours, minutes, seconds, milliseconds, and ticks.

The same works for cmdlets too…

This one is a quick and easy one. We have multiple IIS boxes, which each generate large usage logs on a daily basis. To save space, and for analytics off-server, we compress the log files. Having found Rar is one of the better compression algorithms, we simply shell out to rar, compress the logs, and cleanup older logs.

This could probably do with some cleaning up as there are probably some slightly more efficient ways of doing some of this, but it works.

We’ve had a stubborn Windows 2003 server that we’ve been trying to get Office 2007 installed for a while. It has a couple of things wrong with it which have made the install substantially harder.

The first being an MSI speed issue. Because the server plays host to about 120+ custom applications, all installed using MSI, any further MSI installs simply drag whilst it reads through all the keys, does its validation and backup checks, and then proceeds.

The second issue we have is that this box has a partially installed, corrupted Office 2007 install already on here. Attempting to do “upgrades” fail, and uninstalling just barfs with a delightful error. This resulted in some stumbling to figure out how to get it off, and start again.

Fortunately Microsoft do frequently document things like this, you just have to find the right search terms to find it, which luckily I did, in the form of KB928218. This article gives you a list of the registry keys you have to go through and remove, and you should be all well and good. Unfortunately, this list is over 200 registry keys on our count, and quite a list all over the place. This is where Powershell comes in handy…

As the server itself doesn’t (yet) have Powershell installed, I ran the commands from my laptop, which required remote registry access. Powershell makes use of the .Net libraries to do this:

The LocalMachine reference here is the part of the registry we want to open, which is equivalent to HKEY_LOCAL_MACHINE. This now gives us a registry object in the HKLM section of the registry. From here, there are a bunch of functions we can use to fetch the keys we need. The first is OpenSubKey. This opens a subkey in the $reg object.

This assigns $regKey to the subkey. The boolean flag $true on the end is to make sure the key is opened in writable mode. This will be required to modify the contents of the key.

Per the KB article, Microsoft say to delete all of the 12.0 subkey from under here. I had originally used the following:

This resulted in an error about not being allowed to delete subkeys. Then I stumbled upon the much more suitable DeleteSubKeyTree, which, as the name hints, deletes the tree. So the new code became:

This was repeated for the various other subkeys Microsoft recommended deleting. It became more complex when it asks you to delete keys that match some of the characters below, and use a * as the match. For example:

Note In the following registry keys, the asterisk character (*) represents one or more characters in the subkey name. HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\*0FF1CE*

Fortunately, Powershell is very good at string manipulation, and queries, so this was a bit of a case for a where, and foreach loop.

It looks pretty messy, but it’s actually pretty easy if you understand pipes (the | character). The above line 24 boils down to:

• Get the subkey names
• Find only the ones that have 0FF1CE in them (yes I know that’s office with a zero and a one)
• For each one that matches, go delete it, and its children

See, not so hard.

There are some parts of this that venture KB into other areas of the registry, so the syntax changes in the opening command, for example:

This opens HKEY_CLASSES_ROOT. You can use CurrentUser to access HKEY_CURRENT_USER too.

That’s about it for remote registry access and fixing this blasted Office install. As a side, the script saved me about 30 minutes of bashing through about 200+ registry keys for cleaning up.

Below is the full script, with the delete statements commented out. If you have 2007 installed, give it a run, see what happens (please double check that all the deletes are in fact commented out, this script was used against a server to fix an install issue, and did have active delete statements in, and I cannot remember if I got all the deletes).

With Windows 2008, the task scheduler is setup differently than earlier. You can use the Com object Schedule.Service to access information about tasks.

First we need to get the Com object to talk to the server. This is easily done with a “new object”.

The first line gets you the com object, the second connects to the server. You can put in a server address to query remote servers too.

As 2008 uses task folders, you have to get the folder to search inside. If you don’t organize, this can be “\” or you can use the folder name.

Then get the tasks from that folder.

$tasks is now an array of tasks, which you can walk through, or query. Now we have a task listing, we can figure out the state of each task using a foreach.

The state column is based on the values here. So we know that 4 is running for example.

I used this information to build a script that allowed my Nagios server to manage alerts based on our build server running the build process.