Whilst driving back from our 4th of July celebrations in San Antonio, I got alerted to a server running low on disk space.  As it hadn’t hit critical, and there was still a fair bit of space left, I decided to wait until I got home.  When I looked at the server, I pulled up my trusted space analyzer, TreeSize.

A quick analysis of the drive, and a report showed that there was a hefty chunk of space lost to the WinSXS directory, and 8GB of Files in the root of C:.  This was a little odd, I know I usually use c:\temp from time to time, but never the root.  To find out what was there, I first had to show hidden and system files.  To do this, go to Tools, then Folder Options, then onto the View tab.  There are 3 options there that we need to change.

1. Change “Hidden files and folders” to “Show hidden files and folders”
2. Uncheck “Hide extensions for known file types”
3. Uncheck “Hide protected operating system files (Recommended)”

I highly recommend doing 2 all the time, you never know when somebody might sent a .txt.exe file.  Changing 3 will prompt that it isn’t recommended, but do it anyway, the files I was having issues with were still hidden after changing the first two.

After making these changes, two files appeared that were the culprits to the lost disk space.  Pagefile.sys and Hiberfil.sys.  I’ll leave the explanation of the Pagefile to the experts.  Hiberfil.sys is a file that is used to handle the state of memory, and running applications when you hibernate your computer. 

What is hibernation?

Hibernation is a heavy sleep state for computers where all the running processes are frozen, the memory is locked, and all the states are saved into that file, then the computer is powered down.  When the computer is turned back on, the operating system sees this state file, and starts to reload everything from there.  This makes it appear like the computer was never even turned off.  The applications are resumed, everything is back to the state it was before hibernation was kicked off.

Why is that bad?

Generally speaking, it isn’t.  In fact, it’s a great thing for home computers.  When you’re done reading your email at night, if you hibernate your computer, and hop into bed, your computer will be off, saving power.  However this isn’t so good for servers.  In 95% of cases, servers don’t ever need to hibernate.  They usually have tasks, and operations that they need to run all the time, which means putting them in a state of hibernation is a bad thing as the tasks won’t be run.

How do you stop this?

Usually, servers have this feature turned off, but for some reason this particular one had it enabled.  I’ve done some looking around, and it seems it’s may have actually been a default option to have it enabled, but for some (lucky) reason a handful of our other servers don’t have it on.  In earlier versions of Windows, disabling the hibernation option was a case of removing a check mark from a box.  This was done in the Control Panel under Power Options, and the Hibernate tab.  However, in Windows 2008 this option was removed, and you now have to use a command line to manage this advanced option.

Within about 10 seconds of issuing that command, the Hiberfil.sys file disappeared, and I recovered 4GB of disk space.  You can read more about the powercfg command here.

Since IIS6, or there abouts, IIS will not host a file that it does not understand, or doesn’t have a MIME handler for. This is a security feature, but can cause some unknown issues…

We use some customer facing training software by Skillsoft. They produce a Java application which allows you to record voice, and screen shows for training, and follow up with a questionnaire after. Unfortunately because it’s Java, when something goes wrong, it’s very blackbox((A term coined to explain that you can see the outsides, but have no idea what’s going on inside)) like, making it hard to diagnose any issues. In this case, the splash screen would load, and that’s where it’d stop.

Fortunately, the Java console does provide enough information to give us a hint as to what might be wrong. When opening a Java application, the little console app appears in the notification area next to the clock. This allows you to see if the app throws any errors. In our case, it was complaining that a file with the extension .properties was missing from the server (IIS was throwing a 404 message). A quick showed it existed in the directory structure.

This is because IIS doesn’t know what a .properties file is, or how it should be treated. This is easily remedied. In IIS manager, either under the site, or at the root level, go to the MIME option, and add it. In this case, the file extension was .properties, and the MIME Type was specified as application/octet-stream. If it’s done at the root server level, it applies to all sites. If you do it on a site, it only applies to the site you changed it on. No resets, or such, required. Now the file is served properly.

Microsoft has this documented on KB326965. Whilst this applies to IIS6, the same applies to IIS7 as well. To save some waiting in IIS7, you can use the appcmd to add the type for you…

This applies the setting at the root server level, and is usually inherited down to all sites.

Part of our application at work sends email notifications to various uses when certain criteria are met. This is great, except testing is difficult as you really don’t want to send out mails to valid customers, so we disable relaying outbound, and the local mail server just chomps on it a bit, then decides to not deliver it and drops it to the local file system. This is great, except in a the matter of days, we have 100k message files. This becomes entirely unmanageable with Windows, so PowerShell is here to help…

There are plenty of articles on file management with PowerShell, pretty much all of them revolving around the Get-ChildItem function. This usually works pretty well, but is terrible on memory when it comes to handling this many files, and working with filters (only selecting files that are X days old).

This is when my brain clicked to a PowerShell.com tip a few weeks back about Windows Management Instrumentation (WMI) based filters. I won’t go into the dark recesses of my brain, and try an explain why I have a weird ability to remember some weird function that I used 5 years ago, but I’ll save you some work and introduce you to the WMI class CIM_DataFile. The cool thing about using WMI to do this, you can use the server side filters to do most of the dirty work for you. So lets see the example:

Okay, that was pretty easy… right? That just gets all the files in the folder C:\inetpub\mailroot\badmail on the server yourremoteserver. Now we want to limit it to just keeping the mail from the last 3 days. Before I did into the code that does this, the date format on CIM_DataFile through PowerShell is a little ‘odd’. Don’t believe me? Try this:

You will probably end up with something that looks like this:

Okay, not too odd, it’s really just the date in the format YYYYmmddhhmmss.nnnnnn-tzo. n is milliseconds, and tzo is the timezone offset in minutes. In the above example, 300 would be 5 hours, 360 is 6 hours. This means, for the query, we have to modify slightly.

You might be wondering what that weird stuff on line 3 is, it’s the PowerShell equivalent to C#’s string.Format. This takes the date we had, and reformats it nicely for WMI to use. This is much nicer, gets WMI to do all the grunt work, and is much more friendly on the server (memory/CPU). Next, the delete…

As I mentioned at the beginning, it’s possible to use the Get-ChildItem function to do the same thing (except it’s not remote), so I figured I’d show how too…

The problem you can see here, the filtering is done after all the items are fetched, rather than restricting the fetched list to begin with. This cannot be executed remotely either (unless using network shares), so running this from a central job server is a little more tricky.

I’m sticking with the WMI method for now, it’s fast, memory and CPU friendly, and does the job just as nicely. How do you guys manage remote files with PowerShell? Any hints and tips?

Edit: Thanks to Steven Klassen, I edited some of the code. Apparently the CodeColorer plugin was escaping my escape and only showing a single backslash on the end of some of the commands.

A feature I’ve long awaited in Microsoft SQL (TSQL) has been a function similar to MySQL’s REPLACE function. For those that haven’t used it, the REPLACE function allows you to do an insert of a new record, or replace an existing record if one matches the primary key. In SQL 2008, Microsoft introduced a function called MERGE. This is like REPLACE, but a whole bunch of extra goodies…

About MySQL Replace

Before I get into the fun stuff of MERGE, lets look at an example of MySQL’s REPLACE function. We’ll do this by example, as everybody loves a good sample. We’ll take a company that has a ticket tracking system, with a table containing the employee ID number, and their names. We have an employee, Susan Johnson, who is about to get married to her long time high-school sweetheart, Scott Drew. Susan’s employee ID is 55458. There are two ways which we could update her last name in the database, the basic UPDATE statement, which will look like this:

Then there is the REPLACE method, which looks similar to an INSERT:

In the event that HR had not done a good job, the REPLACE into statement would create a record if one did not exist for Susan, or if a record did exist for her, it would update her last name. Pretty cool huh? There are some caveats to this function, so take a look at the documentation, and the user comments.

T-SQL MERGE

Microsoft did a good job with this new function, making it quite flexible. For a detailed look into it, you should read the documentation, it’s quite extensive, and gives a complex example, but for us, we’ll stick with our simple example of Susan Johnson.

Okay, that’s an awful lot of code that you managed to do with a single line in MySQL, but unlike MySQL the T-SQL code allows you to match on any field, not just a primary key like employee ID. Lets boost up the example, and show you what we can really do here…

For this, I’m going to use a sample of what I do nearly every day. We get requests to add vehicles to a store’s data, or update data if it already exists. Originally this would require two separate runs, an insert (using LEFT JOINs to find the missing target data), and an update.

Now with this example, we can see I do a lot more than match on the primary ID of the table (which in our case is actually vehicle_id). This update would not have been possible with MySQL because it would have required you to use the vehicle_id from the vehicle_data table, which the new_customer_vehicles table did not have. Whilst it still looks like a fair bit of code, it is now a single statement over 2 different statements to achieve the same thing. There is something else handy about the MERGE function too, you get to do something with the unmatched data in the target side. Taking the same as above, we’ll assume the customer sent us data straight from their system, and what they gave us must match, which means stuff we have showing, should no longer be showing. We do this by changing the status ID. Appending the following code allows us to tweak the records that don’t match…

Now that is a handy, and powerful function. I dare say I’ll be using it a lot more now I’ve discovered it, and might even start poking our developers to start integrating it into places where it’d work better than what they are currently using.

This is just one of the cool things they added to SQL 2008 that I like, what other things have they added that you like too?

Every now and again, we have issues where a developer will have their laptop rebuilt, and forget they have files checked out in Team Foundation Server (TFS). This isn’t very helpful as sometimes they cannot undo their check outs. Visual Studio doesn’t give you access to undo a checkout in another workspace if it’s not your workspace, however the TF.exe command does. Here’s how:

This will happily undo the change on that particular file. If you have a whole directory to revert the changes on, that’s easy too…

You must remember when reverting somebody else’s change to use the project URL, not the physical path on your local machine, otherwise you’ll get an error like this:

For more details on the tf.exe undo command, see the Microsoft documentation here. Full details on arguments for the tf.exe command can be found here too.

This is a quick and handy one for server monitoring, and tracking down that process that is using all your CPU. It makes use of WMI counters.

Okay, this one is a mouthful of pipes, and formatting to make it easier to see. But the breakdown is pretty easy…

• Get the process information
• Exclude _Total and Idle
• Sort the output by descending processor usage
• Only get the first 5 rows
• Format the results into a table, with the columns Name, Process ID, and processor usage.

Output looks something like this:

Another handy hint here, the back-tick is PowerShell’s way of allowing you to line-wrap. This makes it a little easier to read, especially when giving it as samples like above.

I’ll be bundling parts of this for Nagios to return what processes are using the most processor when I get CPU alerts, allowing me to make quicker judgments on what I need to do to correct the issue.

Curious about how long your script took to execute? How about just that cmdlet? Powershell has a built in function for you.

• Open Powershell command prompt
• Use the command Measure-Command like so:

Enjoy the output broken down by days, hours, minutes, seconds, milliseconds, and ticks.

The same works for cmdlets too…

This one is a quick and easy one. We have multiple IIS boxes, which each generate large usage logs on a daily basis. To save space, and for analytics off-server, we compress the log files. Having found Rar is one of the better compression algorithms, we simply shell out to rar, compress the logs, and cleanup older logs.

This could probably do with some cleaning up as there are probably some slightly more efficient ways of doing some of this, but it works.

We’ve had a stubborn Windows 2003 server that we’ve been trying to get Office 2007 installed for a while. It has a couple of things wrong with it which have made the install substantially harder.

The first being an MSI speed issue. Because the server plays host to about 120+ custom applications, all installed using MSI, any further MSI installs simply drag whilst it reads through all the keys, does its validation and backup checks, and then proceeds.

The second issue we have is that this box has a partially installed, corrupted Office 2007 install already on here. Attempting to do “upgrades” fail, and uninstalling just barfs with a delightful error. This resulted in some stumbling to figure out how to get it off, and start again.

Fortunately Microsoft do frequently document things like this, you just have to find the right search terms to find it, which luckily I did, in the form of KB928218. This article gives you a list of the registry keys you have to go through and remove, and you should be all well and good. Unfortunately, this list is over 200 registry keys on our count, and quite a list all over the place. This is where Powershell comes in handy…

As the server itself doesn’t (yet) have Powershell installed, I ran the commands from my laptop, which required remote registry access. Powershell makes use of the .Net libraries to do this:

The LocalMachine reference here is the part of the registry we want to open, which is equivalent to HKEY_LOCAL_MACHINE. This now gives us a registry object in the HKLM section of the registry. From here, there are a bunch of functions we can use to fetch the keys we need. The first is OpenSubKey. This opens a subkey in the $reg object.

This assigns $regKey to the subkey. The boolean flag $true on the end is to make sure the key is opened in writable mode. This will be required to modify the contents of the key.

Per the KB article, Microsoft say to delete all of the 12.0 subkey from under here. I had originally used the following:

This resulted in an error about not being allowed to delete subkeys. Then I stumbled upon the much more suitable DeleteSubKeyTree, which, as the name hints, deletes the tree. So the new code became:

This was repeated for the various other subkeys Microsoft recommended deleting. It became more complex when it asks you to delete keys that match some of the characters below, and use a * as the match. For example:

Note In the following registry keys, the asterisk character (*) represents one or more characters in the subkey name. HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\*0FF1CE*

Fortunately, Powershell is very good at string manipulation, and queries, so this was a bit of a case for a where, and foreach loop.

It looks pretty messy, but it’s actually pretty easy if you understand pipes (the | character). The above line 24 boils down to:

• Get the subkey names
• Find only the ones that have 0FF1CE in them (yes I know that’s office with a zero and a one)
• For each one that matches, go delete it, and its children

See, not so hard.

There are some parts of this that venture KB into other areas of the registry, so the syntax changes in the opening command, for example:

This opens HKEY_CLASSES_ROOT. You can use CurrentUser to access HKEY_CURRENT_USER too.

That’s about it for remote registry access and fixing this blasted Office install. As a side, the script saved me about 30 minutes of bashing through about 200+ registry keys for cleaning up.

Below is the full script, with the delete statements commented out. If you have 2007 installed, give it a run, see what happens (please double check that all the deletes are in fact commented out, this script was used against a server to fix an install issue, and did have active delete statements in, and I cannot remember if I got all the deletes).

For the last day or so, we’ve been working on rolling out the next release of our code to our QA environment. When it came time to build a new set of solutions I stumbled across a weird validation error.

This error popped up on the setup applications for several new programs. Doing some searching I stumbled across a couple of posts that hinted on removing the project output from the setup, and trying again. This stopped the errors on the setup application, but not the actual application itself.

Then I saw a hint in the errors list…

Checking into the app.config file, I stumbled across the cause. It looks like somebody had pasted code that had been “fixed” up by Outlook, or word, and broken a comment. Fixing this, resolved both the app compilation, and the setup validation error.