In Exchange 2003 hiding accounts from the GAL used to handled by an Active Directory (AD) user attribute called msExchHideFromAddressLists. When this was set to TRUE, the user would be hidden from the GAL. Our HR applications toggle this flag for users that are disabled to hide them away from other users.

This process all worked fine for a long time, until Exchange 2007 rolled around. I guess there was plenty of push to allow you to hide a user from all the GALs, but still allow specific GALs to have those users in. So Microsoft introduced a new AD user attribute called showInAddressBook. The problem now appears that if you toggle the msExchHideFromAddressList, but have a value set for showInAddressBook, the user accounts are no longer hidden in the GAL mentioned in the latter attribute.

Can anybody see where this is going? Yup, it appears that all the user accounts were getting the default GALs assigned to the showInAddressBook attribute, so even when they were having the option to hide the user, they were still showing up¹. This was causing problems as people that were disabled/terminated were still showing up, and causing some confusions and concerns.

I started to poke around, and bashed together a quick PowerShell script that will walk through all disabled users that have a showInAddressBook attribute, it’ll then wipe out that attribute.

$ldapFilter = '(&(objectClass=user)(userAccountControl:1.2.840.113556.1.4.803:=2)(showInAddressBook=*))'
$ldap=[ADSI]"LDAP://OU=Disabled Accounts,DC=DOMAIN,DC=TLD,DC=PVT"
 
$srch = New-Object DirectoryServices.DirectorySearcher($ldap)
$srch.Filter = $ldapFilter
$foundUsers = $srch.findAll()
 
$foundUsers | % {
       $adUser = $_
       $props = $adUser.Properties
       $objUser = [ADSI]$("LDAP://{0}" -f $props.get_Item('distinguishedName').Item(0))
       $objUser.PutEX(1,'ShowInAddressBook', $null)
       $objUser.SetInfo()
}

If you’ve not seen LDAP queries before, they work by starting with the operator (and, or, etc), and then the objects that they apply to. So in the example above, it reads as such:

(objectClass=user) AND (userAccountControl:1.2.840.113556.1.4.803:=2) AND (showInAddressBook=*)

It can get a little more complicated when you start stringing together multiple options such as AND and OR operators, and various combinations of them. In this example, we’re going for pretty simple.

I then used the .NET libraries System.DirectoryServices.DirectorySearcher. This uses the LDAP query specified, and returns all matching results. Next was a case of walking through the results, and fetching a DirectoryEntry object to edit the properties. In this case we’re setting it to $null which removes it.

After letting this script run over about 25k users disabled users, it cleared up the fluff in the GAL, and made HR happy.

In the enterprise licensed version of RDM you are given the ability to add “remote management” interface details to a host configuration. In our environment, that remote management interface is iLO, and is available from a dedicated IP address, over HTTPS, giving you access to a remote console as well as power management features. RDM handles this with a small tweak to the XML file adding another element under the connection meta information.

The XML that RDM is looking for is like this:

<Connection>
  <MetaInformation>
    <ServerRemoteManagementUrl>https://hostname-ilo</ServerRemoteManagementUrl>
  </MetaInformation>
</Connection>

I’ve removed most of the information, which you can see in the previous post.

As we’re trying to be careful with the file, we need to first validate the XML has a MetaInformation element, and then an existing ServerRemoteManagementUrl element. If one, or neither, exist, then they get created. Not all hosts have iLO interfaces, such as virtual machines, so we need to verify the presence of a DNS record first, and then only create the entry if it exists.

$ilo_str = "https://{0}-ilo"
$ilo_host = "{0}-ilo"
 
[XML]$conns = Get-Content c:\temp\connections.xml
 
$nodes = $conns.ArrayOfConnection.SelectNodes("Connection[ConnectionType[.='RDPConfigured']]")
 
if ($nodes -eq $null) {
     continue;
}
 
$nodes | %{
     $node_name = $_.Name
     $meta = $_.SelectSingleNode("./MetaInformation")
     if ($meta -eq $null) {
          $meta = $conns.CreateElement("MetaInformation")
          $_.AppendChild($meta)
     }
 
     $ilo = $meta.SelectSingleNode("./ServerRemoteManagementUrl")
 
     if ($ilo -eq $null) {
 
          $dns = $null;
          try {
               $dns = [System.Net.Dns]::GetHostAddresses($($ilo_host -f $node_name))
          }
          catch [System.Exception]
          {
            ## Doing Nothing ##
          }
 
          if ($dns -ne $null) {
               $ilo = $conns.CreateElement("ServerRemoteManagementUrl")
               $ilo.InnerText = $ilo_str -f $node_name
               $meta.AppendChild($ilo)
          }
 
     }
 
}
 
$conns.Save("C:\temp\connections_2.xml")

Again, working with a copy of the original file, I use some crafty XPath queries again to only select connections that are RDP. I then loop through the connections/nodes, and extract the name. Lines 14-18 test for the presence of the MetaInformation element, and create it if it doesn’t exist. Line 20 checks for the ServerRemoteManagementUrl element, if it’s not there, it creates it proceeds with DNS validation.

Lines 24-31 perform a DNS lookup, unfortunately it returns an exception rather than a $null or empty object, so I had to throw in some quick dummy catch code that doesn’t really do anything. If a DNS record is returned it creates the new element, and adds it to the MetaInformation element. For the final step, I saved it to a second file so I could do a comparison between the files to make sure it did as I expected.

One thing to note about adding elements to an XML document is that the CreateElement function (lines 16 and 34) are not executed against the node you are adding the element to, they are executed against the document root. This is so that the element gets all the correct name space information. You then append your element to the existing element.

Remote Desktop Manager (RDM) uses an XML file for its configuration, which makes editing it really easy. To clean up the invalid descriptions, I used a little PowerShell and some XML know-how. Here is an example entry I need to clean up…

  <Connection>
    <ConnectionType>RDPConfigured</ConnectionType>
    <Events />
    <Group>MyDomain\App Servers\DEV</Group>
    <ID>73146eeb-caf9-4579-a146-41f7330261a6</ID>
    <MetaInformation />
    <Name>SERVER1</Name>
    <ScreenSize>R1280x800</ScreenSize>
    <Stamp>5f8a9830-fc2e-440e-a72b-f889d5b17a5b</Stamp>
    <Url>SERVER1</Url>
    <Description>HP Command View EVA</Description>
  </Connection>

And here is the PowerShell that is used to cleanup the file.

[xml]$xml = Get-Content C:\Temp\Connections.XML
 
$node = $conns.SelectSingleNode("//Description[.='HP Command View EVA']")
while ($node -ne $null) {
    $node.ParentNode.RemoveChild($node)
    $node = $conns.SelectSingleNode("//Description[.='HP Command View EVA']")
}
 
$xml.save("C:\Temp\Connections.XML")

Pretty simple, but here is how it works. The first line is pretty obvious, it’s getting the content of the file¹. It then explicitly converts the array object into XML using [xml]. The next bit is where it gets a little harder, and requires a little knowledge of XPath syntax. The code is looking to select a single node, that has the name “Description”, with the data in it that says ‘HP Command View EVA’. If it’s found, it’ll return a XMLElement object, otherwise $node ends up being $null. This gives us the ability to wrap the search in a loop, and remove the elements we don’t need. To remove the element, you have to tell the parent node to remove it, so you ask the node to go back to the parent to remove itself, a little weird, but it works. The final step is to go back and save it to a file.

The hardest bit about handling XML is knowing how XPath stuff works, once that is understood, the rest is usually pretty easy. PowerShell treats XML as an object, so it’s easy to figure out what you can do with the objects using Get-Member.

After scheduling some down time to replace the memory, I went to slide the server out on the rails, and discovered that the HP ProLiant DL585 G1 series servers has the health status board on the top of the chassis. This confused me because all the other servers we have have either a pop out tray with the information on, or some kind of LCD up from that reports the status.

What caught me by surprise was the fact that the server had the full hardware layout instructions on the top of the server. It’s been a long time since I’ve seen this. Granted it is an old server (G1 class servers are getting a little long in the tooth), but it was nice to see the full details right where I needed them, and not having to go hunting through HPs terrible support site for the details.

What’s even more interesting to me was the hardware health status board. On the newer servers, this is usually smaller than a credit card, and about as thick as 3 quarters stacked on top of each other. In this DL585, the board is the size of a large network card, with multiple LEDs feeding clear plastic shafts that pipe to another set on the bottom of the chassis top.

I find it interesting to see how hardware configurations have grown, and changed over time, even the very small things. I don’t see many servers with instructions on any more, probably a money saver for the vendors. I also don’t see much in the way of big health boards any more, managing to squeeze so much into a tiny board now. The advancement in chips and technologies has made the server realm quite interesting to poke around in.

Calling in the libraries from the .NET framework; this task is actually really easy.

$src = "http://localhost/test.cmp"
$dst = "F:\Downloads\junk\test_{0}.cmp"
$web = New-Object System.Net.WebClient
 
$web.Headers.Add([System.Net.HttpRequestHeader]::AcceptEncoding, "gzip")
 
1..100 | %{
	$web.DownloadFile($src, $dst -f $_ )
}

The script is relatively self-explanatory. $src and $dst are the source and destination files. For the destination I’ve used a formatted string allowing me to inject values into the string using C# style formatting. I create a new object using the System.Net.WebClient type. I then loop 100 times and download the same file, saving it to a different destination name each time.

The above code includes setting an “Accept-Encoding” header to the request. Because I was testing a gzip compression issue on IIS I needed this header, otherwise WebClient just requests the raw data. The caveat to this is that the file being written out to the $dst path is actually a gzip compressed file and not the actual data. This is fine for my testing because it made it quick and easy to see if compression had worked. I was downloading a file containing 300KB of Lorem Ipsum text. If it the IIS compression worked the file would be much smaller. I’ll do another blog post soon about handling the gzip data and turning it back to the same as the source file.

I could have done the same using the BITS service, but BITS would probably have fixed the gzip’d file so I’d have to do more work to determine if compression had actually worked (examining logs, network trace, IIS trace, etc). This worked out nicely.

As a side note, the “1..100” code is a shortcut for generating a sequence of numbers from 1 to 100. If you opened a powershell prompt, typed “1..100” and hit enter it’d spew out all the digits from 1 through to 100. Passing this on to foreach-object we can turn it into a loop. The other way to do this would be to use a for, do, or while loop.

for( $i=0; $i -lt 100; $i++) {
  $web.DownloadFile($src, $dst -f $i)
}
 
$i=0
do {
  $web.DownloadFile($src,$dst -f $i)
  $i++
} until ($i -eq 100)
 
$i = 0
while ($i -lt 100) {
  $web.DownloadFile($src, $dst -f $i)
  $i++
}

In this case, I’m referring to the design of HPs DL365 G5 servers. Particularly the decision for the location of the storage controller’s battery. I understand that 1U servers have some very specific, and complex requirements, such as:

• Must fit all the stuff in a small space
• Must have good air flow
• Must provide easy access to hot-swap parts

Earlier this month we got an alarm on a server that the battery had gone bad, and was no longer holding a charge. I scheduled downtime, made sure we had a battery on hand, put in a change ticket, and set to work.

After powering down the server, and removing the top cover, I set out to find the battery. The storage controller was next to the memory banks, and had 3 cables coming from it. 2 were obviously for the HDD back planes, the third, a little thinner, disappeared behind some fans, and into the front of the case.

If you take a look at the picture to the right, and look above the center fan bank, you can see a thin black cable disappearing under the chassis. A closer look is in this picture. From the front, it’s hiding right above the middle drive behind this grill.

To remove the fan bank is a project on its own. Dell uses nice quick release containers, squeeze some colored tabs, and stuff just pops apart. To get these out, I had to use a flathead screwdriver to pry the metal tabs out the way. Well, they’re not really tabs. They were about 2 inches long, and about the same deep, and not flexible in the slightest.

Once removed, it should be a simple case of pulling on a small black tab. This is where the design flaw comes in (as if the fact its hidden in there to begin with isn’t bad enough). These storage batteries have a tendency to swell as they get older, even more so as they reach the end of their life. If you look at the picture to the left, you can see the bulge already. Why is this a problem? Well 2 reasons, the first being fairly obvious… If it’s bulging, there is a chance for it to explode, or leak, all over the inside of the server, in particular onto the hard drives below it. The second, is the design flaw, the battery is nicely wedged between 2 sheets of rivited metal, and when the battery swells, it does NOT want to come out.

It can take some ‘gentle’ pursuasion to evict the battery from it’s sanctuary without puncturing the first cell on the little board. In this case, I had to use a flat head screwdriver, and go through the grill at the front of the chassis above the drives to push it, while I carefully pulled from the other side at the same time. There is a few more pictures of the swollen battery here, but this one shows it pretty well.

The battery was probably put in that location because it was the only free space left. Little consideration was taken for maintenance, or hardware faults, such as swelling batteries. I believe the G7 series has this issue resolved though.

• New job
• More volunteer work with CERT
• More volunteer work at school

Generally as busy as before, but at the same time, a different kind of busy.

Job

The transition to a new job was probably one of the harder decisions I’ve had to make in a long time, but now I’ve settled in, I don’t regret it at all. I’m getting my hands on a lot more stuff I didn’t have access to before, as well as taking lead on projects that were originally left to HQ folk.

The environment is a lot larger. I’ve gone from a handful of few racks with gear to 20+ full racks, not counting the volume of VM guests we have running.

Documentation is pretty limited, so I’m working on updating it all, starting with a full data center audit. I’m also working on a Lync pilot, as well as updating monitoring and other projects.

CERT

I’ll have a complete post on this one later. In the meantime, you can read Christopher Webber’s post on CERT and Sys Admins.

With tornadoes striking the DFW metroplex recently, we were asked to help with some cleanup operations. Post and pictures on that later.

So lots of new exciting stuff floating around, learning new stuff, getting my fingers into new toys and such. I’ll post more stuff as it comes up.

A few weeks ago I stumbled on an odd issue where Eclipse wasn’t keeping in-sync with what TFS Anywhere was doing. For example, we keep our Nagios configs checked into TFS for version control. I was using the Mac, and had added several new contact files. When I was back using my Windows machine, I told Eclipse to trigger the “Get Latest” function on TFS Anywhere, which it dutifully did, then threw an error…

The item $/infrastructure/Nagios/etc/objects/contacts/newcontact.cfg already exists

It repeated this error for each of the new contacts I had added. What appeared to be happening is TFS Anywhere was getting the files, notifying Eclipse the files were there, which triggered the project to add them to the file list, which then in turn triggered Eclipse to try an add them to TFS. A nice little circle going there. I thought maybe it was just a quirk and closed the project. When I reopened the project, the same error occurred again, this time I didn’t even tell Eclipse to do any updates, this occurred just on opening the project.

After some skimming around, I discovered this is likely a bug in TFS Anywhere, and the simple solution is to tell Eclipse to update the server information. This is done using the “Team” context menu (right click the project), then selecting “Refresh Server Information”. Here is the reference post I found the details on.

The first task is to get the ADO.Net provider, which I found over here. The provider hasn’t been updated in a while, but still seems to work just fine. Next is loading the libraries, this is a case of calling the Assembly loader, so the code looks like this:

[void][System.Reflection.Assembly]::LoadFrom("C:\Temp\System.Data.SQLite.DLL")

Remember to update the path to reflect the actual location. I have [void] at the beginning to hide the output of the library being loaded. If you are having issues, try removing it to see any errors.

Next is just like using any data object.

Add-Type @'
public class Subscription
{
	public string url;
	public string title;
}
'@
 
[Void][System.Reflection.Assembly]::LoadFrom("C:\temp\System.Data.SQLite.DLL")
$cn = New-Object -TypeName System.Data.SQLite.SQLiteConnection
$cn.ConnectionString = "Data Source=C:\temp\config\database.sqlite"
$cn.Open()
 
$cm = New-Object -TypeName System.Data.Sqlite.SQLiteCommand
$sql = "select title,url from channels"
$cm.Connection = $cn
$cm.CommandText = $sql
 
$dr = $cm.ExecuteReader()
 
$subs = @()
 
while ($dr.Read()) {
    $sub = New-Object Subscription
	$sub.url = $dr.GetString(1)
	$sub.title = $dr.GetString(0)
 
	$subs + $sub
}
 
$subs

In this code, I’m creating a custom object to hold the subscription data in it, loading the libraries, and querying the “channels” table from the file c:\temp\config\database.sqlite, and then outputting the Title, and URL for the feed.

The great thing about PowerShell is it’s ability to pull in, and use any of the .NET libraries, making it incredibly flexible for quick scripts, as well as more in-depth work. The above script took all of about 5 minutes, and got my podcast list back.