While trying to help a co-worker today, I stumbled across a documentation error on Microsoft’s TechNet in relation to Set-DnsServerResourceRecord. The example uses multiple variable initialization, which unfortunately ends up making pointers.

There can be a lot of confusion over when to use single and double quotes in PowerShell. Not to worry, this confusion carries over in a lot of programming and scripting languages, such as Perl. I figured, after seeing this tweet, to give a quick run down of when to use which one.

The steps to replace the SSL certificate in your Microsoft Office Web Apps farm seem to be fairly simply, but we stumbled on an odd issue where it was complaining on some of the farm’s member servers that the certificate couldn’t be found.

We’re in the process of enabling a new password reset portal, which requires additional licensing features in Office 365. There is no “apply all” button in Office 365, so we have to do this via script to tens of thousands of user accounts. The problem with this is some form of feedback to the user running the script. PowerShell has some really handy built in features, such as a progress bar. It’s amazing how something so simple can make you feel a little better, and actively see how things are moving along. Progress bars in PowerShell are incredibly simple.

One of the things about new platforms is you get to learn new technology. One of the bad things about new technology is that a lot of your old methods might not apply anymore, need to be revamped, or redesigned completely. Over the last few months I’ve been working on a Cisco UCS platform deployment for work. This has been quite exciting as it’s all new gear, and it’s implementing stuff that we should have been able to implement with our HP BladeSystem C7000 gear.

A year or so ago we’d upgraded our vCenter from 4.1 to 5.1, and with this upgrade, and some features built into our SAN, we got access VAAI. For example, removing a VM guest would tell the SAN that the guest had been removed, and if the data store had been thinly provisioned from the SAN, it’d clean up and shrink down the space used (in theory).

Another feature we discovered was something called “fast copy”. In layman’s understanding of this feature, when a storage vMotion request was created, the SAN was notified of the request, and the SAN would process the copying of the bits in the background. This is handy because it stops the data from being sent from SAN to host to SAN again. This causes a good speed up with regards to moving machines around.

In the process of migrating our mailboxes from our on-premise Exchange servers to Office 365, we had to rewrite the mailbox enable scripts. This script keys off of our HR database, does some magic, then calls Enable-Mailbox on Exchange 2010 servers. To update this to support creating mailboxes in Office 365, we needed to set user licenses, and use the Enable-RemoteMailbox command in Exchange 2013¹.

One of the quirks we stumbled upon is a bug in the Exchange 2013 tools that didn’t allow it to identify the domain for the remote routing address. This is what we’d get:

[PS] C:\ > Enable-RemoteMailbox Jonathan.Angliss

The address '@mytenant.mail.onmicrosoft.com' is invalid: "@mytenant.mail.onmicrosoft.com" isn't a valid SMTP address. The domain name can't contain spaces and it has to have a prefix and a suffix, such as example.com.
    + CategoryInfo          : NotSpecified: (:) [Enable-RemoteMailbox], DataValidationException
    + FullyQualifiedErrorId : [Server=Exchsvr01,RequestId=190c9764-d8bd-446e-ac43-7c80bcc54eea,TimeStamp=6/3/2014 1:19:33 PM] [FailureCategory=Cmdlet-DataValidationException] 730D5E7F,Microsoft.Exchange.Management.RecipientTasks.EnableRemoteMailbox
    + PSComputerName        : Exchsvr01

According to the Microsoft documentation for Enable-RemoteMailbox you should be able to specify just the sAMAccountName as an argument, the rest should be calculated.

The remote routing address doesn’t need to be specified because mail flow between the on-premises organization and the service has been configured. Using this configuration, the Enable-RemoteMailbox cmdlet automatically calculates the SMTP address of the mailbox to be used with the RemoteRoutingAddress parameter. — Microsoft TechNet

This apparently isn’t the case, so some tweaking was needed. We called upon Get-ADUser to retreive the account, and fill in the rest.

[PS] C:\ > Get-ADUser jonathan.angliss | Enable-RemoteMailbox $_.sAMAccountName -RemoteRoutingAddress "$($_.sAMAccountName)@mytenant.mail.onmicrosoft.com"

As this is part of a script, the content is slightly different, but you can see how it works. We used Get-ADUser earlier in the script to pull other user data to calculate licensing requirements, but if you’re doing this as a one off and are seeing the error then you could just as easily do this:

[PS] C:\ > Enable-RemoteMailbox jonathan.angliss -RemoteRoutingAddress 'jonathan.angliss@mytenant.mail.onmicrosoft.com'

Hat tip goes to Steve Goodman for posting similar work, and getting me back on track.

In our transition to using Offce365 for email services, we’ve had some interesting discoveries. Some of them are revolving around Dynamic Distribution Lists (DDLs). These are groups which have members identified at time of delivery of emails, and are based on various styles of queries. We usually use PowerShell style queries to build the groups, but LDAP works, as does simple queries based on fixed parameters.

One of the interesting observations is that Exchange will tack extra query parameters into the DDL to exclude system mailboxes. For example the following query string:

(RecipientType -eq 'UserMailbox') -and (CustomAttribute4 -eq 'Plano')

Will actually result in the follow query:

((((RecipientType -eq 'UserMailbox') -and (CustomAttribute4 -eq 'Plano')) -and (-not(Name -like 'System{*')) 
-and (-not(Name -like 'CAS_{*')) -and )-not(-not(RecintTypeDetailsValue -eq 'MailboxPlan')) 
-and (-not(RecipientTypeDetailsValue -eq 'DiscoveryMailbox')) 
-and (-not(RecipientTypeDetailsValue -eq 'ArbitrationMailbox')))

This forces Exchange to exclude any of the system mailboxes for delivery, which is what you want it to do. The problem is, this additional data varies from version to version, and it’s not always backwards compatible. One of the observations is that in Exchange 2013 they introduced a RecipientTypeDetailsValue of PublicFolderMailbox. This is great, except that value is invalid in 2010. What does that mean?

Let’s try an example. From one of our on-prem 2013 hyrbid servers, we’re going to create a new distribution group with the initial query we gave as an example above…

New-DynamicDistributionGroup -Name 'JATestDist' -RecipientFilter "(RecipientType -eq 'UserMailbox') -and (CustomAttribute4 -eq 'Plano')"

Now it’s created, lets see what we have for a query parameter:

PS C:\> Get-DynamicDistributionGroup -Identity 'JATestDist' | Select RecipientFilter | fl


RecipientFilter : ((((RecipientType -eq 'UserMailbox') -and (CustomAttribute4 -eq 'Plano'))) -and (-not(Name -like
                  'SystemMailbox{*')) -and (-not(Name -like 'CAS_{*')) -and (-not(RecipientTypeDetailsValue -eq
                  'MailboxPlan')) -and (-not(RecipientTypeDetailsValue -eq 'DiscoveryMailbox')) -and
                  (-not(RecipientTypeDetailsValue -eq 'PublicFolderMailbox')) -and (-not(RecipientTypeDetailsValue -eq
                  'ArbitrationMailbox')))

As we can see, a whole bunch of extra options, including our PublicFolderMailbox option. Lets test to see what users we get back…

[PS] C:\>$dist1 = Get-DynamicDistributionGroup -Identity 'JATestDist'
[PS] C:\>Get-Recipient -RecipientPreviewFilter $dist1.RecipientFilter
$dist1 = Get-DynamicDistributionGroup -Identity 'JATestDist'
Get-Recipient -RecipientPreviewFilter $dist1.RecipientFilter

Name                                                        RecipientType
----                                                        -------------
Angliss, Jon                                                UserMailbox

Okay, so we get results back, no big deal right? Now I’m going to go to a 2010 server, and without changing anything, I’m going to see what results we get back…

[PS] C:\>$dist1 = Get-DynamicDistributionGroup -Identity 'JATestDist'
[PS] C:\>Get-Recipient -RecipientPreviewFilter $dist1.RecipientFilter
The recipient preview filter string "((((RecipientType -eq 'UserMailbox') -and (CustomAttribute4 -eq 'Plano'))) -and (-not(Name -like 'SystemMailbox{*')) -and (-not(Name -like 'CAS_{*')) -and (-not(RecipientTypeDetailsValue -eq 'MailboxPlan')) -and (-not(RecipientTypeDetailsValue -eq 'DiscoveryMailbox')) -and (-not(RecipientTypeDetailsValue -eq 'PublicFolderMailbox')) -and (-not(RecipientTypeDetailsValue -eq 'ArbitrationMailbox')))" is neither a valid OPath filter nor a valid LDAP filter. Use the -RecipientPreviewFilter parameter with either a valid OPath filter string or a valid LDAP filter string.
    + CategoryInfo          : InvalidArgument: (:) [Get-Recipient], ArgumentException
    + FullyQualifiedErrorId : 79B22B5B,Microsoft.Exchange.Management.RecipientTasks.GetRecipient

Now we see an error. This is because 2010 doesn’t like PublicFolderMailbox as a RecipientTypeDetailsValue, and as such, throws it out as an error. So we have to go back to the 2010 server and edit the query, and reset it to be what we wanted originally:

[PS] C:\>Set-DynamicDistributionGroup -Identity 'JATestDist' -RecipientFilter "(RecipientType -eq 'UserMailbox') -and (CustomAttribute4 -eq 'Plano')"
[PS] C:\>$dist1 = Get-DynamicDistributionGroup -Identity 'JATestDist'
[PS] C:\>Get-Recipient -RecipientPreviewFilter $dist1.RecipientFilter
$dist1 = Get-DynamicDistributionGroup -Identity 'JATestDist'
Get-Recipient -RecipientPreviewFilter $dist1.RecipientFilter

Name                                                        RecipientType
----                                                        -------------
Angliss, Jon                                                UserMailbox

This same query is happy on the 2013 servers as well, however, it will attempt delivery to a Public Mailbox if your other query parameters allow for it. In the example above, we set the RecipientType to be a very specific value, so this shouldn’t allow for this to happen anyway.

One other observation, when migrating your queries to be Office 365 Hybrid complaint, you will also need to include the RecipientType of MailUser. For example:

((RecipientType -eq 'UserMailbox') -or (RecipientType -eq 'MailUser')) -and (CustomAttribute4 -eq 'Plano')

Mailboxes that are migrated change their RecipientType to be MailUser.

There are lots of other fun things about DDLs that you’ll have to be aware of, which I shall cover in a separate post, but this is one of the fun gotchas I discovered in a mixed environment that’ll impact people using Exchange 2010 and 2013 in the same environment.

While setting up SolarWinds SAM AppInsight for Exchange, I stumbled across a small Exchange setup bug where it’s not correctly deploying all the counters for the server roles that are being used. When SAM does the checks for the performance counter, you’ll see an error like the following:

'Average Document Indexing Time'
'Performance counter not found'

The solution is fairly simple, you have to copy the counters from the install media, and register them using PowerShell. The one caveat is that the counters aren’t on all the install media, so if you have CU3 setup files for example, they are not there. You have to go back to the original install DVD and get them from there. Here are the steps:

1. Find the missing performance counter files on the install media, usually in <install media>\setup\perf
2. For the above counter, the files needed are:
   1. IndexAgentPerfCounters.h
   2. IndexAgentPerfCounters.ini
   3. IndexAgentPerfCounters.xml
3. Copy the files to <install path>\Setup\perf
4. Open a PowerShell prompt with elevated privileges
5. Execute the following commands:

Add-PSSnapin Microsoft.Exchange.Management.PowerShell.Setup
New-PerfCounters -DefinitionFileName "<install path>\Setup\Perf\IndexAgentPerfCounters.xml"

Obviously adjust <install path> to be the correct path.

This is documented on the SAM product blog (here) and in the SolarWinds Knowledge Base (here), with the missing step of Add-PSSnapin.

This post was going to be one of those rant posts about not following instructions, and then I realized this is a common problem that a lot of people have issues with, not just in the IT world. It is the importance of knowing what L and R is referring to. By L and R, I mean left and right.

This might seem silly and trivial, because we all know our left hand and our right hand, and in general when somebody says it’s in the left hand side of the cabinet, you know exactly where to look. But what happens if you are working on something that can be reached/used from both sides? Is that the left hand side relative to the front, or the back?

This comes up with other things too. How many times have you gone to a car mechanic and said “my left brake squeals when I apply the brakes”? Is that the left side when looking at the car, or driving the car? This is why you’ll find a lot of mechanics refer to the sides as driver and passenger, there is no confusion there.

The whole point of this post is about the importance of documenting exactly what is referred to as L and R, because it makes a great deal of difference when you are putting rails on a server. Why you might ask? It’s all about server removal…

A lot of rail kits consist of 3 major parts, the server portion, the cabinet/rack portion, and the slider. The server portion is usually screwed/clipped onto the server, and is stationary. The cabinet/rack portion is also stationary, and attached to the cabinet/rack. Then there is the slider portion. This portion is “attached” to the server and cabinet portions, has ball bearings, and allows the server to slide in and out of the rack. It slides on tracks in the cabinet portion, and the server portion slides on bearings to slide out. This allows for people to work on the server without having to completely remove the server.

Also part of the slider is usually a catch. This catch is to stop you from pulling the server completely off the rails and having it come crashing down to the floor. Something most people don’t want to happen. And it is with this part of the rails that it is important to know what is L and what is R. This catch usually has an orientation so that it can “clip” into the slider rail, and pushing the catch down allows the server rail to slide out of the slider rail. If you mount the server rail on the wrong side, the catch either doesn’t work properly, or becomes impossible to remove.

Here is an example of one of those catches…

If looking at this, you cannot figure out how this works, here is another picture with arrows. Arrows make everything easier to understand…

When you pull the server out, it moves in the direction of the top arrow (orange). Near the end of the slider rail is a small block, this block (shows as a green blob) moves along the server rail in the direction of the bottom arrow (green). As it gets to the catch, it pushes it up, and the spring in the catch pushes it back down when the block moves into the void. Because of the shape of the void, the green blob is prohibited from moving any further, and stops the server sliding off the end of the rail.

If you need to actually remove the server from the rails completely, you simply pull the catch up, which moves the blob outside the void of the catch, and pull the server forward. If you put the rail on upside down, instead of the block catching on the void in the catch, it actually stops when it hits the mount point of the catch. This is why it’s important to know which way around to mount the rails (note the little L next to the screw).

This situation caused myself and a co-worker some struggles as we could not get the server unmounted from the rails. Ultimately we ended up having to unscrew the rails from the rack, with the server still attached, fully extend the rails, and then bend them in so that we could pull the server out of the rack. Fortunately this was a sever that is well past EoL, so this wasn’t a hard decision to make, or live with.

That all being said, it is important to make documentation as clear and concise as possible. Images are very useful in this situation. A server we put in place of this one had really clear documentation, and the rails themselves even had pictures of the configuration, essentially saying “This rail goes on the left here” with a picture of where the rail was located in relation to the server.

So next time you’re writing documentation for something, and there is an opportunity for ambiguity, clear up the documentation and remove any doubt.