Updating multiple Exchange 2010 Receive Connectors at once

Although I haven’t really delved into it properly yet, PowerShell is an awesome tool to have in your toolkit when managing Exchange. When we deployed Exchange 2010, I supplied the sysadmins in our other offices with a list of PowerShell commands to configure their servers appropriately. This saved a massive amount of time and hassle.

I recently had to bump up the incoming message size limit on our incoming SMTP (from Messagelabs) Receive Connectors to 26MB on five different servers.

Back in the days of Exchange 2003, embarrassingly for me, not that long ago; the same task would have required much mouse clicking and searching for settings. Now, in Exchange 2010, since I’ve followed a naming convention for the connectors across all of the servers, it’s easy to retrieve all of the relevant ones at once:

[PS] C:>Get-ReceiveConnector |Where-Object  {$_.Identity -like "*Messagelabs*"} | fl identity

Identity : SITE1-MAIL-CAS1Incoming SMTP from Messagelabs SITE1-MAIL-CAS1 Identity : SITE1-MAIL-CAS2Incoming SMTP from Messagelabs SITE1-MAIL-CAS2 Identity : SITE2-MAILIncoming SMTP from Messagelabs SITE2-MAIL Identity : SITE3-MAILIncoming SMTP from Messagelabs SITE3-MAIL Identity : SITE4-MAILIncoming SMTP from Messagelabs SITE4-MAIL Identity : SITE5-MAILIncoming SMTP from Messagelabs SITE5-MAIL

Updating the MaxMessageSize on all of the connectors is then as simple as issuing this one liner:

Get-ReceiveConnector | Where-Object  {$_.Identity -like "*Messagelabs*"} | Set-ReceiveConnector -MaxMessageSize 26mb

The great thing with many PowerShell commandlets is that you can “get” the objects you’re after, and then pipe the results into the command that makes the changes.

This is just a really simple example, and most experienced Exchange admins would probably laugh at the fact that I’ve documented something this simple – but I just wanted to highlight how easy it is to make “bulk” changes with PowerShell and EMS, the Exchange Management Shell.

Remember that you might want to run commands such as this with the “-WhatIf” switch just to check what’s going to be affected before going ahead:

[PS] C:>Get-ReceiveConnector |Where-Object  {$_.Identity -like "*Messagelabs*"} |Set-ReceiveConnector -MaxMessageSize 26mb -WhatIf
What if: Configuring Receive connector "SITE1-MAIL-CAS1Incoming SMTP from Messagelabs SITE1-MAIL-CAS1".
What if: Configuring Receive connector "SITE1-MAIL-CAS2Incoming SMTP from Messagelabs SITE1-MAIL-CAS2".
What if: Configuring Receive connector "SITE2-MAILIncoming SMTP from Messagelabs SITE2-MAIL".
What if: Configuring Receive connector "SITE3-MAILIncoming SMTP from Messagelabs SITE3-MAIL".
What if: Configuring Receive connector "SITE4-MAILIncoming SMTP from Messagelabs SITE4-MAIL".
What if: Configuring Receive connector "SITE5-MAILIncoming SMTP from Messagelabs SITE5-MAIL".

Failing Exchange Transport Service after Policy Patrol 8 upgrade

Just had an issue on one of my Exchange 2010 (SP2 RU3) CAS servers when upgrading Policy Patrol from v7.x to v8.0. The install/upgrade completed successfully, but I noticed the following errors in the Application log afterwards:

Error 1052, MSExchange Extensibility
Failed to create agent factory for the agent 'Policy Patrol Email Edge' with error 'Failed to create type 'RedEarthSoftware.PolicyPatrol.Email.Agent.EdgeTransportFactory' from assembly 'C:Program Files (x86)Red Earth SoftwarePolicy Patrol EmailAgentPolicyPatrol.Email.Agent.dll' due to error 'Invalid agent assembly path.'.'. Please verify the corresponding transport agent assembly and dependencies with correct version are installed.

Error 16023, MSExchangeTransport
Microsoft Exchange couldn't start transport agents. The Microsoft Exchange Transport service will be stopped. Exception details: Failed to create type 'RedEarthSoftware.PolicyPatrol.Email.Agent.EdgeTransportFactory' from assembly 'C:Program Files (x86)Red Earth SoftwarePolicy Patrol EmailAgentPolicyPatrol.Email.Agent.dll' due to error 'Invalid agent assembly path.'. : Microsoft.Exchange.Data.ExchangeConfigurationException: Failed to create type 'RedEarthSoftware.PolicyPatrol.Email.Agent.EdgeTransportFactory' from assembly 'C:Program Files (x86)Red Earth SoftwarePolicy Patrol EmailAgentPolicyPatrol.Email.Agent.dll' due to error 'Invalid agent assembly path.'. ---> System.ArgumentException: Invalid agent assembly path.
   --- End of inner exception stack trace ---
   at Microsoft.Exchange.Data.Transport.Internal.MExRuntime.FactoryTable.CreateAgentFactory(AgentInfo agentInfo)
   at Microsoft.Exchange.Data.Transport.Internal.MExRuntime.FactoryTable..ctor(IEnumerable agents)
   at Microsoft.Exchange.Data.Transport.Internal.MExRuntime.RuntimeSettings..ctor(MExConfiguration config, String agentGroup)
   at Microsoft.Exchange.Data.Transport.Internal.MExRuntime.MExRuntime.Initialize(String configFile, String agentGroup, Boolean isBridgeHead, String installPath)
   at Microsoft.Exchange.Transport.Extensibility.AgentComponent.Load()

The Microsoft Exchange Transport service was failing because it seemed that the transport agent configuration hadn’t been updated as part of the upgrade. This was caused by issues with the Exchange Management Shell on the server in question. I found this in the Policy Patrol install logs:

ERROR:The Windows PowerShell snap-in 'Microsoft.Exchange.Management.PowerShell.Admin' is not installed on this machine.
ERROR:The Windows PowerShell snap-in 'Microsoft.Exchange.Management.PowerShell.E2010' is not installed on this machine.

Luckily, I had another server to reference, so this is what I did to resolve the problem:

  1. Stopped the Policy Patrol services
  2. Opened a new EMS (Exchange Management Shell) window and did this:
    1. Uninstall-TransportAgent “Policy Patrol Email Edge”
    2. Uninstall-TransportAgent “Policy Patrol Email Hub”
  3. Closed the EMS window, and restarted the Microsoft Exchange Transport service
  4. Opened a new EMS window, and did the following:
    1. Install-TransportAgent -Name “Policy Patrol Email Edge” -TransportAgentFactory “RedEarthSoftware.PolicyPatrol.Email.Engine.Ex2007.EdgeAgentFactory” -AssemblyPath “C:Program Files (x86)Red Earth SoftwarePolicy Patrol EmailPolicyPatrol.Email.Engine.Ex2007.Sink.dll”
    2. Install-TransportAgent -Name “Policy Patrol Email Hub” -TransportAgentFactory “RedEarthSoftware.PolicyPatrol.Email.Engine.Ex2007.PostCatAgentFactory” -AssemblyPath “C:Program Files (x86)Red Earth SoftwarePolicy Patrol EmailPolicyPatrol.Email.Engine.Ex2007.Sink.dll”
  5. Restarted the Microsoft Exchange Transport service
  6. Opened a new EMS window
    1. Enable-TransportAgent “Policy Patrol Email Edge”
    2. Enable-TransportAgent “Policy Patrol Email Hub”
  7. Restarted the Microsoft Exchange Transport service
  8. Started the Policy Patrol services

That’s it. No more issues loading the transport agents, and no more crashing Microsoft Exchange Transport service.

Update – 10/07/2012: Added some info about the cause of the problem.

ActiveSync woes–“Cannot get mail” and the case of the endless re-sync

cannotgetmailWe recently experienced a really bizarre issue with our ActiveSync infrastructure. Users started complaining that their contacts were disappearing, and that their inboxes would re-synchronise constantly. All items in the inbox would disappear, and then reappear, starting with the oldest item. Some items were even dated at the Unix epoch. Users on iOS would get an error screen “Cannot get mail”, and downloading emails would time out or take a very long time.

We’re set up with TMG in our DMZ, which then sends traffic to a pair of CAS servers internally. We’ve been running Exchange 2010 SP2 and 2003 in co-existence for some time now, as some of our national offices are still in the process of migrating users across.

Our troubleshooting covered all areas, from looking at ActiveSync logs from IIS, running the Test-ExchangeConnectivity scripts, to testing on the devices themselves – you name it, we tried it. Here’s a quick way to turn up the logging level on ActiveSync using PowerShell:

Get-EventLogLevel | Where-Object {$_.Identity -like "MSExchange ActiveSync*"} | Set-EventLogLevel -Level High

The usual suggestions of permissions on the user account in AD and various other settings were not relevant. We even investigated the possibility that the problem could be caused by users still on iOS 4.0, which was known to cause issues and unusually high server load.

We then noticed that the TMG box would experience timeouts when requesting DNS resolution from our internal DNS servers. There were also errors from the TMG connectivity verifiers for AD that the LDAP servers were unreachable. This pointed to some sort of connectivity issue between TMG and and the CAS servers. Circumventing the TMG box by VPN’ing in or connecting via our corporate WiFi seemed to resolve the issue.

Upon inspection of our Netscreen 25 firewall, we noticed a lot of error messages about the source IP session limit being exceeded:juniperlog

This is by design. It turned out that our DMZ had previously had IP based session limits set to a threshold of 128 sessions. This limit was being exceeded by the large number of ActiveSync users we now have. We bumped up that number to 512, and our problems are now resolved.

Funnily enough, while I was troubleshooting this issue, two ActiveSync troubleshooting-related articles appeared in my RSS reader of choice, Google Reader:

  1. The Exchange Team Blog: A script to troubleshoot issues with Exchange ActiveSync
  2. MSExchangeGuru.com: Troubleshooting Exchange ActiveSync and reading IIS logs

They’re both certainly worth reading, and are a great starting point if you’re experiencing ActiveSync issues.

Propagate mailbox folder permissions

I recently had the requirement where somebody required another person to access a specific folder in their mailbox. We didn’t want to grant full mailbox access. This normally isn’t a problem as we’d just set the permissions on the individual folders in Outlook, but the user in question had an extensive folder structure set up.

I found several tools that were capable of doing this, but I wasn’t going to pay USD$500+ for a tool I was going to use once. In addition to that, some tools didn’t work against Exchange 2010. That equates to a pretty bad investment for our company as we’re currently migrating from Exchange 2003 to Exchange 2010.

I also toyed with the following ideas:

  1. Doing it in Powershell: I’m no Powershell guru, so it would take some time – plus I’d need to move the user’s mailbox to Exchange 2010 before I could use Powershell
  2. Using EWS and write a desktop app in C# (Interesting, but too time consuming)

The Microsoft Exchange Server Public Folder DAV-based Administration Tool (PFDAVAdmin) can be used to do exactly this (and a myriad of other things). Note that the tool has also been renamed and updated to work with Exchange 2010 (including EX2010 SP1).

I’ll be working with the legacy WebDAV version in this example, as the mailbox in question still resides on the Exchange 2003 server.

Andrew Shugg raised a good point in the comments section:

Worth noting that the “classic” PFDAVadmin requires .NET Framework 1.1, and if you try to install that on a modern server system (e.g. Windows Server 2008, Windows SBS 2008) you’ll get warnings about it breaking things in IIS.

If possible, install PFDAVadmin and the .NET Framework 1.1 on a desktop system or non-critical server.

  1. Downloadand extract PFDAVAdmin
  2. Run the tool, PFDAVAdmin.exe
  3. Go to File, Connect, specify the connection details, and connect
  4. Drill down to the mailbox in question, and then to Top of Information Store
  5. Locate the starting-level folder that you’re going to assign permissions to. Ensure that the permissions are correct at that level (right-click, Folder Permissions)
  6. Right-click on that same folder again, and select Propagate ACEs
  7. Select the ACEs (Access Control Entries) that you wish to propagate
  8. Leave the other settings on their defaults, so we’re Adding/Replacing the specific ACE on subfolders
  9. The tool will then run through all of the folders. In my case, this user had hundreds of folders that needed to be modified:

The same process can be used to remove the permissions later on.

Quite handy.

Delete emails from Journal mailbox before they’re journaled

Setting up a new journal mailbox that has Enterprise Vault Journal Archiving turned on, and I don’t want emails from our monitoring system to go into the journal archive.

When I tried logging onto the ECP as that user I’d get the OWA language and time zone prompt screen, and then an error “Bad Request”, so I couldn’t create a rule via the GUI.

This is what I ended up doing:

New-InboxRule -Name "Delete Monitoring Messages"
 -Mailbox domainjournalaccount
 -from monitoring@domain.local -DeleteMessage $true

(wrapped for legibility)

The only problem now is that the messages go to the Deleted Items folder in that mailbox, but the mailbox management process will take care of that.

The rules take effect before Enterprise Vault picks up the email, so this works well. Not sure about any performance impact this may have, but I’m only running it in a small environment.

To view the inbox rules that are configured for the mailbox, do the following:

Get-InboxRule -Mailbox domainjournalaccount

Or in detail:

Get-InboxRule -Mailbox domainjournalaccount | Format-List

NLB on a Hyper-V host

Just a quick gotcha I came up against when I was trying to set up a CAS array on my Exchange 2010 boxes. I kept getting errors when trying to set up a unicast NLB cluster, and the nodes wouldn’t converge.

I ended up figuring out that I had to enable MAC address spoofing on the virtual NICs. The cluster needs to assign the same MAC to both cluster NICs, and this isn’t possible without having that checkbox ticked. This feature is only available in Hyper-V since Server 2008 R2 was released.

I was going to write a more detailed article about it, but as usual, Paul Cunningham’s great blog has it covered already in detail with screenshots.