Exchange 2013 and 2016 OAB

Exchange 2013 has gone through an evolution in the way the Offline Address Book is generated and maintained. While most of the changes are advantageous, I ran into a chink in the armor while working with a customer who was having an issue with Exchange 2013 DAG replication. We were having issues with their Exchange 2013 servers and one node would crash or lose connection and break the DAG replication. This would leave the source database mounted and the destination (the second database copy) would end up in a “Failed and Suspended” state. What ended up happening is about a day later if a user tried to manually download the address book, they would not be able to and would get a 0x80190194 error:


Troublshooting
So what is this? what does the error message mean? Well, let’s look at the log files for OAB downloading. In Exchange Server 2013, these files are located in the %ExchangeInstall%\Logging\OABDownload directory. Open up the latest log file in the directory and see what errors are being reported by the client. In our case, we had these error messages:

Notice the error “IsLocalOABFileCurrent:ManifestEnforcementDisabledAndLocalFileTooOld”. The file location is correct and shows where the current OAB files are. I verified the file location manually but also noticed the date discrepancy (the blue rectangles in the above image) which shows the date the attempted download was made (upper left) and the date of the files (lower right). Because the files were not changed recently the error message was downloaded and the client could not download the files.
To summarize – we have an OAB that is not updating and clients that cannot update their local copies because the copies are too old. Now the problem is becoming visible to all users and will eventually become a support issue for the client.
Further Troubleshooting
Let’s see where the OAB generating mailbox is and if the database is mounted. We should also check to make sure we can update the OAB using PowerShell. So, first, check to see where the mailbox is:

Get-Mailbox -Arbitration | where-Object {$_.PersistedCapabilities -Like "*OabGen*"}  | ft name,servername,database -auto

Which gives us this result:

Then we can check and the database is mounted:

Next we can try to update the files for the OAB. Check the OAB file location to confirm the dates on the files:

The we can run this command (assuming you have one OAB):

get-offlineaddressbook| update-offlineaddressbook

In the case of my client, the files never changed. Even tried restarting the Exchange Mailbox Assistant service without any luck.
Next we renamed the folder containing the OAB files (the folder with a name that looks like a GUID). Once renamed, we tried to update the address book and restart the Mailbox Assistant service. No new files were generated.
The Fix
After some more troubleshooting in the event logs and diagnostic logging, I noticed that the database copy for Database02, where to OAB mailbox resided, had one copy that was in a ‘Failed and Suspended’ state.

We tried to fix this copy and were unable to do so with a reseed or a resume. So we decided to remove the copy and then create a new database copy. After we removed, we tested the OAB to see if this would make a difference.
Fixed
OK. So we now have a fix. With a single, good database copy, we can now generate, update and download the Offline Address Book.
Although Microsoft touts that the address book can now reside in multiple locations (CU 7+) and the generating mailbox can be replicated via a DAG:
“This new infrastructure can also take advantage of the high availability architecture of Exchange 2013. The OAB mailbox can be deployed on a mailbox database that has multiple copies, thereby mitigating a failure mode that occurred in previous releases.”
Ironic in that if the OAB generating mailbox resides in a database that has a failed copy, then it could potentially lead to failed downloads.
Exchange 2016 OAB
After running into this issue, I fired up my Exchange 2016 RTM lab and I was able to reproduce the issue with a good copy and a failed and suspended copy of the database where the OAB generating mailbox is located. So this is not limited to Exchange 2013, but the architecture upon which it is based. I will be updating this article to see if I can find a version where this did not occur. I am thinking either pre-CU7 or Pre-CU5 would be the place to find it.
Further Reading
Exchange 2013 CU 5 OAB Improvements
Exchange 2013 CU 7 OAB Improvements
Offline Address Book Procedures
Original article on Exchange 2013 OAB
Manage OAB in Exchange 2013
——————————————————————————————————
Need to learn PowerShell for Exchange Server 2016 or 2019? Look no further!
On sale now at http://www.PracticalPowerShell.com: