Recently Microsoft has made some changes where you are required to run a few extra commands prior to upgrading your Exchange 2016 servers to the latest Cumulative Update (CU) 14. The change relates to permissions needed for Exchange and the first change was made in CU12 for Exchange 2016 – KB4490059. A further change was introduced in Exchange 2016 CU13 – June 2019 Quarterly Updates article.. In the article, the wording is a bit unclear as to what steps we need to take (see bolded words):
In order to apply these changes, a directory admin will need to run the cumulative update setup program we are releasing today with the /PrepareAD parameter. When multiple Exchange versions co-exist in a single Active Directory forest, the cumulative update matching the latest version of Exchange deployed should be used. Setup will automatically run /PrepareDomain in the domain where /PrepareAD is executed. Environments with multiple domains in the forest will need to run the cumulative update setup program using the /PrepareDomain parameter in all domains in the forest. These steps will update the rights granted to Exchange Servers in the Active Directory to meet the new permissions scope. More information on /PrepareAD and /PrepareDomain is available at this link.
Having just performed two of these updates and reading comments from others that have experiences this issue, let me clarify the experience and what you should do to avoid any of these missteps (and others):
(1) Run ‘Setup.exe /PrepareAD /IAcceptExchangeServerLicenseTerms’ on a server with at least .NET 4.7.2 (Exchange CU13+) and in the SAME site as the Domain Controller that has the Schema Master role.
(2) Run ‘Setup.exe’ from the Exchange version you are upgrading to (CU14 is the latest at the moment).
(3) Run the ‘Setup.exe /PrepareDomain’ process as well as this seems to smooth out some issues.
(4) If you are upgrading an Exchange server in a remote site (and not in the same site as the Schema Master) first for testing, wait for Active Directory replication to occur BEFORE running setup.
Those are some general guidelines that should help out with this change.
Hybrid Considerations
In addition to the above, there are some additional notes I would like to make and they have to do with two different scenarios where we have Exchange in coexistence or hybrid with Exchange Online. One scenario is that we have a mixed environment of Exchange 2010/3 and Exchange 2016 and we are upgrading Exchange 2016 to the latest CU. The second scenario is when we have Exchange 2010/3 and we are adding a new Exchange 2016 server (CU14 for this example).
Second Scenario- Upgrading Exchange to latest CU
(1) Enter maintenance mode for the Exchange server to be upgraded.
(2) If there is a need to upgrade .NET – clear the cache (NEED ARTICLE) or reboot. A reboot can also solve the problem where the server may have a reboot pending and the administrator did not know this. Why reboot? Why clear the cache? .NET can take an abnormally long time to install (hours) if this is not done and if your downtime window isn’t big enough, you could be put in a tough spot.
(3) Run – ‘Setup /PrepareAD /IAcceptExchangeServerLicenseTerms’
(4) Wait for AD Replication
(5) Install Exchange 2016 CU update to the existing server.
(6) Reboot upgraded server
(7) Exit maintenance mode.
Second Scenario – Adding first Exchange 2016 server
What we find is that not only do we need to run the typical /PrepareAD and /PrepareDomain switches for the Setup prep work, but we also need to specify a tenant switch that is used to specify the tenant config. The issue lies in the fact that if you run the setup, and there is no mention of this in CU14 release notes, a strange error occurs.
Exchange Setup Log Errors:
“[10/02/2019 20:06:53.0776] [0] Starting hybrid configuration checks…
[10/02/2019 20:06:53.0792] [0] Running the Office 365 tenant hybrid test.
[10/02/2019 20:06:53.0792] [0] The hybrid detection engine will try to connect to Office 365 using the provided tenant credentials to run the Get-OrganizationConfig cmdlet.
[10/02/2019 20:06:53.0823] [0] Opening runspace to https://ps.outlook.com/powershell-liveid?serializationLevel=Full.
[10/02/2019 20:06:55.0995] [0] Session=Tenant Cmdlet=Get-OrganizationConfig START
[10/02/2019 20:07:07.0339] [0] Session=Tenant Cmdlet=Get-OrganizationConfig FINISH Time=11343.7764ms
[10/02/2019 20:07:07.0495] [0] Session=Tenant Total Cmdlet Time=11.3437764s
[10/02/2019 20:07:07.0511] [0] Microsoft.Exchange.Management.Deployment.HybridConfigurationDetection.HybridConfigurationDetectionException: Deserialization fails due to one SerializationException: System.Runtime.Serialization.SerializationException: Unable to find assembly ‘Microsoft.M365.Core.Utility, Version=18.0.0.0, Culture=neutral, PublicKeyToken=5a24b4a52c5686bd’.
at System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo.GetAssembly()
at System.Runtime.Serialization.Formatters.Binary.ObjectReader.GetType(BinaryAssemblyInfo assemblyInfo, String name)
at System.Runtime.Serialization.Formatters.Binary.ObjectMap..ctor(String objectName, String[] memberNames, BinaryTypeEnum[] binaryTypeEnumA, Object[] typeInformationA, Int32[] memberAssemIds, ObjectReader objectReader, Int32 objectId, BinaryAssemblyInfo assemblyInfo, SizedArray assemIdToAssemblyTable)
at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWithMapTyped(BinaryObjectWithMapTyped record)
at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run()
at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(HeaderHandler handler, __BinaryParser serParser, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
at Microsoft.Exchange.Data.SerializationTypeConverter.DeserializeObject(Object sourceValue, Type destinationType) —> System.InvalidCastException: Deserialization fails due to one SerializationException: System.Runtime.Serialization.SerializationException: Unable to find assembly ‘Microsoft.M365.Core.Utility, Version=18.0.0.0, Culture=neutral, PublicKeyToken=5a24b4a52c5686bd’.
at System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo.GetAssembly()
at System.Runtime.Serialization.Formatters.Binary.ObjectReader.GetType(BinaryAssemblyInfo assemblyInfo, String name)
at System.Runtime.Serialization.Formatters.Binary.ObjectMap..ctor(String objectName, String[] memberNames, BinaryTypeEnum[] binaryTypeEnumA, Object[] typeInformationA, Int32[] memberAssemIds, ObjectReader objectReader, Int32 objectId, BinaryAssemblyInfo assemblyInfo, SizedArray assemIdToAssemblyTable)
at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWithMapTyped(BinaryObjectWithMapTyped record)
at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run()
at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(HeaderHandler handler, __BinaryParser serParser, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
at Microsoft.Exchange.Data.SerializationTypeConverter.DeserializeObject(Object sourceValue, Type destinationType) —> System.Runtime.Serialization.SerializationException: Unable to find assembly ‘Microsoft.M365.Core.Utility, Version=18.0.0.0, Culture=neutral, PublicKeyToken=5a24b4a52c5686bd’.
at System.Runtime.Serialization.Formatters.Binary.BinaryAssemblyInfo.GetAssembly()
at System.Runtime.Serialization.Formatters.Binary.ObjectReader.GetType(BinaryAssemblyInfo assemblyInfo, String name)
at System.Runtime.Serialization.Formatters.Binary.ObjectMap..ctor(String objectName, String[] memberNames, BinaryTypeEnum[] binaryTypeEnumA, Object[] typeInformationA, Int32[] memberAssemIds, ObjectReader objectReader, Int32 objectId, BinaryAssemblyInfo assemblyInfo, SizedArray assemIdToAssemblyTable)
at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.ReadObjectWithMapTyped(BinaryObjectWithMapTyped record)
at System.Runtime.Serialization.Formatters.Binary.__BinaryParser.Run()
at System.Runtime.Serialization.Formatters.Binary.ObjectReader.Deserialize(HeaderHandler handler, __BinaryParser serParser, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Deserialize(Stream serializationStream, HeaderHandler handler, Boolean fCheck, Boolean isCrossAppDomain, IMethodCallMessage methodCallMessage)
at Microsoft.Exchange.Data.SerializationTypeConverter.DeserializeObject(Object sourceValue, Type destinationType)
— End of inner exception stack trace —
at Microsoft.Exchange.Data.SerializationTypeConverter.DeserializeObject(Object sourceValue, Type destinationType)
at Microsoft.Exchange.Configuration.MonadDataProvider.MonadCommand.Deserialize(PSObject psObject)
at Microsoft.Exchange.Management.Deployment.HybridConfigurationDetection.RemotePowershellSession.RunOneCommand[T](String command, Dictionary`2 parameters, Boolean ignoreNotFoundErrors)
at Microsoft.Exchange.Management.Deployment.HybridConfigurationDetection.TenantHybridDetectionCmdlet.GetOrganizationConfig()
at Microsoft.Exchange.Management.Deployment.HybridConfigurationDetection.HybridConfigurationDetection.RunTenantHybridTest(PSCredential psCredential, String organizationConfigHash)
— End of inner exception stack trace —
at Microsoft.Exchange.Management.Deployment.HybridConfigurationDetection.HybridConfigurationDetection.RunTenantHybridTest(PSCredential psCredential, String organizationConfigHash)
at Microsoft.Exchange.Setup.GUI.HybridConfigurationStatusPage.StartHybridTest()
[10/02/2019 20:07:07.0511] [0] Finished loading screen HybridConfigurationStatusPage.
[10/02/2019 20:07:09.0854] [0] Finished loading screen HybridConfigurationCredentialPage.”
What this means is that we need to run the /prepareAD switch (because of the permission changes). When we do this we get an error that we need to run the ‘/TenantOrganizationConfig’ switch. We can find information on that process here:
Reference article – Error when you run Setup /PrepareSchema to prepare the schema for an existing Exchange hybrid environment
So we need to perform the following steps:
(1) Connect to Exchange Online with PowerShell
(2) Export the config – Get-OrganizationConfig | Export-Clixml -Path MyTenantOrganizationConfig.XML
(3) Run this on the server you are installing Exchange 2016 on ‘Setup.exe /PrepareAD /IAcceptExchangeServerLicenseTerms /TenantOrganizationConfig MyTenantOrganizationConfig.XML’
(4) Then run ‘Setup.exe /PrepareDomain’
(5) Retry Exchange 2016 CU14 install
Now Exchange should install without issue.
After the install, reboot and exit server maintenance mode for Exchange.
Summary
With or without hybrid, we need to at least run ‘Setup.exe /PrepareAD /IAcceptExchangeServerLicenseTerms’ when adding or upgrading an Exchange 2016 to your Exchange environment. This needs to be done prior to running the GUI for the Exchange installation.
Further Reading
* 250 Hello – Exchange 2016 CU13
A blog about PowerShell! … and Office 365
