This is an article written by our friend Stoyan Chalakov. We found out a few months ago we had been researching the same strange installation issues of SCOM at customers of ours. We discussed that researching and finding the solution took a long while, because you need to find the exact issue keywords. So it is a smart idea to blog about it separately again to make it easier for people to find it. You can run into the same issue while trying to install SCOM 2016 or SCOM 2019 or do an in-place upgrade to one of these products if you have taken action on the security protocols in your network to stop using older ones as you will see below. Now lets quickly get into the blog post!
This article is intended to help all, who are installing the newest version of System Center Operations Manager (SCOM) – 2019 (this applies also to SCOM 2016) and face the same issue, I faced many times during the last couple of months. Before going into details, let me say that there is already a blog post on the topic regarding SCOM 2016, written by Nathan Gau. The reason I am describing this all over again is because I would like to remind you that the problem exists also in SCOM 2019 and also ease the troubleshooting for you by providing some additional log output. This should make it easier for you to find the information in the Internet.
One other consideration would be that with the increased security requirements in the IT sphere, this behavior will be experienced much more often than before.
The problem
While installing SCOM, when you get to point, where you need to specify your service accounts, the following error is displayed, and you are unable to proceed with installation (The option “Next” in the installation wizard is greyed out):
“One or more accounts provided could not be validated. Please provide valid user names and passwords.”
Challenges
- Troubleshooting
Before doing the usual review of the installation logs I made sure that the account names and their passwords are entered properly. Usually I copy the password and paste it in the “Password” field, but this time I even tried entering it manually. One more thing I did is to ensure that the passwords have no special characters, which can somehow fail the verification. As a next step I did what the standard troubleshooting process requires – reviewing the installation logs and in particular r the OpsMgrSetupWizard.log. All the logs are found under %LocalAppData%\SCOM\Log or by default c:\users\<UserName>\Appdata\Local\SCOM\Logs where <UserName> is the name of the user, who does the installation.
In my particular case, the confusion came after reviewing the installation logs. When you try to verify the accounts in the GUI, you get the error and if you open the “OpsMgrSetupWizard.log” the only thing you see is:
[16:41:07]: Always: :Entering Page: AccountsInformationPage
[16:44:29]: Info: :Info:AccountsInformationPage: In OnNextFinalValidationsDoWork to validate account access.
[16:44:29]: Error: :Error:Failed to log in with account DOMAIN\MSA_ACCOUNT
[16:44:29]: Error: :Error:Failed to log in with account DOMAIN\SDK_ACCOUNT
[16:44:29]: Info: :Info:AccountsInformationPage: Async account validation thread returned to UI thread.
The part that helped me find the Nathan’s Blog is logged always around a minute later. So, at the time you are doing troubleshooting this info is not available in the log. I am not sure if it gets added when you close the wizard or just after some time, no matter what you do, but in both cases, this is not helpful. This are the install records that you find a while after getting the error:
[17:20:17]: Error: :Error:Failed to log in with account DOMAIN\SDK_ACCOUNT
[17:20:17]: Info: :Info:AccountsInformationPage: Async account validation thread returned to UI thread.
[17:24:28]: Error: :UserName format is incorrect. It should contain one and only one \as a separator character.
[17:25:43]: Info: :Info:AccountsInformationPage: In OnNextFinalValidationsDoWork to validate account access.
[17:25:43]: Error: :GetCrackNameResult() DS_NAME_RESULT_ITEM crack failed with error = DS_NAME_ERROR_NOT_FOUND
[17:25:43]: Error: :ValidateEssentialsAdministratorAccount() failed to crack NT4 format.
[17:25:43]: Info: :ValidateEssentialsAdministratorAccount() Try to crack account with directory searcher.
[17:25:43]: Error: :Error:Failed to log in with account DOMAIN\MSA_ACCOUNT
[17:25:43]: Error: :GetCrackNameResult() DS_NAME_RESULT_ITEM crack failed with error = DS_NAME_ERROR_NOT_FOUND
[17:25:43]: Error: :ValidateEssentialsAdministratorAccount() failed to crack NT4 format.
[17:25:43]: Info: :ValidateEssentialsAdministratorAccount() Try to crack account with directory searcher.
The problem in my case, the first time I encountered this behavior, is that I open the logs right away after I face the error in the console, so the entries above were still not present.
- Security
The other challenge in this case is the Security aspect. Paying attention to Security related recommendations or implementing Security best practices is nonnegotiable in the modern IT. So, when talking about Active Directory authentication and the Kerberos protocol in particular, all Microsoft recommendations should be followed.
The things is, Microsoft states that RC4 Kerberos encryption is not that secure and even recommends disabling it when it comes to security hardening of domain members:
From KB 4492348
“RC4 encryption is considered less secure than the newer encryption types, AES128-CTS-HMAC-SHA1-96 and AES256-CTS-HMAC-SHA1-96. Security guides such as the Windows 10 Security Technical Implementation Guide provide instructions for improving the security of a computer by configuring it to use only AES128 and/or AES256 encryption (see Kerberos encryption types must be configured to prevent the use of DES and RC4 encryption suites).”
- Information or Warning during the installation process
It is nowhere indicated that the SCOM 2019 uses an older (or unsecure) Kerberos encryption type. The wizard also doesn’t show any Information or Warnings. In high secure environments, this leads to confusion as the wizard (GUI) only shows that that the accounts cannot be verified. I would love to see some kind of notification about this.
Solution
- Workaround
Currently there isn’t a “clean” solution to this problem, only a workaround, but Microsoft is already aware of that and hopefully it will be fixed soon enough. Until then, the way to solve this would be temporary enabling RC4 for Kerberos until the last management server is installed. How you can do that? There are different ways – using a registry key:
Registry Hive: HKEY_LOCAL_MACHINE
Registry Path: \SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System\Kerberos\Parameters\
Value Name: SupportedEncryptionTypes
or a Group Policy setting:
Network security: Configure encryption types allowed for Kerberos
https://docs.microsoft.com/en-us/windows/security/threat-protection/security-policy-settings/network-security-configure-encryption-types-allowed-for-kerberos
There is one more thing you need to be aware of. Depending on configuration of your environment, you may have to set this policy at the domain level to apply to the OU, where your SCOM server are located or, you may have to set this policy at the organizational unit (OU) of the domain controllers in your domain.
- Solution
The SCOM product group is aware of the behavior and the impact it could have on the installation process. The reason they are aware of that is that approximately 1 year ago this has been reported on the SCOM User Voice page, which is being actively monitored by Microsoft and contains all types of feedback (ideas, bug fix requests, feature requests), related to SCOM:
SCOM Installer Failure with RC4 Protocol Disabled
https://systemcenterom.uservoice.com/forums/293064-general-operations-manager-feedback/suggestions/37258615-scom-installer-failure-with-rc4-protocol-disabled?tracking_code=25ee37b0f7e7ca2a79a440993a8b865e
The great thing about this particular request is that its status has recently (on the 10.03.2020) been changed to “Under Review”, so we will expect this to reviewed by the SCOM Product Group and hopefully fixed as soon as possible. If you want to speed up the process, I suggest you go to the SCOM User Voice and upvote this request.
Conclusion
A few important takeaways:
- First things first – keep your environment secure and disable RC4 for Kerberos
- If you encounter this and need to enable RC4 encryption in your environment, do this only temporary, during the setup process of your servers and disable it afterwards.
- If you want to help shaping the product (Operations Manager) or speed up the implementation of a specific request, please post it on the SCOM User Voice site and/or upvote the existing requests.
- Make sure your do a thorough research on the Internet, using the related information from the installation logs.
References
SCOM Installer Failure with RC4 Protocol Disabled
https://nathangau.wordpress.com/2018/06/22/scom-installer-failure-with-rc4-protocol-disabled/
SCOM Installer Failure with RC4 Protocol Disabled
https://systemcenterom.uservoice.com/forums/293064-general-operations-manager-feedback/suggestions/37258615-scom-installer-failure-with-rc4-protocol-disabled?tracking_code=25ee37b0f7e7ca2a79a440993a8b865e
AD DS: Security: Kerberos “Unsupported etype” error when accessing a resource in a trusted domain
https://support.microsoft.com/hu-hu/help/4492348/kerberos-unsupported-etype-error-when-authenticating-across-trust
See also
Tough Questions Answered: Can I disable RC4 Etype for Kerberos on Windows 10?
https://techcommunity.microsoft.com/t5/itops-talk-blog/tough-questions-answered-can-i-disable-rc4-etype-for-kerberos-on/ba-p/382718
Note from Bob:
At one of my customers we were at we at the minimum needed to enable RC4 for the database servers and the management servers involved to get this to work. I do not know what the settings on domain controllers was. Because the encryption settings were set through a GPO we only needed to manually change it in registry and do our actions quickly before the GPO gave it an override back again. In the registry key noted above we put decimal value 2147483644 on the management server we were installing or upgrading and the database servers involved (so OperationsManager and OperationsManagerDW database hosting servers). If you change anything in registry, make sure you remember the previous value.
Happy monitoring!