Had a strange case today. On some servers in a DMZ we wanted to upgrade the existing SCOM 2007 R2 agents to SCOM 2012 SP1. However something strange happened during upgrade and somehow it looked like both the old agent and the new agent were installed, but not working. Upon removing of the old agent entry from the control panel we still found the SCOM 2012 agent unable to connect and even to start the service (we had both cases actually on a few machines). We also had one machine where we first removed the old agent followed by a reboot and a clean install of the new agent and still had below problems.
Next course of action is to go to Control Panel and uninstall the SCOM 2012 agent. Only to receive an error like the one below. Failed to load Intercept Library.
Troubleshooting led to all kinds of things 😀 B)
First because I couldn’t remove the agent, I thought lets try to heal it first.
Lets go through a few items:
- The system environment variable Path still contained the path to the 2007 agent folder.
- The state directory of the agent found in the registry was pointing to the 2007 folder at hklmsystemcurrentcontrolsetserviceshealthserviceparameters
- There were a few old entries in the registry pointing to wrong folder names. However looking at these they seemed not to be that important. Cleaned them anyway.
Try to start the SCOM agent.
Results in two errors seen below. Especially the 2147221164 error seemed to be the main one and we got that one on all the machines.
The error 2147221164 can be caused by different things and I have some links at the bottom of this post that have some relation. However in this case it was something different. We were missing a registry entry for a connector.
We ran the following registry file to merge into the registry:
Windows Registry Editor Version 5.00
@=”C:\Program Files\System Center Operations Manager\Agent\MOMConnector.dll”
Thats it :p
Well, at least the SCOM agent service starts after this. However we are immediately greeted in the Operations Manager event log by hundreds of 4507 errors. They talk about not being able to register a class ID in the registry (the agent tries to register some entries in the HKEY_classes_rootCLSID for different types of workflows). And with hundreds I mean hundreds of these errors.
So I guess we get back to uninstalling the agent again, because this can not be good.
This time I went out and got the log file where we uninstall the agent. I see the following entries:
CExecutorHelper::GetPath: FAILED: Executor not registered.. Error Code: 0x80070057.
UninstallAPMMonitoringConfiguration: Failed to load Intercept Scripting Library. Error Code: 0x80004005.
MSI (s) (B8!D8) [14:19:26:606]: Product: System Center 2012 – Operations Manager Agent — Failed to load Intercept Scripting Library
Well, I have looked and could not find anything about this error “Failed to load Intercept Scripting Library” or anything in these lines basically.
What we can see in here is what I expected when I saw this error. Intercept Studio is the old name for the APM piece of the SCOM agent. And we can see it is trying to uninstall that piece (or at least gather all related info).
At this point I decided to call out to my fellow MVP’s and MS Product Team members. Somebody must have an idea 🙄
And sure enough, out of the Product Group there came a suggestion. He actually looked at the first line in the error (Executor not registered). So he suggested to register an APM dll and see if that got it moving.
Alright here goes:
In written form:
- In a (elevated) command prompt on the server go to the directory where the SCOM agent APM component is installed.
C:Program FilesSystem Center Operations ManagerAgentAPMDOTNETAgent
- Next register the DLL
Now try to uninstall the SCOM agent again.
Next we close all open programs and clean out the SCOM agent related directories in “C:Program Files”. I am guessing they were left over after the issues.
Next we could proceed to install the SCOm 2012 SP1 agent and UR3 and run momcertimport.exe to import the already existing certificate. And everytyhing was running again!
A very big thank you to Vlad, Daniele and Vitaly for getting me in touch with Dmitry, who gave the suggestions pointing to the solution!! And of course a very big thank you to Dmitry himself for trying to think of ways how this could happen and what might be done to get it rolling again! B) :>> 💡
Some of the articles I have used in troubleshooting and which I freely interpreted during troubleshooting and trying to solve this:
AT least some of the things mentioned in these linked articles pointed me to somewhat similar situations for smaller pieces of the puzzle and where to look for troubleshooting.
Hope this helps the very few that might run into a similar issue.