FIM CM / MIM CM Certificate Management service account certificate renewals

references:

https://docs.microsoft.com/en-us/previous-versions/mim/hh149034(v=ws.10)

https://blogs.technet.microsoft.com/iamsupport/2016/08/03/support-tip-fim-cm-2010-mim-cm-2016-admin-key-diversification-and-certificate-renewal/

Internally FIM/MIM Certificate management has 5 service accounts.  3 of these accounts have certificates stored within their personal certificate store on your application server.  Each certificate uses a unique template that was created during the installation of the application.  As with all certificates, they do eventual expired (based on the settings in your template).

The 3 accounts that have certificates are the Key Recovery Agent, the Enroll Agent, and the CLM agent accounts.  If you are unsure of what accounts are which, go to this folder \Program Files\Microsoft Forefront Identity Manager\2010\Certificate Management\web on your CM server and open the web.config file.  Look for the section that is labelled "CLM users" and find the entries with CLM.RecoveryAgent.Username, Clm.EnrollAgent.Username, and CLM.Agent.Username.  Keep this file open as we need to make changes to it later.

Once you have the accounts identified, ensure you have the correct password for the account.  You can test them using ldp.exe.  If you don't have the password, first go through the password reset process.

With each of the accounts, you will need to open MMC doing a runas.  Open one for each of the three accounts.  Add the certificates snapin with the Current User option.  Expand this, expand personal, and click on certificates.  Unless you have gone through several certificates already, there should only be one in there.

Identify the key that you want to replace, and do an export of each one.  Select PCKS#12 format with "include all certificates in the certificate path if possible" and "export all extended properties" options.  Set a password and export to a file.  This will give you a backup of the key just in case you need it again.

If you read the second article linked above, you will see that the CLMAgent key needs to be renewed with the same key, otherwise it will break previously issued smartcards.  So you can do a renewal of the existing certificate by right clicking the certificate -> all tasks -> advanced operations -> renew this certificate with the same key.  Click next/enroll/finish.  You can do this for each of the 3 certificates.  Once you have the new certificate (you will see an updated expiration date), open the certificates, go to the details tab, find the thumbprint value and make a copy of each new certificate's thumbprint.  

Note: when copying the thumbprint value, you will end up with some invisible unicode character at the beginning of the string.  Paste the thumbprint to notepad, go to the start of the string and hit Del once.  This should get rid of it.  Remove all spaces between the hex values.  To validate that the special char has been removed, copy and paste the whole string into a command prompt and look for any box shaped character.  If there are none, then the string is properly cleaned up.

Once you have all certificates renewed, and your thumbprints gathered, go to the web.config file for the CM application.

Look for Clm.SigningCertificate.Hash.  Replace the current value with the new thumbnail of the ClmAgent certificate

Look for Clm.ValidSigningCertificate.Hashes.  Add the new thumbnail of the ClmAgent certificate to this as a comma seperated list.

Look for Clm.SmartCard.ExchangeCertificate.Hash.  Replace this with the ClmAgent certificate hash.

Search for Clm.EnrollAgent.Certificate.Hash.  Replace this with the EnrollAgent certificate hash.

Go to your certificate authority server.  Open the certificate authority utility, rightclick the CA name, open properties.  Look for the policy module tab, click properties.  Go to the signature certificates tab.  Add a new hash and enter the ClmAgent thumbnail here.  Restart certificate services.

On your CM server, run IISRESET.

If you use recovery agent's, follow the additional steps mentioned in the first link above.

Service Account password resets for FIM CM / MIM CM service accounts

Microsoft's identity manager - Certificate management product has several different service accounts associated with its internal functions as well as an IIS application pool account.  For best practices, it is always good to periodically change service account passwords.  For this product, the account passwords are not configured on windows services, or other easily identifiable locations, so automated password management tools won't be very helpful.

To start with, you want to identify all of your service accounts what roles they perform.  If you are unsure, logon to your CM server and open up the:  \Program Files\Microsoft Forefront Identity Manager\2010\Certificate Management\web folder.  Open the web.config file and look for the section labelled "CLM USERS".  Under this you will find keys for usernames for each component.

Open up a command prompt and go to the CM folder path, and BIN subfolder.  In this folder, there is a tool called clmutil, which will be used to enter the new account passwords.  The account name values in this tool don't perfectly line up with what is in the webconfig.  They are

WebConfig      ->    clmutil
AuthzAgent            authAgent
Agent                      agent
CAManager            caMngr
RecoveryAgent       krAgent
EnrollAgent            enrollagent

Start by going to Active Directory and creating a new strong password for each account.  Make note of the passwords for each username, and ensure you match up your usernames to the roles above.

For each of the accounts, run the clmutil, ex:

clmutil -setacctpwd authAgent  "mynewPassword"

Once you have entered the new account passwords matching each of the service account's roles, open up IIS administration.  Look in the application pools for clmAppPool.  Check the identity of the pool.  For the service account associated with that, do a password reset in Active Directory.  Open the advanced settings for the application pool, click the ... button on the identity value.  Click the set button.  Enter the username and the new password, then click ok, ok, ok.

Now that you have reset all of the passwords, run an iisreset on the server.  Ensure everything is working after that.  If you have additional FIM/MIM CM server nodes, you will need to enter the passwords on each one.

How to pull a full list of users to certificates and card mappings from FIM CM / MIM CM

If you want to collect a report that combines usernames to certificate serial numbers, linked to the card serial number, along with the date the card was issued, you can use this query on your FIM CM database:

Select
u.unc_user_nt4_name, c.cert_issued_serial_number, s.sc_serial_number,
s.sc_manufacturer_id, q.req_submitted_dt,
replace(replace(replace(replace(replace(s.sc_status,'1','Assigned'), '2', 'Active'), '3', 'Disabled'), '4', 'Suspended'), '5', 'Retired') as Status

from dbo.Certificates as C left join dbo.ProfileCertificates  as p
on c.cert_id = p.pc_cert_id 

right outer join dbo.Profiles as r
on p.pc_profile_uuid = r.profile_uuid

 inner join dbo.UserNameCache as u on r.pr_assigned_user_uuid = u.unc_user_uuid

right outer join dbo.Smartcards as s on
r.pr_sc_uuid = s.sc_uuid

right outer join (select * from dbo.Requests where req_type=1) as q
on q.req_sc_uuid = s.sc_uuid

order by unc_user_nt4_name,sc_serial_number,req_submitted_dt

FIM or MIM certificate manager client tracing (Cmclient)

I have found that most of the posts you can find regarding setting up trace logging on the CM Client side don't really work.  These are the steps that MS provided to me in a recent case.

1) Open regedit
   a) create key:  [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\CLM\adk]
   b) create key: [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\CLM\adk\TraceError]
      i)  Create Reg_DWORD with name "*", and value 0x1
      ii)  create REG_DWORD with name "C:\Program Files (x86)\Internet Explorer\IEXPLORE.EXE", with value 0x1
     iii) create REG_DWORD with name "c:\Program Files (x86)\Microsoft Forefront Identity Manager\2010\CM Client\bin\clmProfileUpdate.exe", with value 0x1

   b) create key [HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Microsoft\CLM\adk\TraceError\Folder]
    i) create REG_SZ with name "C:\Program Files (x86)\Internet Explorer\iexplore.exe" with value "C:\cmtraces"   (or wherever you want the log file to be created)
    ii)  create REG_SZ with name "C:\Program Files (x86)\Microsoft Forefront Identity Manager\2010\CM Client\bin\clmProfileUpdate.exe"  with value "C:\cmtraces"

Grant permissions to everyone on that file location that you are logging to.

Search the registry under HKCU for "TraceError".  If that exists under a folder structure that has a CLM key folder above it, delete the whole CLM key.


The above registry keys look like its very x86 specific, so similar keys may be needed for x64 client versions or IE.

FIM/MIM certificate manager, can't enroll due to wrong number of attributes

Certificate manager can be picky about x86 and x64 versions of Internet Explorer and the CM client software.  In the past, I've had cases where the x64 client needed to be installed and x64 IE used.  Other times both needed to be x86.  Recently I hit this "wrong number of arguments" error while issuing a card in a newly built environment.  The issue was resolved by running the x64 bit version of IE with the x86 version of the client installed.  For newer windows OS, the link to IE should default to the x64, but older ones may point to x86.  In the two "program files" directories, you can get to the correct .exe version if you really want to make sure you have the right one.


Trace:


"2019-07-07 18:48:53.85 -07" "Microsoft.Clm.BusinessLayer.SmartCard.SmartCard" "Void EnrollFailed(Microsoft.Clm.Common.Requests.Request, System.String, Boolean, System.String)" "CONTOSO\CertAdminGuy" "CONTOSO\CertAdminGuy" 0x000004D8 0x00000004

Enroll sequence failed. Smart Card UUID: c1e07453-4610-46cf-b9f2-4e0b819eda80 OnIntall: False Error: Wrong number of arguments or invalid property assignment

"2019-07-07 18:48:53.95 -07" "Microsoft.Clm.Web.Modules.BaseCspUserControl" "Void ProcessClientMessage(Microsoft.Clm.BusinessLayer.SmartCard.BaseCsp.Serialization.bcspClientMsg)" "CONTOSO\CertAdminGuy" "CONTOSO\CertAdminGuy" 0x000004D8 0x00000004


1) Exception Information

*********************************************
Exception Type: Microsoft.Clm.BusinessLayer.SmartCard.BaseCsp.ClientException
Number: -2146827838
Message: Wrong number of arguments or invalid property assignment
Data: System.Collections.ListDictionaryInternal
TargetSite: Microsoft.Clm.BusinessLayer.SmartCard.BaseCsp.ServerMessage Fail()
HelpLink: NULL
Source: Microsoft.Clm.BusinessLayer



StackTrace Information

*********************************************
   at Microsoft.Clm.BusinessLayer.SmartCard.BaseCsp.EnrollProtocol.Fail()
   at Microsoft.Clm.BusinessLayer.SmartCard.BaseCsp.Protocol.HandleError()
   at Microsoft.Clm.BusinessLayer.SmartCard.BaseCsp.Protocol.ProcessClientMessage()
   at Microsoft.Clm.BusinessLayer.SmartCard.BaseCsp.Protocol.ProcessClientMessage(Guid requestUuid, bcspClientMsg clientMsg)
   at Microsoft.Clm.Web.Modules.BaseCspUserControl.ProcessClientMessage(bcspClientMsg msg)

Upgrade FIM 2010R2 Certificate Manager to MIM 2016 SP1

I recently ran this scenario through a lab, where I had a Windows 2008R2 server with a FIM 2010R2 instance at a pretty low patch level, along with a cm client installed on the same machine.  Going through the upgrade, I followed roughly these steps:

1) Upgrade of the FIM 2010R2 server components to the last released patch level (4.1.3766.0).  This step may not have been necessary, but I did it just in case its relevant.

2) Install .net 4 framework on the CM server and CA if they don't have it already.  It is a prerequisite for MIM 2016 SP1

3) Install of MIM 2016 SP1 CM server component over the top of the existing installation.  The component list of the installer defaults to try to install the CA module component.  You can uncheck that.  There will be a notice about reusing the database and warning regarding that.  The install itself went smoothly.  The only issue on the CM server was that the IIS application pool's identity was changed from my service account to LocalSystem.  So this will need to be manually fixed.  After install of the CM web server components, run the same installer on the certificate authority that you point to.  Again, this was smooth.  You will need to unselect the installer components that you don't want.  There's a checkbox option regarding keeping your old settings.  Restart the cert services when done.  For both of these components, I don't believe a reboot was required (all card operations still worked), however if you run some other installer afterwards it may tell you that a reboot is needed.

4) At this point I did an in-place OS upgrade to 2012R2 on the CM server.  Again, no issues here.  All the components for the server and the cm client were fine after the upgrade.  The only issue was IE security settings for the trusted zone were changed and I had to do some fixes there.

5) Patch both the CM server and CA server to the latest patch level.  In my case I took it directly to 4.5.412.0.  Again, no reboots we required.  Everything continued working fine for card operations that I tested throughout the whole process.

Also note: I did not upgrade the CM client during any of the stages above, while I was still able to perform card enrollment/renewal and such on my physical cards at each step.  For the CM client upgrade, you can't directly upgrade from FIM2010 versions to MIM2016 versions.  You need to uninstall the old client and then put on the 2016 SP1 version.  Surprisingly with these two operations, neither of them required a reboot.  I tried taking this directly to the same MIM patch version of the server, but ran into DLL dependency errors.  So I went to v4.4.1749.0 instead (reboot required).  The few MIM patches released since then have various .net framework version and visual c++ install dependencies, so I'm assuming the client might be impacted by these dependencies, while the server component installers and patches did not require them.

Unattended installation of FIM CM client

I was going through the unattended install guide for FIM components at technet.  Since they put them all together, but don't clearly separate all of the options, it makes it challenging to find the correct option for specifying servers in the FIM CM client's dialog box which requests you to provide the list of FIM component servers that you connect to.  After playing with a few options, I found SITELOCK_DOMAIN is the correct choice.  You can install with this:

msiexec /i "CM Client.msi" /q ADDLOCAL=CMClient,ChangePin,AppletManagement,SelfServiceControl,ProfileUpdateControl SITELOCK_DOMAIN=fimcmportal.contoso.com

If you have more than one site, seperate them by semi-colon and quote it:

msiexec /i "CM Client.msi" /q ADDLOCAL=CMClient,ChangePin,AppletManagement,SelfServiceControl,ProfileUpdateControl SITELOCK_DOMAIN="fimcmportal.contoso.com;cmportal.contoso.com;fimportal.contoso.com"

FIM CM Portal problems

Lately I have been working a lot with the FIM CM portal in support of end users trying to perform self service operations in mixed environment of OS's and versions of IE.  Below are some problems seen, and some suggested workarounds that may help others with the same issues.  From experience, it looks like the portal has problems caused by ActiveX security settings, IE compatibility mode required as well as FIM CM client architecture support issues.

Some ways to get around problems with FIM CM portal:

1) CM portal site is in trusted sites, yet user is getting repeat prompts for logon to the page.  The OS security logs on the portal server show success, yet IIS is not accepting it and ends up at access denied.

Solution to try:  Internet Explorer options -> Security tab ->  Check "Enable Protected Mode", and set security levels to Low.  Restart IE


2)  User is able to get into FIM CM portal, but whenever they click on an operation, nothing happens.  Problem with javascript in the links

Solutions to try:
a) Set compatibility mode for the site.  In newer versions of IE, you can find this in the tools menu
b) Internet Explorer options -> Security tab -> Set security level to Low.  Refresh the page


3) BaseCSP error on smart card operations

Solutions to try:
a)  For XP machines, ensure the BaseCSP hotfix is installed (KB909520)
b)  Ensure the FIM CM Client is installed (displays as "Forefront Identity Manager CM Client" in add/remove programs)
c)  Internet Explorer options -> Security tab -> Set security level to Low.  Refresh the page
d)  If the client machine is x64 bit OS
    1) Check the version of the installed FIM client (what "program files" folder is it under, x86 or the main one).  Try to run the IE version that matches the version of the FIM client
    2)  (IE11) Internet options -> Advanced tab -> Check "Enable 64-bit processes for enhanced protected mode" or "Enabled Enhanced Protected Mode" if you don't have the first option.  Do this if you have the 64 bit FIM CM Client and are running IE 64bit, yet it still fails.
    3)  If you have the 64 bit client installed, and both versions of IE fail, and step #2 isn't available, remove the 64 bit client and install the 32bit client
e)  Ensure ActiveX filtering is off.  This may show up in the address bar as an icon saying components are filtered.  Or you can look in the tools menu to see if it is checked (not all versions of IE have this)
e)  Repair or reinstall the FIM CM client


4) Slow/hanging operations.  Look at 64bit/32bit IE as mentioned in BaseCSP problems.  You may want to try the other version of IE.

FIM Certificate manager portal "Value does not fall within the expected range"

Recently I ran into an unusual issue with a FIM certificate manager portal installation occasionally throwing the "Value does not fall within the expected range" error when doing searches.  Some searches would work all of the time, while others would fail all of the time.


The error:

1) Exception Information
*********************************************
Exception Type: System.ArgumentException
Message: Value does not fall within the expected range.
ParamName: NULL
Data: System.Collections.ListDictionaryInternal
TargetSite: Int32 SecurityDescriptorToBinarySD(Microsoft.Clm.Security.Structs.VariantIDispatch, IntPtr ByRef, UInt32 ByRef, System.String, System.String, System.String, UInt32)
HelpLink: NULL
Source: Microsoft.Clm.Security.Authorization

StackTrace Information
*********************************************
   at Microsoft.Clm.Security.NativeMethods.SecurityDescriptorToBinarySD(VariantIDispatch vVarSecDes, IntPtr& ppSecurityDescriptor, UInt32& pdwSDLength, String pszServerName, String userName, String passWord, UInt32 dwFlags)
   at Microsoft.Clm.Security.Authorization.SecurityDescriptor.ConvertToByteArray(DirectoryEntry entry)



After digging through various components, checking AD, etc, a pattern seemed to emerge.  Whenever the search result should have returned results of users that were in a specific OU, it would fail.  While results that gave only results in other OU's would work.  On checking the metadata for the OU, the ntSecurityDescriptor had recently changed right around the time that the FIM CM portal started throwing errors.  A large number of property management ACE's had been added, which pushed the size of the ACL too high for the system to deal with.  According to Microsoft, the max size for an ACL is 64k.  My previous post shows how easy it can be to hit that limit when you get too fine grained in your entries.  Removing the added ACE's resolved the issue.