Troubleshooting the Sharepoint 2010 User Profile Service Application

The Sharepoint 2010 User Profile Service (UPS) application allows the Sharepoint administrator great flexibility and is a "must have" feature if you are taking your Sharepoint to the next level.

Unfortunately UPS has a dark side that too many administrators have to face at one time or another, more often than not it is related to the "Forefront Identity Management Service" and the "Forefront Identity Manager Synchronization Service" service.



Some Tips

• Upgrade to Sharepoint 2010 SP1 and the August 2011 CU at before attempting to resolve problems. Both of these updates resolve a number of issues that might impact the User Profile Service application.
• Don't ever try to change the "Forefront Identity Manager Service" or "Forefront Identity Manager Synchronization Service" settings manually from the Services MMC snap-in. It simply doesn't work as SP needs to do a great deal of configuration.
• Sharepoint can be super slow, sometimes you need to wait 10-15 minutes for things to happen, so when you click start and nothing happens, wait 15 minutes then check again.
• If you are re-creating the User Profile Service application, it is a good idea to use different names for the databases, the service application itself and the application pool. This will ensure there are no conflicts with any old settings that may be floating around in your Sharepoint configuration or registry.
• Be IISRESET "happy". It is a good idea to perform IISRESET's after major parts of the setup process. I normally follow a pattern such as: Start/Create the service, wait 10 minutes, IISRESET, next step. This will ensure all of Sharepoint is "on the same page" before moving forward to the next step of the process.

Possible Issues and Resolutions

Problem:
After starting the "User Profile Synchronization Service" from Central Administration only the "Forefront Identity Manager Synchronization Service" starts, or both services fail to start.

Resolution:
If you don't have too much already setup in the UPS or it is your first time setting it up, it can be much easier to delete the UPS, stop the Synchronization services and then recreate it, than mess around. See my rebuild process below.


Problem:
"Forefront Identity Manager" source logs an Event ID 3 in event logs.

.Net SqlClient Data Provider: System.Data.SqlClient.SqlException: HostId is not registered

Resolution:
Most of the time simply restarting the "Forefront Identity Manage Service" and "Forefront Identity Manager Synchronization Service" from the services MMC snap-in will resolve this issue. If either of them is in a Disabled state or the problem persists after a service restart, then I recommend rebuilding the UPS from scratch as per my instructions below.


Problem:
"ILM Web Service Configuration" source logs an Event ID 234 in event logs.

ILM Certificate could not be created: Cert step 2 could not be created: C:\Program Files\Microsoft Office Servers\14.0\Tools\MakeCert.exe -pe -sr LocalMachine -ss My -a sha1 -n CN="ForefrontIdentityManager" -sky exchange -pe -in "ForefrontIdentityManager" -ir localmachine -is root

Resolution:
If you have tried to provision the "User Profile Synchronization Service" a number of times you might see this error. It occurs because there are multiple "ForefrontIdentityManager" certificates stored in the Certificate store.

Firstly you need to stop the "User Profile Synchronization Service" under Sharepoint Central Administration > System Settings > Manage Services on Server.

Then open an MMC console, add a Certificates snap-in, select Computer Account. Check the Personal, Trusted Root Certification Authorities and Trusted People stores for duplicate "ForefrontIdentityManager" certificates and delete ALL the FIM certificates.

Next under Sharepoint Central Administration > System Settings > Manage Services on Server, Start the "User Profile Synchronization Service" again, you will be prompted for the password of the Sharepoint service account it is using. It should successfully restart and create a new certificate without conflicts.


Problem:
"Microsoft Resource Management Service" source logs a Event ID 0 in event logs.

Service cannot be started. System.InvalidOperationException: Cannot find the X.509 certificate using the following search criteria: StoreName 'My', StoreLocation 'LocalMachine'

Resolution:
The "User Profile Synchronization Service" can't find a certificate when it is starting. You need to stop and then start this service. Firstly you need to stop the "User Profile Synchronization Service" under Sharepoint Central Administration > System Settings > Manage Services on Server. After waiting 5 minutes, perform an IISRESET and then press Start to restart the service. You will be prompted for the password of the Sharepoint service account it using.


Problem:
"Microsoft.ResourceManagement.ServiceHealthSource" source logs an Event ID 2 in event logs.

The Forefront Identity Manager Service could not bind to its endpoints.  This failure prevents clients from communicating with the Web services.

Resolution:
You can try restarting the "Forefront Identity Manage Service" and "Forefront Identity Manager Synchronization Service" from the services MMC snap-in. If this does not work then I recommend rebuilding the UPS from scratch as per my instructions below.


Problem:
"Forefront Identity Manager" source logs an Event ID 3 in event logs.

.Net SqlClient Data Provider: System.Data.SqlClient.SqlException: Cannot open database "DBNAME" requested by the login. The login failed.
Login failed for user 'DOMAIN\service-spsql'

Resolution:
This issue occurs if you have recreated the User Profile Service application and the database settings have not updated in the registry. It is normally a smart idea to stop the "User Profile Synchronization Service" under Sharepoint Central Administration > System Settings > Manage Services on Server, wait 5 minutes, then restart it.

If it is simply a database name wrong you can edit it in the registry under the following paths.

HKLM\system\currentcontrolset\services\FIMService

HKLM\system\currentcontrolset001\services\FIMService

HKLM\system\currentcontrolset002\services\FIMService

HKLM\system\currentcontrolset\services\FIMSynchronizationService

HKLM\system\currentcontrolset001\services\FIMSynchronizationService

HKLM\system\currentcontrolset002\services\FIMSynchronizationService

The main two values you will want to look at are "DatabaseName" and "DatabaseServer", ensure those two are correct then restart the "Forefront Identity Manage Service" and "Forefront Identity Manager Synchronization Service" from the services MMC snap-in.

If this doesn't work, then stopping the "User Profile Synchronization Service"and then restarting it from Central Administrator is your best solution.


Problem:
 "User Profile Service" source logs an Event ID 1511 in event logs. The "event user" will be one of your Sharepoint service accounts.

Windows cannot find the local profile and is logging you on with a temporary profile. Changes you make to this profile will be lost when you log off.

Resolution:
This is a fairly common problem and one of the easier ones to fix. Firstly go into your IIS management console > Application Pools and search for any application pools that have the same username as the event log user. Stop all of those application pools and then run an IISRESET.

Then from the command line run the commands. The first command adds the problem user account to the local administrator group (to aide the profile creation) and the second command creates the user profile.

net localgroup administrators DOMAIN\AppPoolAccount /add
runas /u:DOMAIN\AppPoolAccount /profile cmd

When this process is complete remove the user from the local administrators group.

net localgroup administrators DOMAIN\AppPoolAccount /delete

Then you will need to restart all the IIS Application Pools you previously stopped.


Problem:
After adding a "Synchronization Connector" to the UPS you can no longer get into "Manage User Profiles". When clicking "Manage User Profiles" the web browser simply times out with no errors in the event log or ULS logs.

Resolution:
Unfortunately I am still struggling with this one and have no resolution. On the other hand it seems to make no difference, unless you want to map custom properties, which you can do manually through the Forefront Identity Manager console from the Sharepoint server desktop. While it is annoying to not be able to access, it doesn't seem to have any functional restrictions, in my environment at least.


Recreating the User Profile Service application

1. First we need to get rid of the broken instance of UPS.

a. Under Central Administration > System Settings > Manage services on server, stop the "User Profile Service" and "User Profile Synchronization Service"

b.  Under Central Administration > Application Management > Manage service application, delete the "User Profile Service application"

c.  On the Sharepoint server itself, open an MMC console, add a Certificates snap-in, select Computer Account. Check the Personal, Trusted Root Certification Authorities and Trusted People stores for "ForefrontIdentityManager" certificates and delete ALL the FIM certificates.

d.  Open a Sharepoint 2010 Management Shell, issue the command get-spserviceapplicationpool. Remove any service application pools that are associated with previous UPS applications with the command:

remove-spserviceapplicationpool "PoolName"

e.  Delete any pending timer jobs related to the UPS synchronization service provisioning. Under Central Administration >Monitoring > Check job status, check the Running section for any related jobs and delete them.

f.  Wait 15 minutes and then issue an IISRESET before proceeding to the next step.

2. Under Central Administration > System Settings > Manage services on server, start the "User Profile Service.

3.  Under Central Administration > Application Management > Manage service application, create a new UPS. Use a different name than you did for the previous UPS instance, different database names and a different application pool name. Wait 15 minutes then issue an IISRESET.

4. Under Central Administration > System Settings > Manage services on server, start the User Profile Synchronization service. Wait 15 minutes and if both FIM services are running from an MMC services snap-in as below, issue an IISRESET.

If you get this far and have no problems opening your UPS application from Central Administration > Application Management > Manage service application then congratulations, more than likely you have resolved your problems.

You can now proceed to adding Synchronization connectors and bringing those attributes in from Active Directory.

If you are still having issues getting your UPS to work after following all these steps, then the Sharepoint MSDN forums are a great place to get help from Sharepoint gurus with much more experience than myself.