TMG 2010 Certificate Enrollment Fails - RPC server is unavailable

During some recent TMG 2010 maintenance, we noticed the TMG computer certificate had expired a few months earlier, this should never occur as our Active Directory CA should re-issue a certificate when expiration is near.

We first attempted to manually re-request the certificate via the Certificates MMC snap-in and were presented with an error saying "The RPC server is unavailable."

'

Our eventlogs suggested that a DCOM problem may have occurred. After checking DCOM and the RPC Service we were unable to uncover any issues.

The Solution

It turns out TMG itself introduces this error intentionally. The "Enable strict RPC compliance" setting, which is enabled by default, blocks the RPC functionality required for AD based certificate enrollment to work. Fortunately the fix is straight forward.

1. Open your TMG console 

2. Navigate to "Firewall Policy" 

3. Right click "Firewall Policy", select "All Tasks", select "System Policy" and finally select "Edit System Policy"
 

4. Under "Authentication Services" select "Active Directory 

5. Untick "Enforce strict RPC compliance
 

6. Click "OK" 

7. Apply the policy changes

You should of course do the appropriate research before disabling this setting, in some super high security environments you may not wish to disable RPC compliance, however in our environment it made no difference.

After waiting a few minutes for the policy changes to occur our problem was resolved, certificate enrollment once again worked perfectly.