TMG 2010 Certificate Enrollment Fails - RPC server is unavailable

During some recent TMG 2010 maintenance, we noticed the TMG computer certificate had expired a few months earlier, this should never occur as our Active Directory CA should re-issue a certificate when expiration is near.

We first attempted to manually re-request the certificate via the Certificates MMC snap-in and were presented with an error saying "The RPC server is unavailable."

'

Our eventlogs suggested that a DCOM problem may have occurred. After checking DCOM and the RPC Service we were unable to uncover any issues.

The Solution

It turns out TMG itself introduces this error intentionally. The "Enable strict RPC compliance" setting, which is enabled by default, blocks the RPC functionality required for AD based certificate enrollment to work. Fortunately the fix is straight forward.

1. Open your TMG console 

2. Navigate to "Firewall Policy" 

3. Right click "Firewall Policy", select "All Tasks", select "System Policy" and finally select "Edit System Policy"
 

4. Under "Authentication Services" select "Active Directory 

5. Untick "Enforce strict RPC compliance
 

6. Click "OK" 

7. Apply the policy changes

You should of course do the appropriate research before disabling this setting, in some super high security environments you may not wish to disable RPC compliance, however in our environment it made no difference.

After waiting a few minutes for the policy changes to occur our problem was resolved, certificate enrollment once again worked perfectly.

Configuring Forefront UAG trunks to support Yubico YubiRadius OTP authentication

In the process of preparing some of my external services for Yubikey integration I have been faced with a few problems, integration with Forefront UAG is no exception.

Adding the YubiRadius radius server to UAG as an authentication server is rediculously easy. Open the desired trunk properties, go to the authentication tab, add a new radius authentication server and put in your server IP and secret key.

After spending all of 2 minutes configuring YubiRadius as an authentication provider for one of my existing trunks I attempted to login and was repeatedly met with a generic UAG "Access Denied" screen.

I jumped onto the YubiRadius box via SSH and restarted freeradius in foreground vebrose debug mode by starting it with freeradius -f -X. Freeradius gave me a vital clue, UAG was only passing the first 20 characters of the OTP to the YubiRadius server, so of course YubiRadius was replying to UAG with access denied.



Fixing the Issue

The problem occurs because by default UAG only allows 20 characters in the password field, any more than 20 are automatically truncated back to 20 before being passed to the authentication server. In most instances this would be fine, but for OTP's it simply doesn't work. Luckily for us, the fix is a piece of cake.

1. Log into your UAG box and open the following folder "%programfiles%\Microsoft Forefront Unified Access Gateway\von\InternalSite\samples"

2. Copy the customDefault.inc from the samples folder to "%programfiles%\Microsoft Forefront Unified Access Gateway\von\InternalSite\inc\CustomUpdate"

3. Edit the customDefault.inc and change the PasswordLimit field to 50 (or more if you are using a custom OTP length), as per below. You may even need to consider a length closer to 70 characters if you are using a shared field for Active Directory password and OTP.

4. Open a command prompt and issue an iisreset

Done like a dinner, your UAG server should now pass the full OTP token to Freeradius, to which it can properly validate if the token is authentic or not.

Customizing the UAG SP1 logon page

Microsoft Forefront UAG is a great product for adding a bit more security to the publishing of internal websites. The ability to screen the login process, apply some basic IDS and NAC is very handy indeed.

Users wanting to take UAG to the next level might consider customizing their login landing page, to give a more corporate feel to their external sites. Olivier Detilleux published a great tutorial explaining how this process works but unfortunately from a number of Technet posts with users asking questions articles its evident some of the detail in Olivier's article is lost on some users.



How does the customization work? 

Although the process changed with SP1, it is probably easier now than it was before.

1. Navigate to "C:\program files\Microsoft Forefront Unified Access Gateway\von\InternalSite"

2. Create your custom headertopr.gif and place it in "C:\program files\Microsoft Forefront Unified Access Gateway\von\InternalSite\Images\CustomUpdate"

3. Copy  "C:\program files\Microsoft Forefront Unified Access Gateway\von\InternalSite\inc\logo.inc" to "C:\program files\Microsoft Forefront Unified Access Gateway\von\InternalSite\inc\CustomUpdate\logo.inc"

4. Rename "C:\program files\Microsoft Forefront Unified Access Gateway\von\InternalSite\inc\CustomUpdate\logo.inc" to "C:\program files\Microsoft Forefront Unified Access Gateway\von\InternalSite\inc\CustomUpdate\<trunkname><issecure 0 or 1>logo.inc"

For example if your trunk is called "OWA" and it is a https trunk, then your custom logo.inc would be called "OWA1logo.inc" with the 1 indicating https, 0 is used for http.

5. Now you need to remove the "if" scripting at the top of this file. This scripting is used in the original logo.inc to detect your custom "<trunkname><issecure 0 or 1>logo.inc" and if it is used in a custom logo.inc it will cause an error message.

To do this from the top of your custom logo.inc please remove

<%'include file for title
' xxxxxxxxxxxxxxxxxxxxxxx DO NOT EDIT THIS FILE xxxxxxxxxxxxxxxxxxxxxxxx
' A.O.detectionDOSFix - Store include file names in Application and not in Session.
if Application(g_site_name&g_secure&LOGO_INC) <> FILE_NOT_EXIST then
    include Application(g_site_name&g_secure&LOGO_INC)
else%>

 

and from the bottom of the file remove 

<%end if%> 

6. Then you can make the customizations to your custom logo.inc, such as inserting your own header image as per Olivier's tutorial.