DES encryption, Kerberos and 2008 Server

When dealing with Windows 2008 servers as domain controllers, mixed with legacy applications, you may run in to a problem with encryption support. Older software and platforms may be set to use DES encryption. This has been disabled by default on newer Microsoft OS's including 2008R2 and Windows 7. There are ways to get the support turned back on, though for security reasons this is not recommended. DES uses a weaker key than the other available methods, and most systems should support the windows standard of RC4-HMAC.

Identification of DES usage in your environment:

1) Netmon. Using the older Microsoft version of netmon, you can monitor your domain controllers and look for kerberos traffic that is using encryption other than RC4-HMAC. To do this, run a capture and create a display filter as follows:

AND
|---Kerberos: Encryption type (Etype[0]) <> 0x17
|---Protocol == KERBEROS
|---Any <--> Any

This will show all Kerberos traffic that is not using the standard. This may also find other types such as AES if you are using the latest and greatest. Since there are several types of DES encryption formats for Kerberos, this filter method is the simplest, but you can also create a multiple set of OR statements on that Etype value. Refer to the RFC's to get the values of each.

2) Kinit: If you are already seeing problems with kerberos authentication for certain applications, you can use kinit with debugging options to request a ticket for that service. This can show you if you are getting a bad Encryption type error. Here is one example using java and a keytab:

java -Dsun.security.krb5.debug=true -Dsun.security.krb5.krb5debug=all sun.security.krb5.internal.tools.Kinit -k -t HTTP.keytab HTTP/myservice@MYDOMAIN


-result snip-
Found unsupported keytype (23) for
HTTP/myservice@MYDOMAIN
-/result snip-


3) Searching for DES enabled AD accounts:
Ldap filter: (&(|(objectcategory=user)(objectcategory=computer))(userAccountControl:1.2.840.113556.1.4.804:=2097152))

This will find all DES enabled computer and user accounts in your search scope. You can search the serviceprincipalname attribute to see what applications may be using DES.

4) Kerberos event log events on the DC's



5) Errors in application logs.  ETYPE not supported and similar.  Some logs will show types of encryption configured.  Check java versions (1.4.x and earlier don't support RC4-HMAC).  Check krb5.ini and krb5.conf files for encryption type configurations.

Fixing the problem

There are two ways to go about getting around this problem during an upgrade. First, you can try to identify all of the uses of DES and get rid of them, or you can enabled DES support.

To get rid of DES usage, look for capability in the application for RC4-HMAC or other support encryption standard. Check the application, check the versions of java it uses, etc. If it is java, anything 1.5 and higher supports RC4. Check all the krb5.conf files to ensure any supplied enctype values allow for RC4-HMAC


[libdefaults]
default_tkt_enctypes = rc4-hmac
default_tgs_enctypes = rc4-hmac
permitted_enctypes = rc4-hmac

Do your testing and push the appropriate application owners to do what they can to get away from DES. If keytab's are in use, ensure they are recreated with the appropriate encryption.

If you have applications that cannot get rid of DES, you can look at the steps required to enable DES support on the OS. There are two parts to this. First you will need to patch your 2008 domain controllers with KB978055. This gives the DC the ability to issue DES tickets. If your clients are windows 7 or 2008R2 server themselves, they will need to have some configuration changes. This can be done by a registry fix, or pushed by group policy. Refer to this article for that. When changing the client settings, be careful that you allow all of the required encryption types. If you use a GPO to turn on DES, and don't specify anything else, your machine will only use DES.

UPDATE (2/11/2012): After having the above patch fail to install with an error that it is not applicable, it seems that the patch was rolled into Windows 2008R2 service pack 1. So if you have the service pack installed, you should be fine on the domain controller side.

UPDATE (10/2/2012):  Apparently not all versions of service pack 1 would have this fix.  General distribution release versions of the SP won't have it.  You can check http://support.microsoft.com/?id=2425227 and http://support.microsoft.com/?id=2029058 to get hotfixes for updating the KDC service.  Also ensure that your group policy allowed encryption types include the Future encryption types.  Don't ask why, but for some reason in my testing having everything but that causes DES to fail in some case, while checking it causes it to suddenly work.