Windows Time Service event 46 - access denied

I recently worked on a case where a domain controller came online with its clock time several hours out of sync (virtualized DC).  In this case, when looking at the system log, during the service start up events, there was a critical error for the windows time service:


- <Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
- <System>
<Provider Name="Microsoft-Windows-Time-Service" Guid="{06EDCFEB-0FD0-4E53-ACCA-A6F8BBF81BCB}" />
<EventID>46</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>0</Task>
<Opcode>0</Opcode>
<Keywords>0x8000000000000000</Keywords>
<TimeCreated SystemTime="2014-02-21T07:25:24.140175500Z" />
<EventRecordID>260042</EventRecordID>
<Correlation />
<Execution ProcessID="452" ThreadID="3648" />
<Channel>System</Channel>
<Computer>MyDC1.contoso.com</Computer>
<Security UserID="S-1-5-19" />
</System>
- <EventData Name="TMP_EVENT_ERROR_SHUTDOWN">
<Data Name="ErrorMessage">0x80070005: Access is denied.</Data>
</EventData>
</Event>


Googling around came up with some details that this error can occur when netlogon service is not started.  Going back to the log showed a Service Control Manager 7022 netlogon service hung during startup.  After a few weeks back and forth with microsoft with netlogon tracing and memory dumps, it just came down to the fact that there were a lot of subnets being processed.  The servers being effected by the slow netlogon startup were all low spec virtualized domain controllers, so they weren't going to perform at their best anyways.  During the service startup, all subnets must be read into memory, which can take a while.  There is also no registry tweaks or configuration changes to get around this...other than cleaning up subnets.  The one thing that we had thought of before the whole case was, if time service needs netlogon running for it to function, why isn't it configured with service dependencies.  Even though the OS doesn't do this by default, some registry hacking will allow you to add a DependsOnService value to the w32time service key to ensure netlogon is started before time service tries to start.  This can be pushed through GPO as well.  For a .REG file you can use this:

Windows Registry Editor Version 5.00

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\services\W32Time]
"DependOnService"=hex(7):6e,00,65,00,74,00,6c,00,6f,00,67,00,6f,00,6e,00,00,00,\
  00,00

Finding large time changes (windows)

When you are looking at time sync problems on newer Microsoft OS's (2008+), there are several places that may show useful information. Looking in the system log, you can find various events from the source: Time-Service, which tell you what server you are syncing with, if the servers are not available, if you domain controller is advertising time, and other various issues. In addition to that, another source: Kernel-General, may have some useful information. In Event ID #1 of this source, you will see occasional clock changes on the system. It gives both the old DateTime and the new one. This helps show you when large changes to the clock happen, so you can help historically see problematic servers. So, to collect and view this information in a more useful way, I came up with this example:


get-winevent -FilterHashtable @{logname="system"; providername="Microsoft-Windows-Kernel-General"; ID=1}|select -first 100 -Property TimeCreated,Properties,MachineName | foreach {
     $comp = $_.machinename
     $timeskew = new-timespan -start $_.properties[0].value -end $_.properties[1].value
     $timeskew = [int][math]::abs($timeskew.totalminutes)
     new-object PSObject -property @{
            Machine=$comp
           EventDate = $_.TimeCreated
          TimeDiffMinutes = $timeskew
   }
}|where {$_.TimeDiffMinutes -gt 2}

Here we use Get-WinEvent with a filterhash table to get the events we want. I'm just looking at a limited result here. In each event there are 2 properties which contain the two DateTime values. I'm putting that into a timespan to pull the difference in minutes, removing any negative value and printing out the machinename, Timeskew in minutes and when the change was done. You can add -computer to the initial Get-Winevent to run a list of machines.

W32Time event 47 manually configured peer

Recently I was dealing with some SCOM events for time services on a few machines in the same domain.  When checking the machines, I came across this error:

Time Provider NtpClient: No valid response has been received from  manually configured peer 10.0.0.1 after 8 attempts to contact it. This peer will be discarded as a time source and NtpClient will attempt to discover a new peer  with this DNS name.

On seeing this, I thought this domain may have been configured with manual peers and NTP as the client's provider.  When looking at the registry though, all I was seeing was the typical time.windows.com ntp server setting and source was NT5DS.  So I was stuck for a while thinking, the source should be the domain, and this IP address that I'm seeing is not a domain controller, never was a domain controller, and isn't even pinging.  So I tried manual peer configuration with NTP as the provider on a server, but I hit the same issue with the same error.  Searching the registry for both a host name and the IP came up with nothing.  Searching gpresult for the IP/hostname came up with nothing.  Eventually, I dug a bit further in to the "gpresult /scope COMPUTER /Z" output and found an NTP serverr was set in there.  So apparently this type of GPO setting does not push itself to the register, and just quietly overrides whatever is in the registry.  The reason I couldn't find the IP/hostname in the gpresult the first time was that it comes out in gpresult as an array of byte values.

So anyways, GPO edited, gupdate /force, w32tm /resync...and its all back to normal.