Friday, July 22, 2011

Powershell sometimes less powerful than a snail

Today I was working on a simple old log file deletion script to run against remote machines. Since they are legacy boxes, they don't have powershell remoting capabilities, so I was going for basic access via admin shares. I wanted to clean up files that contained date data in the file name, so it was a pretty simple, wild card match delete operations and a few date operations...something that should take a few seconds to throw together. Just to be on the safe side, I tried running my work through PowerGui Script editor in debug mode and ended up with a hang on a deletion operation. That's wierd since its only handling a few hundred files, something cmd's del command would knock out in no time. It seems with the changes that came with powershell, .NET integration and the object oriented nature of how DEL and DIR were replaced with remove-item and get-childitem, the operations in the background became ridiculously inefficient in some cases. In my example I'm accessing a server that has a ping response time of 249ms from my machine where I'm running the debug. From another machine (my script jobs server) I have a 4ms response time to the target. Notice how this works for me:

From the 4ms server doing a get-childitem operation on a folder with 940 files in it

Friday, July 22, 2011 2:04:50 AM
Friday, July 22, 2011 2:05:02 AM


12 second is a bit slow, but tolerable. Lets see how cmd compares:

get-date; cmd /C "dir \\remoteserver\c$\mydir >c:\temp\somebsfile.txt"; get-date

Friday, July 22, 2011 2:18:08 AM
Friday, July 22, 2011 2:18:09 AM


1 second or less. Much nicer. So, how about that debug machine 249ms away from the target:

Lets start with cmd /C DIR, because my Powershell get-childitem has been running so long already:

Friday, July 22, 2011 3:08:54 PM
Friday, July 22, 2011 3:09:01 PM


and we're waiting for powershell.....

waiting....

15 minutes gone by.....

still waiting....

is this still processing????

firing up netmon.....

yeah its still pulling data over SMB....

SMB query path info every 300ms or so...

comes out like this

Friday, July 22, 2011 3:05:05 PM
Friday, July 22, 2011 3:30:16 PM


Bottom line, powershell remoting is probably a better way if available, otherwise failback to CMD.

3 comments:

  1. You know I wrote something just like this recently to deal with IIS logs though didn't notice any performance issues. The difference I think was that I was moving not deleting so perhaps that is why it wasn't a problem for me...

    I just now posted my script here if you're intested:

    http://danstechnotes.blogspot.com/2011/08/powershell-script-to-archive-iis-log.html

    I'd be interested to see how your script turned out.

    ReplyDelete
  2. Yours looks like it would do roughly the same kind of operations on a remote system. Mine actually runs ok the way I have it, which is running it from a server in the same datacenter that my target servers are in. When the network delay between the system running the script and the target machine is low, then it runs ok. When trying it on a link with more delay it causes some issues.

    ReplyDelete
  3. Ah yeah good old CIFS... Sounds like you need some riverbeds!

    ReplyDelete