Website automation for monitoring (oracle access manager 10, diagnostics)

I wanted to share an example that I had building a script for monitoring a web application by looking at its diagnostic page. To get to this page, there are one or more layers of logons, cookies that need to be collected, and forms to fill in before you get to the actual diagnostics information. To build this, I first looked at powershell, but there seemed to be no easy method of doing web interactions involving cookies. Apparently you need to extend the webclient class to allow cookies, which would require developing an app instead of just using built in commands. After looking around a bit, I found PERL LWP which is simple and easy to use. To get going with a complicated web transaction, you can use tools like fiddler or iehttpheaders to walk yourself through the transaction manually and gather the data. This will show you where cookies are used, and what information is passed in POST requests, as well as all of the URL's to use. Strip out all of the image download requests and similar javascript junk you don't need, then create a step by step walk through this with corresponding PERL LWP methods. The results can be processed as text, or any other method you can come up with using other parsing or modules operations. In my example, I just do some text matching and counting to determine what is a successful check and what is a failure.

use LWP::UserAgent;
use HTTP::Cookies;

#put args processing in here
if ($#ARGV < 3) {
 print "usage: runaccess-diag.pl user password server domain";
 exit 1;
}

$user = $ARGV[0];
$pass = $ARGV[1];
$server = $ARGV[2];
$domain = $ARGV[3];
$debug = $ARGV[4];



#################################################################################################
###This url is where you would end up if you tried to directly access the diagnostics page without
###logging in.  This will keep us out of frames and keep it all simple.
#################################################################################

$url = "http://" . $server . '/identity/oblix/apps/admin/bin/front_page_admin.cgi?program=commonLogin&returnUrl=..%2F..%2F..%2F..%2F..%2Faccess%2Foblix%2Fapps%2Fadmin%2Fbin%2Fsysmgmt.cgi%3FloginTry%3D1%26pluginName%3Dsysmgmt%26program%3DgenDiagnostics&backUrl=..%2F..%2F..%2F..%2F..%2Faccess%2Foblix%2Fapps%2Fadmin%2Fbin%2Fsysmgmt.cgi';

#####################################################################################
#Define a few other URLS for handling webgate and logoff, as well as diagnostics page code
#####################################################################################

$starturl = "http://" . $server . "/access/oblix/apps/admin/bin/front_page_admin.cgi";

$altLoginurl = "http://" . $server . "/login/OAMlogin.htm";

$webgateurl = "http://" . $server . "/access/oblix/apps/webgate/bin/webgate.dll";

$diagurl = "http://" . $server . "/access/oblix/apps/admin/bin/sysmgmt.cgi";

$endurl = "http://" . $server . "/access/oblix/lang/en-us/logout.html";

$secondaryurl = "http://" . $server . "/access/oblix/apps/admin/bin/sysmgmt.cgi?loginTry=1&pluginName=sysmgmt&program=genDiagnostics";


$cookie_jar = HTTP::Cookies->new(file => "c:\temp\cookie.lwp");
$browser = LWP::UserAgent->new;
$browser->cookie_jar( $cookie_jar);

######################################################
#open up diag url initially to get logon page cookie##
######################################################

$response = $browser->get($starturl);
sleep 5;
$response = $browser->get($altLoginurl);

if (!($response->content =~ /document.loginform.password.onkeypress/)) {
 print "
Could not load access page for $server
\n";
 if ($debug) { print $response->content;}
 exit 2;
}

if ($debug) { print $response->content; }

########################################
#POST back to form with logon details ##

#NOTE: Not all access servers are the same
# They use different POST values and logon
# methods, some requireing webgate interaction
########################################


$response = $browser->post($url,[ 'fromloginpage' => "true", 'comp' => "", 'login' => $user, 'password' => $pass, 'LoginDomain' => $domain]);
if ($debug) { print "\n\n5\n" . $response->content; } 
$response = $browser->get($secondaryurl);
if ($debug) { print "\n\n6\n" . $response->content; }

if ((!($response->content =~ /Please select Access Server/))){
 #$response = $browser->get($secondaryurl);
 $response = $browser->get($altLoginurl);
 if ($debug) { print "\n\n7\n" . $response->content; }
 $response = $browser->post($webgateurl,[ 'fromloginpage' => "true", 'comp' => "", 'uid' => $user, 'password' => $pass]);
 if ($debug) { print "\n\n8\n" . $response->content; }
 $response = $browser->get($secondaryurl);
 if ($debug) { print "\n\n9\n" . $response->content; }
 if ((!($response->content =~ /Please select Access Server/))){
  print "
Logon failure for $server
\n";
  print "\n\n10\n" . $response->content;
  exit 2;
 }
}


##################################################
#POST back again to run the diagnostics          #
#
#That 'Program' var is not a typo in the script, #
#problem is in the access code
##################################################
$response = $browser->post($diagurl,[ 'program' => "generateDiagnositcsReportPage", 'allAsServers' => "true", 'as_server' => "true"]);
 if ($debug) { print "\n\n\n" . $response->content; }


#################################################
#READ results of content from diagnostics page ##
#################################################
$results = $response->content;
 if ($debug) { print "\n\n\n" . $response->content; }

#################
#Do UP and DOWN status match counting.  For a valid server
#it will be UP with its overall status and UP for 3 components
#and down for 3 components. 
#
#This is not the best checking, ok for 2 servers 
###############

$match = ">Up<";
$UPcount = () = $results =~ /$match/g;

$match = ">Down<";
$DOWNCount = () = $results =~ /$match/g;



if ($UPcount < ($DOWNCount +2)) {
 print "
$server: Diagnostics showing failure
";
 print $results;
 my $response = $browser->get($endurl);
 exit 3;
} else {
 my $response = $browser->get($endurl);
 exit 0;
}