Created
Monitoring Replication Queue Status in an automated fashion
While monitoring any CQ instance,it is necessary to track the status of replication queues to check if replication queue is blocked or not, or to check if the size of queue is within a certain range.One can manually monitor replication and reverse replication queues by looking at their respective status pages. Details are available here.
The Problem
In a load testing run or longevity run, one will have to manually go to above mentioned URL, check the data and record it. This too at regular intervals, so that he can create meaningful trends out of it. But this will become a hectic process if done manually
The Solution
getReplicationQueueStatus is a Perl based script which can be used to keep track of status of replication and reverse replication queues.This script regularly polls Replication Agent and Reverse Replication Agent status page and dumps the required data in the form of csv on stdout which can be redirected to a file if required. The output of this script can be used to analyze whether queue grew abnormally during the test run or was blocked at any point of time or not.
Perquisites
- Perl should be installed and present in default PATH
- Curl should be installed and present in default PATH
Script Execution
To get started, modify the script properties to point to your server.
#Polling interval
$interval = 4;
# URL to hit For Replication Queue Status: curl [http://admin:admin@]:/etc/replication/agents.author/publish.html]
$fwd_target = "/etc/replication/agents.author/publish.html";
# URL to hit For Reverse Replication Queue Status: curl [http://admin:admin@]:/etc/replication/agents.author/publish_reverse.html]
$rev_target = "/etc/replication/agents.author/publish_reverse.html";
#Host Name or IP and Port of machine running CQ instance.
$host = ":";
#Admin Credentials of CQ Instance
$id = "admin:admin@";
#1st Part of Curl Command. Specify curl parameters here.
$curl = "/usr/bin/curl \-s \-o \-";
And simply start the script.
Usage
./getReplicationQueueStatus> replication_output.log 2>&1
Sample Output
# Replication Queue Details From Server date,time,repQueueStatus,repQueueSize,revRepQueueStatus,revRepQueueSize 2012-06-11,06:49:16,idle,0,idle,0 2012-06-11,06:49:23,active,2,idle,0 2012-06-11,06:49:30,idle,0,active,1 2012-06-11,06:49:37,idle,0,idle,0
Points to be noted
- Note that the attached script currently monitors a single author-publish instance .If more than one replication agent/reverse replication agents are configured, script needs to be modified accordingly.
- The above statistics can also be measured using JMX. More details are available here.
Script
The perl script, getReplicationQueueStatus, is pasted below for use
#!/usr/bin/perl
#*************************************************************************
# ADOBE SYSTEMS INCORPORATED
# Copyright 2012 Adobe Systems Incorporated
# All Rights Reserved.
#
# NOTICE: Adobe permits you to use, modify, and distribute this file in
# accordance with the terms of the Adobe license agreement accompanying it.
# If you have received this file from a source other than Adobe,
# then your use, modification, or distribution of it requires the prior
# written permission of Adobe.
#*************************************************************************
# URL to hit For Replication Queue Status: curl http://admin:admin@:/etc/replication/agents.author/publish.html
# URL to hit For Reverse Replication Queue Status: curl http://admin:admin@:/etc/replication/agents.author/publish_reverse.html
#
#Data logged:date,time,repstat,repq,rrepstat,rrepq
#Polling interval
$interval = 4;
# URL to hit For Replication Queue Status:http://admin:admin@:/etc/replication/agents.author/publish.html
$fwd_target = "/etc/replication/agents.author/publish.html";
# URL to hit For Reverse Replication Queue Status:http://admin:admin@:/etc/replication/agents.author/publish_reverse.html
$rev_target = "/etc/replication/agents.author/publish_reverse.html";
#Host Name or IP and Port of machine running CQ instance.
$host = ":";
#Admin Credentials of CQ Instance
$id = "admin:admin@";
#1st Part of Curl Command. Specify curl parameters here.
$curl = "/usr/bin/curl -s -o -";
sub scan
{
local $status;
local ( $target ) = @_;
$stream = "$curl http://$id$host$target";
$status = "?,?";
die "can't execute curl operation" unless open DATA, "$stream|";
while () {
#Queue Status can be active/blocked/idle
if (/\Queue is active - 90 pending
#
if (/cq-agent-queue-active">Queue is active - (\d+) pending/) { $status = "active,$1"; }
# Queue is blocked - 20 pending
#
if (/cq-agent-queue-blocked">Queue is blocked - (\d+) pending/) { $status = "blocked,$1"; }
# Queue is idle
#
if (/cq-agent-queue-idle">Queue is idle/) { $status = "idle,0"; }
last;
}
}
close DATA;
return $status;
}
#Function to get date for time stamp in logs
sub timestamp
{
local($date);
local($sec,$min,$hour,$mday,$mon,$year) = localtime;
$year += 1900;
$mon += 1;
$date = sprintf("%04d-%02d-%02d,%02d:%02d:%02d",
$year, $mon, $mday, $hour, $min, $sec);
return $date;
}
#Main function which will poll Publish and Publish_Reverse page in pre-defined interval.
print "# Replication Queue Details From $host\n";
print "date,time,repQueueStatus,repQueueSize,revRepQueueStatus,revRepQueueSize\n";
while (1) {
#get timestamp
$ts = ×tamp;
#get replication queue stats
$fst = &scan($fwd_target);
#get reverse replication queue stats
$rst = &scan($rev_target);
print "$ts,$fst,$rst\n";
sleep $interval;
}
COMMENTS
-
informative post, its so useful, keep it up
-
wow, i dont know that, this is a great share, thank you
-
nice share, it open my mind even more now, thanks
-
i dont know at first that it will be a useful information, but really, thank you for posting it
-
This information really helped me understand Things beyond my expertise. websites like this expanding knowledge everyone and I hope something like this will continue to grow
-
i dont know that, this is a great share, thank you
-
awesome information man, thx
-
what a great post, thx for the share
-
Thanks for keeping us informed
-
I like it that we can now monitor the replication queue status. Before it was just a hassle. Thanks
-
Its great to see you taking the time to share this information
-
I was searching threw the code earilier and this was really helpful to find where I went wrong. Thanks for actually posting the code for some of us.
Comments (12)