Created

Jul 23, 2012

Monitoring Replication Queue Status in an automated fashion

Posted by Sneh Arora

While monitoring any CQ instance,it is necessary to track the status of replication queues to check if replication queue is blocked or not, or to check if  the size of queue is within a certain range.One can manually monitor replication and reverse replication queues by looking at their respective status pages. Details are available here.


The Problem

In a load testing run or longevity run, one will have to manually go to above mentioned URL, check the data and record it. This too at regular intervals, so that he can create meaningful trends out of it. But this will become a hectic process if done manually

The Solution

getReplicationQueueStatus is a Perl based script which can be used to keep track of status of replication and reverse replication queues.This script regularly polls Replication Agent and Reverse Replication Agent status page and dumps the required data in the form of csv on stdout which can be redirected to a file if required. The output of this script can be used to analyze whether queue grew abnormally during the test run or was blocked at any point of time or not.

Perquisites

  • Perl should be installed and present in default PATH
  • Curl should be installed and present in default PATH

Script Execution

To get started, modify the script properties to point to your server.






#Polling interval
$interval = 4;

# URL to hit For Replication Queue Status: curl  [http://admin:admin@]:/etc/replication/agents.author/publish.html]
$fwd_target = "/etc/replication/agents.author/publish.html";

# URL to hit For Reverse Replication Queue Status: curl  [http://admin:admin@]:/etc/replication/agents.author/publish_reverse.html]
$rev_target = "/etc/replication/agents.author/publish_reverse.html";

#Host Name or IP and Port of machine running CQ instance.
$host = ":";

#Admin Credentials of CQ Instance
$id = "admin:admin@";

#1st Part of Curl Command. Specify curl parameters here.
$curl = "/usr/bin/curl \-s \-o \-";

And simply start the script.

Usage

./getReplicationQueueStatus> replication_output.log 2>&1

Sample Output

# Replication Queue Details From Server date,time,repQueueStatus,repQueueSize,revRepQueueStatus,revRepQueueSize 2012-06-11,06:49:16,idle,0,idle,0 2012-06-11,06:49:23,active,2,idle,0 2012-06-11,06:49:30,idle,0,active,1 2012-06-11,06:49:37,idle,0,idle,0

Points to be noted

  1. Note that the attached script currently monitors a single author-publish instance .If more than one replication agent/reverse replication agents are configured, script needs to be modified accordingly.
  2. The above statistics can also be measured using JMX. More details are available here.

Script

The perl script, getReplicationQueueStatus, is pasted below for use


#!/usr/bin/perl
#*************************************************************************
# ADOBE SYSTEMS INCORPORATED
# Copyright 2012 Adobe Systems Incorporated
# All Rights Reserved.
# 
# NOTICE:  Adobe permits you to use, modify, and distribute this file in 
# accordance with the terms of the Adobe license agreement accompanying it.  
# If you have received this file from a source other than Adobe, 
# then your use, modification, or distribution of it requires the prior 
# written permission of Adobe.
#*************************************************************************
# URL to hit For Replication Queue Status: curl  http://admin:admin@:/etc/replication/agents.author/publish.html
# URL to hit For Reverse Replication Queue Status: curl  http://admin:admin@:/etc/replication/agents.author/publish_reverse.html
# 
#Data logged:date,time,repstat,repq,rrepstat,rrepq

#Polling interval
$interval = 4;
# URL to hit For Replication Queue Status:http://admin:admin@:/etc/replication/agents.author/publish.html
$fwd_target = "/etc/replication/agents.author/publish.html";
# URL to hit For Reverse Replication Queue Status:http://admin:admin@:/etc/replication/agents.author/publish_reverse.html
$rev_target = "/etc/replication/agents.author/publish_reverse.html";

#Host Name or IP and Port of machine running CQ instance.
$host = ":";

#Admin Credentials of CQ Instance
$id = "admin:admin@";

#1st Part of Curl Command. Specify curl parameters here.
$curl = "/usr/bin/curl -s -o -";



sub scan
{
local $status;

local ( $target ) = @_;

$stream = "$curl http://$id$host$target";

$status = "?,?";

die "can't execute curl operation" unless open DATA, "$stream|";
while () {
#Queue Status can be active/blocked/idle
  if (/\
Queue is active - 90 pending
# if (/cq-agent-queue-active">Queue is active - (\d+) pending/) { $status = "active,$1"; } #
  • Queue is blocked - 20 pending
  • # if (/cq-agent-queue-blocked">Queue is blocked - (\d+) pending/) { $status = "blocked,$1"; } #
  • Queue is idle
  • # if (/cq-agent-queue-idle">Queue is idle/) { $status = "idle,0"; } last; } } close DATA; return $status; } #Function to get date for time stamp in logs sub timestamp { local($date); local($sec,$min,$hour,$mday,$mon,$year) = localtime; $year += 1900; $mon += 1; $date = sprintf("%04d-%02d-%02d,%02d:%02d:%02d", $year, $mon, $mday, $hour, $min, $sec); return $date; } #Main function which will poll Publish and Publish_Reverse page in pre-defined interval. print "# Replication Queue Details From $host\n"; print "date,time,repQueueStatus,repQueueSize,revRepQueueStatus,revRepQueueSize\n"; while (1) { #get timestamp $ts = ×tamp; #get replication queue stats $fst = &scan($fwd_target); #get reverse replication queue stats $rst = &scan($rev_target); print "$ts,$fst,$rst\n"; sleep $interval; }

    COMMENTS

    • By jual propolis murah - 12:02 PM on Apr 05, 2013   Reply
      informative post, its so useful, keep it up
    • By manfaat madu - 12:03 PM on Apr 05, 2013   Reply
      wow, i dont know that, this is a great share, thank you
    • By belly bands for dogs - 12:03 PM on Apr 05, 2013   Reply
      nice share, it open my mind even more now, thanks
    • By wooden dog crates - 12:03 PM on Apr 05, 2013   Reply
      i dont know at first that it will be a useful information, but really, thank you for posting it
    • By bee propolis hd - 12:03 PM on Apr 05, 2013   Reply
      This information really helped me understand Things beyond my expertise. websites like this expanding knowledge everyone and I hope something like this will continue to grow
    • By dien dan seo - 5:47 AM on Apr 15, 2013   Reply
      i dont know that, this is a great share, thank you
    • By hdi bee propolis - 9:26 AM on Apr 21, 2013   Reply
      awesome information man, thx
    • By manfaat madu asli - 10:22 AM on Apr 27, 2013   Reply
      what a great post, thx for the share
    • By photo booth nj - 5:10 PM on Apr 27, 2013   Reply
      Thanks for keeping us informed
    • By limo nj - 6:45 PM on Apr 27, 2013   Reply
      I like it that we can now monitor the replication queue status. Before it was just a hassle. Thanks
    • By best wedding photographer nj - 9:54 PM on Apr 27, 2013   Reply
      Its great to see you taking the time to share this information
    • By nyc limo rental - 5:26 PM on May 02, 2013   Reply
      I was searching threw the code earilier and this was really helpful to find where I went wrong. Thanks for actually posting the code for some of us.