Jump to content


Monitor P4D with Datadog

datadog server monitor linux

  • Please log in to reply
3 replies to this topic

#1 JasonDeArte

JasonDeArte

    Newbie

  • Members
  • Pip
  • 2 posts

Posted 08 August 2017 - 08:18 PM

Hello,
How do you monitor the uptime of a p4d server process on a Linux box?

I use DataDog Agent & usually I would install one of their various client integrations for Jenkins or GitLab. https://docs.datadog...m/integrations/

But there are none for helix p4d.

What is the most reliable method for a process running on the same machine as p4d to determine that p4d is running & gather helix specific performance stats?

#2 Matt Janulewicz

Matt Janulewicz

    Advanced Member

  • Members
  • PipPipPip
  • 133 posts
  • LocationSan Francisco, CA

Posted 08 August 2017 - 08:32 PM

We use Zabbix for monitoring but the idea is probably the same. Here's a smattering of commands we run on our hosts to produce values that are sent to the Zabbix server each minute.

The quickest and dirtiest way to get basic info is by parsing the output of 'p4 info'. If the server is not up, no output. And you don't need a login ticket to run that.


We are in a commit/edge architecture so we collect info about journals and whatnot this way:

journal_raw_output=`/p4/common/bin/p4master_run 1 /p4/1/bin/p4_1 pull -l -j`
archive_raw_output=`/p4/common/bin/p4master_run 1 /p4/1/bin/p4_1 pull -l -s`

Then parse that output into a bunch of data points:

journal_update=`echo $journal_raw_output | awk -v RS='' '{ print $25,$26 }' | sed 's/\.//' | xargs -n1 -I{} date +%s -d"{}"`
master_journal_update=`echo $journal_raw_output | awk -v RS='' '{ print $33,$34 }' | xargs -n1 -I{} date +%s -d"{}"`
journal_counter=`echo $journal_raw_output | awk '{ print $7 }' | sed 's/\,$//'`
master_journal_counter=`echo $journal_raw_output | awk '{ print $16 }' | sed 's/\,$//'`
journal_sequence=`echo $journal_raw_output | awk '{ print $9 }' | sed 's/\.$//'`
master_journal_sequence=`echo $journal_raw_output | awk '{ print $18 }' | sed 's/\.$//'`
active_archive_transfers=`echo $archive_raw_output | awk '{ print $3 }'`
queued_archive_transfers=`echo $archive_raw_output | awk '{ print $4 }' | sed 's/active\///'`
journal_update_delay=$(($master_journal_update-$journal_update))
journal_counter_offset=$(($master_journal_counter-$journal_counter))
journal_sequence_offset=$(($master_journal_sequence-$journal_sequence))


Uptime and p4d version are easily parseable from 'p4 info' output:

uptime=`/p4/common/bin/p4master_run 1 /p4/1/bin/p4_1 info | /bin/grep 'uptime' | /bin/awk '{print $3}'`
version=`/p4/common/bin/p4master_run 1 /p4/1/bin/p4_1 info | /bin/grep 'version' | /bin/awk '{print $3}'`


As well as license info from our commit server:

users=`/p4/common/bin/p4master_run 1 /p4/1/bin/p4_1 users | /usr/bin/wc -l`
licenses=`/p4/common/bin/p4master_run 1 /p4/1/bin/p4_1 info | /bin/grep 'Server license:' | /bin/awk '{print $6}'`


We don't collect any performance stats from Perforce in general, we're mostly concerned about that journal delay as well as general filesystem churning, CPU overload, number of p4d processes, stuff like that, but 'p4 -ztag info' might give some interesting stats to track. It's all Greek to me but maybe there's something useful in there, no idea. :)
-Matt Janulewicz
Staff SCM Engineer, Perforce Administrator
Dolby Laboratories, Inc.
1275 Market St.
San Francisco, CA 94103, USA
majanu@dolby.com

#3 Chris_M

Chris_M

    Newbie

  • Members
  • Pip
  • 2 posts

Posted 12 January 2018 - 03:35 PM

Realise this thread is a while old but...

P4Nagios is your friend.

https://swarm.worksh...tware-p4nagios/

The script contained in there is brilliant for keeping an eye on your Linux P4D hosts.  We also use monit (https://mmonit.com/) to capture and alert on CPU, RAM, child process counts etc.

Those parsing commands Matt shared are pretty good too (read as "I've half inched them" ;)

#4 P4Reg

P4Reg

    Advanced Member

  • Staff Moderators
  • 87 posts

Posted 12 January 2018 - 04:25 PM

Just adding a couple of extra comments here about parsing output (it won't add anything extra to what Matt said, but can make for shorter code)

The -F formatting option that p4 has can streamline a command and often removes the need to parse the standard output of p4 commands using sed/awk etc.

..So
uptime=`/p4/common/bin/p4master_run 1 /p4/1/bin/p4_1 info | /bin/grep 'uptime' | /bin/awk '{print $3}'


..shortens to:

uptime=`p4 -ztag -F %serverUptime% info`

..and
version=`/p4/common/bin/p4master_run 1 /p4/1/bin/p4_1 info | /bin/grep 'version' | /bin/awk '{print $3}'`


..shortens to

p4 -ztag -F %serverVersion% info

You can read more about the -F option in  https://www.perforce.../fun-formatting and in this Knowledge Base articleFormatting p4 command output using the -F global option with examples

Equally useful for simplified on the fly replacement of fields in forms is the --field client side option (rather than the more traditional "p4 user -o | sed "s/some item on the form/to something else/" | p4 user -i ) :

  https://www.perforce...nd-line-roundup






Also tagged with one or more of these keywords: datadog, server, monitor, linux

0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users