Add phusion Passenger plugin

Gather metric by parsing XMLoutput of `passenger-status` utility.
More information of this utility:
https://www.phusionpassenger.com/library/admin/apache/overall_status_report.html

closes #522
This commit is contained in:
Vinh 2016-01-14 17:09:38 -08:00 committed by Cameron Sparr
parent 551db20657
commit 1388b1b58b
7 changed files with 693 additions and 0 deletions

View File

@ -10,6 +10,7 @@
- [#494](https://github.com/influxdata/telegraf/pull/494): Graphite output plugin. Thanks @titilambert!
- AMQP SSL support. Thanks @ekini!
- [#539](https://github.com/influxdata/telegraf/pull/539): Reload config on SIGHUP. Thanks @titilambert!
- [#522](https://github.com/influxdata/telegraf/pull/522): Phusion passenger input plugin. Thanks @kureikain!
### Bugfixes
- [#506](https://github.com/influxdb/telegraf/pull/506): Ping input doesn't return response time metric when timeout. Thanks @titilambert!

1
Godeps
View File

@ -46,6 +46,7 @@ github.com/wvanbergen/kafka 1a8639a45164fcc245d5c7b4bd3ccfbd1a0ffbf3
github.com/wvanbergen/kazoo-go 0f768712ae6f76454f987c3356177e138df258f8
golang.org/x/crypto 3760e016850398b85094c4c99e955b8c3dea5711
golang.org/x/net 99ca920b6037ef77af8a11297150f7f0d8f4ef80
golang.org/x/text cf4986612c83df6c55578ba198316d1684a9a287
gopkg.in/dancannon/gorethink.v1 e2cef022d0495329dfb0635991de76efcab5cf50
gopkg.in/fatih/pool.v2 cba550ebf9bce999a02e963296d4bc7a486cb715
gopkg.in/mgo.v2 e30de8ac9ae3b30df7065f766c71f88bba7d4e49

View File

@ -154,6 +154,7 @@ Currently implemented sources:
* nginx
* nsq
* phpfpm
* phusion passenger
* ping
* postgresql
* procstat

View File

@ -20,6 +20,7 @@ import (
_ "github.com/influxdb/telegraf/plugins/inputs/mysql"
_ "github.com/influxdb/telegraf/plugins/inputs/nginx"
_ "github.com/influxdb/telegraf/plugins/inputs/nsq"
_ "github.com/influxdb/telegraf/plugins/inputs/passenger"
_ "github.com/influxdb/telegraf/plugins/inputs/phpfpm"
_ "github.com/influxdb/telegraf/plugins/inputs/ping"
_ "github.com/influxdb/telegraf/plugins/inputs/postgresql"

View File

@ -0,0 +1,138 @@
# Telegraf plugin: passenger
Get phusion passenger stat using their command line utility
`passenger-status`
# Measurements
Meta:
- tags:
* name
* passenger_version
* pid
* code_revision
Measurement names:
- passenger:
* Tags: `passenger_version`
* Fields:
- process_count
- max
- capacity_used
- get_wait_list_size
- passenger_supergroup:
* Tags: `name`
* Fields:
- get_wait_list_size
- capacity_used
- passenger_group:
* Tags:
- name
- app_root
- app_type
* Fields:
- get_wait_list_size
- capacity_used
- processes_being_spawned
- passenger_process:
* Tags:
- group_name
- app_root
- supergroup_name
- pid
- code_revision
- life_status
- process_group_id
* Field:
- concurrency
- sessions
- busyness
- processed
- spawner_creation_time
- spawn_start_time
- spawn_end_time
- last_used
- uptime
- cpu
- rss
- pss
- private_dirty
- swap
- real_memory
- vmsize
# Example output
Using this configuration:
```
[[inputs.passenger]]
# Path of passenger-status.
#
# Plugin gather metric via parsing XML output of passenger-status
# More information about the tool:
# https://www.phusionpassenger.com/library/admin/apache/overall_status_report.html
#
#
# If no path is specified, then the plugin simply execute passenger-status
# hopefully it can be found in your PATH
command = "passenger-status -v --show=xml"
```
When run with:
```
./telegraf -config telegraf.conf -test -input-filter passenger
```
It produces:
```
> passenger,passenger_version=5.0.17 capacity_used=23i,get_wait_list_size=0i,max=23i,process_count=23i 1452984112799414257
> passenger_supergroup,name=/var/app/current/public capacity_used=23i,get_wait_list_size=0i 1452984112799496977
> passenger_group,app_root=/var/app/current,app_type=rack,name=/var/app/current/public capacity_used=23i,get_wait_list_size=0i,processes_being_spawned=0i 1452984112799527021
> passenger_process,app_root=/var/app/current,code_revision=899ac7f,group_name=/var/app/current/public,life_status=ALIVE,pid=11553,process_group_id=13608,supergroup_name=/var/app/current/public busyness=0i,concurrency=1i,cpu=58i,last_used=1452747071764940i,private_dirty=314900i,processed=951i,pss=319391i,real_memory=314900i,rss=418548i,sessions=0i,spawn_end_time=1452746845013365i,spawn_start_time=1452746844946982i,spawner_creation_time=1452746835922747i,swap=0i,uptime=226i,vmsize=1563580i 1452984112799571490
> passenger_process,app_root=/var/app/current,code_revision=899ac7f,group_name=/var/app/current/public,life_status=ALIVE,pid=11563,process_group_id=13608,supergroup_name=/var/app/current/public busyness=2147483647i,concurrency=1i,cpu=47i,last_used=1452747071709179i,private_dirty=309240i,processed=756i,pss=314036i,real_memory=309240i,rss=418296i,sessions=1i,spawn_end_time=1452746845172460i,spawn_start_time=1452746845136882i,spawner_creation_time=1452746835922747i,swap=0i,uptime=226i,vmsize=1563608i 1452984112799638581
```
# Note
You have to ensure that you can run the `passenger-status` command under
telegraf user. Depend on how you install and configure passenger, this
maybe an issue for you. If you are using passenger standlone, or compile
yourself, it is straight forward. However, if you are using gem and
`rvm`, it maybe harder to get this right.
Such as with `rvm`, you can use this command:
```
~/.rvm/bin/rvm default do passenger-status -v --show=xml
```
You can use `&` and `;` in the shell command to run comlicated shell command
in order to get the passenger-status such as load the rvm shell, source the
path
```
command = "source .rvm/scripts/rvm && passenger-status -v --show=xml"
```
Anyway, just ensure that you can run the command under `telegraf` user, and it
has to produce XML output.

View File

@ -0,0 +1,250 @@
package passenger
import (
"bytes"
"encoding/xml"
"fmt"
"os/exec"
"strconv"
"strings"
"github.com/influxdb/telegraf/plugins/inputs"
"golang.org/x/net/html/charset"
)
type passenger struct {
Command string
}
func (p *passenger) parseCommand() (string, []string) {
var arguments []string
if !strings.Contains(p.Command, " ") {
return p.Command, arguments
}
arguments = strings.Split(p.Command, " ")
if len(arguments) == 1 {
return arguments[0], arguments[1:]
}
return arguments[0], arguments[1:]
}
type info struct {
Passenger_version string `xml:"passenger_version"`
Process_count int `xml:"process_count"`
Capacity_used int `xml:"capacity_used"`
Get_wait_list_size int `xml:"get_wait_list_size"`
Max int `xml:"max"`
Supergroups struct {
Supergroup []struct {
Name string `xml:"name"`
Get_wait_list_size int `xml:"get_wait_list_size"`
Capacity_used int `xml:"capacity_used"`
Group []struct {
Name string `xml:"name"`
AppRoot string `xml:"app_root"`
AppType string `xml:"app_type"`
Enabled_process_count int `xml:"enabled_process_count"`
Disabling_process_count int `xml:"disabling_process_count"`
Disabled_process_count int `xml:"disabled_process_count"`
Capacity_used int `xml:"capacity_used"`
Get_wait_list_size int `xml:"get_wait_list_size"`
Processes_being_spawned int `xml:"processes_being_spawned"`
Processes struct {
Process []*process `xml:"process"`
} `xml:"processes"`
} `xml:"group"`
} `xml:"supergroup"`
} `xml:"supergroups"`
}
type process struct {
Pid int `xml:"pid"`
Concurrency int `xml:"concurrency"`
Sessions int `xml:"sessions"`
Busyness int `xml:"busyness"`
Processed int `xml:"processed"`
Spawner_creation_time int64 `xml:"spawner_creation_time"`
Spawn_start_time int64 `xml:"spawn_start_time"`
Spawn_end_time int64 `xml:"spawn_end_time"`
Last_used int64 `xml:"last_used"`
Uptime string `xml:"uptime"`
Code_revision string `xml:"code_revision"`
Life_status string `xml:"life_status"`
Enabled string `xml:"enabled"`
Has_metrics bool `xml:"has_metrics"`
Cpu int64 `xml:"cpu"`
Rss int64 `xml:"rss"`
Pss int64 `xml:"pss"`
Private_dirty int64 `xml:"private_dirty"`
Swap int64 `xml:"swap"`
Real_memory int64 `xml:"real_memory"`
Vmsize int64 `xml:"vmsize"`
Process_group_id string `xml:"process_group_id"`
}
func (p *process) getUptime() int64 {
if p.Uptime == "" {
return 0
}
timeSlice := strings.Split(p.Uptime, " ")
var uptime int64
uptime = 0
for _, v := range timeSlice {
switch {
case strings.HasSuffix(v, "d"):
iValue := strings.TrimSuffix(v, "d")
value, err := strconv.ParseInt(iValue, 10, 64)
if err == nil {
uptime += value * (24 * 60 * 60)
}
case strings.HasSuffix(v, "h"):
iValue := strings.TrimSuffix(v, "y")
value, err := strconv.ParseInt(iValue, 10, 64)
if err == nil {
uptime += value * (60 * 60)
}
case strings.HasSuffix(v, "m"):
iValue := strings.TrimSuffix(v, "m")
value, err := strconv.ParseInt(iValue, 10, 64)
if err == nil {
uptime += value * 60
}
case strings.HasSuffix(v, "s"):
iValue := strings.TrimSuffix(v, "s")
value, err := strconv.ParseInt(iValue, 10, 64)
if err == nil {
uptime += value
}
}
}
return uptime
}
var sampleConfig = `
# Path of passenger-status.
#
# Plugin gather metric via parsing XML output of passenger-status
# More information about the tool:
# https://www.phusionpassenger.com/library/admin/apache/overall_status_report.html
#
#
# If no path is specified, then the plugin simply execute passenger-status
# hopefully it can be found in your PATH
command = "passenger-status -v --show=xml"
`
func (r *passenger) SampleConfig() string {
return sampleConfig
}
func (r *passenger) Description() string {
return "Read metrics of passenger using passenger-status"
}
func (g *passenger) Gather(acc inputs.Accumulator) error {
if g.Command == "" {
g.Command = "passenger-status -v --show=xml"
}
cmd, args := g.parseCommand()
out, err := exec.Command(cmd, args...).Output()
if err != nil {
return err
}
if err = importMetric(out, acc); err != nil {
return err
}
return nil
}
func importMetric(stat []byte, acc inputs.Accumulator) error {
var p info
decoder := xml.NewDecoder(bytes.NewReader(stat))
decoder.CharsetReader = charset.NewReaderLabel
if err := decoder.Decode(&p); err != nil {
return fmt.Errorf("Cannot parse input with error: %v\n", err)
}
tags := map[string]string{
"passenger_version": p.Passenger_version,
}
fields := map[string]interface{}{
"process_count": p.Process_count,
"max": p.Max,
"capacity_used": p.Capacity_used,
"get_wait_list_size": p.Get_wait_list_size,
}
acc.AddFields("passenger", fields, tags)
for _, sg := range p.Supergroups.Supergroup {
tags := map[string]string{
"name": sg.Name,
}
fields := map[string]interface{}{
"get_wait_list_size": sg.Get_wait_list_size,
"capacity_used": sg.Capacity_used,
}
acc.AddFields("passenger_supergroup", fields, tags)
for _, group := range sg.Group {
tags := map[string]string{
"name": group.Name,
"app_root": group.AppRoot,
"app_type": group.AppType,
}
fields := map[string]interface{}{
"get_wait_list_size": group.Get_wait_list_size,
"capacity_used": group.Capacity_used,
"processes_being_spawned": group.Processes_being_spawned,
}
acc.AddFields("passenger_group", fields, tags)
for _, process := range group.Processes.Process {
tags := map[string]string{
"group_name": group.Name,
"app_root": group.AppRoot,
"supergroup_name": sg.Name,
"pid": fmt.Sprintf("%d", process.Pid),
"code_revision": process.Code_revision,
"life_status": process.Life_status,
"process_group_id": process.Process_group_id,
}
fields := map[string]interface{}{
"concurrency": process.Concurrency,
"sessions": process.Sessions,
"busyness": process.Busyness,
"processed": process.Processed,
"spawner_creation_time": process.Spawner_creation_time,
"spawn_start_time": process.Spawn_start_time,
"spawn_end_time": process.Spawn_end_time,
"last_used": process.Last_used,
"uptime": process.getUptime(),
"cpu": process.Cpu,
"rss": process.Rss,
"pss": process.Pss,
"private_dirty": process.Private_dirty,
"swap": process.Swap,
"real_memory": process.Real_memory,
"vmsize": process.Vmsize,
}
acc.AddFields("passenger_process", fields, tags)
}
}
}
return nil
}
func init() {
inputs.Add("passenger", func() inputs.Input {
return &passenger{}
})
}

View File

@ -0,0 +1,301 @@
package passenger
import (
"fmt"
"io/ioutil"
"os"
"testing"
"github.com/influxdb/telegraf/testutil"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
func fakePassengerStatus(stat string) {
content := fmt.Sprintf("#!/bin/sh\ncat << EOF\n%s\nEOF", stat)
ioutil.WriteFile("/tmp/passenger-status", []byte(content), 0700)
}
func teardown() {
os.Remove("/tmp/passenger-status")
}
func Test_Invalid_Passenger_Status_Cli(t *testing.T) {
r := &passenger{
Command: "an-invalid-command passenger-status",
}
var acc testutil.Accumulator
err := r.Gather(&acc)
require.Error(t, err)
assert.Equal(t, err.Error(), `exec: "an-invalid-command": executable file not found in $PATH`)
}
func Test_Invalid_Xml(t *testing.T) {
fakePassengerStatus("invalid xml")
defer teardown()
r := &passenger{
Command: "/tmp/passenger-status",
}
var acc testutil.Accumulator
err := r.Gather(&acc)
require.Error(t, err)
assert.Equal(t, err.Error(), "Cannot parse input with error: EOF\n")
}
// We test this by ensure that the error message match the path of default cli
func Test_Default_Config_Load_Default_Command(t *testing.T) {
fakePassengerStatus("invalid xml")
defer teardown()
r := &passenger{}
var acc testutil.Accumulator
err := r.Gather(&acc)
require.Error(t, err)
assert.Equal(t, err.Error(), "exec: \"passenger-status\": executable file not found in $PATH")
}
func TestPassengerGenerateMetric(t *testing.T) {
fakePassengerStatus(sampleStat)
defer teardown()
//Now we tested again above server, with our authentication data
r := &passenger{
Command: "/tmp/passenger-status",
}
var acc testutil.Accumulator
err := r.Gather(&acc)
require.NoError(t, err)
tags := map[string]string{
"passenger_version": "5.0.17",
}
fields := map[string]interface{}{
"process_count": 23,
"max": 23,
"capacity_used": 23,
"get_wait_list_size": 3,
}
acc.AssertContainsTaggedFields(t, "passenger", fields, tags)
tags = map[string]string{
"name": "/var/app/current/public",
"app_root": "/var/app/current",
"app_type": "rack",
}
fields = map[string]interface{}{
"processes_being_spawned": 2,
"capacity_used": 23,
"get_wait_list_size": 3,
}
acc.AssertContainsTaggedFields(t, "passenger_group", fields, tags)
tags = map[string]string{
"name": "/var/app/current/public",
}
fields = map[string]interface{}{
"capacity_used": 23,
"get_wait_list_size": 3,
}
acc.AssertContainsTaggedFields(t, "passenger_supergroup", fields, tags)
tags = map[string]string{
"app_root": "/var/app/current",
"group_name": "/var/app/current/public",
"supergroup_name": "/var/app/current/public",
"pid": "11553",
"code_revision": "899ac7f",
"life_status": "ALIVE",
"process_group_id": "13608",
}
fields = map[string]interface{}{
"concurrency": 1,
"sessions": 0,
"busyness": 0,
"processed": 951,
"spawner_creation_time": int64(1452746835922747),
"spawn_start_time": int64(1452746844946982),
"spawn_end_time": int64(1452746845013365),
"last_used": int64(1452747071764940),
"uptime": int64(226), // in seconds of 3m 46s
"cpu": int64(58),
"rss": int64(418548),
"pss": int64(319391),
"private_dirty": int64(314900),
"swap": int64(0),
"real_memory": int64(314900),
"vmsize": int64(1563580),
}
acc.AssertContainsTaggedFields(t, "passenger_process", fields, tags)
}
var sampleStat = `
<?xml version="1.0" encoding="iso8859-1" ?>
<?xml version="1.0" encoding="UTF-8"?>
<info version="3">
<passenger_version>5.0.17</passenger_version>
<group_count>1</group_count>
<process_count>23</process_count>
<max>23</max>
<capacity_used>23</capacity_used>
<get_wait_list_size>3</get_wait_list_size>
<get_wait_list />
<supergroups>
<supergroup>
<name>/var/app/current/public</name>
<state>READY</state>
<get_wait_list_size>3</get_wait_list_size>
<capacity_used>23</capacity_used>
<secret>foo</secret>
<group default="true">
<name>/var/app/current/public</name>
<component_name>/var/app/current/public</component_name>
<app_root>/var/app/current</app_root>
<app_type>rack</app_type>
<environment>production</environment>
<uuid>QQUrbCVYxbJYpfgyDOwJ</uuid>
<enabled_process_count>23</enabled_process_count>
<disabling_process_count>0</disabling_process_count>
<disabled_process_count>0</disabled_process_count>
<capacity_used>23</capacity_used>
<get_wait_list_size>3</get_wait_list_size>
<disable_wait_list_size>0</disable_wait_list_size>
<processes_being_spawned>2</processes_being_spawned>
<secret>foo</secret>
<api_key>foo</api_key>
<life_status>ALIVE</life_status>
<user>axcoto</user>
<uid>1001</uid>
<group>axcoto</group>
<gid>1001</gid>
<options>
<app_root>/var/app/current</app_root>
<app_group_name>/var/app/current/public</app_group_name>
<app_type>rack</app_type>
<start_command>/var/app/.rvm/gems/ruby-2.2.0-p645/gems/passenger-5.0.17/helper-scripts/rack-loader.rb</start_command>
<startup_file>config.ru</startup_file>
<process_title>Passenger RubyApp</process_title>
<log_level>3</log_level>
<start_timeout>90000</start_timeout>
<environment>production</environment>
<base_uri>/</base_uri>
<spawn_method>smart</spawn_method>
<default_user>nobody</default_user>
<default_group>nogroup</default_group>
<ruby>/var/app/.rvm/gems/ruby-2.2.0-p645/wrappers/ruby</ruby>
<python>python</python>
<nodejs>node</nodejs>
<ust_router_address>unix:/tmp/passenger.eKFdvdC/agents.s/ust_router</ust_router_address>
<ust_router_username>logging</ust_router_username>
<ust_router_password>foo</ust_router_password>
<debugger>false</debugger>
<analytics>false</analytics>
<api_key>foo</api_key>
<min_processes>22</min_processes>
<max_processes>0</max_processes>
<max_preloader_idle_time>300</max_preloader_idle_time>
<max_out_of_band_work_instances>1</max_out_of_band_work_instances>
</options>
<processes>
<process>
<pid>11553</pid>
<sticky_session_id>378579907</sticky_session_id>
<gupid>17173df-PoNT3J9HCf</gupid>
<concurrency>1</concurrency>
<sessions>0</sessions>
<busyness>0</busyness>
<processed>951</processed>
<spawner_creation_time>1452746835922747</spawner_creation_time>
<spawn_start_time>1452746844946982</spawn_start_time>
<spawn_end_time>1452746845013365</spawn_end_time>
<last_used>1452747071764940</last_used>
<last_used_desc>0s ago</last_used_desc>
<uptime>3m 46s</uptime>
<code_revision>899ac7f</code_revision>
<life_status>ALIVE</life_status>
<enabled>ENABLED</enabled>
<has_metrics>true</has_metrics>
<cpu>58</cpu>
<rss>418548</rss>
<pss>319391</pss>
<private_dirty>314900</private_dirty>
<swap>0</swap>
<real_memory>314900</real_memory>
<vmsize>1563580</vmsize>
<process_group_id>13608</process_group_id>
<command>Passenger RubyApp: /var/app/current/public</command>
<sockets>
<socket>
<name>main</name>
<address>unix:/tmp/passenger.eKFdvdC/apps.s/ruby.UWF6zkRJ71aoMXPxpknpWVfC1POFqgWZzbEsdz5v0G46cSSMxJ3GHLFhJaUrK2I</address>
<protocol>session</protocol>
<concurrency>1</concurrency>
<sessions>0</sessions>
</socket>
<socket>
<name>http</name>
<address>tcp://127.0.0.1:49888</address>
<protocol>http</protocol>
<concurrency>1</concurrency>
<sessions>0</sessions>
</socket>
</sockets>
</process>
<process>
<pid>11563</pid>
<sticky_session_id>1549681201</sticky_session_id>
<gupid>17173df-pX5iJOipd8</gupid>
<concurrency>1</concurrency>
<sessions>1</sessions>
<busyness>2147483647</busyness>
<processed>756</processed>
<spawner_creation_time>1452746835922747</spawner_creation_time>
<spawn_start_time>1452746845136882</spawn_start_time>
<spawn_end_time>1452746845172460</spawn_end_time>
<last_used>1452747071709179</last_used>
<last_used_desc>0s ago</last_used_desc>
<uptime>3m 46s</uptime>
<code_revision>899ac7f</code_revision>
<life_status>ALIVE</life_status>
<enabled>ENABLED</enabled>
<has_metrics>true</has_metrics>
<cpu>47</cpu>
<rss>418296</rss>
<pss>314036</pss>
<private_dirty>309240</private_dirty>
<swap>0</swap>
<real_memory>309240</real_memory>
<vmsize>1563608</vmsize>
<process_group_id>13608</process_group_id>
<command>Passenger RubyApp: /var/app/current/public</command>
<sockets>
<socket>
<name>main</name>
<address>unix:/tmp/passenger.eKFdvdC/apps.s/ruby.PVCh7TmvCi9knqhba2vG5qXrlHGEIwhGrxnUvRbIAD6SPz9m0G7YlJ8HEsREHY3</address>
<protocol>session</protocol>
<concurrency>1</concurrency>
<sessions>1</sessions>
</socket>
<socket>
<name>http</name>
<address>tcp://127.0.0.1:52783</address>
<protocol>http</protocol>
<concurrency>1</concurrency>
<sessions>0</sessions>
</socket>
</sockets>
</process>
</processes>
</group>
</supergroup>
</supergroups>
</info>`