dom
/
telegraf
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity
telegraf/plugins/inputs/varnish
History
Lukasz Jagiello e3e6fa4f1d
Sudo support for ceph/varnish plugin 
In many default installations ceph and varnish binaries require root
permissions or specific user permissions to actually gather metrics.

Although it's possible to instrumented system configuration to either
run telegraf with root privileges or share the same user/permissions as
measured software it's not a default setup.

In my opinion ability to run binary with `sudo` makes entire setup
simpler without additional interaction.

My changes allow to use configuration like:
```
binary = "/usr/bin/sudo /usr/bin/varnishstat"
ceph_binary = "/usr/bin/sudo /usr/bin/ceph"
```

With a proper sudo configuration it will simplify amount of changes
needed to collect metrics.

I've also changed type of varnish metrics. It was `int` but in varnish source
code we can find all metrics are actually uint64
(https://github.com/varnishcache/varnish-cache/blob/master/include/tbl/vsc_f_main.h)

I hit that issue with some metrics:
```
~# varnishstat -1 | grep -i MAIN.n_objectcore
MAIN.n_objectcore      18446744073709550877          .   objectcore structs made
```
 		2016-05-25 23:49:42 +00:00
..
README.md Adding Varnish HTTP Cache input plugin (#1173) 2016-05-24 11:06:25 +01:00
varnish.go Sudo support for ceph/varnish plugin 2016-05-25 23:49:42 +00:00
varnish_test.go Adding Varnish HTTP Cache input plugin (#1173) 2016-05-24 11:06:25 +01:00
varnish_windows.go Adding Varnish HTTP Cache input plugin (#1173) 2016-05-24 11:06:25 +01:00

README.md

Varnish Input Plugin

This plugin gathers stats from Varnish HTTP Cache

Configuration:

 # A plugin to collect stats from Varnish HTTP Cache
 [[inputs.varnish]]
   ## The default location of the varnishstat binary can be overridden with:
   binary = "/usr/bin/varnishstat"

   ## By default, telegraf gathers stats for 3 metric points.
   ## Setting stats will override the defaults shown below.
   ## stats may also be set to ["all"], which will collect all stats
   stats = ["MAIN.cache_hit", "MAIN.cache_miss", "MAIN.uptime"]

Measurements & Fields:

This is the full list of stats provided by varnish. Stats will be grouped by their capitalized prefix (eg MAIN, MEMPOOL, etc). In the output, the prefix will be used as a tag, and removed from field names.

  • varnish
    • MAIN.uptime (int, count, Child process uptime)
    • MAIN.sess_conn (int, count, Sessions accepted)
    • MAIN.sess_drop (int, count, Sessions dropped)
    • MAIN.sess_fail (int, count, Session accept failures)
    • MAIN.sess_pipe_overflow (int, count, Session pipe overflow)
    • MAIN.client_req_400 (int, count, Client requests received,)
    • MAIN.client_req_411 (int, count, Client requests received,)
    • MAIN.client_req_413 (int, count, Client requests received,)
    • MAIN.client_req_417 (int, count, Client requests received,)
    • MAIN.client_req (int, count, Good client requests)
    • MAIN.cache_hit (int, count, Cache hits)
    • MAIN.cache_hitpass (int, count, Cache hits for)
    • MAIN.cache_miss (int, count, Cache misses)
    • MAIN.backend_conn (int, count, Backend conn. success)
    • MAIN.backend_unhealthy (int, count, Backend conn. not)
    • MAIN.backend_busy (int, count, Backend conn. too)
    • MAIN.backend_fail (int, count, Backend conn. failures)
    • MAIN.backend_reuse (int, count, Backend conn. reuses)
    • MAIN.backend_toolate (int, count, Backend conn. was)
    • MAIN.backend_recycle (int, count, Backend conn. recycles)
    • MAIN.backend_retry (int, count, Backend conn. retry)
    • MAIN.fetch_head (int, count, Fetch no body)
    • MAIN.fetch_length (int, count, Fetch with Length)
    • MAIN.fetch_chunked (int, count, Fetch chunked)
    • MAIN.fetch_eof (int, count, Fetch EOF)
    • MAIN.fetch_bad (int, count, Fetch bad T- E)
    • MAIN.fetch_close (int, count, Fetch wanted close)
    • MAIN.fetch_oldhttp (int, count, Fetch pre HTTP/1.1)
    • MAIN.fetch_zero (int, count, Fetch zero len)
    • MAIN.fetch_1xx (int, count, Fetch no body)
    • MAIN.fetch_204 (int, count, Fetch no body)
    • MAIN.fetch_304 (int, count, Fetch no body)
    • MAIN.fetch_failed (int, count, Fetch failed (all)
    • MAIN.fetch_no_thread (int, count, Fetch failed (no)
    • MAIN.pools (int, count, Number of thread)
    • MAIN.threads (int, count, Total number of)
    • MAIN.threads_limited (int, count, Threads hit max)
    • MAIN.threads_created (int, count, Threads created)
    • MAIN.threads_destroyed (int, count, Threads destroyed)
    • MAIN.threads_failed (int, count, Thread creation failed)
    • MAIN.thread_queue_len (int, count, Length of session)
    • MAIN.busy_sleep (int, count, Number of requests)
    • MAIN.busy_wakeup (int, count, Number of requests)
    • MAIN.sess_queued (int, count, Sessions queued for)
    • MAIN.sess_dropped (int, count, Sessions dropped for)
    • MAIN.n_object (int, count, object structs made)
    • MAIN.n_vampireobject (int, count, unresurrected objects)
    • MAIN.n_objectcore (int, count, objectcore structs made)
    • MAIN.n_objecthead (int, count, objecthead structs made)
    • MAIN.n_waitinglist (int, count, waitinglist structs made)
    • MAIN.n_backend (int, count, Number of backends)
    • MAIN.n_expired (int, count, Number of expired)
    • MAIN.n_lru_nuked (int, count, Number of LRU)
    • MAIN.n_lru_moved (int, count, Number of LRU)
    • MAIN.losthdr (int, count, HTTP header overflows)
    • MAIN.s_sess (int, count, Total sessions seen)
    • MAIN.s_req (int, count, Total requests seen)
    • MAIN.s_pipe (int, count, Total pipe sessions)
    • MAIN.s_pass (int, count, Total pass- ed requests)
    • MAIN.s_fetch (int, count, Total backend fetches)
    • MAIN.s_synth (int, count, Total synthethic responses)
    • MAIN.s_req_hdrbytes (int, count, Request header bytes)
    • MAIN.s_req_bodybytes (int, count, Request body bytes)
    • MAIN.s_resp_hdrbytes (int, count, Response header bytes)
    • MAIN.s_resp_bodybytes (int, count, Response body bytes)
    • MAIN.s_pipe_hdrbytes (int, count, Pipe request header)
    • MAIN.s_pipe_in (int, count, Piped bytes from)
    • MAIN.s_pipe_out (int, count, Piped bytes to)
    • MAIN.sess_closed (int, count, Session Closed)
    • MAIN.sess_pipeline (int, count, Session Pipeline)
    • MAIN.sess_readahead (int, count, Session Read Ahead)
    • MAIN.sess_herd (int, count, Session herd)
    • MAIN.shm_records (int, count, SHM records)
    • MAIN.shm_writes (int, count, SHM writes)
    • MAIN.shm_flushes (int, count, SHM flushes due)
    • MAIN.shm_cont (int, count, SHM MTX contention)
    • MAIN.shm_cycles (int, count, SHM cycles through)
    • MAIN.sms_nreq (int, count, SMS allocator requests)
    • MAIN.sms_nobj (int, count, SMS outstanding allocations)
    • MAIN.sms_nbytes (int, count, SMS outstanding bytes)
    • MAIN.sms_balloc (int, count, SMS bytes allocated)
    • MAIN.sms_bfree (int, count, SMS bytes freed)
    • MAIN.backend_req (int, count, Backend requests made)
    • MAIN.n_vcl (int, count, Number of loaded)
    • MAIN.n_vcl_avail (int, count, Number of VCLs)
    • MAIN.n_vcl_discard (int, count, Number of discarded)
    • MAIN.bans (int, count, Count of bans)
    • MAIN.bans_completed (int, count, Number of bans)
    • MAIN.bans_obj (int, count, Number of bans)
    • MAIN.bans_req (int, count, Number of bans)
    • MAIN.bans_added (int, count, Bans added)
    • MAIN.bans_deleted (int, count, Bans deleted)
    • MAIN.bans_tested (int, count, Bans tested against)
    • MAIN.bans_obj_killed (int, count, Objects killed by)
    • MAIN.bans_lurker_tested (int, count, Bans tested against)
    • MAIN.bans_tests_tested (int, count, Ban tests tested)
    • MAIN.bans_lurker_tests_tested (int, count, Ban tests tested)
    • MAIN.bans_lurker_obj_killed (int, count, Objects killed by)
    • MAIN.bans_dups (int, count, Bans superseded by)
    • MAIN.bans_lurker_contention (int, count, Lurker gave way)
    • MAIN.bans_persisted_bytes (int, count, Bytes used by)
    • MAIN.bans_persisted_fragmentation (int, count, Extra bytes in)
    • MAIN.n_purges (int, count, Number of purge)
    • MAIN.n_obj_purged (int, count, Number of purged)
    • MAIN.exp_mailed (int, count, Number of objects)
    • MAIN.exp_received (int, count, Number of objects)
    • MAIN.hcb_nolock (int, count, HCB Lookups without)
    • MAIN.hcb_lock (int, count, HCB Lookups with)
    • MAIN.hcb_insert (int, count, HCB Inserts)
    • MAIN.esi_errors (int, count, ESI parse errors)
    • MAIN.esi_warnings (int, count, ESI parse warnings)
    • MAIN.vmods (int, count, Loaded VMODs)
    • MAIN.n_gzip (int, count, Gzip operations)
    • MAIN.n_gunzip (int, count, Gunzip operations)
    • MAIN.vsm_free (int, count, Free VSM space)
    • MAIN.vsm_used (int, count, Used VSM space)
    • MAIN.vsm_cooling (int, count, Cooling VSM space)
    • MAIN.vsm_overflow (int, count, Overflow VSM space)
    • MAIN.vsm_overflowed (int, count, Overflowed VSM space)
    • MGT.uptime (int, count, Management process uptime)
    • MGT.child_start (int, count, Child process started)
    • MGT.child_exit (int, count, Child process normal)
    • MGT.child_stop (int, count, Child process unexpected)
    • MGT.child_died (int, count, Child process died)
    • MGT.child_dump (int, count, Child process core)
    • MGT.child_panic (int, count, Child process panic)
    • MEMPOOL.vbc.live (int, count, In use)
    • MEMPOOL.vbc.pool (int, count, In Pool)
    • MEMPOOL.vbc.sz_wanted (int, count, Size requested)
    • MEMPOOL.vbc.sz_needed (int, count, Size allocated)
    • MEMPOOL.vbc.allocs (int, count, Allocations )
    • MEMPOOL.vbc.frees (int, count, Frees )
    • MEMPOOL.vbc.recycle (int, count, Recycled from pool)
    • MEMPOOL.vbc.timeout (int, count, Timed out from)
    • MEMPOOL.vbc.toosmall (int, count, Too small to)
    • MEMPOOL.vbc.surplus (int, count, Too many for)
    • MEMPOOL.vbc.randry (int, count, Pool ran dry)
    • MEMPOOL.busyobj.live (int, count, In use)
    • MEMPOOL.busyobj.pool (int, count, In Pool)
    • MEMPOOL.busyobj.sz_wanted (int, count, Size requested)
    • MEMPOOL.busyobj.sz_needed (int, count, Size allocated)
    • MEMPOOL.busyobj.allocs (int, count, Allocations )
    • MEMPOOL.busyobj.frees (int, count, Frees )
    • MEMPOOL.busyobj.recycle (int, count, Recycled from pool)
    • MEMPOOL.busyobj.timeout (int, count, Timed out from)
    • MEMPOOL.busyobj.toosmall (int, count, Too small to)
    • MEMPOOL.busyobj.surplus (int, count, Too many for)
    • MEMPOOL.busyobj.randry (int, count, Pool ran dry)
    • MEMPOOL.req0.live (int, count, In use)
    • MEMPOOL.req0.pool (int, count, In Pool)
    • MEMPOOL.req0.sz_wanted (int, count, Size requested)
    • MEMPOOL.req0.sz_needed (int, count, Size allocated)
    • MEMPOOL.req0.allocs (int, count, Allocations )
    • MEMPOOL.req0.frees (int, count, Frees )
    • MEMPOOL.req0.recycle (int, count, Recycled from pool)
    • MEMPOOL.req0.timeout (int, count, Timed out from)
    • MEMPOOL.req0.toosmall (int, count, Too small to)
    • MEMPOOL.req0.surplus (int, count, Too many for)
    • MEMPOOL.req0.randry (int, count, Pool ran dry)
    • MEMPOOL.sess0.live (int, count, In use)
    • MEMPOOL.sess0.pool (int, count, In Pool)
    • MEMPOOL.sess0.sz_wanted (int, count, Size requested)
    • MEMPOOL.sess0.sz_needed (int, count, Size allocated)
    • MEMPOOL.sess0.allocs (int, count, Allocations )
    • MEMPOOL.sess0.frees (int, count, Frees )
    • MEMPOOL.sess0.recycle (int, count, Recycled from pool)
    • MEMPOOL.sess0.timeout (int, count, Timed out from)
    • MEMPOOL.sess0.toosmall (int, count, Too small to)
    • MEMPOOL.sess0.surplus (int, count, Too many for)
    • MEMPOOL.sess0.randry (int, count, Pool ran dry)
    • MEMPOOL.req1.live (int, count, In use)
    • MEMPOOL.req1.pool (int, count, In Pool)
    • MEMPOOL.req1.sz_wanted (int, count, Size requested)
    • MEMPOOL.req1.sz_needed (int, count, Size allocated)
    • MEMPOOL.req1.allocs (int, count, Allocations )
    • MEMPOOL.req1.frees (int, count, Frees )
    • MEMPOOL.req1.recycle (int, count, Recycled from pool)
    • MEMPOOL.req1.timeout (int, count, Timed out from)
    • MEMPOOL.req1.toosmall (int, count, Too small to)
    • MEMPOOL.req1.surplus (int, count, Too many for)
    • MEMPOOL.req1.randry (int, count, Pool ran dry)
    • MEMPOOL.sess1.live (int, count, In use)
    • MEMPOOL.sess1.pool (int, count, In Pool)
    • MEMPOOL.sess1.sz_wanted (int, count, Size requested)
    • MEMPOOL.sess1.sz_needed (int, count, Size allocated)
    • MEMPOOL.sess1.allocs (int, count, Allocations )
    • MEMPOOL.sess1.frees (int, count, Frees )
    • MEMPOOL.sess1.recycle (int, count, Recycled from pool)
    • MEMPOOL.sess1.timeout (int, count, Timed out from)
    • MEMPOOL.sess1.toosmall (int, count, Too small to)
    • MEMPOOL.sess1.surplus (int, count, Too many for)
    • MEMPOOL.sess1.randry (int, count, Pool ran dry)
    • SMA.s0.c_req (int, count, Allocator requests)
    • SMA.s0.c_fail (int, count, Allocator failures)
    • SMA.s0.c_bytes (int, count, Bytes allocated)
    • SMA.s0.c_freed (int, count, Bytes freed)
    • SMA.s0.g_alloc (int, count, Allocations outstanding)
    • SMA.s0.g_bytes (int, count, Bytes outstanding)
    • SMA.s0.g_space (int, count, Bytes available)
    • SMA.Transient.c_req (int, count, Allocator requests)
    • SMA.Transient.c_fail (int, count, Allocator failures)
    • SMA.Transient.c_bytes (int, count, Bytes allocated)
    • SMA.Transient.c_freed (int, count, Bytes freed)
    • SMA.Transient.g_alloc (int, count, Allocations outstanding)
    • SMA.Transient.g_bytes (int, count, Bytes outstanding)
    • SMA.Transient.g_space (int, count, Bytes available)
    • VBE.default(127.0.0.1,,8080).vcls (int, count, VCL references)
    • VBE.default(127.0.0.1,,8080).happy (int, count, Happy health probes)
    • VBE.default(127.0.0.1,,8080).bereq_hdrbytes (int, count, Request header bytes)
    • VBE.default(127.0.0.1,,8080).bereq_bodybytes (int, count, Request body bytes)
    • VBE.default(127.0.0.1,,8080).beresp_hdrbytes (int, count, Response header bytes)
    • VBE.default(127.0.0.1,,8080).beresp_bodybytes (int, count, Response body bytes)
    • VBE.default(127.0.0.1,,8080).pipe_hdrbytes (int, count, Pipe request header)
    • VBE.default(127.0.0.1,,8080).pipe_out (int, count, Piped bytes to)
    • VBE.default(127.0.0.1,,8080).pipe_in (int, count, Piped bytes from)
    • LCK.sms.creat (int, count, Created locks)
    • LCK.sms.destroy (int, count, Destroyed locks)
    • LCK.sms.locks (int, count, Lock Operations)
    • LCK.smp.creat (int, count, Created locks)
    • LCK.smp.destroy (int, count, Destroyed locks)
    • LCK.smp.locks (int, count, Lock Operations)
    • LCK.sma.creat (int, count, Created locks)
    • LCK.sma.destroy (int, count, Destroyed locks)
    • LCK.sma.locks (int, count, Lock Operations)
    • LCK.smf.creat (int, count, Created locks)
    • LCK.smf.destroy (int, count, Destroyed locks)
    • LCK.smf.locks (int, count, Lock Operations)
    • LCK.hsl.creat (int, count, Created locks)
    • LCK.hsl.destroy (int, count, Destroyed locks)
    • LCK.hsl.locks (int, count, Lock Operations)
    • LCK.hcb.creat (int, count, Created locks)
    • LCK.hcb.destroy (int, count, Destroyed locks)
    • LCK.hcb.locks (int, count, Lock Operations)
    • LCK.hcl.creat (int, count, Created locks)
    • LCK.hcl.destroy (int, count, Destroyed locks)
    • LCK.hcl.locks (int, count, Lock Operations)
    • LCK.vcl.creat (int, count, Created locks)
    • LCK.vcl.destroy (int, count, Destroyed locks)
    • LCK.vcl.locks (int, count, Lock Operations)
    • LCK.sessmem.creat (int, count, Created locks)
    • LCK.sessmem.destroy (int, count, Destroyed locks)
    • LCK.sessmem.locks (int, count, Lock Operations)
    • LCK.sess.creat (int, count, Created locks)
    • LCK.sess.destroy (int, count, Destroyed locks)
    • LCK.sess.locks (int, count, Lock Operations)
    • LCK.wstat.creat (int, count, Created locks)
    • LCK.wstat.destroy (int, count, Destroyed locks)
    • LCK.wstat.locks (int, count, Lock Operations)
    • LCK.herder.creat (int, count, Created locks)
    • LCK.herder.destroy (int, count, Destroyed locks)
    • LCK.herder.locks (int, count, Lock Operations)
    • LCK.wq.creat (int, count, Created locks)
    • LCK.wq.destroy (int, count, Destroyed locks)
    • LCK.wq.locks (int, count, Lock Operations)
    • LCK.objhdr.creat (int, count, Created locks)
    • LCK.objhdr.destroy (int, count, Destroyed locks)
    • LCK.objhdr.locks (int, count, Lock Operations)
    • LCK.exp.creat (int, count, Created locks)
    • LCK.exp.destroy (int, count, Destroyed locks)
    • LCK.exp.locks (int, count, Lock Operations)
    • LCK.lru.creat (int, count, Created locks)
    • LCK.lru.destroy (int, count, Destroyed locks)
    • LCK.lru.locks (int, count, Lock Operations)
    • LCK.cli.creat (int, count, Created locks)
    • LCK.cli.destroy (int, count, Destroyed locks)
    • LCK.cli.locks (int, count, Lock Operations)
    • LCK.ban.creat (int, count, Created locks)
    • LCK.ban.destroy (int, count, Destroyed locks)
    • LCK.ban.locks (int, count, Lock Operations)
    • LCK.vbp.creat (int, count, Created locks)
    • LCK.vbp.destroy (int, count, Destroyed locks)
    • LCK.vbp.locks (int, count, Lock Operations)
    • LCK.backend.creat (int, count, Created locks)
    • LCK.backend.destroy (int, count, Destroyed locks)
    • LCK.backend.locks (int, count, Lock Operations)
    • LCK.vcapace.creat (int, count, Created locks)
    • LCK.vcapace.destroy (int, count, Destroyed locks)
    • LCK.vcapace.locks (int, count, Lock Operations)
    • LCK.nbusyobj.creat (int, count, Created locks)
    • LCK.nbusyobj.destroy (int, count, Destroyed locks)
    • LCK.nbusyobj.locks (int, count, Lock Operations)
    • LCK.busyobj.creat (int, count, Created locks)
    • LCK.busyobj.destroy (int, count, Destroyed locks)
    • LCK.busyobj.locks (int, count, Lock Operations)
    • LCK.mempool.creat (int, count, Created locks)
    • LCK.mempool.destroy (int, count, Destroyed locks)
    • LCK.mempool.locks (int, count, Lock Operations)
    • LCK.vxid.creat (int, count, Created locks)
    • LCK.vxid.destroy (int, count, Destroyed locks)
    • LCK.vxid.locks (int, count, Lock Operations)
    • LCK.pipestat.creat (int, count, Created locks)
    • LCK.pipestat.destroy (int, count, Destroyed locks)
    • LCK.pipestat.locks (int, count, Lock Operations)

Tags:

As indicated above, the prefix of a varnish stat will be used as it's 'section' tag. So section tag may have one of the following values:

  • section:
    • MAIN
    • MGT
    • MEMPOOL
    • SMA
    • VBE
    • LCK

Example Output:

 telegraf -test -config etc/telegraf.conf  -input-filter varnish
* Plugin: varnish, Collection 1
> varnish,host=rpercy-VirtualBox,section=MAIN cache_hit=0i,cache_miss=0i,uptime=8416i 1462765437090957980