Compare commits
1 Commits
1.0.0-beta
...
0.13.2
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
359d617b8f |
4
.gitattributes
vendored
4
.gitattributes
vendored
@@ -1,4 +1,2 @@
|
||||
CHANGELOG.md merge=union
|
||||
README.md merge=union
|
||||
plugins/inputs/all/all.go merge=union
|
||||
plugins/outputs/all/all.go merge=union
|
||||
|
||||
|
||||
2
.github/ISSUE_TEMPLATE.md
vendored
2
.github/ISSUE_TEMPLATE.md
vendored
@@ -11,8 +11,6 @@ Erase the other section and everything on and above this line.
|
||||
|
||||
## Bug report
|
||||
|
||||
### Relevant telegraf.conf:
|
||||
|
||||
### System info:
|
||||
|
||||
[Include Telegraf version, operating system name, and other relevant details]
|
||||
|
||||
77
CHANGELOG.md
77
CHANGELOG.md
@@ -1,79 +1,4 @@
|
||||
## v1.0
|
||||
|
||||
### Features
|
||||
|
||||
### Bugfixes
|
||||
|
||||
## v1.0 beta 2 [2016-06-21]
|
||||
|
||||
### Features
|
||||
|
||||
- [#1340](https://github.com/influxdata/telegraf/issues/1340): statsd: do not log every dropped metric.
|
||||
- [#1368](https://github.com/influxdata/telegraf/pull/1368): Add precision rounding to all metrics on collection.
|
||||
- [#1390](https://github.com/influxdata/telegraf/pull/1390): Add support for Tengine
|
||||
- [#1320](https://github.com/influxdata/telegraf/pull/1320): Logparser input plugin for parsing grok-style log patterns.
|
||||
|
||||
### Bugfixes
|
||||
|
||||
- [#1330](https://github.com/influxdata/telegraf/issues/1330): Fix exec plugin panic when using single binary.
|
||||
- [#1336](https://github.com/influxdata/telegraf/issues/1336): Fixed incorrect prometheus metrics source selection.
|
||||
- [#1112](https://github.com/influxdata/telegraf/issues/1112): Set default Zookeeper chroot to empty string.
|
||||
- [#1335](https://github.com/influxdata/telegraf/issues/1335): Fix overall ping timeout to be calculated based on per-ping timeout.
|
||||
- [#1374](https://github.com/influxdata/telegraf/pull/1374): Change "default" retention policy to "".
|
||||
- [#1377](https://github.com/influxdata/telegraf/issues/1377): Graphite output mangling '%' character.
|
||||
|
||||
## v1.0 beta 1 [2016-06-07]
|
||||
|
||||
### Release Notes
|
||||
|
||||
- `flush_jitter` behavior has been changed. The random jitter will now be
|
||||
evaluated at every flush interval, rather than once at startup. This makes it
|
||||
consistent with the behavior of `collection_jitter`.
|
||||
|
||||
- All AWS plugins now utilize a standard mechanism for evaluating credentials.
|
||||
This allows all AWS plugins to support environment variables, shared credential
|
||||
files & profiles, and role assumptions. See the specific plugin README for
|
||||
details.
|
||||
|
||||
- The AWS CloudWatch input plugin can now declare a wildcard value for a metric
|
||||
dimension. This causes the plugin to read all metrics that contain the specified
|
||||
dimension key regardless of value. This is used to export collections of metrics
|
||||
without having to know the dimension values ahead of time.
|
||||
|
||||
- The AWS CloudWatch input plugin can now be configured with the `cache_ttl`
|
||||
attribute. This configures the TTL of the internal metric cache. This is useful
|
||||
in conjunction with wildcard dimension values as it will control the amount of
|
||||
time before a new metric is included by the plugin.
|
||||
|
||||
### Features
|
||||
- [#1262](https://github.com/influxdata/telegraf/pull/1261): Add graylog input pluging.
|
||||
- [#1294](https://github.com/influxdata/telegraf/pull/1294): consul input plugin. Thanks @harnash
|
||||
- [#1164](https://github.com/influxdata/telegraf/pull/1164): conntrack input plugin. Thanks @robinpercy!
|
||||
- [#1165](https://github.com/influxdata/telegraf/pull/1165): vmstat input plugin. Thanks @jshim-xm!
|
||||
- [#1247](https://github.com/influxdata/telegraf/pull/1247): rollbar input plugin. Thanks @francois2metz and @cduez!
|
||||
- [#1208](https://github.com/influxdata/telegraf/pull/1208): Standardized AWS credentials evaluation & wildcard CloudWatch dimensions. Thanks @johnrengelman!
|
||||
- [#1264](https://github.com/influxdata/telegraf/pull/1264): Add SSL config options to http_response plugin.
|
||||
- [#1272](https://github.com/influxdata/telegraf/pull/1272): graphite parser: add ability to specify multiple tag keys, for consistency with influxdb parser.
|
||||
- [#1265](https://github.com/influxdata/telegraf/pull/1265): Make dns lookups for chrony configurable. Thanks @zbindenren!
|
||||
- [#1275](https://github.com/influxdata/telegraf/pull/1275): Allow wildcard filtering of varnish stats.
|
||||
- [#1142](https://github.com/influxdata/telegraf/pull/1142): Support for glob patterns in exec plugin commands configuration.
|
||||
- [#1278](https://github.com/influxdata/telegraf/pull/1278): RabbitMQ input: made url parameter optional by using DefaultURL (http://localhost:15672) if not specified
|
||||
- [#1197](https://github.com/influxdata/telegraf/pull/1197): Limit AWS GetMetricStatistics requests to 10 per second.
|
||||
- [#1278](https://github.com/influxdata/telegraf/pull/1278) & [#1288](https://github.com/influxdata/telegraf/pull/1288) & [#1295](https://github.com/influxdata/telegraf/pull/1295): RabbitMQ/Apache/InfluxDB inputs: made url(s) parameter optional by using reasonable input defaults if not specified
|
||||
- [#1296](https://github.com/influxdata/telegraf/issues/1296): Refactor of flush_jitter argument.
|
||||
- [#1213](https://github.com/influxdata/telegraf/issues/1213): Add inactive & active memory to mem plugin.
|
||||
|
||||
### Bugfixes
|
||||
|
||||
- [#1252](https://github.com/influxdata/telegraf/pull/1252) & [#1279](https://github.com/influxdata/telegraf/pull/1279): Fix systemd service. Thanks @zbindenren & @PierreF!
|
||||
- [#1221](https://github.com/influxdata/telegraf/pull/1221): Fix influxdb n_shards counter.
|
||||
- [#1258](https://github.com/influxdata/telegraf/pull/1258): Fix potential kernel plugin integer parse error.
|
||||
- [#1268](https://github.com/influxdata/telegraf/pull/1268): Fix potential influxdb input type assertion panic.
|
||||
- [#1283](https://github.com/influxdata/telegraf/pull/1283): Still send processes metrics if a process exited during metric collection.
|
||||
- [#1297](https://github.com/influxdata/telegraf/issues/1297): disk plugin panic when usage grab fails.
|
||||
- [#1316](https://github.com/influxdata/telegraf/pull/1316): Removed leaked "database" tag on redis metrics. Thanks @PierreF!
|
||||
- [#1323](https://github.com/influxdata/telegraf/issues/1323): Processes plugin: fix potential error with /proc/net/stat directory.
|
||||
- [#1322](https://github.com/influxdata/telegraf/issues/1322): Fix rare RHEL 5.2 panic in gopsutil diskio gathering function.
|
||||
## v1.0 [unreleased]
|
||||
|
||||
## v0.13.1 [2016-05-24]
|
||||
|
||||
|
||||
@@ -212,8 +212,8 @@ func (s *Simple) Close() error {
|
||||
}
|
||||
|
||||
func (s *Simple) Write(metrics []telegraf.Metric) error {
|
||||
for _, metric := range metrics {
|
||||
// write `metric` to the output sink here
|
||||
for _, pt := range points {
|
||||
// write `pt` to the output sink here
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
6
Godeps
6
Godeps
@@ -16,14 +16,13 @@ github.com/eapache/go-resiliency b86b1ec0dd4209a588dc1285cdd471e73525c0b3
|
||||
github.com/eapache/queue ded5959c0d4e360646dc9e9908cff48666781367
|
||||
github.com/eclipse/paho.mqtt.golang 0f7a459f04f13a41b7ed752d47944528d4bf9a86
|
||||
github.com/go-sql-driver/mysql 1fca743146605a172a266e1654e01e5cd5669bee
|
||||
github.com/gobwas/glob 49571a1557cd20e6a2410adc6421f85b66c730b5
|
||||
github.com/gobwas/glob d877f6352135181470c40c73ebb81aefa22115fa
|
||||
github.com/golang/protobuf 552c7b9542c194800fd493123b3798ef0a832032
|
||||
github.com/golang/snappy 427fb6fc07997f43afa32f35e850833760e489a7
|
||||
github.com/gonuts/go-shellquote e842a11b24c6abfb3dd27af69a17f482e4b483c2
|
||||
github.com/gorilla/context 1ea25387ff6f684839d82767c1733ff4d4d15d0a
|
||||
github.com/gorilla/mux c9e326e2bdec29039a3761c07bece13133863e1e
|
||||
github.com/hailocab/go-hostpool e80d13ce29ede4452c43dea11e79b9bc8a15b478
|
||||
github.com/hashicorp/consul 5aa90455ce78d4d41578bafc86305e6e6b28d7d2
|
||||
github.com/hpcloud/tail b2940955ab8b26e19d43a43c4da0475dd81bdb56
|
||||
github.com/influxdata/config b79f6829346b8d6e78ba73544b1e1038f1f1c9da
|
||||
github.com/influxdata/influxdb e094138084855d444195b252314dfee9eae34cab
|
||||
@@ -43,11 +42,10 @@ github.com/prometheus/client_model fa8ad6fec33561be4280a8f0514318c79d7f6cb6
|
||||
github.com/prometheus/common e8eabff8812b05acf522b45fdcd725a785188e37
|
||||
github.com/prometheus/procfs 406e5b7bfd8201a36e2bb5f7bdae0b03380c2ce8
|
||||
github.com/samuel/go-zookeeper 218e9c81c0dd8b3b18172b2bbfad92cc7d6db55f
|
||||
github.com/shirou/gopsutil 586bb697f3ec9f8ec08ffefe18f521a64534037c
|
||||
github.com/shirou/gopsutil 83c6e72cbdef6e8ada934549abf700ff0ba96776
|
||||
github.com/soniah/gosnmp b1b4f885b12c5dcbd021c5cee1c904110de6db7d
|
||||
github.com/streadway/amqp b4f3ceab0337f013208d31348b578d83c0064744
|
||||
github.com/stretchr/testify 1f4a1643a57e798696635ea4c126e9127adb7d3c
|
||||
github.com/vjeantet/grok 83bfdfdfd1a8146795b28e547a8e3c8b28a466c2
|
||||
github.com/wvanbergen/kafka 46f9a1cf3f670edec492029fadded9c2d9e18866
|
||||
github.com/wvanbergen/kazoo-go 0f768712ae6f76454f987c3356177e138df258f8
|
||||
github.com/zensqlmonitor/go-mssqldb ffe5510c6fa5e15e6d983210ab501c815b56b363
|
||||
|
||||
6
Makefile
6
Makefile
@@ -64,6 +64,7 @@ endif
|
||||
docker run --name memcached -p "11211:11211" -d memcached
|
||||
docker run --name postgres -p "5432:5432" -d postgres
|
||||
docker run --name rabbitmq -p "15672:15672" -p "5672:5672" -d rabbitmq:3-management
|
||||
docker run --name opentsdb -p "4242:4242" -d petergrace/opentsdb-docker
|
||||
docker run --name redis -p "6379:6379" -d redis
|
||||
docker run --name aerospike -p "3000:3000" -d aerospike
|
||||
docker run --name nsq -p "4150:4150" -d nsqio/nsq /nsqd
|
||||
@@ -78,6 +79,7 @@ docker-run-circle:
|
||||
-e ADVERTISED_PORT=9092 \
|
||||
-p "2181:2181" -p "9092:9092" \
|
||||
-d spotify/kafka
|
||||
docker run --name opentsdb -p "4242:4242" -d petergrace/opentsdb-docker
|
||||
docker run --name aerospike -p "3000:3000" -d aerospike
|
||||
docker run --name nsq -p "4150:4150" -d nsqio/nsq /nsqd
|
||||
docker run --name mqtt -p "1883:1883" -d ncarlier/mqtt
|
||||
@@ -86,8 +88,8 @@ docker-run-circle:
|
||||
|
||||
# Kill all docker containers, ignore errors
|
||||
docker-kill:
|
||||
-docker kill nsq aerospike redis rabbitmq postgres memcached mysql kafka mqtt riemann snmp
|
||||
-docker rm nsq aerospike redis rabbitmq postgres memcached mysql kafka mqtt riemann snmp
|
||||
-docker kill nsq aerospike redis opentsdb rabbitmq postgres memcached mysql kafka mqtt riemann snmp
|
||||
-docker rm nsq aerospike redis opentsdb rabbitmq postgres memcached mysql kafka mqtt riemann snmp
|
||||
|
||||
# Run full unit tests using docker containers (includes setup and teardown)
|
||||
test: vet docker-kill docker-run
|
||||
|
||||
24
README.md
24
README.md
@@ -20,12 +20,12 @@ new plugins.
|
||||
### Linux deb and rpm Packages:
|
||||
|
||||
Latest:
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf_1.0.0-beta2_amd64.deb
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-1.0.0_beta2.x86_64.rpm
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf_0.13.2_amd64.deb
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.2.x86_64.rpm
|
||||
|
||||
Latest (arm):
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf_1.0.0-beta2_armhf.deb
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-1.0.0_beta2.armhf.rpm
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf_0.13.2_armhf.deb
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.2.armhf.rpm
|
||||
|
||||
##### Package Instructions:
|
||||
|
||||
@@ -46,14 +46,14 @@ to use this repo to install & update telegraf.
|
||||
### Linux tarballs:
|
||||
|
||||
Latest:
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-1.0.0-beta2_linux_amd64.tar.gz
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-1.0.0-beta2_linux_i386.tar.gz
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-1.0.0-beta2_linux_armhf.tar.gz
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.2_linux_amd64.tar.gz
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.2_linux_i386.tar.gz
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.2_linux_armhf.tar.gz
|
||||
|
||||
### FreeBSD tarball:
|
||||
|
||||
Latest:
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-1.0.0-beta2_freebsd_amd64.tar.gz
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.2_freebsd_amd64.tar.gz
|
||||
|
||||
### Ansible Role:
|
||||
|
||||
@@ -69,7 +69,8 @@ brew install telegraf
|
||||
### Windows Binaries (EXPERIMENTAL)
|
||||
|
||||
Latest:
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-1.0.0-beta2_windows_amd64.zip
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.2_windows_amd64.zip
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.2_windows_i386.zip
|
||||
|
||||
### From Source:
|
||||
|
||||
@@ -144,8 +145,6 @@ Currently implemented sources:
|
||||
* [cassandra](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/cassandra)
|
||||
* [ceph](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/ceph)
|
||||
* [chrony](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/chrony)
|
||||
* [consul](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/consul)
|
||||
* [conntrack](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/conntrack)
|
||||
* [couchbase](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/couchbase)
|
||||
* [couchdb](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/couchdb)
|
||||
* [disque](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/disque)
|
||||
@@ -206,7 +205,6 @@ Currently implemented sources:
|
||||
* swap
|
||||
* processes
|
||||
* kernel (/proc/stat)
|
||||
* kernel (/proc/vmstat)
|
||||
|
||||
Telegraf can also collect metrics via the following service plugins:
|
||||
|
||||
@@ -218,7 +216,6 @@ Telegraf can also collect metrics via the following service plugins:
|
||||
* [kafka_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/kafka_consumer)
|
||||
* [nats_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nats_consumer)
|
||||
* [github_webhooks](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/github_webhooks)
|
||||
* [rollbar_webhooks](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/rollbar_webhooks)
|
||||
|
||||
We'll be adding support for many more over the coming months. Read on if you
|
||||
want to add support for another service or third-party API.
|
||||
@@ -233,7 +230,6 @@ want to add support for another service or third-party API.
|
||||
* [datadog](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/datadog)
|
||||
* [file](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/file)
|
||||
* [graphite](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/graphite)
|
||||
* [graylog](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/graylog)
|
||||
* [instrumental](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/instrumental)
|
||||
* [kafka](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/kafka)
|
||||
* [librato](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/librato)
|
||||
|
||||
@@ -18,8 +18,4 @@ type Accumulator interface {
|
||||
|
||||
Debug() bool
|
||||
SetDebug(enabled bool)
|
||||
|
||||
SetPrecision(precision, interval time.Duration)
|
||||
|
||||
DisablePrecision()
|
||||
}
|
||||
|
||||
@@ -17,7 +17,6 @@ func NewAccumulator(
|
||||
acc := accumulator{}
|
||||
acc.metrics = metrics
|
||||
acc.inputConfig = inputConfig
|
||||
acc.precision = time.Nanosecond
|
||||
return &acc
|
||||
}
|
||||
|
||||
@@ -33,8 +32,6 @@ type accumulator struct {
|
||||
inputConfig *internal_models.InputConfig
|
||||
|
||||
prefix string
|
||||
|
||||
precision time.Duration
|
||||
}
|
||||
|
||||
func (ac *accumulator) Add(
|
||||
@@ -144,7 +141,6 @@ func (ac *accumulator) AddFields(
|
||||
} else {
|
||||
timestamp = time.Now()
|
||||
}
|
||||
timestamp = timestamp.Round(ac.precision)
|
||||
|
||||
if ac.prefix != "" {
|
||||
measurement = ac.prefix + measurement
|
||||
@@ -177,31 +173,6 @@ func (ac *accumulator) SetTrace(trace bool) {
|
||||
ac.trace = trace
|
||||
}
|
||||
|
||||
// SetPrecision takes two time.Duration objects. If the first is non-zero,
|
||||
// it sets that as the precision. Otherwise, it takes the second argument
|
||||
// as the order of time that the metrics should be rounded to, with the
|
||||
// maximum being 1s.
|
||||
func (ac *accumulator) SetPrecision(precision, interval time.Duration) {
|
||||
if precision > 0 {
|
||||
ac.precision = precision
|
||||
return
|
||||
}
|
||||
switch {
|
||||
case interval >= time.Second:
|
||||
ac.precision = time.Second
|
||||
case interval >= time.Millisecond:
|
||||
ac.precision = time.Millisecond
|
||||
case interval >= time.Microsecond:
|
||||
ac.precision = time.Microsecond
|
||||
default:
|
||||
ac.precision = time.Nanosecond
|
||||
}
|
||||
}
|
||||
|
||||
func (ac *accumulator) DisablePrecision() {
|
||||
ac.precision = time.Nanosecond
|
||||
}
|
||||
|
||||
func (ac *accumulator) setDefaultTags(tags map[string]string) {
|
||||
ac.defaultTags = tags
|
||||
}
|
||||
|
||||
@@ -38,128 +38,6 @@ func TestAdd(t *testing.T) {
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddNoPrecisionWithInterval(t *testing.T) {
|
||||
a := accumulator{}
|
||||
now := time.Date(2006, time.February, 10, 12, 0, 0, 82912748, time.UTC)
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
a.SetPrecision(0, time.Second)
|
||||
a.Add("acctest", float64(101), map[string]string{})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800000000000)),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddNoIntervalWithPrecision(t *testing.T) {
|
||||
a := accumulator{}
|
||||
now := time.Date(2006, time.February, 10, 12, 0, 0, 82912748, time.UTC)
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
a.SetPrecision(time.Second, time.Millisecond)
|
||||
a.Add("acctest", float64(101), map[string]string{})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800000000000)),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddDisablePrecision(t *testing.T) {
|
||||
a := accumulator{}
|
||||
now := time.Date(2006, time.February, 10, 12, 0, 0, 82912748, time.UTC)
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
a.SetPrecision(time.Second, time.Millisecond)
|
||||
a.DisablePrecision()
|
||||
a.Add("acctest", float64(101), map[string]string{})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800082912748)),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestDifferentPrecisions(t *testing.T) {
|
||||
a := accumulator{}
|
||||
now := time.Date(2006, time.February, 10, 12, 0, 0, 82912748, time.UTC)
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
a.SetPrecision(0, time.Second)
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800000000000)),
|
||||
actual)
|
||||
|
||||
a.SetPrecision(0, time.Millisecond)
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800083000000)),
|
||||
actual)
|
||||
|
||||
a.SetPrecision(0, time.Microsecond)
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800082913000)),
|
||||
actual)
|
||||
|
||||
a.SetPrecision(0, time.Nanosecond)
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800082912748)),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddDefaultTags(t *testing.T) {
|
||||
a := accumulator{}
|
||||
a.addDefaultTag("default", "tag")
|
||||
|
||||
@@ -1,15 +1,17 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
cryptorand "crypto/rand"
|
||||
"fmt"
|
||||
"log"
|
||||
"math/big"
|
||||
"math/rand"
|
||||
"os"
|
||||
"runtime"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/internal/config"
|
||||
"github.com/influxdata/telegraf/internal/models"
|
||||
)
|
||||
@@ -113,18 +115,27 @@ func (a *Agent) gatherer(
|
||||
ticker := time.NewTicker(interval)
|
||||
defer ticker.Stop()
|
||||
|
||||
jitter := a.Config.Agent.CollectionJitter.Duration.Nanoseconds()
|
||||
|
||||
for {
|
||||
var outerr error
|
||||
start := time.Now()
|
||||
|
||||
acc := NewAccumulator(input.Config, metricC)
|
||||
acc.SetDebug(a.Config.Agent.Debug)
|
||||
acc.SetPrecision(a.Config.Agent.Precision.Duration,
|
||||
a.Config.Agent.Interval.Duration)
|
||||
acc.setDefaultTags(a.Config.Tags)
|
||||
|
||||
internal.RandomSleep(a.Config.Agent.CollectionJitter.Duration, shutdown)
|
||||
if jitter != 0 {
|
||||
nanoSleep := rand.Int63n(jitter)
|
||||
d, err := time.ParseDuration(fmt.Sprintf("%dns", nanoSleep))
|
||||
if err != nil {
|
||||
log.Printf("Jittering collection interval failed for plugin %s",
|
||||
input.Name)
|
||||
} else {
|
||||
time.Sleep(d)
|
||||
}
|
||||
}
|
||||
|
||||
start := time.Now()
|
||||
gatherWithTimeout(shutdown, input, acc, interval)
|
||||
elapsed := time.Since(start)
|
||||
|
||||
@@ -203,8 +214,6 @@ func (a *Agent) Test() error {
|
||||
for _, input := range a.Config.Inputs {
|
||||
acc := NewAccumulator(input.Config, metricC)
|
||||
acc.SetTrace(true)
|
||||
acc.SetPrecision(a.Config.Agent.Precision.Duration,
|
||||
a.Config.Agent.Interval.Duration)
|
||||
acc.setDefaultTags(a.Config.Tags)
|
||||
|
||||
fmt.Printf("* Plugin: %s, Collection 1\n", input.Name)
|
||||
@@ -265,7 +274,6 @@ func (a *Agent) flusher(shutdown chan struct{}, metricC chan telegraf.Metric) er
|
||||
a.flush()
|
||||
return nil
|
||||
case <-ticker.C:
|
||||
internal.RandomSleep(a.Config.Agent.FlushJitter.Duration, shutdown)
|
||||
a.flush()
|
||||
case m := <-metricC:
|
||||
for _, o := range a.Config.Outputs {
|
||||
@@ -275,10 +283,35 @@ func (a *Agent) flusher(shutdown chan struct{}, metricC chan telegraf.Metric) er
|
||||
}
|
||||
}
|
||||
|
||||
// jitterInterval applies the the interval jitter to the flush interval using
|
||||
// crypto/rand number generator
|
||||
func jitterInterval(ininterval, injitter time.Duration) time.Duration {
|
||||
var jitter int64
|
||||
outinterval := ininterval
|
||||
if injitter.Nanoseconds() != 0 {
|
||||
maxjitter := big.NewInt(injitter.Nanoseconds())
|
||||
if j, err := cryptorand.Int(cryptorand.Reader, maxjitter); err == nil {
|
||||
jitter = j.Int64()
|
||||
}
|
||||
outinterval = time.Duration(jitter + ininterval.Nanoseconds())
|
||||
}
|
||||
|
||||
if outinterval.Nanoseconds() < time.Duration(500*time.Millisecond).Nanoseconds() {
|
||||
log.Printf("Flush interval %s too low, setting to 500ms\n", outinterval)
|
||||
outinterval = time.Duration(500 * time.Millisecond)
|
||||
}
|
||||
|
||||
return outinterval
|
||||
}
|
||||
|
||||
// Run runs the agent daemon, gathering every Interval
|
||||
func (a *Agent) Run(shutdown chan struct{}) error {
|
||||
var wg sync.WaitGroup
|
||||
|
||||
a.Config.Agent.FlushInterval.Duration = jitterInterval(
|
||||
a.Config.Agent.FlushInterval.Duration,
|
||||
a.Config.Agent.FlushJitter.Duration)
|
||||
|
||||
log.Printf("Agent Config: Interval:%s, Debug:%#v, Quiet:%#v, Hostname:%#v, "+
|
||||
"Flush Interval:%s \n",
|
||||
a.Config.Agent.Interval.Duration, a.Config.Agent.Debug, a.Config.Agent.Quiet,
|
||||
@@ -293,9 +326,6 @@ func (a *Agent) Run(shutdown chan struct{}) error {
|
||||
case telegraf.ServiceInput:
|
||||
acc := NewAccumulator(input.Config, metricC)
|
||||
acc.SetDebug(a.Config.Agent.Debug)
|
||||
// Service input plugins should set their own precision of their
|
||||
// metrics.
|
||||
acc.DisablePrecision()
|
||||
acc.setDefaultTags(a.Config.Tags)
|
||||
if err := p.Start(acc); err != nil {
|
||||
log.Printf("Service for input %s failed to start, exiting\n%s\n",
|
||||
|
||||
@@ -2,6 +2,7 @@ package agent
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf/internal/config"
|
||||
|
||||
@@ -109,3 +110,75 @@ func TestAgent_LoadOutput(t *testing.T) {
|
||||
a, _ = NewAgent(c)
|
||||
assert.Equal(t, 3, len(a.Config.Outputs))
|
||||
}
|
||||
|
||||
func TestAgent_ZeroJitter(t *testing.T) {
|
||||
flushinterval := jitterInterval(time.Duration(10*time.Second),
|
||||
time.Duration(0*time.Second))
|
||||
|
||||
actual := flushinterval.Nanoseconds()
|
||||
exp := time.Duration(10 * time.Second).Nanoseconds()
|
||||
|
||||
if actual != exp {
|
||||
t.Errorf("Actual %v, expected %v", actual, exp)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_ZeroInterval(t *testing.T) {
|
||||
min := time.Duration(500 * time.Millisecond).Nanoseconds()
|
||||
max := time.Duration(5 * time.Second).Nanoseconds()
|
||||
|
||||
for i := 0; i < 1000; i++ {
|
||||
flushinterval := jitterInterval(time.Duration(0*time.Second),
|
||||
time.Duration(5*time.Second))
|
||||
actual := flushinterval.Nanoseconds()
|
||||
|
||||
if actual > max {
|
||||
t.Errorf("Didn't expect interval %d to be > %d", actual, max)
|
||||
break
|
||||
}
|
||||
if actual < min {
|
||||
t.Errorf("Didn't expect interval %d to be < %d", actual, min)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_ZeroBoth(t *testing.T) {
|
||||
flushinterval := jitterInterval(time.Duration(0*time.Second),
|
||||
time.Duration(0*time.Second))
|
||||
|
||||
actual := flushinterval
|
||||
exp := time.Duration(500 * time.Millisecond)
|
||||
|
||||
if actual != exp {
|
||||
t.Errorf("Actual %v, expected %v", actual, exp)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_JitterMax(t *testing.T) {
|
||||
max := time.Duration(32 * time.Second).Nanoseconds()
|
||||
|
||||
for i := 0; i < 1000; i++ {
|
||||
flushinterval := jitterInterval(time.Duration(30*time.Second),
|
||||
time.Duration(2*time.Second))
|
||||
actual := flushinterval.Nanoseconds()
|
||||
if actual > max {
|
||||
t.Errorf("Didn't expect interval %d to be > %d", actual, max)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_JitterMin(t *testing.T) {
|
||||
min := time.Duration(30 * time.Second).Nanoseconds()
|
||||
|
||||
for i := 0; i < 1000; i++ {
|
||||
flushinterval := jitterInterval(time.Duration(30*time.Second),
|
||||
time.Duration(2*time.Second))
|
||||
actual := flushinterval.Nanoseconds()
|
||||
if actual < min {
|
||||
t.Errorf("Didn't expect interval %d to be < %d", actual, min)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -186,59 +186,49 @@ name of the plugin.
|
||||
# Graphite:
|
||||
|
||||
The Graphite data format translates graphite _dot_ buckets directly into
|
||||
telegraf measurement names, with a single value field, and without any tags.
|
||||
By default, the separator is left as ".", but this can be changed using the
|
||||
"separator" argument. For more advanced options,
|
||||
Telegraf supports specifying "templates" to translate
|
||||
telegraf measurement names, with a single value field, and without any tags. For
|
||||
more advanced options, Telegraf supports specifying "templates" to translate
|
||||
graphite buckets into Telegraf metrics.
|
||||
|
||||
Templates are of the form:
|
||||
#### Separator:
|
||||
|
||||
You can specify a separator to use for the parsed metrics.
|
||||
By default, it will leave the metrics with a "." separator.
|
||||
Setting `separator = "_"` will translate:
|
||||
|
||||
```
|
||||
"host.mytag.mytag.measurement.measurement.field*"
|
||||
cpu.usage.idle 99
|
||||
=> cpu_usage_idle value=99
|
||||
```
|
||||
|
||||
Where the following keywords exist:
|
||||
|
||||
1. `measurement`: specifies that this section of the graphite bucket corresponds
|
||||
to the measurement name. This can be specified multiple times.
|
||||
2. `field`: specifies that this section of the graphite bucket corresponds
|
||||
to the field name. This can be specified multiple times.
|
||||
3. `measurement*`: specifies that all remaining elements of the graphite bucket
|
||||
correspond to the measurement name.
|
||||
4. `field*`: specifies that all remaining elements of the graphite bucket
|
||||
correspond to the field name.
|
||||
|
||||
Any part of the template that is not a keyword is treated as a tag key. This
|
||||
can also be specified multiple times.
|
||||
|
||||
NOTE: `field*` cannot be used in conjunction with `measurement*`!
|
||||
|
||||
#### Measurement & Tag Templates:
|
||||
#### Measurement/Tag Templates:
|
||||
|
||||
The most basic template is to specify a single transformation to apply to all
|
||||
incoming metrics. So the following template:
|
||||
incoming metrics. _measurement_ is a special keyword that tells Telegraf which
|
||||
parts of the graphite bucket to combine into the measurement name. It can have a
|
||||
trailing `*` to indicate that the remainder of the metric should be used.
|
||||
Other words are considered tag keys. So the following template:
|
||||
|
||||
```toml
|
||||
templates = [
|
||||
"region.region.measurement*"
|
||||
"region.measurement*"
|
||||
]
|
||||
```
|
||||
|
||||
would result in the following Graphite -> Telegraf transformation.
|
||||
|
||||
```
|
||||
us.west.cpu.load 100
|
||||
=> cpu.load,region=us.west value=100
|
||||
us-west.cpu.load 100
|
||||
=> cpu.load,region=us-west value=100
|
||||
```
|
||||
|
||||
#### Field Templates:
|
||||
|
||||
There is also a _field_ keyword, which can only be specified once.
|
||||
The field keyword tells Telegraf to give the metric that field name.
|
||||
So the following template:
|
||||
|
||||
```toml
|
||||
separator = "_"
|
||||
templates = [
|
||||
"measurement.measurement.field.field.region"
|
||||
]
|
||||
@@ -247,26 +237,24 @@ templates = [
|
||||
would result in the following Graphite -> Telegraf transformation.
|
||||
|
||||
```
|
||||
cpu.usage.idle.percent.eu-east 100
|
||||
=> cpu_usage,region=eu-east idle_percent=100
|
||||
cpu.usage.idle.percent.us-west 100
|
||||
=> cpu_usage,region=us-west idle_percent=100
|
||||
```
|
||||
|
||||
The field key can also be derived from all remaining elements of the graphite
|
||||
bucket by specifying `field*`:
|
||||
|
||||
The field key can also be derived from the second "half" of the input metric-name by specifying ```field*```:
|
||||
```toml
|
||||
separator = "_"
|
||||
templates = [
|
||||
"measurement.measurement.region.field*"
|
||||
]
|
||||
```
|
||||
|
||||
which would result in the following Graphite -> Telegraf transformation.
|
||||
would result in the following Graphite -> Telegraf transformation.
|
||||
|
||||
```
|
||||
cpu.usage.eu-east.idle.percentage 100
|
||||
=> cpu_usage,region=eu-east idle_percentage=100
|
||||
cpu.usage.us-west.idle.percentage 100
|
||||
=> cpu_usage,region=us-west idle_percentage=100
|
||||
```
|
||||
(This cannot be used in conjunction with "measurement*"!)
|
||||
|
||||
#### Filter Templates:
|
||||
|
||||
@@ -283,8 +271,8 @@ templates = [
|
||||
which would result in the following transformation:
|
||||
|
||||
```
|
||||
cpu.load.eu-east 100
|
||||
=> cpu_load,region=eu-east value=100
|
||||
cpu.load.us-west 100
|
||||
=> cpu_load,region=us-west value=100
|
||||
|
||||
mem.cached.localhost 256
|
||||
=> mem_cached,host=localhost value=256
|
||||
@@ -306,8 +294,8 @@ templates = [
|
||||
would result in the following Graphite -> Telegraf transformation.
|
||||
|
||||
```
|
||||
cpu.usage.idle.eu-east 100
|
||||
=> cpu_usage,region=eu-east,datacenter=1a idle=100
|
||||
cpu.usage.idle.us-west 100
|
||||
=> cpu_usage,region=us-west,datacenter=1a idle=100
|
||||
```
|
||||
|
||||
There are many more options available,
|
||||
@@ -338,12 +326,12 @@ There are many more options available,
|
||||
## similar to the line protocol format. There can be only one default template.
|
||||
## Templates support below format:
|
||||
## 1. filter + template
|
||||
## 2. filter + template + extra tag(s)
|
||||
## 2. filter + template + extra tag
|
||||
## 3. filter + template with field key
|
||||
## 4. default template
|
||||
templates = [
|
||||
"*.app env.service.resource.measurement",
|
||||
"stats.* .host.measurement* region=eu-east,agent=sensu",
|
||||
"stats.* .host.measurement* region=us-west,agent=sensu",
|
||||
"stats2.* .host.measurement.field",
|
||||
"measurement*"
|
||||
]
|
||||
|
||||
@@ -52,11 +52,6 @@
|
||||
## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
|
||||
flush_jitter = "0s"
|
||||
|
||||
## By default, precision will be set to the same timestamp order as the
|
||||
## collection interval, with the maximum being 1s.
|
||||
## Precision will NOT be used for service inputs, such as logparser and statsd.
|
||||
## Valid values are "Nns", "Nus" (or "Nµs"), "Nms", "Ns".
|
||||
precision = ""
|
||||
## Run telegraf in debug mode
|
||||
debug = false
|
||||
## Run telegraf in quiet mode
|
||||
@@ -80,9 +75,12 @@
|
||||
urls = ["http://localhost:8086"] # required
|
||||
## The target database for metrics (telegraf will create it if not exists).
|
||||
database = "telegraf" # required
|
||||
## Precision of writes, valid values are "ns", "us" (or "µs"), "ms", "s", "m", "h".
|
||||
## note: using "s" precision greatly improves InfluxDB compression.
|
||||
precision = "s"
|
||||
|
||||
## Retention policy to write to. Empty string writes to the default rp.
|
||||
retention_policy = ""
|
||||
## Retention policy to write to.
|
||||
retention_policy = "default"
|
||||
## Write consistency (clusters only), can be: "any", "one", "quorom", "all"
|
||||
write_consistency = "any"
|
||||
|
||||
@@ -108,10 +106,10 @@
|
||||
# [[outputs.amon]]
|
||||
# ## Amon Server Key
|
||||
# server_key = "my-server-key" # required.
|
||||
#
|
||||
#
|
||||
# ## Amon Instance URL
|
||||
# amon_instance = "https://youramoninstance" # required
|
||||
#
|
||||
#
|
||||
# ## Connection timeout.
|
||||
# # timeout = "5s"
|
||||
|
||||
@@ -127,21 +125,21 @@
|
||||
# ## Telegraf tag to use as a routing key
|
||||
# ## ie, if this tag exists, it's value will be used as the routing key
|
||||
# routing_tag = "host"
|
||||
#
|
||||
#
|
||||
# ## InfluxDB retention policy
|
||||
# # retention_policy = "default"
|
||||
# ## InfluxDB database
|
||||
# # database = "telegraf"
|
||||
# ## InfluxDB precision
|
||||
# # precision = "s"
|
||||
#
|
||||
#
|
||||
# ## Optional SSL Config
|
||||
# # ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# # ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# # ssl_key = "/etc/telegraf/key.pem"
|
||||
# ## Use SSL but skip chain & host verification
|
||||
# # insecure_skip_verify = false
|
||||
#
|
||||
#
|
||||
# ## Data format to output.
|
||||
# ## Each data format has it's own unique set of configuration options, read
|
||||
# ## more about them here:
|
||||
@@ -153,22 +151,16 @@
|
||||
# [[outputs.cloudwatch]]
|
||||
# ## Amazon REGION
|
||||
# region = 'us-east-1'
|
||||
#
|
||||
#
|
||||
# ## Amazon Credentials
|
||||
# ## Credentials are loaded in the following order
|
||||
# ## 1) Assumed credentials via STS if role_arn is specified
|
||||
# ## 2) explicit credentials from 'access_key' and 'secret_key'
|
||||
# ## 3) shared profile from 'profile'
|
||||
# ## 4) environment variables
|
||||
# ## 5) shared credentials file
|
||||
# ## 6) EC2 Instance Profile
|
||||
# ## 1) explicit credentials from 'access_key' and 'secret_key'
|
||||
# ## 2) environment variables
|
||||
# ## 3) shared credentials file
|
||||
# ## 4) EC2 Instance Profile
|
||||
# #access_key = ""
|
||||
# #secret_key = ""
|
||||
# #token = ""
|
||||
# #role_arn = ""
|
||||
# #profile = ""
|
||||
# #shared_credential_file = ""
|
||||
#
|
||||
#
|
||||
# ## Namespace for the CloudWatch MetricDatums
|
||||
# namespace = 'InfluxData/Telegraf'
|
||||
|
||||
@@ -177,7 +169,7 @@
|
||||
# [[outputs.datadog]]
|
||||
# ## Datadog API key
|
||||
# apikey = "my-secret-key" # required.
|
||||
#
|
||||
#
|
||||
# ## Connection timeout.
|
||||
# # timeout = "5s"
|
||||
|
||||
@@ -186,7 +178,7 @@
|
||||
# [[outputs.file]]
|
||||
# ## Files to write to, "stdout" is a specially handled file.
|
||||
# files = ["stdout", "/tmp/metrics.out"]
|
||||
#
|
||||
#
|
||||
# ## Data format to output.
|
||||
# ## Each data format has it's own unique set of configuration options, read
|
||||
# ## more about them here:
|
||||
@@ -207,12 +199,6 @@
|
||||
# timeout = 2
|
||||
|
||||
|
||||
# # Send telegraf metrics to graylog(s)
|
||||
# [[outputs.graylog]]
|
||||
# ## Udp endpoint for your graylog instance.
|
||||
# servers = ["127.0.0.1:12201", "192.168.1.1:12201"]
|
||||
|
||||
|
||||
# # Configuration for sending metrics to an Instrumental project
|
||||
# [[outputs.instrumental]]
|
||||
# ## Project API Token (required)
|
||||
@@ -237,14 +223,14 @@
|
||||
# ## Telegraf tag to use as a routing key
|
||||
# ## ie, if this tag exists, it's value will be used as the routing key
|
||||
# routing_tag = "host"
|
||||
#
|
||||
#
|
||||
# ## CompressionCodec represents the various compression codecs recognized by
|
||||
# ## Kafka in messages.
|
||||
# ## 0 : No compression
|
||||
# ## 1 : Gzip compression
|
||||
# ## 2 : Snappy compression
|
||||
# compression_codec = 0
|
||||
#
|
||||
#
|
||||
# ## RequiredAcks is used in Produce Requests to tell the broker how many
|
||||
# ## replica acknowledgements it must see before responding
|
||||
# ## 0 : the producer never waits for an acknowledgement from the broker.
|
||||
@@ -260,17 +246,17 @@
|
||||
# ## guarantee that no messages will be lost as long as at least one in
|
||||
# ## sync replica remains.
|
||||
# required_acks = -1
|
||||
#
|
||||
#
|
||||
# ## The total number of times to retry sending a message
|
||||
# max_retry = 3
|
||||
#
|
||||
#
|
||||
# ## Optional SSL Config
|
||||
# # ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# # ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# # ssl_key = "/etc/telegraf/key.pem"
|
||||
# ## Use SSL but skip chain & host verification
|
||||
# # insecure_skip_verify = false
|
||||
#
|
||||
#
|
||||
# ## Data format to output.
|
||||
# ## Each data format has it's own unique set of configuration options, read
|
||||
# ## more about them here:
|
||||
@@ -282,22 +268,16 @@
|
||||
# [[outputs.kinesis]]
|
||||
# ## Amazon REGION of kinesis endpoint.
|
||||
# region = "ap-southeast-2"
|
||||
#
|
||||
#
|
||||
# ## Amazon Credentials
|
||||
# ## Credentials are loaded in the following order
|
||||
# ## 1) Assumed credentials via STS if role_arn is specified
|
||||
# ## 2) explicit credentials from 'access_key' and 'secret_key'
|
||||
# ## 3) shared profile from 'profile'
|
||||
# ## 4) environment variables
|
||||
# ## 5) shared credentials file
|
||||
# ## 6) EC2 Instance Profile
|
||||
# ## 1) explicit credentials from 'access_key' and 'secret_key'
|
||||
# ## 2) environment variables
|
||||
# ## 3) shared credentials file
|
||||
# ## 4) EC2 Instance Profile
|
||||
# #access_key = ""
|
||||
# #secret_key = ""
|
||||
# #token = ""
|
||||
# #role_arn = ""
|
||||
# #profile = ""
|
||||
# #shared_credential_file = ""
|
||||
#
|
||||
#
|
||||
# ## Kinesis StreamName must exist prior to starting telegraf.
|
||||
# streamname = "StreamName"
|
||||
# ## PartitionKey as used for sharding data.
|
||||
@@ -332,23 +312,23 @@
|
||||
# # Configuration for MQTT server to send metrics to
|
||||
# [[outputs.mqtt]]
|
||||
# servers = ["localhost:1883"] # required.
|
||||
#
|
||||
#
|
||||
# ## MQTT outputs send metrics to this topic format
|
||||
# ## "<topic_prefix>/<hostname>/<pluginname>/"
|
||||
# ## ex: prefix/web01.example.com/mem
|
||||
# topic_prefix = "telegraf"
|
||||
#
|
||||
#
|
||||
# ## username and password to connect MQTT server.
|
||||
# # username = "telegraf"
|
||||
# # password = "metricsmetricsmetricsmetrics"
|
||||
#
|
||||
#
|
||||
# ## Optional SSL Config
|
||||
# # ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# # ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# # ssl_key = "/etc/telegraf/key.pem"
|
||||
# ## Use SSL but skip chain & host verification
|
||||
# # insecure_skip_verify = false
|
||||
#
|
||||
#
|
||||
# ## Data format to output.
|
||||
# ## Each data format has it's own unique set of configuration options, read
|
||||
# ## more about them here:
|
||||
@@ -362,7 +342,7 @@
|
||||
# server = "localhost:4150"
|
||||
# ## NSQ topic for producer messages
|
||||
# topic = "telegraf"
|
||||
#
|
||||
#
|
||||
# ## Data format to output.
|
||||
# ## Each data format has it's own unique set of configuration options, read
|
||||
# ## more about them here:
|
||||
@@ -374,14 +354,14 @@
|
||||
# [[outputs.opentsdb]]
|
||||
# ## prefix for metrics keys
|
||||
# prefix = "my.specific.prefix."
|
||||
#
|
||||
#
|
||||
# ## Telnet Mode ##
|
||||
# ## DNS name of the OpenTSDB server in telnet mode
|
||||
# host = "opentsdb.example.com"
|
||||
#
|
||||
#
|
||||
# ## Port of the OpenTSDB server in telnet mode
|
||||
# port = 4242
|
||||
#
|
||||
#
|
||||
# ## Debug true - Prints OpenTSDB communication
|
||||
# debug = false
|
||||
|
||||
@@ -474,7 +454,6 @@
|
||||
# # Read Apache status information (mod_status)
|
||||
# [[inputs.apache]]
|
||||
# ## An array of Apache status URI to gather stats.
|
||||
# ## Default is "http://localhost/server-status?auto".
|
||||
# urls = ["http://localhost/server-status?auto"]
|
||||
|
||||
|
||||
@@ -483,7 +462,7 @@
|
||||
# ## Bcache sets path
|
||||
# ## If not specified, then default is:
|
||||
# bcachePath = "/sys/fs/bcache"
|
||||
#
|
||||
#
|
||||
# ## By default, telegraf gather stats for all bcache devices
|
||||
# ## Setting devices will restrict the stats to the specified
|
||||
# ## bcache devices.
|
||||
@@ -511,17 +490,17 @@
|
||||
# # Collects performance metrics from the MON and OSD nodes in a Ceph storage cluster.
|
||||
# [[inputs.ceph]]
|
||||
# ## All configuration values are optional, defaults are shown below
|
||||
#
|
||||
#
|
||||
# ## location of ceph binary
|
||||
# ceph_binary = "/usr/bin/ceph"
|
||||
#
|
||||
#
|
||||
# ## directory in which to look for socket files
|
||||
# socket_dir = "/var/run/ceph"
|
||||
#
|
||||
#
|
||||
# ## prefix of MON and OSD socket files, used to determine socket type
|
||||
# mon_prefix = "ceph-mon"
|
||||
# osd_prefix = "ceph-osd"
|
||||
#
|
||||
#
|
||||
# ## suffix used to identify socket files
|
||||
# socket_suffix = "asok"
|
||||
|
||||
@@ -530,39 +509,29 @@
|
||||
# [[inputs.cloudwatch]]
|
||||
# ## Amazon Region
|
||||
# region = 'us-east-1'
|
||||
#
|
||||
#
|
||||
# ## Amazon Credentials
|
||||
# ## Credentials are loaded in the following order
|
||||
# ## 1) Assumed credentials via STS if role_arn is specified
|
||||
# ## 2) explicit credentials from 'access_key' and 'secret_key'
|
||||
# ## 3) shared profile from 'profile'
|
||||
# ## 4) environment variables
|
||||
# ## 5) shared credentials file
|
||||
# ## 6) EC2 Instance Profile
|
||||
# ## 1) explicit credentials from 'access_key' and 'secret_key'
|
||||
# ## 2) environment variables
|
||||
# ## 3) shared credentials file
|
||||
# ## 4) EC2 Instance Profile
|
||||
# #access_key = ""
|
||||
# #secret_key = ""
|
||||
# #token = ""
|
||||
# #role_arn = ""
|
||||
# #profile = ""
|
||||
# #shared_credential_file = ""
|
||||
#
|
||||
#
|
||||
# ## Requested CloudWatch aggregation Period (required - must be a multiple of 60s)
|
||||
# period = '1m'
|
||||
#
|
||||
#
|
||||
# ## Collection Delay (required - must account for metrics availability via CloudWatch API)
|
||||
# delay = '1m'
|
||||
#
|
||||
#
|
||||
# ## Recomended: use metric 'interval' that is a multiple of 'period' to avoid
|
||||
# ## gaps or overlap in pulled data
|
||||
# interval = '1m'
|
||||
#
|
||||
# ## Configure the TTL for the internal cache of metrics.
|
||||
# ## Defaults to 1 hr if not specified
|
||||
# #cache_ttl = '10m'
|
||||
#
|
||||
#
|
||||
# ## Metric Statistic Namespace (required)
|
||||
# namespace = 'AWS/ELB'
|
||||
#
|
||||
#
|
||||
# ## Metrics to Pull (optional)
|
||||
# ## Defaults to all Metrics in Namespace if nothing is provided
|
||||
# ## Refreshes Namespace available metrics every 1h
|
||||
@@ -575,23 +544,6 @@
|
||||
# # value = 'p-example'
|
||||
|
||||
|
||||
# # Gather health check statuses from services registered in Consul
|
||||
# [[inputs.consul]]
|
||||
# ## Most of these values defaults to the one configured on a Consul's agent level.
|
||||
# ## Optional Consul server address (default: "localhost")
|
||||
# # address = "localhost"
|
||||
# ## Optional URI scheme for the Consul server (default: "http")
|
||||
# # scheme = "http"
|
||||
# ## Optional ACL token used in every request (default: "")
|
||||
# # token = ""
|
||||
# ## Optional username used for request HTTP Basic Authentication (default: "")
|
||||
# # username = ""
|
||||
# ## Optional password used for HTTP Basic Authentication (default: "")
|
||||
# # password = ""
|
||||
# ## Optional data centre to query the health checks from (default: "")
|
||||
# # datacentre = ""
|
||||
|
||||
|
||||
# # Read metrics from one or many couchbase clusters
|
||||
# [[inputs.couchbase]]
|
||||
# ## specify servers via a url matching:
|
||||
@@ -626,17 +578,17 @@
|
||||
# [[inputs.dns_query]]
|
||||
# ## servers to query
|
||||
# servers = ["8.8.8.8"] # required
|
||||
#
|
||||
#
|
||||
# ## Domains or subdomains to query. "."(root) is default
|
||||
# domains = ["."] # optional
|
||||
#
|
||||
#
|
||||
# ## Query record type. Default is "A"
|
||||
# ## Posible values: A, AAAA, CNAME, MX, NS, PTR, TXT, SOA, SPF, SRV.
|
||||
# record_type = "A" # optional
|
||||
#
|
||||
#
|
||||
# ## Dns server port. 53 is default
|
||||
# port = 53 # optional
|
||||
#
|
||||
#
|
||||
# ## Query timeout in seconds. Default is 2 seconds
|
||||
# timeout = 2 # optional
|
||||
|
||||
@@ -672,11 +624,11 @@
|
||||
# [[inputs.elasticsearch]]
|
||||
# ## specify a list of one or more Elasticsearch servers
|
||||
# servers = ["http://localhost:9200"]
|
||||
#
|
||||
#
|
||||
# ## set local to false when you want to read the indices stats from all nodes
|
||||
# ## within the cluster
|
||||
# local = true
|
||||
#
|
||||
#
|
||||
# ## set cluster_health to true when you want to also obtain cluster level stats
|
||||
# cluster_health = false
|
||||
|
||||
@@ -684,18 +636,14 @@
|
||||
# # Read metrics from one or more commands that can output to stdout
|
||||
# [[inputs.exec]]
|
||||
# ## Commands array
|
||||
# commands = [
|
||||
# "/tmp/test.sh",
|
||||
# "/usr/bin/mycollector --foo=bar",
|
||||
# "/tmp/collect_*.sh"
|
||||
# ]
|
||||
#
|
||||
# commands = ["/tmp/test.sh", "/usr/bin/mycollector --foo=bar"]
|
||||
#
|
||||
# ## Timeout for each command to complete.
|
||||
# timeout = "5s"
|
||||
#
|
||||
#
|
||||
# ## measurement name suffix (for separating different commands)
|
||||
# name_suffix = "_mycollector"
|
||||
#
|
||||
#
|
||||
# ## Data format to consume.
|
||||
# ## Each data format has it's own unique set of configuration options, read
|
||||
# ## more about them here:
|
||||
@@ -719,48 +667,11 @@
|
||||
# md5 = false
|
||||
|
||||
|
||||
# # Read flattened metrics from one or more GrayLog HTTP endpoints
|
||||
# [[inputs.graylog]]
|
||||
# ## API endpoint, currently supported API:
|
||||
# ##
|
||||
# ## - multiple (Ex http://<host>:12900/system/metrics/multiple)
|
||||
# ## - namespace (Ex http://<host>:12900/system/metrics/namespace/{namespace})
|
||||
# ##
|
||||
# ## For namespace endpoint, the metrics array will be ignored for that call.
|
||||
# ## Endpoint can contain namespace and multiple type calls.
|
||||
# ##
|
||||
# ## Please check http://[graylog-server-ip]:12900/api-browser for full list
|
||||
# ## of endpoints
|
||||
# servers = [
|
||||
# "http://[graylog-server-ip]:12900/system/metrics/multiple",
|
||||
# ]
|
||||
#
|
||||
# ## Metrics list
|
||||
# ## List of metrics can be found on Graylog webservice documentation.
|
||||
# ## Or by hitting the the web service api at:
|
||||
# ## http://[graylog-host]:12900/system/metrics
|
||||
# metrics = [
|
||||
# "jvm.cl.loaded",
|
||||
# "jvm.memory.pools.Metaspace.committed"
|
||||
# ]
|
||||
#
|
||||
# ## Username and password
|
||||
# username = ""
|
||||
# password = ""
|
||||
#
|
||||
# ## Optional SSL Config
|
||||
# # ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# # ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# # ssl_key = "/etc/telegraf/key.pem"
|
||||
# ## Use SSL but skip chain & host verification
|
||||
# # insecure_skip_verify = false
|
||||
|
||||
|
||||
# # Read metrics of haproxy, via socket or csv stats page
|
||||
# [[inputs.haproxy]]
|
||||
# ## An array of address to gather stats about. Specify an ip on hostname
|
||||
# ## with optional port. ie localhost, 10.10.3.33:1936, etc.
|
||||
#
|
||||
#
|
||||
# ## If no servers are specified, then default to 127.0.0.1:1936
|
||||
# servers = ["http://myhaproxy.com:1936", "http://anotherhaproxy.com:1936"]
|
||||
# ## Or you can also use local socket
|
||||
@@ -784,48 +695,41 @@
|
||||
# # body = '''
|
||||
# # {'fake':'data'}
|
||||
# # '''
|
||||
#
|
||||
# ## Optional SSL Config
|
||||
# # ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# # ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# # ssl_key = "/etc/telegraf/key.pem"
|
||||
# ## Use SSL but skip chain & host verification
|
||||
# # insecure_skip_verify = false
|
||||
|
||||
|
||||
# # Read flattened metrics from one or more JSON HTTP endpoints
|
||||
# [[inputs.httpjson]]
|
||||
# ## NOTE This plugin only reads numerical measurements, strings and booleans
|
||||
# ## will be ignored.
|
||||
#
|
||||
#
|
||||
# ## a name for the service being polled
|
||||
# name = "webserver_stats"
|
||||
#
|
||||
#
|
||||
# ## URL of each server in the service's cluster
|
||||
# servers = [
|
||||
# "http://localhost:9999/stats/",
|
||||
# "http://localhost:9998/stats/",
|
||||
# ]
|
||||
#
|
||||
#
|
||||
# ## HTTP method to use: GET or POST (case-sensitive)
|
||||
# method = "GET"
|
||||
#
|
||||
#
|
||||
# ## List of tag names to extract from top-level of JSON server response
|
||||
# # tag_keys = [
|
||||
# # "my_tag_1",
|
||||
# # "my_tag_2"
|
||||
# # ]
|
||||
#
|
||||
#
|
||||
# ## HTTP parameters (all values must be strings)
|
||||
# [inputs.httpjson.parameters]
|
||||
# event_type = "cpu_spike"
|
||||
# threshold = "0.75"
|
||||
#
|
||||
#
|
||||
# ## HTTP Header parameters (all values must be strings)
|
||||
# # [inputs.httpjson.headers]
|
||||
# # X-Auth-Token = "my-xauth-token"
|
||||
# # apiVersion = "v1"
|
||||
#
|
||||
#
|
||||
# ## Optional SSL Config
|
||||
# # ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# # ssl_cert = "/etc/telegraf/cert.pem"
|
||||
@@ -839,9 +743,8 @@
|
||||
# ## Works with InfluxDB debug endpoints out of the box,
|
||||
# ## but other services can use this format too.
|
||||
# ## See the influxdb plugin's README for more details.
|
||||
#
|
||||
#
|
||||
# ## Multiple URLs from which to read InfluxDB-formatted JSON
|
||||
# ## Default is "http://localhost:8086/debug/vars".
|
||||
# urls = [
|
||||
# "http://localhost:8086/debug/vars"
|
||||
# ]
|
||||
@@ -861,7 +764,7 @@
|
||||
# [[inputs.jolokia]]
|
||||
# ## This is the context root used to compose the jolokia url
|
||||
# context = "/jolokia"
|
||||
#
|
||||
#
|
||||
# ## This specifies the mode used
|
||||
# # mode = "proxy"
|
||||
# #
|
||||
@@ -871,8 +774,8 @@
|
||||
# # [inputs.jolokia.proxy]
|
||||
# # host = "127.0.0.1"
|
||||
# # port = "8080"
|
||||
#
|
||||
#
|
||||
#
|
||||
#
|
||||
# ## List of servers exposing jolokia read service
|
||||
# [[inputs.jolokia.servers]]
|
||||
# name = "as-server-01"
|
||||
@@ -880,7 +783,7 @@
|
||||
# port = "8080"
|
||||
# # username = "myuser"
|
||||
# # password = "mypassword"
|
||||
#
|
||||
#
|
||||
# ## List of metrics collected on above servers
|
||||
# ## Each metric consists in a name, a jmx path and either
|
||||
# ## a pass or drop slice attribute.
|
||||
@@ -889,13 +792,13 @@
|
||||
# name = "heap_memory_usage"
|
||||
# mbean = "java.lang:type=Memory"
|
||||
# attribute = "HeapMemoryUsage"
|
||||
#
|
||||
#
|
||||
# ## This collect thread counts metrics.
|
||||
# [[inputs.jolokia.metrics]]
|
||||
# name = "thread_count"
|
||||
# mbean = "java.lang:type=Threading"
|
||||
# attribute = "TotalStartedThreadCount,ThreadCount,DaemonThreadCount,PeakThreadCount"
|
||||
#
|
||||
#
|
||||
# ## This collect number of class loaded/unloaded counts metrics.
|
||||
# [[inputs.jolokia.metrics]]
|
||||
# name = "class_count"
|
||||
@@ -1048,7 +951,7 @@
|
||||
# address = "github.com:80"
|
||||
# ## Set timeout
|
||||
# timeout = "1s"
|
||||
#
|
||||
#
|
||||
# ## Optional string sent to the server
|
||||
# # send = "ssh"
|
||||
# ## Optional expected string in answer
|
||||
@@ -1140,7 +1043,7 @@
|
||||
# count = 1 # required
|
||||
# ## interval, in s, at which to ping. 0 == default (ping -i <PING_INTERVAL>)
|
||||
# ping_interval = 0.0
|
||||
# ## per-ping timeout, in s. 0 == no timeout (ping -W <TIMEOUT>)
|
||||
# ## ping timeout, in s. 0 == no timeout (ping -W <TIMEOUT>)
|
||||
# timeout = 1.0
|
||||
# ## interface to send ping from (ping -I <INTERFACE>)
|
||||
# interface = ""
|
||||
@@ -1162,7 +1065,7 @@
|
||||
# ## to grab metrics for.
|
||||
# ##
|
||||
# address = "host=localhost user=postgres sslmode=disable"
|
||||
#
|
||||
#
|
||||
# ## A list of databases to pull metrics about. If not specified, metrics for all
|
||||
# ## databases are gathered.
|
||||
# # databases = ["app_production", "testing"]
|
||||
@@ -1244,7 +1147,7 @@
|
||||
# # pattern = "nginx"
|
||||
# ## user as argument for pgrep (ie, pgrep -u <user>)
|
||||
# # user = "nginx"
|
||||
#
|
||||
#
|
||||
# ## override for process_name
|
||||
# ## This is optional; default is sourced from /proc/<pid>/status
|
||||
# # process_name = "bar"
|
||||
@@ -1258,7 +1161,7 @@
|
||||
# [[inputs.prometheus]]
|
||||
# ## An array of urls to scrape metrics from.
|
||||
# urls = ["http://localhost:9100/metrics"]
|
||||
#
|
||||
#
|
||||
# ## Use SSL but skip chain & host verification
|
||||
# # insecure_skip_verify = false
|
||||
# ## Use bearer token for authorization
|
||||
@@ -1273,11 +1176,11 @@
|
||||
|
||||
# # Read metrics from one or many RabbitMQ servers via the management API
|
||||
# [[inputs.rabbitmq]]
|
||||
# # url = "http://localhost:15672"
|
||||
# url = "http://localhost:15672" # required
|
||||
# # name = "rmq-server-1" # optional tag
|
||||
# # username = "guest"
|
||||
# # password = "guest"
|
||||
#
|
||||
#
|
||||
# ## A list of nodes to pull metrics about. If not specified, metrics for
|
||||
# ## all nodes are gathered.
|
||||
# # nodes = ["rabbit@node1", "rabbit@node2"]
|
||||
@@ -1341,7 +1244,7 @@
|
||||
# collect = ["mybulk", "sysservices", "sysdescr"]
|
||||
# # Simple list of OIDs to get, in addition to "collect"
|
||||
# get_oids = []
|
||||
#
|
||||
#
|
||||
# [[inputs.snmp.host]]
|
||||
# address = "192.168.2.3:161"
|
||||
# community = "public"
|
||||
@@ -1353,31 +1256,31 @@
|
||||
# "ifNumber",
|
||||
# ".1.3.6.1.2.1.1.3.0",
|
||||
# ]
|
||||
#
|
||||
#
|
||||
# [[inputs.snmp.get]]
|
||||
# name = "ifnumber"
|
||||
# oid = "ifNumber"
|
||||
#
|
||||
#
|
||||
# [[inputs.snmp.get]]
|
||||
# name = "interface_speed"
|
||||
# oid = "ifSpeed"
|
||||
# instance = "0"
|
||||
#
|
||||
#
|
||||
# [[inputs.snmp.get]]
|
||||
# name = "sysuptime"
|
||||
# oid = ".1.3.6.1.2.1.1.3.0"
|
||||
# unit = "second"
|
||||
#
|
||||
#
|
||||
# [[inputs.snmp.bulk]]
|
||||
# name = "mybulk"
|
||||
# max_repetition = 127
|
||||
# oid = ".1.3.6.1.2.1.1"
|
||||
#
|
||||
#
|
||||
# [[inputs.snmp.bulk]]
|
||||
# name = "ifoutoctets"
|
||||
# max_repetition = 127
|
||||
# oid = "ifOutOctets"
|
||||
#
|
||||
#
|
||||
# [[inputs.snmp.host]]
|
||||
# address = "192.168.2.13:161"
|
||||
# #address = "127.0.0.1:161"
|
||||
@@ -1390,19 +1293,19 @@
|
||||
# [[inputs.snmp.host.table]]
|
||||
# name = "iftable3"
|
||||
# include_instances = ["enp5s0", "eth1"]
|
||||
#
|
||||
#
|
||||
# # SNMP TABLEs
|
||||
# # table without mapping neither subtables
|
||||
# [[inputs.snmp.table]]
|
||||
# name = "iftable1"
|
||||
# oid = ".1.3.6.1.2.1.31.1.1.1"
|
||||
#
|
||||
#
|
||||
# # table without mapping but with subtables
|
||||
# [[inputs.snmp.table]]
|
||||
# name = "iftable2"
|
||||
# oid = ".1.3.6.1.2.1.31.1.1.1"
|
||||
# sub_tables = [".1.3.6.1.2.1.2.2.1.13"]
|
||||
#
|
||||
#
|
||||
# # table with mapping but without subtables
|
||||
# [[inputs.snmp.table]]
|
||||
# name = "iftable3"
|
||||
@@ -1410,7 +1313,7 @@
|
||||
# # if empty. get all instances
|
||||
# mapping_table = ".1.3.6.1.2.1.31.1.1.1.1"
|
||||
# # if empty, get all subtables
|
||||
#
|
||||
#
|
||||
# # table with both mapping and subtables
|
||||
# [[inputs.snmp.table]]
|
||||
# name = "iftable4"
|
||||
@@ -1453,33 +1356,32 @@
|
||||
# [[inputs.varnish]]
|
||||
# ## The default location of the varnishstat binary can be overridden with:
|
||||
# binary = "/usr/bin/varnishstat"
|
||||
#
|
||||
#
|
||||
# ## By default, telegraf gather stats for 3 metric points.
|
||||
# ## Setting stats will override the defaults shown below.
|
||||
# ## Glob matching can be used, ie, stats = ["MAIN.*"]
|
||||
# ## stats may also be set to ["*"], which will collect all stats
|
||||
# ## stats may also be set to ["all"], which will collect all stats
|
||||
# stats = ["MAIN.cache_hit", "MAIN.cache_miss", "MAIN.uptime"]
|
||||
|
||||
|
||||
# # Read metrics of ZFS from arcstats, zfetchstats, vdev_cache_stats, and pools
|
||||
# # Read metrics of ZFS from arcstats, zfetchstats and vdev_cache_stats
|
||||
# [[inputs.zfs]]
|
||||
# ## ZFS kstat path. Ignored on FreeBSD
|
||||
# ## ZFS kstat path
|
||||
# ## If not specified, then default is:
|
||||
# # kstatPath = "/proc/spl/kstat/zfs"
|
||||
#
|
||||
# kstatPath = "/proc/spl/kstat/zfs"
|
||||
#
|
||||
# ## By default, telegraf gather all zfs stats
|
||||
# ## If not specified, then default is:
|
||||
# # kstatMetrics = ["arcstats", "zfetchstats", "vdev_cache_stats"]
|
||||
#
|
||||
# kstatMetrics = ["arcstats", "zfetchstats", "vdev_cache_stats"]
|
||||
#
|
||||
# ## By default, don't gather zpool stats
|
||||
# # poolMetrics = false
|
||||
# poolMetrics = false
|
||||
|
||||
|
||||
# # Reads 'mntr' stats from one or many zookeeper servers
|
||||
# [[inputs.zookeeper]]
|
||||
# ## An array of address to gather stats about. Specify an ip or hostname
|
||||
# ## with port. ie localhost:2181, 10.0.0.1:2181, etc.
|
||||
#
|
||||
#
|
||||
# ## If no servers are specified, then localhost is used as the host.
|
||||
# ## If no port is specified, 2181 is used
|
||||
# servers = [":2181"]
|
||||
@@ -1503,12 +1405,12 @@
|
||||
# ## an array of Zookeeper connection strings
|
||||
# zookeeper_peers = ["localhost:2181"]
|
||||
# ## Zookeeper Chroot
|
||||
# zookeeper_chroot = ""
|
||||
# zookeeper_chroot = "/"
|
||||
# ## the name of the consumer group
|
||||
# consumer_group = "telegraf_metrics_consumers"
|
||||
# ## Offset (must be either "oldest" or "newest")
|
||||
# offset = "oldest"
|
||||
#
|
||||
#
|
||||
# ## Data format to consume.
|
||||
# ## Each data format has it's own unique set of configuration options, read
|
||||
# ## more about them here:
|
||||
@@ -1516,66 +1418,37 @@
|
||||
# data_format = "influx"
|
||||
|
||||
|
||||
# # Stream and parse log file(s).
|
||||
# [[inputs.logparser]]
|
||||
# ## Log files to parse.
|
||||
# ## These accept standard unix glob matching rules, but with the addition of
|
||||
# ## ** as a "super asterisk". ie:
|
||||
# ## /var/log/**.log -> recursively find all .log files in /var/log
|
||||
# ## /var/log/*/*.log -> find all .log files with a parent dir in /var/log
|
||||
# ## /var/log/apache.log -> only tail the apache log file
|
||||
# files = ["/var/log/influxdb/influxdb.log"]
|
||||
# ## Read file from beginning.
|
||||
# from_beginning = false
|
||||
#
|
||||
# ## Parse logstash-style "grok" patterns:
|
||||
# ## Telegraf built-in parsing patterns: https://goo.gl/dkay10
|
||||
# [inputs.logparser.grok]
|
||||
# ## This is a list of patterns to check the given log file(s) for.
|
||||
# ## Note that adding patterns here increases processing time. The most
|
||||
# ## efficient configuration is to have one pattern per logparser.
|
||||
# ## Other common built-in patterns are:
|
||||
# ## %{COMMON_LOG_FORMAT} (plain apache & nginx access logs)
|
||||
# ## %{COMBINED_LOG_FORMAT} (access logs + referrer & agent)
|
||||
# patterns = ["%{INFLUXDB_HTTPD_LOG}"]
|
||||
# ## Full path(s) to custom pattern files.
|
||||
# custom_pattern_files = []
|
||||
# ## Custom patterns can also be defined here. Put one pattern per line.
|
||||
# custom_patterns = '''
|
||||
# '''
|
||||
|
||||
|
||||
# # Read metrics from MQTT topic(s)
|
||||
# [[inputs.mqtt_consumer]]
|
||||
# servers = ["localhost:1883"]
|
||||
# ## MQTT QoS, must be 0, 1, or 2
|
||||
# qos = 0
|
||||
#
|
||||
#
|
||||
# ## Topics to subscribe to
|
||||
# topics = [
|
||||
# "telegraf/host01/cpu",
|
||||
# "telegraf/+/mem",
|
||||
# "sensors/#",
|
||||
# ]
|
||||
#
|
||||
#
|
||||
# # if true, messages that can't be delivered while the subscriber is offline
|
||||
# # will be delivered when it comes back (such as on service restart).
|
||||
# # NOTE: if true, client_id MUST be set
|
||||
# persistent_session = false
|
||||
# # If empty, a random client ID will be generated.
|
||||
# client_id = ""
|
||||
#
|
||||
#
|
||||
# ## username and password to connect MQTT server.
|
||||
# # username = "telegraf"
|
||||
# # password = "metricsmetricsmetricsmetrics"
|
||||
#
|
||||
#
|
||||
# ## Optional SSL Config
|
||||
# # ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# # ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# # ssl_key = "/etc/telegraf/key.pem"
|
||||
# ## Use SSL but skip chain & host verification
|
||||
# # insecure_skip_verify = false
|
||||
#
|
||||
#
|
||||
# ## Data format to consume.
|
||||
# ## Each data format has it's own unique set of configuration options, read
|
||||
# ## more about them here:
|
||||
@@ -1593,7 +1466,7 @@
|
||||
# subjects = ["telegraf"]
|
||||
# ## name a queue group
|
||||
# queue_group = "telegraf_consumers"
|
||||
#
|
||||
#
|
||||
# ## Data format to consume.
|
||||
# ## Each data format has it's own unique set of configuration options, read
|
||||
# ## more about them here:
|
||||
@@ -1601,12 +1474,6 @@
|
||||
# data_format = "influx"
|
||||
|
||||
|
||||
# # A Rollbar Webhook Event collector
|
||||
# [[inputs.rollbar_webhooks]]
|
||||
# ## Address and port to host Webhook listener on
|
||||
# service_address = ":1619"
|
||||
|
||||
|
||||
# # Statsd Server
|
||||
# [[inputs.statsd]]
|
||||
# ## Address and port to host UDP listener on
|
||||
@@ -1621,24 +1488,24 @@
|
||||
# delete_timings = true
|
||||
# ## Percentiles to calculate for timing & histogram stats
|
||||
# percentiles = [90]
|
||||
#
|
||||
#
|
||||
# ## separator to use between elements of a statsd metric
|
||||
# metric_separator = "_"
|
||||
#
|
||||
#
|
||||
# ## Parses tags in the datadog statsd format
|
||||
# ## http://docs.datadoghq.com/guides/dogstatsd/
|
||||
# parse_data_dog_tags = false
|
||||
#
|
||||
#
|
||||
# ## Statsd data translation templates, more info can be read here:
|
||||
# ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md#graphite
|
||||
# # templates = [
|
||||
# # "cpu.* measurement*"
|
||||
# # ]
|
||||
#
|
||||
#
|
||||
# ## Number of UDP messages allowed to queue up, once filled,
|
||||
# ## the statsd server will start dropping packets
|
||||
# allowed_pending_messages = 10000
|
||||
#
|
||||
#
|
||||
# ## Number of timing/histogram values to track per-measurement in the
|
||||
# ## calculation of percentiles. Raising this limit increases the accuracy
|
||||
# ## of percentiles but also increases the memory usage and cpu time.
|
||||
@@ -1659,7 +1526,7 @@
|
||||
# files = ["/var/mymetrics.out"]
|
||||
# ## Read file from beginning.
|
||||
# from_beginning = false
|
||||
#
|
||||
#
|
||||
# ## Data format to consume.
|
||||
# ## Each data format has it's own unique set of configuration options, read
|
||||
# ## more about them here:
|
||||
@@ -1671,14 +1538,14 @@
|
||||
# [[inputs.tcp_listener]]
|
||||
# ## Address and port to host TCP listener on
|
||||
# service_address = ":8094"
|
||||
#
|
||||
#
|
||||
# ## Number of TCP messages allowed to queue up. Once filled, the
|
||||
# ## TCP listener will start dropping packets.
|
||||
# allowed_pending_messages = 10000
|
||||
#
|
||||
#
|
||||
# ## Maximum number of concurrent TCP connections to allow
|
||||
# max_tcp_connections = 250
|
||||
#
|
||||
#
|
||||
# ## Data format to consume.
|
||||
# ## Each data format has it's own unique set of configuration options, read
|
||||
# ## more about them here:
|
||||
@@ -1690,11 +1557,11 @@
|
||||
# [[inputs.udp_listener]]
|
||||
# ## Address and port to host UDP listener on
|
||||
# service_address = ":8092"
|
||||
#
|
||||
#
|
||||
# ## Number of UDP messages allowed to queue up. Once filled, the
|
||||
# ## UDP listener will start dropping packets.
|
||||
# allowed_pending_messages = 10000
|
||||
#
|
||||
#
|
||||
# ## Data format to consume.
|
||||
# ## Each data format has it's own unique set of configuration options, read
|
||||
# ## more about them here:
|
||||
|
||||
@@ -1,79 +0,0 @@
|
||||
package filter
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"github.com/gobwas/glob"
|
||||
)
|
||||
|
||||
type Filter interface {
|
||||
Match(string) bool
|
||||
}
|
||||
|
||||
// CompileFilter takes a list of string filters and returns a Filter interface
|
||||
// for matching a given string against the filter list. The filter list
|
||||
// supports glob matching too, ie:
|
||||
//
|
||||
// f, _ := CompileFilter([]string{"cpu", "mem", "net*"})
|
||||
// f.Match("cpu") // true
|
||||
// f.Match("network") // true
|
||||
// f.Match("memory") // false
|
||||
//
|
||||
func CompileFilter(filters []string) (Filter, error) {
|
||||
// return if there is nothing to compile
|
||||
if len(filters) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
// check if we can compile a non-glob filter
|
||||
noGlob := true
|
||||
for _, filter := range filters {
|
||||
if hasMeta(filter) {
|
||||
noGlob = false
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
switch {
|
||||
case noGlob:
|
||||
// return non-globbing filter if not needed.
|
||||
return compileFilterNoGlob(filters), nil
|
||||
case len(filters) == 1:
|
||||
return glob.Compile(filters[0])
|
||||
default:
|
||||
return glob.Compile("{" + strings.Join(filters, ",") + "}")
|
||||
}
|
||||
}
|
||||
|
||||
// hasMeta reports whether path contains any magic glob characters.
|
||||
func hasMeta(s string) bool {
|
||||
return strings.IndexAny(s, "*?[") >= 0
|
||||
}
|
||||
|
||||
type filter struct {
|
||||
m map[string]struct{}
|
||||
}
|
||||
|
||||
func (f *filter) Match(s string) bool {
|
||||
_, ok := f.m[s]
|
||||
return ok
|
||||
}
|
||||
|
||||
type filtersingle struct {
|
||||
s string
|
||||
}
|
||||
|
||||
func (f *filtersingle) Match(s string) bool {
|
||||
return f.s == s
|
||||
}
|
||||
|
||||
func compileFilterNoGlob(filters []string) Filter {
|
||||
if len(filters) == 1 {
|
||||
return &filtersingle{s: filters[0]}
|
||||
}
|
||||
out := filter{m: make(map[string]struct{})}
|
||||
for _, filter := range filters {
|
||||
out.m[filter] = struct{}{}
|
||||
}
|
||||
return &out
|
||||
}
|
||||
@@ -1,96 +0,0 @@
|
||||
package filter
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestCompileFilter(t *testing.T) {
|
||||
f, err := CompileFilter([]string{})
|
||||
assert.NoError(t, err)
|
||||
assert.Nil(t, f)
|
||||
|
||||
f, err = CompileFilter([]string{"cpu"})
|
||||
assert.NoError(t, err)
|
||||
assert.True(t, f.Match("cpu"))
|
||||
assert.False(t, f.Match("cpu0"))
|
||||
assert.False(t, f.Match("mem"))
|
||||
|
||||
f, err = CompileFilter([]string{"cpu*"})
|
||||
assert.NoError(t, err)
|
||||
assert.True(t, f.Match("cpu"))
|
||||
assert.True(t, f.Match("cpu0"))
|
||||
assert.False(t, f.Match("mem"))
|
||||
|
||||
f, err = CompileFilter([]string{"cpu", "mem"})
|
||||
assert.NoError(t, err)
|
||||
assert.True(t, f.Match("cpu"))
|
||||
assert.False(t, f.Match("cpu0"))
|
||||
assert.True(t, f.Match("mem"))
|
||||
|
||||
f, err = CompileFilter([]string{"cpu", "mem", "net*"})
|
||||
assert.NoError(t, err)
|
||||
assert.True(t, f.Match("cpu"))
|
||||
assert.False(t, f.Match("cpu0"))
|
||||
assert.True(t, f.Match("mem"))
|
||||
assert.True(t, f.Match("network"))
|
||||
}
|
||||
|
||||
var benchbool bool
|
||||
|
||||
func BenchmarkFilterSingleNoGlobFalse(b *testing.B) {
|
||||
f, _ := CompileFilter([]string{"cpu"})
|
||||
var tmp bool
|
||||
for n := 0; n < b.N; n++ {
|
||||
tmp = f.Match("network")
|
||||
}
|
||||
benchbool = tmp
|
||||
}
|
||||
|
||||
func BenchmarkFilterSingleNoGlobTrue(b *testing.B) {
|
||||
f, _ := CompileFilter([]string{"cpu"})
|
||||
var tmp bool
|
||||
for n := 0; n < b.N; n++ {
|
||||
tmp = f.Match("cpu")
|
||||
}
|
||||
benchbool = tmp
|
||||
}
|
||||
|
||||
func BenchmarkFilter(b *testing.B) {
|
||||
f, _ := CompileFilter([]string{"cpu", "mem", "net*"})
|
||||
var tmp bool
|
||||
for n := 0; n < b.N; n++ {
|
||||
tmp = f.Match("network")
|
||||
}
|
||||
benchbool = tmp
|
||||
}
|
||||
|
||||
func BenchmarkFilterNoGlob(b *testing.B) {
|
||||
f, _ := CompileFilter([]string{"cpu", "mem", "net"})
|
||||
var tmp bool
|
||||
for n := 0; n < b.N; n++ {
|
||||
tmp = f.Match("net")
|
||||
}
|
||||
benchbool = tmp
|
||||
}
|
||||
|
||||
func BenchmarkFilter2(b *testing.B) {
|
||||
f, _ := CompileFilter([]string{"aa", "bb", "c", "ad", "ar", "at", "aq",
|
||||
"aw", "az", "axxx", "ab", "cpu", "mem", "net*"})
|
||||
var tmp bool
|
||||
for n := 0; n < b.N; n++ {
|
||||
tmp = f.Match("network")
|
||||
}
|
||||
benchbool = tmp
|
||||
}
|
||||
|
||||
func BenchmarkFilter2NoGlob(b *testing.B) {
|
||||
f, _ := CompileFilter([]string{"aa", "bb", "c", "ad", "ar", "at", "aq",
|
||||
"aw", "az", "axxx", "ab", "cpu", "mem", "net"})
|
||||
var tmp bool
|
||||
for n := 0; n < b.N; n++ {
|
||||
tmp = f.Match("net")
|
||||
}
|
||||
benchbool = tmp
|
||||
}
|
||||
@@ -1,49 +0,0 @@
|
||||
package aws
|
||||
|
||||
import (
|
||||
"github.com/aws/aws-sdk-go/aws"
|
||||
"github.com/aws/aws-sdk-go/aws/client"
|
||||
"github.com/aws/aws-sdk-go/aws/credentials"
|
||||
"github.com/aws/aws-sdk-go/aws/credentials/stscreds"
|
||||
"github.com/aws/aws-sdk-go/aws/session"
|
||||
)
|
||||
|
||||
type CredentialConfig struct {
|
||||
Region string
|
||||
AccessKey string
|
||||
SecretKey string
|
||||
RoleARN string
|
||||
Profile string
|
||||
Filename string
|
||||
Token string
|
||||
}
|
||||
|
||||
func (c *CredentialConfig) Credentials() client.ConfigProvider {
|
||||
if c.RoleARN != "" {
|
||||
return c.assumeCredentials()
|
||||
} else {
|
||||
return c.rootCredentials()
|
||||
}
|
||||
}
|
||||
|
||||
func (c *CredentialConfig) rootCredentials() client.ConfigProvider {
|
||||
config := &aws.Config{
|
||||
Region: aws.String(c.Region),
|
||||
}
|
||||
if c.AccessKey != "" || c.SecretKey != "" {
|
||||
config.Credentials = credentials.NewStaticCredentials(c.AccessKey, c.SecretKey, c.Token)
|
||||
} else if c.Profile != "" || c.Filename != "" {
|
||||
config.Credentials = credentials.NewSharedCredentials(c.Filename, c.Profile)
|
||||
}
|
||||
|
||||
return session.New(config)
|
||||
}
|
||||
|
||||
func (c *CredentialConfig) assumeCredentials() client.ConfigProvider {
|
||||
rootCredentials := c.rootCredentials()
|
||||
config := &aws.Config{
|
||||
Region: aws.String(c.Region),
|
||||
}
|
||||
config.Credentials = stscreds.NewCredentials(rootCredentials, c.RoleARN)
|
||||
return session.New(config)
|
||||
}
|
||||
@@ -58,6 +58,7 @@ func NewConfig() *Config {
|
||||
Interval: internal.Duration{Duration: 10 * time.Second},
|
||||
RoundInterval: true,
|
||||
FlushInterval: internal.Duration{Duration: 10 * time.Second},
|
||||
FlushJitter: internal.Duration{Duration: 5 * time.Second},
|
||||
},
|
||||
|
||||
Tags: make(map[string]string),
|
||||
@@ -77,14 +78,6 @@ type AgentConfig struct {
|
||||
// ie, if Interval=10s then always collect on :00, :10, :20, etc.
|
||||
RoundInterval bool
|
||||
|
||||
// By default, precision will be set to the same timestamp order as the
|
||||
// collection interval, with the maximum being 1s.
|
||||
// ie, when interval = "10s", precision will be "1s"
|
||||
// when interval = "250ms", precision will be "1ms"
|
||||
// Precision will NOT be used for service inputs. It is up to each individual
|
||||
// service input to set the timestamp at the appropriate precision.
|
||||
Precision internal.Duration
|
||||
|
||||
// CollectionJitter is used to jitter the collection by a random amount.
|
||||
// Each plugin will sleep for a random time within jitter before collecting.
|
||||
// This can be used to avoid many plugins querying things like sysfs at the
|
||||
@@ -116,10 +109,11 @@ type AgentConfig struct {
|
||||
// does _not_ deactivate FlushInterval.
|
||||
FlushBufferWhenFull bool
|
||||
|
||||
// TODO(cam): Remove UTC and parameter, they are no longer
|
||||
// TODO(cam): Remove UTC and Precision parameters, they are no longer
|
||||
// valid for the agent config. Leaving them here for now for backwards-
|
||||
// compatability
|
||||
UTC bool `toml:"utc"`
|
||||
UTC bool `toml:"utc"`
|
||||
Precision string
|
||||
|
||||
// Debug is the option for running in debug mode
|
||||
Debug bool
|
||||
@@ -216,11 +210,6 @@ var header = `# Telegraf Configuration
|
||||
## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
|
||||
flush_jitter = "0s"
|
||||
|
||||
## By default, precision will be set to the same timestamp order as the
|
||||
## collection interval, with the maximum being 1s.
|
||||
## Precision will NOT be used for service inputs, such as logparser and statsd.
|
||||
## Valid values are "Nns", "Nus" (or "Nµs"), "Nms", "Ns".
|
||||
precision = ""
|
||||
## Run telegraf in debug mode
|
||||
debug = false
|
||||
## Run telegraf in quiet mode
|
||||
@@ -368,7 +357,7 @@ func printConfig(name string, p printer, op string, commented bool) {
|
||||
fmt.Print("\n")
|
||||
continue
|
||||
}
|
||||
fmt.Print(strings.TrimRight(comment+line, " ") + "\n")
|
||||
fmt.Print(comment + line + "\n")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,37 +0,0 @@
|
||||
package errchan
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type ErrChan struct {
|
||||
C chan error
|
||||
}
|
||||
|
||||
// New returns an error channel of max length 'n'
|
||||
// errors can be sent to the ErrChan.C channel, and will be returned when
|
||||
// ErrChan.Error() is called.
|
||||
func New(n int) *ErrChan {
|
||||
return &ErrChan{
|
||||
C: make(chan error, n),
|
||||
}
|
||||
}
|
||||
|
||||
// Error closes the ErrChan.C channel and returns an error if there are any
|
||||
// non-nil errors, otherwise returns nil.
|
||||
func (e *ErrChan) Error() error {
|
||||
close(e.C)
|
||||
|
||||
var out string
|
||||
for err := range e.C {
|
||||
if err != nil {
|
||||
out += "[" + err.Error() + "], "
|
||||
}
|
||||
}
|
||||
|
||||
if out != "" {
|
||||
return fmt.Errorf("Errors encountered: " + strings.TrimRight(out, ", "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -10,7 +10,6 @@ import (
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"log"
|
||||
"math/big"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strconv"
|
||||
@@ -206,27 +205,3 @@ func WaitTimeout(c *exec.Cmd, timeout time.Duration) error {
|
||||
return TimeoutErr
|
||||
}
|
||||
}
|
||||
|
||||
// RandomSleep will sleep for a random amount of time up to max.
|
||||
// If the shutdown channel is closed, it will return before it has finished
|
||||
// sleeping.
|
||||
func RandomSleep(max time.Duration, shutdown chan struct{}) {
|
||||
if max == 0 {
|
||||
return
|
||||
}
|
||||
maxSleep := big.NewInt(max.Nanoseconds())
|
||||
|
||||
var sleepns int64
|
||||
if j, err := rand.Int(rand.Reader, maxSleep); err == nil {
|
||||
sleepns = j.Int64()
|
||||
}
|
||||
|
||||
t := time.NewTimer(time.Nanosecond * time.Duration(sleepns))
|
||||
select {
|
||||
case <-t.C:
|
||||
return
|
||||
case <-shutdown:
|
||||
t.Stop()
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
@@ -106,28 +106,3 @@ func TestRunError(t *testing.T) {
|
||||
|
||||
assert.Error(t, err)
|
||||
}
|
||||
|
||||
func TestRandomSleep(t *testing.T) {
|
||||
// test that zero max returns immediately
|
||||
s := time.Now()
|
||||
RandomSleep(time.Duration(0), make(chan struct{}))
|
||||
elapsed := time.Since(s)
|
||||
assert.True(t, elapsed < time.Millisecond)
|
||||
|
||||
// test that max sleep is respected
|
||||
s = time.Now()
|
||||
RandomSleep(time.Millisecond*50, make(chan struct{}))
|
||||
elapsed = time.Since(s)
|
||||
assert.True(t, elapsed < time.Millisecond*50)
|
||||
|
||||
// test that shutdown is respected
|
||||
s = time.Now()
|
||||
shutdown := make(chan struct{})
|
||||
go func() {
|
||||
time.Sleep(time.Millisecond * 100)
|
||||
close(shutdown)
|
||||
}()
|
||||
RandomSleep(time.Second, shutdown)
|
||||
elapsed = time.Since(s)
|
||||
assert.True(t, elapsed < time.Millisecond*150)
|
||||
}
|
||||
|
||||
@@ -1,59 +0,0 @@
|
||||
package limiter
|
||||
|
||||
import (
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// NewRateLimiter returns a rate limiter that will will emit from the C
|
||||
// channel only 'n' times every 'rate' seconds.
|
||||
func NewRateLimiter(n int, rate time.Duration) *rateLimiter {
|
||||
r := &rateLimiter{
|
||||
C: make(chan bool),
|
||||
rate: rate,
|
||||
n: n,
|
||||
shutdown: make(chan bool),
|
||||
}
|
||||
r.wg.Add(1)
|
||||
go r.limiter()
|
||||
return r
|
||||
}
|
||||
|
||||
type rateLimiter struct {
|
||||
C chan bool
|
||||
rate time.Duration
|
||||
n int
|
||||
|
||||
shutdown chan bool
|
||||
wg sync.WaitGroup
|
||||
}
|
||||
|
||||
func (r *rateLimiter) Stop() {
|
||||
close(r.shutdown)
|
||||
r.wg.Wait()
|
||||
close(r.C)
|
||||
}
|
||||
|
||||
func (r *rateLimiter) limiter() {
|
||||
defer r.wg.Done()
|
||||
ticker := time.NewTicker(r.rate)
|
||||
defer ticker.Stop()
|
||||
counter := 0
|
||||
for {
|
||||
select {
|
||||
case <-r.shutdown:
|
||||
return
|
||||
case <-ticker.C:
|
||||
counter = 0
|
||||
default:
|
||||
if counter < r.n {
|
||||
select {
|
||||
case r.C <- true:
|
||||
counter++
|
||||
case <-r.shutdown:
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,54 +0,0 @@
|
||||
package limiter
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestRateLimiter(t *testing.T) {
|
||||
r := NewRateLimiter(5, time.Second)
|
||||
ticker := time.NewTicker(time.Millisecond * 75)
|
||||
|
||||
// test that we can only get 5 receives from the rate limiter
|
||||
counter := 0
|
||||
outer:
|
||||
for {
|
||||
select {
|
||||
case <-r.C:
|
||||
counter++
|
||||
case <-ticker.C:
|
||||
break outer
|
||||
}
|
||||
}
|
||||
|
||||
assert.Equal(t, 5, counter)
|
||||
r.Stop()
|
||||
// verify that the Stop function closes the channel.
|
||||
_, ok := <-r.C
|
||||
assert.False(t, ok)
|
||||
}
|
||||
|
||||
func TestRateLimiterMultipleIterations(t *testing.T) {
|
||||
r := NewRateLimiter(5, time.Millisecond*50)
|
||||
ticker := time.NewTicker(time.Millisecond * 250)
|
||||
|
||||
// test that we can get 15 receives from the rate limiter
|
||||
counter := 0
|
||||
outer:
|
||||
for {
|
||||
select {
|
||||
case <-ticker.C:
|
||||
break outer
|
||||
case <-r.C:
|
||||
counter++
|
||||
}
|
||||
}
|
||||
|
||||
assert.True(t, counter > 10)
|
||||
r.Stop()
|
||||
// verify that the Stop function closes the channel.
|
||||
_, ok := <-r.C
|
||||
assert.False(t, ok)
|
||||
}
|
||||
@@ -2,79 +2,81 @@ package internal_models
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/gobwas/glob"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/filter"
|
||||
)
|
||||
|
||||
// TagFilter is the name of a tag, and the values on which to filter
|
||||
type TagFilter struct {
|
||||
Name string
|
||||
Filter []string
|
||||
filter filter.Filter
|
||||
filter glob.Glob
|
||||
}
|
||||
|
||||
// Filter containing drop/pass and tagdrop/tagpass rules
|
||||
type Filter struct {
|
||||
NameDrop []string
|
||||
nameDrop filter.Filter
|
||||
nameDrop glob.Glob
|
||||
NamePass []string
|
||||
namePass filter.Filter
|
||||
namePass glob.Glob
|
||||
|
||||
FieldDrop []string
|
||||
fieldDrop filter.Filter
|
||||
fieldDrop glob.Glob
|
||||
FieldPass []string
|
||||
fieldPass filter.Filter
|
||||
fieldPass glob.Glob
|
||||
|
||||
TagDrop []TagFilter
|
||||
TagPass []TagFilter
|
||||
|
||||
TagExclude []string
|
||||
tagExclude filter.Filter
|
||||
tagExclude glob.Glob
|
||||
TagInclude []string
|
||||
tagInclude filter.Filter
|
||||
tagInclude glob.Glob
|
||||
|
||||
IsActive bool
|
||||
}
|
||||
|
||||
// Compile all Filter lists into filter.Filter objects.
|
||||
// Compile all Filter lists into glob.Glob objects.
|
||||
func (f *Filter) CompileFilter() error {
|
||||
var err error
|
||||
f.nameDrop, err = filter.CompileFilter(f.NameDrop)
|
||||
f.nameDrop, err = compileFilter(f.NameDrop)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'namedrop', %s", err)
|
||||
}
|
||||
f.namePass, err = filter.CompileFilter(f.NamePass)
|
||||
f.namePass, err = compileFilter(f.NamePass)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'namepass', %s", err)
|
||||
}
|
||||
|
||||
f.fieldDrop, err = filter.CompileFilter(f.FieldDrop)
|
||||
f.fieldDrop, err = compileFilter(f.FieldDrop)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'fielddrop', %s", err)
|
||||
}
|
||||
f.fieldPass, err = filter.CompileFilter(f.FieldPass)
|
||||
f.fieldPass, err = compileFilter(f.FieldPass)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'fieldpass', %s", err)
|
||||
}
|
||||
|
||||
f.tagExclude, err = filter.CompileFilter(f.TagExclude)
|
||||
f.tagExclude, err = compileFilter(f.TagExclude)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'tagexclude', %s", err)
|
||||
}
|
||||
f.tagInclude, err = filter.CompileFilter(f.TagInclude)
|
||||
f.tagInclude, err = compileFilter(f.TagInclude)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'taginclude', %s", err)
|
||||
}
|
||||
|
||||
for i, _ := range f.TagDrop {
|
||||
f.TagDrop[i].filter, err = filter.CompileFilter(f.TagDrop[i].Filter)
|
||||
f.TagDrop[i].filter, err = compileFilter(f.TagDrop[i].Filter)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'tagdrop', %s", err)
|
||||
}
|
||||
}
|
||||
for i, _ := range f.TagPass {
|
||||
f.TagPass[i].filter, err = filter.CompileFilter(f.TagPass[i].Filter)
|
||||
f.TagPass[i].filter, err = compileFilter(f.TagPass[i].Filter)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'tagpass', %s", err)
|
||||
}
|
||||
@@ -82,6 +84,20 @@ func (f *Filter) CompileFilter() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func compileFilter(filter []string) (glob.Glob, error) {
|
||||
if len(filter) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
var g glob.Glob
|
||||
var err error
|
||||
if len(filter) == 1 {
|
||||
g, err = glob.Compile(filter[0])
|
||||
} else {
|
||||
g, err = glob.Compile("{" + strings.Join(filter, ",") + "}")
|
||||
}
|
||||
return g, err
|
||||
}
|
||||
|
||||
func (f *Filter) ShouldMetricPass(metric telegraf.Metric) bool {
|
||||
if f.ShouldNamePass(metric.Name()) && f.ShouldTagsPass(metric.Tags()) {
|
||||
return true
|
||||
|
||||
@@ -253,6 +253,51 @@ func TestFilter_TagDrop(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestFilter_CompileFilterError(t *testing.T) {
|
||||
f := Filter{
|
||||
NameDrop: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
NamePass: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
FieldDrop: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
FieldPass: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
TagExclude: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
TagInclude: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
filters := []TagFilter{
|
||||
TagFilter{
|
||||
Name: "cpu",
|
||||
Filter: []string{"{foobar}"},
|
||||
}}
|
||||
f = Filter{
|
||||
TagDrop: filters,
|
||||
}
|
||||
require.Error(t, f.CompileFilter())
|
||||
filters = []TagFilter{
|
||||
TagFilter{
|
||||
Name: "cpu",
|
||||
Filter: []string{"{foobar}"},
|
||||
}}
|
||||
f = Filter{
|
||||
TagPass: filters,
|
||||
}
|
||||
require.Error(t, f.CompileFilter())
|
||||
}
|
||||
|
||||
func TestFilter_ShouldMetricsPass(t *testing.T) {
|
||||
m := testutil.TestMetric(1, "testmetric")
|
||||
f := Filter{
|
||||
|
||||
@@ -45,9 +45,14 @@ func NewMetric(
|
||||
name string,
|
||||
tags map[string]string,
|
||||
fields map[string]interface{},
|
||||
t time.Time,
|
||||
t ...time.Time,
|
||||
) (Metric, error) {
|
||||
pt, err := client.NewPoint(name, tags, fields, t)
|
||||
var T time.Time
|
||||
if len(t) > 0 {
|
||||
T = t[0]
|
||||
}
|
||||
|
||||
pt, err := client.NewPoint(name, tags, fields, T)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -51,6 +51,23 @@ func TestNewMetricString(t *testing.T) {
|
||||
assert.Equal(t, lineProtoPrecision, m.PrecisionString("s"))
|
||||
}
|
||||
|
||||
func TestNewMetricStringNoTime(t *testing.T) {
|
||||
tags := map[string]string{
|
||||
"host": "localhost",
|
||||
}
|
||||
fields := map[string]interface{}{
|
||||
"usage_idle": float64(99),
|
||||
}
|
||||
m, err := NewMetric("cpu", tags, fields)
|
||||
assert.NoError(t, err)
|
||||
|
||||
lineProto := fmt.Sprintf("cpu,host=localhost usage_idle=99")
|
||||
assert.Equal(t, lineProto, m.String())
|
||||
|
||||
lineProtoPrecision := fmt.Sprintf("cpu,host=localhost usage_idle=99")
|
||||
assert.Equal(t, lineProtoPrecision, m.PrecisionString("s"))
|
||||
}
|
||||
|
||||
func TestNewMetricFailNaN(t *testing.T) {
|
||||
now := time.Now()
|
||||
|
||||
|
||||
@@ -8,8 +8,6 @@ import (
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/ceph"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/chrony"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/cloudwatch"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/conntrack"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/consul"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/couchbase"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/couchdb"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/disque"
|
||||
@@ -20,7 +18,6 @@ import (
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/exec"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/filestat"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/github_webhooks"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/graylog"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/haproxy"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/http_response"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/httpjson"
|
||||
@@ -29,7 +26,6 @@ import (
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/jolokia"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/kafka_consumer"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/leofs"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/logparser"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/lustre2"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/mailchimp"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/memcached"
|
||||
@@ -57,7 +53,6 @@ import (
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/redis"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/rethinkdb"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/riak"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/rollbar_webhooks"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/sensors"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/snmp"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/sqlserver"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Telegraf plugin: Apache
|
||||
|
||||
#### Plugin arguments:
|
||||
- **urls** []string: List of apache-status URLs to collect from. Default is "http://localhost/server-status?auto".
|
||||
- **urls** []string: List of apache-status URLs to collect from.
|
||||
|
||||
#### Description
|
||||
|
||||
|
||||
@@ -21,7 +21,6 @@ type Apache struct {
|
||||
|
||||
var sampleConfig = `
|
||||
## An array of Apache status URI to gather stats.
|
||||
## Default is "http://localhost/server-status?auto".
|
||||
urls = ["http://localhost/server-status?auto"]
|
||||
`
|
||||
|
||||
@@ -34,10 +33,6 @@ func (n *Apache) Description() string {
|
||||
}
|
||||
|
||||
func (n *Apache) Gather(acc telegraf.Accumulator) error {
|
||||
if len(n.Urls) == 0 {
|
||||
n.Urls = []string{"http://localhost/server-status?auto"}
|
||||
}
|
||||
|
||||
var wg sync.WaitGroup
|
||||
var outerr error
|
||||
|
||||
|
||||
@@ -40,7 +40,7 @@ is computed for the new frequency, with weights depending on these accuracies. I
|
||||
measurements from the reference source follow a consistent trend, the residual will be
|
||||
driven to zero over time.
|
||||
- Skew - This is the estimated error bound on the frequency.
|
||||
- Root delay - This is the total of the network path delays to the stratum-1 computer
|
||||
- Root delay -This is the total of the network path delays to the stratum-1 computer
|
||||
from which the computer is ultimately synchronised. In certain extreme situations, this
|
||||
value can be negative. (This can arise in a symmetric peer arrangement where the computers’
|
||||
frequencies are not tracking each other and the network delay is very short relative to the
|
||||
@@ -56,8 +56,7 @@ Delete second or Not synchronised.
|
||||
```toml
|
||||
# Get standard chrony metrics, requires chronyc executable.
|
||||
[[inputs.chrony]]
|
||||
## If true, chronyc tries to perform a DNS lookup for the time server.
|
||||
# dns_lookup = false
|
||||
# no configuration
|
||||
```
|
||||
|
||||
### Measurements & Fields:
|
||||
|
||||
@@ -20,8 +20,7 @@ var (
|
||||
)
|
||||
|
||||
type Chrony struct {
|
||||
DNSLookup bool `toml:"dns_lookup"`
|
||||
path string
|
||||
path string
|
||||
}
|
||||
|
||||
func (*Chrony) Description() string {
|
||||
@@ -29,24 +28,14 @@ func (*Chrony) Description() string {
|
||||
}
|
||||
|
||||
func (*Chrony) SampleConfig() string {
|
||||
return `
|
||||
## If true, chronyc tries to perform a DNS lookup for the time server.
|
||||
# dns_lookup = false
|
||||
`
|
||||
return ""
|
||||
}
|
||||
|
||||
func (c *Chrony) Gather(acc telegraf.Accumulator) error {
|
||||
if len(c.path) == 0 {
|
||||
return errors.New("chronyc not found: verify that chrony is installed and that chronyc is in your PATH")
|
||||
}
|
||||
|
||||
flags := []string{}
|
||||
if !c.DNSLookup {
|
||||
flags = append(flags, "-n")
|
||||
}
|
||||
flags = append(flags, "tracking")
|
||||
|
||||
cmd := execCommand(c.path, flags...)
|
||||
cmd := execCommand(c.path, "tracking")
|
||||
out, err := internal.CombinedOutputTimeout(cmd, time.Second*5)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to run command %s: %s - %s", strings.Join(cmd.Args, " "), err, string(out))
|
||||
|
||||
@@ -42,15 +42,6 @@ func TestGather(t *testing.T) {
|
||||
}
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "chrony", fields, tags)
|
||||
|
||||
// test with dns lookup
|
||||
c.DNSLookup = true
|
||||
err = c.Gather(&acc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "chrony", fields, tags)
|
||||
|
||||
}
|
||||
|
||||
// fackeExecCommand is a helper function that mock
|
||||
@@ -72,9 +63,8 @@ func TestHelperProcess(t *testing.T) {
|
||||
return
|
||||
}
|
||||
|
||||
lookup := "Reference ID : 192.168.1.22 (ntp.example.com)\n"
|
||||
noLookup := "Reference ID : 192.168.1.22 (192.168.1.22)\n"
|
||||
mockData := `Stratum : 3
|
||||
mockData := `Reference ID : 192.168.1.22 (ntp.example.com)
|
||||
Stratum : 3
|
||||
Ref time (UTC) : Thu May 12 14:27:07 2016
|
||||
System time : 0.000020390 seconds fast of NTP time
|
||||
Last offset : +0.000012651 seconds
|
||||
@@ -94,12 +84,8 @@ Leap status : Normal
|
||||
// /tmp/go-build970079519/…/_test/integration.test -test.run=TestHelperProcess --
|
||||
cmd, args := args[3], args[4:]
|
||||
|
||||
if cmd == "chronyc" {
|
||||
if args[0] == "tracking" {
|
||||
fmt.Fprint(os.Stdout, lookup+mockData)
|
||||
} else {
|
||||
fmt.Fprint(os.Stdout, noLookup+mockData)
|
||||
}
|
||||
if cmd == "chronyc" && args[0] == "tracking" {
|
||||
fmt.Fprint(os.Stdout, mockData)
|
||||
} else {
|
||||
fmt.Fprint(os.Stdout, "command not found")
|
||||
os.Exit(1)
|
||||
|
||||
@@ -6,12 +6,9 @@ This plugin will pull Metric Statistics from Amazon CloudWatch.
|
||||
|
||||
This plugin uses a credential chain for Authentication with the CloudWatch
|
||||
API endpoint. In the following order the plugin will attempt to authenticate.
|
||||
1. Assumed credentials via STS if `role_arn` attribute is specified (source credentials are evaluated from subsequent rules)
|
||||
2. Explicit credentials from `access_key`, `secret_key`, and `token` attributes
|
||||
3. Shared profile from `profile` attribute
|
||||
4. [Environment Variables](https://github.com/aws/aws-sdk-go/wiki/configuring-sdk#environment-variables)
|
||||
5. [Shared Credentials](https://github.com/aws/aws-sdk-go/wiki/configuring-sdk#shared-credentials-file)
|
||||
6. [EC2 Instance Profile](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html)
|
||||
1. [IAMS Role](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html)
|
||||
2. [Environment Variables](https://github.com/aws/aws-sdk-go/wiki/configuring-sdk#environment-variables)
|
||||
3. [Shared Credentials](https://github.com/aws/aws-sdk-go/wiki/configuring-sdk#shared-credentials-file)
|
||||
|
||||
### Configuration:
|
||||
|
||||
@@ -27,7 +24,7 @@ API endpoint. In the following order the plugin will attempt to authenticate.
|
||||
delay = '1m'
|
||||
|
||||
## Override global run interval (optional - defaults to global interval)
|
||||
## Recomended: use metric 'interval' that is a multiple of 'period' to avoid
|
||||
## Recomended: use metric 'interval' that is a multiple of 'period' to avoid
|
||||
## gaps or overlap in pulled data
|
||||
interval = '1m'
|
||||
|
||||
@@ -39,15 +36,11 @@ API endpoint. In the following order the plugin will attempt to authenticate.
|
||||
## Refreshes Namespace available metrics every 1h
|
||||
[[inputs.cloudwatch.metrics]]
|
||||
names = ['Latency', 'RequestCount']
|
||||
|
||||
|
||||
## Dimension filters for Metric (optional)
|
||||
[[inputs.cloudwatch.metrics.dimensions]]
|
||||
name = 'LoadBalancerName'
|
||||
value = 'p-example'
|
||||
|
||||
[[inputs.cloudwatch.metrics.dimensions]]
|
||||
name = 'AvailabilityZone'
|
||||
value = '*'
|
||||
```
|
||||
#### Requirements and Terminology
|
||||
|
||||
@@ -59,39 +52,6 @@ Plugin Configuration utilizes [CloudWatch concepts](http://docs.aws.amazon.com/A
|
||||
- `names` must be valid CloudWatch [Metric](http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/cloudwatch_concepts.html#Metric) names
|
||||
- `dimensions` must be valid CloudWatch [Dimension](http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/cloudwatch_concepts.html#Dimension) name/value pairs
|
||||
|
||||
Omitting or specifying a value of `'*'` for a dimension value configures all available metrics that contain a dimension with the specified name
|
||||
to be retrieved. If specifying >1 dimension, then the metric must contain *all* the configured dimensions where the the value of the
|
||||
wildcard dimension is ignored.
|
||||
|
||||
Example:
|
||||
```
|
||||
[[inputs.cloudwatch.metrics]]
|
||||
names = ['Latency']
|
||||
|
||||
## Dimension filters for Metric (optional)
|
||||
[[inputs.cloudwatch.metrics.dimensions]]
|
||||
name = 'LoadBalancerName'
|
||||
value = 'p-example'
|
||||
|
||||
[[inputs.cloudwatch.metrics.dimensions]]
|
||||
name = 'AvailabilityZone'
|
||||
value = '*'
|
||||
```
|
||||
|
||||
If the following ELBs are available:
|
||||
- name: `p-example`, availabilityZone: `us-east-1a`
|
||||
- name: `p-example`, availabilityZone: `us-east-1b`
|
||||
- name: `q-example`, availabilityZone: `us-east-1a`
|
||||
- name: `q-example`, availabilityZone: `us-east-1b`
|
||||
|
||||
|
||||
Then 2 metrics will be output:
|
||||
- name: `p-example`, availabilityZone: `us-east-1a`
|
||||
- name: `p-example`, availabilityZone: `us-east-1b`
|
||||
|
||||
If the `AvailabilityZone` wildcard dimension was omitted, then a single metric (name: `p-example`)
|
||||
would be exported containing the aggregate values of the ELB across availability zones.
|
||||
|
||||
#### Restrictions and Limitations
|
||||
- CloudWatch metrics are not available instantly via the CloudWatch API. You should adjust your collection `delay` to account for this lag in metrics availability based on your [monitoring subscription level](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-new.html)
|
||||
- CloudWatch API usage incurs cost - see [GetMetricStatistics Pricing](https://aws.amazon.com/cloudwatch/pricing/)
|
||||
|
||||
@@ -3,36 +3,28 @@ package cloudwatch
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/aws/aws-sdk-go/aws"
|
||||
"github.com/aws/aws-sdk-go/aws/credentials"
|
||||
"github.com/aws/aws-sdk-go/aws/session"
|
||||
|
||||
"github.com/aws/aws-sdk-go/service/cloudwatch"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
internalaws "github.com/influxdata/telegraf/internal/config/aws"
|
||||
"github.com/influxdata/telegraf/internal/errchan"
|
||||
"github.com/influxdata/telegraf/internal/limiter"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
type (
|
||||
CloudWatch struct {
|
||||
Region string `toml:"region"`
|
||||
AccessKey string `toml:"access_key"`
|
||||
SecretKey string `toml:"secret_key"`
|
||||
RoleARN string `toml:"role_arn"`
|
||||
Profile string `toml:"profile"`
|
||||
Filename string `toml:"shared_credential_file"`
|
||||
Token string `toml:"token"`
|
||||
|
||||
Region string `toml:"region"`
|
||||
AccessKey string `toml:"access_key"`
|
||||
SecretKey string `toml:"secret_key"`
|
||||
Period internal.Duration `toml:"period"`
|
||||
Delay internal.Duration `toml:"delay"`
|
||||
Namespace string `toml:"namespace"`
|
||||
Metrics []*Metric `toml:"metrics"`
|
||||
CacheTTL internal.Duration `toml:"cache_ttl"`
|
||||
client cloudwatchClient
|
||||
metricCache *MetricCache
|
||||
}
|
||||
@@ -66,18 +58,12 @@ func (c *CloudWatch) SampleConfig() string {
|
||||
|
||||
## Amazon Credentials
|
||||
## Credentials are loaded in the following order
|
||||
## 1) Assumed credentials via STS if role_arn is specified
|
||||
## 2) explicit credentials from 'access_key' and 'secret_key'
|
||||
## 3) shared profile from 'profile'
|
||||
## 4) environment variables
|
||||
## 5) shared credentials file
|
||||
## 6) EC2 Instance Profile
|
||||
## 1) explicit credentials from 'access_key' and 'secret_key'
|
||||
## 2) environment variables
|
||||
## 3) shared credentials file
|
||||
## 4) EC2 Instance Profile
|
||||
#access_key = ""
|
||||
#secret_key = ""
|
||||
#token = ""
|
||||
#role_arn = ""
|
||||
#profile = ""
|
||||
#shared_credential_file = ""
|
||||
|
||||
## Requested CloudWatch aggregation Period (required - must be a multiple of 60s)
|
||||
period = '1m'
|
||||
@@ -89,10 +75,6 @@ func (c *CloudWatch) SampleConfig() string {
|
||||
## gaps or overlap in pulled data
|
||||
interval = '1m'
|
||||
|
||||
## Configure the TTL for the internal cache of metrics.
|
||||
## Defaults to 1 hr if not specified
|
||||
#cache_ttl = '10m'
|
||||
|
||||
## Metric Statistic Namespace (required)
|
||||
namespace = 'AWS/ELB'
|
||||
|
||||
@@ -124,40 +106,20 @@ func (c *CloudWatch) Gather(acc telegraf.Accumulator) error {
|
||||
if c.Metrics != nil {
|
||||
metrics = []*cloudwatch.Metric{}
|
||||
for _, m := range c.Metrics {
|
||||
if !hasWilcard(m.Dimensions) {
|
||||
dimensions := make([]*cloudwatch.Dimension, len(m.Dimensions))
|
||||
for k, d := range m.Dimensions {
|
||||
fmt.Printf("Dimension [%s]:[%s]\n", d.Name, d.Value)
|
||||
dimensions[k] = &cloudwatch.Dimension{
|
||||
Name: aws.String(d.Name),
|
||||
Value: aws.String(d.Value),
|
||||
}
|
||||
}
|
||||
for _, name := range m.MetricNames {
|
||||
metrics = append(metrics, &cloudwatch.Metric{
|
||||
Namespace: aws.String(c.Namespace),
|
||||
MetricName: aws.String(name),
|
||||
Dimensions: dimensions,
|
||||
})
|
||||
}
|
||||
} else {
|
||||
allMetrics, err := c.fetchNamespaceMetrics()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, name := range m.MetricNames {
|
||||
for _, metric := range allMetrics {
|
||||
if isSelected(metric, m.Dimensions) {
|
||||
metrics = append(metrics, &cloudwatch.Metric{
|
||||
Namespace: aws.String(c.Namespace),
|
||||
MetricName: aws.String(name),
|
||||
Dimensions: metric.Dimensions,
|
||||
})
|
||||
}
|
||||
}
|
||||
dimensions := make([]*cloudwatch.Dimension, len(m.Dimensions))
|
||||
for k, d := range m.Dimensions {
|
||||
dimensions[k] = &cloudwatch.Dimension{
|
||||
Name: aws.String(d.Name),
|
||||
Value: aws.String(d.Value),
|
||||
}
|
||||
}
|
||||
|
||||
for _, name := range m.MetricNames {
|
||||
metrics = append(metrics, &cloudwatch.Metric{
|
||||
Namespace: aws.String(c.Namespace),
|
||||
MetricName: aws.String(name),
|
||||
Dimensions: dimensions,
|
||||
})
|
||||
}
|
||||
}
|
||||
} else {
|
||||
var err error
|
||||
@@ -168,35 +130,30 @@ func (c *CloudWatch) Gather(acc telegraf.Accumulator) error {
|
||||
}
|
||||
|
||||
metricCount := len(metrics)
|
||||
errChan := errchan.New(metricCount)
|
||||
var errChan = make(chan error, metricCount)
|
||||
|
||||
now := time.Now()
|
||||
|
||||
// limit concurrency or we can easily exhaust user connection limit
|
||||
// see cloudwatch API request limits:
|
||||
// http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/cloudwatch_limits.html
|
||||
lmtr := limiter.NewRateLimiter(10, time.Second)
|
||||
defer lmtr.Stop()
|
||||
var wg sync.WaitGroup
|
||||
wg.Add(len(metrics))
|
||||
for _, m := range metrics {
|
||||
<-lmtr.C
|
||||
go func(inm *cloudwatch.Metric) {
|
||||
defer wg.Done()
|
||||
c.gatherMetric(acc, inm, now, errChan.C)
|
||||
}(m)
|
||||
}
|
||||
wg.Wait()
|
||||
semaphore := make(chan byte, 64)
|
||||
|
||||
return errChan.Error()
|
||||
for _, m := range metrics {
|
||||
semaphore <- 0x1
|
||||
go c.gatherMetric(acc, m, now, semaphore, errChan)
|
||||
}
|
||||
|
||||
for i := 1; i <= metricCount; i++ {
|
||||
err := <-errChan
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("cloudwatch", func() telegraf.Input {
|
||||
ttl, _ := time.ParseDuration("1hr")
|
||||
return &CloudWatch{
|
||||
CacheTTL: internal.Duration{Duration: ttl},
|
||||
}
|
||||
return &CloudWatch{}
|
||||
})
|
||||
}
|
||||
|
||||
@@ -204,18 +161,14 @@ func init() {
|
||||
* Initialize CloudWatch client
|
||||
*/
|
||||
func (c *CloudWatch) initializeCloudWatch() error {
|
||||
credentialConfig := &internalaws.CredentialConfig{
|
||||
Region: c.Region,
|
||||
AccessKey: c.AccessKey,
|
||||
SecretKey: c.SecretKey,
|
||||
RoleARN: c.RoleARN,
|
||||
Profile: c.Profile,
|
||||
Filename: c.Filename,
|
||||
Token: c.Token,
|
||||
config := &aws.Config{
|
||||
Region: aws.String(c.Region),
|
||||
}
|
||||
if c.AccessKey != "" || c.SecretKey != "" {
|
||||
config.Credentials = credentials.NewStaticCredentials(c.AccessKey, c.SecretKey, "")
|
||||
}
|
||||
configProvider := credentialConfig.Credentials()
|
||||
|
||||
c.client = cloudwatch.New(configProvider)
|
||||
c.client = cloudwatch.New(session.New(config))
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -250,10 +203,11 @@ func (c *CloudWatch) fetchNamespaceMetrics() (metrics []*cloudwatch.Metric, err
|
||||
more = token != nil
|
||||
}
|
||||
|
||||
cacheTTL, _ := time.ParseDuration("1hr")
|
||||
c.metricCache = &MetricCache{
|
||||
Metrics: metrics,
|
||||
Fetched: time.Now(),
|
||||
TTL: c.CacheTTL.Duration,
|
||||
TTL: cacheTTL,
|
||||
}
|
||||
|
||||
return
|
||||
@@ -262,16 +216,12 @@ func (c *CloudWatch) fetchNamespaceMetrics() (metrics []*cloudwatch.Metric, err
|
||||
/*
|
||||
* Gather given Metric and emit any error
|
||||
*/
|
||||
func (c *CloudWatch) gatherMetric(
|
||||
acc telegraf.Accumulator,
|
||||
metric *cloudwatch.Metric,
|
||||
now time.Time,
|
||||
errChan chan error,
|
||||
) {
|
||||
func (c *CloudWatch) gatherMetric(acc telegraf.Accumulator, metric *cloudwatch.Metric, now time.Time, semaphore chan byte, errChan chan error) {
|
||||
params := c.getStatisticsInput(metric, now)
|
||||
resp, err := c.client.GetMetricStatistics(params)
|
||||
if err != nil {
|
||||
errChan <- err
|
||||
<-semaphore
|
||||
return
|
||||
}
|
||||
|
||||
@@ -308,6 +258,7 @@ func (c *CloudWatch) gatherMetric(
|
||||
}
|
||||
|
||||
errChan <- nil
|
||||
<-semaphore
|
||||
}
|
||||
|
||||
/*
|
||||
@@ -358,32 +309,3 @@ func (c *CloudWatch) getStatisticsInput(metric *cloudwatch.Metric, now time.Time
|
||||
func (c *MetricCache) IsValid() bool {
|
||||
return c.Metrics != nil && time.Since(c.Fetched) < c.TTL
|
||||
}
|
||||
|
||||
func hasWilcard(dimensions []*Dimension) bool {
|
||||
for _, d := range dimensions {
|
||||
if d.Value == "" || d.Value == "*" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isSelected(metric *cloudwatch.Metric, dimensions []*Dimension) bool {
|
||||
if len(metric.Dimensions) != len(dimensions) {
|
||||
return false
|
||||
}
|
||||
for _, d := range dimensions {
|
||||
selected := false
|
||||
for _, d2 := range metric.Dimensions {
|
||||
if d.Name == *d2.Name {
|
||||
if d.Value == "" || d.Value == "*" || d.Value == *d2.Value {
|
||||
selected = true
|
||||
}
|
||||
}
|
||||
}
|
||||
if !selected {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
@@ -1,56 +0,0 @@
|
||||
# Conntrack Plugin
|
||||
|
||||
Collects stats from Netfilter's conntrack-tools.
|
||||
|
||||
The conntrack-tools provide a mechanism for tracking various aspects of
|
||||
network connections as they are processed by netfilter. At runtime,
|
||||
conntrack exposes many of those connection statistics within /proc/sys/net.
|
||||
Depending on your kernel version, these files can be found in either
|
||||
/proc/sys/net/ipv4/netfilter or /proc/sys/net/netfilter and will be
|
||||
prefixed with either ip_ or nf_. This plugin reads the files specified
|
||||
in its configuration and publishes each one as a field, with the prefix
|
||||
normalized to ip_.
|
||||
|
||||
In order to simplify configuration in a heterogeneous environment, a superset
|
||||
of directory and filenames can be specified. Any locations that don't exist
|
||||
will be ignored.
|
||||
|
||||
For more information on conntrack-tools, see the
|
||||
[Netfilter Documentation](http://conntrack-tools.netfilter.org/).
|
||||
|
||||
|
||||
### Configuration:
|
||||
|
||||
```toml
|
||||
# Collects conntrack stats from the configured directories and files.
|
||||
[[inputs.conntrack]]
|
||||
## The following defaults would work with multiple versions of conntrack.
|
||||
## Note the nf_ and ip_ filename prefixes are mutually exclusive across
|
||||
## kernel versions, as are the directory locations.
|
||||
|
||||
## Superset of filenames to look for within the conntrack dirs.
|
||||
## Missing files will be ignored.
|
||||
files = ["ip_conntrack_count","ip_conntrack_max",
|
||||
"nf_conntrack_count","nf_conntrack_max"]
|
||||
|
||||
## Directories to search within for the conntrack files above.
|
||||
## Missing directrories will be ignored.
|
||||
dirs = ["/proc/sys/net/ipv4/netfilter","/proc/sys/net/netfilter"]
|
||||
```
|
||||
|
||||
### Measurements & Fields:
|
||||
|
||||
- conntrack
|
||||
- ip_conntrack_count (int, count): the number of entries in the conntrack table
|
||||
- ip_conntrack_max (int, size): the max capacity of the conntrack table
|
||||
|
||||
### Tags:
|
||||
|
||||
This input does not use tags.
|
||||
|
||||
### Example Output:
|
||||
|
||||
```
|
||||
$ ./telegraf -config telegraf.conf -input-filter conntrack -test
|
||||
conntrack,host=myhost ip_conntrack_count=2,ip_conntrack_max=262144 1461620427667995735
|
||||
```
|
||||
@@ -1,119 +0,0 @@
|
||||
// +build linux
|
||||
|
||||
package conntrack
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"log"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
type Conntrack struct {
|
||||
Path string
|
||||
Dirs []string
|
||||
Files []string
|
||||
}
|
||||
|
||||
const (
|
||||
inputName = "conntrack"
|
||||
)
|
||||
|
||||
var dfltDirs = []string{
|
||||
"/proc/sys/net/ipv4/netfilter",
|
||||
"/proc/sys/net/netfilter",
|
||||
}
|
||||
|
||||
var dfltFiles = []string{
|
||||
"ip_conntrack_count",
|
||||
"ip_conntrack_max",
|
||||
"nf_conntrack_count",
|
||||
"nf_conntrack_max",
|
||||
}
|
||||
|
||||
func (c *Conntrack) setDefaults() {
|
||||
if len(c.Dirs) == 0 {
|
||||
c.Dirs = dfltDirs
|
||||
}
|
||||
|
||||
if len(c.Files) == 0 {
|
||||
c.Files = dfltFiles
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Conntrack) Description() string {
|
||||
return "Collects conntrack stats from the configured directories and files."
|
||||
}
|
||||
|
||||
var sampleConfig = `
|
||||
## The following defaults would work with multiple versions of conntrack.
|
||||
## Note the nf_ and ip_ filename prefixes are mutually exclusive across
|
||||
## kernel versions, as are the directory locations.
|
||||
|
||||
## Superset of filenames to look for within the conntrack dirs.
|
||||
## Missing files will be ignored.
|
||||
files = ["ip_conntrack_count","ip_conntrack_max",
|
||||
"nf_conntrack_count","nf_conntrack_max"]
|
||||
|
||||
## Directories to search within for the conntrack files above.
|
||||
## Missing directrories will be ignored.
|
||||
dirs = ["/proc/sys/net/ipv4/netfilter","/proc/sys/net/netfilter"]
|
||||
`
|
||||
|
||||
func (c *Conntrack) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
|
||||
func (c *Conntrack) Gather(acc telegraf.Accumulator) error {
|
||||
c.setDefaults()
|
||||
|
||||
var metricKey string
|
||||
fields := make(map[string]interface{})
|
||||
|
||||
for _, dir := range c.Dirs {
|
||||
for _, file := range c.Files {
|
||||
// NOTE: no system will have both nf_ and ip_ prefixes,
|
||||
// so we're safe to branch on suffix only.
|
||||
parts := strings.SplitN(file, "_", 2)
|
||||
if len(parts) < 2 {
|
||||
continue
|
||||
}
|
||||
metricKey = "ip_" + parts[1]
|
||||
|
||||
fName := filepath.Join(dir, file)
|
||||
if _, err := os.Stat(fName); err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
contents, err := ioutil.ReadFile(fName)
|
||||
if err != nil {
|
||||
log.Printf("failed to read file '%s': %v", fName, err)
|
||||
}
|
||||
|
||||
v := strings.TrimSpace(string(contents))
|
||||
fields[metricKey], err = strconv.ParseFloat(v, 64)
|
||||
if err != nil {
|
||||
log.Printf("failed to parse metric, expected number but "+
|
||||
" found '%s': %v", v, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if len(fields) == 0 {
|
||||
return fmt.Errorf("Conntrack input failed to collect metrics. " +
|
||||
"Is the conntrack kernel module loaded?")
|
||||
}
|
||||
|
||||
acc.AddFields(inputName, fields, nil)
|
||||
return nil
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add(inputName, func() telegraf.Input { return &Conntrack{} })
|
||||
}
|
||||
@@ -1,3 +0,0 @@
|
||||
// +build !linux
|
||||
|
||||
package conntrack
|
||||
@@ -1,90 +0,0 @@
|
||||
// +build linux
|
||||
|
||||
package conntrack
|
||||
|
||||
import (
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"io/ioutil"
|
||||
"os"
|
||||
"path"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func restoreDflts(savedFiles, savedDirs []string) {
|
||||
dfltFiles = savedFiles
|
||||
dfltDirs = savedDirs
|
||||
}
|
||||
|
||||
func TestNoFilesFound(t *testing.T) {
|
||||
defer restoreDflts(dfltFiles, dfltDirs)
|
||||
|
||||
dfltFiles = []string{"baz.txt"}
|
||||
dfltDirs = []string{"./foo/bar"}
|
||||
c := &Conntrack{}
|
||||
acc := &testutil.Accumulator{}
|
||||
err := c.Gather(acc)
|
||||
|
||||
assert.EqualError(t, err, "Conntrack input failed to collect metrics. "+
|
||||
"Is the conntrack kernel module loaded?")
|
||||
}
|
||||
|
||||
func TestDefaultsUsed(t *testing.T) {
|
||||
defer restoreDflts(dfltFiles, dfltDirs)
|
||||
tmpdir, err := ioutil.TempDir("", "tmp1")
|
||||
assert.NoError(t, err)
|
||||
defer os.Remove(tmpdir)
|
||||
|
||||
tmpFile, err := ioutil.TempFile(tmpdir, "ip_conntrack_count")
|
||||
assert.NoError(t, err)
|
||||
|
||||
dfltDirs = []string{tmpdir}
|
||||
fname := path.Base(tmpFile.Name())
|
||||
dfltFiles = []string{fname}
|
||||
|
||||
count := 1234321
|
||||
ioutil.WriteFile(tmpFile.Name(), []byte(strconv.Itoa(count)), 0660)
|
||||
c := &Conntrack{}
|
||||
acc := &testutil.Accumulator{}
|
||||
|
||||
c.Gather(acc)
|
||||
acc.AssertContainsFields(t, inputName, map[string]interface{}{
|
||||
fname: float64(count)})
|
||||
}
|
||||
|
||||
func TestConfigsUsed(t *testing.T) {
|
||||
defer restoreDflts(dfltFiles, dfltDirs)
|
||||
tmpdir, err := ioutil.TempDir("", "tmp1")
|
||||
assert.NoError(t, err)
|
||||
defer os.Remove(tmpdir)
|
||||
|
||||
cntFile, err := ioutil.TempFile(tmpdir, "nf_conntrack_count")
|
||||
maxFile, err := ioutil.TempFile(tmpdir, "nf_conntrack_max")
|
||||
assert.NoError(t, err)
|
||||
|
||||
dfltDirs = []string{tmpdir}
|
||||
cntFname := path.Base(cntFile.Name())
|
||||
maxFname := path.Base(maxFile.Name())
|
||||
dfltFiles = []string{cntFname, maxFname}
|
||||
|
||||
count := 1234321
|
||||
max := 9999999
|
||||
ioutil.WriteFile(cntFile.Name(), []byte(strconv.Itoa(count)), 0660)
|
||||
ioutil.WriteFile(maxFile.Name(), []byte(strconv.Itoa(max)), 0660)
|
||||
c := &Conntrack{}
|
||||
acc := &testutil.Accumulator{}
|
||||
|
||||
c.Gather(acc)
|
||||
|
||||
fix := func(s string) string {
|
||||
return strings.Replace(s, "nf_", "ip_", 1)
|
||||
}
|
||||
|
||||
acc.AssertContainsFields(t, inputName,
|
||||
map[string]interface{}{
|
||||
fix(cntFname): float64(count),
|
||||
fix(maxFname): float64(max),
|
||||
})
|
||||
}
|
||||
@@ -1,46 +0,0 @@
|
||||
# Telegraf Input Plugin: Consul
|
||||
|
||||
This plugin will collect statistics about all helath checks registered in the Consul. It uses [Consul API](https://www.consul.io/docs/agent/http/health.html#health_state)
|
||||
to query the data. It will not report the [telemetry](https://www.consul.io/docs/agent/telemetry.html) but Consul can report those stats already using StatsD protocol if needed.
|
||||
|
||||
## Configuration:
|
||||
|
||||
```
|
||||
# Gather health check statuses from services registered in Consul
|
||||
[[inputs.consul]]
|
||||
## Most of these values defaults to the one configured on a Consul's agent level.
|
||||
## Optional Consul server address (default: "")
|
||||
# address = ""
|
||||
## Optional URI scheme for the Consul server (default: "")
|
||||
# scheme = ""
|
||||
## Optional ACL token used in every request (default: "")
|
||||
# token = ""
|
||||
## Optional username used for request HTTP Basic Authentication (default: "")
|
||||
# username = ""
|
||||
## Optional password used for HTTP Basic Authentication (default: "")
|
||||
# password = ""
|
||||
## Optional data centre to query the health checks from (default: "")
|
||||
# datacentre = ""
|
||||
```
|
||||
|
||||
## Measurements:
|
||||
|
||||
### Consul:
|
||||
Tags:
|
||||
- node: on which node check/service is registered on
|
||||
- service_name: name of the service (this is the service name not the service ID)
|
||||
|
||||
Fields:
|
||||
- check_id
|
||||
- check_name
|
||||
- service_id
|
||||
- status
|
||||
|
||||
## Example output
|
||||
|
||||
```
|
||||
$ telegraf --config ./telegraf.conf -input-filter consul -test
|
||||
* Plugin: consul, Collection 1
|
||||
> consul_health_checks,host=wolfpit,node=consul-server-node check_id="serfHealth",check_name="Serf Health Status",service_id="",status="passing" 1464698464486439902
|
||||
> consul_health_checks,host=wolfpit,node=consul-server-node,service_name=www.example.com check_id="service:www-example-com.test01",check_name="Service 'www.example.com' check",service_id="www-example-com.test01",status="critical" 1464698464486519036
|
||||
```
|
||||
@@ -1,136 +0,0 @@
|
||||
package consul
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
"github.com/hashicorp/consul/api"
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
type Consul struct {
|
||||
Address string
|
||||
Scheme string
|
||||
Token string
|
||||
Username string
|
||||
Password string
|
||||
Datacentre string
|
||||
|
||||
// Path to CA file
|
||||
SSLCA string `toml:"ssl_ca"`
|
||||
// Path to host cert file
|
||||
SSLCert string `toml:"ssl_cert"`
|
||||
// Path to cert key file
|
||||
SSLKey string `toml:"ssl_key"`
|
||||
// Use SSL but skip chain & host verification
|
||||
InsecureSkipVerify bool
|
||||
|
||||
// client used to connect to Consul agnet
|
||||
client *api.Client
|
||||
}
|
||||
|
||||
var sampleConfig = `
|
||||
## Most of these values defaults to the one configured on a Consul's agent level.
|
||||
## Optional Consul server address (default: "localhost")
|
||||
# address = "localhost"
|
||||
## Optional URI scheme for the Consul server (default: "http")
|
||||
# scheme = "http"
|
||||
## Optional ACL token used in every request (default: "")
|
||||
# token = ""
|
||||
## Optional username used for request HTTP Basic Authentication (default: "")
|
||||
# username = ""
|
||||
## Optional password used for HTTP Basic Authentication (default: "")
|
||||
# password = ""
|
||||
## Optional data centre to query the health checks from (default: "")
|
||||
# datacentre = ""
|
||||
`
|
||||
|
||||
func (c *Consul) Description() string {
|
||||
return "Gather health check statuses from services registered in Consul"
|
||||
}
|
||||
|
||||
func (c *Consul) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
|
||||
func (c *Consul) createAPIClient() (*api.Client, error) {
|
||||
config := api.DefaultConfig()
|
||||
|
||||
if c.Address != "" {
|
||||
config.Address = c.Address
|
||||
}
|
||||
|
||||
if c.Scheme != "" {
|
||||
config.Scheme = c.Scheme
|
||||
}
|
||||
|
||||
if c.Datacentre != "" {
|
||||
config.Datacenter = c.Datacentre
|
||||
}
|
||||
|
||||
if c.Username != "" {
|
||||
config.HttpAuth = &api.HttpBasicAuth{
|
||||
Username: c.Username,
|
||||
Password: c.Password,
|
||||
}
|
||||
}
|
||||
|
||||
tlsCfg, err := internal.GetTLSConfig(
|
||||
c.SSLCert, c.SSLKey, c.SSLCA, c.InsecureSkipVerify)
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
config.HttpClient.Transport = &http.Transport{
|
||||
TLSClientConfig: tlsCfg,
|
||||
}
|
||||
|
||||
return api.NewClient(config)
|
||||
}
|
||||
|
||||
func (c *Consul) GatherHealthCheck(acc telegraf.Accumulator, checks []*api.HealthCheck) {
|
||||
for _, check := range checks {
|
||||
record := make(map[string]interface{})
|
||||
tags := make(map[string]string)
|
||||
|
||||
record["check_id"] = check.CheckID
|
||||
record["check_name"] = check.Name
|
||||
record["service_id"] = check.ServiceID
|
||||
record["status"] = check.Status
|
||||
|
||||
tags["node"] = check.Node
|
||||
tags["service_name"] = check.ServiceName
|
||||
|
||||
acc.AddFields("consul_health_checks", record, tags)
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Consul) Gather(acc telegraf.Accumulator) error {
|
||||
if c.client == nil {
|
||||
newClient, err := c.createAPIClient()
|
||||
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
c.client = newClient
|
||||
}
|
||||
|
||||
checks, _, err := c.client.Health().State("any", nil)
|
||||
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
c.GatherHealthCheck(acc, checks)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("consul", func() telegraf.Input {
|
||||
return &Consul{}
|
||||
})
|
||||
}
|
||||
@@ -1,42 +0,0 @@
|
||||
package consul
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/hashicorp/consul/api"
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
)
|
||||
|
||||
var sampleChecks = []*api.HealthCheck{
|
||||
&api.HealthCheck{
|
||||
Node: "localhost",
|
||||
CheckID: "foo.health123",
|
||||
Name: "foo.health",
|
||||
Status: "passing",
|
||||
Notes: "lorem ipsum",
|
||||
Output: "OK",
|
||||
ServiceID: "foo.123",
|
||||
ServiceName: "foo",
|
||||
},
|
||||
}
|
||||
|
||||
func TestGatherHealtCheck(t *testing.T) {
|
||||
expectedFields := map[string]interface{}{
|
||||
"check_id": "foo.health123",
|
||||
"check_name": "foo.health",
|
||||
"status": "passing",
|
||||
"service_id": "foo.123",
|
||||
}
|
||||
|
||||
expectedTags := map[string]string{
|
||||
"node": "localhost",
|
||||
"service_name": "foo",
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
|
||||
consul := &Consul{}
|
||||
consul.GatherHealthCheck(&acc, sampleChecks)
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "consul_health_checks", expectedFields, expectedTags)
|
||||
}
|
||||
@@ -2,13 +2,14 @@ package elasticsearch
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal/errchan"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
jsonparser "github.com/influxdata/telegraf/plugins/parsers/json"
|
||||
)
|
||||
@@ -101,7 +102,7 @@ func (e *Elasticsearch) Description() string {
|
||||
// Gather reads the stats from Elasticsearch and writes it to the
|
||||
// Accumulator.
|
||||
func (e *Elasticsearch) Gather(acc telegraf.Accumulator) error {
|
||||
errChan := errchan.New(len(e.Servers))
|
||||
errChan := make(chan error, len(e.Servers))
|
||||
var wg sync.WaitGroup
|
||||
wg.Add(len(e.Servers))
|
||||
|
||||
@@ -115,7 +116,7 @@ func (e *Elasticsearch) Gather(acc telegraf.Accumulator) error {
|
||||
url = s + statsPath
|
||||
}
|
||||
if err := e.gatherNodeStats(url, acc); err != nil {
|
||||
errChan.C <- err
|
||||
errChan <- err
|
||||
return
|
||||
}
|
||||
if e.ClusterHealth {
|
||||
@@ -125,7 +126,17 @@ func (e *Elasticsearch) Gather(acc telegraf.Accumulator) error {
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
return errChan.Error()
|
||||
close(errChan)
|
||||
// Get all errors and return them as one giant error
|
||||
errStrings := []string{}
|
||||
for err := range errChan {
|
||||
errStrings = append(errStrings, err.Error())
|
||||
}
|
||||
|
||||
if len(errStrings) == 0 {
|
||||
return nil
|
||||
}
|
||||
return errors.New(strings.Join(errStrings, "\n"))
|
||||
}
|
||||
|
||||
func (e *Elasticsearch) gatherNodeStats(url string, acc telegraf.Accumulator) error {
|
||||
|
||||
@@ -6,20 +6,14 @@ Please also see: [Telegraf Input Data Formats](https://github.com/influxdata/tel
|
||||
|
||||
#### Configuration
|
||||
|
||||
In this example a script called ```/tmp/test.sh```, a script called ```/tmp/test2.sh```, and
|
||||
all scripts matching glob pattern ```/tmp/collect_*.sh``` are configured for ```[[inputs.exec]]```
|
||||
in JSON format. Glob patterns are matched on every run, so adding new scripts that match the pattern
|
||||
will cause them to be picked up immediately.
|
||||
In this example a script called ```/tmp/test.sh``` and a script called ```/tmp/test2.sh```
|
||||
are configured for ```[[inputs.exec]]``` in JSON format.
|
||||
|
||||
```toml
|
||||
```
|
||||
# Read flattened metrics from one or more commands that output JSON to stdout
|
||||
[[inputs.exec]]
|
||||
# Shell/commands array
|
||||
# Full command line to executable with parameters, or a glob pattern to run all matching files.
|
||||
commands = ["/tmp/test.sh", "/tmp/test2.sh", "/tmp/collect_*.sh"]
|
||||
|
||||
## Timeout for each command to complete.
|
||||
timeout = "5s"
|
||||
commands = ["/tmp/test.sh", "/tmp/test2.sh"]
|
||||
|
||||
# Data format to consume.
|
||||
# NOTE json only reads numerical measurements, strings and booleans are ignored.
|
||||
@@ -27,6 +21,26 @@ will cause them to be picked up immediately.
|
||||
|
||||
# measurement name suffix (for separating different commands)
|
||||
name_suffix = "_mycollector"
|
||||
|
||||
## Below configuration will be used for data_format = "graphite", can be ignored for other data_format
|
||||
## If matching multiple measurement files, this string will be used to join the matched values.
|
||||
#separator = "."
|
||||
|
||||
## Each template line requires a template pattern. It can have an optional
|
||||
## filter before the template and separated by spaces. It can also have optional extra
|
||||
## tags following the template. Multiple tags should be separated by commas and no spaces
|
||||
## similar to the line protocol format. The can be only one default template.
|
||||
## Templates support below format:
|
||||
## 1. filter + template
|
||||
## 2. filter + template + extra tag
|
||||
## 3. filter + template with field key
|
||||
## 4. default template
|
||||
#templates = [
|
||||
# "*.app env.service.resource.measurement",
|
||||
# "stats.* .host.measurement* region=us-west,agent=sensu",
|
||||
# "stats2.* .host.measurement.field",
|
||||
# "measurement*"
|
||||
#]
|
||||
```
|
||||
|
||||
Other options for modifying the measurement names are:
|
||||
@@ -65,7 +79,7 @@ in influx line-protocol format.
|
||||
|
||||
#### Configuration
|
||||
|
||||
```toml
|
||||
```
|
||||
[[inputs.exec]]
|
||||
# Shell/commands array
|
||||
# compatible with old version
|
||||
@@ -73,9 +87,6 @@ in influx line-protocol format.
|
||||
# command = "/usr/bin/line_protocol_collector"
|
||||
commands = ["/usr/bin/line_protocol_collector","/tmp/test2.sh"]
|
||||
|
||||
## Timeout for each command to complete.
|
||||
timeout = "5s"
|
||||
|
||||
# Data format to consume.
|
||||
# NOTE json only reads numerical measurements, strings and booleans are ignored.
|
||||
data_format = "influx"
|
||||
@@ -109,16 +120,12 @@ We can also change the data_format to "graphite" to use the metrics collecting s
|
||||
In this example a script called /tmp/test.sh and a script called /tmp/test2.sh are configured for [[inputs.exec]] in graphite format.
|
||||
|
||||
#### Configuration
|
||||
|
||||
```toml
|
||||
```
|
||||
# Read flattened metrics from one or more commands that output JSON to stdout
|
||||
[[inputs.exec]]
|
||||
# Shell/commands array
|
||||
commands = ["/tmp/test.sh","/tmp/test2.sh"]
|
||||
|
||||
## Timeout for each command to complete.
|
||||
timeout = "5s"
|
||||
|
||||
# Data format to consume.
|
||||
# NOTE json only reads numerical measurements, strings and booleans are ignored.
|
||||
data_format = "graphite"
|
||||
@@ -173,3 +180,4 @@ sensu.metric.net.server0.eth0.rx_dropped 0 1444234982
|
||||
The templates configuration will be used to parse the graphite metrics to support influxdb/opentsdb tagging store engines.
|
||||
|
||||
More detail information about templates, please refer to [The graphite Input](https://github.com/influxdata/influxdb/blob/master/services/graphite/README.md)
|
||||
|
||||
|
||||
@@ -4,8 +4,6 @@ import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"syscall"
|
||||
"time"
|
||||
@@ -14,7 +12,6 @@ import (
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/internal/errchan"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"github.com/influxdata/telegraf/plugins/parsers"
|
||||
"github.com/influxdata/telegraf/plugins/parsers/nagios"
|
||||
@@ -22,11 +19,7 @@ import (
|
||||
|
||||
const sampleConfig = `
|
||||
## Commands array
|
||||
commands = [
|
||||
"/tmp/test.sh",
|
||||
"/usr/bin/mycollector --foo=bar",
|
||||
"/tmp/collect_*.sh"
|
||||
]
|
||||
commands = ["/tmp/test.sh", "/usr/bin/mycollector --foo=bar"]
|
||||
|
||||
## Timeout for each command to complete.
|
||||
timeout = "5s"
|
||||
@@ -157,45 +150,23 @@ func (e *Exec) Gather(acc telegraf.Accumulator) error {
|
||||
e.Command = ""
|
||||
}
|
||||
|
||||
commands := make([]string, 0, len(e.Commands))
|
||||
for _, pattern := range e.Commands {
|
||||
cmdAndArgs := strings.SplitN(pattern, " ", 2)
|
||||
if len(cmdAndArgs) == 0 {
|
||||
continue
|
||||
}
|
||||
e.errChan = make(chan error, len(e.Commands))
|
||||
|
||||
matches, err := filepath.Glob(cmdAndArgs[0])
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if len(matches) == 0 {
|
||||
// There were no matches with the glob pattern, so let's assume
|
||||
// that the command is in PATH and just run it as it is
|
||||
commands = append(commands, pattern)
|
||||
} else {
|
||||
// There were matches, so we'll append each match together with
|
||||
// the arguments to the commands slice
|
||||
for _, match := range matches {
|
||||
if len(cmdAndArgs) == 1 {
|
||||
commands = append(commands, match)
|
||||
} else {
|
||||
commands = append(commands,
|
||||
strings.Join([]string{match, cmdAndArgs[1]}, " "))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
errChan := errchan.New(len(commands))
|
||||
e.errChan = errChan.C
|
||||
|
||||
e.wg.Add(len(commands))
|
||||
for _, command := range commands {
|
||||
e.wg.Add(len(e.Commands))
|
||||
for _, command := range e.Commands {
|
||||
go e.ProcessCommand(command, acc)
|
||||
}
|
||||
e.wg.Wait()
|
||||
return errChan.Error()
|
||||
|
||||
select {
|
||||
default:
|
||||
close(e.errChan)
|
||||
return nil
|
||||
case err := <-e.errChan:
|
||||
close(e.errChan)
|
||||
return err
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
func init() {
|
||||
|
||||
@@ -169,51 +169,3 @@ func TestLineProtocolParseMultiple(t *testing.T) {
|
||||
acc.AssertContainsTaggedFields(t, "cpu", fields, tags)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecCommandWithGlob(t *testing.T) {
|
||||
parser, _ := parsers.NewValueParser("metric", "string", nil)
|
||||
e := NewExec()
|
||||
e.Commands = []string{"/bin/ech* metric_value"}
|
||||
e.SetParser(parser)
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := e.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"value": "metric_value",
|
||||
}
|
||||
acc.AssertContainsFields(t, "metric", fields)
|
||||
}
|
||||
|
||||
func TestExecCommandWithoutGlob(t *testing.T) {
|
||||
parser, _ := parsers.NewValueParser("metric", "string", nil)
|
||||
e := NewExec()
|
||||
e.Commands = []string{"/bin/echo metric_value"}
|
||||
e.SetParser(parser)
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := e.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"value": "metric_value",
|
||||
}
|
||||
acc.AssertContainsFields(t, "metric", fields)
|
||||
}
|
||||
|
||||
func TestExecCommandWithoutGlobAndPath(t *testing.T) {
|
||||
parser, _ := parsers.NewValueParser("metric", "string", nil)
|
||||
e := NewExec()
|
||||
e.Commands = []string{"echo metric_value"}
|
||||
e.SetParser(parser)
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := e.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"value": "metric_value",
|
||||
}
|
||||
acc.AssertContainsFields(t, "metric", fields)
|
||||
}
|
||||
|
||||
@@ -1,55 +0,0 @@
|
||||
# GrayLog plugin
|
||||
|
||||
The Graylog plugin can collect data from remote Graylog service URLs.
|
||||
|
||||
Plugin currently support two type of end points:-
|
||||
|
||||
- multiple (Ex http://[graylog-server-ip]:12900/system/metrics/multiple)
|
||||
- namespace (Ex http://[graylog-server-ip]:12900/system/metrics/namespace/{namespace})
|
||||
|
||||
End Point can be a mixe of one multiple end point and several namespaces end points
|
||||
|
||||
|
||||
Note: if namespace end point specified metrics array will be ignored for that call.
|
||||
|
||||
### Configuration:
|
||||
|
||||
```toml
|
||||
# Read flattened metrics from one or more GrayLog HTTP endpoints
|
||||
[[inputs.graylog]]
|
||||
## API endpoint, currently supported API:
|
||||
##
|
||||
## - multiple (Ex http://<host>:12900/system/metrics/multiple)
|
||||
## - namespace (Ex http://<host>:12900/system/metrics/namespace/{namespace})
|
||||
##
|
||||
## For namespace endpoint, the metrics array will be ignored for that call.
|
||||
## Endpoint can contain namespace and multiple type calls.
|
||||
##
|
||||
## Please check http://[graylog-server-ip]:12900/api-browser for full list
|
||||
## of endpoints
|
||||
servers = [
|
||||
"http://[graylog-server-ip]:12900/system/metrics/multiple",
|
||||
]
|
||||
|
||||
## Metrics list
|
||||
## List of metrics can be found on Graylog webservice documentation.
|
||||
## Or by hitting the the web service api at:
|
||||
## http://[graylog-host]:12900/system/metrics
|
||||
metrics = [
|
||||
"jvm.cl.loaded",
|
||||
"jvm.memory.pools.Metaspace.committed"
|
||||
]
|
||||
|
||||
## Username and password
|
||||
username = ""
|
||||
password = ""
|
||||
|
||||
## Optional SSL Config
|
||||
# ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# ssl_key = "/etc/telegraf/key.pem"
|
||||
## Use SSL but skip chain & host verification
|
||||
# insecure_skip_verify = false
|
||||
```
|
||||
|
||||
Please refer to GrayLog metrics api browser for full metric end points http://host:12900/api-browser
|
||||
@@ -1,312 +0,0 @@
|
||||
package graylog
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
type ResponseMetrics struct {
|
||||
total int
|
||||
Metrics []Metric `json:"metrics"`
|
||||
}
|
||||
|
||||
type Metric struct {
|
||||
FullName string `json:"full_name"`
|
||||
Name string `json:"name"`
|
||||
Type string `json:"type"`
|
||||
Fields map[string]interface{} `json:"metric"`
|
||||
}
|
||||
|
||||
type GrayLog struct {
|
||||
Servers []string
|
||||
Metrics []string
|
||||
Username string
|
||||
Password string
|
||||
|
||||
// Path to CA file
|
||||
SSLCA string `toml:"ssl_ca"`
|
||||
// Path to host cert file
|
||||
SSLCert string `toml:"ssl_cert"`
|
||||
// Path to cert key file
|
||||
SSLKey string `toml:"ssl_key"`
|
||||
// Use SSL but skip chain & host verification
|
||||
InsecureSkipVerify bool
|
||||
|
||||
client HTTPClient
|
||||
}
|
||||
|
||||
type HTTPClient interface {
|
||||
// Returns the result of an http request
|
||||
//
|
||||
// Parameters:
|
||||
// req: HTTP request object
|
||||
//
|
||||
// Returns:
|
||||
// http.Response: HTTP respons object
|
||||
// error : Any error that may have occurred
|
||||
MakeRequest(req *http.Request) (*http.Response, error)
|
||||
|
||||
SetHTTPClient(client *http.Client)
|
||||
HTTPClient() *http.Client
|
||||
}
|
||||
|
||||
type Messagebody struct {
|
||||
Metrics []string `json:"metrics"`
|
||||
}
|
||||
|
||||
type RealHTTPClient struct {
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
func (c *RealHTTPClient) MakeRequest(req *http.Request) (*http.Response, error) {
|
||||
return c.client.Do(req)
|
||||
}
|
||||
|
||||
func (c *RealHTTPClient) SetHTTPClient(client *http.Client) {
|
||||
c.client = client
|
||||
}
|
||||
|
||||
func (c *RealHTTPClient) HTTPClient() *http.Client {
|
||||
return c.client
|
||||
}
|
||||
|
||||
var sampleConfig = `
|
||||
## API endpoint, currently supported API:
|
||||
##
|
||||
## - multiple (Ex http://<host>:12900/system/metrics/multiple)
|
||||
## - namespace (Ex http://<host>:12900/system/metrics/namespace/{namespace})
|
||||
##
|
||||
## For namespace endpoint, the metrics array will be ignored for that call.
|
||||
## Endpoint can contain namespace and multiple type calls.
|
||||
##
|
||||
## Please check http://[graylog-server-ip]:12900/api-browser for full list
|
||||
## of endpoints
|
||||
servers = [
|
||||
"http://[graylog-server-ip]:12900/system/metrics/multiple",
|
||||
]
|
||||
|
||||
## Metrics list
|
||||
## List of metrics can be found on Graylog webservice documentation.
|
||||
## Or by hitting the the web service api at:
|
||||
## http://[graylog-host]:12900/system/metrics
|
||||
metrics = [
|
||||
"jvm.cl.loaded",
|
||||
"jvm.memory.pools.Metaspace.committed"
|
||||
]
|
||||
|
||||
## Username and password
|
||||
username = ""
|
||||
password = ""
|
||||
|
||||
## Optional SSL Config
|
||||
# ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# ssl_key = "/etc/telegraf/key.pem"
|
||||
## Use SSL but skip chain & host verification
|
||||
# insecure_skip_verify = false
|
||||
`
|
||||
|
||||
func (h *GrayLog) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
|
||||
func (h *GrayLog) Description() string {
|
||||
return "Read flattened metrics from one or more GrayLog HTTP endpoints"
|
||||
}
|
||||
|
||||
// Gathers data for all servers.
|
||||
func (h *GrayLog) Gather(acc telegraf.Accumulator) error {
|
||||
var wg sync.WaitGroup
|
||||
|
||||
if h.client.HTTPClient() == nil {
|
||||
tlsCfg, err := internal.GetTLSConfig(
|
||||
h.SSLCert, h.SSLKey, h.SSLCA, h.InsecureSkipVerify)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
tr := &http.Transport{
|
||||
ResponseHeaderTimeout: time.Duration(3 * time.Second),
|
||||
TLSClientConfig: tlsCfg,
|
||||
}
|
||||
client := &http.Client{
|
||||
Transport: tr,
|
||||
Timeout: time.Duration(4 * time.Second),
|
||||
}
|
||||
h.client.SetHTTPClient(client)
|
||||
}
|
||||
|
||||
errorChannel := make(chan error, len(h.Servers))
|
||||
|
||||
for _, server := range h.Servers {
|
||||
wg.Add(1)
|
||||
go func(server string) {
|
||||
defer wg.Done()
|
||||
if err := h.gatherServer(acc, server); err != nil {
|
||||
errorChannel <- err
|
||||
}
|
||||
}(server)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
close(errorChannel)
|
||||
|
||||
// Get all errors and return them as one giant error
|
||||
errorStrings := []string{}
|
||||
for err := range errorChannel {
|
||||
errorStrings = append(errorStrings, err.Error())
|
||||
}
|
||||
|
||||
if len(errorStrings) == 0 {
|
||||
return nil
|
||||
}
|
||||
return errors.New(strings.Join(errorStrings, "\n"))
|
||||
}
|
||||
|
||||
// Gathers data from a particular server
|
||||
// Parameters:
|
||||
// acc : The telegraf Accumulator to use
|
||||
// serverURL: endpoint to send request to
|
||||
// service : the service being queried
|
||||
//
|
||||
// Returns:
|
||||
// error: Any error that may have occurred
|
||||
func (h *GrayLog) gatherServer(
|
||||
acc telegraf.Accumulator,
|
||||
serverURL string,
|
||||
) error {
|
||||
resp, _, err := h.sendRequest(serverURL)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
requestURL, err := url.Parse(serverURL)
|
||||
host, port, _ := net.SplitHostPort(requestURL.Host)
|
||||
var dat ResponseMetrics
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := json.Unmarshal([]byte(resp), &dat); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, m_item := range dat.Metrics {
|
||||
fields := make(map[string]interface{})
|
||||
tags := map[string]string{
|
||||
"server": host,
|
||||
"port": port,
|
||||
"name": m_item.Name,
|
||||
"type": m_item.Type,
|
||||
}
|
||||
h.flatten(m_item.Fields, fields, "")
|
||||
acc.AddFields(m_item.FullName, fields, tags)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Flatten JSON hierarchy to produce field name and field value
|
||||
// Parameters:
|
||||
// item: Item map to flatten
|
||||
// fields: Map to store generated fields.
|
||||
// id: Prefix for top level metric (empty string "")
|
||||
// Returns:
|
||||
// void
|
||||
func (h *GrayLog) flatten(item map[string]interface{}, fields map[string]interface{}, id string) {
|
||||
if id != "" {
|
||||
id = id + "_"
|
||||
}
|
||||
for k, i := range item {
|
||||
switch i.(type) {
|
||||
case int:
|
||||
fields[id+k] = i.(float64)
|
||||
case float64:
|
||||
fields[id+k] = i.(float64)
|
||||
case map[string]interface{}:
|
||||
h.flatten(i.(map[string]interface{}), fields, id+k)
|
||||
default:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Sends an HTTP request to the server using the GrayLog object's HTTPClient.
|
||||
// Parameters:
|
||||
// serverURL: endpoint to send request to
|
||||
//
|
||||
// Returns:
|
||||
// string: body of the response
|
||||
// error : Any error that may have occurred
|
||||
func (h *GrayLog) sendRequest(serverURL string) (string, float64, error) {
|
||||
headers := map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
"Accept": "application/json",
|
||||
}
|
||||
method := "GET"
|
||||
content := bytes.NewBufferString("")
|
||||
headers["Authorization"] = "Basic " + base64.URLEncoding.EncodeToString([]byte(h.Username+":"+h.Password))
|
||||
// Prepare URL
|
||||
requestURL, err := url.Parse(serverURL)
|
||||
if err != nil {
|
||||
return "", -1, fmt.Errorf("Invalid server URL \"%s\"", serverURL)
|
||||
}
|
||||
if strings.Contains(requestURL.String(), "multiple") {
|
||||
m := &Messagebody{Metrics: h.Metrics}
|
||||
http_body, err := json.Marshal(m)
|
||||
if err != nil {
|
||||
return "", -1, fmt.Errorf("Invalid list of Metrics %s", h.Metrics)
|
||||
}
|
||||
method = "POST"
|
||||
content = bytes.NewBuffer(http_body)
|
||||
}
|
||||
req, err := http.NewRequest(method, requestURL.String(), content)
|
||||
if err != nil {
|
||||
return "", -1, err
|
||||
}
|
||||
// Add header parameters
|
||||
for k, v := range headers {
|
||||
req.Header.Add(k, v)
|
||||
}
|
||||
start := time.Now()
|
||||
resp, err := h.client.MakeRequest(req)
|
||||
if err != nil {
|
||||
return "", -1, err
|
||||
}
|
||||
|
||||
defer resp.Body.Close()
|
||||
responseTime := time.Since(start).Seconds()
|
||||
|
||||
body, err := ioutil.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return string(body), responseTime, err
|
||||
}
|
||||
|
||||
// Process response
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
err = fmt.Errorf("Response from url \"%s\" has status code %d (%s), expected %d (%s)",
|
||||
requestURL.String(),
|
||||
resp.StatusCode,
|
||||
http.StatusText(resp.StatusCode),
|
||||
http.StatusOK,
|
||||
http.StatusText(http.StatusOK))
|
||||
return string(body), responseTime, err
|
||||
}
|
||||
return string(body), responseTime, err
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("graylog", func() telegraf.Input {
|
||||
return &GrayLog{
|
||||
client: &RealHTTPClient{},
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -1,199 +0,0 @@
|
||||
package graylog
|
||||
|
||||
import (
|
||||
"io/ioutil"
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
const validJSON = `
|
||||
{
|
||||
"total": 3,
|
||||
"metrics": [
|
||||
{
|
||||
"full_name": "jvm.cl.loaded",
|
||||
"metric": {
|
||||
"value": 18910
|
||||
},
|
||||
"name": "loaded",
|
||||
"type": "gauge"
|
||||
},
|
||||
{
|
||||
"full_name": "jvm.memory.pools.Metaspace.committed",
|
||||
"metric": {
|
||||
"value": 108040192
|
||||
},
|
||||
"name": "committed",
|
||||
"type": "gauge"
|
||||
},
|
||||
{
|
||||
"full_name": "org.graylog2.shared.journal.KafkaJournal.writeTime",
|
||||
"metric": {
|
||||
"time": {
|
||||
"min": 99
|
||||
},
|
||||
"rate": {
|
||||
"total": 10,
|
||||
"mean": 2
|
||||
},
|
||||
"duration_unit": "microseconds",
|
||||
"rate_unit": "events/second"
|
||||
},
|
||||
"name": "writeTime",
|
||||
"type": "hdrtimer"
|
||||
}
|
||||
]
|
||||
}`
|
||||
|
||||
var validTags = map[string]map[string]string{
|
||||
"jvm.cl.loaded": {
|
||||
"name": "loaded",
|
||||
"type": "gauge",
|
||||
"port": "12900",
|
||||
"server": "localhost",
|
||||
},
|
||||
"jvm.memory.pools.Metaspace.committed": {
|
||||
"name": "committed",
|
||||
"type": "gauge",
|
||||
"port": "12900",
|
||||
"server": "localhost",
|
||||
},
|
||||
"org.graylog2.shared.journal.KafkaJournal.writeTime": {
|
||||
"name": "writeTime",
|
||||
"type": "hdrtimer",
|
||||
"port": "12900",
|
||||
"server": "localhost",
|
||||
},
|
||||
}
|
||||
|
||||
var expectedFields = map[string]map[string]interface{}{
|
||||
"jvm.cl.loaded": {
|
||||
"value": float64(18910),
|
||||
},
|
||||
"jvm.memory.pools.Metaspace.committed": {
|
||||
"value": float64(108040192),
|
||||
},
|
||||
"org.graylog2.shared.journal.KafkaJournal.writeTime": {
|
||||
"time_min": float64(99),
|
||||
"rate_total": float64(10),
|
||||
"rate_mean": float64(2),
|
||||
},
|
||||
}
|
||||
|
||||
const invalidJSON = "I don't think this is JSON"
|
||||
|
||||
const empty = ""
|
||||
|
||||
type mockHTTPClient struct {
|
||||
responseBody string
|
||||
statusCode int
|
||||
}
|
||||
|
||||
// Mock implementation of MakeRequest. Usually returns an http.Response with
|
||||
// hard-coded responseBody and statusCode. However, if the request uses a
|
||||
// nonstandard method, it uses status code 405 (method not allowed)
|
||||
func (c *mockHTTPClient) MakeRequest(req *http.Request) (*http.Response, error) {
|
||||
resp := http.Response{}
|
||||
resp.StatusCode = c.statusCode
|
||||
|
||||
// basic error checking on request method
|
||||
allowedMethods := []string{"GET", "HEAD", "POST", "PUT", "DELETE", "TRACE", "CONNECT"}
|
||||
methodValid := false
|
||||
for _, method := range allowedMethods {
|
||||
if req.Method == method {
|
||||
methodValid = true
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if !methodValid {
|
||||
resp.StatusCode = 405 // Method not allowed
|
||||
}
|
||||
|
||||
resp.Body = ioutil.NopCloser(strings.NewReader(c.responseBody))
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
func (c *mockHTTPClient) SetHTTPClient(_ *http.Client) {
|
||||
}
|
||||
|
||||
func (c *mockHTTPClient) HTTPClient() *http.Client {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Generates a pointer to an HttpJson object that uses a mock HTTP client.
|
||||
// Parameters:
|
||||
// response : Body of the response that the mock HTTP client should return
|
||||
// statusCode: HTTP status code the mock HTTP client should return
|
||||
//
|
||||
// Returns:
|
||||
// *HttpJson: Pointer to an HttpJson object that uses the generated mock HTTP client
|
||||
func genMockGrayLog(response string, statusCode int) []*GrayLog {
|
||||
return []*GrayLog{
|
||||
&GrayLog{
|
||||
client: &mockHTTPClient{responseBody: response, statusCode: statusCode},
|
||||
Servers: []string{
|
||||
"http://localhost:12900/system/metrics/multiple",
|
||||
},
|
||||
Metrics: []string{
|
||||
"jvm.memory.pools.Metaspace.committed",
|
||||
"jvm.cl.loaded",
|
||||
"org.graylog2.shared.journal.KafkaJournal.writeTime",
|
||||
},
|
||||
Username: "test",
|
||||
Password: "test",
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// Test that the proper values are ignored or collected
|
||||
func TestNormalResponse(t *testing.T) {
|
||||
graylog := genMockGrayLog(validJSON, 200)
|
||||
|
||||
for _, service := range graylog {
|
||||
var acc testutil.Accumulator
|
||||
err := service.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
for k, v := range expectedFields {
|
||||
acc.AssertContainsTaggedFields(t, k, v, validTags[k])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Test response to HTTP 500
|
||||
func TestHttpJson500(t *testing.T) {
|
||||
graylog := genMockGrayLog(validJSON, 500)
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := graylog[0].Gather(&acc)
|
||||
|
||||
assert.NotNil(t, err)
|
||||
assert.Equal(t, 0, acc.NFields())
|
||||
}
|
||||
|
||||
// Test response to malformed JSON
|
||||
func TestHttpJsonBadJson(t *testing.T) {
|
||||
graylog := genMockGrayLog(invalidJSON, 200)
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := graylog[0].Gather(&acc)
|
||||
|
||||
assert.NotNil(t, err)
|
||||
assert.Equal(t, 0, acc.NFields())
|
||||
}
|
||||
|
||||
// Test response to empty string as response objectgT
|
||||
func TestHttpJsonEmptyResponse(t *testing.T) {
|
||||
graylog := genMockGrayLog(empty, 200)
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := graylog[0].Gather(&acc)
|
||||
|
||||
assert.NotNil(t, err)
|
||||
assert.Equal(t, 0, acc.NFields())
|
||||
}
|
||||
@@ -3,6 +3,8 @@ package haproxy
|
||||
import (
|
||||
"encoding/csv"
|
||||
"fmt"
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
@@ -11,10 +13,6 @@ import (
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal/errchan"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
//CSV format: https://cbonte.github.io/haproxy-dconv/configuration-1.5.html#9.1
|
||||
@@ -115,17 +113,20 @@ func (g *haproxy) Gather(acc telegraf.Accumulator) error {
|
||||
}
|
||||
|
||||
var wg sync.WaitGroup
|
||||
errChan := errchan.New(len(g.Servers))
|
||||
wg.Add(len(g.Servers))
|
||||
for _, server := range g.Servers {
|
||||
|
||||
var outerr error
|
||||
|
||||
for _, serv := range g.Servers {
|
||||
wg.Add(1)
|
||||
go func(serv string) {
|
||||
defer wg.Done()
|
||||
errChan.C <- g.gatherServer(serv, acc)
|
||||
}(server)
|
||||
outerr = g.gatherServer(serv, acc)
|
||||
}(serv)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
return errChan.Error()
|
||||
|
||||
return outerr
|
||||
}
|
||||
|
||||
func (g *haproxy) gatherServerSocket(addr string, acc telegraf.Accumulator) error {
|
||||
|
||||
@@ -22,13 +22,6 @@ This input plugin will test HTTP/HTTPS connections.
|
||||
# body = '''
|
||||
# {'fake':'data'}
|
||||
# '''
|
||||
|
||||
## Optional SSL Config
|
||||
# ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# ssl_key = "/etc/telegraf/key.pem"
|
||||
## Use SSL but skip chain & host verification
|
||||
# insecure_skip_verify = false
|
||||
```
|
||||
|
||||
### Measurements & Fields:
|
||||
|
||||
@@ -21,15 +21,6 @@ type HTTPResponse struct {
|
||||
ResponseTimeout internal.Duration
|
||||
Headers map[string]string
|
||||
FollowRedirects bool
|
||||
|
||||
// Path to CA file
|
||||
SSLCA string `toml:"ssl_ca"`
|
||||
// Path to host cert file
|
||||
SSLCert string `toml:"ssl_cert"`
|
||||
// Path to cert key file
|
||||
SSLKey string `toml:"ssl_key"`
|
||||
// Use SSL but skip chain & host verification
|
||||
InsecureSkipVerify bool
|
||||
}
|
||||
|
||||
// Description returns the plugin Description
|
||||
@@ -53,13 +44,6 @@ var sampleConfig = `
|
||||
# body = '''
|
||||
# {'fake':'data'}
|
||||
# '''
|
||||
|
||||
## Optional SSL Config
|
||||
# ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# ssl_key = "/etc/telegraf/key.pem"
|
||||
## Use SSL but skip chain & host verification
|
||||
# insecure_skip_verify = false
|
||||
`
|
||||
|
||||
// SampleConfig returns the plugin SampleConfig
|
||||
@@ -72,27 +56,17 @@ var ErrRedirectAttempted = errors.New("redirect")
|
||||
|
||||
// CreateHttpClient creates an http client which will timeout at the specified
|
||||
// timeout period and can follow redirects if specified
|
||||
func (h *HTTPResponse) createHttpClient() (*http.Client, error) {
|
||||
tlsCfg, err := internal.GetTLSConfig(
|
||||
h.SSLCert, h.SSLKey, h.SSLCA, h.InsecureSkipVerify)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tr := &http.Transport{
|
||||
ResponseHeaderTimeout: h.ResponseTimeout.Duration,
|
||||
TLSClientConfig: tlsCfg,
|
||||
}
|
||||
func CreateHttpClient(followRedirects bool, ResponseTimeout time.Duration) *http.Client {
|
||||
client := &http.Client{
|
||||
Transport: tr,
|
||||
Timeout: h.ResponseTimeout.Duration,
|
||||
Timeout: ResponseTimeout,
|
||||
}
|
||||
|
||||
if h.FollowRedirects == false {
|
||||
if followRedirects == false {
|
||||
client.CheckRedirect = func(req *http.Request, via []*http.Request) error {
|
||||
return ErrRedirectAttempted
|
||||
}
|
||||
}
|
||||
return client, nil
|
||||
return client
|
||||
}
|
||||
|
||||
// HTTPGather gathers all fields and returns any errors it encounters
|
||||
@@ -100,10 +74,7 @@ func (h *HTTPResponse) HTTPGather() (map[string]interface{}, error) {
|
||||
// Prepare fields
|
||||
fields := make(map[string]interface{})
|
||||
|
||||
client, err := h.createHttpClient()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
client := CreateHttpClient(h.FollowRedirects, h.ResponseTimeout.Duration)
|
||||
|
||||
var body io.Reader
|
||||
if h.Body != "" {
|
||||
|
||||
@@ -15,7 +15,6 @@ InfluxDB-formatted endpoints. See below for more information.
|
||||
## See the influxdb plugin's README for more details.
|
||||
|
||||
## Multiple URLs from which to read InfluxDB-formatted JSON
|
||||
## Default is "http://localhost:8086/debug/vars".
|
||||
urls = [
|
||||
"http://localhost:8086/debug/vars"
|
||||
]
|
||||
|
||||
@@ -28,7 +28,6 @@ func (*InfluxDB) SampleConfig() string {
|
||||
## See the influxdb plugin's README for more details.
|
||||
|
||||
## Multiple URLs from which to read InfluxDB-formatted JSON
|
||||
## Default is "http://localhost:8086/debug/vars".
|
||||
urls = [
|
||||
"http://localhost:8086/debug/vars"
|
||||
]
|
||||
@@ -36,9 +35,6 @@ func (*InfluxDB) SampleConfig() string {
|
||||
}
|
||||
|
||||
func (i *InfluxDB) Gather(acc telegraf.Accumulator) error {
|
||||
if len(i.URLs) == 0 {
|
||||
i.URLs = []string{"http://localhost:8086/debug/vars"}
|
||||
}
|
||||
errorChannel := make(chan error, len(i.URLs))
|
||||
|
||||
var wg sync.WaitGroup
|
||||
@@ -161,45 +157,43 @@ func (i *InfluxDB) gatherURL(
|
||||
return err
|
||||
}
|
||||
|
||||
if keyStr, ok := key.(string); ok {
|
||||
if keyStr == "memstats" {
|
||||
var m memstats
|
||||
if err := dec.Decode(&m); err != nil {
|
||||
continue
|
||||
}
|
||||
acc.AddFields("influxdb_memstats",
|
||||
map[string]interface{}{
|
||||
"alloc": m.Alloc,
|
||||
"total_alloc": m.TotalAlloc,
|
||||
"sys": m.Sys,
|
||||
"lookups": m.Lookups,
|
||||
"mallocs": m.Mallocs,
|
||||
"frees": m.Frees,
|
||||
"heap_alloc": m.HeapAlloc,
|
||||
"heap_sys": m.HeapSys,
|
||||
"heap_idle": m.HeapIdle,
|
||||
"heap_inuse": m.HeapInuse,
|
||||
"heap_released": m.HeapReleased,
|
||||
"heap_objects": m.HeapObjects,
|
||||
"stack_inuse": m.StackInuse,
|
||||
"stack_sys": m.StackSys,
|
||||
"mspan_inuse": m.MSpanInuse,
|
||||
"mspan_sys": m.MSpanSys,
|
||||
"mcache_inuse": m.MCacheInuse,
|
||||
"mcache_sys": m.MCacheSys,
|
||||
"buck_hash_sys": m.BuckHashSys,
|
||||
"gc_sys": m.GCSys,
|
||||
"other_sys": m.OtherSys,
|
||||
"next_gc": m.NextGC,
|
||||
"last_gc": m.LastGC,
|
||||
"pause_total_ns": m.PauseTotalNs,
|
||||
"num_gc": m.NumGC,
|
||||
"gcc_pu_fraction": m.GCCPUFraction,
|
||||
},
|
||||
map[string]string{
|
||||
"url": url,
|
||||
})
|
||||
if key.(string) == "memstats" {
|
||||
var m memstats
|
||||
if err := dec.Decode(&m); err != nil {
|
||||
continue
|
||||
}
|
||||
acc.AddFields("influxdb_memstats",
|
||||
map[string]interface{}{
|
||||
"alloc": m.Alloc,
|
||||
"total_alloc": m.TotalAlloc,
|
||||
"sys": m.Sys,
|
||||
"lookups": m.Lookups,
|
||||
"mallocs": m.Mallocs,
|
||||
"frees": m.Frees,
|
||||
"heap_alloc": m.HeapAlloc,
|
||||
"heap_sys": m.HeapSys,
|
||||
"heap_idle": m.HeapIdle,
|
||||
"heap_inuse": m.HeapInuse,
|
||||
"heap_released": m.HeapReleased,
|
||||
"heap_objects": m.HeapObjects,
|
||||
"stack_inuse": m.StackInuse,
|
||||
"stack_sys": m.StackSys,
|
||||
"mspan_inuse": m.MSpanInuse,
|
||||
"mspan_sys": m.MSpanSys,
|
||||
"mcache_inuse": m.MCacheInuse,
|
||||
"mcache_sys": m.MCacheSys,
|
||||
"buck_hash_sys": m.BuckHashSys,
|
||||
"gc_sys": m.GCSys,
|
||||
"other_sys": m.OtherSys,
|
||||
"next_gc": m.NextGC,
|
||||
"last_gc": m.LastGC,
|
||||
"pause_total_ns": m.PauseTotalNs,
|
||||
"num_gc": m.NumGC,
|
||||
"gcc_pu_fraction": m.GCCPUFraction,
|
||||
},
|
||||
map[string]string{
|
||||
"url": url,
|
||||
})
|
||||
}
|
||||
|
||||
// Attempt to parse a whole object into a point.
|
||||
@@ -210,16 +204,16 @@ func (i *InfluxDB) gatherURL(
|
||||
continue
|
||||
}
|
||||
|
||||
if p.Name == "shard" {
|
||||
shardCounter++
|
||||
}
|
||||
|
||||
// If the object was a point, but was not fully initialized,
|
||||
// ignore it and move on.
|
||||
if p.Name == "" || p.Tags == nil || p.Values == nil || len(p.Values) == 0 {
|
||||
continue
|
||||
}
|
||||
|
||||
if p.Name == "shard" {
|
||||
shardCounter++
|
||||
}
|
||||
|
||||
// Add a tag to indicate the source of the data.
|
||||
p.Tags["url"] = url
|
||||
|
||||
|
||||
@@ -112,7 +112,7 @@ func TestInfluxDB(t *testing.T) {
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "influxdb",
|
||||
map[string]interface{}{
|
||||
"n_shards": 1,
|
||||
"n_shards": 2,
|
||||
}, map[string]string{})
|
||||
}
|
||||
|
||||
|
||||
@@ -50,7 +50,7 @@ var sampleConfig = `
|
||||
## an array of Zookeeper connection strings
|
||||
zookeeper_peers = ["localhost:2181"]
|
||||
## Zookeeper Chroot
|
||||
zookeeper_chroot = ""
|
||||
zookeeper_chroot = "/"
|
||||
## the name of the consumer group
|
||||
consumer_group = "telegraf_metrics_consumers"
|
||||
## Offset (must be either "oldest" or "newest")
|
||||
|
||||
@@ -1,89 +0,0 @@
|
||||
# logparser Input Plugin
|
||||
|
||||
The logparser plugin streams and parses the given logfiles. Currently it only
|
||||
has the capability of parsing "grok" patterns from logfiles, which also supports
|
||||
regex patterns.
|
||||
|
||||
### Configuration:
|
||||
|
||||
```toml
|
||||
[[inputs.logparser]]
|
||||
## Log files to parse.
|
||||
## These accept standard unix glob matching rules, but with the addition of
|
||||
## ** as a "super asterisk". ie:
|
||||
## /var/log/**.log -> recursively find all .log files in /var/log
|
||||
## /var/log/*/*.log -> find all .log files with a parent dir in /var/log
|
||||
## /var/log/apache.log -> only tail the apache log file
|
||||
files = ["/var/log/influxdb/influxdb.log"]
|
||||
## Read file from beginning.
|
||||
from_beginning = false
|
||||
|
||||
## Parse logstash-style "grok" patterns:
|
||||
## Telegraf builtin parsing patterns: https://goo.gl/dkay10
|
||||
[inputs.logparser.grok]
|
||||
## This is a list of patterns to check the given log file(s) for.
|
||||
## Note that adding patterns here increases processing time. The most
|
||||
## efficient configuration is to have one file & pattern per logparser.
|
||||
patterns = ["%{INFLUXDB_HTTPD_LOG}"]
|
||||
## Full path(s) to custom pattern files.
|
||||
custom_pattern_files = []
|
||||
## Custom patterns can also be defined here. Put one pattern per line.
|
||||
custom_patterns = '''
|
||||
'''
|
||||
```
|
||||
|
||||
## Grok Parser
|
||||
|
||||
The grok parser uses a slightly modified version of logstash "grok" patterns,
|
||||
with the format `%{<capture_syntax>[:<semantic_name>][:<modifier>]}`
|
||||
|
||||
|
||||
Telegraf has many of it's own
|
||||
[built-in patterns](https://github.com/influxdata/telegraf/blob/master/plugins/inputs/logparser/grok/patterns/influx-patterns),
|
||||
as well as supporting
|
||||
[logstash's builtin patterns](https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/grok-patterns).
|
||||
|
||||
|
||||
The best way to get acquainted with grok patterns is to read the logstash docs,
|
||||
which are available here:
|
||||
https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html
|
||||
|
||||
|
||||
If you need help building patterns to match your logs,
|
||||
you will find the http://grokdebug.herokuapp.com application quite useful!
|
||||
|
||||
|
||||
By default all named captures are converted into string fields.
|
||||
Modifiers can be used to convert captures to other types or tags.
|
||||
Timestamp modifiers can be used to convert captures to the timestamp of the
|
||||
parsed metric.
|
||||
|
||||
|
||||
- Available modifiers:
|
||||
- string (default if nothing is specified)
|
||||
- int
|
||||
- float
|
||||
- duration (ie, 5.23ms gets converted to int nanoseconds)
|
||||
- tag (converts the field into a tag)
|
||||
- drop (drops the field completely)
|
||||
- Timestamp modifiers:
|
||||
- ts-ansic ("Mon Jan _2 15:04:05 2006")
|
||||
- ts-unix ("Mon Jan _2 15:04:05 MST 2006")
|
||||
- ts-ruby ("Mon Jan 02 15:04:05 -0700 2006")
|
||||
- ts-rfc822 ("02 Jan 06 15:04 MST")
|
||||
- ts-rfc822z ("02 Jan 06 15:04 -0700")
|
||||
- ts-rfc850 ("Monday, 02-Jan-06 15:04:05 MST")
|
||||
- ts-rfc1123 ("Mon, 02 Jan 2006 15:04:05 MST")
|
||||
- ts-rfc1123z ("Mon, 02 Jan 2006 15:04:05 -0700")
|
||||
- ts-rfc3339 ("2006-01-02T15:04:05Z07:00")
|
||||
- ts-rfc3339nano ("2006-01-02T15:04:05.999999999Z07:00")
|
||||
- ts-httpd ("02/Jan/2006:15:04:05 -0700")
|
||||
- ts-epoch (seconds since unix epoch)
|
||||
- ts-epochnano (nanoseconds since unix epoch)
|
||||
- ts-"CUSTOM"
|
||||
|
||||
|
||||
CUSTOM time layouts must be within quotes and be the representation of the
|
||||
"reference time", which is `Mon Jan 2 15:04:05 -0700 MST 2006`
|
||||
See https://golang.org/pkg/time/#Parse for more details.
|
||||
|
||||
@@ -1,373 +0,0 @@
|
||||
package grok
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"fmt"
|
||||
"log"
|
||||
"os"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/vjeantet/grok"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
)
|
||||
|
||||
var timeFormats = map[string]string{
|
||||
"ts-ansic": "Mon Jan _2 15:04:05 2006",
|
||||
"ts-unix": "Mon Jan _2 15:04:05 MST 2006",
|
||||
"ts-ruby": "Mon Jan 02 15:04:05 -0700 2006",
|
||||
"ts-rfc822": "02 Jan 06 15:04 MST",
|
||||
"ts-rfc822z": "02 Jan 06 15:04 -0700", // RFC822 with numeric zone
|
||||
"ts-rfc850": "Monday, 02-Jan-06 15:04:05 MST",
|
||||
"ts-rfc1123": "Mon, 02 Jan 2006 15:04:05 MST",
|
||||
"ts-rfc1123z": "Mon, 02 Jan 2006 15:04:05 -0700", // RFC1123 with numeric zone
|
||||
"ts-rfc3339": "2006-01-02T15:04:05Z07:00",
|
||||
"ts-rfc3339nano": "2006-01-02T15:04:05.999999999Z07:00",
|
||||
"ts-httpd": "02/Jan/2006:15:04:05 -0700",
|
||||
"ts-epoch": "EPOCH",
|
||||
"ts-epochnano": "EPOCH_NANO",
|
||||
}
|
||||
|
||||
const (
|
||||
INT = "int"
|
||||
TAG = "tag"
|
||||
FLOAT = "float"
|
||||
STRING = "string"
|
||||
DURATION = "duration"
|
||||
DROP = "drop"
|
||||
)
|
||||
|
||||
var (
|
||||
// matches named captures that contain a type.
|
||||
// ie,
|
||||
// %{NUMBER:bytes:int}
|
||||
// %{IPORHOST:clientip:tag}
|
||||
// %{HTTPDATE:ts1:ts-http}
|
||||
// %{HTTPDATE:ts2:ts-"02 Jan 06 15:04"}
|
||||
typedRe = regexp.MustCompile(`%{\w+:(\w+):(ts-".+"|t?s?-?\w+)}`)
|
||||
// matches a plain pattern name. ie, %{NUMBER}
|
||||
patternOnlyRe = regexp.MustCompile(`%{(\w+)}`)
|
||||
)
|
||||
|
||||
type Parser struct {
|
||||
Patterns []string
|
||||
CustomPatterns string
|
||||
CustomPatternFiles []string
|
||||
|
||||
// typeMap is a map of patterns -> capture name -> modifier,
|
||||
// ie, {
|
||||
// "%{TESTLOG}":
|
||||
// {
|
||||
// "bytes": "int",
|
||||
// "clientip": "tag"
|
||||
// }
|
||||
// }
|
||||
typeMap map[string]map[string]string
|
||||
// tsMap is a map of patterns -> capture name -> timestamp layout.
|
||||
// ie, {
|
||||
// "%{TESTLOG}":
|
||||
// {
|
||||
// "httptime": "02/Jan/2006:15:04:05 -0700"
|
||||
// }
|
||||
// }
|
||||
tsMap map[string]map[string]string
|
||||
// patterns is a map of all of the parsed patterns from CustomPatterns
|
||||
// and CustomPatternFiles.
|
||||
// ie, {
|
||||
// "DURATION": "%{NUMBER}[nuµm]?s"
|
||||
// "RESPONSE_CODE": "%{NUMBER:rc:tag}"
|
||||
// }
|
||||
patterns map[string]string
|
||||
|
||||
g *grok.Grok
|
||||
tsModder *tsModder
|
||||
}
|
||||
|
||||
func (p *Parser) Compile() error {
|
||||
p.typeMap = make(map[string]map[string]string)
|
||||
p.tsMap = make(map[string]map[string]string)
|
||||
p.patterns = make(map[string]string)
|
||||
p.tsModder = &tsModder{}
|
||||
var err error
|
||||
p.g, err = grok.NewWithConfig(&grok.Config{NamedCapturesOnly: true})
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
p.CustomPatterns = DEFAULT_PATTERNS + p.CustomPatterns
|
||||
|
||||
if len(p.CustomPatterns) != 0 {
|
||||
scanner := bufio.NewScanner(strings.NewReader(p.CustomPatterns))
|
||||
p.addCustomPatterns(scanner)
|
||||
}
|
||||
|
||||
for _, filename := range p.CustomPatternFiles {
|
||||
file, err := os.Open(filename)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
scanner := bufio.NewScanner(bufio.NewReader(file))
|
||||
p.addCustomPatterns(scanner)
|
||||
}
|
||||
|
||||
return p.compileCustomPatterns()
|
||||
}
|
||||
|
||||
func (p *Parser) ParseLine(line string) (telegraf.Metric, error) {
|
||||
var err error
|
||||
var values map[string]string
|
||||
// the matching pattern string
|
||||
var patternName string
|
||||
for _, pattern := range p.Patterns {
|
||||
if values, err = p.g.Parse(pattern, line); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if len(values) != 0 {
|
||||
patternName = pattern
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if len(values) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
fields := make(map[string]interface{})
|
||||
tags := make(map[string]string)
|
||||
timestamp := time.Now()
|
||||
for k, v := range values {
|
||||
if k == "" || v == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
var t string
|
||||
// check if pattern has some modifiers
|
||||
if types, ok := p.typeMap[patternName]; ok {
|
||||
t = types[k]
|
||||
}
|
||||
// if we didn't find a modifier, check if we have a timestamp layout
|
||||
if t == "" {
|
||||
if ts, ok := p.tsMap[patternName]; ok {
|
||||
// check if the modifier is a timestamp layout
|
||||
if layout, ok := ts[k]; ok {
|
||||
t = layout
|
||||
}
|
||||
}
|
||||
}
|
||||
// if we didn't find a type OR timestamp modifier, assume string
|
||||
if t == "" {
|
||||
t = STRING
|
||||
}
|
||||
|
||||
switch t {
|
||||
case INT:
|
||||
iv, err := strconv.ParseInt(v, 10, 64)
|
||||
if err != nil {
|
||||
log.Printf("ERROR parsing %s to int: %s", v, err)
|
||||
} else {
|
||||
fields[k] = iv
|
||||
}
|
||||
case FLOAT:
|
||||
fv, err := strconv.ParseFloat(v, 64)
|
||||
if err != nil {
|
||||
log.Printf("ERROR parsing %s to float: %s", v, err)
|
||||
} else {
|
||||
fields[k] = fv
|
||||
}
|
||||
case DURATION:
|
||||
d, err := time.ParseDuration(v)
|
||||
if err != nil {
|
||||
log.Printf("ERROR parsing %s to duration: %s", v, err)
|
||||
} else {
|
||||
fields[k] = int64(d)
|
||||
}
|
||||
case TAG:
|
||||
tags[k] = v
|
||||
case STRING:
|
||||
fields[k] = strings.Trim(v, `"`)
|
||||
case "EPOCH":
|
||||
iv, err := strconv.ParseInt(v, 10, 64)
|
||||
if err != nil {
|
||||
log.Printf("ERROR parsing %s to int: %s", v, err)
|
||||
} else {
|
||||
timestamp = time.Unix(iv, 0)
|
||||
}
|
||||
case "EPOCH_NANO":
|
||||
iv, err := strconv.ParseInt(v, 10, 64)
|
||||
if err != nil {
|
||||
log.Printf("ERROR parsing %s to int: %s", v, err)
|
||||
} else {
|
||||
timestamp = time.Unix(0, iv)
|
||||
}
|
||||
case DROP:
|
||||
// goodbye!
|
||||
default:
|
||||
ts, err := time.Parse(t, v)
|
||||
if err == nil {
|
||||
timestamp = ts
|
||||
} else {
|
||||
log.Printf("ERROR parsing %s to time layout [%s]: %s", v, t, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return telegraf.NewMetric("logparser_grok", tags, fields, p.tsModder.tsMod(timestamp))
|
||||
}
|
||||
|
||||
func (p *Parser) addCustomPatterns(scanner *bufio.Scanner) {
|
||||
for scanner.Scan() {
|
||||
line := strings.TrimSpace(scanner.Text())
|
||||
if len(line) > 0 && line[0] != '#' {
|
||||
names := strings.SplitN(line, " ", 2)
|
||||
p.patterns[names[0]] = names[1]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (p *Parser) compileCustomPatterns() error {
|
||||
var err error
|
||||
// check if the pattern contains a subpattern that is already defined
|
||||
// replace it with the subpattern for modifier inheritance.
|
||||
for i := 0; i < 2; i++ {
|
||||
for name, pattern := range p.patterns {
|
||||
subNames := patternOnlyRe.FindAllStringSubmatch(pattern, -1)
|
||||
for _, subName := range subNames {
|
||||
if subPattern, ok := p.patterns[subName[1]]; ok {
|
||||
pattern = strings.Replace(pattern, subName[0], subPattern, 1)
|
||||
}
|
||||
}
|
||||
p.patterns[name] = pattern
|
||||
}
|
||||
}
|
||||
|
||||
// check if pattern contains modifiers. Parse them out if it does.
|
||||
for name, pattern := range p.patterns {
|
||||
if typedRe.MatchString(pattern) {
|
||||
// this pattern has modifiers, so parse out the modifiers
|
||||
pattern, err = p.parseTypedCaptures(name, pattern)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
p.patterns[name] = pattern
|
||||
}
|
||||
}
|
||||
|
||||
return p.g.AddPatternsFromMap(p.patterns)
|
||||
}
|
||||
|
||||
// parseTypedCaptures parses the capture types, and then deletes the type from
|
||||
// the line so that it is a valid "grok" pattern again.
|
||||
// ie,
|
||||
// %{NUMBER:bytes:int} => %{NUMBER:bytes} (stores %{NUMBER}->bytes->int)
|
||||
// %{IPORHOST:clientip:tag} => %{IPORHOST:clientip} (stores %{IPORHOST}->clientip->tag)
|
||||
func (p *Parser) parseTypedCaptures(name, pattern string) (string, error) {
|
||||
matches := typedRe.FindAllStringSubmatch(pattern, -1)
|
||||
|
||||
// grab the name of the capture pattern
|
||||
patternName := "%{" + name + "}"
|
||||
// create type map for this pattern
|
||||
p.typeMap[patternName] = make(map[string]string)
|
||||
p.tsMap[patternName] = make(map[string]string)
|
||||
|
||||
// boolean to verify that each pattern only has a single ts- data type.
|
||||
hasTimestamp := false
|
||||
for _, match := range matches {
|
||||
// regex capture 1 is the name of the capture
|
||||
// regex capture 2 is the type of the capture
|
||||
if strings.HasPrefix(match[2], "ts-") {
|
||||
if hasTimestamp {
|
||||
return pattern, fmt.Errorf("logparser pattern compile error: "+
|
||||
"Each pattern is allowed only one named "+
|
||||
"timestamp data type. pattern: %s", pattern)
|
||||
}
|
||||
if f, ok := timeFormats[match[2]]; ok {
|
||||
p.tsMap[patternName][match[1]] = f
|
||||
} else {
|
||||
p.tsMap[patternName][match[1]] = strings.TrimSuffix(strings.TrimPrefix(match[2], `ts-"`), `"`)
|
||||
}
|
||||
hasTimestamp = true
|
||||
} else {
|
||||
p.typeMap[patternName][match[1]] = match[2]
|
||||
}
|
||||
|
||||
// the modifier is not a valid part of a "grok" pattern, so remove it
|
||||
// from the pattern.
|
||||
pattern = strings.Replace(pattern, ":"+match[2]+"}", "}", 1)
|
||||
}
|
||||
|
||||
return pattern, nil
|
||||
}
|
||||
|
||||
// tsModder is a struct for incrementing identical timestamps of log lines
|
||||
// so that we don't push identical metrics that will get overwritten.
|
||||
type tsModder struct {
|
||||
dupe time.Time
|
||||
last time.Time
|
||||
incr time.Duration
|
||||
incrn time.Duration
|
||||
rollover time.Duration
|
||||
}
|
||||
|
||||
// tsMod increments the given timestamp one unit more from the previous
|
||||
// duplicate timestamp.
|
||||
// the increment unit is determined as the next smallest time unit below the
|
||||
// most significant time unit of ts.
|
||||
// ie, if the input is at ms precision, it will increment it 1µs.
|
||||
func (t *tsModder) tsMod(ts time.Time) time.Time {
|
||||
defer func() { t.last = ts }()
|
||||
// don't mod the time if we don't need to
|
||||
if t.last.IsZero() || ts.IsZero() {
|
||||
t.incrn = 0
|
||||
t.rollover = 0
|
||||
return ts
|
||||
}
|
||||
if !ts.Equal(t.last) && !ts.Equal(t.dupe) {
|
||||
t.incr = 0
|
||||
t.incrn = 0
|
||||
t.rollover = 0
|
||||
return ts
|
||||
}
|
||||
|
||||
if ts.Equal(t.last) {
|
||||
t.dupe = ts
|
||||
}
|
||||
|
||||
if ts.Equal(t.dupe) && t.incr == time.Duration(0) {
|
||||
tsNano := ts.UnixNano()
|
||||
|
||||
d := int64(10)
|
||||
counter := 1
|
||||
for {
|
||||
a := tsNano % d
|
||||
if a > 0 {
|
||||
break
|
||||
}
|
||||
d = d * 10
|
||||
counter++
|
||||
}
|
||||
|
||||
switch {
|
||||
case counter <= 6:
|
||||
t.incr = time.Nanosecond
|
||||
case counter <= 9:
|
||||
t.incr = time.Microsecond
|
||||
case counter > 9:
|
||||
t.incr = time.Millisecond
|
||||
}
|
||||
}
|
||||
|
||||
t.incrn++
|
||||
if t.incrn == 999 && t.incr > time.Nanosecond {
|
||||
t.rollover = t.incr * t.incrn
|
||||
t.incrn = 1
|
||||
t.incr = t.incr / 1000
|
||||
if t.incr < time.Nanosecond {
|
||||
t.incr = time.Nanosecond
|
||||
}
|
||||
}
|
||||
return ts.Add(t.incr*t.incrn + t.rollover)
|
||||
}
|
||||
@@ -1,508 +0,0 @@
|
||||
package grok
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
var benchM telegraf.Metric
|
||||
|
||||
func Benchmark_ParseLine_CommonLogFormat(b *testing.B) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{COMMON_LOG_FORMAT}"},
|
||||
}
|
||||
p.Compile()
|
||||
|
||||
var m telegraf.Metric
|
||||
for n := 0; n < b.N; n++ {
|
||||
m, _ = p.ParseLine(`127.0.0.1 user-identifier frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326`)
|
||||
}
|
||||
benchM = m
|
||||
}
|
||||
|
||||
func Benchmark_ParseLine_CombinedLogFormat(b *testing.B) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{COMBINED_LOG_FORMAT}"},
|
||||
}
|
||||
p.Compile()
|
||||
|
||||
var m telegraf.Metric
|
||||
for n := 0; n < b.N; n++ {
|
||||
m, _ = p.ParseLine(`127.0.0.1 user-identifier frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 "-" "Mozilla"`)
|
||||
}
|
||||
benchM = m
|
||||
}
|
||||
|
||||
func Benchmark_ParseLine_InfluxLog(b *testing.B) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{INFLUXDB_HTTPD_LOG}"},
|
||||
}
|
||||
p.Compile()
|
||||
|
||||
var m telegraf.Metric
|
||||
for n := 0; n < b.N; n++ {
|
||||
m, _ = p.ParseLine(`[httpd] 192.168.1.1 - - [14/Jun/2016:11:33:29 +0100] "POST /write?consistency=any&db=telegraf&precision=ns&rp= HTTP/1.1" 204 0 "-" "InfluxDBClient" 6f61bc44-321b-11e6-8050-000000000000 2513`)
|
||||
}
|
||||
benchM = m
|
||||
}
|
||||
|
||||
func Benchmark_ParseLine_InfluxLog_NoMatch(b *testing.B) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{INFLUXDB_HTTPD_LOG}"},
|
||||
}
|
||||
p.Compile()
|
||||
|
||||
var m telegraf.Metric
|
||||
for n := 0; n < b.N; n++ {
|
||||
m, _ = p.ParseLine(`[retention] 2016/06/14 14:38:24 retention policy shard deletion check commencing`)
|
||||
}
|
||||
benchM = m
|
||||
}
|
||||
|
||||
func Benchmark_ParseLine_CustomPattern(b *testing.B) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{TEST_LOG_A}", "%{TEST_LOG_B}"},
|
||||
CustomPatterns: `
|
||||
DURATION %{NUMBER}[nuµm]?s
|
||||
RESPONSE_CODE %{NUMBER:response_code:tag}
|
||||
RESPONSE_TIME %{DURATION:response_time:duration}
|
||||
TEST_LOG_A %{NUMBER:myfloat:float} %{RESPONSE_CODE} %{IPORHOST:clientip} %{RESPONSE_TIME}
|
||||
`,
|
||||
}
|
||||
p.Compile()
|
||||
|
||||
var m telegraf.Metric
|
||||
for n := 0; n < b.N; n++ {
|
||||
m, _ = p.ParseLine(`[04/Jun/2016:12:41:45 +0100] 1.25 200 192.168.1.1 5.432µs 101`)
|
||||
}
|
||||
benchM = m
|
||||
}
|
||||
|
||||
func TestBuiltinInfluxdbHttpd(t *testing.T) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{INFLUXDB_HTTPD_LOG}"},
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
|
||||
// Parse an influxdb POST request
|
||||
m, err := p.ParseLine(`[httpd] ::1 - - [14/Jun/2016:11:33:29 +0100] "POST /write?consistency=any&db=telegraf&precision=ns&rp= HTTP/1.1" 204 0 "-" "InfluxDBClient" 6f61bc44-321b-11e6-8050-000000000000 2513`)
|
||||
require.NotNil(t, m)
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t,
|
||||
map[string]interface{}{
|
||||
"resp_bytes": int64(0),
|
||||
"auth": "-",
|
||||
"client_ip": "::1",
|
||||
"resp_code": int64(204),
|
||||
"http_version": float64(1.1),
|
||||
"ident": "-",
|
||||
"referrer": "-",
|
||||
"request": "/write?consistency=any&db=telegraf&precision=ns&rp=",
|
||||
"response_time_us": int64(2513),
|
||||
"agent": "InfluxDBClient",
|
||||
},
|
||||
m.Fields())
|
||||
assert.Equal(t, map[string]string{"verb": "POST"}, m.Tags())
|
||||
|
||||
// Parse an influxdb GET request
|
||||
m, err = p.ParseLine(`[httpd] ::1 - - [14/Jun/2016:12:10:02 +0100] "GET /query?db=telegraf&q=SELECT+bytes%2Cresponse_time_us+FROM+logparser_grok+WHERE+http_method+%3D+%27GET%27+AND+response_time_us+%3E+0+AND+time+%3E+now%28%29+-+1h HTTP/1.1" 200 578 "http://localhost:8083/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36" 8a3806f1-3220-11e6-8006-000000000000 988`)
|
||||
require.NotNil(t, m)
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t,
|
||||
map[string]interface{}{
|
||||
"resp_bytes": int64(578),
|
||||
"auth": "-",
|
||||
"client_ip": "::1",
|
||||
"resp_code": int64(200),
|
||||
"http_version": float64(1.1),
|
||||
"ident": "-",
|
||||
"referrer": "http://localhost:8083/",
|
||||
"request": "/query?db=telegraf&q=SELECT+bytes%2Cresponse_time_us+FROM+logparser_grok+WHERE+http_method+%3D+%27GET%27+AND+response_time_us+%3E+0+AND+time+%3E+now%28%29+-+1h",
|
||||
"response_time_us": int64(988),
|
||||
"agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.84 Safari/537.36",
|
||||
},
|
||||
m.Fields())
|
||||
assert.Equal(t, map[string]string{"verb": "GET"}, m.Tags())
|
||||
}
|
||||
|
||||
// common log format
|
||||
// 127.0.0.1 user-identifier frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326
|
||||
func TestBuiltinCommonLogFormat(t *testing.T) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{COMMON_LOG_FORMAT}"},
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
|
||||
// Parse an influxdb POST request
|
||||
m, err := p.ParseLine(`127.0.0.1 user-identifier frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326`)
|
||||
require.NotNil(t, m)
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t,
|
||||
map[string]interface{}{
|
||||
"resp_bytes": int64(2326),
|
||||
"auth": "frank",
|
||||
"client_ip": "127.0.0.1",
|
||||
"resp_code": int64(200),
|
||||
"http_version": float64(1.0),
|
||||
"ident": "user-identifier",
|
||||
"request": "/apache_pb.gif",
|
||||
},
|
||||
m.Fields())
|
||||
assert.Equal(t, map[string]string{"verb": "GET"}, m.Tags())
|
||||
}
|
||||
|
||||
// combined log format
|
||||
// 127.0.0.1 user-identifier frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 "-" "Mozilla"
|
||||
func TestBuiltinCombinedLogFormat(t *testing.T) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{COMBINED_LOG_FORMAT}"},
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
|
||||
// Parse an influxdb POST request
|
||||
m, err := p.ParseLine(`127.0.0.1 user-identifier frank [10/Oct/2000:13:55:36 -0700] "GET /apache_pb.gif HTTP/1.0" 200 2326 "-" "Mozilla"`)
|
||||
require.NotNil(t, m)
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t,
|
||||
map[string]interface{}{
|
||||
"resp_bytes": int64(2326),
|
||||
"auth": "frank",
|
||||
"client_ip": "127.0.0.1",
|
||||
"resp_code": int64(200),
|
||||
"http_version": float64(1.0),
|
||||
"ident": "user-identifier",
|
||||
"request": "/apache_pb.gif",
|
||||
"referrer": "-",
|
||||
"agent": "Mozilla",
|
||||
},
|
||||
m.Fields())
|
||||
assert.Equal(t, map[string]string{"verb": "GET"}, m.Tags())
|
||||
}
|
||||
|
||||
func TestCompileStringAndParse(t *testing.T) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{TEST_LOG_A}", "%{TEST_LOG_B}"},
|
||||
CustomPatterns: `
|
||||
DURATION %{NUMBER}[nuµm]?s
|
||||
RESPONSE_CODE %{NUMBER:response_code:tag}
|
||||
RESPONSE_TIME %{DURATION:response_time:duration}
|
||||
TEST_LOG_A %{NUMBER:myfloat:float} %{RESPONSE_CODE} %{IPORHOST:clientip} %{RESPONSE_TIME}
|
||||
`,
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
|
||||
metricA, err := p.ParseLine(`1.25 200 192.168.1.1 5.432µs`)
|
||||
require.NotNil(t, metricA)
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t,
|
||||
map[string]interface{}{
|
||||
"clientip": "192.168.1.1",
|
||||
"myfloat": float64(1.25),
|
||||
"response_time": int64(5432),
|
||||
},
|
||||
metricA.Fields())
|
||||
assert.Equal(t, map[string]string{"response_code": "200"}, metricA.Tags())
|
||||
}
|
||||
|
||||
func TestParseEpochNano(t *testing.T) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{MYAPP}"},
|
||||
CustomPatterns: `
|
||||
MYAPP %{POSINT:ts:ts-epochnano} response_time=%{POSINT:response_time:int} mymetric=%{NUMBER:metric:float}
|
||||
`,
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
|
||||
metricA, err := p.ParseLine(`1466004605359052000 response_time=20821 mymetric=10890.645`)
|
||||
require.NotNil(t, metricA)
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t,
|
||||
map[string]interface{}{
|
||||
"response_time": int64(20821),
|
||||
"metric": float64(10890.645),
|
||||
},
|
||||
metricA.Fields())
|
||||
assert.Equal(t, map[string]string{}, metricA.Tags())
|
||||
assert.Equal(t, time.Unix(0, 1466004605359052000), metricA.Time())
|
||||
}
|
||||
|
||||
func TestParseEpoch(t *testing.T) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{MYAPP}"},
|
||||
CustomPatterns: `
|
||||
MYAPP %{POSINT:ts:ts-epoch} response_time=%{POSINT:response_time:int} mymetric=%{NUMBER:metric:float}
|
||||
`,
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
|
||||
metricA, err := p.ParseLine(`1466004605 response_time=20821 mymetric=10890.645`)
|
||||
require.NotNil(t, metricA)
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t,
|
||||
map[string]interface{}{
|
||||
"response_time": int64(20821),
|
||||
"metric": float64(10890.645),
|
||||
},
|
||||
metricA.Fields())
|
||||
assert.Equal(t, map[string]string{}, metricA.Tags())
|
||||
assert.Equal(t, time.Unix(1466004605, 0), metricA.Time())
|
||||
}
|
||||
|
||||
func TestParseEpochErrors(t *testing.T) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{MYAPP}"},
|
||||
CustomPatterns: `
|
||||
MYAPP %{WORD:ts:ts-epoch} response_time=%{POSINT:response_time:int} mymetric=%{NUMBER:metric:float}
|
||||
`,
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
|
||||
_, err := p.ParseLine(`foobar response_time=20821 mymetric=10890.645`)
|
||||
assert.NoError(t, err)
|
||||
|
||||
p = &Parser{
|
||||
Patterns: []string{"%{MYAPP}"},
|
||||
CustomPatterns: `
|
||||
MYAPP %{WORD:ts:ts-epochnano} response_time=%{POSINT:response_time:int} mymetric=%{NUMBER:metric:float}
|
||||
`,
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
|
||||
_, err = p.ParseLine(`foobar response_time=20821 mymetric=10890.645`)
|
||||
assert.NoError(t, err)
|
||||
}
|
||||
|
||||
func TestCompileFileAndParse(t *testing.T) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{TEST_LOG_A}", "%{TEST_LOG_B}"},
|
||||
CustomPatternFiles: []string{"./testdata/test-patterns"},
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
|
||||
metricA, err := p.ParseLine(`[04/Jun/2016:12:41:45 +0100] 1.25 200 192.168.1.1 5.432µs 101`)
|
||||
require.NotNil(t, metricA)
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t,
|
||||
map[string]interface{}{
|
||||
"clientip": "192.168.1.1",
|
||||
"myfloat": float64(1.25),
|
||||
"response_time": int64(5432),
|
||||
"myint": int64(101),
|
||||
},
|
||||
metricA.Fields())
|
||||
assert.Equal(t, map[string]string{"response_code": "200"}, metricA.Tags())
|
||||
assert.Equal(t,
|
||||
time.Date(2016, time.June, 4, 12, 41, 45, 0, time.FixedZone("foo", 60*60)).Nanosecond(),
|
||||
metricA.Time().Nanosecond())
|
||||
|
||||
metricB, err := p.ParseLine(`[04/06/2016--12:41:45] 1.25 mystring dropme nomodifier`)
|
||||
require.NotNil(t, metricB)
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t,
|
||||
map[string]interface{}{
|
||||
"myfloat": 1.25,
|
||||
"mystring": "mystring",
|
||||
"nomodifier": "nomodifier",
|
||||
},
|
||||
metricB.Fields())
|
||||
assert.Equal(t, map[string]string{}, metricB.Tags())
|
||||
assert.Equal(t,
|
||||
time.Date(2016, time.June, 4, 12, 41, 45, 0, time.FixedZone("foo", 60*60)).Nanosecond(),
|
||||
metricB.Time().Nanosecond())
|
||||
}
|
||||
|
||||
func TestCompileNoModifiersAndParse(t *testing.T) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{TEST_LOG_C}"},
|
||||
CustomPatterns: `
|
||||
DURATION %{NUMBER}[nuµm]?s
|
||||
TEST_LOG_C %{NUMBER:myfloat} %{NUMBER} %{IPORHOST:clientip} %{DURATION:rt}
|
||||
`,
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
|
||||
metricA, err := p.ParseLine(`1.25 200 192.168.1.1 5.432µs`)
|
||||
require.NotNil(t, metricA)
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t,
|
||||
map[string]interface{}{
|
||||
"clientip": "192.168.1.1",
|
||||
"myfloat": "1.25",
|
||||
"rt": "5.432µs",
|
||||
},
|
||||
metricA.Fields())
|
||||
assert.Equal(t, map[string]string{}, metricA.Tags())
|
||||
}
|
||||
|
||||
func TestCompileNoNamesAndParse(t *testing.T) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{TEST_LOG_C}"},
|
||||
CustomPatterns: `
|
||||
DURATION %{NUMBER}[nuµm]?s
|
||||
TEST_LOG_C %{NUMBER} %{NUMBER} %{IPORHOST} %{DURATION}
|
||||
`,
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
|
||||
metricA, err := p.ParseLine(`1.25 200 192.168.1.1 5.432µs`)
|
||||
require.Nil(t, metricA)
|
||||
assert.NoError(t, err)
|
||||
}
|
||||
|
||||
func TestParseNoMatch(t *testing.T) {
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{TEST_LOG_A}", "%{TEST_LOG_B}"},
|
||||
CustomPatternFiles: []string{"./testdata/test-patterns"},
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
|
||||
metricA, err := p.ParseLine(`[04/Jun/2016:12:41:45 +0100] notnumber 200 192.168.1.1 5.432µs 101`)
|
||||
assert.NoError(t, err)
|
||||
assert.Nil(t, metricA)
|
||||
}
|
||||
|
||||
func TestCompileErrors(t *testing.T) {
|
||||
// Compile fails because there are multiple timestamps:
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{TEST_LOG_A}", "%{TEST_LOG_B}"},
|
||||
CustomPatterns: `
|
||||
TEST_LOG_A %{HTTPDATE:ts1:ts-httpd} %{HTTPDATE:ts2:ts-httpd} %{NUMBER:mynum:int}
|
||||
`,
|
||||
}
|
||||
assert.Error(t, p.Compile())
|
||||
|
||||
// Compile fails because file doesn't exist:
|
||||
p = &Parser{
|
||||
Patterns: []string{"%{TEST_LOG_A}", "%{TEST_LOG_B}"},
|
||||
CustomPatternFiles: []string{"/tmp/foo/bar/baz"},
|
||||
}
|
||||
assert.Error(t, p.Compile())
|
||||
}
|
||||
|
||||
func TestParseErrors(t *testing.T) {
|
||||
// Parse fails because the pattern doesn't exist
|
||||
p := &Parser{
|
||||
Patterns: []string{"%{TEST_LOG_B}"},
|
||||
CustomPatterns: `
|
||||
TEST_LOG_A %{HTTPDATE:ts:ts-httpd} %{WORD:myword:int} %{}
|
||||
`,
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
_, err := p.ParseLine(`[04/Jun/2016:12:41:45 +0100] notnumber 200 192.168.1.1 5.432µs 101`)
|
||||
assert.Error(t, err)
|
||||
|
||||
// Parse fails because myword is not an int
|
||||
p = &Parser{
|
||||
Patterns: []string{"%{TEST_LOG_A}"},
|
||||
CustomPatterns: `
|
||||
TEST_LOG_A %{HTTPDATE:ts:ts-httpd} %{WORD:myword:int}
|
||||
`,
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
_, err = p.ParseLine(`04/Jun/2016:12:41:45 +0100 notnumber`)
|
||||
assert.Error(t, err)
|
||||
|
||||
// Parse fails because myword is not a float
|
||||
p = &Parser{
|
||||
Patterns: []string{"%{TEST_LOG_A}"},
|
||||
CustomPatterns: `
|
||||
TEST_LOG_A %{HTTPDATE:ts:ts-httpd} %{WORD:myword:float}
|
||||
`,
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
_, err = p.ParseLine(`04/Jun/2016:12:41:45 +0100 notnumber`)
|
||||
assert.Error(t, err)
|
||||
|
||||
// Parse fails because myword is not a duration
|
||||
p = &Parser{
|
||||
Patterns: []string{"%{TEST_LOG_A}"},
|
||||
CustomPatterns: `
|
||||
TEST_LOG_A %{HTTPDATE:ts:ts-httpd} %{WORD:myword:duration}
|
||||
`,
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
_, err = p.ParseLine(`04/Jun/2016:12:41:45 +0100 notnumber`)
|
||||
assert.Error(t, err)
|
||||
|
||||
// Parse fails because the time layout is wrong.
|
||||
p = &Parser{
|
||||
Patterns: []string{"%{TEST_LOG_A}"},
|
||||
CustomPatterns: `
|
||||
TEST_LOG_A %{HTTPDATE:ts:ts-unix} %{WORD:myword:duration}
|
||||
`,
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
_, err = p.ParseLine(`04/Jun/2016:12:41:45 +0100 notnumber`)
|
||||
assert.Error(t, err)
|
||||
}
|
||||
|
||||
func TestTsModder(t *testing.T) {
|
||||
tsm := &tsModder{}
|
||||
|
||||
reftime := time.Date(2006, time.December, 1, 1, 1, 1, int(time.Millisecond), time.UTC)
|
||||
modt := tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime, modt)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime.Add(time.Microsecond*1), modt)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime.Add(time.Microsecond*2), modt)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime.Add(time.Microsecond*3), modt)
|
||||
|
||||
reftime = time.Date(2006, time.December, 1, 1, 1, 1, int(time.Microsecond), time.UTC)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime, modt)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime.Add(time.Nanosecond*1), modt)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime.Add(time.Nanosecond*2), modt)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime.Add(time.Nanosecond*3), modt)
|
||||
|
||||
reftime = time.Date(2006, time.December, 1, 1, 1, 1, int(time.Microsecond)*999, time.UTC)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime, modt)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime.Add(time.Nanosecond*1), modt)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime.Add(time.Nanosecond*2), modt)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime.Add(time.Nanosecond*3), modt)
|
||||
|
||||
reftime = time.Date(2006, time.December, 1, 1, 1, 1, 0, time.UTC)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime, modt)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime.Add(time.Millisecond*1), modt)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime.Add(time.Millisecond*2), modt)
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime.Add(time.Millisecond*3), modt)
|
||||
|
||||
reftime = time.Time{}
|
||||
modt = tsm.tsMod(reftime)
|
||||
assert.Equal(t, reftime, modt)
|
||||
}
|
||||
|
||||
func TestTsModder_Rollover(t *testing.T) {
|
||||
tsm := &tsModder{}
|
||||
|
||||
reftime := time.Date(2006, time.December, 1, 1, 1, 1, int(time.Millisecond), time.UTC)
|
||||
modt := tsm.tsMod(reftime)
|
||||
for i := 1; i < 1000; i++ {
|
||||
modt = tsm.tsMod(reftime)
|
||||
}
|
||||
assert.Equal(t, reftime.Add(time.Microsecond*999+time.Nanosecond), modt)
|
||||
|
||||
reftime = time.Date(2006, time.December, 1, 1, 1, 1, int(time.Microsecond), time.UTC)
|
||||
modt = tsm.tsMod(reftime)
|
||||
for i := 1; i < 1001; i++ {
|
||||
modt = tsm.tsMod(reftime)
|
||||
}
|
||||
assert.Equal(t, reftime.Add(time.Nanosecond*1000), modt)
|
||||
}
|
||||
@@ -1,80 +0,0 @@
|
||||
package grok
|
||||
|
||||
// THIS SHOULD BE KEPT IN-SYNC WITH patterns/influx-patterns
|
||||
const DEFAULT_PATTERNS = `
|
||||
# Captures are a slightly modified version of logstash "grok" patterns, with
|
||||
# the format %{<capture syntax>[:<semantic name>][:<modifier>]}
|
||||
# By default all named captures are converted into string fields.
|
||||
# Modifiers can be used to convert captures to other types or tags.
|
||||
# Timestamp modifiers can be used to convert captures to the timestamp of the
|
||||
# parsed metric.
|
||||
|
||||
# View logstash grok pattern docs here:
|
||||
# https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html
|
||||
# All default logstash patterns are supported, these can be viewed here:
|
||||
# https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/grok-patterns
|
||||
|
||||
# Available modifiers:
|
||||
# string (default if nothing is specified)
|
||||
# int
|
||||
# float
|
||||
# duration (ie, 5.23ms gets converted to int nanoseconds)
|
||||
# tag (converts the field into a tag)
|
||||
# drop (drops the field completely)
|
||||
# Timestamp modifiers:
|
||||
# ts-ansic ("Mon Jan _2 15:04:05 2006")
|
||||
# ts-unix ("Mon Jan _2 15:04:05 MST 2006")
|
||||
# ts-ruby ("Mon Jan 02 15:04:05 -0700 2006")
|
||||
# ts-rfc822 ("02 Jan 06 15:04 MST")
|
||||
# ts-rfc822z ("02 Jan 06 15:04 -0700")
|
||||
# ts-rfc850 ("Monday, 02-Jan-06 15:04:05 MST")
|
||||
# ts-rfc1123 ("Mon, 02 Jan 2006 15:04:05 MST")
|
||||
# ts-rfc1123z ("Mon, 02 Jan 2006 15:04:05 -0700")
|
||||
# ts-rfc3339 ("2006-01-02T15:04:05Z07:00")
|
||||
# ts-rfc3339nano ("2006-01-02T15:04:05.999999999Z07:00")
|
||||
# ts-httpd ("02/Jan/2006:15:04:05 -0700")
|
||||
# ts-epoch (seconds since unix epoch)
|
||||
# ts-epochnano (nanoseconds since unix epoch)
|
||||
# ts-"CUSTOM"
|
||||
# CUSTOM time layouts must be within quotes and be the representation of the
|
||||
# "reference time", which is Mon Jan 2 15:04:05 -0700 MST 2006
|
||||
# See https://golang.org/pkg/time/#Parse for more details.
|
||||
|
||||
# Example log file pattern, example log looks like this:
|
||||
# [04/Jun/2016:12:41:45 +0100] 1.25 200 192.168.1.1 5.432µs
|
||||
# Breakdown of the DURATION pattern below:
|
||||
# NUMBER is a builtin logstash grok pattern matching float & int numbers.
|
||||
# [nuµm]? is a regex specifying 0 or 1 of the characters within brackets.
|
||||
# s is also regex, this pattern must end in "s".
|
||||
# so DURATION will match something like '5.324ms' or '6.1µs' or '10s'
|
||||
DURATION %{NUMBER}[nuµm]?s
|
||||
RESPONSE_CODE %{NUMBER:response_code:tag}
|
||||
RESPONSE_TIME %{DURATION:response_time_ns:duration}
|
||||
EXAMPLE_LOG \[%{HTTPDATE:ts:ts-httpd}\] %{NUMBER:myfloat:float} %{RESPONSE_CODE} %{IPORHOST:clientip} %{RESPONSE_TIME}
|
||||
|
||||
# Wider-ranging username matching vs. logstash built-in %{USER}
|
||||
NGUSERNAME [a-zA-Z\.\@\-\+_%]+
|
||||
NGUSER %{NGUSERNAME}
|
||||
|
||||
##
|
||||
## COMMON LOG PATTERNS
|
||||
##
|
||||
|
||||
# InfluxDB log patterns
|
||||
CLIENT (?:%{IPORHOST}|%{HOSTPORT}|::1)
|
||||
INFLUXDB_HTTPD_LOG \[httpd\] %{COMBINED_LOG_FORMAT} %{UUID:uuid:drop} %{NUMBER:response_time_us:int}
|
||||
|
||||
# apache & nginx logs, this is also known as the "common log format"
|
||||
# see https://en.wikipedia.org/wiki/Common_Log_Format
|
||||
COMMON_LOG_FORMAT %{CLIENT:client_ip} %{NGUSER:ident} %{NGUSER:auth} \[%{HTTPDATE:ts:ts-httpd}\] "(?:%{WORD:verb:tag} %{NOTSPACE:request}(?: HTTP/%{NUMBER:http_version:float})?|%{DATA})" %{NUMBER:resp_code:int} (?:%{NUMBER:resp_bytes:int}|-)
|
||||
|
||||
# Combined log format is the same as the common log format but with the addition
|
||||
# of two quoted strings at the end for "referrer" and "agent"
|
||||
# See Examples at http://httpd.apache.org/docs/current/mod/mod_log_config.html
|
||||
COMBINED_LOG_FORMAT %{COMMON_LOG_FORMAT} %{QS:referrer} %{QS:agent}
|
||||
|
||||
# HTTPD log formats
|
||||
HTTPD20_ERRORLOG \[%{HTTPDERROR_DATE:timestamp}\] \[%{LOGLEVEL:loglevel:tag}\] (?:\[client %{IPORHOST:clientip}\] ){0,1}%{GREEDYDATA:errormsg}
|
||||
HTTPD24_ERRORLOG \[%{HTTPDERROR_DATE:timestamp}\] \[%{WORD:module}:%{LOGLEVEL:loglevel:tag}\] \[pid %{POSINT:pid:int}:tid %{NUMBER:tid:int}\]( \(%{POSINT:proxy_errorcode:int}\)%{DATA:proxy_errormessage}:)?( \[client %{IPORHOST:client}:%{POSINT:clientport}\])? %{DATA:errorcode}: %{GREEDYDATA:message}
|
||||
HTTPD_ERRORLOG %{HTTPD20_ERRORLOG}|%{HTTPD24_ERRORLOG}
|
||||
`
|
||||
@@ -1,75 +0,0 @@
|
||||
# Captures are a slightly modified version of logstash "grok" patterns, with
|
||||
# the format %{<capture syntax>[:<semantic name>][:<modifier>]}
|
||||
# By default all named captures are converted into string fields.
|
||||
# Modifiers can be used to convert captures to other types or tags.
|
||||
# Timestamp modifiers can be used to convert captures to the timestamp of the
|
||||
# parsed metric.
|
||||
|
||||
# View logstash grok pattern docs here:
|
||||
# https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html
|
||||
# All default logstash patterns are supported, these can be viewed here:
|
||||
# https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/grok-patterns
|
||||
|
||||
# Available modifiers:
|
||||
# string (default if nothing is specified)
|
||||
# int
|
||||
# float
|
||||
# duration (ie, 5.23ms gets converted to int nanoseconds)
|
||||
# tag (converts the field into a tag)
|
||||
# drop (drops the field completely)
|
||||
# Timestamp modifiers:
|
||||
# ts-ansic ("Mon Jan _2 15:04:05 2006")
|
||||
# ts-unix ("Mon Jan _2 15:04:05 MST 2006")
|
||||
# ts-ruby ("Mon Jan 02 15:04:05 -0700 2006")
|
||||
# ts-rfc822 ("02 Jan 06 15:04 MST")
|
||||
# ts-rfc822z ("02 Jan 06 15:04 -0700")
|
||||
# ts-rfc850 ("Monday, 02-Jan-06 15:04:05 MST")
|
||||
# ts-rfc1123 ("Mon, 02 Jan 2006 15:04:05 MST")
|
||||
# ts-rfc1123z ("Mon, 02 Jan 2006 15:04:05 -0700")
|
||||
# ts-rfc3339 ("2006-01-02T15:04:05Z07:00")
|
||||
# ts-rfc3339nano ("2006-01-02T15:04:05.999999999Z07:00")
|
||||
# ts-httpd ("02/Jan/2006:15:04:05 -0700")
|
||||
# ts-epoch (seconds since unix epoch)
|
||||
# ts-epochnano (nanoseconds since unix epoch)
|
||||
# ts-"CUSTOM"
|
||||
# CUSTOM time layouts must be within quotes and be the representation of the
|
||||
# "reference time", which is Mon Jan 2 15:04:05 -0700 MST 2006
|
||||
# See https://golang.org/pkg/time/#Parse for more details.
|
||||
|
||||
# Example log file pattern, example log looks like this:
|
||||
# [04/Jun/2016:12:41:45 +0100] 1.25 200 192.168.1.1 5.432µs
|
||||
# Breakdown of the DURATION pattern below:
|
||||
# NUMBER is a builtin logstash grok pattern matching float & int numbers.
|
||||
# [nuµm]? is a regex specifying 0 or 1 of the characters within brackets.
|
||||
# s is also regex, this pattern must end in "s".
|
||||
# so DURATION will match something like '5.324ms' or '6.1µs' or '10s'
|
||||
DURATION %{NUMBER}[nuµm]?s
|
||||
RESPONSE_CODE %{NUMBER:response_code:tag}
|
||||
RESPONSE_TIME %{DURATION:response_time_ns:duration}
|
||||
EXAMPLE_LOG \[%{HTTPDATE:ts:ts-httpd}\] %{NUMBER:myfloat:float} %{RESPONSE_CODE} %{IPORHOST:clientip} %{RESPONSE_TIME}
|
||||
|
||||
# Wider-ranging username matching vs. logstash built-in %{USER}
|
||||
NGUSERNAME [a-zA-Z\.\@\-\+_%]+
|
||||
NGUSER %{NGUSERNAME}
|
||||
|
||||
##
|
||||
## COMMON LOG PATTERNS
|
||||
##
|
||||
|
||||
# InfluxDB log patterns
|
||||
CLIENT (?:%{IPORHOST}|%{HOSTPORT}|::1)
|
||||
INFLUXDB_HTTPD_LOG \[httpd\] %{COMBINED_LOG_FORMAT} %{UUID:uuid:drop} %{NUMBER:response_time_us:int}
|
||||
|
||||
# apache & nginx logs, this is also known as the "common log format"
|
||||
# see https://en.wikipedia.org/wiki/Common_Log_Format
|
||||
COMMON_LOG_FORMAT %{CLIENT:client_ip} %{NGUSER:ident} %{NGUSER:auth} \[%{HTTPDATE:ts:ts-httpd}\] "(?:%{WORD:verb:tag} %{NOTSPACE:request}(?: HTTP/%{NUMBER:http_version:float})?|%{DATA})" %{NUMBER:resp_code:int} (?:%{NUMBER:resp_bytes:int}|-)
|
||||
|
||||
# Combined log format is the same as the common log format but with the addition
|
||||
# of two quoted strings at the end for "referrer" and "agent"
|
||||
# See Examples at http://httpd.apache.org/docs/current/mod/mod_log_config.html
|
||||
COMBINED_LOG_FORMAT %{COMMON_LOG_FORMAT} %{QS:referrer} %{QS:agent}
|
||||
|
||||
# HTTPD log formats
|
||||
HTTPD20_ERRORLOG \[%{HTTPDERROR_DATE:timestamp}\] \[%{LOGLEVEL:loglevel:tag}\] (?:\[client %{IPORHOST:clientip}\] ){0,1}%{GREEDYDATA:errormsg}
|
||||
HTTPD24_ERRORLOG \[%{HTTPDERROR_DATE:timestamp}\] \[%{WORD:module}:%{LOGLEVEL:loglevel:tag}\] \[pid %{POSINT:pid:int}:tid %{NUMBER:tid:int}\]( \(%{POSINT:proxy_errorcode:int}\)%{DATA:proxy_errormessage}:)?( \[client %{IPORHOST:client}:%{POSINT:clientport}\])? %{DATA:errorcode}: %{GREEDYDATA:message}
|
||||
HTTPD_ERRORLOG %{HTTPD20_ERRORLOG}|%{HTTPD24_ERRORLOG}
|
||||
@@ -1,14 +0,0 @@
|
||||
# Test A log line:
|
||||
# [04/Jun/2016:12:41:45 +0100] 1.25 200 192.168.1.1 5.432µs 101
|
||||
DURATION %{NUMBER}[nuµm]?s
|
||||
RESPONSE_CODE %{NUMBER:response_code:tag}
|
||||
RESPONSE_TIME %{DURATION:response_time:duration}
|
||||
TEST_LOG_A \[%{HTTPDATE:timestamp:ts-httpd}\] %{NUMBER:myfloat:float} %{RESPONSE_CODE} %{IPORHOST:clientip} %{RESPONSE_TIME} %{NUMBER:myint:int}
|
||||
|
||||
# Test B log line:
|
||||
# [04/06/2016--12:41:45] 1.25 mystring dropme nomodifier
|
||||
TEST_TIMESTAMP %{MONTHDAY}/%{MONTHNUM}/%{YEAR}--%{TIME}
|
||||
TEST_LOG_B \[%{TEST_TIMESTAMP:timestamp:ts-"02/01/2006--15:04:05"}\] %{NUMBER:myfloat:float} %{WORD:mystring:string} %{WORD:dropme:drop} %{WORD:nomodifier}
|
||||
|
||||
TEST_TIMESTAMP %{MONTHDAY}/%{MONTHNUM}/%{YEAR}--%{TIME}
|
||||
TEST_LOG_BAD \[%{TEST_TIMESTAMP:timestamp:ts-"02/01/2006--15:04:05"}\] %{NUMBER:myfloat:float} %{WORD:mystring:int} %{WORD:dropme:drop} %{WORD:nomodifier}
|
||||
@@ -1 +0,0 @@
|
||||
[04/Jun/2016:12:41:45 +0100] 1.25 200 192.168.1.1 5.432µs 101
|
||||
@@ -1 +0,0 @@
|
||||
[04/06/2016--12:41:45] 1.25 mystring dropme nomodifier
|
||||
@@ -1,228 +0,0 @@
|
||||
package logparser
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"log"
|
||||
"reflect"
|
||||
"sync"
|
||||
|
||||
"github.com/hpcloud/tail"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal/globpath"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
|
||||
// Parsers
|
||||
"github.com/influxdata/telegraf/plugins/inputs/logparser/grok"
|
||||
)
|
||||
|
||||
type LogParser interface {
|
||||
ParseLine(line string) (telegraf.Metric, error)
|
||||
Compile() error
|
||||
}
|
||||
|
||||
type LogParserPlugin struct {
|
||||
Files []string
|
||||
FromBeginning bool
|
||||
|
||||
tailers []*tail.Tail
|
||||
lines chan string
|
||||
done chan struct{}
|
||||
wg sync.WaitGroup
|
||||
acc telegraf.Accumulator
|
||||
parsers []LogParser
|
||||
|
||||
sync.Mutex
|
||||
|
||||
GrokParser *grok.Parser `toml:"grok"`
|
||||
}
|
||||
|
||||
const sampleConfig = `
|
||||
## Log files to parse.
|
||||
## These accept standard unix glob matching rules, but with the addition of
|
||||
## ** as a "super asterisk". ie:
|
||||
## /var/log/**.log -> recursively find all .log files in /var/log
|
||||
## /var/log/*/*.log -> find all .log files with a parent dir in /var/log
|
||||
## /var/log/apache.log -> only tail the apache log file
|
||||
files = ["/var/log/influxdb/influxdb.log"]
|
||||
## Read file from beginning.
|
||||
from_beginning = false
|
||||
|
||||
## Parse logstash-style "grok" patterns:
|
||||
## Telegraf built-in parsing patterns: https://goo.gl/dkay10
|
||||
[inputs.logparser.grok]
|
||||
## This is a list of patterns to check the given log file(s) for.
|
||||
## Note that adding patterns here increases processing time. The most
|
||||
## efficient configuration is to have one pattern per logparser.
|
||||
## Other common built-in patterns are:
|
||||
## %{COMMON_LOG_FORMAT} (plain apache & nginx access logs)
|
||||
## %{COMBINED_LOG_FORMAT} (access logs + referrer & agent)
|
||||
patterns = ["%{INFLUXDB_HTTPD_LOG}"]
|
||||
## Full path(s) to custom pattern files.
|
||||
custom_pattern_files = []
|
||||
## Custom patterns can also be defined here. Put one pattern per line.
|
||||
custom_patterns = '''
|
||||
'''
|
||||
`
|
||||
|
||||
func (l *LogParserPlugin) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
|
||||
func (l *LogParserPlugin) Description() string {
|
||||
return "Stream and parse log file(s)."
|
||||
}
|
||||
|
||||
func (l *LogParserPlugin) Gather(acc telegraf.Accumulator) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (l *LogParserPlugin) Start(acc telegraf.Accumulator) error {
|
||||
l.Lock()
|
||||
defer l.Unlock()
|
||||
|
||||
l.acc = acc
|
||||
l.lines = make(chan string, 1000)
|
||||
l.done = make(chan struct{})
|
||||
|
||||
// Looks for fields which implement LogParser interface
|
||||
l.parsers = []LogParser{}
|
||||
s := reflect.ValueOf(l).Elem()
|
||||
for i := 0; i < s.NumField(); i++ {
|
||||
f := s.Field(i)
|
||||
|
||||
if !f.CanInterface() {
|
||||
continue
|
||||
}
|
||||
|
||||
if lpPlugin, ok := f.Interface().(LogParser); ok {
|
||||
if reflect.ValueOf(lpPlugin).IsNil() {
|
||||
continue
|
||||
}
|
||||
l.parsers = append(l.parsers, lpPlugin)
|
||||
}
|
||||
}
|
||||
|
||||
if len(l.parsers) == 0 {
|
||||
return fmt.Errorf("ERROR: logparser input plugin: no parser defined.")
|
||||
}
|
||||
|
||||
// compile log parser patterns:
|
||||
for _, parser := range l.parsers {
|
||||
if err := parser.Compile(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
var seek tail.SeekInfo
|
||||
if !l.FromBeginning {
|
||||
seek.Whence = 2
|
||||
seek.Offset = 0
|
||||
}
|
||||
|
||||
l.wg.Add(1)
|
||||
go l.parser()
|
||||
|
||||
var errS string
|
||||
// Create a "tailer" for each file
|
||||
for _, filepath := range l.Files {
|
||||
g, err := globpath.Compile(filepath)
|
||||
if err != nil {
|
||||
log.Printf("ERROR Glob %s failed to compile, %s", filepath, err)
|
||||
}
|
||||
for file, _ := range g.Match() {
|
||||
tailer, err := tail.TailFile(file,
|
||||
tail.Config{
|
||||
ReOpen: true,
|
||||
Follow: true,
|
||||
Location: &seek,
|
||||
})
|
||||
if err != nil {
|
||||
errS += err.Error() + " "
|
||||
continue
|
||||
}
|
||||
// create a goroutine for each "tailer"
|
||||
l.wg.Add(1)
|
||||
go l.receiver(tailer)
|
||||
l.tailers = append(l.tailers, tailer)
|
||||
}
|
||||
}
|
||||
|
||||
if errS != "" {
|
||||
return fmt.Errorf(errS)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// receiver is launched as a goroutine to continuously watch a tailed logfile
|
||||
// for changes and send any log lines down the l.lines channel.
|
||||
func (l *LogParserPlugin) receiver(tailer *tail.Tail) {
|
||||
defer l.wg.Done()
|
||||
|
||||
var line *tail.Line
|
||||
for line = range tailer.Lines {
|
||||
if line.Err != nil {
|
||||
log.Printf("ERROR tailing file %s, Error: %s\n",
|
||||
tailer.Filename, line.Err)
|
||||
continue
|
||||
}
|
||||
|
||||
select {
|
||||
case <-l.done:
|
||||
case l.lines <- line.Text:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// parser is launched as a goroutine to watch the l.lines channel.
|
||||
// when a line is available, parser parses it and adds the metric(s) to the
|
||||
// accumulator.
|
||||
func (l *LogParserPlugin) parser() {
|
||||
defer l.wg.Done()
|
||||
|
||||
var m telegraf.Metric
|
||||
var err error
|
||||
var line string
|
||||
for {
|
||||
select {
|
||||
case <-l.done:
|
||||
return
|
||||
case line = <-l.lines:
|
||||
if line == "" || line == "\n" {
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
for _, parser := range l.parsers {
|
||||
m, err = parser.ParseLine(line)
|
||||
if err == nil {
|
||||
if m != nil {
|
||||
l.acc.AddFields(m.Name(), m.Fields(), m.Tags(), m.Time())
|
||||
}
|
||||
} else {
|
||||
log.Printf("Malformed log line in [%s], Error: %s\n", line, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (l *LogParserPlugin) Stop() {
|
||||
l.Lock()
|
||||
defer l.Unlock()
|
||||
|
||||
for _, t := range l.tailers {
|
||||
err := t.Stop()
|
||||
if err != nil {
|
||||
log.Printf("ERROR stopping tail on file %s\n", t.Filename)
|
||||
}
|
||||
t.Cleanup()
|
||||
}
|
||||
close(l.done)
|
||||
l.wg.Wait()
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("logparser", func() telegraf.Input {
|
||||
return &LogParserPlugin{}
|
||||
})
|
||||
}
|
||||
@@ -1,116 +0,0 @@
|
||||
package logparser
|
||||
|
||||
import (
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
|
||||
"github.com/influxdata/telegraf/plugins/inputs/logparser/grok"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestStartNoParsers(t *testing.T) {
|
||||
logparser := &LogParserPlugin{
|
||||
FromBeginning: true,
|
||||
Files: []string{"grok/testdata/*.log"},
|
||||
}
|
||||
|
||||
acc := testutil.Accumulator{}
|
||||
assert.Error(t, logparser.Start(&acc))
|
||||
}
|
||||
|
||||
func TestGrokParseLogFilesNonExistPattern(t *testing.T) {
|
||||
thisdir := getCurrentDir()
|
||||
p := &grok.Parser{
|
||||
Patterns: []string{"%{FOOBAR}"},
|
||||
CustomPatternFiles: []string{thisdir + "grok/testdata/test-patterns"},
|
||||
}
|
||||
|
||||
logparser := &LogParserPlugin{
|
||||
FromBeginning: true,
|
||||
Files: []string{thisdir + "grok/testdata/*.log"},
|
||||
GrokParser: p,
|
||||
}
|
||||
|
||||
acc := testutil.Accumulator{}
|
||||
assert.NoError(t, logparser.Start(&acc))
|
||||
|
||||
time.Sleep(time.Millisecond * 500)
|
||||
logparser.Stop()
|
||||
}
|
||||
|
||||
func TestGrokParseLogFiles(t *testing.T) {
|
||||
thisdir := getCurrentDir()
|
||||
p := &grok.Parser{
|
||||
Patterns: []string{"%{TEST_LOG_A}", "%{TEST_LOG_B}"},
|
||||
CustomPatternFiles: []string{thisdir + "grok/testdata/test-patterns"},
|
||||
}
|
||||
|
||||
logparser := &LogParserPlugin{
|
||||
FromBeginning: true,
|
||||
Files: []string{thisdir + "grok/testdata/*.log"},
|
||||
GrokParser: p,
|
||||
}
|
||||
|
||||
acc := testutil.Accumulator{}
|
||||
assert.NoError(t, logparser.Start(&acc))
|
||||
|
||||
time.Sleep(time.Millisecond * 500)
|
||||
logparser.Stop()
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "logparser_grok",
|
||||
map[string]interface{}{
|
||||
"clientip": "192.168.1.1",
|
||||
"myfloat": float64(1.25),
|
||||
"response_time": int64(5432),
|
||||
"myint": int64(101),
|
||||
},
|
||||
map[string]string{"response_code": "200"})
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "logparser_grok",
|
||||
map[string]interface{}{
|
||||
"myfloat": 1.25,
|
||||
"mystring": "mystring",
|
||||
"nomodifier": "nomodifier",
|
||||
},
|
||||
map[string]string{})
|
||||
}
|
||||
|
||||
func TestGrokParseLogFilesOneBad(t *testing.T) {
|
||||
thisdir := getCurrentDir()
|
||||
p := &grok.Parser{
|
||||
Patterns: []string{"%{TEST_LOG_A}", "%{TEST_LOG_BAD}"},
|
||||
CustomPatternFiles: []string{thisdir + "grok/testdata/test-patterns"},
|
||||
}
|
||||
assert.NoError(t, p.Compile())
|
||||
|
||||
logparser := &LogParserPlugin{
|
||||
FromBeginning: true,
|
||||
Files: []string{thisdir + "grok/testdata/*.log"},
|
||||
GrokParser: p,
|
||||
}
|
||||
|
||||
acc := testutil.Accumulator{}
|
||||
assert.NoError(t, logparser.Start(&acc))
|
||||
|
||||
time.Sleep(time.Millisecond * 500)
|
||||
logparser.Stop()
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "logparser_grok",
|
||||
map[string]interface{}{
|
||||
"clientip": "192.168.1.1",
|
||||
"myfloat": float64(1.25),
|
||||
"response_time": int64(5432),
|
||||
"myint": int64(101),
|
||||
},
|
||||
map[string]string{"response_code": "200"})
|
||||
}
|
||||
|
||||
func getCurrentDir() string {
|
||||
_, filename, _, _ := runtime.Caller(1)
|
||||
return strings.Replace(filename, "logparser_test.go", "", 1)
|
||||
}
|
||||
@@ -33,7 +33,7 @@ func (s *Server) gatherData(acc telegraf.Accumulator) error {
|
||||
result_repl := &ReplSetStatus{}
|
||||
err = s.Session.DB("admin").Run(bson.D{{"replSetGetStatus", 1}}, result_repl)
|
||||
if err != nil {
|
||||
log.Println("Not gathering replica set status, member not in replica set (" + err.Error() + ")")
|
||||
log.Println("Not gathering replica set status, member not in replica set")
|
||||
}
|
||||
|
||||
jumbo_chunks, _ := s.Session.DB("config").C("chunks").Find(bson.M{"jumbo": true}).Count()
|
||||
|
||||
@@ -78,9 +78,9 @@ type ReplSetStatus struct {
|
||||
|
||||
// ReplSetMember stores information related to a replica set member
|
||||
type ReplSetMember struct {
|
||||
Name string `bson:"name"`
|
||||
State int64 `bson:"state"`
|
||||
OptimeDate *bson.MongoTimestamp `bson:"optimeDate"`
|
||||
Name string `bson:"name"`
|
||||
State int64 `bson:"state"`
|
||||
Optime *bson.MongoTimestamp `bson:"optime"`
|
||||
}
|
||||
|
||||
// WiredTiger stores information related to the WiredTiger storage engine.
|
||||
@@ -663,9 +663,9 @@ func NewStatLine(oldMongo, newMongo MongoStatus, key string, all bool, sampleSec
|
||||
}
|
||||
}
|
||||
|
||||
if me.OptimeDate != nil && master.OptimeDate != nil && me.State == 2 {
|
||||
if me.Optime != nil && master.Optime != nil && me.State == 2 {
|
||||
// MongoTimestamp type is int64 where the first 32bits are the unix timestamp
|
||||
lag := int64(*master.OptimeDate>>32 - *me.OptimeDate>>32)
|
||||
lag := int64(*master.Optime>>32 - *me.Optime>>32)
|
||||
if lag < 0 {
|
||||
returnVal.ReplLag = 0
|
||||
} else {
|
||||
|
||||
@@ -53,13 +53,13 @@ This plugin gathers the statistic data from MySQL server
|
||||
## gather metrics from SHOW BINARY LOGS command output
|
||||
gather_binary_logs = false
|
||||
#
|
||||
## gather metrics from PERFORMANCE_SCHEMA.TABLE_IO_WAITS_SUMMARY_BY_TABLE
|
||||
## gather metrics from PERFORMANCE_SCHEMA.TABLE_IO_WAITS_SUMMART_BY_TABLE
|
||||
gather_table_io_waits = false
|
||||
#
|
||||
## gather metrics from PERFORMANCE_SCHEMA.TABLE_LOCK_WAITS
|
||||
gather_table_lock_waits = false
|
||||
#
|
||||
## gather metrics from PERFORMANCE_SCHEMA.TABLE_IO_WAITS_SUMMARY_BY_INDEX_USAGE
|
||||
## gather metrics from PERFORMANCE_SCHEMA.TABLE_IO_WAITS_SUMMART_BY_INDEX_USAGE
|
||||
gather_index_io_waits = false
|
||||
#
|
||||
## gather metrics from PERFORMANCE_SCHEMA.EVENT_WAITS
|
||||
|
||||
@@ -97,12 +97,11 @@ func (n *Nginx) gatherUrl(addr *url.URL, acc telegraf.Accumulator) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data := strings.Fields(line)
|
||||
data := strings.SplitN(strings.TrimSpace(line), " ", 3)
|
||||
accepts, err := strconv.ParseUint(data[0], 10, 64)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
handled, err := strconv.ParseUint(data[1], 10, 64)
|
||||
if err != nil {
|
||||
return err
|
||||
@@ -117,7 +116,7 @@ func (n *Nginx) gatherUrl(addr *url.URL, acc telegraf.Accumulator) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
data = strings.Fields(line)
|
||||
data = strings.SplitN(strings.TrimSpace(line), " ", 6)
|
||||
reading, err := strconv.ParseUint(data[1], 10, 64)
|
||||
if err != nil {
|
||||
return err
|
||||
|
||||
@@ -13,18 +13,12 @@ import (
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
const nginxSampleResponse = `
|
||||
const sampleResponse = `
|
||||
Active connections: 585
|
||||
server accepts handled requests
|
||||
85340 85340 35085
|
||||
Reading: 4 Writing: 135 Waiting: 446
|
||||
`
|
||||
const tengineSampleResponse = `
|
||||
Active connections: 403
|
||||
server accepts handled requests request_time
|
||||
853 8533 3502 1546565864
|
||||
Reading: 8 Writing: 125 Waiting: 946
|
||||
`
|
||||
|
||||
// Verify that nginx tags are properly parsed based on the server
|
||||
func TestNginxTags(t *testing.T) {
|
||||
@@ -42,9 +36,7 @@ func TestNginxGeneratesMetrics(t *testing.T) {
|
||||
var rsp string
|
||||
|
||||
if r.URL.Path == "/stub_status" {
|
||||
rsp = nginxSampleResponse
|
||||
} else if r.URL.Path == "/tengine_status" {
|
||||
rsp = tengineSampleResponse
|
||||
rsp = sampleResponse
|
||||
} else {
|
||||
panic("Cannot handle request")
|
||||
}
|
||||
@@ -57,20 +49,12 @@ func TestNginxGeneratesMetrics(t *testing.T) {
|
||||
Urls: []string{fmt.Sprintf("%s/stub_status", ts.URL)},
|
||||
}
|
||||
|
||||
nt := &Nginx{
|
||||
Urls: []string{fmt.Sprintf("%s/tengine_status", ts.URL)},
|
||||
}
|
||||
var acc testutil.Accumulator
|
||||
|
||||
var acc_nginx testutil.Accumulator
|
||||
var acc_tengine testutil.Accumulator
|
||||
err := n.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
err_nginx := n.Gather(&acc_nginx)
|
||||
err_tengine := nt.Gather(&acc_tengine)
|
||||
|
||||
require.NoError(t, err_nginx)
|
||||
require.NoError(t, err_tengine)
|
||||
|
||||
fields_nginx := map[string]interface{}{
|
||||
fields := map[string]interface{}{
|
||||
"active": uint64(585),
|
||||
"accepts": uint64(85340),
|
||||
"handled": uint64(85340),
|
||||
@@ -79,17 +63,6 @@ func TestNginxGeneratesMetrics(t *testing.T) {
|
||||
"writing": uint64(135),
|
||||
"waiting": uint64(446),
|
||||
}
|
||||
|
||||
fields_tengine := map[string]interface{}{
|
||||
"active": uint64(403),
|
||||
"accepts": uint64(853),
|
||||
"handled": uint64(8533),
|
||||
"requests": uint64(3502),
|
||||
"reading": uint64(8),
|
||||
"writing": uint64(125),
|
||||
"waiting": uint64(946),
|
||||
}
|
||||
|
||||
addr, err := url.Parse(ts.URL)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
@@ -108,6 +81,5 @@ func TestNginxGeneratesMetrics(t *testing.T) {
|
||||
}
|
||||
|
||||
tags := map[string]string{"server": host, "port": port}
|
||||
acc_nginx.AssertContainsTaggedFields(t, "nginx", fields_nginx, tags)
|
||||
acc_tengine.AssertContainsTaggedFields(t, "nginx", fields_tengine, tags)
|
||||
acc.AssertContainsTaggedFields(t, "nginx", fields, tags)
|
||||
}
|
||||
|
||||
@@ -28,7 +28,7 @@ type Ping struct {
|
||||
// Number of pings to send (ping -c <COUNT>)
|
||||
Count int
|
||||
|
||||
// Ping timeout, in seconds. 0 means no timeout (ping -W <TIMEOUT>)
|
||||
// Ping timeout, in seconds. 0 means no timeout (ping -t <TIMEOUT>)
|
||||
Timeout float64
|
||||
|
||||
// Interface to send ping from (ping -I <INTERFACE>)
|
||||
@@ -55,7 +55,7 @@ const sampleConfig = `
|
||||
count = 1 # required
|
||||
## interval, in s, at which to ping. 0 == default (ping -i <PING_INTERVAL>)
|
||||
ping_interval = 0.0
|
||||
## per-ping timeout, in s. 0 == no timeout (ping -W <TIMEOUT>)
|
||||
## ping timeout, in s. 0 == no timeout (ping -W <TIMEOUT>)
|
||||
timeout = 1.0
|
||||
## interface to send ping from (ping -I <INTERFACE>)
|
||||
interface = ""
|
||||
@@ -76,8 +76,7 @@ func (p *Ping) Gather(acc telegraf.Accumulator) error {
|
||||
go func(u string) {
|
||||
defer wg.Done()
|
||||
args := p.args(u)
|
||||
totalTimeout := float64(p.Count)*p.Timeout + float64(p.Count-1)*p.PingInterval
|
||||
out, err := p.pingHost(totalTimeout, args...)
|
||||
out, err := p.pingHost(p.Timeout, args...)
|
||||
if err != nil {
|
||||
// Combine go err + stderr output
|
||||
errorChannel <- errors.New(
|
||||
@@ -139,8 +138,8 @@ func (p *Ping) args(url string) []string {
|
||||
}
|
||||
if p.Timeout > 0 {
|
||||
switch runtime.GOOS {
|
||||
case "darwin":
|
||||
args = append(args, "-W", strconv.FormatFloat(p.Timeout/1000, 'f', 1, 64))
|
||||
case "darwin", "freebsd":
|
||||
args = append(args, "-t", strconv.FormatFloat(p.Timeout, 'f', 1, 64))
|
||||
case "linux":
|
||||
args = append(args, "-W", strconv.FormatFloat(p.Timeout, 'f', 1, 64))
|
||||
default:
|
||||
|
||||
@@ -10,7 +10,6 @@ import (
|
||||
"io"
|
||||
"math"
|
||||
"mime"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
|
||||
@@ -75,13 +74,13 @@ func (p *PrometheusParser) Parse(buf []byte) ([]telegraf.Metric, error) {
|
||||
if mf.GetType() == dto.MetricType_SUMMARY {
|
||||
// summary metric
|
||||
fields = makeQuantiles(m)
|
||||
fields["count"] = float64(m.GetSummary().GetSampleCount())
|
||||
fields["count"] = float64(m.GetHistogram().GetSampleCount())
|
||||
fields["sum"] = float64(m.GetSummary().GetSampleSum())
|
||||
} else if mf.GetType() == dto.MetricType_HISTOGRAM {
|
||||
// historgram metric
|
||||
fields = makeBuckets(m)
|
||||
fields["count"] = float64(m.GetHistogram().GetSampleCount())
|
||||
fields["sum"] = float64(m.GetHistogram().GetSampleSum())
|
||||
fields["sum"] = float64(m.GetSummary().GetSampleSum())
|
||||
|
||||
} else {
|
||||
// standard metric
|
||||
@@ -89,13 +88,7 @@ func (p *PrometheusParser) Parse(buf []byte) ([]telegraf.Metric, error) {
|
||||
}
|
||||
// converting to telegraf metric
|
||||
if len(fields) > 0 {
|
||||
var t time.Time
|
||||
if m.TimestampMs != nil && *m.TimestampMs > 0 {
|
||||
t = time.Unix(0, *m.TimestampMs*1000000)
|
||||
} else {
|
||||
t = time.Now()
|
||||
}
|
||||
metric, err := telegraf.NewMetric(metricName, tags, fields, t)
|
||||
metric, err := telegraf.NewMetric(metricName, tags, fields)
|
||||
if err == nil {
|
||||
metrics = append(metrics, metric)
|
||||
}
|
||||
|
||||
@@ -138,7 +138,7 @@ func TestParseValidPrometheus(t *testing.T) {
|
||||
"0.5": 552048.506,
|
||||
"0.9": 5.876804288e+06,
|
||||
"0.99": 5.876804288e+06,
|
||||
"count": 9.0,
|
||||
"count": 0.0,
|
||||
"sum": 1.8909097205e+07,
|
||||
}, metrics[0].Fields())
|
||||
assert.Equal(t, map[string]string{"handler": "prometheus"}, metrics[0].Tags())
|
||||
@@ -151,7 +151,7 @@ func TestParseValidPrometheus(t *testing.T) {
|
||||
assert.Equal(t, map[string]interface{}{
|
||||
"500000": 2000.0,
|
||||
"count": 2025.0,
|
||||
"sum": 1.02726334e+08,
|
||||
"sum": 0.0,
|
||||
"250000": 1997.0,
|
||||
"2e+06": 2012.0,
|
||||
"4e+06": 2017.0,
|
||||
|
||||
@@ -5,11 +5,9 @@ import (
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal/errchan"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
@@ -104,7 +102,7 @@ type gatherFunc func(r *RabbitMQ, acc telegraf.Accumulator, errChan chan error)
|
||||
var gatherFunctions = []gatherFunc{gatherOverview, gatherNodes, gatherQueues}
|
||||
|
||||
var sampleConfig = `
|
||||
# url = "http://localhost:15672"
|
||||
url = "http://localhost:15672" # required
|
||||
# name = "rmq-server-1" # optional tag
|
||||
# username = "guest"
|
||||
# password = "guest"
|
||||
@@ -131,24 +129,23 @@ func (r *RabbitMQ) Gather(acc telegraf.Accumulator) error {
|
||||
}
|
||||
}
|
||||
|
||||
var wg sync.WaitGroup
|
||||
wg.Add(len(gatherFunctions))
|
||||
errChan := errchan.New(len(gatherFunctions))
|
||||
for _, f := range gatherFunctions {
|
||||
go func(gf gatherFunc) {
|
||||
defer wg.Done()
|
||||
gf(r, acc, errChan.C)
|
||||
}(f)
|
||||
}
|
||||
wg.Wait()
|
||||
var errChan = make(chan error, len(gatherFunctions))
|
||||
|
||||
return errChan.Error()
|
||||
for _, f := range gatherFunctions {
|
||||
go f(r, acc, errChan)
|
||||
}
|
||||
|
||||
for i := 1; i <= len(gatherFunctions); i++ {
|
||||
err := <-errChan
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (r *RabbitMQ) requestJSON(u string, target interface{}) error {
|
||||
if r.URL == "" {
|
||||
r.URL = DefaultURL
|
||||
}
|
||||
u = fmt.Sprintf("%s%s", r.URL, u)
|
||||
|
||||
req, err := http.NewRequest("GET", u, nil)
|
||||
|
||||
@@ -241,14 +241,10 @@ func gatherKeyspaceLine(
|
||||
name string,
|
||||
line string,
|
||||
acc telegraf.Accumulator,
|
||||
global_tags map[string]string,
|
||||
tags map[string]string,
|
||||
) {
|
||||
if strings.Contains(line, "keys=") {
|
||||
fields := make(map[string]interface{})
|
||||
tags := make(map[string]string)
|
||||
for k, v := range global_tags {
|
||||
tags[k] = v
|
||||
}
|
||||
tags["database"] = name
|
||||
dbparts := strings.Split(line, ",")
|
||||
for _, dbp := range dbparts {
|
||||
|
||||
@@ -35,7 +35,6 @@ func TestRedis_ParseMetrics(t *testing.T) {
|
||||
err := gatherInfoOutput(rdr, &acc, tags)
|
||||
require.NoError(t, err)
|
||||
|
||||
tags = map[string]string{"host": "redis.net", "role": "master"}
|
||||
fields := map[string]interface{}{
|
||||
"uptime": uint64(238),
|
||||
"clients": uint64(1),
|
||||
@@ -71,14 +70,13 @@ func TestRedis_ParseMetrics(t *testing.T) {
|
||||
"used_cpu_user_children": float64(0.00),
|
||||
"keyspace_hitrate": float64(0.50),
|
||||
}
|
||||
keyspaceTags := map[string]string{"host": "redis.net", "role": "master", "database": "db0"}
|
||||
keyspaceFields := map[string]interface{}{
|
||||
"avg_ttl": uint64(0),
|
||||
"expires": uint64(0),
|
||||
"keys": uint64(2),
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "redis", fields, tags)
|
||||
acc.AssertContainsTaggedFields(t, "redis_keyspace", keyspaceFields, keyspaceTags)
|
||||
acc.AssertContainsTaggedFields(t, "redis_keyspace", keyspaceFields, tags)
|
||||
}
|
||||
|
||||
const testOutput = `# Server
|
||||
|
||||
@@ -1,47 +0,0 @@
|
||||
# rollbar_webhooks
|
||||
|
||||
This is a Telegraf service plugin that listens for events kicked off by Rollbar Webhooks service and persists data from them into configured outputs. To set up the listener first generate the proper configuration:
|
||||
```sh
|
||||
$ telegraf -sample-config -input-filter rollbar_webhooks -output-filter influxdb > config.conf.new
|
||||
```
|
||||
Change the config file to point to the InfluxDB server you are using and adjust the settings to match your environment. Once that is complete:
|
||||
```sh
|
||||
$ cp config.conf.new /etc/telegraf/telegraf.conf
|
||||
$ sudo service telegraf start
|
||||
```
|
||||
Once the server is running you should configure your Rollbar's Webhooks to point at the `rollbar_webhooks` service. To do this go to `rollbar.com/` and click `Settings > Notifications > Webhook`. In the resulting page set `URL` to `http://<my_ip>:1619`, and click on `Enable Webhook Integration`.
|
||||
|
||||
## Events
|
||||
|
||||
The titles of the following sections are links to the full payloads and details for each event. The body contains what information from the event is persisted. The format is as follows:
|
||||
```
|
||||
# TAGS
|
||||
* 'tagKey' = `tagValue` type
|
||||
# FIELDS
|
||||
* 'fieldKey' = `fieldValue` type
|
||||
```
|
||||
The tag values and field values show the place on the incoming JSON object where the data is sourced from.
|
||||
|
||||
See [webhook doc](https://rollbar.com/docs/webhooks/)
|
||||
|
||||
#### `new_item` event
|
||||
|
||||
**Tags:**
|
||||
* 'event' = `event.event_name` string
|
||||
* 'environment' = `event.data.item.environment` string
|
||||
* 'project_id = `event.data.item.project_id` int
|
||||
* 'language' = `event.data.item.last_occurence.language` string
|
||||
* 'level' = `event.data.item.last_occurence.level` string
|
||||
|
||||
**Fields:**
|
||||
* 'id' = `event.data.item.id` int
|
||||
|
||||
#### `deploy` event
|
||||
|
||||
**Tags:**
|
||||
* 'event' = `event.event_name` string
|
||||
* 'environment' = `event.data.deploy.environment` string
|
||||
* 'project_id = `event.data.deploy.project_id` int
|
||||
|
||||
**Fields:**
|
||||
* 'id' = `event.data.item.id` int
|
||||
@@ -1,119 +0,0 @@
|
||||
package rollbar_webhooks
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"log"
|
||||
"net/http"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/gorilla/mux"
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
func init() {
|
||||
inputs.Add("rollbar_webhooks", func() telegraf.Input { return NewRollbarWebhooks() })
|
||||
}
|
||||
|
||||
type RollbarWebhooks struct {
|
||||
ServiceAddress string
|
||||
// Lock for the struct
|
||||
sync.Mutex
|
||||
// Events buffer to store events between Gather calls
|
||||
events []Event
|
||||
}
|
||||
|
||||
func NewRollbarWebhooks() *RollbarWebhooks {
|
||||
return &RollbarWebhooks{}
|
||||
}
|
||||
|
||||
func (rb *RollbarWebhooks) SampleConfig() string {
|
||||
return `
|
||||
## Address and port to host Webhook listener on
|
||||
service_address = ":1619"
|
||||
`
|
||||
}
|
||||
|
||||
func (rb *RollbarWebhooks) Description() string {
|
||||
return "A Rollbar Webhook Event collector"
|
||||
}
|
||||
|
||||
func (rb *RollbarWebhooks) Gather(acc telegraf.Accumulator) error {
|
||||
rb.Lock()
|
||||
defer rb.Unlock()
|
||||
for _, event := range rb.events {
|
||||
acc.AddFields("rollbar_webhooks", event.Fields(), event.Tags(), time.Now())
|
||||
}
|
||||
rb.events = make([]Event, 0)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (rb *RollbarWebhooks) Listen() {
|
||||
r := mux.NewRouter()
|
||||
r.HandleFunc("/", rb.eventHandler).Methods("POST")
|
||||
err := http.ListenAndServe(fmt.Sprintf("%s", rb.ServiceAddress), r)
|
||||
if err != nil {
|
||||
log.Printf("Error starting server: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func (rb *RollbarWebhooks) Start(_ telegraf.Accumulator) error {
|
||||
go rb.Listen()
|
||||
log.Printf("Started the rollbar_webhooks service on %s\n", rb.ServiceAddress)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (rb *RollbarWebhooks) Stop() {
|
||||
log.Println("Stopping the rbWebhooks service")
|
||||
}
|
||||
|
||||
func (rb *RollbarWebhooks) eventHandler(w http.ResponseWriter, r *http.Request) {
|
||||
defer r.Body.Close()
|
||||
data, err := ioutil.ReadAll(r.Body)
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
dummyEvent := &DummyEvent{}
|
||||
err = json.Unmarshal(data, dummyEvent)
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
|
||||
event, err := NewEvent(dummyEvent, data)
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
return
|
||||
}
|
||||
|
||||
rb.Lock()
|
||||
rb.events = append(rb.events, event)
|
||||
rb.Unlock()
|
||||
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}
|
||||
|
||||
func generateEvent(event Event, data []byte) (Event, error) {
|
||||
err := json.Unmarshal(data, event)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return event, nil
|
||||
}
|
||||
|
||||
func NewEvent(dummyEvent *DummyEvent, data []byte) (Event, error) {
|
||||
switch dummyEvent.EventName {
|
||||
case "new_item":
|
||||
return generateEvent(&NewItem{}, data)
|
||||
case "deploy":
|
||||
return generateEvent(&Deploy{}, data)
|
||||
default:
|
||||
return nil, errors.New("Not implemented type: " + dummyEvent.EventName)
|
||||
}
|
||||
}
|
||||
@@ -1,78 +0,0 @@
|
||||
package rollbar_webhooks
|
||||
|
||||
import "strconv"
|
||||
|
||||
type Event interface {
|
||||
Tags() map[string]string
|
||||
Fields() map[string]interface{}
|
||||
}
|
||||
|
||||
type DummyEvent struct {
|
||||
EventName string `json:"event_name"`
|
||||
}
|
||||
|
||||
type NewItemDataItemLastOccurence struct {
|
||||
Language string `json:"language"`
|
||||
Level string `json:"level"`
|
||||
}
|
||||
|
||||
type NewItemDataItem struct {
|
||||
Id int `json:"id"`
|
||||
Environment string `json:"environment"`
|
||||
ProjectId int `json:"project_id"`
|
||||
LastOccurence NewItemDataItemLastOccurence `json:"last_occurrence"`
|
||||
}
|
||||
|
||||
type NewItemData struct {
|
||||
Item NewItemDataItem `json:"item"`
|
||||
}
|
||||
|
||||
type NewItem struct {
|
||||
EventName string `json:"event_name"`
|
||||
Data NewItemData `json:"data"`
|
||||
}
|
||||
|
||||
func (ni *NewItem) Tags() map[string]string {
|
||||
return map[string]string{
|
||||
"event": ni.EventName,
|
||||
"environment": ni.Data.Item.Environment,
|
||||
"project_id": strconv.Itoa(ni.Data.Item.ProjectId),
|
||||
"language": ni.Data.Item.LastOccurence.Language,
|
||||
"level": ni.Data.Item.LastOccurence.Level,
|
||||
}
|
||||
}
|
||||
|
||||
func (ni *NewItem) Fields() map[string]interface{} {
|
||||
return map[string]interface{}{
|
||||
"id": ni.Data.Item.Id,
|
||||
}
|
||||
}
|
||||
|
||||
type DeployDataDeploy struct {
|
||||
Id int `json:"id"`
|
||||
Environment string `json:"environment"`
|
||||
ProjectId int `json:"project_id"`
|
||||
}
|
||||
|
||||
type DeployData struct {
|
||||
Deploy DeployDataDeploy `json:"deploy"`
|
||||
}
|
||||
|
||||
type Deploy struct {
|
||||
EventName string `json:"event_name"`
|
||||
Data DeployData `json:"data"`
|
||||
}
|
||||
|
||||
func (ni *Deploy) Tags() map[string]string {
|
||||
return map[string]string{
|
||||
"event": ni.EventName,
|
||||
"environment": ni.Data.Deploy.Environment,
|
||||
"project_id": strconv.Itoa(ni.Data.Deploy.ProjectId),
|
||||
}
|
||||
}
|
||||
|
||||
func (ni *Deploy) Fields() map[string]interface{} {
|
||||
return map[string]interface{}{
|
||||
"id": ni.Data.Deploy.Id,
|
||||
}
|
||||
}
|
||||
@@ -1,96 +0,0 @@
|
||||
package rollbar_webhooks
|
||||
|
||||
func NewItemJSON() string {
|
||||
return `
|
||||
{
|
||||
"event_name": "new_item",
|
||||
"data": {
|
||||
"item": {
|
||||
"public_item_id": null,
|
||||
"integrations_data": {},
|
||||
"last_activated_timestamp": 1382655421,
|
||||
"unique_occurrences": null,
|
||||
"id": 272716944,
|
||||
"environment": "production",
|
||||
"title": "testing aobg98wrwe",
|
||||
"last_occurrence_id": 481761639,
|
||||
"last_occurrence_timestamp": 1382655421,
|
||||
"platform": 0,
|
||||
"first_occurrence_timestamp": 1382655421,
|
||||
"project_id": 90,
|
||||
"resolved_in_version": null,
|
||||
"status": 1,
|
||||
"hash": "c595b2ae0af9b397bb6bdafd57104ac4d5f6b382",
|
||||
"last_occurrence": {
|
||||
"body": {
|
||||
"message": {
|
||||
"body": "testing aobg98wrwe"
|
||||
}
|
||||
},
|
||||
"uuid": "d2036647-e0b7-4cad-bc98-934831b9b6d1",
|
||||
"language": "python",
|
||||
"level": "error",
|
||||
"timestamp": 1382655421,
|
||||
"server": {
|
||||
"host": "dev",
|
||||
"argv": [
|
||||
""
|
||||
]
|
||||
},
|
||||
"environment": "production",
|
||||
"framework": "unknown",
|
||||
"notifier": {
|
||||
"version": "0.5.12",
|
||||
"name": "pyrollbar"
|
||||
},
|
||||
"metadata": {
|
||||
"access_token": "",
|
||||
"debug": {
|
||||
"routes": {
|
||||
"start_time": 1382212080401,
|
||||
"counters": {
|
||||
"post_item": 3274122
|
||||
}
|
||||
}
|
||||
},
|
||||
"customer_timestamp": 1382655421,
|
||||
"api_server_hostname": "web6"
|
||||
}
|
||||
},
|
||||
"framework": 0,
|
||||
"total_occurrences": 1,
|
||||
"level": 40,
|
||||
"counter": 4,
|
||||
"first_occurrence_id": 481761639,
|
||||
"activating_occurrence_id": 481761639
|
||||
}
|
||||
}
|
||||
}`
|
||||
}
|
||||
|
||||
func DeployJSON() string {
|
||||
return `
|
||||
{
|
||||
"event_name": "deploy",
|
||||
"data": {
|
||||
"deploy": {
|
||||
"comment": "deploying webs",
|
||||
"user_id": 1,
|
||||
"finish_time": 1382656039,
|
||||
"start_time": 1382656038,
|
||||
"id": 187585,
|
||||
"environment": "production",
|
||||
"project_id": 90,
|
||||
"local_username": "brian",
|
||||
"revision": "e4b9b7db860b2e5ac799f8c06b9498b71ab270bb"
|
||||
}
|
||||
}
|
||||
}`
|
||||
}
|
||||
|
||||
func UnknowJSON() string {
|
||||
return `
|
||||
{
|
||||
"event_name": "roger"
|
||||
}`
|
||||
}
|
||||
@@ -1,74 +0,0 @@
|
||||
package rollbar_webhooks
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
)
|
||||
|
||||
func postWebhooks(rb *RollbarWebhooks, eventBody string) *httptest.ResponseRecorder {
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(eventBody))
|
||||
w := httptest.NewRecorder()
|
||||
w.Code = 500
|
||||
|
||||
rb.eventHandler(w, req)
|
||||
|
||||
return w
|
||||
}
|
||||
|
||||
func TestNewItem(t *testing.T) {
|
||||
var acc testutil.Accumulator
|
||||
rb := NewRollbarWebhooks()
|
||||
resp := postWebhooks(rb, NewItemJSON())
|
||||
if resp.Code != http.StatusOK {
|
||||
t.Errorf("POST new_item returned HTTP status code %v.\nExpected %v", resp.Code, http.StatusOK)
|
||||
}
|
||||
rb.Gather(&acc)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"id": 272716944,
|
||||
}
|
||||
|
||||
tags := map[string]string{
|
||||
"event": "new_item",
|
||||
"environment": "production",
|
||||
"project_id": "90",
|
||||
"language": "python",
|
||||
"level": "error",
|
||||
}
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "rollbar_webhooks", fields, tags)
|
||||
}
|
||||
|
||||
func TestDeploy(t *testing.T) {
|
||||
var acc testutil.Accumulator
|
||||
rb := NewRollbarWebhooks()
|
||||
resp := postWebhooks(rb, DeployJSON())
|
||||
if resp.Code != http.StatusOK {
|
||||
t.Errorf("POST deploy returned HTTP status code %v.\nExpected %v", resp.Code, http.StatusOK)
|
||||
}
|
||||
rb.Gather(&acc)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"id": 187585,
|
||||
}
|
||||
|
||||
tags := map[string]string{
|
||||
"event": "deploy",
|
||||
"environment": "production",
|
||||
"project_id": "90",
|
||||
}
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "rollbar_webhooks", fields, tags)
|
||||
}
|
||||
|
||||
func TestUnknowItem(t *testing.T) {
|
||||
rb := NewRollbarWebhooks()
|
||||
resp := postWebhooks(rb, UnknowJSON())
|
||||
if resp.Code != http.StatusOK {
|
||||
t.Errorf("POST unknow returned HTTP status code %v.\nExpected %v", resp.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
@@ -27,8 +27,7 @@ const (
|
||||
defaultSeparator = "_"
|
||||
)
|
||||
|
||||
var dropwarn = "ERROR: statsd message queue full. " +
|
||||
"We have dropped %d messages so far. " +
|
||||
var dropwarn = "ERROR: Message queue full. Discarding line [%s] " +
|
||||
"You may want to increase allowed_pending_messages in the config\n"
|
||||
|
||||
var prevInstance *Statsd
|
||||
@@ -66,8 +65,6 @@ type Statsd struct {
|
||||
|
||||
sync.Mutex
|
||||
wg sync.WaitGroup
|
||||
// drops tracks the number of dropped metrics.
|
||||
drops int
|
||||
|
||||
// Channel for all incoming statsd packets
|
||||
in chan []byte
|
||||
@@ -294,10 +291,7 @@ func (s *Statsd) udpListen() error {
|
||||
select {
|
||||
case s.in <- bufCopy:
|
||||
default:
|
||||
s.drops++
|
||||
if s.drops == 1 || s.drops%s.AllowedPendingMessages == 0 {
|
||||
log.Printf(dropwarn, s.drops)
|
||||
}
|
||||
log.Printf(dropwarn, string(buf[:n]))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,225 +0,0 @@
|
||||
# Kernel VMStat Input Plugin
|
||||
|
||||
The kernel_vmstat plugin gathers virtual memory statistics
|
||||
by reading /proc/vmstat. For a full list of available fields see the
|
||||
/proc/vmstat section of the [proc man page](http://man7.org/linux/man-pages/man5/proc.5.html).
|
||||
For a better idea of what each field represents, see the
|
||||
[vmstat man page](http://linux.die.net/man/8/vmstat).
|
||||
|
||||
|
||||
```
|
||||
/proc/vmstat
|
||||
kernel/system statistics. Common entries include (from http://www.linuxinsight.com/proc_vmstat.html):
|
||||
|
||||
Number of pages that are dirty, under writeback or unstable:
|
||||
|
||||
nr_dirty 1550
|
||||
nr_writeback 0
|
||||
nr_unstable 0
|
||||
|
||||
Number of pages allocated to page tables, mapped by files or allocated by the kernel slab allocator:
|
||||
|
||||
nr_page_table_pages 699
|
||||
nr_mapped 139596
|
||||
nr_slab 42723
|
||||
|
||||
Number of pageins and pageouts (since the last boot):
|
||||
|
||||
pgpgin 33754195
|
||||
pgpgout 38985992
|
||||
|
||||
Number of swapins and swapouts (since the last boot):
|
||||
|
||||
pswpin 2473
|
||||
pswpout 2995
|
||||
|
||||
Number of page allocations per zone (since the last boot):
|
||||
|
||||
pgalloc_high 0
|
||||
pgalloc_normal 110123213
|
||||
pgalloc_dma32 0
|
||||
pgalloc_dma 415219
|
||||
|
||||
Number of page frees, activations and deactivations (since the last boot):
|
||||
|
||||
pgfree 110549163
|
||||
pgactivate 4509729
|
||||
pgdeactivate 2136215
|
||||
|
||||
Number of minor and major page faults (since the last boot):
|
||||
|
||||
pgfault 80663722
|
||||
pgmajfault 49813
|
||||
|
||||
Number of page refills (per zone, since the last boot):
|
||||
|
||||
pgrefill_high 0
|
||||
pgrefill_normal 5817500
|
||||
pgrefill_dma32 0
|
||||
pgrefill_dma 149176
|
||||
|
||||
Number of page steals (per zone, since the last boot):
|
||||
|
||||
pgsteal_high 0
|
||||
pgsteal_normal 10421346
|
||||
pgsteal_dma32 0
|
||||
pgsteal_dma 142196
|
||||
|
||||
Number of pages scanned by the kswapd daemon (per zone, since the last boot):
|
||||
|
||||
pgscan_kswapd_high 0
|
||||
pgscan_kswapd_normal 10491424
|
||||
pgscan_kswapd_dma32 0
|
||||
pgscan_kswapd_dma 156130
|
||||
|
||||
Number of pages reclaimed directly (per zone, since the last boot):
|
||||
|
||||
pgscan_direct_high 0
|
||||
pgscan_direct_normal 11904
|
||||
pgscan_direct_dma32 0
|
||||
pgscan_direct_dma 225
|
||||
|
||||
Number of pages reclaimed via inode freeing (since the last boot):
|
||||
|
||||
pginodesteal 11
|
||||
|
||||
Number of slab objects scanned (since the last boot):
|
||||
|
||||
slabs_scanned 8926976
|
||||
|
||||
Number of pages reclaimed by kswapd (since the last boot):
|
||||
|
||||
kswapd_steal 10551674
|
||||
|
||||
Number of pages reclaimed by kswapd via inode freeing (since the last boot):
|
||||
|
||||
kswapd_inodesteal 338730
|
||||
|
||||
Number of kswapd's calls to page reclaim (since the last boot):
|
||||
|
||||
pageoutrun 181908
|
||||
|
||||
Number of direct reclaim calls (since the last boot):
|
||||
|
||||
allocstall 160
|
||||
|
||||
Miscellaneous statistics:
|
||||
|
||||
pgrotated 3781
|
||||
nr_bounce 0
|
||||
```
|
||||
|
||||
### Configuration:
|
||||
|
||||
```toml
|
||||
# Get kernel statistics from /proc/vmstat
|
||||
[[inputs.kernel_vmstat]]
|
||||
# no configuration
|
||||
```
|
||||
|
||||
### Measurements & Fields:
|
||||
|
||||
- kernel_vmstat
|
||||
- nr_free_pages (integer, `nr_free_pages`)
|
||||
- nr_inactive_anon (integer, `nr_inactive_anon`)
|
||||
- nr_active_anon (integer, `nr_active_anon`)
|
||||
- nr_inactive_file (integer, `nr_inactive_file`)
|
||||
- nr_active_file (integer, `nr_active_file`)
|
||||
- nr_unevictable (integer, `nr_unevictable`)
|
||||
- nr_mlock (integer, `nr_mlock`)
|
||||
- nr_anon_pages (integer, `nr_anon_pages`)
|
||||
- nr_mapped (integer, `nr_mapped`)
|
||||
- nr_file_pages (integer, `nr_file_pages`)
|
||||
- nr_dirty (integer, `nr_dirty`)
|
||||
- nr_writeback (integer, `nr_writeback`)
|
||||
- nr_slab_reclaimable (integer, `nr_slab_reclaimable`)
|
||||
- nr_slab_unreclaimable (integer, `nr_slab_unreclaimable`)
|
||||
- nr_page_table_pages (integer, `nr_page_table_pages`)
|
||||
- nr_kernel_stack (integer, `nr_kernel_stack`)
|
||||
- nr_unstable (integer, `nr_unstable`)
|
||||
- nr_bounce (integer, `nr_bounce`)
|
||||
- nr_vmscan_write (integer, `nr_vmscan_write`)
|
||||
- nr_writeback_temp (integer, `nr_writeback_temp`)
|
||||
- nr_isolated_anon (integer, `nr_isolated_anon`)
|
||||
- nr_isolated_file (integer, `nr_isolated_file`)
|
||||
- nr_shmem (integer, `nr_shmem`)
|
||||
- numa_hit (integer, `numa_hit`)
|
||||
- numa_miss (integer, `numa_miss`)
|
||||
- numa_foreign (integer, `numa_foreign`)
|
||||
- numa_interleave (integer, `numa_interleave`)
|
||||
- numa_local (integer, `numa_local`)
|
||||
- numa_other (integer, `numa_other`)
|
||||
- nr_anon_transparent_hugepages (integer, `nr_anon_transparent_hugepages`)
|
||||
- pgpgin (integer, `pgpgin`)
|
||||
- pgpgout (integer, `pgpgout`)
|
||||
- pswpin (integer, `pswpin`)
|
||||
- pswpout (integer, `pswpout`)
|
||||
- pgalloc_dma (integer, `pgalloc_dma`)
|
||||
- pgalloc_dma32 (integer, `pgalloc_dma32`)
|
||||
- pgalloc_normal (integer, `pgalloc_normal`)
|
||||
- pgalloc_movable (integer, `pgalloc_movable`)
|
||||
- pgfree (integer, `pgfree`)
|
||||
- pgactivate (integer, `pgactivate`)
|
||||
- pgdeactivate (integer, `pgdeactivate`)
|
||||
- pgfault (integer, `pgfault`)
|
||||
- pgmajfault (integer, `pgmajfault`)
|
||||
- pgrefill_dma (integer, `pgrefill_dma`)
|
||||
- pgrefill_dma32 (integer, `pgrefill_dma32`)
|
||||
- pgrefill_normal (integer, `pgrefill_normal`)
|
||||
- pgrefill_movable (integer, `pgrefill_movable`)
|
||||
- pgsteal_dma (integer, `pgsteal_dma`)
|
||||
- pgsteal_dma32 (integer, `pgsteal_dma32`)
|
||||
- pgsteal_normal (integer, `pgsteal_normal`)
|
||||
- pgsteal_movable (integer, `pgsteal_movable`)
|
||||
- pgscan_kswapd_dma (integer, `pgscan_kswapd_dma`)
|
||||
- pgscan_kswapd_dma32 (integer, `pgscan_kswapd_dma32`)
|
||||
- pgscan_kswapd_normal (integer, `pgscan_kswapd_normal`)
|
||||
- pgscan_kswapd_movable (integer, `pgscan_kswapd_movable`)
|
||||
- pgscan_direct_dma (integer, `pgscan_direct_dma`)
|
||||
- pgscan_direct_dma32 (integer, `pgscan_direct_dma32`)
|
||||
- pgscan_direct_normal (integer, `pgscan_direct_normal`)
|
||||
- pgscan_direct_movable (integer, `pgscan_direct_movable`)
|
||||
- zone_reclaim_failed (integer, `zone_reclaim_failed`)
|
||||
- pginodesteal (integer, `pginodesteal`)
|
||||
- slabs_scanned (integer, `slabs_scanned`)
|
||||
- kswapd_steal (integer, `kswapd_steal`)
|
||||
- kswapd_inodesteal (integer, `kswapd_inodesteal`)
|
||||
- kswapd_low_wmark_hit_quickly (integer, `kswapd_low_wmark_hit_quickly`)
|
||||
- kswapd_high_wmark_hit_quickly (integer, `kswapd_high_wmark_hit_quickly`)
|
||||
- kswapd_skip_congestion_wait (integer, `kswapd_skip_congestion_wait`)
|
||||
- pageoutrun (integer, `pageoutrun`)
|
||||
- allocstall (integer, `allocstall`)
|
||||
- pgrotated (integer, `pgrotated`)
|
||||
- compact_blocks_moved (integer, `compact_blocks_moved`)
|
||||
- compact_pages_moved (integer, `compact_pages_moved`)
|
||||
- compact_pagemigrate_failed (integer, `compact_pagemigrate_failed`)
|
||||
- compact_stall (integer, `compact_stall`)
|
||||
- compact_fail (integer, `compact_fail`)
|
||||
- compact_success (integer, `compact_success`)
|
||||
- htlb_buddy_alloc_success (integer, `htlb_buddy_alloc_success`)
|
||||
- htlb_buddy_alloc_fail (integer, `htlb_buddy_alloc_fail`)
|
||||
- unevictable_pgs_culled (integer, `unevictable_pgs_culled`)
|
||||
- unevictable_pgs_scanned (integer, `unevictable_pgs_scanned`)
|
||||
- unevictable_pgs_rescued (integer, `unevictable_pgs_rescued`)
|
||||
- unevictable_pgs_mlocked (integer, `unevictable_pgs_mlocked`)
|
||||
- unevictable_pgs_munlocked (integer, `unevictable_pgs_munlocked`)
|
||||
- unevictable_pgs_cleared (integer, `unevictable_pgs_cleared`)
|
||||
- unevictable_pgs_stranded (integer, `unevictable_pgs_stranded`)
|
||||
- unevictable_pgs_mlockfreed (integer, `unevictable_pgs_mlockfreed`)
|
||||
- thp_fault_alloc (integer, `thp_fault_alloc`)
|
||||
- thp_fault_fallback (integer, `thp_fault_fallback`)
|
||||
- thp_collapse_alloc (integer, `thp_collapse_alloc`)
|
||||
- thp_collapse_alloc_failed (integer, `thp_collapse_alloc_failed`)
|
||||
- thp_split (integer, `thp_split`)
|
||||
|
||||
### Tags:
|
||||
|
||||
None
|
||||
|
||||
### Example Output:
|
||||
|
||||
```
|
||||
$ telegraf -config ~/ws/telegraf.conf -input-filter kernel_vmstat -test
|
||||
* Plugin: kernel_vmstat, Collection 1
|
||||
> kernel_vmstat allocstall=81496i,compact_blocks_moved=238196i,compact_fail=135220i,compact_pagemigrate_failed=0i,compact_pages_moved=6370588i,compact_stall=142092i,compact_success=6872i,htlb_buddy_alloc_fail=0i,htlb_buddy_alloc_success=0i,kswapd_high_wmark_hit_quickly=25439i,kswapd_inodesteal=29770874i,kswapd_low_wmark_hit_quickly=8756i,kswapd_skip_congestion_wait=0i,kswapd_steal=291534428i,nr_active_anon=2515657i,nr_active_file=2244914i,nr_anon_pages=1358675i,nr_anon_transparent_hugepages=2034i,nr_bounce=0i,nr_dirty=5690i,nr_file_pages=5153546i,nr_free_pages=78730i,nr_inactive_anon=426259i,nr_inactive_file=2366791i,nr_isolated_anon=0i,nr_isolated_file=0i,nr_kernel_stack=579i,nr_mapped=558821i,nr_mlock=0i,nr_page_table_pages=11115i,nr_shmem=541689i,nr_slab_reclaimable=459806i,nr_slab_unreclaimable=47859i,nr_unevictable=0i,nr_unstable=0i,nr_vmscan_write=6206i,nr_writeback=0i,nr_writeback_temp=0i,numa_foreign=0i,numa_hit=5113399878i,numa_interleave=35793i,numa_local=5113399878i,numa_miss=0i,numa_other=0i,pageoutrun=505006i,pgactivate=375664931i,pgalloc_dma=0i,pgalloc_dma32=122480220i,pgalloc_movable=0i,pgalloc_normal=5233176719i,pgdeactivate=122735906i,pgfault=8699921410i,pgfree=5359765021i,pginodesteal=9188431i,pgmajfault=122210i,pgpgin=219717626i,pgpgout=3495885510i,pgrefill_dma=0i,pgrefill_dma32=1180010i,pgrefill_movable=0i,pgrefill_normal=119866676i,pgrotated=60620i,pgscan_direct_dma=0i,pgscan_direct_dma32=12256i,pgscan_direct_movable=0i,pgscan_direct_normal=31501600i,pgscan_kswapd_dma=0i,pgscan_kswapd_dma32=4480608i,pgscan_kswapd_movable=0i,pgscan_kswapd_normal=287857984i,pgsteal_dma=0i,pgsteal_dma32=4466436i,pgsteal_movable=0i,pgsteal_normal=318463755i,pswpin=2092i,pswpout=6206i,slabs_scanned=93775616i,thp_collapse_alloc=24857i,thp_collapse_alloc_failed=102214i,thp_fault_alloc=346219i,thp_fault_fallback=895453i,thp_split=9817i,unevictable_pgs_cleared=0i,unevictable_pgs_culled=1531i,unevictable_pgs_mlocked=6988i,unevictable_pgs_mlockfreed=0i,unevictable_pgs_munlocked=6988i,unevictable_pgs_rescued=5426i,unevictable_pgs_scanned=0i,unevictable_pgs_stranded=0i,zone_reclaim_failed=0i 1459455200071462843
|
||||
```
|
||||
@@ -18,7 +18,7 @@ It is supposed to be used to monitor actual memory usage in a cross platform fas
|
||||
designed for informational purposes only.
|
||||
- **free**: memory not being used at all (zeroed) that is readily available; note
|
||||
that this doesn't reflect the actual memory available (use 'available' instead).
|
||||
- **used_percent**: the percentage usage calculated as `used / total * 100`
|
||||
- **used_percent**: the percentage usage calculated as `(total - used) / total * 100`
|
||||
|
||||
## Measurements:
|
||||
#### Raw Memory measurements:
|
||||
|
||||
@@ -44,35 +44,35 @@ func (k *Kernel) Gather(acc telegraf.Accumulator) error {
|
||||
for i, field := range dataFields {
|
||||
switch {
|
||||
case bytes.Equal(field, interrupts):
|
||||
m, err := strconv.ParseInt(string(dataFields[i+1]), 10, 64)
|
||||
m, err := strconv.Atoi(string(dataFields[i+1]))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fields["interrupts"] = int64(m)
|
||||
case bytes.Equal(field, context_switches):
|
||||
m, err := strconv.ParseInt(string(dataFields[i+1]), 10, 64)
|
||||
m, err := strconv.Atoi(string(dataFields[i+1]))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fields["context_switches"] = int64(m)
|
||||
case bytes.Equal(field, processes_forked):
|
||||
m, err := strconv.ParseInt(string(dataFields[i+1]), 10, 64)
|
||||
m, err := strconv.Atoi(string(dataFields[i+1]))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fields["processes_forked"] = int64(m)
|
||||
case bytes.Equal(field, boot_time):
|
||||
m, err := strconv.ParseInt(string(dataFields[i+1]), 10, 64)
|
||||
m, err := strconv.Atoi(string(dataFields[i+1]))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
fields["boot_time"] = int64(m)
|
||||
case bytes.Equal(field, disk_pages):
|
||||
in, err := strconv.ParseInt(string(dataFields[i+1]), 10, 64)
|
||||
in, err := strconv.Atoi(string(dataFields[i+1]))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
out, err := strconv.ParseInt(string(dataFields[i+2]), 10, 64)
|
||||
out, err := strconv.Atoi(string(dataFields[i+2]))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -1,78 +0,0 @@
|
||||
// +build linux
|
||||
|
||||
package system
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"os"
|
||||
"strconv"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
type KernelVmstat struct {
|
||||
statFile string
|
||||
}
|
||||
|
||||
func (k *KernelVmstat) Description() string {
|
||||
return "Get kernel statistics from /proc/vmstat"
|
||||
}
|
||||
|
||||
func (k *KernelVmstat) SampleConfig() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (k *KernelVmstat) Gather(acc telegraf.Accumulator) error {
|
||||
data, err := k.getProcVmstat()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
fields := make(map[string]interface{})
|
||||
|
||||
dataFields := bytes.Fields(data)
|
||||
for i, field := range dataFields {
|
||||
|
||||
// dataFields is an array of {"stat1_name", "stat1_value", "stat2_name",
|
||||
// "stat2_value", ...}
|
||||
// We only want the even number index as that contain the stat name.
|
||||
if i%2 == 0 {
|
||||
// Convert the stat value into an integer.
|
||||
m, err := strconv.Atoi(string(dataFields[i+1]))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
fields[string(field)] = int64(m)
|
||||
}
|
||||
}
|
||||
|
||||
acc.AddFields("kernel_vmstat", fields, map[string]string{})
|
||||
return nil
|
||||
}
|
||||
|
||||
func (k *KernelVmstat) getProcVmstat() ([]byte, error) {
|
||||
if _, err := os.Stat(k.statFile); os.IsNotExist(err) {
|
||||
return nil, fmt.Errorf("kernel_vmstat: %s does not exist!", k.statFile)
|
||||
} else if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
data, err := ioutil.ReadFile(k.statFile)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
return data, nil
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("kernel_vmstat", func() telegraf.Input {
|
||||
return &KernelVmstat{
|
||||
statFile: "/proc/vmstat",
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -1,315 +0,0 @@
|
||||
// +build linux
|
||||
|
||||
package system
|
||||
|
||||
import (
|
||||
"io/ioutil"
|
||||
"os"
|
||||
"testing"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestFullVmStatProcFile(t *testing.T) {
|
||||
tmpfile := makeFakeStatFile([]byte(vmStatFile_Full))
|
||||
defer os.Remove(tmpfile)
|
||||
|
||||
k := KernelVmstat{
|
||||
statFile: tmpfile,
|
||||
}
|
||||
|
||||
acc := testutil.Accumulator{}
|
||||
err := k.Gather(&acc)
|
||||
assert.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"nr_free_pages": int64(78730),
|
||||
"nr_inactive_anon": int64(426259),
|
||||
"nr_active_anon": int64(2515657),
|
||||
"nr_inactive_file": int64(2366791),
|
||||
"nr_active_file": int64(2244914),
|
||||
"nr_unevictable": int64(0),
|
||||
"nr_mlock": int64(0),
|
||||
"nr_anon_pages": int64(1358675),
|
||||
"nr_mapped": int64(558821),
|
||||
"nr_file_pages": int64(5153546),
|
||||
"nr_dirty": int64(5690),
|
||||
"nr_writeback": int64(0),
|
||||
"nr_slab_reclaimable": int64(459806),
|
||||
"nr_slab_unreclaimable": int64(47859),
|
||||
"nr_page_table_pages": int64(11115),
|
||||
"nr_kernel_stack": int64(579),
|
||||
"nr_unstable": int64(0),
|
||||
"nr_bounce": int64(0),
|
||||
"nr_vmscan_write": int64(6206),
|
||||
"nr_writeback_temp": int64(0),
|
||||
"nr_isolated_anon": int64(0),
|
||||
"nr_isolated_file": int64(0),
|
||||
"nr_shmem": int64(541689),
|
||||
"numa_hit": int64(5113399878),
|
||||
"numa_miss": int64(0),
|
||||
"numa_foreign": int64(0),
|
||||
"numa_interleave": int64(35793),
|
||||
"numa_local": int64(5113399878),
|
||||
"numa_other": int64(0),
|
||||
"nr_anon_transparent_hugepages": int64(2034),
|
||||
"pgpgin": int64(219717626),
|
||||
"pgpgout": int64(3495885510),
|
||||
"pswpin": int64(2092),
|
||||
"pswpout": int64(6206),
|
||||
"pgalloc_dma": int64(0),
|
||||
"pgalloc_dma32": int64(122480220),
|
||||
"pgalloc_normal": int64(5233176719),
|
||||
"pgalloc_movable": int64(0),
|
||||
"pgfree": int64(5359765021),
|
||||
"pgactivate": int64(375664931),
|
||||
"pgdeactivate": int64(122735906),
|
||||
"pgfault": int64(8699921410),
|
||||
"pgmajfault": int64(122210),
|
||||
"pgrefill_dma": int64(0),
|
||||
"pgrefill_dma32": int64(1180010),
|
||||
"pgrefill_normal": int64(119866676),
|
||||
"pgrefill_movable": int64(0),
|
||||
"pgsteal_dma": int64(0),
|
||||
"pgsteal_dma32": int64(4466436),
|
||||
"pgsteal_normal": int64(318463755),
|
||||
"pgsteal_movable": int64(0),
|
||||
"pgscan_kswapd_dma": int64(0),
|
||||
"pgscan_kswapd_dma32": int64(4480608),
|
||||
"pgscan_kswapd_normal": int64(287857984),
|
||||
"pgscan_kswapd_movable": int64(0),
|
||||
"pgscan_direct_dma": int64(0),
|
||||
"pgscan_direct_dma32": int64(12256),
|
||||
"pgscan_direct_normal": int64(31501600),
|
||||
"pgscan_direct_movable": int64(0),
|
||||
"zone_reclaim_failed": int64(0),
|
||||
"pginodesteal": int64(9188431),
|
||||
"slabs_scanned": int64(93775616),
|
||||
"kswapd_steal": int64(291534428),
|
||||
"kswapd_inodesteal": int64(29770874),
|
||||
"kswapd_low_wmark_hit_quickly": int64(8756),
|
||||
"kswapd_high_wmark_hit_quickly": int64(25439),
|
||||
"kswapd_skip_congestion_wait": int64(0),
|
||||
"pageoutrun": int64(505006),
|
||||
"allocstall": int64(81496),
|
||||
"pgrotated": int64(60620),
|
||||
"compact_blocks_moved": int64(238196),
|
||||
"compact_pages_moved": int64(6370588),
|
||||
"compact_pagemigrate_failed": int64(0),
|
||||
"compact_stall": int64(142092),
|
||||
"compact_fail": int64(135220),
|
||||
"compact_success": int64(6872),
|
||||
"htlb_buddy_alloc_success": int64(0),
|
||||
"htlb_buddy_alloc_fail": int64(0),
|
||||
"unevictable_pgs_culled": int64(1531),
|
||||
"unevictable_pgs_scanned": int64(0),
|
||||
"unevictable_pgs_rescued": int64(5426),
|
||||
"unevictable_pgs_mlocked": int64(6988),
|
||||
"unevictable_pgs_munlocked": int64(6988),
|
||||
"unevictable_pgs_cleared": int64(0),
|
||||
"unevictable_pgs_stranded": int64(0),
|
||||
"unevictable_pgs_mlockfreed": int64(0),
|
||||
"thp_fault_alloc": int64(346219),
|
||||
"thp_fault_fallback": int64(895453),
|
||||
"thp_collapse_alloc": int64(24857),
|
||||
"thp_collapse_alloc_failed": int64(102214),
|
||||
"thp_split": int64(9817),
|
||||
}
|
||||
acc.AssertContainsFields(t, "kernel_vmstat", fields)
|
||||
}
|
||||
|
||||
func TestPartialVmStatProcFile(t *testing.T) {
|
||||
tmpfile := makeFakeStatFile([]byte(vmStatFile_Partial))
|
||||
defer os.Remove(tmpfile)
|
||||
|
||||
k := KernelVmstat{
|
||||
statFile: tmpfile,
|
||||
}
|
||||
|
||||
acc := testutil.Accumulator{}
|
||||
err := k.Gather(&acc)
|
||||
assert.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"unevictable_pgs_culled": int64(1531),
|
||||
"unevictable_pgs_scanned": int64(0),
|
||||
"unevictable_pgs_rescued": int64(5426),
|
||||
"unevictable_pgs_mlocked": int64(6988),
|
||||
"unevictable_pgs_munlocked": int64(6988),
|
||||
"unevictable_pgs_cleared": int64(0),
|
||||
"unevictable_pgs_stranded": int64(0),
|
||||
"unevictable_pgs_mlockfreed": int64(0),
|
||||
"thp_fault_alloc": int64(346219),
|
||||
"thp_fault_fallback": int64(895453),
|
||||
"thp_collapse_alloc": int64(24857),
|
||||
"thp_collapse_alloc_failed": int64(102214),
|
||||
"thp_split": int64(9817),
|
||||
}
|
||||
acc.AssertContainsFields(t, "kernel_vmstat", fields)
|
||||
}
|
||||
|
||||
func TestInvalidVmStatProcFile1(t *testing.T) {
|
||||
tmpfile := makeFakeStatFile([]byte(vmStatFile_Invalid))
|
||||
defer os.Remove(tmpfile)
|
||||
|
||||
k := KernelVmstat{
|
||||
statFile: tmpfile,
|
||||
}
|
||||
|
||||
acc := testutil.Accumulator{}
|
||||
err := k.Gather(&acc)
|
||||
assert.Error(t, err)
|
||||
}
|
||||
|
||||
func TestNoVmStatProcFile(t *testing.T) {
|
||||
tmpfile := makeFakeStatFile([]byte(vmStatFile_Invalid))
|
||||
os.Remove(tmpfile)
|
||||
|
||||
k := KernelVmstat{
|
||||
statFile: tmpfile,
|
||||
}
|
||||
|
||||
acc := testutil.Accumulator{}
|
||||
err := k.Gather(&acc)
|
||||
assert.Error(t, err)
|
||||
assert.Contains(t, err.Error(), "does not exist")
|
||||
}
|
||||
|
||||
const vmStatFile_Full = `nr_free_pages 78730
|
||||
nr_inactive_anon 426259
|
||||
nr_active_anon 2515657
|
||||
nr_inactive_file 2366791
|
||||
nr_active_file 2244914
|
||||
nr_unevictable 0
|
||||
nr_mlock 0
|
||||
nr_anon_pages 1358675
|
||||
nr_mapped 558821
|
||||
nr_file_pages 5153546
|
||||
nr_dirty 5690
|
||||
nr_writeback 0
|
||||
nr_slab_reclaimable 459806
|
||||
nr_slab_unreclaimable 47859
|
||||
nr_page_table_pages 11115
|
||||
nr_kernel_stack 579
|
||||
nr_unstable 0
|
||||
nr_bounce 0
|
||||
nr_vmscan_write 6206
|
||||
nr_writeback_temp 0
|
||||
nr_isolated_anon 0
|
||||
nr_isolated_file 0
|
||||
nr_shmem 541689
|
||||
numa_hit 5113399878
|
||||
numa_miss 0
|
||||
numa_foreign 0
|
||||
numa_interleave 35793
|
||||
numa_local 5113399878
|
||||
numa_other 0
|
||||
nr_anon_transparent_hugepages 2034
|
||||
pgpgin 219717626
|
||||
pgpgout 3495885510
|
||||
pswpin 2092
|
||||
pswpout 6206
|
||||
pgalloc_dma 0
|
||||
pgalloc_dma32 122480220
|
||||
pgalloc_normal 5233176719
|
||||
pgalloc_movable 0
|
||||
pgfree 5359765021
|
||||
pgactivate 375664931
|
||||
pgdeactivate 122735906
|
||||
pgfault 8699921410
|
||||
pgmajfault 122210
|
||||
pgrefill_dma 0
|
||||
pgrefill_dma32 1180010
|
||||
pgrefill_normal 119866676
|
||||
pgrefill_movable 0
|
||||
pgsteal_dma 0
|
||||
pgsteal_dma32 4466436
|
||||
pgsteal_normal 318463755
|
||||
pgsteal_movable 0
|
||||
pgscan_kswapd_dma 0
|
||||
pgscan_kswapd_dma32 4480608
|
||||
pgscan_kswapd_normal 287857984
|
||||
pgscan_kswapd_movable 0
|
||||
pgscan_direct_dma 0
|
||||
pgscan_direct_dma32 12256
|
||||
pgscan_direct_normal 31501600
|
||||
pgscan_direct_movable 0
|
||||
zone_reclaim_failed 0
|
||||
pginodesteal 9188431
|
||||
slabs_scanned 93775616
|
||||
kswapd_steal 291534428
|
||||
kswapd_inodesteal 29770874
|
||||
kswapd_low_wmark_hit_quickly 8756
|
||||
kswapd_high_wmark_hit_quickly 25439
|
||||
kswapd_skip_congestion_wait 0
|
||||
pageoutrun 505006
|
||||
allocstall 81496
|
||||
pgrotated 60620
|
||||
compact_blocks_moved 238196
|
||||
compact_pages_moved 6370588
|
||||
compact_pagemigrate_failed 0
|
||||
compact_stall 142092
|
||||
compact_fail 135220
|
||||
compact_success 6872
|
||||
htlb_buddy_alloc_success 0
|
||||
htlb_buddy_alloc_fail 0
|
||||
unevictable_pgs_culled 1531
|
||||
unevictable_pgs_scanned 0
|
||||
unevictable_pgs_rescued 5426
|
||||
unevictable_pgs_mlocked 6988
|
||||
unevictable_pgs_munlocked 6988
|
||||
unevictable_pgs_cleared 0
|
||||
unevictable_pgs_stranded 0
|
||||
unevictable_pgs_mlockfreed 0
|
||||
thp_fault_alloc 346219
|
||||
thp_fault_fallback 895453
|
||||
thp_collapse_alloc 24857
|
||||
thp_collapse_alloc_failed 102214
|
||||
thp_split 9817`
|
||||
|
||||
const vmStatFile_Partial = `unevictable_pgs_culled 1531
|
||||
unevictable_pgs_scanned 0
|
||||
unevictable_pgs_rescued 5426
|
||||
unevictable_pgs_mlocked 6988
|
||||
unevictable_pgs_munlocked 6988
|
||||
unevictable_pgs_cleared 0
|
||||
unevictable_pgs_stranded 0
|
||||
unevictable_pgs_mlockfreed 0
|
||||
thp_fault_alloc 346219
|
||||
thp_fault_fallback 895453
|
||||
thp_collapse_alloc 24857
|
||||
thp_collapse_alloc_failed 102214
|
||||
thp_split 9817`
|
||||
|
||||
// invalid thp_split measurement
|
||||
const vmStatFile_Invalid = `unevictable_pgs_culled 1531
|
||||
unevictable_pgs_scanned 0
|
||||
unevictable_pgs_rescued 5426
|
||||
unevictable_pgs_mlocked 6988
|
||||
unevictable_pgs_munlocked 6988
|
||||
unevictable_pgs_cleared 0
|
||||
unevictable_pgs_stranded 0
|
||||
unevictable_pgs_mlockfreed 0
|
||||
thp_fault_alloc 346219
|
||||
thp_fault_fallback 895453
|
||||
thp_collapse_alloc 24857
|
||||
thp_collapse_alloc_failed 102214
|
||||
thp_split abcd`
|
||||
|
||||
func makeFakeVmStatFile(content []byte) string {
|
||||
tmpfile, err := ioutil.TempFile("", "kernel_vmstat_test")
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
if _, err := tmpfile.Write(content); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
if err := tmpfile.Close(); err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
return tmpfile.Name()
|
||||
}
|
||||
@@ -30,8 +30,6 @@ func (s *MemStats) Gather(acc telegraf.Accumulator) error {
|
||||
"free": vm.Free,
|
||||
"cached": vm.Cached,
|
||||
"buffered": vm.Buffers,
|
||||
"active": vm.Active,
|
||||
"inactive": vm.Inactive,
|
||||
"used_percent": 100 * float64(vm.Used) / float64(vm.Total),
|
||||
"available_percent": 100 * float64(vm.Available) / float64(vm.Total),
|
||||
}
|
||||
|
||||
@@ -19,8 +19,8 @@ func TestMemStats(t *testing.T) {
|
||||
Available: 7600,
|
||||
Used: 5000,
|
||||
Free: 1235,
|
||||
Active: 8134,
|
||||
Inactive: 1124,
|
||||
// Active: 8134,
|
||||
// Inactive: 1124,
|
||||
// Buffers: 771,
|
||||
// Cached: 4312,
|
||||
// Wired: 134,
|
||||
@@ -52,8 +52,6 @@ func TestMemStats(t *testing.T) {
|
||||
"free": uint64(1235),
|
||||
"cached": uint64(0),
|
||||
"buffered": uint64(0),
|
||||
"active": uint64(8134),
|
||||
"inactive": uint64(1124),
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "mem", memfields, make(map[string]string))
|
||||
|
||||
|
||||
@@ -9,7 +9,7 @@ import (
|
||||
"log"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"path"
|
||||
"runtime"
|
||||
"strconv"
|
||||
|
||||
@@ -19,7 +19,7 @@ import (
|
||||
|
||||
type Processes struct {
|
||||
execPS func() ([]byte, error)
|
||||
readProcFile func(filename string) ([]byte, error)
|
||||
readProcFile func(statFile string) ([]byte, error)
|
||||
|
||||
forcePS bool
|
||||
forceProc bool
|
||||
@@ -128,15 +128,18 @@ func (p *Processes) gatherFromPS(fields map[string]interface{}) error {
|
||||
|
||||
// get process states from /proc/(pid)/stat files
|
||||
func (p *Processes) gatherFromProc(fields map[string]interface{}) error {
|
||||
filenames, err := filepath.Glob("/proc/[0-9]*/stat")
|
||||
files, err := ioutil.ReadDir("/proc")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
for _, filename := range filenames {
|
||||
_, err := os.Stat(filename)
|
||||
for _, file := range files {
|
||||
if !file.IsDir() {
|
||||
continue
|
||||
}
|
||||
|
||||
data, err := p.readProcFile(filename)
|
||||
statFile := path.Join("/proc", file.Name(), "stat")
|
||||
data, err := p.readProcFile(statFile)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -153,7 +156,7 @@ func (p *Processes) gatherFromProc(fields map[string]interface{}) error {
|
||||
|
||||
stats := bytes.Fields(data)
|
||||
if len(stats) < 3 {
|
||||
return fmt.Errorf("Something is terribly wrong with %s", filename)
|
||||
return fmt.Errorf("Something is terribly wrong with %s", statFile)
|
||||
}
|
||||
switch stats[0][0] {
|
||||
case 'R':
|
||||
@@ -170,7 +173,7 @@ func (p *Processes) gatherFromProc(fields map[string]interface{}) error {
|
||||
fields["paging"] = fields["paging"].(int64) + int64(1)
|
||||
default:
|
||||
log.Printf("processes: Unknown state [ %s ] in file %s",
|
||||
string(stats[0][0]), filename)
|
||||
string(stats[0][0]), statFile)
|
||||
}
|
||||
fields["total"] = fields["total"].(int64) + int64(1)
|
||||
|
||||
@@ -184,12 +187,15 @@ func (p *Processes) gatherFromProc(fields map[string]interface{}) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func readProcFile(filename string) ([]byte, error) {
|
||||
data, err := ioutil.ReadFile(filename)
|
||||
func readProcFile(statFile string) ([]byte, error) {
|
||||
if _, err := os.Stat(statFile); os.IsNotExist(err) {
|
||||
return nil, nil
|
||||
} else if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
data, err := ioutil.ReadFile(statFile)
|
||||
if err != nil {
|
||||
if os.IsNotExist(err) {
|
||||
return nil, nil
|
||||
}
|
||||
return nil, err
|
||||
}
|
||||
|
||||
|
||||
@@ -84,10 +84,10 @@ func (s *systemPS) DiskUsage(
|
||||
mountpoint := os.Getenv("HOST_MOUNT_PREFIX") + p.Mountpoint
|
||||
if _, err := os.Stat(mountpoint); err == nil {
|
||||
du, err := disk.Usage(mountpoint)
|
||||
du.Path = p.Mountpoint
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
du.Path = p.Mountpoint
|
||||
// If the mount point is a member of the exclude set,
|
||||
// don't gather info on it.
|
||||
_, ok := fstypeExcludeSet[p.Fstype]
|
||||
|
||||
@@ -95,7 +95,6 @@ func (t *Tail) Start(acc telegraf.Accumulator) error {
|
||||
continue
|
||||
}
|
||||
// create a goroutine for each "tailer"
|
||||
t.wg.Add(1)
|
||||
go t.receiver(tailer)
|
||||
t.tailers = append(t.tailers, tailer)
|
||||
}
|
||||
@@ -110,6 +109,7 @@ func (t *Tail) Start(acc telegraf.Accumulator) error {
|
||||
// this is launched as a goroutine to continuously watch a tailed logfile
|
||||
// for changes, parse any incoming msgs, and add to the accumulator.
|
||||
func (t *Tail) receiver(tailer *tail.Tail) {
|
||||
t.wg.Add(1)
|
||||
defer t.wg.Done()
|
||||
|
||||
var m telegraf.Metric
|
||||
|
||||
@@ -32,8 +32,7 @@ func TestTailFromBeginning(t *testing.T) {
|
||||
_, err = tmpfile.WriteString("cpu,mytag=foo usage_idle=100\n")
|
||||
require.NoError(t, err)
|
||||
require.NoError(t, tt.Gather(&acc))
|
||||
// arbitrary sleep to wait for message to show up
|
||||
time.Sleep(time.Millisecond * 250)
|
||||
time.Sleep(time.Millisecond * 50)
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "cpu",
|
||||
map[string]interface{}{
|
||||
|
||||
@@ -29,8 +29,6 @@ type TcpListener struct {
|
||||
// is an available bool in accept, then we are below the maximum and can
|
||||
// accept the connection
|
||||
accept chan bool
|
||||
// drops tracks the number of dropped metrics.
|
||||
drops int
|
||||
|
||||
// track the listener here so we can close it in Stop()
|
||||
listener *net.TCPListener
|
||||
@@ -41,8 +39,7 @@ type TcpListener struct {
|
||||
acc telegraf.Accumulator
|
||||
}
|
||||
|
||||
var dropwarn = "ERROR: tcp_listener message queue full. " +
|
||||
"We have dropped %d messages so far. " +
|
||||
var dropwarn = "ERROR: Message queue full. Discarding metric [%s], " +
|
||||
"You may want to increase allowed_pending_messages in the config\n"
|
||||
|
||||
const sampleConfig = `
|
||||
@@ -215,10 +212,7 @@ func (t *TcpListener) handler(conn *net.TCPConn, id string) {
|
||||
select {
|
||||
case t.in <- bufCopy:
|
||||
default:
|
||||
t.drops++
|
||||
if t.drops == 1 || t.drops%t.AllowedPendingMessages == 0 {
|
||||
log.Printf(dropwarn, t.drops)
|
||||
}
|
||||
log.Printf(dropwarn, scanner.Text())
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -25,8 +25,6 @@ type UdpListener struct {
|
||||
|
||||
in chan []byte
|
||||
done chan struct{}
|
||||
// drops tracks the number of dropped metrics.
|
||||
drops int
|
||||
|
||||
parser parsers.Parser
|
||||
|
||||
@@ -40,8 +38,7 @@ type UdpListener struct {
|
||||
// https://en.wikipedia.org/wiki/User_Datagram_Protocol#Packet_structure
|
||||
const UDP_MAX_PACKET_SIZE int = 64 * 1024
|
||||
|
||||
var dropwarn = "ERROR: udp_listener message queue full. " +
|
||||
"We have dropped %d messages so far. " +
|
||||
var dropwarn = "ERROR: Message queue full. Discarding line [%s] " +
|
||||
"You may want to increase allowed_pending_messages in the config\n"
|
||||
|
||||
const sampleConfig = `
|
||||
@@ -128,10 +125,7 @@ func (u *UdpListener) udpListen() error {
|
||||
select {
|
||||
case u.in <- bufCopy:
|
||||
default:
|
||||
u.drops++
|
||||
if u.drops == 1 || u.drops%u.AllowedPendingMessages == 0 {
|
||||
log.Printf(dropwarn, u.drops)
|
||||
}
|
||||
log.Printf(dropwarn, string(bufCopy))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -22,299 +22,299 @@ This is the full list of stats provided by varnish. Stats will be grouped by the
|
||||
MEMPOOL, etc). In the output, the prefix will be used as a tag, and removed from field names.
|
||||
|
||||
- varnish
|
||||
- MAIN.uptime (uint64, count, Child process uptime)
|
||||
- MAIN.sess_conn (uint64, count, Sessions accepted)
|
||||
- MAIN.sess_drop (uint64, count, Sessions dropped)
|
||||
- MAIN.sess_fail (uint64, count, Session accept failures)
|
||||
- MAIN.sess_pipe_overflow (uint64, count, Session pipe overflow)
|
||||
- MAIN.client_req_400 (uint64, count, Client requests received,)
|
||||
- MAIN.client_req_411 (uint64, count, Client requests received,)
|
||||
- MAIN.client_req_413 (uint64, count, Client requests received,)
|
||||
- MAIN.client_req_417 (uint64, count, Client requests received,)
|
||||
- MAIN.client_req (uint64, count, Good client requests)
|
||||
- MAIN.cache_hit (uint64, count, Cache hits)
|
||||
- MAIN.cache_hitpass (uint64, count, Cache hits for)
|
||||
- MAIN.cache_miss (uint64, count, Cache misses)
|
||||
- MAIN.backend_conn (uint64, count, Backend conn. success)
|
||||
- MAIN.backend_unhealthy (uint64, count, Backend conn. not)
|
||||
- MAIN.backend_busy (uint64, count, Backend conn. too)
|
||||
- MAIN.backend_fail (uint64, count, Backend conn. failures)
|
||||
- MAIN.backend_reuse (uint64, count, Backend conn. reuses)
|
||||
- MAIN.backend_toolate (uint64, count, Backend conn. was)
|
||||
- MAIN.backend_recycle (uint64, count, Backend conn. recycles)
|
||||
- MAIN.backend_retry (uint64, count, Backend conn. retry)
|
||||
- MAIN.fetch_head (uint64, count, Fetch no body)
|
||||
- MAIN.fetch_length (uint64, count, Fetch with Length)
|
||||
- MAIN.fetch_chunked (uint64, count, Fetch chunked)
|
||||
- MAIN.fetch_eof (uint64, count, Fetch EOF)
|
||||
- MAIN.fetch_bad (uint64, count, Fetch bad T- E)
|
||||
- MAIN.fetch_close (uint64, count, Fetch wanted close)
|
||||
- MAIN.fetch_oldhttp (uint64, count, Fetch pre HTTP/1.1)
|
||||
- MAIN.fetch_zero (uint64, count, Fetch zero len)
|
||||
- MAIN.fetch_1xx (uint64, count, Fetch no body)
|
||||
- MAIN.fetch_204 (uint64, count, Fetch no body)
|
||||
- MAIN.fetch_304 (uint64, count, Fetch no body)
|
||||
- MAIN.fetch_failed (uint64, count, Fetch failed (all)
|
||||
- MAIN.fetch_no_thread (uint64, count, Fetch failed (no)
|
||||
- MAIN.pools (uint64, count, Number of thread)
|
||||
- MAIN.threads (uint64, count, Total number of)
|
||||
- MAIN.threads_limited (uint64, count, Threads hit max)
|
||||
- MAIN.threads_created (uint64, count, Threads created)
|
||||
- MAIN.threads_destroyed (uint64, count, Threads destroyed)
|
||||
- MAIN.threads_failed (uint64, count, Thread creation failed)
|
||||
- MAIN.thread_queue_len (uint64, count, Length of session)
|
||||
- MAIN.busy_sleep (uint64, count, Number of requests)
|
||||
- MAIN.busy_wakeup (uint64, count, Number of requests)
|
||||
- MAIN.sess_queued (uint64, count, Sessions queued for)
|
||||
- MAIN.sess_dropped (uint64, count, Sessions dropped for)
|
||||
- MAIN.n_object (uint64, count, object structs made)
|
||||
- MAIN.n_vampireobject (uint64, count, unresurrected objects)
|
||||
- MAIN.n_objectcore (uint64, count, objectcore structs made)
|
||||
- MAIN.n_objecthead (uint64, count, objecthead structs made)
|
||||
- MAIN.n_waitinglist (uint64, count, waitinglist structs made)
|
||||
- MAIN.n_backend (uint64, count, Number of backends)
|
||||
- MAIN.n_expired (uint64, count, Number of expired)
|
||||
- MAIN.n_lru_nuked (uint64, count, Number of LRU)
|
||||
- MAIN.n_lru_moved (uint64, count, Number of LRU)
|
||||
- MAIN.losthdr (uint64, count, HTTP header overflows)
|
||||
- MAIN.s_sess (uint64, count, Total sessions seen)
|
||||
- MAIN.s_req (uint64, count, Total requests seen)
|
||||
- MAIN.s_pipe (uint64, count, Total pipe sessions)
|
||||
- MAIN.s_pass (uint64, count, Total pass- ed requests)
|
||||
- MAIN.s_fetch (uint64, count, Total backend fetches)
|
||||
- MAIN.s_synth (uint64, count, Total synthethic responses)
|
||||
- MAIN.s_req_hdrbytes (uint64, count, Request header bytes)
|
||||
- MAIN.s_req_bodybytes (uint64, count, Request body bytes)
|
||||
- MAIN.s_resp_hdrbytes (uint64, count, Response header bytes)
|
||||
- MAIN.s_resp_bodybytes (uint64, count, Response body bytes)
|
||||
- MAIN.s_pipe_hdrbytes (uint64, count, Pipe request header)
|
||||
- MAIN.s_pipe_in (uint64, count, Piped bytes from)
|
||||
- MAIN.s_pipe_out (uint64, count, Piped bytes to)
|
||||
- MAIN.sess_closed (uint64, count, Session Closed)
|
||||
- MAIN.sess_pipeline (uint64, count, Session Pipeline)
|
||||
- MAIN.sess_readahead (uint64, count, Session Read Ahead)
|
||||
- MAIN.sess_herd (uint64, count, Session herd)
|
||||
- MAIN.shm_records (uint64, count, SHM records)
|
||||
- MAIN.shm_writes (uint64, count, SHM writes)
|
||||
- MAIN.shm_flushes (uint64, count, SHM flushes due)
|
||||
- MAIN.shm_cont (uint64, count, SHM MTX contention)
|
||||
- MAIN.shm_cycles (uint64, count, SHM cycles through)
|
||||
- MAIN.sms_nreq (uint64, count, SMS allocator requests)
|
||||
- MAIN.sms_nobj (uint64, count, SMS outstanding allocations)
|
||||
- MAIN.sms_nbytes (uint64, count, SMS outstanding bytes)
|
||||
- MAIN.sms_balloc (uint64, count, SMS bytes allocated)
|
||||
- MAIN.sms_bfree (uint64, count, SMS bytes freed)
|
||||
- MAIN.backend_req (uint64, count, Backend requests made)
|
||||
- MAIN.n_vcl (uint64, count, Number of loaded)
|
||||
- MAIN.n_vcl_avail (uint64, count, Number of VCLs)
|
||||
- MAIN.n_vcl_discard (uint64, count, Number of discarded)
|
||||
- MAIN.bans (uint64, count, Count of bans)
|
||||
- MAIN.bans_completed (uint64, count, Number of bans)
|
||||
- MAIN.bans_obj (uint64, count, Number of bans)
|
||||
- MAIN.bans_req (uint64, count, Number of bans)
|
||||
- MAIN.bans_added (uint64, count, Bans added)
|
||||
- MAIN.bans_deleted (uint64, count, Bans deleted)
|
||||
- MAIN.bans_tested (uint64, count, Bans tested against)
|
||||
- MAIN.bans_obj_killed (uint64, count, Objects killed by)
|
||||
- MAIN.bans_lurker_tested (uint64, count, Bans tested against)
|
||||
- MAIN.bans_tests_tested (uint64, count, Ban tests tested)
|
||||
- MAIN.bans_lurker_tests_tested (uint64, count, Ban tests tested)
|
||||
- MAIN.bans_lurker_obj_killed (uint64, count, Objects killed by)
|
||||
- MAIN.bans_dups (uint64, count, Bans superseded by)
|
||||
- MAIN.bans_lurker_contention (uint64, count, Lurker gave way)
|
||||
- MAIN.bans_persisted_bytes (uint64, count, Bytes used by)
|
||||
- MAIN.bans_persisted_fragmentation (uint64, count, Extra bytes in)
|
||||
- MAIN.n_purges (uint64, count, Number of purge)
|
||||
- MAIN.n_obj_purged (uint64, count, Number of purged)
|
||||
- MAIN.exp_mailed (uint64, count, Number of objects)
|
||||
- MAIN.exp_received (uint64, count, Number of objects)
|
||||
- MAIN.hcb_nolock (uint64, count, HCB Lookups without)
|
||||
- MAIN.hcb_lock (uint64, count, HCB Lookups with)
|
||||
- MAIN.hcb_insert (uint64, count, HCB Inserts)
|
||||
- MAIN.esi_errors (uint64, count, ESI parse errors)
|
||||
- MAIN.esi_warnings (uint64, count, ESI parse warnings)
|
||||
- MAIN.vmods (uint64, count, Loaded VMODs)
|
||||
- MAIN.n_gzip (uint64, count, Gzip operations)
|
||||
- MAIN.n_gunzip (uint64, count, Gunzip operations)
|
||||
- MAIN.vsm_free (uint64, count, Free VSM space)
|
||||
- MAIN.vsm_used (uint64, count, Used VSM space)
|
||||
- MAIN.vsm_cooling (uint64, count, Cooling VSM space)
|
||||
- MAIN.vsm_overflow (uint64, count, Overflow VSM space)
|
||||
- MAIN.vsm_overflowed (uint64, count, Overflowed VSM space)
|
||||
- MGT.uptime (uint64, count, Management process uptime)
|
||||
- MGT.child_start (uint64, count, Child process started)
|
||||
- MGT.child_exit (uint64, count, Child process normal)
|
||||
- MGT.child_stop (uint64, count, Child process unexpected)
|
||||
- MGT.child_died (uint64, count, Child process died)
|
||||
- MGT.child_dump (uint64, count, Child process core)
|
||||
- MGT.child_panic (uint64, count, Child process panic)
|
||||
- MEMPOOL.vbc.live (uint64, count, In use)
|
||||
- MEMPOOL.vbc.pool (uint64, count, In Pool)
|
||||
- MEMPOOL.vbc.sz_wanted (uint64, count, Size requested)
|
||||
- MEMPOOL.vbc.sz_needed (uint64, count, Size allocated)
|
||||
- MEMPOOL.vbc.allocs (uint64, count, Allocations )
|
||||
- MEMPOOL.vbc.frees (uint64, count, Frees )
|
||||
- MEMPOOL.vbc.recycle (uint64, count, Recycled from pool)
|
||||
- MEMPOOL.vbc.timeout (uint64, count, Timed out from)
|
||||
- MEMPOOL.vbc.toosmall (uint64, count, Too small to)
|
||||
- MEMPOOL.vbc.surplus (uint64, count, Too many for)
|
||||
- MEMPOOL.vbc.randry (uint64, count, Pool ran dry)
|
||||
- MEMPOOL.busyobj.live (uint64, count, In use)
|
||||
- MEMPOOL.busyobj.pool (uint64, count, In Pool)
|
||||
- MEMPOOL.busyobj.sz_wanted (uint64, count, Size requested)
|
||||
- MEMPOOL.busyobj.sz_needed (uint64, count, Size allocated)
|
||||
- MEMPOOL.busyobj.allocs (uint64, count, Allocations )
|
||||
- MEMPOOL.busyobj.frees (uint64, count, Frees )
|
||||
- MEMPOOL.busyobj.recycle (uint64, count, Recycled from pool)
|
||||
- MEMPOOL.busyobj.timeout (uint64, count, Timed out from)
|
||||
- MEMPOOL.busyobj.toosmall (uint64, count, Too small to)
|
||||
- MEMPOOL.busyobj.surplus (uint64, count, Too many for)
|
||||
- MEMPOOL.busyobj.randry (uint64, count, Pool ran dry)
|
||||
- MEMPOOL.req0.live (uint64, count, In use)
|
||||
- MEMPOOL.req0.pool (uint64, count, In Pool)
|
||||
- MEMPOOL.req0.sz_wanted (uint64, count, Size requested)
|
||||
- MEMPOOL.req0.sz_needed (uint64, count, Size allocated)
|
||||
- MEMPOOL.req0.allocs (uint64, count, Allocations )
|
||||
- MEMPOOL.req0.frees (uint64, count, Frees )
|
||||
- MEMPOOL.req0.recycle (uint64, count, Recycled from pool)
|
||||
- MEMPOOL.req0.timeout (uint64, count, Timed out from)
|
||||
- MEMPOOL.req0.toosmall (uint64, count, Too small to)
|
||||
- MEMPOOL.req0.surplus (uint64, count, Too many for)
|
||||
- MEMPOOL.req0.randry (uint64, count, Pool ran dry)
|
||||
- MEMPOOL.sess0.live (uint64, count, In use)
|
||||
- MEMPOOL.sess0.pool (uint64, count, In Pool)
|
||||
- MEMPOOL.sess0.sz_wanted (uint64, count, Size requested)
|
||||
- MEMPOOL.sess0.sz_needed (uint64, count, Size allocated)
|
||||
- MEMPOOL.sess0.allocs (uint64, count, Allocations )
|
||||
- MEMPOOL.sess0.frees (uint64, count, Frees )
|
||||
- MEMPOOL.sess0.recycle (uint64, count, Recycled from pool)
|
||||
- MEMPOOL.sess0.timeout (uint64, count, Timed out from)
|
||||
- MEMPOOL.sess0.toosmall (uint64, count, Too small to)
|
||||
- MEMPOOL.sess0.surplus (uint64, count, Too many for)
|
||||
- MEMPOOL.sess0.randry (uint64, count, Pool ran dry)
|
||||
- MEMPOOL.req1.live (uint64, count, In use)
|
||||
- MEMPOOL.req1.pool (uint64, count, In Pool)
|
||||
- MEMPOOL.req1.sz_wanted (uint64, count, Size requested)
|
||||
- MEMPOOL.req1.sz_needed (uint64, count, Size allocated)
|
||||
- MEMPOOL.req1.allocs (uint64, count, Allocations )
|
||||
- MEMPOOL.req1.frees (uint64, count, Frees )
|
||||
- MEMPOOL.req1.recycle (uint64, count, Recycled from pool)
|
||||
- MEMPOOL.req1.timeout (uint64, count, Timed out from)
|
||||
- MEMPOOL.req1.toosmall (uint64, count, Too small to)
|
||||
- MEMPOOL.req1.surplus (uint64, count, Too many for)
|
||||
- MEMPOOL.req1.randry (uint64, count, Pool ran dry)
|
||||
- MEMPOOL.sess1.live (uint64, count, In use)
|
||||
- MEMPOOL.sess1.pool (uint64, count, In Pool)
|
||||
- MEMPOOL.sess1.sz_wanted (uint64, count, Size requested)
|
||||
- MEMPOOL.sess1.sz_needed (uint64, count, Size allocated)
|
||||
- MEMPOOL.sess1.allocs (uint64, count, Allocations )
|
||||
- MEMPOOL.sess1.frees (uint64, count, Frees )
|
||||
- MEMPOOL.sess1.recycle (uint64, count, Recycled from pool)
|
||||
- MEMPOOL.sess1.timeout (uint64, count, Timed out from)
|
||||
- MEMPOOL.sess1.toosmall (uint64, count, Too small to)
|
||||
- MEMPOOL.sess1.surplus (uint64, count, Too many for)
|
||||
- MEMPOOL.sess1.randry (uint64, count, Pool ran dry)
|
||||
- SMA.s0.c_req (uint64, count, Allocator requests)
|
||||
- SMA.s0.c_fail (uint64, count, Allocator failures)
|
||||
- SMA.s0.c_bytes (uint64, count, Bytes allocated)
|
||||
- SMA.s0.c_freed (uint64, count, Bytes freed)
|
||||
- SMA.s0.g_alloc (uint64, count, Allocations outstanding)
|
||||
- SMA.s0.g_bytes (uint64, count, Bytes outstanding)
|
||||
- SMA.s0.g_space (uint64, count, Bytes available)
|
||||
- SMA.Transient.c_req (uint64, count, Allocator requests)
|
||||
- SMA.Transient.c_fail (uint64, count, Allocator failures)
|
||||
- SMA.Transient.c_bytes (uint64, count, Bytes allocated)
|
||||
- SMA.Transient.c_freed (uint64, count, Bytes freed)
|
||||
- SMA.Transient.g_alloc (uint64, count, Allocations outstanding)
|
||||
- SMA.Transient.g_bytes (uint64, count, Bytes outstanding)
|
||||
- SMA.Transient.g_space (uint64, count, Bytes available)
|
||||
- VBE.default(127.0.0.1,,8080).vcls (uint64, count, VCL references)
|
||||
- VBE.default(127.0.0.1,,8080).happy (uint64, count, Happy health probes)
|
||||
- VBE.default(127.0.0.1,,8080).bereq_hdrbytes (uint64, count, Request header bytes)
|
||||
- VBE.default(127.0.0.1,,8080).bereq_bodybytes (uint64, count, Request body bytes)
|
||||
- VBE.default(127.0.0.1,,8080).beresp_hdrbytes (uint64, count, Response header bytes)
|
||||
- VBE.default(127.0.0.1,,8080).beresp_bodybytes (uint64, count, Response body bytes)
|
||||
- VBE.default(127.0.0.1,,8080).pipe_hdrbytes (uint64, count, Pipe request header)
|
||||
- VBE.default(127.0.0.1,,8080).pipe_out (uint64, count, Piped bytes to)
|
||||
- VBE.default(127.0.0.1,,8080).pipe_in (uint64, count, Piped bytes from)
|
||||
- LCK.sms.creat (uint64, count, Created locks)
|
||||
- LCK.sms.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.sms.locks (uint64, count, Lock Operations)
|
||||
- LCK.smp.creat (uint64, count, Created locks)
|
||||
- LCK.smp.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.smp.locks (uint64, count, Lock Operations)
|
||||
- LCK.sma.creat (uint64, count, Created locks)
|
||||
- LCK.sma.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.sma.locks (uint64, count, Lock Operations)
|
||||
- LCK.smf.creat (uint64, count, Created locks)
|
||||
- LCK.smf.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.smf.locks (uint64, count, Lock Operations)
|
||||
- LCK.hsl.creat (uint64, count, Created locks)
|
||||
- LCK.hsl.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.hsl.locks (uint64, count, Lock Operations)
|
||||
- LCK.hcb.creat (uint64, count, Created locks)
|
||||
- LCK.hcb.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.hcb.locks (uint64, count, Lock Operations)
|
||||
- LCK.hcl.creat (uint64, count, Created locks)
|
||||
- LCK.hcl.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.hcl.locks (uint64, count, Lock Operations)
|
||||
- LCK.vcl.creat (uint64, count, Created locks)
|
||||
- LCK.vcl.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.vcl.locks (uint64, count, Lock Operations)
|
||||
- LCK.sessmem.creat (uint64, count, Created locks)
|
||||
- LCK.sessmem.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.sessmem.locks (uint64, count, Lock Operations)
|
||||
- LCK.sess.creat (uint64, count, Created locks)
|
||||
- LCK.sess.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.sess.locks (uint64, count, Lock Operations)
|
||||
- LCK.wstat.creat (uint64, count, Created locks)
|
||||
- LCK.wstat.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.wstat.locks (uint64, count, Lock Operations)
|
||||
- LCK.herder.creat (uint64, count, Created locks)
|
||||
- LCK.herder.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.herder.locks (uint64, count, Lock Operations)
|
||||
- LCK.wq.creat (uint64, count, Created locks)
|
||||
- LCK.wq.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.wq.locks (uint64, count, Lock Operations)
|
||||
- LCK.objhdr.creat (uint64, count, Created locks)
|
||||
- LCK.objhdr.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.objhdr.locks (uint64, count, Lock Operations)
|
||||
- LCK.exp.creat (uint64, count, Created locks)
|
||||
- LCK.exp.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.exp.locks (uint64, count, Lock Operations)
|
||||
- LCK.lru.creat (uint64, count, Created locks)
|
||||
- LCK.lru.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.lru.locks (uint64, count, Lock Operations)
|
||||
- LCK.cli.creat (uint64, count, Created locks)
|
||||
- LCK.cli.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.cli.locks (uint64, count, Lock Operations)
|
||||
- LCK.ban.creat (uint64, count, Created locks)
|
||||
- LCK.ban.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.ban.locks (uint64, count, Lock Operations)
|
||||
- LCK.vbp.creat (uint64, count, Created locks)
|
||||
- LCK.vbp.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.vbp.locks (uint64, count, Lock Operations)
|
||||
- LCK.backend.creat (uint64, count, Created locks)
|
||||
- LCK.backend.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.backend.locks (uint64, count, Lock Operations)
|
||||
- LCK.vcapace.creat (uint64, count, Created locks)
|
||||
- LCK.vcapace.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.vcapace.locks (uint64, count, Lock Operations)
|
||||
- LCK.nbusyobj.creat (uint64, count, Created locks)
|
||||
- LCK.nbusyobj.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.nbusyobj.locks (uint64, count, Lock Operations)
|
||||
- LCK.busyobj.creat (uint64, count, Created locks)
|
||||
- LCK.busyobj.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.busyobj.locks (uint64, count, Lock Operations)
|
||||
- LCK.mempool.creat (uint64, count, Created locks)
|
||||
- LCK.mempool.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.mempool.locks (uint64, count, Lock Operations)
|
||||
- LCK.vxid.creat (uint64, count, Created locks)
|
||||
- LCK.vxid.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.vxid.locks (uint64, count, Lock Operations)
|
||||
- LCK.pipestat.creat (uint64, count, Created locks)
|
||||
- LCK.pipestat.destroy (uint64, count, Destroyed locks)
|
||||
- LCK.pipestat.locks (uint64, count, Lock Operations)
|
||||
- MAIN.uptime (int, count, Child process uptime)
|
||||
- MAIN.sess_conn (int, count, Sessions accepted)
|
||||
- MAIN.sess_drop (int, count, Sessions dropped)
|
||||
- MAIN.sess_fail (int, count, Session accept failures)
|
||||
- MAIN.sess_pipe_overflow (int, count, Session pipe overflow)
|
||||
- MAIN.client_req_400 (int, count, Client requests received,)
|
||||
- MAIN.client_req_411 (int, count, Client requests received,)
|
||||
- MAIN.client_req_413 (int, count, Client requests received,)
|
||||
- MAIN.client_req_417 (int, count, Client requests received,)
|
||||
- MAIN.client_req (int, count, Good client requests)
|
||||
- MAIN.cache_hit (int, count, Cache hits)
|
||||
- MAIN.cache_hitpass (int, count, Cache hits for)
|
||||
- MAIN.cache_miss (int, count, Cache misses)
|
||||
- MAIN.backend_conn (int, count, Backend conn. success)
|
||||
- MAIN.backend_unhealthy (int, count, Backend conn. not)
|
||||
- MAIN.backend_busy (int, count, Backend conn. too)
|
||||
- MAIN.backend_fail (int, count, Backend conn. failures)
|
||||
- MAIN.backend_reuse (int, count, Backend conn. reuses)
|
||||
- MAIN.backend_toolate (int, count, Backend conn. was)
|
||||
- MAIN.backend_recycle (int, count, Backend conn. recycles)
|
||||
- MAIN.backend_retry (int, count, Backend conn. retry)
|
||||
- MAIN.fetch_head (int, count, Fetch no body)
|
||||
- MAIN.fetch_length (int, count, Fetch with Length)
|
||||
- MAIN.fetch_chunked (int, count, Fetch chunked)
|
||||
- MAIN.fetch_eof (int, count, Fetch EOF)
|
||||
- MAIN.fetch_bad (int, count, Fetch bad T- E)
|
||||
- MAIN.fetch_close (int, count, Fetch wanted close)
|
||||
- MAIN.fetch_oldhttp (int, count, Fetch pre HTTP/1.1)
|
||||
- MAIN.fetch_zero (int, count, Fetch zero len)
|
||||
- MAIN.fetch_1xx (int, count, Fetch no body)
|
||||
- MAIN.fetch_204 (int, count, Fetch no body)
|
||||
- MAIN.fetch_304 (int, count, Fetch no body)
|
||||
- MAIN.fetch_failed (int, count, Fetch failed (all)
|
||||
- MAIN.fetch_no_thread (int, count, Fetch failed (no)
|
||||
- MAIN.pools (int, count, Number of thread)
|
||||
- MAIN.threads (int, count, Total number of)
|
||||
- MAIN.threads_limited (int, count, Threads hit max)
|
||||
- MAIN.threads_created (int, count, Threads created)
|
||||
- MAIN.threads_destroyed (int, count, Threads destroyed)
|
||||
- MAIN.threads_failed (int, count, Thread creation failed)
|
||||
- MAIN.thread_queue_len (int, count, Length of session)
|
||||
- MAIN.busy_sleep (int, count, Number of requests)
|
||||
- MAIN.busy_wakeup (int, count, Number of requests)
|
||||
- MAIN.sess_queued (int, count, Sessions queued for)
|
||||
- MAIN.sess_dropped (int, count, Sessions dropped for)
|
||||
- MAIN.n_object (int, count, object structs made)
|
||||
- MAIN.n_vampireobject (int, count, unresurrected objects)
|
||||
- MAIN.n_objectcore (int, count, objectcore structs made)
|
||||
- MAIN.n_objecthead (int, count, objecthead structs made)
|
||||
- MAIN.n_waitinglist (int, count, waitinglist structs made)
|
||||
- MAIN.n_backend (int, count, Number of backends)
|
||||
- MAIN.n_expired (int, count, Number of expired)
|
||||
- MAIN.n_lru_nuked (int, count, Number of LRU)
|
||||
- MAIN.n_lru_moved (int, count, Number of LRU)
|
||||
- MAIN.losthdr (int, count, HTTP header overflows)
|
||||
- MAIN.s_sess (int, count, Total sessions seen)
|
||||
- MAIN.s_req (int, count, Total requests seen)
|
||||
- MAIN.s_pipe (int, count, Total pipe sessions)
|
||||
- MAIN.s_pass (int, count, Total pass- ed requests)
|
||||
- MAIN.s_fetch (int, count, Total backend fetches)
|
||||
- MAIN.s_synth (int, count, Total synthethic responses)
|
||||
- MAIN.s_req_hdrbytes (int, count, Request header bytes)
|
||||
- MAIN.s_req_bodybytes (int, count, Request body bytes)
|
||||
- MAIN.s_resp_hdrbytes (int, count, Response header bytes)
|
||||
- MAIN.s_resp_bodybytes (int, count, Response body bytes)
|
||||
- MAIN.s_pipe_hdrbytes (int, count, Pipe request header)
|
||||
- MAIN.s_pipe_in (int, count, Piped bytes from)
|
||||
- MAIN.s_pipe_out (int, count, Piped bytes to)
|
||||
- MAIN.sess_closed (int, count, Session Closed)
|
||||
- MAIN.sess_pipeline (int, count, Session Pipeline)
|
||||
- MAIN.sess_readahead (int, count, Session Read Ahead)
|
||||
- MAIN.sess_herd (int, count, Session herd)
|
||||
- MAIN.shm_records (int, count, SHM records)
|
||||
- MAIN.shm_writes (int, count, SHM writes)
|
||||
- MAIN.shm_flushes (int, count, SHM flushes due)
|
||||
- MAIN.shm_cont (int, count, SHM MTX contention)
|
||||
- MAIN.shm_cycles (int, count, SHM cycles through)
|
||||
- MAIN.sms_nreq (int, count, SMS allocator requests)
|
||||
- MAIN.sms_nobj (int, count, SMS outstanding allocations)
|
||||
- MAIN.sms_nbytes (int, count, SMS outstanding bytes)
|
||||
- MAIN.sms_balloc (int, count, SMS bytes allocated)
|
||||
- MAIN.sms_bfree (int, count, SMS bytes freed)
|
||||
- MAIN.backend_req (int, count, Backend requests made)
|
||||
- MAIN.n_vcl (int, count, Number of loaded)
|
||||
- MAIN.n_vcl_avail (int, count, Number of VCLs)
|
||||
- MAIN.n_vcl_discard (int, count, Number of discarded)
|
||||
- MAIN.bans (int, count, Count of bans)
|
||||
- MAIN.bans_completed (int, count, Number of bans)
|
||||
- MAIN.bans_obj (int, count, Number of bans)
|
||||
- MAIN.bans_req (int, count, Number of bans)
|
||||
- MAIN.bans_added (int, count, Bans added)
|
||||
- MAIN.bans_deleted (int, count, Bans deleted)
|
||||
- MAIN.bans_tested (int, count, Bans tested against)
|
||||
- MAIN.bans_obj_killed (int, count, Objects killed by)
|
||||
- MAIN.bans_lurker_tested (int, count, Bans tested against)
|
||||
- MAIN.bans_tests_tested (int, count, Ban tests tested)
|
||||
- MAIN.bans_lurker_tests_tested (int, count, Ban tests tested)
|
||||
- MAIN.bans_lurker_obj_killed (int, count, Objects killed by)
|
||||
- MAIN.bans_dups (int, count, Bans superseded by)
|
||||
- MAIN.bans_lurker_contention (int, count, Lurker gave way)
|
||||
- MAIN.bans_persisted_bytes (int, count, Bytes used by)
|
||||
- MAIN.bans_persisted_fragmentation (int, count, Extra bytes in)
|
||||
- MAIN.n_purges (int, count, Number of purge)
|
||||
- MAIN.n_obj_purged (int, count, Number of purged)
|
||||
- MAIN.exp_mailed (int, count, Number of objects)
|
||||
- MAIN.exp_received (int, count, Number of objects)
|
||||
- MAIN.hcb_nolock (int, count, HCB Lookups without)
|
||||
- MAIN.hcb_lock (int, count, HCB Lookups with)
|
||||
- MAIN.hcb_insert (int, count, HCB Inserts)
|
||||
- MAIN.esi_errors (int, count, ESI parse errors)
|
||||
- MAIN.esi_warnings (int, count, ESI parse warnings)
|
||||
- MAIN.vmods (int, count, Loaded VMODs)
|
||||
- MAIN.n_gzip (int, count, Gzip operations)
|
||||
- MAIN.n_gunzip (int, count, Gunzip operations)
|
||||
- MAIN.vsm_free (int, count, Free VSM space)
|
||||
- MAIN.vsm_used (int, count, Used VSM space)
|
||||
- MAIN.vsm_cooling (int, count, Cooling VSM space)
|
||||
- MAIN.vsm_overflow (int, count, Overflow VSM space)
|
||||
- MAIN.vsm_overflowed (int, count, Overflowed VSM space)
|
||||
- MGT.uptime (int, count, Management process uptime)
|
||||
- MGT.child_start (int, count, Child process started)
|
||||
- MGT.child_exit (int, count, Child process normal)
|
||||
- MGT.child_stop (int, count, Child process unexpected)
|
||||
- MGT.child_died (int, count, Child process died)
|
||||
- MGT.child_dump (int, count, Child process core)
|
||||
- MGT.child_panic (int, count, Child process panic)
|
||||
- MEMPOOL.vbc.live (int, count, In use)
|
||||
- MEMPOOL.vbc.pool (int, count, In Pool)
|
||||
- MEMPOOL.vbc.sz_wanted (int, count, Size requested)
|
||||
- MEMPOOL.vbc.sz_needed (int, count, Size allocated)
|
||||
- MEMPOOL.vbc.allocs (int, count, Allocations )
|
||||
- MEMPOOL.vbc.frees (int, count, Frees )
|
||||
- MEMPOOL.vbc.recycle (int, count, Recycled from pool)
|
||||
- MEMPOOL.vbc.timeout (int, count, Timed out from)
|
||||
- MEMPOOL.vbc.toosmall (int, count, Too small to)
|
||||
- MEMPOOL.vbc.surplus (int, count, Too many for)
|
||||
- MEMPOOL.vbc.randry (int, count, Pool ran dry)
|
||||
- MEMPOOL.busyobj.live (int, count, In use)
|
||||
- MEMPOOL.busyobj.pool (int, count, In Pool)
|
||||
- MEMPOOL.busyobj.sz_wanted (int, count, Size requested)
|
||||
- MEMPOOL.busyobj.sz_needed (int, count, Size allocated)
|
||||
- MEMPOOL.busyobj.allocs (int, count, Allocations )
|
||||
- MEMPOOL.busyobj.frees (int, count, Frees )
|
||||
- MEMPOOL.busyobj.recycle (int, count, Recycled from pool)
|
||||
- MEMPOOL.busyobj.timeout (int, count, Timed out from)
|
||||
- MEMPOOL.busyobj.toosmall (int, count, Too small to)
|
||||
- MEMPOOL.busyobj.surplus (int, count, Too many for)
|
||||
- MEMPOOL.busyobj.randry (int, count, Pool ran dry)
|
||||
- MEMPOOL.req0.live (int, count, In use)
|
||||
- MEMPOOL.req0.pool (int, count, In Pool)
|
||||
- MEMPOOL.req0.sz_wanted (int, count, Size requested)
|
||||
- MEMPOOL.req0.sz_needed (int, count, Size allocated)
|
||||
- MEMPOOL.req0.allocs (int, count, Allocations )
|
||||
- MEMPOOL.req0.frees (int, count, Frees )
|
||||
- MEMPOOL.req0.recycle (int, count, Recycled from pool)
|
||||
- MEMPOOL.req0.timeout (int, count, Timed out from)
|
||||
- MEMPOOL.req0.toosmall (int, count, Too small to)
|
||||
- MEMPOOL.req0.surplus (int, count, Too many for)
|
||||
- MEMPOOL.req0.randry (int, count, Pool ran dry)
|
||||
- MEMPOOL.sess0.live (int, count, In use)
|
||||
- MEMPOOL.sess0.pool (int, count, In Pool)
|
||||
- MEMPOOL.sess0.sz_wanted (int, count, Size requested)
|
||||
- MEMPOOL.sess0.sz_needed (int, count, Size allocated)
|
||||
- MEMPOOL.sess0.allocs (int, count, Allocations )
|
||||
- MEMPOOL.sess0.frees (int, count, Frees )
|
||||
- MEMPOOL.sess0.recycle (int, count, Recycled from pool)
|
||||
- MEMPOOL.sess0.timeout (int, count, Timed out from)
|
||||
- MEMPOOL.sess0.toosmall (int, count, Too small to)
|
||||
- MEMPOOL.sess0.surplus (int, count, Too many for)
|
||||
- MEMPOOL.sess0.randry (int, count, Pool ran dry)
|
||||
- MEMPOOL.req1.live (int, count, In use)
|
||||
- MEMPOOL.req1.pool (int, count, In Pool)
|
||||
- MEMPOOL.req1.sz_wanted (int, count, Size requested)
|
||||
- MEMPOOL.req1.sz_needed (int, count, Size allocated)
|
||||
- MEMPOOL.req1.allocs (int, count, Allocations )
|
||||
- MEMPOOL.req1.frees (int, count, Frees )
|
||||
- MEMPOOL.req1.recycle (int, count, Recycled from pool)
|
||||
- MEMPOOL.req1.timeout (int, count, Timed out from)
|
||||
- MEMPOOL.req1.toosmall (int, count, Too small to)
|
||||
- MEMPOOL.req1.surplus (int, count, Too many for)
|
||||
- MEMPOOL.req1.randry (int, count, Pool ran dry)
|
||||
- MEMPOOL.sess1.live (int, count, In use)
|
||||
- MEMPOOL.sess1.pool (int, count, In Pool)
|
||||
- MEMPOOL.sess1.sz_wanted (int, count, Size requested)
|
||||
- MEMPOOL.sess1.sz_needed (int, count, Size allocated)
|
||||
- MEMPOOL.sess1.allocs (int, count, Allocations )
|
||||
- MEMPOOL.sess1.frees (int, count, Frees )
|
||||
- MEMPOOL.sess1.recycle (int, count, Recycled from pool)
|
||||
- MEMPOOL.sess1.timeout (int, count, Timed out from)
|
||||
- MEMPOOL.sess1.toosmall (int, count, Too small to)
|
||||
- MEMPOOL.sess1.surplus (int, count, Too many for)
|
||||
- MEMPOOL.sess1.randry (int, count, Pool ran dry)
|
||||
- SMA.s0.c_req (int, count, Allocator requests)
|
||||
- SMA.s0.c_fail (int, count, Allocator failures)
|
||||
- SMA.s0.c_bytes (int, count, Bytes allocated)
|
||||
- SMA.s0.c_freed (int, count, Bytes freed)
|
||||
- SMA.s0.g_alloc (int, count, Allocations outstanding)
|
||||
- SMA.s0.g_bytes (int, count, Bytes outstanding)
|
||||
- SMA.s0.g_space (int, count, Bytes available)
|
||||
- SMA.Transient.c_req (int, count, Allocator requests)
|
||||
- SMA.Transient.c_fail (int, count, Allocator failures)
|
||||
- SMA.Transient.c_bytes (int, count, Bytes allocated)
|
||||
- SMA.Transient.c_freed (int, count, Bytes freed)
|
||||
- SMA.Transient.g_alloc (int, count, Allocations outstanding)
|
||||
- SMA.Transient.g_bytes (int, count, Bytes outstanding)
|
||||
- SMA.Transient.g_space (int, count, Bytes available)
|
||||
- VBE.default(127.0.0.1,,8080).vcls (int, count, VCL references)
|
||||
- VBE.default(127.0.0.1,,8080).happy (int, count, Happy health probes)
|
||||
- VBE.default(127.0.0.1,,8080).bereq_hdrbytes (int, count, Request header bytes)
|
||||
- VBE.default(127.0.0.1,,8080).bereq_bodybytes (int, count, Request body bytes)
|
||||
- VBE.default(127.0.0.1,,8080).beresp_hdrbytes (int, count, Response header bytes)
|
||||
- VBE.default(127.0.0.1,,8080).beresp_bodybytes (int, count, Response body bytes)
|
||||
- VBE.default(127.0.0.1,,8080).pipe_hdrbytes (int, count, Pipe request header)
|
||||
- VBE.default(127.0.0.1,,8080).pipe_out (int, count, Piped bytes to)
|
||||
- VBE.default(127.0.0.1,,8080).pipe_in (int, count, Piped bytes from)
|
||||
- LCK.sms.creat (int, count, Created locks)
|
||||
- LCK.sms.destroy (int, count, Destroyed locks)
|
||||
- LCK.sms.locks (int, count, Lock Operations)
|
||||
- LCK.smp.creat (int, count, Created locks)
|
||||
- LCK.smp.destroy (int, count, Destroyed locks)
|
||||
- LCK.smp.locks (int, count, Lock Operations)
|
||||
- LCK.sma.creat (int, count, Created locks)
|
||||
- LCK.sma.destroy (int, count, Destroyed locks)
|
||||
- LCK.sma.locks (int, count, Lock Operations)
|
||||
- LCK.smf.creat (int, count, Created locks)
|
||||
- LCK.smf.destroy (int, count, Destroyed locks)
|
||||
- LCK.smf.locks (int, count, Lock Operations)
|
||||
- LCK.hsl.creat (int, count, Created locks)
|
||||
- LCK.hsl.destroy (int, count, Destroyed locks)
|
||||
- LCK.hsl.locks (int, count, Lock Operations)
|
||||
- LCK.hcb.creat (int, count, Created locks)
|
||||
- LCK.hcb.destroy (int, count, Destroyed locks)
|
||||
- LCK.hcb.locks (int, count, Lock Operations)
|
||||
- LCK.hcl.creat (int, count, Created locks)
|
||||
- LCK.hcl.destroy (int, count, Destroyed locks)
|
||||
- LCK.hcl.locks (int, count, Lock Operations)
|
||||
- LCK.vcl.creat (int, count, Created locks)
|
||||
- LCK.vcl.destroy (int, count, Destroyed locks)
|
||||
- LCK.vcl.locks (int, count, Lock Operations)
|
||||
- LCK.sessmem.creat (int, count, Created locks)
|
||||
- LCK.sessmem.destroy (int, count, Destroyed locks)
|
||||
- LCK.sessmem.locks (int, count, Lock Operations)
|
||||
- LCK.sess.creat (int, count, Created locks)
|
||||
- LCK.sess.destroy (int, count, Destroyed locks)
|
||||
- LCK.sess.locks (int, count, Lock Operations)
|
||||
- LCK.wstat.creat (int, count, Created locks)
|
||||
- LCK.wstat.destroy (int, count, Destroyed locks)
|
||||
- LCK.wstat.locks (int, count, Lock Operations)
|
||||
- LCK.herder.creat (int, count, Created locks)
|
||||
- LCK.herder.destroy (int, count, Destroyed locks)
|
||||
- LCK.herder.locks (int, count, Lock Operations)
|
||||
- LCK.wq.creat (int, count, Created locks)
|
||||
- LCK.wq.destroy (int, count, Destroyed locks)
|
||||
- LCK.wq.locks (int, count, Lock Operations)
|
||||
- LCK.objhdr.creat (int, count, Created locks)
|
||||
- LCK.objhdr.destroy (int, count, Destroyed locks)
|
||||
- LCK.objhdr.locks (int, count, Lock Operations)
|
||||
- LCK.exp.creat (int, count, Created locks)
|
||||
- LCK.exp.destroy (int, count, Destroyed locks)
|
||||
- LCK.exp.locks (int, count, Lock Operations)
|
||||
- LCK.lru.creat (int, count, Created locks)
|
||||
- LCK.lru.destroy (int, count, Destroyed locks)
|
||||
- LCK.lru.locks (int, count, Lock Operations)
|
||||
- LCK.cli.creat (int, count, Created locks)
|
||||
- LCK.cli.destroy (int, count, Destroyed locks)
|
||||
- LCK.cli.locks (int, count, Lock Operations)
|
||||
- LCK.ban.creat (int, count, Created locks)
|
||||
- LCK.ban.destroy (int, count, Destroyed locks)
|
||||
- LCK.ban.locks (int, count, Lock Operations)
|
||||
- LCK.vbp.creat (int, count, Created locks)
|
||||
- LCK.vbp.destroy (int, count, Destroyed locks)
|
||||
- LCK.vbp.locks (int, count, Lock Operations)
|
||||
- LCK.backend.creat (int, count, Created locks)
|
||||
- LCK.backend.destroy (int, count, Destroyed locks)
|
||||
- LCK.backend.locks (int, count, Lock Operations)
|
||||
- LCK.vcapace.creat (int, count, Created locks)
|
||||
- LCK.vcapace.destroy (int, count, Destroyed locks)
|
||||
- LCK.vcapace.locks (int, count, Lock Operations)
|
||||
- LCK.nbusyobj.creat (int, count, Created locks)
|
||||
- LCK.nbusyobj.destroy (int, count, Destroyed locks)
|
||||
- LCK.nbusyobj.locks (int, count, Lock Operations)
|
||||
- LCK.busyobj.creat (int, count, Created locks)
|
||||
- LCK.busyobj.destroy (int, count, Destroyed locks)
|
||||
- LCK.busyobj.locks (int, count, Lock Operations)
|
||||
- LCK.mempool.creat (int, count, Created locks)
|
||||
- LCK.mempool.destroy (int, count, Destroyed locks)
|
||||
- LCK.mempool.locks (int, count, Lock Operations)
|
||||
- LCK.vxid.creat (int, count, Created locks)
|
||||
- LCK.vxid.destroy (int, count, Destroyed locks)
|
||||
- LCK.vxid.locks (int, count, Lock Operations)
|
||||
- LCK.pipestat.creat (int, count, Created locks)
|
||||
- LCK.pipestat.destroy (int, count, Destroyed locks)
|
||||
- LCK.pipestat.locks (int, count, Lock Operations)
|
||||
|
||||
|
||||
### Tags:
|
||||
|
||||
@@ -10,36 +10,33 @@ import (
|
||||
"os/exec"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/filter"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"time"
|
||||
)
|
||||
|
||||
type runner func(cmdName string) (*bytes.Buffer, error)
|
||||
const (
|
||||
kwAll = "all"
|
||||
)
|
||||
|
||||
// Varnish is used to store configuration values
|
||||
type Varnish struct {
|
||||
Stats []string
|
||||
Binary string
|
||||
|
||||
filter filter.Filter
|
||||
run runner
|
||||
}
|
||||
|
||||
var defaultStats = []string{"MAIN.cache_hit", "MAIN.cache_miss", "MAIN.uptime"}
|
||||
var defaultBinary = "/usr/bin/varnishstat"
|
||||
|
||||
var sampleConfig = `
|
||||
var varnishSampleConfig = `
|
||||
## The default location of the varnishstat binary can be overridden with:
|
||||
binary = "/usr/bin/varnishstat"
|
||||
|
||||
## By default, telegraf gather stats for 3 metric points.
|
||||
## Setting stats will override the defaults shown below.
|
||||
## Glob matching can be used, ie, stats = ["MAIN.*"]
|
||||
## stats may also be set to ["*"], which will collect all stats
|
||||
## stats may also be set to ["all"], which will collect all stats
|
||||
stats = ["MAIN.cache_hit", "MAIN.cache_miss", "MAIN.uptime"]
|
||||
`
|
||||
|
||||
@@ -49,11 +46,44 @@ func (s *Varnish) Description() string {
|
||||
|
||||
// SampleConfig displays configuration instructions
|
||||
func (s *Varnish) SampleConfig() string {
|
||||
return sampleConfig
|
||||
return fmt.Sprintf(varnishSampleConfig, strings.Join(defaultStats, "\",\""))
|
||||
}
|
||||
|
||||
func (s *Varnish) setDefaults() {
|
||||
if len(s.Stats) == 0 {
|
||||
s.Stats = defaultStats
|
||||
}
|
||||
|
||||
if s.Binary == "" {
|
||||
s.Binary = defaultBinary
|
||||
}
|
||||
}
|
||||
|
||||
// Builds a filter function that will indicate whether a given stat should
|
||||
// be reported
|
||||
func (s *Varnish) statsFilter() func(string) bool {
|
||||
s.setDefaults()
|
||||
|
||||
// Build a set for constant-time lookup of whether stats should be included
|
||||
filter := make(map[string]struct{})
|
||||
for _, s := range s.Stats {
|
||||
filter[s] = struct{}{}
|
||||
}
|
||||
|
||||
// Create a function that respects the kwAll by always returning true
|
||||
// if it is set
|
||||
return func(stat string) bool {
|
||||
if s.Stats[0] == kwAll {
|
||||
return true
|
||||
}
|
||||
|
||||
_, found := filter[stat]
|
||||
return found
|
||||
}
|
||||
}
|
||||
|
||||
// Shell out to varnish_stat and return the output
|
||||
func varnishRunner(cmdName string) (*bytes.Buffer, error) {
|
||||
var varnishStat = func(cmdName string) (*bytes.Buffer, error) {
|
||||
cmdArgs := []string{"-1"}
|
||||
|
||||
cmd := exec.Command(cmdName, cmdArgs...)
|
||||
@@ -74,27 +104,13 @@ func varnishRunner(cmdName string) (*bytes.Buffer, error) {
|
||||
// 'section' tag and all stats that share that prefix will be reported as fields
|
||||
// with that tag
|
||||
func (s *Varnish) Gather(acc telegraf.Accumulator) error {
|
||||
if s.filter == nil {
|
||||
var err error
|
||||
if len(s.Stats) == 0 {
|
||||
s.filter, err = filter.CompileFilter(defaultStats)
|
||||
} else {
|
||||
// legacy support, change "all" -> "*":
|
||||
if s.Stats[0] == "all" {
|
||||
s.Stats[0] = "*"
|
||||
}
|
||||
s.filter, err = filter.CompileFilter(s.Stats)
|
||||
}
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
out, err := s.run(s.Binary)
|
||||
s.setDefaults()
|
||||
out, err := varnishStat(s.Binary)
|
||||
if err != nil {
|
||||
return fmt.Errorf("error gathering metrics: %s", err)
|
||||
}
|
||||
|
||||
statsFilter := s.statsFilter()
|
||||
sectionMap := make(map[string]map[string]interface{})
|
||||
scanner := bufio.NewScanner(out)
|
||||
for scanner.Scan() {
|
||||
@@ -109,7 +125,7 @@ func (s *Varnish) Gather(acc telegraf.Accumulator) error {
|
||||
stat := cols[0]
|
||||
value := cols[1]
|
||||
|
||||
if s.filter != nil && !s.filter.Match(stat) {
|
||||
if !statsFilter(stat) {
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -122,7 +138,7 @@ func (s *Varnish) Gather(acc telegraf.Accumulator) error {
|
||||
sectionMap[section] = make(map[string]interface{})
|
||||
}
|
||||
|
||||
sectionMap[section][field], err = strconv.ParseUint(value, 10, 64)
|
||||
sectionMap[section][field], err = strconv.Atoi(value)
|
||||
if err != nil {
|
||||
fmt.Fprintf(os.Stderr, "Expected a numeric value for %s = %v\n",
|
||||
stat, value)
|
||||
@@ -144,9 +160,5 @@ func (s *Varnish) Gather(acc telegraf.Accumulator) error {
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("varnish", func() telegraf.Input {
|
||||
return &Varnish{
|
||||
run: varnishRunner,
|
||||
}
|
||||
})
|
||||
inputs.Add("varnish", func() telegraf.Input { return &Varnish{} })
|
||||
}
|
||||
|
||||
@@ -17,12 +17,38 @@ func fakeVarnishStat(output string) func(string) (*bytes.Buffer, error) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestGather(t *testing.T) {
|
||||
acc := &testutil.Accumulator{}
|
||||
v := &Varnish{
|
||||
run: fakeVarnishStat(smOutput),
|
||||
Stats: []string{"*"},
|
||||
func TestConfigsUsed(t *testing.T) {
|
||||
saved := varnishStat
|
||||
defer func() {
|
||||
varnishStat = saved
|
||||
}()
|
||||
|
||||
expecations := map[string]string{
|
||||
"": defaultBinary,
|
||||
"/foo/bar/baz": "/foo/bar/baz",
|
||||
}
|
||||
|
||||
for in, expected := range expecations {
|
||||
varnishStat = func(actual string) (*bytes.Buffer, error) {
|
||||
assert.Equal(t, expected, actual)
|
||||
return &bytes.Buffer{}, nil
|
||||
}
|
||||
|
||||
acc := &testutil.Accumulator{}
|
||||
v := &Varnish{Binary: in}
|
||||
v.Gather(acc)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGather(t *testing.T) {
|
||||
saved := varnishStat
|
||||
defer func() {
|
||||
varnishStat = saved
|
||||
}()
|
||||
varnishStat = fakeVarnishStat(smOutput)
|
||||
|
||||
acc := &testutil.Accumulator{}
|
||||
v := &Varnish{Stats: []string{"all"}}
|
||||
v.Gather(acc)
|
||||
|
||||
acc.HasMeasurement("varnish")
|
||||
@@ -34,11 +60,14 @@ func TestGather(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestParseFullOutput(t *testing.T) {
|
||||
saved := varnishStat
|
||||
defer func() {
|
||||
varnishStat = saved
|
||||
}()
|
||||
varnishStat = fakeVarnishStat(fullOutput)
|
||||
|
||||
acc := &testutil.Accumulator{}
|
||||
v := &Varnish{
|
||||
run: fakeVarnishStat(fullOutput),
|
||||
Stats: []string{"*"},
|
||||
}
|
||||
v := &Varnish{Stats: []string{"all"}}
|
||||
err := v.Gather(acc)
|
||||
|
||||
assert.NoError(t, err)
|
||||
@@ -48,24 +77,15 @@ func TestParseFullOutput(t *testing.T) {
|
||||
assert.Equal(t, 293, len(flat))
|
||||
}
|
||||
|
||||
func TestFilterSomeStats(t *testing.T) {
|
||||
acc := &testutil.Accumulator{}
|
||||
v := &Varnish{
|
||||
run: fakeVarnishStat(fullOutput),
|
||||
Stats: []string{"MGT.*", "VBE.*"},
|
||||
}
|
||||
err := v.Gather(acc)
|
||||
|
||||
assert.NoError(t, err)
|
||||
acc.HasMeasurement("varnish")
|
||||
flat := flatten(acc.Metrics)
|
||||
assert.Len(t, acc.Metrics, 2)
|
||||
assert.Equal(t, 16, len(flat))
|
||||
}
|
||||
|
||||
func TestFieldConfig(t *testing.T) {
|
||||
saved := varnishStat
|
||||
defer func() {
|
||||
varnishStat = saved
|
||||
}()
|
||||
varnishStat = fakeVarnishStat(fullOutput)
|
||||
|
||||
expect := map[string]int{
|
||||
"*": 293,
|
||||
"all": 293,
|
||||
"": 0, // default
|
||||
"MAIN.uptime": 1,
|
||||
"MEMPOOL.req0.sz_needed,MAIN.fetch_bad": 2,
|
||||
@@ -73,10 +93,7 @@ func TestFieldConfig(t *testing.T) {
|
||||
|
||||
for fieldCfg, expected := range expect {
|
||||
acc := &testutil.Accumulator{}
|
||||
v := &Varnish{
|
||||
run: fakeVarnishStat(fullOutput),
|
||||
Stats: strings.Split(fieldCfg, ","),
|
||||
}
|
||||
v := &Varnish{Stats: strings.Split(fieldCfg, ",")}
|
||||
err := v.Gather(acc)
|
||||
|
||||
assert.NoError(t, err)
|
||||
@@ -113,18 +130,18 @@ MEMPOOL.vbc.sz_wanted 88 . Size requested
|
||||
|
||||
var parsedSmOutput = map[string]map[string]interface{}{
|
||||
"MAIN": map[string]interface{}{
|
||||
"uptime": uint64(895),
|
||||
"cache_hit": uint64(95),
|
||||
"cache_miss": uint64(5),
|
||||
"uptime": 895,
|
||||
"cache_hit": 95,
|
||||
"cache_miss": 5,
|
||||
},
|
||||
"MGT": map[string]interface{}{
|
||||
"uptime": uint64(896),
|
||||
"child_start": uint64(1),
|
||||
"uptime": 896,
|
||||
"child_start": 1,
|
||||
},
|
||||
"MEMPOOL": map[string]interface{}{
|
||||
"vbc.live": uint64(0),
|
||||
"vbc.pool": uint64(10),
|
||||
"vbc.sz_wanted": uint64(88),
|
||||
"vbc.live": 0,
|
||||
"vbc.pool": 10,
|
||||
"vbc.sz_wanted": 88,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
@@ -1,294 +1,227 @@
|
||||
# ZFS plugin
|
||||
# Telegraf plugin: zfs
|
||||
|
||||
This ZFS plugin provides metrics from your ZFS filesystems. It supports ZFS on
|
||||
Linux and FreeBSD. It gets ZFS stat from `/proc/spl/kstat/zfs` on Linux and
|
||||
from `sysctl` and `zpool` on FreeBSD.
|
||||
Get ZFS stat from /proc/spl/kstat/zfs
|
||||
|
||||
### Configuration:
|
||||
# Measurements
|
||||
|
||||
```toml
|
||||
[[inputs.zfs]]
|
||||
## ZFS kstat path. Ignored on FreeBSD
|
||||
## If not specified, then default is:
|
||||
# kstatPath = "/proc/spl/kstat/zfs"
|
||||
Meta:
|
||||
|
||||
## By default, telegraf gather all zfs stats
|
||||
## If not specified, then default is:
|
||||
# kstatMetrics = ["arcstats", "zfetchstats", "vdev_cache_stats"]
|
||||
- tags: `pools=POOL1::POOL2`
|
||||
|
||||
## By default, don't gather zpool stats
|
||||
# poolMetrics = false
|
||||
```
|
||||
Measurement names:
|
||||
|
||||
### Measurements & Fields:
|
||||
|
||||
By default this plugin collects metrics about **Arc**, **Zfetch**, and
|
||||
**Vdev cache**. All these metrics are either counters or measure sizes
|
||||
in bytes. These metrics will be in the `zfs` measurement with the field
|
||||
names listed bellow.
|
||||
|
||||
If `poolMetrics` is enabled then additional metrics will be gathered for
|
||||
each pool.
|
||||
|
||||
- zfs
|
||||
With fields listed bellow.
|
||||
|
||||
#### Arc Stats
|
||||
|
||||
- arcstats_allocated (FreeBSD only)
|
||||
- arcstats_anon_evict_data (Linux only)
|
||||
- arcstats_anon_evict_metadata (Linux only)
|
||||
- arcstats_anon_evictable_data (FreeBSD only)
|
||||
- arcstats_anon_evictable_metadata (FreeBSD only)
|
||||
- arcstats_anon_size
|
||||
- arcstats_arc_loaned_bytes (Linux only)
|
||||
- arcstats_arc_meta_limit
|
||||
- arcstats_arc_meta_max
|
||||
- arcstats_arc_meta_min (FreeBSD only)
|
||||
- arcstats_arc_meta_used
|
||||
- arcstats_arc_no_grow (Linux only)
|
||||
- arcstats_arc_prune (Linux only)
|
||||
- arcstats_arc_tempreserve (Linux only)
|
||||
- arcstats_c
|
||||
- arcstats_c_max
|
||||
- arcstats_c_min
|
||||
- arcstats_data_size
|
||||
- arcstats_deleted
|
||||
- arcstats_hits
|
||||
- arcstats_misses
|
||||
- arcstats_demand_data_hits
|
||||
- arcstats_demand_data_misses
|
||||
- arcstats_demand_hit_predictive_prefetch (FreeBSD only)
|
||||
- arcstats_demand_metadata_hits
|
||||
- arcstats_demand_metadata_misses
|
||||
- arcstats_duplicate_buffers
|
||||
- arcstats_duplicate_buffers_size
|
||||
- arcstats_duplicate_reads
|
||||
- arcstats_evict_l2_cached
|
||||
- arcstats_evict_l2_eligible
|
||||
- arcstats_evict_l2_ineligible
|
||||
- arcstats_evict_l2_skip (FreeBSD only)
|
||||
- arcstats_evict_not_enough (FreeBSD only)
|
||||
- arcstats_evict_skip
|
||||
- arcstats_hash_chain_max
|
||||
- arcstats_hash_chains
|
||||
- arcstats_hash_collisions
|
||||
- arcstats_hash_elements
|
||||
- arcstats_hash_elements_max
|
||||
- arcstats_hdr_size
|
||||
- arcstats_hits
|
||||
- arcstats_l2_abort_lowmem
|
||||
- arcstats_l2_asize
|
||||
- arcstats_l2_cdata_free_on_write
|
||||
- arcstats_l2_cksum_bad
|
||||
- arcstats_l2_compress_failures
|
||||
- arcstats_l2_compress_successes
|
||||
- arcstats_l2_compress_zeros
|
||||
- arcstats_l2_evict_l1cached (FreeBSD only)
|
||||
- arcstats_l2_evict_lock_retry
|
||||
- arcstats_l2_evict_reading
|
||||
- arcstats_l2_feeds
|
||||
- arcstats_l2_free_on_write
|
||||
- arcstats_l2_hdr_size
|
||||
- arcstats_l2_hits
|
||||
- arcstats_l2_io_error
|
||||
- arcstats_l2_misses
|
||||
- arcstats_l2_read_bytes
|
||||
- arcstats_l2_rw_clash
|
||||
- arcstats_l2_size
|
||||
- arcstats_l2_write_buffer_bytes_scanned (FreeBSD only)
|
||||
- arcstats_l2_write_buffer_iter (FreeBSD only)
|
||||
- arcstats_l2_write_buffer_list_iter (FreeBSD only)
|
||||
- arcstats_l2_write_buffer_list_null_iter (FreeBSD only)
|
||||
- arcstats_l2_write_bytes
|
||||
- arcstats_l2_write_full (FreeBSD only)
|
||||
- arcstats_l2_write_in_l2 (FreeBSD only)
|
||||
- arcstats_l2_write_io_in_progress (FreeBSD only)
|
||||
- arcstats_l2_write_not_cacheable (FreeBSD only)
|
||||
- arcstats_l2_write_passed_headroom (FreeBSD only)
|
||||
- arcstats_l2_write_pios (FreeBSD only)
|
||||
- arcstats_l2_write_spa_mismatch (FreeBSD only)
|
||||
- arcstats_l2_write_trylock_fail (FreeBSD only)
|
||||
- arcstats_l2_writes_done
|
||||
- arcstats_l2_writes_error
|
||||
- arcstats_l2_writes_hdr_miss (Linux only)
|
||||
- arcstats_l2_writes_lock_retry (FreeBSD only)
|
||||
- arcstats_l2_writes_sent
|
||||
- arcstats_memory_direct_count (Linux only)
|
||||
- arcstats_memory_indirect_count (Linux only)
|
||||
- arcstats_memory_throttle_count
|
||||
- arcstats_meta_size (Linux only)
|
||||
- arcstats_mfu_evict_data (Linux only)
|
||||
- arcstats_mfu_evict_metadata (Linux only)
|
||||
- arcstats_mfu_ghost_evict_data (Linux only)
|
||||
- arcstats_mfu_ghost_evict_metadata (Linux only)
|
||||
- arcstats_metadata_size (FreeBSD only)
|
||||
- arcstats_mfu_evictable_data (FreeBSD only)
|
||||
- arcstats_mfu_evictable_metadata (FreeBSD only)
|
||||
- arcstats_mfu_ghost_evictable_data (FreeBSD only)
|
||||
- arcstats_mfu_ghost_evictable_metadata (FreeBSD only)
|
||||
- arcstats_mfu_ghost_hits
|
||||
- arcstats_mfu_ghost_size
|
||||
- arcstats_mfu_hits
|
||||
- arcstats_mfu_size
|
||||
- arcstats_misses
|
||||
- arcstats_mru_evict_data (Linux only)
|
||||
- arcstats_mru_evict_metadata (Linux only)
|
||||
- arcstats_mru_ghost_evict_data (Linux only)
|
||||
- arcstats_mru_ghost_evict_metadata (Linux only)
|
||||
- arcstats_mru_evictable_data (FreeBSD only)
|
||||
- arcstats_mru_evictable_metadata (FreeBSD only)
|
||||
- arcstats_mru_ghost_evictable_data (FreeBSD only)
|
||||
- arcstats_mru_ghost_evictable_metadata (FreeBSD only)
|
||||
- arcstats_mru_ghost_hits
|
||||
- arcstats_mru_ghost_size
|
||||
- arcstats_mru_hits
|
||||
- arcstats_mru_size
|
||||
- arcstats_mutex_miss
|
||||
- arcstats_other_size
|
||||
- arcstats_p
|
||||
- arcstats_prefetch_data_hits
|
||||
- arcstats_prefetch_data_misses
|
||||
- arcstats_prefetch_metadata_hits
|
||||
- arcstats_prefetch_metadata_misses
|
||||
- arcstats_recycle_miss (Linux only)
|
||||
- arcstats_mru_hits
|
||||
- arcstats_mru_ghost_hits
|
||||
- arcstats_mfu_hits
|
||||
- arcstats_mfu_ghost_hits
|
||||
- arcstats_deleted
|
||||
- arcstats_recycle_miss
|
||||
- arcstats_mutex_miss
|
||||
- arcstats_evict_skip
|
||||
- arcstats_evict_l2_cached
|
||||
- arcstats_evict_l2_eligible
|
||||
- arcstats_evict_l2_ineligible
|
||||
- arcstats_hash_elements
|
||||
- arcstats_hash_elements_max
|
||||
- arcstats_hash_collisions
|
||||
- arcstats_hash_chains
|
||||
- arcstats_hash_chain_max
|
||||
- arcstats_p
|
||||
- arcstats_c
|
||||
- arcstats_c_min
|
||||
- arcstats_c_max
|
||||
- arcstats_size
|
||||
- arcstats_sync_wait_for_async (FreeBSD only)
|
||||
|
||||
#### Zfetch Stats
|
||||
|
||||
- zfetchstats_bogus_streams (Linux only)
|
||||
- zfetchstats_colinear_hits (Linux only)
|
||||
- zfetchstats_colinear_misses (Linux only)
|
||||
- arcstats_hdr_size
|
||||
- arcstats_data_size
|
||||
- arcstats_meta_size
|
||||
- arcstats_other_size
|
||||
- arcstats_anon_size
|
||||
- arcstats_anon_evict_data
|
||||
- arcstats_anon_evict_metadata
|
||||
- arcstats_mru_size
|
||||
- arcstats_mru_evict_data
|
||||
- arcstats_mru_evict_metadata
|
||||
- arcstats_mru_ghost_size
|
||||
- arcstats_mru_ghost_evict_data
|
||||
- arcstats_mru_ghost_evict_metadata
|
||||
- arcstats_mfu_size
|
||||
- arcstats_mfu_evict_data
|
||||
- arcstats_mfu_evict_metadata
|
||||
- arcstats_mfu_ghost_size
|
||||
- arcstats_mfu_ghost_evict_data
|
||||
- arcstats_mfu_ghost_evict_metadata
|
||||
- arcstats_l2_hits
|
||||
- arcstats_l2_misses
|
||||
- arcstats_l2_feeds
|
||||
- arcstats_l2_rw_clash
|
||||
- arcstats_l2_read_bytes
|
||||
- arcstats_l2_write_bytes
|
||||
- arcstats_l2_writes_sent
|
||||
- arcstats_l2_writes_done
|
||||
- arcstats_l2_writes_error
|
||||
- arcstats_l2_writes_hdr_miss
|
||||
- arcstats_l2_evict_lock_retry
|
||||
- arcstats_l2_evict_reading
|
||||
- arcstats_l2_free_on_write
|
||||
- arcstats_l2_cdata_free_on_write
|
||||
- arcstats_l2_abort_lowmem
|
||||
- arcstats_l2_cksum_bad
|
||||
- arcstats_l2_io_error
|
||||
- arcstats_l2_size
|
||||
- arcstats_l2_asize
|
||||
- arcstats_l2_hdr_size
|
||||
- arcstats_l2_compress_successes
|
||||
- arcstats_l2_compress_zeros
|
||||
- arcstats_l2_compress_failures
|
||||
- arcstats_memory_throttle_count
|
||||
- arcstats_duplicate_buffers
|
||||
- arcstats_duplicate_buffers_size
|
||||
- arcstats_duplicate_reads
|
||||
- arcstats_memory_direct_count
|
||||
- arcstats_memory_indirect_count
|
||||
- arcstats_arc_no_grow
|
||||
- arcstats_arc_tempreserve
|
||||
- arcstats_arc_loaned_bytes
|
||||
- arcstats_arc_prune
|
||||
- arcstats_arc_meta_used
|
||||
- arcstats_arc_meta_limit
|
||||
- arcstats_arc_meta_max
|
||||
- zfetchstats_hits
|
||||
- zfetchstats_max_streams (FreeBSD only)
|
||||
- zfetchstats_misses
|
||||
- zfetchstats_reclaim_failures (Linux only)
|
||||
- zfetchstats_reclaim_successes (Linux only)
|
||||
- zfetchstats_streams_noresets (Linux only)
|
||||
- zfetchstats_streams_resets (Linux only)
|
||||
- zfetchstats_stride_hits (Linux only)
|
||||
- zfetchstats_stride_misses (Linux only)
|
||||
|
||||
#### Vdev Cache Stats
|
||||
|
||||
- zfetchstats_colinear_hits
|
||||
- zfetchstats_colinear_misses
|
||||
- zfetchstats_stride_hits
|
||||
- zfetchstats_stride_misses
|
||||
- zfetchstats_reclaim_successes
|
||||
- zfetchstats_reclaim_failures
|
||||
- zfetchstats_streams_resets
|
||||
- zfetchstats_streams_noresets
|
||||
- zfetchstats_bogus_streams
|
||||
- vdev_cache_stats_delegations
|
||||
- vdev_cache_stats_hits
|
||||
- vdev_cache_stats_misses
|
||||
|
||||
#### Pool Metrics (optional)
|
||||
|
||||
On Linux:
|
||||
|
||||
- zfs_pool
|
||||
- nread (integer, )
|
||||
- nwritten (integer, )
|
||||
- reads (integer, )
|
||||
- writes (integer, )
|
||||
- wtime (integer, )
|
||||
- wlentime (integer, )
|
||||
- wupdate (integer, )
|
||||
- rtime (integer, )
|
||||
- rlentime (integer, )
|
||||
- rupdate (integer, )
|
||||
- wcnt (integer, )
|
||||
- rcnt (integer, )
|
||||
|
||||
On FreeBSD:
|
||||
|
||||
- zfs_pool
|
||||
- allocated (integer, bytes)
|
||||
- capacity (integer, bytes)
|
||||
- dedupratio (float, ratio)
|
||||
- free (integer, bytes)
|
||||
- size (integer, bytes)
|
||||
- fragmentation (integer, percent)
|
||||
|
||||
### Tags:
|
||||
|
||||
- ZFS stats (`zfs`) will have the following tag:
|
||||
- pools - A `::` concatenated list of all ZFS pools on the machine.
|
||||
|
||||
- Pool metrics (`zfs_pool`) will have the following tag:
|
||||
- pool - with the name of the pool which the metrics are for.
|
||||
- health - the health status of the pool. (FreeBSD only)
|
||||
|
||||
### Example Output:
|
||||
|
||||
```
|
||||
$ ./telegraf -config telegraf.conf -input-filter zfs -test
|
||||
* Plugin: zfs, Collection 1
|
||||
> zfs_pool,health=ONLINE,pool=zroot allocated=1578590208i,capacity=2i,dedupratio=1,fragmentation=1i,free=64456531968i,size=66035122176i 1464473103625653908
|
||||
> zfs,pools=zroot arcstats_allocated=4167764i,arcstats_anon_evictable_data=0i,arcstats_anon_evictable_metadata=0i,arcstats_anon_size=16896i,arcstats_arc_meta_limit=10485760i,arcstats_arc_meta_max=115269568i,arcstats_arc_meta_min=8388608i,arcstats_arc_meta_used=51977456i,arcstats_c=16777216i,arcstats_c_max=41943040i,arcstats_c_min=16777216i,arcstats_data_size=0i,arcstats_deleted=1699340i,arcstats_demand_data_hits=14836131i,arcstats_demand_data_misses=2842945i,arcstats_demand_hit_predictive_prefetch=0i,arcstats_demand_metadata_hits=1655006i,arcstats_demand_metadata_misses=830074i,arcstats_duplicate_buffers=0i,arcstats_duplicate_buffers_size=0i,arcstats_duplicate_reads=123i,arcstats_evict_l2_cached=0i,arcstats_evict_l2_eligible=332172623872i,arcstats_evict_l2_ineligible=6168576i,arcstats_evict_l2_skip=0i,arcstats_evict_not_enough=12189444i,arcstats_evict_skip=195190764i,arcstats_hash_chain_max=2i,arcstats_hash_chains=10i,arcstats_hash_collisions=43134i,arcstats_hash_elements=2268i,arcstats_hash_elements_max=6136i,arcstats_hdr_size=565632i,arcstats_hits=16515778i,arcstats_l2_abort_lowmem=0i,arcstats_l2_asize=0i,arcstats_l2_cdata_free_on_write=0i,arcstats_l2_cksum_bad=0i,arcstats_l2_compress_failures=0i,arcstats_l2_compress_successes=0i,arcstats_l2_compress_zeros=0i,arcstats_l2_evict_l1cached=0i,arcstats_l2_evict_lock_retry=0i,arcstats_l2_evict_reading=0i,arcstats_l2_feeds=0i,arcstats_l2_free_on_write=0i,arcstats_l2_hdr_size=0i,arcstats_l2_hits=0i,arcstats_l2_io_error=0i,arcstats_l2_misses=0i,arcstats_l2_read_bytes=0i,arcstats_l2_rw_clash=0i,arcstats_l2_size=0i,arcstats_l2_write_buffer_bytes_scanned=0i,arcstats_l2_write_buffer_iter=0i,arcstats_l2_write_buffer_list_iter=0i,arcstats_l2_write_buffer_list_null_iter=0i,arcstats_l2_write_bytes=0i,arcstats_l2_write_full=0i,arcstats_l2_write_in_l2=0i,arcstats_l2_write_io_in_progress=0i,arcstats_l2_write_not_cacheable=380i,arcstats_l2_write_passed_headroom=0i,arcstats_l2_write_pios=0i,arcstats_l2_write_spa_mismatch=0i,arcstats_l2_write_trylock_fail=0i,arcstats_l2_writes_done=0i,arcstats_l2_writes_error=0i,arcstats_l2_writes_lock_retry=0i,arcstats_l2_writes_sent=0i,arcstats_memory_throttle_count=0i,arcstats_metadata_size=17014784i,arcstats_mfu_evictable_data=0i,arcstats_mfu_evictable_metadata=16384i,arcstats_mfu_ghost_evictable_data=5723648i,arcstats_mfu_ghost_evictable_metadata=10709504i,arcstats_mfu_ghost_hits=1315619i,arcstats_mfu_ghost_size=16433152i,arcstats_mfu_hits=7646611i,arcstats_mfu_size=305152i,arcstats_misses=3676993i,arcstats_mru_evictable_data=0i,arcstats_mru_evictable_metadata=0i,arcstats_mru_ghost_evictable_data=0i,arcstats_mru_ghost_evictable_metadata=80896i,arcstats_mru_ghost_hits=324250i,arcstats_mru_ghost_size=80896i,arcstats_mru_hits=8844526i,arcstats_mru_size=16693248i,arcstats_mutex_miss=354023i,arcstats_other_size=34397040i,arcstats_p=4172800i,arcstats_prefetch_data_hits=0i,arcstats_prefetch_data_misses=0i,arcstats_prefetch_metadata_hits=24641i,arcstats_prefetch_metadata_misses=3974i,arcstats_size=51977456i,arcstats_sync_wait_for_async=0i,vdev_cache_stats_delegations=779i,vdev_cache_stats_hits=323123i,vdev_cache_stats_misses=59929i,zfetchstats_hits=0i,zfetchstats_max_streams=0i,zfetchstats_misses=0i 1464473103634124908
|
||||
```
|
||||
|
||||
### Description
|
||||
|
||||
A short description for some of the metrics.
|
||||
```
|
||||
arcstats_hits
|
||||
Total amount of cache hits in the arc.
|
||||
|
||||
#### Arc Stats
|
||||
arcstats_misses
|
||||
Total amount of cache misses in the arc.
|
||||
|
||||
`arcstats_hits` Total amount of cache hits in the arc.
|
||||
arcstats_demand_data_hits
|
||||
Amount of cache hits for demand data, this is what matters (is good) for your application/share.
|
||||
|
||||
`arcstats_misses` Total amount of cache misses in the arc.
|
||||
arcstats_demand_data_misses
|
||||
Amount of cache misses for demand data, this is what matters (is bad) for your application/share.
|
||||
|
||||
`arcstats_demand_data_hits` Amount of cache hits for demand data, this is what matters (is good) for your application/share.
|
||||
arcstats_demand_metadata_hits
|
||||
Ammount of cache hits for demand metadata, this matters (is good) for getting filesystem data (ls,find,…)
|
||||
|
||||
`arcstats_demand_data_misses` Amount of cache misses for demand data, this is what matters (is bad) for your application/share.
|
||||
arcstats_demand_metadata_misses
|
||||
Ammount of cache misses for demand metadata, this matters (is bad) for getting filesystem data (ls,find,…)
|
||||
|
||||
`arcstats_demand_metadata_hits` Amount of cache hits for demand metadata, this matters (is good) for getting filesystem data (ls,find,…)
|
||||
arcstats_prefetch_data_hits
|
||||
The zfs prefetcher tried to prefetch somethin, but it was allready cached (boring)
|
||||
|
||||
`arcstats_demand_metadata_misses` Amount of cache misses for demand metadata, this matters (is bad) for getting filesystem data (ls,find,…)
|
||||
arcstats_prefetch_data_misses
|
||||
The zfs prefetcher prefetched something which was not in the cache (good job, could become a demand hit in the future)
|
||||
|
||||
`arcstats_prefetch_data_hits` The zfs prefetcher tried to prefetch something, but it was already cached (boring)
|
||||
arcstats_prefetch_metadata_hits
|
||||
Same as above, but for metadata
|
||||
|
||||
`arcstats_prefetch_data_misses` The zfs prefetcher prefetched something which was not in the cache (good job, could become a demand hit in the future)
|
||||
arcstats_prefetch_metadata_misses
|
||||
Same as above, but for metadata
|
||||
|
||||
`arcstats_prefetch_metadata_hits` Same as above, but for metadata
|
||||
arcstats_mru_hits
|
||||
Cache hit in the “most recently used cache”, we move this to the mfu cache.
|
||||
|
||||
`arcstats_prefetch_metadata_misses` Same as above, but for metadata
|
||||
arcstats_mru_ghost_hits
|
||||
Cache hit in the “most recently used ghost list” we had this item in the cache, but evicted it, maybe we should increase the mru cache size.
|
||||
|
||||
`arcstats_mru_hits` Cache hit in the “most recently used cache”, we move this to the mfu cache.
|
||||
arcstats_mfu_hits
|
||||
Cache hit in the “most freqently used cache” we move this to the begining of the mfu cache.
|
||||
|
||||
`arcstats_mru_ghost_hits` Cache hit in the “most recently used ghost list” we had this item in the cache, but evicted it, maybe we should increase the mru cache size.
|
||||
arcstats_mfu_ghost_hits
|
||||
Cache hit in the “most frequently used ghost list” we had this item in the cache, but evicted it, maybe we should increase the mfu cache size.
|
||||
|
||||
`arcstats_mfu_hits` Cache hit in the “most frequently used cache” we move this to the beginning of the mfu cache.
|
||||
arcstats_allocated
|
||||
New data is written to the cache.
|
||||
|
||||
`arcstats_mfu_ghost_hits` Cache hit in the “most frequently used ghost list” we had this item in the cache, but evicted it, maybe we should increase the mfu cache size.
|
||||
arcstats_deleted
|
||||
Old data is evicted (deleted) from the cache.
|
||||
|
||||
`arcstats_allocated` New data is written to the cache.
|
||||
arcstats_evict_l2_cached
|
||||
We evicted something from the arc, but its still cached in the l2 if we need it.
|
||||
|
||||
`arcstats_deleted` Old data is evicted (deleted) from the cache.
|
||||
arcstats_evict_l2_eligible
|
||||
We evicted something from the arc, and it’s not in the l2 this is sad. (maybe we hadn’t had enough time to store it there)
|
||||
|
||||
`arcstats_evict_l2_cached` We evicted something from the arc, but its still cached in the l2 if we need it.
|
||||
arcstats_evict_l2_ineligible
|
||||
We evicted something which cannot be stored in the l2.
|
||||
Reasons could be:
|
||||
We have multiple pools, we evicted something from a pool whithot an l2 device.
|
||||
The zfs property secondarycache.
|
||||
|
||||
`arcstats_evict_l2_eligible` We evicted something from the arc, and it’s not in the l2 this is sad. (maybe we hadn’t had enough time to store it there)
|
||||
arcstats_c
|
||||
Arc target size, this is the size the system thinks the arc should have.
|
||||
|
||||
`arcstats_evict_l2_ineligible` We evicted something which cannot be stored in the l2.
|
||||
Reasons could be:
|
||||
- We have multiple pools, we evicted something from a pool whithout an l2 device.
|
||||
- The zfs property secondary cache.
|
||||
arcstats_size
|
||||
Total size of the arc.
|
||||
|
||||
`arcstats_c` Arc target size, this is the size the system thinks the arc should have.
|
||||
arcstats_l2_hits
|
||||
Hits to the L2 cache. (It was not in the arc, but in the l2 cache)
|
||||
|
||||
`arcstats_size` Total size of the arc.
|
||||
arcstats_l2_misses
|
||||
Miss to the L2 cache. (It was not in the arc, and not in the l2 cache)
|
||||
|
||||
`arcstats_l2_hits` Hits to the L2 cache. (It was not in the arc, but in the l2 cache)
|
||||
arcstats_l2_size
|
||||
Size of the l2 cache.
|
||||
|
||||
`arcstats_l2_misses` Miss to the L2 cache. (It was not in the arc, and not in the l2 cache)
|
||||
arcstats_l2_hdr_size
|
||||
Size of the metadata in the arc (ram) used to manage (lookup if someting is in the l2) the l2 cache.
|
||||
|
||||
`arcstats_l2_size` Size of the l2 cache.
|
||||
|
||||
`arcstats_l2_hdr_size` Size of the metadata in the arc (ram) used to manage (lookup if something is in the l2) the l2 cache.
|
||||
|
||||
#### Zfetch Stats
|
||||
zfetchstats_hits
|
||||
Counts the number of cache hits, to items wich are in the cache because of the prefetcher.
|
||||
|
||||
`zfetchstats_hits` Counts the number of cache hits, to items which are in the cache because of the prefetcher.
|
||||
zfetchstats_colinear_hits
|
||||
Counts the number of cache hits, to items wich are in the cache because of the prefetcher (prefetched linear reads)
|
||||
|
||||
`zfetchstats_colinear_hits` Counts the number of cache hits, to items which are in the cache because of the prefetcher (prefetched linear reads)
|
||||
zfetchstats_stride_hits
|
||||
Counts the number of cache hits, to items wich are in the cache because of the prefetcher (prefetched stride reads)
|
||||
|
||||
`zfetchstats_stride_hits` Counts the number of cache hits, to items which are in the cache because of the prefetcher (prefetched stride reads)
|
||||
|
||||
#### Vdev Cache Stats
|
||||
|
||||
`vdev_cache_stats_hits` Hits to the vdev (device level) cache.
|
||||
vdev_cache_stats_hits
|
||||
Hits to the vdev (device level) cache.
|
||||
|
||||
vdev_cache_stats_misses
|
||||
Misses to the vdev (device level) cache.
|
||||
```
|
||||
|
||||
# Default config
|
||||
|
||||
```
|
||||
[zfs]
|
||||
# ZFS kstat path
|
||||
# If not specified, then default is:
|
||||
# kstatPath = "/proc/spl/kstat/zfs"
|
||||
#
|
||||
# By default, telegraf gather all zfs stats
|
||||
# If not specified, then default is:
|
||||
# kstatMetrics = ["arcstats", "zfetchstats", "vdev_cache_stats"]
|
||||
```
|
||||
|
||||
`vdev_cache_stats_misses` Misses to the vdev (device level) cache.
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user