Compare commits
1 Commits
0.13.0-rc3
...
0.10.3-win
| Author | SHA1 | Date | |
|---|---|---|---|
|
0074696e67 |
2
.gitattributes
vendored
2
.gitattributes
vendored
@@ -1,2 +0,0 @@
|
||||
CHANGELOG.md merge=union
|
||||
|
||||
235
CHANGELOG.md
235
CHANGELOG.md
@@ -1,246 +1,13 @@
|
||||
## v0.13 [2016-05-09]
|
||||
|
||||
### Release Notes
|
||||
|
||||
- **Breaking change** in jolokia plugin. See
|
||||
https://github.com/influxdata/telegraf/blob/master/plugins/inputs/jolokia/README.md
|
||||
for updated configuration. The plugin will now support proxy mode and will make
|
||||
POST requests.
|
||||
|
||||
- New [agent] configuration option: `metric_batch_size`. This option tells
|
||||
telegraf the maximum batch size to allow to accumulate before sending a flush
|
||||
to the configured outputs. `metric_buffer_limit` now refers to the absolute
|
||||
maximum number of metrics that will accumulate before metrics are dropped.
|
||||
|
||||
- There is no longer an option to
|
||||
`flush_buffer_when_full`, this is now the default and only behavior of telegraf.
|
||||
|
||||
- **Breaking Change**: docker plugin tags. The cont_id tag no longer exists, it
|
||||
will now be a field, and be called container_id. Additionally, cont_image and
|
||||
cont_name are being renamed to container_image and container_name.
|
||||
|
||||
- **Breaking Change**: docker plugin measurements. The `docker_cpu`, `docker_mem`,
|
||||
`docker_blkio` and `docker_net` measurements are being renamed to
|
||||
`docker_container_cpu`, `docker_container_mem`, `docker_container_blkio` and
|
||||
`docker_container_net`. Why? Because these metrics are
|
||||
specifically tracking per-container stats. The problem with per-container stats,
|
||||
in some use-cases, is that if containers are short-lived AND names are not
|
||||
kept consistent, then the series cardinality will balloon very quickly.
|
||||
So adding "container" to each metric will:
|
||||
(1) make it more clear that these metrics are per-container, and
|
||||
(2) allow users to easily drop per-container metrics if cardinality is an
|
||||
issue (`namedrop = ["docker_container_*"]`)
|
||||
|
||||
- `tagexclude` and `taginclude` are now available, which can be used to remove
|
||||
tags from measurements on inputs and outputs. See
|
||||
[the configuration doc](https://github.com/influxdata/telegraf/blob/master/docs/CONFIGURATION.md)
|
||||
for more details.
|
||||
|
||||
- **Measurement filtering:** All measurement filters now match based on glob
|
||||
only. Previously there was an undocumented behavior where filters would match
|
||||
based on _prefix_ in addition to globs. This means that a filter like
|
||||
`fielddrop = ["time_"]` will need to be changed to `fielddrop = ["time_*"]`
|
||||
|
||||
- **datadog**: measurement and field names will no longer have `_` replaced by `.`
|
||||
|
||||
- The following plugins have changed their tags to _not_ overwrite the host tag:
|
||||
- cassandra: `host -> cassandra_host`
|
||||
- disque: `host -> disque_host`
|
||||
- rethinkdb: `host -> rethinkdb_host`
|
||||
|
||||
- **Breaking Change**: The `win_perf_counters` input has been changed to
|
||||
sanitize field names, replacing `/Sec` and `/sec` with `_persec`, as well as
|
||||
spaces with underscores. This is needed because Graphite doesn't like slashes
|
||||
and spaces, and was failing to accept metrics that had them.
|
||||
The `/[sS]ec` -> `_persec` is just to make things clearer and uniform.
|
||||
|
||||
- **Breaking Change**: snmp plugin. The `host` tag of the snmp plugin has been
|
||||
changed to the `snmp_host` tag.
|
||||
|
||||
- The `disk` input plugin can now be configured with the `HOST_MOUNT_PREFIX` environment variable.
|
||||
This value is prepended to any mountpaths discovered before retrieving stats.
|
||||
It is not included on the report path. This is necessary for reporting host disk stats when running from within a container.
|
||||
|
||||
### Features
|
||||
|
||||
- [#1031](https://github.com/influxdata/telegraf/pull/1031): Jolokia plugin proxy mode. Thanks @saiello!
|
||||
- [#1017](https://github.com/influxdata/telegraf/pull/1017): taginclude and tagexclude arguments.
|
||||
- [#1015](https://github.com/influxdata/telegraf/pull/1015): Docker plugin schema refactor.
|
||||
- [#889](https://github.com/influxdata/telegraf/pull/889): Improved MySQL plugin. Thanks @maksadbek!
|
||||
- [#1060](https://github.com/influxdata/telegraf/pull/1060): TTL metrics added to MongoDB input plugin
|
||||
- [#1056](https://github.com/influxdata/telegraf/pull/1056): Don't allow inputs to overwrite host tags.
|
||||
- [#1035](https://github.com/influxdata/telegraf/issues/1035): Add `user`, `exe`, `pidfile` tags to procstat plugin.
|
||||
- [#1041](https://github.com/influxdata/telegraf/issues/1041): Add `n_cpus` field to the system plugin.
|
||||
- [#1072](https://github.com/influxdata/telegraf/pull/1072): New Input Plugin: filestat.
|
||||
- [#1066](https://github.com/influxdata/telegraf/pull/1066): Replication lag metrics for MongoDB input plugin
|
||||
- [#1086](https://github.com/influxdata/telegraf/pull/1086): Ability to specify AWS keys in config file. Thanks @johnrengleman!
|
||||
- [#1096](https://github.com/influxdata/telegraf/pull/1096): Performance refactor of running output buffers.
|
||||
- [#967](https://github.com/influxdata/telegraf/issues/967): Buffer logging improvements.
|
||||
- [#1107](https://github.com/influxdata/telegraf/issues/1107): Support lustre2 job stats. Thanks @hanleyja!
|
||||
- [#1122](https://github.com/influxdata/telegraf/pull/1122): Support setting config path through env variable and default paths.
|
||||
- [#1128](https://github.com/influxdata/telegraf/pull/1128): MongoDB jumbo chunks metric for MongoDB input plugin
|
||||
- [#1146](https://github.com/influxdata/telegraf/pull/1146): HAProxy socket support. Thanks weshmashian!
|
||||
|
||||
### Bugfixes
|
||||
|
||||
- [#1050](https://github.com/influxdata/telegraf/issues/1050): jolokia plugin - do not overwrite host tag. Thanks @saiello!
|
||||
- [#921](https://github.com/influxdata/telegraf/pull/921): mqtt_consumer stops gathering metrics. Thanks @chaton78!
|
||||
- [#1013](https://github.com/influxdata/telegraf/pull/1013): Close dead riemann output connections. Thanks @echupriyanov!
|
||||
- [#1012](https://github.com/influxdata/telegraf/pull/1012): Set default tags in test accumulator.
|
||||
- [#1024](https://github.com/influxdata/telegraf/issues/1024): Don't replace `.` with `_` in datadog output.
|
||||
- [#1058](https://github.com/influxdata/telegraf/issues/1058): Fix possible leaky TCP connections in influxdb output.
|
||||
- [#1044](https://github.com/influxdata/telegraf/pull/1044): Fix SNMP OID possible collisions. Thanks @relip
|
||||
- [#1022](https://github.com/influxdata/telegraf/issues/1022): Dont error deb/rpm install on systemd errors.
|
||||
- [#1078](https://github.com/influxdata/telegraf/issues/1078): Use default AWS credential chain.
|
||||
- [#1070](https://github.com/influxdata/telegraf/issues/1070): SQL Server input. Fix datatype conversion.
|
||||
- [#1089](https://github.com/influxdata/telegraf/issues/1089): Fix leaky TCP connections in phpfpm plugin.
|
||||
- [#914](https://github.com/influxdata/telegraf/issues/914): Telegraf can drop metrics on full buffers.
|
||||
- [#1098](https://github.com/influxdata/telegraf/issues/1098): Sanitize invalid OpenTSDB characters.
|
||||
- [#1110](https://github.com/influxdata/telegraf/pull/1110): Sanitize * to - in graphite serializer. Thanks @goodeggs!
|
||||
- [#1118](https://github.com/influxdata/telegraf/pull/1118): Sanitize Counter names for `win_perf_counters` input.
|
||||
- [#1125](https://github.com/influxdata/telegraf/pull/1125): Wrap all exec command runners with a timeout, so hung os processes don't halt Telegraf.
|
||||
- [#1113](https://github.com/influxdata/telegraf/pull/1113): Set MaxRetry and RequiredAcks defaults in Kafka output.
|
||||
- [#1090](https://github.com/influxdata/telegraf/issues/1090): [agent] and [global_tags] config sometimes not getting applied.
|
||||
- [#1133](https://github.com/influxdata/telegraf/issues/1133): Use a timeout for docker list & stat cmds.
|
||||
- [#1052](https://github.com/influxdata/telegraf/issues/1052): Docker panic fix when decode fails.
|
||||
- [#1136](https://github.com/influxdata/telegraf/pull/1136): "DELAYED" Inserts were deprecated in MySQL 5.6.6. Thanks @PierreF
|
||||
|
||||
## v0.12.1 [2016-04-14]
|
||||
|
||||
### Release Notes
|
||||
- Breaking change in the dovecot input plugin. See Features section below.
|
||||
- Graphite output templates are now supported. See
|
||||
https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md#graphite
|
||||
- Possible breaking change for the librato and graphite outputs. Telegraf will
|
||||
no longer insert field names when the field is simply named `value`. This is
|
||||
because the `value` field is redundant in the graphite/librato context.
|
||||
|
||||
### Features
|
||||
- [#1009](https://github.com/influxdata/telegraf/pull/1009): Cassandra input plugin. Thanks @subhachandrachandra!
|
||||
- [#976](https://github.com/influxdata/telegraf/pull/976): Reduce allocations in the UDP and statsd inputs.
|
||||
- [#979](https://github.com/influxdata/telegraf/pull/979): Reduce allocations in the TCP listener.
|
||||
- [#992](https://github.com/influxdata/telegraf/pull/992): Refactor allocations in TCP/UDP listeners.
|
||||
- [#935](https://github.com/influxdata/telegraf/pull/935): AWS Cloudwatch input plugin. Thanks @joshhardy & @ljosa!
|
||||
- [#943](https://github.com/influxdata/telegraf/pull/943): http_response input plugin. Thanks @Lswith!
|
||||
- [#939](https://github.com/influxdata/telegraf/pull/939): sysstat input plugin. Thanks @zbindenren!
|
||||
- [#998](https://github.com/influxdata/telegraf/pull/998): **breaking change** enabled global, user and ip queries in dovecot plugin. Thanks @mikif70!
|
||||
- [#1001](https://github.com/influxdata/telegraf/pull/1001): Graphite serializer templates.
|
||||
- [#1008](https://github.com/influxdata/telegraf/pull/1008): Adding memstats metrics to the influxdb plugin.
|
||||
|
||||
### Bugfixes
|
||||
- [#968](https://github.com/influxdata/telegraf/issues/968): Processes plugin gets unknown state when spaces are in (command name)
|
||||
- [#969](https://github.com/influxdata/telegraf/pull/969): ipmi_sensors: allow : in password. Thanks @awaw!
|
||||
- [#972](https://github.com/influxdata/telegraf/pull/972): dovecot: remove extra newline in dovecot command. Thanks @mrannanj!
|
||||
- [#645](https://github.com/influxdata/telegraf/issues/645): docker plugin i/o error on closed pipe. Thanks @tripledes!
|
||||
|
||||
## v0.12.0 [2016-04-05]
|
||||
|
||||
### Features
|
||||
- [#951](https://github.com/influxdata/telegraf/pull/951): Parse environment variables in the config file.
|
||||
- [#948](https://github.com/influxdata/telegraf/pull/948): Cleanup config file and make default package version include all plugins (but commented).
|
||||
- [#927](https://github.com/influxdata/telegraf/pull/927): Adds parsing of tags to the statsd input when using DataDog's dogstatsd extension
|
||||
- [#863](https://github.com/influxdata/telegraf/pull/863): AMQP output: allow external auth. Thanks @ekini!
|
||||
- [#707](https://github.com/influxdata/telegraf/pull/707): Improved prometheus plugin. Thanks @titilambert!
|
||||
- [#878](https://github.com/influxdata/telegraf/pull/878): Added json serializer. Thanks @ch3lo!
|
||||
- [#880](https://github.com/influxdata/telegraf/pull/880): Add the ability to specify the bearer token to the prometheus plugin. Thanks @jchauncey!
|
||||
- [#882](https://github.com/influxdata/telegraf/pull/882): Fixed SQL Server Plugin issues
|
||||
- [#849](https://github.com/influxdata/telegraf/issues/849): Adding ability to parse single values as an input data type.
|
||||
- [#844](https://github.com/influxdata/telegraf/pull/844): postgres_extensible plugin added. Thanks @menardorama!
|
||||
- [#866](https://github.com/influxdata/telegraf/pull/866): couchbase input plugin. Thanks @ljosa!
|
||||
- [#789](https://github.com/influxdata/telegraf/pull/789): Support multiple field specification and `field*` in graphite templates. Thanks @chrusty!
|
||||
- [#762](https://github.com/influxdata/telegraf/pull/762): Nagios parser for the exec plugin. Thanks @titilambert!
|
||||
- [#848](https://github.com/influxdata/telegraf/issues/848): Provide option to omit host tag from telegraf agent.
|
||||
- [#928](https://github.com/influxdata/telegraf/pull/928): Deprecating the statsd "convert_names" options, expose separator config.
|
||||
- [#919](https://github.com/influxdata/telegraf/pull/919): ipmi_sensor input plugin. Thanks @ebookbug!
|
||||
- [#945](https://github.com/influxdata/telegraf/pull/945): KAFKA output: codec, acks, and retry configuration. Thanks @framiere!
|
||||
|
||||
### Bugfixes
|
||||
- [#890](https://github.com/influxdata/telegraf/issues/890): Create TLS config even if only ssl_ca is provided.
|
||||
- [#884](https://github.com/influxdata/telegraf/issues/884): Do not call write method if there are 0 metrics to write.
|
||||
- [#898](https://github.com/influxdata/telegraf/issues/898): Put database name in quotes, fixes special characters in the database name.
|
||||
- [#656](https://github.com/influxdata/telegraf/issues/656): No longer run `lsof` on linux to get netstat data, fixes permissions issue.
|
||||
- [#907](https://github.com/influxdata/telegraf/issues/907): Fix prometheus invalid label/measurement name key.
|
||||
- [#841](https://github.com/influxdata/telegraf/issues/841): Fix memcached unix socket panic.
|
||||
- [#873](https://github.com/influxdata/telegraf/issues/873): Fix SNMP plugin sometimes not returning metrics. Thanks @titiliambert!
|
||||
- [#934](https://github.com/influxdata/telegraf/pull/934): phpfpm: Fix fcgi uri path. Thanks @rudenkovk!
|
||||
- [#805](https://github.com/influxdata/telegraf/issues/805): Kafka consumer stops gathering after i/o timeout.
|
||||
- [#959](https://github.com/influxdata/telegraf/pull/959): reduce mongodb & prometheus collection timeouts. Thanks @PierreF!
|
||||
|
||||
## v0.11.1 [2016-03-17]
|
||||
|
||||
### Release Notes
|
||||
- Primarily this release was cut to fix [#859](https://github.com/influxdata/telegraf/issues/859)
|
||||
|
||||
### Features
|
||||
- [#747](https://github.com/influxdata/telegraf/pull/747): Start telegraf on install & remove on uninstall. Thanks @pierref!
|
||||
- [#794](https://github.com/influxdata/telegraf/pull/794): Add service reload ability. Thanks @entertainyou!
|
||||
|
||||
### Bugfixes
|
||||
- [#852](https://github.com/influxdata/telegraf/issues/852): Windows zip package fix
|
||||
- [#859](https://github.com/influxdata/telegraf/issues/859): httpjson plugin panic
|
||||
|
||||
## v0.11.0 [2016-03-15]
|
||||
|
||||
### Release Notes
|
||||
|
||||
### Features
|
||||
- [#692](https://github.com/influxdata/telegraf/pull/770): Support InfluxDB retention policies
|
||||
- [#771](https://github.com/influxdata/telegraf/pull/771): Default timeouts for input plugns. Thanks @PierreF!
|
||||
- [#758](https://github.com/influxdata/telegraf/pull/758): UDP Listener input plugin, thanks @whatyouhide!
|
||||
- [#769](https://github.com/influxdata/telegraf/issues/769): httpjson plugin: allow specifying SSL configuration.
|
||||
- [#735](https://github.com/influxdata/telegraf/pull/735): SNMP Table feature. Thanks @titilambert!
|
||||
- [#754](https://github.com/influxdata/telegraf/pull/754): docker plugin: adding `docker info` metrics to output. Thanks @titilambert!
|
||||
- [#788](https://github.com/influxdata/telegraf/pull/788): -input-list and -output-list command-line options. Thanks @ebookbug!
|
||||
- [#778](https://github.com/influxdata/telegraf/pull/778): Adding a TCP input listener.
|
||||
- [#797](https://github.com/influxdata/telegraf/issues/797): Provide option for persistent MQTT consumer client sessions.
|
||||
- [#799](https://github.com/influxdata/telegraf/pull/799): Add number of threads for procstat input plugin. Thanks @titilambert!
|
||||
- [#776](https://github.com/influxdata/telegraf/pull/776): Add Zookeeper chroot option to kafka_consumer. Thanks @prune998!
|
||||
- [#811](https://github.com/influxdata/telegraf/pull/811): Add processes plugin for classifying total procs on system. Thanks @titilambert!
|
||||
- [#235](https://github.com/influxdata/telegraf/issues/235): Add number of users to the `system` input plugin.
|
||||
- [#826](https://github.com/influxdata/telegraf/pull/826): "kernel" linux plugin for /proc/stat metrics (context switches, interrupts, etc.)
|
||||
- [#847](https://github.com/influxdata/telegraf/pull/847): `ntpq`: Input plugin for running ntp query executable and gathering metrics.
|
||||
|
||||
### Bugfixes
|
||||
- [#748](https://github.com/influxdata/telegraf/issues/748): Fix sensor plugin split on ":"
|
||||
- [#722](https://github.com/influxdata/telegraf/pull/722): Librato output plugin fixes. Thanks @chrusty!
|
||||
- [#745](https://github.com/influxdata/telegraf/issues/745): Fix Telegraf toml parse panic on large config files. Thanks @titilambert!
|
||||
- [#781](https://github.com/influxdata/telegraf/pull/781): Fix mqtt_consumer username not being set. Thanks @chaton78!
|
||||
- [#786](https://github.com/influxdata/telegraf/pull/786): Fix mqtt output username not being set. Thanks @msangoi!
|
||||
- [#773](https://github.com/influxdata/telegraf/issues/773): Fix duplicate measurements in snmp plugin. Thanks @titilambert!
|
||||
- [#708](https://github.com/influxdata/telegraf/issues/708): packaging: build ARM package
|
||||
- [#713](https://github.com/influxdata/telegraf/issues/713): packaging: insecure permissions error on log directory
|
||||
- [#816](https://github.com/influxdata/telegraf/issues/816): Fix phpfpm panic if fcgi endpoint unreachable.
|
||||
- [#828](https://github.com/influxdata/telegraf/issues/828): fix net_response plugin overwriting host tag.
|
||||
- [#821](https://github.com/influxdata/telegraf/issues/821): Remove postgres password from server tag. Thanks @menardorama!
|
||||
|
||||
## v0.10.4.1
|
||||
|
||||
### Release Notes
|
||||
- Bug in the build script broke deb and rpm packages.
|
||||
|
||||
### Bugfixes
|
||||
- [#750](https://github.com/influxdata/telegraf/issues/750): deb package broken
|
||||
- [#752](https://github.com/influxdata/telegraf/issues/752): rpm package broken
|
||||
|
||||
## v0.10.4 [2016-02-24]
|
||||
|
||||
### Release Notes
|
||||
- The pass/drop parameters have been renamed to fielddrop/fieldpass parameters,
|
||||
to more accurately indicate their purpose.
|
||||
- There are also now namedrop/namepass parameters for passing/dropping based
|
||||
on the metric _name_.
|
||||
- Experimental windows builds now available.
|
||||
## v0.10.4 [unreleased]
|
||||
|
||||
### Features
|
||||
- [#727](https://github.com/influxdata/telegraf/pull/727): riak input, thanks @jcoene!
|
||||
- [#694](https://github.com/influxdata/telegraf/pull/694): DNS Query input, thanks @mjasion!
|
||||
- [#724](https://github.com/influxdata/telegraf/pull/724): username matching for procstat input, thanks @zorel!
|
||||
- [#736](https://github.com/influxdata/telegraf/pull/736): Ignore dummy filesystems from disk plugin. Thanks @PierreF!
|
||||
- [#737](https://github.com/influxdata/telegraf/pull/737): Support multiple fields for statsd input. Thanks @mattheath!
|
||||
|
||||
### Bugfixes
|
||||
- [#701](https://github.com/influxdata/telegraf/pull/701): output write count shouldnt print in quiet mode.
|
||||
- [#746](https://github.com/influxdata/telegraf/pull/746): httpjson plugin: Fix HTTP GET parameters.
|
||||
|
||||
## v0.10.3 [2016-02-18]
|
||||
|
||||
|
||||
@@ -80,7 +80,7 @@ func (s *Simple) SampleConfig() string {
|
||||
return "ok = true # indicate if everything is fine"
|
||||
}
|
||||
|
||||
func (s *Simple) Gather(acc telegraf.Accumulator) error {
|
||||
func (s *Simple) Gather(acc inputs.Accumulator) error {
|
||||
if s.Ok {
|
||||
acc.Add("state", "pretty good", nil)
|
||||
} else {
|
||||
@@ -114,7 +114,7 @@ creating the `Parser` object.
|
||||
You should also add the following to your SampleConfig() return:
|
||||
|
||||
```toml
|
||||
## Data format to consume.
|
||||
## Data format to consume. This can be "json", "influx" or "graphite"
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
|
||||
@@ -244,7 +244,7 @@ instantiating and creating the `Serializer` object.
|
||||
You should also add the following to your SampleConfig() return:
|
||||
|
||||
```toml
|
||||
## Data format to output.
|
||||
## Data format to output. This can be "influx" or "graphite"
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
|
||||
|
||||
77
Godeps
77
Godeps
@@ -1,58 +1,53 @@
|
||||
github.com/Shopify/sarama 8aadb476e66ca998f2f6bb3c993e9a2daa3666b9
|
||||
github.com/Sirupsen/logrus 219c8cb75c258c552e999735be6df753ffc7afdc
|
||||
github.com/amir/raidman 53c1b967405155bfc8758557863bf2e14f814687
|
||||
github.com/aws/aws-sdk-go 13a12060f716145019378a10e2806c174356b857
|
||||
github.com/beorn7/perks 3ac7bf7a47d159a033b107610db8a1b6575507a4
|
||||
git.eclipse.org/gitroot/paho/org.eclipse.paho.mqtt.golang.git 617c801af238c3af2d9e72c5d4a0f02edad03ce5
|
||||
github.com/Shopify/sarama d37c73f2b2bce85f7fa16b6a550d26c5372892ef
|
||||
github.com/Sirupsen/logrus f7f79f729e0fbe2fcc061db48a9ba0263f588252
|
||||
github.com/amir/raidman 6a8e089bbe32e6b907feae5ba688841974b3c339
|
||||
github.com/aws/aws-sdk-go 87b1e60a50b09e4812dee560b33a238f67305804
|
||||
github.com/beorn7/perks b965b613227fddccbfffe13eae360ed3fa822f8d
|
||||
github.com/cenkalti/backoff 4dc77674aceaabba2c7e3da25d4c823edfb73f99
|
||||
github.com/couchbase/go-couchbase cb664315a324d87d19c879d9cc67fda6be8c2ac1
|
||||
github.com/couchbase/gomemcached a5ea6356f648fec6ab89add00edd09151455b4b2
|
||||
github.com/couchbase/goutils 5823a0cbaaa9008406021dc5daf80125ea30bba6
|
||||
github.com/dancannon/gorethink e7cac92ea2bc52638791a021f212145acfedb1fc
|
||||
github.com/dancannon/gorethink 6f088135ff288deb9d5546f4c71919207f891a70
|
||||
github.com/davecgh/go-spew 5215b55f46b2b919f50a1df0eaa5886afe4e3b3d
|
||||
github.com/docker/engine-api 8924d6900370b4c7e7984be5adc61f50a80d7537
|
||||
github.com/docker/go-connections f549a9393d05688dff0992ef3efd8bbe6c628aeb
|
||||
github.com/docker/go-units 5d2041e26a699eaca682e2ea41c8f891e1060444
|
||||
github.com/eapache/go-resiliency b86b1ec0dd4209a588dc1285cdd471e73525c0b3
|
||||
github.com/eapache/queue ded5959c0d4e360646dc9e9908cff48666781367
|
||||
github.com/eclipse/paho.mqtt.golang 0f7a459f04f13a41b7ed752d47944528d4bf9a86
|
||||
github.com/go-sql-driver/mysql 1fca743146605a172a266e1654e01e5cd5669bee
|
||||
github.com/gobwas/glob d877f6352135181470c40c73ebb81aefa22115fa
|
||||
github.com/golang/protobuf 552c7b9542c194800fd493123b3798ef0a832032
|
||||
github.com/golang/snappy 427fb6fc07997f43afa32f35e850833760e489a7
|
||||
github.com/fsouza/go-dockerclient 7b651349f9479f5114913eefbfd3c4eeddd79ab4
|
||||
github.com/go-ini/ini afbd495e5aaea13597b5e14fe514ddeaa4d76fc3
|
||||
github.com/go-sql-driver/mysql 7c7f556282622f94213bc028b4d0a7b6151ba239
|
||||
github.com/golang/protobuf 6aaa8d47701fa6cf07e914ec01fde3d4a1fe79c3
|
||||
github.com/golang/snappy 723cc1e459b8eea2dea4583200fd60757d40097a
|
||||
github.com/gonuts/go-shellquote e842a11b24c6abfb3dd27af69a17f482e4b483c2
|
||||
github.com/gorilla/context 1ea25387ff6f684839d82767c1733ff4d4d15d0a
|
||||
github.com/gorilla/mux c9e326e2bdec29039a3761c07bece13133863e1e
|
||||
github.com/gorilla/context 1c83b3eabd45b6d76072b66b746c20815fb2872d
|
||||
github.com/gorilla/mux 26a6070f849969ba72b72256e9f14cf519751690
|
||||
github.com/hailocab/go-hostpool e80d13ce29ede4452c43dea11e79b9bc8a15b478
|
||||
github.com/hpcloud/tail b2940955ab8b26e19d43a43c4da0475dd81bdb56
|
||||
github.com/influxdata/config b79f6829346b8d6e78ba73544b1e1038f1f1c9da
|
||||
github.com/influxdata/influxdb e094138084855d444195b252314dfee9eae34cab
|
||||
github.com/influxdata/toml af4df43894b16e3fd2b788d01bd27ad0776ef2d0
|
||||
github.com/klauspost/crc32 19b0b332c9e4516a6370a0456e6182c3b5036720
|
||||
github.com/lib/pq e182dc4027e2ded4b19396d638610f2653295f36
|
||||
github.com/influxdata/config bae7cb98197d842374d3b8403905924094930f24
|
||||
github.com/influxdata/influxdb ef571fc104dc24b77cd3710c156cd95e5cfd7aa5
|
||||
github.com/jmespath/go-jmespath c01cf91b011868172fdcd9f41838e80c9d716264
|
||||
github.com/klauspost/crc32 999f3125931f6557b991b2f8472172bdfa578d38
|
||||
github.com/lib/pq 8ad2b298cadd691a77015666a5372eae5dbfac8f
|
||||
github.com/matttproud/golang_protobuf_extensions d0c3fe89de86839aecf2e0579c40ba3bb336a453
|
||||
github.com/miekg/dns cce6c130cdb92c752850880fd285bea1d64439dd
|
||||
github.com/mreiferson/go-snappystream 028eae7ab5c4c9e2d1cb4c4ca1e53259bbe7e504
|
||||
github.com/naoina/go-stringutil 6b638e95a32d0c1131db0e7fe83775cbea4a0d0b
|
||||
github.com/nats-io/nats b13fc9d12b0b123ebc374e6b808c6228ae4234a3
|
||||
github.com/nats-io/nuid 4f84f5f3b2786224e336af2e13dba0a0a80b76fa
|
||||
github.com/nsqio/go-nsq 0b80d6f05e15ca1930e0c5e1d540ed627e299980
|
||||
github.com/opencontainers/runc 89ab7f2ccc1e45ddf6485eaa802c35dcf321dfc8
|
||||
github.com/prometheus/client_golang 18acf9993a863f4c4b40612e19cdd243e7c86831
|
||||
github.com/naoina/toml 751171607256bb66e64c9f0220c00662420c38e9
|
||||
github.com/nats-io/nats 6a83f1a633cfbfd90aa648ac99fb38c06a8b40df
|
||||
github.com/nsqio/go-nsq 2118015c120962edc5d03325c680daf3163a8b5f
|
||||
github.com/pmezard/go-difflib 792786c7400a136282c1664665ae0a8db921c6c2
|
||||
github.com/prometheus/client_golang 67994f177195311c3ea3d4407ed0175e34a4256f
|
||||
github.com/prometheus/client_model fa8ad6fec33561be4280a8f0514318c79d7f6cb6
|
||||
github.com/prometheus/common e8eabff8812b05acf522b45fdcd725a785188e37
|
||||
github.com/prometheus/common 14ca1097bbe21584194c15e391a9dab95ad42a59
|
||||
github.com/prometheus/procfs 406e5b7bfd8201a36e2bb5f7bdae0b03380c2ce8
|
||||
github.com/samuel/go-zookeeper 218e9c81c0dd8b3b18172b2bbfad92cc7d6db55f
|
||||
github.com/shirou/gopsutil 37d89088411de59a4ef9fc340afa0e89dfcb4ea9
|
||||
github.com/shirou/gopsutil e77438504d45b9985c99a75730fe65220ceea00e
|
||||
github.com/soniah/gosnmp b1b4f885b12c5dcbd021c5cee1c904110de6db7d
|
||||
github.com/streadway/amqp b4f3ceab0337f013208d31348b578d83c0064744
|
||||
github.com/stretchr/testify 1f4a1643a57e798696635ea4c126e9127adb7d3c
|
||||
github.com/wvanbergen/kafka 46f9a1cf3f670edec492029fadded9c2d9e18866
|
||||
github.com/stretchr/objx 1a9d0bb9f541897e62256577b352fdbc1fb4fd94
|
||||
github.com/stretchr/testify f390dcf405f7b83c997eac1b06768bb9f44dec18
|
||||
github.com/wvanbergen/kafka 1a8639a45164fcc245d5c7b4bd3ccfbd1a0ffbf3
|
||||
github.com/wvanbergen/kazoo-go 0f768712ae6f76454f987c3356177e138df258f8
|
||||
github.com/zensqlmonitor/go-mssqldb ffe5510c6fa5e15e6d983210ab501c815b56b363
|
||||
golang.org/x/crypto 5dc8cb4b8a8eb076cbb5a06bc3b8682c15bdbbd3
|
||||
golang.org/x/net 6acef71eb69611914f7a30939ea9f6e194c78172
|
||||
golang.org/x/text a71fd10341b064c10f4a81ceac72bcf70f26ea34
|
||||
gopkg.in/dancannon/gorethink.v1 7d1af5be49cb5ecc7b177bf387d232050299d6ef
|
||||
golang.org/x/crypto 1f22c0103821b9390939b6776727195525381532
|
||||
golang.org/x/net 04b9de9b512f58addf28c9853d50ebef61c3953e
|
||||
golang.org/x/text 6d3c22c4525a4da167968fa2479be5524d2e8bd0
|
||||
gopkg.in/dancannon/gorethink.v1 6f088135ff288deb9d5546f4c71919207f891a70
|
||||
gopkg.in/fatih/pool.v2 cba550ebf9bce999a02e963296d4bc7a486cb715
|
||||
gopkg.in/mgo.v2 d90005c5262a3463800497ea5a89aed5fe22c886
|
||||
gopkg.in/yaml.v2 a83829b6f1293c91addabc89d0571c246397bbf4
|
||||
gopkg.in/mgo.v2 03c9f3ee4c14c8e51ee521a6a7d0425658dd6f64
|
||||
gopkg.in/yaml.v2 f7716cbe52baa25d2e9b0d0da546fcf909fc16b4
|
||||
github.com/miekg/dns e0d84d97e59bcb6561eae269c4e94d25b66822cb
|
||||
@@ -1,59 +1,56 @@
|
||||
github.com/Microsoft/go-winio 9f57cbbcbcb41dea496528872a4f0e37a4f7ae98
|
||||
github.com/Shopify/sarama 8aadb476e66ca998f2f6bb3c993e9a2daa3666b9
|
||||
github.com/Sirupsen/logrus 219c8cb75c258c552e999735be6df753ffc7afdc
|
||||
git.eclipse.org/gitroot/paho/org.eclipse.paho.mqtt.golang.git 617c801af238c3af2d9e72c5d4a0f02edad03ce5
|
||||
github.com/Shopify/sarama d37c73f2b2bce85f7fa16b6a550d26c5372892ef
|
||||
github.com/Sirupsen/logrus f7f79f729e0fbe2fcc061db48a9ba0263f588252
|
||||
github.com/StackExchange/wmi f3e2bae1e0cb5aef83e319133eabfee30013a4a5
|
||||
github.com/amir/raidman 53c1b967405155bfc8758557863bf2e14f814687
|
||||
github.com/aws/aws-sdk-go 13a12060f716145019378a10e2806c174356b857
|
||||
github.com/beorn7/perks 3ac7bf7a47d159a033b107610db8a1b6575507a4
|
||||
github.com/amir/raidman 6a8e089bbe32e6b907feae5ba688841974b3c339
|
||||
github.com/aws/aws-sdk-go 87b1e60a50b09e4812dee560b33a238f67305804
|
||||
github.com/beorn7/perks b965b613227fddccbfffe13eae360ed3fa822f8d
|
||||
github.com/cenkalti/backoff 4dc77674aceaabba2c7e3da25d4c823edfb73f99
|
||||
github.com/couchbase/go-couchbase cb664315a324d87d19c879d9cc67fda6be8c2ac1
|
||||
github.com/couchbase/gomemcached a5ea6356f648fec6ab89add00edd09151455b4b2
|
||||
github.com/couchbase/goutils 5823a0cbaaa9008406021dc5daf80125ea30bba6
|
||||
github.com/dancannon/gorethink e7cac92ea2bc52638791a021f212145acfedb1fc
|
||||
github.com/dancannon/gorethink 6f088135ff288deb9d5546f4c71919207f891a70
|
||||
github.com/davecgh/go-spew 5215b55f46b2b919f50a1df0eaa5886afe4e3b3d
|
||||
github.com/docker/engine-api 8924d6900370b4c7e7984be5adc61f50a80d7537
|
||||
github.com/docker/go-connections f549a9393d05688dff0992ef3efd8bbe6c628aeb
|
||||
github.com/docker/go-units 5d2041e26a699eaca682e2ea41c8f891e1060444
|
||||
github.com/eapache/go-resiliency b86b1ec0dd4209a588dc1285cdd471e73525c0b3
|
||||
github.com/eapache/queue ded5959c0d4e360646dc9e9908cff48666781367
|
||||
github.com/eclipse/paho.mqtt.golang 0f7a459f04f13a41b7ed752d47944528d4bf9a86
|
||||
github.com/fsouza/go-dockerclient 7b651349f9479f5114913eefbfd3c4eeddd79ab4
|
||||
github.com/go-ini/ini afbd495e5aaea13597b5e14fe514ddeaa4d76fc3
|
||||
github.com/go-ole/go-ole 50055884d646dd9434f16bbb5c9801749b9bafe4
|
||||
github.com/go-sql-driver/mysql 1fca743146605a172a266e1654e01e5cd5669bee
|
||||
github.com/golang/protobuf 552c7b9542c194800fd493123b3798ef0a832032
|
||||
github.com/golang/snappy 427fb6fc07997f43afa32f35e850833760e489a7
|
||||
github.com/go-sql-driver/mysql 7c7f556282622f94213bc028b4d0a7b6151ba239
|
||||
github.com/golang/protobuf 6aaa8d47701fa6cf07e914ec01fde3d4a1fe79c3
|
||||
github.com/golang/snappy 723cc1e459b8eea2dea4583200fd60757d40097a
|
||||
github.com/gonuts/go-shellquote e842a11b24c6abfb3dd27af69a17f482e4b483c2
|
||||
github.com/gorilla/context 1ea25387ff6f684839d82767c1733ff4d4d15d0a
|
||||
github.com/gorilla/mux c9e326e2bdec29039a3761c07bece13133863e1e
|
||||
github.com/gorilla/context 1c83b3eabd45b6d76072b66b746c20815fb2872d
|
||||
github.com/gorilla/mux 26a6070f849969ba72b72256e9f14cf519751690
|
||||
github.com/hailocab/go-hostpool e80d13ce29ede4452c43dea11e79b9bc8a15b478
|
||||
github.com/influxdata/config b79f6829346b8d6e78ba73544b1e1038f1f1c9da
|
||||
github.com/influxdata/influxdb e3fef5593c21644f2b43af55d6e17e70910b0e48
|
||||
github.com/influxdata/toml af4df43894b16e3fd2b788d01bd27ad0776ef2d0
|
||||
github.com/klauspost/crc32 19b0b332c9e4516a6370a0456e6182c3b5036720
|
||||
github.com/lib/pq e182dc4027e2ded4b19396d638610f2653295f36
|
||||
github.com/influxdata/config bae7cb98197d842374d3b8403905924094930f24
|
||||
github.com/influxdata/influxdb ef571fc104dc24b77cd3710c156cd95e5cfd7aa5
|
||||
github.com/jmespath/go-jmespath c01cf91b011868172fdcd9f41838e80c9d716264
|
||||
github.com/klauspost/crc32 999f3125931f6557b991b2f8472172bdfa578d38
|
||||
github.com/lib/pq 8ad2b298cadd691a77015666a5372eae5dbfac8f
|
||||
github.com/lxn/win 9a7734ea4db26bc593d52f6a8a957afdad39c5c1
|
||||
github.com/matttproud/golang_protobuf_extensions d0c3fe89de86839aecf2e0579c40ba3bb336a453
|
||||
github.com/miekg/dns cce6c130cdb92c752850880fd285bea1d64439dd
|
||||
github.com/miekg/dns e0d84d97e59bcb6561eae269c4e94d25b66822cb
|
||||
github.com/mreiferson/go-snappystream 028eae7ab5c4c9e2d1cb4c4ca1e53259bbe7e504
|
||||
github.com/naoina/go-stringutil 6b638e95a32d0c1131db0e7fe83775cbea4a0d0b
|
||||
github.com/nats-io/nats b13fc9d12b0b123ebc374e6b808c6228ae4234a3
|
||||
github.com/nats-io/nuid 4f84f5f3b2786224e336af2e13dba0a0a80b76fa
|
||||
github.com/nsqio/go-nsq 0b80d6f05e15ca1930e0c5e1d540ed627e299980
|
||||
github.com/prometheus/client_golang 18acf9993a863f4c4b40612e19cdd243e7c86831
|
||||
github.com/naoina/toml 751171607256bb66e64c9f0220c00662420c38e9
|
||||
github.com/nats-io/nats 6a83f1a633cfbfd90aa648ac99fb38c06a8b40df
|
||||
github.com/nsqio/go-nsq 2118015c120962edc5d03325c680daf3163a8b5f
|
||||
github.com/pmezard/go-difflib 792786c7400a136282c1664665ae0a8db921c6c2
|
||||
github.com/prometheus/client_golang 67994f177195311c3ea3d4407ed0175e34a4256f
|
||||
github.com/prometheus/client_model fa8ad6fec33561be4280a8f0514318c79d7f6cb6
|
||||
github.com/prometheus/common e8eabff8812b05acf522b45fdcd725a785188e37
|
||||
github.com/prometheus/common 14ca1097bbe21584194c15e391a9dab95ad42a59
|
||||
github.com/prometheus/procfs 406e5b7bfd8201a36e2bb5f7bdae0b03380c2ce8
|
||||
github.com/samuel/go-zookeeper 218e9c81c0dd8b3b18172b2bbfad92cc7d6db55f
|
||||
github.com/shirou/gopsutil 1f32ce1bb380845be7f5d174ac641a2c592c0c42
|
||||
github.com/shirou/gopsutil e77438504d45b9985c99a75730fe65220ceea00e
|
||||
github.com/shirou/w32 ada3ba68f000aa1b58580e45c9d308fe0b7fc5c5
|
||||
github.com/soniah/gosnmp b1b4f885b12c5dcbd021c5cee1c904110de6db7d
|
||||
github.com/streadway/amqp b4f3ceab0337f013208d31348b578d83c0064744
|
||||
github.com/stretchr/testify 1f4a1643a57e798696635ea4c126e9127adb7d3c
|
||||
github.com/wvanbergen/kafka 46f9a1cf3f670edec492029fadded9c2d9e18866
|
||||
github.com/stretchr/objx 1a9d0bb9f541897e62256577b352fdbc1fb4fd94
|
||||
github.com/stretchr/testify f390dcf405f7b83c997eac1b06768bb9f44dec18
|
||||
github.com/wvanbergen/kafka 1a8639a45164fcc245d5c7b4bd3ccfbd1a0ffbf3
|
||||
github.com/wvanbergen/kazoo-go 0f768712ae6f76454f987c3356177e138df258f8
|
||||
github.com/zensqlmonitor/go-mssqldb ffe5510c6fa5e15e6d983210ab501c815b56b363
|
||||
golang.org/x/net 6acef71eb69611914f7a30939ea9f6e194c78172
|
||||
golang.org/x/text a71fd10341b064c10f4a81ceac72bcf70f26ea34
|
||||
gopkg.in/dancannon/gorethink.v1 7d1af5be49cb5ecc7b177bf387d232050299d6ef
|
||||
golang.org/x/net 04b9de9b512f58addf28c9853d50ebef61c3953e
|
||||
golang.org/x/text 6d3c22c4525a4da167968fa2479be5524d2e8bd0
|
||||
gopkg.in/dancannon/gorethink.v1 6f088135ff288deb9d5546f4c71919207f891a70
|
||||
gopkg.in/fatih/pool.v2 cba550ebf9bce999a02e963296d4bc7a486cb715
|
||||
gopkg.in/mgo.v2 d90005c5262a3463800497ea5a89aed5fe22c886
|
||||
gopkg.in/yaml.v2 a83829b6f1293c91addabc89d0571c246397bbf4
|
||||
gopkg.in/mgo.v2 03c9f3ee4c14c8e51ee521a6a7d0425658dd6f64
|
||||
gopkg.in/yaml.v2 f7716cbe52baa25d2e9b0d0da546fcf909fc16b4
|
||||
|
||||
10
Makefile
10
Makefile
@@ -14,21 +14,21 @@ windows: prepare-windows build-windows
|
||||
|
||||
# Only run the build (no dependency grabbing)
|
||||
build:
|
||||
go install -ldflags "-X main.version=$(VERSION)" ./...
|
||||
go install -ldflags "-X main.Version=$(VERSION)" ./...
|
||||
|
||||
build-windows:
|
||||
go build -o telegraf.exe -ldflags \
|
||||
"-X main.version=$(VERSION)" \
|
||||
"-X main.Version=$(VERSION)" \
|
||||
./cmd/telegraf/telegraf.go
|
||||
|
||||
build-for-docker:
|
||||
CGO_ENABLED=0 GOOS=linux go build -installsuffix cgo -o telegraf -ldflags \
|
||||
"-s -X main.version=$(VERSION)" \
|
||||
CGO_ENABLED=0 GOOS=linux go -o telegraf -ldflags \
|
||||
"-X main.Version=$(VERSION)" \
|
||||
./cmd/telegraf/telegraf.go
|
||||
|
||||
# Build with race detector
|
||||
dev: prepare
|
||||
go build -race -ldflags "-X main.version=$(VERSION)" ./...
|
||||
go build -race -ldflags "-X main.Version=$(VERSION)" ./...
|
||||
|
||||
# run package script
|
||||
package:
|
||||
|
||||
196
README.md
196
README.md
@@ -17,15 +17,26 @@ new plugins.
|
||||
|
||||
## Installation:
|
||||
|
||||
NOTE: Telegraf 0.10.x is **not** backwards-compatible with previous versions
|
||||
of telegraf, both in the database layout and the configuration file. 0.2.x
|
||||
will continue to be supported, see below for download links.
|
||||
|
||||
For more details on the differences between Telegraf 0.2.x and 0.10.x, see
|
||||
the [release blog post](https://influxdata.com/blog/announcing-telegraf-0-10-0/).
|
||||
|
||||
### Linux deb and rpm Packages:
|
||||
|
||||
Latest:
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf_0.13.0-1_amd64.deb
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1.x86_64.rpm
|
||||
* http://get.influxdb.org/telegraf/telegraf_0.10.3-1_amd64.deb
|
||||
* http://get.influxdb.org/telegraf/telegraf-0.10.3-1.x86_64.rpm
|
||||
|
||||
Latest (arm):
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf_0.13.0-1_armhf.deb
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1.armhf.rpm
|
||||
* http://get.influxdb.org/telegraf/telegraf_0.10.3-1_arm.deb
|
||||
* http://get.influxdb.org/telegraf/telegraf-0.10.3-1.arm.rpm
|
||||
|
||||
0.2.x:
|
||||
* http://get.influxdb.org/telegraf/telegraf_0.2.4_amd64.deb
|
||||
* http://get.influxdb.org/telegraf/telegraf-0.2.4-1.x86_64.rpm
|
||||
|
||||
##### Package Instructions:
|
||||
|
||||
@@ -39,40 +50,35 @@ controlled via `systemctl [action] telegraf`
|
||||
### yum/apt Repositories:
|
||||
|
||||
There is a yum/apt repo available for the whole InfluxData stack, see
|
||||
[here](https://docs.influxdata.com/influxdb/v0.10/introduction/installation/#installation)
|
||||
for instructions on setting up the repo. Once it is configured, you will be able
|
||||
to use this repo to install & update telegraf.
|
||||
[here](https://docs.influxdata.com/influxdb/v0.9/introduction/installation/#installation)
|
||||
for instructions, replacing the `influxdb` package name with `telegraf`.
|
||||
|
||||
### Linux tarballs:
|
||||
|
||||
Latest:
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1_linux_amd64.tar.gz
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1_linux_i386.tar.gz
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1_linux_armhf.tar.gz
|
||||
* http://get.influxdb.org/telegraf/telegraf-0.10.3-1_linux_amd64.tar.gz
|
||||
* http://get.influxdb.org/telegraf/telegraf-0.10.3-1_linux_i386.tar.gz
|
||||
* http://get.influxdb.org/telegraf/telegraf-0.10.3-1_linux_arm.tar.gz
|
||||
|
||||
0.2.x:
|
||||
* http://get.influxdb.org/telegraf/telegraf_linux_amd64_0.2.4.tar.gz
|
||||
* http://get.influxdb.org/telegraf/telegraf_linux_386_0.2.4.tar.gz
|
||||
* http://get.influxdb.org/telegraf/telegraf_linux_arm_0.2.4.tar.gz
|
||||
|
||||
##### tarball Instructions:
|
||||
|
||||
To install the full directory structure with config file, run:
|
||||
|
||||
```
|
||||
sudo tar -C / -zxvf ./telegraf-0.13.0-1_linux_amd64.tar.gz
|
||||
sudo tar -C / -zxvf ./telegraf-0.10.3-1_linux_amd64.tar.gz
|
||||
```
|
||||
|
||||
To extract only the binary, run:
|
||||
|
||||
```
|
||||
tar -zxvf telegraf-0.13.0-1_linux_amd64.tar.gz --strip-components=3 ./usr/bin/telegraf
|
||||
tar -zxvf telegraf-0.10.3-1_linux_amd64.tar.gz --strip-components=3 ./usr/bin/telegraf
|
||||
```
|
||||
|
||||
### FreeBSD tarball:
|
||||
|
||||
Latest:
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1_freebsd_amd64.tar.gz
|
||||
|
||||
##### tarball Instructions:
|
||||
|
||||
See linux instructions above.
|
||||
|
||||
### Ansible Role:
|
||||
|
||||
Ansible role: https://github.com/rossmcdonald/telegraf
|
||||
@@ -84,12 +90,6 @@ brew update
|
||||
brew install telegraf
|
||||
```
|
||||
|
||||
### Windows Binaries (EXPERIMENTAL)
|
||||
|
||||
Latest:
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1_windows_amd64.zip
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1_windows_i386.zip
|
||||
|
||||
### From Source:
|
||||
|
||||
Telegraf manages dependencies via [gdm](https://github.com/sparrc/gdm),
|
||||
@@ -156,60 +156,51 @@ more information on each, please look at the directory of the same name in
|
||||
|
||||
Currently implemented sources:
|
||||
|
||||
* [aws cloudwatch](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/cloudwatch)
|
||||
* [aerospike](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/aerospike)
|
||||
* [apache](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/apache)
|
||||
* [bcache](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/bcache)
|
||||
* [cassandra](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/cassandra)
|
||||
* [couchbase](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/couchbase)
|
||||
* [couchdb](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/couchdb)
|
||||
* [disque](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/disque)
|
||||
* [dns query time](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/dns_query)
|
||||
* [docker](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/docker)
|
||||
* [dovecot](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/dovecot)
|
||||
* [elasticsearch](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/elasticsearch)
|
||||
* [exec](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/exec) (generic executable plugin, support JSON, influx, graphite and nagios)
|
||||
* [filestat](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/filestat)
|
||||
* [haproxy](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/haproxy)
|
||||
* [http_response](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/http_response)
|
||||
* [httpjson](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/httpjson) (generic JSON-emitting http service plugin)
|
||||
* [influxdb](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/influxdb)
|
||||
* [ipmi_sensor](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/ipmi_sensor)
|
||||
* [jolokia](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/jolokia)
|
||||
* [leofs](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/leofs)
|
||||
* [lustre2](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/lustre2)
|
||||
* [mailchimp](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/mailchimp)
|
||||
* [memcached](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/memcached)
|
||||
* [mesos](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/mesos)
|
||||
* [mongodb](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/mongodb)
|
||||
* [mysql](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/mysql)
|
||||
* [net_response](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/net_response)
|
||||
* [nginx](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nginx)
|
||||
* [nsq](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nsq)
|
||||
* [ntpq](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/ntpq)
|
||||
* [phpfpm](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/phpfpm)
|
||||
* [phusion passenger](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/passenger)
|
||||
* [ping](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/ping)
|
||||
* [postgresql](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/postgresql)
|
||||
* [postgresql_extensible](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/postgresql_extensible)
|
||||
* [powerdns](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/powerdns)
|
||||
* [procstat](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/procstat)
|
||||
* [prometheus](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/prometheus)
|
||||
* [puppetagent](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/puppetagent)
|
||||
* [rabbitmq](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/rabbitmq)
|
||||
* [raindrops](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/raindrops)
|
||||
* [redis](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/redis)
|
||||
* [rethinkdb](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/rethinkdb)
|
||||
* [riak](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/riak)
|
||||
* [sensors ](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/sensors) (only available if built from source)
|
||||
* [snmp](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/snmp)
|
||||
* [sql server](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/sqlserver) (microsoft)
|
||||
* [twemproxy](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/twemproxy)
|
||||
* [zfs](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/zfs)
|
||||
* [zookeeper](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/zookeeper)
|
||||
* [win_perf_counters ](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/win_perf_counters) (windows performance counters)
|
||||
* [sysstat](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/sysstat)
|
||||
* [system](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/system)
|
||||
* aerospike
|
||||
* apache
|
||||
* bcache
|
||||
* couchdb
|
||||
* disque
|
||||
* dns query time
|
||||
* docker
|
||||
* dovecot
|
||||
* elasticsearch
|
||||
* exec (generic executable plugin, support JSON, influx and graphite)
|
||||
* haproxy
|
||||
* httpjson (generic JSON-emitting http service plugin)
|
||||
* influxdb
|
||||
* jolokia
|
||||
* leofs
|
||||
* lustre2
|
||||
* mailchimp
|
||||
* memcached
|
||||
* mesos
|
||||
* mongodb
|
||||
* mysql
|
||||
* net_response
|
||||
* nginx
|
||||
* nsq
|
||||
* phpfpm
|
||||
* phusion passenger
|
||||
* ping
|
||||
* postgresql
|
||||
* powerdns
|
||||
* procstat
|
||||
* prometheus
|
||||
* puppetagent
|
||||
* rabbitmq
|
||||
* raindrops
|
||||
* redis
|
||||
* rethinkdb
|
||||
* riak
|
||||
* sensors (only available if built from source)
|
||||
* snmp
|
||||
* sql server (microsoft)
|
||||
* twemproxy
|
||||
* zfs
|
||||
* zookeeper
|
||||
* win_perf_counters (windows performance counters)
|
||||
* system
|
||||
* cpu
|
||||
* mem
|
||||
* net
|
||||
@@ -217,39 +208,34 @@ Currently implemented sources:
|
||||
* disk
|
||||
* diskio
|
||||
* swap
|
||||
* processes
|
||||
* kernel (/proc/stat)
|
||||
|
||||
Telegraf can also collect metrics via the following service plugins:
|
||||
|
||||
* [statsd](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/statsd)
|
||||
* [udp_listener](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/udp_listener)
|
||||
* [tcp_listener](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/tcp_listener)
|
||||
* [mqtt_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/mqtt_consumer)
|
||||
* [kafka_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/kafka_consumer)
|
||||
* [nats_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nats_consumer)
|
||||
* [github_webhooks](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/github_webhooks)
|
||||
* statsd
|
||||
* mqtt_consumer
|
||||
* kafka_consumer
|
||||
* nats_consumer
|
||||
* github_webhooks
|
||||
|
||||
We'll be adding support for many more over the coming months. Read on if you
|
||||
want to add support for another service or third-party API.
|
||||
|
||||
## Supported Output Plugins
|
||||
|
||||
* [influxdb](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/influxdb)
|
||||
* [amon](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/amon)
|
||||
* [amqp](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/amqp)
|
||||
* [aws kinesis](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/kinesis)
|
||||
* [aws cloudwatch](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/cloudwatch)
|
||||
* [datadog](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/datadog)
|
||||
* [file](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/file)
|
||||
* [graphite](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/graphite)
|
||||
* [kafka](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/kafka)
|
||||
* [librato](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/librato)
|
||||
* [mqtt](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/mqtt)
|
||||
* [nsq](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/nsq)
|
||||
* [opentsdb](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/opentsdb)
|
||||
* [prometheus](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/prometheus_client)
|
||||
* [riemann](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/riemann)
|
||||
* influxdb
|
||||
* amon
|
||||
* amqp
|
||||
* aws kinesis
|
||||
* aws cloudwatch
|
||||
* datadog
|
||||
* graphite
|
||||
* kafka
|
||||
* librato
|
||||
* mqtt
|
||||
* nsq
|
||||
* opentsdb
|
||||
* prometheus
|
||||
* riemann
|
||||
|
||||
## Contributing
|
||||
|
||||
|
||||
@@ -84,15 +84,18 @@ func (ac *accumulator) AddFields(
|
||||
if tags == nil {
|
||||
tags = make(map[string]string)
|
||||
}
|
||||
// Apply daemon-wide tags if set
|
||||
for k, v := range ac.defaultTags {
|
||||
tags[k] = v
|
||||
}
|
||||
// Apply plugin-wide tags if set
|
||||
for k, v := range ac.inputConfig.Tags {
|
||||
tags[k] = v
|
||||
if _, ok := tags[k]; !ok {
|
||||
tags[k] = v
|
||||
}
|
||||
}
|
||||
// Apply daemon-wide tags if set
|
||||
for k, v := range ac.defaultTags {
|
||||
if _, ok := tags[k]; !ok {
|
||||
tags[k] = v
|
||||
}
|
||||
}
|
||||
ac.inputConfig.Filter.FilterTags(tags)
|
||||
|
||||
result := make(map[string]interface{})
|
||||
for k, v := range fields {
|
||||
@@ -102,6 +105,7 @@ func (ac *accumulator) AddFields(
|
||||
continue
|
||||
}
|
||||
}
|
||||
result[k] = v
|
||||
|
||||
// Validate uint64 and float64 fields
|
||||
switch val := v.(type) {
|
||||
@@ -112,7 +116,6 @@ func (ac *accumulator) AddFields(
|
||||
} else {
|
||||
result[k] = int64(9223372036854775807)
|
||||
}
|
||||
continue
|
||||
case float64:
|
||||
// NaNs are invalid values in influxdb, skip measurement
|
||||
if math.IsNaN(val) || math.IsInf(val, 0) {
|
||||
@@ -124,8 +127,6 @@ func (ac *accumulator) AddFields(
|
||||
continue
|
||||
}
|
||||
}
|
||||
|
||||
result[k] = v
|
||||
}
|
||||
fields = nil
|
||||
if len(result) == 0 {
|
||||
@@ -167,8 +168,5 @@ func (ac *accumulator) setDefaultTags(tags map[string]string) {
|
||||
}
|
||||
|
||||
func (ac *accumulator) addDefaultTag(key, value string) {
|
||||
if ac.defaultTags == nil {
|
||||
ac.defaultTags = make(map[string]string)
|
||||
}
|
||||
ac.defaultTags[key] = value
|
||||
}
|
||||
|
||||
@@ -1,334 +0,0 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"math"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal/models"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestAdd(t *testing.T) {
|
||||
a := accumulator{}
|
||||
now := time.Now()
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
a.Add("acctest", float64(101), map[string]string{})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", now.UnixNano()),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddDefaultTags(t *testing.T) {
|
||||
a := accumulator{}
|
||||
a.addDefaultTag("default", "tag")
|
||||
now := time.Now()
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
a.Add("acctest", float64(101), map[string]string{})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest,default=tag value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test,default=tag value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test,default=tag value=101 %d", now.UnixNano()),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddFields(t *testing.T) {
|
||||
a := accumulator{}
|
||||
now := time.Now()
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"usage": float64(99),
|
||||
}
|
||||
a.AddFields("acctest", fields, map[string]string{})
|
||||
a.AddFields("acctest", fields, map[string]string{"acc": "test"})
|
||||
a.AddFields("acctest", fields, map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest usage=99")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test usage=99")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test usage=99 %d", now.UnixNano()),
|
||||
actual)
|
||||
}
|
||||
|
||||
// Test that all Inf fields get dropped, and not added to metrics channel
|
||||
func TestAddInfFields(t *testing.T) {
|
||||
inf := math.Inf(1)
|
||||
ninf := math.Inf(-1)
|
||||
|
||||
a := accumulator{}
|
||||
now := time.Now()
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"usage": inf,
|
||||
"nusage": ninf,
|
||||
}
|
||||
a.AddFields("acctest", fields, map[string]string{})
|
||||
a.AddFields("acctest", fields, map[string]string{"acc": "test"})
|
||||
a.AddFields("acctest", fields, map[string]string{"acc": "test"}, now)
|
||||
|
||||
assert.Len(t, a.metrics, 0)
|
||||
|
||||
// test that non-inf fields are kept and not dropped
|
||||
fields["notinf"] = float64(100)
|
||||
a.AddFields("acctest", fields, map[string]string{})
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest notinf=100")
|
||||
}
|
||||
|
||||
// Test that nan fields are dropped and not added
|
||||
func TestAddNaNFields(t *testing.T) {
|
||||
nan := math.NaN()
|
||||
|
||||
a := accumulator{}
|
||||
now := time.Now()
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"usage": nan,
|
||||
}
|
||||
a.AddFields("acctest", fields, map[string]string{})
|
||||
a.AddFields("acctest", fields, map[string]string{"acc": "test"})
|
||||
a.AddFields("acctest", fields, map[string]string{"acc": "test"}, now)
|
||||
|
||||
assert.Len(t, a.metrics, 0)
|
||||
|
||||
// test that non-nan fields are kept and not dropped
|
||||
fields["notnan"] = float64(100)
|
||||
a.AddFields("acctest", fields, map[string]string{})
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest notnan=100")
|
||||
}
|
||||
|
||||
func TestAddUint64Fields(t *testing.T) {
|
||||
a := accumulator{}
|
||||
now := time.Now()
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"usage": uint64(99),
|
||||
}
|
||||
a.AddFields("acctest", fields, map[string]string{})
|
||||
a.AddFields("acctest", fields, map[string]string{"acc": "test"})
|
||||
a.AddFields("acctest", fields, map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest usage=99i")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test usage=99i")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test usage=99i %d", now.UnixNano()),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddUint64Overflow(t *testing.T) {
|
||||
a := accumulator{}
|
||||
now := time.Now()
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"usage": uint64(9223372036854775808),
|
||||
}
|
||||
a.AddFields("acctest", fields, map[string]string{})
|
||||
a.AddFields("acctest", fields, map[string]string{"acc": "test"})
|
||||
a.AddFields("acctest", fields, map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest usage=9223372036854775807i")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test usage=9223372036854775807i")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test usage=9223372036854775807i %d", now.UnixNano()),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddInts(t *testing.T) {
|
||||
a := accumulator{}
|
||||
a.addDefaultTag("default", "tag")
|
||||
now := time.Now()
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
a.Add("acctest", int(101), map[string]string{})
|
||||
a.Add("acctest", int32(101), map[string]string{"acc": "test"})
|
||||
a.Add("acctest", int64(101), map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest,default=tag value=101i")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test,default=tag value=101i")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test,default=tag value=101i %d", now.UnixNano()),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddFloats(t *testing.T) {
|
||||
a := accumulator{}
|
||||
a.addDefaultTag("default", "tag")
|
||||
now := time.Now()
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
a.Add("acctest", float32(101), map[string]string{"acc": "test"})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test,default=tag value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test,default=tag value=101 %d", now.UnixNano()),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddStrings(t *testing.T) {
|
||||
a := accumulator{}
|
||||
a.addDefaultTag("default", "tag")
|
||||
now := time.Now()
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
a.Add("acctest", "test", map[string]string{"acc": "test"})
|
||||
a.Add("acctest", "foo", map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test,default=tag value=\"test\"")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test,default=tag value=\"foo\" %d", now.UnixNano()),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddBools(t *testing.T) {
|
||||
a := accumulator{}
|
||||
a.addDefaultTag("default", "tag")
|
||||
now := time.Now()
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
a.Add("acctest", true, map[string]string{"acc": "test"})
|
||||
a.Add("acctest", false, map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test,default=tag value=true")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test,default=tag value=false %d", now.UnixNano()),
|
||||
actual)
|
||||
}
|
||||
|
||||
// Test that tag filters get applied to metrics.
|
||||
func TestAccFilterTags(t *testing.T) {
|
||||
a := accumulator{}
|
||||
now := time.Now()
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
filter := internal_models.Filter{
|
||||
TagExclude: []string{"acc"},
|
||||
}
|
||||
assert.NoError(t, filter.CompileFilter())
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
a.inputConfig.Filter = filter
|
||||
|
||||
a.Add("acctest", float64(101), map[string]string{})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Contains(t, actual, "acctest value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest value=101 %d", now.UnixNano()),
|
||||
actual)
|
||||
}
|
||||
@@ -27,19 +27,17 @@ func NewAgent(config *config.Config) (*Agent, error) {
|
||||
Config: config,
|
||||
}
|
||||
|
||||
if !a.Config.Agent.OmitHostname {
|
||||
if a.Config.Agent.Hostname == "" {
|
||||
hostname, err := os.Hostname()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
a.Config.Agent.Hostname = hostname
|
||||
if a.Config.Agent.Hostname == "" {
|
||||
hostname, err := os.Hostname()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
config.Tags["host"] = a.Config.Agent.Hostname
|
||||
a.Config.Agent.Hostname = hostname
|
||||
}
|
||||
|
||||
config.Tags["host"] = a.Config.Agent.Hostname
|
||||
|
||||
return a, nil
|
||||
}
|
||||
|
||||
@@ -221,7 +219,6 @@ func (a *Agent) Test() error {
|
||||
for _, input := range a.Config.Inputs {
|
||||
acc := NewAccumulator(input.Config, metricC)
|
||||
acc.SetDebug(true)
|
||||
acc.setDefaultTags(a.Config.Tags)
|
||||
|
||||
fmt.Printf("* Plugin: %s, Collection 1\n", input.Name)
|
||||
if input.Config.Interval != 0 {
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
"github.com/stretchr/testify/assert"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
@@ -10,18 +11,8 @@ import (
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/all"
|
||||
// needing to load the outputs
|
||||
_ "github.com/influxdata/telegraf/plugins/outputs/all"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestAgent_OmitHostname(t *testing.T) {
|
||||
c := config.NewConfig()
|
||||
c.Agent.OmitHostname = true
|
||||
_, err := NewAgent(c)
|
||||
assert.NoError(t, err)
|
||||
assert.NotContains(t, c.Tags, "host")
|
||||
}
|
||||
|
||||
func TestAgent_LoadPlugin(t *testing.T) {
|
||||
c := config.NewConfig()
|
||||
c.InputFilters = []string{"mysql"}
|
||||
|
||||
@@ -4,9 +4,9 @@ machine:
|
||||
post:
|
||||
- sudo service zookeeper stop
|
||||
- go version
|
||||
- go version | grep 1.6.2 || sudo rm -rf /usr/local/go
|
||||
- wget https://storage.googleapis.com/golang/go1.6.2.linux-amd64.tar.gz
|
||||
- sudo tar -C /usr/local -xzf go1.6.2.linux-amd64.tar.gz
|
||||
- go version | grep 1.5.3 || sudo rm -rf /usr/local/go
|
||||
- wget https://storage.googleapis.com/golang/go1.5.3.linux-amd64.tar.gz
|
||||
- sudo tar -C /usr/local -xzf go1.5.3.linux-amd64.tar.gz
|
||||
- go version
|
||||
|
||||
dependencies:
|
||||
|
||||
@@ -11,9 +11,8 @@ import (
|
||||
|
||||
"github.com/influxdata/telegraf/agent"
|
||||
"github.com/influxdata/telegraf/internal/config"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/all"
|
||||
"github.com/influxdata/telegraf/plugins/outputs"
|
||||
_ "github.com/influxdata/telegraf/plugins/outputs/all"
|
||||
)
|
||||
|
||||
@@ -31,14 +30,11 @@ var fSampleConfig = flag.Bool("sample-config", false,
|
||||
var fPidfile = flag.String("pidfile", "", "file to write our pid to")
|
||||
var fInputFilters = flag.String("input-filter", "",
|
||||
"filter the inputs to enable, separator is :")
|
||||
var fInputList = flag.Bool("input-list", false,
|
||||
"print available input plugins.")
|
||||
var fOutputFilters = flag.String("output-filter", "",
|
||||
"filter the outputs to enable, separator is :")
|
||||
var fOutputList = flag.Bool("output-list", false,
|
||||
"print available output plugins.")
|
||||
var fUsage = flag.String("usage", "",
|
||||
"print usage for a plugin, ie, 'telegraf -usage mysql'")
|
||||
|
||||
var fInputFiltersLegacy = flag.String("filter", "",
|
||||
"filter the inputs to enable, separator is :")
|
||||
var fOutputFiltersLegacy = flag.String("outputfilter", "",
|
||||
@@ -46,13 +42,9 @@ var fOutputFiltersLegacy = flag.String("outputfilter", "",
|
||||
var fConfigDirectoryLegacy = flag.String("configdirectory", "",
|
||||
"directory containing additional *.conf files")
|
||||
|
||||
// Telegraf version, populated linker.
|
||||
// ie, -ldflags "-X main.version=`git describe --always --tags`"
|
||||
var (
|
||||
version string
|
||||
commit string
|
||||
branch string
|
||||
)
|
||||
// Telegraf version
|
||||
// -ldflags "-X main.Version=`git describe --always --tags`"
|
||||
var Version string
|
||||
|
||||
const usage = `Telegraf, The plugin-driven server agent for collecting and reporting metrics.
|
||||
|
||||
@@ -67,21 +59,12 @@ The flags are:
|
||||
-sample-config print out full sample configuration to stdout
|
||||
-config-directory directory containing additional *.conf files
|
||||
-input-filter filter the input plugins to enable, separator is :
|
||||
-input-list print all the plugins inputs
|
||||
-output-filter filter the output plugins to enable, separator is :
|
||||
-output-list print all the available outputs
|
||||
-usage print usage for a plugin, ie, 'telegraf -usage mysql'
|
||||
-debug print metrics as they're generated to stdout
|
||||
-quiet run in quiet mode
|
||||
-version print the version to stdout
|
||||
|
||||
In addition to the -config flag, telegraf will also load the config file from
|
||||
an environment variable or default location. Precedence is:
|
||||
1. -config flag
|
||||
2. $TELEGRAF_CONFIG_PATH environment variable
|
||||
3. $HOME/.telegraf/telegraf.conf
|
||||
4. /etc/telegraf/telegraf.conf
|
||||
|
||||
Examples:
|
||||
|
||||
# generate a telegraf config file:
|
||||
@@ -107,12 +90,13 @@ func main() {
|
||||
reload <- false
|
||||
flag.Usage = func() { usageExit(0) }
|
||||
flag.Parse()
|
||||
args := flag.Args()
|
||||
|
||||
if flag.NFlag() == 0 {
|
||||
usageExit(0)
|
||||
}
|
||||
|
||||
var inputFilters []string
|
||||
if *fInputFiltersLegacy != "" {
|
||||
fmt.Printf("WARNING '--filter' flag is deprecated, please use" +
|
||||
" '--input-filter'")
|
||||
inputFilter := strings.TrimSpace(*fInputFiltersLegacy)
|
||||
inputFilters = strings.Split(":"+inputFilter+":", ":")
|
||||
}
|
||||
@@ -123,8 +107,6 @@ func main() {
|
||||
|
||||
var outputFilters []string
|
||||
if *fOutputFiltersLegacy != "" {
|
||||
fmt.Printf("WARNING '--outputfilter' flag is deprecated, please use" +
|
||||
" '--output-filter'")
|
||||
outputFilter := strings.TrimSpace(*fOutputFiltersLegacy)
|
||||
outputFilters = strings.Split(":"+outputFilter+":", ":")
|
||||
}
|
||||
@@ -133,36 +115,8 @@ func main() {
|
||||
outputFilters = strings.Split(":"+outputFilter+":", ":")
|
||||
}
|
||||
|
||||
if len(args) > 0 {
|
||||
switch args[0] {
|
||||
case "version":
|
||||
v := fmt.Sprintf("Telegraf - version %s", version)
|
||||
fmt.Println(v)
|
||||
return
|
||||
case "config":
|
||||
config.PrintSampleConfig(inputFilters, outputFilters)
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
if *fOutputList {
|
||||
fmt.Println("Available Output Plugins:")
|
||||
for k, _ := range outputs.Outputs {
|
||||
fmt.Printf(" %s\n", k)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if *fInputList {
|
||||
fmt.Println("Available Input Plugins:")
|
||||
for k, _ := range inputs.Inputs {
|
||||
fmt.Printf(" %s\n", k)
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
if *fVersion {
|
||||
v := fmt.Sprintf("Telegraf - version %s", version)
|
||||
v := fmt.Sprintf("Telegraf - Version %s", Version)
|
||||
fmt.Println(v)
|
||||
return
|
||||
}
|
||||
@@ -181,19 +135,25 @@ func main() {
|
||||
return
|
||||
}
|
||||
|
||||
// If no other options are specified, load the config file and run.
|
||||
c := config.NewConfig()
|
||||
c.OutputFilters = outputFilters
|
||||
c.InputFilters = inputFilters
|
||||
err := c.LoadConfig(*fConfig)
|
||||
if err != nil {
|
||||
fmt.Println(err)
|
||||
var (
|
||||
c *config.Config
|
||||
err error
|
||||
)
|
||||
|
||||
if *fConfig != "" {
|
||||
c = config.NewConfig()
|
||||
c.OutputFilters = outputFilters
|
||||
c.InputFilters = inputFilters
|
||||
err = c.LoadConfig(*fConfig)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
} else {
|
||||
fmt.Println("You must specify a config file. See telegraf --help")
|
||||
os.Exit(1)
|
||||
}
|
||||
|
||||
if *fConfigDirectoryLegacy != "" {
|
||||
fmt.Printf("WARNING '--configdirectory' flag is deprecated, please use" +
|
||||
" '--config-directory'")
|
||||
err = c.LoadDirectory(*fConfigDirectoryLegacy)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
@@ -255,7 +215,7 @@ func main() {
|
||||
}
|
||||
}()
|
||||
|
||||
log.Printf("Starting Telegraf (version %s)\n", version)
|
||||
log.Printf("Starting Telegraf (version %s)\n", Version)
|
||||
log.Printf("Loaded outputs: %s", strings.Join(c.OutputNames(), " "))
|
||||
log.Printf("Loaded inputs: %s", strings.Join(c.InputNames(), " "))
|
||||
log.Printf("Tags enabled: %s", c.ListTags())
|
||||
|
||||
@@ -3,31 +3,16 @@
|
||||
## Generating a Configuration File
|
||||
|
||||
A default Telegraf config file can be generated using the -sample-config flag:
|
||||
|
||||
```
|
||||
telegraf -sample-config > telegraf.conf
|
||||
```
|
||||
`telegraf -sample-config > telegraf.conf`
|
||||
|
||||
To generate a file with specific inputs and outputs, you can use the
|
||||
-input-filter and -output-filter flags:
|
||||
|
||||
```
|
||||
telegraf -sample-config -input-filter cpu:mem:net:swap -output-filter influxdb:kafka
|
||||
```
|
||||
|
||||
You can see the latest config file with all available plugins here:
|
||||
[telegraf.conf](https://github.com/influxdata/telegraf/blob/master/etc/telegraf.conf)
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Environment variables can be used anywhere in the config file, simply prepend
|
||||
them with $. For strings the variable must be within quotes (ie, "$STR_VAR"),
|
||||
for numbers and booleans they should be plain (ie, $INT_VAR, $BOOL_VAR)
|
||||
`telegraf -sample-config -input-filter cpu:mem:net:swap -output-filter influxdb:kafka`
|
||||
|
||||
## `[global_tags]` Configuration
|
||||
|
||||
Global tags can be specified in the `[global_tags]` section of the config file
|
||||
in key="value" format. All metrics being gathered on this host will be tagged
|
||||
Global tags can be specific in the `[global_tags]` section of the config file in
|
||||
key="value" format. All metrics being gathered on this host will be tagged
|
||||
with the tags specified here.
|
||||
|
||||
## `[agent]` Configuration
|
||||
@@ -38,12 +23,8 @@ config.
|
||||
* **interval**: Default data collection interval for all inputs
|
||||
* **round_interval**: Rounds collection interval to 'interval'
|
||||
ie, if interval="10s" then always collect on :00, :10, :20, etc.
|
||||
* **metric_batch_size**: Telegraf will send metrics to output in batch of at
|
||||
most metric_batch_size metrics.
|
||||
* **metric_buffer_limit**: Telegraf will cache metric_buffer_limit metrics
|
||||
for each output, and will flush this buffer on a successful write.
|
||||
This should be a multiple of metric_batch_size and could not be less
|
||||
than 2 times metric_batch_size.
|
||||
* **collection_jitter**: Collection jitter is used to jitter
|
||||
the collection by a random amount.
|
||||
Each plugin will sleep for a random time within jitter before collecting.
|
||||
@@ -60,35 +41,9 @@ ie, a jitter of 5s and flush_interval 10s means flushes will happen every 10-15s
|
||||
* **quiet**: Run telegraf in quiet mode.
|
||||
* **hostname**: Override default hostname, if empty use os.Hostname().
|
||||
|
||||
#### Measurement Filtering
|
||||
## `[inputs.xxx]` Configuration
|
||||
|
||||
Filters can be configured per input or output, see below for examples.
|
||||
|
||||
* **namepass**: An array of strings that is used to filter metrics generated by the
|
||||
current input. Each string in the array is tested as a glob match against
|
||||
measurement names and if it matches, the field is emitted.
|
||||
* **namedrop**: The inverse of pass, if a measurement name matches, it is not emitted.
|
||||
* **fieldpass**: An array of strings that is used to filter metrics generated by the
|
||||
current input. Each string in the array is tested as a glob match against field names
|
||||
and if it matches, the field is emitted. fieldpass is not available for outputs.
|
||||
* **fielddrop**: The inverse of pass, if a field name matches, it is not emitted.
|
||||
fielddrop is not available for outputs.
|
||||
* **tagpass**: tag names and arrays of strings that are used to filter
|
||||
measurements by the current input. Each string in the array is tested as a glob
|
||||
match against the tag name, and if it matches the measurement is emitted.
|
||||
* **tagdrop**: The inverse of tagpass. If a tag matches, the measurement is not
|
||||
emitted. This is tested on measurements that have passed the tagpass test.
|
||||
* **tagexclude**: tagexclude can be used to exclude a tag from measurement(s).
|
||||
As opposed to tagdrop, which will drop an entire measurement based on it's
|
||||
tags, tagexclude simply strips the given tag keys from the measurement. This
|
||||
can be used on inputs & outputs, but it is _recommended_ to be used on inputs,
|
||||
as it is more efficient to filter out tags at the ingestion point.
|
||||
* **taginclude**: taginclude is the inverse of tagexclude. It will only include
|
||||
the tag keys in the final measurement.
|
||||
|
||||
## Input Configuration
|
||||
|
||||
Some configuration options are configurable per input:
|
||||
There are some configuration options that are configurable per input:
|
||||
|
||||
* **name_override**: Override the base name of the measurement.
|
||||
(Default is the name of the input).
|
||||
@@ -99,6 +54,24 @@ Some configuration options are configurable per input:
|
||||
global interval, but if one particular input should be run less or more often,
|
||||
you can configure that here.
|
||||
|
||||
#### Input Filters
|
||||
|
||||
There are also filters that can be configured per input:
|
||||
|
||||
* **namepass**: An array of strings that is used to filter metrics generated by the
|
||||
current input. Each string in the array is tested as a glob match against
|
||||
measurement names and if it matches, the field is emitted.
|
||||
* **namedrop**: The inverse of pass, if a measurement name matches, it is not emitted.
|
||||
* **fieldpass**: An array of strings that is used to filter metrics generated by the
|
||||
current input. Each string in the array is tested as a glob match against field names
|
||||
and if it matches, the field is emitted.
|
||||
* **fielddrop**: The inverse of pass, if a field name matches, it is not emitted.
|
||||
* **tagpass**: tag names and arrays of strings that are used to filter
|
||||
measurements by the current input. Each string in the array is tested as a glob
|
||||
match against the tag name, and if it matches the measurement is emitted.
|
||||
* **tagdrop**: The inverse of tagpass. If a tag matches, the measurement is not
|
||||
emitted. This is tested on measurements that have passed the tagpass test.
|
||||
|
||||
#### Input Configuration Examples
|
||||
|
||||
This is a full working config that will output CPU data to an InfluxDB instance
|
||||
@@ -124,7 +97,7 @@ fields which begin with `time_`.
|
||||
percpu = true
|
||||
totalcpu = false
|
||||
# filter all fields beginning with 'time_'
|
||||
fielddrop = ["time_*"]
|
||||
drop = ["time_*"]
|
||||
```
|
||||
|
||||
#### Input Config: tagpass and tagdrop
|
||||
@@ -133,7 +106,7 @@ fields which begin with `time_`.
|
||||
[[inputs.cpu]]
|
||||
percpu = true
|
||||
totalcpu = false
|
||||
fielddrop = ["cpu_time"]
|
||||
drop = ["cpu_time"]
|
||||
# Don't collect CPU data for cpu6 & cpu7
|
||||
[inputs.cpu.tagdrop]
|
||||
cpu = [ "cpu6", "cpu7" ]
|
||||
@@ -168,26 +141,12 @@ fields which begin with `time_`.
|
||||
# Drop all metrics about containers for kubelet
|
||||
[[inputs.prometheus]]
|
||||
urls = ["http://kube-node-1:4194/metrics"]
|
||||
namedrop = ["container_*"]
|
||||
namedrop = ["container_"]
|
||||
|
||||
# Only store rest client related metrics for kubelet
|
||||
[[inputs.prometheus]]
|
||||
urls = ["http://kube-node-1:4194/metrics"]
|
||||
namepass = ["rest_client_*"]
|
||||
```
|
||||
|
||||
#### Input Config: taginclude and tagexclude
|
||||
|
||||
```toml
|
||||
# Only include the "cpu" tag in the measurements for the cpu plugin.
|
||||
[[inputs.cpu]]
|
||||
percpu = true
|
||||
totalcpu = true
|
||||
taginclude = ["cpu"]
|
||||
|
||||
# Exclude the "fstype" tag from the measurements for the disk plugin.
|
||||
[[inputs.disk]]
|
||||
tagexclude = ["fstype"]
|
||||
namepass = ["rest_client_"]
|
||||
```
|
||||
|
||||
#### Input config: prefix, suffix, and override
|
||||
@@ -215,9 +174,6 @@ This will emit measurements with the name `foobar`
|
||||
This plugin will emit measurements with two additional tags: `tag1=foo` and
|
||||
`tag2=bar`
|
||||
|
||||
NOTE: Order matters, the `[inputs.cpu.tags]` table must be at the _end_ of the
|
||||
plugin definition.
|
||||
|
||||
```toml
|
||||
[[inputs.cpu]]
|
||||
percpu = false
|
||||
@@ -243,15 +199,18 @@ to avoid measurement collisions:
|
||||
percpu = true
|
||||
totalcpu = false
|
||||
name_override = "percpu_usage"
|
||||
fielddrop = ["cpu_time*"]
|
||||
drop = ["cpu_time*"]
|
||||
```
|
||||
|
||||
## Output Configuration
|
||||
## `[outputs.xxx]` Configuration
|
||||
|
||||
Telegraf also supports specifying multiple output sinks to send data to,
|
||||
configuring each output sink is different, but examples can be
|
||||
found by running `telegraf -sample-config`.
|
||||
|
||||
Outputs also support the same configurable options as inputs
|
||||
(namepass, namedrop, tagpass, tagdrop)
|
||||
|
||||
```toml
|
||||
[[outputs.influxdb]]
|
||||
urls = [ "http://localhost:8086" ]
|
||||
|
||||
@@ -1,13 +1,5 @@
|
||||
# Telegraf Input Data Formats
|
||||
|
||||
Telegraf is able to parse the following input data formats into metrics:
|
||||
|
||||
1. [InfluxDB Line Protocol](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md#influx)
|
||||
1. [JSON](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md#json)
|
||||
1. [Graphite](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md#graphite)
|
||||
1. [Value](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md#value), ie: 45 or "booyah"
|
||||
1. [Nagios](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md#nagios) (exec input only)
|
||||
|
||||
Telegraf metrics, like InfluxDB
|
||||
[points](https://docs.influxdata.com/influxdb/v0.10/write_protocols/line/),
|
||||
are a combination of four basic parts:
|
||||
@@ -39,7 +31,7 @@ example, in the exec plugin:
|
||||
## measurement name suffix (for separating different commands)
|
||||
name_suffix = "_mycollector"
|
||||
|
||||
## Data format to consume.
|
||||
## Data format to consume. This can be "json", "influx" or "graphite"
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
|
||||
@@ -51,7 +43,7 @@ example, in the exec plugin:
|
||||
Each data_format has an additional set of configuration options available, which
|
||||
I'll go over below.
|
||||
|
||||
# Influx:
|
||||
## Influx:
|
||||
|
||||
There are no additional configuration options for InfluxDB line-protocol. The
|
||||
metrics are parsed directly into Telegraf metrics.
|
||||
@@ -66,28 +58,23 @@ metrics are parsed directly into Telegraf metrics.
|
||||
## measurement name suffix (for separating different commands)
|
||||
name_suffix = "_mycollector"
|
||||
|
||||
## Data format to consume.
|
||||
## Data format to consume. This can be "json", "influx" or "graphite"
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
|
||||
data_format = "influx"
|
||||
```
|
||||
|
||||
# JSON:
|
||||
## JSON:
|
||||
|
||||
The JSON data format flattens JSON into metric _fields_.
|
||||
NOTE: Only numerical values are converted to fields, and they are converted
|
||||
into a float. strings are ignored unless specified as a tag_key (see below).
|
||||
|
||||
So for example, this JSON:
|
||||
The JSON data format flattens JSON into metric _fields_. For example, this JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"a": 5,
|
||||
"b": {
|
||||
"c": 6
|
||||
},
|
||||
"ignored": "I'm a string"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
@@ -116,7 +103,7 @@ For example, if you had this configuration:
|
||||
## measurement name suffix (for separating different commands)
|
||||
name_suffix = "_mycollector"
|
||||
|
||||
## Data format to consume.
|
||||
## Data format to consume. This can be "json", "influx" or "graphite"
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
|
||||
@@ -147,43 +134,7 @@ Your Telegraf metrics would get tagged with "my_tag_1"
|
||||
exec_mycollector,my_tag_1=foo a=5,b_c=6
|
||||
```
|
||||
|
||||
# Value:
|
||||
|
||||
The "value" data format translates single values into Telegraf metrics. This
|
||||
is done by assigning a measurement name and setting a single field ("value")
|
||||
as the parsed metric.
|
||||
|
||||
#### Value Configuration:
|
||||
|
||||
You **must** tell Telegraf what type of metric to collect by using the
|
||||
`data_type` configuration option. Available options are:
|
||||
|
||||
1. integer
|
||||
2. float or long
|
||||
3. string
|
||||
4. boolean
|
||||
|
||||
**Note:** It is also recommended that you set `name_override` to a measurement
|
||||
name that makes sense for your metric, otherwise it will just be set to the
|
||||
name of the plugin.
|
||||
|
||||
```toml
|
||||
[[inputs.exec]]
|
||||
## Commands array
|
||||
commands = ["cat /proc/sys/kernel/random/entropy_avail"]
|
||||
|
||||
## override the default metric name of "exec"
|
||||
name_override = "entropy_available"
|
||||
|
||||
## Data format to consume.
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
|
||||
data_format = "value"
|
||||
data_type = "integer" # required
|
||||
```
|
||||
|
||||
# Graphite:
|
||||
## Graphite:
|
||||
|
||||
The Graphite data format translates graphite _dot_ buckets directly into
|
||||
telegraf measurement names, with a single value field, and without any tags. For
|
||||
@@ -230,32 +181,17 @@ So the following template:
|
||||
|
||||
```toml
|
||||
templates = [
|
||||
"measurement.measurement.field.field.region"
|
||||
"measurement.measurement.field.region"
|
||||
]
|
||||
```
|
||||
|
||||
would result in the following Graphite -> Telegraf transformation.
|
||||
|
||||
```
|
||||
cpu.usage.idle.percent.us-west 100
|
||||
=> cpu_usage,region=us-west idle_percent=100
|
||||
cpu.usage.idle.us-west 100
|
||||
=> cpu_usage,region=us-west idle=100
|
||||
```
|
||||
|
||||
The field key can also be derived from the second "half" of the input metric-name by specifying ```field*```:
|
||||
```toml
|
||||
templates = [
|
||||
"measurement.measurement.region.field*"
|
||||
]
|
||||
```
|
||||
|
||||
would result in the following Graphite -> Telegraf transformation.
|
||||
|
||||
```
|
||||
cpu.usage.us-west.idle.percentage 100
|
||||
=> cpu_usage,region=us-west idle_percentage=100
|
||||
```
|
||||
(This cannot be used in conjunction with "measurement*"!)
|
||||
|
||||
#### Filter Templates:
|
||||
|
||||
Users can also filter the template(s) to use based on the name of the bucket,
|
||||
@@ -311,7 +247,7 @@ There are many more options available,
|
||||
## measurement name suffix (for separating different commands)
|
||||
name_suffix = "_mycollector"
|
||||
|
||||
## Data format to consume.
|
||||
## Data format to consume. This can be "json", "influx" or "graphite" (line-protocol)
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
|
||||
@@ -336,27 +272,3 @@ There are many more options available,
|
||||
"measurement*"
|
||||
]
|
||||
```
|
||||
|
||||
# Nagios:
|
||||
|
||||
There are no additional configuration options for Nagios line-protocol. The
|
||||
metrics are parsed directly into Telegraf metrics.
|
||||
|
||||
Note: Nagios Input Data Formats is only supported in `exec` input plugin.
|
||||
|
||||
#### Nagios Configuration:
|
||||
|
||||
```toml
|
||||
[[inputs.exec]]
|
||||
## Commands array
|
||||
commands = ["/usr/lib/nagios/plugins/check_load", "-w 5,6,7 -c 7,8,9"]
|
||||
|
||||
## measurement name suffix (for separating different commands)
|
||||
name_suffix = "_mycollector"
|
||||
|
||||
## Data format to consume.
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
|
||||
data_format = "nagios"
|
||||
```
|
||||
|
||||
@@ -1,11 +1,5 @@
|
||||
# Telegraf Output Data Formats
|
||||
|
||||
Telegraf is able to serialize metrics into the following output data formats:
|
||||
|
||||
1. [InfluxDB Line Protocol](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md#influx)
|
||||
1. [JSON](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md#json)
|
||||
1. [Graphite](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md#graphite)
|
||||
|
||||
Telegraf metrics, like InfluxDB
|
||||
[points](https://docs.influxdata.com/influxdb/v0.10/write_protocols/line/),
|
||||
are a combination of four basic parts:
|
||||
@@ -35,7 +29,7 @@ config option, for example, in the `file` output plugin:
|
||||
## Files to write to, "stdout" is a specially handled file.
|
||||
files = ["stdout"]
|
||||
|
||||
## Data format to output.
|
||||
## Data format to output. This can be "influx" or "graphite"
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
|
||||
@@ -47,46 +41,34 @@ config option, for example, in the `file` output plugin:
|
||||
Each data_format has an additional set of configuration options available, which
|
||||
I'll go over below.
|
||||
|
||||
# Influx:
|
||||
## Influx:
|
||||
|
||||
There are no additional configuration options for InfluxDB line-protocol. The
|
||||
metrics are serialized directly into InfluxDB line-protocol.
|
||||
|
||||
### Influx Configuration:
|
||||
#### Influx Configuration:
|
||||
|
||||
```toml
|
||||
[[outputs.file]]
|
||||
## Files to write to, "stdout" is a specially handled file.
|
||||
files = ["stdout", "/tmp/metrics.out"]
|
||||
|
||||
## Data format to output.
|
||||
## Data format to output. This can be "influx" or "graphite"
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
|
||||
data_format = "influx"
|
||||
```
|
||||
|
||||
# Graphite:
|
||||
## Graphite:
|
||||
|
||||
The Graphite data format translates Telegraf metrics into _dot_ buckets. A
|
||||
template can be specified for the output of Telegraf metrics into Graphite
|
||||
buckets. The default template is:
|
||||
The Graphite data format translates Telegraf metrics into _dot_ buckets.
|
||||
The format is:
|
||||
|
||||
```
|
||||
template = "host.tags.measurement.field"
|
||||
[prefix].[host tag].[all tags (alphabetical)].[measurement name].[field name] value timestamp
|
||||
```
|
||||
|
||||
In the above template, we have four parts:
|
||||
|
||||
1. _host_ is a tag key. This can be any tag key that is in the Telegraf
|
||||
metric(s). If the key doesn't exist, it will be ignored. If it does exist, the
|
||||
tag value will be filled in.
|
||||
1. _tags_ is a special keyword that outputs all remaining tag values, separated
|
||||
by dots and in alphabetical order (by tag key). These will be filled after all
|
||||
tag keys are filled.
|
||||
1. _measurement_ is a special keyword that outputs the measurement name.
|
||||
1. _field_ is a special keyword that outputs the field name.
|
||||
|
||||
Which means the following influx metric -> graphite conversion would happen:
|
||||
|
||||
```
|
||||
@@ -96,55 +78,20 @@ tars.cpu-total.us-east-1.cpu.usage_user 0.89 1455320690
|
||||
tars.cpu-total.us-east-1.cpu.usage_idle 98.09 1455320690
|
||||
```
|
||||
|
||||
### Graphite Configuration:
|
||||
`prefix` is a configuration option when using the graphite output data format.
|
||||
|
||||
#### Graphite Configuration:
|
||||
|
||||
```toml
|
||||
[[outputs.file]]
|
||||
## Files to write to, "stdout" is a specially handled file.
|
||||
files = ["stdout", "/tmp/metrics.out"]
|
||||
|
||||
## Data format to output.
|
||||
## Data format to output. This can be "influx" or "graphite"
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
|
||||
data_format = "graphite"
|
||||
data_format = "influx"
|
||||
|
||||
# prefix each graphite bucket
|
||||
prefix = "telegraf"
|
||||
# graphite template
|
||||
template = "host.tags.measurement.field"
|
||||
```
|
||||
|
||||
# JSON:
|
||||
|
||||
The JSON data format serialized Telegraf metrics in json format. The format is:
|
||||
|
||||
```json
|
||||
{
|
||||
"fields":{
|
||||
"field_1":30,
|
||||
"field_2":4,
|
||||
"field_N":59,
|
||||
"n_images":660
|
||||
},
|
||||
"name":"docker",
|
||||
"tags":{
|
||||
"host":"raynor"
|
||||
},
|
||||
"timestamp":1458229140
|
||||
}
|
||||
```
|
||||
|
||||
### JSON Configuration:
|
||||
|
||||
```toml
|
||||
[[outputs.file]]
|
||||
## Files to write to, "stdout" is a specially handled file.
|
||||
files = ["stdout", "/tmp/metrics.out"]
|
||||
|
||||
## Data format to output.
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
|
||||
data_format = "json"
|
||||
```
|
||||
|
||||
@@ -28,5 +28,6 @@
|
||||
- github.com/wvanbergen/kazoo-go [MIT LICENSE](https://github.com/wvanbergen/kazoo-go/blob/master/MIT-LICENSE)
|
||||
- gopkg.in/dancannon/gorethink.v1 [APACHE LICENSE](https://github.com/dancannon/gorethink/blob/v1.1.2/LICENSE)
|
||||
- gopkg.in/mgo.v2 [BSD LICENSE](https://github.com/go-mgo/mgo/blob/v2/LICENSE)
|
||||
- golang.org/x/crypto/ [BSD LICENSE](https://github.com/golang/crypto/blob/master/LICENSE)
|
||||
- golang.org/x/crypto/* [BSD LICENSE](https://github.com/golang/crypto/blob/master/LICENSE)
|
||||
- internal Glob function [MIT LICENSE](https://github.com/ryanuber/go-glob/blob/master/LICENSE)
|
||||
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
# Running Telegraf as a Windows Service
|
||||
|
||||
If you have tried to install Go binaries as Windows Services with the **sc.exe**
|
||||
tool you may have seen that the service errors and stops running after a while.
|
||||
|
||||
**NSSM** (the Non-Sucking Service Manager) is a tool that helps you in a
|
||||
[number of scenarios](http://nssm.cc/scenarios) including running Go binaries
|
||||
that were not specifically designed to run only in Windows platforms.
|
||||
|
||||
## NSSM Installation via Chocolatey
|
||||
|
||||
You can install [Chocolatey](https://chocolatey.org/) and [NSSM](http://nssm.cc/)
|
||||
with these commands
|
||||
|
||||
```powershell
|
||||
iex ((new-object net.webclient).DownloadString('https://chocolatey.org/install.ps1'))
|
||||
choco install -y nssm
|
||||
```
|
||||
|
||||
## Installing Telegraf as a Windows Service with NSSM
|
||||
|
||||
You can download the latest Telegraf Windows binaries (still Experimental at
|
||||
the moment) from [the Telegraf Github repo](https://github.com/influxdata/telegraf).
|
||||
|
||||
Then you can create a C:\telegraf folder, unzip the binary there and modify the
|
||||
**telegraf.conf** sample to allocate the metrics you want to send to **InfluxDB**.
|
||||
|
||||
Once you have NSSM installed in your system, the process is quite straightforward.
|
||||
You only need to type this command in your Windows shell
|
||||
|
||||
```powershell
|
||||
nssm install Telegraf c:\telegraf\telegraf.exe -config c:\telegraf\telegraf.config
|
||||
```
|
||||
|
||||
And now your service will be installed in Windows and you will be able to start and
|
||||
stop it gracefully
|
||||
1441
etc/telegraf.conf
1441
etc/telegraf.conf
File diff suppressed because it is too large
Load Diff
@@ -1,164 +0,0 @@
|
||||
# Telegraf configuration
|
||||
|
||||
# Telegraf is entirely plugin driven. All metrics are gathered from the
|
||||
# declared inputs, and sent to the declared outputs.
|
||||
|
||||
# Plugins must be declared in here to be active.
|
||||
# To deactivate a plugin, comment out the name and any variables.
|
||||
|
||||
# Use 'telegraf -config telegraf.conf -test' to see what metrics a config
|
||||
# file would generate.
|
||||
|
||||
# Global tags can be specified here in key="value" format.
|
||||
[global_tags]
|
||||
# dc = "us-east-1" # will tag all metrics with dc=us-east-1
|
||||
# rack = "1a"
|
||||
|
||||
# Configuration for telegraf agent
|
||||
[agent]
|
||||
## Default data collection interval for all inputs
|
||||
interval = "10s"
|
||||
## Rounds collection interval to 'interval'
|
||||
## ie, if interval="10s" then always collect on :00, :10, :20, etc.
|
||||
round_interval = true
|
||||
|
||||
## Telegraf will cache metric_buffer_limit metrics for each output, and will
|
||||
## flush this buffer on a successful write.
|
||||
metric_buffer_limit = 1000
|
||||
## Flush the buffer whenever full, regardless of flush_interval.
|
||||
flush_buffer_when_full = true
|
||||
|
||||
## Collection jitter is used to jitter the collection by a random amount.
|
||||
## Each plugin will sleep for a random time within jitter before collecting.
|
||||
## This can be used to avoid many plugins querying things like sysfs at the
|
||||
## same time, which can have a measurable effect on the system.
|
||||
collection_jitter = "0s"
|
||||
|
||||
## Default flushing interval for all outputs. You shouldn't set this below
|
||||
## interval. Maximum flush_interval will be flush_interval + flush_jitter
|
||||
flush_interval = "10s"
|
||||
## Jitter the flush interval by a random amount. This is primarily to avoid
|
||||
## large write spikes for users running a large number of telegraf instances.
|
||||
## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
|
||||
flush_jitter = "0s"
|
||||
|
||||
## Run telegraf in debug mode
|
||||
debug = false
|
||||
## Run telegraf in quiet mode
|
||||
quiet = false
|
||||
## Override default hostname, if empty use os.Hostname()
|
||||
hostname = ""
|
||||
|
||||
|
||||
###############################################################################
|
||||
# OUTPUTS #
|
||||
###############################################################################
|
||||
|
||||
# Configuration for influxdb server to send metrics to
|
||||
[[outputs.influxdb]]
|
||||
# The full HTTP or UDP endpoint URL for your InfluxDB instance.
|
||||
# Multiple urls can be specified but it is assumed that they are part of the same
|
||||
# cluster, this means that only ONE of the urls will be written to each interval.
|
||||
# urls = ["udp://localhost:8089"] # UDP endpoint example
|
||||
urls = ["http://localhost:8086"] # required
|
||||
# The target database for metrics (telegraf will create it if not exists)
|
||||
database = "telegraf" # required
|
||||
# Precision of writes, valid values are "ns", "us" (or "µs"), "ms", "s", "m", "h".
|
||||
# note: using second precision greatly helps InfluxDB compression
|
||||
precision = "s"
|
||||
|
||||
## Write timeout (for the InfluxDB client), formatted as a string.
|
||||
## If not provided, will default to 5s. 0s means no timeout (not recommended).
|
||||
timeout = "5s"
|
||||
# username = "telegraf"
|
||||
# password = "metricsmetricsmetricsmetrics"
|
||||
# Set the user agent for HTTP POSTs (can be useful for log differentiation)
|
||||
# user_agent = "telegraf"
|
||||
# Set UDP payload size, defaults to InfluxDB UDP Client default (512 bytes)
|
||||
# udp_payload = 512
|
||||
|
||||
|
||||
###############################################################################
|
||||
# INPUTS #
|
||||
###############################################################################
|
||||
|
||||
# Windows Performance Counters plugin.
|
||||
# These are the recommended method of monitoring system metrics on windows,
|
||||
# as the regular system plugins (inputs.cpu, inputs.mem, etc.) rely on WMI,
|
||||
# which utilizes a lot of system resources.
|
||||
#
|
||||
# See more configuration examples at:
|
||||
# https://github.com/influxdata/telegraf/tree/master/plugins/inputs/win_perf_counters
|
||||
|
||||
[[inputs.win_perf_counters]]
|
||||
[[inputs.win_perf_counters.object]]
|
||||
# Processor usage, alternative to native, reports on a per core.
|
||||
ObjectName = "Processor"
|
||||
Instances = ["*"]
|
||||
Counters = ["% Idle Time", "% Interrupt Time", "% Privileged Time", "% User Time", "% Processor Time"]
|
||||
Measurement = "win_cpu"
|
||||
#IncludeTotal=false #Set to true to include _Total instance when querying for all (*).
|
||||
|
||||
[[inputs.win_perf_counters.object]]
|
||||
# Disk times and queues
|
||||
ObjectName = "LogicalDisk"
|
||||
Instances = ["*"]
|
||||
Counters = ["% Idle Time", "% Disk Time","% Disk Read Time", "% Disk Write Time", "% User Time", "Current Disk Queue Length"]
|
||||
Measurement = "win_disk"
|
||||
#IncludeTotal=false #Set to true to include _Total instance when querying for all (*).
|
||||
|
||||
[[inputs.win_perf_counters.object]]
|
||||
ObjectName = "System"
|
||||
Counters = ["Context Switches/sec","System Calls/sec"]
|
||||
Instances = ["------"]
|
||||
Measurement = "win_system"
|
||||
#IncludeTotal=false #Set to true to include _Total instance when querying for all (*).
|
||||
|
||||
[[inputs.win_perf_counters.object]]
|
||||
# Example query where the Instance portion must be removed to get data back, such as from the Memory object.
|
||||
ObjectName = "Memory"
|
||||
Counters = ["Available Bytes","Cache Faults/sec","Demand Zero Faults/sec","Page Faults/sec","Pages/sec","Transition Faults/sec","Pool Nonpaged Bytes","Pool Paged Bytes"]
|
||||
Instances = ["------"] # Use 6 x - to remove the Instance bit from the query.
|
||||
Measurement = "win_mem"
|
||||
#IncludeTotal=false #Set to true to include _Total instance when querying for all (*).
|
||||
|
||||
|
||||
# Windows system plugins using WMI (disabled by default, using
|
||||
# win_perf_counters over WMI is recommended)
|
||||
|
||||
# Read metrics about cpu usage
|
||||
#[[inputs.cpu]]
|
||||
## Whether to report per-cpu stats or not
|
||||
#percpu = true
|
||||
## Whether to report total system cpu stats or not
|
||||
#totalcpu = true
|
||||
## Comment this line if you want the raw CPU time metrics
|
||||
#fielddrop = ["time_*"]
|
||||
|
||||
# Read metrics about disk usage by mount point
|
||||
#[[inputs.disk]]
|
||||
## By default, telegraf gather stats for all mountpoints.
|
||||
## Setting mountpoints will restrict the stats to the specified mountpoints.
|
||||
## mount_points=["/"]
|
||||
|
||||
## Ignore some mountpoints by filesystem type. For example (dev)tmpfs (usually
|
||||
## present on /run, /var/run, /dev/shm or /dev).
|
||||
#ignore_fs = ["tmpfs", "devtmpfs"]
|
||||
|
||||
# Read metrics about disk IO by device
|
||||
#[[inputs.diskio]]
|
||||
## By default, telegraf will gather stats for all devices including
|
||||
## disk partitions.
|
||||
## Setting devices will restrict the stats to the specified devices.
|
||||
## devices = ["sda", "sdb"]
|
||||
## Uncomment the following line if you do not need disk serial numbers.
|
||||
## skip_serial_number = true
|
||||
|
||||
# Read metrics about memory usage
|
||||
#[[inputs.mem]]
|
||||
# no configuration
|
||||
|
||||
# Read metrics about swap memory usage
|
||||
#[[inputs.swap]]
|
||||
# no configuration
|
||||
|
||||
@@ -1,77 +0,0 @@
|
||||
package buffer
|
||||
|
||||
import (
|
||||
"github.com/influxdata/telegraf"
|
||||
)
|
||||
|
||||
// Buffer is an object for storing metrics in a circular buffer.
|
||||
type Buffer struct {
|
||||
buf chan telegraf.Metric
|
||||
// total dropped metrics
|
||||
drops int
|
||||
// total metrics added
|
||||
total int
|
||||
}
|
||||
|
||||
// NewBuffer returns a Buffer
|
||||
// size is the maximum number of metrics that Buffer will cache. If Add is
|
||||
// called when the buffer is full, then the oldest metric(s) will be dropped.
|
||||
func NewBuffer(size int) *Buffer {
|
||||
return &Buffer{
|
||||
buf: make(chan telegraf.Metric, size),
|
||||
}
|
||||
}
|
||||
|
||||
// IsEmpty returns true if Buffer is empty.
|
||||
func (b *Buffer) IsEmpty() bool {
|
||||
return len(b.buf) == 0
|
||||
}
|
||||
|
||||
// Len returns the current length of the buffer.
|
||||
func (b *Buffer) Len() int {
|
||||
return len(b.buf)
|
||||
}
|
||||
|
||||
// Drops returns the total number of dropped metrics that have occured in this
|
||||
// buffer since instantiation.
|
||||
func (b *Buffer) Drops() int {
|
||||
return b.drops
|
||||
}
|
||||
|
||||
// Total returns the total number of metrics that have been added to this buffer.
|
||||
func (b *Buffer) Total() int {
|
||||
return b.total
|
||||
}
|
||||
|
||||
// Add adds metrics to the buffer.
|
||||
func (b *Buffer) Add(metrics ...telegraf.Metric) {
|
||||
for i, _ := range metrics {
|
||||
b.total++
|
||||
select {
|
||||
case b.buf <- metrics[i]:
|
||||
default:
|
||||
b.drops++
|
||||
<-b.buf
|
||||
b.buf <- metrics[i]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Batch returns a batch of metrics of size batchSize.
|
||||
// the batch will be of maximum length batchSize. It can be less than batchSize,
|
||||
// if the length of Buffer is less than batchSize.
|
||||
func (b *Buffer) Batch(batchSize int) []telegraf.Metric {
|
||||
n := min(len(b.buf), batchSize)
|
||||
out := make([]telegraf.Metric, n)
|
||||
for i := 0; i < n; i++ {
|
||||
out[i] = <-b.buf
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
func min(a, b int) int {
|
||||
if b < a {
|
||||
return b
|
||||
}
|
||||
return a
|
||||
}
|
||||
@@ -1,94 +0,0 @@
|
||||
package buffer
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
var metricList = []telegraf.Metric{
|
||||
testutil.TestMetric(2, "mymetric1"),
|
||||
testutil.TestMetric(1, "mymetric2"),
|
||||
testutil.TestMetric(11, "mymetric3"),
|
||||
testutil.TestMetric(15, "mymetric4"),
|
||||
testutil.TestMetric(8, "mymetric5"),
|
||||
}
|
||||
|
||||
func BenchmarkAddMetrics(b *testing.B) {
|
||||
buf := NewBuffer(10000)
|
||||
m := testutil.TestMetric(1, "mymetric")
|
||||
for n := 0; n < b.N; n++ {
|
||||
buf.Add(m)
|
||||
}
|
||||
}
|
||||
|
||||
func TestNewBufferBasicFuncs(t *testing.T) {
|
||||
b := NewBuffer(10)
|
||||
|
||||
assert.True(t, b.IsEmpty())
|
||||
assert.Zero(t, b.Len())
|
||||
assert.Zero(t, b.Drops())
|
||||
assert.Zero(t, b.Total())
|
||||
|
||||
m := testutil.TestMetric(1, "mymetric")
|
||||
b.Add(m)
|
||||
assert.False(t, b.IsEmpty())
|
||||
assert.Equal(t, b.Len(), 1)
|
||||
assert.Equal(t, b.Drops(), 0)
|
||||
assert.Equal(t, b.Total(), 1)
|
||||
|
||||
b.Add(metricList...)
|
||||
assert.False(t, b.IsEmpty())
|
||||
assert.Equal(t, b.Len(), 6)
|
||||
assert.Equal(t, b.Drops(), 0)
|
||||
assert.Equal(t, b.Total(), 6)
|
||||
}
|
||||
|
||||
func TestDroppingMetrics(t *testing.T) {
|
||||
b := NewBuffer(10)
|
||||
|
||||
// Add up to the size of the buffer
|
||||
b.Add(metricList...)
|
||||
b.Add(metricList...)
|
||||
assert.False(t, b.IsEmpty())
|
||||
assert.Equal(t, b.Len(), 10)
|
||||
assert.Equal(t, b.Drops(), 0)
|
||||
assert.Equal(t, b.Total(), 10)
|
||||
|
||||
// Add 5 more and verify they were dropped
|
||||
b.Add(metricList...)
|
||||
assert.False(t, b.IsEmpty())
|
||||
assert.Equal(t, b.Len(), 10)
|
||||
assert.Equal(t, b.Drops(), 5)
|
||||
assert.Equal(t, b.Total(), 15)
|
||||
}
|
||||
|
||||
func TestGettingBatches(t *testing.T) {
|
||||
b := NewBuffer(20)
|
||||
|
||||
// Verify that the buffer returned is smaller than requested when there are
|
||||
// not as many items as requested.
|
||||
b.Add(metricList...)
|
||||
batch := b.Batch(10)
|
||||
assert.Len(t, batch, 5)
|
||||
|
||||
// Verify that the buffer is now empty
|
||||
assert.True(t, b.IsEmpty())
|
||||
assert.Zero(t, b.Len())
|
||||
assert.Zero(t, b.Drops())
|
||||
assert.Equal(t, b.Total(), 5)
|
||||
|
||||
// Verify that the buffer returned is not more than the size requested
|
||||
b.Add(metricList...)
|
||||
batch = b.Batch(3)
|
||||
assert.Len(t, batch, 3)
|
||||
|
||||
// Verify that buffer is not empty
|
||||
assert.False(t, b.IsEmpty())
|
||||
assert.Equal(t, b.Len(), 2)
|
||||
assert.Equal(t, b.Drops(), 0)
|
||||
assert.Equal(t, b.Total(), 10)
|
||||
}
|
||||
@@ -1,14 +1,11 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"log"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"sort"
|
||||
"strings"
|
||||
"time"
|
||||
@@ -22,20 +19,7 @@ import (
|
||||
"github.com/influxdata/telegraf/plugins/serializers"
|
||||
|
||||
"github.com/influxdata/config"
|
||||
"github.com/influxdata/toml"
|
||||
"github.com/influxdata/toml/ast"
|
||||
)
|
||||
|
||||
var (
|
||||
// Default input plugins
|
||||
inputDefaults = []string{"cpu", "mem", "swap", "system", "kernel",
|
||||
"processes", "disk", "diskio"}
|
||||
|
||||
// Default output plugins
|
||||
outputDefaults = []string{"influxdb"}
|
||||
|
||||
// envVarRe is a regex to find environment variables in the config file
|
||||
envVarRe = regexp.MustCompile(`\$\w+`)
|
||||
"github.com/naoina/toml/ast"
|
||||
)
|
||||
|
||||
// Config specifies the URL/user/password for the database that telegraf
|
||||
@@ -93,15 +77,9 @@ type AgentConfig struct {
|
||||
// ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
|
||||
FlushJitter internal.Duration
|
||||
|
||||
// MetricBatchSize is the maximum number of metrics that is wrote to an
|
||||
// output plugin in one call.
|
||||
MetricBatchSize int
|
||||
|
||||
// MetricBufferLimit is the max number of metrics that each output plugin
|
||||
// will cache. The buffer is cleared when a successful write occurs. When
|
||||
// full, the oldest metrics will be overwritten. This number should be a
|
||||
// multiple of MetricBatchSize. Due to current implementation, this could
|
||||
// not be less than 2 times MetricBatchSize.
|
||||
// full, the oldest metrics will be overwritten.
|
||||
MetricBufferLimit int
|
||||
|
||||
// FlushBufferWhenFull tells Telegraf to flush the metric buffer whenever
|
||||
@@ -119,9 +97,8 @@ type AgentConfig struct {
|
||||
Debug bool
|
||||
|
||||
// Quiet is the option for running in quiet mode
|
||||
Quiet bool
|
||||
Hostname string
|
||||
OmitHostname bool
|
||||
Quiet bool
|
||||
Hostname string
|
||||
}
|
||||
|
||||
// Inputs returns a list of strings of the configured inputs.
|
||||
@@ -157,28 +134,20 @@ func (c *Config) ListTags() string {
|
||||
}
|
||||
|
||||
var header = `# Telegraf Configuration
|
||||
#
|
||||
|
||||
# Telegraf is entirely plugin driven. All metrics are gathered from the
|
||||
# declared inputs, and sent to the declared outputs.
|
||||
#
|
||||
|
||||
# Plugins must be declared in here to be active.
|
||||
# To deactivate a plugin, comment out the name and any variables.
|
||||
#
|
||||
|
||||
# Use 'telegraf -config telegraf.conf -test' to see what metrics a config
|
||||
# file would generate.
|
||||
#
|
||||
# Environment variables can be used anywhere in this config file, simply prepend
|
||||
# them with $. For strings the variable must be within quotes (ie, "$STR_VAR"),
|
||||
# for numbers and booleans they should be plain (ie, $INT_VAR, $BOOL_VAR)
|
||||
|
||||
|
||||
# Global tags can be specified here in key="value" format.
|
||||
[global_tags]
|
||||
# dc = "us-east-1" # will tag all metrics with dc=us-east-1
|
||||
# rack = "1a"
|
||||
## Environment variables can be used as tags, and throughout the config file
|
||||
# user = "$USER"
|
||||
|
||||
|
||||
# Configuration for telegraf agent
|
||||
[agent]
|
||||
@@ -188,13 +157,11 @@ var header = `# Telegraf Configuration
|
||||
## ie, if interval="10s" then always collect on :00, :10, :20, etc.
|
||||
round_interval = true
|
||||
|
||||
## Telegraf will send metrics to outputs in batches of at
|
||||
## most metric_batch_size metrics.
|
||||
metric_batch_size = 1000
|
||||
## For failed writes, telegraf will cache metric_buffer_limit metrics for each
|
||||
## output, and will flush this buffer on a successful write. Oldest metrics
|
||||
## are dropped first when this buffer fills.
|
||||
## Telegraf will cache metric_buffer_limit metrics for each output, and will
|
||||
## flush this buffer on a successful write.
|
||||
metric_buffer_limit = 10000
|
||||
## Flush the buffer whenever full, regardless of flush_interval.
|
||||
flush_buffer_when_full = true
|
||||
|
||||
## Collection jitter is used to jitter the collection by a random amount.
|
||||
## Each plugin will sleep for a random time within jitter before collecting.
|
||||
@@ -216,111 +183,34 @@ var header = `# Telegraf Configuration
|
||||
quiet = false
|
||||
## Override default hostname, if empty use os.Hostname()
|
||||
hostname = ""
|
||||
## If set to true, do no set the "host" tag in the telegraf agent.
|
||||
omit_hostname = false
|
||||
|
||||
|
||||
###############################################################################
|
||||
# OUTPUT PLUGINS #
|
||||
###############################################################################
|
||||
#
|
||||
# OUTPUTS:
|
||||
#
|
||||
|
||||
`
|
||||
|
||||
var inputHeader = `
|
||||
|
||||
###############################################################################
|
||||
# INPUT PLUGINS #
|
||||
###############################################################################
|
||||
var pluginHeader = `
|
||||
#
|
||||
# INPUTS:
|
||||
#
|
||||
`
|
||||
|
||||
var serviceInputHeader = `
|
||||
|
||||
###############################################################################
|
||||
# SERVICE INPUT PLUGINS #
|
||||
###############################################################################
|
||||
#
|
||||
# SERVICE INPUTS:
|
||||
#
|
||||
`
|
||||
|
||||
// PrintSampleConfig prints the sample config
|
||||
func PrintSampleConfig(inputFilters []string, outputFilters []string) {
|
||||
func PrintSampleConfig(pluginFilters []string, outputFilters []string) {
|
||||
fmt.Printf(header)
|
||||
|
||||
if len(outputFilters) != 0 {
|
||||
printFilteredOutputs(outputFilters, false)
|
||||
} else {
|
||||
printFilteredOutputs(outputDefaults, false)
|
||||
// Print non-default outputs, commented
|
||||
var pnames []string
|
||||
for pname := range outputs.Outputs {
|
||||
if !sliceContains(pname, outputDefaults) {
|
||||
pnames = append(pnames, pname)
|
||||
}
|
||||
}
|
||||
sort.Strings(pnames)
|
||||
printFilteredOutputs(pnames, true)
|
||||
}
|
||||
|
||||
fmt.Printf(inputHeader)
|
||||
if len(inputFilters) != 0 {
|
||||
printFilteredInputs(inputFilters, false)
|
||||
} else {
|
||||
printFilteredInputs(inputDefaults, false)
|
||||
// Print non-default inputs, commented
|
||||
var pnames []string
|
||||
for pname := range inputs.Inputs {
|
||||
if !sliceContains(pname, inputDefaults) {
|
||||
pnames = append(pnames, pname)
|
||||
}
|
||||
}
|
||||
sort.Strings(pnames)
|
||||
printFilteredInputs(pnames, true)
|
||||
}
|
||||
}
|
||||
|
||||
func printFilteredInputs(inputFilters []string, commented bool) {
|
||||
// Filter inputs
|
||||
var pnames []string
|
||||
for pname := range inputs.Inputs {
|
||||
if sliceContains(pname, inputFilters) {
|
||||
pnames = append(pnames, pname)
|
||||
}
|
||||
}
|
||||
sort.Strings(pnames)
|
||||
|
||||
// cache service inputs to print them at the end
|
||||
servInputs := make(map[string]telegraf.ServiceInput)
|
||||
// for alphabetical looping:
|
||||
servInputNames := []string{}
|
||||
|
||||
// Print Inputs
|
||||
for _, pname := range pnames {
|
||||
creator := inputs.Inputs[pname]
|
||||
input := creator()
|
||||
|
||||
switch p := input.(type) {
|
||||
case telegraf.ServiceInput:
|
||||
servInputs[pname] = p
|
||||
servInputNames = append(servInputNames, pname)
|
||||
continue
|
||||
}
|
||||
|
||||
printConfig(pname, input, "inputs", commented)
|
||||
}
|
||||
|
||||
// Print Service Inputs
|
||||
if len(servInputs) == 0 {
|
||||
return
|
||||
}
|
||||
sort.Strings(servInputNames)
|
||||
fmt.Printf(serviceInputHeader)
|
||||
for _, name := range servInputNames {
|
||||
printConfig(name, servInputs[name], "inputs", commented)
|
||||
}
|
||||
}
|
||||
|
||||
func printFilteredOutputs(outputFilters []string, commented bool) {
|
||||
// Filter outputs
|
||||
var onames []string
|
||||
for oname := range outputs.Outputs {
|
||||
if sliceContains(oname, outputFilters) {
|
||||
if len(outputFilters) == 0 || sliceContains(oname, outputFilters) {
|
||||
onames = append(onames, oname)
|
||||
}
|
||||
}
|
||||
@@ -330,7 +220,38 @@ func printFilteredOutputs(outputFilters []string, commented bool) {
|
||||
for _, oname := range onames {
|
||||
creator := outputs.Outputs[oname]
|
||||
output := creator()
|
||||
printConfig(oname, output, "outputs", commented)
|
||||
printConfig(oname, output, "outputs")
|
||||
}
|
||||
|
||||
// Filter inputs
|
||||
var pnames []string
|
||||
for pname := range inputs.Inputs {
|
||||
if len(pluginFilters) == 0 || sliceContains(pname, pluginFilters) {
|
||||
pnames = append(pnames, pname)
|
||||
}
|
||||
}
|
||||
sort.Strings(pnames)
|
||||
|
||||
// Print Inputs
|
||||
fmt.Printf(pluginHeader)
|
||||
servInputs := make(map[string]telegraf.ServiceInput)
|
||||
for _, pname := range pnames {
|
||||
creator := inputs.Inputs[pname]
|
||||
input := creator()
|
||||
|
||||
switch p := input.(type) {
|
||||
case telegraf.ServiceInput:
|
||||
servInputs[pname] = p
|
||||
continue
|
||||
}
|
||||
|
||||
printConfig(pname, input, "inputs")
|
||||
}
|
||||
|
||||
// Print Service Inputs
|
||||
fmt.Printf(serviceInputHeader)
|
||||
for name, input := range servInputs {
|
||||
printConfig(name, input, "inputs")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -339,26 +260,13 @@ type printer interface {
|
||||
SampleConfig() string
|
||||
}
|
||||
|
||||
func printConfig(name string, p printer, op string, commented bool) {
|
||||
comment := ""
|
||||
if commented {
|
||||
comment = "# "
|
||||
}
|
||||
fmt.Printf("\n%s# %s\n%s[[%s.%s]]", comment, p.Description(), comment,
|
||||
op, name)
|
||||
|
||||
func printConfig(name string, p printer, op string) {
|
||||
fmt.Printf("\n# %s\n[[%s.%s]]", p.Description(), op, name)
|
||||
config := p.SampleConfig()
|
||||
if config == "" {
|
||||
fmt.Printf("\n%s # no configuration\n\n", comment)
|
||||
fmt.Printf("\n # no configuration\n")
|
||||
} else {
|
||||
lines := strings.Split(config, "\n")
|
||||
for i, line := range lines {
|
||||
if i == 0 || i == len(lines)-1 {
|
||||
fmt.Print("\n")
|
||||
continue
|
||||
}
|
||||
fmt.Print(comment + line + "\n")
|
||||
}
|
||||
fmt.Printf(config)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -374,7 +282,7 @@ func sliceContains(name string, list []string) bool {
|
||||
// PrintInputConfig prints the config usage of a single input.
|
||||
func PrintInputConfig(name string) error {
|
||||
if creator, ok := inputs.Inputs[name]; ok {
|
||||
printConfig(name, creator(), "inputs", false)
|
||||
printConfig(name, creator(), "inputs")
|
||||
} else {
|
||||
return errors.New(fmt.Sprintf("Input %s not found", name))
|
||||
}
|
||||
@@ -384,7 +292,7 @@ func PrintInputConfig(name string) error {
|
||||
// PrintOutputConfig prints the config usage of a single output.
|
||||
func PrintOutputConfig(name string) error {
|
||||
if creator, ok := outputs.Outputs[name]; ok {
|
||||
printConfig(name, creator(), "outputs", false)
|
||||
printConfig(name, creator(), "outputs")
|
||||
} else {
|
||||
return errors.New(fmt.Sprintf("Output %s not found", name))
|
||||
}
|
||||
@@ -412,91 +320,46 @@ func (c *Config) LoadDirectory(path string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Try to find a default config file at these locations (in order):
|
||||
// 1. $TELEGRAF_CONFIG_PATH
|
||||
// 2. $HOME/.telegraf/telegraf.conf
|
||||
// 3. /etc/telegraf/telegraf.conf
|
||||
//
|
||||
func getDefaultConfigPath() (string, error) {
|
||||
envfile := os.Getenv("TELEGRAF_CONFIG_PATH")
|
||||
homefile := os.ExpandEnv("${HOME}/.telegraf/telegraf.conf")
|
||||
etcfile := "/etc/telegraf/telegraf.conf"
|
||||
for _, path := range []string{envfile, homefile, etcfile} {
|
||||
if _, err := os.Stat(path); err == nil {
|
||||
log.Printf("Using config file: %s", path)
|
||||
return path, nil
|
||||
}
|
||||
}
|
||||
|
||||
// if we got here, we didn't find a file in a default location
|
||||
return "", fmt.Errorf("No config file specified, and could not find one"+
|
||||
" in $TELEGRAF_CONFIG_PATH, %s, or %s", homefile, etcfile)
|
||||
}
|
||||
|
||||
// LoadConfig loads the given config file and applies it to c
|
||||
func (c *Config) LoadConfig(path string) error {
|
||||
var err error
|
||||
if path == "" {
|
||||
if path, err = getDefaultConfigPath(); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
tbl, err := parseFile(path)
|
||||
tbl, err := config.ParseFile(path)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error parsing %s, %s", path, err)
|
||||
return err
|
||||
}
|
||||
|
||||
// Parse tags tables first:
|
||||
for _, tableName := range []string{"tags", "global_tags"} {
|
||||
if val, ok := tbl.Fields[tableName]; ok {
|
||||
subTable, ok := val.(*ast.Table)
|
||||
if !ok {
|
||||
return fmt.Errorf("%s: invalid configuration", path)
|
||||
}
|
||||
if err = config.UnmarshalTable(subTable, c.Tags); err != nil {
|
||||
log.Printf("Could not parse [global_tags] config\n")
|
||||
return fmt.Errorf("Error parsing %s, %s", path, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Parse agent table:
|
||||
if val, ok := tbl.Fields["agent"]; ok {
|
||||
subTable, ok := val.(*ast.Table)
|
||||
if !ok {
|
||||
return fmt.Errorf("%s: invalid configuration", path)
|
||||
}
|
||||
if err = config.UnmarshalTable(subTable, c.Agent); err != nil {
|
||||
log.Printf("Could not parse [agent] config\n")
|
||||
return fmt.Errorf("Error parsing %s, %s", path, err)
|
||||
}
|
||||
}
|
||||
|
||||
// Parse all the rest of the plugins:
|
||||
for name, val := range tbl.Fields {
|
||||
subTable, ok := val.(*ast.Table)
|
||||
if !ok {
|
||||
return fmt.Errorf("%s: invalid configuration", path)
|
||||
return errors.New("invalid configuration")
|
||||
}
|
||||
|
||||
switch name {
|
||||
case "agent", "global_tags", "tags":
|
||||
case "agent":
|
||||
if err = config.UnmarshalTable(subTable, c.Agent); err != nil {
|
||||
log.Printf("Could not parse [agent] config\n")
|
||||
return err
|
||||
}
|
||||
case "global_tags", "tags":
|
||||
if err = config.UnmarshalTable(subTable, c.Tags); err != nil {
|
||||
log.Printf("Could not parse [global_tags] config\n")
|
||||
return err
|
||||
}
|
||||
case "outputs":
|
||||
for pluginName, pluginVal := range subTable.Fields {
|
||||
switch pluginSubTable := pluginVal.(type) {
|
||||
case *ast.Table:
|
||||
if err = c.addOutput(pluginName, pluginSubTable); err != nil {
|
||||
return fmt.Errorf("Error parsing %s, %s", path, err)
|
||||
return err
|
||||
}
|
||||
case []*ast.Table:
|
||||
for _, t := range pluginSubTable {
|
||||
if err = c.addOutput(pluginName, t); err != nil {
|
||||
return fmt.Errorf("Error parsing %s, %s", path, err)
|
||||
return err
|
||||
}
|
||||
}
|
||||
default:
|
||||
return fmt.Errorf("Unsupported config format: %s, file %s",
|
||||
pluginName, path)
|
||||
return fmt.Errorf("Unsupported config format: %s",
|
||||
pluginName)
|
||||
}
|
||||
}
|
||||
case "inputs", "plugins":
|
||||
@@ -504,50 +367,30 @@ func (c *Config) LoadConfig(path string) error {
|
||||
switch pluginSubTable := pluginVal.(type) {
|
||||
case *ast.Table:
|
||||
if err = c.addInput(pluginName, pluginSubTable); err != nil {
|
||||
return fmt.Errorf("Error parsing %s, %s", path, err)
|
||||
return err
|
||||
}
|
||||
case []*ast.Table:
|
||||
for _, t := range pluginSubTable {
|
||||
if err = c.addInput(pluginName, t); err != nil {
|
||||
return fmt.Errorf("Error parsing %s, %s", path, err)
|
||||
return err
|
||||
}
|
||||
}
|
||||
default:
|
||||
return fmt.Errorf("Unsupported config format: %s, file %s",
|
||||
pluginName, path)
|
||||
return fmt.Errorf("Unsupported config format: %s",
|
||||
pluginName)
|
||||
}
|
||||
}
|
||||
// Assume it's an input input for legacy config file support if no other
|
||||
// identifiers are present
|
||||
default:
|
||||
if err = c.addInput(name, subTable); err != nil {
|
||||
return fmt.Errorf("Error parsing %s, %s", path, err)
|
||||
return err
|
||||
}
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// parseFile loads a TOML configuration from a provided path and
|
||||
// returns the AST produced from the TOML parser. When loading the file, it
|
||||
// will find environment variables and replace them.
|
||||
func parseFile(fpath string) (*ast.Table, error) {
|
||||
contents, err := ioutil.ReadFile(fpath)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
env_vars := envVarRe.FindAll(contents, -1)
|
||||
for _, env_var := range env_vars {
|
||||
env_val := os.Getenv(strings.TrimPrefix(string(env_var), "$"))
|
||||
if env_val != "" {
|
||||
contents = bytes.Replace(contents, env_var, []byte(env_val), 1)
|
||||
}
|
||||
}
|
||||
|
||||
return toml.Parse(contents)
|
||||
}
|
||||
|
||||
func (c *Config) addOutput(name string, table *ast.Table) error {
|
||||
if len(c.OutputFilters) > 0 && !sliceContains(name, c.OutputFilters) {
|
||||
return nil
|
||||
@@ -578,8 +421,11 @@ func (c *Config) addOutput(name string, table *ast.Table) error {
|
||||
return err
|
||||
}
|
||||
|
||||
ro := internal_models.NewRunningOutput(name, output, outputConfig,
|
||||
c.Agent.MetricBatchSize, c.Agent.MetricBufferLimit)
|
||||
ro := internal_models.NewRunningOutput(name, output, outputConfig)
|
||||
if c.Agent.MetricBufferLimit > 0 {
|
||||
ro.MetricBufferLimit = c.Agent.MetricBufferLimit
|
||||
}
|
||||
ro.FlushBufferWhenFull = c.Agent.FlushBufferWhenFull
|
||||
c.Outputs = append(c.Outputs, ro)
|
||||
return nil
|
||||
}
|
||||
@@ -630,9 +476,9 @@ func (c *Config) addInput(name string, table *ast.Table) error {
|
||||
|
||||
// buildFilter builds a Filter
|
||||
// (tagpass/tagdrop/namepass/namedrop/fieldpass/fielddrop) to
|
||||
// be inserted into the internal_models.OutputConfig/internal_models.InputConfig
|
||||
// to be used for glob filtering on tags and measurements
|
||||
func buildFilter(tbl *ast.Table) (internal_models.Filter, error) {
|
||||
// be inserted into the internal_models.OutputConfig/internal_models.InputConfig to be used for prefix
|
||||
// filtering on tags and measurements
|
||||
func buildFilter(tbl *ast.Table) internal_models.Filter {
|
||||
f := internal_models.Filter{}
|
||||
|
||||
if node, ok := tbl.Fields["namepass"]; ok {
|
||||
@@ -731,33 +577,6 @@ func buildFilter(tbl *ast.Table) (internal_models.Filter, error) {
|
||||
}
|
||||
}
|
||||
|
||||
if node, ok := tbl.Fields["tagexclude"]; ok {
|
||||
if kv, ok := node.(*ast.KeyValue); ok {
|
||||
if ary, ok := kv.Value.(*ast.Array); ok {
|
||||
for _, elem := range ary.Value {
|
||||
if str, ok := elem.(*ast.String); ok {
|
||||
f.TagExclude = append(f.TagExclude, str.Value)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if node, ok := tbl.Fields["taginclude"]; ok {
|
||||
if kv, ok := node.(*ast.KeyValue); ok {
|
||||
if ary, ok := kv.Value.(*ast.Array); ok {
|
||||
for _, elem := range ary.Value {
|
||||
if str, ok := elem.(*ast.String); ok {
|
||||
f.TagInclude = append(f.TagInclude, str.Value)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
if err := f.CompileFilter(); err != nil {
|
||||
return f, err
|
||||
}
|
||||
|
||||
delete(tbl.Fields, "namedrop")
|
||||
delete(tbl.Fields, "namepass")
|
||||
delete(tbl.Fields, "fielddrop")
|
||||
@@ -766,9 +585,7 @@ func buildFilter(tbl *ast.Table) (internal_models.Filter, error) {
|
||||
delete(tbl.Fields, "pass")
|
||||
delete(tbl.Fields, "tagdrop")
|
||||
delete(tbl.Fields, "tagpass")
|
||||
delete(tbl.Fields, "tagexclude")
|
||||
delete(tbl.Fields, "taginclude")
|
||||
return f, nil
|
||||
return f
|
||||
}
|
||||
|
||||
// buildInput parses input specific items from the ast.Table,
|
||||
@@ -827,11 +644,7 @@ func buildInput(name string, tbl *ast.Table) (*internal_models.InputConfig, erro
|
||||
delete(tbl.Fields, "name_override")
|
||||
delete(tbl.Fields, "interval")
|
||||
delete(tbl.Fields, "tags")
|
||||
var err error
|
||||
cp.Filter, err = buildFilter(tbl)
|
||||
if err != nil {
|
||||
return cp, err
|
||||
}
|
||||
cp.Filter = buildFilter(tbl)
|
||||
return cp, nil
|
||||
}
|
||||
|
||||
@@ -888,21 +701,12 @@ func buildParser(name string, tbl *ast.Table) (parsers.Parser, error) {
|
||||
}
|
||||
}
|
||||
|
||||
if node, ok := tbl.Fields["data_type"]; ok {
|
||||
if kv, ok := node.(*ast.KeyValue); ok {
|
||||
if str, ok := kv.Value.(*ast.String); ok {
|
||||
c.DataType = str.Value
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
c.MetricName = name
|
||||
|
||||
delete(tbl.Fields, "data_format")
|
||||
delete(tbl.Fields, "separator")
|
||||
delete(tbl.Fields, "templates")
|
||||
delete(tbl.Fields, "tag_keys")
|
||||
delete(tbl.Fields, "data_type")
|
||||
|
||||
return parsers.NewParser(c)
|
||||
}
|
||||
@@ -933,32 +737,18 @@ func buildSerializer(name string, tbl *ast.Table) (serializers.Serializer, error
|
||||
}
|
||||
}
|
||||
|
||||
if node, ok := tbl.Fields["template"]; ok {
|
||||
if kv, ok := node.(*ast.KeyValue); ok {
|
||||
if str, ok := kv.Value.(*ast.String); ok {
|
||||
c.Template = str.Value
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
delete(tbl.Fields, "data_format")
|
||||
delete(tbl.Fields, "prefix")
|
||||
delete(tbl.Fields, "template")
|
||||
return serializers.NewSerializer(c)
|
||||
}
|
||||
|
||||
// buildOutput parses output specific items from the ast.Table,
|
||||
// builds the filter and returns an
|
||||
// buildOutput parses output specific items from the ast.Table, builds the filter and returns an
|
||||
// internal_models.OutputConfig to be inserted into internal_models.RunningInput
|
||||
// Note: error exists in the return for future calls that might require error
|
||||
func buildOutput(name string, tbl *ast.Table) (*internal_models.OutputConfig, error) {
|
||||
filter, err := buildFilter(tbl)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
oc := &internal_models.OutputConfig{
|
||||
Name: name,
|
||||
Filter: filter,
|
||||
Filter: buildFilter(tbl),
|
||||
}
|
||||
// Outputs don't support FieldDrop/FieldPass, so set to NameDrop/NamePass
|
||||
if len(oc.Filter.FieldDrop) > 0 {
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
package config
|
||||
|
||||
import (
|
||||
"os"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
@@ -11,54 +10,9 @@ import (
|
||||
"github.com/influxdata/telegraf/plugins/inputs/memcached"
|
||||
"github.com/influxdata/telegraf/plugins/inputs/procstat"
|
||||
"github.com/influxdata/telegraf/plugins/parsers"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestConfig_LoadSingleInputWithEnvVars(t *testing.T) {
|
||||
c := NewConfig()
|
||||
err := os.Setenv("MY_TEST_SERVER", "192.168.1.1")
|
||||
assert.NoError(t, err)
|
||||
err = os.Setenv("TEST_INTERVAL", "10s")
|
||||
assert.NoError(t, err)
|
||||
c.LoadConfig("./testdata/single_plugin_env_vars.toml")
|
||||
|
||||
memcached := inputs.Inputs["memcached"]().(*memcached.Memcached)
|
||||
memcached.Servers = []string{"192.168.1.1"}
|
||||
|
||||
filter := internal_models.Filter{
|
||||
NameDrop: []string{"metricname2"},
|
||||
NamePass: []string{"metricname1"},
|
||||
FieldDrop: []string{"other", "stuff"},
|
||||
FieldPass: []string{"some", "strings"},
|
||||
TagDrop: []internal_models.TagFilter{
|
||||
internal_models.TagFilter{
|
||||
Name: "badtag",
|
||||
Filter: []string{"othertag"},
|
||||
},
|
||||
},
|
||||
TagPass: []internal_models.TagFilter{
|
||||
internal_models.TagFilter{
|
||||
Name: "goodtag",
|
||||
Filter: []string{"mytag"},
|
||||
},
|
||||
},
|
||||
IsActive: true,
|
||||
}
|
||||
assert.NoError(t, filter.CompileFilter())
|
||||
mConfig := &internal_models.InputConfig{
|
||||
Name: "memcached",
|
||||
Filter: filter,
|
||||
Interval: 10 * time.Second,
|
||||
}
|
||||
mConfig.Tags = make(map[string]string)
|
||||
|
||||
assert.Equal(t, memcached, c.Inputs[0].Input,
|
||||
"Testdata did not produce a correct memcached struct.")
|
||||
assert.Equal(t, mConfig, c.Inputs[0].Config,
|
||||
"Testdata did not produce correct memcached metadata.")
|
||||
}
|
||||
|
||||
func TestConfig_LoadSingleInput(t *testing.T) {
|
||||
c := NewConfig()
|
||||
c.LoadConfig("./testdata/single_plugin.toml")
|
||||
@@ -66,29 +20,27 @@ func TestConfig_LoadSingleInput(t *testing.T) {
|
||||
memcached := inputs.Inputs["memcached"]().(*memcached.Memcached)
|
||||
memcached.Servers = []string{"localhost"}
|
||||
|
||||
filter := internal_models.Filter{
|
||||
NameDrop: []string{"metricname2"},
|
||||
NamePass: []string{"metricname1"},
|
||||
FieldDrop: []string{"other", "stuff"},
|
||||
FieldPass: []string{"some", "strings"},
|
||||
TagDrop: []internal_models.TagFilter{
|
||||
internal_models.TagFilter{
|
||||
Name: "badtag",
|
||||
Filter: []string{"othertag"},
|
||||
},
|
||||
},
|
||||
TagPass: []internal_models.TagFilter{
|
||||
internal_models.TagFilter{
|
||||
Name: "goodtag",
|
||||
Filter: []string{"mytag"},
|
||||
},
|
||||
},
|
||||
IsActive: true,
|
||||
}
|
||||
assert.NoError(t, filter.CompileFilter())
|
||||
mConfig := &internal_models.InputConfig{
|
||||
Name: "memcached",
|
||||
Filter: filter,
|
||||
Name: "memcached",
|
||||
Filter: internal_models.Filter{
|
||||
NameDrop: []string{"metricname2"},
|
||||
NamePass: []string{"metricname1"},
|
||||
FieldDrop: []string{"other", "stuff"},
|
||||
FieldPass: []string{"some", "strings"},
|
||||
TagDrop: []internal_models.TagFilter{
|
||||
internal_models.TagFilter{
|
||||
Name: "badtag",
|
||||
Filter: []string{"othertag"},
|
||||
},
|
||||
},
|
||||
TagPass: []internal_models.TagFilter{
|
||||
internal_models.TagFilter{
|
||||
Name: "goodtag",
|
||||
Filter: []string{"mytag"},
|
||||
},
|
||||
},
|
||||
IsActive: true,
|
||||
},
|
||||
Interval: 5 * time.Second,
|
||||
}
|
||||
mConfig.Tags = make(map[string]string)
|
||||
@@ -113,29 +65,27 @@ func TestConfig_LoadDirectory(t *testing.T) {
|
||||
memcached := inputs.Inputs["memcached"]().(*memcached.Memcached)
|
||||
memcached.Servers = []string{"localhost"}
|
||||
|
||||
filter := internal_models.Filter{
|
||||
NameDrop: []string{"metricname2"},
|
||||
NamePass: []string{"metricname1"},
|
||||
FieldDrop: []string{"other", "stuff"},
|
||||
FieldPass: []string{"some", "strings"},
|
||||
TagDrop: []internal_models.TagFilter{
|
||||
internal_models.TagFilter{
|
||||
Name: "badtag",
|
||||
Filter: []string{"othertag"},
|
||||
},
|
||||
},
|
||||
TagPass: []internal_models.TagFilter{
|
||||
internal_models.TagFilter{
|
||||
Name: "goodtag",
|
||||
Filter: []string{"mytag"},
|
||||
},
|
||||
},
|
||||
IsActive: true,
|
||||
}
|
||||
assert.NoError(t, filter.CompileFilter())
|
||||
mConfig := &internal_models.InputConfig{
|
||||
Name: "memcached",
|
||||
Filter: filter,
|
||||
Name: "memcached",
|
||||
Filter: internal_models.Filter{
|
||||
NameDrop: []string{"metricname2"},
|
||||
NamePass: []string{"metricname1"},
|
||||
FieldDrop: []string{"other", "stuff"},
|
||||
FieldPass: []string{"some", "strings"},
|
||||
TagDrop: []internal_models.TagFilter{
|
||||
internal_models.TagFilter{
|
||||
Name: "badtag",
|
||||
Filter: []string{"othertag"},
|
||||
},
|
||||
},
|
||||
TagPass: []internal_models.TagFilter{
|
||||
internal_models.TagFilter{
|
||||
Name: "goodtag",
|
||||
Filter: []string{"mytag"},
|
||||
},
|
||||
},
|
||||
IsActive: true,
|
||||
},
|
||||
Interval: 5 * time.Second,
|
||||
}
|
||||
mConfig.Tags = make(map[string]string)
|
||||
|
||||
@@ -1,11 +0,0 @@
|
||||
[[inputs.memcached]]
|
||||
servers = ["$MY_TEST_SERVER"]
|
||||
namepass = ["metricname1"]
|
||||
namedrop = ["metricname2"]
|
||||
fieldpass = ["some", "strings"]
|
||||
fielddrop = ["other", "stuff"]
|
||||
interval = "$TEST_INTERVAL"
|
||||
[inputs.memcached.tagpass]
|
||||
goodtag = ["mytag"]
|
||||
[inputs.memcached.tagdrop]
|
||||
badtag = ["othertag"]
|
||||
@@ -1,98 +0,0 @@
|
||||
package globpath
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
|
||||
"github.com/gobwas/glob"
|
||||
)
|
||||
|
||||
var sepStr = fmt.Sprintf("%v", string(os.PathSeparator))
|
||||
|
||||
type GlobPath struct {
|
||||
path string
|
||||
hasMeta bool
|
||||
g glob.Glob
|
||||
root string
|
||||
}
|
||||
|
||||
func Compile(path string) (*GlobPath, error) {
|
||||
out := GlobPath{
|
||||
hasMeta: hasMeta(path),
|
||||
path: path,
|
||||
}
|
||||
|
||||
// if there are no glob meta characters in the path, don't bother compiling
|
||||
// a glob object or finding the root directory. (see short-circuit in Match)
|
||||
if !out.hasMeta {
|
||||
return &out, nil
|
||||
}
|
||||
|
||||
var err error
|
||||
if out.g, err = glob.Compile(path, os.PathSeparator); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// Get the root directory for this filepath
|
||||
out.root = findRootDir(path)
|
||||
return &out, nil
|
||||
}
|
||||
|
||||
func (g *GlobPath) Match() map[string]os.FileInfo {
|
||||
if !g.hasMeta {
|
||||
out := make(map[string]os.FileInfo)
|
||||
info, err := os.Stat(g.path)
|
||||
if !os.IsNotExist(err) {
|
||||
out[g.path] = info
|
||||
}
|
||||
return out
|
||||
}
|
||||
return walkFilePath(g.root, g.g)
|
||||
}
|
||||
|
||||
// walk the filepath from the given root and return a list of files that match
|
||||
// the given glob.
|
||||
func walkFilePath(root string, g glob.Glob) map[string]os.FileInfo {
|
||||
matchedFiles := make(map[string]os.FileInfo)
|
||||
walkfn := func(path string, info os.FileInfo, _ error) error {
|
||||
if g.Match(path) {
|
||||
matchedFiles[path] = info
|
||||
}
|
||||
return nil
|
||||
}
|
||||
filepath.Walk(root, walkfn)
|
||||
return matchedFiles
|
||||
}
|
||||
|
||||
// find the root dir of the given path (could include globs).
|
||||
// ie:
|
||||
// /var/log/telegraf.conf -> /var/log
|
||||
// /home/** -> /home
|
||||
// /home/*/** -> /home
|
||||
// /lib/share/*/*/**.txt -> /lib/share
|
||||
func findRootDir(path string) string {
|
||||
pathItems := strings.Split(path, sepStr)
|
||||
out := sepStr
|
||||
for i, item := range pathItems {
|
||||
if i == len(pathItems)-1 {
|
||||
break
|
||||
}
|
||||
if item == "" {
|
||||
continue
|
||||
}
|
||||
if hasMeta(item) {
|
||||
break
|
||||
}
|
||||
out += item + sepStr
|
||||
}
|
||||
if out != "/" {
|
||||
out = strings.TrimSuffix(out, "/")
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
// hasMeta reports whether path contains any magic glob characters.
|
||||
func hasMeta(path string) bool {
|
||||
return strings.IndexAny(path, "*?[") >= 0
|
||||
}
|
||||
@@ -1,62 +0,0 @@
|
||||
package globpath
|
||||
|
||||
import (
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestCompileAndMatch(t *testing.T) {
|
||||
dir := getTestdataDir()
|
||||
// test super asterisk
|
||||
g1, err := Compile(dir + "/**")
|
||||
require.NoError(t, err)
|
||||
// test single asterisk
|
||||
g2, err := Compile(dir + "/*.log")
|
||||
require.NoError(t, err)
|
||||
// test no meta characters (file exists)
|
||||
g3, err := Compile(dir + "/log1.log")
|
||||
require.NoError(t, err)
|
||||
// test file that doesn't exist
|
||||
g4, err := Compile(dir + "/i_dont_exist.log")
|
||||
require.NoError(t, err)
|
||||
// test super asterisk that doesn't exist
|
||||
g5, err := Compile(dir + "/dir_doesnt_exist/**")
|
||||
require.NoError(t, err)
|
||||
|
||||
matches := g1.Match()
|
||||
assert.Len(t, matches, 3)
|
||||
matches = g2.Match()
|
||||
assert.Len(t, matches, 2)
|
||||
matches = g3.Match()
|
||||
assert.Len(t, matches, 1)
|
||||
matches = g4.Match()
|
||||
assert.Len(t, matches, 0)
|
||||
matches = g5.Match()
|
||||
assert.Len(t, matches, 0)
|
||||
}
|
||||
|
||||
func TestFindRootDir(t *testing.T) {
|
||||
tests := []struct {
|
||||
input string
|
||||
output string
|
||||
}{
|
||||
{"/var/log/telegraf.conf", "/var/log"},
|
||||
{"/home/**", "/home"},
|
||||
{"/home/*/**", "/home"},
|
||||
{"/lib/share/*/*/**.txt", "/lib/share"},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
actual := findRootDir(test.input)
|
||||
assert.Equal(t, test.output, actual)
|
||||
}
|
||||
}
|
||||
|
||||
func getTestdataDir() string {
|
||||
_, filename, _, _ := runtime.Caller(1)
|
||||
return strings.Replace(filename, "globpath_test.go", "testdata", 1)
|
||||
}
|
||||
0
internal/globpath/testdata/log1.log
vendored
0
internal/globpath/testdata/log1.log
vendored
0
internal/globpath/testdata/log2.log
vendored
0
internal/globpath/testdata/log2.log
vendored
5
internal/globpath/testdata/test.conf
vendored
5
internal/globpath/testdata/test.conf
vendored
@@ -1,5 +0,0 @@
|
||||
# this is a fake testing config file
|
||||
# for testing the filestat plugin
|
||||
|
||||
option1 = "foo"
|
||||
option2 = "bar"
|
||||
@@ -2,29 +2,19 @@ package internal
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"bytes"
|
||||
"crypto/rand"
|
||||
"crypto/tls"
|
||||
"crypto/x509"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"log"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
)
|
||||
|
||||
const alphanum string = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz"
|
||||
|
||||
var (
|
||||
TimeoutErr = errors.New("Command timed out.")
|
||||
|
||||
NotImplementedError = errors.New("not implemented yet")
|
||||
)
|
||||
|
||||
// Duration just wraps time.Duration
|
||||
type Duration struct {
|
||||
Duration time.Duration
|
||||
@@ -42,6 +32,8 @@ func (d *Duration) UnmarshalTOML(b []byte) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
var NotImplementedError = errors.New("not implemented yet")
|
||||
|
||||
// ReadLines reads contents from a file and splits them by new lines.
|
||||
// A convenience wrapper to ReadLinesOffsetN(filename, 0, -1).
|
||||
func ReadLines(filename string) ([]string, error) {
|
||||
@@ -94,15 +86,15 @@ func GetTLSConfig(
|
||||
SSLCert, SSLKey, SSLCA string,
|
||||
InsecureSkipVerify bool,
|
||||
) (*tls.Config, error) {
|
||||
if SSLCert == "" && SSLKey == "" && SSLCA == "" && !InsecureSkipVerify {
|
||||
return nil, nil
|
||||
}
|
||||
t := &tls.Config{}
|
||||
if SSLCert != "" && SSLKey != "" && SSLCA != "" {
|
||||
cert, err := tls.LoadX509KeyPair(SSLCert, SSLKey)
|
||||
if err != nil {
|
||||
return nil, errors.New(fmt.Sprintf(
|
||||
"Could not load TLS client key/certificate: %s",
|
||||
err))
|
||||
}
|
||||
|
||||
t := &tls.Config{
|
||||
InsecureSkipVerify: InsecureSkipVerify,
|
||||
}
|
||||
|
||||
if SSLCA != "" {
|
||||
caCert, err := ioutil.ReadFile(SSLCA)
|
||||
if err != nil {
|
||||
return nil, errors.New(fmt.Sprintf("Could not load TLS CA: %s",
|
||||
@@ -111,83 +103,75 @@ func GetTLSConfig(
|
||||
|
||||
caCertPool := x509.NewCertPool()
|
||||
caCertPool.AppendCertsFromPEM(caCert)
|
||||
t.RootCAs = caCertPool
|
||||
}
|
||||
|
||||
if SSLCert != "" && SSLKey != "" {
|
||||
cert, err := tls.LoadX509KeyPair(SSLCert, SSLKey)
|
||||
if err != nil {
|
||||
return nil, errors.New(fmt.Sprintf(
|
||||
"Could not load TLS client key/certificate: %s",
|
||||
err))
|
||||
t = &tls.Config{
|
||||
Certificates: []tls.Certificate{cert},
|
||||
RootCAs: caCertPool,
|
||||
InsecureSkipVerify: InsecureSkipVerify,
|
||||
}
|
||||
} else {
|
||||
if InsecureSkipVerify {
|
||||
t.InsecureSkipVerify = true
|
||||
} else {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
t.Certificates = []tls.Certificate{cert}
|
||||
t.BuildNameToCertificate()
|
||||
}
|
||||
|
||||
// will be nil by default if nothing is provided
|
||||
return t, nil
|
||||
}
|
||||
|
||||
// SnakeCase converts the given string to snake case following the Golang format:
|
||||
// acronyms are converted to lower-case and preceded by an underscore.
|
||||
func SnakeCase(in string) string {
|
||||
runes := []rune(in)
|
||||
length := len(runes)
|
||||
// Glob will test a string pattern, potentially containing globs, against a
|
||||
// subject string. The result is a simple true/false, determining whether or
|
||||
// not the glob pattern matched the subject text.
|
||||
//
|
||||
// Adapted from https://github.com/ryanuber/go-glob/blob/master/glob.go
|
||||
// thanks Ryan Uber!
|
||||
func Glob(pattern, measurement string) bool {
|
||||
// Empty pattern can only match empty subject
|
||||
if pattern == "" {
|
||||
return measurement == pattern
|
||||
}
|
||||
|
||||
var out []rune
|
||||
for i := 0; i < length; i++ {
|
||||
if i > 0 && unicode.IsUpper(runes[i]) && ((i+1 < length && unicode.IsLower(runes[i+1])) || unicode.IsLower(runes[i-1])) {
|
||||
out = append(out, '_')
|
||||
// If the pattern _is_ a glob, it matches everything
|
||||
if pattern == "*" {
|
||||
return true
|
||||
}
|
||||
|
||||
parts := strings.Split(pattern, "*")
|
||||
|
||||
if len(parts) == 1 {
|
||||
// No globs in pattern, so test for match
|
||||
return pattern == measurement
|
||||
}
|
||||
|
||||
leadingGlob := strings.HasPrefix(pattern, "*")
|
||||
trailingGlob := strings.HasSuffix(pattern, "*")
|
||||
end := len(parts) - 1
|
||||
|
||||
for i, part := range parts {
|
||||
switch i {
|
||||
case 0:
|
||||
if leadingGlob {
|
||||
continue
|
||||
}
|
||||
if !strings.HasPrefix(measurement, part) {
|
||||
return false
|
||||
}
|
||||
case end:
|
||||
if len(measurement) > 0 {
|
||||
return trailingGlob || strings.HasSuffix(measurement, part)
|
||||
}
|
||||
default:
|
||||
if !strings.Contains(measurement, part) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
out = append(out, unicode.ToLower(runes[i]))
|
||||
|
||||
// Trim evaluated text from measurement as we loop over the pattern.
|
||||
idx := strings.Index(measurement, part) + len(part)
|
||||
measurement = measurement[idx:]
|
||||
}
|
||||
|
||||
return string(out)
|
||||
}
|
||||
|
||||
// CombinedOutputTimeout runs the given command with the given timeout and
|
||||
// returns the combined output of stdout and stderr.
|
||||
// If the command times out, it attempts to kill the process.
|
||||
func CombinedOutputTimeout(c *exec.Cmd, timeout time.Duration) ([]byte, error) {
|
||||
var b bytes.Buffer
|
||||
c.Stdout = &b
|
||||
c.Stderr = &b
|
||||
if err := c.Start(); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
err := WaitTimeout(c, timeout)
|
||||
return b.Bytes(), err
|
||||
}
|
||||
|
||||
// RunTimeout runs the given command with the given timeout.
|
||||
// If the command times out, it attempts to kill the process.
|
||||
func RunTimeout(c *exec.Cmd, timeout time.Duration) error {
|
||||
if err := c.Start(); err != nil {
|
||||
return err
|
||||
}
|
||||
return WaitTimeout(c, timeout)
|
||||
}
|
||||
|
||||
// WaitTimeout waits for the given command to finish with a timeout.
|
||||
// It assumes the command has already been started.
|
||||
// If the command times out, it attempts to kill the process.
|
||||
func WaitTimeout(c *exec.Cmd, timeout time.Duration) error {
|
||||
timer := time.NewTimer(timeout)
|
||||
done := make(chan error)
|
||||
go func() { done <- c.Wait() }()
|
||||
select {
|
||||
case err := <-done:
|
||||
timer.Stop()
|
||||
return err
|
||||
case <-timer.C:
|
||||
if err := c.Process.Kill(); err != nil {
|
||||
log.Printf("FATAL error killing process: %s", err)
|
||||
return err
|
||||
}
|
||||
// wait for the command to return after killing it
|
||||
<-done
|
||||
return TimeoutErr
|
||||
}
|
||||
// All parts of the pattern matched
|
||||
return true
|
||||
}
|
||||
|
||||
@@ -1,108 +1,44 @@
|
||||
package internal
|
||||
|
||||
import (
|
||||
"os/exec"
|
||||
"testing"
|
||||
"time"
|
||||
import "testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
type SnakeTest struct {
|
||||
input string
|
||||
output string
|
||||
}
|
||||
|
||||
var tests = []SnakeTest{
|
||||
{"a", "a"},
|
||||
{"snake", "snake"},
|
||||
{"A", "a"},
|
||||
{"ID", "id"},
|
||||
{"MOTD", "motd"},
|
||||
{"Snake", "snake"},
|
||||
{"SnakeTest", "snake_test"},
|
||||
{"APIResponse", "api_response"},
|
||||
{"SnakeID", "snake_id"},
|
||||
{"SnakeIDGoogle", "snake_id_google"},
|
||||
{"LinuxMOTD", "linux_motd"},
|
||||
{"OMGWTFBBQ", "omgwtfbbq"},
|
||||
{"omg_wtf_bbq", "omg_wtf_bbq"},
|
||||
}
|
||||
|
||||
func TestSnakeCase(t *testing.T) {
|
||||
for _, test := range tests {
|
||||
if SnakeCase(test.input) != test.output {
|
||||
t.Errorf(`SnakeCase("%s"), wanted "%s", got \%s"`, test.input, test.output, SnakeCase(test.input))
|
||||
}
|
||||
func testGlobMatch(t *testing.T, pattern, subj string) {
|
||||
if !Glob(pattern, subj) {
|
||||
t.Errorf("%s should match %s", pattern, subj)
|
||||
}
|
||||
}
|
||||
|
||||
var (
|
||||
sleepbin, _ = exec.LookPath("sleep")
|
||||
echobin, _ = exec.LookPath("echo")
|
||||
)
|
||||
|
||||
func TestRunTimeout(t *testing.T) {
|
||||
if sleepbin == "" {
|
||||
t.Skip("'sleep' binary not available on OS, skipping.")
|
||||
func testGlobNoMatch(t *testing.T, pattern, subj string) {
|
||||
if Glob(pattern, subj) {
|
||||
t.Errorf("%s should not match %s", pattern, subj)
|
||||
}
|
||||
cmd := exec.Command(sleepbin, "10")
|
||||
start := time.Now()
|
||||
err := RunTimeout(cmd, time.Millisecond*20)
|
||||
elapsed := time.Since(start)
|
||||
|
||||
assert.Equal(t, TimeoutErr, err)
|
||||
// Verify that command gets killed in 20ms, with some breathing room
|
||||
assert.True(t, elapsed < time.Millisecond*75)
|
||||
}
|
||||
|
||||
func TestCombinedOutputTimeout(t *testing.T) {
|
||||
if sleepbin == "" {
|
||||
t.Skip("'sleep' binary not available on OS, skipping.")
|
||||
}
|
||||
cmd := exec.Command(sleepbin, "10")
|
||||
start := time.Now()
|
||||
_, err := CombinedOutputTimeout(cmd, time.Millisecond*20)
|
||||
elapsed := time.Since(start)
|
||||
|
||||
assert.Equal(t, TimeoutErr, err)
|
||||
// Verify that command gets killed in 20ms, with some breathing room
|
||||
assert.True(t, elapsed < time.Millisecond*75)
|
||||
func TestEmptyPattern(t *testing.T) {
|
||||
testGlobMatch(t, "", "")
|
||||
testGlobNoMatch(t, "", "test")
|
||||
}
|
||||
|
||||
func TestCombinedOutput(t *testing.T) {
|
||||
if echobin == "" {
|
||||
t.Skip("'echo' binary not available on OS, skipping.")
|
||||
}
|
||||
cmd := exec.Command(echobin, "foo")
|
||||
out, err := CombinedOutputTimeout(cmd, time.Second)
|
||||
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, "foo\n", string(out))
|
||||
func TestPatternWithoutGlobs(t *testing.T) {
|
||||
testGlobMatch(t, "test", "test")
|
||||
}
|
||||
|
||||
// test that CombinedOutputTimeout and exec.Cmd.CombinedOutput return
|
||||
// the same output from a failed command.
|
||||
func TestCombinedOutputError(t *testing.T) {
|
||||
if sleepbin == "" {
|
||||
t.Skip("'sleep' binary not available on OS, skipping.")
|
||||
func TestGlob(t *testing.T) {
|
||||
for _, pattern := range []string{
|
||||
"*test", // Leading glob
|
||||
"this*", // Trailing glob
|
||||
"*is*a*", // Lots of globs
|
||||
"**test**", // Double glob characters
|
||||
"**is**a***test*", // Varying number of globs
|
||||
} {
|
||||
testGlobMatch(t, pattern, "this_is_a_test")
|
||||
}
|
||||
cmd := exec.Command(sleepbin, "foo")
|
||||
expected, err := cmd.CombinedOutput()
|
||||
|
||||
cmd2 := exec.Command(sleepbin, "foo")
|
||||
actual, err := CombinedOutputTimeout(cmd2, time.Second)
|
||||
|
||||
assert.Error(t, err)
|
||||
assert.Equal(t, expected, actual)
|
||||
}
|
||||
|
||||
func TestRunError(t *testing.T) {
|
||||
if sleepbin == "" {
|
||||
t.Skip("'sleep' binary not available on OS, skipping.")
|
||||
for _, pattern := range []string{
|
||||
"test*", // Implicit substring match should fail
|
||||
"*is", // Partial match should fail
|
||||
"*no*", // Globs without a match between them should fail
|
||||
} {
|
||||
testGlobNoMatch(t, pattern, "this_is_a_test")
|
||||
}
|
||||
cmd := exec.Command(sleepbin, "foo")
|
||||
err := RunTimeout(cmd, time.Second)
|
||||
|
||||
assert.Error(t, err)
|
||||
}
|
||||
|
||||
@@ -1,104 +1,33 @@
|
||||
package internal_models
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/gobwas/glob"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
)
|
||||
|
||||
// TagFilter is the name of a tag, and the values on which to filter
|
||||
type TagFilter struct {
|
||||
Name string
|
||||
Filter []string
|
||||
filter glob.Glob
|
||||
}
|
||||
|
||||
// Filter containing drop/pass and tagdrop/tagpass rules
|
||||
type Filter struct {
|
||||
NameDrop []string
|
||||
nameDrop glob.Glob
|
||||
NamePass []string
|
||||
namePass glob.Glob
|
||||
|
||||
FieldDrop []string
|
||||
fieldDrop glob.Glob
|
||||
FieldPass []string
|
||||
fieldPass glob.Glob
|
||||
|
||||
TagDrop []TagFilter
|
||||
TagPass []TagFilter
|
||||
|
||||
TagExclude []string
|
||||
tagExclude glob.Glob
|
||||
TagInclude []string
|
||||
tagInclude glob.Glob
|
||||
|
||||
IsActive bool
|
||||
}
|
||||
|
||||
// Compile all Filter lists into glob.Glob objects.
|
||||
func (f *Filter) CompileFilter() error {
|
||||
var err error
|
||||
f.nameDrop, err = compileFilter(f.NameDrop)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'namedrop', %s", err)
|
||||
}
|
||||
f.namePass, err = compileFilter(f.NamePass)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'namepass', %s", err)
|
||||
}
|
||||
|
||||
f.fieldDrop, err = compileFilter(f.FieldDrop)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'fielddrop', %s", err)
|
||||
}
|
||||
f.fieldPass, err = compileFilter(f.FieldPass)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'fieldpass', %s", err)
|
||||
}
|
||||
|
||||
f.tagExclude, err = compileFilter(f.TagExclude)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'tagexclude', %s", err)
|
||||
}
|
||||
f.tagInclude, err = compileFilter(f.TagInclude)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'taginclude', %s", err)
|
||||
}
|
||||
|
||||
for i, _ := range f.TagDrop {
|
||||
f.TagDrop[i].filter, err = compileFilter(f.TagDrop[i].Filter)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'tagdrop', %s", err)
|
||||
}
|
||||
}
|
||||
for i, _ := range f.TagPass {
|
||||
f.TagPass[i].filter, err = compileFilter(f.TagPass[i].Filter)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'tagpass', %s", err)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func compileFilter(filter []string) (glob.Glob, error) {
|
||||
if len(filter) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
var g glob.Glob
|
||||
var err error
|
||||
if len(filter) == 1 {
|
||||
g, err = glob.Compile(filter[0])
|
||||
} else {
|
||||
g, err = glob.Compile("{" + strings.Join(filter, ",") + "}")
|
||||
}
|
||||
return g, err
|
||||
}
|
||||
|
||||
func (f *Filter) ShouldMetricPass(metric telegraf.Metric) bool {
|
||||
func (f Filter) ShouldMetricPass(metric telegraf.Metric) bool {
|
||||
if f.ShouldNamePass(metric.Name()) && f.ShouldTagsPass(metric.Tags()) {
|
||||
return true
|
||||
}
|
||||
@@ -107,51 +36,70 @@ func (f *Filter) ShouldMetricPass(metric telegraf.Metric) bool {
|
||||
|
||||
// ShouldFieldsPass returns true if the metric should pass, false if should drop
|
||||
// based on the drop/pass filter parameters
|
||||
func (f *Filter) ShouldNamePass(key string) bool {
|
||||
if f.namePass != nil {
|
||||
if f.namePass.Match(key) {
|
||||
return true
|
||||
func (f Filter) ShouldNamePass(key string) bool {
|
||||
if f.NamePass != nil {
|
||||
for _, pat := range f.NamePass {
|
||||
// TODO remove HasPrefix check, leaving it for now for legacy support.
|
||||
// Cam, 2015-12-07
|
||||
if strings.HasPrefix(key, pat) || internal.Glob(pat, key) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
if f.nameDrop != nil {
|
||||
if f.nameDrop.Match(key) {
|
||||
return false
|
||||
if f.NameDrop != nil {
|
||||
for _, pat := range f.NameDrop {
|
||||
// TODO remove HasPrefix check, leaving it for now for legacy support.
|
||||
// Cam, 2015-12-07
|
||||
if strings.HasPrefix(key, pat) || internal.Glob(pat, key) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
return true
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// ShouldFieldsPass returns true if the metric should pass, false if should drop
|
||||
// based on the drop/pass filter parameters
|
||||
func (f *Filter) ShouldFieldsPass(key string) bool {
|
||||
if f.fieldPass != nil {
|
||||
if f.fieldPass.Match(key) {
|
||||
return true
|
||||
func (f Filter) ShouldFieldsPass(key string) bool {
|
||||
if f.FieldPass != nil {
|
||||
for _, pat := range f.FieldPass {
|
||||
// TODO remove HasPrefix check, leaving it for now for legacy support.
|
||||
// Cam, 2015-12-07
|
||||
if strings.HasPrefix(key, pat) || internal.Glob(pat, key) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
if f.fieldDrop != nil {
|
||||
if f.fieldDrop.Match(key) {
|
||||
return false
|
||||
if f.FieldDrop != nil {
|
||||
for _, pat := range f.FieldDrop {
|
||||
// TODO remove HasPrefix check, leaving it for now for legacy support.
|
||||
// Cam, 2015-12-07
|
||||
if strings.HasPrefix(key, pat) || internal.Glob(pat, key) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
return true
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
// ShouldTagsPass returns true if the metric should pass, false if should drop
|
||||
// based on the tagdrop/tagpass filter parameters
|
||||
func (f *Filter) ShouldTagsPass(tags map[string]string) bool {
|
||||
func (f Filter) ShouldTagsPass(tags map[string]string) bool {
|
||||
if f.TagPass != nil {
|
||||
for _, pat := range f.TagPass {
|
||||
if pat.filter == nil {
|
||||
continue
|
||||
}
|
||||
if tagval, ok := tags[pat.Name]; ok {
|
||||
if pat.filter.Match(tagval) {
|
||||
return true
|
||||
for _, filter := range pat.Filter {
|
||||
if internal.Glob(filter, tagval) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -160,12 +108,11 @@ func (f *Filter) ShouldTagsPass(tags map[string]string) bool {
|
||||
|
||||
if f.TagDrop != nil {
|
||||
for _, pat := range f.TagDrop {
|
||||
if pat.filter == nil {
|
||||
continue
|
||||
}
|
||||
if tagval, ok := tags[pat.Name]; ok {
|
||||
if pat.filter.Match(tagval) {
|
||||
return false
|
||||
for _, filter := range pat.Filter {
|
||||
if internal.Glob(filter, tagval) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -174,23 +121,3 @@ func (f *Filter) ShouldTagsPass(tags map[string]string) bool {
|
||||
|
||||
return true
|
||||
}
|
||||
|
||||
// Apply TagInclude and TagExclude filters.
|
||||
// modifies the tags map in-place.
|
||||
func (f *Filter) FilterTags(tags map[string]string) {
|
||||
if f.tagInclude != nil {
|
||||
for k, _ := range tags {
|
||||
if !f.tagInclude.Match(k) {
|
||||
delete(tags, k)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if f.tagExclude != nil {
|
||||
for k, _ := range tags {
|
||||
if f.tagExclude.Match(k) {
|
||||
delete(tags, k)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2,11 +2,6 @@ package internal_models
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func TestFilter_Empty(t *testing.T) {
|
||||
@@ -33,7 +28,6 @@ func TestFilter_NamePass(t *testing.T) {
|
||||
f := Filter{
|
||||
NamePass: []string{"foo*", "cpu_usage_idle"},
|
||||
}
|
||||
require.NoError(t, f.CompileFilter())
|
||||
|
||||
passes := []string{
|
||||
"foo",
|
||||
@@ -67,7 +61,6 @@ func TestFilter_NameDrop(t *testing.T) {
|
||||
f := Filter{
|
||||
NameDrop: []string{"foo*", "cpu_usage_idle"},
|
||||
}
|
||||
require.NoError(t, f.CompileFilter())
|
||||
|
||||
drops := []string{
|
||||
"foo",
|
||||
@@ -101,7 +94,6 @@ func TestFilter_FieldPass(t *testing.T) {
|
||||
f := Filter{
|
||||
FieldPass: []string{"foo*", "cpu_usage_idle"},
|
||||
}
|
||||
require.NoError(t, f.CompileFilter())
|
||||
|
||||
passes := []string{
|
||||
"foo",
|
||||
@@ -135,7 +127,6 @@ func TestFilter_FieldDrop(t *testing.T) {
|
||||
f := Filter{
|
||||
FieldDrop: []string{"foo*", "cpu_usage_idle"},
|
||||
}
|
||||
require.NoError(t, f.CompileFilter())
|
||||
|
||||
drops := []string{
|
||||
"foo",
|
||||
@@ -178,7 +169,6 @@ func TestFilter_TagPass(t *testing.T) {
|
||||
f := Filter{
|
||||
TagPass: filters,
|
||||
}
|
||||
require.NoError(t, f.CompileFilter())
|
||||
|
||||
passes := []map[string]string{
|
||||
{"cpu": "cpu-total"},
|
||||
@@ -222,7 +212,6 @@ func TestFilter_TagDrop(t *testing.T) {
|
||||
f := Filter{
|
||||
TagDrop: filters,
|
||||
}
|
||||
require.NoError(t, f.CompileFilter())
|
||||
|
||||
drops := []map[string]string{
|
||||
{"cpu": "cpu-total"},
|
||||
@@ -252,115 +241,3 @@ func TestFilter_TagDrop(t *testing.T) {
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestFilter_CompileFilterError(t *testing.T) {
|
||||
f := Filter{
|
||||
NameDrop: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
NamePass: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
FieldDrop: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
FieldPass: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
TagExclude: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
TagInclude: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
filters := []TagFilter{
|
||||
TagFilter{
|
||||
Name: "cpu",
|
||||
Filter: []string{"{foobar}"},
|
||||
}}
|
||||
f = Filter{
|
||||
TagDrop: filters,
|
||||
}
|
||||
require.Error(t, f.CompileFilter())
|
||||
filters = []TagFilter{
|
||||
TagFilter{
|
||||
Name: "cpu",
|
||||
Filter: []string{"{foobar}"},
|
||||
}}
|
||||
f = Filter{
|
||||
TagPass: filters,
|
||||
}
|
||||
require.Error(t, f.CompileFilter())
|
||||
}
|
||||
|
||||
func TestFilter_ShouldMetricsPass(t *testing.T) {
|
||||
m := testutil.TestMetric(1, "testmetric")
|
||||
f := Filter{
|
||||
NameDrop: []string{"foobar"},
|
||||
}
|
||||
require.NoError(t, f.CompileFilter())
|
||||
require.True(t, f.ShouldMetricPass(m))
|
||||
|
||||
m = testutil.TestMetric(1, "foobar")
|
||||
require.False(t, f.ShouldMetricPass(m))
|
||||
}
|
||||
|
||||
func TestFilter_FilterTagsNoMatches(t *testing.T) {
|
||||
pretags := map[string]string{
|
||||
"host": "localhost",
|
||||
"mytag": "foobar",
|
||||
}
|
||||
f := Filter{
|
||||
TagExclude: []string{"nomatch"},
|
||||
}
|
||||
require.NoError(t, f.CompileFilter())
|
||||
|
||||
f.FilterTags(pretags)
|
||||
assert.Equal(t, map[string]string{
|
||||
"host": "localhost",
|
||||
"mytag": "foobar",
|
||||
}, pretags)
|
||||
|
||||
f = Filter{
|
||||
TagInclude: []string{"nomatch"},
|
||||
}
|
||||
require.NoError(t, f.CompileFilter())
|
||||
|
||||
f.FilterTags(pretags)
|
||||
assert.Equal(t, map[string]string{}, pretags)
|
||||
}
|
||||
|
||||
func TestFilter_FilterTagsMatches(t *testing.T) {
|
||||
pretags := map[string]string{
|
||||
"host": "localhost",
|
||||
"mytag": "foobar",
|
||||
}
|
||||
f := Filter{
|
||||
TagExclude: []string{"ho*"},
|
||||
}
|
||||
require.NoError(t, f.CompileFilter())
|
||||
|
||||
f.FilterTags(pretags)
|
||||
assert.Equal(t, map[string]string{
|
||||
"mytag": "foobar",
|
||||
}, pretags)
|
||||
|
||||
pretags = map[string]string{
|
||||
"host": "localhost",
|
||||
"mytag": "foobar",
|
||||
}
|
||||
f = Filter{
|
||||
TagInclude: []string{"my*"},
|
||||
}
|
||||
require.NoError(t, f.CompileFilter())
|
||||
|
||||
f.FilterTags(pretags)
|
||||
assert.Equal(t, map[string]string{
|
||||
"mytag": "foobar",
|
||||
}, pretags)
|
||||
}
|
||||
|
||||
@@ -2,54 +2,48 @@ package internal_models
|
||||
|
||||
import (
|
||||
"log"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal/buffer"
|
||||
)
|
||||
|
||||
const (
|
||||
// Default size of metrics batch size.
|
||||
DEFAULT_METRIC_BATCH_SIZE = 1000
|
||||
|
||||
// Default number of metrics kept. It should be a multiple of batch size.
|
||||
// Default number of metrics kept between flushes.
|
||||
DEFAULT_METRIC_BUFFER_LIMIT = 10000
|
||||
|
||||
// Limit how many full metric buffers are kept due to failed writes.
|
||||
FULL_METRIC_BUFFERS_LIMIT = 100
|
||||
)
|
||||
|
||||
// RunningOutput contains the output configuration
|
||||
type RunningOutput struct {
|
||||
Name string
|
||||
Output telegraf.Output
|
||||
Config *OutputConfig
|
||||
Quiet bool
|
||||
MetricBufferLimit int
|
||||
MetricBatchSize int
|
||||
Name string
|
||||
Output telegraf.Output
|
||||
Config *OutputConfig
|
||||
Quiet bool
|
||||
MetricBufferLimit int
|
||||
FlushBufferWhenFull bool
|
||||
|
||||
metrics *buffer.Buffer
|
||||
failMetrics *buffer.Buffer
|
||||
metrics []telegraf.Metric
|
||||
tmpmetrics map[int][]telegraf.Metric
|
||||
overwriteI int
|
||||
mapI int
|
||||
|
||||
sync.Mutex
|
||||
}
|
||||
|
||||
func NewRunningOutput(
|
||||
name string,
|
||||
output telegraf.Output,
|
||||
conf *OutputConfig,
|
||||
batchSize int,
|
||||
bufferLimit int,
|
||||
) *RunningOutput {
|
||||
if bufferLimit == 0 {
|
||||
bufferLimit = DEFAULT_METRIC_BUFFER_LIMIT
|
||||
}
|
||||
if batchSize == 0 {
|
||||
batchSize = DEFAULT_METRIC_BATCH_SIZE
|
||||
}
|
||||
ro := &RunningOutput{
|
||||
Name: name,
|
||||
metrics: buffer.NewBuffer(batchSize),
|
||||
failMetrics: buffer.NewBuffer(bufferLimit),
|
||||
metrics: make([]telegraf.Metric, 0),
|
||||
tmpmetrics: make(map[int][]telegraf.Metric),
|
||||
Output: output,
|
||||
Config: conf,
|
||||
MetricBufferLimit: bufferLimit,
|
||||
MetricBatchSize: batchSize,
|
||||
MetricBufferLimit: DEFAULT_METRIC_BUFFER_LIMIT,
|
||||
}
|
||||
return ro
|
||||
}
|
||||
@@ -62,92 +56,76 @@ func (ro *RunningOutput) AddMetric(metric telegraf.Metric) {
|
||||
return
|
||||
}
|
||||
}
|
||||
ro.Lock()
|
||||
defer ro.Unlock()
|
||||
|
||||
// Filter any tagexclude/taginclude parameters before adding metric
|
||||
if len(ro.Config.Filter.TagExclude) != 0 || len(ro.Config.Filter.TagInclude) != 0 {
|
||||
// In order to filter out tags, we need to create a new metric, since
|
||||
// metrics are immutable once created.
|
||||
tags := metric.Tags()
|
||||
fields := metric.Fields()
|
||||
t := metric.Time()
|
||||
name := metric.Name()
|
||||
ro.Config.Filter.FilterTags(tags)
|
||||
// error is not possible if creating from another metric, so ignore.
|
||||
metric, _ = telegraf.NewMetric(name, tags, fields, t)
|
||||
}
|
||||
|
||||
ro.metrics.Add(metric)
|
||||
if ro.metrics.Len() == ro.MetricBatchSize {
|
||||
batch := ro.metrics.Batch(ro.MetricBatchSize)
|
||||
err := ro.write(batch)
|
||||
if err != nil {
|
||||
ro.failMetrics.Add(batch...)
|
||||
if len(ro.metrics) < ro.MetricBufferLimit {
|
||||
ro.metrics = append(ro.metrics, metric)
|
||||
} else {
|
||||
if ro.FlushBufferWhenFull {
|
||||
ro.metrics = append(ro.metrics, metric)
|
||||
tmpmetrics := make([]telegraf.Metric, len(ro.metrics))
|
||||
copy(tmpmetrics, ro.metrics)
|
||||
ro.metrics = make([]telegraf.Metric, 0)
|
||||
err := ro.write(tmpmetrics)
|
||||
if err != nil {
|
||||
log.Printf("ERROR writing full metric buffer to output %s, %s",
|
||||
ro.Name, err)
|
||||
if len(ro.tmpmetrics) == FULL_METRIC_BUFFERS_LIMIT {
|
||||
ro.mapI = 0
|
||||
// overwrite one
|
||||
ro.tmpmetrics[ro.mapI] = tmpmetrics
|
||||
ro.mapI++
|
||||
} else {
|
||||
ro.tmpmetrics[ro.mapI] = tmpmetrics
|
||||
ro.mapI++
|
||||
}
|
||||
}
|
||||
} else {
|
||||
log.Printf("WARNING: overwriting cached metrics, you may want to " +
|
||||
"increase the metric_buffer_limit setting in your [agent] " +
|
||||
"config if you do not wish to overwrite metrics.\n")
|
||||
if ro.overwriteI == len(ro.metrics) {
|
||||
ro.overwriteI = 0
|
||||
}
|
||||
ro.metrics[ro.overwriteI] = metric
|
||||
ro.overwriteI++
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Write writes all cached points to this output.
|
||||
func (ro *RunningOutput) Write() error {
|
||||
if !ro.Quiet {
|
||||
log.Printf("Output [%s] buffer fullness: %d / %d metrics. "+
|
||||
"Total gathered metrics: %d. Total dropped metrics: %d.",
|
||||
ro.Name,
|
||||
ro.failMetrics.Len()+ro.metrics.Len(),
|
||||
ro.MetricBufferLimit,
|
||||
ro.metrics.Total(),
|
||||
ro.metrics.Drops()+ro.failMetrics.Drops())
|
||||
ro.Lock()
|
||||
defer ro.Unlock()
|
||||
err := ro.write(ro.metrics)
|
||||
if err != nil {
|
||||
return err
|
||||
} else {
|
||||
ro.metrics = make([]telegraf.Metric, 0)
|
||||
ro.overwriteI = 0
|
||||
}
|
||||
|
||||
var err error
|
||||
if !ro.failMetrics.IsEmpty() {
|
||||
bufLen := ro.failMetrics.Len()
|
||||
// how many batches of failed writes we need to write.
|
||||
nBatches := bufLen/ro.MetricBatchSize + 1
|
||||
batchSize := ro.MetricBatchSize
|
||||
|
||||
for i := 0; i < nBatches; i++ {
|
||||
// If it's the last batch, only grab the metrics that have not had
|
||||
// a write attempt already (this is primarily to preserve order).
|
||||
if i == nBatches-1 {
|
||||
batchSize = bufLen % ro.MetricBatchSize
|
||||
}
|
||||
batch := ro.failMetrics.Batch(batchSize)
|
||||
// If we've already failed previous writes, don't bother trying to
|
||||
// write to this output again. We are not exiting the loop just so
|
||||
// that we can rotate the metrics to preserve order.
|
||||
if err == nil {
|
||||
err = ro.write(batch)
|
||||
}
|
||||
if err != nil {
|
||||
ro.failMetrics.Add(batch...)
|
||||
}
|
||||
// Write any cached metric buffers that failed previously
|
||||
for i, tmpmetrics := range ro.tmpmetrics {
|
||||
if err := ro.write(tmpmetrics); err != nil {
|
||||
return err
|
||||
} else {
|
||||
delete(ro.tmpmetrics, i)
|
||||
}
|
||||
}
|
||||
|
||||
batch := ro.metrics.Batch(ro.MetricBatchSize)
|
||||
// see comment above about not trying to write to an already failed output.
|
||||
// if ro.failMetrics is empty then err will always be nil at this point.
|
||||
if err == nil {
|
||||
err = ro.write(batch)
|
||||
}
|
||||
if err != nil {
|
||||
ro.failMetrics.Add(batch...)
|
||||
return err
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (ro *RunningOutput) write(metrics []telegraf.Metric) error {
|
||||
if len(metrics) == 0 {
|
||||
return nil
|
||||
}
|
||||
start := time.Now()
|
||||
err := ro.Output.Write(metrics)
|
||||
elapsed := time.Since(start)
|
||||
if err == nil {
|
||||
if !ro.Quiet {
|
||||
log.Printf("Output [%s] wrote batch of %d metrics in %s\n",
|
||||
ro.Name, len(metrics), elapsed)
|
||||
log.Printf("Wrote %d metrics to output %s in %s\n",
|
||||
len(metrics), ro.Name, elapsed)
|
||||
}
|
||||
}
|
||||
return err
|
||||
|
||||
@@ -2,6 +2,7 @@ package internal_models
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"sort"
|
||||
"sync"
|
||||
"testing"
|
||||
|
||||
@@ -28,202 +29,6 @@ var next5 = []telegraf.Metric{
|
||||
testutil.TestMetric(101, "metric10"),
|
||||
}
|
||||
|
||||
// Benchmark adding metrics.
|
||||
func BenchmarkRunningOutputAddWrite(b *testing.B) {
|
||||
conf := &OutputConfig{
|
||||
Filter: Filter{
|
||||
IsActive: false,
|
||||
},
|
||||
}
|
||||
|
||||
m := &perfOutput{}
|
||||
ro := NewRunningOutput("test", m, conf, 1000, 10000)
|
||||
ro.Quiet = true
|
||||
|
||||
for n := 0; n < b.N; n++ {
|
||||
ro.AddMetric(first5[0])
|
||||
ro.Write()
|
||||
}
|
||||
}
|
||||
|
||||
// Benchmark adding metrics.
|
||||
func BenchmarkRunningOutputAddWriteEvery100(b *testing.B) {
|
||||
conf := &OutputConfig{
|
||||
Filter: Filter{
|
||||
IsActive: false,
|
||||
},
|
||||
}
|
||||
|
||||
m := &perfOutput{}
|
||||
ro := NewRunningOutput("test", m, conf, 1000, 10000)
|
||||
ro.Quiet = true
|
||||
|
||||
for n := 0; n < b.N; n++ {
|
||||
ro.AddMetric(first5[0])
|
||||
if n%100 == 0 {
|
||||
ro.Write()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Benchmark adding metrics.
|
||||
func BenchmarkRunningOutputAddFailWrites(b *testing.B) {
|
||||
conf := &OutputConfig{
|
||||
Filter: Filter{
|
||||
IsActive: false,
|
||||
},
|
||||
}
|
||||
|
||||
m := &perfOutput{}
|
||||
m.failWrite = true
|
||||
ro := NewRunningOutput("test", m, conf, 1000, 10000)
|
||||
ro.Quiet = true
|
||||
|
||||
for n := 0; n < b.N; n++ {
|
||||
ro.AddMetric(first5[0])
|
||||
}
|
||||
}
|
||||
|
||||
// Test that NameDrop filters ger properly applied.
|
||||
func TestRunningOutput_DropFilter(t *testing.T) {
|
||||
conf := &OutputConfig{
|
||||
Filter: Filter{
|
||||
IsActive: true,
|
||||
NameDrop: []string{"metric1", "metric2"},
|
||||
},
|
||||
}
|
||||
assert.NoError(t, conf.Filter.CompileFilter())
|
||||
|
||||
m := &mockOutput{}
|
||||
ro := NewRunningOutput("test", m, conf, 1000, 10000)
|
||||
|
||||
for _, metric := range first5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
for _, metric := range next5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
assert.Len(t, m.Metrics(), 0)
|
||||
|
||||
err := ro.Write()
|
||||
assert.NoError(t, err)
|
||||
assert.Len(t, m.Metrics(), 8)
|
||||
}
|
||||
|
||||
// Test that NameDrop filters without a match do nothing.
|
||||
func TestRunningOutput_PassFilter(t *testing.T) {
|
||||
conf := &OutputConfig{
|
||||
Filter: Filter{
|
||||
IsActive: true,
|
||||
NameDrop: []string{"metric1000", "foo*"},
|
||||
},
|
||||
}
|
||||
assert.NoError(t, conf.Filter.CompileFilter())
|
||||
|
||||
m := &mockOutput{}
|
||||
ro := NewRunningOutput("test", m, conf, 1000, 10000)
|
||||
|
||||
for _, metric := range first5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
for _, metric := range next5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
assert.Len(t, m.Metrics(), 0)
|
||||
|
||||
err := ro.Write()
|
||||
assert.NoError(t, err)
|
||||
assert.Len(t, m.Metrics(), 10)
|
||||
}
|
||||
|
||||
// Test that tags are properly included
|
||||
func TestRunningOutput_TagIncludeNoMatch(t *testing.T) {
|
||||
conf := &OutputConfig{
|
||||
Filter: Filter{
|
||||
IsActive: true,
|
||||
TagInclude: []string{"nothing*"},
|
||||
},
|
||||
}
|
||||
assert.NoError(t, conf.Filter.CompileFilter())
|
||||
|
||||
m := &mockOutput{}
|
||||
ro := NewRunningOutput("test", m, conf, 1000, 10000)
|
||||
|
||||
ro.AddMetric(first5[0])
|
||||
assert.Len(t, m.Metrics(), 0)
|
||||
|
||||
err := ro.Write()
|
||||
assert.NoError(t, err)
|
||||
assert.Len(t, m.Metrics(), 1)
|
||||
assert.Empty(t, m.Metrics()[0].Tags())
|
||||
}
|
||||
|
||||
// Test that tags are properly excluded
|
||||
func TestRunningOutput_TagExcludeMatch(t *testing.T) {
|
||||
conf := &OutputConfig{
|
||||
Filter: Filter{
|
||||
IsActive: true,
|
||||
TagExclude: []string{"tag*"},
|
||||
},
|
||||
}
|
||||
assert.NoError(t, conf.Filter.CompileFilter())
|
||||
|
||||
m := &mockOutput{}
|
||||
ro := NewRunningOutput("test", m, conf, 1000, 10000)
|
||||
|
||||
ro.AddMetric(first5[0])
|
||||
assert.Len(t, m.Metrics(), 0)
|
||||
|
||||
err := ro.Write()
|
||||
assert.NoError(t, err)
|
||||
assert.Len(t, m.Metrics(), 1)
|
||||
assert.Len(t, m.Metrics()[0].Tags(), 0)
|
||||
}
|
||||
|
||||
// Test that tags are properly Excluded
|
||||
func TestRunningOutput_TagExcludeNoMatch(t *testing.T) {
|
||||
conf := &OutputConfig{
|
||||
Filter: Filter{
|
||||
IsActive: true,
|
||||
TagExclude: []string{"nothing*"},
|
||||
},
|
||||
}
|
||||
assert.NoError(t, conf.Filter.CompileFilter())
|
||||
|
||||
m := &mockOutput{}
|
||||
ro := NewRunningOutput("test", m, conf, 1000, 10000)
|
||||
|
||||
ro.AddMetric(first5[0])
|
||||
assert.Len(t, m.Metrics(), 0)
|
||||
|
||||
err := ro.Write()
|
||||
assert.NoError(t, err)
|
||||
assert.Len(t, m.Metrics(), 1)
|
||||
assert.Len(t, m.Metrics()[0].Tags(), 1)
|
||||
}
|
||||
|
||||
// Test that tags are properly included
|
||||
func TestRunningOutput_TagIncludeMatch(t *testing.T) {
|
||||
conf := &OutputConfig{
|
||||
Filter: Filter{
|
||||
IsActive: true,
|
||||
TagInclude: []string{"tag*"},
|
||||
},
|
||||
}
|
||||
assert.NoError(t, conf.Filter.CompileFilter())
|
||||
|
||||
m := &mockOutput{}
|
||||
ro := NewRunningOutput("test", m, conf, 1000, 10000)
|
||||
|
||||
ro.AddMetric(first5[0])
|
||||
assert.Len(t, m.Metrics(), 0)
|
||||
|
||||
err := ro.Write()
|
||||
assert.NoError(t, err)
|
||||
assert.Len(t, m.Metrics(), 1)
|
||||
assert.Len(t, m.Metrics()[0].Tags(), 1)
|
||||
}
|
||||
|
||||
// Test that we can write metrics with simple default setup.
|
||||
func TestRunningOutputDefault(t *testing.T) {
|
||||
conf := &OutputConfig{
|
||||
@@ -233,7 +38,7 @@ func TestRunningOutputDefault(t *testing.T) {
|
||||
}
|
||||
|
||||
m := &mockOutput{}
|
||||
ro := NewRunningOutput("test", m, conf, 1000, 10000)
|
||||
ro := NewRunningOutput("test", m, conf)
|
||||
|
||||
for _, metric := range first5 {
|
||||
ro.AddMetric(metric)
|
||||
@@ -248,6 +53,75 @@ func TestRunningOutputDefault(t *testing.T) {
|
||||
assert.Len(t, m.Metrics(), 10)
|
||||
}
|
||||
|
||||
// Test that the first metric gets overwritten if there is a buffer overflow.
|
||||
func TestRunningOutputOverwrite(t *testing.T) {
|
||||
conf := &OutputConfig{
|
||||
Filter: Filter{
|
||||
IsActive: false,
|
||||
},
|
||||
}
|
||||
|
||||
m := &mockOutput{}
|
||||
ro := NewRunningOutput("test", m, conf)
|
||||
ro.MetricBufferLimit = 4
|
||||
|
||||
for _, metric := range first5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
require.Len(t, m.Metrics(), 0)
|
||||
|
||||
err := ro.Write()
|
||||
require.NoError(t, err)
|
||||
require.Len(t, m.Metrics(), 4)
|
||||
|
||||
var expected, actual []string
|
||||
for i, exp := range first5[1:] {
|
||||
expected = append(expected, exp.String())
|
||||
actual = append(actual, m.Metrics()[i].String())
|
||||
}
|
||||
|
||||
sort.Strings(expected)
|
||||
sort.Strings(actual)
|
||||
|
||||
assert.Equal(t, expected, actual)
|
||||
}
|
||||
|
||||
// Test that multiple buffer overflows are handled properly.
|
||||
func TestRunningOutputMultiOverwrite(t *testing.T) {
|
||||
conf := &OutputConfig{
|
||||
Filter: Filter{
|
||||
IsActive: false,
|
||||
},
|
||||
}
|
||||
|
||||
m := &mockOutput{}
|
||||
ro := NewRunningOutput("test", m, conf)
|
||||
ro.MetricBufferLimit = 3
|
||||
|
||||
for _, metric := range first5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
for _, metric := range next5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
require.Len(t, m.Metrics(), 0)
|
||||
|
||||
err := ro.Write()
|
||||
require.NoError(t, err)
|
||||
require.Len(t, m.Metrics(), 3)
|
||||
|
||||
var expected, actual []string
|
||||
for i, exp := range next5[2:] {
|
||||
expected = append(expected, exp.String())
|
||||
actual = append(actual, m.Metrics()[i].String())
|
||||
}
|
||||
|
||||
sort.Strings(expected)
|
||||
sort.Strings(actual)
|
||||
|
||||
assert.Equal(t, expected, actual)
|
||||
}
|
||||
|
||||
// Test that running output doesn't flush until it's full when
|
||||
// FlushBufferWhenFull is set.
|
||||
func TestRunningOutputFlushWhenFull(t *testing.T) {
|
||||
@@ -258,9 +132,11 @@ func TestRunningOutputFlushWhenFull(t *testing.T) {
|
||||
}
|
||||
|
||||
m := &mockOutput{}
|
||||
ro := NewRunningOutput("test", m, conf, 6, 10)
|
||||
ro := NewRunningOutput("test", m, conf)
|
||||
ro.FlushBufferWhenFull = true
|
||||
ro.MetricBufferLimit = 5
|
||||
|
||||
// Fill buffer to 1 under limit
|
||||
// Fill buffer to limit
|
||||
for _, metric := range first5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
@@ -289,7 +165,9 @@ func TestRunningOutputMultiFlushWhenFull(t *testing.T) {
|
||||
}
|
||||
|
||||
m := &mockOutput{}
|
||||
ro := NewRunningOutput("test", m, conf, 4, 12)
|
||||
ro := NewRunningOutput("test", m, conf)
|
||||
ro.FlushBufferWhenFull = true
|
||||
ro.MetricBufferLimit = 4
|
||||
|
||||
// Fill buffer past limit twive
|
||||
for _, metric := range first5 {
|
||||
@@ -299,7 +177,7 @@ func TestRunningOutputMultiFlushWhenFull(t *testing.T) {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
// flushed twice
|
||||
assert.Len(t, m.Metrics(), 8)
|
||||
assert.Len(t, m.Metrics(), 10)
|
||||
}
|
||||
|
||||
func TestRunningOutputWriteFail(t *testing.T) {
|
||||
@@ -311,9 +189,11 @@ func TestRunningOutputWriteFail(t *testing.T) {
|
||||
|
||||
m := &mockOutput{}
|
||||
m.failWrite = true
|
||||
ro := NewRunningOutput("test", m, conf, 4, 12)
|
||||
ro := NewRunningOutput("test", m, conf)
|
||||
ro.FlushBufferWhenFull = true
|
||||
ro.MetricBufferLimit = 4
|
||||
|
||||
// Fill buffer to limit twice
|
||||
// Fill buffer past limit twice
|
||||
for _, metric := range first5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
@@ -336,161 +216,6 @@ func TestRunningOutputWriteFail(t *testing.T) {
|
||||
assert.Len(t, m.Metrics(), 10)
|
||||
}
|
||||
|
||||
// Verify that the order of points is preserved during a write failure.
|
||||
func TestRunningOutputWriteFailOrder(t *testing.T) {
|
||||
conf := &OutputConfig{
|
||||
Filter: Filter{
|
||||
IsActive: false,
|
||||
},
|
||||
}
|
||||
|
||||
m := &mockOutput{}
|
||||
m.failWrite = true
|
||||
ro := NewRunningOutput("test", m, conf, 100, 1000)
|
||||
|
||||
// add 5 metrics
|
||||
for _, metric := range first5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
// no successful flush yet
|
||||
assert.Len(t, m.Metrics(), 0)
|
||||
|
||||
// Write fails
|
||||
err := ro.Write()
|
||||
require.Error(t, err)
|
||||
// no successful flush yet
|
||||
assert.Len(t, m.Metrics(), 0)
|
||||
|
||||
m.failWrite = false
|
||||
// add 5 more metrics
|
||||
for _, metric := range next5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
err = ro.Write()
|
||||
require.NoError(t, err)
|
||||
|
||||
// Verify that 10 metrics were written
|
||||
assert.Len(t, m.Metrics(), 10)
|
||||
// Verify that they are in order
|
||||
expected := append(first5, next5...)
|
||||
assert.Equal(t, expected, m.Metrics())
|
||||
}
|
||||
|
||||
// Verify that the order of points is preserved during many write failures.
|
||||
func TestRunningOutputWriteFailOrder2(t *testing.T) {
|
||||
conf := &OutputConfig{
|
||||
Filter: Filter{
|
||||
IsActive: false,
|
||||
},
|
||||
}
|
||||
|
||||
m := &mockOutput{}
|
||||
m.failWrite = true
|
||||
ro := NewRunningOutput("test", m, conf, 5, 100)
|
||||
|
||||
// add 5 metrics
|
||||
for _, metric := range first5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
// Write fails
|
||||
err := ro.Write()
|
||||
require.Error(t, err)
|
||||
// no successful flush yet
|
||||
assert.Len(t, m.Metrics(), 0)
|
||||
|
||||
// add 5 metrics
|
||||
for _, metric := range next5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
// Write fails
|
||||
err = ro.Write()
|
||||
require.Error(t, err)
|
||||
// no successful flush yet
|
||||
assert.Len(t, m.Metrics(), 0)
|
||||
|
||||
// add 5 metrics
|
||||
for _, metric := range first5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
// Write fails
|
||||
err = ro.Write()
|
||||
require.Error(t, err)
|
||||
// no successful flush yet
|
||||
assert.Len(t, m.Metrics(), 0)
|
||||
|
||||
// add 5 metrics
|
||||
for _, metric := range next5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
// Write fails
|
||||
err = ro.Write()
|
||||
require.Error(t, err)
|
||||
// no successful flush yet
|
||||
assert.Len(t, m.Metrics(), 0)
|
||||
|
||||
m.failWrite = false
|
||||
err = ro.Write()
|
||||
require.NoError(t, err)
|
||||
|
||||
// Verify that 10 metrics were written
|
||||
assert.Len(t, m.Metrics(), 20)
|
||||
// Verify that they are in order
|
||||
expected := append(first5, next5...)
|
||||
expected = append(expected, first5...)
|
||||
expected = append(expected, next5...)
|
||||
assert.Equal(t, expected, m.Metrics())
|
||||
}
|
||||
|
||||
// Verify that the order of points is preserved when there is a remainder
|
||||
// of points for the batch.
|
||||
//
|
||||
// ie, with a batch size of 5:
|
||||
//
|
||||
// 1 2 3 4 5 6 <-- order, failed points
|
||||
// 6 1 2 3 4 5 <-- order, after 1st write failure (1 2 3 4 5 was batch)
|
||||
// 1 2 3 4 5 6 <-- order, after 2nd write failure, (6 was batch)
|
||||
//
|
||||
func TestRunningOutputWriteFailOrder3(t *testing.T) {
|
||||
conf := &OutputConfig{
|
||||
Filter: Filter{
|
||||
IsActive: false,
|
||||
},
|
||||
}
|
||||
|
||||
m := &mockOutput{}
|
||||
m.failWrite = true
|
||||
ro := NewRunningOutput("test", m, conf, 5, 1000)
|
||||
|
||||
// add 5 metrics
|
||||
for _, metric := range first5 {
|
||||
ro.AddMetric(metric)
|
||||
}
|
||||
// no successful flush yet
|
||||
assert.Len(t, m.Metrics(), 0)
|
||||
|
||||
// Write fails
|
||||
err := ro.Write()
|
||||
require.Error(t, err)
|
||||
// no successful flush yet
|
||||
assert.Len(t, m.Metrics(), 0)
|
||||
|
||||
// add and attempt to write a single metric:
|
||||
ro.AddMetric(next5[0])
|
||||
err = ro.Write()
|
||||
require.Error(t, err)
|
||||
|
||||
// unset fail and write metrics
|
||||
m.failWrite = false
|
||||
err = ro.Write()
|
||||
require.NoError(t, err)
|
||||
|
||||
// Verify that 6 metrics were written
|
||||
assert.Len(t, m.Metrics(), 6)
|
||||
// Verify that they are in order
|
||||
expected := append(first5, next5[0])
|
||||
assert.Equal(t, expected, m.Metrics())
|
||||
}
|
||||
|
||||
type mockOutput struct {
|
||||
sync.Mutex
|
||||
|
||||
@@ -538,31 +263,3 @@ func (m *mockOutput) Metrics() []telegraf.Metric {
|
||||
defer m.Unlock()
|
||||
return m.metrics
|
||||
}
|
||||
|
||||
type perfOutput struct {
|
||||
// if true, mock a write failure
|
||||
failWrite bool
|
||||
}
|
||||
|
||||
func (m *perfOutput) Connect() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *perfOutput) Close() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *perfOutput) Description() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (m *perfOutput) SampleConfig() string {
|
||||
return ""
|
||||
}
|
||||
|
||||
func (m *perfOutput) Write(metrics []telegraf.Metric) error {
|
||||
if m.failWrite {
|
||||
return fmt.Errorf("Failed Write!")
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -30,6 +30,8 @@ The example plugin gathers metrics about example things
|
||||
|
||||
### Example Output:
|
||||
|
||||
Give an example `-test` output here
|
||||
|
||||
```
|
||||
$ ./telegraf -config telegraf.conf -input-filter example -test
|
||||
measurement1,tag1=foo,tag2=bar field1=1i,field2=2.1 1453831884664956455
|
||||
|
||||
@@ -4,9 +4,6 @@ import (
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/aerospike"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/apache"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/bcache"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/cassandra"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/cloudwatch"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/couchbase"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/couchdb"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/disque"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/dns_query"
|
||||
@@ -14,13 +11,10 @@ import (
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/dovecot"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/elasticsearch"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/exec"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/filestat"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/github_webhooks"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/haproxy"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/http_response"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/httpjson"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/influxdb"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/ipmi_sensor"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/jolokia"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/kafka_consumer"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/leofs"
|
||||
@@ -35,12 +29,10 @@ import (
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/net_response"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/nginx"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/nsq"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/ntpq"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/passenger"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/phpfpm"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/ping"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/postgresql"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/postgresql_extensible"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/powerdns"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/procstat"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/prometheus"
|
||||
@@ -54,13 +46,9 @@ import (
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/snmp"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/sqlserver"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/statsd"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/sysstat"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/system"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/tail"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/tcp_listener"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/trig"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/twemproxy"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/udp_listener"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/win_perf_counters"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/zfs"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/zookeeper"
|
||||
|
||||
@@ -58,10 +58,7 @@ var tr = &http.Transport{
|
||||
ResponseHeaderTimeout: time.Duration(3 * time.Second),
|
||||
}
|
||||
|
||||
var client = &http.Client{
|
||||
Transport: tr,
|
||||
Timeout: time.Duration(4 * time.Second),
|
||||
}
|
||||
var client = &http.Client{Transport: tr}
|
||||
|
||||
func (n *Apache) gatherUrl(addr *url.URL, acc telegraf.Accumulator) error {
|
||||
resp, err := client.Get(addr.String())
|
||||
|
||||
@@ -1,125 +0,0 @@
|
||||
# Telegraf plugin: Cassandra
|
||||
|
||||
#### Plugin arguments:
|
||||
- **context** string: Context root used for jolokia url
|
||||
- **servers** []string: List of servers with the format "<user:passwd@><host>:port"
|
||||
- **metrics** []string: List of Jmx paths that identify mbeans attributes
|
||||
|
||||
#### Description
|
||||
|
||||
The Cassandra plugin collects Cassandra/JVM metrics exposed as MBean's attributes through jolokia REST endpoint. All metrics are collected for each server configured.
|
||||
|
||||
See: https://jolokia.org/ and [Cassandra Documentation](http://docs.datastax.com/en/cassandra/3.x/cassandra/operations/monitoringCassandraTOC.html)
|
||||
|
||||
# Measurements:
|
||||
Cassandra plugin produces one or more measurements for each metric configured, adding Server's name as `host` tag. More than one measurement is generated when querying table metrics with a wildcard for the keyspace or table name.
|
||||
|
||||
Given a configuration like:
|
||||
|
||||
```toml
|
||||
[[inputs.cassandra]]
|
||||
context = "/jolokia/read"
|
||||
servers = [":8778"]
|
||||
metrics = ["/java.lang:type=Memory/HeapMemoryUsage"]
|
||||
```
|
||||
|
||||
The collected metrics will be:
|
||||
|
||||
```
|
||||
javaMemory,host=myHost,mname=HeapMemoryUsage HeapMemoryUsage_committed=1040187392,HeapMemoryUsage_init=1050673152,HeapMemoryUsage_max=1040187392,HeapMemoryUsage_used=368155000 1459551767230567084
|
||||
```
|
||||
|
||||
# Useful Metrics:
|
||||
|
||||
Here is a list of metrics that might be useful to monitor your cassandra cluster. This was put together from multiple sources on the web.
|
||||
|
||||
- [How to monitor Cassandra performance metrics](https://www.datadoghq.com/blog/how-to-monitor-cassandra-performance-metrics)
|
||||
- [Cassandra Documentation](http://docs.datastax.com/en/cassandra/3.x/cassandra/operations/monitoringCassandraTOC.html)
|
||||
|
||||
####measurement = javaGarbageCollector
|
||||
|
||||
- /java.lang:type=GarbageCollector,name=ConcurrentMarkSweep/CollectionTime
|
||||
- /java.lang:type=GarbageCollector,name=ConcurrentMarkSweep/CollectionCount
|
||||
- /java.lang:type=GarbageCollector,name=ParNew/CollectionTime
|
||||
- /java.lang:type=GarbageCollector,name=ParNew/CollectionCount
|
||||
|
||||
####measurement = javaMemory
|
||||
|
||||
- /java.lang:type=Memory/HeapMemoryUsage
|
||||
- /java.lang:type=Memory/NonHeapMemoryUsage
|
||||
|
||||
####measurement = cassandraCache
|
||||
|
||||
- /org.apache.cassandra.metrics:type=Cache,scope=KeyCache,name=Hit
|
||||
- /org.apache.cassandra.metrics:type=Cache,scope=KeyCache,name=Requests
|
||||
- /org.apache.cassandra.metrics:type=Cache,scope=KeyCache,name=Entries
|
||||
- /org.apache.cassandra.metrics:type=Cache,scope=KeyCache,name=Size
|
||||
- /org.apache.cassandra.metrics:type=Cache,scope=KeyCache,name=Capacity
|
||||
- /org.apache.cassandra.metrics:type=Cache,scope=RowCache,name=Hit
|
||||
- /org.apache.cassandra.metrics:type=Cache,scope=RowCache,name=Requests
|
||||
- /org.apache.cassandra.metrics:type=Cache,scope=RowCache,name=Entries
|
||||
- /org.apache.cassandra.metrics:type=Cache,scope=RowCache,name=Size
|
||||
- /org.apache.cassandra.metrics:type=Cache,scope=RowCache,name=Capacity
|
||||
|
||||
####measurement = cassandraClient
|
||||
|
||||
- /org.apache.cassandra.metrics:type=Client,name=connectedNativeClients
|
||||
|
||||
####measurement = cassandraClientRequest
|
||||
|
||||
- /org.apache.cassandra.metrics:type=ClientRequest,scope=Read,name=TotalLatency
|
||||
- /org.apache.cassandra.metrics:type=ClientRequest,scope=Write,name=TotalLatency
|
||||
- /org.apache.cassandra.metrics:type=ClientRequest,scope=Read,name=Latency
|
||||
- /org.apache.cassandra.metrics:type=ClientRequest,scope=Write,name=Latency
|
||||
- /org.apache.cassandra.metrics:type=ClientRequest,scope=Read,name=Timeouts
|
||||
- /org.apache.cassandra.metrics:type=ClientRequest,scope=Write,name=Timeouts
|
||||
- /org.apache.cassandra.metrics:type=ClientRequest,scope=Read,name=Unavailables
|
||||
- /org.apache.cassandra.metrics:type=ClientRequest,scope=Write,name=Unavailables
|
||||
- /org.apache.cassandra.metrics:type=ClientRequest,scope=Read,name=Failures
|
||||
- /org.apache.cassandra.metrics:type=ClientRequest,scope=Write,name=Failures
|
||||
|
||||
####measurement = cassandraCommitLog
|
||||
|
||||
- /org.apache.cassandra.metrics:type=CommitLog,name=PendingTasks
|
||||
- /org.apache.cassandra.metrics:type=CommitLog,name=TotalCommitLogSize
|
||||
|
||||
####measurement = cassandraCompaction
|
||||
|
||||
- /org.apache.cassandra.metrics:type=Compaction,name=CompletedTask
|
||||
- /org.apache.cassandra.metrics:type=Compaction,name=PendingTasks
|
||||
- /org.apache.cassandra.metrics:type=Compaction,name=TotalCompactionsCompleted
|
||||
- /org.apache.cassandra.metrics:type=Compaction,name=BytesCompacted
|
||||
|
||||
####measurement = cassandraStorage
|
||||
|
||||
- /org.apache.cassandra.metrics:type=Storage,name=Load
|
||||
- /org.apache.cassandra.metrics:type=Storage,name=Exceptions
|
||||
|
||||
####measurement = cassandraTable
|
||||
Using wildcards for "keyspace" and "scope" can create a lot of series as metrics will be reported for every table and keyspace including internal system tables. Specify a keyspace name and/or a table name to limit them.
|
||||
|
||||
- /org.apache.cassandra.metrics:type=Table,keyspace=\*,scope=\*,name=LiveDiskSpaceUsed
|
||||
- /org.apache.cassandra.metrics:type=Table,keyspace=\*,scope=\*,name=TotalDiskSpaceUsed
|
||||
- /org.apache.cassandra.metrics:type=Table,keyspace=\*,scope=\*,name=ReadLatency
|
||||
- /org.apache.cassandra.metrics:type=Table,keyspace=\*,scope=\*,name=CoordinatorReadLatency
|
||||
- /org.apache.cassandra.metrics:type=Table,keyspace=\*,scope=\*,name=WriteLatency
|
||||
- /org.apache.cassandra.metrics:type=Table,keyspace=\*,scope=\*,name=ReadTotalLatency
|
||||
- /org.apache.cassandra.metrics:type=Table,keyspace=\*,scope=\*,name=WriteTotalLatency
|
||||
|
||||
|
||||
####measurement = cassandraThreadPools
|
||||
|
||||
- /org.apache.cassandra.metrics:type=ThreadPools,path=internal,scope=CompactionExecutor,name=ActiveTasks
|
||||
- /org.apache.cassandra.metrics:type=ThreadPools,path=internal,scope=AntiEntropyStage,name=ActiveTasks
|
||||
- /org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=CounterMutationStage,name=PendingTasks
|
||||
- /org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=CounterMutationStage,name=CurrentlyBlockedTasks
|
||||
- /org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=MutationStage,name=PendingTasks
|
||||
- /org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=MutationStage,name=CurrentlyBlockedTasks
|
||||
- /org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadRepairStage,name=PendingTasks
|
||||
- /org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadRepairStage,name=CurrentlyBlockedTasks
|
||||
- /org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=PendingTasks
|
||||
- /org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=ReadStage,name=CurrentlyBlockedTasks
|
||||
- /org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=RequestResponseStage,name=PendingTasks
|
||||
- /org.apache.cassandra.metrics:type=ThreadPools,path=request,scope=RequestResponseStage,name=CurrentlyBlockedTasks
|
||||
|
||||
|
||||
@@ -1,309 +0,0 @@
|
||||
package cassandra
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"io/ioutil"
|
||||
"log"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type JolokiaClient interface {
|
||||
MakeRequest(req *http.Request) (*http.Response, error)
|
||||
}
|
||||
|
||||
type JolokiaClientImpl struct {
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
func (c JolokiaClientImpl) MakeRequest(req *http.Request) (*http.Response, error) {
|
||||
return c.client.Do(req)
|
||||
}
|
||||
|
||||
type Cassandra struct {
|
||||
jClient JolokiaClient
|
||||
Context string
|
||||
Servers []string
|
||||
Metrics []string
|
||||
}
|
||||
|
||||
type javaMetric struct {
|
||||
host string
|
||||
metric string
|
||||
acc telegraf.Accumulator
|
||||
}
|
||||
|
||||
type cassandraMetric struct {
|
||||
host string
|
||||
metric string
|
||||
acc telegraf.Accumulator
|
||||
}
|
||||
|
||||
type jmxMetric interface {
|
||||
addTagsFields(out map[string]interface{})
|
||||
}
|
||||
|
||||
func newJavaMetric(host string, metric string,
|
||||
acc telegraf.Accumulator) *javaMetric {
|
||||
return &javaMetric{host: host, metric: metric, acc: acc}
|
||||
}
|
||||
|
||||
func newCassandraMetric(host string, metric string,
|
||||
acc telegraf.Accumulator) *cassandraMetric {
|
||||
return &cassandraMetric{host: host, metric: metric, acc: acc}
|
||||
}
|
||||
|
||||
func addValuesAsFields(values map[string]interface{}, fields map[string]interface{},
|
||||
mname string) {
|
||||
for k, v := range values {
|
||||
if v != nil {
|
||||
fields[mname+"_"+k] = v
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func parseJmxMetricRequest(mbean string) map[string]string {
|
||||
tokens := make(map[string]string)
|
||||
classAndPairs := strings.Split(mbean, ":")
|
||||
if classAndPairs[0] == "org.apache.cassandra.metrics" {
|
||||
tokens["class"] = "cassandra"
|
||||
} else if classAndPairs[0] == "java.lang" {
|
||||
tokens["class"] = "java"
|
||||
} else {
|
||||
return tokens
|
||||
}
|
||||
pairs := strings.Split(classAndPairs[1], ",")
|
||||
for _, pair := range pairs {
|
||||
p := strings.Split(pair, "=")
|
||||
tokens[p[0]] = p[1]
|
||||
}
|
||||
return tokens
|
||||
}
|
||||
|
||||
func addTokensToTags(tokens map[string]string, tags map[string]string) {
|
||||
for k, v := range tokens {
|
||||
if k == "name" {
|
||||
tags["mname"] = v // name seems to a reserved word in influxdb
|
||||
} else if k == "class" || k == "type" {
|
||||
continue // class and type are used in the metric name
|
||||
} else {
|
||||
tags[k] = v
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (j javaMetric) addTagsFields(out map[string]interface{}) {
|
||||
tags := make(map[string]string)
|
||||
fields := make(map[string]interface{})
|
||||
|
||||
a := out["request"].(map[string]interface{})
|
||||
attribute := a["attribute"].(string)
|
||||
mbean := a["mbean"].(string)
|
||||
|
||||
tokens := parseJmxMetricRequest(mbean)
|
||||
addTokensToTags(tokens, tags)
|
||||
tags["cassandra_host"] = j.host
|
||||
|
||||
if _, ok := tags["mname"]; !ok {
|
||||
//Queries for a single value will not return a "name" tag in the response.
|
||||
tags["mname"] = attribute
|
||||
}
|
||||
|
||||
if values, ok := out["value"]; ok {
|
||||
switch t := values.(type) {
|
||||
case map[string]interface{}:
|
||||
addValuesAsFields(values.(map[string]interface{}), fields, attribute)
|
||||
case interface{}:
|
||||
fields[attribute] = t
|
||||
}
|
||||
j.acc.AddFields(tokens["class"]+tokens["type"], fields, tags)
|
||||
} else {
|
||||
fmt.Printf("Missing key 'value' in '%s' output response\n%v\n",
|
||||
j.metric, out)
|
||||
}
|
||||
}
|
||||
|
||||
func addCassandraMetric(mbean string, c cassandraMetric,
|
||||
values map[string]interface{}) {
|
||||
|
||||
tags := make(map[string]string)
|
||||
fields := make(map[string]interface{})
|
||||
tokens := parseJmxMetricRequest(mbean)
|
||||
addTokensToTags(tokens, tags)
|
||||
tags["cassandra_host"] = c.host
|
||||
addValuesAsFields(values, fields, tags["mname"])
|
||||
c.acc.AddFields(tokens["class"]+tokens["type"], fields, tags)
|
||||
|
||||
}
|
||||
|
||||
func (c cassandraMetric) addTagsFields(out map[string]interface{}) {
|
||||
|
||||
r := out["request"]
|
||||
|
||||
tokens := parseJmxMetricRequest(r.(map[string]interface{})["mbean"].(string))
|
||||
// Requests with wildcards for keyspace or table names will return nested
|
||||
// maps in the json response
|
||||
if tokens["type"] == "Table" && (tokens["keyspace"] == "*" ||
|
||||
tokens["scope"] == "*") {
|
||||
if valuesMap, ok := out["value"]; ok {
|
||||
for k, v := range valuesMap.(map[string]interface{}) {
|
||||
addCassandraMetric(k, c, v.(map[string]interface{}))
|
||||
}
|
||||
} else {
|
||||
fmt.Printf("Missing key 'value' in '%s' output response\n%v\n",
|
||||
c.metric, out)
|
||||
return
|
||||
}
|
||||
} else {
|
||||
if values, ok := out["value"]; ok {
|
||||
addCassandraMetric(r.(map[string]interface{})["mbean"].(string),
|
||||
c, values.(map[string]interface{}))
|
||||
} else {
|
||||
fmt.Printf("Missing key 'value' in '%s' output response\n%v\n",
|
||||
c.metric, out)
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func (j *Cassandra) SampleConfig() string {
|
||||
return `
|
||||
# This is the context root used to compose the jolokia url
|
||||
context = "/jolokia/read"
|
||||
## List of cassandra servers exposing jolokia read service
|
||||
servers = ["myuser:mypassword@10.10.10.1:8778","10.10.10.2:8778",":8778"]
|
||||
## List of metrics collected on above servers
|
||||
## Each metric consists of a jmx path.
|
||||
## This will collect all heap memory usage metrics from the jvm and
|
||||
## ReadLatency metrics for all keyspaces and tables.
|
||||
## "type=Table" in the query works with Cassandra3.0. Older versions might
|
||||
## need to use "type=ColumnFamily"
|
||||
metrics = [
|
||||
"/java.lang:type=Memory/HeapMemoryUsage",
|
||||
"/org.apache.cassandra.metrics:type=Table,keyspace=*,scope=*,name=ReadLatency"
|
||||
]
|
||||
`
|
||||
}
|
||||
|
||||
func (j *Cassandra) Description() string {
|
||||
return "Read Cassandra metrics through Jolokia"
|
||||
}
|
||||
|
||||
func (j *Cassandra) getAttr(requestUrl *url.URL) (map[string]interface{}, error) {
|
||||
// Create + send request
|
||||
req, err := http.NewRequest("GET", requestUrl.String(), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp, err := j.jClient.MakeRequest(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
// Process response
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
err = fmt.Errorf("Response from url \"%s\" has status code %d (%s), expected %d (%s)",
|
||||
requestUrl,
|
||||
resp.StatusCode,
|
||||
http.StatusText(resp.StatusCode),
|
||||
http.StatusOK,
|
||||
http.StatusText(http.StatusOK))
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// read body
|
||||
body, err := ioutil.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
// Unmarshal json
|
||||
var jsonOut map[string]interface{}
|
||||
if err = json.Unmarshal([]byte(body), &jsonOut); err != nil {
|
||||
return nil, errors.New("Error decoding JSON response")
|
||||
}
|
||||
|
||||
return jsonOut, nil
|
||||
}
|
||||
|
||||
func parseServerTokens(server string) map[string]string {
|
||||
serverTokens := make(map[string]string)
|
||||
|
||||
hostAndUser := strings.Split(server, "@")
|
||||
hostPort := ""
|
||||
userPasswd := ""
|
||||
if len(hostAndUser) == 2 {
|
||||
hostPort = hostAndUser[1]
|
||||
userPasswd = hostAndUser[0]
|
||||
} else {
|
||||
hostPort = hostAndUser[0]
|
||||
}
|
||||
hostTokens := strings.Split(hostPort, ":")
|
||||
serverTokens["host"] = hostTokens[0]
|
||||
serverTokens["port"] = hostTokens[1]
|
||||
|
||||
if userPasswd != "" {
|
||||
userTokens := strings.Split(userPasswd, ":")
|
||||
serverTokens["user"] = userTokens[0]
|
||||
serverTokens["passwd"] = userTokens[1]
|
||||
}
|
||||
return serverTokens
|
||||
}
|
||||
|
||||
func (c *Cassandra) Gather(acc telegraf.Accumulator) error {
|
||||
context := c.Context
|
||||
servers := c.Servers
|
||||
metrics := c.Metrics
|
||||
|
||||
for _, server := range servers {
|
||||
for _, metric := range metrics {
|
||||
serverTokens := parseServerTokens(server)
|
||||
|
||||
var m jmxMetric
|
||||
if strings.HasPrefix(metric, "/java.lang:") {
|
||||
m = newJavaMetric(serverTokens["host"], metric, acc)
|
||||
} else if strings.HasPrefix(metric,
|
||||
"/org.apache.cassandra.metrics:") {
|
||||
m = newCassandraMetric(serverTokens["host"], metric, acc)
|
||||
} else {
|
||||
// unsupported metric type
|
||||
log.Printf("Unsupported Cassandra metric [%s], skipping",
|
||||
metric)
|
||||
continue
|
||||
}
|
||||
|
||||
// Prepare URL
|
||||
requestUrl, err := url.Parse("http://" + serverTokens["host"] + ":" +
|
||||
serverTokens["port"] + context + metric)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if serverTokens["user"] != "" && serverTokens["passwd"] != "" {
|
||||
requestUrl.User = url.UserPassword(serverTokens["user"],
|
||||
serverTokens["passwd"])
|
||||
}
|
||||
fmt.Printf("host %s url %s\n", serverTokens["host"], requestUrl)
|
||||
|
||||
out, err := c.getAttr(requestUrl)
|
||||
if out["status"] != 200.0 {
|
||||
fmt.Printf("URL returned with status %v\n", out["status"])
|
||||
continue
|
||||
}
|
||||
m.addTagsFields(out)
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("cassandra", func() telegraf.Input {
|
||||
return &Cassandra{jClient: &JolokiaClientImpl{client: &http.Client{}}}
|
||||
})
|
||||
}
|
||||
@@ -1,286 +0,0 @@
|
||||
package cassandra
|
||||
|
||||
import (
|
||||
_ "fmt"
|
||||
"io/ioutil"
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
"github.com/stretchr/testify/assert"
|
||||
_ "github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
const validJavaMultiValueJSON = `
|
||||
{
|
||||
"request":{
|
||||
"mbean":"java.lang:type=Memory",
|
||||
"attribute":"HeapMemoryUsage",
|
||||
"type":"read"
|
||||
},
|
||||
"value":{
|
||||
"init":67108864,
|
||||
"committed":456130560,
|
||||
"max":477626368,
|
||||
"used":203288528
|
||||
},
|
||||
"timestamp":1446129191,
|
||||
"status":200
|
||||
}`
|
||||
|
||||
const validCassandraMultiValueJSON = `
|
||||
{
|
||||
"request": {
|
||||
"mbean": "org.apache.cassandra.metrics:keyspace=test_keyspace1,name=ReadLatency,scope=test_table,type=Table",
|
||||
"type": "read"},
|
||||
"status": 200,
|
||||
"timestamp": 1458089229,
|
||||
"value": {
|
||||
"999thPercentile": 20.0,
|
||||
"99thPercentile": 10.0,
|
||||
"Count": 400,
|
||||
"DurationUnit": "microseconds",
|
||||
"Max": 30.0,
|
||||
"Mean": null,
|
||||
"MeanRate": 3.0,
|
||||
"Min": 1.0,
|
||||
"RateUnit": "events/second",
|
||||
"StdDev": null
|
||||
}
|
||||
}`
|
||||
|
||||
const validCassandraNestedMultiValueJSON = `
|
||||
{
|
||||
"request": {
|
||||
"mbean": "org.apache.cassandra.metrics:keyspace=test_keyspace1,name=ReadLatency,scope=*,type=Table",
|
||||
"type": "read"},
|
||||
"status": 200,
|
||||
"timestamp": 1458089184,
|
||||
"value": {
|
||||
"org.apache.cassandra.metrics:keyspace=test_keyspace1,name=ReadLatency,scope=test_table1,type=Table":
|
||||
{ "999thPercentile": 1.0,
|
||||
"Count": 100,
|
||||
"DurationUnit": "microseconds",
|
||||
"OneMinuteRate": 1.0,
|
||||
"RateUnit": "events/second",
|
||||
"StdDev": null
|
||||
},
|
||||
"org.apache.cassandra.metrics:keyspace=test_keyspace2,name=ReadLatency,scope=test_table2,type=Table":
|
||||
{ "999thPercentile": 2.0,
|
||||
"Count": 200,
|
||||
"DurationUnit": "microseconds",
|
||||
"OneMinuteRate": 2.0,
|
||||
"RateUnit": "events/second",
|
||||
"StdDev": null
|
||||
}
|
||||
}
|
||||
}`
|
||||
|
||||
const validSingleValueJSON = `
|
||||
{
|
||||
"request":{
|
||||
"path":"used",
|
||||
"mbean":"java.lang:type=Memory",
|
||||
"attribute":"HeapMemoryUsage",
|
||||
"type":"read"
|
||||
},
|
||||
"value":209274376,
|
||||
"timestamp":1446129256,
|
||||
"status":200
|
||||
}`
|
||||
|
||||
const validJavaMultiTypeJSON = `
|
||||
{
|
||||
"request":{
|
||||
"mbean":"java.lang:name=ConcurrentMarkSweep,type=GarbageCollector",
|
||||
"attribute":"CollectionCount",
|
||||
"type":"read"
|
||||
},
|
||||
"value":1,
|
||||
"timestamp":1459316570,
|
||||
"status":200
|
||||
}`
|
||||
|
||||
const invalidJSON = "I don't think this is JSON"
|
||||
|
||||
const empty = ""
|
||||
|
||||
var Servers = []string{"10.10.10.10:8778"}
|
||||
var AuthServers = []string{"user:passwd@10.10.10.10:8778"}
|
||||
var MultipleServers = []string{"10.10.10.10:8778", "10.10.10.11:8778"}
|
||||
var HeapMetric = "/java.lang:type=Memory/HeapMemoryUsage"
|
||||
var ReadLatencyMetric = "/org.apache.cassandra.metrics:type=Table,keyspace=test_keyspace1,scope=test_table,name=ReadLatency"
|
||||
var NestedReadLatencyMetric = "/org.apache.cassandra.metrics:type=Table,keyspace=test_keyspace1,scope=*,name=ReadLatency"
|
||||
var GarbageCollectorMetric1 = "/java.lang:type=GarbageCollector,name=ConcurrentMarkSweep/CollectionCount"
|
||||
var GarbageCollectorMetric2 = "/java.lang:type=GarbageCollector,name=ConcurrentMarkSweep/CollectionTime"
|
||||
var Context = "/jolokia/read"
|
||||
|
||||
type jolokiaClientStub struct {
|
||||
responseBody string
|
||||
statusCode int
|
||||
}
|
||||
|
||||
func (c jolokiaClientStub) MakeRequest(req *http.Request) (*http.Response, error) {
|
||||
resp := http.Response{}
|
||||
resp.StatusCode = c.statusCode
|
||||
resp.Body = ioutil.NopCloser(strings.NewReader(c.responseBody))
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
// Generates a pointer to an HttpJson object that uses a mock HTTP client.
|
||||
// Parameters:
|
||||
// response : Body of the response that the mock HTTP client should return
|
||||
// statusCode: HTTP status code the mock HTTP client should return
|
||||
//
|
||||
// Returns:
|
||||
// *HttpJson: Pointer to an HttpJson object that uses the generated mock HTTP client
|
||||
func genJolokiaClientStub(response string, statusCode int, servers []string, metrics []string) *Cassandra {
|
||||
return &Cassandra{
|
||||
jClient: jolokiaClientStub{responseBody: response, statusCode: statusCode},
|
||||
Context: Context,
|
||||
Servers: servers,
|
||||
Metrics: metrics,
|
||||
}
|
||||
}
|
||||
|
||||
// Test that the proper values are ignored or collected for class=Java
|
||||
func TestHttpJsonJavaMultiValue(t *testing.T) {
|
||||
cassandra := genJolokiaClientStub(validJavaMultiValueJSON, 200,
|
||||
MultipleServers, []string{HeapMetric})
|
||||
|
||||
var acc testutil.Accumulator
|
||||
acc.SetDebug(true)
|
||||
err := cassandra.Gather(&acc)
|
||||
|
||||
assert.Nil(t, err)
|
||||
assert.Equal(t, 2, len(acc.Metrics))
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"HeapMemoryUsage_init": 67108864.0,
|
||||
"HeapMemoryUsage_committed": 456130560.0,
|
||||
"HeapMemoryUsage_max": 477626368.0,
|
||||
"HeapMemoryUsage_used": 203288528.0,
|
||||
}
|
||||
tags1 := map[string]string{
|
||||
"cassandra_host": "10.10.10.10",
|
||||
"mname": "HeapMemoryUsage",
|
||||
}
|
||||
|
||||
tags2 := map[string]string{
|
||||
"cassandra_host": "10.10.10.11",
|
||||
"mname": "HeapMemoryUsage",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "javaMemory", fields, tags1)
|
||||
acc.AssertContainsTaggedFields(t, "javaMemory", fields, tags2)
|
||||
}
|
||||
|
||||
func TestHttpJsonJavaMultiType(t *testing.T) {
|
||||
cassandra := genJolokiaClientStub(validJavaMultiTypeJSON, 200, AuthServers, []string{GarbageCollectorMetric1, GarbageCollectorMetric2})
|
||||
|
||||
var acc testutil.Accumulator
|
||||
acc.SetDebug(true)
|
||||
err := cassandra.Gather(&acc)
|
||||
|
||||
assert.Nil(t, err)
|
||||
assert.Equal(t, 2, len(acc.Metrics))
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"CollectionCount": 1.0,
|
||||
}
|
||||
|
||||
tags := map[string]string{
|
||||
"cassandra_host": "10.10.10.10",
|
||||
"mname": "ConcurrentMarkSweep",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "javaGarbageCollector", fields, tags)
|
||||
}
|
||||
|
||||
// Test that the proper values are ignored or collected
|
||||
func TestHttpJsonOn404(t *testing.T) {
|
||||
|
||||
jolokia := genJolokiaClientStub(validJavaMultiValueJSON, 404, Servers,
|
||||
[]string{HeapMetric})
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := jolokia.Gather(&acc)
|
||||
|
||||
assert.Nil(t, err)
|
||||
assert.Equal(t, 0, len(acc.Metrics))
|
||||
}
|
||||
|
||||
// Test that the proper values are ignored or collected for class=Cassandra
|
||||
func TestHttpJsonCassandraMultiValue(t *testing.T) {
|
||||
cassandra := genJolokiaClientStub(validCassandraMultiValueJSON, 200, Servers, []string{ReadLatencyMetric})
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := cassandra.Gather(&acc)
|
||||
|
||||
assert.Nil(t, err)
|
||||
assert.Equal(t, 1, len(acc.Metrics))
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"ReadLatency_999thPercentile": 20.0,
|
||||
"ReadLatency_99thPercentile": 10.0,
|
||||
"ReadLatency_Count": 400.0,
|
||||
"ReadLatency_DurationUnit": "microseconds",
|
||||
"ReadLatency_Max": 30.0,
|
||||
"ReadLatency_MeanRate": 3.0,
|
||||
"ReadLatency_Min": 1.0,
|
||||
"ReadLatency_RateUnit": "events/second",
|
||||
}
|
||||
|
||||
tags := map[string]string{
|
||||
"cassandra_host": "10.10.10.10",
|
||||
"mname": "ReadLatency",
|
||||
"keyspace": "test_keyspace1",
|
||||
"scope": "test_table",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "cassandraTable", fields, tags)
|
||||
}
|
||||
|
||||
// Test that the proper values are ignored or collected for class=Cassandra with
|
||||
// nested values
|
||||
func TestHttpJsonCassandraNestedMultiValue(t *testing.T) {
|
||||
cassandra := genJolokiaClientStub(validCassandraNestedMultiValueJSON, 200, Servers, []string{NestedReadLatencyMetric})
|
||||
|
||||
var acc testutil.Accumulator
|
||||
acc.SetDebug(true)
|
||||
err := cassandra.Gather(&acc)
|
||||
|
||||
assert.Nil(t, err)
|
||||
assert.Equal(t, 2, len(acc.Metrics))
|
||||
|
||||
fields1 := map[string]interface{}{
|
||||
"ReadLatency_999thPercentile": 1.0,
|
||||
"ReadLatency_Count": 100.0,
|
||||
"ReadLatency_DurationUnit": "microseconds",
|
||||
"ReadLatency_OneMinuteRate": 1.0,
|
||||
"ReadLatency_RateUnit": "events/second",
|
||||
}
|
||||
|
||||
fields2 := map[string]interface{}{
|
||||
"ReadLatency_999thPercentile": 2.0,
|
||||
"ReadLatency_Count": 200.0,
|
||||
"ReadLatency_DurationUnit": "microseconds",
|
||||
"ReadLatency_OneMinuteRate": 2.0,
|
||||
"ReadLatency_RateUnit": "events/second",
|
||||
}
|
||||
|
||||
tags1 := map[string]string{
|
||||
"cassandra_host": "10.10.10.10",
|
||||
"mname": "ReadLatency",
|
||||
"keyspace": "test_keyspace1",
|
||||
"scope": "test_table1",
|
||||
}
|
||||
|
||||
tags2 := map[string]string{
|
||||
"cassandra_host": "10.10.10.10",
|
||||
"mname": "ReadLatency",
|
||||
"keyspace": "test_keyspace2",
|
||||
"scope": "test_table2",
|
||||
}
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "cassandraTable", fields1, tags1)
|
||||
acc.AssertContainsTaggedFields(t, "cassandraTable", fields2, tags2)
|
||||
}
|
||||
@@ -1,86 +0,0 @@
|
||||
# Amazon CloudWatch Statistics Input
|
||||
|
||||
This plugin will pull Metric Statistics from Amazon CloudWatch.
|
||||
|
||||
### Amazon Authentication
|
||||
|
||||
This plugin uses a credential chain for Authentication with the CloudWatch
|
||||
API endpoint. In the following order the plugin will attempt to authenticate.
|
||||
1. [IAMS Role](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html)
|
||||
2. [Environment Variables](https://github.com/aws/aws-sdk-go/wiki/configuring-sdk#environment-variables)
|
||||
3. [Shared Credentials](https://github.com/aws/aws-sdk-go/wiki/configuring-sdk#shared-credentials-file)
|
||||
|
||||
### Configuration:
|
||||
|
||||
```toml
|
||||
[[inputs.cloudwatch]]
|
||||
## Amazon Region (required)
|
||||
region = 'us-east-1'
|
||||
|
||||
## Requested CloudWatch aggregation Period (required - must be a multiple of 60s)
|
||||
period = '1m'
|
||||
|
||||
## Collection Delay (required - must account for metrics availability via CloudWatch API)
|
||||
delay = '1m'
|
||||
|
||||
## Override global run interval (optional - defaults to global interval)
|
||||
## Recomended: use metric 'interval' that is a multiple of 'period' to avoid
|
||||
## gaps or overlap in pulled data
|
||||
interval = '1m'
|
||||
|
||||
## Metric Statistic Namespace (required)
|
||||
namespace = 'AWS/ELB'
|
||||
|
||||
## Metrics to Pull (optional)
|
||||
## Defaults to all Metrics in Namespace if nothing is provided
|
||||
## Refreshes Namespace available metrics every 1h
|
||||
[[inputs.cloudwatch.metrics]]
|
||||
names = ['Latency', 'RequestCount']
|
||||
|
||||
## Dimension filters for Metric (optional)
|
||||
[[inputs.cloudwatch.metrics.dimensions]]
|
||||
name = 'LoadBalancerName'
|
||||
value = 'p-example'
|
||||
```
|
||||
#### Requirements and Terminology
|
||||
|
||||
Plugin Configuration utilizes [CloudWatch concepts](http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/cloudwatch_concepts.html) and access pattern to allow monitoring of any CloudWatch Metric.
|
||||
|
||||
- `region` must be a valid AWS [Region](http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/cloudwatch_concepts.html#CloudWatchRegions) value
|
||||
- `period` must be a valid CloudWatch [Period](http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/cloudwatch_concepts.html#CloudWatchPeriods) value
|
||||
- `namespace` must be a valid CloudWatch [Namespace](http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/cloudwatch_concepts.html#Namespace) value
|
||||
- `names` must be valid CloudWatch [Metric](http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/cloudwatch_concepts.html#Metric) names
|
||||
- `dimensions` must be valid CloudWatch [Dimension](http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/cloudwatch_concepts.html#Dimension) name/value pairs
|
||||
|
||||
#### Restrictions and Limitations
|
||||
- CloudWatch metrics are not available instantly via the CloudWatch API. You should adjust your collection `delay` to account for this lag in metrics availability based on your [monitoring subscription level](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-new.html)
|
||||
- CloudWatch API usage incurs cost - see [GetMetricStatistics Pricing](https://aws.amazon.com/cloudwatch/pricing/)
|
||||
|
||||
### Measurements & Fields:
|
||||
|
||||
Each CloudWatch Namespace monitored records a measurement with fields for each available Metric Statistic
|
||||
Namespace and Metrics are represented in [snake case](https://en.wikipedia.org/wiki/Snake_case)
|
||||
|
||||
- cloudwatch_{namespace}
|
||||
- {metric}_sum (metric Sum value)
|
||||
- {metric}_average (metric Average value)
|
||||
- {metric}_minimum (metric Minimum value)
|
||||
- {metric}_maximum (metric Maximum value)
|
||||
- {metric}_sample_count (metric SampleCount value)
|
||||
|
||||
|
||||
### Tags:
|
||||
Each measurement is tagged with the following identifiers to uniquely identify the associated metric
|
||||
Tag Dimension names are represented in [snake case](https://en.wikipedia.org/wiki/Snake_case)
|
||||
|
||||
- All measurements have the following tags:
|
||||
- region (CloudWatch Region)
|
||||
- unit (CloudWatch Metric Unit)
|
||||
- {dimension-name} (Cloudwatch Dimension value - one for each metric dimension)
|
||||
|
||||
### Example Output:
|
||||
|
||||
```
|
||||
$ ./telegraf -config telegraf.conf -input-filter cloudwatch -test
|
||||
> cloudwatch_aws_elb,load_balancer_name=p-example,region=us-east-1,unit=seconds latency_average=0.004810798017284538,latency_maximum=0.1100282669067383,latency_minimum=0.0006084442138671875,latency_sample_count=4029,latency_sum=19.382705211639404 1459542420000000000
|
||||
```
|
||||
@@ -1,311 +0,0 @@
|
||||
package cloudwatch
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/aws/aws-sdk-go/aws"
|
||||
"github.com/aws/aws-sdk-go/aws/credentials"
|
||||
"github.com/aws/aws-sdk-go/aws/session"
|
||||
|
||||
"github.com/aws/aws-sdk-go/service/cloudwatch"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
type (
|
||||
CloudWatch struct {
|
||||
Region string `toml:"region"`
|
||||
AccessKey string `toml:"access_key"`
|
||||
SecretKey string `toml:"secret_key"`
|
||||
Period internal.Duration `toml:"period"`
|
||||
Delay internal.Duration `toml:"delay"`
|
||||
Namespace string `toml:"namespace"`
|
||||
Metrics []*Metric `toml:"metrics"`
|
||||
client cloudwatchClient
|
||||
metricCache *MetricCache
|
||||
}
|
||||
|
||||
Metric struct {
|
||||
MetricNames []string `toml:"names"`
|
||||
Dimensions []*Dimension `toml:"dimensions"`
|
||||
}
|
||||
|
||||
Dimension struct {
|
||||
Name string `toml:"name"`
|
||||
Value string `toml:"value"`
|
||||
}
|
||||
|
||||
MetricCache struct {
|
||||
TTL time.Duration
|
||||
Fetched time.Time
|
||||
Metrics []*cloudwatch.Metric
|
||||
}
|
||||
|
||||
cloudwatchClient interface {
|
||||
ListMetrics(*cloudwatch.ListMetricsInput) (*cloudwatch.ListMetricsOutput, error)
|
||||
GetMetricStatistics(*cloudwatch.GetMetricStatisticsInput) (*cloudwatch.GetMetricStatisticsOutput, error)
|
||||
}
|
||||
)
|
||||
|
||||
func (c *CloudWatch) SampleConfig() string {
|
||||
return `
|
||||
## Amazon Region
|
||||
region = 'us-east-1'
|
||||
|
||||
## Amazon Credentials
|
||||
## Credentials are loaded in the following order
|
||||
## 1) explicit credentials from 'access_key' and 'secret_key'
|
||||
## 2) environment variables
|
||||
## 3) shared credentials file
|
||||
## 4) EC2 Instance Profile
|
||||
#access_key = ""
|
||||
#secret_key = ""
|
||||
|
||||
## Requested CloudWatch aggregation Period (required - must be a multiple of 60s)
|
||||
period = '1m'
|
||||
|
||||
## Collection Delay (required - must account for metrics availability via CloudWatch API)
|
||||
delay = '1m'
|
||||
|
||||
## Recomended: use metric 'interval' that is a multiple of 'period' to avoid
|
||||
## gaps or overlap in pulled data
|
||||
interval = '1m'
|
||||
|
||||
## Metric Statistic Namespace (required)
|
||||
namespace = 'AWS/ELB'
|
||||
|
||||
## Metrics to Pull (optional)
|
||||
## Defaults to all Metrics in Namespace if nothing is provided
|
||||
## Refreshes Namespace available metrics every 1h
|
||||
#[[inputs.cloudwatch.metrics]]
|
||||
# names = ['Latency', 'RequestCount']
|
||||
#
|
||||
# ## Dimension filters for Metric (optional)
|
||||
# [[inputs.cloudwatch.metrics.dimensions]]
|
||||
# name = 'LoadBalancerName'
|
||||
# value = 'p-example'
|
||||
`
|
||||
}
|
||||
|
||||
func (c *CloudWatch) Description() string {
|
||||
return "Pull Metric Statistics from Amazon CloudWatch"
|
||||
}
|
||||
|
||||
func (c *CloudWatch) Gather(acc telegraf.Accumulator) error {
|
||||
if c.client == nil {
|
||||
c.initializeCloudWatch()
|
||||
}
|
||||
|
||||
var metrics []*cloudwatch.Metric
|
||||
|
||||
// check for provided metric filter
|
||||
if c.Metrics != nil {
|
||||
metrics = []*cloudwatch.Metric{}
|
||||
for _, m := range c.Metrics {
|
||||
dimensions := make([]*cloudwatch.Dimension, len(m.Dimensions))
|
||||
for k, d := range m.Dimensions {
|
||||
dimensions[k] = &cloudwatch.Dimension{
|
||||
Name: aws.String(d.Name),
|
||||
Value: aws.String(d.Value),
|
||||
}
|
||||
}
|
||||
for _, name := range m.MetricNames {
|
||||
metrics = append(metrics, &cloudwatch.Metric{
|
||||
Namespace: aws.String(c.Namespace),
|
||||
MetricName: aws.String(name),
|
||||
Dimensions: dimensions,
|
||||
})
|
||||
}
|
||||
}
|
||||
} else {
|
||||
var err error
|
||||
metrics, err = c.fetchNamespaceMetrics()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
metricCount := len(metrics)
|
||||
var errChan = make(chan error, metricCount)
|
||||
|
||||
now := time.Now()
|
||||
|
||||
// limit concurrency or we can easily exhaust user connection limit
|
||||
semaphore := make(chan byte, 64)
|
||||
|
||||
for _, m := range metrics {
|
||||
semaphore <- 0x1
|
||||
go c.gatherMetric(acc, m, now, semaphore, errChan)
|
||||
}
|
||||
|
||||
for i := 1; i <= metricCount; i++ {
|
||||
err := <-errChan
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("cloudwatch", func() telegraf.Input {
|
||||
return &CloudWatch{}
|
||||
})
|
||||
}
|
||||
|
||||
/*
|
||||
* Initialize CloudWatch client
|
||||
*/
|
||||
func (c *CloudWatch) initializeCloudWatch() error {
|
||||
config := &aws.Config{
|
||||
Region: aws.String(c.Region),
|
||||
}
|
||||
if c.AccessKey != "" || c.SecretKey != "" {
|
||||
config.Credentials = credentials.NewStaticCredentials(c.AccessKey, c.SecretKey, "")
|
||||
}
|
||||
|
||||
c.client = cloudwatch.New(session.New(config))
|
||||
return nil
|
||||
}
|
||||
|
||||
/*
|
||||
* Fetch available metrics for given CloudWatch Namespace
|
||||
*/
|
||||
func (c *CloudWatch) fetchNamespaceMetrics() (metrics []*cloudwatch.Metric, err error) {
|
||||
if c.metricCache != nil && c.metricCache.IsValid() {
|
||||
metrics = c.metricCache.Metrics
|
||||
return
|
||||
}
|
||||
|
||||
metrics = []*cloudwatch.Metric{}
|
||||
|
||||
var token *string
|
||||
for more := true; more; {
|
||||
params := &cloudwatch.ListMetricsInput{
|
||||
Namespace: aws.String(c.Namespace),
|
||||
Dimensions: []*cloudwatch.DimensionFilter{},
|
||||
NextToken: token,
|
||||
MetricName: nil,
|
||||
}
|
||||
|
||||
resp, err := c.client.ListMetrics(params)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
metrics = append(metrics, resp.Metrics...)
|
||||
|
||||
token = resp.NextToken
|
||||
more = token != nil
|
||||
}
|
||||
|
||||
cacheTTL, _ := time.ParseDuration("1hr")
|
||||
c.metricCache = &MetricCache{
|
||||
Metrics: metrics,
|
||||
Fetched: time.Now(),
|
||||
TTL: cacheTTL,
|
||||
}
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
/*
|
||||
* Gather given Metric and emit any error
|
||||
*/
|
||||
func (c *CloudWatch) gatherMetric(acc telegraf.Accumulator, metric *cloudwatch.Metric, now time.Time, semaphore chan byte, errChan chan error) {
|
||||
params := c.getStatisticsInput(metric, now)
|
||||
resp, err := c.client.GetMetricStatistics(params)
|
||||
if err != nil {
|
||||
errChan <- err
|
||||
<-semaphore
|
||||
return
|
||||
}
|
||||
|
||||
for _, point := range resp.Datapoints {
|
||||
tags := map[string]string{
|
||||
"region": c.Region,
|
||||
"unit": snakeCase(*point.Unit),
|
||||
}
|
||||
|
||||
for _, d := range metric.Dimensions {
|
||||
tags[snakeCase(*d.Name)] = *d.Value
|
||||
}
|
||||
|
||||
// record field for each statistic
|
||||
fields := map[string]interface{}{}
|
||||
|
||||
if point.Average != nil {
|
||||
fields[formatField(*metric.MetricName, cloudwatch.StatisticAverage)] = *point.Average
|
||||
}
|
||||
if point.Maximum != nil {
|
||||
fields[formatField(*metric.MetricName, cloudwatch.StatisticMaximum)] = *point.Maximum
|
||||
}
|
||||
if point.Minimum != nil {
|
||||
fields[formatField(*metric.MetricName, cloudwatch.StatisticMinimum)] = *point.Minimum
|
||||
}
|
||||
if point.SampleCount != nil {
|
||||
fields[formatField(*metric.MetricName, cloudwatch.StatisticSampleCount)] = *point.SampleCount
|
||||
}
|
||||
if point.Sum != nil {
|
||||
fields[formatField(*metric.MetricName, cloudwatch.StatisticSum)] = *point.Sum
|
||||
}
|
||||
|
||||
acc.AddFields(formatMeasurement(c.Namespace), fields, tags, *point.Timestamp)
|
||||
}
|
||||
|
||||
errChan <- nil
|
||||
<-semaphore
|
||||
}
|
||||
|
||||
/*
|
||||
* Formatting helpers
|
||||
*/
|
||||
func formatField(metricName string, statistic string) string {
|
||||
return fmt.Sprintf("%s_%s", snakeCase(metricName), snakeCase(statistic))
|
||||
}
|
||||
|
||||
func formatMeasurement(namespace string) string {
|
||||
namespace = strings.Replace(namespace, "/", "_", -1)
|
||||
namespace = snakeCase(namespace)
|
||||
return fmt.Sprintf("cloudwatch_%s", namespace)
|
||||
}
|
||||
|
||||
func snakeCase(s string) string {
|
||||
s = internal.SnakeCase(s)
|
||||
s = strings.Replace(s, "__", "_", -1)
|
||||
return s
|
||||
}
|
||||
|
||||
/*
|
||||
* Map Metric to *cloudwatch.GetMetricStatisticsInput for given timeframe
|
||||
*/
|
||||
func (c *CloudWatch) getStatisticsInput(metric *cloudwatch.Metric, now time.Time) *cloudwatch.GetMetricStatisticsInput {
|
||||
end := now.Add(-c.Delay.Duration)
|
||||
|
||||
input := &cloudwatch.GetMetricStatisticsInput{
|
||||
StartTime: aws.Time(end.Add(-c.Period.Duration)),
|
||||
EndTime: aws.Time(end),
|
||||
MetricName: metric.MetricName,
|
||||
Namespace: metric.Namespace,
|
||||
Period: aws.Int64(int64(c.Period.Duration.Seconds())),
|
||||
Dimensions: metric.Dimensions,
|
||||
Statistics: []*string{
|
||||
aws.String(cloudwatch.StatisticAverage),
|
||||
aws.String(cloudwatch.StatisticMaximum),
|
||||
aws.String(cloudwatch.StatisticMinimum),
|
||||
aws.String(cloudwatch.StatisticSum),
|
||||
aws.String(cloudwatch.StatisticSampleCount)},
|
||||
}
|
||||
return input
|
||||
}
|
||||
|
||||
/*
|
||||
* Check Metric Cache validity
|
||||
*/
|
||||
func (c *MetricCache) IsValid() bool {
|
||||
return c.Metrics != nil && time.Since(c.Fetched) < c.TTL
|
||||
}
|
||||
@@ -1,131 +0,0 @@
|
||||
package cloudwatch
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/aws/aws-sdk-go/aws"
|
||||
"github.com/aws/aws-sdk-go/service/cloudwatch"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
type mockCloudWatchClient struct{}
|
||||
|
||||
func (m *mockCloudWatchClient) ListMetrics(params *cloudwatch.ListMetricsInput) (*cloudwatch.ListMetricsOutput, error) {
|
||||
metric := &cloudwatch.Metric{
|
||||
Namespace: params.Namespace,
|
||||
MetricName: aws.String("Latency"),
|
||||
Dimensions: []*cloudwatch.Dimension{
|
||||
&cloudwatch.Dimension{
|
||||
Name: aws.String("LoadBalancerName"),
|
||||
Value: aws.String("p-example"),
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
result := &cloudwatch.ListMetricsOutput{
|
||||
Metrics: []*cloudwatch.Metric{metric},
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func (m *mockCloudWatchClient) GetMetricStatistics(params *cloudwatch.GetMetricStatisticsInput) (*cloudwatch.GetMetricStatisticsOutput, error) {
|
||||
dataPoint := &cloudwatch.Datapoint{
|
||||
Timestamp: params.EndTime,
|
||||
Minimum: aws.Float64(0.1),
|
||||
Maximum: aws.Float64(0.3),
|
||||
Average: aws.Float64(0.2),
|
||||
Sum: aws.Float64(123),
|
||||
SampleCount: aws.Float64(100),
|
||||
Unit: aws.String("Seconds"),
|
||||
}
|
||||
result := &cloudwatch.GetMetricStatisticsOutput{
|
||||
Label: aws.String("Latency"),
|
||||
Datapoints: []*cloudwatch.Datapoint{dataPoint},
|
||||
}
|
||||
return result, nil
|
||||
}
|
||||
|
||||
func TestGather(t *testing.T) {
|
||||
duration, _ := time.ParseDuration("1m")
|
||||
internalDuration := internal.Duration{
|
||||
Duration: duration,
|
||||
}
|
||||
c := &CloudWatch{
|
||||
Region: "us-east-1",
|
||||
Namespace: "AWS/ELB",
|
||||
Delay: internalDuration,
|
||||
Period: internalDuration,
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
c.client = &mockCloudWatchClient{}
|
||||
|
||||
c.Gather(&acc)
|
||||
|
||||
fields := map[string]interface{}{}
|
||||
fields["latency_minimum"] = 0.1
|
||||
fields["latency_maximum"] = 0.3
|
||||
fields["latency_average"] = 0.2
|
||||
fields["latency_sum"] = 123.0
|
||||
fields["latency_sample_count"] = 100.0
|
||||
|
||||
tags := map[string]string{}
|
||||
tags["unit"] = "seconds"
|
||||
tags["region"] = "us-east-1"
|
||||
tags["load_balancer_name"] = "p-example"
|
||||
|
||||
assert.True(t, acc.HasMeasurement("cloudwatch_aws_elb"))
|
||||
acc.AssertContainsTaggedFields(t, "cloudwatch_aws_elb", fields, tags)
|
||||
|
||||
}
|
||||
|
||||
func TestGenerateStatisticsInputParams(t *testing.T) {
|
||||
d := &cloudwatch.Dimension{
|
||||
Name: aws.String("LoadBalancerName"),
|
||||
Value: aws.String("p-example"),
|
||||
}
|
||||
|
||||
m := &cloudwatch.Metric{
|
||||
MetricName: aws.String("Latency"),
|
||||
Dimensions: []*cloudwatch.Dimension{d},
|
||||
}
|
||||
|
||||
duration, _ := time.ParseDuration("1m")
|
||||
internalDuration := internal.Duration{
|
||||
Duration: duration,
|
||||
}
|
||||
|
||||
c := &CloudWatch{
|
||||
Namespace: "AWS/ELB",
|
||||
Delay: internalDuration,
|
||||
Period: internalDuration,
|
||||
}
|
||||
|
||||
c.initializeCloudWatch()
|
||||
|
||||
now := time.Now()
|
||||
|
||||
params := c.getStatisticsInput(m, now)
|
||||
|
||||
assert.EqualValues(t, *params.EndTime, now.Add(-c.Delay.Duration))
|
||||
assert.EqualValues(t, *params.StartTime, now.Add(-c.Period.Duration).Add(-c.Delay.Duration))
|
||||
assert.Len(t, params.Dimensions, 1)
|
||||
assert.Len(t, params.Statistics, 5)
|
||||
assert.EqualValues(t, *params.Period, 60)
|
||||
}
|
||||
|
||||
func TestMetricsCacheTimeout(t *testing.T) {
|
||||
ttl, _ := time.ParseDuration("5ms")
|
||||
cache := &MetricCache{
|
||||
Metrics: []*cloudwatch.Metric{},
|
||||
Fetched: time.Now(),
|
||||
TTL: ttl,
|
||||
}
|
||||
|
||||
assert.True(t, cache.IsValid())
|
||||
time.Sleep(ttl)
|
||||
assert.False(t, cache.IsValid())
|
||||
}
|
||||
@@ -1,63 +0,0 @@
|
||||
# Telegraf Plugin: Couchbase
|
||||
|
||||
## Configuration:
|
||||
|
||||
```
|
||||
# Read per-node and per-bucket metrics from Couchbase
|
||||
[[inputs.couchbase]]
|
||||
## specify servers via a url matching:
|
||||
## [protocol://][:password]@address[:port]
|
||||
## e.g.
|
||||
## http://couchbase-0.example.com/
|
||||
## http://admin:secret@couchbase-0.example.com:8091/
|
||||
##
|
||||
## If no servers are specified, then localhost is used as the host.
|
||||
## If no protocol is specifed, HTTP is used.
|
||||
## If no port is specified, 8091 is used.
|
||||
servers = ["http://localhost:8091"]
|
||||
```
|
||||
|
||||
## Measurements:
|
||||
|
||||
### couchbase_node
|
||||
|
||||
Tags:
|
||||
- cluster: whatever you called it in `servers` in the configuration, e.g.: `http://couchbase-0.example.com/`
|
||||
- hostname: Couchbase's name for the node and port, e.g., `172.16.10.187:8091`
|
||||
|
||||
Fields:
|
||||
- memory_free (unit: bytes, example: 23181365248.0)
|
||||
- memory_total (unit: bytes, example: 64424656896.0)
|
||||
|
||||
### couchbase_bucket
|
||||
|
||||
Tags:
|
||||
- cluster: whatever you called it in `servers` in the configuration, e.g.: `http://couchbase-0.example.com/`)
|
||||
- bucket: the name of the couchbase bucket, e.g., `blastro-df`
|
||||
|
||||
Fields:
|
||||
- quota_percent_used (unit: percent, example: 68.85424936294555)
|
||||
- ops_per_sec (unit: count, example: 5686.789686789687)
|
||||
- disk_fetches (unit: count, example: 0.0)
|
||||
- item_count (unit: count, example: 943239752.0)
|
||||
- disk_used (unit: bytes, example: 409178772321.0)
|
||||
- data_used (unit: bytes, example: 212179309111.0)
|
||||
- mem_used (unit: bytes, example: 202156957464.0)
|
||||
|
||||
|
||||
## Example output
|
||||
|
||||
```
|
||||
$ telegraf -config telegraf.conf -input-filter couchbase -test
|
||||
* Plugin: couchbase, Collection 1
|
||||
> couchbase_node,cluster=https://couchbase-0.example.com/,hostname=172.16.10.187:8091 memory_free=22927384576,memory_total=64424656896 1458381183695864929
|
||||
> couchbase_node,cluster=https://couchbase-0.example.com/,hostname=172.16.10.65:8091 memory_free=23520161792,memory_total=64424656896 1458381183695972112
|
||||
> couchbase_node,cluster=https://couchbase-0.example.com/,hostname=172.16.13.105:8091 memory_free=23531704320,memory_total=64424656896 1458381183695995259
|
||||
> couchbase_node,cluster=https://couchbase-0.example.com/,hostname=172.16.13.173:8091 memory_free=23628767232,memory_total=64424656896 1458381183696010870
|
||||
> couchbase_node,cluster=https://couchbase-0.example.com/,hostname=172.16.15.120:8091 memory_free=23616692224,memory_total=64424656896 1458381183696027406
|
||||
> couchbase_node,cluster=https://couchbase-0.example.com/,hostname=172.16.8.127:8091 memory_free=23431770112,memory_total=64424656896 1458381183696041040
|
||||
> couchbase_node,cluster=https://couchbase-0.example.com/,hostname=172.16.8.148:8091 memory_free=23811371008,memory_total=64424656896 1458381183696059060
|
||||
> couchbase_bucket,bucket=default,cluster=https://couchbase-0.example.com/ data_used=25743360,disk_fetches=0,disk_used=31744886,item_count=0,mem_used=77729224,ops_per_sec=0,quota_percent_used=10.58976636614118 1458381183696210074
|
||||
> couchbase_bucket,bucket=demoncat,cluster=https://couchbase-0.example.com/ data_used=38157584951,disk_fetches=0,disk_used=62730302441,item_count=14662532,mem_used=24015304256,ops_per_sec=1207.753207753208,quota_percent_used=79.87855353525707 1458381183696242695
|
||||
> couchbase_bucket,bucket=blastro-df,cluster=https://couchbase-0.example.com/ data_used=212552491622,disk_fetches=0,disk_used=413323157621,item_count=944655680,mem_used=202421103760,ops_per_sec=1692.176692176692,quota_percent_used=68.9442170551845 1458381183696272206
|
||||
```
|
||||
@@ -1,104 +0,0 @@
|
||||
package couchbase
|
||||
|
||||
import (
|
||||
couchbase "github.com/couchbase/go-couchbase"
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"sync"
|
||||
)
|
||||
|
||||
type Couchbase struct {
|
||||
Servers []string
|
||||
}
|
||||
|
||||
var sampleConfig = `
|
||||
## specify servers via a url matching:
|
||||
## [protocol://][:password]@address[:port]
|
||||
## e.g.
|
||||
## http://couchbase-0.example.com/
|
||||
## http://admin:secret@couchbase-0.example.com:8091/
|
||||
##
|
||||
## If no servers are specified, then localhost is used as the host.
|
||||
## If no protocol is specifed, HTTP is used.
|
||||
## If no port is specified, 8091 is used.
|
||||
servers = ["http://localhost:8091"]
|
||||
`
|
||||
|
||||
func (r *Couchbase) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
|
||||
func (r *Couchbase) Description() string {
|
||||
return "Read metrics from one or many couchbase clusters"
|
||||
}
|
||||
|
||||
// Reads stats from all configured clusters. Accumulates stats.
|
||||
// Returns one of the errors encountered while gathering stats (if any).
|
||||
func (r *Couchbase) Gather(acc telegraf.Accumulator) error {
|
||||
if len(r.Servers) == 0 {
|
||||
r.gatherServer("http://localhost:8091/", acc, nil)
|
||||
return nil
|
||||
}
|
||||
|
||||
var wg sync.WaitGroup
|
||||
|
||||
var outerr error
|
||||
|
||||
for _, serv := range r.Servers {
|
||||
wg.Add(1)
|
||||
go func(serv string) {
|
||||
defer wg.Done()
|
||||
outerr = r.gatherServer(serv, acc, nil)
|
||||
}(serv)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
|
||||
return outerr
|
||||
}
|
||||
|
||||
func (r *Couchbase) gatherServer(addr string, acc telegraf.Accumulator, pool *couchbase.Pool) error {
|
||||
if pool == nil {
|
||||
client, err := couchbase.Connect(addr)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// `default` is the only possible pool name. It's a
|
||||
// placeholder for a possible future Couchbase feature. See
|
||||
// http://stackoverflow.com/a/16990911/17498.
|
||||
p, err := client.GetPool("default")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
pool = &p
|
||||
}
|
||||
for i := 0; i < len(pool.Nodes); i++ {
|
||||
node := pool.Nodes[i]
|
||||
tags := map[string]string{"cluster": addr, "hostname": node.Hostname}
|
||||
fields := make(map[string]interface{})
|
||||
fields["memory_free"] = node.MemoryFree
|
||||
fields["memory_total"] = node.MemoryTotal
|
||||
acc.AddFields("couchbase_node", fields, tags)
|
||||
}
|
||||
for bucketName, _ := range pool.BucketMap {
|
||||
tags := map[string]string{"cluster": addr, "bucket": bucketName}
|
||||
bs := pool.BucketMap[bucketName].BasicStats
|
||||
fields := make(map[string]interface{})
|
||||
fields["quota_percent_used"] = bs["quotaPercentUsed"]
|
||||
fields["ops_per_sec"] = bs["opsPerSec"]
|
||||
fields["disk_fetches"] = bs["diskFetches"]
|
||||
fields["item_count"] = bs["itemCount"]
|
||||
fields["disk_used"] = bs["diskUsed"]
|
||||
fields["data_used"] = bs["dataUsed"]
|
||||
fields["mem_used"] = bs["memUsed"]
|
||||
acc.AddFields("couchbase_bucket", fields, tags)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("couchbase", func() telegraf.Input {
|
||||
return &Couchbase{}
|
||||
})
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -10,7 +10,6 @@ import (
|
||||
"reflect"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// Schema:
|
||||
@@ -113,18 +112,9 @@ func (c *CouchDB) Gather(accumulator telegraf.Accumulator) error {
|
||||
|
||||
}
|
||||
|
||||
var tr = &http.Transport{
|
||||
ResponseHeaderTimeout: time.Duration(3 * time.Second),
|
||||
}
|
||||
|
||||
var client = &http.Client{
|
||||
Transport: tr,
|
||||
Timeout: time.Duration(4 * time.Second),
|
||||
}
|
||||
|
||||
func (c *CouchDB) fetchAndInsertData(accumulator telegraf.Accumulator, host string) error {
|
||||
|
||||
response, error := client.Get(host)
|
||||
response, error := http.Get(host)
|
||||
if error != nil {
|
||||
return error
|
||||
}
|
||||
|
||||
@@ -9,7 +9,6 @@ import (
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
@@ -24,14 +23,13 @@ type Disque struct {
|
||||
|
||||
var sampleConfig = `
|
||||
## An array of URI to gather stats about. Specify an ip or hostname
|
||||
## with optional port and password.
|
||||
## ie disque://localhost, disque://10.10.3.33:18832, 10.0.0.1:10000, etc.
|
||||
## with optional port and password. ie disque://localhost, disque://10.10.3.33:18832,
|
||||
## 10.0.0.1:10000, etc.
|
||||
|
||||
## If no servers are specified, then localhost is used as the host.
|
||||
servers = ["localhost"]
|
||||
`
|
||||
|
||||
var defaultTimeout = 5 * time.Second
|
||||
|
||||
func (r *Disque) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
@@ -109,7 +107,7 @@ func (g *Disque) gatherServer(addr *url.URL, acc telegraf.Accumulator) error {
|
||||
addr.Host = addr.Host + ":" + defaultPort
|
||||
}
|
||||
|
||||
c, err := net.DialTimeout("tcp", addr.Host, defaultTimeout)
|
||||
c, err := net.Dial("tcp", addr.Host)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Unable to connect to disque server '%s': %s", addr.Host, err)
|
||||
}
|
||||
@@ -134,9 +132,6 @@ func (g *Disque) gatherServer(addr *url.URL, acc telegraf.Accumulator) error {
|
||||
g.c = c
|
||||
}
|
||||
|
||||
// Extend connection
|
||||
g.c.SetDeadline(time.Now().Add(defaultTimeout))
|
||||
|
||||
g.c.Write([]byte("info\r\n"))
|
||||
|
||||
r := bufio.NewReader(g.c)
|
||||
@@ -162,7 +157,7 @@ func (g *Disque) gatherServer(addr *url.URL, acc telegraf.Accumulator) error {
|
||||
var read int
|
||||
|
||||
fields := make(map[string]interface{})
|
||||
tags := map[string]string{"disque_host": addr.String()}
|
||||
tags := map[string]string{"host": addr.String()}
|
||||
for read < sz {
|
||||
line, err := r.ReadString('\n')
|
||||
if err != nil {
|
||||
|
||||
@@ -35,8 +35,7 @@ var sampleConfig = `
|
||||
## Domains or subdomains to query. "."(root) is default
|
||||
domains = ["."] # optional
|
||||
|
||||
## Query record type. Default is "A"
|
||||
## Posible values: A, AAAA, CNAME, MX, NS, PTR, TXT, SOA, SPF, SRV.
|
||||
## Query record type. Posible values: A, AAAA, CNAME, MX, NS, PTR, TXT, SOA, SPF, SRV. Default is "NS"
|
||||
record_type = "A" # optional
|
||||
|
||||
## Dns server port. 53 is default
|
||||
|
||||
@@ -1,42 +1,31 @@
|
||||
package dns_query
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
|
||||
"github.com/miekg/dns"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
var servers = []string{"8.8.8.8"}
|
||||
var domains = []string{"google.com"}
|
||||
var domains = []string{"mjasion.pl"}
|
||||
|
||||
func TestGathering(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("Skipping network-dependent test in short mode.")
|
||||
}
|
||||
var dnsConfig = DnsQuery{
|
||||
Servers: servers,
|
||||
Domains: domains,
|
||||
}
|
||||
var acc testutil.Accumulator
|
||||
|
||||
err := dnsConfig.Gather(&acc)
|
||||
assert.NoError(t, err)
|
||||
metric, ok := acc.Get("dns_query")
|
||||
require.True(t, ok)
|
||||
dnsConfig.Gather(&acc)
|
||||
metric, _ := acc.Get("dns_query")
|
||||
queryTime, _ := metric.Fields["query_time_ms"].(float64)
|
||||
|
||||
assert.NotEqual(t, 0, queryTime)
|
||||
}
|
||||
|
||||
func TestGatheringMxRecord(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("Skipping network-dependent test in short mode.")
|
||||
}
|
||||
var dnsConfig = DnsQuery{
|
||||
Servers: servers,
|
||||
Domains: domains,
|
||||
@@ -44,19 +33,14 @@ func TestGatheringMxRecord(t *testing.T) {
|
||||
var acc testutil.Accumulator
|
||||
dnsConfig.RecordType = "MX"
|
||||
|
||||
err := dnsConfig.Gather(&acc)
|
||||
assert.NoError(t, err)
|
||||
metric, ok := acc.Get("dns_query")
|
||||
require.True(t, ok)
|
||||
dnsConfig.Gather(&acc)
|
||||
metric, _ := acc.Get("dns_query")
|
||||
queryTime, _ := metric.Fields["query_time_ms"].(float64)
|
||||
|
||||
assert.NotEqual(t, 0, queryTime)
|
||||
}
|
||||
|
||||
func TestGatheringRootDomain(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("Skipping network-dependent test in short mode.")
|
||||
}
|
||||
var dnsConfig = DnsQuery{
|
||||
Servers: servers,
|
||||
Domains: []string{"."},
|
||||
@@ -70,10 +54,8 @@ func TestGatheringRootDomain(t *testing.T) {
|
||||
}
|
||||
fields := map[string]interface{}{}
|
||||
|
||||
err := dnsConfig.Gather(&acc)
|
||||
assert.NoError(t, err)
|
||||
metric, ok := acc.Get("dns_query")
|
||||
require.True(t, ok)
|
||||
dnsConfig.Gather(&acc)
|
||||
metric, _ := acc.Get("dns_query")
|
||||
queryTime, _ := metric.Fields["query_time_ms"].(float64)
|
||||
|
||||
fields["query_time_ms"] = queryTime
|
||||
@@ -81,9 +63,6 @@ func TestGatheringRootDomain(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestMetricContainsServerAndDomainAndRecordTypeTags(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("Skipping network-dependent test in short mode.")
|
||||
}
|
||||
var dnsConfig = DnsQuery{
|
||||
Servers: servers,
|
||||
Domains: domains,
|
||||
@@ -91,15 +70,13 @@ func TestMetricContainsServerAndDomainAndRecordTypeTags(t *testing.T) {
|
||||
var acc testutil.Accumulator
|
||||
tags := map[string]string{
|
||||
"server": "8.8.8.8",
|
||||
"domain": "google.com",
|
||||
"domain": "mjasion.pl",
|
||||
"record_type": "NS",
|
||||
}
|
||||
fields := map[string]interface{}{}
|
||||
|
||||
err := dnsConfig.Gather(&acc)
|
||||
assert.NoError(t, err)
|
||||
metric, ok := acc.Get("dns_query")
|
||||
require.True(t, ok)
|
||||
dnsConfig.Gather(&acc)
|
||||
metric, _ := acc.Get("dns_query")
|
||||
queryTime, _ := metric.Fields["query_time_ms"].(float64)
|
||||
|
||||
fields["query_time_ms"] = queryTime
|
||||
@@ -107,9 +84,6 @@ func TestMetricContainsServerAndDomainAndRecordTypeTags(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestGatheringTimeout(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("Skipping network-dependent test in short mode.")
|
||||
}
|
||||
var dnsConfig = DnsQuery{
|
||||
Servers: servers,
|
||||
Domains: domains,
|
||||
|
||||
@@ -5,11 +5,11 @@ docker containers. You can read Docker's documentation for their remote API
|
||||
[here](https://docs.docker.com/engine/reference/api/docker_remote_api_v1.20/#get-container-stats-based-on-resource-usage)
|
||||
|
||||
The docker plugin uses the excellent
|
||||
[docker engine-api](https://github.com/docker/engine-api) library to
|
||||
[fsouza go-dockerclient](https://github.com/fsouza/go-dockerclient) library to
|
||||
gather stats. Documentation for the library can be found
|
||||
[here](https://godoc.org/github.com/docker/engine-api) and documentation
|
||||
[here](https://godoc.org/github.com/fsouza/go-dockerclient) and documentation
|
||||
for the stat structure can be found
|
||||
[here](https://godoc.org/github.com/docker/engine-api/types#Stats)
|
||||
[here](https://godoc.org/github.com/fsouza/go-dockerclient#Stats)
|
||||
|
||||
### Configuration:
|
||||
|
||||
@@ -29,10 +29,10 @@ for the stat structure can be found
|
||||
Every effort was made to preserve the names based on the JSON response from the
|
||||
docker API.
|
||||
|
||||
Note that the docker_container_cpu metric may appear multiple times per collection,
|
||||
based on the availability of per-cpu stats on your system.
|
||||
Note that the docker_cpu metric may appear multiple times per collection, based
|
||||
on the availability of per-cpu stats on your system.
|
||||
|
||||
- docker_container_mem
|
||||
- docker_mem
|
||||
- total_pgmafault
|
||||
- cache
|
||||
- mapped_file
|
||||
@@ -66,8 +66,7 @@ based on the availability of per-cpu stats on your system.
|
||||
- usage
|
||||
- failcnt
|
||||
- limit
|
||||
- container_id
|
||||
- docker_container_cpu
|
||||
- docker_cpu
|
||||
- throttling_periods
|
||||
- throttling_throttled_periods
|
||||
- throttling_throttled_time
|
||||
@@ -75,9 +74,7 @@ based on the availability of per-cpu stats on your system.
|
||||
- usage_in_usermode
|
||||
- usage_system
|
||||
- usage_total
|
||||
- usage_percent
|
||||
- container_id
|
||||
- docker_container_net
|
||||
- docker_net
|
||||
- rx_dropped
|
||||
- rx_bytes
|
||||
- rx_errors
|
||||
@@ -86,8 +83,7 @@ based on the availability of per-cpu stats on your system.
|
||||
- rx_packets
|
||||
- tx_errors
|
||||
- tx_bytes
|
||||
- container_id
|
||||
- docker_container_blkio
|
||||
- docker_blkio
|
||||
- io_service_bytes_recursive_async
|
||||
- io_service_bytes_recursive_read
|
||||
- io_service_bytes_recursive_sync
|
||||
@@ -98,51 +94,18 @@ based on the availability of per-cpu stats on your system.
|
||||
- io_serviced_recursive_sync
|
||||
- io_serviced_recursive_total
|
||||
- io_serviced_recursive_write
|
||||
- container_id
|
||||
- docker_
|
||||
- n_used_file_descriptors
|
||||
- n_cpus
|
||||
- n_containers
|
||||
- n_images
|
||||
- n_goroutines
|
||||
- n_listener_events
|
||||
- memory_total
|
||||
- pool_blocksize
|
||||
- docker_data
|
||||
- available
|
||||
- total
|
||||
- used
|
||||
- docker_metadata
|
||||
- available
|
||||
- total
|
||||
- used
|
||||
|
||||
|
||||
### Tags:
|
||||
|
||||
- docker (memory_total)
|
||||
- unit=bytes
|
||||
- docker (pool_blocksize)
|
||||
- unit=bytes
|
||||
- docker_data
|
||||
- unit=bytes
|
||||
- docker_metadata
|
||||
- unit=bytes
|
||||
|
||||
- docker_container_mem specific:
|
||||
- container_image
|
||||
- container_name
|
||||
- docker_container_cpu specific:
|
||||
- container_image
|
||||
- container_name
|
||||
- All stats have the following tags:
|
||||
- cont_id (container ID)
|
||||
- cont_image (container image)
|
||||
- cont_name (container name)
|
||||
- docker_cpu specific:
|
||||
- cpu
|
||||
- docker_container_net specific:
|
||||
- container_image
|
||||
- container_name
|
||||
- docker_net specific:
|
||||
- network
|
||||
- docker_container_blkio specific:
|
||||
- container_image
|
||||
- container_name
|
||||
- docker_blkio specific:
|
||||
- device
|
||||
|
||||
### Example Output:
|
||||
@@ -150,18 +113,8 @@ based on the availability of per-cpu stats on your system.
|
||||
```
|
||||
% ./telegraf -config ~/ws/telegraf.conf -input-filter docker -test
|
||||
* Plugin: docker, Collection 1
|
||||
> docker n_cpus=8i 1456926671065383978
|
||||
> docker n_used_file_descriptors=15i 1456926671065383978
|
||||
> docker n_containers=7i 1456926671065383978
|
||||
> docker n_images=152i 1456926671065383978
|
||||
> docker n_goroutines=36i 1456926671065383978
|
||||
> docker n_listener_events=0i 1456926671065383978
|
||||
> docker,unit=bytes memory_total=18935443456i 1456926671065383978
|
||||
> docker,unit=bytes pool_blocksize=65540i 1456926671065383978
|
||||
> docker_data,unit=bytes available=24340000000i,total=107400000000i,used=14820000000i 1456926671065383978
|
||||
> docker_metadata,unit=bytes available=2126999999i,total=2146999999i,used=20420000i 145692667106538
|
||||
> docker_container_mem,
|
||||
container_image=spotify/kafka,container_name=kafka \
|
||||
> docker_mem,cont_id=5705ba8ed8fb47527410653d60a8bb2f3af5e62372297c419022a3cc6d45d848,\
|
||||
cont_image=spotify/kafka,cont_name=kafka \
|
||||
active_anon=52568064i,active_file=6926336i,cache=12038144i,fail_count=0i,\
|
||||
hierarchical_memory_limit=9223372036854771712i,inactive_anon=52707328i,\
|
||||
inactive_file=5111808i,limit=1044578304i,mapped_file=10301440i,\
|
||||
@@ -172,21 +125,21 @@ total_inactive_file=5111808i,total_mapped_file=10301440i,total_pgfault=63762i,\
|
||||
total_pgmafault=0i,total_pgpgin=73355i,total_pgpgout=45736i,\
|
||||
total_rss=105275392i,total_rss_huge=4194304i,total_unevictable=0i,\
|
||||
total_writeback=0i,unevictable=0i,usage=117440512i,writeback=0i 1453409536840126713
|
||||
> docker_container_cpu,
|
||||
container_image=spotify/kafka,container_name=kafka,cpu=cpu-total \
|
||||
> docker_cpu,cont_id=5705ba8ed8fb47527410653d60a8bb2f3af5e62372297c419022a3cc6d45d848,\
|
||||
cont_image=spotify/kafka,cont_name=kafka,cpu=cpu-total \
|
||||
throttling_periods=0i,throttling_throttled_periods=0i,\
|
||||
throttling_throttled_time=0i,usage_in_kernelmode=440000000i,\
|
||||
usage_in_usermode=2290000000i,usage_system=84795360000000i,\
|
||||
usage_total=6628208865i 1453409536840126713
|
||||
> docker_container_cpu,
|
||||
container_image=spotify/kafka,container_name=kafka,cpu=cpu0 \
|
||||
> docker_cpu,cont_id=5705ba8ed8fb47527410653d60a8bb2f3af5e62372297c419022a3cc6d45d848,\
|
||||
cont_image=spotify/kafka,cont_name=kafka,cpu=cpu0 \
|
||||
usage_total=6628208865i 1453409536840126713
|
||||
> docker_container_net,\
|
||||
container_image=spotify/kafka,container_name=kafka,network=eth0 \
|
||||
> docker_net,cont_id=5705ba8ed8fb47527410653d60a8bb2f3af5e62372297c419022a3cc6d45d848,\
|
||||
cont_image=spotify/kafka,cont_name=kafka,network=eth0 \
|
||||
rx_bytes=7468i,rx_dropped=0i,rx_errors=0i,rx_packets=94i,tx_bytes=946i,\
|
||||
tx_dropped=0i,tx_errors=0i,tx_packets=13i 1453409536840126713
|
||||
> docker_container_blkio,
|
||||
container_image=spotify/kafka,container_name=kafka,device=8:0 \
|
||||
> docker_blkio,cont_id=5705ba8ed8fb47527410653d60a8bb2f3af5e62372297c419022a3cc6d45d848,\
|
||||
cont_image=spotify/kafka,cont_name=kafka,device=8:0 \
|
||||
io_service_bytes_recursive_async=80216064i,io_service_bytes_recursive_read=79925248i,\
|
||||
io_service_bytes_recursive_sync=77824i,io_service_bytes_recursive_total=80293888i,\
|
||||
io_service_bytes_recursive_write=368640i,io_serviced_recursive_async=6562i,\
|
||||
|
||||
@@ -1,54 +1,25 @@
|
||||
package system
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"log"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"golang.org/x/net/context"
|
||||
|
||||
"github.com/docker/engine-api/client"
|
||||
"github.com/docker/engine-api/types"
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
|
||||
"github.com/fsouza/go-dockerclient"
|
||||
)
|
||||
|
||||
// Docker object
|
||||
type Docker struct {
|
||||
Endpoint string
|
||||
ContainerNames []string
|
||||
Timeout internal.Duration
|
||||
|
||||
client DockerClient
|
||||
client *docker.Client
|
||||
}
|
||||
|
||||
// DockerClient interface, useful for testing
|
||||
type DockerClient interface {
|
||||
Info(ctx context.Context) (types.Info, error)
|
||||
ContainerList(ctx context.Context, options types.ContainerListOptions) ([]types.Container, error)
|
||||
ContainerStats(ctx context.Context, containerID string, stream bool) (io.ReadCloser, error)
|
||||
}
|
||||
|
||||
// KB, MB, GB, TB, PB...human friendly
|
||||
const (
|
||||
KB = 1000
|
||||
MB = 1000 * KB
|
||||
GB = 1000 * MB
|
||||
TB = 1000 * GB
|
||||
PB = 1000 * TB
|
||||
)
|
||||
|
||||
var (
|
||||
sizeRegex = regexp.MustCompile(`^(\d+(\.\d+)*) ?([kKmMgGtTpP])?[bB]?$`)
|
||||
)
|
||||
|
||||
var sampleConfig = `
|
||||
## Docker Endpoint
|
||||
## To use TCP, set endpoint = "tcp://[ip]:[port]"
|
||||
@@ -56,36 +27,30 @@ var sampleConfig = `
|
||||
endpoint = "unix:///var/run/docker.sock"
|
||||
## Only collect metrics for these containers, collect all if empty
|
||||
container_names = []
|
||||
## Timeout for docker list, info, and stats commands
|
||||
timeout = "5s"
|
||||
`
|
||||
|
||||
// Description returns input description
|
||||
func (d *Docker) Description() string {
|
||||
return "Read metrics about docker containers"
|
||||
}
|
||||
|
||||
// SampleConfig prints sampleConfig
|
||||
func (d *Docker) SampleConfig() string { return sampleConfig }
|
||||
|
||||
// Gather starts stats collection
|
||||
func (d *Docker) Gather(acc telegraf.Accumulator) error {
|
||||
if d.client == nil {
|
||||
var c *client.Client
|
||||
var c *docker.Client
|
||||
var err error
|
||||
defaultHeaders := map[string]string{"User-Agent": "engine-api-cli-1.0"}
|
||||
if d.Endpoint == "ENV" {
|
||||
c, err = client.NewEnvClient()
|
||||
c, err = docker.NewClientFromEnv()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
} else if d.Endpoint == "" {
|
||||
c, err = client.NewClient("unix:///var/run/docker.sock", "", nil, defaultHeaders)
|
||||
c, err = docker.NewClient("unix:///var/run/docker.sock")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
} else {
|
||||
c, err = client.NewClient(d.Endpoint, "", nil, defaultHeaders)
|
||||
c, err = docker.NewClient(d.Endpoint)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -93,31 +58,21 @@ func (d *Docker) Gather(acc telegraf.Accumulator) error {
|
||||
d.client = c
|
||||
}
|
||||
|
||||
// Get daemon info
|
||||
err := d.gatherInfo(acc)
|
||||
if err != nil {
|
||||
fmt.Println(err.Error())
|
||||
}
|
||||
|
||||
// List containers
|
||||
opts := types.ContainerListOptions{}
|
||||
ctx, cancel := context.WithTimeout(context.Background(), d.Timeout.Duration)
|
||||
defer cancel()
|
||||
containers, err := d.client.ContainerList(ctx, opts)
|
||||
opts := docker.ListContainersOptions{}
|
||||
containers, err := d.client.ListContainers(opts)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Get container data
|
||||
var wg sync.WaitGroup
|
||||
wg.Add(len(containers))
|
||||
for _, container := range containers {
|
||||
go func(c types.Container) {
|
||||
|
||||
go func(c docker.APIContainers) {
|
||||
defer wg.Done()
|
||||
err := d.gatherContainer(c, acc)
|
||||
if err != nil {
|
||||
log.Printf("Error gathering container %s stats: %s\n",
|
||||
c.Names, err.Error())
|
||||
fmt.Println(err.Error())
|
||||
}
|
||||
}(container)
|
||||
}
|
||||
@@ -126,80 +81,10 @@ func (d *Docker) Gather(acc telegraf.Accumulator) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Docker) gatherInfo(acc telegraf.Accumulator) error {
|
||||
// Init vars
|
||||
dataFields := make(map[string]interface{})
|
||||
metadataFields := make(map[string]interface{})
|
||||
now := time.Now()
|
||||
// Get info from docker daemon
|
||||
ctx, cancel := context.WithTimeout(context.Background(), d.Timeout.Duration)
|
||||
defer cancel()
|
||||
info, err := d.client.Info(ctx)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"n_cpus": info.NCPU,
|
||||
"n_used_file_descriptors": info.NFd,
|
||||
"n_containers": info.Containers,
|
||||
"n_images": info.Images,
|
||||
"n_goroutines": info.NGoroutines,
|
||||
"n_listener_events": info.NEventsListener,
|
||||
}
|
||||
// Add metrics
|
||||
acc.AddFields("docker",
|
||||
fields,
|
||||
nil,
|
||||
now)
|
||||
acc.AddFields("docker",
|
||||
map[string]interface{}{"memory_total": info.MemTotal},
|
||||
map[string]string{"unit": "bytes"},
|
||||
now)
|
||||
// Get storage metrics
|
||||
for _, rawData := range info.DriverStatus {
|
||||
// Try to convert string to int (bytes)
|
||||
value, err := parseSize(rawData[1])
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
name := strings.ToLower(strings.Replace(rawData[0], " ", "_", -1))
|
||||
if name == "pool_blocksize" {
|
||||
// pool blocksize
|
||||
acc.AddFields("docker",
|
||||
map[string]interface{}{"pool_blocksize": value},
|
||||
map[string]string{"unit": "bytes"},
|
||||
now)
|
||||
} else if strings.HasPrefix(name, "data_space_") {
|
||||
// data space
|
||||
fieldName := strings.TrimPrefix(name, "data_space_")
|
||||
dataFields[fieldName] = value
|
||||
} else if strings.HasPrefix(name, "metadata_space_") {
|
||||
// metadata space
|
||||
fieldName := strings.TrimPrefix(name, "metadata_space_")
|
||||
metadataFields[fieldName] = value
|
||||
}
|
||||
}
|
||||
if len(dataFields) > 0 {
|
||||
acc.AddFields("docker_data",
|
||||
dataFields,
|
||||
map[string]string{"unit": "bytes"},
|
||||
now)
|
||||
}
|
||||
if len(metadataFields) > 0 {
|
||||
acc.AddFields("docker_metadata",
|
||||
metadataFields,
|
||||
map[string]string{"unit": "bytes"},
|
||||
now)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func (d *Docker) gatherContainer(
|
||||
container types.Container,
|
||||
container docker.APIContainers,
|
||||
acc telegraf.Accumulator,
|
||||
) error {
|
||||
var v *types.StatsJSON
|
||||
// Parse container name
|
||||
cname := "unknown"
|
||||
if len(container.Names) > 0 {
|
||||
@@ -208,8 +93,9 @@ func (d *Docker) gatherContainer(
|
||||
}
|
||||
|
||||
tags := map[string]string{
|
||||
"container_name": cname,
|
||||
"container_image": container.Image,
|
||||
"cont_id": container.ID,
|
||||
"cont_name": cname,
|
||||
"cont_image": container.Image,
|
||||
}
|
||||
if len(d.ContainerNames) > 0 {
|
||||
if !sliceContains(cname, d.ContainerNames) {
|
||||
@@ -217,36 +103,44 @@ func (d *Docker) gatherContainer(
|
||||
}
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(context.Background(), d.Timeout.Duration)
|
||||
defer cancel()
|
||||
r, err := d.client.ContainerStats(ctx, container.ID, false)
|
||||
if err != nil {
|
||||
log.Printf("Error getting docker stats: %s\n", err.Error())
|
||||
statChan := make(chan *docker.Stats)
|
||||
done := make(chan bool)
|
||||
statOpts := docker.StatsOptions{
|
||||
Stream: false,
|
||||
ID: container.ID,
|
||||
Stats: statChan,
|
||||
Done: done,
|
||||
Timeout: time.Duration(time.Second * 5),
|
||||
}
|
||||
defer r.Close()
|
||||
dec := json.NewDecoder(r)
|
||||
if err = dec.Decode(&v); err != nil {
|
||||
if err == io.EOF {
|
||||
return nil
|
||||
|
||||
go func() {
|
||||
err := d.client.Stats(statOpts)
|
||||
if err != nil {
|
||||
log.Printf("Error getting docker stats: %s\n", err.Error())
|
||||
}
|
||||
return fmt.Errorf("Error decoding: %s", err.Error())
|
||||
}()
|
||||
|
||||
stat := <-statChan
|
||||
close(done)
|
||||
|
||||
if stat == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Add labels to tags
|
||||
for k, label := range container.Labels {
|
||||
tags[k] = label
|
||||
for k, v := range container.Labels {
|
||||
tags[k] = v
|
||||
}
|
||||
|
||||
gatherContainerStats(v, acc, tags, container.ID)
|
||||
gatherContainerStats(stat, acc, tags)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func gatherContainerStats(
|
||||
stat *types.StatsJSON,
|
||||
stat *docker.Stats,
|
||||
acc telegraf.Accumulator,
|
||||
tags map[string]string,
|
||||
id string,
|
||||
) {
|
||||
now := stat.Read
|
||||
|
||||
@@ -255,87 +149,80 @@ func gatherContainerStats(
|
||||
"usage": stat.MemoryStats.Usage,
|
||||
"fail_count": stat.MemoryStats.Failcnt,
|
||||
"limit": stat.MemoryStats.Limit,
|
||||
"total_pgmafault": stat.MemoryStats.Stats["total_pgmajfault"],
|
||||
"cache": stat.MemoryStats.Stats["cache"],
|
||||
"mapped_file": stat.MemoryStats.Stats["mapped_file"],
|
||||
"total_inactive_file": stat.MemoryStats.Stats["total_inactive_file"],
|
||||
"pgpgout": stat.MemoryStats.Stats["pagpgout"],
|
||||
"rss": stat.MemoryStats.Stats["rss"],
|
||||
"total_mapped_file": stat.MemoryStats.Stats["total_mapped_file"],
|
||||
"writeback": stat.MemoryStats.Stats["writeback"],
|
||||
"unevictable": stat.MemoryStats.Stats["unevictable"],
|
||||
"pgpgin": stat.MemoryStats.Stats["pgpgin"],
|
||||
"total_unevictable": stat.MemoryStats.Stats["total_unevictable"],
|
||||
"pgmajfault": stat.MemoryStats.Stats["pgmajfault"],
|
||||
"total_rss": stat.MemoryStats.Stats["total_rss"],
|
||||
"total_rss_huge": stat.MemoryStats.Stats["total_rss_huge"],
|
||||
"total_writeback": stat.MemoryStats.Stats["total_write_back"],
|
||||
"total_inactive_anon": stat.MemoryStats.Stats["total_inactive_anon"],
|
||||
"rss_huge": stat.MemoryStats.Stats["rss_huge"],
|
||||
"hierarchical_memory_limit": stat.MemoryStats.Stats["hierarchical_memory_limit"],
|
||||
"total_pgfault": stat.MemoryStats.Stats["total_pgfault"],
|
||||
"total_active_file": stat.MemoryStats.Stats["total_active_file"],
|
||||
"active_anon": stat.MemoryStats.Stats["active_anon"],
|
||||
"total_active_anon": stat.MemoryStats.Stats["total_active_anon"],
|
||||
"total_pgpgout": stat.MemoryStats.Stats["total_pgpgout"],
|
||||
"total_cache": stat.MemoryStats.Stats["total_cache"],
|
||||
"inactive_anon": stat.MemoryStats.Stats["inactive_anon"],
|
||||
"active_file": stat.MemoryStats.Stats["active_file"],
|
||||
"pgfault": stat.MemoryStats.Stats["pgfault"],
|
||||
"inactive_file": stat.MemoryStats.Stats["inactive_file"],
|
||||
"total_pgpgin": stat.MemoryStats.Stats["total_pgpgin"],
|
||||
"total_pgmafault": stat.MemoryStats.Stats.TotalPgmafault,
|
||||
"cache": stat.MemoryStats.Stats.Cache,
|
||||
"mapped_file": stat.MemoryStats.Stats.MappedFile,
|
||||
"total_inactive_file": stat.MemoryStats.Stats.TotalInactiveFile,
|
||||
"pgpgout": stat.MemoryStats.Stats.Pgpgout,
|
||||
"rss": stat.MemoryStats.Stats.Rss,
|
||||
"total_mapped_file": stat.MemoryStats.Stats.TotalMappedFile,
|
||||
"writeback": stat.MemoryStats.Stats.Writeback,
|
||||
"unevictable": stat.MemoryStats.Stats.Unevictable,
|
||||
"pgpgin": stat.MemoryStats.Stats.Pgpgin,
|
||||
"total_unevictable": stat.MemoryStats.Stats.TotalUnevictable,
|
||||
"pgmajfault": stat.MemoryStats.Stats.Pgmajfault,
|
||||
"total_rss": stat.MemoryStats.Stats.TotalRss,
|
||||
"total_rss_huge": stat.MemoryStats.Stats.TotalRssHuge,
|
||||
"total_writeback": stat.MemoryStats.Stats.TotalWriteback,
|
||||
"total_inactive_anon": stat.MemoryStats.Stats.TotalInactiveAnon,
|
||||
"rss_huge": stat.MemoryStats.Stats.RssHuge,
|
||||
"hierarchical_memory_limit": stat.MemoryStats.Stats.HierarchicalMemoryLimit,
|
||||
"total_pgfault": stat.MemoryStats.Stats.TotalPgfault,
|
||||
"total_active_file": stat.MemoryStats.Stats.TotalActiveFile,
|
||||
"active_anon": stat.MemoryStats.Stats.ActiveAnon,
|
||||
"total_active_anon": stat.MemoryStats.Stats.TotalActiveAnon,
|
||||
"total_pgpgout": stat.MemoryStats.Stats.TotalPgpgout,
|
||||
"total_cache": stat.MemoryStats.Stats.TotalCache,
|
||||
"inactive_anon": stat.MemoryStats.Stats.InactiveAnon,
|
||||
"active_file": stat.MemoryStats.Stats.ActiveFile,
|
||||
"pgfault": stat.MemoryStats.Stats.Pgfault,
|
||||
"inactive_file": stat.MemoryStats.Stats.InactiveFile,
|
||||
"total_pgpgin": stat.MemoryStats.Stats.TotalPgpgin,
|
||||
"usage_percent": calculateMemPercent(stat),
|
||||
"container_id": id,
|
||||
}
|
||||
acc.AddFields("docker_container_mem", memfields, tags, now)
|
||||
acc.AddFields("docker_mem", memfields, tags, now)
|
||||
|
||||
cpufields := map[string]interface{}{
|
||||
"usage_total": stat.CPUStats.CPUUsage.TotalUsage,
|
||||
"usage_in_usermode": stat.CPUStats.CPUUsage.UsageInUsermode,
|
||||
"usage_in_kernelmode": stat.CPUStats.CPUUsage.UsageInKernelmode,
|
||||
"usage_system": stat.CPUStats.SystemUsage,
|
||||
"usage_system": stat.CPUStats.SystemCPUUsage,
|
||||
"throttling_periods": stat.CPUStats.ThrottlingData.Periods,
|
||||
"throttling_throttled_periods": stat.CPUStats.ThrottlingData.ThrottledPeriods,
|
||||
"throttling_throttled_time": stat.CPUStats.ThrottlingData.ThrottledTime,
|
||||
"usage_percent": calculateCPUPercent(stat),
|
||||
"container_id": id,
|
||||
}
|
||||
cputags := copyTags(tags)
|
||||
cputags["cpu"] = "cpu-total"
|
||||
acc.AddFields("docker_container_cpu", cpufields, cputags, now)
|
||||
acc.AddFields("docker_cpu", cpufields, cputags, now)
|
||||
|
||||
for i, percpu := range stat.CPUStats.CPUUsage.PercpuUsage {
|
||||
percputags := copyTags(tags)
|
||||
percputags["cpu"] = fmt.Sprintf("cpu%d", i)
|
||||
fields := map[string]interface{}{
|
||||
"usage_total": percpu,
|
||||
"container_id": id,
|
||||
}
|
||||
acc.AddFields("docker_container_cpu", fields, percputags, now)
|
||||
acc.AddFields("docker_cpu", map[string]interface{}{"usage_total": percpu}, percputags, now)
|
||||
}
|
||||
|
||||
for network, netstats := range stat.Networks {
|
||||
netfields := map[string]interface{}{
|
||||
"rx_dropped": netstats.RxDropped,
|
||||
"rx_bytes": netstats.RxBytes,
|
||||
"rx_errors": netstats.RxErrors,
|
||||
"tx_packets": netstats.TxPackets,
|
||||
"tx_dropped": netstats.TxDropped,
|
||||
"rx_packets": netstats.RxPackets,
|
||||
"tx_errors": netstats.TxErrors,
|
||||
"tx_bytes": netstats.TxBytes,
|
||||
"container_id": id,
|
||||
"rx_dropped": netstats.RxDropped,
|
||||
"rx_bytes": netstats.RxBytes,
|
||||
"rx_errors": netstats.RxErrors,
|
||||
"tx_packets": netstats.TxPackets,
|
||||
"tx_dropped": netstats.TxDropped,
|
||||
"rx_packets": netstats.RxPackets,
|
||||
"tx_errors": netstats.TxErrors,
|
||||
"tx_bytes": netstats.TxBytes,
|
||||
}
|
||||
// Create a new network tag dictionary for the "network" tag
|
||||
nettags := copyTags(tags)
|
||||
nettags["network"] = network
|
||||
acc.AddFields("docker_container_net", netfields, nettags, now)
|
||||
acc.AddFields("docker_net", netfields, nettags, now)
|
||||
}
|
||||
|
||||
gatherBlockIOMetrics(stat, acc, tags, now, id)
|
||||
gatherBlockIOMetrics(stat, acc, tags, now)
|
||||
}
|
||||
|
||||
func calculateMemPercent(stat *types.StatsJSON) float64 {
|
||||
func calculateMemPercent(stat *docker.Stats) float64 {
|
||||
var memPercent = 0.0
|
||||
if stat.MemoryStats.Limit > 0 {
|
||||
memPercent = float64(stat.MemoryStats.Usage) / float64(stat.MemoryStats.Limit) * 100.0
|
||||
@@ -343,11 +230,11 @@ func calculateMemPercent(stat *types.StatsJSON) float64 {
|
||||
return memPercent
|
||||
}
|
||||
|
||||
func calculateCPUPercent(stat *types.StatsJSON) float64 {
|
||||
func calculateCPUPercent(stat *docker.Stats) float64 {
|
||||
var cpuPercent = 0.0
|
||||
// calculate the change for the cpu and system usage of the container in between readings
|
||||
cpuDelta := float64(stat.CPUStats.CPUUsage.TotalUsage) - float64(stat.PreCPUStats.CPUUsage.TotalUsage)
|
||||
systemDelta := float64(stat.CPUStats.SystemUsage) - float64(stat.PreCPUStats.SystemUsage)
|
||||
systemDelta := float64(stat.CPUStats.SystemCPUUsage) - float64(stat.PreCPUStats.SystemCPUUsage)
|
||||
|
||||
if systemDelta > 0.0 && cpuDelta > 0.0 {
|
||||
cpuPercent = (cpuDelta / systemDelta) * float64(len(stat.CPUStats.CPUUsage.PercpuUsage)) * 100.0
|
||||
@@ -356,17 +243,16 @@ func calculateCPUPercent(stat *types.StatsJSON) float64 {
|
||||
}
|
||||
|
||||
func gatherBlockIOMetrics(
|
||||
stat *types.StatsJSON,
|
||||
stat *docker.Stats,
|
||||
acc telegraf.Accumulator,
|
||||
tags map[string]string,
|
||||
now time.Time,
|
||||
id string,
|
||||
) {
|
||||
blkioStats := stat.BlkioStats
|
||||
// Make a map of devices to their block io stats
|
||||
deviceStatMap := make(map[string]map[string]interface{})
|
||||
|
||||
for _, metric := range blkioStats.IoServiceBytesRecursive {
|
||||
for _, metric := range blkioStats.IOServiceBytesRecursive {
|
||||
device := fmt.Sprintf("%d:%d", metric.Major, metric.Minor)
|
||||
_, ok := deviceStatMap[device]
|
||||
if !ok {
|
||||
@@ -377,7 +263,7 @@ func gatherBlockIOMetrics(
|
||||
deviceStatMap[device][field] = metric.Value
|
||||
}
|
||||
|
||||
for _, metric := range blkioStats.IoServicedRecursive {
|
||||
for _, metric := range blkioStats.IOServicedRecursive {
|
||||
device := fmt.Sprintf("%d:%d", metric.Major, metric.Minor)
|
||||
_, ok := deviceStatMap[device]
|
||||
if !ok {
|
||||
@@ -388,45 +274,46 @@ func gatherBlockIOMetrics(
|
||||
deviceStatMap[device][field] = metric.Value
|
||||
}
|
||||
|
||||
for _, metric := range blkioStats.IoQueuedRecursive {
|
||||
for _, metric := range blkioStats.IOQueueRecursive {
|
||||
device := fmt.Sprintf("%d:%d", metric.Major, metric.Minor)
|
||||
field := fmt.Sprintf("io_queue_recursive_%s", strings.ToLower(metric.Op))
|
||||
deviceStatMap[device][field] = metric.Value
|
||||
}
|
||||
|
||||
for _, metric := range blkioStats.IoServiceTimeRecursive {
|
||||
for _, metric := range blkioStats.IOServiceTimeRecursive {
|
||||
device := fmt.Sprintf("%d:%d", metric.Major, metric.Minor)
|
||||
field := fmt.Sprintf("io_service_time_recursive_%s", strings.ToLower(metric.Op))
|
||||
deviceStatMap[device][field] = metric.Value
|
||||
}
|
||||
|
||||
for _, metric := range blkioStats.IoWaitTimeRecursive {
|
||||
for _, metric := range blkioStats.IOWaitTimeRecursive {
|
||||
device := fmt.Sprintf("%d:%d", metric.Major, metric.Minor)
|
||||
field := fmt.Sprintf("io_wait_time_%s", strings.ToLower(metric.Op))
|
||||
deviceStatMap[device][field] = metric.Value
|
||||
}
|
||||
|
||||
for _, metric := range blkioStats.IoMergedRecursive {
|
||||
for _, metric := range blkioStats.IOMergedRecursive {
|
||||
device := fmt.Sprintf("%d:%d", metric.Major, metric.Minor)
|
||||
field := fmt.Sprintf("io_merged_recursive_%s", strings.ToLower(metric.Op))
|
||||
deviceStatMap[device][field] = metric.Value
|
||||
}
|
||||
|
||||
for _, metric := range blkioStats.IoTimeRecursive {
|
||||
for _, metric := range blkioStats.IOTimeRecursive {
|
||||
device := fmt.Sprintf("%d:%d", metric.Major, metric.Minor)
|
||||
deviceStatMap[device]["io_time_recursive"] = metric.Value
|
||||
field := fmt.Sprintf("io_time_recursive_%s", strings.ToLower(metric.Op))
|
||||
deviceStatMap[device][field] = metric.Value
|
||||
}
|
||||
|
||||
for _, metric := range blkioStats.SectorsRecursive {
|
||||
device := fmt.Sprintf("%d:%d", metric.Major, metric.Minor)
|
||||
deviceStatMap[device]["sectors_recursive"] = metric.Value
|
||||
field := fmt.Sprintf("sectors_recursive_%s", strings.ToLower(metric.Op))
|
||||
deviceStatMap[device][field] = metric.Value
|
||||
}
|
||||
|
||||
for device, fields := range deviceStatMap {
|
||||
iotags := copyTags(tags)
|
||||
iotags["device"] = device
|
||||
fields["container_id"] = id
|
||||
acc.AddFields("docker_container_blkio", fields, iotags, now)
|
||||
acc.AddFields("docker_blkio", fields, iotags, now)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -447,31 +334,8 @@ func sliceContains(in string, sl []string) bool {
|
||||
return false
|
||||
}
|
||||
|
||||
// Parses the human-readable size string into the amount it represents.
|
||||
func parseSize(sizeStr string) (int64, error) {
|
||||
matches := sizeRegex.FindStringSubmatch(sizeStr)
|
||||
if len(matches) != 4 {
|
||||
return -1, fmt.Errorf("invalid size: '%s'", sizeStr)
|
||||
}
|
||||
|
||||
size, err := strconv.ParseFloat(matches[1], 64)
|
||||
if err != nil {
|
||||
return -1, err
|
||||
}
|
||||
|
||||
uMap := map[string]int64{"k": KB, "m": MB, "g": GB, "t": TB, "p": PB}
|
||||
unitPrefix := strings.ToLower(matches[3])
|
||||
if mul, ok := uMap[unitPrefix]; ok {
|
||||
size *= float64(mul)
|
||||
}
|
||||
|
||||
return int64(size), nil
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("docker", func() telegraf.Input {
|
||||
return &Docker{
|
||||
Timeout: internal.Duration{Duration: time.Second * 5},
|
||||
}
|
||||
return &Docker{}
|
||||
})
|
||||
}
|
||||
|
||||
@@ -1,19 +1,12 @@
|
||||
package system
|
||||
|
||||
import (
|
||||
"io"
|
||||
"io/ioutil"
|
||||
"strings"
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"golang.org/x/net/context"
|
||||
|
||||
"github.com/docker/engine-api/types"
|
||||
"github.com/docker/engine-api/types/registry"
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
|
||||
"github.com/stretchr/testify/require"
|
||||
"github.com/fsouza/go-dockerclient"
|
||||
)
|
||||
|
||||
func TestDockerGatherContainerStats(t *testing.T) {
|
||||
@@ -21,26 +14,26 @@ func TestDockerGatherContainerStats(t *testing.T) {
|
||||
stats := testStats()
|
||||
|
||||
tags := map[string]string{
|
||||
"container_name": "redis",
|
||||
"container_image": "redis/image",
|
||||
"cont_id": "foobarbaz",
|
||||
"cont_name": "redis",
|
||||
"cont_image": "redis/image",
|
||||
}
|
||||
gatherContainerStats(stats, &acc, tags, "123456789")
|
||||
gatherContainerStats(stats, &acc, tags)
|
||||
|
||||
// test docker_container_net measurement
|
||||
// test docker_net measurement
|
||||
netfields := map[string]interface{}{
|
||||
"rx_dropped": uint64(1),
|
||||
"rx_bytes": uint64(2),
|
||||
"rx_errors": uint64(3),
|
||||
"tx_packets": uint64(4),
|
||||
"tx_dropped": uint64(1),
|
||||
"rx_packets": uint64(2),
|
||||
"tx_errors": uint64(3),
|
||||
"tx_bytes": uint64(4),
|
||||
"container_id": "123456789",
|
||||
"rx_dropped": uint64(1),
|
||||
"rx_bytes": uint64(2),
|
||||
"rx_errors": uint64(3),
|
||||
"tx_packets": uint64(4),
|
||||
"tx_dropped": uint64(1),
|
||||
"rx_packets": uint64(2),
|
||||
"tx_errors": uint64(3),
|
||||
"tx_bytes": uint64(4),
|
||||
}
|
||||
nettags := copyTags(tags)
|
||||
nettags["network"] = "eth0"
|
||||
acc.AssertContainsTaggedFields(t, "docker_container_net", netfields, nettags)
|
||||
acc.AssertContainsTaggedFields(t, "docker_net", netfields, nettags)
|
||||
|
||||
// test docker_blkio measurement
|
||||
blkiotags := copyTags(tags)
|
||||
@@ -48,11 +41,10 @@ func TestDockerGatherContainerStats(t *testing.T) {
|
||||
blkiofields := map[string]interface{}{
|
||||
"io_service_bytes_recursive_read": uint64(100),
|
||||
"io_serviced_recursive_write": uint64(101),
|
||||
"container_id": "123456789",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "docker_container_blkio", blkiofields, blkiotags)
|
||||
acc.AssertContainsTaggedFields(t, "docker_blkio", blkiofields, blkiotags)
|
||||
|
||||
// test docker_container_mem measurement
|
||||
// test docker_mem measurement
|
||||
memfields := map[string]interface{}{
|
||||
"max_usage": uint64(1001),
|
||||
"usage": uint64(1111),
|
||||
@@ -88,12 +80,11 @@ func TestDockerGatherContainerStats(t *testing.T) {
|
||||
"inactive_file": uint64(3),
|
||||
"total_pgpgin": uint64(4),
|
||||
"usage_percent": float64(55.55),
|
||||
"container_id": "123456789",
|
||||
}
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "docker_container_mem", memfields, tags)
|
||||
acc.AssertContainsTaggedFields(t, "docker_mem", memfields, tags)
|
||||
|
||||
// test docker_container_cpu measurement
|
||||
// test docker_cpu measurement
|
||||
cputags := copyTags(tags)
|
||||
cputags["cpu"] = "cpu-total"
|
||||
cpufields := map[string]interface{}{
|
||||
@@ -105,77 +96,74 @@ func TestDockerGatherContainerStats(t *testing.T) {
|
||||
"throttling_throttled_periods": uint64(0),
|
||||
"throttling_throttled_time": uint64(0),
|
||||
"usage_percent": float64(400.0),
|
||||
"container_id": "123456789",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "docker_container_cpu", cpufields, cputags)
|
||||
acc.AssertContainsTaggedFields(t, "docker_cpu", cpufields, cputags)
|
||||
|
||||
cputags["cpu"] = "cpu0"
|
||||
cpu0fields := map[string]interface{}{
|
||||
"usage_total": uint64(1),
|
||||
"container_id": "123456789",
|
||||
"usage_total": uint64(1),
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "docker_container_cpu", cpu0fields, cputags)
|
||||
acc.AssertContainsTaggedFields(t, "docker_cpu", cpu0fields, cputags)
|
||||
|
||||
cputags["cpu"] = "cpu1"
|
||||
cpu1fields := map[string]interface{}{
|
||||
"usage_total": uint64(1002),
|
||||
"container_id": "123456789",
|
||||
"usage_total": uint64(1002),
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "docker_container_cpu", cpu1fields, cputags)
|
||||
acc.AssertContainsTaggedFields(t, "docker_cpu", cpu1fields, cputags)
|
||||
}
|
||||
|
||||
func testStats() *types.StatsJSON {
|
||||
stats := &types.StatsJSON{}
|
||||
stats.Read = time.Now()
|
||||
stats.Networks = make(map[string]types.NetworkStats)
|
||||
func testStats() *docker.Stats {
|
||||
stats := &docker.Stats{
|
||||
Read: time.Now(),
|
||||
Networks: make(map[string]docker.NetworkStats),
|
||||
}
|
||||
|
||||
stats.CPUStats.CPUUsage.PercpuUsage = []uint64{1, 1002}
|
||||
stats.CPUStats.CPUUsage.UsageInUsermode = 100
|
||||
stats.CPUStats.CPUUsage.TotalUsage = 500
|
||||
stats.CPUStats.CPUUsage.UsageInKernelmode = 200
|
||||
stats.CPUStats.SystemUsage = 100
|
||||
stats.CPUStats.SystemCPUUsage = 100
|
||||
stats.CPUStats.ThrottlingData.Periods = 1
|
||||
|
||||
stats.PreCPUStats.CPUUsage.TotalUsage = 400
|
||||
stats.PreCPUStats.SystemUsage = 50
|
||||
stats.PreCPUStats.SystemCPUUsage = 50
|
||||
|
||||
stats.MemoryStats.Stats = make(map[string]uint64)
|
||||
stats.MemoryStats.Stats["total_pgmajfault"] = 0
|
||||
stats.MemoryStats.Stats["cache"] = 0
|
||||
stats.MemoryStats.Stats["mapped_file"] = 0
|
||||
stats.MemoryStats.Stats["total_inactive_file"] = 0
|
||||
stats.MemoryStats.Stats["pagpgout"] = 0
|
||||
stats.MemoryStats.Stats["rss"] = 0
|
||||
stats.MemoryStats.Stats["total_mapped_file"] = 0
|
||||
stats.MemoryStats.Stats["writeback"] = 0
|
||||
stats.MemoryStats.Stats["unevictable"] = 0
|
||||
stats.MemoryStats.Stats["pgpgin"] = 0
|
||||
stats.MemoryStats.Stats["total_unevictable"] = 0
|
||||
stats.MemoryStats.Stats["pgmajfault"] = 0
|
||||
stats.MemoryStats.Stats["total_rss"] = 44
|
||||
stats.MemoryStats.Stats["total_rss_huge"] = 444
|
||||
stats.MemoryStats.Stats["total_write_back"] = 55
|
||||
stats.MemoryStats.Stats["total_inactive_anon"] = 0
|
||||
stats.MemoryStats.Stats["rss_huge"] = 0
|
||||
stats.MemoryStats.Stats["hierarchical_memory_limit"] = 0
|
||||
stats.MemoryStats.Stats["total_pgfault"] = 0
|
||||
stats.MemoryStats.Stats["total_active_file"] = 0
|
||||
stats.MemoryStats.Stats["active_anon"] = 0
|
||||
stats.MemoryStats.Stats["total_active_anon"] = 0
|
||||
stats.MemoryStats.Stats["total_pgpgout"] = 0
|
||||
stats.MemoryStats.Stats["total_cache"] = 0
|
||||
stats.MemoryStats.Stats["inactive_anon"] = 0
|
||||
stats.MemoryStats.Stats["active_file"] = 1
|
||||
stats.MemoryStats.Stats["pgfault"] = 2
|
||||
stats.MemoryStats.Stats["inactive_file"] = 3
|
||||
stats.MemoryStats.Stats["total_pgpgin"] = 4
|
||||
stats.MemoryStats.Stats.TotalPgmafault = 0
|
||||
stats.MemoryStats.Stats.Cache = 0
|
||||
stats.MemoryStats.Stats.MappedFile = 0
|
||||
stats.MemoryStats.Stats.TotalInactiveFile = 0
|
||||
stats.MemoryStats.Stats.Pgpgout = 0
|
||||
stats.MemoryStats.Stats.Rss = 0
|
||||
stats.MemoryStats.Stats.TotalMappedFile = 0
|
||||
stats.MemoryStats.Stats.Writeback = 0
|
||||
stats.MemoryStats.Stats.Unevictable = 0
|
||||
stats.MemoryStats.Stats.Pgpgin = 0
|
||||
stats.MemoryStats.Stats.TotalUnevictable = 0
|
||||
stats.MemoryStats.Stats.Pgmajfault = 0
|
||||
stats.MemoryStats.Stats.TotalRss = 44
|
||||
stats.MemoryStats.Stats.TotalRssHuge = 444
|
||||
stats.MemoryStats.Stats.TotalWriteback = 55
|
||||
stats.MemoryStats.Stats.TotalInactiveAnon = 0
|
||||
stats.MemoryStats.Stats.RssHuge = 0
|
||||
stats.MemoryStats.Stats.HierarchicalMemoryLimit = 0
|
||||
stats.MemoryStats.Stats.TotalPgfault = 0
|
||||
stats.MemoryStats.Stats.TotalActiveFile = 0
|
||||
stats.MemoryStats.Stats.ActiveAnon = 0
|
||||
stats.MemoryStats.Stats.TotalActiveAnon = 0
|
||||
stats.MemoryStats.Stats.TotalPgpgout = 0
|
||||
stats.MemoryStats.Stats.TotalCache = 0
|
||||
stats.MemoryStats.Stats.InactiveAnon = 0
|
||||
stats.MemoryStats.Stats.ActiveFile = 1
|
||||
stats.MemoryStats.Stats.Pgfault = 2
|
||||
stats.MemoryStats.Stats.InactiveFile = 3
|
||||
stats.MemoryStats.Stats.TotalPgpgin = 4
|
||||
|
||||
stats.MemoryStats.MaxUsage = 1001
|
||||
stats.MemoryStats.Usage = 1111
|
||||
stats.MemoryStats.Failcnt = 1
|
||||
stats.MemoryStats.Limit = 2000
|
||||
|
||||
stats.Networks["eth0"] = types.NetworkStats{
|
||||
stats.Networks["eth0"] = docker.NetworkStats{
|
||||
RxDropped: 1,
|
||||
RxBytes: 2,
|
||||
RxErrors: 3,
|
||||
@@ -186,247 +174,23 @@ func testStats() *types.StatsJSON {
|
||||
TxBytes: 4,
|
||||
}
|
||||
|
||||
sbr := types.BlkioStatEntry{
|
||||
sbr := docker.BlkioStatsEntry{
|
||||
Major: 6,
|
||||
Minor: 0,
|
||||
Op: "read",
|
||||
Value: 100,
|
||||
}
|
||||
sr := types.BlkioStatEntry{
|
||||
sr := docker.BlkioStatsEntry{
|
||||
Major: 6,
|
||||
Minor: 0,
|
||||
Op: "write",
|
||||
Value: 101,
|
||||
}
|
||||
|
||||
stats.BlkioStats.IoServiceBytesRecursive = append(
|
||||
stats.BlkioStats.IoServiceBytesRecursive, sbr)
|
||||
stats.BlkioStats.IoServicedRecursive = append(
|
||||
stats.BlkioStats.IoServicedRecursive, sr)
|
||||
stats.BlkioStats.IOServiceBytesRecursive = append(
|
||||
stats.BlkioStats.IOServiceBytesRecursive, sbr)
|
||||
stats.BlkioStats.IOServicedRecursive = append(
|
||||
stats.BlkioStats.IOServicedRecursive, sr)
|
||||
|
||||
return stats
|
||||
}
|
||||
|
||||
type FakeDockerClient struct {
|
||||
}
|
||||
|
||||
func (d FakeDockerClient) Info(ctx context.Context) (types.Info, error) {
|
||||
env := types.Info{
|
||||
Containers: 108,
|
||||
OomKillDisable: false,
|
||||
SystemTime: "2016-02-24T00:55:09.15073105-05:00",
|
||||
NEventsListener: 0,
|
||||
ID: "5WQQ:TFWR:FDNG:OKQ3:37Y4:FJWG:QIKK:623T:R3ME:QTKB:A7F7:OLHD",
|
||||
Debug: false,
|
||||
LoggingDriver: "json-file",
|
||||
KernelVersion: "4.3.0-1-amd64",
|
||||
IndexServerAddress: "https://index.docker.io/v1/",
|
||||
MemTotal: 3840757760,
|
||||
Images: 199,
|
||||
CPUCfsQuota: true,
|
||||
Name: "absol",
|
||||
SwapLimit: false,
|
||||
IPv4Forwarding: true,
|
||||
ExecutionDriver: "native-0.2",
|
||||
ExperimentalBuild: false,
|
||||
CPUCfsPeriod: true,
|
||||
RegistryConfig: ®istry.ServiceConfig{
|
||||
IndexConfigs: map[string]*registry.IndexInfo{
|
||||
"docker.io": {
|
||||
Name: "docker.io",
|
||||
Mirrors: []string{},
|
||||
Official: true,
|
||||
Secure: true,
|
||||
},
|
||||
}, InsecureRegistryCIDRs: []*registry.NetIPNet{{IP: []byte{127, 0, 0, 0}, Mask: []byte{255, 0, 0, 0}}}, Mirrors: []string{}},
|
||||
OperatingSystem: "Linux Mint LMDE (containerized)",
|
||||
BridgeNfIptables: true,
|
||||
HTTPSProxy: "",
|
||||
Labels: []string{},
|
||||
MemoryLimit: false,
|
||||
DriverStatus: [][2]string{{"Pool Name", "docker-8:1-1182287-pool"}, {"Pool Blocksize", "65.54 kB"}, {"Backing Filesystem", "extfs"}, {"Data file", "/dev/loop0"}, {"Metadata file", "/dev/loop1"}, {"Data Space Used", "17.3 GB"}, {"Data Space Total", "107.4 GB"}, {"Data Space Available", "36.53 GB"}, {"Metadata Space Used", "20.97 MB"}, {"Metadata Space Total", "2.147 GB"}, {"Metadata Space Available", "2.127 GB"}, {"Udev Sync Supported", "true"}, {"Deferred Removal Enabled", "false"}, {"Data loop file", "/var/lib/docker/devicemapper/devicemapper/data"}, {"Metadata loop file", "/var/lib/docker/devicemapper/devicemapper/metadata"}, {"Library Version", "1.02.115 (2016-01-25)"}},
|
||||
NFd: 19,
|
||||
HTTPProxy: "",
|
||||
Driver: "devicemapper",
|
||||
NGoroutines: 39,
|
||||
NCPU: 4,
|
||||
DockerRootDir: "/var/lib/docker",
|
||||
NoProxy: "",
|
||||
BridgeNfIP6tables: true,
|
||||
}
|
||||
return env, nil
|
||||
}
|
||||
|
||||
func (d FakeDockerClient) ContainerList(octx context.Context, options types.ContainerListOptions) ([]types.Container, error) {
|
||||
container1 := types.Container{
|
||||
ID: "e2173b9478a6ae55e237d4d74f8bbb753f0817192b5081334dc78476296b7dfb",
|
||||
Names: []string{"/etcd"},
|
||||
Image: "quay.io/coreos/etcd:v2.2.2",
|
||||
Command: "/etcd -name etcd0 -advertise-client-urls http://localhost:2379 -listen-client-urls http://0.0.0.0:2379",
|
||||
Created: 1455941930,
|
||||
Status: "Up 4 hours",
|
||||
Ports: []types.Port{
|
||||
types.Port{
|
||||
PrivatePort: 7001,
|
||||
PublicPort: 0,
|
||||
Type: "tcp",
|
||||
},
|
||||
types.Port{
|
||||
PrivatePort: 4001,
|
||||
PublicPort: 0,
|
||||
Type: "tcp",
|
||||
},
|
||||
types.Port{
|
||||
PrivatePort: 2380,
|
||||
PublicPort: 0,
|
||||
Type: "tcp",
|
||||
},
|
||||
types.Port{
|
||||
PrivatePort: 2379,
|
||||
PublicPort: 2379,
|
||||
Type: "tcp",
|
||||
IP: "0.0.0.0",
|
||||
},
|
||||
},
|
||||
SizeRw: 0,
|
||||
SizeRootFs: 0,
|
||||
}
|
||||
container2 := types.Container{
|
||||
ID: "b7dfbb9478a6ae55e237d4d74f8bbb753f0817192b5081334dc78476296e2173",
|
||||
Names: []string{"/etcd2"},
|
||||
Image: "quay.io/coreos/etcd:v2.2.2",
|
||||
Command: "/etcd -name etcd2 -advertise-client-urls http://localhost:2379 -listen-client-urls http://0.0.0.0:2379",
|
||||
Created: 1455941933,
|
||||
Status: "Up 4 hours",
|
||||
Ports: []types.Port{
|
||||
types.Port{
|
||||
PrivatePort: 7002,
|
||||
PublicPort: 0,
|
||||
Type: "tcp",
|
||||
},
|
||||
types.Port{
|
||||
PrivatePort: 4002,
|
||||
PublicPort: 0,
|
||||
Type: "tcp",
|
||||
},
|
||||
types.Port{
|
||||
PrivatePort: 2381,
|
||||
PublicPort: 0,
|
||||
Type: "tcp",
|
||||
},
|
||||
types.Port{
|
||||
PrivatePort: 2382,
|
||||
PublicPort: 2382,
|
||||
Type: "tcp",
|
||||
IP: "0.0.0.0",
|
||||
},
|
||||
},
|
||||
SizeRw: 0,
|
||||
SizeRootFs: 0,
|
||||
}
|
||||
|
||||
containers := []types.Container{container1, container2}
|
||||
return containers, nil
|
||||
|
||||
//#{e6a96c84ca91a5258b7cb752579fb68826b68b49ff957487695cd4d13c343b44 titilambert/snmpsim /bin/sh -c 'snmpsimd --agent-udpv4-endpoint=0.0.0.0:31161 --process-user=root --process-group=user' 1455724831 Up 4 hours [{31161 31161 udp 0.0.0.0}] 0 0 [/snmp] map[]}]2016/02/24 01:05:01 Gathered metrics, (3s interval), from 1 inputs in 1.233836656s
|
||||
}
|
||||
|
||||
func (d FakeDockerClient) ContainerStats(ctx context.Context, containerID string, stream bool) (io.ReadCloser, error) {
|
||||
var stat io.ReadCloser
|
||||
jsonStat := `{"read":"2016-02-24T11:42:27.472459608-05:00","memory_stats":{"stats":{},"limit":18935443456},"blkio_stats":{"io_service_bytes_recursive":[{"major":252,"minor":1,"op":"Read","value":753664},{"major":252,"minor":1,"op":"Write"},{"major":252,"minor":1,"op":"Sync"},{"major":252,"minor":1,"op":"Async","value":753664},{"major":252,"minor":1,"op":"Total","value":753664}],"io_serviced_recursive":[{"major":252,"minor":1,"op":"Read","value":26},{"major":252,"minor":1,"op":"Write"},{"major":252,"minor":1,"op":"Sync"},{"major":252,"minor":1,"op":"Async","value":26},{"major":252,"minor":1,"op":"Total","value":26}]},"cpu_stats":{"cpu_usage":{"percpu_usage":[17871,4959158,1646137,1231652,11829401,244656,369972,0],"usage_in_usermode":10000000,"total_usage":20298847},"system_cpu_usage":24052607520000000,"throttling_data":{}},"precpu_stats":{"cpu_usage":{"percpu_usage":[17871,4959158,1646137,1231652,11829401,244656,369972,0],"usage_in_usermode":10000000,"total_usage":20298847},"system_cpu_usage":24052599550000000,"throttling_data":{}}}`
|
||||
stat = ioutil.NopCloser(strings.NewReader(jsonStat))
|
||||
return stat, nil
|
||||
}
|
||||
|
||||
func TestDockerGatherInfo(t *testing.T) {
|
||||
var acc testutil.Accumulator
|
||||
client := FakeDockerClient{}
|
||||
d := Docker{client: client}
|
||||
|
||||
err := d.Gather(&acc)
|
||||
|
||||
require.NoError(t, err)
|
||||
|
||||
acc.AssertContainsTaggedFields(t,
|
||||
"docker",
|
||||
map[string]interface{}{
|
||||
"n_listener_events": int(0),
|
||||
"n_cpus": int(4),
|
||||
"n_used_file_descriptors": int(19),
|
||||
"n_containers": int(108),
|
||||
"n_images": int(199),
|
||||
"n_goroutines": int(39),
|
||||
},
|
||||
map[string]string{},
|
||||
)
|
||||
|
||||
acc.AssertContainsTaggedFields(t,
|
||||
"docker_data",
|
||||
map[string]interface{}{
|
||||
"used": int64(17300000000),
|
||||
"total": int64(107400000000),
|
||||
"available": int64(36530000000),
|
||||
},
|
||||
map[string]string{
|
||||
"unit": "bytes",
|
||||
},
|
||||
)
|
||||
acc.AssertContainsTaggedFields(t,
|
||||
"docker_container_cpu",
|
||||
map[string]interface{}{
|
||||
"usage_total": uint64(1231652),
|
||||
"container_id": "b7dfbb9478a6ae55e237d4d74f8bbb753f0817192b5081334dc78476296e2173",
|
||||
},
|
||||
map[string]string{
|
||||
"container_name": "etcd2",
|
||||
"container_image": "quay.io/coreos/etcd:v2.2.2",
|
||||
"cpu": "cpu3",
|
||||
},
|
||||
)
|
||||
acc.AssertContainsTaggedFields(t,
|
||||
"docker_container_mem",
|
||||
map[string]interface{}{
|
||||
"total_pgpgout": uint64(0),
|
||||
"usage_percent": float64(0),
|
||||
"rss": uint64(0),
|
||||
"total_writeback": uint64(0),
|
||||
"active_anon": uint64(0),
|
||||
"total_pgmafault": uint64(0),
|
||||
"total_rss": uint64(0),
|
||||
"total_unevictable": uint64(0),
|
||||
"active_file": uint64(0),
|
||||
"total_mapped_file": uint64(0),
|
||||
"pgpgin": uint64(0),
|
||||
"total_active_file": uint64(0),
|
||||
"total_active_anon": uint64(0),
|
||||
"total_cache": uint64(0),
|
||||
"inactive_anon": uint64(0),
|
||||
"pgmajfault": uint64(0),
|
||||
"total_inactive_anon": uint64(0),
|
||||
"total_rss_huge": uint64(0),
|
||||
"rss_huge": uint64(0),
|
||||
"hierarchical_memory_limit": uint64(0),
|
||||
"pgpgout": uint64(0),
|
||||
"unevictable": uint64(0),
|
||||
"total_inactive_file": uint64(0),
|
||||
"writeback": uint64(0),
|
||||
"total_pgfault": uint64(0),
|
||||
"total_pgpgin": uint64(0),
|
||||
"cache": uint64(0),
|
||||
"mapped_file": uint64(0),
|
||||
"inactive_file": uint64(0),
|
||||
"max_usage": uint64(0),
|
||||
"fail_count": uint64(0),
|
||||
"pgfault": uint64(0),
|
||||
"usage": uint64(0),
|
||||
"limit": uint64(18935443456),
|
||||
"container_id": "b7dfbb9478a6ae55e237d4d74f8bbb753f0817192b5081334dc78476296e2173",
|
||||
},
|
||||
map[string]string{
|
||||
"container_name": "etcd2",
|
||||
"container_image": "quay.io/coreos/etcd:v2.2.2",
|
||||
},
|
||||
)
|
||||
|
||||
//fmt.Print(info)
|
||||
}
|
||||
|
||||
@@ -10,25 +10,20 @@ domains. You can read Dovecot's documentation
|
||||
```
|
||||
# Read metrics about dovecot servers
|
||||
[[inputs.dovecot]]
|
||||
## specify dovecot servers via an address:port list
|
||||
## e.g.
|
||||
## localhost:24242
|
||||
##
|
||||
## If no servers are specified, then localhost is used as the host.
|
||||
# Dovecot servers
|
||||
# specify dovecot servers via an address:port list
|
||||
# e.g.
|
||||
# localhost:24242
|
||||
#
|
||||
# If no servers are specified, then localhost is used as the host.
|
||||
servers = ["localhost:24242"]
|
||||
## Type is one of "user", "domain", "ip", or "global"
|
||||
type = "global"
|
||||
## Wildcard matches like "*.com". An empty string "" is same as "*"
|
||||
## If type = "ip" filters should be <IP/network>
|
||||
filters = [""]
|
||||
# Only collect metrics for these domains, collect all if empty
|
||||
domains = []
|
||||
```
|
||||
|
||||
|
||||
### Tags:
|
||||
server: hostname
|
||||
type: query type
|
||||
ip: ip addr
|
||||
user: username
|
||||
domain: domain name
|
||||
|
||||
|
||||
@@ -38,7 +33,7 @@ domains. You can read Dovecot's documentation
|
||||
last_update time.Time
|
||||
num_logins int64
|
||||
num_cmds int64
|
||||
num_connected_sessions int64 ## not in <user> type
|
||||
num_connected_sessions int64
|
||||
user_cpu float32
|
||||
sys_cpu float32
|
||||
clock_time float64
|
||||
@@ -62,13 +57,11 @@ domains. You can read Dovecot's documentation
|
||||
### Example Output:
|
||||
|
||||
```
|
||||
telegraf -config t.cfg -input-filter dovecot -test
|
||||
telegraf -config telegraf.cfg -input-filter dovecot -test
|
||||
* Plugin: dovecot, Collection 1
|
||||
> dovecot,ip=192.168.0.1,server=dovecot-1.domain.test,type=ip clock_time=0,disk_input=0i,disk_output=0i,invol_cs=0i,last_update="2016-04-08 10:59:47.000208479 +0200 CEST",mail_cache_hits=0i,mail_lookup_attr=0i,mail_lookup_path=0i,mail_read_bytes=0i,mail_read_count=0i,maj_faults=0i,min_faults=0i,num_cmds=12i,num_connected_sessions=0i,num_logins=6i,read_bytes=0i,read_count=0i,reset_timestamp="2016-04-08 10:33:34 +0200 CEST",sys_cpu=0,user_cpu=0,vol_cs=0i,write_bytes=0i,write_count=0i 1460106251633824223
|
||||
* Plugin: dovecot, Collection 1
|
||||
> dovecot,server=dovecot-1.domain.test,type=user,user=user-1@domain.test clock_time=0.00006,disk_input=405504i,disk_output=77824i,invol_cs=67i,last_update="2016-04-08 11:02:55.000111634 +0200 CEST",mail_cache_hits=26i,mail_lookup_attr=0i,mail_lookup_path=6i,mail_read_bytes=86233i,mail_read_count=5i,maj_faults=0i,min_faults=975i,num_cmds=41i,num_logins=3i,read_bytes=368833i,read_count=394i,reset_timestamp="2016-04-08 11:01:32 +0200 CEST",sys_cpu=0.008,user_cpu=0.004,vol_cs=323i,write_bytes=105086i,write_count=176i 1460106256637049167
|
||||
* Plugin: dovecot, Collection 1
|
||||
> dovecot,domain=domain.test,server=dovecot-1.domain.test,type=domain clock_time=100896189179847.7,disk_input=6467588263936i,disk_output=17933680439296i,invol_cs=1194808498i,last_update="2016-04-08 11:04:08.000377367 +0200 CEST",mail_cache_hits=46455781i,mail_lookup_attr=0i,mail_lookup_path=571490i,mail_read_bytes=79287033067i,mail_read_count=491243i,maj_faults=16992i,min_faults=1278442541i,num_cmds=606005i,num_connected_sessions=6597i,num_logins=166381i,read_bytes=30231409780721i,read_count=1624912080i,reset_timestamp="2016-04-08 10:28:45 +0200 CEST",sys_cpu=156440.372,user_cpu=216676.476,vol_cs=2749291157i,write_bytes=17097106707594i,write_count=944448998i 1460106261639672622
|
||||
* Plugin: dovecot, Collection 1
|
||||
> dovecot,server=dovecot-1.domain.test,type=global clock_time=101196971074203.94,disk_input=6493168218112i,disk_output=17978638815232i,invol_cs=1198855447i,last_update="2016-04-08 11:04:13.000379245 +0200 CEST",mail_cache_hits=68192209i,mail_lookup_attr=0i,mail_lookup_path=653861i,mail_read_bytes=86705151847i,mail_read_count=566125i,maj_faults=17208i,min_faults=1286179702i,num_cmds=917469i,num_connected_sessions=8896i,num_logins=174827i,read_bytes=30327690466186i,read_count=1772396430i,reset_timestamp="2016-04-08 10:28:45 +0200 CEST",sys_cpu=157965.692,user_cpu=219337.48,vol_cs=2827615787i,write_bytes=17150837661940i,write_count=992653220i 1460106266642153907
|
||||
```
|
||||
> dovecot,domain=xxxxx.it,server=dovecot--1.mail.sys clock_time=12105746411632.5,disk_input=115285225472i,disk_output=4885067755520i,invol_cs=169701886i,last_update="2016-02-09 08:49:47.000014113 +0100 CET",mail_cache_hits=441828i,mail_lookup_attr=0i,mail_lookup_path=25323i,mail_read_bytes=241188145i,mail_read_count=11719i,maj_faults=3168i,min_faults=321438988i,num_cmds=51635i,num_connected_sessions=2i,num_logins=17149i,read_bytes=7939026951110i,read_count=3716991752i,reset_timestamp="2016-01-28 09:34:36 +0100 CET",sys_cpu=222595.288,user_cpu=267468.08,vol_cs=3288715920i,write_bytes=4483648967059i,write_count=1640646952i 1455004219924838345
|
||||
> dovecot,domain=yyyyy.com,server=dovecot-1.mail.sys clock_time=6650794455331782,disk_input=61957695569920i,disk_output=2638244004487168i,invol_cs=2004805041i,last_update="2016-02-09 08:49:49.000251296 +0100 CET",mail_cache_hits=2499112513i,mail_lookup_attr=506730i,mail_lookup_path=39128227i,mail_read_bytes=1076496874501i,mail_read_count=32615262i,maj_faults=1643304i,min_faults=4216116325i,num_cmds=85785559i,num_connected_sessions=1177i,num_logins=11658255i,read_bytes=4289150974554145i,read_count=1112000703i,reset_timestamp="2016-01-28 09:31:26 +0100 CET",sys_cpu=121125923.032,user_cpu=145561336.428,vol_cs=205451885i,write_bytes=2420130526835796i,write_count=2991367252i 1455004219925152529
|
||||
> dovecot,domain=xxxxx.it,server=dovecot-2.mail.sys clock_time=10710826586999.143,disk_input=79792410624i,disk_output=4496066158592i,invol_cs=150426876i,last_update="2016-02-09 08:48:19.000209134 +0100 CET",mail_cache_hits=5480869i,mail_lookup_attr=0i,mail_lookup_path=122563i,mail_read_bytes=340746273i,mail_read_count=44275i,maj_faults=1722i,min_faults=288071875i,num_cmds=50098i,num_connected_sessions=0i,num_logins=16389i,read_bytes=7259551999517i,read_count=3396625369i,reset_timestamp="2016-01-28 09:31:29 +0100 CET",sys_cpu=200762.792,user_cpu=242477.664,vol_cs=2996657358i,write_bytes=4133381575263i,write_count=1497242759i 1455004219924888283
|
||||
> dovecot,domain=yyyyy.com,server=dovecot-2.mail.sys clock_time=6522131245483702,disk_input=48259150004224i,disk_output=2754333359087616i,invol_cs=2294595260i,last_update="2016-02-09 08:49:49.000251919 +0100 CET",mail_cache_hits=2139113611i,mail_lookup_attr=520276i,mail_lookup_path=37940318i,mail_read_bytes=1088002215022i,mail_read_count=31350271i,maj_faults=994420i,min_faults=1486260543i,num_cmds=40414997i,num_connected_sessions=978i,num_logins=11259672i,read_bytes=4445546612487315i,read_count=1763534543i,reset_timestamp="2016-01-28 09:31:24 +0100 CET",sys_cpu=123655962.668,user_cpu=149259327.032,vol_cs=4215130546i,write_bytes=2531186030222761i,write_count=2186579650i 1455004219925398372
|
||||
```
|
||||
|
||||
|
||||
@@ -4,7 +4,6 @@ import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"io"
|
||||
// "log"
|
||||
"net"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -16,9 +15,8 @@ import (
|
||||
)
|
||||
|
||||
type Dovecot struct {
|
||||
Type string
|
||||
Filters []string
|
||||
Servers []string
|
||||
Domains []string
|
||||
}
|
||||
|
||||
func (d *Dovecot) Description() string {
|
||||
@@ -32,19 +30,10 @@ var sampleConfig = `
|
||||
##
|
||||
## If no servers are specified, then localhost is used as the host.
|
||||
servers = ["localhost:24242"]
|
||||
## Type is one of "user", "domain", "ip", or "global"
|
||||
type = "global"
|
||||
## Wildcard matches like "*.com". An empty string "" is same as "*"
|
||||
## If type = "ip" filters should be <IP/network>
|
||||
filters = [""]
|
||||
## Only collect metrics for these domains, collect all if empty
|
||||
domains = []
|
||||
`
|
||||
|
||||
var defaultTimeout = time.Second * time.Duration(5)
|
||||
|
||||
var validQuery = map[string]bool{
|
||||
"user": true, "domain": true, "global": true, "ip": true,
|
||||
}
|
||||
|
||||
func (d *Dovecot) SampleConfig() string { return sampleConfig }
|
||||
|
||||
const defaultPort = "24242"
|
||||
@@ -52,11 +41,6 @@ const defaultPort = "24242"
|
||||
// Reads stats from all configured servers.
|
||||
func (d *Dovecot) Gather(acc telegraf.Accumulator) error {
|
||||
|
||||
if !validQuery[d.Type] {
|
||||
return fmt.Errorf("Error: %s is not a valid query type\n",
|
||||
d.Type)
|
||||
}
|
||||
|
||||
if len(d.Servers) == 0 {
|
||||
d.Servers = append(d.Servers, "127.0.0.1:24242")
|
||||
}
|
||||
@@ -65,18 +49,18 @@ func (d *Dovecot) Gather(acc telegraf.Accumulator) error {
|
||||
|
||||
var outerr error
|
||||
|
||||
if len(d.Filters) <= 0 {
|
||||
d.Filters = append(d.Filters, "")
|
||||
var domains = make(map[string]bool)
|
||||
|
||||
for _, dom := range d.Domains {
|
||||
domains[dom] = true
|
||||
}
|
||||
|
||||
for _, serv := range d.Servers {
|
||||
for _, filter := range d.Filters {
|
||||
wg.Add(1)
|
||||
go func(serv string, filter string) {
|
||||
defer wg.Done()
|
||||
outerr = d.gatherServer(serv, acc, d.Type, filter)
|
||||
}(serv, filter)
|
||||
}
|
||||
wg.Add(1)
|
||||
go func(serv string) {
|
||||
defer wg.Done()
|
||||
outerr = d.gatherServer(serv, acc, domains)
|
||||
}(serv)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
@@ -84,38 +68,29 @@ func (d *Dovecot) Gather(acc telegraf.Accumulator) error {
|
||||
return outerr
|
||||
}
|
||||
|
||||
func (d *Dovecot) gatherServer(addr string, acc telegraf.Accumulator, qtype string, filter string) error {
|
||||
|
||||
func (d *Dovecot) gatherServer(addr string, acc telegraf.Accumulator, doms map[string]bool) error {
|
||||
_, _, err := net.SplitHostPort(addr)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error: %s on url %s\n", err, addr)
|
||||
}
|
||||
|
||||
c, err := net.DialTimeout("tcp", addr, defaultTimeout)
|
||||
c, err := net.Dial("tcp", addr)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Unable to connect to dovecot server '%s': %s", addr, err)
|
||||
}
|
||||
defer c.Close()
|
||||
|
||||
// Extend connection
|
||||
c.SetDeadline(time.Now().Add(defaultTimeout))
|
||||
|
||||
msg := fmt.Sprintf("EXPORT\t%s", qtype)
|
||||
if len(filter) > 0 {
|
||||
msg += fmt.Sprintf("\t%s=%s", qtype, filter)
|
||||
}
|
||||
msg += "\n"
|
||||
|
||||
c.Write([]byte(msg))
|
||||
c.Write([]byte("EXPORT\tdomain\n\n"))
|
||||
var buf bytes.Buffer
|
||||
io.Copy(&buf, c)
|
||||
// buf := bufio.NewReader(c)
|
||||
|
||||
host, _, _ := net.SplitHostPort(addr)
|
||||
|
||||
return gatherStats(&buf, acc, host, qtype)
|
||||
return gatherStats(&buf, acc, doms, host)
|
||||
}
|
||||
|
||||
func gatherStats(buf *bytes.Buffer, acc telegraf.Accumulator, host string, qtype string) error {
|
||||
func gatherStats(buf *bytes.Buffer, acc telegraf.Accumulator, doms map[string]bool, host string) error {
|
||||
|
||||
lines := strings.Split(buf.String(), "\n")
|
||||
head := strings.Split(lines[0], "\t")
|
||||
@@ -126,18 +101,16 @@ func gatherStats(buf *bytes.Buffer, acc telegraf.Accumulator, host string, qtype
|
||||
continue
|
||||
}
|
||||
val := strings.Split(vals[i], "\t")
|
||||
|
||||
fields := make(map[string]interface{})
|
||||
tags := map[string]string{"server": host, "type": qtype}
|
||||
|
||||
if qtype != "global" {
|
||||
tags[qtype] = val[0]
|
||||
if len(doms) > 0 && !doms[val[0]] {
|
||||
continue
|
||||
}
|
||||
|
||||
tags := map[string]string{"server": host, "domain": val[0]}
|
||||
for n := range val {
|
||||
switch head[n] {
|
||||
case qtype:
|
||||
case "domain":
|
||||
continue
|
||||
// fields[head[n]] = val[n]
|
||||
case "user_cpu", "sys_cpu", "clock_time":
|
||||
fields[head[n]] = secParser(val[n])
|
||||
case "reset_timestamp", "last_update":
|
||||
|
||||
@@ -15,6 +15,17 @@ func TestDovecot(t *testing.T) {
|
||||
t.Skip("Skipping integration test in short mode")
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
tags := map[string]string{"server": "dovecot.test", "domain": "domain.test"}
|
||||
buf := bytes.NewBufferString(sampleStats)
|
||||
|
||||
var doms = map[string]bool{
|
||||
"domain.test": true,
|
||||
}
|
||||
|
||||
err := gatherStats(buf, &acc, doms, "dovecot.test")
|
||||
require.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"reset_timestamp": time.Unix(1453969886, 0),
|
||||
"last_update": time.Unix(1454603963, 39864),
|
||||
@@ -41,79 +52,10 @@ func TestDovecot(t *testing.T) {
|
||||
"mail_cache_hits": int64(1557255080),
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
|
||||
// Test type=global
|
||||
tags := map[string]string{"server": "dovecot.test", "type": "global"}
|
||||
buf := bytes.NewBufferString(sampleGlobal)
|
||||
|
||||
err := gatherStats(buf, &acc, "dovecot.test", "global")
|
||||
require.NoError(t, err)
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "dovecot", fields, tags)
|
||||
|
||||
// Test type=domain
|
||||
tags = map[string]string{"server": "dovecot.test", "type": "domain", "domain": "domain.test"}
|
||||
buf = bytes.NewBufferString(sampleDomain)
|
||||
|
||||
err = gatherStats(buf, &acc, "dovecot.test", "domain")
|
||||
require.NoError(t, err)
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "dovecot", fields, tags)
|
||||
|
||||
// Test type=ip
|
||||
tags = map[string]string{"server": "dovecot.test", "type": "ip", "ip": "192.168.0.100"}
|
||||
buf = bytes.NewBufferString(sampleIp)
|
||||
|
||||
err = gatherStats(buf, &acc, "dovecot.test", "ip")
|
||||
require.NoError(t, err)
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "dovecot", fields, tags)
|
||||
|
||||
// Test type=user
|
||||
fields = map[string]interface{}{
|
||||
"reset_timestamp": time.Unix(1453969886, 0),
|
||||
"last_update": time.Unix(1454603963, 39864),
|
||||
"num_logins": int64(7503897),
|
||||
"num_cmds": int64(52595715),
|
||||
"user_cpu": 1.00831175372e+08,
|
||||
"sys_cpu": 8.3849071112e+07,
|
||||
"clock_time": 4.3260019315281835e+15,
|
||||
"min_faults": int64(763950011),
|
||||
"maj_faults": int64(1112443),
|
||||
"vol_cs": int64(4120386897),
|
||||
"invol_cs": int64(3685239306),
|
||||
"disk_input": int64(41679480946688),
|
||||
"disk_output": int64(1819070669176832),
|
||||
"read_count": int64(2368906465),
|
||||
"read_bytes": int64(2957928122981169),
|
||||
"write_count": int64(3545389615),
|
||||
"write_bytes": int64(1666822498251286),
|
||||
"mail_lookup_path": int64(24396105),
|
||||
"mail_lookup_attr": int64(302845),
|
||||
"mail_read_count": int64(20155768),
|
||||
"mail_read_bytes": int64(669946617705),
|
||||
"mail_cache_hits": int64(1557255080),
|
||||
}
|
||||
|
||||
tags = map[string]string{"server": "dovecot.test", "type": "user", "user": "user.1@domain.test"}
|
||||
buf = bytes.NewBufferString(sampleUser)
|
||||
|
||||
err = gatherStats(buf, &acc, "dovecot.test", "user")
|
||||
require.NoError(t, err)
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "dovecot", fields, tags)
|
||||
|
||||
}
|
||||
|
||||
const sampleGlobal = `reset_timestamp last_update num_logins num_cmds num_connected_sessions user_cpu sys_cpu clock_time min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits
|
||||
1453969886 1454603963.039864 7503897 52595715 1204 100831175.372000 83849071.112000 4326001931528183.495762 763950011 1112443 4120386897 3685239306 41679480946688 1819070669176832 2368906465 2957928122981169 3545389615 1666822498251286 24396105 302845 20155768 669946617705 1557255080`
|
||||
|
||||
const sampleDomain = `domain reset_timestamp last_update num_logins num_cmds num_connected_sessions user_cpu sys_cpu clock_time min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits
|
||||
const sampleStats = `domain reset_timestamp last_update num_logins num_cmds num_connected_sessions user_cpu sys_cpu clock_time min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits
|
||||
domain.bad 1453970076 1454603947.383029 10749 33828 0 177988.524000 148071.772000 7531838964717.193706 212491179 2125 2190386067 112779200 74487934976 3221808119808 2469948401 5237602841760 1091171292 2951966459802 15363 0 2922 136403379 334372
|
||||
domain.test 1453969886 1454603963.039864 7503897 52595715 1204 100831175.372000 83849071.112000 4326001931528183.495762 763950011 1112443 4120386897 3685239306 41679480946688 1819070669176832 2368906465 2957928122981169 3545389615 1666822498251286 24396105 302845 20155768 669946617705 1557255080`
|
||||
|
||||
const sampleIp = `ip reset_timestamp last_update num_logins num_cmds num_connected_sessions user_cpu sys_cpu clock_time min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits
|
||||
192.168.0.100 1453969886 1454603963.039864 7503897 52595715 1204 100831175.372000 83849071.112000 4326001931528183.495762 763950011 1112443 4120386897 3685239306 41679480946688 1819070669176832 2368906465 2957928122981169 3545389615 1666822498251286 24396105 302845 20155768 669946617705 1557255080`
|
||||
|
||||
const sampleUser = `user reset_timestamp last_update num_logins num_cmds user_cpu sys_cpu clock_time min_faults maj_faults vol_cs invol_cs disk_input disk_output read_count read_bytes write_count write_bytes mail_lookup_path mail_lookup_attr mail_read_count mail_read_bytes mail_cache_hits
|
||||
user.1@domain.test 1453969886 1454603963.039864 7503897 52595715 100831175.372000 83849071.112000 4326001931528183.495762 763950011 1112443 4120386897 3685239306 41679480946688 1819070669176832 2368906465 2957928122981169 3545389615 1666822498251286 24396105 302845 20155768 669946617705 1557255080`
|
||||
|
||||
@@ -81,12 +81,7 @@ type Elasticsearch struct {
|
||||
|
||||
// NewElasticsearch return a new instance of Elasticsearch
|
||||
func NewElasticsearch() *Elasticsearch {
|
||||
tr := &http.Transport{ResponseHeaderTimeout: time.Duration(3 * time.Second)}
|
||||
client := &http.Client{
|
||||
Transport: tr,
|
||||
Timeout: time.Duration(4 * time.Second),
|
||||
}
|
||||
return &Elasticsearch{client: client}
|
||||
return &Elasticsearch{client: http.DefaultClient}
|
||||
}
|
||||
|
||||
// SampleConfig returns sample configuration for this plugin.
|
||||
|
||||
@@ -34,9 +34,6 @@ func (t *transportMock) RoundTrip(r *http.Request) (*http.Response, error) {
|
||||
return res, nil
|
||||
}
|
||||
|
||||
func (t *transportMock) CancelRequest(_ *http.Request) {
|
||||
}
|
||||
|
||||
func TestElasticsearch(t *testing.T) {
|
||||
es := NewElasticsearch()
|
||||
es.Servers = []string{"http://example.com:9200"}
|
||||
|
||||
@@ -1,13 +1,28 @@
|
||||
# Exec Input Plugin
|
||||
|
||||
Please also see: [Telegraf Input Data Formats](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md)
|
||||
The exec plugin can execute arbitrary commands which output:
|
||||
|
||||
### Example 1 - JSON
|
||||
* JSON
|
||||
* InfluxDB [line-protocol](https://docs.influxdata.com/influxdb/v0.9/write_protocols/line/)
|
||||
* Graphite [graphite-protocol](http://graphite.readthedocs.org/en/latest/feeding-carbon.html)
|
||||
|
||||
#### Configuration
|
||||
> Graphite understands messages with this format:
|
||||
|
||||
In this example a script called ```/tmp/test.sh``` and a script called ```/tmp/test2.sh```
|
||||
are configured for ```[[inputs.exec]]``` in JSON format.
|
||||
> ```
|
||||
metric_path value timestamp\n
|
||||
```
|
||||
|
||||
> __metric_path__ is the metric namespace that you want to populate.
|
||||
|
||||
> __value__ is the value that you want to assign to the metric at this time.
|
||||
|
||||
> __timestamp__ is the unix epoch time.
|
||||
|
||||
|
||||
If using JSON, only numeric values are parsed and turned into floats. Booleans
|
||||
and strings will be ignored.
|
||||
|
||||
### Configuration
|
||||
|
||||
```
|
||||
# Read flattened metrics from one or more commands that output JSON to stdout
|
||||
@@ -15,7 +30,7 @@ are configured for ```[[inputs.exec]]``` in JSON format.
|
||||
# Shell/commands array
|
||||
commands = ["/tmp/test.sh", "/tmp/test2.sh"]
|
||||
|
||||
# Data format to consume.
|
||||
# Data format to consume. This can be "json", "influx" or "graphite" (line-protocol)
|
||||
# NOTE json only reads numerical measurements, strings and booleans are ignored.
|
||||
data_format = "json"
|
||||
|
||||
@@ -49,6 +64,8 @@ Other options for modifying the measurement names are:
|
||||
name_prefix = "prefix_"
|
||||
```
|
||||
|
||||
### Example 1
|
||||
|
||||
Let's say that we have the above configuration, and mycollector outputs the
|
||||
following JSON:
|
||||
|
||||
@@ -68,16 +85,10 @@ The collected metrics will be stored as fields under the measurement
|
||||
```
|
||||
exec_mycollector a=0.5,b_c=0.1,b_d=5 1452815002357578567
|
||||
```
|
||||
If using JSON, only numeric values are parsed and turned into floats. Booleans
|
||||
and strings will be ignored.
|
||||
|
||||
### Example 2 - Influx Line-Protocol
|
||||
### Example 2
|
||||
|
||||
In this example an application called ```/usr/bin/line_protocol_collector```
|
||||
and a script called ```/tmp/test2.sh``` are configured for ```[[inputs.exec]]```
|
||||
in influx line-protocol format.
|
||||
|
||||
#### Configuration
|
||||
Now let's say we have the following configuration:
|
||||
|
||||
```
|
||||
[[inputs.exec]]
|
||||
@@ -87,12 +98,12 @@ in influx line-protocol format.
|
||||
# command = "/usr/bin/line_protocol_collector"
|
||||
commands = ["/usr/bin/line_protocol_collector","/tmp/test2.sh"]
|
||||
|
||||
# Data format to consume.
|
||||
# Data format to consume. This can be "json" or "influx" (line-protocol)
|
||||
# NOTE json only reads numerical measurements, strings and booleans are ignored.
|
||||
data_format = "influx"
|
||||
```
|
||||
|
||||
The line_protocol_collector application outputs the following line protocol:
|
||||
And line_protocol_collector outputs the following line protocol:
|
||||
|
||||
```
|
||||
cpu,cpu=cpu0,host=foo,datacenter=us-east usage_idle=99,usage_busy=1
|
||||
@@ -107,17 +118,14 @@ cpu,cpu=cpu6,host=foo,datacenter=us-east usage_idle=99,usage_busy=1
|
||||
You will get data in InfluxDB exactly as it is defined above,
|
||||
tags are cpu=cpuN, host=foo, and datacenter=us-east with fields usage_idle
|
||||
and usage_busy. They will receive a timestamp at collection time.
|
||||
Each line must end in \n, just as the Influx line protocol does.
|
||||
|
||||
|
||||
### Example 3 - Graphite
|
||||
### Example 3
|
||||
|
||||
We can also change the data_format to "graphite" to use the metrics collecting scripts such as (compatible with graphite):
|
||||
|
||||
* Nagios [Metrics Plugins](https://exchange.nagios.org/directory/Plugins)
|
||||
* Sensu [Metrics Plugins](https://github.com/sensu-plugins)
|
||||
|
||||
In this example a script called /tmp/test.sh and a script called /tmp/test2.sh are configured for [[inputs.exec]] in graphite format.
|
||||
* Nagios [Mertics Plugins] (https://exchange.nagios.org/directory/Plugins)
|
||||
* Sensu [Mertics Plugins] (https://github.com/sensu-plugins)
|
||||
|
||||
#### Configuration
|
||||
```
|
||||
@@ -126,7 +134,7 @@ In this example a script called /tmp/test.sh and a script called /tmp/test2.sh a
|
||||
# Shell/commands array
|
||||
commands = ["/tmp/test.sh","/tmp/test2.sh"]
|
||||
|
||||
# Data format to consume.
|
||||
# Data format to consume. This can be "json", "influx" or "graphite" (line-protocol)
|
||||
# NOTE json only reads numerical measurements, strings and booleans are ignored.
|
||||
data_format = "graphite"
|
||||
|
||||
@@ -153,17 +161,6 @@ In this example a script called /tmp/test.sh and a script called /tmp/test2.sh a
|
||||
"measurement*"
|
||||
]
|
||||
```
|
||||
Graphite messages are in this format:
|
||||
|
||||
```
|
||||
metric_path value timestamp\n
|
||||
```
|
||||
|
||||
__metric_path__ is the metric namespace that you want to populate.
|
||||
|
||||
__value__ is the value that you want to assign to the metric at this time.
|
||||
|
||||
__timestamp__ is the unix epoch time.
|
||||
|
||||
And test.sh/test2.sh will output:
|
||||
|
||||
@@ -179,5 +176,5 @@ sensu.metric.net.server0.eth0.rx_dropped 0 1444234982
|
||||
|
||||
The templates configuration will be used to parse the graphite metrics to support influxdb/opentsdb tagging store engines.
|
||||
|
||||
More detail information about templates, please refer to [The graphite Input](https://github.com/influxdata/influxdb/blob/master/services/graphite/README.md)
|
||||
More detail information about templates, please refer to [The graphite Input] (https://github.com/influxdata/influxdb/blob/master/services/graphite/README.md)
|
||||
|
||||
|
||||
@@ -5,29 +5,22 @@ import (
|
||||
"fmt"
|
||||
"os/exec"
|
||||
"sync"
|
||||
"syscall"
|
||||
"time"
|
||||
|
||||
"github.com/gonuts/go-shellquote"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"github.com/influxdata/telegraf/plugins/parsers"
|
||||
"github.com/influxdata/telegraf/plugins/parsers/nagios"
|
||||
)
|
||||
|
||||
const sampleConfig = `
|
||||
## Commands array
|
||||
commands = ["/tmp/test.sh", "/usr/bin/mycollector --foo=bar"]
|
||||
|
||||
## Timeout for each command to complete.
|
||||
timeout = "5s"
|
||||
|
||||
## measurement name suffix (for separating different commands)
|
||||
name_suffix = "_mycollector"
|
||||
|
||||
## Data format to consume.
|
||||
## Data format to consume. This can be "json", "influx" or "graphite"
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
|
||||
@@ -37,7 +30,6 @@ const sampleConfig = `
|
||||
type Exec struct {
|
||||
Commands []string
|
||||
Command string
|
||||
Timeout internal.Duration
|
||||
|
||||
parser parsers.Parser
|
||||
|
||||
@@ -49,42 +41,17 @@ type Exec struct {
|
||||
|
||||
func NewExec() *Exec {
|
||||
return &Exec{
|
||||
runner: CommandRunner{},
|
||||
Timeout: internal.Duration{Duration: time.Second * 5},
|
||||
runner: CommandRunner{},
|
||||
}
|
||||
}
|
||||
|
||||
type Runner interface {
|
||||
Run(*Exec, string, telegraf.Accumulator) ([]byte, error)
|
||||
Run(*Exec, string) ([]byte, error)
|
||||
}
|
||||
|
||||
type CommandRunner struct{}
|
||||
|
||||
func AddNagiosState(exitCode error, acc telegraf.Accumulator) error {
|
||||
nagiosState := 0
|
||||
if exitCode != nil {
|
||||
exiterr, ok := exitCode.(*exec.ExitError)
|
||||
if ok {
|
||||
status, ok := exiterr.Sys().(syscall.WaitStatus)
|
||||
if ok {
|
||||
nagiosState = status.ExitStatus()
|
||||
} else {
|
||||
return fmt.Errorf("exec: unable to get nagios plugin exit code")
|
||||
}
|
||||
} else {
|
||||
return fmt.Errorf("exec: unable to get nagios plugin exit code")
|
||||
}
|
||||
}
|
||||
fields := map[string]interface{}{"state": nagiosState}
|
||||
acc.AddFields("nagios_state", fields, nil)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (c CommandRunner) Run(
|
||||
e *Exec,
|
||||
command string,
|
||||
acc telegraf.Accumulator,
|
||||
) ([]byte, error) {
|
||||
func (c CommandRunner) Run(e *Exec, command string) ([]byte, error) {
|
||||
split_cmd, err := shellquote.Split(command)
|
||||
if err != nil || len(split_cmd) == 0 {
|
||||
return nil, fmt.Errorf("exec: unable to parse command, %s", err)
|
||||
@@ -95,18 +62,8 @@ func (c CommandRunner) Run(
|
||||
var out bytes.Buffer
|
||||
cmd.Stdout = &out
|
||||
|
||||
if err := internal.RunTimeout(cmd, e.Timeout.Duration); err != nil {
|
||||
switch e.parser.(type) {
|
||||
case *nagios.NagiosParser:
|
||||
AddNagiosState(err, acc)
|
||||
default:
|
||||
return nil, fmt.Errorf("exec: %s for command '%s'", err, command)
|
||||
}
|
||||
} else {
|
||||
switch e.parser.(type) {
|
||||
case *nagios.NagiosParser:
|
||||
AddNagiosState(nil, acc)
|
||||
}
|
||||
if err := cmd.Run(); err != nil {
|
||||
return nil, fmt.Errorf("exec: %s for command '%s'", err, command)
|
||||
}
|
||||
|
||||
return out.Bytes(), nil
|
||||
@@ -115,7 +72,7 @@ func (c CommandRunner) Run(
|
||||
func (e *Exec) ProcessCommand(command string, acc telegraf.Accumulator) {
|
||||
defer e.wg.Done()
|
||||
|
||||
out, err := e.runner.Run(e, command, acc)
|
||||
out, err := e.runner.Run(e, command)
|
||||
if err != nil {
|
||||
e.errChan <- err
|
||||
return
|
||||
|
||||
@@ -4,7 +4,6 @@ import (
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/parsers"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
@@ -58,7 +57,7 @@ func newRunnerMock(out []byte, err error) Runner {
|
||||
}
|
||||
}
|
||||
|
||||
func (r runnerMock) Run(e *Exec, command string, acc telegraf.Accumulator) ([]byte, error) {
|
||||
func (r runnerMock) Run(e *Exec, command string) ([]byte, error) {
|
||||
if r.err != nil {
|
||||
return nil, r.err
|
||||
}
|
||||
|
||||
@@ -1,37 +0,0 @@
|
||||
# filestat Input Plugin
|
||||
|
||||
The filestat plugin gathers metrics about file existence, size, and other stats.
|
||||
|
||||
### Configuration:
|
||||
|
||||
```toml
|
||||
# Read stats about given file(s)
|
||||
[[inputs.filestat]]
|
||||
## Files to gather stats about.
|
||||
## These accept standard unix glob matching rules, but with the addition of
|
||||
## ** as a "super asterisk". See https://github.com/gobwas/glob.
|
||||
files = ["/etc/telegraf/telegraf.conf", "/var/log/**.log"]
|
||||
## If true, read the entire file and calculate an md5 checksum.
|
||||
md5 = false
|
||||
```
|
||||
|
||||
### Measurements & Fields:
|
||||
|
||||
- filestat
|
||||
- exists (int, 0 | 1)
|
||||
- size_bytes (int, bytes)
|
||||
- md5 (optional, string)
|
||||
|
||||
### Tags:
|
||||
|
||||
- All measurements have the following tags:
|
||||
- file (the path the to file, as specified in the config)
|
||||
|
||||
### Example Output:
|
||||
|
||||
```
|
||||
$ telegraf -config /etc/telegraf/telegraf.conf -input-filter filestat -test
|
||||
* Plugin: filestat, Collection 1
|
||||
> filestat,file=/tmp/foo/bar,host=tyrion exists=0i 1461203374493128216
|
||||
> filestat,file=/Users/sparrc/ws/telegraf.conf,host=tyrion exists=1i,size=47894i 1461203374493199335
|
||||
```
|
||||
@@ -1,125 +0,0 @@
|
||||
package filestat
|
||||
|
||||
import (
|
||||
"crypto/md5"
|
||||
"fmt"
|
||||
"io"
|
||||
"os"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal/globpath"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
const sampleConfig = `
|
||||
## Files to gather stats about.
|
||||
## These accept standard unix glob matching rules, but with the addition of
|
||||
## ** as a "super asterisk". ie:
|
||||
## "/var/log/**.log" -> recursively find all .log files in /var/log
|
||||
## "/var/log/*/*.log" -> find all .log files with a parent dir in /var/log
|
||||
## "/var/log/apache.log" -> just tail the apache log file
|
||||
##
|
||||
## See https://github.com/gobwas/glob for more examples
|
||||
##
|
||||
files = ["/var/log/**.log"]
|
||||
## If true, read the entire file and calculate an md5 checksum.
|
||||
md5 = false
|
||||
`
|
||||
|
||||
type FileStat struct {
|
||||
Md5 bool
|
||||
Files []string
|
||||
|
||||
// maps full file paths to globmatch obj
|
||||
globs map[string]*globpath.GlobPath
|
||||
}
|
||||
|
||||
func NewFileStat() *FileStat {
|
||||
return &FileStat{
|
||||
globs: make(map[string]*globpath.GlobPath),
|
||||
}
|
||||
}
|
||||
|
||||
func (_ *FileStat) Description() string {
|
||||
return "Read stats about given file(s)"
|
||||
}
|
||||
|
||||
func (_ *FileStat) SampleConfig() string { return sampleConfig }
|
||||
|
||||
func (f *FileStat) Gather(acc telegraf.Accumulator) error {
|
||||
var errS string
|
||||
var err error
|
||||
|
||||
for _, filepath := range f.Files {
|
||||
// Get the compiled glob object for this filepath
|
||||
g, ok := f.globs[filepath]
|
||||
if !ok {
|
||||
if g, err = globpath.Compile(filepath); err != nil {
|
||||
errS += err.Error() + " "
|
||||
continue
|
||||
}
|
||||
f.globs[filepath] = g
|
||||
}
|
||||
|
||||
files := g.Match()
|
||||
if len(files) == 0 {
|
||||
acc.AddFields("filestat",
|
||||
map[string]interface{}{
|
||||
"exists": int64(0),
|
||||
},
|
||||
map[string]string{
|
||||
"file": filepath,
|
||||
})
|
||||
continue
|
||||
}
|
||||
|
||||
for fileName, fileInfo := range files {
|
||||
tags := map[string]string{
|
||||
"file": fileName,
|
||||
}
|
||||
fields := map[string]interface{}{
|
||||
"exists": int64(1),
|
||||
"size_bytes": fileInfo.Size(),
|
||||
}
|
||||
|
||||
if f.Md5 {
|
||||
md5, err := getMd5(fileName)
|
||||
if err != nil {
|
||||
errS += err.Error() + " "
|
||||
} else {
|
||||
fields["md5_sum"] = md5
|
||||
}
|
||||
}
|
||||
|
||||
acc.AddFields("filestat", fields, tags)
|
||||
}
|
||||
}
|
||||
|
||||
if errS != "" {
|
||||
return fmt.Errorf(errS)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Read given file and calculate an md5 hash.
|
||||
func getMd5(file string) (string, error) {
|
||||
of, err := os.Open(file)
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
defer of.Close()
|
||||
|
||||
hash := md5.New()
|
||||
_, err = io.Copy(hash, of)
|
||||
if err != nil {
|
||||
// fatal error
|
||||
return "", err
|
||||
}
|
||||
return fmt.Sprintf("%x", hash.Sum(nil)), nil
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("filestat", func() telegraf.Input {
|
||||
return NewFileStat()
|
||||
})
|
||||
}
|
||||
@@ -1,180 +0,0 @@
|
||||
package filestat
|
||||
|
||||
import (
|
||||
"runtime"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestGatherNoMd5(t *testing.T) {
|
||||
dir := getTestdataDir()
|
||||
fs := NewFileStat()
|
||||
fs.Files = []string{
|
||||
dir + "log1.log",
|
||||
dir + "log2.log",
|
||||
"/non/existant/file",
|
||||
}
|
||||
|
||||
acc := testutil.Accumulator{}
|
||||
fs.Gather(&acc)
|
||||
|
||||
tags1 := map[string]string{
|
||||
"file": dir + "log1.log",
|
||||
}
|
||||
fields1 := map[string]interface{}{
|
||||
"size_bytes": int64(0),
|
||||
"exists": int64(1),
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "filestat", fields1, tags1)
|
||||
|
||||
tags2 := map[string]string{
|
||||
"file": dir + "log2.log",
|
||||
}
|
||||
fields2 := map[string]interface{}{
|
||||
"size_bytes": int64(0),
|
||||
"exists": int64(1),
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "filestat", fields2, tags2)
|
||||
|
||||
tags3 := map[string]string{
|
||||
"file": "/non/existant/file",
|
||||
}
|
||||
fields3 := map[string]interface{}{
|
||||
"exists": int64(0),
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "filestat", fields3, tags3)
|
||||
}
|
||||
|
||||
func TestGatherExplicitFiles(t *testing.T) {
|
||||
dir := getTestdataDir()
|
||||
fs := NewFileStat()
|
||||
fs.Md5 = true
|
||||
fs.Files = []string{
|
||||
dir + "log1.log",
|
||||
dir + "log2.log",
|
||||
"/non/existant/file",
|
||||
}
|
||||
|
||||
acc := testutil.Accumulator{}
|
||||
fs.Gather(&acc)
|
||||
|
||||
tags1 := map[string]string{
|
||||
"file": dir + "log1.log",
|
||||
}
|
||||
fields1 := map[string]interface{}{
|
||||
"size_bytes": int64(0),
|
||||
"exists": int64(1),
|
||||
"md5_sum": "d41d8cd98f00b204e9800998ecf8427e",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "filestat", fields1, tags1)
|
||||
|
||||
tags2 := map[string]string{
|
||||
"file": dir + "log2.log",
|
||||
}
|
||||
fields2 := map[string]interface{}{
|
||||
"size_bytes": int64(0),
|
||||
"exists": int64(1),
|
||||
"md5_sum": "d41d8cd98f00b204e9800998ecf8427e",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "filestat", fields2, tags2)
|
||||
|
||||
tags3 := map[string]string{
|
||||
"file": "/non/existant/file",
|
||||
}
|
||||
fields3 := map[string]interface{}{
|
||||
"exists": int64(0),
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "filestat", fields3, tags3)
|
||||
}
|
||||
|
||||
func TestGatherGlob(t *testing.T) {
|
||||
dir := getTestdataDir()
|
||||
fs := NewFileStat()
|
||||
fs.Md5 = true
|
||||
fs.Files = []string{
|
||||
dir + "*.log",
|
||||
}
|
||||
|
||||
acc := testutil.Accumulator{}
|
||||
fs.Gather(&acc)
|
||||
|
||||
tags1 := map[string]string{
|
||||
"file": dir + "log1.log",
|
||||
}
|
||||
fields1 := map[string]interface{}{
|
||||
"size_bytes": int64(0),
|
||||
"exists": int64(1),
|
||||
"md5_sum": "d41d8cd98f00b204e9800998ecf8427e",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "filestat", fields1, tags1)
|
||||
|
||||
tags2 := map[string]string{
|
||||
"file": dir + "log2.log",
|
||||
}
|
||||
fields2 := map[string]interface{}{
|
||||
"size_bytes": int64(0),
|
||||
"exists": int64(1),
|
||||
"md5_sum": "d41d8cd98f00b204e9800998ecf8427e",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "filestat", fields2, tags2)
|
||||
}
|
||||
|
||||
func TestGatherSuperAsterisk(t *testing.T) {
|
||||
dir := getTestdataDir()
|
||||
fs := NewFileStat()
|
||||
fs.Md5 = true
|
||||
fs.Files = []string{
|
||||
dir + "**",
|
||||
}
|
||||
|
||||
acc := testutil.Accumulator{}
|
||||
fs.Gather(&acc)
|
||||
|
||||
tags1 := map[string]string{
|
||||
"file": dir + "log1.log",
|
||||
}
|
||||
fields1 := map[string]interface{}{
|
||||
"size_bytes": int64(0),
|
||||
"exists": int64(1),
|
||||
"md5_sum": "d41d8cd98f00b204e9800998ecf8427e",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "filestat", fields1, tags1)
|
||||
|
||||
tags2 := map[string]string{
|
||||
"file": dir + "log2.log",
|
||||
}
|
||||
fields2 := map[string]interface{}{
|
||||
"size_bytes": int64(0),
|
||||
"exists": int64(1),
|
||||
"md5_sum": "d41d8cd98f00b204e9800998ecf8427e",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "filestat", fields2, tags2)
|
||||
|
||||
tags3 := map[string]string{
|
||||
"file": dir + "test.conf",
|
||||
}
|
||||
fields3 := map[string]interface{}{
|
||||
"size_bytes": int64(104),
|
||||
"exists": int64(1),
|
||||
"md5_sum": "5a7e9b77fa25e7bb411dbd17cf403c1f",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "filestat", fields3, tags3)
|
||||
}
|
||||
|
||||
func TestGetMd5(t *testing.T) {
|
||||
dir := getTestdataDir()
|
||||
md5, err := getMd5(dir + "test.conf")
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, "5a7e9b77fa25e7bb411dbd17cf403c1f", md5)
|
||||
|
||||
md5, err = getMd5("/tmp/foo/bar/fooooo")
|
||||
assert.Error(t, err)
|
||||
}
|
||||
|
||||
func getTestdataDir() string {
|
||||
_, filename, _, _ := runtime.Caller(1)
|
||||
return strings.Replace(filename, "filestat_test.go", "testdata/", 1)
|
||||
}
|
||||
5
plugins/inputs/filestat/testdata/test.conf
vendored
5
plugins/inputs/filestat/testdata/test.conf
vendored
@@ -1,5 +0,0 @@
|
||||
# this is a fake testing config file
|
||||
# for testing the filestat plugin
|
||||
|
||||
option1 = "foo"
|
||||
option2 = "bar"
|
||||
@@ -73,17 +73,14 @@ func (gh *GithubWebhooks) Stop() {
|
||||
|
||||
// Handles the / route
|
||||
func (gh *GithubWebhooks) eventHandler(w http.ResponseWriter, r *http.Request) {
|
||||
defer r.Body.Close()
|
||||
eventType := r.Header["X-Github-Event"][0]
|
||||
data, err := ioutil.ReadAll(r.Body)
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
e, err := NewEvent(data, eventType)
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
gh.Lock()
|
||||
gh.events = append(gh.events, e)
|
||||
|
||||
@@ -6,11 +6,9 @@ import (
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
@@ -49,7 +47,7 @@ const (
|
||||
HF_THROTTLE = 29 //29. throttle [...S]: current throttle percentage for the server, when slowstart is active, or no value if not in slowstart.
|
||||
HF_LBTOT = 30 //30. lbtot [..BS]: total number of times a server was selected, either for new sessions, or when re-dispatching. The server counter is the number of times that server was selected.
|
||||
HF_TRACKED = 31 //31. tracked [...S]: id of proxy/server if tracking is enabled.
|
||||
HF_TYPE = 32 //32. type [LFBS]: (0 = frontend, 1 = backend, 2 = server, 3 = socket/listener)
|
||||
HF_TYPE = 32 //32. type [LFBS]: (0 = frontend, 1 = backend, 2 = server, 3 = socket/listener)
|
||||
HF_RATE = 33 //33. rate [.FBS]: number of sessions per second over last elapsed second
|
||||
HF_RATE_LIM = 34 //34. rate_lim [.F..]: configured limit on new sessions per second
|
||||
HF_RATE_MAX = 35 //35. rate_max [.FBS]: max number of new sessions per second
|
||||
@@ -93,8 +91,8 @@ var sampleConfig = `
|
||||
|
||||
## If no servers are specified, then default to 127.0.0.1:1936
|
||||
servers = ["http://myhaproxy.com:1936", "http://anotherhaproxy.com:1936"]
|
||||
## Or you can also use local socket
|
||||
## servers = ["socket:/run/haproxy/admin.sock"]
|
||||
## Or you can also use local socket(not work yet)
|
||||
## servers = ["socket://run/haproxy/admin.sock"]
|
||||
`
|
||||
|
||||
func (r *haproxy) SampleConfig() string {
|
||||
@@ -129,42 +127,10 @@ func (g *haproxy) Gather(acc telegraf.Accumulator) error {
|
||||
return outerr
|
||||
}
|
||||
|
||||
func (g *haproxy) gatherServerSocket(addr string, acc telegraf.Accumulator) error {
|
||||
var socketPath string
|
||||
socketAddr := strings.Split(addr, ":")
|
||||
|
||||
if len(socketAddr) >= 2 {
|
||||
socketPath = socketAddr[1]
|
||||
} else {
|
||||
socketPath = socketAddr[0]
|
||||
}
|
||||
|
||||
c, err := net.Dial("unix", socketPath)
|
||||
|
||||
if err != nil {
|
||||
return fmt.Errorf("Could not connect to socket '%s': %s", addr, err)
|
||||
}
|
||||
|
||||
_, errw := c.Write([]byte("show stat\n"))
|
||||
|
||||
if errw != nil {
|
||||
return fmt.Errorf("Could not write to socket '%s': %s", addr, errw)
|
||||
}
|
||||
|
||||
return importCsvResult(c, acc, socketPath)
|
||||
}
|
||||
|
||||
func (g *haproxy) gatherServer(addr string, acc telegraf.Accumulator) error {
|
||||
if !strings.HasPrefix(addr, "http") {
|
||||
return g.gatherServerSocket(addr, acc)
|
||||
}
|
||||
|
||||
if g.client == nil {
|
||||
tr := &http.Transport{ResponseHeaderTimeout: time.Duration(3 * time.Second)}
|
||||
client := &http.Client{
|
||||
Transport: tr,
|
||||
Timeout: time.Duration(4 * time.Second),
|
||||
}
|
||||
|
||||
client := &http.Client{}
|
||||
g.client = client
|
||||
}
|
||||
|
||||
|
||||
@@ -1,42 +1,17 @@
|
||||
package haproxy
|
||||
|
||||
import (
|
||||
"crypto/rand"
|
||||
"encoding/binary"
|
||||
"fmt"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
)
|
||||
|
||||
type statServer struct{}
|
||||
|
||||
func (s statServer) serverSocket(l net.Listener) {
|
||||
for {
|
||||
conn, err := l.Accept()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
go func(c net.Conn) {
|
||||
buf := make([]byte, 1024)
|
||||
n, _ := c.Read(buf)
|
||||
|
||||
data := buf[:n]
|
||||
if string(data) == "show stat\n" {
|
||||
c.Write([]byte(csvOutputSample))
|
||||
c.Close()
|
||||
}
|
||||
}(conn)
|
||||
}
|
||||
}
|
||||
|
||||
func TestHaproxyGeneratesMetricsWithAuthentication(t *testing.T) {
|
||||
//We create a fake server to return test data
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
@@ -171,69 +146,6 @@ func TestHaproxyGeneratesMetricsWithoutAuthentication(t *testing.T) {
|
||||
acc.AssertContainsTaggedFields(t, "haproxy", fields, tags)
|
||||
}
|
||||
|
||||
func TestHaproxyGeneratesMetricsUsingSocket(t *testing.T) {
|
||||
var randomNumber int64
|
||||
binary.Read(rand.Reader, binary.LittleEndian, &randomNumber)
|
||||
sock, err := net.Listen("unix", fmt.Sprintf("/tmp/test-haproxy%d.sock", randomNumber))
|
||||
if err != nil {
|
||||
t.Fatal("Cannot initialize socket ")
|
||||
}
|
||||
|
||||
defer sock.Close()
|
||||
|
||||
s := statServer{}
|
||||
go s.serverSocket(sock)
|
||||
|
||||
r := &haproxy{
|
||||
Servers: []string{sock.Addr().String()},
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
|
||||
err = r.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
tags := map[string]string{
|
||||
"proxy": "be_app",
|
||||
"server": sock.Addr().String(),
|
||||
"sv": "host0",
|
||||
}
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"active_servers": uint64(1),
|
||||
"backup_servers": uint64(0),
|
||||
"bin": uint64(510913516),
|
||||
"bout": uint64(2193856571),
|
||||
"check_duration": uint64(10),
|
||||
"cli_abort": uint64(73),
|
||||
"ctime": uint64(2),
|
||||
"downtime": uint64(0),
|
||||
"dresp": uint64(0),
|
||||
"econ": uint64(0),
|
||||
"eresp": uint64(1),
|
||||
"http_response.1xx": uint64(0),
|
||||
"http_response.2xx": uint64(119534),
|
||||
"http_response.3xx": uint64(48051),
|
||||
"http_response.4xx": uint64(2345),
|
||||
"http_response.5xx": uint64(1056),
|
||||
"lbtot": uint64(171013),
|
||||
"qcur": uint64(0),
|
||||
"qmax": uint64(0),
|
||||
"qtime": uint64(0),
|
||||
"rate": uint64(3),
|
||||
"rate_max": uint64(12),
|
||||
"rtime": uint64(312),
|
||||
"scur": uint64(1),
|
||||
"smax": uint64(32),
|
||||
"srv_abort": uint64(1),
|
||||
"stot": uint64(171014),
|
||||
"ttime": uint64(2341),
|
||||
"wredis": uint64(0),
|
||||
"wretr": uint64(1),
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "haproxy", fields, tags)
|
||||
}
|
||||
|
||||
//When not passing server config, we default to localhost
|
||||
//We just want to make sure we did request stat from localhost
|
||||
func TestHaproxyDefaultGetFromLocalhost(t *testing.T) {
|
||||
|
||||
@@ -1,44 +0,0 @@
|
||||
# Example Input Plugin
|
||||
|
||||
This input plugin will test HTTP/HTTPS connections.
|
||||
|
||||
### Configuration:
|
||||
|
||||
```
|
||||
# List of UDP/TCP connections you want to check
|
||||
[[inputs.http_response]]
|
||||
## Server address (default http://localhost)
|
||||
address = "http://github.com"
|
||||
## Set response_timeout (default 5 seconds)
|
||||
response_timeout = 5
|
||||
## HTTP Request Method
|
||||
method = "GET"
|
||||
## HTTP Request Headers
|
||||
[inputs.http_response.headers]
|
||||
Host = github.com
|
||||
## Whether to follow redirects from the server (defaults to false)
|
||||
follow_redirects = true
|
||||
## Optional HTTP Request Body
|
||||
body = '''
|
||||
{'fake':'data'}
|
||||
'''
|
||||
```
|
||||
|
||||
### Measurements & Fields:
|
||||
|
||||
- http_response
|
||||
- response_time (float, seconds)
|
||||
- http_response_code (int) #The code received
|
||||
|
||||
### Tags:
|
||||
|
||||
- All measurements have the following tags:
|
||||
- server
|
||||
- method
|
||||
|
||||
### Example Output:
|
||||
|
||||
```
|
||||
$ ./telegraf -config telegraf.conf -input-filter http_response -test
|
||||
http_response,method=GET,server=http://www.github.com http_response_code=200i,response_time=6.223266528 1459419354977857955
|
||||
```
|
||||
@@ -1,154 +0,0 @@
|
||||
package http_response
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
// HTTPResponse struct
|
||||
type HTTPResponse struct {
|
||||
Address string
|
||||
Body string
|
||||
Method string
|
||||
ResponseTimeout int
|
||||
Headers map[string]string
|
||||
FollowRedirects bool
|
||||
}
|
||||
|
||||
// Description returns the plugin Description
|
||||
func (h *HTTPResponse) Description() string {
|
||||
return "HTTP/HTTPS request given an address a method and a timeout"
|
||||
}
|
||||
|
||||
var sampleConfig = `
|
||||
## Server address (default http://localhost)
|
||||
address = "http://github.com"
|
||||
## Set response_timeout (default 5 seconds)
|
||||
response_timeout = 5
|
||||
## HTTP Request Method
|
||||
method = "GET"
|
||||
## Whether to follow redirects from the server (defaults to false)
|
||||
follow_redirects = true
|
||||
## HTTP Request Headers (all values must be strings)
|
||||
# [inputs.http_response.headers]
|
||||
# Host = "github.com"
|
||||
## Optional HTTP Request Body
|
||||
# body = '''
|
||||
# {'fake':'data'}
|
||||
# '''
|
||||
`
|
||||
|
||||
// SampleConfig returns the plugin SampleConfig
|
||||
func (h *HTTPResponse) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
|
||||
// ErrRedirectAttempted indicates that a redirect occurred
|
||||
var ErrRedirectAttempted = errors.New("redirect")
|
||||
|
||||
// CreateHttpClient creates an http client which will timeout at the specified
|
||||
// timeout period and can follow redirects if specified
|
||||
func CreateHttpClient(followRedirects bool, ResponseTimeout time.Duration) *http.Client {
|
||||
client := &http.Client{
|
||||
Timeout: time.Second * ResponseTimeout,
|
||||
}
|
||||
|
||||
if followRedirects == false {
|
||||
client.CheckRedirect = func(req *http.Request, via []*http.Request) error {
|
||||
return ErrRedirectAttempted
|
||||
}
|
||||
}
|
||||
return client
|
||||
}
|
||||
|
||||
// CreateHeaders takes a map of header strings and puts them
|
||||
// into a http.Header Object
|
||||
func CreateHeaders(headers map[string]string) http.Header {
|
||||
httpHeaders := make(http.Header)
|
||||
for key := range headers {
|
||||
httpHeaders.Add(key, headers[key])
|
||||
}
|
||||
return httpHeaders
|
||||
}
|
||||
|
||||
// HTTPGather gathers all fields and returns any errors it encounters
|
||||
func (h *HTTPResponse) HTTPGather() (map[string]interface{}, error) {
|
||||
// Prepare fields
|
||||
fields := make(map[string]interface{})
|
||||
|
||||
client := CreateHttpClient(h.FollowRedirects, time.Duration(h.ResponseTimeout))
|
||||
|
||||
var body io.Reader
|
||||
if h.Body != "" {
|
||||
body = strings.NewReader(h.Body)
|
||||
}
|
||||
request, err := http.NewRequest(h.Method, h.Address, body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
request.Header = CreateHeaders(h.Headers)
|
||||
|
||||
// Start Timer
|
||||
start := time.Now()
|
||||
resp, err := client.Do(request)
|
||||
if err != nil {
|
||||
if h.FollowRedirects {
|
||||
return nil, err
|
||||
}
|
||||
if urlError, ok := err.(*url.Error); ok &&
|
||||
urlError.Err == ErrRedirectAttempted {
|
||||
err = nil
|
||||
} else {
|
||||
return nil, err
|
||||
}
|
||||
}
|
||||
fields["response_time"] = time.Since(start).Seconds()
|
||||
fields["http_response_code"] = resp.StatusCode
|
||||
return fields, nil
|
||||
}
|
||||
|
||||
// Gather gets all metric fields and tags and returns any errors it encounters
|
||||
func (h *HTTPResponse) Gather(acc telegraf.Accumulator) error {
|
||||
// Set default values
|
||||
if h.ResponseTimeout < 1 {
|
||||
h.ResponseTimeout = 5
|
||||
}
|
||||
// Check send and expected string
|
||||
if h.Method == "" {
|
||||
h.Method = "GET"
|
||||
}
|
||||
if h.Address == "" {
|
||||
h.Address = "http://localhost"
|
||||
}
|
||||
addr, err := url.Parse(h.Address)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if addr.Scheme != "http" && addr.Scheme != "https" {
|
||||
return errors.New("Only http and https are supported")
|
||||
}
|
||||
// Prepare data
|
||||
tags := map[string]string{"server": h.Address, "method": h.Method}
|
||||
var fields map[string]interface{}
|
||||
// Gather data
|
||||
fields, err = h.HTTPGather()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// Add metrics
|
||||
acc.AddFields("http_response", fields, tags)
|
||||
return nil
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("http_response", func() telegraf.Input {
|
||||
return &HTTPResponse{}
|
||||
})
|
||||
}
|
||||
@@ -1,241 +0,0 @@
|
||||
package http_response
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"io/ioutil"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestCreateHeaders(t *testing.T) {
|
||||
fakeHeaders := map[string]string{
|
||||
"Accept": "text/plain",
|
||||
"Content-Type": "application/json",
|
||||
"Cache-Control": "no-cache",
|
||||
}
|
||||
headers := CreateHeaders(fakeHeaders)
|
||||
testHeaders := make(http.Header)
|
||||
testHeaders.Add("Accept", "text/plain")
|
||||
testHeaders.Add("Content-Type", "application/json")
|
||||
testHeaders.Add("Cache-Control", "no-cache")
|
||||
assert.Equal(t, testHeaders, headers)
|
||||
}
|
||||
|
||||
func setUpTestMux() http.Handler {
|
||||
mux := http.NewServeMux()
|
||||
mux.HandleFunc("/redirect", func(w http.ResponseWriter, req *http.Request) {
|
||||
http.Redirect(w, req, "/good", http.StatusMovedPermanently)
|
||||
})
|
||||
mux.HandleFunc("/good", func(w http.ResponseWriter, req *http.Request) {
|
||||
fmt.Fprintf(w, "hit the good page!")
|
||||
})
|
||||
mux.HandleFunc("/badredirect", func(w http.ResponseWriter, req *http.Request) {
|
||||
http.Redirect(w, req, "/badredirect", http.StatusMovedPermanently)
|
||||
})
|
||||
mux.HandleFunc("/mustbepostmethod", func(w http.ResponseWriter, req *http.Request) {
|
||||
if req.Method != "POST" {
|
||||
http.Error(w, "method wasn't post", http.StatusMethodNotAllowed)
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(w, "used post correctly!")
|
||||
})
|
||||
mux.HandleFunc("/musthaveabody", func(w http.ResponseWriter, req *http.Request) {
|
||||
body, err := ioutil.ReadAll(req.Body)
|
||||
req.Body.Close()
|
||||
if err != nil {
|
||||
http.Error(w, "couldn't read request body", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
if string(body) == "" {
|
||||
http.Error(w, "body was empty", http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
fmt.Fprintf(w, "sent a body!")
|
||||
})
|
||||
mux.HandleFunc("/twosecondnap", func(w http.ResponseWriter, req *http.Request) {
|
||||
time.Sleep(time.Second * 2)
|
||||
return
|
||||
})
|
||||
return mux
|
||||
}
|
||||
|
||||
func TestFields(t *testing.T) {
|
||||
mux := setUpTestMux()
|
||||
ts := httptest.NewServer(mux)
|
||||
defer ts.Close()
|
||||
|
||||
h := &HTTPResponse{
|
||||
Address: ts.URL + "/good",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "GET",
|
||||
ResponseTimeout: 20,
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
FollowRedirects: true,
|
||||
}
|
||||
fields, err := h.HTTPGather()
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, fields)
|
||||
if assert.NotNil(t, fields["http_response_code"]) {
|
||||
assert.Equal(t, http.StatusOK, fields["http_response_code"])
|
||||
}
|
||||
assert.NotNil(t, fields["response_time"])
|
||||
|
||||
}
|
||||
|
||||
func TestRedirects(t *testing.T) {
|
||||
mux := setUpTestMux()
|
||||
ts := httptest.NewServer(mux)
|
||||
defer ts.Close()
|
||||
|
||||
h := &HTTPResponse{
|
||||
Address: ts.URL + "/redirect",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "GET",
|
||||
ResponseTimeout: 20,
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
FollowRedirects: true,
|
||||
}
|
||||
fields, err := h.HTTPGather()
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, fields)
|
||||
if assert.NotNil(t, fields["http_response_code"]) {
|
||||
assert.Equal(t, http.StatusOK, fields["http_response_code"])
|
||||
}
|
||||
|
||||
h = &HTTPResponse{
|
||||
Address: ts.URL + "/badredirect",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "GET",
|
||||
ResponseTimeout: 20,
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
FollowRedirects: true,
|
||||
}
|
||||
fields, err = h.HTTPGather()
|
||||
require.Error(t, err)
|
||||
}
|
||||
|
||||
func TestMethod(t *testing.T) {
|
||||
mux := setUpTestMux()
|
||||
ts := httptest.NewServer(mux)
|
||||
defer ts.Close()
|
||||
|
||||
h := &HTTPResponse{
|
||||
Address: ts.URL + "/mustbepostmethod",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "POST",
|
||||
ResponseTimeout: 20,
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
FollowRedirects: true,
|
||||
}
|
||||
fields, err := h.HTTPGather()
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, fields)
|
||||
if assert.NotNil(t, fields["http_response_code"]) {
|
||||
assert.Equal(t, http.StatusOK, fields["http_response_code"])
|
||||
}
|
||||
|
||||
h = &HTTPResponse{
|
||||
Address: ts.URL + "/mustbepostmethod",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "GET",
|
||||
ResponseTimeout: 20,
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
FollowRedirects: true,
|
||||
}
|
||||
fields, err = h.HTTPGather()
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, fields)
|
||||
if assert.NotNil(t, fields["http_response_code"]) {
|
||||
assert.Equal(t, http.StatusMethodNotAllowed, fields["http_response_code"])
|
||||
}
|
||||
|
||||
//check that lowercase methods work correctly
|
||||
h = &HTTPResponse{
|
||||
Address: ts.URL + "/mustbepostmethod",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "head",
|
||||
ResponseTimeout: 20,
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
FollowRedirects: true,
|
||||
}
|
||||
fields, err = h.HTTPGather()
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, fields)
|
||||
if assert.NotNil(t, fields["http_response_code"]) {
|
||||
assert.Equal(t, http.StatusMethodNotAllowed, fields["http_response_code"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestBody(t *testing.T) {
|
||||
mux := setUpTestMux()
|
||||
ts := httptest.NewServer(mux)
|
||||
defer ts.Close()
|
||||
|
||||
h := &HTTPResponse{
|
||||
Address: ts.URL + "/musthaveabody",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "GET",
|
||||
ResponseTimeout: 20,
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
FollowRedirects: true,
|
||||
}
|
||||
fields, err := h.HTTPGather()
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, fields)
|
||||
if assert.NotNil(t, fields["http_response_code"]) {
|
||||
assert.Equal(t, http.StatusOK, fields["http_response_code"])
|
||||
}
|
||||
|
||||
h = &HTTPResponse{
|
||||
Address: ts.URL + "/musthaveabody",
|
||||
Method: "GET",
|
||||
ResponseTimeout: 20,
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
FollowRedirects: true,
|
||||
}
|
||||
fields, err = h.HTTPGather()
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, fields)
|
||||
if assert.NotNil(t, fields["http_response_code"]) {
|
||||
assert.Equal(t, http.StatusBadRequest, fields["http_response_code"])
|
||||
}
|
||||
}
|
||||
|
||||
func TestTimeout(t *testing.T) {
|
||||
mux := setUpTestMux()
|
||||
ts := httptest.NewServer(mux)
|
||||
defer ts.Close()
|
||||
|
||||
h := &HTTPResponse{
|
||||
Address: ts.URL + "/twosecondnap",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "GET",
|
||||
ResponseTimeout: 1,
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
FollowRedirects: true,
|
||||
}
|
||||
_, err := h.HTTPGather()
|
||||
require.Error(t, err)
|
||||
}
|
||||
@@ -6,7 +6,7 @@ For example, if you have a service called _mycollector_, which has HTTP endpoint
|
||||
plugin like this:
|
||||
|
||||
```
|
||||
[[inputs.httpjson]]
|
||||
[[httpjson.services]]
|
||||
name = "mycollector"
|
||||
|
||||
servers = [
|
||||
@@ -24,7 +24,7 @@ plugin like this:
|
||||
You can also specify which keys from server response should be considered tags:
|
||||
|
||||
```
|
||||
[[inputs.httpjson]]
|
||||
[[httpjson.services]]
|
||||
...
|
||||
|
||||
tag_keys = [
|
||||
@@ -36,10 +36,10 @@ You can also specify which keys from server response should be considered tags:
|
||||
You can also specify additional request parameters for the service:
|
||||
|
||||
```
|
||||
[[inputs.httpjson]]
|
||||
[[httpjson.services]]
|
||||
...
|
||||
|
||||
[inputs.httpjson.parameters]
|
||||
[httpjson.services.parameters]
|
||||
event_type = "cpu_spike"
|
||||
threshold = "0.75"
|
||||
|
||||
@@ -48,10 +48,10 @@ You can also specify additional request parameters for the service:
|
||||
You can also specify additional request header parameters for the service:
|
||||
|
||||
```
|
||||
[[inputs.httpjson]]
|
||||
[[httpjson.services]]
|
||||
...
|
||||
|
||||
[inputs.httpjson.headers]
|
||||
[httpjson.services.headers]
|
||||
X-Auth-Token = "my-xauth-token"
|
||||
apiVersion = "v1"
|
||||
```
|
||||
@@ -61,14 +61,18 @@ You can also specify additional request header parameters for the service:
|
||||
Let's say that we have a service named "mycollector" configured like this:
|
||||
|
||||
```
|
||||
[[inputs.httpjson]]
|
||||
name = "mycollector"
|
||||
servers = [
|
||||
"http://my.service.com/_stats"
|
||||
]
|
||||
# HTTP method to use (case-sensitive)
|
||||
method = "GET"
|
||||
tag_keys = ["service"]
|
||||
[httpjson]
|
||||
[[httpjson.services]]
|
||||
name = "mycollector"
|
||||
|
||||
servers = [
|
||||
"http://my.service.com/_stats"
|
||||
]
|
||||
|
||||
# HTTP method to use (case-sensitive)
|
||||
method = "GET"
|
||||
|
||||
tag_keys = ["service"]
|
||||
```
|
||||
|
||||
which responds with the following JSON:
|
||||
@@ -98,21 +102,26 @@ There is also the option to collect JSON from multiple services, here is an
|
||||
example doing that.
|
||||
|
||||
```
|
||||
[[inputs.httpjson]]
|
||||
name = "mycollector1"
|
||||
servers = [
|
||||
"http://my.service1.com/_stats"
|
||||
]
|
||||
# HTTP method to use (case-sensitive)
|
||||
method = "GET"
|
||||
[httpjson]
|
||||
[[httpjson.services]]
|
||||
name = "mycollector1"
|
||||
|
||||
[[inputs.httpjson]]
|
||||
name = "mycollector2"
|
||||
servers = [
|
||||
"http://service.net/json/stats"
|
||||
]
|
||||
# HTTP method to use (case-sensitive)
|
||||
method = "POST"
|
||||
servers = [
|
||||
"http://my.service1.com/_stats"
|
||||
]
|
||||
|
||||
# HTTP method to use (case-sensitive)
|
||||
method = "GET"
|
||||
|
||||
[[httpjson.services]]
|
||||
name = "mycollector2"
|
||||
|
||||
servers = [
|
||||
"http://service.net/json/stats"
|
||||
]
|
||||
|
||||
# HTTP method to use (case-sensitive)
|
||||
method = "POST"
|
||||
```
|
||||
|
||||
The services respond with the following JSON:
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package httpjson
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
@@ -11,7 +12,6 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"github.com/influxdata/telegraf/plugins/parsers"
|
||||
)
|
||||
@@ -23,17 +23,7 @@ type HttpJson struct {
|
||||
TagKeys []string
|
||||
Parameters map[string]string
|
||||
Headers map[string]string
|
||||
|
||||
// Path to CA file
|
||||
SSLCA string `toml:"ssl_ca"`
|
||||
// Path to host cert file
|
||||
SSLCert string `toml:"ssl_cert"`
|
||||
// Path to cert key file
|
||||
SSLKey string `toml:"ssl_key"`
|
||||
// Use SSL but skip chain & host verification
|
||||
InsecureSkipVerify bool
|
||||
|
||||
client HTTPClient
|
||||
client HTTPClient
|
||||
}
|
||||
|
||||
type HTTPClient interface {
|
||||
@@ -46,27 +36,16 @@ type HTTPClient interface {
|
||||
// http.Response: HTTP respons object
|
||||
// error : Any error that may have occurred
|
||||
MakeRequest(req *http.Request) (*http.Response, error)
|
||||
|
||||
SetHTTPClient(client *http.Client)
|
||||
HTTPClient() *http.Client
|
||||
}
|
||||
|
||||
type RealHTTPClient struct {
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
func (c *RealHTTPClient) MakeRequest(req *http.Request) (*http.Response, error) {
|
||||
func (c RealHTTPClient) MakeRequest(req *http.Request) (*http.Response, error) {
|
||||
return c.client.Do(req)
|
||||
}
|
||||
|
||||
func (c *RealHTTPClient) SetHTTPClient(client *http.Client) {
|
||||
c.client = client
|
||||
}
|
||||
|
||||
func (c *RealHTTPClient) HTTPClient() *http.Client {
|
||||
return c.client
|
||||
}
|
||||
|
||||
var sampleConfig = `
|
||||
## NOTE This plugin only reads numerical measurements, strings and booleans
|
||||
## will be ignored.
|
||||
@@ -98,13 +77,6 @@ var sampleConfig = `
|
||||
# [inputs.httpjson.headers]
|
||||
# X-Auth-Token = "my-xauth-token"
|
||||
# apiVersion = "v1"
|
||||
|
||||
## Optional SSL Config
|
||||
# ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# ssl_key = "/etc/telegraf/key.pem"
|
||||
## Use SSL but skip chain & host verification
|
||||
# insecure_skip_verify = false
|
||||
`
|
||||
|
||||
func (h *HttpJson) SampleConfig() string {
|
||||
@@ -119,23 +91,6 @@ func (h *HttpJson) Description() string {
|
||||
func (h *HttpJson) Gather(acc telegraf.Accumulator) error {
|
||||
var wg sync.WaitGroup
|
||||
|
||||
if h.client.HTTPClient() == nil {
|
||||
tlsCfg, err := internal.GetTLSConfig(
|
||||
h.SSLCert, h.SSLKey, h.SSLCA, h.InsecureSkipVerify)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
tr := &http.Transport{
|
||||
ResponseHeaderTimeout: time.Duration(3 * time.Second),
|
||||
TLSClientConfig: tlsCfg,
|
||||
}
|
||||
client := &http.Client{
|
||||
Transport: tr,
|
||||
Timeout: time.Duration(4 * time.Second),
|
||||
}
|
||||
h.client.SetHTTPClient(client)
|
||||
}
|
||||
|
||||
errorChannel := make(chan error, len(h.Servers))
|
||||
|
||||
for _, server := range h.Servers {
|
||||
@@ -227,14 +182,15 @@ func (h *HttpJson) sendRequest(serverURL string) (string, float64, error) {
|
||||
return "", -1, fmt.Errorf("Invalid server URL \"%s\"", serverURL)
|
||||
}
|
||||
|
||||
params := url.Values{}
|
||||
data := url.Values{}
|
||||
|
||||
switch {
|
||||
case h.Method == "GET":
|
||||
params := requestURL.Query()
|
||||
requestURL.RawQuery = params.Encode()
|
||||
for k, v := range h.Parameters {
|
||||
params.Add(k, v)
|
||||
}
|
||||
requestURL.RawQuery = params.Encode()
|
||||
|
||||
case h.Method == "POST":
|
||||
requestURL.RawQuery = ""
|
||||
@@ -244,8 +200,7 @@ func (h *HttpJson) sendRequest(serverURL string) (string, float64, error) {
|
||||
}
|
||||
|
||||
// Create + send request
|
||||
req, err := http.NewRequest(h.Method, requestURL.String(),
|
||||
strings.NewReader(data.Encode()))
|
||||
req, err := http.NewRequest(h.Method, requestURL.String(), bytes.NewBufferString(data.Encode()))
|
||||
if err != nil {
|
||||
return "", -1, err
|
||||
}
|
||||
@@ -289,8 +244,6 @@ func (h *HttpJson) sendRequest(serverURL string) (string, float64, error) {
|
||||
|
||||
func init() {
|
||||
inputs.Add("httpjson", func() telegraf.Input {
|
||||
return &HttpJson{
|
||||
client: &RealHTTPClient{},
|
||||
}
|
||||
return &HttpJson{client: RealHTTPClient{client: &http.Client{}}}
|
||||
})
|
||||
}
|
||||
|
||||
@@ -1,10 +1,8 @@
|
||||
package httpjson
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
@@ -29,75 +27,6 @@ const validJSON = `
|
||||
"another_list": [4]
|
||||
}`
|
||||
|
||||
const validJSON2 = `{
|
||||
"user":{
|
||||
"hash_rate":0,
|
||||
"expected_24h_rewards":0,
|
||||
"total_rewards":0.000595109232,
|
||||
"paid_rewards":0,
|
||||
"unpaid_rewards":0.000595109232,
|
||||
"past_24h_rewards":0,
|
||||
"total_work":"5172625408",
|
||||
"blocks_found":0
|
||||
},
|
||||
"workers":{
|
||||
"brminer.1":{
|
||||
"hash_rate":0,
|
||||
"hash_rate_24h":0,
|
||||
"valid_shares":"6176",
|
||||
"stale_shares":"0",
|
||||
"invalid_shares":"0",
|
||||
"rewards":4.5506464e-5,
|
||||
"rewards_24h":0,
|
||||
"reset_time":1455409950
|
||||
},
|
||||
"brminer.2":{
|
||||
"hash_rate":0,
|
||||
"hash_rate_24h":0,
|
||||
"valid_shares":"0",
|
||||
"stale_shares":"0",
|
||||
"invalid_shares":"0",
|
||||
"rewards":0,
|
||||
"rewards_24h":0,
|
||||
"reset_time":1455936726
|
||||
},
|
||||
"brminer.3":{
|
||||
"hash_rate":0,
|
||||
"hash_rate_24h":0,
|
||||
"valid_shares":"0",
|
||||
"stale_shares":"0",
|
||||
"invalid_shares":"0",
|
||||
"rewards":0,
|
||||
"rewards_24h":0,
|
||||
"reset_time":1455936733
|
||||
}
|
||||
},
|
||||
"pool":{
|
||||
"hash_rate":114100000,
|
||||
"active_users":843,
|
||||
"total_work":"5015346808842682368",
|
||||
"pps_ratio":1.04,
|
||||
"pps_rate":7.655e-9
|
||||
},
|
||||
"network":{
|
||||
"hash_rate":1426117703,
|
||||
"block_number":944895,
|
||||
"time_per_block":156,
|
||||
"difficulty":51825.72835216,
|
||||
"next_difficulty":51916.15249019,
|
||||
"retarget_time":95053
|
||||
},
|
||||
"market":{
|
||||
"ltc_btc":0.00798,
|
||||
"ltc_usd":3.37801,
|
||||
"ltc_eur":3.113,
|
||||
"ltc_gbp":2.32807,
|
||||
"ltc_rub":241.796,
|
||||
"ltc_cny":21.3883,
|
||||
"btc_usd":422.852
|
||||
}
|
||||
}`
|
||||
|
||||
const validJSONTags = `
|
||||
{
|
||||
"value": 15,
|
||||
@@ -125,7 +54,7 @@ type mockHTTPClient struct {
|
||||
// Mock implementation of MakeRequest. Usually returns an http.Response with
|
||||
// hard-coded responseBody and statusCode. However, if the request uses a
|
||||
// nonstandard method, it uses status code 405 (method not allowed)
|
||||
func (c *mockHTTPClient) MakeRequest(req *http.Request) (*http.Response, error) {
|
||||
func (c mockHTTPClient) MakeRequest(req *http.Request) (*http.Response, error) {
|
||||
resp := http.Response{}
|
||||
resp.StatusCode = c.statusCode
|
||||
|
||||
@@ -147,13 +76,6 @@ func (c *mockHTTPClient) MakeRequest(req *http.Request) (*http.Response, error)
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
func (c *mockHTTPClient) SetHTTPClient(_ *http.Client) {
|
||||
}
|
||||
|
||||
func (c *mockHTTPClient) HTTPClient() *http.Client {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Generates a pointer to an HttpJson object that uses a mock HTTP client.
|
||||
// Parameters:
|
||||
// response : Body of the response that the mock HTTP client should return
|
||||
@@ -164,7 +86,7 @@ func (c *mockHTTPClient) HTTPClient() *http.Client {
|
||||
func genMockHttpJson(response string, statusCode int) []*HttpJson {
|
||||
return []*HttpJson{
|
||||
&HttpJson{
|
||||
client: &mockHTTPClient{responseBody: response, statusCode: statusCode},
|
||||
client: mockHTTPClient{responseBody: response, statusCode: statusCode},
|
||||
Servers: []string{
|
||||
"http://server1.example.com/metrics/",
|
||||
"http://server2.example.com/metrics/",
|
||||
@@ -181,7 +103,7 @@ func genMockHttpJson(response string, statusCode int) []*HttpJson {
|
||||
},
|
||||
},
|
||||
&HttpJson{
|
||||
client: &mockHTTPClient{responseBody: response, statusCode: statusCode},
|
||||
client: mockHTTPClient{responseBody: response, statusCode: statusCode},
|
||||
Servers: []string{
|
||||
"http://server3.example.com/metrics/",
|
||||
"http://server4.example.com/metrics/",
|
||||
@@ -227,222 +149,6 @@ func TestHttpJson200(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// Test that GET Parameters from the url string are applied properly
|
||||
func TestHttpJsonGET_URL(t *testing.T) {
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
key := r.FormValue("api_key")
|
||||
assert.Equal(t, "mykey", key)
|
||||
w.WriteHeader(http.StatusOK)
|
||||
fmt.Fprintln(w, validJSON2)
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
a := HttpJson{
|
||||
Servers: []string{ts.URL + "?api_key=mykey"},
|
||||
Name: "",
|
||||
Method: "GET",
|
||||
client: &RealHTTPClient{client: &http.Client{}},
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := a.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
// remove response_time from gathered fields because it's non-deterministic
|
||||
delete(acc.Metrics[0].Fields, "response_time")
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"market_btc_usd": float64(422.852),
|
||||
"market_ltc_btc": float64(0.00798),
|
||||
"market_ltc_cny": float64(21.3883),
|
||||
"market_ltc_eur": float64(3.113),
|
||||
"market_ltc_gbp": float64(2.32807),
|
||||
"market_ltc_rub": float64(241.796),
|
||||
"market_ltc_usd": float64(3.37801),
|
||||
"network_block_number": float64(944895),
|
||||
"network_difficulty": float64(51825.72835216),
|
||||
"network_hash_rate": float64(1.426117703e+09),
|
||||
"network_next_difficulty": float64(51916.15249019),
|
||||
"network_retarget_time": float64(95053),
|
||||
"network_time_per_block": float64(156),
|
||||
"pool_active_users": float64(843),
|
||||
"pool_hash_rate": float64(1.141e+08),
|
||||
"pool_pps_rate": float64(7.655e-09),
|
||||
"pool_pps_ratio": float64(1.04),
|
||||
"user_blocks_found": float64(0),
|
||||
"user_expected_24h_rewards": float64(0),
|
||||
"user_hash_rate": float64(0),
|
||||
"user_paid_rewards": float64(0),
|
||||
"user_past_24h_rewards": float64(0),
|
||||
"user_total_rewards": float64(0.000595109232),
|
||||
"user_unpaid_rewards": float64(0.000595109232),
|
||||
"workers_brminer.1_hash_rate": float64(0),
|
||||
"workers_brminer.1_hash_rate_24h": float64(0),
|
||||
"workers_brminer.1_reset_time": float64(1.45540995e+09),
|
||||
"workers_brminer.1_rewards": float64(4.5506464e-05),
|
||||
"workers_brminer.1_rewards_24h": float64(0),
|
||||
"workers_brminer.2_hash_rate": float64(0),
|
||||
"workers_brminer.2_hash_rate_24h": float64(0),
|
||||
"workers_brminer.2_reset_time": float64(1.455936726e+09),
|
||||
"workers_brminer.2_rewards": float64(0),
|
||||
"workers_brminer.2_rewards_24h": float64(0),
|
||||
"workers_brminer.3_hash_rate": float64(0),
|
||||
"workers_brminer.3_hash_rate_24h": float64(0),
|
||||
"workers_brminer.3_reset_time": float64(1.455936733e+09),
|
||||
"workers_brminer.3_rewards": float64(0),
|
||||
"workers_brminer.3_rewards_24h": float64(0),
|
||||
}
|
||||
|
||||
acc.AssertContainsFields(t, "httpjson", fields)
|
||||
}
|
||||
|
||||
// Test that GET Parameters are applied properly
|
||||
func TestHttpJsonGET(t *testing.T) {
|
||||
params := map[string]string{
|
||||
"api_key": "mykey",
|
||||
}
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
key := r.FormValue("api_key")
|
||||
assert.Equal(t, "mykey", key)
|
||||
w.WriteHeader(http.StatusOK)
|
||||
fmt.Fprintln(w, validJSON2)
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
a := HttpJson{
|
||||
Servers: []string{ts.URL},
|
||||
Name: "",
|
||||
Method: "GET",
|
||||
Parameters: params,
|
||||
client: &RealHTTPClient{client: &http.Client{}},
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := a.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
// remove response_time from gathered fields because it's non-deterministic
|
||||
delete(acc.Metrics[0].Fields, "response_time")
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"market_btc_usd": float64(422.852),
|
||||
"market_ltc_btc": float64(0.00798),
|
||||
"market_ltc_cny": float64(21.3883),
|
||||
"market_ltc_eur": float64(3.113),
|
||||
"market_ltc_gbp": float64(2.32807),
|
||||
"market_ltc_rub": float64(241.796),
|
||||
"market_ltc_usd": float64(3.37801),
|
||||
"network_block_number": float64(944895),
|
||||
"network_difficulty": float64(51825.72835216),
|
||||
"network_hash_rate": float64(1.426117703e+09),
|
||||
"network_next_difficulty": float64(51916.15249019),
|
||||
"network_retarget_time": float64(95053),
|
||||
"network_time_per_block": float64(156),
|
||||
"pool_active_users": float64(843),
|
||||
"pool_hash_rate": float64(1.141e+08),
|
||||
"pool_pps_rate": float64(7.655e-09),
|
||||
"pool_pps_ratio": float64(1.04),
|
||||
"user_blocks_found": float64(0),
|
||||
"user_expected_24h_rewards": float64(0),
|
||||
"user_hash_rate": float64(0),
|
||||
"user_paid_rewards": float64(0),
|
||||
"user_past_24h_rewards": float64(0),
|
||||
"user_total_rewards": float64(0.000595109232),
|
||||
"user_unpaid_rewards": float64(0.000595109232),
|
||||
"workers_brminer.1_hash_rate": float64(0),
|
||||
"workers_brminer.1_hash_rate_24h": float64(0),
|
||||
"workers_brminer.1_reset_time": float64(1.45540995e+09),
|
||||
"workers_brminer.1_rewards": float64(4.5506464e-05),
|
||||
"workers_brminer.1_rewards_24h": float64(0),
|
||||
"workers_brminer.2_hash_rate": float64(0),
|
||||
"workers_brminer.2_hash_rate_24h": float64(0),
|
||||
"workers_brminer.2_reset_time": float64(1.455936726e+09),
|
||||
"workers_brminer.2_rewards": float64(0),
|
||||
"workers_brminer.2_rewards_24h": float64(0),
|
||||
"workers_brminer.3_hash_rate": float64(0),
|
||||
"workers_brminer.3_hash_rate_24h": float64(0),
|
||||
"workers_brminer.3_reset_time": float64(1.455936733e+09),
|
||||
"workers_brminer.3_rewards": float64(0),
|
||||
"workers_brminer.3_rewards_24h": float64(0),
|
||||
}
|
||||
|
||||
acc.AssertContainsFields(t, "httpjson", fields)
|
||||
}
|
||||
|
||||
// Test that POST Parameters are applied properly
|
||||
func TestHttpJsonPOST(t *testing.T) {
|
||||
params := map[string]string{
|
||||
"api_key": "mykey",
|
||||
}
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
body, err := ioutil.ReadAll(r.Body)
|
||||
assert.NoError(t, err)
|
||||
assert.Equal(t, "api_key=mykey", string(body))
|
||||
w.WriteHeader(http.StatusOK)
|
||||
fmt.Fprintln(w, validJSON2)
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
a := HttpJson{
|
||||
Servers: []string{ts.URL},
|
||||
Name: "",
|
||||
Method: "POST",
|
||||
Parameters: params,
|
||||
client: &RealHTTPClient{client: &http.Client{}},
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := a.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
// remove response_time from gathered fields because it's non-deterministic
|
||||
delete(acc.Metrics[0].Fields, "response_time")
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"market_btc_usd": float64(422.852),
|
||||
"market_ltc_btc": float64(0.00798),
|
||||
"market_ltc_cny": float64(21.3883),
|
||||
"market_ltc_eur": float64(3.113),
|
||||
"market_ltc_gbp": float64(2.32807),
|
||||
"market_ltc_rub": float64(241.796),
|
||||
"market_ltc_usd": float64(3.37801),
|
||||
"network_block_number": float64(944895),
|
||||
"network_difficulty": float64(51825.72835216),
|
||||
"network_hash_rate": float64(1.426117703e+09),
|
||||
"network_next_difficulty": float64(51916.15249019),
|
||||
"network_retarget_time": float64(95053),
|
||||
"network_time_per_block": float64(156),
|
||||
"pool_active_users": float64(843),
|
||||
"pool_hash_rate": float64(1.141e+08),
|
||||
"pool_pps_rate": float64(7.655e-09),
|
||||
"pool_pps_ratio": float64(1.04),
|
||||
"user_blocks_found": float64(0),
|
||||
"user_expected_24h_rewards": float64(0),
|
||||
"user_hash_rate": float64(0),
|
||||
"user_paid_rewards": float64(0),
|
||||
"user_past_24h_rewards": float64(0),
|
||||
"user_total_rewards": float64(0.000595109232),
|
||||
"user_unpaid_rewards": float64(0.000595109232),
|
||||
"workers_brminer.1_hash_rate": float64(0),
|
||||
"workers_brminer.1_hash_rate_24h": float64(0),
|
||||
"workers_brminer.1_reset_time": float64(1.45540995e+09),
|
||||
"workers_brminer.1_rewards": float64(4.5506464e-05),
|
||||
"workers_brminer.1_rewards_24h": float64(0),
|
||||
"workers_brminer.2_hash_rate": float64(0),
|
||||
"workers_brminer.2_hash_rate_24h": float64(0),
|
||||
"workers_brminer.2_reset_time": float64(1.455936726e+09),
|
||||
"workers_brminer.2_rewards": float64(0),
|
||||
"workers_brminer.2_rewards_24h": float64(0),
|
||||
"workers_brminer.3_hash_rate": float64(0),
|
||||
"workers_brminer.3_hash_rate_24h": float64(0),
|
||||
"workers_brminer.3_reset_time": float64(1.455936733e+09),
|
||||
"workers_brminer.3_rewards": float64(0),
|
||||
"workers_brminer.3_rewards_24h": float64(0),
|
||||
}
|
||||
|
||||
acc.AssertContainsFields(t, "httpjson", fields)
|
||||
}
|
||||
|
||||
// Test response to HTTP 500
|
||||
func TestHttpJson500(t *testing.T) {
|
||||
httpjson := genMockHttpJson(validJSON, 500)
|
||||
|
||||
@@ -1,41 +1,6 @@
|
||||
# influxdb plugin
|
||||
|
||||
The InfluxDB plugin will collect metrics on the given InfluxDB servers.
|
||||
|
||||
This plugin can also gather metrics from endpoints that expose
|
||||
InfluxDB-formatted endpoints. See below for more information.
|
||||
|
||||
### Configuration:
|
||||
|
||||
```toml
|
||||
# Read InfluxDB-formatted JSON metrics from one or more HTTP endpoints
|
||||
[[inputs.influxdb]]
|
||||
## Works with InfluxDB debug endpoints out of the box,
|
||||
## but other services can use this format too.
|
||||
## See the influxdb plugin's README for more details.
|
||||
|
||||
## Multiple URLs from which to read InfluxDB-formatted JSON
|
||||
urls = [
|
||||
"http://localhost:8086/debug/vars"
|
||||
]
|
||||
```
|
||||
|
||||
### Measurements & Fields
|
||||
|
||||
- influxdb_database
|
||||
- influxdb_httpd
|
||||
- influxdb_measurement
|
||||
- influxdb_memstats
|
||||
- influxdb_shard
|
||||
- influxdb_subscriber
|
||||
- influxdb_tsm1_cache
|
||||
- influxdb_tsm1_wal
|
||||
- influxdb_write
|
||||
|
||||
### InfluxDB-formatted endpoints
|
||||
|
||||
The influxdb plugin can collect InfluxDB-formatted data from JSON endpoints.
|
||||
Whether associated with an Influx database or not.
|
||||
The influxdb plugin collects InfluxDB-formatted data from JSON endpoints.
|
||||
|
||||
With a configuration of:
|
||||
|
||||
@@ -100,11 +65,8 @@ influxdb_transactions,url='http://192.168.2.1:8086/debug/vars' total=100.0,balan
|
||||
|
||||
There are two important details to note about the collected metrics:
|
||||
|
||||
1. Even though the values in JSON are being displayed as integers,
|
||||
the metrics are reported as floats.
|
||||
1. Even though the values in JSON are being displayed as integers, the metrics are reported as floats.
|
||||
JSON encoders usually don't print the fractional part for round floats.
|
||||
Because you cannot change the type of an existing field in InfluxDB,
|
||||
we assume all numbers are floats.
|
||||
Because you cannot change the type of an existing field in InfluxDB, we assume all numbers are floats.
|
||||
|
||||
2. The top-level keys' names (in the example above, `"k1"`, `"k2"`, and `"k3"`)
|
||||
are not considered when recording the metrics.
|
||||
2. The top-level keys' names (in the example above, `"k1"`, `"k2"`, and `"k3"`) are not considered when recording the metrics.
|
||||
|
||||
@@ -7,7 +7,6 @@ import (
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
@@ -71,44 +70,6 @@ type point struct {
|
||||
Values map[string]interface{} `json:"values"`
|
||||
}
|
||||
|
||||
type memstats struct {
|
||||
Alloc int64 `json:"Alloc"`
|
||||
TotalAlloc int64 `json:"TotalAlloc"`
|
||||
Sys int64 `json:"Sys"`
|
||||
Lookups int64 `json:"Lookups"`
|
||||
Mallocs int64 `json:"Mallocs"`
|
||||
Frees int64 `json:"Frees"`
|
||||
HeapAlloc int64 `json:"HeapAlloc"`
|
||||
HeapSys int64 `json:"HeapSys"`
|
||||
HeapIdle int64 `json:"HeapIdle"`
|
||||
HeapInuse int64 `json:"HeapInuse"`
|
||||
HeapReleased int64 `json:"HeapReleased"`
|
||||
HeapObjects int64 `json:"HeapObjects"`
|
||||
StackInuse int64 `json:"StackInuse"`
|
||||
StackSys int64 `json:"StackSys"`
|
||||
MSpanInuse int64 `json:"MSpanInuse"`
|
||||
MSpanSys int64 `json:"MSpanSys"`
|
||||
MCacheInuse int64 `json:"MCacheInuse"`
|
||||
MCacheSys int64 `json:"MCacheSys"`
|
||||
BuckHashSys int64 `json:"BuckHashSys"`
|
||||
GCSys int64 `json:"GCSys"`
|
||||
OtherSys int64 `json:"OtherSys"`
|
||||
NextGC int64 `json:"NextGC"`
|
||||
LastGC int64 `json:"LastGC"`
|
||||
PauseTotalNs int64 `json:"PauseTotalNs"`
|
||||
NumGC int64 `json:"NumGC"`
|
||||
GCCPUFraction float64 `json:"GCCPUFraction"`
|
||||
}
|
||||
|
||||
var tr = &http.Transport{
|
||||
ResponseHeaderTimeout: time.Duration(3 * time.Second),
|
||||
}
|
||||
|
||||
var client = &http.Client{
|
||||
Transport: tr,
|
||||
Timeout: time.Duration(4 * time.Second),
|
||||
}
|
||||
|
||||
// Gathers data from a particular URL
|
||||
// Parameters:
|
||||
// acc : The telegraf Accumulator to use
|
||||
@@ -120,7 +81,7 @@ func (i *InfluxDB) gatherURL(
|
||||
acc telegraf.Accumulator,
|
||||
url string,
|
||||
) error {
|
||||
resp, err := client.Get(url)
|
||||
resp, err := http.Get(url)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -147,52 +108,12 @@ func (i *InfluxDB) gatherURL(
|
||||
break
|
||||
}
|
||||
|
||||
// Read in a string key. We don't do anything with the top-level keys,
|
||||
// so it's discarded.
|
||||
key, err := dec.Token()
|
||||
// Read in a string key. We don't do anything with the top-level keys, so it's discarded.
|
||||
_, err := dec.Token()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
if key.(string) == "memstats" {
|
||||
var m memstats
|
||||
if err := dec.Decode(&m); err != nil {
|
||||
continue
|
||||
}
|
||||
acc.AddFields("influxdb_memstats",
|
||||
map[string]interface{}{
|
||||
"alloc": m.Alloc,
|
||||
"total_alloc": m.TotalAlloc,
|
||||
"sys": m.Sys,
|
||||
"lookups": m.Lookups,
|
||||
"mallocs": m.Mallocs,
|
||||
"frees": m.Frees,
|
||||
"heap_alloc": m.HeapAlloc,
|
||||
"heap_sys": m.HeapSys,
|
||||
"heap_idle": m.HeapIdle,
|
||||
"heap_inuse": m.HeapInuse,
|
||||
"heap_released": m.HeapReleased,
|
||||
"heap_objects": m.HeapObjects,
|
||||
"stack_inuse": m.StackInuse,
|
||||
"stack_sys": m.StackSys,
|
||||
"mspan_inuse": m.MSpanInuse,
|
||||
"mspan_sys": m.MSpanSys,
|
||||
"mcache_inuse": m.MCacheInuse,
|
||||
"mcache_sys": m.MCacheSys,
|
||||
"buck_hash_sys": m.BuckHashSys,
|
||||
"gc_sys": m.GCSys,
|
||||
"other_sys": m.OtherSys,
|
||||
"next_gc": m.NextGC,
|
||||
"last_gc": m.LastGC,
|
||||
"pause_total_ns": m.PauseTotalNs,
|
||||
"num_gc": m.NumGC,
|
||||
"gcc_pu_fraction": m.GCCPUFraction,
|
||||
},
|
||||
map[string]string{
|
||||
"url": url,
|
||||
})
|
||||
}
|
||||
|
||||
// Attempt to parse a whole object into a point.
|
||||
// It might be a non-object, like a string or array.
|
||||
// If we fail to decode it into a point, ignore it and move on.
|
||||
@@ -201,8 +122,7 @@ func (i *InfluxDB) gatherURL(
|
||||
continue
|
||||
}
|
||||
|
||||
// If the object was a point, but was not fully initialized,
|
||||
// ignore it and move on.
|
||||
// If the object was a point, but was not fully initialized, ignore it and move on.
|
||||
if p.Name == "" || p.Tags == nil || p.Values == nil || len(p.Values) == 0 {
|
||||
continue
|
||||
}
|
||||
|
||||
@@ -11,138 +11,7 @@ import (
|
||||
)
|
||||
|
||||
func TestBasic(t *testing.T) {
|
||||
fakeServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path == "/endpoint" {
|
||||
_, _ = w.Write([]byte(basicJSON))
|
||||
} else {
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
}
|
||||
}))
|
||||
defer fakeServer.Close()
|
||||
|
||||
plugin := &influxdb.InfluxDB{
|
||||
URLs: []string{fakeServer.URL + "/endpoint"},
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
require.NoError(t, plugin.Gather(&acc))
|
||||
|
||||
require.Len(t, acc.Metrics, 2)
|
||||
fields := map[string]interface{}{
|
||||
// JSON will truncate floats to integer representations.
|
||||
// Since there's no distinction in JSON, we can't assume it's an int.
|
||||
"i": -1.0,
|
||||
"f": 0.5,
|
||||
"b": true,
|
||||
"s": "string",
|
||||
}
|
||||
tags := map[string]string{
|
||||
"id": "ex1",
|
||||
"url": fakeServer.URL + "/endpoint",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "influxdb_foo", fields, tags)
|
||||
|
||||
fields = map[string]interface{}{
|
||||
"x": "x",
|
||||
}
|
||||
tags = map[string]string{
|
||||
"id": "ex2",
|
||||
"url": fakeServer.URL + "/endpoint",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "influxdb_bar", fields, tags)
|
||||
}
|
||||
|
||||
func TestInfluxDB(t *testing.T) {
|
||||
fakeInfluxServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path == "/endpoint" {
|
||||
_, _ = w.Write([]byte(influxReturn))
|
||||
} else {
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
}
|
||||
}))
|
||||
defer fakeInfluxServer.Close()
|
||||
|
||||
plugin := &influxdb.InfluxDB{
|
||||
URLs: []string{fakeInfluxServer.URL + "/endpoint"},
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
require.NoError(t, plugin.Gather(&acc))
|
||||
|
||||
require.Len(t, acc.Metrics, 33)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"heap_inuse": int64(18046976),
|
||||
"heap_released": int64(3473408),
|
||||
"mspan_inuse": int64(97440),
|
||||
"total_alloc": int64(201739016),
|
||||
"sys": int64(38537464),
|
||||
"mallocs": int64(570251),
|
||||
"frees": int64(381008),
|
||||
"heap_idle": int64(15802368),
|
||||
"pause_total_ns": int64(5132914),
|
||||
"lookups": int64(77),
|
||||
"heap_sys": int64(33849344),
|
||||
"mcache_sys": int64(16384),
|
||||
"next_gc": int64(20843042),
|
||||
"gcc_pu_fraction": float64(4.287178819113636e-05),
|
||||
"other_sys": int64(1229737),
|
||||
"alloc": int64(17034016),
|
||||
"stack_inuse": int64(753664),
|
||||
"stack_sys": int64(753664),
|
||||
"buck_hash_sys": int64(1461583),
|
||||
"gc_sys": int64(1112064),
|
||||
"num_gc": int64(27),
|
||||
"heap_alloc": int64(17034016),
|
||||
"heap_objects": int64(189243),
|
||||
"mspan_sys": int64(114688),
|
||||
"mcache_inuse": int64(4800),
|
||||
"last_gc": int64(1460434886475114239),
|
||||
}
|
||||
|
||||
tags := map[string]string{
|
||||
"url": fakeInfluxServer.URL + "/endpoint",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "influxdb_memstats", fields, tags)
|
||||
}
|
||||
|
||||
func TestErrorHandling(t *testing.T) {
|
||||
badServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path == "/endpoint" {
|
||||
_, _ = w.Write([]byte("not json"))
|
||||
} else {
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
}
|
||||
}))
|
||||
defer badServer.Close()
|
||||
|
||||
plugin := &influxdb.InfluxDB{
|
||||
URLs: []string{badServer.URL + "/endpoint"},
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
require.Error(t, plugin.Gather(&acc))
|
||||
}
|
||||
|
||||
func TestErrorHandling404(t *testing.T) {
|
||||
badServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path == "/endpoint" {
|
||||
_, _ = w.Write([]byte(basicJSON))
|
||||
} else {
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
}
|
||||
}))
|
||||
defer badServer.Close()
|
||||
|
||||
plugin := &influxdb.InfluxDB{
|
||||
URLs: []string{badServer.URL},
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
require.Error(t, plugin.Gather(&acc))
|
||||
}
|
||||
|
||||
const basicJSON = `
|
||||
js := `
|
||||
{
|
||||
"_1": {
|
||||
"name": "foo",
|
||||
@@ -186,48 +55,43 @@ const basicJSON = `
|
||||
}
|
||||
}
|
||||
`
|
||||
fakeServer := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.URL.Path == "/endpoint" {
|
||||
_, _ = w.Write([]byte(js))
|
||||
} else {
|
||||
w.WriteHeader(http.StatusNotFound)
|
||||
}
|
||||
}))
|
||||
defer fakeServer.Close()
|
||||
|
||||
const influxReturn = `
|
||||
{
|
||||
"cluster": {"name": "cluster", "tags": {}, "values": {}},
|
||||
"cmdline": ["influxd"],
|
||||
"cq": {"name": "cq", "tags": {}, "values": {}},
|
||||
"database:_internal": {"name": "database", "tags": {"database": "_internal"}, "values": {"numMeasurements": 8, "numSeries": 12}},
|
||||
"database:udp": {"name": "database", "tags": {"database": "udp"}, "values": {"numMeasurements": 14, "numSeries": 38}},
|
||||
"hh:/Users/csparr/.influxdb/hh": {"name": "hh", "tags": {"path": "/Users/csparr/.influxdb/hh"}, "values": {}},
|
||||
"httpd::8086": {"name": "httpd", "tags": {"bind": ":8086"}, "values": {"req": 7, "reqActive": 1, "reqDurationNs": 4488799}},
|
||||
"measurement:cpu_idle.udp": {"name": "measurement", "tags": {"database": "udp", "measurement": "cpu_idle"}, "values": {"numSeries": 1}},
|
||||
"measurement:cpu_usage.udp": {"name": "measurement", "tags": {"database": "udp", "measurement": "cpu_usage"}, "values": {"numSeries": 1}},
|
||||
"measurement:database._internal": {"name": "measurement", "tags": {"database": "_internal", "measurement": "database"}, "values": {"numSeries": 2}},
|
||||
"measurement:database.udp": {"name": "measurement", "tags": {"database": "udp", "measurement": "database"}, "values": {"numSeries": 2}},
|
||||
"measurement:httpd.udp": {"name": "measurement", "tags": {"database": "udp", "measurement": "httpd"}, "values": {"numSeries": 1}},
|
||||
"measurement:measurement.udp": {"name": "measurement", "tags": {"database": "udp", "measurement": "measurement"}, "values": {"numSeries": 22}},
|
||||
"measurement:mem.udp": {"name": "measurement", "tags": {"database": "udp", "measurement": "mem"}, "values": {"numSeries": 1}},
|
||||
"measurement:net.udp": {"name": "measurement", "tags": {"database": "udp", "measurement": "net"}, "values": {"numSeries": 1}},
|
||||
"measurement:runtime._internal": {"name": "measurement", "tags": {"database": "_internal", "measurement": "runtime"}, "values": {"numSeries": 1}},
|
||||
"measurement:runtime.udp": {"name": "measurement", "tags": {"database": "udp", "measurement": "runtime"}, "values": {"numSeries": 1}},
|
||||
"measurement:shard._internal": {"name": "measurement", "tags": {"database": "_internal", "measurement": "shard"}, "values": {"numSeries": 2}},
|
||||
"measurement:shard.udp": {"name": "measurement", "tags": {"database": "udp", "measurement": "shard"}, "values": {"numSeries": 1}},
|
||||
"measurement:subscriber._internal": {"name": "measurement", "tags": {"database": "_internal", "measurement": "subscriber"}, "values": {"numSeries": 1}},
|
||||
"measurement:subscriber.udp": {"name": "measurement", "tags": {"database": "udp", "measurement": "subscriber"}, "values": {"numSeries": 1}},
|
||||
"measurement:swap_used.udp": {"name": "measurement", "tags": {"database": "udp", "measurement": "swap_used"}, "values": {"numSeries": 1}},
|
||||
"measurement:tsm1_cache._internal": {"name": "measurement", "tags": {"database": "_internal", "measurement": "tsm1_cache"}, "values": {"numSeries": 2}},
|
||||
"measurement:tsm1_cache.udp": {"name": "measurement", "tags": {"database": "udp", "measurement": "tsm1_cache"}, "values": {"numSeries": 2}},
|
||||
"measurement:tsm1_wal._internal": {"name": "measurement", "tags": {"database": "_internal", "measurement": "tsm1_wal"}, "values": {"numSeries": 2}},
|
||||
"measurement:tsm1_wal.udp": {"name": "measurement", "tags": {"database": "udp", "measurement": "tsm1_wal"}, "values": {"numSeries": 2}},
|
||||
"measurement:udp._internal": {"name": "measurement", "tags": {"database": "_internal", "measurement": "udp"}, "values": {"numSeries": 1}},
|
||||
"measurement:write._internal": {"name": "measurement", "tags": {"database": "_internal", "measurement": "write"}, "values": {"numSeries": 1}},
|
||||
"measurement:write.udp": {"name": "measurement", "tags": {"database": "udp", "measurement": "write"}, "values": {"numSeries": 1}},
|
||||
"memstats": {"Alloc":17034016,"TotalAlloc":201739016,"Sys":38537464,"Lookups":77,"Mallocs":570251,"Frees":381008,"HeapAlloc":17034016,"HeapSys":33849344,"HeapIdle":15802368,"HeapInuse":18046976,"HeapReleased":3473408,"HeapObjects":189243,"StackInuse":753664,"StackSys":753664,"MSpanInuse":97440,"MSpanSys":114688,"MCacheInuse":4800,"MCacheSys":16384,"BuckHashSys":1461583,"GCSys":1112064,"OtherSys":1229737,"NextGC":20843042,"LastGC":1460434886475114239,"PauseTotalNs":5132914,"PauseNs":[195052,117751,139370,156933,263089,165249,713747,103904,122015,294408,213753,170864,175845,114221,121563,122409,113098,162219,229257,126726,250774,254235,117206,293588,144279,124306,127053,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"PauseEnd":[1460433856394860455,1460433856398162739,1460433856405888337,1460433856411784017,1460433856417924684,1460433856428385687,1460433856443782908,1460433856456522851,1460433857392743223,1460433866484394564,1460433866494076235,1460433896472438632,1460433957839825106,1460433976473440328,1460434016473413006,1460434096471892794,1460434126470792929,1460434246480428250,1460434366554468369,1460434396471249528,1460434456471205885,1460434476479487292,1460434536471435965,1460434616469784776,1460434736482078216,1460434856544251733,1460434886475114239,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0],"NumGC":27,"GCCPUFraction":4.287178819113636e-05,"EnableGC":true,"DebugGC":false,"BySize":[{"Size":0,"Mallocs":0,"Frees":0},{"Size":8,"Mallocs":1031,"Frees":955},{"Size":16,"Mallocs":308485,"Frees":142064},{"Size":32,"Mallocs":64937,"Frees":54321},{"Size":48,"Mallocs":33012,"Frees":29754},{"Size":64,"Mallocs":20299,"Frees":18173},{"Size":80,"Mallocs":8186,"Frees":7597},{"Size":96,"Mallocs":9806,"Frees":8982},{"Size":112,"Mallocs":5671,"Frees":4850},{"Size":128,"Mallocs":2972,"Frees":2684},{"Size":144,"Mallocs":4106,"Frees":3719},{"Size":160,"Mallocs":1324,"Frees":911},{"Size":176,"Mallocs":2574,"Frees":2391},{"Size":192,"Mallocs":4053,"Frees":3863},{"Size":208,"Mallocs":442,"Frees":307},{"Size":224,"Mallocs":336,"Frees":172},{"Size":240,"Mallocs":143,"Frees":125},{"Size":256,"Mallocs":542,"Frees":497},{"Size":288,"Mallocs":15971,"Frees":14761},{"Size":320,"Mallocs":245,"Frees":30},{"Size":352,"Mallocs":1299,"Frees":1065},{"Size":384,"Mallocs":138,"Frees":2},{"Size":416,"Mallocs":54,"Frees":47},{"Size":448,"Mallocs":75,"Frees":29},{"Size":480,"Mallocs":6,"Frees":4},{"Size":512,"Mallocs":452,"Frees":422},{"Size":576,"Mallocs":486,"Frees":395},{"Size":640,"Mallocs":81,"Frees":67},{"Size":704,"Mallocs":421,"Frees":397},{"Size":768,"Mallocs":469,"Frees":468},{"Size":896,"Mallocs":1049,"Frees":1010},{"Size":1024,"Mallocs":1078,"Frees":960},{"Size":1152,"Mallocs":750,"Frees":498},{"Size":1280,"Mallocs":84,"Frees":72},{"Size":1408,"Mallocs":218,"Frees":187},{"Size":1536,"Mallocs":73,"Frees":48},{"Size":1664,"Mallocs":43,"Frees":30},{"Size":2048,"Mallocs":153,"Frees":57},{"Size":2304,"Mallocs":41,"Frees":30},{"Size":2560,"Mallocs":18,"Frees":15},{"Size":2816,"Mallocs":164,"Frees":157},{"Size":3072,"Mallocs":0,"Frees":0},{"Size":3328,"Mallocs":13,"Frees":6},{"Size":4096,"Mallocs":101,"Frees":82},{"Size":4608,"Mallocs":32,"Frees":26},{"Size":5376,"Mallocs":165,"Frees":151},{"Size":6144,"Mallocs":15,"Frees":9},{"Size":6400,"Mallocs":1,"Frees":1},{"Size":6656,"Mallocs":1,"Frees":0},{"Size":6912,"Mallocs":0,"Frees":0},{"Size":8192,"Mallocs":13,"Frees":13},{"Size":8448,"Mallocs":0,"Frees":0},{"Size":8704,"Mallocs":1,"Frees":1},{"Size":9472,"Mallocs":6,"Frees":4},{"Size":10496,"Mallocs":0,"Frees":0},{"Size":12288,"Mallocs":41,"Frees":35},{"Size":13568,"Mallocs":0,"Frees":0},{"Size":14080,"Mallocs":0,"Frees":0},{"Size":16384,"Mallocs":4,"Frees":4},{"Size":16640,"Mallocs":0,"Frees":0},{"Size":17664,"Mallocs":0,"Frees":0}]},
|
||||
"queryExecutor": {"name": "queryExecutor", "tags": {}, "values": {}},
|
||||
"shard:/Users/csparr/.influxdb/data/_internal/monitor/2:2": {"name": "shard", "tags": {"database": "_internal", "engine": "tsm1", "id": "2", "path": "/Users/csparr/.influxdb/data/_internal/monitor/2", "retentionPolicy": "monitor"}, "values": {}},
|
||||
"shard:/Users/csparr/.influxdb/data/udp/default/1:1": {"name": "shard", "tags": {"database": "udp", "engine": "tsm1", "id": "1", "path": "/Users/csparr/.influxdb/data/udp/default/1", "retentionPolicy": "default"}, "values": {"fieldsCreate": 61, "seriesCreate": 33, "writePointsOk": 3613, "writeReq": 110}},
|
||||
"subscriber": {"name": "subscriber", "tags": {}, "values": {"pointsWritten": 3613}},
|
||||
"tsm1_cache:/Users/csparr/.influxdb/data/_internal/monitor/2": {"name": "tsm1_cache", "tags": {"database": "_internal", "path": "/Users/csparr/.influxdb/data/_internal/monitor/2", "retentionPolicy": "monitor"}, "values": {"WALCompactionTimeMs": 0, "cacheAgeMs": 1103932, "cachedBytes": 0, "diskBytes": 0, "memBytes": 40480, "snapshotCount": 0}},
|
||||
"tsm1_cache:/Users/csparr/.influxdb/data/udp/default/1": {"name": "tsm1_cache", "tags": {"database": "udp", "path": "/Users/csparr/.influxdb/data/udp/default/1", "retentionPolicy": "default"}, "values": {"WALCompactionTimeMs": 0, "cacheAgeMs": 1103029, "cachedBytes": 0, "diskBytes": 0, "memBytes": 2359472, "snapshotCount": 0}},
|
||||
"tsm1_filestore:/Users/csparr/.influxdb/data/_internal/monitor/2": {"name": "tsm1_filestore", "tags": {"database": "_internal", "path": "/Users/csparr/.influxdb/data/_internal/monitor/2", "retentionPolicy": "monitor"}, "values": {}},
|
||||
"tsm1_filestore:/Users/csparr/.influxdb/data/udp/default/1": {"name": "tsm1_filestore", "tags": {"database": "udp", "path": "/Users/csparr/.influxdb/data/udp/default/1", "retentionPolicy": "default"}, "values": {}},
|
||||
"tsm1_wal:/Users/csparr/.influxdb/wal/_internal/monitor/2": {"name": "tsm1_wal", "tags": {"database": "_internal", "path": "/Users/csparr/.influxdb/wal/_internal/monitor/2", "retentionPolicy": "monitor"}, "values": {"currentSegmentDiskBytes": 0, "oldSegmentsDiskBytes": 69532}},
|
||||
"tsm1_wal:/Users/csparr/.influxdb/wal/udp/default/1": {"name": "tsm1_wal", "tags": {"database": "udp", "path": "/Users/csparr/.influxdb/wal/udp/default/1", "retentionPolicy": "default"}, "values": {"currentSegmentDiskBytes": 193728, "oldSegmentsDiskBytes": 1008330}},
|
||||
"write": {"name": "write", "tags": {}, "values": {"pointReq": 3613, "pointReqLocal": 3613, "req": 110, "subWriteOk": 110, "writeOk": 110}}
|
||||
}`
|
||||
plugin := &influxdb.InfluxDB{
|
||||
URLs: []string{fakeServer.URL + "/endpoint"},
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
require.NoError(t, plugin.Gather(&acc))
|
||||
|
||||
require.Len(t, acc.Metrics, 2)
|
||||
fields := map[string]interface{}{
|
||||
// JSON will truncate floats to integer representations.
|
||||
// Since there's no distinction in JSON, we can't assume it's an int.
|
||||
"i": -1.0,
|
||||
"f": 0.5,
|
||||
"b": true,
|
||||
"s": "string",
|
||||
}
|
||||
tags := map[string]string{
|
||||
"id": "ex1",
|
||||
"url": fakeServer.URL + "/endpoint",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "influxdb_foo", fields, tags)
|
||||
|
||||
fields = map[string]interface{}{
|
||||
"x": "x",
|
||||
}
|
||||
tags = map[string]string{
|
||||
"id": "ex2",
|
||||
"url": fakeServer.URL + "/endpoint",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "influxdb_bar", fields, tags)
|
||||
}
|
||||
|
||||
@@ -1,42 +0,0 @@
|
||||
# Telegraf ipmi plugin
|
||||
|
||||
Get bare metal metrics using the command line utility `ipmitool`
|
||||
|
||||
see ipmitool(https://sourceforge.net/projects/ipmitool/files/ipmitool/)
|
||||
|
||||
The plugin will use the following command to collect remote host sensor stats:
|
||||
|
||||
ipmitool -I lan -H 192.168.1.1 -U USERID -P PASSW0RD sdr
|
||||
|
||||
## Measurements
|
||||
|
||||
- ipmi_sensor:
|
||||
|
||||
* Tags: `name`, `server`, `unit`
|
||||
* Fields:
|
||||
- status
|
||||
- value
|
||||
|
||||
## Configuration
|
||||
|
||||
```toml
|
||||
[[inputs.ipmi_sensor]]
|
||||
## specify servers via a url matching:
|
||||
## [username[:password]@][protocol[(address)]]
|
||||
## e.g.
|
||||
## root:passwd@lan(127.0.0.1)
|
||||
##
|
||||
servers = ["USERID:PASSW0RD@lan(10.20.2.203)"]
|
||||
```
|
||||
|
||||
## Output
|
||||
|
||||
```
|
||||
> ipmi_sensor,server=10.20.2.203,unit=degrees_c,name=ambient_temp status=1i,value=20 1458488465012559455
|
||||
> ipmi_sensor,server=10.20.2.203,unit=feet,name=altitude status=1i,value=80 1458488465012688613
|
||||
> ipmi_sensor,server=10.20.2.203,unit=watts,name=avg_power status=1i,value=220 1458488465012776511
|
||||
> ipmi_sensor,server=10.20.2.203,unit=volts,name=planar_3.3v status=1i,value=3.28 1458488465012861875
|
||||
> ipmi_sensor,server=10.20.2.203,unit=volts,name=planar_vbat status=1i,value=3.04 1458488465013072508
|
||||
> ipmi_sensor,server=10.20.2.203,unit=rpm,name=fan_1a_tach status=1i,value=2610 1458488465013137932
|
||||
> ipmi_sensor,server=10.20.2.203,unit=rpm,name=fan_1b_tach status=1i,value=1775 1458488465013279896
|
||||
```
|
||||
@@ -1,35 +0,0 @@
|
||||
package ipmi_sensor
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os/exec"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
)
|
||||
|
||||
type CommandRunner struct{}
|
||||
|
||||
func (t CommandRunner) cmd(conn *Connection, args ...string) *exec.Cmd {
|
||||
path := conn.Path
|
||||
opts := append(conn.options(), args...)
|
||||
|
||||
if path == "" {
|
||||
path = "ipmitool"
|
||||
}
|
||||
|
||||
return exec.Command(path, opts...)
|
||||
}
|
||||
|
||||
func (t CommandRunner) Run(conn *Connection, args ...string) (string, error) {
|
||||
cmd := t.cmd(conn, args...)
|
||||
|
||||
output, err := internal.CombinedOutputTimeout(cmd, time.Second*5)
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("run %s %s: %s (%s)",
|
||||
cmd.Path, strings.Join(cmd.Args, " "), string(output), err)
|
||||
}
|
||||
|
||||
return string(output), err
|
||||
}
|
||||
@@ -1,89 +0,0 @@
|
||||
package ipmi_sensor
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"net"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Connection properties for a Client
|
||||
type Connection struct {
|
||||
Hostname string
|
||||
Username string
|
||||
Password string
|
||||
Path string
|
||||
Port int
|
||||
Interface string
|
||||
}
|
||||
|
||||
func NewConnection(server string) *Connection {
|
||||
conn := &Connection{}
|
||||
inx1 := strings.Index(server, "@")
|
||||
inx2 := strings.Index(server, "(")
|
||||
inx3 := strings.Index(server, ")")
|
||||
|
||||
connstr := server
|
||||
|
||||
if inx1 > 0 {
|
||||
security := server[0:inx1]
|
||||
connstr = server[inx1+1 : len(server)]
|
||||
up := strings.SplitN(security, ":", 2)
|
||||
conn.Username = up[0]
|
||||
conn.Password = up[1]
|
||||
}
|
||||
|
||||
if inx2 > 0 {
|
||||
inx2 = strings.Index(connstr, "(")
|
||||
inx3 = strings.Index(connstr, ")")
|
||||
|
||||
conn.Interface = connstr[0:inx2]
|
||||
conn.Hostname = connstr[inx2+1 : inx3]
|
||||
}
|
||||
|
||||
return conn
|
||||
}
|
||||
|
||||
func (t *Connection) options() []string {
|
||||
intf := t.Interface
|
||||
if intf == "" {
|
||||
intf = "lan"
|
||||
}
|
||||
|
||||
options := []string{
|
||||
"-H", t.Hostname,
|
||||
"-U", t.Username,
|
||||
"-P", t.Password,
|
||||
"-I", intf,
|
||||
}
|
||||
|
||||
if t.Port != 0 {
|
||||
options = append(options, "-p", strconv.Itoa(t.Port))
|
||||
}
|
||||
|
||||
return options
|
||||
}
|
||||
|
||||
// RemoteIP returns the remote (bmc) IP address of the Connection
|
||||
func (c *Connection) RemoteIP() string {
|
||||
if net.ParseIP(c.Hostname) == nil {
|
||||
addrs, err := net.LookupHost(c.Hostname)
|
||||
if err != nil && len(addrs) > 0 {
|
||||
return addrs[0]
|
||||
}
|
||||
}
|
||||
return c.Hostname
|
||||
}
|
||||
|
||||
// LocalIP returns the local (client) IP address of the Connection
|
||||
func (c *Connection) LocalIP() string {
|
||||
conn, err := net.Dial("udp", fmt.Sprintf("%s:%d", c.Hostname, c.Port))
|
||||
if err != nil {
|
||||
// don't bother returning an error, since this value will never
|
||||
// make it to the bmc if we can't connect to it.
|
||||
return c.Hostname
|
||||
}
|
||||
_ = conn.Close()
|
||||
host, _, _ := net.SplitHostPort(conn.LocalAddr().String())
|
||||
return host
|
||||
}
|
||||
@@ -1,129 +0,0 @@
|
||||
package ipmi_sensor
|
||||
|
||||
import (
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
type Ipmi struct {
|
||||
Servers []string
|
||||
runner Runner
|
||||
}
|
||||
|
||||
var sampleConfig = `
|
||||
## specify servers via a url matching:
|
||||
## [username[:password]@][protocol[(address)]]
|
||||
## e.g.
|
||||
## root:passwd@lan(127.0.0.1)
|
||||
##
|
||||
servers = ["USERID:PASSW0RD@lan(192.168.1.1)"]
|
||||
`
|
||||
|
||||
func NewIpmi() *Ipmi {
|
||||
return &Ipmi{
|
||||
runner: CommandRunner{},
|
||||
}
|
||||
}
|
||||
|
||||
func (m *Ipmi) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
|
||||
func (m *Ipmi) Description() string {
|
||||
return "Read metrics from one or many bare metal servers"
|
||||
}
|
||||
|
||||
func (m *Ipmi) Gather(acc telegraf.Accumulator) error {
|
||||
if m.runner == nil {
|
||||
m.runner = CommandRunner{}
|
||||
}
|
||||
for _, serv := range m.Servers {
|
||||
err := m.gatherServer(serv, acc)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (m *Ipmi) gatherServer(serv string, acc telegraf.Accumulator) error {
|
||||
conn := NewConnection(serv)
|
||||
|
||||
res, err := m.runner.Run(conn, "sdr")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// each line will look something like
|
||||
// Planar VBAT | 3.05 Volts | ok
|
||||
lines := strings.Split(res, "\n")
|
||||
for i := 0; i < len(lines); i++ {
|
||||
vals := strings.Split(lines[i], "|")
|
||||
if len(vals) != 3 {
|
||||
continue
|
||||
}
|
||||
|
||||
tags := map[string]string{
|
||||
"server": conn.Hostname,
|
||||
"name": transform(vals[0]),
|
||||
}
|
||||
|
||||
fields := make(map[string]interface{})
|
||||
if strings.EqualFold("ok", trim(vals[2])) {
|
||||
fields["status"] = 1
|
||||
} else {
|
||||
fields["status"] = 0
|
||||
}
|
||||
|
||||
val1 := trim(vals[1])
|
||||
|
||||
if strings.Index(val1, " ") > 0 {
|
||||
// split middle column into value and unit
|
||||
valunit := strings.SplitN(val1, " ", 2)
|
||||
fields["value"] = Atofloat(valunit[0])
|
||||
if len(valunit) > 1 {
|
||||
tags["unit"] = transform(valunit[1])
|
||||
}
|
||||
} else {
|
||||
fields["value"] = 0.0
|
||||
}
|
||||
|
||||
acc.AddFields("ipmi_sensor", fields, tags, time.Now())
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
type Runner interface {
|
||||
Run(conn *Connection, args ...string) (string, error)
|
||||
}
|
||||
|
||||
func Atofloat(val string) float64 {
|
||||
f, err := strconv.ParseFloat(val, 64)
|
||||
if err != nil {
|
||||
return 0.0
|
||||
} else {
|
||||
return f
|
||||
}
|
||||
}
|
||||
|
||||
func trim(s string) string {
|
||||
return strings.TrimSpace(s)
|
||||
}
|
||||
|
||||
func transform(s string) string {
|
||||
s = trim(s)
|
||||
s = strings.ToLower(s)
|
||||
return strings.Replace(s, " ", "_", -1)
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("ipmi_sensor", func() telegraf.Input {
|
||||
return &Ipmi{}
|
||||
})
|
||||
}
|
||||
@@ -1,275 +0,0 @@
|
||||
package ipmi_sensor
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
const serv = "USERID:PASSW0RD@lan(192.168.1.1)"
|
||||
|
||||
const cmdReturn = `
|
||||
Ambient Temp | 20 degrees C | ok
|
||||
Altitude | 80 feet | ok
|
||||
Avg Power | 210 Watts | ok
|
||||
Planar 3.3V | 3.29 Volts | ok
|
||||
Planar 5V | 4.90 Volts | ok
|
||||
Planar 12V | 12.04 Volts | ok
|
||||
Planar VBAT | 3.05 Volts | ok
|
||||
Fan 1A Tach | 2610 RPM | ok
|
||||
Fan 1B Tach | 1775 RPM | ok
|
||||
Fan 2A Tach | 2001 RPM | ok
|
||||
Fan 2B Tach | 1275 RPM | ok
|
||||
Fan 3A Tach | 2929 RPM | ok
|
||||
Fan 3B Tach | 2125 RPM | ok
|
||||
Fan 1 | 0x00 | ok
|
||||
Fan 2 | 0x00 | ok
|
||||
Fan 3 | 0x00 | ok
|
||||
Front Panel | 0x00 | ok
|
||||
Video USB | 0x00 | ok
|
||||
DASD Backplane 1 | 0x00 | ok
|
||||
SAS Riser | 0x00 | ok
|
||||
PCI Riser 1 | 0x00 | ok
|
||||
PCI Riser 2 | 0x00 | ok
|
||||
CPU 1 | 0x00 | ok
|
||||
CPU 2 | 0x00 | ok
|
||||
All CPUs | 0x00 | ok
|
||||
One of The CPUs | 0x00 | ok
|
||||
IOH Temp Status | 0x00 | ok
|
||||
CPU 1 OverTemp | 0x00 | ok
|
||||
CPU 2 OverTemp | 0x00 | ok
|
||||
CPU Fault Reboot | 0x00 | ok
|
||||
Aux Log | 0x00 | ok
|
||||
NMI State | 0x00 | ok
|
||||
ABR Status | 0x00 | ok
|
||||
Firmware Error | 0x00 | ok
|
||||
PCIs | 0x00 | ok
|
||||
CPUs | 0x00 | ok
|
||||
DIMMs | 0x00 | ok
|
||||
Sys Board Fault | 0x00 | ok
|
||||
Power Supply 1 | 0x00 | ok
|
||||
Power Supply 2 | 0x00 | ok
|
||||
PS 1 Fan Fault | 0x00 | ok
|
||||
PS 2 Fan Fault | 0x00 | ok
|
||||
VT Fault | 0x00 | ok
|
||||
Pwr Rail A Fault | 0x00 | ok
|
||||
Pwr Rail B Fault | 0x00 | ok
|
||||
Pwr Rail C Fault | 0x00 | ok
|
||||
Pwr Rail D Fault | 0x00 | ok
|
||||
Pwr Rail E Fault | 0x00 | ok
|
||||
PS 1 Therm Fault | 0x00 | ok
|
||||
PS 2 Therm Fault | 0x00 | ok
|
||||
PS1 12V OV Fault | 0x00 | ok
|
||||
PS2 12V OV Fault | 0x00 | ok
|
||||
PS1 12V UV Fault | 0x00 | ok
|
||||
PS2 12V UV Fault | 0x00 | ok
|
||||
PS1 12V OC Fault | 0x00 | ok
|
||||
PS2 12V OC Fault | 0x00 | ok
|
||||
PS 1 VCO Fault | 0x00 | ok
|
||||
PS 2 VCO Fault | 0x00 | ok
|
||||
Power Unit | 0x00 | ok
|
||||
Cooling Zone 1 | 0x00 | ok
|
||||
Cooling Zone 2 | 0x00 | ok
|
||||
Cooling Zone 3 | 0x00 | ok
|
||||
Drive 0 | 0x00 | ok
|
||||
Drive 1 | 0x00 | ok
|
||||
Drive 2 | 0x00 | ok
|
||||
Drive 3 | 0x00 | ok
|
||||
Drive 4 | 0x00 | ok
|
||||
Drive 5 | 0x00 | ok
|
||||
Drive 6 | 0x00 | ok
|
||||
Drive 7 | 0x00 | ok
|
||||
Drive 8 | 0x00 | ok
|
||||
Drive 9 | 0x00 | ok
|
||||
Drive 10 | 0x00 | ok
|
||||
Drive 11 | 0x00 | ok
|
||||
Drive 12 | 0x00 | ok
|
||||
Drive 13 | 0x00 | ok
|
||||
Drive 14 | 0x00 | ok
|
||||
Drive 15 | 0x00 | ok
|
||||
All DIMMS | 0x00 | ok
|
||||
One of the DIMMs | 0x00 | ok
|
||||
DIMM 1 | 0x00 | ok
|
||||
DIMM 2 | 0x00 | ok
|
||||
DIMM 3 | 0x00 | ok
|
||||
DIMM 4 | 0x00 | ok
|
||||
DIMM 5 | 0x00 | ok
|
||||
DIMM 6 | 0x00 | ok
|
||||
DIMM 7 | 0x00 | ok
|
||||
DIMM 8 | 0x00 | ok
|
||||
DIMM 9 | 0x00 | ok
|
||||
DIMM 10 | 0x00 | ok
|
||||
DIMM 11 | 0x00 | ok
|
||||
DIMM 12 | 0x00 | ok
|
||||
DIMM 13 | 0x00 | ok
|
||||
DIMM 14 | 0x00 | ok
|
||||
DIMM 15 | 0x00 | ok
|
||||
DIMM 16 | 0x00 | ok
|
||||
DIMM 17 | 0x00 | ok
|
||||
DIMM 18 | 0x00 | ok
|
||||
DIMM 1 Temp | 0x00 | ok
|
||||
DIMM 2 Temp | 0x00 | ok
|
||||
DIMM 3 Temp | 0x00 | ok
|
||||
DIMM 4 Temp | 0x00 | ok
|
||||
DIMM 5 Temp | 0x00 | ok
|
||||
DIMM 6 Temp | 0x00 | ok
|
||||
DIMM 7 Temp | 0x00 | ok
|
||||
DIMM 8 Temp | 0x00 | ok
|
||||
DIMM 9 Temp | 0x00 | ok
|
||||
DIMM 10 Temp | 0x00 | ok
|
||||
DIMM 11 Temp | 0x00 | ok
|
||||
DIMM 12 Temp | 0x00 | ok
|
||||
DIMM 13 Temp | 0x00 | ok
|
||||
DIMM 14 Temp | 0x00 | ok
|
||||
DIMM 15 Temp | 0x00 | ok
|
||||
DIMM 16 Temp | 0x00 | ok
|
||||
DIMM 17 Temp | 0x00 | ok
|
||||
DIMM 18 Temp | 0x00 | ok
|
||||
PCI 1 | 0x00 | ok
|
||||
PCI 2 | 0x00 | ok
|
||||
PCI 3 | 0x00 | ok
|
||||
PCI 4 | 0x00 | ok
|
||||
All PCI Error | 0x00 | ok
|
||||
One of PCI Error | 0x00 | ok
|
||||
IPMI Watchdog | 0x00 | ok
|
||||
Host Power | 0x00 | ok
|
||||
DASD Backplane 2 | 0x00 | ok
|
||||
DASD Backplane 3 | Not Readable | ns
|
||||
DASD Backplane 4 | Not Readable | ns
|
||||
Backup Memory | 0x00 | ok
|
||||
Progress | 0x00 | ok
|
||||
Planar Fault | 0x00 | ok
|
||||
SEL Fullness | 0x00 | ok
|
||||
PCI 5 | 0x00 | ok
|
||||
OS RealTime Mod | 0x00 | ok
|
||||
`
|
||||
|
||||
type runnerMock struct {
|
||||
out string
|
||||
err error
|
||||
}
|
||||
|
||||
func newRunnerMock(out string, err error) Runner {
|
||||
return &runnerMock{
|
||||
out: out,
|
||||
err: err,
|
||||
}
|
||||
}
|
||||
|
||||
func (r runnerMock) Run(conn *Connection, args ...string) (out string, err error) {
|
||||
if r.err != nil {
|
||||
return out, r.err
|
||||
}
|
||||
return r.out, nil
|
||||
}
|
||||
|
||||
func TestIpmi(t *testing.T) {
|
||||
i := &Ipmi{
|
||||
Servers: []string{"USERID:PASSW0RD@lan(192.168.1.1)"},
|
||||
runner: newRunnerMock(cmdReturn, nil),
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
|
||||
err := i.Gather(&acc)
|
||||
|
||||
require.NoError(t, err)
|
||||
|
||||
assert.Equal(t, acc.NFields(), 266, "non-numeric measurements should be ignored")
|
||||
|
||||
var tests = []struct {
|
||||
fields map[string]interface{}
|
||||
tags map[string]string
|
||||
}{
|
||||
{
|
||||
map[string]interface{}{
|
||||
"value": float64(20),
|
||||
"status": int(1),
|
||||
},
|
||||
map[string]string{
|
||||
"name": "ambient_temp",
|
||||
"server": "192.168.1.1",
|
||||
"unit": "degrees_c",
|
||||
},
|
||||
},
|
||||
{
|
||||
map[string]interface{}{
|
||||
"value": float64(80),
|
||||
"status": int(1),
|
||||
},
|
||||
map[string]string{
|
||||
"name": "altitude",
|
||||
"server": "192.168.1.1",
|
||||
"unit": "feet",
|
||||
},
|
||||
},
|
||||
{
|
||||
map[string]interface{}{
|
||||
"value": float64(210),
|
||||
"status": int(1),
|
||||
},
|
||||
map[string]string{
|
||||
"name": "avg_power",
|
||||
"server": "192.168.1.1",
|
||||
"unit": "watts",
|
||||
},
|
||||
},
|
||||
{
|
||||
map[string]interface{}{
|
||||
"value": float64(4.9),
|
||||
"status": int(1),
|
||||
},
|
||||
map[string]string{
|
||||
"name": "planar_5v",
|
||||
"server": "192.168.1.1",
|
||||
"unit": "volts",
|
||||
},
|
||||
},
|
||||
{
|
||||
map[string]interface{}{
|
||||
"value": float64(3.05),
|
||||
"status": int(1),
|
||||
},
|
||||
map[string]string{
|
||||
"name": "planar_vbat",
|
||||
"server": "192.168.1.1",
|
||||
"unit": "volts",
|
||||
},
|
||||
},
|
||||
{
|
||||
map[string]interface{}{
|
||||
"value": float64(2610),
|
||||
"status": int(1),
|
||||
},
|
||||
map[string]string{
|
||||
"name": "fan_1a_tach",
|
||||
"server": "192.168.1.1",
|
||||
"unit": "rpm",
|
||||
},
|
||||
},
|
||||
{
|
||||
map[string]interface{}{
|
||||
"value": float64(1775),
|
||||
"status": int(1),
|
||||
},
|
||||
map[string]string{
|
||||
"name": "fan_1b_tach",
|
||||
"server": "192.168.1.1",
|
||||
"unit": "rpm",
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
for _, test := range tests {
|
||||
acc.AssertContainsTaggedFields(t, "ipmi_sensor", test.fields, test.tags)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIpmiConnection(t *testing.T) {
|
||||
conn := NewConnection(serv)
|
||||
assert.Equal(t, "USERID", conn.Username)
|
||||
assert.Equal(t, "lan", conn.Interface)
|
||||
|
||||
}
|
||||
@@ -1,61 +1,51 @@
|
||||
# Telegraf plugin: Jolokia
|
||||
|
||||
#### Configuration
|
||||
|
||||
```toml
|
||||
# Read JMX metrics through Jolokia
|
||||
[[inputs.jolokia]]
|
||||
## This is the context root used to compose the jolokia url
|
||||
context = "/jolokia"
|
||||
|
||||
## This specifies the mode used
|
||||
# mode = "proxy"
|
||||
#
|
||||
## When in proxy mode this section is used to specify further
|
||||
## proxy address configurations.
|
||||
## Remember to change host address to fit your environment.
|
||||
# [inputs.jolokia.proxy]
|
||||
# host = "127.0.0.1"
|
||||
# port = "8080"
|
||||
|
||||
|
||||
## List of servers exposing jolokia read service
|
||||
[[inputs.jolokia.servers]]
|
||||
name = "as-server-01"
|
||||
host = "127.0.0.1"
|
||||
port = "8080"
|
||||
# username = "myuser"
|
||||
# password = "mypassword"
|
||||
|
||||
## List of metrics collected on above servers
|
||||
## Each metric consists in a name, a jmx path and either
|
||||
## a pass or drop slice attribute.
|
||||
## This collect all heap memory usage metrics.
|
||||
[[inputs.jolokia.metrics]]
|
||||
name = "heap_memory_usage"
|
||||
mbean = "java.lang:type=Memory"
|
||||
attribute = "HeapMemoryUsage"
|
||||
|
||||
## This collect thread counts metrics.
|
||||
[[inputs.jolokia.metrics]]
|
||||
name = "thread_count"
|
||||
mbean = "java.lang:type=Threading"
|
||||
attribute = "TotalStartedThreadCount,ThreadCount,DaemonThreadCount,PeakThreadCount"
|
||||
|
||||
## This collect number of class loaded/unloaded counts metrics.
|
||||
[[inputs.jolokia.metrics]]
|
||||
name = "class_count"
|
||||
mbean = "java.lang:type=ClassLoading"
|
||||
attribute = "LoadedClassCount,UnloadedClassCount,TotalLoadedClassCount"
|
||||
```
|
||||
#### Plugin arguments:
|
||||
- **context** string: Context root used of jolokia url
|
||||
- **servers** []Server: List of servers
|
||||
+ **name** string: Server's logical name
|
||||
+ **host** string: Server's ip address or hostname
|
||||
+ **port** string: Server's listening port
|
||||
- **metrics** []Metric
|
||||
+ **name** string: Name of the measure
|
||||
+ **jmx** string: Jmx path that identifies mbeans attributes
|
||||
+ **pass** []string: Attributes to retain when collecting values
|
||||
+ **drop** []string: Attributes to drop when collecting values
|
||||
|
||||
#### Description
|
||||
|
||||
The Jolokia plugin collects JVM metrics exposed as MBean's attributes through
|
||||
jolokia REST endpoint. All metrics are collected for each server configured.
|
||||
The Jolokia plugin collects JVM metrics exposed as MBean's attributes through jolokia REST endpoint. All metrics
|
||||
are collected for each server configured.
|
||||
|
||||
See: https://jolokia.org/
|
||||
|
||||
# Measurements:
|
||||
Jolokia plugin produces one measure for each metric configured,
|
||||
adding Server's `jolokia_name`, `jolokia_host` and `jolokia_port` as tags.
|
||||
Jolokia plugin produces one measure for each metric configured, adding Server's `name`, `host` and `port` as tags.
|
||||
|
||||
Given a configuration like:
|
||||
|
||||
```ini
|
||||
[jolokia]
|
||||
|
||||
[[jolokia.servers]]
|
||||
name = "as-service-1"
|
||||
host = "127.0.0.1"
|
||||
port = "8080"
|
||||
|
||||
[[jolokia.servers]]
|
||||
name = "as-service-2"
|
||||
host = "127.0.0.1"
|
||||
port = "8180"
|
||||
|
||||
[[jolokia.metrics]]
|
||||
name = "heap_memory_usage"
|
||||
jmx = "/java.lang:type=Memory/HeapMemoryUsage"
|
||||
pass = ["used", "max"]
|
||||
```
|
||||
|
||||
The collected metrics will be:
|
||||
|
||||
```
|
||||
jolokia_heap_memory_usage name=as-service-1,host=127.0.0.1,port=8080 used=xxx,max=yyy
|
||||
jolokia_heap_memory_usage name=as-service-2,host=127.0.0.1,port=8180 used=vvv,max=zzz
|
||||
```
|
||||
|
||||
@@ -1,14 +1,12 @@
|
||||
package jolokia
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
@@ -23,10 +21,8 @@ type Server struct {
|
||||
}
|
||||
|
||||
type Metric struct {
|
||||
Name string
|
||||
Mbean string
|
||||
Attribute string
|
||||
Path string
|
||||
Name string
|
||||
Jmx string
|
||||
}
|
||||
|
||||
type JolokiaClient interface {
|
||||
@@ -44,32 +40,20 @@ func (c JolokiaClientImpl) MakeRequest(req *http.Request) (*http.Response, error
|
||||
type Jolokia struct {
|
||||
jClient JolokiaClient
|
||||
Context string
|
||||
Mode string
|
||||
Servers []Server
|
||||
Metrics []Metric
|
||||
Proxy Server
|
||||
}
|
||||
|
||||
const sampleConfig = `
|
||||
func (j *Jolokia) SampleConfig() string {
|
||||
return `
|
||||
## This is the context root used to compose the jolokia url
|
||||
context = "/jolokia"
|
||||
|
||||
## This specifies the mode used
|
||||
# mode = "proxy"
|
||||
#
|
||||
## When in proxy mode this section is used to specify further
|
||||
## proxy address configurations.
|
||||
## Remember to change host address to fit your environment.
|
||||
# [inputs.jolokia.proxy]
|
||||
# host = "127.0.0.1"
|
||||
# port = "8080"
|
||||
|
||||
context = "/jolokia/read"
|
||||
|
||||
## List of servers exposing jolokia read service
|
||||
[[inputs.jolokia.servers]]
|
||||
name = "as-server-01"
|
||||
host = "127.0.0.1"
|
||||
port = "8080"
|
||||
name = "stable"
|
||||
host = "192.168.103.2"
|
||||
port = "8180"
|
||||
# username = "myuser"
|
||||
# password = "mypassword"
|
||||
|
||||
@@ -79,31 +63,20 @@ const sampleConfig = `
|
||||
## This collect all heap memory usage metrics.
|
||||
[[inputs.jolokia.metrics]]
|
||||
name = "heap_memory_usage"
|
||||
mbean = "java.lang:type=Memory"
|
||||
attribute = "HeapMemoryUsage"
|
||||
|
||||
## This collect thread counts metrics.
|
||||
[[inputs.jolokia.metrics]]
|
||||
name = "thread_count"
|
||||
mbean = "java.lang:type=Threading"
|
||||
attribute = "TotalStartedThreadCount,ThreadCount,DaemonThreadCount,PeakThreadCount"
|
||||
|
||||
## This collect number of class loaded/unloaded counts metrics.
|
||||
[[inputs.jolokia.metrics]]
|
||||
name = "class_count"
|
||||
mbean = "java.lang:type=ClassLoading"
|
||||
attribute = "LoadedClassCount,UnloadedClassCount,TotalLoadedClassCount"
|
||||
jmx = "/java.lang:type=Memory/HeapMemoryUsage"
|
||||
`
|
||||
|
||||
func (j *Jolokia) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
|
||||
func (j *Jolokia) Description() string {
|
||||
return "Read JMX metrics through Jolokia"
|
||||
}
|
||||
|
||||
func (j *Jolokia) doRequest(req *http.Request) (map[string]interface{}, error) {
|
||||
func (j *Jolokia) getAttr(requestUrl *url.URL) (map[string]interface{}, error) {
|
||||
// Create + send request
|
||||
req, err := http.NewRequest("GET", requestUrl.String(), nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
resp, err := j.jClient.MakeRequest(req)
|
||||
if err != nil {
|
||||
@@ -114,7 +87,7 @@ func (j *Jolokia) doRequest(req *http.Request) (map[string]interface{}, error) {
|
||||
// Process response
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
err = fmt.Errorf("Response from url \"%s\" has status code %d (%s), expected %d (%s)",
|
||||
req.RequestURI,
|
||||
requestUrl,
|
||||
resp.StatusCode,
|
||||
http.StatusText(resp.StatusCode),
|
||||
http.StatusOK,
|
||||
@@ -134,133 +107,51 @@ func (j *Jolokia) doRequest(req *http.Request) (map[string]interface{}, error) {
|
||||
return nil, errors.New("Error decoding JSON response")
|
||||
}
|
||||
|
||||
if status, ok := jsonOut["status"]; ok {
|
||||
if status != float64(200) {
|
||||
return nil, fmt.Errorf("Not expected status value in response body: %3.f",
|
||||
status)
|
||||
}
|
||||
} else {
|
||||
return nil, fmt.Errorf("Missing status in response body")
|
||||
}
|
||||
|
||||
return jsonOut, nil
|
||||
}
|
||||
|
||||
func (j *Jolokia) prepareRequest(server Server, metric Metric) (*http.Request, error) {
|
||||
var jolokiaUrl *url.URL
|
||||
context := j.Context // Usually "/jolokia"
|
||||
|
||||
// Create bodyContent
|
||||
bodyContent := map[string]interface{}{
|
||||
"type": "read",
|
||||
"mbean": metric.Mbean,
|
||||
}
|
||||
|
||||
if metric.Attribute != "" {
|
||||
bodyContent["attribute"] = metric.Attribute
|
||||
if metric.Path != "" {
|
||||
bodyContent["path"] = metric.Path
|
||||
}
|
||||
}
|
||||
|
||||
// Add target, only in proxy mode
|
||||
if j.Mode == "proxy" {
|
||||
serviceUrl := fmt.Sprintf("service:jmx:rmi:///jndi/rmi://%s:%s/jmxrmi",
|
||||
server.Host, server.Port)
|
||||
|
||||
target := map[string]string{
|
||||
"url": serviceUrl,
|
||||
}
|
||||
|
||||
if server.Username != "" {
|
||||
target["user"] = server.Username
|
||||
}
|
||||
|
||||
if server.Password != "" {
|
||||
target["password"] = server.Password
|
||||
}
|
||||
|
||||
bodyContent["target"] = target
|
||||
|
||||
proxy := j.Proxy
|
||||
|
||||
// Prepare ProxyURL
|
||||
proxyUrl, err := url.Parse("http://" + proxy.Host + ":" + proxy.Port + context)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if proxy.Username != "" || proxy.Password != "" {
|
||||
proxyUrl.User = url.UserPassword(proxy.Username, proxy.Password)
|
||||
}
|
||||
|
||||
jolokiaUrl = proxyUrl
|
||||
|
||||
} else {
|
||||
serverUrl, err := url.Parse("http://" + server.Host + ":" + server.Port + context)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if server.Username != "" || server.Password != "" {
|
||||
serverUrl.User = url.UserPassword(server.Username, server.Password)
|
||||
}
|
||||
|
||||
jolokiaUrl = serverUrl
|
||||
}
|
||||
|
||||
requestBody, err := json.Marshal(bodyContent)
|
||||
|
||||
req, err := http.NewRequest("POST", jolokiaUrl.String(), bytes.NewBuffer(requestBody))
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
req.Header.Add("Content-type", "application/json")
|
||||
|
||||
return req, nil
|
||||
}
|
||||
|
||||
func (j *Jolokia) Gather(acc telegraf.Accumulator) error {
|
||||
context := j.Context //"/jolokia/read"
|
||||
servers := j.Servers
|
||||
metrics := j.Metrics
|
||||
tags := make(map[string]string)
|
||||
|
||||
for _, server := range servers {
|
||||
tags["jolokia_name"] = server.Name
|
||||
tags["jolokia_port"] = server.Port
|
||||
tags["jolokia_host"] = server.Host
|
||||
tags["server"] = server.Name
|
||||
tags["port"] = server.Port
|
||||
tags["host"] = server.Host
|
||||
fields := make(map[string]interface{})
|
||||
|
||||
for _, metric := range metrics {
|
||||
measurement := metric.Name
|
||||
|
||||
req, err := j.prepareRequest(server, metric)
|
||||
measurement := metric.Name
|
||||
jmxPath := metric.Jmx
|
||||
|
||||
// Prepare URL
|
||||
requestUrl, err := url.Parse("http://" + server.Host + ":" +
|
||||
server.Port + context + jmxPath)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if server.Username != "" || server.Password != "" {
|
||||
requestUrl.User = url.UserPassword(server.Username, server.Password)
|
||||
}
|
||||
|
||||
out, err := j.doRequest(req)
|
||||
out, _ := j.getAttr(requestUrl)
|
||||
|
||||
if err != nil {
|
||||
fmt.Printf("Error handling response: %s\n", err)
|
||||
} else {
|
||||
|
||||
if values, ok := out["value"]; ok {
|
||||
switch t := values.(type) {
|
||||
case map[string]interface{}:
|
||||
for k, v := range t {
|
||||
fields[measurement+"_"+k] = v
|
||||
}
|
||||
case interface{}:
|
||||
fields[measurement] = t
|
||||
if values, ok := out["value"]; ok {
|
||||
switch t := values.(type) {
|
||||
case map[string]interface{}:
|
||||
for k, v := range t {
|
||||
fields[measurement+"_"+k] = v
|
||||
}
|
||||
} else {
|
||||
fmt.Printf("Missing key 'value' in output response\n")
|
||||
case interface{}:
|
||||
fields[measurement] = t
|
||||
}
|
||||
|
||||
} else {
|
||||
fmt.Printf("Missing key 'value' in '%s' output response\n",
|
||||
requestUrl.String())
|
||||
}
|
||||
}
|
||||
|
||||
acc.AddFields("jolokia", fields, tags)
|
||||
}
|
||||
|
||||
@@ -269,11 +160,6 @@ func (j *Jolokia) Gather(acc telegraf.Accumulator) error {
|
||||
|
||||
func init() {
|
||||
inputs.Add("jolokia", func() telegraf.Input {
|
||||
tr := &http.Transport{ResponseHeaderTimeout: time.Duration(3 * time.Second)}
|
||||
client := &http.Client{
|
||||
Transport: tr,
|
||||
Timeout: time.Duration(4 * time.Second),
|
||||
}
|
||||
return &Jolokia{jClient: &JolokiaClientImpl{client: client}}
|
||||
return &Jolokia{jClient: &JolokiaClientImpl{client: &http.Client{}}}
|
||||
})
|
||||
}
|
||||
|
||||
@@ -47,10 +47,8 @@ const invalidJSON = "I don't think this is JSON"
|
||||
const empty = ""
|
||||
|
||||
var Servers = []Server{Server{Name: "as1", Host: "127.0.0.1", Port: "8080"}}
|
||||
var HeapMetric = Metric{Name: "heap_memory_usage",
|
||||
Mbean: "java.lang:type=Memory", Attribute: "HeapMemoryUsage"}
|
||||
var UsedHeapMetric = Metric{Name: "heap_memory_usage",
|
||||
Mbean: "java.lang:type=Memory", Attribute: "HeapMemoryUsage"}
|
||||
var HeapMetric = Metric{Name: "heap_memory_usage", Jmx: "/java.lang:type=Memory/HeapMemoryUsage"}
|
||||
var UsedHeapMetric = Metric{Name: "heap_memory_usage", Jmx: "/java.lang:type=Memory/HeapMemoryUsage"}
|
||||
|
||||
type jolokiaClientStub struct {
|
||||
responseBody string
|
||||
@@ -96,9 +94,9 @@ func TestHttpJsonMultiValue(t *testing.T) {
|
||||
"heap_memory_usage_used": 203288528.0,
|
||||
}
|
||||
tags := map[string]string{
|
||||
"jolokia_host": "127.0.0.1",
|
||||
"jolokia_port": "8080",
|
||||
"jolokia_name": "as1",
|
||||
"host": "127.0.0.1",
|
||||
"port": "8080",
|
||||
"server": "as1",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "jolokia", fields, tags)
|
||||
}
|
||||
@@ -116,17 +114,3 @@ func TestHttpJsonOn404(t *testing.T) {
|
||||
assert.Nil(t, err)
|
||||
assert.Equal(t, 0, len(acc.Metrics))
|
||||
}
|
||||
|
||||
// Test that the proper values are ignored or collected
|
||||
func TestHttpInvalidJson(t *testing.T) {
|
||||
|
||||
jolokia := genJolokiaClientStub(invalidJSON, 200, Servers,
|
||||
[]Metric{UsedHeapMetric})
|
||||
|
||||
var acc testutil.Accumulator
|
||||
acc.SetDebug(true)
|
||||
err := jolokia.Gather(&acc)
|
||||
|
||||
assert.Nil(t, err)
|
||||
assert.Equal(t, 0, len(acc.Metrics))
|
||||
}
|
||||
|
||||
@@ -22,8 +22,7 @@ from the same topic in parallel.
|
||||
## Offset (must be either "oldest" or "newest")
|
||||
offset = "oldest"
|
||||
|
||||
## Data format to consume.
|
||||
|
||||
## Data format to consume. This can be "json", "influx" or "graphite"
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
|
||||
|
||||
@@ -14,11 +14,10 @@ import (
|
||||
)
|
||||
|
||||
type Kafka struct {
|
||||
ConsumerGroup string
|
||||
Topics []string
|
||||
ZookeeperPeers []string
|
||||
ZookeeperChroot string
|
||||
Consumer *consumergroup.ConsumerGroup
|
||||
ConsumerGroup string
|
||||
Topics []string
|
||||
ZookeeperPeers []string
|
||||
Consumer *consumergroup.ConsumerGroup
|
||||
|
||||
// Legacy metric buffer support
|
||||
MetricBuffer int
|
||||
@@ -49,14 +48,12 @@ var sampleConfig = `
|
||||
topics = ["telegraf"]
|
||||
## an array of Zookeeper connection strings
|
||||
zookeeper_peers = ["localhost:2181"]
|
||||
## Zookeeper Chroot
|
||||
zookeeper_chroot = "/"
|
||||
## the name of the consumer group
|
||||
consumer_group = "telegraf_metrics_consumers"
|
||||
## Offset (must be either "oldest" or "newest")
|
||||
offset = "oldest"
|
||||
|
||||
## Data format to consume.
|
||||
## Data format to consume. This can be "json", "influx" or "graphite"
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
|
||||
@@ -83,7 +80,6 @@ func (k *Kafka) Start(acc telegraf.Accumulator) error {
|
||||
k.acc = acc
|
||||
|
||||
config := consumergroup.NewConfig()
|
||||
config.Zookeeper.Chroot = k.ZookeeperChroot
|
||||
switch strings.ToLower(k.Offset) {
|
||||
case "oldest", "":
|
||||
config.Offsets.Initial = sarama.OffsetOldest
|
||||
|
||||
@@ -3,16 +3,13 @@ package leofs
|
||||
import (
|
||||
"bufio"
|
||||
"fmt"
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"net/url"
|
||||
"os/exec"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
const oid = ".1.3.6.1.4.1.35450"
|
||||
@@ -178,18 +175,14 @@ func (l *LeoFS) Gather(acc telegraf.Accumulator) error {
|
||||
return outerr
|
||||
}
|
||||
|
||||
func (l *LeoFS) gatherServer(
|
||||
endpoint string,
|
||||
serverType ServerType,
|
||||
acc telegraf.Accumulator,
|
||||
) error {
|
||||
func (l *LeoFS) gatherServer(endpoint string, serverType ServerType, acc telegraf.Accumulator) error {
|
||||
cmd := exec.Command("snmpwalk", "-v2c", "-cpublic", endpoint, oid)
|
||||
stdout, err := cmd.StdoutPipe()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
cmd.Start()
|
||||
defer internal.WaitTimeout(cmd, time.Second*5)
|
||||
defer cmd.Wait()
|
||||
scanner := bufio.NewScanner(stdout)
|
||||
if !scanner.Scan() {
|
||||
return fmt.Errorf("Unable to retrieve the node name")
|
||||
|
||||
@@ -34,13 +34,9 @@ var sampleConfig = `
|
||||
##
|
||||
# ost_procfiles = [
|
||||
# "/proc/fs/lustre/obdfilter/*/stats",
|
||||
# "/proc/fs/lustre/osd-ldiskfs/*/stats",
|
||||
# "/proc/fs/lustre/obdfilter/*/job_stats",
|
||||
# ]
|
||||
# mds_procfiles = [
|
||||
# "/proc/fs/lustre/mdt/*/md_stats",
|
||||
# "/proc/fs/lustre/mdt/*/job_stats",
|
||||
# "/proc/fs/lustre/osd-ldiskfs/*/stats"
|
||||
# ]
|
||||
# mds_procfiles = ["/proc/fs/lustre/mdt/*/md_stats"]
|
||||
`
|
||||
|
||||
/* The wanted fields would be a []string if not for the
|
||||
@@ -86,139 +82,6 @@ var wanted_ost_fields = []*mapping{
|
||||
},
|
||||
}
|
||||
|
||||
var wanted_ost_jobstats_fields = []*mapping{
|
||||
{ // The read line has several fields, so we need to differentiate what they are
|
||||
inProc: "read",
|
||||
field: 3,
|
||||
reportAs: "jobstats_read_calls",
|
||||
},
|
||||
{
|
||||
inProc: "read",
|
||||
field: 7,
|
||||
reportAs: "jobstats_read_min_size",
|
||||
},
|
||||
{
|
||||
inProc: "read",
|
||||
field: 9,
|
||||
reportAs: "jobstats_read_max_size",
|
||||
},
|
||||
{
|
||||
inProc: "read",
|
||||
field: 11,
|
||||
reportAs: "jobstats_read_bytes",
|
||||
},
|
||||
{ // Different inProc for newer versions
|
||||
inProc: "read_bytes",
|
||||
field: 3,
|
||||
reportAs: "jobstats_read_calls",
|
||||
},
|
||||
{
|
||||
inProc: "read_bytes",
|
||||
field: 7,
|
||||
reportAs: "jobstats_read_min_size",
|
||||
},
|
||||
{
|
||||
inProc: "read_bytes",
|
||||
field: 9,
|
||||
reportAs: "jobstats_read_max_size",
|
||||
},
|
||||
{
|
||||
inProc: "read_bytes",
|
||||
field: 11,
|
||||
reportAs: "jobstats_read_bytes",
|
||||
},
|
||||
{ // We need to do the same for the write fields
|
||||
inProc: "write",
|
||||
field: 3,
|
||||
reportAs: "jobstats_write_calls",
|
||||
},
|
||||
{
|
||||
inProc: "write",
|
||||
field: 7,
|
||||
reportAs: "jobstats_write_min_size",
|
||||
},
|
||||
{
|
||||
inProc: "write",
|
||||
field: 9,
|
||||
reportAs: "jobstats_write_max_size",
|
||||
},
|
||||
{
|
||||
inProc: "write",
|
||||
field: 11,
|
||||
reportAs: "jobstats_write_bytes",
|
||||
},
|
||||
{ // Different inProc for newer versions
|
||||
inProc: "write_bytes",
|
||||
field: 3,
|
||||
reportAs: "jobstats_write_calls",
|
||||
},
|
||||
{
|
||||
inProc: "write_bytes",
|
||||
field: 7,
|
||||
reportAs: "jobstats_write_min_size",
|
||||
},
|
||||
{
|
||||
inProc: "write_bytes",
|
||||
field: 9,
|
||||
reportAs: "jobstats_write_max_size",
|
||||
},
|
||||
{
|
||||
inProc: "write_bytes",
|
||||
field: 11,
|
||||
reportAs: "jobstats_write_bytes",
|
||||
},
|
||||
{
|
||||
inProc: "getattr",
|
||||
field: 3,
|
||||
reportAs: "jobstats_ost_getattr",
|
||||
},
|
||||
{
|
||||
inProc: "setattr",
|
||||
field: 3,
|
||||
reportAs: "jobstats_ost_setattr",
|
||||
},
|
||||
{
|
||||
inProc: "punch",
|
||||
field: 3,
|
||||
reportAs: "jobstats_punch",
|
||||
},
|
||||
{
|
||||
inProc: "sync",
|
||||
field: 3,
|
||||
reportAs: "jobstats_ost_sync",
|
||||
},
|
||||
{
|
||||
inProc: "destroy",
|
||||
field: 3,
|
||||
reportAs: "jobstats_destroy",
|
||||
},
|
||||
{
|
||||
inProc: "create",
|
||||
field: 3,
|
||||
reportAs: "jobstats_create",
|
||||
},
|
||||
{
|
||||
inProc: "statfs",
|
||||
field: 3,
|
||||
reportAs: "jobstats_ost_statfs",
|
||||
},
|
||||
{
|
||||
inProc: "get_info",
|
||||
field: 3,
|
||||
reportAs: "jobstats_get_info",
|
||||
},
|
||||
{
|
||||
inProc: "set_info",
|
||||
field: 3,
|
||||
reportAs: "jobstats_set_info",
|
||||
},
|
||||
{
|
||||
inProc: "quotactl",
|
||||
field: 3,
|
||||
reportAs: "jobstats_quotactl",
|
||||
},
|
||||
}
|
||||
|
||||
var wanted_mds_fields = []*mapping{
|
||||
{
|
||||
inProc: "open",
|
||||
@@ -270,89 +133,6 @@ var wanted_mds_fields = []*mapping{
|
||||
},
|
||||
}
|
||||
|
||||
var wanted_mdt_jobstats_fields = []*mapping{
|
||||
{
|
||||
inProc: "open",
|
||||
field: 3,
|
||||
reportAs: "jobstats_open",
|
||||
},
|
||||
{
|
||||
inProc: "close",
|
||||
field: 3,
|
||||
reportAs: "jobstats_close",
|
||||
},
|
||||
{
|
||||
inProc: "mknod",
|
||||
field: 3,
|
||||
reportAs: "jobstats_mknod",
|
||||
},
|
||||
{
|
||||
inProc: "link",
|
||||
field: 3,
|
||||
reportAs: "jobstats_link",
|
||||
},
|
||||
{
|
||||
inProc: "unlink",
|
||||
field: 3,
|
||||
reportAs: "jobstats_unlink",
|
||||
},
|
||||
{
|
||||
inProc: "mkdir",
|
||||
field: 3,
|
||||
reportAs: "jobstats_mkdir",
|
||||
},
|
||||
{
|
||||
inProc: "rmdir",
|
||||
field: 3,
|
||||
reportAs: "jobstats_rmdir",
|
||||
},
|
||||
{
|
||||
inProc: "rename",
|
||||
field: 3,
|
||||
reportAs: "jobstats_rename",
|
||||
},
|
||||
{
|
||||
inProc: "getattr",
|
||||
field: 3,
|
||||
reportAs: "jobstats_getattr",
|
||||
},
|
||||
{
|
||||
inProc: "setattr",
|
||||
field: 3,
|
||||
reportAs: "jobstats_setattr",
|
||||
},
|
||||
{
|
||||
inProc: "getxattr",
|
||||
field: 3,
|
||||
reportAs: "jobstats_getxattr",
|
||||
},
|
||||
{
|
||||
inProc: "setxattr",
|
||||
field: 3,
|
||||
reportAs: "jobstats_setxattr",
|
||||
},
|
||||
{
|
||||
inProc: "statfs",
|
||||
field: 3,
|
||||
reportAs: "jobstats_statfs",
|
||||
},
|
||||
{
|
||||
inProc: "sync",
|
||||
field: 3,
|
||||
reportAs: "jobstats_sync",
|
||||
},
|
||||
{
|
||||
inProc: "samedir_rename",
|
||||
field: 3,
|
||||
reportAs: "jobstats_samedir_rename",
|
||||
},
|
||||
{
|
||||
inProc: "crossdir_rename",
|
||||
field: 3,
|
||||
reportAs: "jobstats_crossdir_rename",
|
||||
},
|
||||
}
|
||||
|
||||
func (l *Lustre2) GetLustreProcStats(fileglob string, wanted_fields []*mapping, acc telegraf.Accumulator) error {
|
||||
files, err := filepath.Glob(fileglob)
|
||||
if err != nil {
|
||||
@@ -363,7 +143,7 @@ func (l *Lustre2) GetLustreProcStats(fileglob string, wanted_fields []*mapping,
|
||||
/* Turn /proc/fs/lustre/obdfilter/<ost_name>/stats and similar
|
||||
* into just the object store target name
|
||||
* Assumpion: the target name is always second to last,
|
||||
* which is true in Lustre 2.1->2.8
|
||||
* which is true in Lustre 2.1->2.5
|
||||
*/
|
||||
path := strings.Split(file, "/")
|
||||
name := path[len(path)-2]
|
||||
@@ -381,21 +161,16 @@ func (l *Lustre2) GetLustreProcStats(fileglob string, wanted_fields []*mapping,
|
||||
|
||||
for _, line := range lines {
|
||||
parts := strings.Fields(line)
|
||||
if strings.HasPrefix(line, "- job_id:") {
|
||||
// Set the job_id explicitly if present
|
||||
fields["jobid"] = parts[2]
|
||||
}
|
||||
|
||||
for _, wanted := range wanted_fields {
|
||||
var data uint64
|
||||
if strings.TrimSuffix(parts[0], ":") == wanted.inProc {
|
||||
if parts[0] == wanted.inProc {
|
||||
wanted_field := wanted.field
|
||||
// if not set, assume field[1]. Shouldn't be field[0], as
|
||||
// that's a string
|
||||
if wanted_field == 0 {
|
||||
wanted_field = 1
|
||||
}
|
||||
data, err = strconv.ParseUint(strings.TrimSuffix((parts[wanted_field]), ","), 10, 64)
|
||||
data, err = strconv.ParseUint((parts[wanted_field]), 10, 64)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -438,12 +213,6 @@ func (l *Lustre2) Gather(acc telegraf.Accumulator) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
// per job statistics are in obdfilter/<ost_name>/job_stats
|
||||
err = l.GetLustreProcStats("/proc/fs/lustre/obdfilter/*/job_stats",
|
||||
wanted_ost_jobstats_fields, acc)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
if len(l.Mds_procfiles) == 0 {
|
||||
@@ -453,31 +222,16 @@ func (l *Lustre2) Gather(acc telegraf.Accumulator) error {
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
// Metadata target job stats
|
||||
err = l.GetLustreProcStats("/proc/fs/lustre/mdt/*/job_stats",
|
||||
wanted_mdt_jobstats_fields, acc)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
for _, procfile := range l.Ost_procfiles {
|
||||
ost_fields := wanted_ost_fields
|
||||
if strings.HasSuffix(procfile, "job_stats") {
|
||||
ost_fields = wanted_ost_jobstats_fields
|
||||
}
|
||||
err := l.GetLustreProcStats(procfile, ost_fields, acc)
|
||||
err := l.GetLustreProcStats(procfile, wanted_ost_fields, acc)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
for _, procfile := range l.Mds_procfiles {
|
||||
mdt_fields := wanted_mds_fields
|
||||
if strings.HasSuffix(procfile, "job_stats") {
|
||||
mdt_fields = wanted_mdt_jobstats_fields
|
||||
}
|
||||
err := l.GetLustreProcStats(procfile, mdt_fields, acc)
|
||||
err := l.GetLustreProcStats(procfile, wanted_mds_fields, acc)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -487,12 +241,6 @@ func (l *Lustre2) Gather(acc telegraf.Accumulator) error {
|
||||
tags := map[string]string{
|
||||
"name": name,
|
||||
}
|
||||
if _, ok := fields["jobid"]; ok {
|
||||
if jobid, ok := fields["jobid"].(string); ok {
|
||||
tags["jobid"] = jobid
|
||||
}
|
||||
delete(fields, "jobid")
|
||||
}
|
||||
acc.AddFields("lustre2", fields, tags)
|
||||
}
|
||||
|
||||
|
||||
@@ -38,23 +38,6 @@ cache_hit 7393729777 samples [pages] 1 1 7393729777
|
||||
cache_miss 11653333250 samples [pages] 1 1 11653333250
|
||||
`
|
||||
|
||||
const obdfilterJobStatsContents = `job_stats:
|
||||
- job_id: testjob1
|
||||
snapshot_time: 1461772761
|
||||
read_bytes: { samples: 1, unit: bytes, min: 4096, max: 4096, sum: 4096 }
|
||||
write_bytes: { samples: 25, unit: bytes, min: 1048576, max: 1048576, sum: 26214400 }
|
||||
getattr: { samples: 0, unit: reqs }
|
||||
setattr: { samples: 0, unit: reqs }
|
||||
punch: { samples: 1, unit: reqs }
|
||||
sync: { samples: 0, unit: reqs }
|
||||
destroy: { samples: 0, unit: reqs }
|
||||
create: { samples: 0, unit: reqs }
|
||||
statfs: { samples: 0, unit: reqs }
|
||||
get_info: { samples: 0, unit: reqs }
|
||||
set_info: { samples: 0, unit: reqs }
|
||||
quotactl: { samples: 0, unit: reqs }
|
||||
`
|
||||
|
||||
const mdtProcContents = `snapshot_time 1438693238.20113 secs.usecs
|
||||
open 1024577037 samples [reqs]
|
||||
close 873243496 samples [reqs]
|
||||
@@ -74,27 +57,6 @@ samedir_rename 259625 samples [reqs]
|
||||
crossdir_rename 369571 samples [reqs]
|
||||
`
|
||||
|
||||
const mdtJobStatsContents = `job_stats:
|
||||
- job_id: testjob1
|
||||
snapshot_time: 1461772761
|
||||
open: { samples: 5, unit: reqs }
|
||||
close: { samples: 4, unit: reqs }
|
||||
mknod: { samples: 6, unit: reqs }
|
||||
link: { samples: 8, unit: reqs }
|
||||
unlink: { samples: 90, unit: reqs }
|
||||
mkdir: { samples: 521, unit: reqs }
|
||||
rmdir: { samples: 520, unit: reqs }
|
||||
rename: { samples: 9, unit: reqs }
|
||||
getattr: { samples: 11, unit: reqs }
|
||||
setattr: { samples: 1, unit: reqs }
|
||||
getxattr: { samples: 3, unit: reqs }
|
||||
setxattr: { samples: 4, unit: reqs }
|
||||
statfs: { samples: 1205, unit: reqs }
|
||||
sync: { samples: 2, unit: reqs }
|
||||
samedir_rename: { samples: 705, unit: reqs }
|
||||
crossdir_rename: { samples: 200, unit: reqs }
|
||||
`
|
||||
|
||||
func TestLustre2GeneratesMetrics(t *testing.T) {
|
||||
|
||||
tempdir := os.TempDir() + "/telegraf/proc/fs/lustre/"
|
||||
@@ -121,7 +83,6 @@ func TestLustre2GeneratesMetrics(t *testing.T) {
|
||||
err = ioutil.WriteFile(obddir+"/"+ost_name+"/stats", []byte(obdfilterProcContents), 0644)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Begin by testing standard Lustre stats
|
||||
m := &Lustre2{
|
||||
Ost_procfiles: []string{obddir + "/*/stats", osddir + "/*/stats"},
|
||||
Mds_procfiles: []string{mdtdir + "/*/md_stats"},
|
||||
@@ -167,82 +128,3 @@ func TestLustre2GeneratesMetrics(t *testing.T) {
|
||||
err = os.RemoveAll(os.TempDir() + "/telegraf")
|
||||
require.NoError(t, err)
|
||||
}
|
||||
|
||||
func TestLustre2GeneratesJobstatsMetrics(t *testing.T) {
|
||||
|
||||
tempdir := os.TempDir() + "/telegraf/proc/fs/lustre/"
|
||||
ost_name := "OST0001"
|
||||
job_name := "testjob1"
|
||||
|
||||
mdtdir := tempdir + "/mdt/"
|
||||
err := os.MkdirAll(mdtdir+"/"+ost_name, 0755)
|
||||
require.NoError(t, err)
|
||||
|
||||
obddir := tempdir + "/obdfilter/"
|
||||
err = os.MkdirAll(obddir+"/"+ost_name, 0755)
|
||||
require.NoError(t, err)
|
||||
|
||||
err = ioutil.WriteFile(mdtdir+"/"+ost_name+"/job_stats", []byte(mdtJobStatsContents), 0644)
|
||||
require.NoError(t, err)
|
||||
|
||||
err = ioutil.WriteFile(obddir+"/"+ost_name+"/job_stats", []byte(obdfilterJobStatsContents), 0644)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Test Lustre Jobstats
|
||||
m := &Lustre2{
|
||||
Ost_procfiles: []string{obddir + "/*/job_stats"},
|
||||
Mds_procfiles: []string{mdtdir + "/*/job_stats"},
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
|
||||
err = m.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
tags := map[string]string{
|
||||
"name": ost_name,
|
||||
"jobid": job_name,
|
||||
}
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"jobstats_read_calls": uint64(1),
|
||||
"jobstats_read_min_size": uint64(4096),
|
||||
"jobstats_read_max_size": uint64(4096),
|
||||
"jobstats_read_bytes": uint64(4096),
|
||||
"jobstats_write_calls": uint64(25),
|
||||
"jobstats_write_min_size": uint64(1048576),
|
||||
"jobstats_write_max_size": uint64(1048576),
|
||||
"jobstats_write_bytes": uint64(26214400),
|
||||
"jobstats_ost_getattr": uint64(0),
|
||||
"jobstats_ost_setattr": uint64(0),
|
||||
"jobstats_punch": uint64(1),
|
||||
"jobstats_ost_sync": uint64(0),
|
||||
"jobstats_destroy": uint64(0),
|
||||
"jobstats_create": uint64(0),
|
||||
"jobstats_ost_statfs": uint64(0),
|
||||
"jobstats_get_info": uint64(0),
|
||||
"jobstats_set_info": uint64(0),
|
||||
"jobstats_quotactl": uint64(0),
|
||||
"jobstats_open": uint64(5),
|
||||
"jobstats_close": uint64(4),
|
||||
"jobstats_mknod": uint64(6),
|
||||
"jobstats_link": uint64(8),
|
||||
"jobstats_unlink": uint64(90),
|
||||
"jobstats_mkdir": uint64(521),
|
||||
"jobstats_rmdir": uint64(520),
|
||||
"jobstats_rename": uint64(9),
|
||||
"jobstats_getattr": uint64(11),
|
||||
"jobstats_setattr": uint64(1),
|
||||
"jobstats_getxattr": uint64(3),
|
||||
"jobstats_setxattr": uint64(4),
|
||||
"jobstats_statfs": uint64(1205),
|
||||
"jobstats_sync": uint64(2),
|
||||
"jobstats_samedir_rename": uint64(705),
|
||||
"jobstats_crossdir_rename": uint64(200),
|
||||
}
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "lustre2", fields, tags)
|
||||
|
||||
err = os.RemoveAll(os.TempDir() + "/telegraf")
|
||||
require.NoError(t, err)
|
||||
}
|
||||
|
||||
@@ -10,7 +10,6 @@ import (
|
||||
"net/url"
|
||||
"regexp"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
const (
|
||||
@@ -121,10 +120,7 @@ func (a *ChimpAPI) GetReport(campaignID string) (Report, error) {
|
||||
}
|
||||
|
||||
func runChimp(api *ChimpAPI, params ReportsParams) ([]byte, error) {
|
||||
client := &http.Client{
|
||||
Transport: api.Transport,
|
||||
Timeout: time.Duration(4 * time.Second),
|
||||
}
|
||||
client := &http.Client{Transport: api.Transport}
|
||||
|
||||
var b bytes.Buffer
|
||||
req, err := http.NewRequest("GET", api.url.String(), &b)
|
||||
|
||||
@@ -94,15 +94,14 @@ func (m *Memcached) gatherServer(
|
||||
acc telegraf.Accumulator,
|
||||
) error {
|
||||
var conn net.Conn
|
||||
var err error
|
||||
if unix {
|
||||
conn, err = net.DialTimeout("unix", address, defaultTimeout)
|
||||
conn, err := net.DialTimeout("unix", address, defaultTimeout)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer conn.Close()
|
||||
} else {
|
||||
_, _, err = net.SplitHostPort(address)
|
||||
_, _, err := net.SplitHostPort(address)
|
||||
if err != nil {
|
||||
address = address + ":11211"
|
||||
}
|
||||
@@ -114,10 +113,6 @@ func (m *Memcached) gatherServer(
|
||||
defer conn.Close()
|
||||
}
|
||||
|
||||
if conn == nil {
|
||||
return fmt.Errorf("Failed to create net connection")
|
||||
}
|
||||
|
||||
// Extend connection
|
||||
conn.SetDeadline(time.Now().Add(defaultTimeout))
|
||||
|
||||
|
||||
@@ -10,7 +10,6 @@ import (
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
@@ -34,16 +33,7 @@ var sampleConfig = `
|
||||
# A list of Mesos masters, default value is localhost:5050.
|
||||
masters = ["localhost:5050"]
|
||||
# Metrics groups to be collected, by default, all enabled.
|
||||
master_collections = [
|
||||
"resources",
|
||||
"master",
|
||||
"system",
|
||||
"slaves",
|
||||
"frameworks",
|
||||
"messages",
|
||||
"evqueue",
|
||||
"registrar",
|
||||
]
|
||||
master_collections = ["resources","master","system","slaves","frameworks","messages","evqueue","registrar"]
|
||||
`
|
||||
|
||||
// SampleConfig returns a sample configuration block
|
||||
@@ -271,15 +261,6 @@ func (m *Mesos) removeGroup(j *map[string]interface{}) {
|
||||
}
|
||||
}
|
||||
|
||||
var tr = &http.Transport{
|
||||
ResponseHeaderTimeout: time.Duration(3 * time.Second),
|
||||
}
|
||||
|
||||
var client = &http.Client{
|
||||
Transport: tr,
|
||||
Timeout: time.Duration(4 * time.Second),
|
||||
}
|
||||
|
||||
// This should not belong to the object
|
||||
func (m *Mesos) gatherMetrics(a string, acc telegraf.Accumulator) error {
|
||||
var jsonOut map[string]interface{}
|
||||
@@ -301,7 +282,7 @@ func (m *Mesos) gatherMetrics(a string, acc telegraf.Accumulator) error {
|
||||
|
||||
ts := strconv.Itoa(m.Timeout) + "ms"
|
||||
|
||||
resp, err := client.Get("http://" + a + "/metrics/snapshot?timeout=" + ts)
|
||||
resp, err := http.Get("http://" + a + "/metrics/snapshot?timeout=" + ts)
|
||||
|
||||
if err != nil {
|
||||
return err
|
||||
|
||||
@@ -1,54 +0,0 @@
|
||||
# Telegraf plugin: MongoDB
|
||||
|
||||
#### Configuration
|
||||
|
||||
```toml
|
||||
[[inputs.mongodb]]
|
||||
## An array of URI to gather stats about. Specify an ip or hostname
|
||||
## with optional port add password. ie,
|
||||
## mongodb://user:auth_key@10.10.3.30:27017,
|
||||
## mongodb://10.10.3.33:18832,
|
||||
## 10.0.0.1:10000, etc.
|
||||
servers = ["127.0.0.1:27017"]
|
||||
```
|
||||
|
||||
For authenticated mongodb istances use connection mongdb connection URI
|
||||
|
||||
```toml
|
||||
[[inputs.mongodb]]
|
||||
servers = ["mongodb://username:password@10.XX.XX.XX:27101/mydatabase?authSource=admin"]
|
||||
```
|
||||
This connection uri may be different based on your environement and mongodb
|
||||
setup. If the user doesn't have the required privilege to execute serverStatus
|
||||
command the you will get this error on telegraf
|
||||
|
||||
```
|
||||
Error in input [mongodb]: not authorized on admin to execute command { serverStatus: 1, recordStats: 0 }
|
||||
```
|
||||
|
||||
#### Description
|
||||
|
||||
The telegraf plugin collects mongodb stats exposed by serverStatus and few more
|
||||
and create a single measurement containing values e.g.
|
||||
* active_reads
|
||||
* active_writes
|
||||
* commands_per_sec
|
||||
* deletes_per_sec
|
||||
* flushes_per_sec
|
||||
* getmores_per_sec
|
||||
* inserts_per_sec
|
||||
* net_in_bytes
|
||||
* net_out_bytes
|
||||
* open_connections
|
||||
* percent_cache_dirty
|
||||
* percent_cache_used
|
||||
* queries_per_sec
|
||||
* queued_reads
|
||||
* queued_writes
|
||||
* resident_megabytes
|
||||
* updates_per_sec
|
||||
* vsize_megabytes
|
||||
* ttl_deletes_per_sec
|
||||
* ttl_passes_per_sec
|
||||
* repl_lag
|
||||
* jumbo_chunks (only if mongos or mongo config)
|
||||
@@ -103,7 +103,7 @@ func (m *MongoDB) gatherServer(server *Server, acc telegraf.Accumulator) error {
|
||||
dialAddrs[0], err.Error())
|
||||
}
|
||||
dialInfo.Direct = true
|
||||
dialInfo.Timeout = 5 * time.Second
|
||||
dialInfo.Timeout = time.Duration(10) * time.Second
|
||||
|
||||
if m.Ssl.Enabled {
|
||||
tlsConfig := &tls.Config{}
|
||||
|
||||
@@ -26,24 +26,22 @@ func NewMongodbData(statLine *StatLine, tags map[string]string) *MongodbData {
|
||||
}
|
||||
|
||||
var DefaultStats = map[string]string{
|
||||
"inserts_per_sec": "Insert",
|
||||
"queries_per_sec": "Query",
|
||||
"updates_per_sec": "Update",
|
||||
"deletes_per_sec": "Delete",
|
||||
"getmores_per_sec": "GetMore",
|
||||
"commands_per_sec": "Command",
|
||||
"flushes_per_sec": "Flushes",
|
||||
"vsize_megabytes": "Virtual",
|
||||
"resident_megabytes": "Resident",
|
||||
"queued_reads": "QueuedReaders",
|
||||
"queued_writes": "QueuedWriters",
|
||||
"active_reads": "ActiveReaders",
|
||||
"active_writes": "ActiveWriters",
|
||||
"net_in_bytes": "NetIn",
|
||||
"net_out_bytes": "NetOut",
|
||||
"open_connections": "NumConnections",
|
||||
"ttl_deletes_per_sec": "DeletedDocuments",
|
||||
"ttl_passes_per_sec": "Passes",
|
||||
"inserts_per_sec": "Insert",
|
||||
"queries_per_sec": "Query",
|
||||
"updates_per_sec": "Update",
|
||||
"deletes_per_sec": "Delete",
|
||||
"getmores_per_sec": "GetMore",
|
||||
"commands_per_sec": "Command",
|
||||
"flushes_per_sec": "Flushes",
|
||||
"vsize_megabytes": "Virtual",
|
||||
"resident_megabytes": "Resident",
|
||||
"queued_reads": "QueuedReaders",
|
||||
"queued_writes": "QueuedWriters",
|
||||
"active_reads": "ActiveReaders",
|
||||
"active_writes": "ActiveWriters",
|
||||
"net_in_bytes": "NetIn",
|
||||
"net_out_bytes": "NetOut",
|
||||
"open_connections": "NumConnections",
|
||||
}
|
||||
|
||||
var DefaultReplStats = map[string]string{
|
||||
@@ -54,11 +52,6 @@ var DefaultReplStats = map[string]string{
|
||||
"repl_getmores_per_sec": "GetMoreR",
|
||||
"repl_commands_per_sec": "CommandR",
|
||||
"member_status": "NodeType",
|
||||
"repl_lag": "ReplLag",
|
||||
}
|
||||
|
||||
var DefaultClusterStats = map[string]string{
|
||||
"jumbo_chunks": "JumboChunksCount",
|
||||
}
|
||||
|
||||
var MmapStats = map[string]string{
|
||||
@@ -78,7 +71,6 @@ func (d *MongodbData) AddDefaultStats() {
|
||||
if d.StatLine.NodeType != "" {
|
||||
d.addStat(statLine, DefaultReplStats)
|
||||
}
|
||||
d.addStat(statLine, DefaultClusterStats)
|
||||
if d.StatLine.StorageEngine == "mmapv1" {
|
||||
d.addStat(statLine, MmapStats)
|
||||
} else if d.StatLine.StorageEngine == "wiredTiger" {
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user