Compare commits
162 Commits
0.13.0-rc3
...
1.0.0-beta
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
03d02fa67a | ||
|
|
b58cd78c79 | ||
|
|
dabb6f5466 | ||
|
|
281a4d5500 | ||
|
|
1c2965703d | ||
|
|
5dc4cce157 | ||
|
|
8c7edeb53b | ||
|
|
1d9745ee98 | ||
|
|
2d6c8767f7 | ||
|
|
b4a6d9c647 | ||
|
|
6afe9ceef1 | ||
|
|
704d9ad76c | ||
|
|
300d9adbd0 | ||
|
|
207c5498e7 | ||
|
|
d5e7439343 | ||
|
|
21add2c799 | ||
|
|
4651ab88ad | ||
|
|
53f40063b3 | ||
|
|
97d92bba67 | ||
|
|
bfdd665435 | ||
|
|
821d3fafa6 | ||
|
|
7c9b312cee | ||
|
|
69ab8a645c | ||
|
|
7b550c11cb | ||
|
|
bb4f18ca88 | ||
|
|
6efe91ea9c | ||
|
|
5f0a63f554 | ||
|
|
d14e7536ab | ||
|
|
c873937356 | ||
|
|
e1c3800cd9 | ||
|
|
c046232425 | ||
|
|
2d4864e126 | ||
|
|
048448aa93 | ||
|
|
755b2ec953 | ||
|
|
f62c493c77 | ||
|
|
a6365a6086 | ||
|
|
f7e057ec55 | ||
|
|
30cc00d11b | ||
|
|
d641c42029 | ||
|
|
9c2ca805da | ||
|
|
b0484d8a0c | ||
|
|
5ddd61d2e2 | ||
|
|
50ea7f4a9d | ||
|
|
b18134a4e3 | ||
|
|
7825df4771 | ||
|
|
d6951dacdc | ||
|
|
e603825e37 | ||
|
|
e3448153e1 | ||
|
|
25848c545a | ||
|
|
3098564896 | ||
|
|
4b6f9b93dd | ||
|
|
2beef21231 | ||
|
|
cb3c54a1ae | ||
|
|
d50a1e83ac | ||
|
|
1f10639222 | ||
|
|
af0979cce5 | ||
|
|
5b43901bd8 | ||
|
|
d7efb7a71d | ||
|
|
4d242836ee | ||
|
|
06cb5a041e | ||
|
|
ea2521bf27 | ||
|
|
4cd1f7a104 | ||
|
|
137843b2f6 | ||
|
|
008ed17a79 | ||
|
|
75e6cb9064 | ||
|
|
ad88a9421a | ||
|
|
346deb30a3 | ||
|
|
8c3d7cd145 | ||
|
|
821b30eb92 | ||
|
|
a362352587 | ||
|
|
94f952787f | ||
|
|
3ff184c061 | ||
|
|
80368e3936 | ||
|
|
2c448e22e1 | ||
|
|
1aabd38eb2 | ||
|
|
675457873a | ||
|
|
8173338f8a | ||
|
|
c4841843a9 | ||
|
|
f08a27be5d | ||
|
|
a4b36d12dd | ||
|
|
c842724b61 | ||
|
|
fb5f40319e | ||
|
|
52b9fc837c | ||
|
|
6f991ec78a | ||
|
|
7921d87a45 | ||
|
|
9f7a758bf9 | ||
|
|
0aff7a0bc1 | ||
|
|
c4cfdb8a25 | ||
|
|
342cfc4087 | ||
|
|
bd1282eddf | ||
|
|
892abec025 | ||
|
|
e809c4e445 | ||
|
|
9ff536d94d | ||
|
|
4f27315720 | ||
|
|
958ef2f872 | ||
|
|
069764f05e | ||
|
|
eeeab5192b | ||
|
|
a7dfbce3d3 | ||
|
|
ed2d1d9bb7 | ||
|
|
0fb2d2ffae | ||
|
|
3af65e7abb | ||
|
|
984b6cb0fb | ||
|
|
ca504a19ec | ||
|
|
c2797c85d1 | ||
|
|
d5add07c0b | ||
|
|
0ebf1c1ad7 | ||
|
|
42d7fc5e16 | ||
|
|
6828fc48e1 | ||
|
|
98d91b1c89 | ||
|
|
9bbdb2d562 | ||
|
|
a8334c3261 | ||
|
|
9144f9630b | ||
|
|
3e4a19539a | ||
|
|
5fe7e6e40e | ||
|
|
58f2ba1247 | ||
|
|
5f3a91bffd | ||
|
|
6351aa5167 | ||
|
|
9966099d1a | ||
|
|
1ef5599361 | ||
|
|
c78b6cdb4e | ||
|
|
d736c7235a | ||
|
|
475252d873 | ||
|
|
e103923430 | ||
|
|
cb59517ceb | ||
|
|
1248934f3e | ||
|
|
204ebf6bf6 | ||
|
|
52d5b19219 | ||
|
|
8e92d3a4a0 | ||
|
|
c44ecf54a5 | ||
|
|
c6699c36d3 | ||
|
|
d6ceae7005 | ||
|
|
4dcb82bf08 | ||
|
|
4f5d5926d9 | ||
|
|
3c5c3b98df | ||
|
|
56aee1ceee | ||
|
|
f176c28a56 | ||
|
|
2e68bd1412 | ||
|
|
35eb65460d | ||
|
|
ab54064689 | ||
|
|
debf7bf149 | ||
|
|
1dbe3b8231 | ||
|
|
b065573e23 | ||
|
|
e94e50181c | ||
|
|
69dfe63809 | ||
|
|
f32916a5bd | ||
|
|
be7ca56872 | ||
|
|
33cacc71b8 | ||
|
|
c292e3931a | ||
|
|
a87d6f0545 | ||
|
|
3a01b6d5b7 | ||
|
|
39df2635bd | ||
|
|
08ecfb8a67 | ||
|
|
a59bf7246a | ||
|
|
281296cd3f | ||
|
|
61d190b1ae | ||
|
|
dc89f029ad | ||
|
|
7557056a31 | ||
|
|
20c45a150c | ||
|
|
46bf0ef271 | ||
|
|
a7b632eb5e | ||
|
|
90a98c76a0 | ||
|
|
12357ee8c5 |
4
.gitattributes
vendored
4
.gitattributes
vendored
@@ -1,2 +1,4 @@
|
||||
CHANGELOG.md merge=union
|
||||
|
||||
README.md merge=union
|
||||
plugins/inputs/all/all.go merge=union
|
||||
plugins/outputs/all/all.go merge=union
|
||||
|
||||
44
.github/ISSUE_TEMPLATE.md
vendored
Normal file
44
.github/ISSUE_TEMPLATE.md
vendored
Normal file
@@ -0,0 +1,44 @@
|
||||
## Directions
|
||||
|
||||
GitHub Issues are reserved for actionable bug reports and feature requests.
|
||||
General questions should be sent to the [InfluxDB mailing list](https://groups.google.com/forum/#!forum/influxdb).
|
||||
|
||||
Before opening an issue, search for similar bug reports or feature requests on GitHub Issues.
|
||||
If no similar issue can be found, fill out either the "Bug Report" or the "Feature Request" section below.
|
||||
Erase the other section and everything on and above this line.
|
||||
|
||||
*Please note, the quickest way to fix a bug is to open a Pull Request.*
|
||||
|
||||
## Bug report
|
||||
|
||||
### Relevant telegraf.conf:
|
||||
|
||||
### System info:
|
||||
|
||||
[Include Telegraf version, operating system name, and other relevant details]
|
||||
|
||||
### Steps to reproduce:
|
||||
|
||||
1. ...
|
||||
2. ...
|
||||
|
||||
### Expected behavior:
|
||||
|
||||
### Actual behavior:
|
||||
|
||||
### Additional info:
|
||||
|
||||
[Include gist of relevant config, logs, etc.]
|
||||
|
||||
|
||||
## Feature Request
|
||||
|
||||
Opening a feature request kicks off a discussion.
|
||||
|
||||
### Proposal:
|
||||
|
||||
### Current behavior:
|
||||
|
||||
### Desired behavior:
|
||||
|
||||
### Use case: [Why is this important (helps with prioritizing requests)]
|
||||
5
.github/PULL_REQUEST_TEMPLATE.md
vendored
Normal file
5
.github/PULL_REQUEST_TEMPLATE.md
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
### Required for all PRs:
|
||||
|
||||
- [ ] CHANGELOG.md updated
|
||||
- [ ] Sign [CLA](https://influxdata.com/community/cla/) (if not already signed)
|
||||
- [ ] README.md updated (if adding a new plugin)
|
||||
1
.gitignore
vendored
1
.gitignore
vendored
@@ -1,3 +1,4 @@
|
||||
build
|
||||
tivan
|
||||
.vagrant
|
||||
/telegraf
|
||||
|
||||
179
CHANGELOG.md
179
CHANGELOG.md
@@ -1,4 +1,181 @@
|
||||
## v0.13 [2016-05-09]
|
||||
## v1.0 [unreleased]
|
||||
|
||||
## v1.0 beta 3 [2016-07-18]
|
||||
|
||||
### Release Notes
|
||||
|
||||
**Breaking Change**: Aerospike main server node measurements have been renamed
|
||||
aerospike_node. Aerospike namespace measurements have been renamed to
|
||||
aerospike_namespace. They will also now be tagged with the node_name
|
||||
that they correspond to. This has been done to differentiate measurements
|
||||
that pertain to node vs. namespace statistics.
|
||||
|
||||
**Breaking Change**: users of github_webhooks must change to the new
|
||||
`[[inputs.webhooks]]` plugin.
|
||||
|
||||
This means that the default github_webhooks config:
|
||||
|
||||
```
|
||||
# A Github Webhook Event collector
|
||||
[[inputs.github_webhooks]]
|
||||
## Address and port to host Webhook listener on
|
||||
service_address = ":1618"
|
||||
```
|
||||
|
||||
should now look like:
|
||||
|
||||
```
|
||||
# A Webhooks Event collector
|
||||
[[inputs.webhooks]]
|
||||
## Address and port to host Webhook listener on
|
||||
service_address = ":1618"
|
||||
|
||||
[inputs.webhooks.github]
|
||||
path = "/"
|
||||
```
|
||||
|
||||
### Features
|
||||
|
||||
- [#1289](https://github.com/influxdata/telegraf/pull/1289): webhooks input plugin. Thanks @francois2metz and @cduez!
|
||||
- [#1247](https://github.com/influxdata/telegraf/pull/1247): rollbar webhook plugin.
|
||||
- [#1408](https://github.com/influxdata/telegraf/pull/1408): mandrill webhook plugin.
|
||||
- [#1402](https://github.com/influxdata/telegraf/pull/1402): docker-machine/boot2docker no longer required for unit tests.
|
||||
- [#1350](https://github.com/influxdata/telegraf/pull/1350): cgroup input plugin.
|
||||
- [#1369](https://github.com/influxdata/telegraf/pull/1369): Add input plugin for consuming metrics from NSQD.
|
||||
- [#1369](https://github.com/influxdata/telegraf/pull/1480): add ability to read redis from a socket.
|
||||
- [#1387](https://github.com/influxdata/telegraf/pull/1387): **Breaking Change** - Redis `role` tag renamed to `replication_role` to avoid global_tags override
|
||||
- [#1437](https://github.com/influxdata/telegraf/pull/1437): Fetching Galera status metrics in MySQL
|
||||
- [#1500](https://github.com/influxdata/telegraf/pull/1500): Aerospike plugin refactored to use official client lib.
|
||||
- [#1434](https://github.com/influxdata/telegraf/pull/1434): Add measurement name arg to logparser plugin.
|
||||
- [#1479](https://github.com/influxdata/telegraf/pull/1479): logparser: change resp_code from a field to a tag.
|
||||
|
||||
### Bugfixes
|
||||
|
||||
- [#1472](https://github.com/influxdata/telegraf/pull/1472): diskio input plugin: set 'skip_serial_number = true' by default to avoid high cardinality.
|
||||
- [#1426](https://github.com/influxdata/telegraf/pull/1426): nil metrics panic fix.
|
||||
- [#1384](https://github.com/influxdata/telegraf/pull/1384): Fix datarace in apache input plugin.
|
||||
- [#1399](https://github.com/influxdata/telegraf/issues/1399): Add `read_repairs` statistics to riak plugin.
|
||||
- [#1405](https://github.com/influxdata/telegraf/issues/1405): Fix memory/connection leak in prometheus input plugin.
|
||||
- [#1378](https://github.com/influxdata/telegraf/issues/1378): Trim BOM from config file for Windows support.
|
||||
- [#1339](https://github.com/influxdata/telegraf/issues/1339): Prometheus client output panic on service reload.
|
||||
- [#1461](https://github.com/influxdata/telegraf/pull/1461): Prometheus parser, protobuf format header fix.
|
||||
- [#1334](https://github.com/influxdata/telegraf/issues/1334): Prometheus output, metric refresh and caching fixes.
|
||||
- [#1432](https://github.com/influxdata/telegraf/issues/1432): Panic fix for multiple graphite outputs under very high load.
|
||||
- [#1412](https://github.com/influxdata/telegraf/pull/1412): Instrumental output has better reconnect behavior
|
||||
- [#1460](https://github.com/influxdata/telegraf/issues/1460): Remove PID from procstat plugin to fix cardinality issues.
|
||||
- [#1427](https://github.com/influxdata/telegraf/issues/1427): Cassandra input: version 2.x "column family" fix.
|
||||
- [#1463](https://github.com/influxdata/telegraf/issues/1463): Shared WaitGroup in Exec plugin
|
||||
- [#1436](https://github.com/influxdata/telegraf/issues/1436): logparser: honor modifiers in "pattern" config.
|
||||
- [#1418](https://github.com/influxdata/telegraf/issues/1418): logparser: error and exit on file permissions/missing errors.
|
||||
|
||||
## v1.0 beta 2 [2016-06-21]
|
||||
|
||||
### Features
|
||||
|
||||
- [#1340](https://github.com/influxdata/telegraf/issues/1340): statsd: do not log every dropped metric.
|
||||
- [#1368](https://github.com/influxdata/telegraf/pull/1368): Add precision rounding to all metrics on collection.
|
||||
- [#1390](https://github.com/influxdata/telegraf/pull/1390): Add support for Tengine
|
||||
- [#1320](https://github.com/influxdata/telegraf/pull/1320): Logparser input plugin for parsing grok-style log patterns.
|
||||
- [#1397](https://github.com/influxdata/telegraf/issues/1397): ElasticSearch: now supports connecting to ElasticSearch via SSL
|
||||
|
||||
### Bugfixes
|
||||
|
||||
- [#1330](https://github.com/influxdata/telegraf/issues/1330): Fix exec plugin panic when using single binary.
|
||||
- [#1336](https://github.com/influxdata/telegraf/issues/1336): Fixed incorrect prometheus metrics source selection.
|
||||
- [#1112](https://github.com/influxdata/telegraf/issues/1112): Set default Zookeeper chroot to empty string.
|
||||
- [#1335](https://github.com/influxdata/telegraf/issues/1335): Fix overall ping timeout to be calculated based on per-ping timeout.
|
||||
- [#1374](https://github.com/influxdata/telegraf/pull/1374): Change "default" retention policy to "".
|
||||
- [#1377](https://github.com/influxdata/telegraf/issues/1377): Graphite output mangling '%' character.
|
||||
- [#1396](https://github.com/influxdata/telegraf/pull/1396): Prometheus input plugin now supports x509 certs authentication
|
||||
|
||||
## v1.0 beta 1 [2016-06-07]
|
||||
|
||||
### Release Notes
|
||||
|
||||
- `flush_jitter` behavior has been changed. The random jitter will now be
|
||||
evaluated at every flush interval, rather than once at startup. This makes it
|
||||
consistent with the behavior of `collection_jitter`.
|
||||
|
||||
- All AWS plugins now utilize a standard mechanism for evaluating credentials.
|
||||
This allows all AWS plugins to support environment variables, shared credential
|
||||
files & profiles, and role assumptions. See the specific plugin README for
|
||||
details.
|
||||
|
||||
- The AWS CloudWatch input plugin can now declare a wildcard value for a metric
|
||||
dimension. This causes the plugin to read all metrics that contain the specified
|
||||
dimension key regardless of value. This is used to export collections of metrics
|
||||
without having to know the dimension values ahead of time.
|
||||
|
||||
- The AWS CloudWatch input plugin can now be configured with the `cache_ttl`
|
||||
attribute. This configures the TTL of the internal metric cache. This is useful
|
||||
in conjunction with wildcard dimension values as it will control the amount of
|
||||
time before a new metric is included by the plugin.
|
||||
|
||||
### Features
|
||||
|
||||
- [#1262](https://github.com/influxdata/telegraf/pull/1261): Add graylog input pluging.
|
||||
- [#1294](https://github.com/influxdata/telegraf/pull/1294): consul input plugin. Thanks @harnash
|
||||
- [#1164](https://github.com/influxdata/telegraf/pull/1164): conntrack input plugin. Thanks @robinpercy!
|
||||
- [#1165](https://github.com/influxdata/telegraf/pull/1165): vmstat input plugin. Thanks @jshim-xm!
|
||||
- [#1208](https://github.com/influxdata/telegraf/pull/1208): Standardized AWS credentials evaluation & wildcard CloudWatch dimensions. Thanks @johnrengelman!
|
||||
- [#1264](https://github.com/influxdata/telegraf/pull/1264): Add SSL config options to http_response plugin.
|
||||
- [#1272](https://github.com/influxdata/telegraf/pull/1272): graphite parser: add ability to specify multiple tag keys, for consistency with influxdb parser.
|
||||
- [#1265](https://github.com/influxdata/telegraf/pull/1265): Make dns lookups for chrony configurable. Thanks @zbindenren!
|
||||
- [#1275](https://github.com/influxdata/telegraf/pull/1275): Allow wildcard filtering of varnish stats.
|
||||
- [#1142](https://github.com/influxdata/telegraf/pull/1142): Support for glob patterns in exec plugin commands configuration.
|
||||
- [#1278](https://github.com/influxdata/telegraf/pull/1278): RabbitMQ input: made url parameter optional by using DefaultURL (http://localhost:15672) if not specified
|
||||
- [#1197](https://github.com/influxdata/telegraf/pull/1197): Limit AWS GetMetricStatistics requests to 10 per second.
|
||||
- [#1278](https://github.com/influxdata/telegraf/pull/1278) & [#1288](https://github.com/influxdata/telegraf/pull/1288) & [#1295](https://github.com/influxdata/telegraf/pull/1295): RabbitMQ/Apache/InfluxDB inputs: made url(s) parameter optional by using reasonable input defaults if not specified
|
||||
- [#1296](https://github.com/influxdata/telegraf/issues/1296): Refactor of flush_jitter argument.
|
||||
- [#1213](https://github.com/influxdata/telegraf/issues/1213): Add inactive & active memory to mem plugin.
|
||||
|
||||
### Bugfixes
|
||||
|
||||
- [#1252](https://github.com/influxdata/telegraf/pull/1252) & [#1279](https://github.com/influxdata/telegraf/pull/1279): Fix systemd service. Thanks @zbindenren & @PierreF!
|
||||
- [#1221](https://github.com/influxdata/telegraf/pull/1221): Fix influxdb n_shards counter.
|
||||
- [#1258](https://github.com/influxdata/telegraf/pull/1258): Fix potential kernel plugin integer parse error.
|
||||
- [#1268](https://github.com/influxdata/telegraf/pull/1268): Fix potential influxdb input type assertion panic.
|
||||
- [#1283](https://github.com/influxdata/telegraf/pull/1283): Still send processes metrics if a process exited during metric collection.
|
||||
- [#1297](https://github.com/influxdata/telegraf/issues/1297): disk plugin panic when usage grab fails.
|
||||
- [#1316](https://github.com/influxdata/telegraf/pull/1316): Removed leaked "database" tag on redis metrics. Thanks @PierreF!
|
||||
- [#1323](https://github.com/influxdata/telegraf/issues/1323): Processes plugin: fix potential error with /proc/net/stat directory.
|
||||
- [#1322](https://github.com/influxdata/telegraf/issues/1322): Fix rare RHEL 5.2 panic in gopsutil diskio gathering function.
|
||||
|
||||
## v0.13.1 [2016-05-24]
|
||||
|
||||
### Release Notes
|
||||
|
||||
- net_response and http_response plugins timeouts will now accept duration
|
||||
strings, ie, "2s" or "500ms".
|
||||
- Input plugin Gathers will no longer be logged by default, but a Gather for
|
||||
_each_ plugin will be logged in Debug mode.
|
||||
- Debug mode will no longer print every point added to the accumulator. This
|
||||
functionality can be duplicated using the `file` output plugin and printing
|
||||
to "stdout".
|
||||
|
||||
### Features
|
||||
|
||||
- [#1173](https://github.com/influxdata/telegraf/pull/1173): varnish input plugin. Thanks @sfox-xmatters!
|
||||
- [#1138](https://github.com/influxdata/telegraf/pull/1138): nstat input plugin. Thanks @Maksadbek!
|
||||
- [#1139](https://github.com/influxdata/telegraf/pull/1139): instrumental output plugin. Thanks @jasonroelofs!
|
||||
- [#1172](https://github.com/influxdata/telegraf/pull/1172): Ceph storage stats. Thanks @robinpercy!
|
||||
- [#1233](https://github.com/influxdata/telegraf/pull/1233): Updated golint gopsutil dependency.
|
||||
- [#1238](https://github.com/influxdata/telegraf/pull/1238): chrony input plugin. Thanks @zbindenren!
|
||||
- [#479](https://github.com/influxdata/telegraf/issues/479): per-plugin execution time added to debug output.
|
||||
- [#1249](https://github.com/influxdata/telegraf/issues/1249): influxdb output: added write_consistency argument.
|
||||
|
||||
### Bugfixes
|
||||
|
||||
- [#1195](https://github.com/influxdata/telegraf/pull/1195): Docker panic on timeout. Thanks @zstyblik!
|
||||
- [#1211](https://github.com/influxdata/telegraf/pull/1211): mongodb input. Fix possible panic. Thanks @kols!
|
||||
- [#1215](https://github.com/influxdata/telegraf/pull/1215): Fix for possible gopsutil-dependent plugin hangs.
|
||||
- [#1228](https://github.com/influxdata/telegraf/pull/1228): Fix service plugin host tag overwrite.
|
||||
- [#1198](https://github.com/influxdata/telegraf/pull/1198): http_response: override request Host header properly
|
||||
- [#1230](https://github.com/influxdata/telegraf/issues/1230): Fix Telegraf process hangup due to a single plugin hanging.
|
||||
- [#1214](https://github.com/influxdata/telegraf/issues/1214): Use TCP timeout argument in net_response plugin.
|
||||
- [#1243](https://github.com/influxdata/telegraf/pull/1243): Logfile not created on systemd.
|
||||
|
||||
## v0.13 [2016-05-11]
|
||||
|
||||
### Release Notes
|
||||
|
||||
|
||||
@@ -114,7 +114,7 @@ creating the `Parser` object.
|
||||
You should also add the following to your SampleConfig() return:
|
||||
|
||||
```toml
|
||||
## Data format to consume.
|
||||
## Data format to consume.
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
|
||||
@@ -212,8 +212,8 @@ func (s *Simple) Close() error {
|
||||
}
|
||||
|
||||
func (s *Simple) Write(metrics []telegraf.Metric) error {
|
||||
for _, pt := range points {
|
||||
// write `pt` to the output sink here
|
||||
for _, metric := range metrics {
|
||||
// write `metric` to the output sink here
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -244,7 +244,7 @@ instantiating and creating the `Serializer` object.
|
||||
You should also add the following to your SampleConfig() return:
|
||||
|
||||
```toml
|
||||
## Data format to output.
|
||||
## Data format to output.
|
||||
## Each data format has it's own unique set of configuration options, read
|
||||
## more about them here:
|
||||
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
|
||||
@@ -290,10 +290,6 @@ To execute Telegraf tests follow these simple steps:
|
||||
instructions
|
||||
- execute `make test`
|
||||
|
||||
**OSX users**: you will need to install `boot2docker` or `docker-machine`.
|
||||
The Makefile will assume that you have a `docker-machine` box called `default` to
|
||||
get the IP address.
|
||||
|
||||
### Unit test troubleshooting
|
||||
|
||||
Try cleaning up your test environment by executing `make docker-kill` and
|
||||
|
||||
9
Godeps
9
Godeps
@@ -1,5 +1,6 @@
|
||||
github.com/Shopify/sarama 8aadb476e66ca998f2f6bb3c993e9a2daa3666b9
|
||||
github.com/Sirupsen/logrus 219c8cb75c258c552e999735be6df753ffc7afdc
|
||||
github.com/aerospike/aerospike-client-go 45863b7fd8640dc12f7fdd397104d97e1986f25a
|
||||
github.com/amir/raidman 53c1b967405155bfc8758557863bf2e14f814687
|
||||
github.com/aws/aws-sdk-go 13a12060f716145019378a10e2806c174356b857
|
||||
github.com/beorn7/perks 3ac7bf7a47d159a033b107610db8a1b6575507a4
|
||||
@@ -16,13 +17,14 @@ github.com/eapache/go-resiliency b86b1ec0dd4209a588dc1285cdd471e73525c0b3
|
||||
github.com/eapache/queue ded5959c0d4e360646dc9e9908cff48666781367
|
||||
github.com/eclipse/paho.mqtt.golang 0f7a459f04f13a41b7ed752d47944528d4bf9a86
|
||||
github.com/go-sql-driver/mysql 1fca743146605a172a266e1654e01e5cd5669bee
|
||||
github.com/gobwas/glob d877f6352135181470c40c73ebb81aefa22115fa
|
||||
github.com/gobwas/glob 49571a1557cd20e6a2410adc6421f85b66c730b5
|
||||
github.com/golang/protobuf 552c7b9542c194800fd493123b3798ef0a832032
|
||||
github.com/golang/snappy 427fb6fc07997f43afa32f35e850833760e489a7
|
||||
github.com/gonuts/go-shellquote e842a11b24c6abfb3dd27af69a17f482e4b483c2
|
||||
github.com/gorilla/context 1ea25387ff6f684839d82767c1733ff4d4d15d0a
|
||||
github.com/gorilla/mux c9e326e2bdec29039a3761c07bece13133863e1e
|
||||
github.com/hailocab/go-hostpool e80d13ce29ede4452c43dea11e79b9bc8a15b478
|
||||
github.com/hashicorp/consul 5aa90455ce78d4d41578bafc86305e6e6b28d7d2
|
||||
github.com/hpcloud/tail b2940955ab8b26e19d43a43c4da0475dd81bdb56
|
||||
github.com/influxdata/config b79f6829346b8d6e78ba73544b1e1038f1f1c9da
|
||||
github.com/influxdata/influxdb e094138084855d444195b252314dfee9eae34cab
|
||||
@@ -42,12 +44,15 @@ github.com/prometheus/client_model fa8ad6fec33561be4280a8f0514318c79d7f6cb6
|
||||
github.com/prometheus/common e8eabff8812b05acf522b45fdcd725a785188e37
|
||||
github.com/prometheus/procfs 406e5b7bfd8201a36e2bb5f7bdae0b03380c2ce8
|
||||
github.com/samuel/go-zookeeper 218e9c81c0dd8b3b18172b2bbfad92cc7d6db55f
|
||||
github.com/shirou/gopsutil 37d89088411de59a4ef9fc340afa0e89dfcb4ea9
|
||||
github.com/shirou/gopsutil 586bb697f3ec9f8ec08ffefe18f521a64534037c
|
||||
github.com/soniah/gosnmp b1b4f885b12c5dcbd021c5cee1c904110de6db7d
|
||||
github.com/sparrc/aerospike-client-go d4bb42d2c2d39dae68e054116f4538af189e05d5
|
||||
github.com/streadway/amqp b4f3ceab0337f013208d31348b578d83c0064744
|
||||
github.com/stretchr/testify 1f4a1643a57e798696635ea4c126e9127adb7d3c
|
||||
github.com/vjeantet/grok 83bfdfdfd1a8146795b28e547a8e3c8b28a466c2
|
||||
github.com/wvanbergen/kafka 46f9a1cf3f670edec492029fadded9c2d9e18866
|
||||
github.com/wvanbergen/kazoo-go 0f768712ae6f76454f987c3356177e138df258f8
|
||||
github.com/yuin/gopher-lua bf3808abd44b1e55143a2d7f08571aaa80db1808
|
||||
github.com/zensqlmonitor/go-mssqldb ffe5510c6fa5e15e6d983210ab501c815b56b363
|
||||
golang.org/x/crypto 5dc8cb4b8a8eb076cbb5a06bc3b8682c15bdbbd3
|
||||
golang.org/x/net 6acef71eb69611914f7a30939ea9f6e194c78172
|
||||
|
||||
24
Makefile
24
Makefile
@@ -1,4 +1,3 @@
|
||||
UNAME := $(shell sh -c 'uname')
|
||||
VERSION := $(shell sh -c 'git describe --always --tags')
|
||||
ifdef GOBIN
|
||||
PATH := $(GOBIN):$(PATH)
|
||||
@@ -26,10 +25,6 @@ build-for-docker:
|
||||
"-s -X main.version=$(VERSION)" \
|
||||
./cmd/telegraf/telegraf.go
|
||||
|
||||
# Build with race detector
|
||||
dev: prepare
|
||||
go build -race -ldflags "-X main.version=$(VERSION)" ./...
|
||||
|
||||
# run package script
|
||||
package:
|
||||
./scripts/build.py --package --version="$(VERSION)" --platform=linux --arch=all --upload
|
||||
@@ -46,27 +41,17 @@ prepare-windows:
|
||||
|
||||
# Run all docker containers necessary for unit tests
|
||||
docker-run:
|
||||
ifeq ($(UNAME), Darwin)
|
||||
docker run --name kafka \
|
||||
-e ADVERTISED_HOST=$(shell sh -c 'boot2docker ip || docker-machine ip default') \
|
||||
-e ADVERTISED_PORT=9092 \
|
||||
-p "2181:2181" -p "9092:9092" \
|
||||
-d spotify/kafka
|
||||
endif
|
||||
ifeq ($(UNAME), Linux)
|
||||
docker run --name kafka \
|
||||
-e ADVERTISED_HOST=localhost \
|
||||
-e ADVERTISED_PORT=9092 \
|
||||
-p "2181:2181" -p "9092:9092" \
|
||||
-d spotify/kafka
|
||||
endif
|
||||
docker run --name mysql -p "3306:3306" -e MYSQL_ALLOW_EMPTY_PASSWORD=yes -d mysql
|
||||
docker run --name memcached -p "11211:11211" -d memcached
|
||||
docker run --name postgres -p "5432:5432" -d postgres
|
||||
docker run --name rabbitmq -p "15672:15672" -p "5672:5672" -d rabbitmq:3-management
|
||||
docker run --name opentsdb -p "4242:4242" -d petergrace/opentsdb-docker
|
||||
docker run --name redis -p "6379:6379" -d redis
|
||||
docker run --name aerospike -p "3000:3000" -d aerospike
|
||||
docker run --name aerospike -p "3000:3000" -d aerospike/aerospike-server
|
||||
docker run --name nsq -p "4150:4150" -d nsqio/nsq /nsqd
|
||||
docker run --name mqtt -p "1883:1883" -d ncarlier/mqtt
|
||||
docker run --name riemann -p "5555:5555" -d blalor/riemann
|
||||
@@ -79,8 +64,7 @@ docker-run-circle:
|
||||
-e ADVERTISED_PORT=9092 \
|
||||
-p "2181:2181" -p "9092:9092" \
|
||||
-d spotify/kafka
|
||||
docker run --name opentsdb -p "4242:4242" -d petergrace/opentsdb-docker
|
||||
docker run --name aerospike -p "3000:3000" -d aerospike
|
||||
docker run --name aerospike -p "3000:3000" -d aerospike/aerospike-server
|
||||
docker run --name nsq -p "4150:4150" -d nsqio/nsq /nsqd
|
||||
docker run --name mqtt -p "1883:1883" -d ncarlier/mqtt
|
||||
docker run --name riemann -p "5555:5555" -d blalor/riemann
|
||||
@@ -88,8 +72,8 @@ docker-run-circle:
|
||||
|
||||
# Kill all docker containers, ignore errors
|
||||
docker-kill:
|
||||
-docker kill nsq aerospike redis opentsdb rabbitmq postgres memcached mysql kafka mqtt riemann snmp
|
||||
-docker rm nsq aerospike redis opentsdb rabbitmq postgres memcached mysql kafka mqtt riemann snmp
|
||||
-docker kill nsq aerospike redis rabbitmq postgres memcached mysql kafka mqtt riemann snmp
|
||||
-docker rm nsq aerospike redis rabbitmq postgres memcached mysql kafka mqtt riemann snmp
|
||||
|
||||
# Run full unit tests using docker containers (includes setup and teardown)
|
||||
test: vet docker-kill docker-run
|
||||
|
||||
55
README.md
55
README.md
@@ -1,4 +1,4 @@
|
||||
# Telegraf [](https://circleci.com/gh/influxdata/telegraf)
|
||||
# Telegraf [](https://circleci.com/gh/influxdata/telegraf) [](https://hub.docker.com/_/telegraf/)
|
||||
|
||||
Telegraf is an agent written in Go for collecting metrics from the system it's
|
||||
running on, or from other services, and writing them into InfluxDB or other
|
||||
@@ -20,12 +20,12 @@ new plugins.
|
||||
### Linux deb and rpm Packages:
|
||||
|
||||
Latest:
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf_0.13.0-1_amd64.deb
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1.x86_64.rpm
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf_1.0.0-beta3_amd64.deb
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-1.0.0_beta3.x86_64.rpm
|
||||
|
||||
Latest (arm):
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf_0.13.0-1_armhf.deb
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1.armhf.rpm
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf_1.0.0-beta3_armhf.deb
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-1.0.0_beta3.armhf.rpm
|
||||
|
||||
##### Package Instructions:
|
||||
|
||||
@@ -46,32 +46,14 @@ to use this repo to install & update telegraf.
|
||||
### Linux tarballs:
|
||||
|
||||
Latest:
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1_linux_amd64.tar.gz
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1_linux_i386.tar.gz
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1_linux_armhf.tar.gz
|
||||
|
||||
##### tarball Instructions:
|
||||
|
||||
To install the full directory structure with config file, run:
|
||||
|
||||
```
|
||||
sudo tar -C / -zxvf ./telegraf-0.13.0-1_linux_amd64.tar.gz
|
||||
```
|
||||
|
||||
To extract only the binary, run:
|
||||
|
||||
```
|
||||
tar -zxvf telegraf-0.13.0-1_linux_amd64.tar.gz --strip-components=3 ./usr/bin/telegraf
|
||||
```
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-1.0.0-beta3_linux_amd64.tar.gz
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-1.0.0-beta3_linux_i386.tar.gz
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-1.0.0-beta3_linux_armhf.tar.gz
|
||||
|
||||
### FreeBSD tarball:
|
||||
|
||||
Latest:
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1_freebsd_amd64.tar.gz
|
||||
|
||||
##### tarball Instructions:
|
||||
|
||||
See linux instructions above.
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-1.0.0-beta3_freebsd_amd64.tar.gz
|
||||
|
||||
### Ansible Role:
|
||||
|
||||
@@ -87,8 +69,7 @@ brew install telegraf
|
||||
### Windows Binaries (EXPERIMENTAL)
|
||||
|
||||
Latest:
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1_windows_amd64.zip
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-0.13.0-1_windows_i386.zip
|
||||
* https://dl.influxdata.com/telegraf/releases/telegraf-1.0.0-beta3_windows_amd64.zip
|
||||
|
||||
### From Source:
|
||||
|
||||
@@ -161,6 +142,10 @@ Currently implemented sources:
|
||||
* [apache](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/apache)
|
||||
* [bcache](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/bcache)
|
||||
* [cassandra](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/cassandra)
|
||||
* [ceph](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/ceph)
|
||||
* [chrony](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/chrony)
|
||||
* [consul](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/consul)
|
||||
* [conntrack](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/conntrack)
|
||||
* [couchbase](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/couchbase)
|
||||
* [couchdb](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/couchdb)
|
||||
* [disque](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/disque)
|
||||
@@ -186,6 +171,7 @@ Currently implemented sources:
|
||||
* [net_response](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/net_response)
|
||||
* [nginx](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nginx)
|
||||
* [nsq](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nsq)
|
||||
* [nstat](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nstat)
|
||||
* [ntpq](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/ntpq)
|
||||
* [phpfpm](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/phpfpm)
|
||||
* [phusion passenger](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/passenger)
|
||||
@@ -205,6 +191,7 @@ Currently implemented sources:
|
||||
* [snmp](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/snmp)
|
||||
* [sql server](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/sqlserver) (microsoft)
|
||||
* [twemproxy](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/twemproxy)
|
||||
* [varnish](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/varnish)
|
||||
* [zfs](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/zfs)
|
||||
* [zookeeper](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/zookeeper)
|
||||
* [win_perf_counters ](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/win_perf_counters) (windows performance counters)
|
||||
@@ -219,16 +206,22 @@ Currently implemented sources:
|
||||
* swap
|
||||
* processes
|
||||
* kernel (/proc/stat)
|
||||
* kernel (/proc/vmstat)
|
||||
|
||||
Telegraf can also collect metrics via the following service plugins:
|
||||
|
||||
* [statsd](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/statsd)
|
||||
* [tail](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/tail)
|
||||
* [udp_listener](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/udp_listener)
|
||||
* [tcp_listener](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/tcp_listener)
|
||||
* [mqtt_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/mqtt_consumer)
|
||||
* [kafka_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/kafka_consumer)
|
||||
* [nats_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nats_consumer)
|
||||
* [github_webhooks](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/github_webhooks)
|
||||
* [webhooks](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/webhooks)
|
||||
* [github](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/webhooks/github)
|
||||
* [mandrill](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/webhooks/mandrill)
|
||||
* [rollbar](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/webhooks/rollbar)
|
||||
* [nsq_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nsq_consumer)
|
||||
|
||||
We'll be adding support for many more over the coming months. Read on if you
|
||||
want to add support for another service or third-party API.
|
||||
@@ -243,6 +236,8 @@ want to add support for another service or third-party API.
|
||||
* [datadog](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/datadog)
|
||||
* [file](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/file)
|
||||
* [graphite](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/graphite)
|
||||
* [graylog](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/graylog)
|
||||
* [instrumental](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/instrumental)
|
||||
* [kafka](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/kafka)
|
||||
* [librato](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/librato)
|
||||
* [mqtt](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/mqtt)
|
||||
|
||||
@@ -18,4 +18,8 @@ type Accumulator interface {
|
||||
|
||||
Debug() bool
|
||||
SetDebug(enabled bool)
|
||||
|
||||
SetPrecision(precision, interval time.Duration)
|
||||
|
||||
DisablePrecision()
|
||||
}
|
||||
|
||||
@@ -4,7 +4,6 @@ import (
|
||||
"fmt"
|
||||
"log"
|
||||
"math"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
@@ -18,21 +17,24 @@ func NewAccumulator(
|
||||
acc := accumulator{}
|
||||
acc.metrics = metrics
|
||||
acc.inputConfig = inputConfig
|
||||
acc.precision = time.Nanosecond
|
||||
return &acc
|
||||
}
|
||||
|
||||
type accumulator struct {
|
||||
sync.Mutex
|
||||
|
||||
metrics chan telegraf.Metric
|
||||
|
||||
defaultTags map[string]string
|
||||
|
||||
debug bool
|
||||
// print every point added to the accumulator
|
||||
trace bool
|
||||
|
||||
inputConfig *internal_models.InputConfig
|
||||
|
||||
prefix string
|
||||
|
||||
precision time.Duration
|
||||
}
|
||||
|
||||
func (ac *accumulator) Add(
|
||||
@@ -84,13 +86,17 @@ func (ac *accumulator) AddFields(
|
||||
if tags == nil {
|
||||
tags = make(map[string]string)
|
||||
}
|
||||
// Apply daemon-wide tags if set
|
||||
for k, v := range ac.defaultTags {
|
||||
tags[k] = v
|
||||
}
|
||||
// Apply plugin-wide tags if set
|
||||
for k, v := range ac.inputConfig.Tags {
|
||||
tags[k] = v
|
||||
if _, ok := tags[k]; !ok {
|
||||
tags[k] = v
|
||||
}
|
||||
}
|
||||
// Apply daemon-wide tags if set
|
||||
for k, v := range ac.defaultTags {
|
||||
if _, ok := tags[k]; !ok {
|
||||
tags[k] = v
|
||||
}
|
||||
}
|
||||
ac.inputConfig.Filter.FilterTags(tags)
|
||||
|
||||
@@ -138,6 +144,7 @@ func (ac *accumulator) AddFields(
|
||||
} else {
|
||||
timestamp = time.Now()
|
||||
}
|
||||
timestamp = timestamp.Round(ac.precision)
|
||||
|
||||
if ac.prefix != "" {
|
||||
measurement = ac.prefix + measurement
|
||||
@@ -148,7 +155,7 @@ func (ac *accumulator) AddFields(
|
||||
log.Printf("Error adding point [%s]: %s\n", measurement, err.Error())
|
||||
return
|
||||
}
|
||||
if ac.debug {
|
||||
if ac.trace {
|
||||
fmt.Println("> " + m.String())
|
||||
}
|
||||
ac.metrics <- m
|
||||
@@ -162,6 +169,39 @@ func (ac *accumulator) SetDebug(debug bool) {
|
||||
ac.debug = debug
|
||||
}
|
||||
|
||||
func (ac *accumulator) Trace() bool {
|
||||
return ac.trace
|
||||
}
|
||||
|
||||
func (ac *accumulator) SetTrace(trace bool) {
|
||||
ac.trace = trace
|
||||
}
|
||||
|
||||
// SetPrecision takes two time.Duration objects. If the first is non-zero,
|
||||
// it sets that as the precision. Otherwise, it takes the second argument
|
||||
// as the order of time that the metrics should be rounded to, with the
|
||||
// maximum being 1s.
|
||||
func (ac *accumulator) SetPrecision(precision, interval time.Duration) {
|
||||
if precision > 0 {
|
||||
ac.precision = precision
|
||||
return
|
||||
}
|
||||
switch {
|
||||
case interval >= time.Second:
|
||||
ac.precision = time.Second
|
||||
case interval >= time.Millisecond:
|
||||
ac.precision = time.Millisecond
|
||||
case interval >= time.Microsecond:
|
||||
ac.precision = time.Microsecond
|
||||
default:
|
||||
ac.precision = time.Nanosecond
|
||||
}
|
||||
}
|
||||
|
||||
func (ac *accumulator) DisablePrecision() {
|
||||
ac.precision = time.Nanosecond
|
||||
}
|
||||
|
||||
func (ac *accumulator) setDefaultTags(tags map[string]string) {
|
||||
ac.defaultTags = tags
|
||||
}
|
||||
|
||||
@@ -38,6 +38,128 @@ func TestAdd(t *testing.T) {
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddNoPrecisionWithInterval(t *testing.T) {
|
||||
a := accumulator{}
|
||||
now := time.Date(2006, time.February, 10, 12, 0, 0, 82912748, time.UTC)
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
a.SetPrecision(0, time.Second)
|
||||
a.Add("acctest", float64(101), map[string]string{})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800000000000)),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddNoIntervalWithPrecision(t *testing.T) {
|
||||
a := accumulator{}
|
||||
now := time.Date(2006, time.February, 10, 12, 0, 0, 82912748, time.UTC)
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
a.SetPrecision(time.Second, time.Millisecond)
|
||||
a.Add("acctest", float64(101), map[string]string{})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800000000000)),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddDisablePrecision(t *testing.T) {
|
||||
a := accumulator{}
|
||||
now := time.Date(2006, time.February, 10, 12, 0, 0, 82912748, time.UTC)
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
a.SetPrecision(time.Second, time.Millisecond)
|
||||
a.DisablePrecision()
|
||||
a.Add("acctest", float64(101), map[string]string{})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"})
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Contains(t, actual, "acctest value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Contains(t, actual, "acctest,acc=test value=101")
|
||||
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800082912748)),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestDifferentPrecisions(t *testing.T) {
|
||||
a := accumulator{}
|
||||
now := time.Date(2006, time.February, 10, 12, 0, 0, 82912748, time.UTC)
|
||||
a.metrics = make(chan telegraf.Metric, 10)
|
||||
defer close(a.metrics)
|
||||
a.inputConfig = &internal_models.InputConfig{}
|
||||
|
||||
a.SetPrecision(0, time.Second)
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
testm := <-a.metrics
|
||||
actual := testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800000000000)),
|
||||
actual)
|
||||
|
||||
a.SetPrecision(0, time.Millisecond)
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800083000000)),
|
||||
actual)
|
||||
|
||||
a.SetPrecision(0, time.Microsecond)
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800082913000)),
|
||||
actual)
|
||||
|
||||
a.SetPrecision(0, time.Nanosecond)
|
||||
a.Add("acctest", float64(101), map[string]string{"acc": "test"}, now)
|
||||
testm = <-a.metrics
|
||||
actual = testm.String()
|
||||
assert.Equal(t,
|
||||
fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800082912748)),
|
||||
actual)
|
||||
}
|
||||
|
||||
func TestAddDefaultTags(t *testing.T) {
|
||||
a := accumulator{}
|
||||
a.addDefaultTag("default", "tag")
|
||||
|
||||
205
agent/agent.go
205
agent/agent.go
@@ -1,17 +1,15 @@
|
||||
package agent
|
||||
|
||||
import (
|
||||
cryptorand "crypto/rand"
|
||||
"fmt"
|
||||
"log"
|
||||
"math/big"
|
||||
"math/rand"
|
||||
"os"
|
||||
"runtime"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/internal/config"
|
||||
"github.com/influxdata/telegraf/internal/models"
|
||||
)
|
||||
@@ -102,93 +100,41 @@ func panicRecover(input *internal_models.RunningInput) {
|
||||
}
|
||||
}
|
||||
|
||||
// gatherParallel runs the inputs that are using the same reporting interval
|
||||
// as the telegraf agent.
|
||||
func (a *Agent) gatherParallel(metricC chan telegraf.Metric) error {
|
||||
var wg sync.WaitGroup
|
||||
|
||||
start := time.Now()
|
||||
counter := 0
|
||||
jitter := a.Config.Agent.CollectionJitter.Duration.Nanoseconds()
|
||||
for _, input := range a.Config.Inputs {
|
||||
if input.Config.Interval != 0 {
|
||||
continue
|
||||
}
|
||||
|
||||
wg.Add(1)
|
||||
counter++
|
||||
go func(input *internal_models.RunningInput) {
|
||||
defer panicRecover(input)
|
||||
defer wg.Done()
|
||||
|
||||
acc := NewAccumulator(input.Config, metricC)
|
||||
acc.SetDebug(a.Config.Agent.Debug)
|
||||
acc.setDefaultTags(a.Config.Tags)
|
||||
|
||||
if jitter != 0 {
|
||||
nanoSleep := rand.Int63n(jitter)
|
||||
d, err := time.ParseDuration(fmt.Sprintf("%dns", nanoSleep))
|
||||
if err != nil {
|
||||
log.Printf("Jittering collection interval failed for plugin %s",
|
||||
input.Name)
|
||||
} else {
|
||||
time.Sleep(d)
|
||||
}
|
||||
}
|
||||
|
||||
if err := input.Input.Gather(acc); err != nil {
|
||||
log.Printf("Error in input [%s]: %s", input.Name, err)
|
||||
}
|
||||
|
||||
}(input)
|
||||
}
|
||||
|
||||
if counter == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
|
||||
elapsed := time.Since(start)
|
||||
if !a.Config.Agent.Quiet {
|
||||
log.Printf("Gathered metrics, (%s interval), from %d inputs in %s\n",
|
||||
a.Config.Agent.Interval.Duration, counter, elapsed)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// gatherSeparate runs the inputs that have been configured with their own
|
||||
// gatherer runs the inputs that have been configured with their own
|
||||
// reporting interval.
|
||||
func (a *Agent) gatherSeparate(
|
||||
func (a *Agent) gatherer(
|
||||
shutdown chan struct{},
|
||||
input *internal_models.RunningInput,
|
||||
interval time.Duration,
|
||||
metricC chan telegraf.Metric,
|
||||
) error {
|
||||
defer panicRecover(input)
|
||||
|
||||
ticker := time.NewTicker(input.Config.Interval)
|
||||
ticker := time.NewTicker(interval)
|
||||
defer ticker.Stop()
|
||||
|
||||
for {
|
||||
var outerr error
|
||||
start := time.Now()
|
||||
|
||||
acc := NewAccumulator(input.Config, metricC)
|
||||
acc.SetDebug(a.Config.Agent.Debug)
|
||||
acc.SetPrecision(a.Config.Agent.Precision.Duration,
|
||||
a.Config.Agent.Interval.Duration)
|
||||
acc.setDefaultTags(a.Config.Tags)
|
||||
|
||||
if err := input.Input.Gather(acc); err != nil {
|
||||
log.Printf("Error in input [%s]: %s", input.Name, err)
|
||||
}
|
||||
internal.RandomSleep(a.Config.Agent.CollectionJitter.Duration, shutdown)
|
||||
|
||||
start := time.Now()
|
||||
gatherWithTimeout(shutdown, input, acc, interval)
|
||||
elapsed := time.Since(start)
|
||||
if !a.Config.Agent.Quiet {
|
||||
log.Printf("Gathered metrics, (separate %s interval), from %s in %s\n",
|
||||
input.Config.Interval, input.Name, elapsed)
|
||||
}
|
||||
|
||||
if outerr != nil {
|
||||
return outerr
|
||||
}
|
||||
if a.Config.Agent.Debug {
|
||||
log.Printf("Input [%s] gathered metrics, (%s interval) in %s\n",
|
||||
input.Name, interval, elapsed)
|
||||
}
|
||||
|
||||
select {
|
||||
case <-shutdown:
|
||||
@@ -199,6 +145,42 @@ func (a *Agent) gatherSeparate(
|
||||
}
|
||||
}
|
||||
|
||||
// gatherWithTimeout gathers from the given input, with the given timeout.
|
||||
// when the given timeout is reached, gatherWithTimeout logs an error message
|
||||
// but continues waiting for it to return. This is to avoid leaving behind
|
||||
// hung processes, and to prevent re-calling the same hung process over and
|
||||
// over.
|
||||
func gatherWithTimeout(
|
||||
shutdown chan struct{},
|
||||
input *internal_models.RunningInput,
|
||||
acc *accumulator,
|
||||
timeout time.Duration,
|
||||
) {
|
||||
ticker := time.NewTicker(timeout)
|
||||
defer ticker.Stop()
|
||||
done := make(chan error)
|
||||
go func() {
|
||||
done <- input.Input.Gather(acc)
|
||||
}()
|
||||
|
||||
for {
|
||||
select {
|
||||
case err := <-done:
|
||||
if err != nil {
|
||||
log.Printf("ERROR in input [%s]: %s", input.Name, err)
|
||||
}
|
||||
return
|
||||
case <-ticker.C:
|
||||
log.Printf("ERROR: input [%s] took longer to collect than "+
|
||||
"collection interval (%s)",
|
||||
input.Name, timeout)
|
||||
continue
|
||||
case <-shutdown:
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Test verifies that we can 'Gather' from all inputs with their configured
|
||||
// Config struct
|
||||
func (a *Agent) Test() error {
|
||||
@@ -220,7 +202,9 @@ func (a *Agent) Test() error {
|
||||
|
||||
for _, input := range a.Config.Inputs {
|
||||
acc := NewAccumulator(input.Config, metricC)
|
||||
acc.SetDebug(true)
|
||||
acc.SetTrace(true)
|
||||
acc.SetPrecision(a.Config.Agent.Precision.Duration,
|
||||
a.Config.Agent.Interval.Duration)
|
||||
acc.setDefaultTags(a.Config.Tags)
|
||||
|
||||
fmt.Printf("* Plugin: %s, Collection 1\n", input.Name)
|
||||
@@ -281,44 +265,40 @@ func (a *Agent) flusher(shutdown chan struct{}, metricC chan telegraf.Metric) er
|
||||
a.flush()
|
||||
return nil
|
||||
case <-ticker.C:
|
||||
internal.RandomSleep(a.Config.Agent.FlushJitter.Duration, shutdown)
|
||||
a.flush()
|
||||
case m := <-metricC:
|
||||
for _, o := range a.Config.Outputs {
|
||||
o.AddMetric(m)
|
||||
for i, o := range a.Config.Outputs {
|
||||
if i == len(a.Config.Outputs)-1 {
|
||||
o.AddMetric(m)
|
||||
} else {
|
||||
o.AddMetric(copyMetric(m))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// jitterInterval applies the the interval jitter to the flush interval using
|
||||
// crypto/rand number generator
|
||||
func jitterInterval(ininterval, injitter time.Duration) time.Duration {
|
||||
var jitter int64
|
||||
outinterval := ininterval
|
||||
if injitter.Nanoseconds() != 0 {
|
||||
maxjitter := big.NewInt(injitter.Nanoseconds())
|
||||
if j, err := cryptorand.Int(cryptorand.Reader, maxjitter); err == nil {
|
||||
jitter = j.Int64()
|
||||
}
|
||||
outinterval = time.Duration(jitter + ininterval.Nanoseconds())
|
||||
func copyMetric(m telegraf.Metric) telegraf.Metric {
|
||||
t := time.Time(m.Time())
|
||||
|
||||
tags := make(map[string]string)
|
||||
fields := make(map[string]interface{})
|
||||
for k, v := range m.Tags() {
|
||||
tags[k] = v
|
||||
}
|
||||
for k, v := range m.Fields() {
|
||||
fields[k] = v
|
||||
}
|
||||
|
||||
if outinterval.Nanoseconds() < time.Duration(500*time.Millisecond).Nanoseconds() {
|
||||
log.Printf("Flush interval %s too low, setting to 500ms\n", outinterval)
|
||||
outinterval = time.Duration(500 * time.Millisecond)
|
||||
}
|
||||
|
||||
return outinterval
|
||||
out, _ := telegraf.NewMetric(m.Name(), tags, fields, t)
|
||||
return out
|
||||
}
|
||||
|
||||
// Run runs the agent daemon, gathering every Interval
|
||||
func (a *Agent) Run(shutdown chan struct{}) error {
|
||||
var wg sync.WaitGroup
|
||||
|
||||
a.Config.Agent.FlushInterval.Duration = jitterInterval(
|
||||
a.Config.Agent.FlushInterval.Duration,
|
||||
a.Config.Agent.FlushJitter.Duration)
|
||||
|
||||
log.Printf("Agent Config: Interval:%s, Debug:%#v, Quiet:%#v, Hostname:%#v, "+
|
||||
"Flush Interval:%s \n",
|
||||
a.Config.Agent.Interval.Duration, a.Config.Agent.Debug, a.Config.Agent.Quiet,
|
||||
@@ -333,6 +313,9 @@ func (a *Agent) Run(shutdown chan struct{}) error {
|
||||
case telegraf.ServiceInput:
|
||||
acc := NewAccumulator(input.Config, metricC)
|
||||
acc.SetDebug(a.Config.Agent.Debug)
|
||||
// Service input plugins should set their own precision of their
|
||||
// metrics.
|
||||
acc.DisablePrecision()
|
||||
acc.setDefaultTags(a.Config.Tags)
|
||||
if err := p.Start(acc); err != nil {
|
||||
log.Printf("Service for input %s failed to start, exiting\n%s\n",
|
||||
@@ -348,7 +331,6 @@ func (a *Agent) Run(shutdown chan struct{}) error {
|
||||
i := int64(a.Config.Agent.Interval.Duration)
|
||||
time.Sleep(time.Duration(i - (time.Now().UnixNano() % i)))
|
||||
}
|
||||
ticker := time.NewTicker(a.Config.Agent.Interval.Duration)
|
||||
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
@@ -359,32 +341,21 @@ func (a *Agent) Run(shutdown chan struct{}) error {
|
||||
}
|
||||
}()
|
||||
|
||||
wg.Add(len(a.Config.Inputs))
|
||||
for _, input := range a.Config.Inputs {
|
||||
// Special handling for inputs that have their own collection interval
|
||||
// configured. Default intervals are handled below with gatherParallel
|
||||
interval := a.Config.Agent.Interval.Duration
|
||||
// overwrite global interval if this plugin has it's own.
|
||||
if input.Config.Interval != 0 {
|
||||
wg.Add(1)
|
||||
go func(input *internal_models.RunningInput) {
|
||||
defer wg.Done()
|
||||
if err := a.gatherSeparate(shutdown, input, metricC); err != nil {
|
||||
log.Printf(err.Error())
|
||||
}
|
||||
}(input)
|
||||
interval = input.Config.Interval
|
||||
}
|
||||
go func(in *internal_models.RunningInput, interv time.Duration) {
|
||||
defer wg.Done()
|
||||
if err := a.gatherer(shutdown, in, interv, metricC); err != nil {
|
||||
log.Printf(err.Error())
|
||||
}
|
||||
}(input, interval)
|
||||
}
|
||||
|
||||
defer wg.Wait()
|
||||
|
||||
for {
|
||||
if err := a.gatherParallel(metricC); err != nil {
|
||||
log.Printf(err.Error())
|
||||
}
|
||||
|
||||
select {
|
||||
case <-shutdown:
|
||||
return nil
|
||||
case <-ticker.C:
|
||||
continue
|
||||
}
|
||||
}
|
||||
wg.Wait()
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -2,7 +2,6 @@ package agent
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf/internal/config"
|
||||
|
||||
@@ -110,75 +109,3 @@ func TestAgent_LoadOutput(t *testing.T) {
|
||||
a, _ = NewAgent(c)
|
||||
assert.Equal(t, 3, len(a.Config.Outputs))
|
||||
}
|
||||
|
||||
func TestAgent_ZeroJitter(t *testing.T) {
|
||||
flushinterval := jitterInterval(time.Duration(10*time.Second),
|
||||
time.Duration(0*time.Second))
|
||||
|
||||
actual := flushinterval.Nanoseconds()
|
||||
exp := time.Duration(10 * time.Second).Nanoseconds()
|
||||
|
||||
if actual != exp {
|
||||
t.Errorf("Actual %v, expected %v", actual, exp)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_ZeroInterval(t *testing.T) {
|
||||
min := time.Duration(500 * time.Millisecond).Nanoseconds()
|
||||
max := time.Duration(5 * time.Second).Nanoseconds()
|
||||
|
||||
for i := 0; i < 1000; i++ {
|
||||
flushinterval := jitterInterval(time.Duration(0*time.Second),
|
||||
time.Duration(5*time.Second))
|
||||
actual := flushinterval.Nanoseconds()
|
||||
|
||||
if actual > max {
|
||||
t.Errorf("Didn't expect interval %d to be > %d", actual, max)
|
||||
break
|
||||
}
|
||||
if actual < min {
|
||||
t.Errorf("Didn't expect interval %d to be < %d", actual, min)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_ZeroBoth(t *testing.T) {
|
||||
flushinterval := jitterInterval(time.Duration(0*time.Second),
|
||||
time.Duration(0*time.Second))
|
||||
|
||||
actual := flushinterval
|
||||
exp := time.Duration(500 * time.Millisecond)
|
||||
|
||||
if actual != exp {
|
||||
t.Errorf("Actual %v, expected %v", actual, exp)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_JitterMax(t *testing.T) {
|
||||
max := time.Duration(32 * time.Second).Nanoseconds()
|
||||
|
||||
for i := 0; i < 1000; i++ {
|
||||
flushinterval := jitterInterval(time.Duration(30*time.Second),
|
||||
time.Duration(2*time.Second))
|
||||
actual := flushinterval.Nanoseconds()
|
||||
if actual > max {
|
||||
t.Errorf("Didn't expect interval %d to be > %d", actual, max)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAgent_JitterMin(t *testing.T) {
|
||||
min := time.Duration(30 * time.Second).Nanoseconds()
|
||||
|
||||
for i := 0; i < 1000; i++ {
|
||||
flushinterval := jitterInterval(time.Duration(30*time.Second),
|
||||
time.Duration(2*time.Second))
|
||||
actual := flushinterval.Nanoseconds()
|
||||
if actual < min {
|
||||
t.Errorf("Didn't expect interval %d to be < %d", actual, min)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -186,49 +186,59 @@ name of the plugin.
|
||||
# Graphite:
|
||||
|
||||
The Graphite data format translates graphite _dot_ buckets directly into
|
||||
telegraf measurement names, with a single value field, and without any tags. For
|
||||
more advanced options, Telegraf supports specifying "templates" to translate
|
||||
telegraf measurement names, with a single value field, and without any tags.
|
||||
By default, the separator is left as ".", but this can be changed using the
|
||||
"separator" argument. For more advanced options,
|
||||
Telegraf supports specifying "templates" to translate
|
||||
graphite buckets into Telegraf metrics.
|
||||
|
||||
#### Separator:
|
||||
|
||||
You can specify a separator to use for the parsed metrics.
|
||||
By default, it will leave the metrics with a "." separator.
|
||||
Setting `separator = "_"` will translate:
|
||||
Templates are of the form:
|
||||
|
||||
```
|
||||
cpu.usage.idle 99
|
||||
=> cpu_usage_idle value=99
|
||||
"host.mytag.mytag.measurement.measurement.field*"
|
||||
```
|
||||
|
||||
#### Measurement/Tag Templates:
|
||||
Where the following keywords exist:
|
||||
|
||||
1. `measurement`: specifies that this section of the graphite bucket corresponds
|
||||
to the measurement name. This can be specified multiple times.
|
||||
2. `field`: specifies that this section of the graphite bucket corresponds
|
||||
to the field name. This can be specified multiple times.
|
||||
3. `measurement*`: specifies that all remaining elements of the graphite bucket
|
||||
correspond to the measurement name.
|
||||
4. `field*`: specifies that all remaining elements of the graphite bucket
|
||||
correspond to the field name.
|
||||
|
||||
Any part of the template that is not a keyword is treated as a tag key. This
|
||||
can also be specified multiple times.
|
||||
|
||||
NOTE: `field*` cannot be used in conjunction with `measurement*`!
|
||||
|
||||
#### Measurement & Tag Templates:
|
||||
|
||||
The most basic template is to specify a single transformation to apply to all
|
||||
incoming metrics. _measurement_ is a special keyword that tells Telegraf which
|
||||
parts of the graphite bucket to combine into the measurement name. It can have a
|
||||
trailing `*` to indicate that the remainder of the metric should be used.
|
||||
Other words are considered tag keys. So the following template:
|
||||
incoming metrics. So the following template:
|
||||
|
||||
```toml
|
||||
templates = [
|
||||
"region.measurement*"
|
||||
"region.region.measurement*"
|
||||
]
|
||||
```
|
||||
|
||||
would result in the following Graphite -> Telegraf transformation.
|
||||
|
||||
```
|
||||
us-west.cpu.load 100
|
||||
=> cpu.load,region=us-west value=100
|
||||
us.west.cpu.load 100
|
||||
=> cpu.load,region=us.west value=100
|
||||
```
|
||||
|
||||
#### Field Templates:
|
||||
|
||||
There is also a _field_ keyword, which can only be specified once.
|
||||
The field keyword tells Telegraf to give the metric that field name.
|
||||
So the following template:
|
||||
|
||||
```toml
|
||||
separator = "_"
|
||||
templates = [
|
||||
"measurement.measurement.field.field.region"
|
||||
]
|
||||
@@ -237,24 +247,26 @@ templates = [
|
||||
would result in the following Graphite -> Telegraf transformation.
|
||||
|
||||
```
|
||||
cpu.usage.idle.percent.us-west 100
|
||||
=> cpu_usage,region=us-west idle_percent=100
|
||||
cpu.usage.idle.percent.eu-east 100
|
||||
=> cpu_usage,region=eu-east idle_percent=100
|
||||
```
|
||||
|
||||
The field key can also be derived from the second "half" of the input metric-name by specifying ```field*```:
|
||||
The field key can also be derived from all remaining elements of the graphite
|
||||
bucket by specifying `field*`:
|
||||
|
||||
```toml
|
||||
separator = "_"
|
||||
templates = [
|
||||
"measurement.measurement.region.field*"
|
||||
]
|
||||
```
|
||||
|
||||
would result in the following Graphite -> Telegraf transformation.
|
||||
which would result in the following Graphite -> Telegraf transformation.
|
||||
|
||||
```
|
||||
cpu.usage.us-west.idle.percentage 100
|
||||
=> cpu_usage,region=us-west idle_percentage=100
|
||||
cpu.usage.eu-east.idle.percentage 100
|
||||
=> cpu_usage,region=eu-east idle_percentage=100
|
||||
```
|
||||
(This cannot be used in conjunction with "measurement*"!)
|
||||
|
||||
#### Filter Templates:
|
||||
|
||||
@@ -271,8 +283,8 @@ templates = [
|
||||
which would result in the following transformation:
|
||||
|
||||
```
|
||||
cpu.load.us-west 100
|
||||
=> cpu_load,region=us-west value=100
|
||||
cpu.load.eu-east 100
|
||||
=> cpu_load,region=eu-east value=100
|
||||
|
||||
mem.cached.localhost 256
|
||||
=> mem_cached,host=localhost value=256
|
||||
@@ -294,8 +306,8 @@ templates = [
|
||||
would result in the following Graphite -> Telegraf transformation.
|
||||
|
||||
```
|
||||
cpu.usage.idle.us-west 100
|
||||
=> cpu_usage,region=us-west,datacenter=1a idle=100
|
||||
cpu.usage.idle.eu-east 100
|
||||
=> cpu_usage,region=eu-east,datacenter=1a idle=100
|
||||
```
|
||||
|
||||
There are many more options available,
|
||||
@@ -326,12 +338,12 @@ There are many more options available,
|
||||
## similar to the line protocol format. There can be only one default template.
|
||||
## Templates support below format:
|
||||
## 1. filter + template
|
||||
## 2. filter + template + extra tag
|
||||
## 2. filter + template + extra tag(s)
|
||||
## 3. filter + template with field key
|
||||
## 4. default template
|
||||
templates = [
|
||||
"*.app env.service.resource.measurement",
|
||||
"stats.* .host.measurement* region=us-west,agent=sensu",
|
||||
"stats.* .host.measurement* region=eu-east,agent=sensu",
|
||||
"stats2.* .host.measurement.field",
|
||||
"measurement*"
|
||||
]
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
79
filter/filter.go
Normal file
79
filter/filter.go
Normal file
@@ -0,0 +1,79 @@
|
||||
package filter
|
||||
|
||||
import (
|
||||
"strings"
|
||||
|
||||
"github.com/gobwas/glob"
|
||||
)
|
||||
|
||||
type Filter interface {
|
||||
Match(string) bool
|
||||
}
|
||||
|
||||
// CompileFilter takes a list of string filters and returns a Filter interface
|
||||
// for matching a given string against the filter list. The filter list
|
||||
// supports glob matching too, ie:
|
||||
//
|
||||
// f, _ := CompileFilter([]string{"cpu", "mem", "net*"})
|
||||
// f.Match("cpu") // true
|
||||
// f.Match("network") // true
|
||||
// f.Match("memory") // false
|
||||
//
|
||||
func CompileFilter(filters []string) (Filter, error) {
|
||||
// return if there is nothing to compile
|
||||
if len(filters) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
|
||||
// check if we can compile a non-glob filter
|
||||
noGlob := true
|
||||
for _, filter := range filters {
|
||||
if hasMeta(filter) {
|
||||
noGlob = false
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
switch {
|
||||
case noGlob:
|
||||
// return non-globbing filter if not needed.
|
||||
return compileFilterNoGlob(filters), nil
|
||||
case len(filters) == 1:
|
||||
return glob.Compile(filters[0])
|
||||
default:
|
||||
return glob.Compile("{" + strings.Join(filters, ",") + "}")
|
||||
}
|
||||
}
|
||||
|
||||
// hasMeta reports whether path contains any magic glob characters.
|
||||
func hasMeta(s string) bool {
|
||||
return strings.IndexAny(s, "*?[") >= 0
|
||||
}
|
||||
|
||||
type filter struct {
|
||||
m map[string]struct{}
|
||||
}
|
||||
|
||||
func (f *filter) Match(s string) bool {
|
||||
_, ok := f.m[s]
|
||||
return ok
|
||||
}
|
||||
|
||||
type filtersingle struct {
|
||||
s string
|
||||
}
|
||||
|
||||
func (f *filtersingle) Match(s string) bool {
|
||||
return f.s == s
|
||||
}
|
||||
|
||||
func compileFilterNoGlob(filters []string) Filter {
|
||||
if len(filters) == 1 {
|
||||
return &filtersingle{s: filters[0]}
|
||||
}
|
||||
out := filter{m: make(map[string]struct{})}
|
||||
for _, filter := range filters {
|
||||
out.m[filter] = struct{}{}
|
||||
}
|
||||
return &out
|
||||
}
|
||||
96
filter/filter_test.go
Normal file
96
filter/filter_test.go
Normal file
@@ -0,0 +1,96 @@
|
||||
package filter
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestCompileFilter(t *testing.T) {
|
||||
f, err := CompileFilter([]string{})
|
||||
assert.NoError(t, err)
|
||||
assert.Nil(t, f)
|
||||
|
||||
f, err = CompileFilter([]string{"cpu"})
|
||||
assert.NoError(t, err)
|
||||
assert.True(t, f.Match("cpu"))
|
||||
assert.False(t, f.Match("cpu0"))
|
||||
assert.False(t, f.Match("mem"))
|
||||
|
||||
f, err = CompileFilter([]string{"cpu*"})
|
||||
assert.NoError(t, err)
|
||||
assert.True(t, f.Match("cpu"))
|
||||
assert.True(t, f.Match("cpu0"))
|
||||
assert.False(t, f.Match("mem"))
|
||||
|
||||
f, err = CompileFilter([]string{"cpu", "mem"})
|
||||
assert.NoError(t, err)
|
||||
assert.True(t, f.Match("cpu"))
|
||||
assert.False(t, f.Match("cpu0"))
|
||||
assert.True(t, f.Match("mem"))
|
||||
|
||||
f, err = CompileFilter([]string{"cpu", "mem", "net*"})
|
||||
assert.NoError(t, err)
|
||||
assert.True(t, f.Match("cpu"))
|
||||
assert.False(t, f.Match("cpu0"))
|
||||
assert.True(t, f.Match("mem"))
|
||||
assert.True(t, f.Match("network"))
|
||||
}
|
||||
|
||||
var benchbool bool
|
||||
|
||||
func BenchmarkFilterSingleNoGlobFalse(b *testing.B) {
|
||||
f, _ := CompileFilter([]string{"cpu"})
|
||||
var tmp bool
|
||||
for n := 0; n < b.N; n++ {
|
||||
tmp = f.Match("network")
|
||||
}
|
||||
benchbool = tmp
|
||||
}
|
||||
|
||||
func BenchmarkFilterSingleNoGlobTrue(b *testing.B) {
|
||||
f, _ := CompileFilter([]string{"cpu"})
|
||||
var tmp bool
|
||||
for n := 0; n < b.N; n++ {
|
||||
tmp = f.Match("cpu")
|
||||
}
|
||||
benchbool = tmp
|
||||
}
|
||||
|
||||
func BenchmarkFilter(b *testing.B) {
|
||||
f, _ := CompileFilter([]string{"cpu", "mem", "net*"})
|
||||
var tmp bool
|
||||
for n := 0; n < b.N; n++ {
|
||||
tmp = f.Match("network")
|
||||
}
|
||||
benchbool = tmp
|
||||
}
|
||||
|
||||
func BenchmarkFilterNoGlob(b *testing.B) {
|
||||
f, _ := CompileFilter([]string{"cpu", "mem", "net"})
|
||||
var tmp bool
|
||||
for n := 0; n < b.N; n++ {
|
||||
tmp = f.Match("net")
|
||||
}
|
||||
benchbool = tmp
|
||||
}
|
||||
|
||||
func BenchmarkFilter2(b *testing.B) {
|
||||
f, _ := CompileFilter([]string{"aa", "bb", "c", "ad", "ar", "at", "aq",
|
||||
"aw", "az", "axxx", "ab", "cpu", "mem", "net*"})
|
||||
var tmp bool
|
||||
for n := 0; n < b.N; n++ {
|
||||
tmp = f.Match("network")
|
||||
}
|
||||
benchbool = tmp
|
||||
}
|
||||
|
||||
func BenchmarkFilter2NoGlob(b *testing.B) {
|
||||
f, _ := CompileFilter([]string{"aa", "bb", "c", "ad", "ar", "at", "aq",
|
||||
"aw", "az", "axxx", "ab", "cpu", "mem", "net"})
|
||||
var tmp bool
|
||||
for n := 0; n < b.N; n++ {
|
||||
tmp = f.Match("net")
|
||||
}
|
||||
benchbool = tmp
|
||||
}
|
||||
49
internal/config/aws/credentials.go
Normal file
49
internal/config/aws/credentials.go
Normal file
@@ -0,0 +1,49 @@
|
||||
package aws
|
||||
|
||||
import (
|
||||
"github.com/aws/aws-sdk-go/aws"
|
||||
"github.com/aws/aws-sdk-go/aws/client"
|
||||
"github.com/aws/aws-sdk-go/aws/credentials"
|
||||
"github.com/aws/aws-sdk-go/aws/credentials/stscreds"
|
||||
"github.com/aws/aws-sdk-go/aws/session"
|
||||
)
|
||||
|
||||
type CredentialConfig struct {
|
||||
Region string
|
||||
AccessKey string
|
||||
SecretKey string
|
||||
RoleARN string
|
||||
Profile string
|
||||
Filename string
|
||||
Token string
|
||||
}
|
||||
|
||||
func (c *CredentialConfig) Credentials() client.ConfigProvider {
|
||||
if c.RoleARN != "" {
|
||||
return c.assumeCredentials()
|
||||
} else {
|
||||
return c.rootCredentials()
|
||||
}
|
||||
}
|
||||
|
||||
func (c *CredentialConfig) rootCredentials() client.ConfigProvider {
|
||||
config := &aws.Config{
|
||||
Region: aws.String(c.Region),
|
||||
}
|
||||
if c.AccessKey != "" || c.SecretKey != "" {
|
||||
config.Credentials = credentials.NewStaticCredentials(c.AccessKey, c.SecretKey, c.Token)
|
||||
} else if c.Profile != "" || c.Filename != "" {
|
||||
config.Credentials = credentials.NewSharedCredentials(c.Filename, c.Profile)
|
||||
}
|
||||
|
||||
return session.New(config)
|
||||
}
|
||||
|
||||
func (c *CredentialConfig) assumeCredentials() client.ConfigProvider {
|
||||
rootCredentials := c.rootCredentials()
|
||||
config := &aws.Config{
|
||||
Region: aws.String(c.Region),
|
||||
}
|
||||
config.Credentials = stscreds.NewCredentials(rootCredentials, c.RoleARN)
|
||||
return session.New(config)
|
||||
}
|
||||
@@ -58,7 +58,6 @@ func NewConfig() *Config {
|
||||
Interval: internal.Duration{Duration: 10 * time.Second},
|
||||
RoundInterval: true,
|
||||
FlushInterval: internal.Duration{Duration: 10 * time.Second},
|
||||
FlushJitter: internal.Duration{Duration: 5 * time.Second},
|
||||
},
|
||||
|
||||
Tags: make(map[string]string),
|
||||
@@ -78,6 +77,14 @@ type AgentConfig struct {
|
||||
// ie, if Interval=10s then always collect on :00, :10, :20, etc.
|
||||
RoundInterval bool
|
||||
|
||||
// By default, precision will be set to the same timestamp order as the
|
||||
// collection interval, with the maximum being 1s.
|
||||
// ie, when interval = "10s", precision will be "1s"
|
||||
// when interval = "250ms", precision will be "1ms"
|
||||
// Precision will NOT be used for service inputs. It is up to each individual
|
||||
// service input to set the timestamp at the appropriate precision.
|
||||
Precision internal.Duration
|
||||
|
||||
// CollectionJitter is used to jitter the collection by a random amount.
|
||||
// Each plugin will sleep for a random time within jitter before collecting.
|
||||
// This can be used to avoid many plugins querying things like sysfs at the
|
||||
@@ -109,11 +116,10 @@ type AgentConfig struct {
|
||||
// does _not_ deactivate FlushInterval.
|
||||
FlushBufferWhenFull bool
|
||||
|
||||
// TODO(cam): Remove UTC and Precision parameters, they are no longer
|
||||
// TODO(cam): Remove UTC and parameter, they are no longer
|
||||
// valid for the agent config. Leaving them here for now for backwards-
|
||||
// compatability
|
||||
UTC bool `toml:"utc"`
|
||||
Precision string
|
||||
UTC bool `toml:"utc"`
|
||||
|
||||
// Debug is the option for running in debug mode
|
||||
Debug bool
|
||||
@@ -210,6 +216,11 @@ var header = `# Telegraf Configuration
|
||||
## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
|
||||
flush_jitter = "0s"
|
||||
|
||||
## By default, precision will be set to the same timestamp order as the
|
||||
## collection interval, with the maximum being 1s.
|
||||
## Precision will NOT be used for service inputs, such as logparser and statsd.
|
||||
## Valid values are "Nns", "Nus" (or "Nµs"), "Nms", "Ns".
|
||||
precision = ""
|
||||
## Run telegraf in debug mode
|
||||
debug = false
|
||||
## Run telegraf in quiet mode
|
||||
@@ -357,7 +368,7 @@ func printConfig(name string, p printer, op string, commented bool) {
|
||||
fmt.Print("\n")
|
||||
continue
|
||||
}
|
||||
fmt.Print(comment + line + "\n")
|
||||
fmt.Print(strings.TrimRight(comment+line, " ") + "\n")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -528,6 +539,13 @@ func (c *Config) LoadConfig(path string) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
// trimBOM trims the Byte-Order-Marks from the beginning of the file.
|
||||
// this is for Windows compatability only.
|
||||
// see https://github.com/influxdata/telegraf/issues/1378
|
||||
func trimBOM(f []byte) []byte {
|
||||
return bytes.TrimPrefix(f, []byte("\xef\xbb\xbf"))
|
||||
}
|
||||
|
||||
// parseFile loads a TOML configuration from a provided path and
|
||||
// returns the AST produced from the TOML parser. When loading the file, it
|
||||
// will find environment variables and replace them.
|
||||
@@ -536,6 +554,8 @@ func parseFile(fpath string) (*ast.Table, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
// ugh windows why
|
||||
contents = trimBOM(contents)
|
||||
|
||||
env_vars := envVarRe.FindAll(contents, -1)
|
||||
for _, env_var := range env_vars {
|
||||
|
||||
37
internal/errchan/errchan.go
Normal file
37
internal/errchan/errchan.go
Normal file
@@ -0,0 +1,37 @@
|
||||
package errchan
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type ErrChan struct {
|
||||
C chan error
|
||||
}
|
||||
|
||||
// New returns an error channel of max length 'n'
|
||||
// errors can be sent to the ErrChan.C channel, and will be returned when
|
||||
// ErrChan.Error() is called.
|
||||
func New(n int) *ErrChan {
|
||||
return &ErrChan{
|
||||
C: make(chan error, n),
|
||||
}
|
||||
}
|
||||
|
||||
// Error closes the ErrChan.C channel and returns an error if there are any
|
||||
// non-nil errors, otherwise returns nil.
|
||||
func (e *ErrChan) Error() error {
|
||||
close(e.C)
|
||||
|
||||
var out string
|
||||
for err := range e.C {
|
||||
if err != nil {
|
||||
out += "[" + err.Error() + "], "
|
||||
}
|
||||
}
|
||||
|
||||
if out != "" {
|
||||
return fmt.Errorf("Errors encountered: " + strings.TrimRight(out, ", "))
|
||||
}
|
||||
return nil
|
||||
}
|
||||
@@ -10,8 +10,10 @@ import (
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"log"
|
||||
"math/big"
|
||||
"os"
|
||||
"os/exec"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
"unicode"
|
||||
@@ -32,12 +34,25 @@ type Duration struct {
|
||||
|
||||
// UnmarshalTOML parses the duration from the TOML config file
|
||||
func (d *Duration) UnmarshalTOML(b []byte) error {
|
||||
dur, err := time.ParseDuration(string(b[1 : len(b)-1]))
|
||||
if err != nil {
|
||||
return err
|
||||
var err error
|
||||
// Parse string duration, ie, "1s"
|
||||
d.Duration, err = time.ParseDuration(string(b[1 : len(b)-1]))
|
||||
if err == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
d.Duration = dur
|
||||
// First try parsing as integer seconds
|
||||
sI, err := strconv.ParseInt(string(b), 10, 64)
|
||||
if err == nil {
|
||||
d.Duration = time.Second * time.Duration(sI)
|
||||
return nil
|
||||
}
|
||||
// Second try parsing as float seconds
|
||||
sF, err := strconv.ParseFloat(string(b), 64)
|
||||
if err == nil {
|
||||
d.Duration = time.Second * time.Duration(sF)
|
||||
return nil
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
@@ -118,8 +133,8 @@ func GetTLSConfig(
|
||||
cert, err := tls.LoadX509KeyPair(SSLCert, SSLKey)
|
||||
if err != nil {
|
||||
return nil, errors.New(fmt.Sprintf(
|
||||
"Could not load TLS client key/certificate: %s",
|
||||
err))
|
||||
"Could not load TLS client key/certificate from %s:%s: %s",
|
||||
SSLKey, SSLCert, err))
|
||||
}
|
||||
|
||||
t.Certificates = []tls.Certificate{cert}
|
||||
@@ -191,3 +206,27 @@ func WaitTimeout(c *exec.Cmd, timeout time.Duration) error {
|
||||
return TimeoutErr
|
||||
}
|
||||
}
|
||||
|
||||
// RandomSleep will sleep for a random amount of time up to max.
|
||||
// If the shutdown channel is closed, it will return before it has finished
|
||||
// sleeping.
|
||||
func RandomSleep(max time.Duration, shutdown chan struct{}) {
|
||||
if max == 0 {
|
||||
return
|
||||
}
|
||||
maxSleep := big.NewInt(max.Nanoseconds())
|
||||
|
||||
var sleepns int64
|
||||
if j, err := rand.Int(rand.Reader, maxSleep); err == nil {
|
||||
sleepns = j.Int64()
|
||||
}
|
||||
|
||||
t := time.NewTimer(time.Nanosecond * time.Duration(sleepns))
|
||||
select {
|
||||
case <-t.C:
|
||||
return
|
||||
case <-shutdown:
|
||||
t.Stop()
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
@@ -106,3 +106,28 @@ func TestRunError(t *testing.T) {
|
||||
|
||||
assert.Error(t, err)
|
||||
}
|
||||
|
||||
func TestRandomSleep(t *testing.T) {
|
||||
// test that zero max returns immediately
|
||||
s := time.Now()
|
||||
RandomSleep(time.Duration(0), make(chan struct{}))
|
||||
elapsed := time.Since(s)
|
||||
assert.True(t, elapsed < time.Millisecond)
|
||||
|
||||
// test that max sleep is respected
|
||||
s = time.Now()
|
||||
RandomSleep(time.Millisecond*50, make(chan struct{}))
|
||||
elapsed = time.Since(s)
|
||||
assert.True(t, elapsed < time.Millisecond*50)
|
||||
|
||||
// test that shutdown is respected
|
||||
s = time.Now()
|
||||
shutdown := make(chan struct{})
|
||||
go func() {
|
||||
time.Sleep(time.Millisecond * 100)
|
||||
close(shutdown)
|
||||
}()
|
||||
RandomSleep(time.Second, shutdown)
|
||||
elapsed = time.Since(s)
|
||||
assert.True(t, elapsed < time.Millisecond*150)
|
||||
}
|
||||
|
||||
59
internal/limiter/limiter.go
Normal file
59
internal/limiter/limiter.go
Normal file
@@ -0,0 +1,59 @@
|
||||
package limiter
|
||||
|
||||
import (
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
// NewRateLimiter returns a rate limiter that will will emit from the C
|
||||
// channel only 'n' times every 'rate' seconds.
|
||||
func NewRateLimiter(n int, rate time.Duration) *rateLimiter {
|
||||
r := &rateLimiter{
|
||||
C: make(chan bool),
|
||||
rate: rate,
|
||||
n: n,
|
||||
shutdown: make(chan bool),
|
||||
}
|
||||
r.wg.Add(1)
|
||||
go r.limiter()
|
||||
return r
|
||||
}
|
||||
|
||||
type rateLimiter struct {
|
||||
C chan bool
|
||||
rate time.Duration
|
||||
n int
|
||||
|
||||
shutdown chan bool
|
||||
wg sync.WaitGroup
|
||||
}
|
||||
|
||||
func (r *rateLimiter) Stop() {
|
||||
close(r.shutdown)
|
||||
r.wg.Wait()
|
||||
close(r.C)
|
||||
}
|
||||
|
||||
func (r *rateLimiter) limiter() {
|
||||
defer r.wg.Done()
|
||||
ticker := time.NewTicker(r.rate)
|
||||
defer ticker.Stop()
|
||||
counter := 0
|
||||
for {
|
||||
select {
|
||||
case <-r.shutdown:
|
||||
return
|
||||
case <-ticker.C:
|
||||
counter = 0
|
||||
default:
|
||||
if counter < r.n {
|
||||
select {
|
||||
case r.C <- true:
|
||||
counter++
|
||||
case <-r.shutdown:
|
||||
return
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
54
internal/limiter/limiter_test.go
Normal file
54
internal/limiter/limiter_test.go
Normal file
@@ -0,0 +1,54 @@
|
||||
package limiter
|
||||
|
||||
import (
|
||||
"testing"
|
||||
"time"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
)
|
||||
|
||||
func TestRateLimiter(t *testing.T) {
|
||||
r := NewRateLimiter(5, time.Second)
|
||||
ticker := time.NewTicker(time.Millisecond * 75)
|
||||
|
||||
// test that we can only get 5 receives from the rate limiter
|
||||
counter := 0
|
||||
outer:
|
||||
for {
|
||||
select {
|
||||
case <-r.C:
|
||||
counter++
|
||||
case <-ticker.C:
|
||||
break outer
|
||||
}
|
||||
}
|
||||
|
||||
assert.Equal(t, 5, counter)
|
||||
r.Stop()
|
||||
// verify that the Stop function closes the channel.
|
||||
_, ok := <-r.C
|
||||
assert.False(t, ok)
|
||||
}
|
||||
|
||||
func TestRateLimiterMultipleIterations(t *testing.T) {
|
||||
r := NewRateLimiter(5, time.Millisecond*50)
|
||||
ticker := time.NewTicker(time.Millisecond * 250)
|
||||
|
||||
// test that we can get 15 receives from the rate limiter
|
||||
counter := 0
|
||||
outer:
|
||||
for {
|
||||
select {
|
||||
case <-ticker.C:
|
||||
break outer
|
||||
case <-r.C:
|
||||
counter++
|
||||
}
|
||||
}
|
||||
|
||||
assert.True(t, counter > 10)
|
||||
r.Stop()
|
||||
// verify that the Stop function closes the channel.
|
||||
_, ok := <-r.C
|
||||
assert.False(t, ok)
|
||||
}
|
||||
@@ -2,81 +2,79 @@ package internal_models
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
|
||||
"github.com/gobwas/glob"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/filter"
|
||||
)
|
||||
|
||||
// TagFilter is the name of a tag, and the values on which to filter
|
||||
type TagFilter struct {
|
||||
Name string
|
||||
Filter []string
|
||||
filter glob.Glob
|
||||
filter filter.Filter
|
||||
}
|
||||
|
||||
// Filter containing drop/pass and tagdrop/tagpass rules
|
||||
type Filter struct {
|
||||
NameDrop []string
|
||||
nameDrop glob.Glob
|
||||
nameDrop filter.Filter
|
||||
NamePass []string
|
||||
namePass glob.Glob
|
||||
namePass filter.Filter
|
||||
|
||||
FieldDrop []string
|
||||
fieldDrop glob.Glob
|
||||
fieldDrop filter.Filter
|
||||
FieldPass []string
|
||||
fieldPass glob.Glob
|
||||
fieldPass filter.Filter
|
||||
|
||||
TagDrop []TagFilter
|
||||
TagPass []TagFilter
|
||||
|
||||
TagExclude []string
|
||||
tagExclude glob.Glob
|
||||
tagExclude filter.Filter
|
||||
TagInclude []string
|
||||
tagInclude glob.Glob
|
||||
tagInclude filter.Filter
|
||||
|
||||
IsActive bool
|
||||
}
|
||||
|
||||
// Compile all Filter lists into glob.Glob objects.
|
||||
// Compile all Filter lists into filter.Filter objects.
|
||||
func (f *Filter) CompileFilter() error {
|
||||
var err error
|
||||
f.nameDrop, err = compileFilter(f.NameDrop)
|
||||
f.nameDrop, err = filter.CompileFilter(f.NameDrop)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'namedrop', %s", err)
|
||||
}
|
||||
f.namePass, err = compileFilter(f.NamePass)
|
||||
f.namePass, err = filter.CompileFilter(f.NamePass)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'namepass', %s", err)
|
||||
}
|
||||
|
||||
f.fieldDrop, err = compileFilter(f.FieldDrop)
|
||||
f.fieldDrop, err = filter.CompileFilter(f.FieldDrop)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'fielddrop', %s", err)
|
||||
}
|
||||
f.fieldPass, err = compileFilter(f.FieldPass)
|
||||
f.fieldPass, err = filter.CompileFilter(f.FieldPass)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'fieldpass', %s", err)
|
||||
}
|
||||
|
||||
f.tagExclude, err = compileFilter(f.TagExclude)
|
||||
f.tagExclude, err = filter.CompileFilter(f.TagExclude)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'tagexclude', %s", err)
|
||||
}
|
||||
f.tagInclude, err = compileFilter(f.TagInclude)
|
||||
f.tagInclude, err = filter.CompileFilter(f.TagInclude)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'taginclude', %s", err)
|
||||
}
|
||||
|
||||
for i, _ := range f.TagDrop {
|
||||
f.TagDrop[i].filter, err = compileFilter(f.TagDrop[i].Filter)
|
||||
f.TagDrop[i].filter, err = filter.CompileFilter(f.TagDrop[i].Filter)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'tagdrop', %s", err)
|
||||
}
|
||||
}
|
||||
for i, _ := range f.TagPass {
|
||||
f.TagPass[i].filter, err = compileFilter(f.TagPass[i].Filter)
|
||||
f.TagPass[i].filter, err = filter.CompileFilter(f.TagPass[i].Filter)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Error compiling 'tagpass', %s", err)
|
||||
}
|
||||
@@ -84,20 +82,6 @@ func (f *Filter) CompileFilter() error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func compileFilter(filter []string) (glob.Glob, error) {
|
||||
if len(filter) == 0 {
|
||||
return nil, nil
|
||||
}
|
||||
var g glob.Glob
|
||||
var err error
|
||||
if len(filter) == 1 {
|
||||
g, err = glob.Compile(filter[0])
|
||||
} else {
|
||||
g, err = glob.Compile("{" + strings.Join(filter, ",") + "}")
|
||||
}
|
||||
return g, err
|
||||
}
|
||||
|
||||
func (f *Filter) ShouldMetricPass(metric telegraf.Metric) bool {
|
||||
if f.ShouldNamePass(metric.Name()) && f.ShouldTagsPass(metric.Tags()) {
|
||||
return true
|
||||
|
||||
@@ -253,51 +253,6 @@ func TestFilter_TagDrop(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestFilter_CompileFilterError(t *testing.T) {
|
||||
f := Filter{
|
||||
NameDrop: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
NamePass: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
FieldDrop: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
FieldPass: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
TagExclude: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
f = Filter{
|
||||
TagInclude: []string{"", ""},
|
||||
}
|
||||
assert.Error(t, f.CompileFilter())
|
||||
filters := []TagFilter{
|
||||
TagFilter{
|
||||
Name: "cpu",
|
||||
Filter: []string{"{foobar}"},
|
||||
}}
|
||||
f = Filter{
|
||||
TagDrop: filters,
|
||||
}
|
||||
require.Error(t, f.CompileFilter())
|
||||
filters = []TagFilter{
|
||||
TagFilter{
|
||||
Name: "cpu",
|
||||
Filter: []string{"{foobar}"},
|
||||
}}
|
||||
f = Filter{
|
||||
TagPass: filters,
|
||||
}
|
||||
require.Error(t, f.CompileFilter())
|
||||
}
|
||||
|
||||
func TestFilter_ShouldMetricsPass(t *testing.T) {
|
||||
m := testutil.TestMetric(1, "testmetric")
|
||||
f := Filter{
|
||||
|
||||
@@ -138,7 +138,7 @@ func (ro *RunningOutput) Write() error {
|
||||
}
|
||||
|
||||
func (ro *RunningOutput) write(metrics []telegraf.Metric) error {
|
||||
if len(metrics) == 0 {
|
||||
if metrics == nil || len(metrics) == 0 {
|
||||
return nil
|
||||
}
|
||||
start := time.Now()
|
||||
|
||||
@@ -45,14 +45,9 @@ func NewMetric(
|
||||
name string,
|
||||
tags map[string]string,
|
||||
fields map[string]interface{},
|
||||
t ...time.Time,
|
||||
t time.Time,
|
||||
) (Metric, error) {
|
||||
var T time.Time
|
||||
if len(t) > 0 {
|
||||
T = t[0]
|
||||
}
|
||||
|
||||
pt, err := client.NewPoint(name, tags, fields, T)
|
||||
pt, err := client.NewPoint(name, tags, fields, t)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
@@ -51,23 +51,6 @@ func TestNewMetricString(t *testing.T) {
|
||||
assert.Equal(t, lineProtoPrecision, m.PrecisionString("s"))
|
||||
}
|
||||
|
||||
func TestNewMetricStringNoTime(t *testing.T) {
|
||||
tags := map[string]string{
|
||||
"host": "localhost",
|
||||
}
|
||||
fields := map[string]interface{}{
|
||||
"usage_idle": float64(99),
|
||||
}
|
||||
m, err := NewMetric("cpu", tags, fields)
|
||||
assert.NoError(t, err)
|
||||
|
||||
lineProto := fmt.Sprintf("cpu,host=localhost usage_idle=99")
|
||||
assert.Equal(t, lineProto, m.String())
|
||||
|
||||
lineProtoPrecision := fmt.Sprintf("cpu,host=localhost usage_idle=99")
|
||||
assert.Equal(t, lineProtoPrecision, m.PrecisionString("s"))
|
||||
}
|
||||
|
||||
func TestNewMetricFailNaN(t *testing.T) {
|
||||
now := time.Now()
|
||||
|
||||
|
||||
@@ -1,104 +1,19 @@
|
||||
package aerospike
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/binary"
|
||||
"fmt"
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"net"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal/errchan"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
|
||||
as "github.com/sparrc/aerospike-client-go"
|
||||
)
|
||||
|
||||
const (
|
||||
MSG_HEADER_SIZE = 8
|
||||
MSG_TYPE = 1 // Info is 1
|
||||
MSG_VERSION = 2
|
||||
)
|
||||
|
||||
var (
|
||||
STATISTICS_COMMAND = []byte("statistics\n")
|
||||
NAMESPACES_COMMAND = []byte("namespaces\n")
|
||||
)
|
||||
|
||||
type aerospikeMessageHeader struct {
|
||||
Version uint8
|
||||
Type uint8
|
||||
DataLen [6]byte
|
||||
}
|
||||
|
||||
type aerospikeMessage struct {
|
||||
aerospikeMessageHeader
|
||||
Data []byte
|
||||
}
|
||||
|
||||
// Taken from aerospike-client-go/types/message.go
|
||||
func (msg *aerospikeMessage) Serialize() []byte {
|
||||
msg.DataLen = msgLenToBytes(int64(len(msg.Data)))
|
||||
buf := bytes.NewBuffer([]byte{})
|
||||
binary.Write(buf, binary.BigEndian, msg.aerospikeMessageHeader)
|
||||
binary.Write(buf, binary.BigEndian, msg.Data[:])
|
||||
return buf.Bytes()
|
||||
}
|
||||
|
||||
type aerospikeInfoCommand struct {
|
||||
msg *aerospikeMessage
|
||||
}
|
||||
|
||||
// Taken from aerospike-client-go/info.go
|
||||
func (nfo *aerospikeInfoCommand) parseMultiResponse() (map[string]string, error) {
|
||||
responses := make(map[string]string)
|
||||
offset := int64(0)
|
||||
begin := int64(0)
|
||||
|
||||
dataLen := int64(len(nfo.msg.Data))
|
||||
|
||||
// Create reusable StringBuilder for performance.
|
||||
for offset < dataLen {
|
||||
b := nfo.msg.Data[offset]
|
||||
|
||||
if b == '\t' {
|
||||
name := nfo.msg.Data[begin:offset]
|
||||
offset++
|
||||
begin = offset
|
||||
|
||||
// Parse field value.
|
||||
for offset < dataLen {
|
||||
if nfo.msg.Data[offset] == '\n' {
|
||||
break
|
||||
}
|
||||
offset++
|
||||
}
|
||||
|
||||
if offset > begin {
|
||||
value := nfo.msg.Data[begin:offset]
|
||||
responses[string(name)] = string(value)
|
||||
} else {
|
||||
responses[string(name)] = ""
|
||||
}
|
||||
offset++
|
||||
begin = offset
|
||||
} else if b == '\n' {
|
||||
if offset > begin {
|
||||
name := nfo.msg.Data[begin:offset]
|
||||
responses[string(name)] = ""
|
||||
}
|
||||
offset++
|
||||
begin = offset
|
||||
} else {
|
||||
offset++
|
||||
}
|
||||
}
|
||||
|
||||
if offset > begin {
|
||||
name := nfo.msg.Data[begin:offset]
|
||||
responses[string(name)] = ""
|
||||
}
|
||||
return responses, nil
|
||||
}
|
||||
|
||||
type Aerospike struct {
|
||||
Servers []string
|
||||
}
|
||||
@@ -115,7 +30,7 @@ func (a *Aerospike) SampleConfig() string {
|
||||
}
|
||||
|
||||
func (a *Aerospike) Description() string {
|
||||
return "Read stats from an aerospike server"
|
||||
return "Read stats from aerospike server(s)"
|
||||
}
|
||||
|
||||
func (a *Aerospike) Gather(acc telegraf.Accumulator) error {
|
||||
@@ -124,214 +39,90 @@ func (a *Aerospike) Gather(acc telegraf.Accumulator) error {
|
||||
}
|
||||
|
||||
var wg sync.WaitGroup
|
||||
|
||||
var outerr error
|
||||
|
||||
errChan := errchan.New(len(a.Servers))
|
||||
wg.Add(len(a.Servers))
|
||||
for _, server := range a.Servers {
|
||||
wg.Add(1)
|
||||
go func(server string) {
|
||||
go func(serv string) {
|
||||
defer wg.Done()
|
||||
outerr = a.gatherServer(server, acc)
|
||||
errChan.C <- a.gatherServer(serv, acc)
|
||||
}(server)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
return outerr
|
||||
return errChan.Error()
|
||||
}
|
||||
|
||||
func (a *Aerospike) gatherServer(host string, acc telegraf.Accumulator) error {
|
||||
aerospikeInfo, err := getMap(STATISTICS_COMMAND, host)
|
||||
func (a *Aerospike) gatherServer(hostport string, acc telegraf.Accumulator) error {
|
||||
host, port, err := net.SplitHostPort(hostport)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Aerospike info failed: %s", err)
|
||||
return err
|
||||
}
|
||||
readAerospikeStats(aerospikeInfo, acc, host, "")
|
||||
namespaces, err := getList(NAMESPACES_COMMAND, host)
|
||||
|
||||
iport, err := strconv.Atoi(port)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Aerospike namespace list failed: %s", err)
|
||||
iport = 3000
|
||||
}
|
||||
for ix := range namespaces {
|
||||
nsInfo, err := getMap([]byte("namespace/"+namespaces[ix]+"\n"), host)
|
||||
if err != nil {
|
||||
return fmt.Errorf("Aerospike namespace '%s' query failed: %s", namespaces[ix], err)
|
||||
|
||||
c, err := as.NewClient(host, iport)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer c.Close()
|
||||
|
||||
nodes := c.GetNodes()
|
||||
for _, n := range nodes {
|
||||
tags := map[string]string{
|
||||
"node_name": n.GetName(),
|
||||
"aerospike_host": hostport,
|
||||
}
|
||||
fields := make(map[string]interface{})
|
||||
stats, err := as.RequestNodeStats(n)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for k, v := range stats {
|
||||
if iv, err := strconv.ParseInt(v, 10, 64); err == nil {
|
||||
fields[strings.Replace(k, "-", "_", -1)] = iv
|
||||
}
|
||||
}
|
||||
acc.AddFields("aerospike_node", fields, tags, time.Now())
|
||||
|
||||
info, err := as.RequestNodeInfo(n, "namespaces")
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
namespaces := strings.Split(info["namespaces"], ";")
|
||||
|
||||
for _, namespace := range namespaces {
|
||||
nTags := copyTags(tags)
|
||||
nTags["namespace"] = namespace
|
||||
nFields := make(map[string]interface{})
|
||||
info, err := as.RequestNodeInfo(n, "namespace/"+namespace)
|
||||
if err != nil {
|
||||
continue
|
||||
}
|
||||
stats := strings.Split(info["namespace/"+namespace], ";")
|
||||
for _, stat := range stats {
|
||||
parts := strings.Split(stat, "=")
|
||||
if len(parts) < 2 {
|
||||
continue
|
||||
}
|
||||
if iv, err := strconv.ParseInt(parts[1], 10, 64); err == nil {
|
||||
nFields[strings.Replace(parts[0], "-", "_", -1)] = iv
|
||||
}
|
||||
}
|
||||
acc.AddFields("aerospike_namespace", nFields, nTags, time.Now())
|
||||
}
|
||||
readAerospikeStats(nsInfo, acc, host, namespaces[ix])
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func getMap(key []byte, host string) (map[string]string, error) {
|
||||
data, err := get(key, host)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("Failed to get data: %s", err)
|
||||
func copyTags(m map[string]string) map[string]string {
|
||||
out := make(map[string]string)
|
||||
for k, v := range m {
|
||||
out[k] = v
|
||||
}
|
||||
parsed, err := unmarshalMapInfo(data, string(key))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("Failed to unmarshal data: %s", err)
|
||||
}
|
||||
|
||||
return parsed, nil
|
||||
}
|
||||
|
||||
func getList(key []byte, host string) ([]string, error) {
|
||||
data, err := get(key, host)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("Failed to get data: %s", err)
|
||||
}
|
||||
parsed, err := unmarshalListInfo(data, string(key))
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("Failed to unmarshal data: %s", err)
|
||||
}
|
||||
|
||||
return parsed, nil
|
||||
}
|
||||
|
||||
func get(key []byte, host string) (map[string]string, error) {
|
||||
var err error
|
||||
var data map[string]string
|
||||
|
||||
asInfo := &aerospikeInfoCommand{
|
||||
msg: &aerospikeMessage{
|
||||
aerospikeMessageHeader: aerospikeMessageHeader{
|
||||
Version: uint8(MSG_VERSION),
|
||||
Type: uint8(MSG_TYPE),
|
||||
DataLen: msgLenToBytes(int64(len(key))),
|
||||
},
|
||||
Data: key,
|
||||
},
|
||||
}
|
||||
|
||||
cmd := asInfo.msg.Serialize()
|
||||
addr, err := net.ResolveTCPAddr("tcp", host)
|
||||
if err != nil {
|
||||
return data, fmt.Errorf("Lookup failed for '%s': %s", host, err)
|
||||
}
|
||||
|
||||
conn, err := net.DialTCP("tcp", nil, addr)
|
||||
if err != nil {
|
||||
return data, fmt.Errorf("Connection failed for '%s': %s", host, err)
|
||||
}
|
||||
defer conn.Close()
|
||||
|
||||
_, err = conn.Write(cmd)
|
||||
if err != nil {
|
||||
return data, fmt.Errorf("Failed to send to '%s': %s", host, err)
|
||||
}
|
||||
|
||||
msgHeader := bytes.NewBuffer(make([]byte, MSG_HEADER_SIZE))
|
||||
_, err = readLenFromConn(conn, msgHeader.Bytes(), MSG_HEADER_SIZE)
|
||||
if err != nil {
|
||||
return data, fmt.Errorf("Failed to read header: %s", err)
|
||||
}
|
||||
err = binary.Read(msgHeader, binary.BigEndian, &asInfo.msg.aerospikeMessageHeader)
|
||||
if err != nil {
|
||||
return data, fmt.Errorf("Failed to unmarshal header: %s", err)
|
||||
}
|
||||
|
||||
msgLen := msgLenFromBytes(asInfo.msg.aerospikeMessageHeader.DataLen)
|
||||
|
||||
if int64(len(asInfo.msg.Data)) != msgLen {
|
||||
asInfo.msg.Data = make([]byte, msgLen)
|
||||
}
|
||||
|
||||
_, err = readLenFromConn(conn, asInfo.msg.Data, len(asInfo.msg.Data))
|
||||
if err != nil {
|
||||
return data, fmt.Errorf("Failed to read from connection to '%s': %s", host, err)
|
||||
}
|
||||
|
||||
data, err = asInfo.parseMultiResponse()
|
||||
if err != nil {
|
||||
return data, fmt.Errorf("Failed to parse response from '%s': %s", host, err)
|
||||
}
|
||||
|
||||
return data, err
|
||||
}
|
||||
|
||||
func readAerospikeStats(
|
||||
stats map[string]string,
|
||||
acc telegraf.Accumulator,
|
||||
host string,
|
||||
namespace string,
|
||||
) {
|
||||
fields := make(map[string]interface{})
|
||||
tags := map[string]string{
|
||||
"aerospike_host": host,
|
||||
"namespace": "_service",
|
||||
}
|
||||
|
||||
if namespace != "" {
|
||||
tags["namespace"] = namespace
|
||||
}
|
||||
for key, value := range stats {
|
||||
// We are going to ignore all string based keys
|
||||
val, err := strconv.ParseInt(value, 10, 64)
|
||||
if err == nil {
|
||||
if strings.Contains(key, "-") {
|
||||
key = strings.Replace(key, "-", "_", -1)
|
||||
}
|
||||
fields[key] = val
|
||||
}
|
||||
}
|
||||
acc.AddFields("aerospike", fields, tags)
|
||||
}
|
||||
|
||||
func unmarshalMapInfo(infoMap map[string]string, key string) (map[string]string, error) {
|
||||
key = strings.TrimSuffix(key, "\n")
|
||||
res := map[string]string{}
|
||||
|
||||
v, exists := infoMap[key]
|
||||
if !exists {
|
||||
return res, fmt.Errorf("Key '%s' missing from info", key)
|
||||
}
|
||||
|
||||
values := strings.Split(v, ";")
|
||||
for i := range values {
|
||||
kv := strings.Split(values[i], "=")
|
||||
if len(kv) > 1 {
|
||||
res[kv[0]] = kv[1]
|
||||
}
|
||||
}
|
||||
|
||||
return res, nil
|
||||
}
|
||||
|
||||
func unmarshalListInfo(infoMap map[string]string, key string) ([]string, error) {
|
||||
key = strings.TrimSuffix(key, "\n")
|
||||
|
||||
v, exists := infoMap[key]
|
||||
if !exists {
|
||||
return []string{}, fmt.Errorf("Key '%s' missing from info", key)
|
||||
}
|
||||
|
||||
values := strings.Split(v, ";")
|
||||
return values, nil
|
||||
}
|
||||
|
||||
func readLenFromConn(c net.Conn, buffer []byte, length int) (total int, err error) {
|
||||
var r int
|
||||
for total < length {
|
||||
r, err = c.Read(buffer[total:length])
|
||||
total += r
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
}
|
||||
return
|
||||
}
|
||||
|
||||
// Taken from aerospike-client-go/types/message.go
|
||||
func msgLenToBytes(DataLen int64) [6]byte {
|
||||
b := make([]byte, 8)
|
||||
binary.BigEndian.PutUint64(b, uint64(DataLen))
|
||||
res := [6]byte{}
|
||||
copy(res[:], b[2:])
|
||||
return res
|
||||
}
|
||||
|
||||
// Taken from aerospike-client-go/types/message.go
|
||||
func msgLenFromBytes(buf [6]byte) int64 {
|
||||
nbytes := append([]byte{0, 0}, buf[:]...)
|
||||
DataLen := binary.BigEndian.Uint64(nbytes)
|
||||
return int64(DataLen)
|
||||
return out
|
||||
}
|
||||
|
||||
func init() {
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
package aerospike
|
||||
|
||||
import (
|
||||
"reflect"
|
||||
"testing"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
@@ -23,96 +22,29 @@ func TestAerospikeStatistics(t *testing.T) {
|
||||
err := a.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
// Only use a few of the metrics
|
||||
asMetrics := []string{
|
||||
"transactions",
|
||||
"stat_write_errs",
|
||||
"stat_read_reqs",
|
||||
"stat_write_reqs",
|
||||
}
|
||||
|
||||
for _, metric := range asMetrics {
|
||||
assert.True(t, acc.HasIntField("aerospike", metric), metric)
|
||||
}
|
||||
|
||||
assert.True(t, acc.HasMeasurement("aerospike_node"))
|
||||
assert.True(t, acc.HasMeasurement("aerospike_namespace"))
|
||||
assert.True(t, acc.HasIntField("aerospike_node", "batch_error"))
|
||||
}
|
||||
|
||||
func TestAerospikeMsgLenFromToBytes(t *testing.T) {
|
||||
var i int64 = 8
|
||||
assert.True(t, i == msgLenFromBytes(msgLenToBytes(i)))
|
||||
}
|
||||
func TestAerospikeStatisticsPartialErr(t *testing.T) {
|
||||
if testing.Short() {
|
||||
t.Skip("Skipping integration test in short mode")
|
||||
}
|
||||
|
||||
a := &Aerospike{
|
||||
Servers: []string{
|
||||
testutil.GetLocalHost() + ":3000",
|
||||
testutil.GetLocalHost() + ":9999",
|
||||
},
|
||||
}
|
||||
|
||||
func TestReadAerospikeStatsNoNamespace(t *testing.T) {
|
||||
// Also test for re-writing
|
||||
var acc testutil.Accumulator
|
||||
stats := map[string]string{
|
||||
"stat-write-errs": "12345",
|
||||
"stat_read_reqs": "12345",
|
||||
}
|
||||
readAerospikeStats(stats, &acc, "host1", "")
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"stat_write_errs": int64(12345),
|
||||
"stat_read_reqs": int64(12345),
|
||||
}
|
||||
tags := map[string]string{
|
||||
"aerospike_host": "host1",
|
||||
"namespace": "_service",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "aerospike", fields, tags)
|
||||
}
|
||||
|
||||
func TestReadAerospikeStatsNamespace(t *testing.T) {
|
||||
var acc testutil.Accumulator
|
||||
stats := map[string]string{
|
||||
"stat_write_errs": "12345",
|
||||
"stat_read_reqs": "12345",
|
||||
}
|
||||
readAerospikeStats(stats, &acc, "host1", "test")
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"stat_write_errs": int64(12345),
|
||||
"stat_read_reqs": int64(12345),
|
||||
}
|
||||
tags := map[string]string{
|
||||
"aerospike_host": "host1",
|
||||
"namespace": "test",
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "aerospike", fields, tags)
|
||||
}
|
||||
|
||||
func TestAerospikeUnmarshalList(t *testing.T) {
|
||||
i := map[string]string{
|
||||
"test": "one;two;three",
|
||||
}
|
||||
|
||||
expected := []string{"one", "two", "three"}
|
||||
|
||||
list, err := unmarshalListInfo(i, "test2")
|
||||
assert.True(t, err != nil)
|
||||
|
||||
list, err = unmarshalListInfo(i, "test")
|
||||
assert.True(t, err == nil)
|
||||
equal := true
|
||||
for ix := range expected {
|
||||
if list[ix] != expected[ix] {
|
||||
equal = false
|
||||
break
|
||||
}
|
||||
}
|
||||
assert.True(t, equal)
|
||||
}
|
||||
|
||||
func TestAerospikeUnmarshalMap(t *testing.T) {
|
||||
i := map[string]string{
|
||||
"test": "key1=value1;key2=value2",
|
||||
}
|
||||
|
||||
expected := map[string]string{
|
||||
"key1": "value1",
|
||||
"key2": "value2",
|
||||
}
|
||||
m, err := unmarshalMapInfo(i, "test")
|
||||
assert.True(t, err == nil)
|
||||
assert.True(t, reflect.DeepEqual(m, expected))
|
||||
err := a.Gather(&acc)
|
||||
require.Error(t, err)
|
||||
|
||||
assert.True(t, acc.HasMeasurement("aerospike_node"))
|
||||
assert.True(t, acc.HasMeasurement("aerospike_namespace"))
|
||||
assert.True(t, acc.HasIntField("aerospike_node", "batch_error"))
|
||||
}
|
||||
|
||||
@@ -5,7 +5,12 @@ import (
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/apache"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/bcache"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/cassandra"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/ceph"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/cgroup"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/chrony"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/cloudwatch"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/conntrack"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/consul"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/couchbase"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/couchdb"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/disque"
|
||||
@@ -15,7 +20,7 @@ import (
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/elasticsearch"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/exec"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/filestat"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/github_webhooks"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/graylog"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/haproxy"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/http_response"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/httpjson"
|
||||
@@ -24,6 +29,7 @@ import (
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/jolokia"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/kafka_consumer"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/leofs"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/logparser"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/lustre2"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/mailchimp"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/memcached"
|
||||
@@ -35,6 +41,8 @@ import (
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/net_response"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/nginx"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/nsq"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/nsq_consumer"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/nstat"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/ntpq"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/passenger"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/phpfpm"
|
||||
@@ -61,6 +69,8 @@ import (
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/trig"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/twemproxy"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/udp_listener"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/varnish"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/webhooks"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/win_perf_counters"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/zfs"
|
||||
_ "github.com/influxdata/telegraf/plugins/inputs/zookeeper"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Telegraf plugin: Apache
|
||||
|
||||
#### Plugin arguments:
|
||||
- **urls** []string: List of apache-status URLs to collect from.
|
||||
- **urls** []string: List of apache-status URLs to collect from. Default is "http://localhost/server-status?auto".
|
||||
|
||||
#### Description
|
||||
|
||||
|
||||
@@ -8,7 +8,6 @@ import (
|
||||
"net/url"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
@@ -21,6 +20,7 @@ type Apache struct {
|
||||
|
||||
var sampleConfig = `
|
||||
## An array of Apache status URI to gather stats.
|
||||
## Default is "http://localhost/server-status?auto".
|
||||
urls = ["http://localhost/server-status?auto"]
|
||||
`
|
||||
|
||||
@@ -33,8 +33,12 @@ func (n *Apache) Description() string {
|
||||
}
|
||||
|
||||
func (n *Apache) Gather(acc telegraf.Accumulator) error {
|
||||
var wg sync.WaitGroup
|
||||
if len(n.Urls) == 0 {
|
||||
n.Urls = []string{"http://localhost/server-status?auto"}
|
||||
}
|
||||
|
||||
var outerr error
|
||||
var errch = make(chan error)
|
||||
|
||||
for _, u := range n.Urls {
|
||||
addr, err := url.Parse(u)
|
||||
@@ -42,14 +46,17 @@ func (n *Apache) Gather(acc telegraf.Accumulator) error {
|
||||
return fmt.Errorf("Unable to parse address '%s': %s", u, err)
|
||||
}
|
||||
|
||||
wg.Add(1)
|
||||
go func(addr *url.URL) {
|
||||
defer wg.Done()
|
||||
outerr = n.gatherUrl(addr, acc)
|
||||
errch <- n.gatherUrl(addr, acc)
|
||||
}(addr)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
// Drain channel, waiting for all requests to finish and save last error.
|
||||
for range n.Urls {
|
||||
if err := <-errch; err != nil {
|
||||
outerr = err
|
||||
}
|
||||
}
|
||||
|
||||
return outerr
|
||||
}
|
||||
|
||||
@@ -36,7 +36,8 @@ func TestHTTPApache(t *testing.T) {
|
||||
defer ts.Close()
|
||||
|
||||
a := Apache{
|
||||
Urls: []string{ts.URL},
|
||||
// Fetch it 2 times to catch possible data races.
|
||||
Urls: []string{ts.URL, ts.URL},
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
|
||||
@@ -148,7 +148,7 @@ func (c cassandraMetric) addTagsFields(out map[string]interface{}) {
|
||||
tokens := parseJmxMetricRequest(r.(map[string]interface{})["mbean"].(string))
|
||||
// Requests with wildcards for keyspace or table names will return nested
|
||||
// maps in the json response
|
||||
if tokens["type"] == "Table" && (tokens["keyspace"] == "*" ||
|
||||
if (tokens["type"] == "Table" || tokens["type"] == "ColumnFamily") && (tokens["keyspace"] == "*" ||
|
||||
tokens["scope"] == "*") {
|
||||
if valuesMap, ok := out["value"]; ok {
|
||||
for k, v := range valuesMap.(map[string]interface{}) {
|
||||
|
||||
109
plugins/inputs/ceph/README.md
Normal file
109
plugins/inputs/ceph/README.md
Normal file
@@ -0,0 +1,109 @@
|
||||
# Ceph Storage Input Plugin
|
||||
|
||||
Collects performance metrics from the MON and OSD nodes in a Ceph storage cluster.
|
||||
|
||||
The plugin works by scanning the configured SocketDir for OSD and MON socket files. When it finds
|
||||
a MON socket, it runs **ceph --admin-daemon $file perfcounters_dump**. For OSDs it runs **ceph --admin-daemon $file perf dump**
|
||||
|
||||
The resulting JSON is parsed and grouped into collections, based on top-level key. Top-level keys are
|
||||
used as collection tags, and all sub-keys are flattened. For example:
|
||||
|
||||
```
|
||||
{
|
||||
"paxos": {
|
||||
"refresh": 9363435,
|
||||
"refresh_latency": {
|
||||
"avgcount": 9363435,
|
||||
"sum": 5378.794002000
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Would be parsed into the following metrics, all of which would be tagged with collection=paxos:
|
||||
|
||||
- refresh = 9363435
|
||||
- refresh_latency.avgcount: 9363435
|
||||
- refresh_latency.sum: 5378.794002000
|
||||
|
||||
|
||||
### Configuration:
|
||||
|
||||
```
|
||||
# Collects performance metrics from the MON and OSD nodes in a Ceph storage cluster.
|
||||
[[inputs.ceph]]
|
||||
## All configuration values are optional, defaults are shown below
|
||||
|
||||
## location of ceph binary
|
||||
ceph_binary = "/usr/bin/ceph"
|
||||
|
||||
## directory in which to look for socket files
|
||||
socket_dir = "/var/run/ceph"
|
||||
|
||||
## prefix of MON and OSD socket files, used to determine socket type
|
||||
mon_prefix = "ceph-mon"
|
||||
osd_prefix = "ceph-osd"
|
||||
|
||||
## suffix used to identify socket files
|
||||
socket_suffix = "asok"
|
||||
```
|
||||
|
||||
### Measurements & Fields:
|
||||
|
||||
All fields are collected under the **ceph** measurement and stored as float64s. For a full list of fields, see the sample perf dumps in ceph_test.go.
|
||||
|
||||
|
||||
### Tags:
|
||||
|
||||
All measurements will have the following tags:
|
||||
|
||||
- type: either 'osd' or 'mon' to indicate which type of node was queried
|
||||
- id: a unique string identifier, parsed from the socket file name for the node
|
||||
- collection: the top-level key under which these fields were reported. Possible values are:
|
||||
- for MON nodes:
|
||||
- cluster
|
||||
- leveldb
|
||||
- mon
|
||||
- paxos
|
||||
- throttle-mon_client_bytes
|
||||
- throttle-mon_daemon_bytes
|
||||
- throttle-msgr_dispatch_throttler-mon
|
||||
- for OSD nodes:
|
||||
- WBThrottle
|
||||
- filestore
|
||||
- leveldb
|
||||
- mutex-FileJournal::completions_lock
|
||||
- mutex-FileJournal::finisher_lock
|
||||
- mutex-FileJournal::write_lock
|
||||
- mutex-FileJournal::writeq_lock
|
||||
- mutex-JOS::ApplyManager::apply_lock
|
||||
- mutex-JOS::ApplyManager::com_lock
|
||||
- mutex-JOS::SubmitManager::lock
|
||||
- mutex-WBThrottle::lock
|
||||
- objecter
|
||||
- osd
|
||||
- recoverystate_perf
|
||||
- throttle-filestore_bytes
|
||||
- throttle-filestore_ops
|
||||
- throttle-msgr_dispatch_throttler-client
|
||||
- throttle-msgr_dispatch_throttler-cluster
|
||||
- throttle-msgr_dispatch_throttler-hb_back_server
|
||||
- throttle-msgr_dispatch_throttler-hb_front_serve
|
||||
- throttle-msgr_dispatch_throttler-hbclient
|
||||
- throttle-msgr_dispatch_throttler-ms_objecter
|
||||
- throttle-objecter_bytes
|
||||
- throttle-objecter_ops
|
||||
- throttle-osd_client_bytes
|
||||
- throttle-osd_client_messages
|
||||
|
||||
|
||||
### Example Output:
|
||||
|
||||
<pre>
|
||||
telegraf -test -config /etc/telegraf/telegraf.conf -config-directory /etc/telegraf/telegraf.d -input-filter ceph
|
||||
* Plugin: ceph, Collection 1
|
||||
> ceph,collection=paxos, id=node-2,role=openstack,type=mon accept_timeout=0,begin=14931264,begin_bytes.avgcount=14931264,begin_bytes.sum=180309683362,begin_keys.avgcount=0,begin_keys.sum=0,begin_latency.avgcount=14931264,begin_latency.sum=9293.29589,collect=1,collect_bytes.avgcount=1,collect_bytes.sum=24,collect_keys.avgcount=1,collect_keys.sum=1,collect_latency.avgcount=1,collect_latency.sum=0.00028,collect_timeout=0,collect_uncommitted=0,commit=14931264,commit_bytes.avgcount=0,commit_bytes.sum=0,commit_keys.avgcount=0,commit_keys.sum=0,commit_latency.avgcount=0,commit_latency.sum=0,lease_ack_timeout=0,lease_timeout=0,new_pn=0,new_pn_latency.avgcount=0,new_pn_latency.sum=0,refresh=14931264,refresh_latency.avgcount=14931264,refresh_latency.sum=8706.98498,restart=4,share_state=0,share_state_bytes.avgcount=0,share_state_bytes.sum=0,share_state_keys.avgcount=0,share_state_keys.sum=0,start_leader=0,start_peon=1,store_state=14931264,store_state_bytes.avgcount=14931264,store_state_bytes.sum=353119959211,store_state_keys.avgcount=14931264,store_state_keys.sum=289807523,store_state_latency.avgcount=14931264,store_state_latency.sum=10952.835724 1462821234814535148
|
||||
> ceph,collection=throttle-mon_client_bytes,id=node-2,type=mon get=1413017,get_or_fail_fail=0,get_or_fail_success=0,get_sum=71211705,max=104857600,put=1413013,put_sum=71211459,take=0,take_sum=0,val=246,wait.avgcount=0,wait.sum=0 1462821234814737219
|
||||
> ceph,collection=throttle-mon_daemon_bytes,id=node-2,type=mon get=4058121,get_or_fail_fail=0,get_or_fail_success=0,get_sum=6027348117,max=419430400,put=4058121,put_sum=6027348117,take=0,take_sum=0,val=0,wait.avgcount=0,wait.sum=0 1462821234814815661
|
||||
> ceph,collection=throttle-msgr_dispatch_throttler-mon,id=node-2,type=mon get=54276277,get_or_fail_fail=0,get_or_fail_success=0,get_sum=370232877040,max=104857600,put=54276277,put_sum=370232877040,take=0,take_sum=0,val=0,wait.avgcount=0,wait.sum=0 1462821234814872064
|
||||
</pre>
|
||||
249
plugins/inputs/ceph/ceph.go
Normal file
249
plugins/inputs/ceph/ceph.go
Normal file
@@ -0,0 +1,249 @@
|
||||
package ceph
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"io/ioutil"
|
||||
"log"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
)
|
||||
|
||||
const (
|
||||
measurement = "ceph"
|
||||
typeMon = "monitor"
|
||||
typeOsd = "osd"
|
||||
osdPrefix = "ceph-osd"
|
||||
monPrefix = "ceph-mon"
|
||||
sockSuffix = "asok"
|
||||
)
|
||||
|
||||
type Ceph struct {
|
||||
CephBinary string
|
||||
OsdPrefix string
|
||||
MonPrefix string
|
||||
SocketDir string
|
||||
SocketSuffix string
|
||||
}
|
||||
|
||||
func (c *Ceph) setDefaults() {
|
||||
if c.CephBinary == "" {
|
||||
c.CephBinary = "/usr/bin/ceph"
|
||||
}
|
||||
|
||||
if c.OsdPrefix == "" {
|
||||
c.OsdPrefix = osdPrefix
|
||||
}
|
||||
|
||||
if c.MonPrefix == "" {
|
||||
c.MonPrefix = monPrefix
|
||||
}
|
||||
|
||||
if c.SocketDir == "" {
|
||||
c.SocketDir = "/var/run/ceph"
|
||||
}
|
||||
|
||||
if c.SocketSuffix == "" {
|
||||
c.SocketSuffix = sockSuffix
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Ceph) Description() string {
|
||||
return "Collects performance metrics from the MON and OSD nodes in a Ceph storage cluster."
|
||||
}
|
||||
|
||||
var sampleConfig = `
|
||||
## All configuration values are optional, defaults are shown below
|
||||
|
||||
## location of ceph binary
|
||||
ceph_binary = "/usr/bin/ceph"
|
||||
|
||||
## directory in which to look for socket files
|
||||
socket_dir = "/var/run/ceph"
|
||||
|
||||
## prefix of MON and OSD socket files, used to determine socket type
|
||||
mon_prefix = "ceph-mon"
|
||||
osd_prefix = "ceph-osd"
|
||||
|
||||
## suffix used to identify socket files
|
||||
socket_suffix = "asok"
|
||||
`
|
||||
|
||||
func (c *Ceph) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
|
||||
func (c *Ceph) Gather(acc telegraf.Accumulator) error {
|
||||
c.setDefaults()
|
||||
sockets, err := findSockets(c)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to find sockets at path '%s': %v", c.SocketDir, err)
|
||||
}
|
||||
|
||||
for _, s := range sockets {
|
||||
dump, err := perfDump(c.CephBinary, s)
|
||||
if err != nil {
|
||||
log.Printf("error reading from socket '%s': %v", s.socket, err)
|
||||
continue
|
||||
}
|
||||
data, err := parseDump(dump)
|
||||
if err != nil {
|
||||
log.Printf("error parsing dump from socket '%s': %v", s.socket, err)
|
||||
continue
|
||||
}
|
||||
for tag, metrics := range *data {
|
||||
acc.AddFields(measurement,
|
||||
map[string]interface{}(metrics),
|
||||
map[string]string{"type": s.sockType, "id": s.sockId, "collection": tag})
|
||||
}
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add(measurement, func() telegraf.Input { return &Ceph{} })
|
||||
}
|
||||
|
||||
var perfDump = func(binary string, socket *socket) (string, error) {
|
||||
cmdArgs := []string{"--admin-daemon", socket.socket}
|
||||
if socket.sockType == typeOsd {
|
||||
cmdArgs = append(cmdArgs, "perf", "dump")
|
||||
} else if socket.sockType == typeMon {
|
||||
cmdArgs = append(cmdArgs, "perfcounters_dump")
|
||||
} else {
|
||||
return "", fmt.Errorf("ignoring unknown socket type: %s", socket.sockType)
|
||||
}
|
||||
|
||||
cmd := exec.Command(binary, cmdArgs...)
|
||||
var out bytes.Buffer
|
||||
cmd.Stdout = &out
|
||||
err := cmd.Run()
|
||||
if err != nil {
|
||||
return "", fmt.Errorf("error running ceph dump: %s", err)
|
||||
}
|
||||
|
||||
return out.String(), nil
|
||||
}
|
||||
|
||||
var findSockets = func(c *Ceph) ([]*socket, error) {
|
||||
listing, err := ioutil.ReadDir(c.SocketDir)
|
||||
if err != nil {
|
||||
return []*socket{}, fmt.Errorf("Failed to read socket directory '%s': %v", c.SocketDir, err)
|
||||
}
|
||||
sockets := make([]*socket, 0, len(listing))
|
||||
for _, info := range listing {
|
||||
f := info.Name()
|
||||
var sockType string
|
||||
var sockPrefix string
|
||||
if strings.HasPrefix(f, c.MonPrefix) {
|
||||
sockType = typeMon
|
||||
sockPrefix = monPrefix
|
||||
}
|
||||
if strings.HasPrefix(f, c.OsdPrefix) {
|
||||
sockType = typeOsd
|
||||
sockPrefix = osdPrefix
|
||||
|
||||
}
|
||||
if sockType == typeOsd || sockType == typeMon {
|
||||
path := filepath.Join(c.SocketDir, f)
|
||||
sockets = append(sockets, &socket{parseSockId(f, sockPrefix, c.SocketSuffix), sockType, path})
|
||||
}
|
||||
}
|
||||
return sockets, nil
|
||||
}
|
||||
|
||||
func parseSockId(fname, prefix, suffix string) string {
|
||||
s := fname
|
||||
s = strings.TrimPrefix(s, prefix)
|
||||
s = strings.TrimSuffix(s, suffix)
|
||||
s = strings.Trim(s, ".-_")
|
||||
return s
|
||||
}
|
||||
|
||||
type socket struct {
|
||||
sockId string
|
||||
sockType string
|
||||
socket string
|
||||
}
|
||||
|
||||
type metric struct {
|
||||
pathStack []string // lifo stack of name components
|
||||
value float64
|
||||
}
|
||||
|
||||
// Pops names of pathStack to build the flattened name for a metric
|
||||
func (m *metric) name() string {
|
||||
buf := bytes.Buffer{}
|
||||
for i := len(m.pathStack) - 1; i >= 0; i-- {
|
||||
if buf.Len() > 0 {
|
||||
buf.WriteString(".")
|
||||
}
|
||||
buf.WriteString(m.pathStack[i])
|
||||
}
|
||||
return buf.String()
|
||||
}
|
||||
|
||||
type metricMap map[string]interface{}
|
||||
|
||||
type taggedMetricMap map[string]metricMap
|
||||
|
||||
// Parses a raw JSON string into a taggedMetricMap
|
||||
// Delegates the actual parsing to newTaggedMetricMap(..)
|
||||
func parseDump(dump string) (*taggedMetricMap, error) {
|
||||
data := make(map[string]interface{})
|
||||
err := json.Unmarshal([]byte(dump), &data)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to parse json: '%s': %v", dump, err)
|
||||
}
|
||||
|
||||
tmm := newTaggedMetricMap(data)
|
||||
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to tag dataset: '%v': %v", tmm, err)
|
||||
}
|
||||
|
||||
return tmm, nil
|
||||
}
|
||||
|
||||
// Builds a TaggedMetricMap out of a generic string map.
|
||||
// The top-level key is used as a tag and all sub-keys are flattened into metrics
|
||||
func newTaggedMetricMap(data map[string]interface{}) *taggedMetricMap {
|
||||
tmm := make(taggedMetricMap)
|
||||
for tag, datapoints := range data {
|
||||
mm := make(metricMap)
|
||||
for _, m := range flatten(datapoints) {
|
||||
mm[m.name()] = m.value
|
||||
}
|
||||
tmm[tag] = mm
|
||||
}
|
||||
return &tmm
|
||||
}
|
||||
|
||||
// Recursively flattens any k-v hierarchy present in data.
|
||||
// Nested keys are flattened into ordered slices associated with a metric value.
|
||||
// The key slices are treated as stacks, and are expected to be reversed and concatenated
|
||||
// when passed as metrics to the accumulator. (see (*metric).name())
|
||||
func flatten(data interface{}) []*metric {
|
||||
var metrics []*metric
|
||||
|
||||
switch val := data.(type) {
|
||||
case float64:
|
||||
metrics = []*metric{&metric{make([]string, 0, 1), val}}
|
||||
case map[string]interface{}:
|
||||
metrics = make([]*metric, 0, len(val))
|
||||
for k, v := range val {
|
||||
for _, m := range flatten(v) {
|
||||
m.pathStack = append(m.pathStack, k)
|
||||
metrics = append(metrics, m)
|
||||
}
|
||||
}
|
||||
default:
|
||||
log.Printf("Ignoring unexpected type '%T' for value %v", val, val)
|
||||
}
|
||||
|
||||
return metrics
|
||||
}
|
||||
682
plugins/inputs/ceph/ceph_test.go
Normal file
682
plugins/inputs/ceph/ceph_test.go
Normal file
@@ -0,0 +1,682 @@
|
||||
package ceph
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"io/ioutil"
|
||||
"os"
|
||||
"path"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
const (
|
||||
epsilon = float64(0.00000001)
|
||||
)
|
||||
|
||||
func TestParseSockId(t *testing.T) {
|
||||
s := parseSockId(sockFile(osdPrefix, 1), osdPrefix, sockSuffix)
|
||||
assert.Equal(t, s, "1")
|
||||
}
|
||||
|
||||
func TestParseMonDump(t *testing.T) {
|
||||
dump, err := parseDump(monPerfDump)
|
||||
assert.NoError(t, err)
|
||||
assert.InEpsilon(t, 5678670180, (*dump)["cluster"]["osd_kb_used"], epsilon)
|
||||
assert.InEpsilon(t, 6866.540527000, (*dump)["paxos"]["store_state_latency.sum"], epsilon)
|
||||
}
|
||||
|
||||
func TestParseOsdDump(t *testing.T) {
|
||||
dump, err := parseDump(osdPerfDump)
|
||||
assert.NoError(t, err)
|
||||
assert.InEpsilon(t, 552132.109360000, (*dump)["filestore"]["commitcycle_interval.sum"], epsilon)
|
||||
assert.Equal(t, float64(0), (*dump)["mutex-FileJournal::finisher_lock"]["wait.avgcount"])
|
||||
}
|
||||
|
||||
func TestGather(t *testing.T) {
|
||||
saveFind := findSockets
|
||||
saveDump := perfDump
|
||||
defer func() {
|
||||
findSockets = saveFind
|
||||
perfDump = saveDump
|
||||
}()
|
||||
|
||||
findSockets = func(c *Ceph) ([]*socket, error) {
|
||||
return []*socket{&socket{"osd.1", typeOsd, ""}}, nil
|
||||
}
|
||||
|
||||
perfDump = func(binary string, s *socket) (string, error) {
|
||||
return osdPerfDump, nil
|
||||
}
|
||||
|
||||
acc := &testutil.Accumulator{}
|
||||
c := &Ceph{}
|
||||
c.Gather(acc)
|
||||
|
||||
}
|
||||
|
||||
func TestFindSockets(t *testing.T) {
|
||||
tmpdir, err := ioutil.TempDir("", "socktest")
|
||||
assert.NoError(t, err)
|
||||
defer func() {
|
||||
err := os.Remove(tmpdir)
|
||||
assert.NoError(t, err)
|
||||
}()
|
||||
c := &Ceph{
|
||||
CephBinary: "foo",
|
||||
SocketDir: tmpdir,
|
||||
}
|
||||
|
||||
c.setDefaults()
|
||||
|
||||
for _, st := range sockTestParams {
|
||||
createTestFiles(tmpdir, st)
|
||||
|
||||
sockets, err := findSockets(c)
|
||||
assert.NoError(t, err)
|
||||
|
||||
for i := 1; i <= st.osds; i++ {
|
||||
assertFoundSocket(t, tmpdir, typeOsd, i, sockets)
|
||||
}
|
||||
|
||||
for i := 1; i <= st.mons; i++ {
|
||||
assertFoundSocket(t, tmpdir, typeMon, i, sockets)
|
||||
}
|
||||
cleanupTestFiles(tmpdir, st)
|
||||
}
|
||||
}
|
||||
|
||||
func assertFoundSocket(t *testing.T, dir, sockType string, i int, sockets []*socket) {
|
||||
var prefix string
|
||||
if sockType == typeOsd {
|
||||
prefix = osdPrefix
|
||||
} else {
|
||||
prefix = monPrefix
|
||||
}
|
||||
expected := path.Join(dir, sockFile(prefix, i))
|
||||
found := false
|
||||
for _, s := range sockets {
|
||||
fmt.Printf("Checking %s\n", s.socket)
|
||||
if s.socket == expected {
|
||||
found = true
|
||||
assert.Equal(t, s.sockType, sockType, "Unexpected socket type for '%s'", s)
|
||||
assert.Equal(t, s.sockId, strconv.Itoa(i))
|
||||
}
|
||||
}
|
||||
assert.True(t, found, "Did not find socket: %s", expected)
|
||||
}
|
||||
|
||||
func sockFile(prefix string, i int) string {
|
||||
return strings.Join([]string{prefix, strconv.Itoa(i), sockSuffix}, ".")
|
||||
}
|
||||
|
||||
func createTestFiles(dir string, st *SockTest) {
|
||||
writeFile := func(prefix string, i int) {
|
||||
f := sockFile(prefix, i)
|
||||
fpath := path.Join(dir, f)
|
||||
ioutil.WriteFile(fpath, []byte(""), 0777)
|
||||
}
|
||||
tstFileApply(st, writeFile)
|
||||
}
|
||||
|
||||
func cleanupTestFiles(dir string, st *SockTest) {
|
||||
rmFile := func(prefix string, i int) {
|
||||
f := sockFile(prefix, i)
|
||||
fpath := path.Join(dir, f)
|
||||
err := os.Remove(fpath)
|
||||
if err != nil {
|
||||
fmt.Printf("Error removing test file %s: %v\n", fpath, err)
|
||||
}
|
||||
}
|
||||
tstFileApply(st, rmFile)
|
||||
}
|
||||
|
||||
func tstFileApply(st *SockTest, fn func(prefix string, i int)) {
|
||||
for i := 1; i <= st.osds; i++ {
|
||||
fn(osdPrefix, i)
|
||||
}
|
||||
for i := 1; i <= st.mons; i++ {
|
||||
fn(monPrefix, i)
|
||||
}
|
||||
}
|
||||
|
||||
type SockTest struct {
|
||||
osds int
|
||||
mons int
|
||||
}
|
||||
|
||||
var sockTestParams = []*SockTest{
|
||||
&SockTest{
|
||||
osds: 2,
|
||||
mons: 2,
|
||||
},
|
||||
&SockTest{
|
||||
mons: 1,
|
||||
},
|
||||
&SockTest{
|
||||
osds: 1,
|
||||
},
|
||||
&SockTest{},
|
||||
}
|
||||
|
||||
var monPerfDump = `
|
||||
{ "cluster": { "num_mon": 2,
|
||||
"num_mon_quorum": 2,
|
||||
"num_osd": 26,
|
||||
"num_osd_up": 26,
|
||||
"num_osd_in": 26,
|
||||
"osd_epoch": 3306,
|
||||
"osd_kb": 11487846448,
|
||||
"osd_kb_used": 5678670180,
|
||||
"osd_kb_avail": 5809176268,
|
||||
"num_pool": 12,
|
||||
"num_pg": 768,
|
||||
"num_pg_active_clean": 768,
|
||||
"num_pg_active": 768,
|
||||
"num_pg_peering": 0,
|
||||
"num_object": 397616,
|
||||
"num_object_degraded": 0,
|
||||
"num_object_unfound": 0,
|
||||
"num_bytes": 2917848227467,
|
||||
"num_mds_up": 0,
|
||||
"num_mds_in": 0,
|
||||
"num_mds_failed": 0,
|
||||
"mds_epoch": 1},
|
||||
"leveldb": { "leveldb_get": 321950312,
|
||||
"leveldb_transaction": 18729922,
|
||||
"leveldb_compact": 0,
|
||||
"leveldb_compact_range": 74141,
|
||||
"leveldb_compact_queue_merge": 0,
|
||||
"leveldb_compact_queue_len": 0},
|
||||
"mon": {},
|
||||
"paxos": { "start_leader": 0,
|
||||
"start_peon": 1,
|
||||
"restart": 4,
|
||||
"refresh": 9363435,
|
||||
"refresh_latency": { "avgcount": 9363435,
|
||||
"sum": 5378.794002000},
|
||||
"begin": 9363435,
|
||||
"begin_keys": { "avgcount": 0,
|
||||
"sum": 0},
|
||||
"begin_bytes": { "avgcount": 9363435,
|
||||
"sum": 110468605489},
|
||||
"begin_latency": { "avgcount": 9363435,
|
||||
"sum": 5850.060682000},
|
||||
"commit": 9363435,
|
||||
"commit_keys": { "avgcount": 0,
|
||||
"sum": 0},
|
||||
"commit_bytes": { "avgcount": 0,
|
||||
"sum": 0},
|
||||
"commit_latency": { "avgcount": 0,
|
||||
"sum": 0.000000000},
|
||||
"collect": 1,
|
||||
"collect_keys": { "avgcount": 1,
|
||||
"sum": 1},
|
||||
"collect_bytes": { "avgcount": 1,
|
||||
"sum": 24},
|
||||
"collect_latency": { "avgcount": 1,
|
||||
"sum": 0.000280000},
|
||||
"collect_uncommitted": 0,
|
||||
"collect_timeout": 0,
|
||||
"accept_timeout": 0,
|
||||
"lease_ack_timeout": 0,
|
||||
"lease_timeout": 0,
|
||||
"store_state": 9363435,
|
||||
"store_state_keys": { "avgcount": 9363435,
|
||||
"sum": 176572789},
|
||||
"store_state_bytes": { "avgcount": 9363435,
|
||||
"sum": 216355887217},
|
||||
"store_state_latency": { "avgcount": 9363435,
|
||||
"sum": 6866.540527000},
|
||||
"share_state": 0,
|
||||
"share_state_keys": { "avgcount": 0,
|
||||
"sum": 0},
|
||||
"share_state_bytes": { "avgcount": 0,
|
||||
"sum": 0},
|
||||
"new_pn": 0,
|
||||
"new_pn_latency": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"throttle-mon_client_bytes": { "val": 246,
|
||||
"max": 104857600,
|
||||
"get": 896030,
|
||||
"get_sum": 45854374,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 896026,
|
||||
"put_sum": 45854128,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"throttle-mon_daemon_bytes": { "val": 0,
|
||||
"max": 419430400,
|
||||
"get": 2773768,
|
||||
"get_sum": 3627676976,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 2773768,
|
||||
"put_sum": 3627676976,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"throttle-msgr_dispatch_throttler-mon": { "val": 0,
|
||||
"max": 104857600,
|
||||
"get": 34504949,
|
||||
"get_sum": 226860281124,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 34504949,
|
||||
"put_sum": 226860281124,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}}}
|
||||
`
|
||||
|
||||
var osdPerfDump = `
|
||||
{ "WBThrottle": { "bytes_dirtied": 28405539,
|
||||
"bytes_wb": 0,
|
||||
"ios_dirtied": 93,
|
||||
"ios_wb": 0,
|
||||
"inodes_dirtied": 86,
|
||||
"inodes_wb": 0},
|
||||
"filestore": { "journal_queue_max_ops": 0,
|
||||
"journal_queue_ops": 0,
|
||||
"journal_ops": 1108008,
|
||||
"journal_queue_max_bytes": 0,
|
||||
"journal_queue_bytes": 0,
|
||||
"journal_bytes": 73233416196,
|
||||
"journal_latency": { "avgcount": 1108008,
|
||||
"sum": 290.981036000},
|
||||
"journal_wr": 1091866,
|
||||
"journal_wr_bytes": { "avgcount": 1091866,
|
||||
"sum": 74925682688},
|
||||
"journal_full": 0,
|
||||
"committing": 0,
|
||||
"commitcycle": 110389,
|
||||
"commitcycle_interval": { "avgcount": 110389,
|
||||
"sum": 552132.109360000},
|
||||
"commitcycle_latency": { "avgcount": 110389,
|
||||
"sum": 178.657804000},
|
||||
"op_queue_max_ops": 50,
|
||||
"op_queue_ops": 0,
|
||||
"ops": 1108008,
|
||||
"op_queue_max_bytes": 104857600,
|
||||
"op_queue_bytes": 0,
|
||||
"bytes": 73226768148,
|
||||
"apply_latency": { "avgcount": 1108008,
|
||||
"sum": 947.742722000},
|
||||
"queue_transaction_latency_avg": { "avgcount": 1108008,
|
||||
"sum": 0.511327000}},
|
||||
"leveldb": { "leveldb_get": 4361221,
|
||||
"leveldb_transaction": 4351276,
|
||||
"leveldb_compact": 0,
|
||||
"leveldb_compact_range": 0,
|
||||
"leveldb_compact_queue_merge": 0,
|
||||
"leveldb_compact_queue_len": 0},
|
||||
"mutex-FileJournal::completions_lock": { "wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"mutex-FileJournal::finisher_lock": { "wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"mutex-FileJournal::write_lock": { "wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"mutex-FileJournal::writeq_lock": { "wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"mutex-JOS::ApplyManager::apply_lock": { "wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"mutex-JOS::ApplyManager::com_lock": { "wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"mutex-JOS::SubmitManager::lock": { "wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"mutex-WBThrottle::lock": { "wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"objecter": { "op_active": 0,
|
||||
"op_laggy": 0,
|
||||
"op_send": 0,
|
||||
"op_send_bytes": 0,
|
||||
"op_resend": 0,
|
||||
"op_ack": 0,
|
||||
"op_commit": 0,
|
||||
"op": 0,
|
||||
"op_r": 0,
|
||||
"op_w": 0,
|
||||
"op_rmw": 0,
|
||||
"op_pg": 0,
|
||||
"osdop_stat": 0,
|
||||
"osdop_create": 0,
|
||||
"osdop_read": 0,
|
||||
"osdop_write": 0,
|
||||
"osdop_writefull": 0,
|
||||
"osdop_append": 0,
|
||||
"osdop_zero": 0,
|
||||
"osdop_truncate": 0,
|
||||
"osdop_delete": 0,
|
||||
"osdop_mapext": 0,
|
||||
"osdop_sparse_read": 0,
|
||||
"osdop_clonerange": 0,
|
||||
"osdop_getxattr": 0,
|
||||
"osdop_setxattr": 0,
|
||||
"osdop_cmpxattr": 0,
|
||||
"osdop_rmxattr": 0,
|
||||
"osdop_resetxattrs": 0,
|
||||
"osdop_tmap_up": 0,
|
||||
"osdop_tmap_put": 0,
|
||||
"osdop_tmap_get": 0,
|
||||
"osdop_call": 0,
|
||||
"osdop_watch": 0,
|
||||
"osdop_notify": 0,
|
||||
"osdop_src_cmpxattr": 0,
|
||||
"osdop_pgls": 0,
|
||||
"osdop_pgls_filter": 0,
|
||||
"osdop_other": 0,
|
||||
"linger_active": 0,
|
||||
"linger_send": 0,
|
||||
"linger_resend": 0,
|
||||
"poolop_active": 0,
|
||||
"poolop_send": 0,
|
||||
"poolop_resend": 0,
|
||||
"poolstat_active": 0,
|
||||
"poolstat_send": 0,
|
||||
"poolstat_resend": 0,
|
||||
"statfs_active": 0,
|
||||
"statfs_send": 0,
|
||||
"statfs_resend": 0,
|
||||
"command_active": 0,
|
||||
"command_send": 0,
|
||||
"command_resend": 0,
|
||||
"map_epoch": 3300,
|
||||
"map_full": 0,
|
||||
"map_inc": 3293,
|
||||
"osd_sessions": 0,
|
||||
"osd_session_open": 0,
|
||||
"osd_session_close": 0,
|
||||
"osd_laggy": 0},
|
||||
"osd": { "opq": 0,
|
||||
"op_wip": 0,
|
||||
"op": 23939,
|
||||
"op_in_bytes": 1245903961,
|
||||
"op_out_bytes": 29103083856,
|
||||
"op_latency": { "avgcount": 23939,
|
||||
"sum": 440.192015000},
|
||||
"op_process_latency": { "avgcount": 23939,
|
||||
"sum": 30.170685000},
|
||||
"op_r": 23112,
|
||||
"op_r_out_bytes": 29103056146,
|
||||
"op_r_latency": { "avgcount": 23112,
|
||||
"sum": 19.373526000},
|
||||
"op_r_process_latency": { "avgcount": 23112,
|
||||
"sum": 14.625928000},
|
||||
"op_w": 549,
|
||||
"op_w_in_bytes": 1245804358,
|
||||
"op_w_rlat": { "avgcount": 549,
|
||||
"sum": 17.022299000},
|
||||
"op_w_latency": { "avgcount": 549,
|
||||
"sum": 418.494610000},
|
||||
"op_w_process_latency": { "avgcount": 549,
|
||||
"sum": 13.316555000},
|
||||
"op_rw": 278,
|
||||
"op_rw_in_bytes": 99603,
|
||||
"op_rw_out_bytes": 27710,
|
||||
"op_rw_rlat": { "avgcount": 278,
|
||||
"sum": 2.213785000},
|
||||
"op_rw_latency": { "avgcount": 278,
|
||||
"sum": 2.323879000},
|
||||
"op_rw_process_latency": { "avgcount": 278,
|
||||
"sum": 2.228202000},
|
||||
"subop": 1074774,
|
||||
"subop_in_bytes": 26841811636,
|
||||
"subop_latency": { "avgcount": 1074774,
|
||||
"sum": 745.509160000},
|
||||
"subop_w": 0,
|
||||
"subop_w_in_bytes": 26841811636,
|
||||
"subop_w_latency": { "avgcount": 1074774,
|
||||
"sum": 745.509160000},
|
||||
"subop_pull": 0,
|
||||
"subop_pull_latency": { "avgcount": 0,
|
||||
"sum": 0.000000000},
|
||||
"subop_push": 0,
|
||||
"subop_push_in_bytes": 0,
|
||||
"subop_push_latency": { "avgcount": 0,
|
||||
"sum": 0.000000000},
|
||||
"pull": 0,
|
||||
"push": 28,
|
||||
"push_out_bytes": 103483392,
|
||||
"push_in": 0,
|
||||
"push_in_bytes": 0,
|
||||
"recovery_ops": 15,
|
||||
"loadavg": 202,
|
||||
"buffer_bytes": 0,
|
||||
"numpg": 18,
|
||||
"numpg_primary": 8,
|
||||
"numpg_replica": 10,
|
||||
"numpg_stray": 0,
|
||||
"heartbeat_to_peers": 10,
|
||||
"heartbeat_from_peers": 0,
|
||||
"map_messages": 7413,
|
||||
"map_message_epochs": 9792,
|
||||
"map_message_epoch_dups": 10105,
|
||||
"messages_delayed_for_map": 83,
|
||||
"stat_bytes": 102123175936,
|
||||
"stat_bytes_used": 49961820160,
|
||||
"stat_bytes_avail": 52161355776,
|
||||
"copyfrom": 0,
|
||||
"tier_promote": 0,
|
||||
"tier_flush": 0,
|
||||
"tier_flush_fail": 0,
|
||||
"tier_try_flush": 0,
|
||||
"tier_try_flush_fail": 0,
|
||||
"tier_evict": 0,
|
||||
"tier_whiteout": 0,
|
||||
"tier_dirty": 230,
|
||||
"tier_clean": 0,
|
||||
"tier_delay": 0,
|
||||
"agent_wake": 0,
|
||||
"agent_skip": 0,
|
||||
"agent_flush": 0,
|
||||
"agent_evict": 0},
|
||||
"recoverystate_perf": { "initial_latency": { "avgcount": 473,
|
||||
"sum": 0.027207000},
|
||||
"started_latency": { "avgcount": 1480,
|
||||
"sum": 9854902.397648000},
|
||||
"reset_latency": { "avgcount": 1953,
|
||||
"sum": 0.096206000},
|
||||
"start_latency": { "avgcount": 1953,
|
||||
"sum": 0.059947000},
|
||||
"primary_latency": { "avgcount": 765,
|
||||
"sum": 4688922.186935000},
|
||||
"peering_latency": { "avgcount": 704,
|
||||
"sum": 1668.652135000},
|
||||
"backfilling_latency": { "avgcount": 0,
|
||||
"sum": 0.000000000},
|
||||
"waitremotebackfillreserved_latency": { "avgcount": 0,
|
||||
"sum": 0.000000000},
|
||||
"waitlocalbackfillreserved_latency": { "avgcount": 0,
|
||||
"sum": 0.000000000},
|
||||
"notbackfilling_latency": { "avgcount": 0,
|
||||
"sum": 0.000000000},
|
||||
"repnotrecovering_latency": { "avgcount": 462,
|
||||
"sum": 5158922.114600000},
|
||||
"repwaitrecoveryreserved_latency": { "avgcount": 15,
|
||||
"sum": 0.008275000},
|
||||
"repwaitbackfillreserved_latency": { "avgcount": 1,
|
||||
"sum": 0.000095000},
|
||||
"RepRecovering_latency": { "avgcount": 16,
|
||||
"sum": 2274.944727000},
|
||||
"activating_latency": { "avgcount": 514,
|
||||
"sum": 261.008520000},
|
||||
"waitlocalrecoveryreserved_latency": { "avgcount": 20,
|
||||
"sum": 0.175422000},
|
||||
"waitremoterecoveryreserved_latency": { "avgcount": 20,
|
||||
"sum": 0.682778000},
|
||||
"recovering_latency": { "avgcount": 20,
|
||||
"sum": 0.697551000},
|
||||
"recovered_latency": { "avgcount": 511,
|
||||
"sum": 0.011038000},
|
||||
"clean_latency": { "avgcount": 503,
|
||||
"sum": 4686961.154278000},
|
||||
"active_latency": { "avgcount": 506,
|
||||
"sum": 4687223.640464000},
|
||||
"replicaactive_latency": { "avgcount": 446,
|
||||
"sum": 5161197.078966000},
|
||||
"stray_latency": { "avgcount": 794,
|
||||
"sum": 4805.105128000},
|
||||
"getinfo_latency": { "avgcount": 704,
|
||||
"sum": 1138.477937000},
|
||||
"getlog_latency": { "avgcount": 678,
|
||||
"sum": 0.036393000},
|
||||
"waitactingchange_latency": { "avgcount": 69,
|
||||
"sum": 59.172893000},
|
||||
"incomplete_latency": { "avgcount": 0,
|
||||
"sum": 0.000000000},
|
||||
"getmissing_latency": { "avgcount": 609,
|
||||
"sum": 0.012288000},
|
||||
"waitupthru_latency": { "avgcount": 576,
|
||||
"sum": 530.106999000}},
|
||||
"throttle-filestore_bytes": { "val": 0,
|
||||
"max": 0,
|
||||
"get": 0,
|
||||
"get_sum": 0,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 0,
|
||||
"put_sum": 0,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"throttle-filestore_ops": { "val": 0,
|
||||
"max": 0,
|
||||
"get": 0,
|
||||
"get_sum": 0,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 0,
|
||||
"put_sum": 0,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"throttle-msgr_dispatch_throttler-client": { "val": 0,
|
||||
"max": 104857600,
|
||||
"get": 130730,
|
||||
"get_sum": 1246039872,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 130730,
|
||||
"put_sum": 1246039872,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"throttle-msgr_dispatch_throttler-cluster": { "val": 0,
|
||||
"max": 104857600,
|
||||
"get": 1108033,
|
||||
"get_sum": 71277949992,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 1108033,
|
||||
"put_sum": 71277949992,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"throttle-msgr_dispatch_throttler-hb_back_server": { "val": 0,
|
||||
"max": 104857600,
|
||||
"get": 18320575,
|
||||
"get_sum": 861067025,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 18320575,
|
||||
"put_sum": 861067025,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"throttle-msgr_dispatch_throttler-hb_front_server": { "val": 0,
|
||||
"max": 104857600,
|
||||
"get": 18320575,
|
||||
"get_sum": 861067025,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 18320575,
|
||||
"put_sum": 861067025,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"throttle-msgr_dispatch_throttler-hbclient": { "val": 0,
|
||||
"max": 104857600,
|
||||
"get": 40479394,
|
||||
"get_sum": 1902531518,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 40479394,
|
||||
"put_sum": 1902531518,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"throttle-msgr_dispatch_throttler-ms_objecter": { "val": 0,
|
||||
"max": 104857600,
|
||||
"get": 0,
|
||||
"get_sum": 0,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 0,
|
||||
"put_sum": 0,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"throttle-objecter_bytes": { "val": 0,
|
||||
"max": 104857600,
|
||||
"get": 0,
|
||||
"get_sum": 0,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 0,
|
||||
"put_sum": 0,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"throttle-objecter_ops": { "val": 0,
|
||||
"max": 1024,
|
||||
"get": 0,
|
||||
"get_sum": 0,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 0,
|
||||
"put_sum": 0,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"throttle-osd_client_bytes": { "val": 0,
|
||||
"max": 524288000,
|
||||
"get": 24241,
|
||||
"get_sum": 1241992581,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 25958,
|
||||
"put_sum": 1241992581,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}},
|
||||
"throttle-osd_client_messages": { "val": 0,
|
||||
"max": 100,
|
||||
"get": 49214,
|
||||
"get_sum": 49214,
|
||||
"get_or_fail_fail": 0,
|
||||
"get_or_fail_success": 0,
|
||||
"take": 0,
|
||||
"take_sum": 0,
|
||||
"put": 49214,
|
||||
"put_sum": 49214,
|
||||
"wait": { "avgcount": 0,
|
||||
"sum": 0.000000000}}}
|
||||
`
|
||||
59
plugins/inputs/cgroup/README.md
Normal file
59
plugins/inputs/cgroup/README.md
Normal file
@@ -0,0 +1,59 @@
|
||||
# CGroup Input Plugin For Telegraf Agent
|
||||
|
||||
This input plugin will capture specific statistics per cgroup.
|
||||
|
||||
Following file formats are supported:
|
||||
|
||||
* Single value
|
||||
|
||||
```
|
||||
VAL\n
|
||||
```
|
||||
|
||||
* New line separated values
|
||||
|
||||
```
|
||||
VAL0\n
|
||||
VAL1\n
|
||||
```
|
||||
|
||||
* Space separated values
|
||||
|
||||
```
|
||||
VAL0 VAL1 ...\n
|
||||
```
|
||||
|
||||
* New line separated key-space-value's
|
||||
|
||||
```
|
||||
KEY0 VAL0\n
|
||||
KEY1 VAL1\n
|
||||
```
|
||||
|
||||
|
||||
### Tags:
|
||||
|
||||
Measurements don't have any specific tags unless you define them at the telegraf level (defaults). We
|
||||
used to have the path listed as a tag, but to keep cardinality in check it's easier to move this
|
||||
value to a field. Thanks @sebito91!
|
||||
|
||||
|
||||
### Configuration:
|
||||
|
||||
```
|
||||
# [[inputs.cgroup]]
|
||||
# paths = [
|
||||
# "/cgroup/memory", # root cgroup
|
||||
# "/cgroup/memory/child1", # container cgroup
|
||||
# "/cgroup/memory/child2/*", # all children cgroups under child2, but not child2 itself
|
||||
# ]
|
||||
# files = ["memory.*usage*", "memory.limit_in_bytes"]
|
||||
|
||||
# [[inputs.cgroup]]
|
||||
# paths = [
|
||||
# "/cgroup/cpu", # root cgroup
|
||||
# "/cgroup/cpu/*", # all container cgroups
|
||||
# "/cgroup/cpu/*/*", # all children cgroups under each container cgroup
|
||||
# ]
|
||||
# files = ["cpuacct.usage", "cpu.cfs_period_us", "cpu.cfs_quota_us"]
|
||||
```
|
||||
35
plugins/inputs/cgroup/cgroup.go
Normal file
35
plugins/inputs/cgroup/cgroup.go
Normal file
@@ -0,0 +1,35 @@
|
||||
package cgroup
|
||||
|
||||
import (
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
type CGroup struct {
|
||||
Paths []string `toml:"paths"`
|
||||
Files []string `toml:"files"`
|
||||
}
|
||||
|
||||
var sampleConfig = `
|
||||
## Directories in which to look for files, globs are supported.
|
||||
# paths = [
|
||||
# "/cgroup/memory",
|
||||
# "/cgroup/memory/child1",
|
||||
# "/cgroup/memory/child2/*",
|
||||
# ]
|
||||
## cgroup stat fields, as file names, globs are supported.
|
||||
## these file names are appended to each path from above.
|
||||
# files = ["memory.*usage*", "memory.limit_in_bytes"]
|
||||
`
|
||||
|
||||
func (g *CGroup) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
|
||||
func (g *CGroup) Description() string {
|
||||
return "Read specific statistics per cgroup"
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("cgroup", func() telegraf.Input { return &CGroup{} })
|
||||
}
|
||||
243
plugins/inputs/cgroup/cgroup_linux.go
Normal file
243
plugins/inputs/cgroup/cgroup_linux.go
Normal file
@@ -0,0 +1,243 @@
|
||||
// +build linux
|
||||
|
||||
package cgroup
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"os"
|
||||
"path"
|
||||
"path/filepath"
|
||||
"regexp"
|
||||
"strconv"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
)
|
||||
|
||||
const metricName = "cgroup"
|
||||
|
||||
func (g *CGroup) Gather(acc telegraf.Accumulator) error {
|
||||
list := make(chan pathInfo)
|
||||
go g.generateDirs(list)
|
||||
|
||||
for dir := range list {
|
||||
if dir.err != nil {
|
||||
return dir.err
|
||||
}
|
||||
if err := g.gatherDir(dir.path, acc); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func (g *CGroup) gatherDir(dir string, acc telegraf.Accumulator) error {
|
||||
fields := make(map[string]interface{})
|
||||
|
||||
list := make(chan pathInfo)
|
||||
go g.generateFiles(dir, list)
|
||||
|
||||
for file := range list {
|
||||
if file.err != nil {
|
||||
return file.err
|
||||
}
|
||||
|
||||
raw, err := ioutil.ReadFile(file.path)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if len(raw) == 0 {
|
||||
continue
|
||||
}
|
||||
|
||||
fd := fileData{data: raw, path: file.path}
|
||||
if err := fd.parse(fields); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
fields["path"] = dir
|
||||
|
||||
acc.AddFields(metricName, fields, nil)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
// ======================================================================
|
||||
|
||||
type pathInfo struct {
|
||||
path string
|
||||
err error
|
||||
}
|
||||
|
||||
func isDir(path string) (bool, error) {
|
||||
result, err := os.Stat(path)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
return result.IsDir(), nil
|
||||
}
|
||||
|
||||
func (g *CGroup) generateDirs(list chan<- pathInfo) {
|
||||
for _, dir := range g.Paths {
|
||||
// getting all dirs that match the pattern 'dir'
|
||||
items, err := filepath.Glob(dir)
|
||||
if err != nil {
|
||||
list <- pathInfo{err: err}
|
||||
return
|
||||
}
|
||||
|
||||
for _, item := range items {
|
||||
ok, err := isDir(item)
|
||||
if err != nil {
|
||||
list <- pathInfo{err: err}
|
||||
return
|
||||
}
|
||||
// supply only dirs
|
||||
if ok {
|
||||
list <- pathInfo{path: item}
|
||||
}
|
||||
}
|
||||
}
|
||||
close(list)
|
||||
}
|
||||
|
||||
func (g *CGroup) generateFiles(dir string, list chan<- pathInfo) {
|
||||
for _, file := range g.Files {
|
||||
// getting all file paths that match the pattern 'dir + file'
|
||||
// path.Base make sure that file variable does not contains part of path
|
||||
items, err := filepath.Glob(path.Join(dir, path.Base(file)))
|
||||
if err != nil {
|
||||
list <- pathInfo{err: err}
|
||||
return
|
||||
}
|
||||
|
||||
for _, item := range items {
|
||||
ok, err := isDir(item)
|
||||
if err != nil {
|
||||
list <- pathInfo{err: err}
|
||||
return
|
||||
}
|
||||
// supply only files not dirs
|
||||
if !ok {
|
||||
list <- pathInfo{path: item}
|
||||
}
|
||||
}
|
||||
}
|
||||
close(list)
|
||||
}
|
||||
|
||||
// ======================================================================
|
||||
|
||||
type fileData struct {
|
||||
data []byte
|
||||
path string
|
||||
}
|
||||
|
||||
func (fd *fileData) format() (*fileFormat, error) {
|
||||
for _, ff := range fileFormats {
|
||||
ok, err := ff.match(fd.data)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if ok {
|
||||
return &ff, nil
|
||||
}
|
||||
}
|
||||
|
||||
return nil, fmt.Errorf("%v: unknown file format", fd.path)
|
||||
}
|
||||
|
||||
func (fd *fileData) parse(fields map[string]interface{}) error {
|
||||
format, err := fd.format()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
format.parser(filepath.Base(fd.path), fields, fd.data)
|
||||
return nil
|
||||
}
|
||||
|
||||
// ======================================================================
|
||||
|
||||
type fileFormat struct {
|
||||
name string
|
||||
pattern string
|
||||
parser func(measurement string, fields map[string]interface{}, b []byte)
|
||||
}
|
||||
|
||||
const keyPattern = "[[:alpha:]_]+"
|
||||
const valuePattern = "[\\d-]+"
|
||||
|
||||
var fileFormats = [...]fileFormat{
|
||||
// VAL\n
|
||||
fileFormat{
|
||||
name: "Single value",
|
||||
pattern: "^" + valuePattern + "\n$",
|
||||
parser: func(measurement string, fields map[string]interface{}, b []byte) {
|
||||
re := regexp.MustCompile("^(" + valuePattern + ")\n$")
|
||||
matches := re.FindAllStringSubmatch(string(b), -1)
|
||||
fields[measurement] = numberOrString(matches[0][1])
|
||||
},
|
||||
},
|
||||
// VAL0\n
|
||||
// VAL1\n
|
||||
// ...
|
||||
fileFormat{
|
||||
name: "New line separated values",
|
||||
pattern: "^(" + valuePattern + "\n){2,}$",
|
||||
parser: func(measurement string, fields map[string]interface{}, b []byte) {
|
||||
re := regexp.MustCompile("(" + valuePattern + ")\n")
|
||||
matches := re.FindAllStringSubmatch(string(b), -1)
|
||||
for i, v := range matches {
|
||||
fields[measurement+"."+strconv.Itoa(i)] = numberOrString(v[1])
|
||||
}
|
||||
},
|
||||
},
|
||||
// VAL0 VAL1 ...\n
|
||||
fileFormat{
|
||||
name: "Space separated values",
|
||||
pattern: "^(" + valuePattern + " )+\n$",
|
||||
parser: func(measurement string, fields map[string]interface{}, b []byte) {
|
||||
re := regexp.MustCompile("(" + valuePattern + ") ")
|
||||
matches := re.FindAllStringSubmatch(string(b), -1)
|
||||
for i, v := range matches {
|
||||
fields[measurement+"."+strconv.Itoa(i)] = numberOrString(v[1])
|
||||
}
|
||||
},
|
||||
},
|
||||
// KEY0 VAL0\n
|
||||
// KEY1 VAL1\n
|
||||
// ...
|
||||
fileFormat{
|
||||
name: "New line separated key-space-value's",
|
||||
pattern: "^(" + keyPattern + " " + valuePattern + "\n)+$",
|
||||
parser: func(measurement string, fields map[string]interface{}, b []byte) {
|
||||
re := regexp.MustCompile("(" + keyPattern + ") (" + valuePattern + ")\n")
|
||||
matches := re.FindAllStringSubmatch(string(b), -1)
|
||||
for _, v := range matches {
|
||||
fields[measurement+"."+v[1]] = numberOrString(v[2])
|
||||
}
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
func numberOrString(s string) interface{} {
|
||||
i, err := strconv.Atoi(s)
|
||||
if err == nil {
|
||||
return i
|
||||
}
|
||||
|
||||
return s
|
||||
}
|
||||
|
||||
func (f fileFormat) match(b []byte) (bool, error) {
|
||||
ok, err := regexp.Match(f.pattern, b)
|
||||
if err != nil {
|
||||
return false, err
|
||||
}
|
||||
if ok {
|
||||
return true, nil
|
||||
}
|
||||
return false, nil
|
||||
}
|
||||
11
plugins/inputs/cgroup/cgroup_notlinux.go
Normal file
11
plugins/inputs/cgroup/cgroup_notlinux.go
Normal file
@@ -0,0 +1,11 @@
|
||||
// +build !linux
|
||||
|
||||
package cgroup
|
||||
|
||||
import (
|
||||
"github.com/influxdata/telegraf"
|
||||
)
|
||||
|
||||
func (g *CGroup) Gather(acc telegraf.Accumulator) error {
|
||||
return nil
|
||||
}
|
||||
194
plugins/inputs/cgroup/cgroup_test.go
Normal file
194
plugins/inputs/cgroup/cgroup_test.go
Normal file
@@ -0,0 +1,194 @@
|
||||
// +build linux
|
||||
|
||||
package cgroup
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"testing"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"reflect"
|
||||
)
|
||||
|
||||
var cg1 = &CGroup{
|
||||
Paths: []string{"testdata/memory"},
|
||||
Files: []string{
|
||||
"memory.empty",
|
||||
"memory.max_usage_in_bytes",
|
||||
"memory.limit_in_bytes",
|
||||
"memory.stat",
|
||||
"memory.use_hierarchy",
|
||||
"notify_on_release",
|
||||
},
|
||||
}
|
||||
|
||||
func assertContainsFields(a *testutil.Accumulator, t *testing.T, measurement string, fieldSet []map[string]interface{}) {
|
||||
a.Lock()
|
||||
defer a.Unlock()
|
||||
|
||||
numEquals := 0
|
||||
for _, p := range a.Metrics {
|
||||
if p.Measurement == measurement {
|
||||
for _, fields := range fieldSet {
|
||||
if reflect.DeepEqual(fields, p.Fields) {
|
||||
numEquals++
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if numEquals != len(fieldSet) {
|
||||
assert.Fail(t, fmt.Sprintf("only %d of %d are equal", numEquals, len(fieldSet)))
|
||||
}
|
||||
}
|
||||
|
||||
func TestCgroupStatistics_1(t *testing.T) {
|
||||
var acc testutil.Accumulator
|
||||
|
||||
err := cg1.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"memory.stat.cache": 1739362304123123123,
|
||||
"memory.stat.rss": 1775325184,
|
||||
"memory.stat.rss_huge": 778043392,
|
||||
"memory.stat.mapped_file": 421036032,
|
||||
"memory.stat.dirty": -307200,
|
||||
"memory.max_usage_in_bytes.0": 0,
|
||||
"memory.max_usage_in_bytes.1": -1,
|
||||
"memory.max_usage_in_bytes.2": 2,
|
||||
"memory.limit_in_bytes": 223372036854771712,
|
||||
"memory.use_hierarchy": "12-781",
|
||||
"notify_on_release": 0,
|
||||
"path": "testdata/memory",
|
||||
}
|
||||
assertContainsFields(&acc, t, "cgroup", []map[string]interface{}{fields})
|
||||
}
|
||||
|
||||
// ======================================================================
|
||||
|
||||
var cg2 = &CGroup{
|
||||
Paths: []string{"testdata/cpu"},
|
||||
Files: []string{"cpuacct.usage_percpu"},
|
||||
}
|
||||
|
||||
func TestCgroupStatistics_2(t *testing.T) {
|
||||
var acc testutil.Accumulator
|
||||
|
||||
err := cg2.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"cpuacct.usage_percpu.0": -1452543795404,
|
||||
"cpuacct.usage_percpu.1": 1376681271659,
|
||||
"cpuacct.usage_percpu.2": 1450950799997,
|
||||
"cpuacct.usage_percpu.3": -1473113374257,
|
||||
"path": "testdata/cpu",
|
||||
}
|
||||
assertContainsFields(&acc, t, "cgroup", []map[string]interface{}{fields})
|
||||
}
|
||||
|
||||
// ======================================================================
|
||||
|
||||
var cg3 = &CGroup{
|
||||
Paths: []string{"testdata/memory/*"},
|
||||
Files: []string{"memory.limit_in_bytes"},
|
||||
}
|
||||
|
||||
func TestCgroupStatistics_3(t *testing.T) {
|
||||
var acc testutil.Accumulator
|
||||
|
||||
err := cg3.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"memory.limit_in_bytes": 223372036854771712,
|
||||
"path": "testdata/memory/group_1",
|
||||
}
|
||||
|
||||
fieldsTwo := map[string]interface{}{
|
||||
"memory.limit_in_bytes": 223372036854771712,
|
||||
"path": "testdata/memory/group_2",
|
||||
}
|
||||
assertContainsFields(&acc, t, "cgroup", []map[string]interface{}{fields, fieldsTwo})
|
||||
}
|
||||
|
||||
// ======================================================================
|
||||
|
||||
var cg4 = &CGroup{
|
||||
Paths: []string{"testdata/memory/*/*", "testdata/memory/group_2"},
|
||||
Files: []string{"memory.limit_in_bytes"},
|
||||
}
|
||||
|
||||
func TestCgroupStatistics_4(t *testing.T) {
|
||||
var acc testutil.Accumulator
|
||||
|
||||
err := cg4.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"memory.limit_in_bytes": 223372036854771712,
|
||||
"path": "testdata/memory/group_1/group_1_1",
|
||||
}
|
||||
|
||||
fieldsTwo := map[string]interface{}{
|
||||
"memory.limit_in_bytes": 223372036854771712,
|
||||
"path": "testdata/memory/group_1/group_1_2",
|
||||
}
|
||||
|
||||
fieldsThree := map[string]interface{}{
|
||||
"memory.limit_in_bytes": 223372036854771712,
|
||||
"path": "testdata/memory/group_2",
|
||||
}
|
||||
|
||||
assertContainsFields(&acc, t, "cgroup", []map[string]interface{}{fields, fieldsTwo, fieldsThree})
|
||||
}
|
||||
|
||||
// ======================================================================
|
||||
|
||||
var cg5 = &CGroup{
|
||||
Paths: []string{"testdata/memory/*/group_1_1"},
|
||||
Files: []string{"memory.limit_in_bytes"},
|
||||
}
|
||||
|
||||
func TestCgroupStatistics_5(t *testing.T) {
|
||||
var acc testutil.Accumulator
|
||||
|
||||
err := cg5.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"memory.limit_in_bytes": 223372036854771712,
|
||||
"path": "testdata/memory/group_1/group_1_1",
|
||||
}
|
||||
|
||||
fieldsTwo := map[string]interface{}{
|
||||
"memory.limit_in_bytes": 223372036854771712,
|
||||
"path": "testdata/memory/group_2/group_1_1",
|
||||
}
|
||||
assertContainsFields(&acc, t, "cgroup", []map[string]interface{}{fields, fieldsTwo})
|
||||
}
|
||||
|
||||
// ======================================================================
|
||||
|
||||
var cg6 = &CGroup{
|
||||
Paths: []string{"testdata/memory"},
|
||||
Files: []string{"memory.us*", "*/memory.kmem.*"},
|
||||
}
|
||||
|
||||
func TestCgroupStatistics_6(t *testing.T) {
|
||||
var acc testutil.Accumulator
|
||||
|
||||
err := cg6.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"memory.usage_in_bytes": 3513667584,
|
||||
"memory.use_hierarchy": "12-781",
|
||||
"memory.kmem.limit_in_bytes": 9223372036854771712,
|
||||
"path": "testdata/memory",
|
||||
}
|
||||
assertContainsFields(&acc, t, "cgroup", []map[string]interface{}{fields})
|
||||
}
|
||||
1
plugins/inputs/cgroup/testdata/blkio/blkio.io_serviced
vendored
Normal file
1
plugins/inputs/cgroup/testdata/blkio/blkio.io_serviced
vendored
Normal file
@@ -0,0 +1 @@
|
||||
Total 0
|
||||
131
plugins/inputs/cgroup/testdata/blkio/blkio.throttle.io_serviced
vendored
Normal file
131
plugins/inputs/cgroup/testdata/blkio/blkio.throttle.io_serviced
vendored
Normal file
@@ -0,0 +1,131 @@
|
||||
11:0 Read 0
|
||||
11:0 Write 0
|
||||
11:0 Sync 0
|
||||
11:0 Async 0
|
||||
11:0 Total 0
|
||||
8:0 Read 49134
|
||||
8:0 Write 216703
|
||||
8:0 Sync 177906
|
||||
8:0 Async 87931
|
||||
8:0 Total 265837
|
||||
7:7 Read 0
|
||||
7:7 Write 0
|
||||
7:7 Sync 0
|
||||
7:7 Async 0
|
||||
7:7 Total 0
|
||||
7:6 Read 0
|
||||
7:6 Write 0
|
||||
7:6 Sync 0
|
||||
7:6 Async 0
|
||||
7:6 Total 0
|
||||
7:5 Read 0
|
||||
7:5 Write 0
|
||||
7:5 Sync 0
|
||||
7:5 Async 0
|
||||
7:5 Total 0
|
||||
7:4 Read 0
|
||||
7:4 Write 0
|
||||
7:4 Sync 0
|
||||
7:4 Async 0
|
||||
7:4 Total 0
|
||||
7:3 Read 0
|
||||
7:3 Write 0
|
||||
7:3 Sync 0
|
||||
7:3 Async 0
|
||||
7:3 Total 0
|
||||
7:2 Read 0
|
||||
7:2 Write 0
|
||||
7:2 Sync 0
|
||||
7:2 Async 0
|
||||
7:2 Total 0
|
||||
7:1 Read 0
|
||||
7:1 Write 0
|
||||
7:1 Sync 0
|
||||
7:1 Async 0
|
||||
7:1 Total 0
|
||||
7:0 Read 0
|
||||
7:0 Write 0
|
||||
7:0 Sync 0
|
||||
7:0 Async 0
|
||||
7:0 Total 0
|
||||
1:15 Read 3
|
||||
1:15 Write 0
|
||||
1:15 Sync 0
|
||||
1:15 Async 3
|
||||
1:15 Total 3
|
||||
1:14 Read 3
|
||||
1:14 Write 0
|
||||
1:14 Sync 0
|
||||
1:14 Async 3
|
||||
1:14 Total 3
|
||||
1:13 Read 3
|
||||
1:13 Write 0
|
||||
1:13 Sync 0
|
||||
1:13 Async 3
|
||||
1:13 Total 3
|
||||
1:12 Read 3
|
||||
1:12 Write 0
|
||||
1:12 Sync 0
|
||||
1:12 Async 3
|
||||
1:12 Total 3
|
||||
1:11 Read 3
|
||||
1:11 Write 0
|
||||
1:11 Sync 0
|
||||
1:11 Async 3
|
||||
1:11 Total 3
|
||||
1:10 Read 3
|
||||
1:10 Write 0
|
||||
1:10 Sync 0
|
||||
1:10 Async 3
|
||||
1:10 Total 3
|
||||
1:9 Read 3
|
||||
1:9 Write 0
|
||||
1:9 Sync 0
|
||||
1:9 Async 3
|
||||
1:9 Total 3
|
||||
1:8 Read 3
|
||||
1:8 Write 0
|
||||
1:8 Sync 0
|
||||
1:8 Async 3
|
||||
1:8 Total 3
|
||||
1:7 Read 3
|
||||
1:7 Write 0
|
||||
1:7 Sync 0
|
||||
1:7 Async 3
|
||||
1:7 Total 3
|
||||
1:6 Read 3
|
||||
1:6 Write 0
|
||||
1:6 Sync 0
|
||||
1:6 Async 3
|
||||
1:6 Total 3
|
||||
1:5 Read 3
|
||||
1:5 Write 0
|
||||
1:5 Sync 0
|
||||
1:5 Async 3
|
||||
1:5 Total 3
|
||||
1:4 Read 3
|
||||
1:4 Write 0
|
||||
1:4 Sync 0
|
||||
1:4 Async 3
|
||||
1:4 Total 3
|
||||
1:3 Read 3
|
||||
1:3 Write 0
|
||||
1:3 Sync 0
|
||||
1:3 Async 3
|
||||
1:3 Total 3
|
||||
1:2 Read 3
|
||||
1:2 Write 0
|
||||
1:2 Sync 0
|
||||
1:2 Async 3
|
||||
1:2 Total 3
|
||||
1:1 Read 3
|
||||
1:1 Write 0
|
||||
1:1 Sync 0
|
||||
1:1 Async 3
|
||||
1:1 Total 3
|
||||
1:0 Read 3
|
||||
1:0 Write 0
|
||||
1:0 Sync 0
|
||||
1:0 Async 3
|
||||
1:0 Total 3
|
||||
Total 265885
|
||||
1
plugins/inputs/cgroup/testdata/cpu/cpu.cfs_quota_us
vendored
Normal file
1
plugins/inputs/cgroup/testdata/cpu/cpu.cfs_quota_us
vendored
Normal file
@@ -0,0 +1 @@
|
||||
-1
|
||||
1
plugins/inputs/cgroup/testdata/cpu/cpuacct.usage_percpu
vendored
Normal file
1
plugins/inputs/cgroup/testdata/cpu/cpuacct.usage_percpu
vendored
Normal file
@@ -0,0 +1 @@
|
||||
-1452543795404 1376681271659 1450950799997 -1473113374257
|
||||
1
plugins/inputs/cgroup/testdata/memory/group_1/group_1_1/memory.limit_in_bytes
vendored
Normal file
1
plugins/inputs/cgroup/testdata/memory/group_1/group_1_1/memory.limit_in_bytes
vendored
Normal file
@@ -0,0 +1 @@
|
||||
223372036854771712
|
||||
5
plugins/inputs/cgroup/testdata/memory/group_1/group_1_1/memory.stat
vendored
Normal file
5
plugins/inputs/cgroup/testdata/memory/group_1/group_1_1/memory.stat
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
cache 1739362304123123123
|
||||
rss 1775325184
|
||||
rss_huge 778043392
|
||||
mapped_file 421036032
|
||||
dirty -307200
|
||||
1
plugins/inputs/cgroup/testdata/memory/group_1/group_1_2/memory.limit_in_bytes
vendored
Normal file
1
plugins/inputs/cgroup/testdata/memory/group_1/group_1_2/memory.limit_in_bytes
vendored
Normal file
@@ -0,0 +1 @@
|
||||
223372036854771712
|
||||
5
plugins/inputs/cgroup/testdata/memory/group_1/group_1_2/memory.stat
vendored
Normal file
5
plugins/inputs/cgroup/testdata/memory/group_1/group_1_2/memory.stat
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
cache 1739362304123123123
|
||||
rss 1775325184
|
||||
rss_huge 778043392
|
||||
mapped_file 421036032
|
||||
dirty -307200
|
||||
1
plugins/inputs/cgroup/testdata/memory/group_1/memory.kmem.limit_in_bytes
vendored
Normal file
1
plugins/inputs/cgroup/testdata/memory/group_1/memory.kmem.limit_in_bytes
vendored
Normal file
@@ -0,0 +1 @@
|
||||
9223372036854771712
|
||||
1
plugins/inputs/cgroup/testdata/memory/group_1/memory.kmem.max_usage_in_bytes
vendored
Normal file
1
plugins/inputs/cgroup/testdata/memory/group_1/memory.kmem.max_usage_in_bytes
vendored
Normal file
@@ -0,0 +1 @@
|
||||
0
|
||||
1
plugins/inputs/cgroup/testdata/memory/group_1/memory.limit_in_bytes
vendored
Normal file
1
plugins/inputs/cgroup/testdata/memory/group_1/memory.limit_in_bytes
vendored
Normal file
@@ -0,0 +1 @@
|
||||
223372036854771712
|
||||
5
plugins/inputs/cgroup/testdata/memory/group_1/memory.stat
vendored
Normal file
5
plugins/inputs/cgroup/testdata/memory/group_1/memory.stat
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
cache 1739362304123123123
|
||||
rss 1775325184
|
||||
rss_huge 778043392
|
||||
mapped_file 421036032
|
||||
dirty -307200
|
||||
1
plugins/inputs/cgroup/testdata/memory/group_2/group_1_1/memory.limit_in_bytes
vendored
Normal file
1
plugins/inputs/cgroup/testdata/memory/group_2/group_1_1/memory.limit_in_bytes
vendored
Normal file
@@ -0,0 +1 @@
|
||||
223372036854771712
|
||||
5
plugins/inputs/cgroup/testdata/memory/group_2/group_1_1/memory.stat
vendored
Normal file
5
plugins/inputs/cgroup/testdata/memory/group_2/group_1_1/memory.stat
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
cache 1739362304123123123
|
||||
rss 1775325184
|
||||
rss_huge 778043392
|
||||
mapped_file 421036032
|
||||
dirty -307200
|
||||
1
plugins/inputs/cgroup/testdata/memory/group_2/memory.limit_in_bytes
vendored
Normal file
1
plugins/inputs/cgroup/testdata/memory/group_2/memory.limit_in_bytes
vendored
Normal file
@@ -0,0 +1 @@
|
||||
223372036854771712
|
||||
5
plugins/inputs/cgroup/testdata/memory/group_2/memory.stat
vendored
Normal file
5
plugins/inputs/cgroup/testdata/memory/group_2/memory.stat
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
cache 1739362304123123123
|
||||
rss 1775325184
|
||||
rss_huge 778043392
|
||||
mapped_file 421036032
|
||||
dirty -307200
|
||||
0
plugins/inputs/cgroup/testdata/memory/memory.empty
vendored
Normal file
0
plugins/inputs/cgroup/testdata/memory/memory.empty
vendored
Normal file
1
plugins/inputs/cgroup/testdata/memory/memory.kmem.limit_in_bytes
vendored
Normal file
1
plugins/inputs/cgroup/testdata/memory/memory.kmem.limit_in_bytes
vendored
Normal file
@@ -0,0 +1 @@
|
||||
9223372036854771712
|
||||
1
plugins/inputs/cgroup/testdata/memory/memory.limit_in_bytes
vendored
Normal file
1
plugins/inputs/cgroup/testdata/memory/memory.limit_in_bytes
vendored
Normal file
@@ -0,0 +1 @@
|
||||
223372036854771712
|
||||
3
plugins/inputs/cgroup/testdata/memory/memory.max_usage_in_bytes
vendored
Normal file
3
plugins/inputs/cgroup/testdata/memory/memory.max_usage_in_bytes
vendored
Normal file
@@ -0,0 +1,3 @@
|
||||
0
|
||||
-1
|
||||
2
|
||||
8
plugins/inputs/cgroup/testdata/memory/memory.numa_stat
vendored
Normal file
8
plugins/inputs/cgroup/testdata/memory/memory.numa_stat
vendored
Normal file
@@ -0,0 +1,8 @@
|
||||
total=858067 N0=858067
|
||||
file=406254 N0=406254
|
||||
anon=451792 N0=451792
|
||||
unevictable=21 N0=21
|
||||
hierarchical_total=858067 N0=858067
|
||||
hierarchical_file=406254 N0=406254
|
||||
hierarchical_anon=451792 N0=451792
|
||||
hierarchical_unevictable=21 N0=21
|
||||
5
plugins/inputs/cgroup/testdata/memory/memory.stat
vendored
Normal file
5
plugins/inputs/cgroup/testdata/memory/memory.stat
vendored
Normal file
@@ -0,0 +1,5 @@
|
||||
cache 1739362304123123123
|
||||
rss 1775325184
|
||||
rss_huge 778043392
|
||||
mapped_file 421036032
|
||||
dirty -307200
|
||||
1
plugins/inputs/cgroup/testdata/memory/memory.usage_in_bytes
vendored
Normal file
1
plugins/inputs/cgroup/testdata/memory/memory.usage_in_bytes
vendored
Normal file
@@ -0,0 +1 @@
|
||||
3513667584
|
||||
1
plugins/inputs/cgroup/testdata/memory/memory.use_hierarchy
vendored
Normal file
1
plugins/inputs/cgroup/testdata/memory/memory.use_hierarchy
vendored
Normal file
@@ -0,0 +1 @@
|
||||
12-781
|
||||
1
plugins/inputs/cgroup/testdata/memory/notify_on_release
vendored
Normal file
1
plugins/inputs/cgroup/testdata/memory/notify_on_release
vendored
Normal file
@@ -0,0 +1 @@
|
||||
0
|
||||
92
plugins/inputs/chrony/README.md
Normal file
92
plugins/inputs/chrony/README.md
Normal file
@@ -0,0 +1,92 @@
|
||||
# chrony Input Plugin
|
||||
|
||||
Get standard chrony metrics, requires chronyc executable.
|
||||
|
||||
Below is the documentation of the various headers returned by `chronyc tracking`.
|
||||
|
||||
- Reference ID - This is the refid and name (or IP address) if available, of the
|
||||
server to which the computer is currently synchronised. If this is 127.127.1.1
|
||||
it means the computer is not synchronised to any external source and that you
|
||||
have the ‘local’ mode operating (via the local command in chronyc (see section local),
|
||||
or the local directive in the ‘/etc/chrony.conf’ file (see section local)).
|
||||
- Stratum - The stratum indicates how many hops away from a computer with an attached
|
||||
reference clock we are. Such a computer is a stratum-1 computer, so the computer in the
|
||||
example is two hops away (i.e. a.b.c is a stratum-2 and is synchronised from a stratum-1).
|
||||
- Ref time - This is the time (UTC) at which the last measurement from the reference
|
||||
source was processed.
|
||||
- System time - In normal operation, chronyd never steps the system clock, because any
|
||||
jump in the timescale can have adverse consequences for certain application programs.
|
||||
Instead, any error in the system clock is corrected by slightly speeding up or slowing
|
||||
down the system clock until the error has been removed, and then returning to the system
|
||||
clock’s normal speed. A consequence of this is that there will be a period when the
|
||||
system clock (as read by other programs using the gettimeofday() system call, or by the
|
||||
date command in the shell) will be different from chronyd's estimate of the current true
|
||||
time (which it reports to NTP clients when it is operating in server mode). The value
|
||||
reported on this line is the difference due to this effect.
|
||||
- Last offset - This is the estimated local offset on the last clock update.
|
||||
- RMS offset - This is a long-term average of the offset value.
|
||||
- Frequency - The ‘frequency’ is the rate by which the system’s clock would be
|
||||
wrong if chronyd was not correcting it. It is expressed in ppm (parts per million).
|
||||
For example, a value of 1ppm would mean that when the system’s clock thinks it has
|
||||
advanced 1 second, it has actually advanced by 1.000001 seconds relative to true time.
|
||||
- Residual freq - This shows the ‘residual frequency’ for the currently selected
|
||||
reference source. This reflects any difference between what the measurements from the
|
||||
reference source indicate the frequency should be and the frequency currently being used.
|
||||
The reason this is not always zero is that a smoothing procedure is applied to the
|
||||
frequency. Each time a measurement from the reference source is obtained and a new
|
||||
residual frequency computed, the estimated accuracy of this residual is compared with the
|
||||
estimated accuracy (see ‘skew’ next) of the existing frequency value. A weighted average
|
||||
is computed for the new frequency, with weights depending on these accuracies. If the
|
||||
measurements from the reference source follow a consistent trend, the residual will be
|
||||
driven to zero over time.
|
||||
- Skew - This is the estimated error bound on the frequency.
|
||||
- Root delay - This is the total of the network path delays to the stratum-1 computer
|
||||
from which the computer is ultimately synchronised. In certain extreme situations, this
|
||||
value can be negative. (This can arise in a symmetric peer arrangement where the computers’
|
||||
frequencies are not tracking each other and the network delay is very short relative to the
|
||||
turn-around time at each computer.)
|
||||
- Root dispersion - This is the total dispersion accumulated through all the computers
|
||||
back to the stratum-1 computer from which the computer is ultimately synchronised.
|
||||
Dispersion is due to system clock resolution, statistical measurement variations etc.
|
||||
- Leap status - This is the leap status, which can be Normal, Insert second,
|
||||
Delete second or Not synchronised.
|
||||
|
||||
### Configuration:
|
||||
|
||||
```toml
|
||||
# Get standard chrony metrics, requires chronyc executable.
|
||||
[[inputs.chrony]]
|
||||
## If true, chronyc tries to perform a DNS lookup for the time server.
|
||||
# dns_lookup = false
|
||||
```
|
||||
|
||||
### Measurements & Fields:
|
||||
|
||||
- chrony
|
||||
- last_offset (float, seconds)
|
||||
- rms_offset (float, seconds)
|
||||
- frequency (float, ppm)
|
||||
- residual_freq (float, ppm)
|
||||
- skew (float, ppm)
|
||||
- root_delay (float, seconds)
|
||||
- root_dispersion (float, seconds)
|
||||
- update_interval (float, seconds)
|
||||
|
||||
### Tags:
|
||||
|
||||
- All measurements have the following tags:
|
||||
- reference_id
|
||||
- stratum
|
||||
- leap_status
|
||||
|
||||
### Example Output:
|
||||
|
||||
```
|
||||
$ telegraf -config telegraf.conf -input-filter chrony -test
|
||||
* Plugin: chrony, Collection 1
|
||||
> chrony,leap_status=normal,reference_id=192.168.1.1,stratum=3 frequency=-35.657,last_offset=-0.000013616,residual_freq=-0,rms_offset=0.000027073,root_delay=0.000644,root_dispersion=0.003444,skew=0.001,update_interval=1031.2 1463750789687639161
|
||||
```
|
||||
|
||||
|
||||
|
||||
|
||||
129
plugins/inputs/chrony/chrony.go
Normal file
129
plugins/inputs/chrony/chrony.go
Normal file
@@ -0,0 +1,129 @@
|
||||
// +build linux
|
||||
|
||||
package chrony
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"os/exec"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
var (
|
||||
execCommand = exec.Command // execCommand is used to mock commands in tests.
|
||||
)
|
||||
|
||||
type Chrony struct {
|
||||
DNSLookup bool `toml:"dns_lookup"`
|
||||
path string
|
||||
}
|
||||
|
||||
func (*Chrony) Description() string {
|
||||
return "Get standard chrony metrics, requires chronyc executable."
|
||||
}
|
||||
|
||||
func (*Chrony) SampleConfig() string {
|
||||
return `
|
||||
## If true, chronyc tries to perform a DNS lookup for the time server.
|
||||
# dns_lookup = false
|
||||
`
|
||||
}
|
||||
|
||||
func (c *Chrony) Gather(acc telegraf.Accumulator) error {
|
||||
if len(c.path) == 0 {
|
||||
return errors.New("chronyc not found: verify that chrony is installed and that chronyc is in your PATH")
|
||||
}
|
||||
|
||||
flags := []string{}
|
||||
if !c.DNSLookup {
|
||||
flags = append(flags, "-n")
|
||||
}
|
||||
flags = append(flags, "tracking")
|
||||
|
||||
cmd := execCommand(c.path, flags...)
|
||||
out, err := internal.CombinedOutputTimeout(cmd, time.Second*5)
|
||||
if err != nil {
|
||||
return fmt.Errorf("failed to run command %s: %s - %s", strings.Join(cmd.Args, " "), err, string(out))
|
||||
}
|
||||
fields, tags, err := processChronycOutput(string(out))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
acc.AddFields("chrony", fields, tags)
|
||||
return nil
|
||||
}
|
||||
|
||||
// processChronycOutput takes in a string output from the chronyc command, like:
|
||||
//
|
||||
// Reference ID : 192.168.1.22 (ntp.example.com)
|
||||
// Stratum : 3
|
||||
// Ref time (UTC) : Thu May 12 14:27:07 2016
|
||||
// System time : 0.000020390 seconds fast of NTP time
|
||||
// Last offset : +0.000012651 seconds
|
||||
// RMS offset : 0.000025577 seconds
|
||||
// Frequency : 16.001 ppm slow
|
||||
// Residual freq : -0.000 ppm
|
||||
// Skew : 0.006 ppm
|
||||
// Root delay : 0.001655 seconds
|
||||
// Root dispersion : 0.003307 seconds
|
||||
// Update interval : 507.2 seconds
|
||||
// Leap status : Normal
|
||||
//
|
||||
// The value on the left side of the colon is used as field name, if the first field on
|
||||
// the right side is a float. If it cannot be parsed as float, it is a tag name.
|
||||
//
|
||||
// Ref time is ignored and all names are converted to snake case.
|
||||
//
|
||||
// It returns (<fields>, <tags>)
|
||||
func processChronycOutput(out string) (map[string]interface{}, map[string]string, error) {
|
||||
tags := map[string]string{}
|
||||
fields := map[string]interface{}{}
|
||||
lines := strings.Split(strings.TrimSpace(out), "\n")
|
||||
for _, line := range lines {
|
||||
stats := strings.Split(line, ":")
|
||||
if len(stats) < 2 {
|
||||
return nil, nil, fmt.Errorf("unexpected output from chronyc, expected ':' in %s", out)
|
||||
}
|
||||
name := strings.ToLower(strings.Replace(strings.TrimSpace(stats[0]), " ", "_", -1))
|
||||
// ignore reference time
|
||||
if strings.Contains(name, "time") {
|
||||
continue
|
||||
}
|
||||
valueFields := strings.Fields(stats[1])
|
||||
if len(valueFields) == 0 {
|
||||
return nil, nil, fmt.Errorf("unexpected output from chronyc: %s", out)
|
||||
}
|
||||
if strings.Contains(strings.ToLower(name), "stratum") {
|
||||
tags["stratum"] = valueFields[0]
|
||||
continue
|
||||
}
|
||||
value, err := strconv.ParseFloat(valueFields[0], 64)
|
||||
if err != nil {
|
||||
tags[name] = strings.ToLower(valueFields[0])
|
||||
continue
|
||||
}
|
||||
if strings.Contains(stats[1], "slow") {
|
||||
value = -value
|
||||
}
|
||||
fields[name] = value
|
||||
}
|
||||
|
||||
return fields, tags, nil
|
||||
}
|
||||
|
||||
func init() {
|
||||
c := Chrony{}
|
||||
path, _ := exec.LookPath("chronyc")
|
||||
if len(path) > 0 {
|
||||
c.path = path
|
||||
}
|
||||
inputs.Add("chrony", func() telegraf.Input {
|
||||
return &c
|
||||
})
|
||||
}
|
||||
3
plugins/inputs/chrony/chrony_notlinux.go
Normal file
3
plugins/inputs/chrony/chrony_notlinux.go
Normal file
@@ -0,0 +1,3 @@
|
||||
// +build !linux
|
||||
|
||||
package chrony
|
||||
109
plugins/inputs/chrony/chrony_test.go
Normal file
109
plugins/inputs/chrony/chrony_test.go
Normal file
@@ -0,0 +1,109 @@
|
||||
// +build linux
|
||||
|
||||
package chrony
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"os"
|
||||
"os/exec"
|
||||
"testing"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
)
|
||||
|
||||
func TestGather(t *testing.T) {
|
||||
c := Chrony{
|
||||
path: "chronyc",
|
||||
}
|
||||
// overwriting exec commands with mock commands
|
||||
execCommand = fakeExecCommand
|
||||
defer func() { execCommand = exec.Command }()
|
||||
var acc testutil.Accumulator
|
||||
|
||||
err := c.Gather(&acc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
tags := map[string]string{
|
||||
"reference_id": "192.168.1.22",
|
||||
"leap_status": "normal",
|
||||
"stratum": "3",
|
||||
}
|
||||
fields := map[string]interface{}{
|
||||
"last_offset": 0.000012651,
|
||||
"rms_offset": 0.000025577,
|
||||
"frequency": -16.001,
|
||||
"residual_freq": 0.0,
|
||||
"skew": 0.006,
|
||||
"root_delay": 0.001655,
|
||||
"root_dispersion": 0.003307,
|
||||
"update_interval": 507.2,
|
||||
}
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "chrony", fields, tags)
|
||||
|
||||
// test with dns lookup
|
||||
c.DNSLookup = true
|
||||
err = c.Gather(&acc)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
acc.AssertContainsTaggedFields(t, "chrony", fields, tags)
|
||||
|
||||
}
|
||||
|
||||
// fackeExecCommand is a helper function that mock
|
||||
// the exec.Command call (and call the test binary)
|
||||
func fakeExecCommand(command string, args ...string) *exec.Cmd {
|
||||
cs := []string{"-test.run=TestHelperProcess", "--", command}
|
||||
cs = append(cs, args...)
|
||||
cmd := exec.Command(os.Args[0], cs...)
|
||||
cmd.Env = []string{"GO_WANT_HELPER_PROCESS=1"}
|
||||
return cmd
|
||||
}
|
||||
|
||||
// TestHelperProcess isn't a real test. It's used to mock exec.Command
|
||||
// For example, if you run:
|
||||
// GO_WANT_HELPER_PROCESS=1 go test -test.run=TestHelperProcess -- chrony tracking
|
||||
// it returns below mockData.
|
||||
func TestHelperProcess(t *testing.T) {
|
||||
if os.Getenv("GO_WANT_HELPER_PROCESS") != "1" {
|
||||
return
|
||||
}
|
||||
|
||||
lookup := "Reference ID : 192.168.1.22 (ntp.example.com)\n"
|
||||
noLookup := "Reference ID : 192.168.1.22 (192.168.1.22)\n"
|
||||
mockData := `Stratum : 3
|
||||
Ref time (UTC) : Thu May 12 14:27:07 2016
|
||||
System time : 0.000020390 seconds fast of NTP time
|
||||
Last offset : +0.000012651 seconds
|
||||
RMS offset : 0.000025577 seconds
|
||||
Frequency : 16.001 ppm slow
|
||||
Residual freq : -0.000 ppm
|
||||
Skew : 0.006 ppm
|
||||
Root delay : 0.001655 seconds
|
||||
Root dispersion : 0.003307 seconds
|
||||
Update interval : 507.2 seconds
|
||||
Leap status : Normal
|
||||
`
|
||||
|
||||
args := os.Args
|
||||
|
||||
// Previous arguments are tests stuff, that looks like :
|
||||
// /tmp/go-build970079519/…/_test/integration.test -test.run=TestHelperProcess --
|
||||
cmd, args := args[3], args[4:]
|
||||
|
||||
if cmd == "chronyc" {
|
||||
if args[0] == "tracking" {
|
||||
fmt.Fprint(os.Stdout, lookup+mockData)
|
||||
} else {
|
||||
fmt.Fprint(os.Stdout, noLookup+mockData)
|
||||
}
|
||||
} else {
|
||||
fmt.Fprint(os.Stdout, "command not found")
|
||||
os.Exit(1)
|
||||
|
||||
}
|
||||
os.Exit(0)
|
||||
}
|
||||
@@ -6,9 +6,12 @@ This plugin will pull Metric Statistics from Amazon CloudWatch.
|
||||
|
||||
This plugin uses a credential chain for Authentication with the CloudWatch
|
||||
API endpoint. In the following order the plugin will attempt to authenticate.
|
||||
1. [IAMS Role](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html)
|
||||
2. [Environment Variables](https://github.com/aws/aws-sdk-go/wiki/configuring-sdk#environment-variables)
|
||||
3. [Shared Credentials](https://github.com/aws/aws-sdk-go/wiki/configuring-sdk#shared-credentials-file)
|
||||
1. Assumed credentials via STS if `role_arn` attribute is specified (source credentials are evaluated from subsequent rules)
|
||||
2. Explicit credentials from `access_key`, `secret_key`, and `token` attributes
|
||||
3. Shared profile from `profile` attribute
|
||||
4. [Environment Variables](https://github.com/aws/aws-sdk-go/wiki/configuring-sdk#environment-variables)
|
||||
5. [Shared Credentials](https://github.com/aws/aws-sdk-go/wiki/configuring-sdk#shared-credentials-file)
|
||||
6. [EC2 Instance Profile](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/iam-roles-for-amazon-ec2.html)
|
||||
|
||||
### Configuration:
|
||||
|
||||
@@ -24,7 +27,7 @@ API endpoint. In the following order the plugin will attempt to authenticate.
|
||||
delay = '1m'
|
||||
|
||||
## Override global run interval (optional - defaults to global interval)
|
||||
## Recomended: use metric 'interval' that is a multiple of 'period' to avoid
|
||||
## Recomended: use metric 'interval' that is a multiple of 'period' to avoid
|
||||
## gaps or overlap in pulled data
|
||||
interval = '1m'
|
||||
|
||||
@@ -36,11 +39,15 @@ API endpoint. In the following order the plugin will attempt to authenticate.
|
||||
## Refreshes Namespace available metrics every 1h
|
||||
[[inputs.cloudwatch.metrics]]
|
||||
names = ['Latency', 'RequestCount']
|
||||
|
||||
|
||||
## Dimension filters for Metric (optional)
|
||||
[[inputs.cloudwatch.metrics.dimensions]]
|
||||
name = 'LoadBalancerName'
|
||||
value = 'p-example'
|
||||
|
||||
[[inputs.cloudwatch.metrics.dimensions]]
|
||||
name = 'AvailabilityZone'
|
||||
value = '*'
|
||||
```
|
||||
#### Requirements and Terminology
|
||||
|
||||
@@ -52,6 +59,39 @@ Plugin Configuration utilizes [CloudWatch concepts](http://docs.aws.amazon.com/A
|
||||
- `names` must be valid CloudWatch [Metric](http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/cloudwatch_concepts.html#Metric) names
|
||||
- `dimensions` must be valid CloudWatch [Dimension](http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/cloudwatch_concepts.html#Dimension) name/value pairs
|
||||
|
||||
Omitting or specifying a value of `'*'` for a dimension value configures all available metrics that contain a dimension with the specified name
|
||||
to be retrieved. If specifying >1 dimension, then the metric must contain *all* the configured dimensions where the the value of the
|
||||
wildcard dimension is ignored.
|
||||
|
||||
Example:
|
||||
```
|
||||
[[inputs.cloudwatch.metrics]]
|
||||
names = ['Latency']
|
||||
|
||||
## Dimension filters for Metric (optional)
|
||||
[[inputs.cloudwatch.metrics.dimensions]]
|
||||
name = 'LoadBalancerName'
|
||||
value = 'p-example'
|
||||
|
||||
[[inputs.cloudwatch.metrics.dimensions]]
|
||||
name = 'AvailabilityZone'
|
||||
value = '*'
|
||||
```
|
||||
|
||||
If the following ELBs are available:
|
||||
- name: `p-example`, availabilityZone: `us-east-1a`
|
||||
- name: `p-example`, availabilityZone: `us-east-1b`
|
||||
- name: `q-example`, availabilityZone: `us-east-1a`
|
||||
- name: `q-example`, availabilityZone: `us-east-1b`
|
||||
|
||||
|
||||
Then 2 metrics will be output:
|
||||
- name: `p-example`, availabilityZone: `us-east-1a`
|
||||
- name: `p-example`, availabilityZone: `us-east-1b`
|
||||
|
||||
If the `AvailabilityZone` wildcard dimension was omitted, then a single metric (name: `p-example`)
|
||||
would be exported containing the aggregate values of the ELB across availability zones.
|
||||
|
||||
#### Restrictions and Limitations
|
||||
- CloudWatch metrics are not available instantly via the CloudWatch API. You should adjust your collection `delay` to account for this lag in metrics availability based on your [monitoring subscription level](http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/using-cloudwatch-new.html)
|
||||
- CloudWatch API usage incurs cost - see [GetMetricStatistics Pricing](https://aws.amazon.com/cloudwatch/pricing/)
|
||||
|
||||
@@ -3,28 +3,36 @@ package cloudwatch
|
||||
import (
|
||||
"fmt"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/aws/aws-sdk-go/aws"
|
||||
"github.com/aws/aws-sdk-go/aws/credentials"
|
||||
"github.com/aws/aws-sdk-go/aws/session"
|
||||
|
||||
"github.com/aws/aws-sdk-go/service/cloudwatch"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
internalaws "github.com/influxdata/telegraf/internal/config/aws"
|
||||
"github.com/influxdata/telegraf/internal/errchan"
|
||||
"github.com/influxdata/telegraf/internal/limiter"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
type (
|
||||
CloudWatch struct {
|
||||
Region string `toml:"region"`
|
||||
AccessKey string `toml:"access_key"`
|
||||
SecretKey string `toml:"secret_key"`
|
||||
Region string `toml:"region"`
|
||||
AccessKey string `toml:"access_key"`
|
||||
SecretKey string `toml:"secret_key"`
|
||||
RoleARN string `toml:"role_arn"`
|
||||
Profile string `toml:"profile"`
|
||||
Filename string `toml:"shared_credential_file"`
|
||||
Token string `toml:"token"`
|
||||
|
||||
Period internal.Duration `toml:"period"`
|
||||
Delay internal.Duration `toml:"delay"`
|
||||
Namespace string `toml:"namespace"`
|
||||
Metrics []*Metric `toml:"metrics"`
|
||||
CacheTTL internal.Duration `toml:"cache_ttl"`
|
||||
client cloudwatchClient
|
||||
metricCache *MetricCache
|
||||
}
|
||||
@@ -58,12 +66,18 @@ func (c *CloudWatch) SampleConfig() string {
|
||||
|
||||
## Amazon Credentials
|
||||
## Credentials are loaded in the following order
|
||||
## 1) explicit credentials from 'access_key' and 'secret_key'
|
||||
## 2) environment variables
|
||||
## 3) shared credentials file
|
||||
## 4) EC2 Instance Profile
|
||||
## 1) Assumed credentials via STS if role_arn is specified
|
||||
## 2) explicit credentials from 'access_key' and 'secret_key'
|
||||
## 3) shared profile from 'profile'
|
||||
## 4) environment variables
|
||||
## 5) shared credentials file
|
||||
## 6) EC2 Instance Profile
|
||||
#access_key = ""
|
||||
#secret_key = ""
|
||||
#token = ""
|
||||
#role_arn = ""
|
||||
#profile = ""
|
||||
#shared_credential_file = ""
|
||||
|
||||
## Requested CloudWatch aggregation Period (required - must be a multiple of 60s)
|
||||
period = '1m'
|
||||
@@ -75,6 +89,10 @@ func (c *CloudWatch) SampleConfig() string {
|
||||
## gaps or overlap in pulled data
|
||||
interval = '1m'
|
||||
|
||||
## Configure the TTL for the internal cache of metrics.
|
||||
## Defaults to 1 hr if not specified
|
||||
#cache_ttl = '10m'
|
||||
|
||||
## Metric Statistic Namespace (required)
|
||||
namespace = 'AWS/ELB'
|
||||
|
||||
@@ -106,20 +124,40 @@ func (c *CloudWatch) Gather(acc telegraf.Accumulator) error {
|
||||
if c.Metrics != nil {
|
||||
metrics = []*cloudwatch.Metric{}
|
||||
for _, m := range c.Metrics {
|
||||
dimensions := make([]*cloudwatch.Dimension, len(m.Dimensions))
|
||||
for k, d := range m.Dimensions {
|
||||
dimensions[k] = &cloudwatch.Dimension{
|
||||
Name: aws.String(d.Name),
|
||||
Value: aws.String(d.Value),
|
||||
if !hasWilcard(m.Dimensions) {
|
||||
dimensions := make([]*cloudwatch.Dimension, len(m.Dimensions))
|
||||
for k, d := range m.Dimensions {
|
||||
fmt.Printf("Dimension [%s]:[%s]\n", d.Name, d.Value)
|
||||
dimensions[k] = &cloudwatch.Dimension{
|
||||
Name: aws.String(d.Name),
|
||||
Value: aws.String(d.Value),
|
||||
}
|
||||
}
|
||||
for _, name := range m.MetricNames {
|
||||
metrics = append(metrics, &cloudwatch.Metric{
|
||||
Namespace: aws.String(c.Namespace),
|
||||
MetricName: aws.String(name),
|
||||
Dimensions: dimensions,
|
||||
})
|
||||
}
|
||||
} else {
|
||||
allMetrics, err := c.fetchNamespaceMetrics()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
for _, name := range m.MetricNames {
|
||||
for _, metric := range allMetrics {
|
||||
if isSelected(metric, m.Dimensions) {
|
||||
metrics = append(metrics, &cloudwatch.Metric{
|
||||
Namespace: aws.String(c.Namespace),
|
||||
MetricName: aws.String(name),
|
||||
Dimensions: metric.Dimensions,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, name := range m.MetricNames {
|
||||
metrics = append(metrics, &cloudwatch.Metric{
|
||||
Namespace: aws.String(c.Namespace),
|
||||
MetricName: aws.String(name),
|
||||
Dimensions: dimensions,
|
||||
})
|
||||
}
|
||||
|
||||
}
|
||||
} else {
|
||||
var err error
|
||||
@@ -130,30 +168,35 @@ func (c *CloudWatch) Gather(acc telegraf.Accumulator) error {
|
||||
}
|
||||
|
||||
metricCount := len(metrics)
|
||||
var errChan = make(chan error, metricCount)
|
||||
errChan := errchan.New(metricCount)
|
||||
|
||||
now := time.Now()
|
||||
|
||||
// limit concurrency or we can easily exhaust user connection limit
|
||||
semaphore := make(chan byte, 64)
|
||||
|
||||
// see cloudwatch API request limits:
|
||||
// http://docs.aws.amazon.com/AmazonCloudWatch/latest/DeveloperGuide/cloudwatch_limits.html
|
||||
lmtr := limiter.NewRateLimiter(10, time.Second)
|
||||
defer lmtr.Stop()
|
||||
var wg sync.WaitGroup
|
||||
wg.Add(len(metrics))
|
||||
for _, m := range metrics {
|
||||
semaphore <- 0x1
|
||||
go c.gatherMetric(acc, m, now, semaphore, errChan)
|
||||
<-lmtr.C
|
||||
go func(inm *cloudwatch.Metric) {
|
||||
defer wg.Done()
|
||||
c.gatherMetric(acc, inm, now, errChan.C)
|
||||
}(m)
|
||||
}
|
||||
wg.Wait()
|
||||
|
||||
for i := 1; i <= metricCount; i++ {
|
||||
err := <-errChan
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return nil
|
||||
return errChan.Error()
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("cloudwatch", func() telegraf.Input {
|
||||
return &CloudWatch{}
|
||||
ttl, _ := time.ParseDuration("1hr")
|
||||
return &CloudWatch{
|
||||
CacheTTL: internal.Duration{Duration: ttl},
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
@@ -161,14 +204,18 @@ func init() {
|
||||
* Initialize CloudWatch client
|
||||
*/
|
||||
func (c *CloudWatch) initializeCloudWatch() error {
|
||||
config := &aws.Config{
|
||||
Region: aws.String(c.Region),
|
||||
}
|
||||
if c.AccessKey != "" || c.SecretKey != "" {
|
||||
config.Credentials = credentials.NewStaticCredentials(c.AccessKey, c.SecretKey, "")
|
||||
credentialConfig := &internalaws.CredentialConfig{
|
||||
Region: c.Region,
|
||||
AccessKey: c.AccessKey,
|
||||
SecretKey: c.SecretKey,
|
||||
RoleARN: c.RoleARN,
|
||||
Profile: c.Profile,
|
||||
Filename: c.Filename,
|
||||
Token: c.Token,
|
||||
}
|
||||
configProvider := credentialConfig.Credentials()
|
||||
|
||||
c.client = cloudwatch.New(session.New(config))
|
||||
c.client = cloudwatch.New(configProvider)
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -203,11 +250,10 @@ func (c *CloudWatch) fetchNamespaceMetrics() (metrics []*cloudwatch.Metric, err
|
||||
more = token != nil
|
||||
}
|
||||
|
||||
cacheTTL, _ := time.ParseDuration("1hr")
|
||||
c.metricCache = &MetricCache{
|
||||
Metrics: metrics,
|
||||
Fetched: time.Now(),
|
||||
TTL: cacheTTL,
|
||||
TTL: c.CacheTTL.Duration,
|
||||
}
|
||||
|
||||
return
|
||||
@@ -216,12 +262,16 @@ func (c *CloudWatch) fetchNamespaceMetrics() (metrics []*cloudwatch.Metric, err
|
||||
/*
|
||||
* Gather given Metric and emit any error
|
||||
*/
|
||||
func (c *CloudWatch) gatherMetric(acc telegraf.Accumulator, metric *cloudwatch.Metric, now time.Time, semaphore chan byte, errChan chan error) {
|
||||
func (c *CloudWatch) gatherMetric(
|
||||
acc telegraf.Accumulator,
|
||||
metric *cloudwatch.Metric,
|
||||
now time.Time,
|
||||
errChan chan error,
|
||||
) {
|
||||
params := c.getStatisticsInput(metric, now)
|
||||
resp, err := c.client.GetMetricStatistics(params)
|
||||
if err != nil {
|
||||
errChan <- err
|
||||
<-semaphore
|
||||
return
|
||||
}
|
||||
|
||||
@@ -258,7 +308,6 @@ func (c *CloudWatch) gatherMetric(acc telegraf.Accumulator, metric *cloudwatch.M
|
||||
}
|
||||
|
||||
errChan <- nil
|
||||
<-semaphore
|
||||
}
|
||||
|
||||
/*
|
||||
@@ -309,3 +358,32 @@ func (c *CloudWatch) getStatisticsInput(metric *cloudwatch.Metric, now time.Time
|
||||
func (c *MetricCache) IsValid() bool {
|
||||
return c.Metrics != nil && time.Since(c.Fetched) < c.TTL
|
||||
}
|
||||
|
||||
func hasWilcard(dimensions []*Dimension) bool {
|
||||
for _, d := range dimensions {
|
||||
if d.Value == "" || d.Value == "*" {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func isSelected(metric *cloudwatch.Metric, dimensions []*Dimension) bool {
|
||||
if len(metric.Dimensions) != len(dimensions) {
|
||||
return false
|
||||
}
|
||||
for _, d := range dimensions {
|
||||
selected := false
|
||||
for _, d2 := range metric.Dimensions {
|
||||
if d.Name == *d2.Name {
|
||||
if d.Value == "" || d.Value == "*" || d.Value == *d2.Value {
|
||||
selected = true
|
||||
}
|
||||
}
|
||||
}
|
||||
if !selected {
|
||||
return false
|
||||
}
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
56
plugins/inputs/conntrack/README.md
Normal file
56
plugins/inputs/conntrack/README.md
Normal file
@@ -0,0 +1,56 @@
|
||||
# Conntrack Plugin
|
||||
|
||||
Collects stats from Netfilter's conntrack-tools.
|
||||
|
||||
The conntrack-tools provide a mechanism for tracking various aspects of
|
||||
network connections as they are processed by netfilter. At runtime,
|
||||
conntrack exposes many of those connection statistics within /proc/sys/net.
|
||||
Depending on your kernel version, these files can be found in either
|
||||
/proc/sys/net/ipv4/netfilter or /proc/sys/net/netfilter and will be
|
||||
prefixed with either ip_ or nf_. This plugin reads the files specified
|
||||
in its configuration and publishes each one as a field, with the prefix
|
||||
normalized to ip_.
|
||||
|
||||
In order to simplify configuration in a heterogeneous environment, a superset
|
||||
of directory and filenames can be specified. Any locations that don't exist
|
||||
will be ignored.
|
||||
|
||||
For more information on conntrack-tools, see the
|
||||
[Netfilter Documentation](http://conntrack-tools.netfilter.org/).
|
||||
|
||||
|
||||
### Configuration:
|
||||
|
||||
```toml
|
||||
# Collects conntrack stats from the configured directories and files.
|
||||
[[inputs.conntrack]]
|
||||
## The following defaults would work with multiple versions of conntrack.
|
||||
## Note the nf_ and ip_ filename prefixes are mutually exclusive across
|
||||
## kernel versions, as are the directory locations.
|
||||
|
||||
## Superset of filenames to look for within the conntrack dirs.
|
||||
## Missing files will be ignored.
|
||||
files = ["ip_conntrack_count","ip_conntrack_max",
|
||||
"nf_conntrack_count","nf_conntrack_max"]
|
||||
|
||||
## Directories to search within for the conntrack files above.
|
||||
## Missing directrories will be ignored.
|
||||
dirs = ["/proc/sys/net/ipv4/netfilter","/proc/sys/net/netfilter"]
|
||||
```
|
||||
|
||||
### Measurements & Fields:
|
||||
|
||||
- conntrack
|
||||
- ip_conntrack_count (int, count): the number of entries in the conntrack table
|
||||
- ip_conntrack_max (int, size): the max capacity of the conntrack table
|
||||
|
||||
### Tags:
|
||||
|
||||
This input does not use tags.
|
||||
|
||||
### Example Output:
|
||||
|
||||
```
|
||||
$ ./telegraf -config telegraf.conf -input-filter conntrack -test
|
||||
conntrack,host=myhost ip_conntrack_count=2,ip_conntrack_max=262144 1461620427667995735
|
||||
```
|
||||
119
plugins/inputs/conntrack/conntrack.go
Normal file
119
plugins/inputs/conntrack/conntrack.go
Normal file
@@ -0,0 +1,119 @@
|
||||
// +build linux
|
||||
|
||||
package conntrack
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"os"
|
||||
"strconv"
|
||||
"strings"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"log"
|
||||
"path/filepath"
|
||||
)
|
||||
|
||||
type Conntrack struct {
|
||||
Path string
|
||||
Dirs []string
|
||||
Files []string
|
||||
}
|
||||
|
||||
const (
|
||||
inputName = "conntrack"
|
||||
)
|
||||
|
||||
var dfltDirs = []string{
|
||||
"/proc/sys/net/ipv4/netfilter",
|
||||
"/proc/sys/net/netfilter",
|
||||
}
|
||||
|
||||
var dfltFiles = []string{
|
||||
"ip_conntrack_count",
|
||||
"ip_conntrack_max",
|
||||
"nf_conntrack_count",
|
||||
"nf_conntrack_max",
|
||||
}
|
||||
|
||||
func (c *Conntrack) setDefaults() {
|
||||
if len(c.Dirs) == 0 {
|
||||
c.Dirs = dfltDirs
|
||||
}
|
||||
|
||||
if len(c.Files) == 0 {
|
||||
c.Files = dfltFiles
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Conntrack) Description() string {
|
||||
return "Collects conntrack stats from the configured directories and files."
|
||||
}
|
||||
|
||||
var sampleConfig = `
|
||||
## The following defaults would work with multiple versions of conntrack.
|
||||
## Note the nf_ and ip_ filename prefixes are mutually exclusive across
|
||||
## kernel versions, as are the directory locations.
|
||||
|
||||
## Superset of filenames to look for within the conntrack dirs.
|
||||
## Missing files will be ignored.
|
||||
files = ["ip_conntrack_count","ip_conntrack_max",
|
||||
"nf_conntrack_count","nf_conntrack_max"]
|
||||
|
||||
## Directories to search within for the conntrack files above.
|
||||
## Missing directrories will be ignored.
|
||||
dirs = ["/proc/sys/net/ipv4/netfilter","/proc/sys/net/netfilter"]
|
||||
`
|
||||
|
||||
func (c *Conntrack) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
|
||||
func (c *Conntrack) Gather(acc telegraf.Accumulator) error {
|
||||
c.setDefaults()
|
||||
|
||||
var metricKey string
|
||||
fields := make(map[string]interface{})
|
||||
|
||||
for _, dir := range c.Dirs {
|
||||
for _, file := range c.Files {
|
||||
// NOTE: no system will have both nf_ and ip_ prefixes,
|
||||
// so we're safe to branch on suffix only.
|
||||
parts := strings.SplitN(file, "_", 2)
|
||||
if len(parts) < 2 {
|
||||
continue
|
||||
}
|
||||
metricKey = "ip_" + parts[1]
|
||||
|
||||
fName := filepath.Join(dir, file)
|
||||
if _, err := os.Stat(fName); err != nil {
|
||||
continue
|
||||
}
|
||||
|
||||
contents, err := ioutil.ReadFile(fName)
|
||||
if err != nil {
|
||||
log.Printf("failed to read file '%s': %v", fName, err)
|
||||
}
|
||||
|
||||
v := strings.TrimSpace(string(contents))
|
||||
fields[metricKey], err = strconv.ParseFloat(v, 64)
|
||||
if err != nil {
|
||||
log.Printf("failed to parse metric, expected number but "+
|
||||
" found '%s': %v", v, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if len(fields) == 0 {
|
||||
return fmt.Errorf("Conntrack input failed to collect metrics. " +
|
||||
"Is the conntrack kernel module loaded?")
|
||||
}
|
||||
|
||||
acc.AddFields(inputName, fields, nil)
|
||||
return nil
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add(inputName, func() telegraf.Input { return &Conntrack{} })
|
||||
}
|
||||
3
plugins/inputs/conntrack/conntrack_notlinux.go
Normal file
3
plugins/inputs/conntrack/conntrack_notlinux.go
Normal file
@@ -0,0 +1,3 @@
|
||||
// +build !linux
|
||||
|
||||
package conntrack
|
||||
90
plugins/inputs/conntrack/conntrack_test.go
Normal file
90
plugins/inputs/conntrack/conntrack_test.go
Normal file
@@ -0,0 +1,90 @@
|
||||
// +build linux
|
||||
|
||||
package conntrack
|
||||
|
||||
import (
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"io/ioutil"
|
||||
"os"
|
||||
"path"
|
||||
"strconv"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func restoreDflts(savedFiles, savedDirs []string) {
|
||||
dfltFiles = savedFiles
|
||||
dfltDirs = savedDirs
|
||||
}
|
||||
|
||||
func TestNoFilesFound(t *testing.T) {
|
||||
defer restoreDflts(dfltFiles, dfltDirs)
|
||||
|
||||
dfltFiles = []string{"baz.txt"}
|
||||
dfltDirs = []string{"./foo/bar"}
|
||||
c := &Conntrack{}
|
||||
acc := &testutil.Accumulator{}
|
||||
err := c.Gather(acc)
|
||||
|
||||
assert.EqualError(t, err, "Conntrack input failed to collect metrics. "+
|
||||
"Is the conntrack kernel module loaded?")
|
||||
}
|
||||
|
||||
func TestDefaultsUsed(t *testing.T) {
|
||||
defer restoreDflts(dfltFiles, dfltDirs)
|
||||
tmpdir, err := ioutil.TempDir("", "tmp1")
|
||||
assert.NoError(t, err)
|
||||
defer os.Remove(tmpdir)
|
||||
|
||||
tmpFile, err := ioutil.TempFile(tmpdir, "ip_conntrack_count")
|
||||
assert.NoError(t, err)
|
||||
|
||||
dfltDirs = []string{tmpdir}
|
||||
fname := path.Base(tmpFile.Name())
|
||||
dfltFiles = []string{fname}
|
||||
|
||||
count := 1234321
|
||||
ioutil.WriteFile(tmpFile.Name(), []byte(strconv.Itoa(count)), 0660)
|
||||
c := &Conntrack{}
|
||||
acc := &testutil.Accumulator{}
|
||||
|
||||
c.Gather(acc)
|
||||
acc.AssertContainsFields(t, inputName, map[string]interface{}{
|
||||
fname: float64(count)})
|
||||
}
|
||||
|
||||
func TestConfigsUsed(t *testing.T) {
|
||||
defer restoreDflts(dfltFiles, dfltDirs)
|
||||
tmpdir, err := ioutil.TempDir("", "tmp1")
|
||||
assert.NoError(t, err)
|
||||
defer os.Remove(tmpdir)
|
||||
|
||||
cntFile, err := ioutil.TempFile(tmpdir, "nf_conntrack_count")
|
||||
maxFile, err := ioutil.TempFile(tmpdir, "nf_conntrack_max")
|
||||
assert.NoError(t, err)
|
||||
|
||||
dfltDirs = []string{tmpdir}
|
||||
cntFname := path.Base(cntFile.Name())
|
||||
maxFname := path.Base(maxFile.Name())
|
||||
dfltFiles = []string{cntFname, maxFname}
|
||||
|
||||
count := 1234321
|
||||
max := 9999999
|
||||
ioutil.WriteFile(cntFile.Name(), []byte(strconv.Itoa(count)), 0660)
|
||||
ioutil.WriteFile(maxFile.Name(), []byte(strconv.Itoa(max)), 0660)
|
||||
c := &Conntrack{}
|
||||
acc := &testutil.Accumulator{}
|
||||
|
||||
c.Gather(acc)
|
||||
|
||||
fix := func(s string) string {
|
||||
return strings.Replace(s, "nf_", "ip_", 1)
|
||||
}
|
||||
|
||||
acc.AssertContainsFields(t, inputName,
|
||||
map[string]interface{}{
|
||||
fix(cntFname): float64(count),
|
||||
fix(maxFname): float64(max),
|
||||
})
|
||||
}
|
||||
46
plugins/inputs/consul/README.md
Normal file
46
plugins/inputs/consul/README.md
Normal file
@@ -0,0 +1,46 @@
|
||||
# Telegraf Input Plugin: Consul
|
||||
|
||||
This plugin will collect statistics about all helath checks registered in the Consul. It uses [Consul API](https://www.consul.io/docs/agent/http/health.html#health_state)
|
||||
to query the data. It will not report the [telemetry](https://www.consul.io/docs/agent/telemetry.html) but Consul can report those stats already using StatsD protocol if needed.
|
||||
|
||||
## Configuration:
|
||||
|
||||
```
|
||||
# Gather health check statuses from services registered in Consul
|
||||
[[inputs.consul]]
|
||||
## Most of these values defaults to the one configured on a Consul's agent level.
|
||||
## Optional Consul server address (default: "")
|
||||
# address = ""
|
||||
## Optional URI scheme for the Consul server (default: "")
|
||||
# scheme = ""
|
||||
## Optional ACL token used in every request (default: "")
|
||||
# token = ""
|
||||
## Optional username used for request HTTP Basic Authentication (default: "")
|
||||
# username = ""
|
||||
## Optional password used for HTTP Basic Authentication (default: "")
|
||||
# password = ""
|
||||
## Optional data centre to query the health checks from (default: "")
|
||||
# datacentre = ""
|
||||
```
|
||||
|
||||
## Measurements:
|
||||
|
||||
### Consul:
|
||||
Tags:
|
||||
- node: on which node check/service is registered on
|
||||
- service_name: name of the service (this is the service name not the service ID)
|
||||
|
||||
Fields:
|
||||
- check_id
|
||||
- check_name
|
||||
- service_id
|
||||
- status
|
||||
|
||||
## Example output
|
||||
|
||||
```
|
||||
$ telegraf --config ./telegraf.conf -input-filter consul -test
|
||||
* Plugin: consul, Collection 1
|
||||
> consul_health_checks,host=wolfpit,node=consul-server-node check_id="serfHealth",check_name="Serf Health Status",service_id="",status="passing" 1464698464486439902
|
||||
> consul_health_checks,host=wolfpit,node=consul-server-node,service_name=www.example.com check_id="service:www-example-com.test01",check_name="Service 'www.example.com' check",service_id="www-example-com.test01",status="critical" 1464698464486519036
|
||||
```
|
||||
136
plugins/inputs/consul/consul.go
Normal file
136
plugins/inputs/consul/consul.go
Normal file
@@ -0,0 +1,136 @@
|
||||
package consul
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
|
||||
"github.com/hashicorp/consul/api"
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
type Consul struct {
|
||||
Address string
|
||||
Scheme string
|
||||
Token string
|
||||
Username string
|
||||
Password string
|
||||
Datacentre string
|
||||
|
||||
// Path to CA file
|
||||
SSLCA string `toml:"ssl_ca"`
|
||||
// Path to host cert file
|
||||
SSLCert string `toml:"ssl_cert"`
|
||||
// Path to cert key file
|
||||
SSLKey string `toml:"ssl_key"`
|
||||
// Use SSL but skip chain & host verification
|
||||
InsecureSkipVerify bool
|
||||
|
||||
// client used to connect to Consul agnet
|
||||
client *api.Client
|
||||
}
|
||||
|
||||
var sampleConfig = `
|
||||
## Most of these values defaults to the one configured on a Consul's agent level.
|
||||
## Optional Consul server address (default: "localhost")
|
||||
# address = "localhost"
|
||||
## Optional URI scheme for the Consul server (default: "http")
|
||||
# scheme = "http"
|
||||
## Optional ACL token used in every request (default: "")
|
||||
# token = ""
|
||||
## Optional username used for request HTTP Basic Authentication (default: "")
|
||||
# username = ""
|
||||
## Optional password used for HTTP Basic Authentication (default: "")
|
||||
# password = ""
|
||||
## Optional data centre to query the health checks from (default: "")
|
||||
# datacentre = ""
|
||||
`
|
||||
|
||||
func (c *Consul) Description() string {
|
||||
return "Gather health check statuses from services registered in Consul"
|
||||
}
|
||||
|
||||
func (c *Consul) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
|
||||
func (c *Consul) createAPIClient() (*api.Client, error) {
|
||||
config := api.DefaultConfig()
|
||||
|
||||
if c.Address != "" {
|
||||
config.Address = c.Address
|
||||
}
|
||||
|
||||
if c.Scheme != "" {
|
||||
config.Scheme = c.Scheme
|
||||
}
|
||||
|
||||
if c.Datacentre != "" {
|
||||
config.Datacenter = c.Datacentre
|
||||
}
|
||||
|
||||
if c.Username != "" {
|
||||
config.HttpAuth = &api.HttpBasicAuth{
|
||||
Username: c.Username,
|
||||
Password: c.Password,
|
||||
}
|
||||
}
|
||||
|
||||
tlsCfg, err := internal.GetTLSConfig(
|
||||
c.SSLCert, c.SSLKey, c.SSLCA, c.InsecureSkipVerify)
|
||||
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
config.HttpClient.Transport = &http.Transport{
|
||||
TLSClientConfig: tlsCfg,
|
||||
}
|
||||
|
||||
return api.NewClient(config)
|
||||
}
|
||||
|
||||
func (c *Consul) GatherHealthCheck(acc telegraf.Accumulator, checks []*api.HealthCheck) {
|
||||
for _, check := range checks {
|
||||
record := make(map[string]interface{})
|
||||
tags := make(map[string]string)
|
||||
|
||||
record["check_id"] = check.CheckID
|
||||
record["check_name"] = check.Name
|
||||
record["service_id"] = check.ServiceID
|
||||
record["status"] = check.Status
|
||||
|
||||
tags["node"] = check.Node
|
||||
tags["service_name"] = check.ServiceName
|
||||
|
||||
acc.AddFields("consul_health_checks", record, tags)
|
||||
}
|
||||
}
|
||||
|
||||
func (c *Consul) Gather(acc telegraf.Accumulator) error {
|
||||
if c.client == nil {
|
||||
newClient, err := c.createAPIClient()
|
||||
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
c.client = newClient
|
||||
}
|
||||
|
||||
checks, _, err := c.client.Health().State("any", nil)
|
||||
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
c.GatherHealthCheck(acc, checks)
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("consul", func() telegraf.Input {
|
||||
return &Consul{}
|
||||
})
|
||||
}
|
||||
42
plugins/inputs/consul/consul_test.go
Normal file
42
plugins/inputs/consul/consul_test.go
Normal file
@@ -0,0 +1,42 @@
|
||||
package consul
|
||||
|
||||
import (
|
||||
"testing"
|
||||
|
||||
"github.com/hashicorp/consul/api"
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
)
|
||||
|
||||
var sampleChecks = []*api.HealthCheck{
|
||||
&api.HealthCheck{
|
||||
Node: "localhost",
|
||||
CheckID: "foo.health123",
|
||||
Name: "foo.health",
|
||||
Status: "passing",
|
||||
Notes: "lorem ipsum",
|
||||
Output: "OK",
|
||||
ServiceID: "foo.123",
|
||||
ServiceName: "foo",
|
||||
},
|
||||
}
|
||||
|
||||
func TestGatherHealtCheck(t *testing.T) {
|
||||
expectedFields := map[string]interface{}{
|
||||
"check_id": "foo.health123",
|
||||
"check_name": "foo.health",
|
||||
"status": "passing",
|
||||
"service_id": "foo.123",
|
||||
}
|
||||
|
||||
expectedTags := map[string]string{
|
||||
"node": "localhost",
|
||||
"service_name": "foo",
|
||||
}
|
||||
|
||||
var acc testutil.Accumulator
|
||||
|
||||
consul := &Consul{}
|
||||
consul.GatherHealthCheck(&acc, sampleChecks)
|
||||
|
||||
acc.AssertContainsTaggedFields(t, "consul_health_checks", expectedFields, expectedTags)
|
||||
}
|
||||
@@ -221,7 +221,7 @@ func (d *Docker) gatherContainer(
|
||||
defer cancel()
|
||||
r, err := d.client.ContainerStats(ctx, container.ID, false)
|
||||
if err != nil {
|
||||
log.Printf("Error getting docker stats: %s\n", err.Error())
|
||||
return fmt.Errorf("Error getting docker stats: %s", err.Error())
|
||||
}
|
||||
defer r.Close()
|
||||
dec := json.NewDecoder(r)
|
||||
|
||||
@@ -1,320 +1,314 @@
|
||||
# Elasticsearch plugin
|
||||
|
||||
#### Plugin arguments:
|
||||
- **servers** []string: list of one or more Elasticsearch servers
|
||||
- **local** boolean: If false, it will read the indices stats from all nodes
|
||||
- **cluster_health** boolean: If true, it will also obtain cluster level stats
|
||||
|
||||
#### Description
|
||||
# Elasticsearch input plugin
|
||||
|
||||
The [elasticsearch](https://www.elastic.co/) plugin queries endpoints to obtain
|
||||
[node](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-nodes-stats.html)
|
||||
and optionally [cluster](https://www.elastic.co/guide/en/elasticsearch/reference/current/cluster-health.html) stats.
|
||||
|
||||
Example:
|
||||
### Configuration:
|
||||
|
||||
```
|
||||
[elasticsearch]
|
||||
[[inputs.elasticsearch]]
|
||||
servers = ["http://localhost:9200"]
|
||||
local = true
|
||||
cluster_health = true
|
||||
|
||||
servers = ["http://localhost:9200"]
|
||||
|
||||
local = true
|
||||
|
||||
cluster_health = true
|
||||
## Optional SSL Config
|
||||
# ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# ssl_key = "/etc/telegraf/key.pem"
|
||||
## Use SSL but skip chain & host verification
|
||||
# insecure_skip_verify = false
|
||||
```
|
||||
|
||||
# Measurements
|
||||
#### cluster measurements (utilizes fields instead of single values):
|
||||
|
||||
contains `status`, `timed_out`, `number_of_nodes`, `number_of_data_nodes`,
|
||||
`active_primary_shards`, `active_shards`, `relocating_shards`,
|
||||
`initializing_shards`, `unassigned_shards` fields
|
||||
- elasticsearch_cluster_health
|
||||
|
||||
contains `status`, `number_of_shards`, `number_of_replicas`,
|
||||
`active_primary_shards`, `active_shards`, `relocating_shards`,
|
||||
`initializing_shards`, `unassigned_shards` fields
|
||||
- elasticsearch_indices
|
||||
|
||||
#### node measurements:
|
||||
### Measurements & Fields:
|
||||
|
||||
field data circuit breaker measurement names:
|
||||
- elasticsearch_breakers_fielddata_estimated_size_in_bytes value=0
|
||||
- elasticsearch_breakers_fielddata_overhead value=1.03
|
||||
- elasticsearch_breakers_fielddata_tripped value=0
|
||||
- elasticsearch_breakers_fielddata_limit_size_in_bytes value=623326003
|
||||
- elasticsearch_breakers_request_estimated_size_in_bytes value=0
|
||||
- elasticsearch_breakers_request_overhead value=1.0
|
||||
- elasticsearch_breakers_request_tripped value=0
|
||||
- elasticsearch_breakers_request_limit_size_in_bytes value=415550668
|
||||
- elasticsearch_breakers_parent_overhead value=1.0
|
||||
- elasticsearch_breakers_parent_tripped value=0
|
||||
- elasticsearch_breakers_parent_limit_size_in_bytes value=727213670
|
||||
- elasticsearch_breakers_parent_estimated_size_in_bytes value=0
|
||||
- elasticsearch_breakers
|
||||
- fielddata_estimated_size_in_bytes value=0
|
||||
- fielddata_overhead value=1.03
|
||||
- fielddata_tripped value=0
|
||||
- fielddata_limit_size_in_bytes value=623326003
|
||||
- request_estimated_size_in_bytes value=0
|
||||
- request_overhead value=1.0
|
||||
- request_tripped value=0
|
||||
- request_limit_size_in_bytes value=415550668
|
||||
- parent_overhead value=1.0
|
||||
- parent_tripped value=0
|
||||
- parent_limit_size_in_bytes value=727213670
|
||||
- parent_estimated_size_in_bytes value=0
|
||||
|
||||
File system information, data path, free disk space, read/write measurement names:
|
||||
- elasticsearch_fs_timestamp value=1436460392946
|
||||
- elasticsearch_fs_total_free_in_bytes value=16909316096
|
||||
- elasticsearch_fs_total_available_in_bytes value=15894814720
|
||||
- elasticsearch_fs_total_total_in_bytes value=19507089408
|
||||
- elasticsearch_fs
|
||||
- timestamp value=1436460392946
|
||||
- total_free_in_bytes value=16909316096
|
||||
- total_available_in_bytes value=15894814720
|
||||
- total_total_in_bytes value=19507089408
|
||||
|
||||
indices size, document count, indexing and deletion times, search times,
|
||||
field cache size, merges and flushes measurement names:
|
||||
- elasticsearch_indices_id_cache_memory_size_in_bytes value=0
|
||||
- elasticsearch_indices_completion_size_in_bytes value=0
|
||||
- elasticsearch_indices_suggest_total value=0
|
||||
- elasticsearch_indices_suggest_time_in_millis value=0
|
||||
- elasticsearch_indices_suggest_current value=0
|
||||
- elasticsearch_indices_query_cache_memory_size_in_bytes value=0
|
||||
- elasticsearch_indices_query_cache_evictions value=0
|
||||
- elasticsearch_indices_query_cache_hit_count value=0
|
||||
- elasticsearch_indices_query_cache_miss_count value=0
|
||||
- elasticsearch_indices_store_size_in_bytes value=37715234
|
||||
- elasticsearch_indices_store_throttle_time_in_millis value=215
|
||||
- elasticsearch_indices_merges_current_docs value=0
|
||||
- elasticsearch_indices_merges_current_size_in_bytes value=0
|
||||
- elasticsearch_indices_merges_total value=133
|
||||
- elasticsearch_indices_merges_total_time_in_millis value=21060
|
||||
- elasticsearch_indices_merges_total_docs value=203672
|
||||
- elasticsearch_indices_merges_total_size_in_bytes value=142900226
|
||||
- elasticsearch_indices_merges_current value=0
|
||||
- elasticsearch_indices_filter_cache_memory_size_in_bytes value=7384
|
||||
- elasticsearch_indices_filter_cache_evictions value=0
|
||||
- elasticsearch_indices_indexing_index_total value=84790
|
||||
- elasticsearch_indices_indexing_index_time_in_millis value=29680
|
||||
- elasticsearch_indices_indexing_index_current value=0
|
||||
- elasticsearch_indices_indexing_noop_update_total value=0
|
||||
- elasticsearch_indices_indexing_throttle_time_in_millis value=0
|
||||
- elasticsearch_indices_indexing_delete_tota value=13879
|
||||
- elasticsearch_indices_indexing_delete_time_in_millis value=1139
|
||||
- elasticsearch_indices_indexing_delete_current value=0
|
||||
- elasticsearch_indices_get_exists_time_in_millis value=0
|
||||
- elasticsearch_indices_get_missing_total value=1
|
||||
- elasticsearch_indices_get_missing_time_in_millis value=2
|
||||
- elasticsearch_indices_get_current value=0
|
||||
- elasticsearch_indices_get_total value=1
|
||||
- elasticsearch_indices_get_time_in_millis value=2
|
||||
- elasticsearch_indices_get_exists_total value=0
|
||||
- elasticsearch_indices_refresh_total value=1076
|
||||
- elasticsearch_indices_refresh_total_time_in_millis value=20078
|
||||
- elasticsearch_indices_percolate_current value=0
|
||||
- elasticsearch_indices_percolate_memory_size_in_bytes value=-1
|
||||
- elasticsearch_indices_percolate_queries value=0
|
||||
- elasticsearch_indices_percolate_total value=0
|
||||
- elasticsearch_indices_percolate_time_in_millis value=0
|
||||
- elasticsearch_indices_translog_operations value=17702
|
||||
- elasticsearch_indices_translog_size_in_bytes value=17
|
||||
- elasticsearch_indices_recovery_current_as_source value=0
|
||||
- elasticsearch_indices_recovery_current_as_target value=0
|
||||
- elasticsearch_indices_recovery_throttle_time_in_millis value=0
|
||||
- elasticsearch_indices_docs_count value=29652
|
||||
- elasticsearch_indices_docs_deleted value=5229
|
||||
- elasticsearch_indices_flush_total_time_in_millis value=2401
|
||||
- elasticsearch_indices_flush_total value=115
|
||||
- elasticsearch_indices_fielddata_memory_size_in_bytes value=12996
|
||||
- elasticsearch_indices_fielddata_evictions value=0
|
||||
- elasticsearch_indices_search_fetch_current value=0
|
||||
- elasticsearch_indices_search_open_contexts value=0
|
||||
- elasticsearch_indices_search_query_total value=1452
|
||||
- elasticsearch_indices_search_query_time_in_millis value=5695
|
||||
- elasticsearch_indices_search_query_current value=0
|
||||
- elasticsearch_indices_search_fetch_total value=414
|
||||
- elasticsearch_indices_search_fetch_time_in_millis value=146
|
||||
- elasticsearch_indices_warmer_current value=0
|
||||
- elasticsearch_indices_warmer_total value=2319
|
||||
- elasticsearch_indices_warmer_total_time_in_millis value=448
|
||||
- elasticsearch_indices_segments_count value=134
|
||||
- elasticsearch_indices_segments_memory_in_bytes value=1285212
|
||||
- elasticsearch_indices_segments_index_writer_memory_in_bytes value=0
|
||||
- elasticsearch_indices_segments_index_writer_max_memory_in_bytes value=172368955
|
||||
- elasticsearch_indices_segments_version_map_memory_in_bytes value=611844
|
||||
- elasticsearch_indices_segments_fixed_bit_set_memory_in_bytes value=0
|
||||
- elasticsearch_indices
|
||||
- id_cache_memory_size_in_bytes value=0
|
||||
- completion_size_in_bytes value=0
|
||||
- suggest_total value=0
|
||||
- suggest_time_in_millis value=0
|
||||
- suggest_current value=0
|
||||
- query_cache_memory_size_in_bytes value=0
|
||||
- query_cache_evictions value=0
|
||||
- query_cache_hit_count value=0
|
||||
- query_cache_miss_count value=0
|
||||
- store_size_in_bytes value=37715234
|
||||
- store_throttle_time_in_millis value=215
|
||||
- merges_current_docs value=0
|
||||
- merges_current_size_in_bytes value=0
|
||||
- merges_total value=133
|
||||
- merges_total_time_in_millis value=21060
|
||||
- merges_total_docs value=203672
|
||||
- merges_total_size_in_bytes value=142900226
|
||||
- merges_current value=0
|
||||
- filter_cache_memory_size_in_bytes value=7384
|
||||
- filter_cache_evictions value=0
|
||||
- indexing_index_total value=84790
|
||||
- indexing_index_time_in_millis value=29680
|
||||
- indexing_index_current value=0
|
||||
- indexing_noop_update_total value=0
|
||||
- indexing_throttle_time_in_millis value=0
|
||||
- indexing_delete_tota value=13879
|
||||
- indexing_delete_time_in_millis value=1139
|
||||
- indexing_delete_current value=0
|
||||
- get_exists_time_in_millis value=0
|
||||
- get_missing_total value=1
|
||||
- get_missing_time_in_millis value=2
|
||||
- get_current value=0
|
||||
- get_total value=1
|
||||
- get_time_in_millis value=2
|
||||
- get_exists_total value=0
|
||||
- refresh_total value=1076
|
||||
- refresh_total_time_in_millis value=20078
|
||||
- percolate_current value=0
|
||||
- percolate_memory_size_in_bytes value=-1
|
||||
- percolate_queries value=0
|
||||
- percolate_total value=0
|
||||
- percolate_time_in_millis value=0
|
||||
- translog_operations value=17702
|
||||
- translog_size_in_bytes value=17
|
||||
- recovery_current_as_source value=0
|
||||
- recovery_current_as_target value=0
|
||||
- recovery_throttle_time_in_millis value=0
|
||||
- docs_count value=29652
|
||||
- docs_deleted value=5229
|
||||
- flush_total_time_in_millis value=2401
|
||||
- flush_total value=115
|
||||
- fielddata_memory_size_in_bytes value=12996
|
||||
- fielddata_evictions value=0
|
||||
- search_fetch_current value=0
|
||||
- search_open_contexts value=0
|
||||
- search_query_total value=1452
|
||||
- search_query_time_in_millis value=5695
|
||||
- search_query_current value=0
|
||||
- search_fetch_total value=414
|
||||
- search_fetch_time_in_millis value=146
|
||||
- warmer_current value=0
|
||||
- warmer_total value=2319
|
||||
- warmer_total_time_in_millis value=448
|
||||
- segments_count value=134
|
||||
- segments_memory_in_bytes value=1285212
|
||||
- segments_index_writer_memory_in_bytes value=0
|
||||
- segments_index_writer_max_memory_in_bytes value=172368955
|
||||
- segments_version_map_memory_in_bytes value=611844
|
||||
- segments_fixed_bit_set_memory_in_bytes value=0
|
||||
|
||||
HTTP connection measurement names:
|
||||
- elasticsearch_http_current_open value=3
|
||||
- elasticsearch_http_total_opened value=3
|
||||
- elasticsearch_http
|
||||
- current_open value=3
|
||||
- total_opened value=3
|
||||
|
||||
JVM stats, memory pool information, garbage collection, buffer pools measurement names:
|
||||
- elasticsearch_jvm_timestamp value=1436460392945
|
||||
- elasticsearch_jvm_uptime_in_millis value=202245
|
||||
- elasticsearch_jvm_mem_non_heap_used_in_bytes value=39634576
|
||||
- elasticsearch_jvm_mem_non_heap_committed_in_bytes value=40841216
|
||||
- elasticsearch_jvm_mem_pools_young_max_in_bytes value=279183360
|
||||
- elasticsearch_jvm_mem_pools_young_peak_used_in_bytes value=71630848
|
||||
- elasticsearch_jvm_mem_pools_young_peak_max_in_bytes value=279183360
|
||||
- elasticsearch_jvm_mem_pools_young_used_in_bytes value=32685760
|
||||
- elasticsearch_jvm_mem_pools_survivor_peak_used_in_bytes value=8912888
|
||||
- elasticsearch_jvm_mem_pools_survivor_peak_max_in_bytes value=34865152
|
||||
- elasticsearch_jvm_mem_pools_survivor_used_in_bytes value=8912880
|
||||
- elasticsearch_jvm_mem_pools_survivor_max_in_bytes value=34865152
|
||||
- elasticsearch_jvm_mem_pools_old_peak_max_in_bytes value=724828160
|
||||
- elasticsearch_jvm_mem_pools_old_used_in_bytes value=11110928
|
||||
- elasticsearch_jvm_mem_pools_old_max_in_bytes value=724828160
|
||||
- elasticsearch_jvm_mem_pools_old_peak_used_in_bytes value=14354608
|
||||
- elasticsearch_jvm_mem_heap_used_in_bytes value=52709568
|
||||
- elasticsearch_jvm_mem_heap_used_percent value=5
|
||||
- elasticsearch_jvm_mem_heap_committed_in_bytes value=259522560
|
||||
- elasticsearch_jvm_mem_heap_max_in_bytes value=1038876672
|
||||
- elasticsearch_jvm_threads_peak_count value=45
|
||||
- elasticsearch_jvm_threads_count value=44
|
||||
- elasticsearch_jvm_gc_collectors_young_collection_count value=2
|
||||
- elasticsearch_jvm_gc_collectors_young_collection_time_in_millis value=98
|
||||
- elasticsearch_jvm_gc_collectors_old_collection_count value=1
|
||||
- elasticsearch_jvm_gc_collectors_old_collection_time_in_millis value=24
|
||||
- elasticsearch_jvm_buffer_pools_direct_count value=40
|
||||
- elasticsearch_jvm_buffer_pools_direct_used_in_bytes value=6304239
|
||||
- elasticsearch_jvm_buffer_pools_direct_total_capacity_in_bytes value=6304239
|
||||
- elasticsearch_jvm_buffer_pools_mapped_count value=0
|
||||
- elasticsearch_jvm_buffer_pools_mapped_used_in_bytes value=0
|
||||
- elasticsearch_jvm_buffer_pools_mapped_total_capacity_in_bytes value=0
|
||||
- elasticsearch_jvm
|
||||
- timestamp value=1436460392945
|
||||
- uptime_in_millis value=202245
|
||||
- mem_non_heap_used_in_bytes value=39634576
|
||||
- mem_non_heap_committed_in_bytes value=40841216
|
||||
- mem_pools_young_max_in_bytes value=279183360
|
||||
- mem_pools_young_peak_used_in_bytes value=71630848
|
||||
- mem_pools_young_peak_max_in_bytes value=279183360
|
||||
- mem_pools_young_used_in_bytes value=32685760
|
||||
- mem_pools_survivor_peak_used_in_bytes value=8912888
|
||||
- mem_pools_survivor_peak_max_in_bytes value=34865152
|
||||
- mem_pools_survivor_used_in_bytes value=8912880
|
||||
- mem_pools_survivor_max_in_bytes value=34865152
|
||||
- mem_pools_old_peak_max_in_bytes value=724828160
|
||||
- mem_pools_old_used_in_bytes value=11110928
|
||||
- mem_pools_old_max_in_bytes value=724828160
|
||||
- mem_pools_old_peak_used_in_bytes value=14354608
|
||||
- mem_heap_used_in_bytes value=52709568
|
||||
- mem_heap_used_percent value=5
|
||||
- mem_heap_committed_in_bytes value=259522560
|
||||
- mem_heap_max_in_bytes value=1038876672
|
||||
- threads_peak_count value=45
|
||||
- threads_count value=44
|
||||
- gc_collectors_young_collection_count value=2
|
||||
- gc_collectors_young_collection_time_in_millis value=98
|
||||
- gc_collectors_old_collection_count value=1
|
||||
- gc_collectors_old_collection_time_in_millis value=24
|
||||
- buffer_pools_direct_count value=40
|
||||
- buffer_pools_direct_used_in_bytes value=6304239
|
||||
- buffer_pools_direct_total_capacity_in_bytes value=6304239
|
||||
- buffer_pools_mapped_count value=0
|
||||
- buffer_pools_mapped_used_in_bytes value=0
|
||||
- buffer_pools_mapped_total_capacity_in_bytes value=0
|
||||
|
||||
TCP information measurement names:
|
||||
- elasticsearch_network_tcp_in_errs value=0
|
||||
- elasticsearch_network_tcp_passive_opens value=16
|
||||
- elasticsearch_network_tcp_curr_estab value=29
|
||||
- elasticsearch_network_tcp_in_segs value=113
|
||||
- elasticsearch_network_tcp_out_segs value=97
|
||||
- elasticsearch_network_tcp_retrans_segs value=0
|
||||
- elasticsearch_network_tcp_attempt_fails value=0
|
||||
- elasticsearch_network_tcp_active_opens value=13
|
||||
- elasticsearch_network_tcp_estab_resets value=0
|
||||
- elasticsearch_network_tcp_out_rsts value=0
|
||||
- elasticsearch_network
|
||||
- tcp_in_errs value=0
|
||||
- tcp_passive_opens value=16
|
||||
- tcp_curr_estab value=29
|
||||
- tcp_in_segs value=113
|
||||
- tcp_out_segs value=97
|
||||
- tcp_retrans_segs value=0
|
||||
- tcp_attempt_fails value=0
|
||||
- tcp_active_opens value=13
|
||||
- tcp_estab_resets value=0
|
||||
- tcp_out_rsts value=0
|
||||
|
||||
Operating system stats, load average, cpu, mem, swap measurement names:
|
||||
- elasticsearch_os_swap_used_in_bytes value=0
|
||||
- elasticsearch_os_swap_free_in_bytes value=487997440
|
||||
- elasticsearch_os_timestamp value=1436460392944
|
||||
- elasticsearch_os_uptime_in_millis value=25092
|
||||
- elasticsearch_os_cpu_sys value=0
|
||||
- elasticsearch_os_cpu_user value=0
|
||||
- elasticsearch_os_cpu_idle value=99
|
||||
- elasticsearch_os_cpu_usage value=0
|
||||
- elasticsearch_os_cpu_stolen value=0
|
||||
- elasticsearch_os_mem_free_percent value=74
|
||||
- elasticsearch_os_mem_used_percent value=25
|
||||
- elasticsearch_os_mem_actual_free_in_bytes value=1565470720
|
||||
- elasticsearch_os_mem_actual_used_in_bytes value=534159360
|
||||
- elasticsearch_os_mem_free_in_bytes value=477761536
|
||||
- elasticsearch_os_mem_used_in_bytes value=1621868544
|
||||
- elasticsearch_os
|
||||
- swap_used_in_bytes value=0
|
||||
- swap_free_in_bytes value=487997440
|
||||
- timestamp value=1436460392944
|
||||
- uptime_in_millis value=25092
|
||||
- cpu_sys value=0
|
||||
- cpu_user value=0
|
||||
- cpu_idle value=99
|
||||
- cpu_usage value=0
|
||||
- cpu_stolen value=0
|
||||
- mem_free_percent value=74
|
||||
- mem_used_percent value=25
|
||||
- mem_actual_free_in_bytes value=1565470720
|
||||
- mem_actual_used_in_bytes value=534159360
|
||||
- mem_free_in_bytes value=477761536
|
||||
- mem_used_in_bytes value=1621868544
|
||||
|
||||
Process statistics, memory consumption, cpu usage, open file descriptors measurement names:
|
||||
- elasticsearch_process_mem_resident_in_bytes value=246382592
|
||||
- elasticsearch_process_mem_share_in_bytes value=18747392
|
||||
- elasticsearch_process_mem_total_virtual_in_bytes value=4747890688
|
||||
- elasticsearch_process_timestamp value=1436460392945
|
||||
- elasticsearch_process_open_file_descriptors value=160
|
||||
- elasticsearch_process_cpu_total_in_millis value=15480
|
||||
- elasticsearch_process_cpu_percent value=2
|
||||
- elasticsearch_process_cpu_sys_in_millis value=1870
|
||||
- elasticsearch_process_cpu_user_in_millis value=13610
|
||||
- elasticsearch_process
|
||||
- mem_resident_in_bytes value=246382592
|
||||
- mem_share_in_bytes value=18747392
|
||||
- mem_total_virtual_in_bytes value=4747890688
|
||||
- timestamp value=1436460392945
|
||||
- open_file_descriptors value=160
|
||||
- cpu_total_in_millis value=15480
|
||||
- cpu_percent value=2
|
||||
- cpu_sys_in_millis value=1870
|
||||
- cpu_user_in_millis value=13610
|
||||
|
||||
Statistics about each thread pool, including current size, queue and rejected tasks measurement names:
|
||||
- elasticsearch_thread_pool_merge_threads value=6
|
||||
- elasticsearch_thread_pool_merge_queue value=4
|
||||
- elasticsearch_thread_pool_merge_active value=5
|
||||
- elasticsearch_thread_pool_merge_rejected value=2
|
||||
- elasticsearch_thread_pool_merge_largest value=5
|
||||
- elasticsearch_thread_pool_merge_completed value=1
|
||||
- elasticsearch_thread_pool_bulk_threads value=4
|
||||
- elasticsearch_thread_pool_bulk_queue value=5
|
||||
- elasticsearch_thread_pool_bulk_active value=7
|
||||
- elasticsearch_thread_pool_bulk_rejected value=3
|
||||
- elasticsearch_thread_pool_bulk_largest value=1
|
||||
- elasticsearch_thread_pool_bulk_completed value=4
|
||||
- elasticsearch_thread_pool_warmer_threads value=2
|
||||
- elasticsearch_thread_pool_warmer_queue value=7
|
||||
- elasticsearch_thread_pool_warmer_active value=3
|
||||
- elasticsearch_thread_pool_warmer_rejected value=2
|
||||
- elasticsearch_thread_pool_warmer_largest value=3
|
||||
- elasticsearch_thread_pool_warmer_completed value=1
|
||||
- elasticsearch_thread_pool_get_largest value=2
|
||||
- elasticsearch_thread_pool_get_completed value=1
|
||||
- elasticsearch_thread_pool_get_threads value=1
|
||||
- elasticsearch_thread_pool_get_queue value=8
|
||||
- elasticsearch_thread_pool_get_active value=4
|
||||
- elasticsearch_thread_pool_get_rejected value=3
|
||||
- elasticsearch_thread_pool_index_threads value=6
|
||||
- elasticsearch_thread_pool_index_queue value=8
|
||||
- elasticsearch_thread_pool_index_active value=4
|
||||
- elasticsearch_thread_pool_index_rejected value=2
|
||||
- elasticsearch_thread_pool_index_largest value=3
|
||||
- elasticsearch_thread_pool_index_completed value=6
|
||||
- elasticsearch_thread_pool_suggest_threads value=2
|
||||
- elasticsearch_thread_pool_suggest_queue value=7
|
||||
- elasticsearch_thread_pool_suggest_active value=2
|
||||
- elasticsearch_thread_pool_suggest_rejected value=1
|
||||
- elasticsearch_thread_pool_suggest_largest value=8
|
||||
- elasticsearch_thread_pool_suggest_completed value=3
|
||||
- elasticsearch_thread_pool_fetch_shard_store_queue value=7
|
||||
- elasticsearch_thread_pool_fetch_shard_store_active value=4
|
||||
- elasticsearch_thread_pool_fetch_shard_store_rejected value=2
|
||||
- elasticsearch_thread_pool_fetch_shard_store_largest value=4
|
||||
- elasticsearch_thread_pool_fetch_shard_store_completed value=1
|
||||
- elasticsearch_thread_pool_fetch_shard_store_threads value=1
|
||||
- elasticsearch_thread_pool_management_threads value=2
|
||||
- elasticsearch_thread_pool_management_queue value=3
|
||||
- elasticsearch_thread_pool_management_active value=1
|
||||
- elasticsearch_thread_pool_management_rejected value=6
|
||||
- elasticsearch_thread_pool_management_largest value=2
|
||||
- elasticsearch_thread_pool_management_completed value=22
|
||||
- elasticsearch_thread_pool_percolate_queue value=23
|
||||
- elasticsearch_thread_pool_percolate_active value=13
|
||||
- elasticsearch_thread_pool_percolate_rejected value=235
|
||||
- elasticsearch_thread_pool_percolate_largest value=23
|
||||
- elasticsearch_thread_pool_percolate_completed value=33
|
||||
- elasticsearch_thread_pool_percolate_threads value=123
|
||||
- elasticsearch_thread_pool_listener_active value=4
|
||||
- elasticsearch_thread_pool_listener_rejected value=8
|
||||
- elasticsearch_thread_pool_listener_largest value=1
|
||||
- elasticsearch_thread_pool_listener_completed value=1
|
||||
- elasticsearch_thread_pool_listener_threads value=1
|
||||
- elasticsearch_thread_pool_listener_queue value=2
|
||||
- elasticsearch_thread_pool_search_rejected value=7
|
||||
- elasticsearch_thread_pool_search_largest value=2
|
||||
- elasticsearch_thread_pool_search_completed value=4
|
||||
- elasticsearch_thread_pool_search_threads value=5
|
||||
- elasticsearch_thread_pool_search_queue value=7
|
||||
- elasticsearch_thread_pool_search_active value=2
|
||||
- elasticsearch_thread_pool_fetch_shard_started_threads value=3
|
||||
- elasticsearch_thread_pool_fetch_shard_started_queue value=1
|
||||
- elasticsearch_thread_pool_fetch_shard_started_active value=5
|
||||
- elasticsearch_thread_pool_fetch_shard_started_rejected value=6
|
||||
- elasticsearch_thread_pool_fetch_shard_started_largest value=4
|
||||
- elasticsearch_thread_pool_fetch_shard_started_completed value=54
|
||||
- elasticsearch_thread_pool_refresh_rejected value=4
|
||||
- elasticsearch_thread_pool_refresh_largest value=8
|
||||
- elasticsearch_thread_pool_refresh_completed value=3
|
||||
- elasticsearch_thread_pool_refresh_threads value=23
|
||||
- elasticsearch_thread_pool_refresh_queue value=7
|
||||
- elasticsearch_thread_pool_refresh_active value=3
|
||||
- elasticsearch_thread_pool_optimize_threads value=3
|
||||
- elasticsearch_thread_pool_optimize_queue value=4
|
||||
- elasticsearch_thread_pool_optimize_active value=1
|
||||
- elasticsearch_thread_pool_optimize_rejected value=2
|
||||
- elasticsearch_thread_pool_optimize_largest value=7
|
||||
- elasticsearch_thread_pool_optimize_completed value=3
|
||||
- elasticsearch_thread_pool_snapshot_largest value=1
|
||||
- elasticsearch_thread_pool_snapshot_completed value=0
|
||||
- elasticsearch_thread_pool_snapshot_threads value=8
|
||||
- elasticsearch_thread_pool_snapshot_queue value=5
|
||||
- elasticsearch_thread_pool_snapshot_active value=6
|
||||
- elasticsearch_thread_pool_snapshot_rejected value=2
|
||||
- elasticsearch_thread_pool_generic_threads value=1
|
||||
- elasticsearch_thread_pool_generic_queue value=4
|
||||
- elasticsearch_thread_pool_generic_active value=6
|
||||
- elasticsearch_thread_pool_generic_rejected value=3
|
||||
- elasticsearch_thread_pool_generic_largest value=2
|
||||
- elasticsearch_thread_pool_generic_completed value=27
|
||||
- elasticsearch_thread_pool_flush_threads value=3
|
||||
- elasticsearch_thread_pool_flush_queue value=8
|
||||
- elasticsearch_thread_pool_flush_active value=0
|
||||
- elasticsearch_thread_pool_flush_rejected value=1
|
||||
- elasticsearch_thread_pool_flush_largest value=5
|
||||
- elasticsearch_thread_pool_flush_completed value=3
|
||||
- elasticsearch_thread_pool
|
||||
- merge_threads value=6
|
||||
- merge_queue value=4
|
||||
- merge_active value=5
|
||||
- merge_rejected value=2
|
||||
- merge_largest value=5
|
||||
- merge_completed value=1
|
||||
- bulk_threads value=4
|
||||
- bulk_queue value=5
|
||||
- bulk_active value=7
|
||||
- bulk_rejected value=3
|
||||
- bulk_largest value=1
|
||||
- bulk_completed value=4
|
||||
- warmer_threads value=2
|
||||
- warmer_queue value=7
|
||||
- warmer_active value=3
|
||||
- warmer_rejected value=2
|
||||
- warmer_largest value=3
|
||||
- warmer_completed value=1
|
||||
- get_largest value=2
|
||||
- get_completed value=1
|
||||
- get_threads value=1
|
||||
- get_queue value=8
|
||||
- get_active value=4
|
||||
- get_rejected value=3
|
||||
- index_threads value=6
|
||||
- index_queue value=8
|
||||
- index_active value=4
|
||||
- index_rejected value=2
|
||||
- index_largest value=3
|
||||
- index_completed value=6
|
||||
- suggest_threads value=2
|
||||
- suggest_queue value=7
|
||||
- suggest_active value=2
|
||||
- suggest_rejected value=1
|
||||
- suggest_largest value=8
|
||||
- suggest_completed value=3
|
||||
- fetch_shard_store_queue value=7
|
||||
- fetch_shard_store_active value=4
|
||||
- fetch_shard_store_rejected value=2
|
||||
- fetch_shard_store_largest value=4
|
||||
- fetch_shard_store_completed value=1
|
||||
- fetch_shard_store_threads value=1
|
||||
- management_threads value=2
|
||||
- management_queue value=3
|
||||
- management_active value=1
|
||||
- management_rejected value=6
|
||||
- management_largest value=2
|
||||
- management_completed value=22
|
||||
- percolate_queue value=23
|
||||
- percolate_active value=13
|
||||
- percolate_rejected value=235
|
||||
- percolate_largest value=23
|
||||
- percolate_completed value=33
|
||||
- percolate_threads value=123
|
||||
- listener_active value=4
|
||||
- listener_rejected value=8
|
||||
- listener_largest value=1
|
||||
- listener_completed value=1
|
||||
- listener_threads value=1
|
||||
- listener_queue value=2
|
||||
- search_rejected value=7
|
||||
- search_largest value=2
|
||||
- search_completed value=4
|
||||
- search_threads value=5
|
||||
- search_queue value=7
|
||||
- search_active value=2
|
||||
- fetch_shard_started_threads value=3
|
||||
- fetch_shard_started_queue value=1
|
||||
- fetch_shard_started_active value=5
|
||||
- fetch_shard_started_rejected value=6
|
||||
- fetch_shard_started_largest value=4
|
||||
- fetch_shard_started_completed value=54
|
||||
- refresh_rejected value=4
|
||||
- refresh_largest value=8
|
||||
- refresh_completed value=3
|
||||
- refresh_threads value=23
|
||||
- refresh_queue value=7
|
||||
- refresh_active value=3
|
||||
- optimize_threads value=3
|
||||
- optimize_queue value=4
|
||||
- optimize_active value=1
|
||||
- optimize_rejected value=2
|
||||
- optimize_largest value=7
|
||||
- optimize_completed value=3
|
||||
- snapshot_largest value=1
|
||||
- snapshot_completed value=0
|
||||
- snapshot_threads value=8
|
||||
- snapshot_queue value=5
|
||||
- snapshot_active value=6
|
||||
- snapshot_rejected value=2
|
||||
- generic_threads value=1
|
||||
- generic_queue value=4
|
||||
- generic_active value=6
|
||||
- generic_rejected value=3
|
||||
- generic_largest value=2
|
||||
- generic_completed value=27
|
||||
- flush_threads value=3
|
||||
- flush_queue value=8
|
||||
- flush_active value=0
|
||||
- flush_rejected value=1
|
||||
- flush_largest value=5
|
||||
- flush_completed value=3
|
||||
|
||||
Transport statistics about sent and received bytes in cluster communication measurement names:
|
||||
- elasticsearch_transport_server_open value=13
|
||||
- elasticsearch_transport_rx_count value=6
|
||||
- elasticsearch_transport_rx_size_in_bytes value=1380
|
||||
- elasticsearch_transport_tx_count value=6
|
||||
- elasticsearch_transport_tx_size_in_bytes value=1380
|
||||
- elasticsearch_transport
|
||||
- server_open value=13
|
||||
- rx_count value=6
|
||||
- rx_size_in_bytes value=1380
|
||||
- tx_count value=6
|
||||
- tx_size_in_bytes value=1380
|
||||
|
||||
@@ -2,14 +2,14 @@ package elasticsearch
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"net/http"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/internal/errchan"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
jsonparser "github.com/influxdata/telegraf/plugins/parsers/json"
|
||||
)
|
||||
@@ -68,25 +68,31 @@ const sampleConfig = `
|
||||
|
||||
## set cluster_health to true when you want to also obtain cluster level stats
|
||||
cluster_health = false
|
||||
|
||||
## Optional SSL Config
|
||||
# ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# ssl_key = "/etc/telegraf/key.pem"
|
||||
## Use SSL but skip chain & host verification
|
||||
# insecure_skip_verify = false
|
||||
`
|
||||
|
||||
// Elasticsearch is a plugin to read stats from one or many Elasticsearch
|
||||
// servers.
|
||||
type Elasticsearch struct {
|
||||
Local bool
|
||||
Servers []string
|
||||
ClusterHealth bool
|
||||
client *http.Client
|
||||
Local bool
|
||||
Servers []string
|
||||
ClusterHealth bool
|
||||
SSLCA string `toml:"ssl_ca"` // Path to CA file
|
||||
SSLCert string `toml:"ssl_cert"` // Path to host cert file
|
||||
SSLKey string `toml:"ssl_key"` // Path to cert key file
|
||||
InsecureSkipVerify bool // Use SSL but skip chain & host verification
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
// NewElasticsearch return a new instance of Elasticsearch
|
||||
func NewElasticsearch() *Elasticsearch {
|
||||
tr := &http.Transport{ResponseHeaderTimeout: time.Duration(3 * time.Second)}
|
||||
client := &http.Client{
|
||||
Transport: tr,
|
||||
Timeout: time.Duration(4 * time.Second),
|
||||
}
|
||||
return &Elasticsearch{client: client}
|
||||
return &Elasticsearch{}
|
||||
}
|
||||
|
||||
// SampleConfig returns sample configuration for this plugin.
|
||||
@@ -102,7 +108,16 @@ func (e *Elasticsearch) Description() string {
|
||||
// Gather reads the stats from Elasticsearch and writes it to the
|
||||
// Accumulator.
|
||||
func (e *Elasticsearch) Gather(acc telegraf.Accumulator) error {
|
||||
errChan := make(chan error, len(e.Servers))
|
||||
if e.client == nil {
|
||||
client, err := e.createHttpClient()
|
||||
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
e.client = client
|
||||
}
|
||||
|
||||
errChan := errchan.New(len(e.Servers))
|
||||
var wg sync.WaitGroup
|
||||
wg.Add(len(e.Servers))
|
||||
|
||||
@@ -116,7 +131,7 @@ func (e *Elasticsearch) Gather(acc telegraf.Accumulator) error {
|
||||
url = s + statsPath
|
||||
}
|
||||
if err := e.gatherNodeStats(url, acc); err != nil {
|
||||
errChan <- err
|
||||
errChan.C <- err
|
||||
return
|
||||
}
|
||||
if e.ClusterHealth {
|
||||
@@ -126,17 +141,24 @@ func (e *Elasticsearch) Gather(acc telegraf.Accumulator) error {
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
close(errChan)
|
||||
// Get all errors and return them as one giant error
|
||||
errStrings := []string{}
|
||||
for err := range errChan {
|
||||
errStrings = append(errStrings, err.Error())
|
||||
return errChan.Error()
|
||||
}
|
||||
|
||||
func (e *Elasticsearch) createHttpClient() (*http.Client, error) {
|
||||
tlsCfg, err := internal.GetTLSConfig(e.SSLCert, e.SSLKey, e.SSLCA, e.InsecureSkipVerify)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tr := &http.Transport{
|
||||
ResponseHeaderTimeout: time.Duration(3 * time.Second),
|
||||
TLSClientConfig: tlsCfg,
|
||||
}
|
||||
client := &http.Client{
|
||||
Transport: tr,
|
||||
Timeout: time.Duration(4 * time.Second),
|
||||
}
|
||||
|
||||
if len(errStrings) == 0 {
|
||||
return nil
|
||||
}
|
||||
return errors.New(strings.Join(errStrings, "\n"))
|
||||
return client, nil
|
||||
}
|
||||
|
||||
func (e *Elasticsearch) gatherNodeStats(url string, acc telegraf.Accumulator) error {
|
||||
|
||||
@@ -38,7 +38,7 @@ func (t *transportMock) CancelRequest(_ *http.Request) {
|
||||
}
|
||||
|
||||
func TestElasticsearch(t *testing.T) {
|
||||
es := NewElasticsearch()
|
||||
es := newElasticsearchWithClient()
|
||||
es.Servers = []string{"http://example.com:9200"}
|
||||
es.client.Transport = newTransportMock(http.StatusOK, statsResponse)
|
||||
|
||||
@@ -67,7 +67,7 @@ func TestElasticsearch(t *testing.T) {
|
||||
}
|
||||
|
||||
func TestGatherClusterStats(t *testing.T) {
|
||||
es := NewElasticsearch()
|
||||
es := newElasticsearchWithClient()
|
||||
es.Servers = []string{"http://example.com:9200"}
|
||||
es.ClusterHealth = true
|
||||
es.client.Transport = newTransportMock(http.StatusOK, clusterResponse)
|
||||
@@ -87,3 +87,9 @@ func TestGatherClusterStats(t *testing.T) {
|
||||
v2IndexExpected,
|
||||
map[string]string{"index": "v2"})
|
||||
}
|
||||
|
||||
func newElasticsearchWithClient() *Elasticsearch {
|
||||
es := NewElasticsearch()
|
||||
es.client = &http.Client{}
|
||||
return es
|
||||
}
|
||||
|
||||
@@ -6,14 +6,20 @@ Please also see: [Telegraf Input Data Formats](https://github.com/influxdata/tel
|
||||
|
||||
#### Configuration
|
||||
|
||||
In this example a script called ```/tmp/test.sh``` and a script called ```/tmp/test2.sh```
|
||||
are configured for ```[[inputs.exec]]``` in JSON format.
|
||||
In this example a script called ```/tmp/test.sh```, a script called ```/tmp/test2.sh```, and
|
||||
all scripts matching glob pattern ```/tmp/collect_*.sh``` are configured for ```[[inputs.exec]]```
|
||||
in JSON format. Glob patterns are matched on every run, so adding new scripts that match the pattern
|
||||
will cause them to be picked up immediately.
|
||||
|
||||
```
|
||||
```toml
|
||||
# Read flattened metrics from one or more commands that output JSON to stdout
|
||||
[[inputs.exec]]
|
||||
# Shell/commands array
|
||||
commands = ["/tmp/test.sh", "/tmp/test2.sh"]
|
||||
# Full command line to executable with parameters, or a glob pattern to run all matching files.
|
||||
commands = ["/tmp/test.sh", "/tmp/test2.sh", "/tmp/collect_*.sh"]
|
||||
|
||||
## Timeout for each command to complete.
|
||||
timeout = "5s"
|
||||
|
||||
# Data format to consume.
|
||||
# NOTE json only reads numerical measurements, strings and booleans are ignored.
|
||||
@@ -21,26 +27,6 @@ are configured for ```[[inputs.exec]]``` in JSON format.
|
||||
|
||||
# measurement name suffix (for separating different commands)
|
||||
name_suffix = "_mycollector"
|
||||
|
||||
## Below configuration will be used for data_format = "graphite", can be ignored for other data_format
|
||||
## If matching multiple measurement files, this string will be used to join the matched values.
|
||||
#separator = "."
|
||||
|
||||
## Each template line requires a template pattern. It can have an optional
|
||||
## filter before the template and separated by spaces. It can also have optional extra
|
||||
## tags following the template. Multiple tags should be separated by commas and no spaces
|
||||
## similar to the line protocol format. The can be only one default template.
|
||||
## Templates support below format:
|
||||
## 1. filter + template
|
||||
## 2. filter + template + extra tag
|
||||
## 3. filter + template with field key
|
||||
## 4. default template
|
||||
#templates = [
|
||||
# "*.app env.service.resource.measurement",
|
||||
# "stats.* .host.measurement* region=us-west,agent=sensu",
|
||||
# "stats2.* .host.measurement.field",
|
||||
# "measurement*"
|
||||
#]
|
||||
```
|
||||
|
||||
Other options for modifying the measurement names are:
|
||||
@@ -79,7 +65,7 @@ in influx line-protocol format.
|
||||
|
||||
#### Configuration
|
||||
|
||||
```
|
||||
```toml
|
||||
[[inputs.exec]]
|
||||
# Shell/commands array
|
||||
# compatible with old version
|
||||
@@ -87,6 +73,9 @@ in influx line-protocol format.
|
||||
# command = "/usr/bin/line_protocol_collector"
|
||||
commands = ["/usr/bin/line_protocol_collector","/tmp/test2.sh"]
|
||||
|
||||
## Timeout for each command to complete.
|
||||
timeout = "5s"
|
||||
|
||||
# Data format to consume.
|
||||
# NOTE json only reads numerical measurements, strings and booleans are ignored.
|
||||
data_format = "influx"
|
||||
@@ -120,12 +109,16 @@ We can also change the data_format to "graphite" to use the metrics collecting s
|
||||
In this example a script called /tmp/test.sh and a script called /tmp/test2.sh are configured for [[inputs.exec]] in graphite format.
|
||||
|
||||
#### Configuration
|
||||
```
|
||||
|
||||
```toml
|
||||
# Read flattened metrics from one or more commands that output JSON to stdout
|
||||
[[inputs.exec]]
|
||||
# Shell/commands array
|
||||
commands = ["/tmp/test.sh","/tmp/test2.sh"]
|
||||
|
||||
## Timeout for each command to complete.
|
||||
timeout = "5s"
|
||||
|
||||
# Data format to consume.
|
||||
# NOTE json only reads numerical measurements, strings and booleans are ignored.
|
||||
data_format = "graphite"
|
||||
@@ -180,4 +173,3 @@ sensu.metric.net.server0.eth0.rx_dropped 0 1444234982
|
||||
The templates configuration will be used to parse the graphite metrics to support influxdb/opentsdb tagging store engines.
|
||||
|
||||
More detail information about templates, please refer to [The graphite Input](https://github.com/influxdata/influxdb/blob/master/services/graphite/README.md)
|
||||
|
||||
|
||||
@@ -4,6 +4,8 @@ import (
|
||||
"bytes"
|
||||
"fmt"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strings"
|
||||
"sync"
|
||||
"syscall"
|
||||
"time"
|
||||
@@ -12,6 +14,7 @@ import (
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/internal/errchan"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"github.com/influxdata/telegraf/plugins/parsers"
|
||||
"github.com/influxdata/telegraf/plugins/parsers/nagios"
|
||||
@@ -19,7 +22,11 @@ import (
|
||||
|
||||
const sampleConfig = `
|
||||
## Commands array
|
||||
commands = ["/tmp/test.sh", "/usr/bin/mycollector --foo=bar"]
|
||||
commands = [
|
||||
"/tmp/test.sh",
|
||||
"/usr/bin/mycollector --foo=bar",
|
||||
"/tmp/collect_*.sh"
|
||||
]
|
||||
|
||||
## Timeout for each command to complete.
|
||||
timeout = "5s"
|
||||
@@ -41,8 +48,6 @@ type Exec struct {
|
||||
|
||||
parser parsers.Parser
|
||||
|
||||
wg sync.WaitGroup
|
||||
|
||||
runner Runner
|
||||
errChan chan error
|
||||
}
|
||||
@@ -112,8 +117,8 @@ func (c CommandRunner) Run(
|
||||
return out.Bytes(), nil
|
||||
}
|
||||
|
||||
func (e *Exec) ProcessCommand(command string, acc telegraf.Accumulator) {
|
||||
defer e.wg.Done()
|
||||
func (e *Exec) ProcessCommand(command string, acc telegraf.Accumulator, wg *sync.WaitGroup) {
|
||||
defer wg.Done()
|
||||
|
||||
out, err := e.runner.Run(e, command, acc)
|
||||
if err != nil {
|
||||
@@ -144,29 +149,52 @@ func (e *Exec) SetParser(parser parsers.Parser) {
|
||||
}
|
||||
|
||||
func (e *Exec) Gather(acc telegraf.Accumulator) error {
|
||||
var wg sync.WaitGroup
|
||||
// Legacy single command support
|
||||
if e.Command != "" {
|
||||
e.Commands = append(e.Commands, e.Command)
|
||||
e.Command = ""
|
||||
}
|
||||
|
||||
e.errChan = make(chan error, len(e.Commands))
|
||||
commands := make([]string, 0, len(e.Commands))
|
||||
for _, pattern := range e.Commands {
|
||||
cmdAndArgs := strings.SplitN(pattern, " ", 2)
|
||||
if len(cmdAndArgs) == 0 {
|
||||
continue
|
||||
}
|
||||
|
||||
e.wg.Add(len(e.Commands))
|
||||
for _, command := range e.Commands {
|
||||
go e.ProcessCommand(command, acc)
|
||||
}
|
||||
e.wg.Wait()
|
||||
matches, err := filepath.Glob(cmdAndArgs[0])
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
select {
|
||||
default:
|
||||
close(e.errChan)
|
||||
return nil
|
||||
case err := <-e.errChan:
|
||||
close(e.errChan)
|
||||
return err
|
||||
if len(matches) == 0 {
|
||||
// There were no matches with the glob pattern, so let's assume
|
||||
// that the command is in PATH and just run it as it is
|
||||
commands = append(commands, pattern)
|
||||
} else {
|
||||
// There were matches, so we'll append each match together with
|
||||
// the arguments to the commands slice
|
||||
for _, match := range matches {
|
||||
if len(cmdAndArgs) == 1 {
|
||||
commands = append(commands, match)
|
||||
} else {
|
||||
commands = append(commands,
|
||||
strings.Join([]string{match, cmdAndArgs[1]}, " "))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
errChan := errchan.New(len(commands))
|
||||
e.errChan = errChan.C
|
||||
|
||||
wg.Add(len(commands))
|
||||
for _, command := range commands {
|
||||
go e.ProcessCommand(command, acc, &wg)
|
||||
}
|
||||
wg.Wait()
|
||||
return errChan.Error()
|
||||
}
|
||||
|
||||
func init() {
|
||||
|
||||
@@ -169,3 +169,51 @@ func TestLineProtocolParseMultiple(t *testing.T) {
|
||||
acc.AssertContainsTaggedFields(t, "cpu", fields, tags)
|
||||
}
|
||||
}
|
||||
|
||||
func TestExecCommandWithGlob(t *testing.T) {
|
||||
parser, _ := parsers.NewValueParser("metric", "string", nil)
|
||||
e := NewExec()
|
||||
e.Commands = []string{"/bin/ech* metric_value"}
|
||||
e.SetParser(parser)
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := e.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"value": "metric_value",
|
||||
}
|
||||
acc.AssertContainsFields(t, "metric", fields)
|
||||
}
|
||||
|
||||
func TestExecCommandWithoutGlob(t *testing.T) {
|
||||
parser, _ := parsers.NewValueParser("metric", "string", nil)
|
||||
e := NewExec()
|
||||
e.Commands = []string{"/bin/echo metric_value"}
|
||||
e.SetParser(parser)
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := e.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"value": "metric_value",
|
||||
}
|
||||
acc.AssertContainsFields(t, "metric", fields)
|
||||
}
|
||||
|
||||
func TestExecCommandWithoutGlobAndPath(t *testing.T) {
|
||||
parser, _ := parsers.NewValueParser("metric", "string", nil)
|
||||
e := NewExec()
|
||||
e.Commands = []string{"echo metric_value"}
|
||||
e.SetParser(parser)
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := e.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
|
||||
fields := map[string]interface{}{
|
||||
"value": "metric_value",
|
||||
}
|
||||
acc.AssertContainsFields(t, "metric", fields)
|
||||
}
|
||||
|
||||
@@ -1,338 +0,0 @@
|
||||
package github_webhooks
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"log"
|
||||
"net/http"
|
||||
"sync"
|
||||
|
||||
"github.com/gorilla/mux"
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
func init() {
|
||||
inputs.Add("github_webhooks", func() telegraf.Input { return &GithubWebhooks{} })
|
||||
}
|
||||
|
||||
type GithubWebhooks struct {
|
||||
ServiceAddress string
|
||||
// Lock for the struct
|
||||
sync.Mutex
|
||||
// Events buffer to store events between Gather calls
|
||||
events []Event
|
||||
}
|
||||
|
||||
func NewGithubWebhooks() *GithubWebhooks {
|
||||
return &GithubWebhooks{}
|
||||
}
|
||||
|
||||
func (gh *GithubWebhooks) SampleConfig() string {
|
||||
return `
|
||||
## Address and port to host Webhook listener on
|
||||
service_address = ":1618"
|
||||
`
|
||||
}
|
||||
|
||||
func (gh *GithubWebhooks) Description() string {
|
||||
return "A Github Webhook Event collector"
|
||||
}
|
||||
|
||||
// Writes the points from <-gh.in to the Accumulator
|
||||
func (gh *GithubWebhooks) Gather(acc telegraf.Accumulator) error {
|
||||
gh.Lock()
|
||||
defer gh.Unlock()
|
||||
for _, event := range gh.events {
|
||||
p := event.NewMetric()
|
||||
acc.AddFields("github_webhooks", p.Fields(), p.Tags(), p.Time())
|
||||
}
|
||||
gh.events = make([]Event, 0)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (gh *GithubWebhooks) Listen() {
|
||||
r := mux.NewRouter()
|
||||
r.HandleFunc("/", gh.eventHandler).Methods("POST")
|
||||
err := http.ListenAndServe(fmt.Sprintf("%s", gh.ServiceAddress), r)
|
||||
if err != nil {
|
||||
log.Printf("Error starting server: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
func (gh *GithubWebhooks) Start(_ telegraf.Accumulator) error {
|
||||
go gh.Listen()
|
||||
log.Printf("Started the github_webhooks service on %s\n", gh.ServiceAddress)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (gh *GithubWebhooks) Stop() {
|
||||
log.Println("Stopping the ghWebhooks service")
|
||||
}
|
||||
|
||||
// Handles the / route
|
||||
func (gh *GithubWebhooks) eventHandler(w http.ResponseWriter, r *http.Request) {
|
||||
defer r.Body.Close()
|
||||
eventType := r.Header["X-Github-Event"][0]
|
||||
data, err := ioutil.ReadAll(r.Body)
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
e, err := NewEvent(data, eventType)
|
||||
if err != nil {
|
||||
w.WriteHeader(http.StatusBadRequest)
|
||||
return
|
||||
}
|
||||
gh.Lock()
|
||||
gh.events = append(gh.events, e)
|
||||
gh.Unlock()
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}
|
||||
|
||||
func newCommitComment(data []byte) (Event, error) {
|
||||
commitCommentStruct := CommitCommentEvent{}
|
||||
err := json.Unmarshal(data, &commitCommentStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return commitCommentStruct, nil
|
||||
}
|
||||
|
||||
func newCreate(data []byte) (Event, error) {
|
||||
createStruct := CreateEvent{}
|
||||
err := json.Unmarshal(data, &createStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return createStruct, nil
|
||||
}
|
||||
|
||||
func newDelete(data []byte) (Event, error) {
|
||||
deleteStruct := DeleteEvent{}
|
||||
err := json.Unmarshal(data, &deleteStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return deleteStruct, nil
|
||||
}
|
||||
|
||||
func newDeployment(data []byte) (Event, error) {
|
||||
deploymentStruct := DeploymentEvent{}
|
||||
err := json.Unmarshal(data, &deploymentStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return deploymentStruct, nil
|
||||
}
|
||||
|
||||
func newDeploymentStatus(data []byte) (Event, error) {
|
||||
deploymentStatusStruct := DeploymentStatusEvent{}
|
||||
err := json.Unmarshal(data, &deploymentStatusStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return deploymentStatusStruct, nil
|
||||
}
|
||||
|
||||
func newFork(data []byte) (Event, error) {
|
||||
forkStruct := ForkEvent{}
|
||||
err := json.Unmarshal(data, &forkStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return forkStruct, nil
|
||||
}
|
||||
|
||||
func newGollum(data []byte) (Event, error) {
|
||||
gollumStruct := GollumEvent{}
|
||||
err := json.Unmarshal(data, &gollumStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return gollumStruct, nil
|
||||
}
|
||||
|
||||
func newIssueComment(data []byte) (Event, error) {
|
||||
issueCommentStruct := IssueCommentEvent{}
|
||||
err := json.Unmarshal(data, &issueCommentStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return issueCommentStruct, nil
|
||||
}
|
||||
|
||||
func newIssues(data []byte) (Event, error) {
|
||||
issuesStruct := IssuesEvent{}
|
||||
err := json.Unmarshal(data, &issuesStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return issuesStruct, nil
|
||||
}
|
||||
|
||||
func newMember(data []byte) (Event, error) {
|
||||
memberStruct := MemberEvent{}
|
||||
err := json.Unmarshal(data, &memberStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return memberStruct, nil
|
||||
}
|
||||
|
||||
func newMembership(data []byte) (Event, error) {
|
||||
membershipStruct := MembershipEvent{}
|
||||
err := json.Unmarshal(data, &membershipStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return membershipStruct, nil
|
||||
}
|
||||
|
||||
func newPageBuild(data []byte) (Event, error) {
|
||||
pageBuildEvent := PageBuildEvent{}
|
||||
err := json.Unmarshal(data, &pageBuildEvent)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return pageBuildEvent, nil
|
||||
}
|
||||
|
||||
func newPublic(data []byte) (Event, error) {
|
||||
publicEvent := PublicEvent{}
|
||||
err := json.Unmarshal(data, &publicEvent)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return publicEvent, nil
|
||||
}
|
||||
|
||||
func newPullRequest(data []byte) (Event, error) {
|
||||
pullRequestStruct := PullRequestEvent{}
|
||||
err := json.Unmarshal(data, &pullRequestStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return pullRequestStruct, nil
|
||||
}
|
||||
|
||||
func newPullRequestReviewComment(data []byte) (Event, error) {
|
||||
pullRequestReviewCommentStruct := PullRequestReviewCommentEvent{}
|
||||
err := json.Unmarshal(data, &pullRequestReviewCommentStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return pullRequestReviewCommentStruct, nil
|
||||
}
|
||||
|
||||
func newPush(data []byte) (Event, error) {
|
||||
pushStruct := PushEvent{}
|
||||
err := json.Unmarshal(data, &pushStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return pushStruct, nil
|
||||
}
|
||||
|
||||
func newRelease(data []byte) (Event, error) {
|
||||
releaseStruct := ReleaseEvent{}
|
||||
err := json.Unmarshal(data, &releaseStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return releaseStruct, nil
|
||||
}
|
||||
|
||||
func newRepository(data []byte) (Event, error) {
|
||||
repositoryStruct := RepositoryEvent{}
|
||||
err := json.Unmarshal(data, &repositoryStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return repositoryStruct, nil
|
||||
}
|
||||
|
||||
func newStatus(data []byte) (Event, error) {
|
||||
statusStruct := StatusEvent{}
|
||||
err := json.Unmarshal(data, &statusStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return statusStruct, nil
|
||||
}
|
||||
|
||||
func newTeamAdd(data []byte) (Event, error) {
|
||||
teamAddStruct := TeamAddEvent{}
|
||||
err := json.Unmarshal(data, &teamAddStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return teamAddStruct, nil
|
||||
}
|
||||
|
||||
func newWatch(data []byte) (Event, error) {
|
||||
watchStruct := WatchEvent{}
|
||||
err := json.Unmarshal(data, &watchStruct)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return watchStruct, nil
|
||||
}
|
||||
|
||||
type newEventError struct {
|
||||
s string
|
||||
}
|
||||
|
||||
func (e *newEventError) Error() string {
|
||||
return e.s
|
||||
}
|
||||
|
||||
func NewEvent(r []byte, t string) (Event, error) {
|
||||
log.Printf("New %v event recieved", t)
|
||||
switch t {
|
||||
case "commit_comment":
|
||||
return newCommitComment(r)
|
||||
case "create":
|
||||
return newCreate(r)
|
||||
case "delete":
|
||||
return newDelete(r)
|
||||
case "deployment":
|
||||
return newDeployment(r)
|
||||
case "deployment_status":
|
||||
return newDeploymentStatus(r)
|
||||
case "fork":
|
||||
return newFork(r)
|
||||
case "gollum":
|
||||
return newGollum(r)
|
||||
case "issue_comment":
|
||||
return newIssueComment(r)
|
||||
case "issues":
|
||||
return newIssues(r)
|
||||
case "member":
|
||||
return newMember(r)
|
||||
case "membership":
|
||||
return newMembership(r)
|
||||
case "page_build":
|
||||
return newPageBuild(r)
|
||||
case "public":
|
||||
return newPublic(r)
|
||||
case "pull_request":
|
||||
return newPullRequest(r)
|
||||
case "pull_request_review_comment":
|
||||
return newPullRequestReviewComment(r)
|
||||
case "push":
|
||||
return newPush(r)
|
||||
case "release":
|
||||
return newRelease(r)
|
||||
case "repository":
|
||||
return newRepository(r)
|
||||
case "status":
|
||||
return newStatus(r)
|
||||
case "team_add":
|
||||
return newTeamAdd(r)
|
||||
case "watch":
|
||||
return newWatch(r)
|
||||
}
|
||||
return nil, &newEventError{"Not a recgonized event type"}
|
||||
}
|
||||
@@ -1,237 +0,0 @@
|
||||
package github_webhooks
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestCommitCommentEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := CommitCommentEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "commit_comment")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeleteEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := DeleteEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "delete")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeploymentEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := DeploymentEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "deployment")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestDeploymentStatusEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := DeploymentStatusEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "deployment_status")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestForkEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := ForkEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "fork")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestGollumEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := GollumEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "gollum")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIssueCommentEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := IssueCommentEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "issue_comment")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestIssuesEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := IssuesEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "issues")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMemberEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := MemberEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "member")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestMembershipEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := MembershipEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "membership")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPageBuildEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := PageBuildEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "page_build")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPublicEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := PublicEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "public")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPullRequestReviewCommentEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := PullRequestReviewCommentEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "pull_request_review_comment")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestPushEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := PushEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "push")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReleaseEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := ReleaseEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "release")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRepositoryEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := RepositoryEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "repository")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestStatusEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
|
||||
jsonString := StatusEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "status")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTeamAddEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := TeamAddEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "team_add")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
|
||||
func TestWatchEvent(t *testing.T) {
|
||||
gh := NewGithubWebhooks()
|
||||
jsonString := WatchEventJSON()
|
||||
req, _ := http.NewRequest("POST", "/", strings.NewReader(jsonString))
|
||||
req.Header.Add("X-Github-Event", "watch")
|
||||
w := httptest.NewRecorder()
|
||||
gh.eventHandler(w, req)
|
||||
if w.Code != http.StatusOK {
|
||||
t.Errorf("POST commit_comment returned HTTP status code %v.\nExpected %v", w.Code, http.StatusOK)
|
||||
}
|
||||
}
|
||||
55
plugins/inputs/graylog/README.md
Normal file
55
plugins/inputs/graylog/README.md
Normal file
@@ -0,0 +1,55 @@
|
||||
# GrayLog plugin
|
||||
|
||||
The Graylog plugin can collect data from remote Graylog service URLs.
|
||||
|
||||
Plugin currently support two type of end points:-
|
||||
|
||||
- multiple (Ex http://[graylog-server-ip]:12900/system/metrics/multiple)
|
||||
- namespace (Ex http://[graylog-server-ip]:12900/system/metrics/namespace/{namespace})
|
||||
|
||||
End Point can be a mixe of one multiple end point and several namespaces end points
|
||||
|
||||
|
||||
Note: if namespace end point specified metrics array will be ignored for that call.
|
||||
|
||||
### Configuration:
|
||||
|
||||
```toml
|
||||
# Read flattened metrics from one or more GrayLog HTTP endpoints
|
||||
[[inputs.graylog]]
|
||||
## API endpoint, currently supported API:
|
||||
##
|
||||
## - multiple (Ex http://<host>:12900/system/metrics/multiple)
|
||||
## - namespace (Ex http://<host>:12900/system/metrics/namespace/{namespace})
|
||||
##
|
||||
## For namespace endpoint, the metrics array will be ignored for that call.
|
||||
## Endpoint can contain namespace and multiple type calls.
|
||||
##
|
||||
## Please check http://[graylog-server-ip]:12900/api-browser for full list
|
||||
## of endpoints
|
||||
servers = [
|
||||
"http://[graylog-server-ip]:12900/system/metrics/multiple",
|
||||
]
|
||||
|
||||
## Metrics list
|
||||
## List of metrics can be found on Graylog webservice documentation.
|
||||
## Or by hitting the the web service api at:
|
||||
## http://[graylog-host]:12900/system/metrics
|
||||
metrics = [
|
||||
"jvm.cl.loaded",
|
||||
"jvm.memory.pools.Metaspace.committed"
|
||||
]
|
||||
|
||||
## Username and password
|
||||
username = ""
|
||||
password = ""
|
||||
|
||||
## Optional SSL Config
|
||||
# ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# ssl_key = "/etc/telegraf/key.pem"
|
||||
## Use SSL but skip chain & host verification
|
||||
# insecure_skip_verify = false
|
||||
```
|
||||
|
||||
Please refer to GrayLog metrics api browser for full metric end points http://host:12900/api-browser
|
||||
312
plugins/inputs/graylog/graylog.go
Normal file
312
plugins/inputs/graylog/graylog.go
Normal file
@@ -0,0 +1,312 @@
|
||||
package graylog
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"encoding/base64"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io/ioutil"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
type ResponseMetrics struct {
|
||||
total int
|
||||
Metrics []Metric `json:"metrics"`
|
||||
}
|
||||
|
||||
type Metric struct {
|
||||
FullName string `json:"full_name"`
|
||||
Name string `json:"name"`
|
||||
Type string `json:"type"`
|
||||
Fields map[string]interface{} `json:"metric"`
|
||||
}
|
||||
|
||||
type GrayLog struct {
|
||||
Servers []string
|
||||
Metrics []string
|
||||
Username string
|
||||
Password string
|
||||
|
||||
// Path to CA file
|
||||
SSLCA string `toml:"ssl_ca"`
|
||||
// Path to host cert file
|
||||
SSLCert string `toml:"ssl_cert"`
|
||||
// Path to cert key file
|
||||
SSLKey string `toml:"ssl_key"`
|
||||
// Use SSL but skip chain & host verification
|
||||
InsecureSkipVerify bool
|
||||
|
||||
client HTTPClient
|
||||
}
|
||||
|
||||
type HTTPClient interface {
|
||||
// Returns the result of an http request
|
||||
//
|
||||
// Parameters:
|
||||
// req: HTTP request object
|
||||
//
|
||||
// Returns:
|
||||
// http.Response: HTTP respons object
|
||||
// error : Any error that may have occurred
|
||||
MakeRequest(req *http.Request) (*http.Response, error)
|
||||
|
||||
SetHTTPClient(client *http.Client)
|
||||
HTTPClient() *http.Client
|
||||
}
|
||||
|
||||
type Messagebody struct {
|
||||
Metrics []string `json:"metrics"`
|
||||
}
|
||||
|
||||
type RealHTTPClient struct {
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
func (c *RealHTTPClient) MakeRequest(req *http.Request) (*http.Response, error) {
|
||||
return c.client.Do(req)
|
||||
}
|
||||
|
||||
func (c *RealHTTPClient) SetHTTPClient(client *http.Client) {
|
||||
c.client = client
|
||||
}
|
||||
|
||||
func (c *RealHTTPClient) HTTPClient() *http.Client {
|
||||
return c.client
|
||||
}
|
||||
|
||||
var sampleConfig = `
|
||||
## API endpoint, currently supported API:
|
||||
##
|
||||
## - multiple (Ex http://<host>:12900/system/metrics/multiple)
|
||||
## - namespace (Ex http://<host>:12900/system/metrics/namespace/{namespace})
|
||||
##
|
||||
## For namespace endpoint, the metrics array will be ignored for that call.
|
||||
## Endpoint can contain namespace and multiple type calls.
|
||||
##
|
||||
## Please check http://[graylog-server-ip]:12900/api-browser for full list
|
||||
## of endpoints
|
||||
servers = [
|
||||
"http://[graylog-server-ip]:12900/system/metrics/multiple",
|
||||
]
|
||||
|
||||
## Metrics list
|
||||
## List of metrics can be found on Graylog webservice documentation.
|
||||
## Or by hitting the the web service api at:
|
||||
## http://[graylog-host]:12900/system/metrics
|
||||
metrics = [
|
||||
"jvm.cl.loaded",
|
||||
"jvm.memory.pools.Metaspace.committed"
|
||||
]
|
||||
|
||||
## Username and password
|
||||
username = ""
|
||||
password = ""
|
||||
|
||||
## Optional SSL Config
|
||||
# ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# ssl_key = "/etc/telegraf/key.pem"
|
||||
## Use SSL but skip chain & host verification
|
||||
# insecure_skip_verify = false
|
||||
`
|
||||
|
||||
func (h *GrayLog) SampleConfig() string {
|
||||
return sampleConfig
|
||||
}
|
||||
|
||||
func (h *GrayLog) Description() string {
|
||||
return "Read flattened metrics from one or more GrayLog HTTP endpoints"
|
||||
}
|
||||
|
||||
// Gathers data for all servers.
|
||||
func (h *GrayLog) Gather(acc telegraf.Accumulator) error {
|
||||
var wg sync.WaitGroup
|
||||
|
||||
if h.client.HTTPClient() == nil {
|
||||
tlsCfg, err := internal.GetTLSConfig(
|
||||
h.SSLCert, h.SSLKey, h.SSLCA, h.InsecureSkipVerify)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
tr := &http.Transport{
|
||||
ResponseHeaderTimeout: time.Duration(3 * time.Second),
|
||||
TLSClientConfig: tlsCfg,
|
||||
}
|
||||
client := &http.Client{
|
||||
Transport: tr,
|
||||
Timeout: time.Duration(4 * time.Second),
|
||||
}
|
||||
h.client.SetHTTPClient(client)
|
||||
}
|
||||
|
||||
errorChannel := make(chan error, len(h.Servers))
|
||||
|
||||
for _, server := range h.Servers {
|
||||
wg.Add(1)
|
||||
go func(server string) {
|
||||
defer wg.Done()
|
||||
if err := h.gatherServer(acc, server); err != nil {
|
||||
errorChannel <- err
|
||||
}
|
||||
}(server)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
close(errorChannel)
|
||||
|
||||
// Get all errors and return them as one giant error
|
||||
errorStrings := []string{}
|
||||
for err := range errorChannel {
|
||||
errorStrings = append(errorStrings, err.Error())
|
||||
}
|
||||
|
||||
if len(errorStrings) == 0 {
|
||||
return nil
|
||||
}
|
||||
return errors.New(strings.Join(errorStrings, "\n"))
|
||||
}
|
||||
|
||||
// Gathers data from a particular server
|
||||
// Parameters:
|
||||
// acc : The telegraf Accumulator to use
|
||||
// serverURL: endpoint to send request to
|
||||
// service : the service being queried
|
||||
//
|
||||
// Returns:
|
||||
// error: Any error that may have occurred
|
||||
func (h *GrayLog) gatherServer(
|
||||
acc telegraf.Accumulator,
|
||||
serverURL string,
|
||||
) error {
|
||||
resp, _, err := h.sendRequest(serverURL)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
requestURL, err := url.Parse(serverURL)
|
||||
host, port, _ := net.SplitHostPort(requestURL.Host)
|
||||
var dat ResponseMetrics
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if err := json.Unmarshal([]byte(resp), &dat); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, m_item := range dat.Metrics {
|
||||
fields := make(map[string]interface{})
|
||||
tags := map[string]string{
|
||||
"server": host,
|
||||
"port": port,
|
||||
"name": m_item.Name,
|
||||
"type": m_item.Type,
|
||||
}
|
||||
h.flatten(m_item.Fields, fields, "")
|
||||
acc.AddFields(m_item.FullName, fields, tags)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// Flatten JSON hierarchy to produce field name and field value
|
||||
// Parameters:
|
||||
// item: Item map to flatten
|
||||
// fields: Map to store generated fields.
|
||||
// id: Prefix for top level metric (empty string "")
|
||||
// Returns:
|
||||
// void
|
||||
func (h *GrayLog) flatten(item map[string]interface{}, fields map[string]interface{}, id string) {
|
||||
if id != "" {
|
||||
id = id + "_"
|
||||
}
|
||||
for k, i := range item {
|
||||
switch i.(type) {
|
||||
case int:
|
||||
fields[id+k] = i.(float64)
|
||||
case float64:
|
||||
fields[id+k] = i.(float64)
|
||||
case map[string]interface{}:
|
||||
h.flatten(i.(map[string]interface{}), fields, id+k)
|
||||
default:
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Sends an HTTP request to the server using the GrayLog object's HTTPClient.
|
||||
// Parameters:
|
||||
// serverURL: endpoint to send request to
|
||||
//
|
||||
// Returns:
|
||||
// string: body of the response
|
||||
// error : Any error that may have occurred
|
||||
func (h *GrayLog) sendRequest(serverURL string) (string, float64, error) {
|
||||
headers := map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
"Accept": "application/json",
|
||||
}
|
||||
method := "GET"
|
||||
content := bytes.NewBufferString("")
|
||||
headers["Authorization"] = "Basic " + base64.URLEncoding.EncodeToString([]byte(h.Username+":"+h.Password))
|
||||
// Prepare URL
|
||||
requestURL, err := url.Parse(serverURL)
|
||||
if err != nil {
|
||||
return "", -1, fmt.Errorf("Invalid server URL \"%s\"", serverURL)
|
||||
}
|
||||
if strings.Contains(requestURL.String(), "multiple") {
|
||||
m := &Messagebody{Metrics: h.Metrics}
|
||||
http_body, err := json.Marshal(m)
|
||||
if err != nil {
|
||||
return "", -1, fmt.Errorf("Invalid list of Metrics %s", h.Metrics)
|
||||
}
|
||||
method = "POST"
|
||||
content = bytes.NewBuffer(http_body)
|
||||
}
|
||||
req, err := http.NewRequest(method, requestURL.String(), content)
|
||||
if err != nil {
|
||||
return "", -1, err
|
||||
}
|
||||
// Add header parameters
|
||||
for k, v := range headers {
|
||||
req.Header.Add(k, v)
|
||||
}
|
||||
start := time.Now()
|
||||
resp, err := h.client.MakeRequest(req)
|
||||
if err != nil {
|
||||
return "", -1, err
|
||||
}
|
||||
|
||||
defer resp.Body.Close()
|
||||
responseTime := time.Since(start).Seconds()
|
||||
|
||||
body, err := ioutil.ReadAll(resp.Body)
|
||||
if err != nil {
|
||||
return string(body), responseTime, err
|
||||
}
|
||||
|
||||
// Process response
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
err = fmt.Errorf("Response from url \"%s\" has status code %d (%s), expected %d (%s)",
|
||||
requestURL.String(),
|
||||
resp.StatusCode,
|
||||
http.StatusText(resp.StatusCode),
|
||||
http.StatusOK,
|
||||
http.StatusText(http.StatusOK))
|
||||
return string(body), responseTime, err
|
||||
}
|
||||
return string(body), responseTime, err
|
||||
}
|
||||
|
||||
func init() {
|
||||
inputs.Add("graylog", func() telegraf.Input {
|
||||
return &GrayLog{
|
||||
client: &RealHTTPClient{},
|
||||
}
|
||||
})
|
||||
}
|
||||
199
plugins/inputs/graylog/graylog_test.go
Normal file
199
plugins/inputs/graylog/graylog_test.go
Normal file
@@ -0,0 +1,199 @@
|
||||
package graylog
|
||||
|
||||
import (
|
||||
"io/ioutil"
|
||||
"net/http"
|
||||
"strings"
|
||||
"testing"
|
||||
|
||||
"github.com/influxdata/telegraf/testutil"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
const validJSON = `
|
||||
{
|
||||
"total": 3,
|
||||
"metrics": [
|
||||
{
|
||||
"full_name": "jvm.cl.loaded",
|
||||
"metric": {
|
||||
"value": 18910
|
||||
},
|
||||
"name": "loaded",
|
||||
"type": "gauge"
|
||||
},
|
||||
{
|
||||
"full_name": "jvm.memory.pools.Metaspace.committed",
|
||||
"metric": {
|
||||
"value": 108040192
|
||||
},
|
||||
"name": "committed",
|
||||
"type": "gauge"
|
||||
},
|
||||
{
|
||||
"full_name": "org.graylog2.shared.journal.KafkaJournal.writeTime",
|
||||
"metric": {
|
||||
"time": {
|
||||
"min": 99
|
||||
},
|
||||
"rate": {
|
||||
"total": 10,
|
||||
"mean": 2
|
||||
},
|
||||
"duration_unit": "microseconds",
|
||||
"rate_unit": "events/second"
|
||||
},
|
||||
"name": "writeTime",
|
||||
"type": "hdrtimer"
|
||||
}
|
||||
]
|
||||
}`
|
||||
|
||||
var validTags = map[string]map[string]string{
|
||||
"jvm.cl.loaded": {
|
||||
"name": "loaded",
|
||||
"type": "gauge",
|
||||
"port": "12900",
|
||||
"server": "localhost",
|
||||
},
|
||||
"jvm.memory.pools.Metaspace.committed": {
|
||||
"name": "committed",
|
||||
"type": "gauge",
|
||||
"port": "12900",
|
||||
"server": "localhost",
|
||||
},
|
||||
"org.graylog2.shared.journal.KafkaJournal.writeTime": {
|
||||
"name": "writeTime",
|
||||
"type": "hdrtimer",
|
||||
"port": "12900",
|
||||
"server": "localhost",
|
||||
},
|
||||
}
|
||||
|
||||
var expectedFields = map[string]map[string]interface{}{
|
||||
"jvm.cl.loaded": {
|
||||
"value": float64(18910),
|
||||
},
|
||||
"jvm.memory.pools.Metaspace.committed": {
|
||||
"value": float64(108040192),
|
||||
},
|
||||
"org.graylog2.shared.journal.KafkaJournal.writeTime": {
|
||||
"time_min": float64(99),
|
||||
"rate_total": float64(10),
|
||||
"rate_mean": float64(2),
|
||||
},
|
||||
}
|
||||
|
||||
const invalidJSON = "I don't think this is JSON"
|
||||
|
||||
const empty = ""
|
||||
|
||||
type mockHTTPClient struct {
|
||||
responseBody string
|
||||
statusCode int
|
||||
}
|
||||
|
||||
// Mock implementation of MakeRequest. Usually returns an http.Response with
|
||||
// hard-coded responseBody and statusCode. However, if the request uses a
|
||||
// nonstandard method, it uses status code 405 (method not allowed)
|
||||
func (c *mockHTTPClient) MakeRequest(req *http.Request) (*http.Response, error) {
|
||||
resp := http.Response{}
|
||||
resp.StatusCode = c.statusCode
|
||||
|
||||
// basic error checking on request method
|
||||
allowedMethods := []string{"GET", "HEAD", "POST", "PUT", "DELETE", "TRACE", "CONNECT"}
|
||||
methodValid := false
|
||||
for _, method := range allowedMethods {
|
||||
if req.Method == method {
|
||||
methodValid = true
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if !methodValid {
|
||||
resp.StatusCode = 405 // Method not allowed
|
||||
}
|
||||
|
||||
resp.Body = ioutil.NopCloser(strings.NewReader(c.responseBody))
|
||||
return &resp, nil
|
||||
}
|
||||
|
||||
func (c *mockHTTPClient) SetHTTPClient(_ *http.Client) {
|
||||
}
|
||||
|
||||
func (c *mockHTTPClient) HTTPClient() *http.Client {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Generates a pointer to an HttpJson object that uses a mock HTTP client.
|
||||
// Parameters:
|
||||
// response : Body of the response that the mock HTTP client should return
|
||||
// statusCode: HTTP status code the mock HTTP client should return
|
||||
//
|
||||
// Returns:
|
||||
// *HttpJson: Pointer to an HttpJson object that uses the generated mock HTTP client
|
||||
func genMockGrayLog(response string, statusCode int) []*GrayLog {
|
||||
return []*GrayLog{
|
||||
&GrayLog{
|
||||
client: &mockHTTPClient{responseBody: response, statusCode: statusCode},
|
||||
Servers: []string{
|
||||
"http://localhost:12900/system/metrics/multiple",
|
||||
},
|
||||
Metrics: []string{
|
||||
"jvm.memory.pools.Metaspace.committed",
|
||||
"jvm.cl.loaded",
|
||||
"org.graylog2.shared.journal.KafkaJournal.writeTime",
|
||||
},
|
||||
Username: "test",
|
||||
Password: "test",
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// Test that the proper values are ignored or collected
|
||||
func TestNormalResponse(t *testing.T) {
|
||||
graylog := genMockGrayLog(validJSON, 200)
|
||||
|
||||
for _, service := range graylog {
|
||||
var acc testutil.Accumulator
|
||||
err := service.Gather(&acc)
|
||||
require.NoError(t, err)
|
||||
for k, v := range expectedFields {
|
||||
acc.AssertContainsTaggedFields(t, k, v, validTags[k])
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Test response to HTTP 500
|
||||
func TestHttpJson500(t *testing.T) {
|
||||
graylog := genMockGrayLog(validJSON, 500)
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := graylog[0].Gather(&acc)
|
||||
|
||||
assert.NotNil(t, err)
|
||||
assert.Equal(t, 0, acc.NFields())
|
||||
}
|
||||
|
||||
// Test response to malformed JSON
|
||||
func TestHttpJsonBadJson(t *testing.T) {
|
||||
graylog := genMockGrayLog(invalidJSON, 200)
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := graylog[0].Gather(&acc)
|
||||
|
||||
assert.NotNil(t, err)
|
||||
assert.Equal(t, 0, acc.NFields())
|
||||
}
|
||||
|
||||
// Test response to empty string as response objectgT
|
||||
func TestHttpJsonEmptyResponse(t *testing.T) {
|
||||
graylog := genMockGrayLog(empty, 200)
|
||||
|
||||
var acc testutil.Accumulator
|
||||
err := graylog[0].Gather(&acc)
|
||||
|
||||
assert.NotNil(t, err)
|
||||
assert.Equal(t, 0, acc.NFields())
|
||||
}
|
||||
@@ -3,8 +3,6 @@ package haproxy
|
||||
import (
|
||||
"encoding/csv"
|
||||
"fmt"
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
"io"
|
||||
"net"
|
||||
"net/http"
|
||||
@@ -13,6 +11,10 @@ import (
|
||||
"strings"
|
||||
"sync"
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal/errchan"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
//CSV format: https://cbonte.github.io/haproxy-dconv/configuration-1.5.html#9.1
|
||||
@@ -113,20 +115,17 @@ func (g *haproxy) Gather(acc telegraf.Accumulator) error {
|
||||
}
|
||||
|
||||
var wg sync.WaitGroup
|
||||
|
||||
var outerr error
|
||||
|
||||
for _, serv := range g.Servers {
|
||||
wg.Add(1)
|
||||
errChan := errchan.New(len(g.Servers))
|
||||
wg.Add(len(g.Servers))
|
||||
for _, server := range g.Servers {
|
||||
go func(serv string) {
|
||||
defer wg.Done()
|
||||
outerr = g.gatherServer(serv, acc)
|
||||
}(serv)
|
||||
errChan.C <- g.gatherServer(serv, acc)
|
||||
}(server)
|
||||
}
|
||||
|
||||
wg.Wait()
|
||||
|
||||
return outerr
|
||||
return errChan.Error()
|
||||
}
|
||||
|
||||
func (g *haproxy) gatherServerSocket(addr string, acc telegraf.Accumulator) error {
|
||||
|
||||
@@ -5,23 +5,30 @@ This input plugin will test HTTP/HTTPS connections.
|
||||
### Configuration:
|
||||
|
||||
```
|
||||
# List of UDP/TCP connections you want to check
|
||||
# HTTP/HTTPS request given an address a method and a timeout
|
||||
[[inputs.http_response]]
|
||||
## Server address (default http://localhost)
|
||||
address = "http://github.com"
|
||||
## Set response_timeout (default 5 seconds)
|
||||
response_timeout = 5
|
||||
response_timeout = "5s"
|
||||
## HTTP Request Method
|
||||
method = "GET"
|
||||
## HTTP Request Headers
|
||||
[inputs.http_response.headers]
|
||||
Host = github.com
|
||||
## Whether to follow redirects from the server (defaults to false)
|
||||
follow_redirects = true
|
||||
## HTTP Request Headers (all values must be strings)
|
||||
# [inputs.http_response.headers]
|
||||
# Host = "github.com"
|
||||
## Optional HTTP Request Body
|
||||
body = '''
|
||||
{'fake':'data'}
|
||||
'''
|
||||
# body = '''
|
||||
# {'fake':'data'}
|
||||
# '''
|
||||
|
||||
## Optional SSL Config
|
||||
# ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# ssl_key = "/etc/telegraf/key.pem"
|
||||
## Use SSL but skip chain & host verification
|
||||
# insecure_skip_verify = false
|
||||
```
|
||||
|
||||
### Measurements & Fields:
|
||||
|
||||
@@ -9,6 +9,7 @@ import (
|
||||
"time"
|
||||
|
||||
"github.com/influxdata/telegraf"
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
"github.com/influxdata/telegraf/plugins/inputs"
|
||||
)
|
||||
|
||||
@@ -17,9 +18,18 @@ type HTTPResponse struct {
|
||||
Address string
|
||||
Body string
|
||||
Method string
|
||||
ResponseTimeout int
|
||||
ResponseTimeout internal.Duration
|
||||
Headers map[string]string
|
||||
FollowRedirects bool
|
||||
|
||||
// Path to CA file
|
||||
SSLCA string `toml:"ssl_ca"`
|
||||
// Path to host cert file
|
||||
SSLCert string `toml:"ssl_cert"`
|
||||
// Path to cert key file
|
||||
SSLKey string `toml:"ssl_key"`
|
||||
// Use SSL but skip chain & host verification
|
||||
InsecureSkipVerify bool
|
||||
}
|
||||
|
||||
// Description returns the plugin Description
|
||||
@@ -31,7 +41,7 @@ var sampleConfig = `
|
||||
## Server address (default http://localhost)
|
||||
address = "http://github.com"
|
||||
## Set response_timeout (default 5 seconds)
|
||||
response_timeout = 5
|
||||
response_timeout = "5s"
|
||||
## HTTP Request Method
|
||||
method = "GET"
|
||||
## Whether to follow redirects from the server (defaults to false)
|
||||
@@ -43,6 +53,13 @@ var sampleConfig = `
|
||||
# body = '''
|
||||
# {'fake':'data'}
|
||||
# '''
|
||||
|
||||
## Optional SSL Config
|
||||
# ssl_ca = "/etc/telegraf/ca.pem"
|
||||
# ssl_cert = "/etc/telegraf/cert.pem"
|
||||
# ssl_key = "/etc/telegraf/key.pem"
|
||||
## Use SSL but skip chain & host verification
|
||||
# insecure_skip_verify = false
|
||||
`
|
||||
|
||||
// SampleConfig returns the plugin SampleConfig
|
||||
@@ -55,27 +72,27 @@ var ErrRedirectAttempted = errors.New("redirect")
|
||||
|
||||
// CreateHttpClient creates an http client which will timeout at the specified
|
||||
// timeout period and can follow redirects if specified
|
||||
func CreateHttpClient(followRedirects bool, ResponseTimeout time.Duration) *http.Client {
|
||||
func (h *HTTPResponse) createHttpClient() (*http.Client, error) {
|
||||
tlsCfg, err := internal.GetTLSConfig(
|
||||
h.SSLCert, h.SSLKey, h.SSLCA, h.InsecureSkipVerify)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
tr := &http.Transport{
|
||||
ResponseHeaderTimeout: h.ResponseTimeout.Duration,
|
||||
TLSClientConfig: tlsCfg,
|
||||
}
|
||||
client := &http.Client{
|
||||
Timeout: time.Second * ResponseTimeout,
|
||||
Transport: tr,
|
||||
Timeout: h.ResponseTimeout.Duration,
|
||||
}
|
||||
|
||||
if followRedirects == false {
|
||||
if h.FollowRedirects == false {
|
||||
client.CheckRedirect = func(req *http.Request, via []*http.Request) error {
|
||||
return ErrRedirectAttempted
|
||||
}
|
||||
}
|
||||
return client
|
||||
}
|
||||
|
||||
// CreateHeaders takes a map of header strings and puts them
|
||||
// into a http.Header Object
|
||||
func CreateHeaders(headers map[string]string) http.Header {
|
||||
httpHeaders := make(http.Header)
|
||||
for key := range headers {
|
||||
httpHeaders.Add(key, headers[key])
|
||||
}
|
||||
return httpHeaders
|
||||
return client, nil
|
||||
}
|
||||
|
||||
// HTTPGather gathers all fields and returns any errors it encounters
|
||||
@@ -83,7 +100,10 @@ func (h *HTTPResponse) HTTPGather() (map[string]interface{}, error) {
|
||||
// Prepare fields
|
||||
fields := make(map[string]interface{})
|
||||
|
||||
client := CreateHttpClient(h.FollowRedirects, time.Duration(h.ResponseTimeout))
|
||||
client, err := h.createHttpClient()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
var body io.Reader
|
||||
if h.Body != "" {
|
||||
@@ -93,7 +113,13 @@ func (h *HTTPResponse) HTTPGather() (map[string]interface{}, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
request.Header = CreateHeaders(h.Headers)
|
||||
|
||||
for key, val := range h.Headers {
|
||||
request.Header.Add(key, val)
|
||||
if key == "Host" {
|
||||
request.Host = val
|
||||
}
|
||||
}
|
||||
|
||||
// Start Timer
|
||||
start := time.Now()
|
||||
@@ -117,8 +143,8 @@ func (h *HTTPResponse) HTTPGather() (map[string]interface{}, error) {
|
||||
// Gather gets all metric fields and tags and returns any errors it encounters
|
||||
func (h *HTTPResponse) Gather(acc telegraf.Accumulator) error {
|
||||
// Set default values
|
||||
if h.ResponseTimeout < 1 {
|
||||
h.ResponseTimeout = 5
|
||||
if h.ResponseTimeout.Duration < time.Second {
|
||||
h.ResponseTimeout.Duration = time.Second * 5
|
||||
}
|
||||
// Check send and expected string
|
||||
if h.Method == "" {
|
||||
|
||||
@@ -2,28 +2,17 @@ package http_response
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
"io/ioutil"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestCreateHeaders(t *testing.T) {
|
||||
fakeHeaders := map[string]string{
|
||||
"Accept": "text/plain",
|
||||
"Content-Type": "application/json",
|
||||
"Cache-Control": "no-cache",
|
||||
}
|
||||
headers := CreateHeaders(fakeHeaders)
|
||||
testHeaders := make(http.Header)
|
||||
testHeaders.Add("Accept", "text/plain")
|
||||
testHeaders.Add("Content-Type", "application/json")
|
||||
testHeaders.Add("Cache-Control", "no-cache")
|
||||
assert.Equal(t, testHeaders, headers)
|
||||
}
|
||||
"github.com/influxdata/telegraf/internal"
|
||||
|
||||
"github.com/stretchr/testify/assert"
|
||||
"github.com/stretchr/testify/require"
|
||||
)
|
||||
|
||||
func setUpTestMux() http.Handler {
|
||||
mux := http.NewServeMux()
|
||||
@@ -63,6 +52,33 @@ func setUpTestMux() http.Handler {
|
||||
return mux
|
||||
}
|
||||
|
||||
func TestHeaders(t *testing.T) {
|
||||
ts := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
cHeader := r.Header.Get("Content-Type")
|
||||
assert.Equal(t, "Hello", r.Host)
|
||||
assert.Equal(t, "application/json", cHeader)
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}))
|
||||
defer ts.Close()
|
||||
|
||||
h := &HTTPResponse{
|
||||
Address: ts.URL,
|
||||
Method: "GET",
|
||||
ResponseTimeout: internal.Duration{Duration: time.Second * 2},
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
"Host": "Hello",
|
||||
},
|
||||
}
|
||||
fields, err := h.HTTPGather()
|
||||
require.NoError(t, err)
|
||||
assert.NotEmpty(t, fields)
|
||||
if assert.NotNil(t, fields["http_response_code"]) {
|
||||
assert.Equal(t, http.StatusOK, fields["http_response_code"])
|
||||
}
|
||||
assert.NotNil(t, fields["response_time"])
|
||||
}
|
||||
|
||||
func TestFields(t *testing.T) {
|
||||
mux := setUpTestMux()
|
||||
ts := httptest.NewServer(mux)
|
||||
@@ -72,7 +88,7 @@ func TestFields(t *testing.T) {
|
||||
Address: ts.URL + "/good",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "GET",
|
||||
ResponseTimeout: 20,
|
||||
ResponseTimeout: internal.Duration{Duration: time.Second * 20},
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
@@ -85,7 +101,6 @@ func TestFields(t *testing.T) {
|
||||
assert.Equal(t, http.StatusOK, fields["http_response_code"])
|
||||
}
|
||||
assert.NotNil(t, fields["response_time"])
|
||||
|
||||
}
|
||||
|
||||
func TestRedirects(t *testing.T) {
|
||||
@@ -97,7 +112,7 @@ func TestRedirects(t *testing.T) {
|
||||
Address: ts.URL + "/redirect",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "GET",
|
||||
ResponseTimeout: 20,
|
||||
ResponseTimeout: internal.Duration{Duration: time.Second * 20},
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
@@ -114,7 +129,7 @@ func TestRedirects(t *testing.T) {
|
||||
Address: ts.URL + "/badredirect",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "GET",
|
||||
ResponseTimeout: 20,
|
||||
ResponseTimeout: internal.Duration{Duration: time.Second * 20},
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
@@ -133,7 +148,7 @@ func TestMethod(t *testing.T) {
|
||||
Address: ts.URL + "/mustbepostmethod",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "POST",
|
||||
ResponseTimeout: 20,
|
||||
ResponseTimeout: internal.Duration{Duration: time.Second * 20},
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
@@ -150,7 +165,7 @@ func TestMethod(t *testing.T) {
|
||||
Address: ts.URL + "/mustbepostmethod",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "GET",
|
||||
ResponseTimeout: 20,
|
||||
ResponseTimeout: internal.Duration{Duration: time.Second * 20},
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
@@ -168,7 +183,7 @@ func TestMethod(t *testing.T) {
|
||||
Address: ts.URL + "/mustbepostmethod",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "head",
|
||||
ResponseTimeout: 20,
|
||||
ResponseTimeout: internal.Duration{Duration: time.Second * 20},
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
@@ -191,7 +206,7 @@ func TestBody(t *testing.T) {
|
||||
Address: ts.URL + "/musthaveabody",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "GET",
|
||||
ResponseTimeout: 20,
|
||||
ResponseTimeout: internal.Duration{Duration: time.Second * 20},
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
@@ -207,7 +222,7 @@ func TestBody(t *testing.T) {
|
||||
h = &HTTPResponse{
|
||||
Address: ts.URL + "/musthaveabody",
|
||||
Method: "GET",
|
||||
ResponseTimeout: 20,
|
||||
ResponseTimeout: internal.Duration{Duration: time.Second * 20},
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
@@ -230,7 +245,7 @@ func TestTimeout(t *testing.T) {
|
||||
Address: ts.URL + "/twosecondnap",
|
||||
Body: "{ 'test': 'data'}",
|
||||
Method: "GET",
|
||||
ResponseTimeout: 1,
|
||||
ResponseTimeout: internal.Duration{Duration: time.Second * 1},
|
||||
Headers: map[string]string{
|
||||
"Content-Type": "application/json",
|
||||
},
|
||||
|
||||
@@ -15,6 +15,7 @@ InfluxDB-formatted endpoints. See below for more information.
|
||||
## See the influxdb plugin's README for more details.
|
||||
|
||||
## Multiple URLs from which to read InfluxDB-formatted JSON
|
||||
## Default is "http://localhost:8086/debug/vars".
|
||||
urls = [
|
||||
"http://localhost:8086/debug/vars"
|
||||
]
|
||||
@@ -22,16 +23,78 @@ InfluxDB-formatted endpoints. See below for more information.
|
||||
|
||||
### Measurements & Fields
|
||||
|
||||
- influxdb
|
||||
- n_shards
|
||||
- influxdb_database
|
||||
- influxdb_httpd
|
||||
- influxdb_measurement
|
||||
- influxdb_memstats
|
||||
- heap_inuse
|
||||
- heap_released
|
||||
- mspan_inuse
|
||||
- total_alloc
|
||||
- sys
|
||||
- mallocs
|
||||
- frees
|
||||
- heap_idle
|
||||
- pause_total_ns
|
||||
- lookups
|
||||
- heap_sys
|
||||
- mcache_sys
|
||||
- next_gc
|
||||
- gcc_pu_fraction
|
||||
- other_sys
|
||||
- alloc
|
||||
- stack_inuse
|
||||
- stack_sys
|
||||
- buck_hash_sys
|
||||
- gc_sys
|
||||
- num_gc
|
||||
- heap_alloc
|
||||
- heap_objects
|
||||
- mspan_sys
|
||||
- mcache_inuse
|
||||
- last_gc
|
||||
- influxdb_shard
|
||||
- influxdb_subscriber
|
||||
- influxdb_tsm1_cache
|
||||
- influxdb_tsm1_wal
|
||||
- influxdb_write
|
||||
|
||||
### Example Output:
|
||||
|
||||
```
|
||||
telegraf -config ~/ws/telegraf.conf -input-filter influxdb -test
|
||||
* Plugin: influxdb, Collection 1
|
||||
> influxdb_database,database=_internal,host=tyrion,url=http://localhost:8086/debug/vars numMeasurements=10,numSeries=29 1463590500247354636
|
||||
> influxdb_httpd,bind=:8086,host=tyrion,url=http://localhost:8086/debug/vars req=7,reqActive=1,reqDurationNs=14227734 1463590500247354636
|
||||
> influxdb_measurement,database=_internal,host=tyrion,measurement=database,url=http://localhost:8086/debug/vars numSeries=1 1463590500247354636
|
||||
> influxdb_measurement,database=_internal,host=tyrion,measurement=httpd,url=http://localhost:8086/debug/vars numSeries=1 1463590500247354636
|
||||
> influxdb_measurement,database=_internal,host=tyrion,measurement=measurement,url=http://localhost:8086/debug/vars numSeries=10 1463590500247354636
|
||||
> influxdb_measurement,database=_internal,host=tyrion,measurement=runtime,url=http://localhost:8086/debug/vars numSeries=1 1463590500247354636
|
||||
> influxdb_measurement,database=_internal,host=tyrion,measurement=shard,url=http://localhost:8086/debug/vars numSeries=4 1463590500247354636
|
||||
> influxdb_measurement,database=_internal,host=tyrion,measurement=subscriber,url=http://localhost:8086/debug/vars numSeries=1 1463590500247354636
|
||||
> influxdb_measurement,database=_internal,host=tyrion,measurement=tsm1_cache,url=http://localhost:8086/debug/vars numSeries=4 1463590500247354636
|
||||
> influxdb_measurement,database=_internal,host=tyrion,measurement=tsm1_filestore,url=http://localhost:8086/debug/vars numSeries=2 1463590500247354636
|
||||
> influxdb_measurement,database=_internal,host=tyrion,measurement=tsm1_wal,url=http://localhost:8086/debug/vars numSeries=4 1463590500247354636
|
||||
> influxdb_measurement,database=_internal,host=tyrion,measurement=write,url=http://localhost:8086/debug/vars numSeries=1 1463590500247354636
|
||||
> influxdb_memstats,host=tyrion,url=http://localhost:8086/debug/vars alloc=7642384i,buck_hash_sys=1463471i,frees=1169558i,gc_sys=653312i,gcc_pu_fraction=0.00003825652361068311,heap_alloc=7642384i,heap_idle=9912320i,heap_inuse=9125888i,heap_objects=48276i,heap_released=0i,heap_sys=19038208i,last_gc=1463590480877651621i,lookups=90i,mallocs=1217834i,mcache_inuse=4800i,mcache_sys=16384i,mspan_inuse=70920i,mspan_sys=81920i,next_gc=11679787i,num_gc=141i,other_sys=1244233i,pause_total_ns=24034027i,stack_inuse=884736i,stack_sys=884736i,sys=23382264i,total_alloc=679012200i 1463590500277918755
|
||||
> influxdb_shard,database=_internal,engine=tsm1,host=tyrion,id=4,path=/Users/sparrc/.influxdb/data/_internal/monitor/4,retentionPolicy=monitor,url=http://localhost:8086/debug/vars fieldsCreate=65,seriesCreate=26,writePointsOk=7274,writeReq=280 1463590500247354636
|
||||
> influxdb_subscriber,host=tyrion,url=http://localhost:8086/debug/vars pointsWritten=7274 1463590500247354636
|
||||
> influxdb_tsm1_cache,database=_internal,host=tyrion,path=/Users/sparrc/.influxdb/data/_internal/monitor/1,retentionPolicy=monitor,url=http://localhost:8086/debug/vars WALCompactionTimeMs=0,cacheAgeMs=2809192,cachedBytes=0,diskBytes=0,memBytes=0,snapshotCount=0 1463590500247354636
|
||||
> influxdb_tsm1_cache,database=_internal,host=tyrion,path=/Users/sparrc/.influxdb/data/_internal/monitor/2,retentionPolicy=monitor,url=http://localhost:8086/debug/vars WALCompactionTimeMs=0,cacheAgeMs=2809184,cachedBytes=0,diskBytes=0,memBytes=0,snapshotCount=0 1463590500247354636
|
||||
> influxdb_tsm1_cache,database=_internal,host=tyrion,path=/Users/sparrc/.influxdb/data/_internal/monitor/3,retentionPolicy=monitor,url=http://localhost:8086/debug/vars WALCompactionTimeMs=0,cacheAgeMs=2809180,cachedBytes=0,diskBytes=0,memBytes=42368,snapshotCount=0 1463590500247354636
|
||||
> influxdb_tsm1_cache,database=_internal,host=tyrion,path=/Users/sparrc/.influxdb/data/_internal/monitor/4,retentionPolicy=monitor,url=http://localhost:8086/debug/vars WALCompactionTimeMs=0,cacheAgeMs=2799155,cachedBytes=0,diskBytes=0,memBytes=331216,snapshotCount=0 1463590500247354636
|
||||
> influxdb_tsm1_filestore,database=_internal,host=tyrion,path=/Users/sparrc/.influxdb/data/_internal/monitor/1,retentionPolicy=monitor,url=http://localhost:8086/debug/vars diskBytes=37892 1463590500247354636
|
||||
> influxdb_tsm1_filestore,database=_internal,host=tyrion,path=/Users/sparrc/.influxdb/data/_internal/monitor/2,retentionPolicy=monitor,url=http://localhost:8086/debug/vars diskBytes=52907 1463590500247354636
|
||||
> influxdb_tsm1_wal,database=_internal,host=tyrion,path=/Users/sparrc/.influxdb/wal/_internal/monitor/1,retentionPolicy=monitor,url=http://localhost:8086/debug/vars currentSegmentDiskBytes=0,oldSegmentsDiskBytes=0 1463590500247354636
|
||||
> influxdb_tsm1_wal,database=_internal,host=tyrion,path=/Users/sparrc/.influxdb/wal/_internal/monitor/2,retentionPolicy=monitor,url=http://localhost:8086/debug/vars currentSegmentDiskBytes=0,oldSegmentsDiskBytes=0 1463590500247354636
|
||||
> influxdb_tsm1_wal,database=_internal,host=tyrion,path=/Users/sparrc/.influxdb/wal/_internal/monitor/3,retentionPolicy=monitor,url=http://localhost:8086/debug/vars currentSegmentDiskBytes=0,oldSegmentsDiskBytes=65651 1463590500247354636
|
||||
> influxdb_tsm1_wal,database=_internal,host=tyrion,path=/Users/sparrc/.influxdb/wal/_internal/monitor/4,retentionPolicy=monitor,url=http://localhost:8086/debug/vars currentSegmentDiskBytes=495687,oldSegmentsDiskBytes=0 1463590500247354636
|
||||
> influxdb_write,host=tyrion,url=http://localhost:8086/debug/vars pointReq=7274,pointReqLocal=7274,req=280,subWriteOk=280,writeOk=280 1463590500247354636
|
||||
> influxdb_shard,host=tyrion n_shards=4i 1463590500247354636
|
||||
```
|
||||
|
||||
### InfluxDB-formatted endpoints
|
||||
|
||||
The influxdb plugin can collect InfluxDB-formatted data from JSON endpoints.
|
||||
@@ -46,65 +109,3 @@ With a configuration of:
|
||||
"http://192.168.2.1:8086/debug/vars"
|
||||
]
|
||||
```
|
||||
|
||||
And if 127.0.0.1 responds with this JSON:
|
||||
|
||||
```json
|
||||
{
|
||||
"k1": {
|
||||
"name": "fruit",
|
||||
"tags": {
|
||||
"kind": "apple"
|
||||
},
|
||||
"values": {
|
||||
"inventory": 371,
|
||||
"sold": 112
|
||||
}
|
||||
},
|
||||
"k2": {
|
||||
"name": "fruit",
|
||||
"tags": {
|
||||
"kind": "banana"
|
||||
},
|
||||
"values": {
|
||||
"inventory": 1000,
|
||||
"sold": 403
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
And if 192.168.2.1 responds like so:
|
||||
|
||||
```json
|
||||
{
|
||||
"k3": {
|
||||
"name": "transactions",
|
||||
"tags": {},
|
||||
"values": {
|
||||
"total": 100,
|
||||
"balance": 184.75
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Then the collected metrics will be:
|
||||
|
||||
```
|
||||
influxdb_fruit,url='http://127.0.0.1:8086/debug/vars',kind='apple' inventory=371.0,sold=112.0
|
||||
influxdb_fruit,url='http://127.0.0.1:8086/debug/vars',kind='banana' inventory=1000.0,sold=403.0
|
||||
|
||||
influxdb_transactions,url='http://192.168.2.1:8086/debug/vars' total=100.0,balance=184.75
|
||||
```
|
||||
|
||||
There are two important details to note about the collected metrics:
|
||||
|
||||
1. Even though the values in JSON are being displayed as integers,
|
||||
the metrics are reported as floats.
|
||||
JSON encoders usually don't print the fractional part for round floats.
|
||||
Because you cannot change the type of an existing field in InfluxDB,
|
||||
we assume all numbers are floats.
|
||||
|
||||
2. The top-level keys' names (in the example above, `"k1"`, `"k2"`, and `"k3"`)
|
||||
are not considered when recording the metrics.
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user