dom
/
telegraf
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Documentation improvements 

- fully document aggregator and processor plugins
- improve readme.md

closes #1989
This commit is contained in:
Cameron Sparr 2016-11-04 14:12:22 +00:00
parent 5d3850c44e
commit 1c7715780e
4 changed files with 233 additions and 111 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

229
README.md
View File

@ -1,15 +1,23 @@
# Telegraf [![Circle CI](https://circleci.com/gh/influxdata/telegraf.svg?style=svg)](https://circleci.com/gh/influxdata/telegraf) [![Docker pulls](https://img.shields.io/docker/pulls/library/telegraf.svg)](https://hub.docker.com/_/telegraf/) # Telegraf [![Circle CI](https://circleci.com/gh/influxdata/telegraf.svg?style=svg)](https://circleci.com/gh/influxdata/telegraf) [![Docker pulls](https://img.shields.io/docker/pulls/library/telegraf.svg)](https://hub.docker.com/_/telegraf/)
Telegraf is an agent written in Go for collecting metrics from the system it's Telegraf is an agent written in Go for collecting, processing, aggregating,
running on, or from other services, and writing them into InfluxDB or other and writing metrics.
[outputs](https://github.com/influxdata/telegraf#supported-output-plugins).
Design goals are to have a minimal memory footprint with a plugin system so Design goals are to have a minimal memory footprint with a plugin system so
that developers in the community can easily add support for collecting metrics that developers in the community can easily add support for collecting metrics
from well known services (like Hadoop, Postgres, or Redis) and third party from well known services (like Hadoop, Postgres, or Redis) and third party
APIs (like Mailchimp, AWS CloudWatch, or Google Analytics). APIs (like Mailchimp, AWS CloudWatch, or Google Analytics).
New input and output plugins are designed to be easy to contribute, Telegraf is plugin-driven and has the concept of 4 distinct plugins:
1. [Input Plugins](#input-plugins) collect metrics from the system, services, or 3rd party APIs
2. [Processor Plugins](#processor-plugins) transform, decorate, and/or filter metrics
3. [Aggregator Plugins](#aggregator-plugins) create aggregate metrics (e.g. mean, min, max, quantiles, etc.)
4. [Output Plugins](#output-plugins) write metrics to various destinations
For more information on Processor and Aggregator plugins please [read this](./docs/AGGREGATORS_AND_PROCESSORS.md).
New plugins are designed to be easy to contribute,
we'll eagerly accept pull we'll eagerly accept pull
requests and will manage the set of plugins that Telegraf supports. requests and will manage the set of plugins that Telegraf supports.
See the [contributing guide](CONTRIBUTING.md) for instructions on writing See the [contributing guide](CONTRIBUTING.md) for instructions on writing
@ -39,7 +47,7 @@ controlled via `systemctl [action] telegraf`
### yum/apt Repositories: ### yum/apt Repositories:
There is a yum/apt repo available for the whole InfluxData stack, see There is a yum/apt repo available for the whole InfluxData stack, see
[here](https://docs.influxdata.com/influxdb/v0.10/introduction/installation/#installation) [here](https://docs.influxdata.com/influxdb/latest/introduction/installation/#installation)
for instructions on setting up the repo. Once it is configured, you will be able for instructions on setting up the repo. Once it is configured, you will be able
to use this repo to install & update telegraf. to use this repo to install & update telegraf.
@ -127,77 +135,71 @@ telegraf --config telegraf.conf -input-filter cpu:mem -output-filter influxdb
See the [configuration guide](docs/CONFIGURATION.md) for a rundown of the more advanced See the [configuration guide](docs/CONFIGURATION.md) for a rundown of the more advanced
configuration options. configuration options.
## Supported Input Plugins ## Input Plugins
Telegraf currently has support for collecting metrics from many sources. For * [aws cloudwatch](./plugins/inputs/cloudwatch)
more information on each, please look at the directory of the same name in * [aerospike](./plugins/inputs/aerospike)
`plugins/inputs`. * [apache](./plugins/inputs/apache)
* [bcache](./plugins/inputs/bcache)
Currently implemented sources: * [cassandra](./plugins/inputs/cassandra)
* [ceph](./plugins/inputs/ceph)
* [aws cloudwatch](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/cloudwatch) * [chrony](./plugins/inputs/chrony)
* [aerospike](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/aerospike) * [consul](./plugins/inputs/consul)
* [apache](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/apache) * [conntrack](./plugins/inputs/conntrack)
* [bcache](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/bcache) * [couchbase](./plugins/inputs/couchbase)
* [cassandra](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/cassandra) * [couchdb](./plugins/inputs/couchdb)
* [ceph](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/ceph) * [disque](./plugins/inputs/disque)
* [chrony](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/chrony) * [dns query time](./plugins/inputs/dns_query)
* [consul](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/consul) * [docker](./plugins/inputs/docker)
* [conntrack](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/conntrack) * [dovecot](./plugins/inputs/dovecot)
* [couchbase](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/couchbase) * [elasticsearch](./plugins/inputs/elasticsearch)
* [couchdb](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/couchdb) * [exec](./plugins/inputs/exec) (generic executable plugin, support JSON, influx, graphite and nagios)
* [disque](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/disque) * [filestat](./plugins/inputs/filestat)
* [dns query time](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/dns_query) * [haproxy](./plugins/inputs/haproxy)
* [docker](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/docker) * [hddtemp](./plugins/inputs/hddtemp)
* [dovecot](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/dovecot) * [http_response](./plugins/inputs/http_response)
* [elasticsearch](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/elasticsearch) * [httpjson](./plugins/inputs/httpjson) (generic JSON-emitting http service plugin)
* [exec](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/exec) (generic executable plugin, support JSON, influx, graphite and nagios) * [influxdb](./plugins/inputs/influxdb)
* [filestat](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/filestat) * [ipmi_sensor](./plugins/inputs/ipmi_sensor)
* [haproxy](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/haproxy) * [iptables](./plugins/inputs/iptables)
* [hddtemp](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/hddtemp) * [jolokia](./plugins/inputs/jolokia)
* [http_response](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/http_response) * [leofs](./plugins/inputs/leofs)
* [httpjson](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/httpjson) (generic JSON-emitting http service plugin) * [lustre2](./plugins/inputs/lustre2)
* [influxdb](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/influxdb) * [mailchimp](./plugins/inputs/mailchimp)
* [ipmi_sensor](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/ipmi_sensor) * [memcached](./plugins/inputs/memcached)
* [iptables](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/iptables) * [mesos](./plugins/inputs/mesos)
* [jolokia](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/jolokia) * [mongodb](./plugins/inputs/mongodb)
* [leofs](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/leofs) * [mysql](./plugins/inputs/mysql)
* [lustre2](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/lustre2) * [net_response](./plugins/inputs/net_response)
* [mailchimp](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/mailchimp) * [nginx](./plugins/inputs/nginx)
* [memcached](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/memcached) * [nsq](./plugins/inputs/nsq)
* [mesos](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/mesos) * [nstat](./plugins/inputs/nstat)
* [mongodb](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/mongodb) * [ntpq](./plugins/inputs/ntpq)
* [mysql](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/mysql) * [phpfpm](./plugins/inputs/phpfpm)
* [net_response](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/net_response) * [phusion passenger](./plugins/inputs/passenger)
* [nginx](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nginx) * [ping](./plugins/inputs/ping)
* [nsq](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nsq) * [postgresql](./plugins/inputs/postgresql)
* [nstat](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nstat) * [postgresql_extensible](./plugins/inputs/postgresql_extensible)
* [ntpq](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/ntpq) * [powerdns](./plugins/inputs/powerdns)
* [phpfpm](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/phpfpm) * [procstat](./plugins/inputs/procstat)
* [phusion passenger](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/passenger) * [prometheus](./plugins/inputs/prometheus)
* [ping](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/ping) * [puppetagent](./plugins/inputs/puppetagent)
* [postgresql](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/postgresql) * [rabbitmq](./plugins/inputs/rabbitmq)
* [postgresql_extensible](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/postgresql_extensible) * [raindrops](./plugins/inputs/raindrops)
* [powerdns](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/powerdns) * [redis](./plugins/inputs/redis)
* [procstat](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/procstat) * [rethinkdb](./plugins/inputs/rethinkdb)
* [prometheus](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/prometheus) * [riak](./plugins/inputs/riak)
* [puppetagent](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/puppetagent) * [sensors](./plugins/inputs/sensors)
* [rabbitmq](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/rabbitmq) * [snmp](./plugins/inputs/snmp)
* [raindrops](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/raindrops) * [snmp_legacy](./plugins/inputs/snmp_legacy)
* [redis](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/redis) * [sql server](./plugins/inputs/sqlserver) (microsoft)
* [rethinkdb](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/rethinkdb) * [twemproxy](./plugins/inputs/twemproxy)
* [riak](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/riak) * [varnish](./plugins/inputs/varnish)
* [sensors](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/sensors) * [zfs](./plugins/inputs/zfs)
* [snmp](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/snmp) * [zookeeper](./plugins/inputs/zookeeper)
* [snmp_legacy](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/snmp_legacy) * [win_perf_counters ](./plugins/inputs/win_perf_counters) (windows performance counters)
* [sql server](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/sqlserver) (microsoft) * [sysstat](./plugins/inputs/sysstat)
* [twemproxy](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/twemproxy) * [system](./plugins/inputs/system)
* [varnish](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/varnish)
* [zfs](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/zfs)
* [zookeeper](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/zookeeper)
* [win_perf_counters ](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/win_perf_counters) (windows performance counters)
* [sysstat](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/sysstat)
* [system](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/system)
* cpu * cpu
* mem * mem
* net * net
@ -211,45 +213,50 @@ Currently implemented sources:
Telegraf can also collect metrics via the following service plugins: Telegraf can also collect metrics via the following service plugins:
* [http_listener](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/http_listener) * [http_listener](./plugins/inputs/http_listener)
* [kafka_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/kafka_consumer) * [kafka_consumer](./plugins/inputs/kafka_consumer)
* [mqtt_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/mqtt_consumer) * [mqtt_consumer](./plugins/inputs/mqtt_consumer)
* [nats_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nats_consumer) * [nats_consumer](./plugins/inputs/nats_consumer)
* [nsq_consumer](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/nsq_consumer) * [nsq_consumer](./plugins/inputs/nsq_consumer)
* [logparser](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/logparser) * [logparser](./plugins/inputs/logparser)
* [statsd](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/statsd) * [statsd](./plugins/inputs/statsd)
* [tail](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/tail) * [tail](./plugins/inputs/tail)
* [tcp_listener](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/tcp_listener) * [tcp_listener](./plugins/inputs/tcp_listener)
* [udp_listener](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/udp_listener) * [udp_listener](./plugins/inputs/udp_listener)
* [webhooks](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/webhooks) * [webhooks](./plugins/inputs/webhooks)
* [filestack](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/webhooks/filestack) * [filestack](./plugins/inputs/webhooks/filestack)
* [github](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/webhooks/github) * [github](./plugins/inputs/webhooks/github)
* [mandrill](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/webhooks/mandrill) * [mandrill](./plugins/inputs/webhooks/mandrill)
* [rollbar](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/webhooks/rollbar) * [rollbar](./plugins/inputs/webhooks/rollbar)
We'll be adding support for many more over the coming months. Read on if you ## Processor Plugins
want to add support for another service or third-party API.
## Supported Output Plugins * [printer](./plugins/processors/printer)
* [influxdb](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/influxdb) ## Aggregator Plugins
* [amon](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/amon)
* [amqp](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/amqp) * [minmax](./plugins/aggregators/minmax)
* [aws kinesis](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/kinesis)
* [aws cloudwatch](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/cloudwatch) ## Output Plugins
* [datadog](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/datadog)
* [file](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/file) * [influxdb](./plugins/outputs/influxdb)
* [graphite](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/graphite) * [amon](./plugins/outputs/amon)
* [graylog](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/graylog) * [amqp](./plugins/outputs/amqp)
* [instrumental](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/instrumental) * [aws kinesis](./plugins/outputs/kinesis)
* [kafka](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/kafka) * [aws cloudwatch](./plugins/outputs/cloudwatch)
* [librato](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/librato) * [datadog](./plugins/outputs/datadog)
* [mqtt](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/mqtt) * [file](./plugins/outputs/file)
* [nats](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/nats) * [graphite](./plugins/outputs/graphite)
* [nsq](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/nsq) * [graylog](./plugins/outputs/graylog)
* [opentsdb](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/opentsdb) * [instrumental](./plugins/outputs/instrumental)
* [prometheus](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/prometheus_client) * [kafka](./plugins/outputs/kafka)
* [riemann](https://github.com/influxdata/telegraf/tree/master/plugins/outputs/riemann) * [librato](./plugins/outputs/librato)
* [mqtt](./plugins/outputs/mqtt)
* [nats](./plugins/outputs/nats)
* [nsq](./plugins/outputs/nsq)
* [opentsdb](./plugins/outputs/opentsdb)
* [prometheus](./plugins/outputs/prometheus_client)
* [riemann](./plugins/outputs/riemann)
## Contributing ## Contributing

59
docs/AGGREGATORS_AND_PROCESSORS.md Normal file
View File

@ -0,0 +1,59 @@
# Telegraf Aggregator & Processor Plugins
As of release 1.1.0, Telegraf has the concept of Aggregator and Processor Plugins.
These plugins sit in-between Input & Output plugins, aggregating and processing
metrics as they pass through Telegraf:
```
┌───────────┐
│ │
│ CPU │───┐
│ │ │
└───────────┘ │
│
┌───────────┐ │ ┌───────────┐
│ │ │ │ │
│ Memory │───┤ ┌──▶│ InfluxDB │
│ │ │ │ │ │
└───────────┘ │ ┌─────────────┐ ┌─────────────┐ │ └───────────┘
│ │ │ │Aggregate │ │
┌───────────┐ │ │Process │ │ - mean │ │ ┌───────────┐
│ │ │ │ - transform │ │ - quantiles │ │ │ │
│ MySQL │───┼───▶│ - decorate │────▶│ - min/max │───┼──▶│ File │
│ │ │ │ - filter │ │ - count │ │ │ │
└───────────┘ │ │ │ │ │ │ └───────────┘
│ └─────────────┘ └─────────────┘ │
┌───────────┐ │ │ ┌───────────┐
│ │ │ │ │ │
│ SNMP │───┤ └──▶│ Kafka │
│ │ │ │ │
└───────────┘ │ └───────────┘
│
┌───────────┐ │
│ │ │
│ Docker │───┘
│ │
└───────────┘
```
Both Aggregators and Processors analyze metrics as they pass through Telegraf.
**Processor** plugins process metrics as they pass through and immediately emit
results based on the values they process. For example, this could be printing
all metrics or adding a tag to all metrics that pass through.
**Aggregator** plugins, on the other hand, are a bit more complicated. Aggregators
are typically for emitting new _aggregate_ metrics, such as a running mean,
minimum, maximum, quantiles, or standard deviation. For this reason, all _aggregator_
plugins are configured with a `period`. The `period` is the size of the window
of metrics that each _aggregate_ represents. In other words, the emitted
_aggregate_ metric will be the aggregated value of the past `period` seconds.
Since many users will only care about their aggregates and not every single metric
gathered, there is also a `drop_original` argument, which tells Telegraf to only
emit the aggregates and not the original metrics.
**NOTE** That since aggregators only aggregate metrics within their period, that
historical data is not supported. In other words, if your metric timestamp is more
than `now() - period` in the past, it will not be aggregated. If this is a feature
that you need, please comment on this [github issue](https://github.com/influxdata/telegraf/issues/1992)

42
plugins/aggregators/minmax/README.md Normal file
View File

@ -0,0 +1,42 @@
# MinMax Aggregator Plugin
The minmax aggregator plugin aggregates min & max values of each field it sees,
emitting the aggrate every `period` seconds.
### Configuration:
```toml
# Keep the aggregate min/max of each metric passing through.
[[aggregators.minmax]]
## General Aggregator Arguments:
## The period on which to flush & clear the aggregator.
period = "30s"
## If true, the original metric will be dropped by the
## aggregator and will not get sent to the output plugins.
drop_original = false
```
### Measurements & Fields:
- measurement1
- field1_max
- field1_min
### Tags:
No tags are applied by this aggregator.
### Example Output:
```
$ telegraf --config telegraf.conf --quiet
system,host=tars load1=1.72 1475583980000000000
system,host=tars load1=1.6 1475583990000000000
system,host=tars load1=1.66 1475584000000000000
system,host=tars load1=1.63 1475584010000000000
system,host=tars load1_max=1.72,load1_min=1.6 1475584010000000000
system,host=tars load1=1.46 1475584020000000000
system,host=tars load1=1.39 1475584030000000000
system,host=tars load1=1.41 1475584040000000000
system,host=tars load1_max=1.46,load1_min=1.39 1475584040000000000
```

14
plugins/processors/printer/README.md Normal file
View File

@ -0,0 +1,14 @@
# Printer Processor Plugin
The printer processor plugin simple prints every metric passing through it.
### Configuration:
```toml
# Print all metrics that pass through this filter.
[[processors.printer]]
```
### Tags:
No tags are applied by this processor.