telegraf/docs/AGGREGATORS_AND_PROCESSORS.md

Telegraf Aggregator & Processor Plugins

As of release 1.1.0, Telegraf has the concept of Aggregator and Processor Plugins.

These plugins sit in-between Input & Output plugins, aggregating and processing metrics as they pass through Telegraf:

┌───────────┐
│           │
│    CPU    │───┐
│           │   │
└───────────┘   │
                │
┌───────────┐   │                                              ┌───────────┐
│           │   │                                              │           │
│  Memory   │───┤                                          ┌──▶│ InfluxDB  │
│           │   │                                          │   │           │
└───────────┘   │    ┌─────────────┐     ┌─────────────┐   │   └───────────┘
                │    │             │     │Aggregate    │   │
┌───────────┐   │    │Process      │     │ - mean      │   │   ┌───────────┐
│           │   │    │ - transform │     │ - quantiles │   │   │           │
│   MySQL   │───┼───▶│ - decorate  │────▶│ - min/max   │───┼──▶│   File    │
│           │   │    │ - filter    │     │ - count     │   │   │           │
└───────────┘   │    │             │     │             │   │   └───────────┘
                │    └─────────────┘     └─────────────┘   │
┌───────────┐   │                                          │   ┌───────────┐
│           │   │                                          │   │           │
│   SNMP    │───┤                                          └──▶│   Kafka   │
│           │   │                                              │           │
└───────────┘   │                                              └───────────┘
                │
┌───────────┐   │
│           │   │
│  Docker   │───┘
│           │
└───────────┘

Both Aggregators and Processors analyze metrics as they pass through Telegraf.

Use measurement filtering to control which metrics are passed through a processor or aggregator. If a metric is filtered out the metric bypasses the plugin and is passed downstream to the next plugin.

Processor plugins process metrics as they pass through and immediately emit results based on the values they process. For example, this could be printing all metrics or adding a tag to all metrics that pass through.

Aggregator plugins, on the other hand, are a bit more complicated. Aggregators are typically for emitting new aggregate metrics, such as a running mean, minimum, maximum, quantiles, or standard deviation. For this reason, all aggregator plugins are configured with a period. The period is the size of the window of metrics that each aggregate represents. In other words, the emitted aggregate metric will be the aggregated value of the past period seconds. Since many users will only care about their aggregates and not every single metric gathered, there is also a drop_original argument, which tells Telegraf to only emit the aggregates and not the original metrics.

NOTE That since aggregators only aggregate metrics within their period, that historical data is not supported. In other words, if your metric timestamp is more than now() - period in the past, it will not be aggregated. If this is a feature that you need, please comment on this github issue