2016-02-12 22:09:34 +00:00
|
|
|
# Telegraf Output Data Formats
|
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
Telegraf is able to serialize metrics into the following output data formats:
|
|
|
|
|
|
|
|
1. [InfluxDB Line Protocol](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md#influx)
|
|
|
|
1. [JSON](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md#json)
|
|
|
|
1. [Graphite](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md#graphite)
|
|
|
|
|
2016-02-12 22:09:34 +00:00
|
|
|
Telegraf metrics, like InfluxDB
|
|
|
|
[points](https://docs.influxdata.com/influxdb/v0.10/write_protocols/line/),
|
|
|
|
are a combination of four basic parts:
|
|
|
|
|
|
|
|
1. Measurement Name
|
|
|
|
1. Tags
|
|
|
|
1. Fields
|
|
|
|
1. Timestamp
|
|
|
|
|
|
|
|
In InfluxDB line protocol, these 4 parts are easily defined in textual form:
|
2016-02-13 18:50:43 +00:00
|
|
|
|
|
|
|
```
|
|
|
|
measurement_name[,tag1=val1,...] field1=val1[,field2=val2,...] [timestamp]
|
|
|
|
```
|
2016-02-12 22:09:34 +00:00
|
|
|
|
|
|
|
For Telegraf outputs that write textual data (such as `kafka`, `mqtt`, and `file`),
|
|
|
|
InfluxDB line protocol was originally the only available output format. But now
|
2016-02-13 18:50:43 +00:00
|
|
|
we are normalizing telegraf metric "serializers" into a
|
|
|
|
[plugin-like interface](https://github.com/influxdata/telegraf/tree/master/plugins/serializers)
|
|
|
|
across all output plugins that can support it.
|
|
|
|
You will be able to identify a plugin that supports different data formats
|
|
|
|
by the presence of a `data_format`
|
|
|
|
config option, for example, in the `file` output plugin:
|
2016-02-12 22:09:34 +00:00
|
|
|
|
|
|
|
```toml
|
|
|
|
[[outputs.file]]
|
2016-02-18 21:26:51 +00:00
|
|
|
## Files to write to, "stdout" is a specially handled file.
|
2016-02-13 18:50:43 +00:00
|
|
|
files = ["stdout"]
|
2016-02-12 22:09:34 +00:00
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
## Data format to output.
|
2017-04-27 21:59:18 +00:00
|
|
|
## Each data format has its own unique set of configuration options, read
|
2016-02-18 21:26:51 +00:00
|
|
|
## more about them here:
|
|
|
|
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
|
2016-02-12 22:09:34 +00:00
|
|
|
data_format = "influx"
|
|
|
|
|
2016-02-18 21:26:51 +00:00
|
|
|
## Additional configuration options go here
|
2016-02-12 22:09:34 +00:00
|
|
|
```
|
|
|
|
|
|
|
|
Each data_format has an additional set of configuration options available, which
|
|
|
|
I'll go over below.
|
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
# Influx:
|
2016-02-12 22:09:34 +00:00
|
|
|
|
|
|
|
There are no additional configuration options for InfluxDB line-protocol. The
|
2016-02-12 23:52:33 +00:00
|
|
|
metrics are serialized directly into InfluxDB line-protocol.
|
2016-02-12 22:09:34 +00:00
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
### Influx Configuration:
|
2016-02-12 22:09:34 +00:00
|
|
|
|
|
|
|
```toml
|
|
|
|
[[outputs.file]]
|
2016-02-18 21:26:51 +00:00
|
|
|
## Files to write to, "stdout" is a specially handled file.
|
2016-02-12 22:09:34 +00:00
|
|
|
files = ["stdout", "/tmp/metrics.out"]
|
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
## Data format to output.
|
2017-04-27 21:59:18 +00:00
|
|
|
## Each data format has its own unique set of configuration options, read
|
2016-02-18 21:26:51 +00:00
|
|
|
## more about them here:
|
|
|
|
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
|
2016-02-12 22:09:34 +00:00
|
|
|
data_format = "influx"
|
|
|
|
```
|
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
# Graphite:
|
2016-02-12 22:09:34 +00:00
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
The Graphite data format translates Telegraf metrics into _dot_ buckets. A
|
|
|
|
template can be specified for the output of Telegraf metrics into Graphite
|
|
|
|
buckets. The default template is:
|
2016-02-12 22:09:34 +00:00
|
|
|
|
|
|
|
```
|
2016-04-08 22:04:45 +00:00
|
|
|
template = "host.tags.measurement.field"
|
2016-02-12 22:09:34 +00:00
|
|
|
```
|
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
In the above template, we have four parts:
|
|
|
|
|
|
|
|
1. _host_ is a tag key. This can be any tag key that is in the Telegraf
|
|
|
|
metric(s). If the key doesn't exist, it will be ignored. If it does exist, the
|
|
|
|
tag value will be filled in.
|
|
|
|
1. _tags_ is a special keyword that outputs all remaining tag values, separated
|
|
|
|
by dots and in alphabetical order (by tag key). These will be filled after all
|
|
|
|
tag keys are filled.
|
|
|
|
1. _measurement_ is a special keyword that outputs the measurement name.
|
|
|
|
1. _field_ is a special keyword that outputs the field name.
|
|
|
|
|
2016-02-12 22:09:34 +00:00
|
|
|
Which means the following influx metric -> graphite conversion would happen:
|
|
|
|
|
|
|
|
```
|
2016-02-12 23:52:33 +00:00
|
|
|
cpu,cpu=cpu-total,dc=us-east-1,host=tars usage_idle=98.09,usage_user=0.89 1455320660004257758
|
2016-02-12 22:09:34 +00:00
|
|
|
=>
|
2016-02-12 23:52:33 +00:00
|
|
|
tars.cpu-total.us-east-1.cpu.usage_user 0.89 1455320690
|
|
|
|
tars.cpu-total.us-east-1.cpu.usage_idle 98.09 1455320690
|
2016-02-12 22:09:34 +00:00
|
|
|
```
|
|
|
|
|
2017-08-29 22:59:38 +00:00
|
|
|
Fields with non-numeric values will be skipped.
|
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
### Graphite Configuration:
|
2016-02-12 22:09:34 +00:00
|
|
|
|
|
|
|
```toml
|
|
|
|
[[outputs.file]]
|
2016-02-18 21:26:51 +00:00
|
|
|
## Files to write to, "stdout" is a specially handled file.
|
2016-02-12 22:09:34 +00:00
|
|
|
files = ["stdout", "/tmp/metrics.out"]
|
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
## Data format to output.
|
2017-04-27 21:59:18 +00:00
|
|
|
## Each data format has its own unique set of configuration options, read
|
2016-02-18 21:26:51 +00:00
|
|
|
## more about them here:
|
|
|
|
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
|
2016-04-08 22:04:45 +00:00
|
|
|
data_format = "graphite"
|
2016-02-12 22:09:34 +00:00
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
# prefix each graphite bucket
|
2016-02-12 22:09:34 +00:00
|
|
|
prefix = "telegraf"
|
2016-04-08 22:04:45 +00:00
|
|
|
# graphite template
|
|
|
|
template = "host.tags.measurement.field"
|
2016-02-12 22:09:34 +00:00
|
|
|
```
|
2016-03-17 17:50:39 +00:00
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
# JSON:
|
2016-03-17 17:50:39 +00:00
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
The JSON data format serialized Telegraf metrics in json format. The format is:
|
2016-03-17 17:50:39 +00:00
|
|
|
|
|
|
|
```json
|
|
|
|
{
|
|
|
|
"fields":{
|
|
|
|
"field_1":30,
|
|
|
|
"field_2":4,
|
|
|
|
"field_N":59,
|
|
|
|
"n_images":660
|
|
|
|
},
|
|
|
|
"name":"docker",
|
|
|
|
"tags":{
|
|
|
|
"host":"raynor"
|
|
|
|
},
|
|
|
|
"timestamp":1458229140
|
|
|
|
}
|
|
|
|
```
|
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
### JSON Configuration:
|
2016-03-17 17:50:39 +00:00
|
|
|
|
|
|
|
```toml
|
|
|
|
[[outputs.file]]
|
|
|
|
## Files to write to, "stdout" is a specially handled file.
|
|
|
|
files = ["stdout", "/tmp/metrics.out"]
|
|
|
|
|
2016-04-08 22:04:45 +00:00
|
|
|
## Data format to output.
|
2017-04-27 21:59:18 +00:00
|
|
|
## Each data format has its own unique set of configuration options, read
|
2016-03-17 17:50:39 +00:00
|
|
|
## more about them here:
|
|
|
|
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
|
|
|
|
data_format = "json"
|
2017-03-30 00:12:29 +00:00
|
|
|
json_timestamp_units = "1ns"
|
2016-03-17 17:50:39 +00:00
|
|
|
```
|
2017-03-30 00:12:29 +00:00
|
|
|
|
|
|
|
By default, the timestamp that is output in JSON data format serialized Telegraf
|
|
|
|
metrics is in seconds. The precision of this timestamp can be adjusted for any output
|
|
|
|
by adding the optional `json_timestamp_units` parameter to the configuration for
|
|
|
|
that output. This parameter can be used to set the timestamp units to nanoseconds (`ns`),
|
|
|
|
microseconds (`us` or `µs`), milliseconds (`ms`), or seconds (`s`). Note that this
|
|
|
|
parameter will be truncated to the nearest power of 10 that, so if the `json_timestamp_units`
|
|
|
|
are set to `15ms` the timestamps for the JSON format serialized Telegraf metrics will be
|
|
|
|
output in hundredths of a second (`10ms`).
|