telegraf/docs/CONFIGURATION.md

# Telegraf Configuration

## Generating a Configuration File

A default Telegraf config file can be generated using the -sample-config flag:
`telegraf -sample-config > telegraf.conf`

To generate a file with specific inputs and outputs, you can use the
-input-filter and -output-filter flags:
`telegraf -sample-config -input-filter cpu:mem:net:swap -output-filter influxdb:kafka`

## Environment Variables

Environment variables can be used anywhere in the config file, simply prepend
them with $. For strings the variable must be within quotes (ie, "$STR_VAR"),
for numbers and booleans they should be plain (ie, $INT_VAR, $BOOL_VAR)

## `[global_tags]` Configuration

Global tags can be specific in the `[global_tags]` section of the config file in
key="value" format. All metrics being gathered on this host will be tagged
with the tags specified here.

## `[agent]` Configuration

Telegraf has a few options you can configure under the `agent` section of the
config.

* **interval**: Default data collection interval for all inputs
* **round_interval**: Rounds collection interval to 'interval'
ie, if interval="10s" then always collect on :00, :10, :20, etc.
* **metric_buffer_limit**: Telegraf will cache metric_buffer_limit metrics
for each output, and will flush this buffer on a successful write.
* **collection_jitter**: Collection jitter is used to jitter
the collection by a random amount.
Each plugin will sleep for a random time within jitter before collecting.
This can be used to avoid many plugins querying things like sysfs at the
same time, which can have a measurable effect on the system.
* **flush_interval**: Default data flushing interval for all outputs.
You should not set this below
interval. Maximum flush_interval will be flush_interval + flush_jitter
* **flush_jitter**: Jitter the flush interval by a random amount.
This is primarily to avoid
large write spikes for users running a large number of telegraf instances.
ie, a jitter of 5s and flush_interval 10s means flushes will happen every 10-15s.
* **debug**: Run telegraf in debug mode.
* **quiet**: Run telegraf in quiet mode.
* **hostname**: Override default hostname, if empty use os.Hostname().

## `[inputs.xxx]` Configuration

There are some configuration options that are configurable per input:

* **name_override**: Override the base name of the measurement.
(Default is the name of the input).
* **name_prefix**: Specifies a prefix to attach to the measurement name.
* **name_suffix**: Specifies a suffix to attach to the measurement name.
* **tags**: A map of tags to apply to a specific input's measurements.
* **interval**: How often to gather this metric. Normal plugins use a single
global interval, but if one particular input should be run less or more often,
you can configure that here.

#### Input Filters

There are also filters that can be configured per input:

* **namepass**: An array of strings that is used to filter metrics generated by the
current input. Each string in the array is tested as a glob match against
measurement names and if it matches, the field is emitted.
* **namedrop**: The inverse of pass, if a measurement name matches, it is not emitted.
* **fieldpass**: An array of strings that is used to filter metrics generated by the
current input. Each string in the array is tested as a glob match against field names
and if it matches, the field is emitted.
* **fielddrop**: The inverse of pass, if a field name matches, it is not emitted.
* **tagpass**: tag names and arrays of strings that are used to filter
measurements by the current input. Each string in the array is tested as a glob
match against the tag name, and if it matches the measurement is emitted.
* **tagdrop**: The inverse of tagpass. If a tag matches, the measurement is not
emitted. This is tested on measurements that have passed the tagpass test.

#### Input Configuration Examples

This is a full working config that will output CPU data to an InfluxDB instance
at 192.168.59.103:8086, tagging measurements with dc="denver-1". It will output
measurements at a 10s interval and will collect per-cpu data, dropping any
fields which begin with `time_`.

```toml
[global_tags]
  dc = "denver-1"

[agent]
  interval = "10s"

# OUTPUTS
[[outputs.influxdb]]
  url = "http://192.168.59.103:8086" # required.
  database = "telegraf" # required.
  precision = "s"

# INPUTS
[[inputs.cpu]]
  percpu = true
  totalcpu = false
  # filter all fields beginning with 'time_'
  fielddrop = ["time_*"]
```

#### Input Config: tagpass and tagdrop

```toml
[[inputs.cpu]]
  percpu = true
  totalcpu = false
  fielddrop = ["cpu_time"]
  # Don't collect CPU data for cpu6 & cpu7
  [inputs.cpu.tagdrop]
    cpu = [ "cpu6", "cpu7" ]

[[inputs.disk]]
  [inputs.disk.tagpass]
    # tagpass conditions are OR, not AND.
    # If the (filesystem is ext4 or xfs) OR (the path is /opt or /home)
    # then the metric passes
    fstype = [ "ext4", "xfs" ]
    # Globs can also be used on the tag values
    path = [ "/opt", "/home*" ]
```

#### Input Config: fieldpass and fielddrop

```toml
# Drop all metrics for guest & steal CPU usage
[[inputs.cpu]]
  percpu = false
  totalcpu = true
  fielddrop = ["usage_guest", "usage_steal"]

# Only store inode related metrics for disks
[[inputs.disk]]
  fieldpass = ["inodes*"]
```

#### Input Config: namepass and namedrop

```toml
# Drop all metrics about containers for kubelet
[[inputs.prometheus]]
  urls = ["http://kube-node-1:4194/metrics"]
  namedrop = ["container_*"]

# Only store rest client related metrics for kubelet
[[inputs.prometheus]]
  urls = ["http://kube-node-1:4194/metrics"]
  namepass = ["rest_client_*"]
```

#### Input config: prefix, suffix, and override

This plugin will emit measurements with the name `cpu_total`

```toml
[[inputs.cpu]]
  name_suffix = "_total"
  percpu = false
  totalcpu = true
```

This will emit measurements with the name `foobar`

```toml
[[inputs.cpu]]
  name_override = "foobar"
  percpu = false
  totalcpu = true
```

#### Input config: tags

This plugin will emit measurements with two additional tags: `tag1=foo` and
`tag2=bar`

```toml
[[inputs.cpu]]
  percpu = false
  totalcpu = true
  [inputs.cpu.tags]
    tag1 = "foo"
    tag2 = "bar"
```

#### Multiple inputs of the same type

Additional inputs (or outputs) of the same type can be specified,
just define more instances in the config file. It is highly recommended that
you utilize `name_override`, `name_prefix`, or `name_suffix` config options
to avoid measurement collisions:

```toml
[[inputs.cpu]]
  percpu = false
  totalcpu = true

[[inputs.cpu]]
  percpu = true
  totalcpu = false
  name_override = "percpu_usage"
  fielddrop = ["cpu_time*"]
```

## `[outputs.xxx]` Configuration

Telegraf also supports specifying multiple output sinks to send data to,
configuring each output sink is different, but examples can be
found by running `telegraf -sample-config`.

Outputs also support the same configurable options as inputs
(namepass, namedrop, tagpass, tagdrop)

```toml
[[outputs.influxdb]]
  urls = [ "http://localhost:8086" ]
  database = "telegraf"
  precision = "s"
  # Drop all measurements that start with "aerospike"
  namedrop = ["aerospike*"]

[[outputs.influxdb]]
  urls = [ "http://localhost:8086" ]
  database = "telegraf-aerospike-data"
  precision = "s"
  # Only accept aerospike data:
  namepass = ["aerospike*"]

[[outputs.influxdb]]
  urls = [ "http://localhost:8086" ]
  database = "telegraf-cpu0-data"
  precision = "s"
  # Only store measurements where the tag "cpu" matches the value "cpu0"
  [outputs.influxdb.tagpass]
    cpu = ["cpu0"]
```

## Etcd Configuration

### Command line parameters

Telegraf option related to Etcd:
```
  -etcd               etcd urls where configuration is stored (comma separated)
  -etcdfolder         etcd folder where configuration is stored and read
  -etcdwriteconfigdir store the following config dir to etcd
  -etcdwriteconfig    store the following config file to etcd
  -etcderaseconfig    erase all telegraf config in etcd
  -etcdwritelabel     store config file to etcd with this label
  -etcdreadlabels     read config from etcd using labels (comma-separated)
```

### Features

* Main config file can be loaded in Etcd
* Each agent try automatically to find its own key in Etcd (/telegraf/hosts/HOSTNAME)
* Labels can be configured in config files, so labels could be from Etcd
* Etcd config watcher: that reload Telegraf when a change is detected in Etcd.
* You can write all configuration of ALL your Telegraf agent in a folder then send it to Etcd.

### Put simple configuration in Etcd


#### Create main configuration file

Create a `myconf.conf` file, which will be stored in Etcd, with the following content:

```
[tags]
  dc = "us-east-1"

[agent]
  interval = "10s"
  round_interval = true
  flush_interval = "10s"
  flush_jitter = "0s"
  debug = false
  hostname = ""


[[outputs.influxdb]]
  urls = ["http://localhost:8086"]
  database = "telegraf"
  precision = "s"


[[inputs.cpu]]
  percpu = true
  totalcpu = true
  drop = ["cpu_time*"]
```

#### Send the configuration to Etcd

Send this file to Etcd using the label **mylabel**

```
$ telegraf  -etcd http://127.0.0.1:2379 -etcdwritelabel mylabel -etcdwriteconfig myconf.conf
2016/03/08 19:50:27 Config written with key /telegraf/labels/mylabel
```

NOTE: By default, configuration is stored in the folder `/telegraf` in Etcd.
      You can change it using `-etcdfolder` flag


#### Check if your configuration in Etcd

You can check if data is really written in Etcd with

```
$ etcdctl get /telegraf/labels/mylabel
[tags]
  dc = "us-east-1"

[agent]
  interval = "10s"
  round_interval = true
  flush_interval = "10s"
  flush_jitter = "0s"
  debug = false
  hostname = ""


[[outputs.influxdb]]
  urls = ["http://localhost:8086"]
  database = "telegraf"
  precision = "s"


[[inputs.cpu]]
  percpu = true
  totalcpu = true
  drop = ["cpu_time*"]
```

#### 

Now any telegraf agent can load this config use the label **mylabel**

```
$ telegraf  -etcd http://127.0.0.1:2379 -etcdreadlabels mylabel
Config read with label mylabel
2016/03/08 19:54:52 WARNING: [etcd] 100: Key not found (/telegraf/main) [245]
2016/03/08 19:54:52 WARNING: [etcd] 100: Key not found (/telegraf/hosts) [245]
2016/03/08 19:54:52 Database creation failed: Get http://localhost:8086/query?db=&q=CREATE+DATABASE+IF+NOT+EXISTS+telegraf: dial tcp [::1]:8086: getsockopt: connection refused
2016/03/08 19:54:52 Starting Telegraf (version 0.10.4.1-49-g07a3a07)
2016/03/08 19:54:52 Loaded outputs: influxdb
2016/03/08 19:54:52 Loaded inputs: cpu
2016/03/08 19:54:52 Tags enabled: dc=us-east-1 host=hostname
2016/03/08 19:54:52 Agent Config: Interval:10s, Debug:false, Quiet:false, Hostname:"hostname", Flush Interval:10s
```

NOTE: By default, configuration is read in the folder `/telegraf` in Etcd.
      You can change it using `-etcdfolder` flag


#### Read configuration from Etcd

1. `/telegraf/main` key
2. `/telegraf/hosts/HOSTNAME` key
3. `/telegraf/labels/LABEL1` key
4. `/telegraf/labels/LABEL2` key

(First means less importance)

### Put configuration folder in Etcd

#### Create configuration folder structure

You can set your configuration in a folder like this:

```
testdata/test1
├── hosts
│   ├── myserver1.conf
│   └── myserver2.conf
├── labels
│   ├── influx.conf
│   ├── network2.conf
│   └── network.conf
└── main.conf
```

The following table how files and folder are stored in Etcd:

| Files                   | Etcd                | Details
|-------------------------|---------------------|-----------------------------------------------------------------------|
|                         | `/telegraf        ` |                                                                       |
| `/main.conf`            | `├── main         ` | Equivalent to main config file load with `-config` option             |
|                         | `├── hosts/       ` |                                                                       |
| `/host/myserver1.conf`  | `│   ├── myserver1` | Configuration specific for Telegraf agent with hostname **myserver1** |
| `/host/myserver2.conf`  | `│   └── myserver2` | Configuration specific for Telegraf agent with hostname **myserver2** |
|                         | `└── labels/      ` |                                                                       |
| `/labels/influx.conf`   | `    ├── influx   ` | Configuration loaded by hosts with etcd label **influx**              |
| `/labels/network2.conf` | `    ├── network2 ` | Configuration loaded by hosts with etcd label **network2**            |
| `/labels/network.conf`  | `    └── network  ` | Configuration loaded by hosts with etcd label **network**             |


(An example is available [here](../internal/etcd/testdata/test1))

#### Send configuration to Etcd

Then you can send your configuration folder to Etcd:

```
$ telegraf -etcd http://127.0.0.1:2379 -etcdwriteconfigdir internal/etcd/testdata/test1/
2016/03/08 20:14:28 Config written with key /telegraf/hosts/myserver1
2016/03/08 20:14:28 Config written with key /telegraf/labels/influx
2016/03/08 20:14:28 Config written with key /telegraf/labels/network
2016/03/08 20:14:28 Config written with key /telegraf/labels/network2
2016/03/08 20:14:28 Config written with key /telegraf/main
```
#### Read configuration from Etcd

Then you can start all your telegraf agents like this

```
$ telegraf  -etcd http://127.0.0.1:2379 -etcdreadlabels=influx,network
2016/03/08 20:17:25 Config read from etcd with labels influx,network
2016/03/08 20:17:25 Database creation failed: Get http://localhost:8086/query?db=&q=CREATE+DATABASE+IF+NOT+EXISTS+telegraf: dial tcp [::1]:8086: getsockopt: connection refused
2016/03/08 20:17:25 Starting Telegraf (version 0.10.4.1-49-g07a3a07)
2016/03/08 20:17:25 Loaded outputs: influxdb
2016/03/08 20:17:25 Loaded inputs: net
2016/03/08 20:17:25 Tags enabled: dc=us-east-1 host=myserver1
2016/03/08 20:17:25 Agent Config: Interval:2s, Debug:false, Quiet:false, Hostname:"myserver1", Flush Interval:10s 
2016/03/08 20:17:26 Gathered metrics, (2s interval), from 1 inputs in 1.382394ms
```