434 lines
14 KiB
Markdown
434 lines
14 KiB
Markdown
# Telegraf Configuration
|
||
|
||
## Generating a Configuration File
|
||
|
||
A default Telegraf config file can be generated using the -sample-config flag:
|
||
`telegraf -sample-config > telegraf.conf`
|
||
|
||
To generate a file with specific inputs and outputs, you can use the
|
||
-input-filter and -output-filter flags:
|
||
`telegraf -sample-config -input-filter cpu:mem:net:swap -output-filter influxdb:kafka`
|
||
|
||
## Environment Variables
|
||
|
||
Environment variables can be used anywhere in the config file, simply prepend
|
||
them with $. For strings the variable must be within quotes (ie, "$STR_VAR"),
|
||
for numbers and booleans they should be plain (ie, $INT_VAR, $BOOL_VAR)
|
||
|
||
## `[global_tags]` Configuration
|
||
|
||
Global tags can be specific in the `[global_tags]` section of the config file in
|
||
key="value" format. All metrics being gathered on this host will be tagged
|
||
with the tags specified here.
|
||
|
||
## `[agent]` Configuration
|
||
|
||
Telegraf has a few options you can configure under the `agent` section of the
|
||
config.
|
||
|
||
* **interval**: Default data collection interval for all inputs
|
||
* **round_interval**: Rounds collection interval to 'interval'
|
||
ie, if interval="10s" then always collect on :00, :10, :20, etc.
|
||
* **metric_buffer_limit**: Telegraf will cache metric_buffer_limit metrics
|
||
for each output, and will flush this buffer on a successful write.
|
||
* **collection_jitter**: Collection jitter is used to jitter
|
||
the collection by a random amount.
|
||
Each plugin will sleep for a random time within jitter before collecting.
|
||
This can be used to avoid many plugins querying things like sysfs at the
|
||
same time, which can have a measurable effect on the system.
|
||
* **flush_interval**: Default data flushing interval for all outputs.
|
||
You should not set this below
|
||
interval. Maximum flush_interval will be flush_interval + flush_jitter
|
||
* **flush_jitter**: Jitter the flush interval by a random amount.
|
||
This is primarily to avoid
|
||
large write spikes for users running a large number of telegraf instances.
|
||
ie, a jitter of 5s and flush_interval 10s means flushes will happen every 10-15s.
|
||
* **debug**: Run telegraf in debug mode.
|
||
* **quiet**: Run telegraf in quiet mode.
|
||
* **hostname**: Override default hostname, if empty use os.Hostname().
|
||
|
||
## `[inputs.xxx]` Configuration
|
||
|
||
There are some configuration options that are configurable per input:
|
||
|
||
* **name_override**: Override the base name of the measurement.
|
||
(Default is the name of the input).
|
||
* **name_prefix**: Specifies a prefix to attach to the measurement name.
|
||
* **name_suffix**: Specifies a suffix to attach to the measurement name.
|
||
* **tags**: A map of tags to apply to a specific input's measurements.
|
||
* **interval**: How often to gather this metric. Normal plugins use a single
|
||
global interval, but if one particular input should be run less or more often,
|
||
you can configure that here.
|
||
|
||
#### Input Filters
|
||
|
||
There are also filters that can be configured per input:
|
||
|
||
* **namepass**: An array of strings that is used to filter metrics generated by the
|
||
current input. Each string in the array is tested as a glob match against
|
||
measurement names and if it matches, the field is emitted.
|
||
* **namedrop**: The inverse of pass, if a measurement name matches, it is not emitted.
|
||
* **fieldpass**: An array of strings that is used to filter metrics generated by the
|
||
current input. Each string in the array is tested as a glob match against field names
|
||
and if it matches, the field is emitted.
|
||
* **fielddrop**: The inverse of pass, if a field name matches, it is not emitted.
|
||
* **tagpass**: tag names and arrays of strings that are used to filter
|
||
measurements by the current input. Each string in the array is tested as a glob
|
||
match against the tag name, and if it matches the measurement is emitted.
|
||
* **tagdrop**: The inverse of tagpass. If a tag matches, the measurement is not
|
||
emitted. This is tested on measurements that have passed the tagpass test.
|
||
|
||
#### Input Configuration Examples
|
||
|
||
This is a full working config that will output CPU data to an InfluxDB instance
|
||
at 192.168.59.103:8086, tagging measurements with dc="denver-1". It will output
|
||
measurements at a 10s interval and will collect per-cpu data, dropping any
|
||
fields which begin with `time_`.
|
||
|
||
```toml
|
||
[global_tags]
|
||
dc = "denver-1"
|
||
|
||
[agent]
|
||
interval = "10s"
|
||
|
||
# OUTPUTS
|
||
[[outputs.influxdb]]
|
||
url = "http://192.168.59.103:8086" # required.
|
||
database = "telegraf" # required.
|
||
precision = "s"
|
||
|
||
# INPUTS
|
||
[[inputs.cpu]]
|
||
percpu = true
|
||
totalcpu = false
|
||
# filter all fields beginning with 'time_'
|
||
fielddrop = ["time_*"]
|
||
```
|
||
|
||
#### Input Config: tagpass and tagdrop
|
||
|
||
```toml
|
||
[[inputs.cpu]]
|
||
percpu = true
|
||
totalcpu = false
|
||
fielddrop = ["cpu_time"]
|
||
# Don't collect CPU data for cpu6 & cpu7
|
||
[inputs.cpu.tagdrop]
|
||
cpu = [ "cpu6", "cpu7" ]
|
||
|
||
[[inputs.disk]]
|
||
[inputs.disk.tagpass]
|
||
# tagpass conditions are OR, not AND.
|
||
# If the (filesystem is ext4 or xfs) OR (the path is /opt or /home)
|
||
# then the metric passes
|
||
fstype = [ "ext4", "xfs" ]
|
||
# Globs can also be used on the tag values
|
||
path = [ "/opt", "/home*" ]
|
||
```
|
||
|
||
#### Input Config: fieldpass and fielddrop
|
||
|
||
```toml
|
||
# Drop all metrics for guest & steal CPU usage
|
||
[[inputs.cpu]]
|
||
percpu = false
|
||
totalcpu = true
|
||
fielddrop = ["usage_guest", "usage_steal"]
|
||
|
||
# Only store inode related metrics for disks
|
||
[[inputs.disk]]
|
||
fieldpass = ["inodes*"]
|
||
```
|
||
|
||
#### Input Config: namepass and namedrop
|
||
|
||
```toml
|
||
# Drop all metrics about containers for kubelet
|
||
[[inputs.prometheus]]
|
||
urls = ["http://kube-node-1:4194/metrics"]
|
||
namedrop = ["container_*"]
|
||
|
||
# Only store rest client related metrics for kubelet
|
||
[[inputs.prometheus]]
|
||
urls = ["http://kube-node-1:4194/metrics"]
|
||
namepass = ["rest_client_*"]
|
||
```
|
||
|
||
#### Input config: prefix, suffix, and override
|
||
|
||
This plugin will emit measurements with the name `cpu_total`
|
||
|
||
```toml
|
||
[[inputs.cpu]]
|
||
name_suffix = "_total"
|
||
percpu = false
|
||
totalcpu = true
|
||
```
|
||
|
||
This will emit measurements with the name `foobar`
|
||
|
||
```toml
|
||
[[inputs.cpu]]
|
||
name_override = "foobar"
|
||
percpu = false
|
||
totalcpu = true
|
||
```
|
||
|
||
#### Input config: tags
|
||
|
||
This plugin will emit measurements with two additional tags: `tag1=foo` and
|
||
`tag2=bar`
|
||
|
||
```toml
|
||
[[inputs.cpu]]
|
||
percpu = false
|
||
totalcpu = true
|
||
[inputs.cpu.tags]
|
||
tag1 = "foo"
|
||
tag2 = "bar"
|
||
```
|
||
|
||
#### Multiple inputs of the same type
|
||
|
||
Additional inputs (or outputs) of the same type can be specified,
|
||
just define more instances in the config file. It is highly recommended that
|
||
you utilize `name_override`, `name_prefix`, or `name_suffix` config options
|
||
to avoid measurement collisions:
|
||
|
||
```toml
|
||
[[inputs.cpu]]
|
||
percpu = false
|
||
totalcpu = true
|
||
|
||
[[inputs.cpu]]
|
||
percpu = true
|
||
totalcpu = false
|
||
name_override = "percpu_usage"
|
||
fielddrop = ["cpu_time*"]
|
||
```
|
||
|
||
## `[outputs.xxx]` Configuration
|
||
|
||
Telegraf also supports specifying multiple output sinks to send data to,
|
||
configuring each output sink is different, but examples can be
|
||
found by running `telegraf -sample-config`.
|
||
|
||
Outputs also support the same configurable options as inputs
|
||
(namepass, namedrop, tagpass, tagdrop)
|
||
|
||
```toml
|
||
[[outputs.influxdb]]
|
||
urls = [ "http://localhost:8086" ]
|
||
database = "telegraf"
|
||
precision = "s"
|
||
# Drop all measurements that start with "aerospike"
|
||
namedrop = ["aerospike*"]
|
||
|
||
[[outputs.influxdb]]
|
||
urls = [ "http://localhost:8086" ]
|
||
database = "telegraf-aerospike-data"
|
||
precision = "s"
|
||
# Only accept aerospike data:
|
||
namepass = ["aerospike*"]
|
||
|
||
[[outputs.influxdb]]
|
||
urls = [ "http://localhost:8086" ]
|
||
database = "telegraf-cpu0-data"
|
||
precision = "s"
|
||
# Only store measurements where the tag "cpu" matches the value "cpu0"
|
||
[outputs.influxdb.tagpass]
|
||
cpu = ["cpu0"]
|
||
```
|
||
|
||
## Etcd Configuration
|
||
|
||
### Command line parameters
|
||
|
||
Telegraf option related to Etcd:
|
||
```
|
||
-etcd etcd urls where configuration is stored (comma separated)
|
||
-etcdfolder etcd folder where configuration is stored and read
|
||
-etcdwriteconfigdir store the following config dir to etcd
|
||
-etcdwriteconfig store the following config file to etcd
|
||
-etcderaseconfig erase all telegraf config in etcd
|
||
-etcdwritelabel store config file to etcd with this label
|
||
-etcdreadlabels read config from etcd using labels (comma-separated)
|
||
```
|
||
|
||
### Features
|
||
|
||
* Main config file can be loaded in Etcd
|
||
* Each agent try automatically to find its own key in Etcd (/telegraf/hosts/HOSTNAME)
|
||
* Labels can be configured in config files, so labels could be from Etcd
|
||
* Etcd config watcher: that reload Telegraf when a change is detected in Etcd.
|
||
* You can write all configuration of ALL your Telegraf agent in a folder then send it to Etcd.
|
||
|
||
### Put simple configuration in Etcd
|
||
|
||
|
||
#### Create main configuration file
|
||
|
||
Create a `myconf.conf` file, which will be stored in Etcd, with the following content:
|
||
|
||
```
|
||
[tags]
|
||
dc = "us-east-1"
|
||
|
||
[agent]
|
||
interval = "10s"
|
||
round_interval = true
|
||
flush_interval = "10s"
|
||
flush_jitter = "0s"
|
||
debug = false
|
||
hostname = ""
|
||
|
||
|
||
[[outputs.influxdb]]
|
||
urls = ["http://localhost:8086"]
|
||
database = "telegraf"
|
||
precision = "s"
|
||
|
||
|
||
[[inputs.cpu]]
|
||
percpu = true
|
||
totalcpu = true
|
||
drop = ["cpu_time*"]
|
||
```
|
||
|
||
#### Send the configuration to Etcd
|
||
|
||
Send this file to Etcd using the label **mylabel**
|
||
|
||
```
|
||
$ telegraf -etcd http://127.0.0.1:2379 -etcdwritelabel mylabel -etcdwriteconfig myconf.conf
|
||
2016/03/08 19:50:27 Config written with key /telegraf/labels/mylabel
|
||
```
|
||
|
||
NOTE: By default, configuration is stored in the folder `/telegraf` in Etcd.
|
||
You can change it using `-etcdfolder` flag
|
||
|
||
|
||
#### Check if your configuration in Etcd
|
||
|
||
You can check if data is really written in Etcd with
|
||
|
||
```
|
||
$ etcdctl get /telegraf/labels/mylabel
|
||
[tags]
|
||
dc = "us-east-1"
|
||
|
||
[agent]
|
||
interval = "10s"
|
||
round_interval = true
|
||
flush_interval = "10s"
|
||
flush_jitter = "0s"
|
||
debug = false
|
||
hostname = ""
|
||
|
||
|
||
[[outputs.influxdb]]
|
||
urls = ["http://localhost:8086"]
|
||
database = "telegraf"
|
||
precision = "s"
|
||
|
||
|
||
[[inputs.cpu]]
|
||
percpu = true
|
||
totalcpu = true
|
||
drop = ["cpu_time*"]
|
||
```
|
||
|
||
####
|
||
|
||
Now any telegraf agent can load this config use the label **mylabel**
|
||
|
||
```
|
||
$ telegraf -etcd http://127.0.0.1:2379 -etcdreadlabels mylabel
|
||
Config read with label mylabel
|
||
2016/03/08 19:54:52 WARNING: [etcd] 100: Key not found (/telegraf/main) [245]
|
||
2016/03/08 19:54:52 WARNING: [etcd] 100: Key not found (/telegraf/hosts) [245]
|
||
2016/03/08 19:54:52 Database creation failed: Get http://localhost:8086/query?db=&q=CREATE+DATABASE+IF+NOT+EXISTS+telegraf: dial tcp [::1]:8086: getsockopt: connection refused
|
||
2016/03/08 19:54:52 Starting Telegraf (version 0.10.4.1-49-g07a3a07)
|
||
2016/03/08 19:54:52 Loaded outputs: influxdb
|
||
2016/03/08 19:54:52 Loaded inputs: cpu
|
||
2016/03/08 19:54:52 Tags enabled: dc=us-east-1 host=hostname
|
||
2016/03/08 19:54:52 Agent Config: Interval:10s, Debug:false, Quiet:false, Hostname:"hostname", Flush Interval:10s
|
||
```
|
||
|
||
NOTE: By default, configuration is read in the folder `/telegraf` in Etcd.
|
||
You can change it using `-etcdfolder` flag
|
||
|
||
|
||
#### Read configuration from Etcd
|
||
|
||
1. `/telegraf/main` key
|
||
2. `/telegraf/hosts/HOSTNAME` key
|
||
3. `/telegraf/labels/LABEL1` key
|
||
4. `/telegraf/labels/LABEL2` key
|
||
|
||
(First means less importance)
|
||
|
||
### Put configuration folder in Etcd
|
||
|
||
#### Create configuration folder structure
|
||
|
||
You can set your configuration in a folder like this:
|
||
|
||
```
|
||
testdata/test1
|
||
├── hosts
|
||
│ ├── myserver1.conf
|
||
│ └── myserver2.conf
|
||
├── labels
|
||
│ ├── influx.conf
|
||
│ ├── network2.conf
|
||
│ └── network.conf
|
||
└── main.conf
|
||
```
|
||
|
||
The following table how files and folder are stored in Etcd:
|
||
|
||
| Files | Etcd | Details
|
||
|-------------------------|---------------------|-----------------------------------------------------------------------|
|
||
| | `/telegraf ` | |
|
||
| `/main.conf` | `├── main ` | Equivalent to main config file load with `-config` option |
|
||
| | `├── hosts/ ` | |
|
||
| `/host/myserver1.conf` | `│ ├── myserver1` | Configuration specific for Telegraf agent with hostname **myserver1** |
|
||
| `/host/myserver2.conf` | `│ └── myserver2` | Configuration specific for Telegraf agent with hostname **myserver2** |
|
||
| | `└── labels/ ` | |
|
||
| `/labels/influx.conf` | ` ├── influx ` | Configuration loaded by hosts with etcd label **influx** |
|
||
| `/labels/network2.conf` | ` ├── network2 ` | Configuration loaded by hosts with etcd label **network2** |
|
||
| `/labels/network.conf` | ` └── network ` | Configuration loaded by hosts with etcd label **network** |
|
||
|
||
|
||
(An example is available [here](../internal/etcd/testdata/test1))
|
||
|
||
#### Send configuration to Etcd
|
||
|
||
Then you can send your configuration folder to Etcd:
|
||
|
||
```
|
||
$ telegraf -etcd http://127.0.0.1:2379 -etcdwriteconfigdir internal/etcd/testdata/test1/
|
||
2016/03/08 20:14:28 Config written with key /telegraf/hosts/myserver1
|
||
2016/03/08 20:14:28 Config written with key /telegraf/labels/influx
|
||
2016/03/08 20:14:28 Config written with key /telegraf/labels/network
|
||
2016/03/08 20:14:28 Config written with key /telegraf/labels/network2
|
||
2016/03/08 20:14:28 Config written with key /telegraf/main
|
||
```
|
||
#### Read configuration from Etcd
|
||
|
||
Then you can start all your telegraf agents like this
|
||
|
||
```
|
||
$ telegraf -etcd http://127.0.0.1:2379 -etcdreadlabels=influx,network
|
||
2016/03/08 20:17:25 Config read from etcd with labels influx,network
|
||
2016/03/08 20:17:25 Database creation failed: Get http://localhost:8086/query?db=&q=CREATE+DATABASE+IF+NOT+EXISTS+telegraf: dial tcp [::1]:8086: getsockopt: connection refused
|
||
2016/03/08 20:17:25 Starting Telegraf (version 0.10.4.1-49-g07a3a07)
|
||
2016/03/08 20:17:25 Loaded outputs: influxdb
|
||
2016/03/08 20:17:25 Loaded inputs: net
|
||
2016/03/08 20:17:25 Tags enabled: dc=us-east-1 host=myserver1
|
||
2016/03/08 20:17:25 Agent Config: Interval:2s, Debug:false, Quiet:false, Hostname:"myserver1", Flush Interval:10s
|
||
2016/03/08 20:17:26 Gathered metrics, (2s interval), from 1 inputs in 1.382394ms
|
||
```
|