62 lines
1.9 KiB
Markdown
62 lines
1.9 KiB
Markdown
# Kafka Consumer Input Plugin
|
|
|
|
The [Kafka](http://kafka.apache.org/) consumer plugin polls a specified Kafka
|
|
topic and adds messages to InfluxDB. The plugin assumes messages follow the
|
|
line protocol. [Consumer Group](http://godoc.org/github.com/wvanbergen/kafka/consumergroup)
|
|
is used to talk to the Kafka cluster so multiple instances of telegraf can read
|
|
from the same topic in parallel.
|
|
|
|
For old kafka version (< 0.8), please use the kafka_consumer_legacy input plugin
|
|
and use the old zookeeper connection method.
|
|
|
|
## Configuration
|
|
|
|
```toml
|
|
# Read metrics from Kafka topic(s)
|
|
[[inputs.kafka_consumer]]
|
|
## kafka servers
|
|
brokers = ["localhost:9092"]
|
|
## topic(s) to consume
|
|
topics = ["telegraf"]
|
|
|
|
## Optional Client id
|
|
# client_id = "Telegraf"
|
|
|
|
## Set the minimal supported Kafka version. Setting this enables the use of new
|
|
## Kafka features and APIs. Of particular interest, lz4 compression
|
|
## requires at least version 0.10.0.0.
|
|
## ex: version = "1.1.0"
|
|
# version = ""
|
|
|
|
## Optional TLS Config
|
|
# tls_ca = "/etc/telegraf/ca.pem"
|
|
# tls_cert = "/etc/telegraf/cert.pem"
|
|
# tls_key = "/etc/telegraf/key.pem"
|
|
## Use TLS but skip chain & host verification
|
|
# insecure_skip_verify = false
|
|
|
|
## Optional SASL Config
|
|
# sasl_username = "kafka"
|
|
# sasl_password = "secret"
|
|
|
|
## the name of the consumer group
|
|
consumer_group = "telegraf_metrics_consumers"
|
|
## Offset (must be either "oldest" or "newest")
|
|
offset = "oldest"
|
|
|
|
## Data format to consume.
|
|
## Each data format has its own unique set of configuration options, read
|
|
## more about them here:
|
|
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
|
|
data_format = "influx"
|
|
|
|
## Maximum length of a message to consume, in bytes (default 0/unlimited);
|
|
## larger messages are dropped
|
|
max_message_len = 1000000
|
|
```
|
|
|
|
## Testing
|
|
|
|
Running integration tests requires running Zookeeper & Kafka. See Makefile
|
|
for kafka container command.
|