telegraf/plugins/inputs/kafka_consumer/README.md

51 lines
1.6 KiB
Markdown
Raw Normal View History

2016-02-09 22:03:46 +00:00
# Kafka Consumer Input Plugin
2015-07-08 19:39:23 +00:00
The [Kafka](http://kafka.apache.org/) consumer plugin polls a specified Kafka
topic and adds messages to InfluxDB. The plugin assumes messages follow the
line protocol. [Consumer Group](http://godoc.org/github.com/wvanbergen/kafka/consumergroup)
is used to talk to the Kafka cluster so multiple instances of telegraf can read
from the same topic in parallel.
For old kafka version (< 0.8), please use the kafka_consumer_legacy input plugin
and use the old zookeeper connection method.
2016-02-09 22:03:46 +00:00
## Configuration
```toml
# Read metrics from Kafka topic(s)
[[inputs.kafka_consumer]]
## topic(s) to consume
2016-02-09 22:03:46 +00:00
topics = ["telegraf"]
brokers = ["localhost:9092"]
## the name of the consumer group
2016-02-09 22:03:46 +00:00
consumer_group = "telegraf_metrics_consumers"
## Offset (must be either "oldest" or "newest")
2016-02-09 22:03:46 +00:00
offset = "oldest"
## Optional SSL Config
# ssl_ca = "/etc/telegraf/ca.pem"
# ssl_cert = "/etc/telegraf/cert.pem"
# ssl_key = "/etc/telegraf/key.pem"
## Use SSL but skip chain & host verification
# insecure_skip_verify = false
## Optional SASL Config
# sasl_username = "kafka"
# sasl_password = "secret"
## Data format to consume.
2017-04-27 21:59:18 +00:00
## Each data format has its own unique set of configuration options, read
## more about them here:
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
2016-02-09 22:03:46 +00:00
data_format = "influx"
## Maximum length of a message to consume, in bytes (default 0/unlimited);
## larger messages are dropped
max_message_len = 65536
2016-02-09 22:03:46 +00:00
```
2015-07-08 19:39:23 +00:00
## Testing
Running integration tests requires running Zookeeper & Kafka. See Makefile
for kafka container command.