telegraf/plugins/inputs/kafka_consumer
Daniel Nelson 613de8a80d
Remove documentation in kafka_consumer for metric_buffer
2017-05-03 11:51:49 -07:00
..
README.md Remove documentation in kafka_consumer for metric_buffer 2017-05-03 11:51:49 -07:00
kafka_consumer.go Add max_message_len in kafka_consumer input (#2636) 2017-04-11 12:05:39 -07:00
kafka_consumer_integration_test.go use AddError everywhere (#2372) 2017-04-24 11:13:26 -07:00
kafka_consumer_test.go use AddError everywhere (#2372) 2017-04-24 11:13:26 -07:00

README.md

Kafka Consumer Input Plugin

The Kafka consumer plugin polls a specified Kafka topic and adds messages to InfluxDB. The plugin assumes messages follow the line protocol. Consumer Group is used to talk to the Kafka cluster so multiple instances of telegraf can read from the same topic in parallel.

Configuration

# Read metrics from Kafka topic(s)
[[inputs.kafka_consumer]]
  ## topic(s) to consume
  topics = ["telegraf"]
  ## an array of Zookeeper connection strings
  zookeeper_peers = ["localhost:2181"]
  ## Zookeeper Chroot
  zookeeper_chroot = ""
  ## the name of the consumer group
  consumer_group = "telegraf_metrics_consumers"
  ## Offset (must be either "oldest" or "newest")
  offset = "oldest"

  ## Data format to consume.
  ## Each data format has its own unique set of configuration options, read
  ## more about them here:
  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
  data_format = "influx"

  ## Maximum length of a message to consume, in bytes (default 0/unlimited);
  ## larger messages are dropped
  max_message_len = 65536

Testing

Running integration tests requires running Zookeeper & Kafka. See Makefile for kafka container command.