Commit Graph

40 Commits

Author SHA1 Message Date
Daniel Nelson aea09b3a20
Use sarama built in support for consumer groups (#6172) 2019-07-29 20:41:12 -07:00
Greg d0a6051fd7 Prevent panic when marking the offset in kafka_consumer (#5118) 2018-12-10 15:55:58 -08:00
Sebastien Le Digabel 448c98d82e Add ability to tag metrics with topic in kafka_consumer (#5038) 2018-11-28 16:29:26 -08:00
Daniel Nelson 6e5c2f8bb6
Remove outputs blocking inputs when output is slow (#4938) 2018-11-05 13:34:28 -08:00
Daniel Nelson 2b026374ec
Update kafka_consumer sample config in README 2018-08-23 11:46:41 -07:00
maxunt 2729378b7f Add name, time, path and string field options to JSON parser (#4351) 2018-08-22 19:26:48 -07:00
Daniel Nelson 886d8cc840
Drop message batches in kafka output if too large (#4565) 2018-08-17 13:51:21 -07:00
Daniel Nelson 49a5dea536
Update client_id in kafka input and output readme 2018-07-13 14:00:25 -07:00
Rion c8f00030dd Add support for setting kafka client id (#4418) 2018-07-13 13:53:56 -07:00
Daniel Nelson de355b76d6
Simplify testing with TLS (#4095) 2018-05-04 16:33:23 -07:00
Daniel Nelson 4dfe2312d0
Switch skipped kafka test 2017-07-18 18:18:57 -07:00
Daniel Nelson 4c534433aa
Skip kafka_consumer_integration_test due to issue on CircleCI 2017-06-07 18:31:52 -07:00
Seuf 2092443cd7 Add Kafka 0.9+ consumer support (#2487) 2017-06-07 18:22:28 -07:00
Daniel Nelson 613de8a80d
Remove documentation in kafka_consumer for metric_buffer 2017-05-03 11:51:49 -07:00
Damien Krotkine f5c890cc1d reflect zookeeper chroot config in readme (#2759) 2017-05-03 11:50:08 -07:00
Daniel Nelson 052e88ad5e
Fix grammar 2017-04-27 14:59:18 -07:00
Patrick Hemmer 06baf7cf78 use AddError everywhere (#2372) 2017-04-24 11:13:26 -07:00
Nick Irvine 0193cbee51 Add max_message_len in kafka_consumer input (#2636) 2017-04-11 12:05:39 -07:00
Patrick Hemmer 1402c158b7 remove sleep from tests (#2555) 2017-03-24 12:03:36 -07:00
Emil Haugbergsmyr 81f95e7a29 Fixes change in Kafka consumer input plugin (#2222)
* Fixes change to the error api in the kafka project.

* Updated test to reflect the change.

* Update kafka to match master branch.
2017-01-11 16:24:09 +00:00
Cameron Sparr e5c7a71d8e Fix unit tests for new metric implementation 2016-12-01 18:17:02 +00:00
Cameron Sparr c7834209d2 Major Logging Overhaul
in this commit:

- centralize logging output handler.
- set global Info/Debug/Error log levels based on config file or flags.
- remove per-plugin debug arg handling.
- add a I!, D!, or E! to every log message.
- add configuration option to specify where to send logs.

closes #1786
2016-10-03 17:13:03 +01:00
Cameron Sparr 9b2e2cc41f kafka panic: Check that error is non-nil before
fixes #1764
2016-09-14 08:54:22 +01:00
Cameron Sparr 2fe427b3b3 mongodb input: fix version 2.2 panic
closes #1628
2016-09-06 11:58:06 +01:00
Cameron Sparr d6951dacdc Remove docker-machine/boot2docker dependencies & references 2016-06-22 17:25:01 +01:00
Cameron Sparr 137843b2f6 Change default zookeeper chroot to empty string
closes #1112
2016-06-10 12:07:36 +01:00
Cameron Sparr 4dd364e1c3 Update all readme instances of data formats 2016-04-05 14:42:20 -06:00
Cameron Sparr e436b2d720 Cleanup & standardize config file
changes:

- -sample-config will now comment out all but a few default plugins.
- config file parse errors will output path to bad conf file.
- cleanup 80-char line-length and some other style issues.
- default package conf file will now have all plugins, but commented
  out.

closes #199
closes #944
2016-04-01 10:59:53 -06:00
Prune Sebastien THOMAS bd3d0c330f parsed with gofmt
closes #776
2016-03-07 18:48:02 +01:00
Prune Sebastien THOMAS 0060df9877 added zookeeper_chroot option
added a plugin option zookeeper_chroot to set up the kafka endpoint in zookeeper, which may not be / (default).
This chroot is then configured in the consumergroup config.Zookeeper.Chroot

This is workaround the fact that this plugins does not handle the urls like    "zookeeper_server:port/chroot"
As the peers are stored in an array, it makes no sens to have them beeing URL. Peers should all be members of the same cluster, so they all have the same chroot.
2016-03-07 18:46:10 +01:00
Cameron Sparr 8d2e5f0bda Seems to be a toml parse bug around triple pounds 2016-02-18 14:36:03 -07:00
Cameron Sparr 7def6663bd Root directory cleanup 2016-02-18 13:37:36 -07:00
Cameron Sparr ee468be696 Flush based on buffer size rather than time
this includes:
- Add Accumulator to the Start() function of service inputs
- For message consumer plugins, use the Accumulator to constantly add
  metrics and make Gather a dummy function
- rework unit tests to match this new behavior.
- make "flush_buffer_when_full" a config option that defaults to true

closes #666
2016-02-16 22:25:22 -07:00
Cameron Sparr 8d0f50a6fd MQTT Consumer Input plugin 2016-02-12 11:13:32 -07:00
Cameron Sparr c09884c686 Fixup some URL typos 2016-02-08 21:36:53 -07:00
Cameron Sparr e619493ece Implementing generic parser plugins and documentation
This constitutes a large change in how we will parse different data
formats going forward (for the plugins that support it)

This is working off @henrypfhu's changes.
2016-02-08 21:08:44 -07:00
Cameron Sparr c549ab907a Throughout telegraf, use telegraf.Metric rather than client.Point
closes #599
2016-01-27 23:47:32 -07:00
Cameron Sparr 9c0d14bb60 Create public models for telegraf metrics, accumlator, plugins
This will basically make the root directory a place for storing the
major telegraf interfaces, which will make telegraf's godoc looks quite
a bit nicer. And make it easier for contributors to lookup the few data
types that they actually care about.

closes #564
2016-01-27 15:42:50 -07:00
Jack Zampolin 0cdf1b07e9 Fix issue 524 2016-01-20 10:57:35 -08:00
Cameron Sparr 9c5db1057d renaming plugins -> inputs 2016-01-07 15:04:30 -07:00