Commit Graph

49 Commits

Author SHA1 Message Date
Florent Ramière cf9b794662 Add compression/acks/retry conf to Kafka output plugin
The following configuration is now possible

	## CompressionCodec represents the various compression codecs
recognized by Kafka in messages.
	##  "none" : No compression
	##  "gzip" : Gzip compression
	##  "snappy" : Snappy compression
	# compression_codec = "none"

	##  RequiredAcks is used in Produce Requests to tell the broker how
many replica acknowledgements it must see before responding
	##  "none" : the producer never waits for an acknowledgement from the
broker. This option provides the lowest latency but the weakest
durability guarantees (some data will be lost when a server fails).
	##  "leader" : the producer gets an acknowledgement after the leader
replica has received the data. This option provides better durability
as the client waits until the server acknowledges the request as
successful (only messages that were written to the now-dead leader but
not yet replicated will be lost).
	##  "leader_and_replicas" : the producer gets an acknowledgement after
all in-sync replicas have received the data. This option provides the
best durability, we guarantee that no messages will be lost as long as
at least one in sync replica remains.
	# required_acks = "leader_and_replicas"

	##  The total number of times to retry sending a message
	# max_retry = "3"
2016-04-08 18:34:52 +02:00
Cameron Sparr f26a824945 Use github paho mqtt client instead of gerrit
this might fix #921
2016-04-08 18:34:52 +02:00
Cameron Sparr 1cc65cfa12 Don't log every string metric that prometheus doesnt support 2016-04-08 18:34:51 +02:00
JP 3b1fa24160 add verifyValue func for datadog and librato, bail if no good
closes #906
2016-04-08 18:34:51 +02:00
Cameron Sparr 87bdff4321 Fix prometheus label names, and dont panic if invalid
fixes #907
2016-04-08 18:34:51 +02:00
Chris Goller ec2d410d23 Add optional parameters to influxdb output README 2016-04-08 18:34:51 +02:00
Cameron Sparr 649d03e3d7 influxdb output: quote the database name
closes #898
2016-04-08 18:34:51 +02:00
david birdsong d01f6f0bd2 special case 'value'
it usually connotes a single value type metric, appending just clutters

closes #793
2016-04-08 18:34:51 +02:00
david birdsong dcfafd3033 fix prometheus output
if i understand the prometheus data model correctly, the current output
for this plugin is unusable

prometheus only accepts a single value per measurement. prior to this change, the range loop
causes a measurement to end up w/ a random value

for instance:

net,dc=sjc1,grp_dashboard=1,grp_home=1,grp_hwy_fetcher=1,grp_web_admin=1,host=sjc1-b4-8,hw=app,interface=docker0,state=live
bytes_recv=477596i,bytes_sent=152963303i,drop_in=0i,drop_out=0i,err_in=0i,err_out=0i,packets_recv=7231i,packets_sent=11460i
1457121990003778992

this 'net' measurent  would have all it's tags copied to prometheus
labels, but any of 152963303, or 0, or 7231 as a value for
'net' depending on which field is last in the map iteration

this change expands the fields into new measurements by appending
the field name to the influxdb measurement name.

ie, the above example results with 'net' dropped and new measurements
to take it's place:
	net_bytes_recv
	net_bytes_sent
	net_drop_in
	net_err_in
	net_packets_recv
	net_packets_sent

i hope this can be merged, i love telegraf's composability of tags and
filtering
2016-04-08 18:34:51 +02:00
JP cc10985cac sanitize known issue characters from graphite tag name 2016-04-08 18:34:51 +02:00
JP 0682a95c48 replace @ character with - for librato 2016-04-08 18:34:50 +02:00
Eugene Dementiev a7b875d99b [amqp output] Allow external auth (cert-based tls auth)
closes #863
2016-04-08 18:34:49 +02:00
Thibault Cohen 1fbee019fc Improve prometheus plugin
closes #707
2016-04-08 18:34:49 +02:00
Cameron Sparr 77513a871f github wh: return from eventHandler when err != nil
closes #837
2016-04-08 18:34:49 +02:00
Cameron Sparr 34c4f4bf53 influxdb output: If all write fails, trigger a reconnect
closes #836
2016-04-08 18:34:49 +02:00
Manuel Sangoi 0cf2e605ae Do not ignore username option for mqtt output 2016-04-08 18:34:47 +02:00
Cameron Sparr 49a29c0c09 Support specifying influxdb retention policy
closes #692
2016-04-08 18:34:46 +02:00
bastard c85a86d69a Fixing Librato plugin
closes #722
2016-04-08 18:34:46 +02:00
Cameron Sparr de26b44c15 influxdb output: try to connect on write if there are no conns 2016-04-08 18:34:46 +02:00
Cameron Sparr 9813420d25 InfluxDB output should not default to 'no timeout' for http writes
default to 5s instead, since even if it times out we will cache the
points and move on

closes #685
2016-04-08 18:34:44 +02:00
Cameron Sparr 659357c1d0 Provide default args: percpu=true and totalcpu=true for cpu plugin
Also if outputs.file is empty, write to stdout

closes #720
2016-04-08 18:34:44 +02:00
Cameron Sparr 0dbb52f9d6 Seems to be a toml parse bug around triple pounds 2016-04-08 18:34:44 +02:00
Cameron Sparr 132f89e3fe Root directory cleanup 2016-04-08 18:34:44 +02:00
Dragostin Yanev (netixen) 87027fbacd pugins/outputs/influxdb: Prevent runtime panic.
- Check and return error from NewBatchPoints to prevent runtime panic if
   user provides an unparsable precision time unit in config.
- Provide correct sample config precision examples.
- Update etc/telegraf.conf precision comment.

closes #715
2016-04-08 18:34:44 +02:00
Cameron Sparr 716b375ab6 Start service plugins immediately, fix off-by-one bug 2016-04-08 18:34:43 +02:00
Cameron Sparr dc6cd5e714 Small readme formattings 2016-04-08 18:34:43 +02:00
Cameron Sparr 0415e8d19e Add support for specifying SSL config for influxdb output
closes #191
2016-04-08 18:34:42 +02:00
Cameron Sparr 5013290094 Add Serializer plugins, and 'file' output plugin 2016-04-08 18:34:42 +02:00
Cameron Sparr a9b91c7daa Add Serializer plugins, and 'file' output plugin 2016-04-08 18:34:42 +02:00
Cameron Sparr 668e16bb36 MQTT Consumer Input plugin 2016-04-08 18:34:42 +02:00
Dhruv Bansal 893357f01e Updated Riemann output:
* Customizable 'separator' option instead of hard-coded '_'

* String values are sent as "State" instead of "Metric", preventing
  Riemann from rejecting them

* Riemann service name is set to an (ugly) combination of input name &
  (sorted) tags' values...this allows connecting different events for
  the same input together on the Riemann side

closes #642
2016-02-09 11:17:07 -07:00
Cameron Sparr fc7fa4b6c5 Cleanup comments and indentation in config file 2016-02-09 11:01:50 -07:00
Cameron Sparr fb75db2f1f re-arrange and cleanup graphite output test 2016-02-09 11:01:13 -07:00
Cameron Sparr e619493ece Implementing generic parser plugins and documentation
This constitutes a large change in how we will parse different data
formats going forward (for the plugins that support it)

This is working off @henrypfhu's changes.
2016-02-08 21:08:44 -07:00
Cameron Sparr bd9c5b6995 mqtt output: cleanup, implement TLS
Also normalize TLS config across all output plugins and normalize
comment strings as well.
2016-02-04 10:44:37 -07:00
Cameron Sparr 38c4111e6c Add unit tests for the root telegraf package 2016-01-29 16:01:34 -07:00
Cameron Sparr 10c4e4f63f Fix datadog json marshalling
fixes #607
2016-01-28 16:12:33 -07:00
Cameron Sparr d835c19fce Insert . between msrmnt and field name in datadog output
fixes #600
2016-01-28 12:04:26 -07:00
Cameron Sparr 9d4b55be19 Include all tag values in graphite output
closes #595
2016-01-28 10:58:35 -07:00
Cameron Sparr c549ab907a Throughout telegraf, use telegraf.Metric rather than client.Point
closes #599
2016-01-27 23:47:32 -07:00
Cameron Sparr 9c0d14bb60 Create public models for telegraf metrics, accumlator, plugins
This will basically make the root directory a place for storing the
major telegraf interfaces, which will make telegraf's godoc looks quite
a bit nicer. And make it easier for contributors to lookup the few data
types that they actually care about.

closes #564
2016-01-27 15:42:50 -07:00
Cameron Sparr 3da390682d Kinesis output shouldn't return an error for no reason 2016-01-22 17:32:36 -07:00
Stephen Kwong e0dc1ef5bd Add Cloudwatch output
closes #553
2016-01-21 09:11:52 -07:00
Jack Zampolin 0cdf1b07e9 Fix issue 524 2016-01-20 10:57:35 -08:00
Hannu Valtonen c313af1b24 kafka: Add support for using TLS authentication for the kafka output
With the advent of Kafka 0.9.0+ it is possible to set up TLS client
certificate based authentication to limit access to Kafka.

Four new configuration variables are specified for setting up the
authentication. If they're not set the behavior stays the same as
before the change.

closes #541
2016-01-18 11:17:01 -07:00
Eugene Dementiev 37b96c192b output amqp: Add ssl support
closes #536
2016-01-16 17:31:05 -07:00
Cameron Sparr 8cbdf0f907 Tweak config messages for graphite. Update changelog and readme
closes #494
2016-01-16 17:29:02 -07:00
Thibault Cohen ef5c630d3a Add Graphite output 2016-01-16 17:19:27 -07:00
Cameron Sparr 9c5db1057d renaming plugins -> inputs 2016-01-07 15:04:30 -07:00