Only split metrics if there is an udp output (#2799)

closes #2739

CHANGELOG.md

@@ -41,6 +41,9 @@ be deprecated eventually.
 
 ### Features
 
+- [#2721](https://github.com/influxdata/telegraf/pull/2721): Added SASL options for kafka output plugin.
+- [#2723](https://github.com/influxdata/telegraf/pull/2723): Added SSL configuration for input haproxy.
+- [#2494](https://github.com/influxdata/telegraf/pull/2494): Add interrupts input plugin.
 - [#2094](https://github.com/influxdata/telegraf/pull/2094): Add generic socket listener & writer.
 - [#2204](https://github.com/influxdata/telegraf/pull/2204): Extend http_response to support searching for a substring in response. Return 1 if found, else 0.
 - [#2137](https://github.com/influxdata/telegraf/pull/2137): Added userstats to mysql input plugin.
@@ -56,10 +59,32 @@ be deprecated eventually.
 - [#2339](https://github.com/influxdata/telegraf/pull/2339): Increment gather_errors for all errors emitted by inputs.
 - [#2071](https://github.com/influxdata/telegraf/issues/2071): Use official docker SDK.
 - [#1678](https://github.com/influxdata/telegraf/pull/1678): Add AMQP consumer input plugin
+- [#2512](https://github.com/influxdata/telegraf/pull/2512): Added pprof tool.
 - [#2501](https://github.com/influxdata/telegraf/pull/2501): Support DEAD(X) state in system input plugin.
+- [#2522](https://github.com/influxdata/telegraf/pull/2522): Add support for mongodb client certificates.
+- [#1948](https://github.com/influxdata/telegraf/pull/1948): Support adding SNMP table indexes as tags.
+- [#2332](https://github.com/influxdata/telegraf/pull/2332): Add Elasticsearch 5.x output
+- [#2587](https://github.com/influxdata/telegraf/pull/2587): Add json timestamp units configurability
+- [#2597](https://github.com/influxdata/telegraf/issues/2597): Add support for Linux sysctl-fs metrics.
+- [#2425](https://github.com/influxdata/telegraf/pull/2425): Support to include/exclude docker container labels as tags
+- [#1667](https://github.com/influxdata/telegraf/pull/1667): dmcache input plugin
+- [#2637](https://github.com/influxdata/telegraf/issues/2637): Add support for precision in http_listener
+- [#2636](https://github.com/influxdata/telegraf/pull/2636): Add `message_len_max` option to `kafka_consumer` input
+- [#1100](https://github.com/influxdata/telegraf/issues/1100): Add collectd parser
+- [#1820](https://github.com/influxdata/telegraf/issues/1820): easier plugin testing without outputs
+- [#2493](https://github.com/influxdata/telegraf/pull/2493): Check signature in the GitHub webhook plugin
+- [#2038](https://github.com/influxdata/telegraf/issues/2038): Add papertrail support to webhooks
+- [#2253](https://github.com/influxdata/telegraf/pull/2253): Change jolokia plugin to use bulk requests.
+- [#2575](https://github.com/influxdata/telegraf/issues/2575) Add diskio input for Darwin
+- [#2705](https://github.com/influxdata/telegraf/pull/2705): Kinesis output: add use_random_partitionkey option
+- [#2635](https://github.com/influxdata/telegraf/issues/2635): add tcp keep-alive to socket_listener & socket_writer
+- [#2031](https://github.com/influxdata/telegraf/pull/2031): Add Kapacitor input plugin
+- [#2732](https://github.com/influxdata/telegraf/pull/2732): Use go 1.8.1
+- [#2712](https://github.com/influxdata/telegraf/issues/2712): Documentation for rabbitmq input plugin
 
 ### Bugfixes
 
+- [#2633](https://github.com/influxdata/telegraf/pull/2633): ipmi_sensor: allow @ symbol in password
 - [#2077](https://github.com/influxdata/telegraf/issues/2077): SQL Server Input - Arithmetic overflow error converting numeric to data type int.
 - [#2262](https://github.com/influxdata/telegraf/issues/2262): Flush jitter can inhibit metric collection.
 - [#2318](https://github.com/influxdata/telegraf/issues/2318): haproxy input - Add missing fields.
@@ -67,6 +92,7 @@ be deprecated eventually.
 - [#2356](https://github.com/influxdata/telegraf/issues/2356): cpu input panic when /proc/stat is empty.
 - [#2341](https://github.com/influxdata/telegraf/issues/2341): telegraf swallowing panics in --test mode.
 - [#2358](https://github.com/influxdata/telegraf/pull/2358): Create pidfile with 644 permissions & defer file deletion.
+- [#2360](https://github.com/influxdata/telegraf/pull/2360): Fixed install/remove of telegraf on non-systemd Debian/Ubuntu systems
 - [#2282](https://github.com/influxdata/telegraf/issues/2282): Reloading telegraf freezes prometheus output.
 - [#2390](https://github.com/influxdata/telegraf/issues/2390): Empty tag value causes error on InfluxDB output.
 - [#2380](https://github.com/influxdata/telegraf/issues/2380): buffer_size field value is negative number from "internal" plugin.
@@ -78,7 +104,23 @@ be deprecated eventually.
 - [#2483](https://github.com/influxdata/telegraf/pull/2483): Fix win_perf_counters capping values at 100.
 - [#2498](https://github.com/influxdata/telegraf/pull/2498): Exporting Ipmi.Path to be set by config.
 - [#2500](https://github.com/influxdata/telegraf/pull/2500): Remove warning if parse empty content
+- [#2520](https://github.com/influxdata/telegraf/pull/2520): Update default value for Cloudwatch rate limit
 - [#2513](https://github.com/influxdata/telegraf/issues/2513): create /etc/telegraf/telegraf.d directory in tarball.
+- [#2541](https://github.com/influxdata/telegraf/issues/2541): Return error on unsupported serializer data format.
+- [#1827](https://github.com/influxdata/telegraf/issues/1827): Fix Windows Performance Counters multi instance identifier
+- [#2576](https://github.com/influxdata/telegraf/pull/2576): Add write timeout to Riemann output
+- [#2596](https://github.com/influxdata/telegraf/pull/2596): fix timestamp parsing on prometheus plugin
+- [#2610](https://github.com/influxdata/telegraf/pull/2610): Fix deadlock when output cannot write
+- [#2410](https://github.com/influxdata/telegraf/issues/2410): Fix connection leak in postgresql.
+- [#2628](https://github.com/influxdata/telegraf/issues/2628): Set default measurement name for snmp input.
+- [#2649](https://github.com/influxdata/telegraf/pull/2649): Improve performance of diskio with many disks
+- [#2671](https://github.com/influxdata/telegraf/issues/2671): The internal input plugin uses the wrong units for `heap_objects`
+- [#2684](https://github.com/influxdata/telegraf/pull/2684): Fix ipmi_sensor config is shared between all plugin instances
+- [#2450](https://github.com/influxdata/telegraf/issues/2450): Network statistics not collected when system has alias interfaces
+- [#1911](https://github.com/influxdata/telegraf/issues/1911): Sysstat plugin needs LANG=C or similar locale
+- [#2528](https://github.com/influxdata/telegraf/issues/2528): File output closes standard streams on reload.
+- [#2603](https://github.com/influxdata/telegraf/issues/2603): AMQP output disconnect blocks all outputs
+- [#2706](https://github.com/influxdata/telegraf/issues/2706): Improve documentation for redis input plugin
 
 ## v1.2.1 [2017-02-01]
 

CONTRIBUTING.md

@@ -124,7 +124,7 @@ You should also add the following to your SampleConfig() return:
 
 ```toml
   ## Data format to consume.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
   data_format = "influx"
@@ -254,7 +254,7 @@ You should also add the following to your SampleConfig() return:
 
 ```toml
   ## Data format to output.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
   data_format = "influx"

Godeps

@@ -1,3 +1,4 @@
+collectd.org 2ce144541b8903101fb8f1483cc0497a68798122
 github.com/Shopify/sarama 574d3147eee384229bf96a5d12c207fe7b5234f3
 github.com/Sirupsen/logrus 61e43dc76f7ee59a82bdf3d71033dc12bea4c77d
 github.com/aerospike/aerospike-client-go 95e1ad7791bdbca44707fedbb29be42024900d9c
@@ -21,10 +22,10 @@ github.com/golang/snappy 7db9049039a047d955fe8c19b83c8ff5abd765c7
 github.com/gorilla/mux 392c28fe23e1c45ddba891b0320b3b5df220beea
 github.com/hailocab/go-hostpool e80d13ce29ede4452c43dea11e79b9bc8a15b478
 github.com/hashicorp/consul 63d2fc68239b996096a1c55a0d4b400ea4c2583f
-github.com/hpcloud/tail 915e5feba042395f5fda4dbe9c0e99aeab3088b3
+github.com/influxdata/tail a395bf99fe07c233f41fba0735fa2b13b58588ea
 github.com/influxdata/toml 5d1d907f22ead1cd47adde17ceec5bda9cacaf8f
 github.com/influxdata/wlog 7c63b0a71ef8300adc255344d275e10e5c3a71ec
-github.com/jackc/pgx c8080fc4a1bfa44bf90383ad0fdce2f68b7d313c
+github.com/jackc/pgx b84338d7d62598f75859b2b146d830b22f1b9ec8
 github.com/kardianos/osext c2c54e542fb797ad986b31721e1baedf214ca413
 github.com/kardianos/service 6d3a0ee7d3425d9d835debc51a0ca1ffa28f4893
 github.com/kballard/go-shellquote d8ec1a69a250a17bb0e419c386eac1f3711dc142
@@ -44,11 +45,12 @@ github.com/prometheus/common dd2f054febf4a6c00f2343686efb775948a8bff4
 github.com/prometheus/procfs 1878d9fbb537119d24b21ca07effd591627cd160
 github.com/rcrowley/go-metrics 1f30fe9094a513ce4c700b9a54458bbb0c96996c
 github.com/samuel/go-zookeeper 1d7be4effb13d2d908342d349d71a284a7542693
-github.com/shirou/gopsutil d371ba1293cb48fedc6850526ea48b3846c54f2c
+github.com/satori/go.uuid 5bf94b69c6b68ee1b541973bb8e1144db23a194b
+github.com/shirou/gopsutil 70693b6a3da51a8a686d31f1b346077bbc066062
 github.com/soniah/gosnmp 5ad50dc75ab389f8a1c9f8a67d3a1cd85f67ed15
 github.com/streadway/amqp 63795daa9a446c920826655f26ba31c81c860fd6
 github.com/stretchr/testify 4d4bfba8f1d1027c4fdbe371823030df51419987
-github.com/vjeantet/grok 83bfdfdfd1a8146795b28e547a8e3c8b28a466c2
+github.com/vjeantet/grok d73e972b60935c7fec0b4ffbc904ed39ecaf7efe
 github.com/wvanbergen/kafka bc265fedb9ff5b5c5d3c0fdcef4a819b3523d3ee
 github.com/wvanbergen/kazoo-go 968957352185472eacb69215fa3dbfcfdbac1096
 github.com/yuin/gopher-lua 66c871e454fcf10251c61bf8eff02d0978cae75a
@@ -59,4 +61,5 @@ golang.org/x/text 506f9d5c962f284575e88337e7d9296d27e729d3
 gopkg.in/dancannon/gorethink.v1 edc7a6a68e2d8015f5ffe1b2560eed989f8a45be
 gopkg.in/fatih/pool.v2 6e328e67893eb46323ad06f0e92cb9536babbabc
 gopkg.in/mgo.v2 3f83fa5005286a7fe593b055f0d7771a7dce4655
+gopkg.in/olivere/elastic.v5 ee3ebceab960cf68ab9a89ee6d78c031ef5b4a4e
 gopkg.in/yaml.v2 4c78c975fe7c825c6d1466c42be594d1d6f3aba6

Makefile

@@ -51,6 +51,7 @@ docker-run:
 		-e ADVERTISED_PORT=9092 \
 		-p "2181:2181" -p "9092:9092" \
 		-d spotify/kafka
+	docker run --name elasticsearch -p "9200:9200" -p "9300:9300" -d elasticsearch:5
 	docker run --name mysql -p "3306:3306" -e MYSQL_ALLOW_EMPTY_PASSWORD=yes -d mysql
 	docker run --name memcached -p "11211:11211" -d memcached
 	docker run --name postgres -p "5432:5432" -d postgres
@@ -69,6 +70,7 @@ docker-run-circle:
 		-e ADVERTISED_PORT=9092 \
 		-p "2181:2181" -p "9092:9092" \
 		-d spotify/kafka
+	docker run --name elasticsearch -p "9200:9200" -p "9300:9300" -d elasticsearch:5
 	docker run --name nsq -p "4150:4150" -d nsqio/nsq /nsqd
 	docker run --name mqtt -p "1883:1883" -d ncarlier/mqtt
 	docker run --name riemann -p "5555:5555" -d stealthly/docker-riemann
@@ -76,8 +78,8 @@ docker-run-circle:
 
 # Kill all docker containers, ignore errors
 docker-kill:
-	-docker kill nsq aerospike redis rabbitmq postgres memcached mysql kafka mqtt riemann nats
-	-docker rm nsq aerospike redis rabbitmq postgres memcached mysql kafka mqtt riemann nats
+	-docker kill nsq aerospike redis rabbitmq postgres memcached mysql kafka mqtt riemann nats elasticsearch
+	-docker rm nsq aerospike redis rabbitmq postgres memcached mysql kafka mqtt riemann nats elasticsearch
 
 # Run full unit tests using docker containers (includes setup and teardown)
 test: vet docker-kill docker-run

README.md

@@ -111,6 +111,7 @@ configuration options.
 * [couchbase](./plugins/inputs/couchbase)
 * [couchdb](./plugins/inputs/couchdb)
 * [disque](./plugins/inputs/disque)
+* [dmcache](./plugins/inputs/dmcache)
 * [dns query time](./plugins/inputs/dns_query)
 * [docker](./plugins/inputs/docker)
 * [dovecot](./plugins/inputs/dovecot)
@@ -123,9 +124,11 @@ configuration options.
 * [httpjson](./plugins/inputs/httpjson) (generic JSON-emitting http service plugin)
 * [internal](./plugins/inputs/internal)
 * [influxdb](./plugins/inputs/influxdb)
+* [interrupts](./plugins/inputs/interrupts)
 * [ipmi_sensor](./plugins/inputs/ipmi_sensor)
 * [iptables](./plugins/inputs/iptables)
 * [jolokia](./plugins/inputs/jolokia)
+* [kapacitor](./plugins/inputs/kapacitor)
 * [kubernetes](./plugins/inputs/kubernetes)
 * [leofs](./plugins/inputs/leofs)
 * [lustre2](./plugins/inputs/lustre2)
@@ -174,6 +177,7 @@ configuration options.
     * processes
     * kernel (/proc/stat)
     * kernel (/proc/vmstat)
+    * linux_sysctl_fs (/proc/sys/fs)
 
 Telegraf can also collect metrics via the following service plugins:
 
@@ -193,6 +197,17 @@ Telegraf can also collect metrics via the following service plugins:
   * [github](./plugins/inputs/webhooks/github)
   * [mandrill](./plugins/inputs/webhooks/mandrill)
   * [rollbar](./plugins/inputs/webhooks/rollbar)
+  * [papertrail](./plugins/inputs/webhooks/papertrail)
+
+Telegraf is able to parse the following input data formats into metrics, these
+formats may be used with input plugins supporting the `data_format` option:
+
+* [InfluxDB Line Protocol](./docs/DATA_FORMATS_INPUT.md#influx)
+* [JSON](./docs/DATA_FORMATS_INPUT.md#json)
+* [Graphite](./docs/DATA_FORMATS_INPUT.md#graphite)
+* [Value](./docs/DATA_FORMATS_INPUT.md#value)
+* [Nagios](./docs/DATA_FORMATS_INPUT.md#nagios)
+* [Collectd](./docs/DATA_FORMATS_INPUT.md#collectd)
 
 ## Processor Plugins
 
@@ -211,6 +226,7 @@ Telegraf can also collect metrics via the following service plugins:
 * [aws cloudwatch](./plugins/outputs/cloudwatch)
 * [datadog](./plugins/outputs/datadog)
 * [discard](./plugins/outputs/discard)
+* [elasticsearch](./plugins/outputs/elasticsearch)
 * [file](./plugins/outputs/file)
 * [graphite](./plugins/outputs/graphite)
 * [graylog](./plugins/outputs/graylog)

circle.yml

@@ -1,13 +1,11 @@
 machine:
+  go:
+    version: 1.8.1
   services:
     - docker
-  post:
-    - sudo service zookeeper stop
-    - go version
-    - sudo rm -rf /usr/local/go
-    - wget https://storage.googleapis.com/golang/go1.8.linux-amd64.tar.gz
-    - sudo tar -C /usr/local -xzf go1.8.linux-amd64.tar.gz
-    - go version
+    - memcached
+    - redis
+    - rabbitmq-server
 
 dependencies:
   override:

cmd/telegraf/telegraf.go

@@ -4,6 +4,8 @@ import (
 	"flag"
 	"fmt"
 	"log"
+	"net/http"
+	_ "net/http/pprof" // Comment this line to disable pprof endpoint.
 	"os"
 	"os/signal"
 	"runtime"
@@ -24,6 +26,8 @@ import (
 
 var fDebug = flag.Bool("debug", false,
 	"turn on debug logging")
+var pprofAddr = flag.String("pprof-addr", "",
+	"pprof address to listen on, not activate pprof if empty")
 var fQuiet = flag.Bool("quiet", false,
 	"run in quiet mode")
 var fTest = flag.Bool("test", false, "gather metrics, print them out, and exit")
@@ -87,6 +91,7 @@ The commands & flags are:
   --output-filter     filter the output plugins to enable, separator is :
   --usage             print usage for a plugin, ie, 'telegraf --usage mysql'
   --debug             print metrics as they're generated to stdout
+  --pprof-addr        pprof address to listen on, format: localhost:6060 or :6060
   --quiet             run in quiet mode
 
 Examples:
@@ -105,6 +110,9 @@ Examples:
 
   # run telegraf, enabling the cpu & memory input, and influxdb output plugins
   telegraf --config telegraf.conf --input-filter cpu:mem --output-filter influxdb
+
+  # run telegraf with pprof
+  telegraf --config telegraf.conf --pprof-addr localhost:6060
 `
 
 var stop chan struct{}
@@ -136,7 +144,7 @@ func reloadLoop(
 				log.Fatal("E! " + err.Error())
 			}
 		}
-		if len(c.Outputs) == 0 {
+		if !*fTest && len(c.Outputs) == 0 {
 			log.Fatalf("E! Error: no outputs found, did you provide a valid config file?")
 		}
 		if len(c.Inputs) == 0 {
@@ -267,6 +275,23 @@ func main() {
 		processorFilters = strings.Split(":"+strings.TrimSpace(*fProcessorFilters)+":", ":")
 	}
 
+	if *pprofAddr != "" {
+		go func() {
+			pprofHostPort := *pprofAddr
+			parts := strings.Split(pprofHostPort, ":")
+			if len(parts) == 2 && parts[0] == "" {
+				pprofHostPort = fmt.Sprintf("localhost:%s", parts[1])
+			}
+			pprofHostPort = "http://" + pprofHostPort + "/debug/pprof"
+
+			log.Printf("I! Starting pprof HTTP server at: %s", pprofHostPort)
+
+			if err := http.ListenAndServe(*pprofAddr, nil); err != nil {
+				log.Fatal("E! " + err.Error())
+			}
+		}()
+	}
+
 	if len(args) > 0 {
 		switch args[0] {
 		case "version":

docs/CONFIGURATION.md

@@ -70,7 +70,7 @@ ie, a jitter of 5s and flush_interval 10s means flushes will happen every 10-15s
 as the collection interval, with the maximum being 1s. Precision will NOT
 be used for service inputs, such as logparser and statsd. Valid values are
 "ns", "us" (or "µs"), "ms", "s".
-* **logfile**: Specify the log file name. The empty string means to log to stdout.
+* **logfile**: Specify the log file name. The empty string means to log to stderr.
 * **debug**: Run telegraf in debug mode.
 * **quiet**: Run telegraf in quiet mode (error messages only).
 * **hostname**: Override default hostname, if empty use os.Hostname().
@@ -124,31 +124,40 @@ is not specified then processor execution order will be random.
 Filters can be configured per input, output, processor, or aggregator,
 see below for examples.
 
-* **namepass**: An array of strings that is used to filter metrics generated by the
-current input. Each string in the array is tested as a glob match against
-measurement names and if it matches, the field is emitted.
-* **namedrop**: The inverse of pass, if a measurement name matches, it is not emitted.
-* **fieldpass**: An array of strings that is used to filter metrics generated by the
-current input. Each string in the array is tested as a glob match against field names
-and if it matches, the field is emitted. fieldpass is not available for outputs.
-* **fielddrop**: The inverse of pass, if a field name matches, it is not emitted.
-fielddrop is not available for outputs.
-* **tagpass**: tag names and arrays of strings that are used to filter
-measurements by the current input. Each string in the array is tested as a glob
-match against the tag name, and if it matches the measurement is emitted.
-* **tagdrop**: The inverse of tagpass. If a tag matches, the measurement is not
-emitted. This is tested on measurements that have passed the tagpass test.
-* **tagexclude**: tagexclude can be used to exclude a tag from measurement(s).
-As opposed to tagdrop, which will drop an entire measurement based on it's
-tags, tagexclude simply strips the given tag keys from the measurement. This
-can be used on inputs & outputs, but it is _recommended_ to be used on inputs,
-as it is more efficient to filter out tags at the ingestion point.
-* **taginclude**: taginclude is the inverse of tagexclude. It will only include
-the tag keys in the final measurement.
+* **namepass**:
+An array of glob pattern strings.  Only points whose measurement name matches
+a pattern in this list are emitted.
+* **namedrop**:
+The inverse of `namepass`.  If a match is found the point is discarded. This
+is tested on points after they have passed the `namepass` test.
+* **fieldpass**:
+An array of glob pattern strings.  Only fields whose field key matches a
+pattern in this list are emitted.  Not available for outputs.
+* **fielddrop**:
+The inverse of `fieldpass`. Fields with a field key matching one of the
+patterns will be discarded from the point.  Not available for outputs.
+* **tagpass**:
+A table mapping tag keys to arrays of glob pattern strings.  Only points
+that contain a tag key in the table and a tag value matching one of its
+patterns is emitted.
+* **tagdrop**:
+The inverse of `tagpass`.  If a match is found the point is discarded. This
+is tested on points after they have passed the `tagpass` test.
+* **taginclude**:
+An array of glob pattern strings.  Only tags with a tag key matching one of
+the patterns are emitted.  In contrast to `tagpass`, which will pass an entire
+point based on its tag, `taginclude` removes all non matching tags from the
+point.  This filter can be used on both inputs & outputs, but it is
+_recommended_ to be used on inputs, as it is more efficient to filter out tags
+at the ingestion point.
+* **tagexclude**:
+The inverse of `taginclude`. Tags with a tag key matching one of the patterns
+will be discarded from the point.
 
-**NOTE** `tagpass` and `tagdrop` parameters must be defined at the _end_ of
-the plugin definition, otherwise subsequent plugin config options will be
-interpreted as part of the tagpass/tagdrop map.
+**NOTE** Due to the way TOML is parsed, `tagpass` and `tagdrop` parameters
+must be defined at the _end_ of the plugin definition, otherwise subsequent
+plugin config options will be interpreted as part of the tagpass/tagdrop
+tables.
 
 #### Input Configuration Examples
 

docs/DATA_FORMATS_INPUT.md

@@ -7,6 +7,7 @@ Telegraf is able to parse the following input data formats into metrics:
 1. [Graphite](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md#graphite)
 1. [Value](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md#value), ie: 45 or "booyah"
 1. [Nagios](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md#nagios) (exec input only)
+1. [Collectd](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md#collectd)
 
 Telegraf metrics, like InfluxDB
 [points](https://docs.influxdata.com/influxdb/v0.10/write_protocols/line/),
@@ -40,7 +41,7 @@ example, in the exec plugin:
   name_suffix = "_mycollector"
 
   ## Data format to consume.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
   data_format = "json"
@@ -67,7 +68,7 @@ metrics are parsed directly into Telegraf metrics.
   name_suffix = "_mycollector"
 
   ## Data format to consume.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
   data_format = "influx"
@@ -117,7 +118,7 @@ For example, if you had this configuration:
   name_suffix = "_mycollector"
 
   ## Data format to consume.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
   data_format = "json"
@@ -161,7 +162,7 @@ For example, if the following configuration:
   name_suffix = "_mycollector"
 
   ## Data format to consume.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
   data_format = "json"
@@ -232,7 +233,7 @@ name of the plugin.
   name_override = "entropy_available"
 
   ## Data format to consume.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
   data_format = "value"
@@ -390,7 +391,7 @@ There are many more options available,
   name_suffix = "_mycollector"
 
   ## Data format to consume.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
   data_format = "graphite"
@@ -427,14 +428,54 @@ Note: Nagios Input Data Formats is only supported in `exec` input plugin.
 ```toml
 [[inputs.exec]]
   ## Commands array
-  commands = ["/usr/lib/nagios/plugins/check_load", "-w 5,6,7 -c 7,8,9"]
+  commands = ["/usr/lib/nagios/plugins/check_load -w 5,6,7 -c 7,8,9"]
 
   ## measurement name suffix (for separating different commands)
   name_suffix = "_mycollector"
 
   ## Data format to consume.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
   data_format = "nagios"
 ```
+
+# Collectd:
+
+The collectd format parses the collectd binary network protocol.  Tags are
+created for host, instance, type, and type instance.  All collectd values are
+added as float64 fields.
+
+For more information about the binary network protocol see
+[here](https://collectd.org/wiki/index.php/Binary_protocol).
+
+You can control the cryptographic settings with parser options.  Create an
+authentication file and set `collectd_auth_file` to the path of the file, then
+set the desired security level in `collectd_security_level`.
+
+Additional information including client setup can be found
+[here](https://collectd.org/wiki/index.php/Networking_introduction#Cryptographic_setup).
+
+You can also change the path to the typesdb or add additional typesdb using
+`collectd_typesdb`.
+
+#### Collectd Configuration:
+
+```toml
+[[inputs.socket_listener]]
+  service_address = "udp://127.0.0.1:25826"
+  name_prefix = "collectd_"
+
+  ## Data format to consume.
+  ## Each data format has its own unique set of configuration options, read
+  ## more about them here:
+  ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
+  data_format = "collectd"
+
+  ## Authentication file for cryptographic security levels
+  collectd_auth_file = "/etc/collectd/auth_file"
+  ## One of none (default), sign, or encrypt
+  collectd_security_level = "encrypt"
+  ## Path of to TypesDB specifications
+  collectd_typesdb = ["/usr/share/collectd/types.db"]
+```

docs/DATA_FORMATS_OUTPUT.md

@@ -36,7 +36,7 @@ config option, for example, in the `file` output plugin:
   files = ["stdout"]
 
   ## Data format to output.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
   data_format = "influx"
@@ -60,7 +60,7 @@ metrics are serialized directly into InfluxDB line-protocol.
   files = ["stdout", "/tmp/metrics.out"]
 
   ## Data format to output.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
   data_format = "influx"
@@ -104,7 +104,7 @@ tars.cpu-total.us-east-1.cpu.usage_idle 98.09 1455320690
   files = ["stdout", "/tmp/metrics.out"]
 
   ## Data format to output.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
   data_format = "graphite"
@@ -143,8 +143,18 @@ The JSON data format serialized Telegraf metrics in json format. The format is:
   files = ["stdout", "/tmp/metrics.out"]
 
   ## Data format to output.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
   data_format = "json"
+  json_timestamp_units = "1ns"
 ```
+
+By default, the timestamp that is output in JSON data format serialized Telegraf
+metrics is in seconds. The precision of this timestamp can be adjusted for any output
+by adding the optional `json_timestamp_units` parameter to the configuration for
+that output. This parameter can be used to set the timestamp units to  nanoseconds (`ns`),
+microseconds (`us` or `µs`), milliseconds (`ms`), or seconds (`s`). Note that this
+parameter will be truncated to the nearest power of 10 that, so if the `json_timestamp_units`
+are set to `15ms` the timestamps for the JSON format serialized Telegraf metrics will be
+output in hundredths of a second (`10ms`).

docs/LICENSE_OF_DEPENDENCIES.md

@@ -1,4 +1,5 @@
 # List
+- collectd.org [MIT LICENSE](https://github.com/collectd/go-collectd/blob/master/LICENSE)
 - github.com/Shopify/sarama [MIT LICENSE](https://github.com/Shopify/sarama/blob/master/MIT-LICENSE)
 - github.com/Sirupsen/logrus [MIT LICENSE](https://github.com/Sirupsen/logrus/blob/master/LICENSE)
 - github.com/armon/go-metrics [MIT LICENSE](https://github.com/armon/go-metrics/blob/master/LICENSE)
@@ -30,4 +31,3 @@
 - gopkg.in/dancannon/gorethink.v1 [APACHE LICENSE](https://github.com/dancannon/gorethink/blob/v1.1.2/LICENSE)
 - gopkg.in/mgo.v2 [BSD LICENSE](https://github.com/go-mgo/mgo/blob/v2/LICENSE)
 - golang.org/x/crypto/ [BSD LICENSE](https://github.com/golang/crypto/blob/master/LICENSE)
-

docs/PROFILING.md

@@ -0,0 +1,24 @@
+# Telegraf profiling
+
+Telegraf uses the standard package `net/http/pprof`. This package serves via its HTTP server runtime profiling data in the format expected by the pprof visualization tool.
+
+By default, the profiling is turned off.
+
+To enable profiling you need to specify address to config parameter `pprof-addr`, for example:
+
+```
+telegraf --config telegraf.conf --pprof-addr localhost:6060
+```
+
+There are several paths to get different profiling information:
+
+To look at the heap profile:
+
+`go tool pprof http://localhost:6060/debug/pprof/heap`
+
+or to look at a 30-second CPU profile:
+
+`go tool pprof http://localhost:6060/debug/pprof/profile?seconds=30`
+
+To view all available profiles, open `http://localhost:6060/debug/pprof/` in your browser.
+

etc/telegraf.conf

@@ -55,10 +55,13 @@
   ## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
   flush_jitter = "0s"
 
-  ## By default, precision will be set to the same timestamp order as the
-  ## collection interval, with the maximum being 1s.
-  ## Precision will NOT be used for service inputs, such as logparser and statsd.
-  ## Valid values are "ns", "us" (or "µs"), "ms", "s".
+  ## By default or when set to "0s", precision will be set to the same
+  ## timestamp order as the collection interval, with the maximum being 1s.
+  ##   ie, when interval = "10s", precision will be "1s"
+  ##       when interval = "250ms", precision will be "1ms"
+  ## Precision will NOT be used for service inputs. It is up to each individual
+  ## service input to set the timestamp at the appropriate precision.
+  ## Valid time units are "ns", "us" (or "µs"), "ms", "s".
   precision = ""
 
   ## Logging configuration:
@@ -81,7 +84,10 @@
 
 # Configuration for influxdb server to send metrics to
 [[outputs.influxdb]]
-  ## The full HTTP or UDP endpoint URL for your InfluxDB instance.
+  ## The HTTP or UDP URL for your InfluxDB instance.  Each item should be
+  ## of the form:
+  ##   scheme "://" host [ ":" port]
+  ##
   ## Multiple urls can be specified as part of the same cluster,
   ## this means that only ONE of the urls will be written to each interval.
   # urls = ["udp://localhost:8089"] # UDP endpoint example
@@ -89,7 +95,8 @@
   ## The target database for metrics (telegraf will create it if not exists).
   database = "telegraf" # required
 
-  ## Retention policy to write to. Empty string writes to the default rp.
+  ## Name of existing retention policy to write to.  Empty string writes to
+  ## the default retention policy.
   retention_policy = ""
   ## Write consistency (clusters only), can be: "any", "one", "quorum", "all"
   write_consistency = "any"
@@ -131,9 +138,11 @@
 #   ## AMQP exchange
 #   exchange = "telegraf"
 #   ## Auth method. PLAIN and EXTERNAL are supported
+#   ## Using EXTERNAL requires enabling the rabbitmq_auth_mechanism_ssl plugin as
+#   ## described here: https://www.rabbitmq.com/plugins.html
 #   # auth_method = "PLAIN"
 #   ## Telegraf tag to use as a routing key
-#   ##  ie, if this tag exists, it's value will be used as the routing key
+#   ##  ie, if this tag exists, its value will be used as the routing key
 #   routing_tag = "host"
 #
 #   ## InfluxDB retention policy
@@ -141,6 +150,10 @@
 #   ## InfluxDB database
 #   # database = "telegraf"
 #
+#   ## Write timeout, formatted as a string.  If not provided, will default
+#   ## to 5s. 0s means no timeout (not recommended).
+#   # timeout = "5s"
+#
 #   ## Optional SSL Config
 #   # ssl_ca = "/etc/telegraf/ca.pem"
 #   # ssl_cert = "/etc/telegraf/cert.pem"
@@ -149,7 +162,7 @@
 #   # insecure_skip_verify = false
 #
 #   ## Data format to output.
-#   ## Each data format has it's own unique set of configuration options, read
+#   ## Each data format has its own unique set of configuration options, read
 #   ## more about them here:
 #   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
 #   data_format = "influx"
@@ -193,13 +206,52 @@
 #   # no configuration
 
 
+# # Configuration for Elasticsearch to send metrics to.
+# [[outputs.elasticsearch]]
+#   ## The full HTTP endpoint URL for your Elasticsearch instance
+#   ## Multiple urls can be specified as part of the same cluster,
+#   ## this means that only ONE of the urls will be written to each interval.
+#   urls = [ "http://node1.es.example.com:9200" ] # required.
+#   ## Elasticsearch client timeout, defaults to "5s" if not set.
+#   timeout = "5s"
+#   ## Set to true to ask Elasticsearch a list of all cluster nodes,
+#   ## thus it is not necessary to list all nodes in the urls config option.
+#   enable_sniffer = false
+#   ## Set the interval to check if the Elasticsearch nodes are available
+#   ## Setting to "0s" will disable the health check (not recommended in production)
+#   health_check_interval = "10s"
+#   ## HTTP basic authentication details (eg. when using Shield)
+#   # username = "telegraf"
+#   # password = "mypassword"
+#
+#   ## Index Config
+#   ## The target index for metrics (Elasticsearch will create if it not exists).
+#   ## You can use the date specifiers below to create indexes per time frame.
+#   ## The metric timestamp will be used to decide the destination index name
+#   # %Y - year (2016)
+#   # %y - last two digits of year (00..99)
+#   # %m - month (01..12)
+#   # %d - day of month (e.g., 01)
+#   # %H - hour (00..23)
+#   index_name = "telegraf-%Y.%m.%d" # required.
+#
+#   ## Template Config
+#   ## Set to true if you want telegraf to manage its index template.
+#   ## If enabled it will create a recommended index template for telegraf indexes
+#   manage_template = true
+#   ## The template name used for telegraf indexes
+#   template_name = "telegraf"
+#   ## Set to true if you want telegraf to overwrite an existing template
+#   overwrite_template = false
+
+
 # # Send telegraf metrics to file(s)
 # [[outputs.file]]
 #   ## Files to write to, "stdout" is a specially handled file.
 #   files = ["stdout", "/tmp/metrics.out"]
 #
 #   ## Data format to output.
-#   ## Each data format has it's own unique set of configuration options, read
+#   ## Each data format has its own unique set of configuration options, read
 #   ## more about them here:
 #   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
 #   data_format = "influx"
@@ -248,7 +300,7 @@
 #   ## Kafka topic for producer messages
 #   topic = "telegraf"
 #   ## Telegraf tag to use as a routing key
-#   ##  ie, if this tag exists, it's value will be used as the routing key
+#   ##  ie, if this tag exists, its value will be used as the routing key
 #   routing_tag = "host"
 #
 #   ## CompressionCodec represents the various compression codecs recognized by
@@ -284,8 +336,12 @@
 #   ## Use SSL but skip chain & host verification
 #   # insecure_skip_verify = false
 #
+#   ## Optional SASL Config
+#   # sasl_username = "kafka"
+#   # sasl_password = "secret"
+#
 #   ## Data format to output.
-#   ## Each data format has it's own unique set of configuration options, read
+#   ## Each data format has its own unique set of configuration options, read
 #   ## more about them here:
 #   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
 #   data_format = "influx"
@@ -315,9 +371,14 @@
 #   streamname = "StreamName"
 #   ## PartitionKey as used for sharding data.
 #   partitionkey = "PartitionKey"
+#   ## If set the paritionKey will be a random UUID on every put.
+#   ## This allows for scaling across multiple shards in a stream.
+#   ## This will cause issues with ordering.
+#   use_random_partitionkey = false
+#
 #
 #   ## Data format to output.
-#   ## Each data format has it's own unique set of configuration options, read
+#   ## Each data format has its own unique set of configuration options, read
 #   ## more about them here:
 #   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
 #   data_format = "influx"
@@ -369,7 +430,7 @@
 #   # insecure_skip_verify = false
 #
 #   ## Data format to output.
-#   ## Each data format has it's own unique set of configuration options, read
+#   ## Each data format has its own unique set of configuration options, read
 #   ## more about them here:
 #   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
 #   data_format = "influx"
@@ -393,7 +454,7 @@
 #   # insecure_skip_verify = false
 #
 #   ## Data format to output.
-#   ## Each data format has it's own unique set of configuration options, read
+#   ## Each data format has its own unique set of configuration options, read
 #   ## more about them here:
 #   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
 #   data_format = "influx"
@@ -407,7 +468,7 @@
 #   topic = "telegraf"
 #
 #   ## Data format to output.
-#   ## Each data format has it's own unique set of configuration options, read
+#   ## Each data format has its own unique set of configuration options, read
 #   ## more about them here:
 #   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
 #   data_format = "influx"
@@ -443,7 +504,7 @@
 #   # expiration_interval = "60s"
 
 
-# # Configuration for Riemann server to send metrics to
+# # Configuration for the Riemann server to send metrics to
 # [[outputs.riemann]]
 #   ## The full TCP or UDP URL of the Riemann server
 #   url = "tcp://localhost:5555"
@@ -472,9 +533,12 @@
 #
 #   ## Description for Riemann event
 #   # description_text = "metrics collected from telegraf"
+#
+#   ## Riemann client write timeout, defaults to "5s" if not set.
+#   # timeout = "5s"
 
 
-# # Configuration for the legacy Riemann plugin
+# # Configuration for the Riemann server to send metrics to
 # [[outputs.riemann_legacy]]
 #   ## URL of server
 #   url = "localhost:5555"
@@ -484,6 +548,33 @@
 #   separator = " "
 
 
+# # Generic socket writer capable of handling multiple socket types.
+# [[outputs.socket_writer]]
+#   ## URL to connect to
+#   # address = "tcp://127.0.0.1:8094"
+#   # address = "tcp://example.com:http"
+#   # address = "tcp4://127.0.0.1:8094"
+#   # address = "tcp6://127.0.0.1:8094"
+#   # address = "tcp6://[2001:db8::1]:8094"
+#   # address = "udp://127.0.0.1:8094"
+#   # address = "udp4://127.0.0.1:8094"
+#   # address = "udp6://127.0.0.1:8094"
+#   # address = "unix:///tmp/telegraf.sock"
+#   # address = "unixgram:///tmp/telegraf.sock"
+#
+#   ## Period between keep alive probes.
+#   ## Only applies to TCP sockets.
+#   ## 0 disables keep alive probes.
+#   ## Defaults to the OS configuration.
+#   # keep_alive_period = "5m"
+#
+#   ## Data format to generate.
+#   ## Each data format has its own unique set of configuration options, read
+#   ## more about them here:
+#   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
+#   # data_format = "influx"
+
+
 
 ###############################################################################
 #                            PROCESSOR PLUGINS                                #
@@ -531,7 +622,7 @@
 
   ## Ignore some mountpoints by filesystem type. For example (dev)tmpfs (usually
   ## present on /run, /var/run, /dev/shm or /dev).
-  ignore_fs = ["tmpfs", "devtmpfs"]
+  ignore_fs = ["tmpfs", "devtmpfs", "devfs"]
 
 
 # Read metrics about disk IO by device
@@ -542,6 +633,23 @@
   # devices = ["sda", "sdb"]
   ## Uncomment the following line if you need disk serial numbers.
   # skip_serial_number = false
+  #
+  ## On systems which support it, device metadata can be added in the form of
+  ## tags.
+  ## Currently only Linux is supported via udev properties. You can view
+  ## available properties for a device by running:
+  ## 'udevadm info -q property -n /dev/sda'
+  # device_tags = ["ID_FS_TYPE", "ID_FS_USAGE"]
+  #
+  ## Using the same metadata source as device_tags, you can also customize the
+  ## name of the device via templates.
+  ## The 'name_templates' parameter is a list of templates to try and apply to
+  ## the device. The template may contain variables in the form of '$PROPERTY' or
+  ## '${PROPERTY}'. The first template which does not contain any variables not
+  ## present for the device is used as the device name tag.
+  ## The typical use case is for LVM volumes, to get the VG/LV name instead of
+  ## the near-meaningless DM-0 name.
+  # name_templates = ["$ID_FS_LABEL","$DM_VG_NAME/$DM_LV_NAME"]
 
 
 # Get kernel statistics from /proc/stat
@@ -658,7 +766,7 @@
 #   gather_admin_socket_stats = true
 #
 #   ## Whether to gather statistics via ceph commands
-#   gather_cluster_stats = true
+#   gather_cluster_stats = false
 
 
 # # Read specific statistics per cgroup
@@ -677,6 +785,12 @@
 #   # files = ["memory.*usage*", "memory.limit_in_bytes"]
 
 
+# # Get standard chrony metrics, requires chronyc executable.
+# [[inputs.chrony]]
+#   ## If true, chronyc tries to perform a DNS lookup for the time server.
+#   # dns_lookup = false
+
+
 # # Pull Metric Statistics from Amazon CloudWatch
 # [[inputs.cloudwatch]]
 #   ## Amazon Region
@@ -722,9 +836,10 @@
 #   namespace = "AWS/ELB"
 #
 #   ## Maximum requests per second. Note that the global default AWS rate limit is
-#   ## 10 reqs/sec, so if you define multiple namespaces, these should add up to a
-#   ## maximum of 10. Optional - default value is 10.
-#   ratelimit = 10
+#   ## 400 reqs/sec, so if you define multiple namespaces, these should add up to a
+#   ## maximum of 400. Optional - default value is 200.
+#   ## See http://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html
+#   ratelimit = 200
 #
 #   ## Metrics to Pull (optional)
 #   ## Defaults to all Metrics in Namespace if nothing is provided
@@ -738,6 +853,22 @@
 #   #    value = "p-example"
 
 
+# # Collects conntrack stats from the configured directories and files.
+# [[inputs.conntrack]]
+#    ## The following defaults would work with multiple versions of conntrack.
+#    ## Note the nf_ and ip_ filename prefixes are mutually exclusive across
+#    ## kernel versions, as are the directory locations.
+#
+#    ## Superset of filenames to look for within the conntrack dirs.
+#    ## Missing files will be ignored.
+#    files = ["ip_conntrack_count","ip_conntrack_max",
+#             "nf_conntrack_count","nf_conntrack_max"]
+#
+#    ## Directories to search within for the conntrack files above.
+#    ## Missing directrories will be ignored.
+#    dirs = ["/proc/sys/net/ipv4/netfilter","/proc/sys/net/netfilter"]
+
+
 # # Gather health check statuses from services registered in Consul
 # [[inputs.consul]]
 #   ## Most of these values defaults to the one configured on a Consul's agent level.
@@ -785,6 +916,12 @@
 #   servers = ["localhost"]
 
 
+# # Provide a native collection for dmsetup based statistics for dm-cache
+# [[inputs.dmcache]]
+#   ## Whether to report per-device stats or not
+#   per_device = true
+
+
 # # Query given DNS server and gives statistics
 # [[inputs.dns_query]]
 #   ## servers to query
@@ -821,6 +958,10 @@
 #   ## Whether to report for each container total blkio and network stats or not
 #   total = false
 #
+#   ## docker labels to include and exclude as tags.  Globs accepted.
+#   ## Note that an empty array for both will include all labels as tags
+#   docker_label_include = []
+#   docker_label_exclude = []
 
 
 # # Read statistics from one or many dovecot servers
@@ -884,7 +1025,7 @@
 #   name_suffix = "_mycollector"
 #
 #   ## Data format to consume.
-#   ## Each data format has it's own unique set of configuration options, read
+#   ## Each data format has its own unique set of configuration options, read
 #   ## more about them here:
 #   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
 #   data_format = "influx"
@@ -949,14 +1090,39 @@
 #   ## with optional port. ie localhost, 10.10.3.33:1936, etc.
 #   ## Make sure you specify the complete path to the stats endpoint
 #   ## including the protocol, ie http://10.10.3.33:1936/haproxy?stats
-#   #
+#
 #   ## If no servers are specified, then default to 127.0.0.1:1936/haproxy?stats
 #   servers = ["http://myhaproxy.com:1936/haproxy?stats"]
-#   ##
+#
 #   ## You can also use local socket with standard wildcard globbing.
 #   ## Server address not starting with 'http' will be treated as a possible
 #   ## socket, so both examples below are valid.
-#   ## servers = ["socket:/run/haproxy/admin.sock", "/run/haproxy/*.sock"]
+#   # servers = ["socket:/run/haproxy/admin.sock", "/run/haproxy/*.sock"]
+#
+#   ## By default, some of the fields are renamed from what haproxy calls them.
+#   ## Setting this option to true results in the plugin keeping the original
+#   ## field names.
+#   # keep_field_names = true
+#
+#   ## Optional SSL Config
+#   # ssl_ca = "/etc/telegraf/ca.pem"
+#   # ssl_cert = "/etc/telegraf/cert.pem"
+#   # ssl_key = "/etc/telegraf/key.pem"
+#   ## Use SSL but skip chain & host verification
+#   # insecure_skip_verify = false
+
+
+# # Monitor disks' temperatures using hddtemp
+# [[inputs.hddtemp]]
+#   ## By default, telegraf gathers temps data from all disks detected by the
+#   ## hddtemp.
+#   ##
+#   ## Only collect temps from the selected disks.
+#   ##
+#   ## A * as the device name will return the temperature values of all disks.
+#   ##
+#   # address = "127.0.0.1:7634"
+#   # devices = ["sda", "*"]
 
 
 # # HTTP/HTTPS request given an address a method and a timeout
@@ -977,6 +1143,11 @@
 #   # {'fake':'data'}
 #   # '''
 #
+#   ## Optional substring or regex match in body of the response
+#   ## response_string_match = "\"service_status\": \"up\""
+#   ## response_string_match = "ok"
+#   ## response_string_match = "\".*_status\".?:.?\"up\""
+#
 #   ## Optional SSL Config
 #   # ssl_ca = "/etc/telegraf/ca.pem"
 #   # ssl_cert = "/etc/telegraf/cert.pem"
@@ -990,7 +1161,10 @@
 #   ## NOTE This plugin only reads numerical measurements, strings and booleans
 #   ## will be ignored.
 #
-#   ## a name for the service being polled
+#   ## Name for the service being polled.  Will be appended to the name of the
+#   ## measurement e.g. httpjson_webserver_stats
+#   ##
+#   ## Deprecated (1.3.0): Use name_override, name_suffix, name_prefix instead.
 #   name = "webserver_stats"
 #
 #   ## URL of each server in the service's cluster
@@ -1010,12 +1184,14 @@
 #   #   "my_tag_2"
 #   # ]
 #
-#   ## HTTP parameters (all values must be strings)
-#   [inputs.httpjson.parameters]
-#     event_type = "cpu_spike"
-#     threshold = "0.75"
+#   ## HTTP parameters (all values must be strings).  For "GET" requests, data
+#   ## will be included in the query.  For "POST" requests, data will be included
+#   ## in the request body as "x-www-form-urlencoded".
+#   # [inputs.httpjson.parameters]
+#   #   event_type = "cpu_spike"
+#   #   threshold = "0.75"
 #
-#   ## HTTP Header parameters (all values must be strings)
+#   ## HTTP Headers (all values must be strings)
 #   # [inputs.httpjson.headers]
 #   #   X-Auth-Token = "my-xauth-token"
 #   #   apiVersion = "v1"
@@ -1050,14 +1226,44 @@
 #   # collect_memstats = true
 
 
-# # Read metrics from one or many bare metal servers
+# # This plugin gathers interrupts data from /proc/interrupts and /proc/softirqs.
+# [[inputs.interrupts]]
+#   ## To filter which IRQs to collect, make use of tagpass / tagdrop, i.e.
+#   # [inputs.interrupts.tagdrop]
+#     # irq = [ "NET_RX", "TASKLET" ]
+
+
+# # Read metrics from the bare metal servers via IPMI
 # [[inputs.ipmi_sensor]]
-#   ## specify servers via a url matching:
+#   ## optionally specify the path to the ipmitool executable
+#   # path = "/usr/bin/ipmitool"
+#   #
+#   ## optionally specify one or more servers via a url matching
 #   ##  [username[:password]@][protocol[(address)]]
 #   ##  e.g.
 #   ##    root:passwd@lan(127.0.0.1)
 #   ##
-#   servers = ["USERID:PASSW0RD@lan(192.168.1.1)"]
+#   ## if no servers are specified, local machine sensor stats will be queried
+#   ##
+#   # servers = ["USERID:PASSW0RD@lan(192.168.1.1)"]
+
+
+# # Gather packets and bytes throughput from iptables
+# [[inputs.iptables]]
+#   ## iptables require root access on most systems.
+#   ## Setting 'use_sudo' to true will make use of sudo to run iptables.
+#   ## Users must configure sudo to allow telegraf user to run iptables with no password.
+#   ## iptables can be restricted to only list command "iptables -nvL".
+#   use_sudo = false
+#   ## Setting 'use_lock' to true runs iptables with the "-w" option.
+#   ## Adjust your sudo settings appropriately if using this option ("iptables -wnvl")
+#   use_lock = false
+#   ## defines the table to monitor:
+#   table = "filter"
+#   ## defines the chains to monitor.
+#   ## NOTE: iptables rules without a comment will not be monitored.
+#   ## Read the plugin documentation for more information.
+#   chains = [ "INPUT" ]
 
 
 # # Read JMX metrics through Jolokia
@@ -1087,6 +1293,13 @@
 #   ## Includes connection time, any redirects, and reading the response body.
 #   # client_timeout = "4s"
 #
+#   ## Attribute delimiter
+#   ##
+#   ## When multiple attributes are returned for a single
+#   ## [inputs.jolokia.metrics], the field name is a concatenation of the metric
+#   ## name, and the attribute name, separated by the given delimiter.
+#   # delimiter = "_"
+#
 #   ## List of servers exposing jolokia read service
 #   [[inputs.jolokia.servers]]
 #     name = "as-server-01"
@@ -1117,6 +1330,23 @@
 #     attribute = "LoadedClassCount,UnloadedClassCount,TotalLoadedClassCount"
 
 
+# # Read Kapacitor-formatted JSON metrics from one or more HTTP endpoints
+# [[inputs.kapacitor]]
+#   ## Multiple URLs from which to read Kapacitor-formatted JSON
+#   ## Default is "http://localhost:9092/kapacitor/v1/debug/vars".
+#   urls = [
+#     "http://localhost:9092/kapacitor/v1/debug/vars"
+#   ]
+#
+#   ## Time limit for http requests
+#   timeout = "5s"
+
+
+# # Get kernel statistics from /proc/vmstat
+# [[inputs.kernel_vmstat]]
+#   # no configuration
+
+
 # # Read metrics from the kubernetes kubelet api
 # [[inputs.kubernetes]]
 #   ## URL for the kubelet
@@ -1140,6 +1370,11 @@
 #   servers = ["127.0.0.1:4021"]
 
 
+# # Provides Linux sysctl fs metrics
+# [[inputs.linux_sysctl_fs]]
+#   # no configuration
+
+
 # # Read metrics from local Lustre service on OST, MDS
 # [[inputs.lustre2]]
 #   ## An array of /proc globs to search for Lustre stats
@@ -1216,6 +1451,13 @@
 #   ##   10.0.0.1:10000, etc.
 #   servers = ["127.0.0.1:27017"]
 #   gather_perdb_stats = false
+#
+#   ## Optional SSL Config
+#   # ssl_ca = "/etc/telegraf/ca.pem"
+#   # ssl_cert = "/etc/telegraf/cert.pem"
+#   # ssl_key = "/etc/telegraf/key.pem"
+#   ## Use SSL but skip chain & host verification
+#   # insecure_skip_verify = false
 
 
 # # Read metrics from one or many mysql servers
@@ -1243,9 +1485,15 @@
 #   ## gather thread state counts from INFORMATION_SCHEMA.PROCESSLIST
 #   gather_process_list                       = true
 #   #
+#   ## gather thread state counts from INFORMATION_SCHEMA.USER_STATISTICS
+#   gather_user_statistics                    = true
+#   #
 #   ## gather auto_increment columns and max values from information schema
 #   gather_info_schema_auto_inc               = true
 #   #
+#   ## gather metrics from INFORMATION_SCHEMA.INNODB_METRICS
+#   gather_innodb_metrics                     = true
+#   #
 #   ## gather metrics from SHOW SLAVE STATUS command output
 #   gather_slave_status                       = true
 #   #
@@ -1383,7 +1631,7 @@
 #   ## NOTE: this plugin forks the ping command. You may need to set capabilities
 #   ## via setcap cap_net_raw+p /bin/ping
 #   #
-#   ## urls to ping
+#   ## List of urls to ping
 #   urls = ["www.google.com"] # required
 #   ## number of pings to send per collection (ping -c <COUNT>)
 #   # count = 1
@@ -1417,7 +1665,7 @@
 #   # ignored_databases = ["postgres", "template0", "template1"]
 #
 #   ## A list of databases to pull metrics about. If not specified, metrics for all
-#   ## databases are gathered.  Do NOT use with the 'ignore_databases' option.
+#   ## databases are gathered.  Do NOT use with the 'ignored_databases' option.
 #   # databases = ["app_production", "testing"]
 
 
@@ -1599,6 +1847,13 @@
 #   servers = ["http://localhost:8098"]
 
 
+# # Monitor sensors, requires lm-sensors package
+# [[inputs.sensors]]
+#   ## Remove numbers from field names.
+#   ## If true, a field name like 'temp1_input' will be changed to 'temp_input'.
+#   # remove_numbers = true
+
+
 # # Retrieves SNMP values from remote agents
 # [[inputs.snmp]]
 #   agents = [ "127.0.0.1:161" ]
@@ -1775,6 +2030,68 @@
 #   # ]
 
 
+# # Sysstat metrics collector
+# [[inputs.sysstat]]
+#   ## Path to the sadc command.
+#   #
+#   ## Common Defaults:
+#   ##   Debian/Ubuntu: /usr/lib/sysstat/sadc
+#   ##   Arch:          /usr/lib/sa/sadc
+#   ##   RHEL/CentOS:   /usr/lib64/sa/sadc
+#   sadc_path = "/usr/lib/sa/sadc" # required
+#   #
+#   #
+#   ## Path to the sadf command, if it is not in PATH
+#   # sadf_path = "/usr/bin/sadf"
+#   #
+#   #
+#   ## Activities is a list of activities, that are passed as argument to the
+#   ## sadc collector utility (e.g: DISK, SNMP etc...)
+#   ## The more activities that are added, the more data is collected.
+#   # activities = ["DISK"]
+#   #
+#   #
+#   ## Group metrics to measurements.
+#   ##
+#   ## If group is false each metric will be prefixed with a description
+#   ## and represents itself a measurement.
+#   ##
+#   ## If Group is true, corresponding metrics are grouped to a single measurement.
+#   # group = true
+#   #
+#   #
+#   ## Options for the sadf command. The values on the left represent the sadf
+#   ## options and the values on the right their description (wich are used for
+#   ## grouping and prefixing metrics).
+#   ##
+#   ## Run 'sar -h' or 'man sar' to find out the supported options for your
+#   ## sysstat version.
+#   [inputs.sysstat.options]
+#     -C = "cpu"
+#     -B = "paging"
+#     -b = "io"
+#     -d = "disk"             # requires DISK activity
+#     "-n ALL" = "network"
+#     "-P ALL" = "per_cpu"
+#     -q = "queue"
+#     -R = "mem"
+#     -r = "mem_util"
+#     -S = "swap_util"
+#     -u = "cpu_util"
+#     -v = "inode"
+#     -W = "swap"
+#     -w = "task"
+#   #  -H = "hugepages"        # only available for newer linux distributions
+#   #  "-I ALL" = "interrupts" # requires INT activity
+#   #
+#   #
+#   ## Device tags can be used to add additional tags for devices.
+#   ## For example the configuration below adds a tag vg with value rootvg for
+#   ## all metrics with sda devices.
+#   # [[inputs.sysstat.device_tags.sda]]
+#   #  vg = "rootvg"
+
+
 # # Inserts sine and cosine waves for demonstration purposes
 # [[inputs.trig]]
 #   ## Set the amplitude
@@ -1830,6 +2147,39 @@
 #                            SERVICE INPUT PLUGINS                            #
 ###############################################################################
 
+# # AMQP consumer plugin
+# [[inputs.amqp_consumer]]
+#   ## AMQP url
+#   url = "amqp://localhost:5672/influxdb"
+#   ## AMQP exchange
+#   exchange = "telegraf"
+#   ## AMQP queue name
+#   queue = "telegraf"
+#   ## Binding Key
+#   binding_key = "#"
+#
+#   ## Maximum number of messages server should give to the worker.
+#   prefetch_count = 50
+#
+#   ## Auth method. PLAIN and EXTERNAL are supported
+#   ## Using EXTERNAL requires enabling the rabbitmq_auth_mechanism_ssl plugin as
+#   ## described here: https://www.rabbitmq.com/plugins.html
+#   # auth_method = "PLAIN"
+#
+#   ## Optional SSL Config
+#   # ssl_ca = "/etc/telegraf/ca.pem"
+#   # ssl_cert = "/etc/telegraf/cert.pem"
+#   # ssl_key = "/etc/telegraf/key.pem"
+#   ## Use SSL but skip chain & host verification
+#   # insecure_skip_verify = false
+#
+#   ## Data format to output.
+#   ## Each data format has its own unique set of configuration options, read
+#   ## more about them here:
+#   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
+#   data_format = "influx"
+
+
 # # Influx HTTP write listener
 # [[inputs.http_listener]]
 #   ## Address and port to host HTTP listener on
@@ -1863,10 +2213,14 @@
 #   offset = "oldest"
 #
 #   ## Data format to consume.
-#   ## Each data format has it's own unique set of configuration options, read
+#   ## Each data format has its own unique set of configuration options, read
 #   ## more about them here:
 #   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
 #   data_format = "influx"
+#
+#   ## Maximum length of a message to consume, in bytes (default 0/unlimited);
+#   ## larger messages are dropped
+#   max_message_len = 65536
 
 
 # # Stream and parse log file(s).
@@ -1878,7 +2232,9 @@
 #   ##   /var/log/*/*.log    -> find all .log files with a parent dir in /var/log
 #   ##   /var/log/apache.log -> only tail the apache log file
 #   files = ["/var/log/apache/access.log"]
-#   ## Read file from beginning.
+#   ## Read files that currently exist from the beginning. Files that are created
+#   ## while telegraf is running (and that match the "files" globs) will always
+#   ## be read from the beginning.
 #   from_beginning = false
 #
 #   ## Parse logstash-style "grok" patterns:
@@ -1932,7 +2288,7 @@
 #   # insecure_skip_verify = false
 #
 #   ## Data format to consume.
-#   ## Each data format has it's own unique set of configuration options, read
+#   ## Each data format has its own unique set of configuration options, read
 #   ## more about them here:
 #   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
 #   data_format = "influx"
@@ -1955,7 +2311,7 @@
 #   # pending_bytes_limit = 67108864
 #
 #   ## Data format to consume.
-#   ## Each data format has it's own unique set of configuration options, read
+#   ## Each data format has its own unique set of configuration options, read
 #   ## more about them here:
 #   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
 #   data_format = "influx"
@@ -1970,12 +2326,50 @@
 #   max_in_flight = 100
 #
 #   ## Data format to consume.
-#   ## Each data format has it's own unique set of configuration options, read
+#   ## Each data format has its own unique set of configuration options, read
 #   ## more about them here:
 #   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
 #   data_format = "influx"
 
 
+# # Generic socket listener capable of handling multiple socket types.
+# [[inputs.socket_listener]]
+#   ## URL to listen on
+#   # service_address = "tcp://:8094"
+#   # service_address = "tcp://127.0.0.1:http"
+#   # service_address = "tcp4://:8094"
+#   # service_address = "tcp6://:8094"
+#   # service_address = "tcp6://[2001:db8::1]:8094"
+#   # service_address = "udp://:8094"
+#   # service_address = "udp4://:8094"
+#   # service_address = "udp6://:8094"
+#   # service_address = "unix:///tmp/telegraf.sock"
+#   # service_address = "unixgram:///tmp/telegraf.sock"
+#
+#   ## Maximum number of concurrent connections.
+#   ## Only applies to stream sockets (e.g. TCP).
+#   ## 0 (default) is unlimited.
+#   # max_connections = 1024
+#
+#   ## Maximum socket buffer size in bytes.
+#   ## For stream sockets, once the buffer fills up, the sender will start backing up.
+#   ## For datagram sockets, once the buffer fills up, metrics will start dropping.
+#   ## Defaults to the OS default.
+#   # read_buffer_size = 65535
+#
+#   ## Period between keep alive probes.
+#   ## Only applies to TCP sockets.
+#   ## 0 disables keep alive probes.
+#   ## Defaults to the OS configuration.
+#   # keep_alive_period = "5m"
+#
+#   ## Data format to consume.
+#   ## Each data format has its own unique set of configuration options, read
+#   ## more about them here:
+#   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
+#   # data_format = "influx"
+
+
 # # Statsd Server
 # [[inputs.statsd]]
 #   ## Address and port to host UDP listener on
@@ -2037,7 +2431,7 @@
 #   pipe = false
 #
 #   ## Data format to consume.
-#   ## Each data format has it's own unique set of configuration options, read
+#   ## Each data format has its own unique set of configuration options, read
 #   ## more about them here:
 #   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
 #   data_format = "influx"
@@ -2045,41 +2439,16 @@
 
 # # Generic TCP listener
 # [[inputs.tcp_listener]]
-#   ## Address and port to host TCP listener on
-#   # service_address = ":8094"
-#
-#   ## Number of TCP messages allowed to queue up. Once filled, the
-#   ## TCP listener will start dropping packets.
-#   # allowed_pending_messages = 10000
-#
-#   ## Maximum number of concurrent TCP connections to allow
-#   # max_tcp_connections = 250
-#
-#   ## Data format to consume.
-#   ## Each data format has it's own unique set of configuration options, read
-#   ## more about them here:
-#   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
-#   data_format = "influx"
+#   # DEPRECATED: the TCP listener plugin has been deprecated in favor of the
+#   # socket_listener plugin
+#   # see https://github.com/influxdata/telegraf/tree/master/plugins/inputs/socket_listener
 
 
 # # Generic UDP listener
 # [[inputs.udp_listener]]
-#   ## Address and port to host UDP listener on
-#   # service_address = ":8092"
-#
-#   ## Number of UDP messages allowed to queue up. Once filled, the
-#   ## UDP listener will start dropping packets.
-#   # allowed_pending_messages = 10000
-#
-#   ## Set the buffer size of the UDP connection outside of OS default (in bytes)
-#   ## If set to 0, take OS default
-#   udp_buffer_size = 16777216
-#
-#   ## Data format to consume.
-#   ## Each data format has it's own unique set of configuration options, read
-#   ## more about them here:
-#   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
-#   data_format = "influx"
+#   # DEPRECATED: the TCP listener plugin has been deprecated in favor of the
+#   # socket_listener plugin
+#   # see https://github.com/influxdata/telegraf/tree/master/plugins/inputs/socket_listener
 
 
 # # A Webhooks Event collector
@@ -2092,10 +2461,14 @@
 #
 #   [inputs.webhooks.github]
 #     path = "/github"
+#     # secret = ""
 #
 #   [inputs.webhooks.mandrill]
 #     path = "/mandrill"
 #
 #   [inputs.webhooks.rollbar]
 #     path = "/rollbar"
+#
+#   [inputs.webhooks.papertrail]
+#     path = "/papertrail"
 

internal/buffer/buffer.go

@@ -45,9 +45,11 @@ func (b *Buffer) Add(metrics ...telegraf.Metric) {
 		select {
 		case b.buf <- metrics[i]:
 		default:
+			b.mu.Lock()
 			MetricsDropped.Incr(1)
 			<-b.buf
 			b.buf <- metrics[i]
+			b.mu.Unlock()
 		}
 	}
 }

internal/config/config.go

@@ -6,6 +6,7 @@ import (
 	"fmt"
 	"io/ioutil"
 	"log"
+	"math"
 	"os"
 	"path/filepath"
 	"regexp"
@@ -84,8 +85,8 @@ type AgentConfig struct {
 	//     ie, if Interval=10s then always collect on :00, :10, :20, etc.
 	RoundInterval bool
 
-	// By default, precision will be set to the same timestamp order as the
-	// collection interval, with the maximum being 1s.
+	// By default or when set to "0s", precision will be set to the same
+	// timestamp order as the collection interval, with the maximum being 1s.
 	//   ie, when interval = "10s", precision will be "1s"
 	//       when interval = "250ms", precision will be "1ms"
 	// Precision will NOT be used for service inputs. It is up to each individual
@@ -229,10 +230,13 @@ var header = `# Telegraf Configuration
   ## ie, a jitter of 5s and interval 10s means flushes will happen every 10-15s
   flush_jitter = "0s"
 
-  ## By default, precision will be set to the same timestamp order as the
-  ## collection interval, with the maximum being 1s.
-  ## Precision will NOT be used for service inputs, such as logparser and statsd.
-  ## Valid values are "ns", "us" (or "µs"), "ms", "s".
+  ## By default or when set to "0s", precision will be set to the same
+  ## timestamp order as the collection interval, with the maximum being 1s.
+  ##   ie, when interval = "10s", precision will be "1s"
+  ##       when interval = "250ms", precision will be "1ms"
+  ## Precision will NOT be used for service inputs. It is up to each individual
+  ## service input to set the timestamp at the appropriate precision.
+  ## Valid time units are "ns", "us" (or "µs"), "ms", "s".
   precision = ""
 
   ## Logging configuration:
@@ -1229,6 +1233,34 @@ func buildParser(name string, tbl *ast.Table) (parsers.Parser, error) {
 		}
 	}
 
+	if node, ok := tbl.Fields["collectd_auth_file"]; ok {
+		if kv, ok := node.(*ast.KeyValue); ok {
+			if str, ok := kv.Value.(*ast.String); ok {
+				c.CollectdAuthFile = str.Value
+			}
+		}
+	}
+
+	if node, ok := tbl.Fields["collectd_security_level"]; ok {
+		if kv, ok := node.(*ast.KeyValue); ok {
+			if str, ok := kv.Value.(*ast.String); ok {
+				c.CollectdSecurityLevel = str.Value
+			}
+		}
+	}
+
+	if node, ok := tbl.Fields["collectd_typesdb"]; ok {
+		if kv, ok := node.(*ast.KeyValue); ok {
+			if ary, ok := kv.Value.(*ast.Array); ok {
+				for _, elem := range ary.Value {
+					if str, ok := elem.(*ast.String); ok {
+						c.CollectdTypesDB = append(c.CollectdTypesDB, str.Value)
+					}
+				}
+			}
+		}
+	}
+
 	c.MetricName = name
 
 	delete(tbl.Fields, "data_format")
@@ -1236,6 +1268,9 @@ func buildParser(name string, tbl *ast.Table) (parsers.Parser, error) {
 	delete(tbl.Fields, "templates")
 	delete(tbl.Fields, "tag_keys")
 	delete(tbl.Fields, "data_type")
+	delete(tbl.Fields, "collectd_auth_file")
+	delete(tbl.Fields, "collectd_security_level")
+	delete(tbl.Fields, "collectd_typesdb")
 
 	return parsers.NewParser(c)
 }
@@ -1244,7 +1279,7 @@ func buildParser(name string, tbl *ast.Table) (parsers.Parser, error) {
 // a serializers.Serializer object, and creates it, which can then be added onto
 // an Output object.
 func buildSerializer(name string, tbl *ast.Table) (serializers.Serializer, error) {
-	c := &serializers.Config{}
+	c := &serializers.Config{TimestampUnits: time.Duration(1 * time.Second)}
 
 	if node, ok := tbl.Fields["data_format"]; ok {
 		if kv, ok := node.(*ast.KeyValue); ok {
@@ -1274,9 +1309,26 @@ func buildSerializer(name string, tbl *ast.Table) (serializers.Serializer, error
 		}
 	}
 
+	if node, ok := tbl.Fields["json_timestamp_units"]; ok {
+		if kv, ok := node.(*ast.KeyValue); ok {
+			if str, ok := kv.Value.(*ast.String); ok {
+				timestampVal, err := time.ParseDuration(str.Value)
+				if err != nil {
+					return nil, fmt.Errorf("Unable to parse json_timestamp_units as a duration, %s", err)
+				}
+				// now that we have a duration, truncate it to the nearest
+				// power of ten (just in case)
+				nearest_exponent := int64(math.Log10(float64(timestampVal.Nanoseconds())))
+				new_nanoseconds := int64(math.Pow(10.0, float64(nearest_exponent)))
+				c.TimestampUnits = time.Duration(new_nanoseconds)
+			}
+		}
+	}
+
 	delete(tbl.Fields, "data_format")
 	delete(tbl.Fields, "prefix")
 	delete(tbl.Fields, "template")
+	delete(tbl.Fields, "json_timestamp_units")
 	return serializers.NewSerializer(c)
 }
 

internal/config/testdata/telegraf-agent.toml

@@ -60,7 +60,7 @@
   # Kafka topic for producer messages
   topic = "telegraf"
   # Telegraf tag to use as a routing key
-  #  ie, if this tag exists, it's value will be used as the routing key
+  #  ie, if this tag exists, its value will be used as the routing key
   routing_tag = "host"
 
 

internal/errchan/errchan.go

@@ -1,37 +0,0 @@
-package errchan
-
-import (
-	"fmt"
-	"strings"
-)
-
-type ErrChan struct {
-	C chan error
-}
-
-// New returns an error channel of max length 'n'
-// errors can be sent to the ErrChan.C channel, and will be returned when
-// ErrChan.Error() is called.
-func New(n int) *ErrChan {
-	return &ErrChan{
-		C: make(chan error, n),
-	}
-}
-
-// Error closes the ErrChan.C channel and returns an error if there are any
-// non-nil errors, otherwise returns nil.
-func (e *ErrChan) Error() error {
-	close(e.C)
-
-	var out string
-	for err := range e.C {
-		if err != nil {
-			out += "[" + err.Error() + "], "
-		}
-	}
-
-	if out != "" {
-		return fmt.Errorf("Errors encountered: " + strings.TrimRight(out, ", "))
-	}
-	return nil
-}

logger/logger.go

@@ -4,11 +4,14 @@ import (
 	"io"
 	"log"
 	"os"
+	"regexp"
 	"time"
 
 	"github.com/influxdata/wlog"
 )
 
+var prefixRegex = regexp.MustCompile("^[DIWE]!")
+
 // newTelegrafWriter returns a logging-wrapped writer.
 func newTelegrafWriter(w io.Writer) io.Writer {
 	return &telegrafLog{
@@ -21,7 +24,13 @@ type telegrafLog struct {
 }
 
 func (t *telegrafLog) Write(b []byte) (n int, err error) {
-	return t.writer.Write(append([]byte(time.Now().UTC().Format(time.RFC3339)+" "), b...))
+	var line []byte
+	if !prefixRegex.Match(b) {
+		line = append([]byte(time.Now().UTC().Format(time.RFC3339)+" I! "), b...)
+	} else {
+		line = append([]byte(time.Now().UTC().Format(time.RFC3339)+" "), b...)
+	}
+	return t.writer.Write(line)
 }
 
 // SetupLogging configures the logging output.

logger/logger_test.go

@@ -51,6 +51,19 @@ func TestErrorWriteLogToFile(t *testing.T) {
 	assert.Equal(t, f[19:], []byte("Z E! TEST\n"))
 }
 
+func TestAddDefaultLogLevel(t *testing.T) {
+	tmpfile, err := ioutil.TempFile("", "")
+	assert.NoError(t, err)
+	defer func() { os.Remove(tmpfile.Name()) }()
+
+	SetupLogging(true, false, tmpfile.Name())
+	log.Printf("TEST")
+
+	f, err := ioutil.ReadFile(tmpfile.Name())
+	assert.NoError(t, err)
+	assert.Equal(t, f[19:], []byte("Z I! TEST\n"))
+}
+
 func BenchmarkTelegrafLogWrite(b *testing.B) {
 	var msg = []byte("test")
 	var buf bytes.Buffer

metric/parse.go

@@ -4,6 +4,7 @@ import (
 	"bytes"
 	"errors"
 	"fmt"
+	"strconv"
 	"time"
 
 	"github.com/influxdata/telegraf"
@@ -40,10 +41,18 @@ const (
 )
 
 func Parse(buf []byte) ([]telegraf.Metric, error) {
-	return ParseWithDefaultTime(buf, time.Now())
+	return ParseWithDefaultTimePrecision(buf, time.Now(), "")
 }
 
 func ParseWithDefaultTime(buf []byte, t time.Time) ([]telegraf.Metric, error) {
+	return ParseWithDefaultTimePrecision(buf, t, "")
+}
+
+func ParseWithDefaultTimePrecision(
+	buf []byte,
+	t time.Time,
+	precision string,
+) ([]telegraf.Metric, error) {
 	if len(buf) == 0 {
 		return []telegraf.Metric{}, nil
 	}
@@ -63,7 +72,7 @@ func ParseWithDefaultTime(buf []byte, t time.Time) ([]telegraf.Metric, error) {
 			continue
 		}
 
-		m, err := parseMetric(buf[i:i+j], t)
+		m, err := parseMetric(buf[i:i+j], t, precision)
 		if err != nil {
 			i += j + 1 // increment i past the previous newline
 			errStr += " " + err.Error()
@@ -80,7 +89,10 @@ func ParseWithDefaultTime(buf []byte, t time.Time) ([]telegraf.Metric, error) {
 	return metrics, nil
 }
 
-func parseMetric(buf []byte, defaultTime time.Time) (telegraf.Metric, error) {
+func parseMetric(buf []byte,
+	defaultTime time.Time,
+	precision string,
+) (telegraf.Metric, error) {
 	var dTime string
 	// scan the first block which is measurement[,tag1=value1,tag2=value=2...]
 	pos, key, err := scanKey(buf, 0)
@@ -114,9 +126,23 @@ func parseMetric(buf []byte, defaultTime time.Time) (telegraf.Metric, error) {
 		return nil, err
 	}
 
+	// apply precision multiplier
+	var nsec int64
+	multiplier := getPrecisionMultiplier(precision)
+	if multiplier > 1 {
+		tsint, err := parseIntBytes(ts, 10, 64)
+		if err != nil {
+			return nil, err
+		}
+
+		nsec := multiplier * tsint
+		ts = []byte(strconv.FormatInt(nsec, 10))
+	}
+
 	m := &metric{
 		fields: fields,
 		t:      ts,
+		nsec:   nsec,
 	}
 
 	// parse out the measurement name
@@ -628,3 +654,21 @@ func makeError(reason string, buf []byte, i int) error {
 	return fmt.Errorf("metric parsing error, reason: [%s], buffer: [%s], index: [%d]",
 		reason, buf, i)
 }
+
+// getPrecisionMultiplier will return a multiplier for the precision specified.
+func getPrecisionMultiplier(precision string) int64 {
+	d := time.Nanosecond
+	switch precision {
+	case "u":
+		d = time.Microsecond
+	case "ms":
+		d = time.Millisecond
+	case "s":
+		d = time.Second
+	case "m":
+		d = time.Minute
+	case "h":
+		d = time.Hour
+	}
+	return int64(d)
+}

metric/parse_test.go

@@ -364,6 +364,27 @@ func TestParseNegativeTimestamps(t *testing.T) {
 	}
 }
 
+func TestParsePrecision(t *testing.T) {
+	for _, tt := range []struct {
+		line      string
+		precision string
+		expected  int64
+	}{
+		{"test v=42 1491847420", "s", 1491847420000000000},
+		{"test v=42 1491847420123", "ms", 1491847420123000000},
+		{"test v=42 1491847420123456", "u", 1491847420123456000},
+		{"test v=42 1491847420123456789", "ns", 1491847420123456789},
+
+		{"test v=42 1491847420123456789", "1s", 1491847420123456789},
+		{"test v=42 1491847420123456789", "asdf", 1491847420123456789},
+	} {
+		metrics, err := ParseWithDefaultTimePrecision(
+			[]byte(tt.line+"\n"), time.Now(), tt.precision)
+		assert.NoError(t, err, tt)
+		assert.Equal(t, tt.expected, metrics[0].UnixNano())
+	}
+}
+
 func TestParseMaxKeyLength(t *testing.T) {
 	key := ""
 	for {

plugins/inputs/EXAMPLE_README.md

@@ -1,6 +1,8 @@
 # Example Input Plugin
 
-The example plugin gathers metrics about example things
+The example plugin gathers metrics about example things.  This description
+explains at a high level what the plugin does and provides links to where
+additional information can be found.
 
 ### Configuration:
 
@@ -12,7 +14,8 @@ The example plugin gathers metrics about example things
 
 ### Measurements & Fields:
 
-<optional description>
+Here you should add an optional description and links to where the user can
+get more information about the measurements.
 
 - measurement1
     - field1 (type, unit)
@@ -27,11 +30,14 @@ The example plugin gathers metrics about example things
     - tag2
 - measurement2 has the following tags:
     - tag3
-    
+
 ### Sample Queries:
 
-These are some useful queries (to generate dashboards or other) to run against data from this plugin:
+This section should contain some useful InfluxDB queries that can be used to
+get started with the plugin or to generate dashboards.  For each query listed,
+describe at a high level what data is returned.
 
+Get the max, mean, and min for the measurement in the last hour:
 ```
 SELECT max(field1), mean(field1), min(field1) FROM measurement1 WHERE tag1=bar AND time > now() - 1h GROUP BY tag
 ```
@@ -39,7 +45,7 @@ SELECT max(field1), mean(field1), min(field1) FROM measurement1 WHERE tag1=bar A
 ### Example Output:
 
 ```
-$ ./telegraf -config telegraf.conf -input-filter example -test
+$ telegraf -input-filter example -test
 measurement1,tag1=foo,tag2=bar field1=1i,field2=2.1 1453831884664956455
 measurement2,tag1=foo,tag2=bar,tag3=baz field3=1i 1453831884664956455
 ```

plugins/inputs/aerospike/aerospike.go

@@ -10,7 +10,6 @@ import (
 	"time"
 
 	"github.com/influxdata/telegraf"
-	"github.com/influxdata/telegraf/internal/errchan"
 	"github.com/influxdata/telegraf/plugins/inputs"
 
 	as "github.com/aerospike/aerospike-client-go"
@@ -41,17 +40,16 @@ func (a *Aerospike) Gather(acc telegraf.Accumulator) error {
 	}
 
 	var wg sync.WaitGroup
-	errChan := errchan.New(len(a.Servers))
 	wg.Add(len(a.Servers))
 	for _, server := range a.Servers {
 		go func(serv string) {
 			defer wg.Done()
-			errChan.C <- a.gatherServer(serv, acc)
+			acc.AddError(a.gatherServer(serv, acc))
 		}(server)
 	}
 
 	wg.Wait()
-	return errChan.Error()
+	return nil
 }
 
 func (a *Aerospike) gatherServer(hostport string, acc telegraf.Accumulator) error {

plugins/inputs/aerospike/aerospike_test.go

@@ -19,7 +19,7 @@ func TestAerospikeStatistics(t *testing.T) {
 
 	var acc testutil.Accumulator
 
-	err := a.Gather(&acc)
+	err := acc.GatherError(a.Gather)
 	require.NoError(t, err)
 
 	assert.True(t, acc.HasMeasurement("aerospike_node"))
@@ -41,8 +41,7 @@ func TestAerospikeStatisticsPartialErr(t *testing.T) {
 
 	var acc testutil.Accumulator
 
-	err := a.Gather(&acc)
-	require.Error(t, err)
+	require.Error(t, acc.GatherError(a.Gather))
 
 	assert.True(t, acc.HasMeasurement("aerospike_node"))
 	assert.True(t, acc.HasMeasurement("aerospike_namespace"))

plugins/inputs/all/all.go

@@ -15,6 +15,7 @@ import (
 	_ "github.com/influxdata/telegraf/plugins/inputs/couchbase"
 	_ "github.com/influxdata/telegraf/plugins/inputs/couchdb"
 	_ "github.com/influxdata/telegraf/plugins/inputs/disque"
+	_ "github.com/influxdata/telegraf/plugins/inputs/dmcache"
 	_ "github.com/influxdata/telegraf/plugins/inputs/dns_query"
 	_ "github.com/influxdata/telegraf/plugins/inputs/docker"
 	_ "github.com/influxdata/telegraf/plugins/inputs/dovecot"
@@ -29,10 +30,12 @@ import (
 	_ "github.com/influxdata/telegraf/plugins/inputs/httpjson"
 	_ "github.com/influxdata/telegraf/plugins/inputs/influxdb"
 	_ "github.com/influxdata/telegraf/plugins/inputs/internal"
+	_ "github.com/influxdata/telegraf/plugins/inputs/interrupts"
 	_ "github.com/influxdata/telegraf/plugins/inputs/ipmi_sensor"
 	_ "github.com/influxdata/telegraf/plugins/inputs/iptables"
 	_ "github.com/influxdata/telegraf/plugins/inputs/jolokia"
 	_ "github.com/influxdata/telegraf/plugins/inputs/kafka_consumer"
+	_ "github.com/influxdata/telegraf/plugins/inputs/kapacitor"
 	_ "github.com/influxdata/telegraf/plugins/inputs/kubernetes"
 	_ "github.com/influxdata/telegraf/plugins/inputs/leofs"
 	_ "github.com/influxdata/telegraf/plugins/inputs/logparser"

plugins/inputs/amqp_consumer/README.md

@@ -40,7 +40,7 @@ The following defaults are known to work with RabbitMQ:
   # insecure_skip_verify = false
 
   ## Data format to output.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
   data_format = "influx"

plugins/inputs/amqp_consumer/amqp_consumer.go

@@ -86,7 +86,7 @@ func (a *AMQPConsumer) SampleConfig() string {
   # insecure_skip_verify = false
 
   ## Data format to output.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_OUTPUT.md
   data_format = "influx"

plugins/inputs/apache/apache.go

@@ -8,6 +8,7 @@ import (
 	"net/url"
 	"strconv"
 	"strings"
+	"sync"
 	"time"
 
 	"github.com/influxdata/telegraf"
@@ -65,28 +66,23 @@ func (n *Apache) Gather(acc telegraf.Accumulator) error {
 		n.ResponseTimeout.Duration = time.Second * 5
 	}
 
-	var outerr error
-	var errch = make(chan error)
-
+	var wg sync.WaitGroup
+	wg.Add(len(n.Urls))
 	for _, u := range n.Urls {
 		addr, err := url.Parse(u)
 		if err != nil {
-			return fmt.Errorf("Unable to parse address '%s': %s", u, err)
+			acc.AddError(fmt.Errorf("Unable to parse address '%s': %s", u, err))
+			continue
 		}
 
 		go func(addr *url.URL) {
-			errch <- n.gatherUrl(addr, acc)
+			defer wg.Done()
+			acc.AddError(n.gatherUrl(addr, acc))
 		}(addr)
 	}
 
-	// Drain channel, waiting for all requests to finish and save last error.
-	for range n.Urls {
-		if err := <-errch; err != nil {
-			outerr = err
-		}
-	}
-
-	return outerr
+	wg.Wait()
+	return nil
 }
 
 func (n *Apache) gatherUrl(addr *url.URL, acc telegraf.Accumulator) error {

plugins/inputs/apache/apache_test.go

@@ -41,7 +41,7 @@ func TestHTTPApache(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	err := a.Gather(&acc)
+	err := acc.GatherError(a.Gather)
 	require.NoError(t, err)
 
 	fields := map[string]interface{}{

plugins/inputs/cassandra/cassandra.go

@@ -7,7 +7,6 @@ import (
 	"github.com/influxdata/telegraf"
 	"github.com/influxdata/telegraf/plugins/inputs"
 	"io/ioutil"
-	"log"
 	"net/http"
 	"net/url"
 	"strings"
@@ -123,8 +122,8 @@ func (j javaMetric) addTagsFields(out map[string]interface{}) {
 		}
 		j.acc.AddFields(tokens["class"]+tokens["type"], fields, tags)
 	} else {
-		fmt.Printf("Missing key 'value' in '%s' output response\n%v\n",
-			j.metric, out)
+		j.acc.AddError(fmt.Errorf("Missing key 'value' in '%s' output response\n%v\n",
+			j.metric, out))
 	}
 }
 
@@ -155,8 +154,8 @@ func (c cassandraMetric) addTagsFields(out map[string]interface{}) {
 				addCassandraMetric(k, c, v.(map[string]interface{}))
 			}
 		} else {
-			fmt.Printf("Missing key 'value' in '%s' output response\n%v\n",
-				c.metric, out)
+			c.acc.AddError(fmt.Errorf("Missing key 'value' in '%s' output response\n%v\n",
+				c.metric, out))
 			return
 		}
 	} else {
@@ -164,8 +163,8 @@ func (c cassandraMetric) addTagsFields(out map[string]interface{}) {
 			addCassandraMetric(r.(map[string]interface{})["mbean"].(string),
 				c, values.(map[string]interface{}))
 		} else {
-			fmt.Printf("Missing key 'value' in '%s' output response\n%v\n",
-				c.metric, out)
+			c.acc.AddError(fmt.Errorf("Missing key 'value' in '%s' output response\n%v\n",
+				c.metric, out))
 			return
 		}
 	}
@@ -274,8 +273,8 @@ func (c *Cassandra) Gather(acc telegraf.Accumulator) error {
 				m = newCassandraMetric(serverTokens["host"], metric, acc)
 			} else {
 				// unsupported metric type
-				log.Printf("I! Unsupported Cassandra metric [%s], skipping",
-					metric)
+				acc.AddError(fmt.Errorf("E! Unsupported Cassandra metric [%s], skipping",
+					metric))
 				continue
 			}
 
@@ -283,7 +282,8 @@ func (c *Cassandra) Gather(acc telegraf.Accumulator) error {
 			requestUrl, err := url.Parse("http://" + serverTokens["host"] + ":" +
 				serverTokens["port"] + context + metric)
 			if err != nil {
-				return err
+				acc.AddError(err)
+				continue
 			}
 			if serverTokens["user"] != "" && serverTokens["passwd"] != "" {
 				requestUrl.User = url.UserPassword(serverTokens["user"],
@@ -291,8 +291,12 @@ func (c *Cassandra) Gather(acc telegraf.Accumulator) error {
 			}
 
 			out, err := c.getAttr(requestUrl)
+			if err != nil {
+				acc.AddError(err)
+				continue
+			}
 			if out["status"] != 200.0 {
-				fmt.Printf("URL returned with status %v\n", out["status"])
+				acc.AddError(fmt.Errorf("URL returned with status %v\n", out["status"]))
 				continue
 			}
 			m.addTagsFields(out)

plugins/inputs/cassandra/cassandra_test.go

@@ -151,7 +151,7 @@ func TestHttpJsonJavaMultiValue(t *testing.T) {
 
 	var acc testutil.Accumulator
 	acc.SetDebug(true)
-	err := cassandra.Gather(&acc)
+	err := acc.GatherError(cassandra.Gather)
 
 	assert.Nil(t, err)
 	assert.Equal(t, 2, len(acc.Metrics))
@@ -180,7 +180,7 @@ func TestHttpJsonJavaMultiType(t *testing.T) {
 
 	var acc testutil.Accumulator
 	acc.SetDebug(true)
-	err := cassandra.Gather(&acc)
+	err := acc.GatherError(cassandra.Gather)
 
 	assert.Nil(t, err)
 	assert.Equal(t, 2, len(acc.Metrics))
@@ -197,16 +197,17 @@ func TestHttpJsonJavaMultiType(t *testing.T) {
 }
 
 // Test that the proper values are ignored or collected
-func TestHttpJsonOn404(t *testing.T) {
+func TestHttp404(t *testing.T) {
 
-	jolokia := genJolokiaClientStub(validJavaMultiValueJSON, 404, Servers,
+	jolokia := genJolokiaClientStub(invalidJSON, 404, Servers,
 		[]string{HeapMetric})
 
 	var acc testutil.Accumulator
-	err := jolokia.Gather(&acc)
+	err := acc.GatherError(jolokia.Gather)
 
-	assert.Nil(t, err)
+	assert.Error(t, err)
 	assert.Equal(t, 0, len(acc.Metrics))
+	assert.Contains(t, err.Error(), "has status code 404")
 }
 
 // Test that the proper values are ignored or collected for class=Cassandra
@@ -214,7 +215,7 @@ func TestHttpJsonCassandraMultiValue(t *testing.T) {
 	cassandra := genJolokiaClientStub(validCassandraMultiValueJSON, 200, Servers, []string{ReadLatencyMetric})
 
 	var acc testutil.Accumulator
-	err := cassandra.Gather(&acc)
+	err := acc.GatherError(cassandra.Gather)
 
 	assert.Nil(t, err)
 	assert.Equal(t, 1, len(acc.Metrics))
@@ -246,7 +247,7 @@ func TestHttpJsonCassandraNestedMultiValue(t *testing.T) {
 
 	var acc testutil.Accumulator
 	acc.SetDebug(true)
-	err := cassandra.Gather(&acc)
+	err := acc.GatherError(cassandra.Gather)
 
 	assert.Nil(t, err)
 	assert.Equal(t, 2, len(acc.Metrics))

plugins/inputs/ceph/ceph.go

@@ -101,12 +101,12 @@ func (c *Ceph) gatherAdminSocketStats(acc telegraf.Accumulator) error {
 	for _, s := range sockets {
 		dump, err := perfDump(c.CephBinary, s)
 		if err != nil {
-			log.Printf("E! error reading from socket '%s': %v", s.socket, err)
+			acc.AddError(fmt.Errorf("E! error reading from socket '%s': %v", s.socket, err))
 			continue
 		}
 		data, err := parseDump(dump)
 		if err != nil {
-			log.Printf("E! error parsing dump from socket '%s': %v", s.socket, err)
+			acc.AddError(fmt.Errorf("E! error parsing dump from socket '%s': %v", s.socket, err))
 			continue
 		}
 		for tag, metrics := range data {

plugins/inputs/cgroup/cgroup_linux.go

@@ -22,10 +22,11 @@ func (g *CGroup) Gather(acc telegraf.Accumulator) error {
 
 	for dir := range list {
 		if dir.err != nil {
-			return dir.err
+			acc.AddError(dir.err)
+			continue
 		}
 		if err := g.gatherDir(dir.path, acc); err != nil {
-			return err
+			acc.AddError(err)
 		}
 	}
 

plugins/inputs/cgroup/cgroup_test.go

@@ -24,7 +24,7 @@ var cg1 = &CGroup{
 func TestCgroupStatistics_1(t *testing.T) {
 	var acc testutil.Accumulator
 
-	err := cg1.Gather(&acc)
+	err := acc.GatherError(cg1.Gather)
 	require.NoError(t, err)
 
 	tags := map[string]string{
@@ -56,7 +56,7 @@ var cg2 = &CGroup{
 func TestCgroupStatistics_2(t *testing.T) {
 	var acc testutil.Accumulator
 
-	err := cg2.Gather(&acc)
+	err := acc.GatherError(cg2.Gather)
 	require.NoError(t, err)
 
 	tags := map[string]string{
@@ -81,7 +81,7 @@ var cg3 = &CGroup{
 func TestCgroupStatistics_3(t *testing.T) {
 	var acc testutil.Accumulator
 
-	err := cg3.Gather(&acc)
+	err := acc.GatherError(cg3.Gather)
 	require.NoError(t, err)
 
 	tags := map[string]string{
@@ -108,7 +108,7 @@ var cg4 = &CGroup{
 func TestCgroupStatistics_4(t *testing.T) {
 	var acc testutil.Accumulator
 
-	err := cg4.Gather(&acc)
+	err := acc.GatherError(cg4.Gather)
 	require.NoError(t, err)
 
 	tags := map[string]string{
@@ -140,7 +140,7 @@ var cg5 = &CGroup{
 func TestCgroupStatistics_5(t *testing.T) {
 	var acc testutil.Accumulator
 
-	err := cg5.Gather(&acc)
+	err := acc.GatherError(cg5.Gather)
 	require.NoError(t, err)
 
 	tags := map[string]string{
@@ -167,7 +167,7 @@ var cg6 = &CGroup{
 func TestCgroupStatistics_6(t *testing.T) {
 	var acc testutil.Accumulator
 
-	err := cg6.Gather(&acc)
+	err := acc.GatherError(cg6.Gather)
 	require.NoError(t, err)
 
 	tags := map[string]string{

plugins/inputs/cloudwatch/README.md

@@ -42,9 +42,10 @@ API endpoint. In the following order the plugin will attempt to authenticate.
   namespace = "AWS/ELB"
 
   ## Maximum requests per second. Note that the global default AWS rate limit is
-  ## 10 reqs/sec, so if you define multiple namespaces, these should add up to a
-  ## maximum of 10. Optional - default value is 10.
-  ratelimit = 10
+  ## 400 reqs/sec, so if you define multiple namespaces, these should add up to a
+  ## maximum of 400. Optional - default value is 200.
+  ## See http://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html
+  ratelimit = 200
 
   ## Metrics to Pull (optional)
   ## Defaults to all Metrics in Namespace if nothing is provided

plugins/inputs/cloudwatch/cloudwatch.go

@@ -13,7 +13,6 @@ import (
 	"github.com/influxdata/telegraf"
 	"github.com/influxdata/telegraf/internal"
 	internalaws "github.com/influxdata/telegraf/internal/config/aws"
-	"github.com/influxdata/telegraf/internal/errchan"
 	"github.com/influxdata/telegraf/internal/limiter"
 	"github.com/influxdata/telegraf/plugins/inputs"
 )
@@ -105,9 +104,10 @@ func (c *CloudWatch) SampleConfig() string {
   namespace = "AWS/ELB"
 
   ## Maximum requests per second. Note that the global default AWS rate limit is
-  ## 10 reqs/sec, so if you define multiple namespaces, these should add up to a
-  ## maximum of 10. Optional - default value is 10.
-  ratelimit = 10
+  ## 400 reqs/sec, so if you define multiple namespaces, these should add up to a
+  ## maximum of 400. Optional - default value is 200.
+  ## See http://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_limits.html
+  ratelimit = 200
 
   ## Metrics to Pull (optional)
   ## Defaults to all Metrics in Namespace if nothing is provided
@@ -185,8 +185,6 @@ func (c *CloudWatch) Gather(acc telegraf.Accumulator) error {
 	if err != nil {
 		return err
 	}
-	metricCount := len(metrics)
-	errChan := errchan.New(metricCount)
 
 	now := time.Now()
 
@@ -201,12 +199,12 @@ func (c *CloudWatch) Gather(acc telegraf.Accumulator) error {
 		<-lmtr.C
 		go func(inm *cloudwatch.Metric) {
 			defer wg.Done()
-			c.gatherMetric(acc, inm, now, errChan.C)
+			acc.AddError(c.gatherMetric(acc, inm, now))
 		}(m)
 	}
 	wg.Wait()
 
-	return errChan.Error()
+	return nil
 }
 
 func init() {
@@ -214,7 +212,7 @@ func init() {
 		ttl, _ := time.ParseDuration("1hr")
 		return &CloudWatch{
 			CacheTTL:  internal.Duration{Duration: ttl},
-			RateLimit: 10,
+			RateLimit: 200,
 		}
 	})
 }
@@ -284,13 +282,11 @@ func (c *CloudWatch) gatherMetric(
 	acc telegraf.Accumulator,
 	metric *cloudwatch.Metric,
 	now time.Time,
-	errChan chan error,
-) {
+) error {
 	params := c.getStatisticsInput(metric, now)
 	resp, err := c.client.GetMetricStatistics(params)
 	if err != nil {
-		errChan <- err
-		return
+		return err
 	}
 
 	for _, point := range resp.Datapoints {
@@ -325,7 +321,7 @@ func (c *CloudWatch) gatherMetric(
 		acc.AddFields(formatMeasurement(c.Namespace), fields, tags, *point.Timestamp)
 	}
 
-	errChan <- nil
+	return nil
 }
 
 /*

plugins/inputs/cloudwatch/cloudwatch_test.go

@@ -58,13 +58,13 @@ func TestGather(t *testing.T) {
 		Namespace: "AWS/ELB",
 		Delay:     internalDuration,
 		Period:    internalDuration,
-		RateLimit: 10,
+		RateLimit: 200,
 	}
 
 	var acc testutil.Accumulator
 	c.client = &mockGatherCloudWatchClient{}
 
-	c.Gather(&acc)
+	acc.GatherError(c.Gather)
 
 	fields := map[string]interface{}{}
 	fields["latency_minimum"] = 0.1
@@ -146,7 +146,7 @@ func TestSelectMetrics(t *testing.T) {
 		Namespace: "AWS/ELB",
 		Delay:     internalDuration,
 		Period:    internalDuration,
-		RateLimit: 10,
+		RateLimit: 200,
 		Metrics: []*Metric{
 			&Metric{
 				MetricNames: []string{"Latency", "RequestCount"},
@@ -207,14 +207,13 @@ func TestGenerateStatisticsInputParams(t *testing.T) {
 }
 
 func TestMetricsCacheTimeout(t *testing.T) {
-	ttl, _ := time.ParseDuration("5ms")
 	cache := &MetricCache{
 		Metrics: []*cloudwatch.Metric{},
 		Fetched: time.Now(),
-		TTL:     ttl,
+		TTL:     time.Minute,
 	}
 
 	assert.True(t, cache.IsValid())
-	time.Sleep(ttl)
+	cache.Fetched = time.Now().Add(-time.Minute)
 	assert.False(t, cache.IsValid())
 }

plugins/inputs/conntrack/conntrack.go

@@ -11,7 +11,6 @@ import (
 
 	"github.com/influxdata/telegraf"
 	"github.com/influxdata/telegraf/plugins/inputs"
-	"log"
 	"path/filepath"
 )
 
@@ -93,15 +92,15 @@ func (c *Conntrack) Gather(acc telegraf.Accumulator) error {
 
 			contents, err := ioutil.ReadFile(fName)
 			if err != nil {
-				log.Printf("E! failed to read file '%s': %v", fName, err)
+				acc.AddError(fmt.Errorf("E! failed to read file '%s': %v", fName, err))
 				continue
 			}
 
 			v := strings.TrimSpace(string(contents))
 			fields[metricKey], err = strconv.ParseFloat(v, 64)
 			if err != nil {
-				log.Printf("E! failed to parse metric, expected number but "+
-					" found '%s': %v", v, err)
+				acc.AddError(fmt.Errorf("E! failed to parse metric, expected number but "+
+					" found '%s': %v", v, err))
 			}
 		}
 	}

plugins/inputs/couchbase/couchbase.go

@@ -42,19 +42,17 @@ func (r *Couchbase) Gather(acc telegraf.Accumulator) error {
 
 	var wg sync.WaitGroup
 
-	var outerr error
-
 	for _, serv := range r.Servers {
 		wg.Add(1)
 		go func(serv string) {
 			defer wg.Done()
-			outerr = r.gatherServer(serv, acc, nil)
+			acc.AddError(r.gatherServer(serv, acc, nil))
 		}(serv)
 	}
 
 	wg.Wait()
 
-	return outerr
+	return nil
 }
 
 func (r *Couchbase) gatherServer(addr string, acc telegraf.Accumulator, pool *couchbase.Pool) error {

plugins/inputs/couchdb/couchdb.go

@@ -2,13 +2,11 @@ package couchdb
 
 import (
 	"encoding/json"
-	"errors"
 	"fmt"
 	"github.com/influxdata/telegraf"
 	"github.com/influxdata/telegraf/plugins/inputs"
 	"net/http"
 	"reflect"
-	"strings"
 	"sync"
 	"time"
 )
@@ -83,34 +81,20 @@ func (*CouchDB) SampleConfig() string {
 }
 
 func (c *CouchDB) Gather(accumulator telegraf.Accumulator) error {
-	errorChannel := make(chan error, len(c.HOSTs))
 	var wg sync.WaitGroup
 	for _, u := range c.HOSTs {
 		wg.Add(1)
 		go func(host string) {
 			defer wg.Done()
 			if err := c.fetchAndInsertData(accumulator, host); err != nil {
-				errorChannel <- fmt.Errorf("[host=%s]: %s", host, err)
+				accumulator.AddError(fmt.Errorf("[host=%s]: %s", host, err))
 			}
 		}(u)
 	}
 
 	wg.Wait()
-	close(errorChannel)
-
-	// If there weren't any errors, we can return nil now.
-	if len(errorChannel) == 0 {
-		return nil
-	}
-
-	// There were errors, so join them all together as one big error.
-	errorStrings := make([]string, 0, len(errorChannel))
-	for err := range errorChannel {
-		errorStrings = append(errorStrings, err.Error())
-	}
-
-	return errors.New(strings.Join(errorStrings, "\n"))
 
+	return nil
 }
 
 var tr = &http.Transport{

plugins/inputs/couchdb/couchdb_test.go

@@ -316,5 +316,5 @@ func TestBasic(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	require.NoError(t, plugin.Gather(&acc))
+	require.NoError(t, acc.GatherError(plugin.Gather))
 }

plugins/inputs/disque/disque.go

@@ -75,12 +75,11 @@ func (g *Disque) Gather(acc telegraf.Accumulator) error {
 
 	var wg sync.WaitGroup
 
-	var outerr error
-
 	for _, serv := range g.Servers {
 		u, err := url.Parse(serv)
 		if err != nil {
-			return fmt.Errorf("Unable to parse to address '%s': %s", serv, err)
+			acc.AddError(fmt.Errorf("Unable to parse to address '%s': %s", serv, err))
+			continue
 		} else if u.Scheme == "" {
 			// fallback to simple string based address (i.e. "10.0.0.1:10000")
 			u.Scheme = "tcp"
@@ -90,13 +89,13 @@ func (g *Disque) Gather(acc telegraf.Accumulator) error {
 		wg.Add(1)
 		go func(serv string) {
 			defer wg.Done()
-			outerr = g.gatherServer(u, acc)
+			acc.AddError(g.gatherServer(u, acc))
 		}(serv)
 	}
 
 	wg.Wait()
 
-	return outerr
+	return nil
 }
 
 const defaultPort = "7711"

plugins/inputs/disque/disque_test.go

@@ -51,7 +51,7 @@ func TestDisqueGeneratesMetrics(t *testing.T) {
 
 	var acc testutil.Accumulator
 
-	err = r.Gather(&acc)
+	err = acc.GatherError(r.Gather)
 	require.NoError(t, err)
 
 	fields := map[string]interface{}{
@@ -117,7 +117,7 @@ func TestDisqueCanPullStatsFromMultipleServers(t *testing.T) {
 
 	var acc testutil.Accumulator
 
-	err = r.Gather(&acc)
+	err = acc.GatherError(r.Gather)
 	require.NoError(t, err)
 
 	fields := map[string]interface{}{

plugins/inputs/dmcache/README.md

@@ -0,0 +1,47 @@
+# DMCache Input Plugin
+
+This plugin provide a native collection for dmsetup based statistics for dm-cache.
+
+This plugin requires sudo, that is why you should setup and be sure that the telegraf is able to execute sudo without a password.
+
+`sudo /sbin/dmsetup status --target cache` is the full command that telegraf will run for debugging purposes.
+
+### Configuration
+
+```toml
+[[inputs.dmcache]]
+  ## Whether to report per-device stats or not
+  per_device = true
+```
+
+### Measurements & Fields:
+
+- dmcache
+    - length
+    - target
+    - metadata_blocksize
+    - metadata_used
+    - metadata_total
+    - cache_blocksize
+    - cache_used
+    - cache_total
+    - read_hits
+    - read_misses
+    - write_hits
+    - write_misses
+    - demotions
+    - promotions
+    - dirty
+
+### Tags:
+
+- All measurements have the following tags:
+    - device
+
+### Example Output:
+
+```
+$ ./telegraf --test --config /etc/telegraf/telegraf.conf --input-filter dmcache
+* Plugin: inputs.dmcache, Collection 1
+> dmcache,device=example cache_blocksize=0i,read_hits=995134034411520i,read_misses=916807089127424i,write_hits=195107267543040i,metadata_used=12861440i,write_misses=563725346013184i,promotions=3265223720960i,dirty=0i,metadata_blocksize=0i,cache_used=1099511627776ii,cache_total=0i,length=0i,metadata_total=1073741824i,demotions=3265223720960i 1491482035000000000
+```

plugins/inputs/dmcache/dmcache.go

@@ -0,0 +1,33 @@
+package dmcache
+
+import (
+	"github.com/influxdata/telegraf"
+	"github.com/influxdata/telegraf/plugins/inputs"
+)
+
+type DMCache struct {
+	PerDevice        bool `toml:"per_device"`
+	getCurrentStatus func() ([]string, error)
+}
+
+var sampleConfig = `
+  ## Whether to report per-device stats or not
+  per_device = true
+`
+
+func (c *DMCache) SampleConfig() string {
+	return sampleConfig
+}
+
+func (c *DMCache) Description() string {
+	return "Provide a native collection for dmsetup based statistics for dm-cache"
+}
+
+func init() {
+	inputs.Add("dmcache", func() telegraf.Input {
+		return &DMCache{
+			PerDevice:        true,
+			getCurrentStatus: dmSetupStatus,
+		}
+	})
+}

plugins/inputs/dmcache/dmcache_linux.go

@@ -0,0 +1,190 @@
+// +build linux
+
+package dmcache
+
+import (
+	"os/exec"
+	"strconv"
+	"strings"
+
+	"errors"
+
+	"github.com/influxdata/telegraf"
+)
+
+const metricName = "dmcache"
+
+type cacheStatus struct {
+	device            string
+	length            int
+	target            string
+	metadataBlocksize int
+	metadataUsed      int
+	metadataTotal     int
+	cacheBlocksize    int
+	cacheUsed         int
+	cacheTotal        int
+	readHits          int
+	readMisses        int
+	writeHits         int
+	writeMisses       int
+	demotions         int
+	promotions        int
+	dirty             int
+}
+
+func (c *DMCache) Gather(acc telegraf.Accumulator) error {
+	outputLines, err := c.getCurrentStatus()
+	if err != nil {
+		return err
+	}
+
+	totalStatus := cacheStatus{}
+
+	for _, s := range outputLines {
+		status, err := parseDMSetupStatus(s)
+		if err != nil {
+			return err
+		}
+
+		if c.PerDevice {
+			tags := map[string]string{"device": status.device}
+			acc.AddFields(metricName, toFields(status), tags)
+		}
+		aggregateStats(&totalStatus, status)
+	}
+
+	acc.AddFields(metricName, toFields(totalStatus), map[string]string{"device": "all"})
+
+	return nil
+}
+
+func parseDMSetupStatus(line string) (cacheStatus, error) {
+	var err error
+	parseError := errors.New("Output from dmsetup could not be parsed")
+	status := cacheStatus{}
+	values := strings.Fields(line)
+	if len(values) < 15 {
+		return cacheStatus{}, parseError
+	}
+
+	status.device = strings.TrimRight(values[0], ":")
+	status.length, err = strconv.Atoi(values[2])
+	if err != nil {
+		return cacheStatus{}, err
+	}
+	status.target = values[3]
+	status.metadataBlocksize, err = strconv.Atoi(values[4])
+	if err != nil {
+		return cacheStatus{}, err
+	}
+	metadata := strings.Split(values[5], "/")
+	if len(metadata) != 2 {
+		return cacheStatus{}, parseError
+	}
+	status.metadataUsed, err = strconv.Atoi(metadata[0])
+	if err != nil {
+		return cacheStatus{}, err
+	}
+	status.metadataTotal, err = strconv.Atoi(metadata[1])
+	if err != nil {
+		return cacheStatus{}, err
+	}
+	status.cacheBlocksize, err = strconv.Atoi(values[6])
+	if err != nil {
+		return cacheStatus{}, err
+	}
+	cache := strings.Split(values[7], "/")
+	if len(cache) != 2 {
+		return cacheStatus{}, parseError
+	}
+	status.cacheUsed, err = strconv.Atoi(cache[0])
+	if err != nil {
+		return cacheStatus{}, err
+	}
+	status.cacheTotal, err = strconv.Atoi(cache[1])
+	if err != nil {
+		return cacheStatus{}, err
+	}
+	status.readHits, err = strconv.Atoi(values[8])
+	if err != nil {
+		return cacheStatus{}, err
+	}
+	status.readMisses, err = strconv.Atoi(values[9])
+	if err != nil {
+		return cacheStatus{}, err
+	}
+	status.writeHits, err = strconv.Atoi(values[10])
+	if err != nil {
+		return cacheStatus{}, err
+	}
+	status.writeMisses, err = strconv.Atoi(values[11])
+	if err != nil {
+		return cacheStatus{}, err
+	}
+	status.demotions, err = strconv.Atoi(values[12])
+	if err != nil {
+		return cacheStatus{}, err
+	}
+	status.promotions, err = strconv.Atoi(values[13])
+	if err != nil {
+		return cacheStatus{}, err
+	}
+	status.dirty, err = strconv.Atoi(values[14])
+	if err != nil {
+		return cacheStatus{}, err
+	}
+
+	return status, nil
+}
+
+func aggregateStats(totalStatus *cacheStatus, status cacheStatus) {
+	totalStatus.length += status.length
+	totalStatus.metadataBlocksize += status.metadataBlocksize
+	totalStatus.metadataUsed += status.metadataUsed
+	totalStatus.metadataTotal += status.metadataTotal
+	totalStatus.cacheBlocksize += status.cacheBlocksize
+	totalStatus.cacheUsed += status.cacheUsed
+	totalStatus.cacheTotal += status.cacheTotal
+	totalStatus.readHits += status.readHits
+	totalStatus.readMisses += status.readMisses
+	totalStatus.writeHits += status.writeHits
+	totalStatus.writeMisses += status.writeMisses
+	totalStatus.demotions += status.demotions
+	totalStatus.promotions += status.promotions
+	totalStatus.dirty += status.dirty
+}
+
+func toFields(status cacheStatus) map[string]interface{} {
+	fields := make(map[string]interface{})
+	fields["length"] = status.length
+	fields["metadata_blocksize"] = status.metadataBlocksize
+	fields["metadata_used"] = status.metadataUsed
+	fields["metadata_total"] = status.metadataTotal
+	fields["cache_blocksize"] = status.cacheBlocksize
+	fields["cache_used"] = status.cacheUsed
+	fields["cache_total"] = status.cacheTotal
+	fields["read_hits"] = status.readHits
+	fields["read_misses"] = status.readMisses
+	fields["write_hits"] = status.writeHits
+	fields["write_misses"] = status.writeMisses
+	fields["demotions"] = status.demotions
+	fields["promotions"] = status.promotions
+	fields["dirty"] = status.dirty
+	return fields
+}
+
+func dmSetupStatus() ([]string, error) {
+	out, err := exec.Command("/bin/sh", "-c", "sudo /sbin/dmsetup status --target cache").Output()
+	if err != nil {
+		return nil, err
+	}
+	if string(out) == "No devices found\n" {
+		return []string{}, nil
+	}
+
+	outString := strings.TrimRight(string(out), "\n")
+	status := strings.Split(outString, "\n")
+
+	return status, nil
+}

plugins/inputs/dmcache/dmcache_notlinux.go

@@ -0,0 +1,15 @@
+// +build !linux
+
+package dmcache
+
+import (
+	"github.com/influxdata/telegraf"
+)
+
+func (c *DMCache) Gather(acc telegraf.Accumulator) error {
+	return nil
+}
+
+func dmSetupStatus() ([]string, error) {
+	return []string{}, nil
+}

plugins/inputs/dmcache/dmcache_test.go

@@ -0,0 +1,169 @@
+package dmcache
+
+import (
+	"errors"
+	"testing"
+
+	"github.com/influxdata/telegraf/testutil"
+	"github.com/stretchr/testify/require"
+)
+
+var (
+	measurement              = "dmcache"
+	badFormatOutput          = []string{"cs-1: 0 4883791872 cache 8 1018/1501122 512 7/464962 139 352643 "}
+	good2DevicesFormatOutput = []string{
+		"cs-1: 0 4883791872 cache 8 1018/1501122 512 7/464962 139 352643 15 46 0 7 0 1 writeback 2 migration_threshold 2048 mq 10 random_threshold 4 sequential_threshold 512 discard_promote_adjustment 1 read_promote_adjustment 4 write_promote_adjustment 8",
+		"cs-2: 0 4294967296 cache 8 72352/1310720 128 26/24327168 2409 286 265 524682 0 0 0 1 writethrough 2 migration_threshold 2048 mq 10 random_threshold 4 sequential_threshold 512 discard_promote_adjustment 1 read_promote_adjustment 4 write_promote_adjustment 8",
+	}
+)
+
+func TestPerDeviceGoodOutput(t *testing.T) {
+	var acc testutil.Accumulator
+	var plugin = &DMCache{
+		PerDevice: true,
+		getCurrentStatus: func() ([]string, error) {
+			return good2DevicesFormatOutput, nil
+		},
+	}
+
+	err := plugin.Gather(&acc)
+	require.NoError(t, err)
+
+	tags1 := map[string]string{
+		"device": "cs-1",
+	}
+	fields1 := map[string]interface{}{
+		"length":             4883791872,
+		"metadata_blocksize": 8,
+		"metadata_used":      1018,
+		"metadata_total":     1501122,
+		"cache_blocksize":    512,
+		"cache_used":         7,
+		"cache_total":        464962,
+		"read_hits":          139,
+		"read_misses":        352643,
+		"write_hits":         15,
+		"write_misses":       46,
+		"demotions":          0,
+		"promotions":         7,
+		"dirty":              0,
+	}
+	acc.AssertContainsTaggedFields(t, measurement, fields1, tags1)
+
+	tags2 := map[string]string{
+		"device": "cs-2",
+	}
+	fields2 := map[string]interface{}{
+		"length":             4294967296,
+		"metadata_blocksize": 8,
+		"metadata_used":      72352,
+		"metadata_total":     1310720,
+		"cache_blocksize":    128,
+		"cache_used":         26,
+		"cache_total":        24327168,
+		"read_hits":          2409,
+		"read_misses":        286,
+		"write_hits":         265,
+		"write_misses":       524682,
+		"demotions":          0,
+		"promotions":         0,
+		"dirty":              0,
+	}
+	acc.AssertContainsTaggedFields(t, measurement, fields2, tags2)
+
+	tags3 := map[string]string{
+		"device": "all",
+	}
+
+	fields3 := map[string]interface{}{
+		"length":             9178759168,
+		"metadata_blocksize": 16,
+		"metadata_used":      73370,
+		"metadata_total":     2811842,
+		"cache_blocksize":    640,
+		"cache_used":         33,
+		"cache_total":        24792130,
+		"read_hits":          2548,
+		"read_misses":        352929,
+		"write_hits":         280,
+		"write_misses":       524728,
+		"demotions":          0,
+		"promotions":         7,
+		"dirty":              0,
+	}
+	acc.AssertContainsTaggedFields(t, measurement, fields3, tags3)
+}
+
+func TestNotPerDeviceGoodOutput(t *testing.T) {
+	var acc testutil.Accumulator
+	var plugin = &DMCache{
+		PerDevice: false,
+		getCurrentStatus: func() ([]string, error) {
+			return good2DevicesFormatOutput, nil
+		},
+	}
+
+	err := plugin.Gather(&acc)
+	require.NoError(t, err)
+
+	tags := map[string]string{
+		"device": "all",
+	}
+
+	fields := map[string]interface{}{
+		"length":             9178759168,
+		"metadata_blocksize": 16,
+		"metadata_used":      73370,
+		"metadata_total":     2811842,
+		"cache_blocksize":    640,
+		"cache_used":         33,
+		"cache_total":        24792130,
+		"read_hits":          2548,
+		"read_misses":        352929,
+		"write_hits":         280,
+		"write_misses":       524728,
+		"demotions":          0,
+		"promotions":         7,
+		"dirty":              0,
+	}
+	acc.AssertContainsTaggedFields(t, measurement, fields, tags)
+}
+
+func TestNoDevicesOutput(t *testing.T) {
+	var acc testutil.Accumulator
+	var plugin = &DMCache{
+		PerDevice: true,
+		getCurrentStatus: func() ([]string, error) {
+			return []string{}, nil
+		},
+	}
+
+	err := plugin.Gather(&acc)
+	require.NoError(t, err)
+}
+
+func TestErrorDuringGettingStatus(t *testing.T) {
+	var acc testutil.Accumulator
+	var plugin = &DMCache{
+		PerDevice: true,
+		getCurrentStatus: func() ([]string, error) {
+			return nil, errors.New("dmsetup doesn't exist")
+		},
+	}
+
+	err := plugin.Gather(&acc)
+	require.Error(t, err)
+}
+
+func TestBadFormatOfStatus(t *testing.T) {
+	var acc testutil.Accumulator
+	var plugin = &DMCache{
+		PerDevice: true,
+		getCurrentStatus: func() ([]string, error) {
+			return badFormatOutput, nil
+		},
+	}
+
+	err := plugin.Gather(&acc)
+	require.Error(t, err)
+}

plugins/inputs/dns_query/dns_query.go

@@ -9,7 +9,6 @@ import (
 	"time"
 
 	"github.com/influxdata/telegraf"
-	"github.com/influxdata/telegraf/internal/errchan"
 	"github.com/influxdata/telegraf/plugins/inputs"
 )
 
@@ -58,11 +57,10 @@ func (d *DnsQuery) Description() string {
 func (d *DnsQuery) Gather(acc telegraf.Accumulator) error {
 	d.setDefaultValues()
 
-	errChan := errchan.New(len(d.Domains) * len(d.Servers))
 	for _, domain := range d.Domains {
 		for _, server := range d.Servers {
 			dnsQueryTime, err := d.getDnsQueryTime(domain, server)
-			errChan.C <- err
+			acc.AddError(err)
 			tags := map[string]string{
 				"server":      server,
 				"domain":      domain,
@@ -74,7 +72,7 @@ func (d *DnsQuery) Gather(acc telegraf.Accumulator) error {
 		}
 	}
 
-	return errChan.Error()
+	return nil
 }
 
 func (d *DnsQuery) setDefaultValues() {

plugins/inputs/dns_query/dns_query_test.go

@@ -24,7 +24,7 @@ func TestGathering(t *testing.T) {
 	}
 	var acc testutil.Accumulator
 
-	err := dnsConfig.Gather(&acc)
+	err := acc.GatherError(dnsConfig.Gather)
 	assert.NoError(t, err)
 	metric, ok := acc.Get("dns_query")
 	require.True(t, ok)
@@ -44,7 +44,7 @@ func TestGatheringMxRecord(t *testing.T) {
 	var acc testutil.Accumulator
 	dnsConfig.RecordType = "MX"
 
-	err := dnsConfig.Gather(&acc)
+	err := acc.GatherError(dnsConfig.Gather)
 	assert.NoError(t, err)
 	metric, ok := acc.Get("dns_query")
 	require.True(t, ok)
@@ -70,7 +70,7 @@ func TestGatheringRootDomain(t *testing.T) {
 	}
 	fields := map[string]interface{}{}
 
-	err := dnsConfig.Gather(&acc)
+	err := acc.GatherError(dnsConfig.Gather)
 	assert.NoError(t, err)
 	metric, ok := acc.Get("dns_query")
 	require.True(t, ok)
@@ -96,7 +96,7 @@ func TestMetricContainsServerAndDomainAndRecordTypeTags(t *testing.T) {
 	}
 	fields := map[string]interface{}{}
 
-	err := dnsConfig.Gather(&acc)
+	err := acc.GatherError(dnsConfig.Gather)
 	assert.NoError(t, err)
 	metric, ok := acc.Get("dns_query")
 	require.True(t, ok)
@@ -121,7 +121,7 @@ func TestGatheringTimeout(t *testing.T) {
 
 	channel := make(chan error, 1)
 	go func() {
-		channel <- dnsConfig.Gather(&acc)
+		channel <- acc.GatherError(dnsConfig.Gather)
 	}()
 	select {
 	case res := <-channel:

plugins/inputs/docker/README.md

@@ -30,6 +30,12 @@ for the stat structure can be found
   perdevice = true
   ## Whether to report for each container total blkio and network stats or not
   total = false
+  
+  ## docker labels to include and exclude as tags.  Globs accepted.
+  ## Note that an empty array for both will include all labels as tags
+  docker_label_include = []
+  docker_label_exclude = []
+  
 ```
 
 ### Measurements & Fields:
@@ -130,30 +136,32 @@ based on the availability of per-cpu stats on your system.
 
 
 ### Tags:
-
+#### Docker Engine tags
 - docker (memory_total)
     - unit=bytes
+    - engine_host
 - docker (pool_blocksize)
     - unit=bytes
+    - engine_host
 - docker_data
     - unit=bytes
+    - engine_host
 - docker_metadata
     - unit=bytes
+    - engine_host
 
+#### Docker Container tags
+- Tags on all containers:
+    - engine_host
+    - container_image
+    - container_name
+    - container_version
 - docker_container_mem specific:
-    - container_image
-    - container_name
 - docker_container_cpu specific:
-    - container_image
-    - container_name
     - cpu
 - docker_container_net specific:
-    - container_image
-    - container_name
     - network
 - docker_container_blkio specific:
-    - container_image
-    - container_name
     - device
 
 ### Example Output:

plugins/inputs/docker/docker.go

@@ -5,7 +5,6 @@ import (
 	"encoding/json"
 	"fmt"
 	"io"
-	"log"
 	"regexp"
 	"strconv"
 	"strings"
@@ -14,24 +13,34 @@ import (
 
 	"github.com/docker/docker/api/types"
 	"github.com/docker/docker/client"
-
 	"github.com/influxdata/telegraf"
+	"github.com/influxdata/telegraf/filter"
 	"github.com/influxdata/telegraf/internal"
 	"github.com/influxdata/telegraf/plugins/inputs"
 )
 
+type DockerLabelFilter struct {
+	labelInclude filter.Filter
+	labelExclude filter.Filter
+}
+
 // Docker object
 type Docker struct {
 	Endpoint       string
 	ContainerNames []string
 	Timeout        internal.Duration
-	PerDevice      bool `toml:"perdevice"`
-	Total          bool `toml:"total"`
+	PerDevice      bool     `toml:"perdevice"`
+	Total          bool     `toml:"total"`
+	LabelInclude   []string `toml:"docker_label_include"`
+	LabelExclude   []string `toml:"docker_label_exclude"`
+
+	LabelFilter DockerLabelFilter
 
 	client      *client.Client
 	engine_host string
 
-	testing bool
+	testing             bool
+	labelFiltersCreated bool
 }
 
 // infoWrapper wraps client.Client.List for testing.
@@ -99,6 +108,10 @@ var sampleConfig = `
   ## Whether to report for each container total blkio and network stats or not
   total = false
 
+  ## docker labels to include and exclude as tags.  Globs accepted.
+  ## Note that an empty array for both will include all labels as tags
+  docker_label_include = []
+  docker_label_exclude = []
 `
 
 // Description returns input description
@@ -133,11 +146,19 @@ func (d *Docker) Gather(acc telegraf.Accumulator) error {
 		}
 		d.client = c
 	}
+	// Create label filters if not already created
+	if !d.labelFiltersCreated {
+		err := d.createLabelFilters()
+		if err != nil {
+			return err
+		}
+		d.labelFiltersCreated = true
+	}
 
 	// Get daemon info
 	err := d.gatherInfo(acc)
 	if err != nil {
-		fmt.Println(err.Error())
+		acc.AddError(err)
 	}
 
 	// List containers
@@ -157,8 +178,8 @@ func (d *Docker) Gather(acc telegraf.Accumulator) error {
 			defer wg.Done()
 			err := d.gatherContainer(c, acc)
 			if err != nil {
-				log.Printf("E! Error gathering container %s stats: %s\n",
-					c.Names, err.Error())
+				acc.AddError(fmt.Errorf("E! Error gathering container %s stats: %s\n",
+					c.Names, err.Error()))
 			}
 		}(container)
 	}
@@ -293,7 +314,11 @@ func (d *Docker) gatherContainer(
 
 	// Add labels to tags
 	for k, label := range container.Labels {
-		tags[k] = label
+		if len(d.LabelInclude) == 0 || d.LabelFilter.labelInclude.Match(k) {
+			if len(d.LabelExclude) == 0 || !d.LabelFilter.labelExclude.Match(k) {
+				tags[k] = label
+			}
+		}
 	}
 
 	gatherContainerStats(v, acc, tags, container.ID, d.PerDevice, d.Total)
@@ -599,11 +624,32 @@ func parseSize(sizeStr string) (int64, error) {
 	return int64(size), nil
 }
 
+func (d *Docker) createLabelFilters() error {
+	if len(d.LabelInclude) != 0 && d.LabelFilter.labelInclude == nil {
+		var err error
+		d.LabelFilter.labelInclude, err = filter.Compile(d.LabelInclude)
+		if err != nil {
+			return err
+		}
+	}
+
+	if len(d.LabelExclude) != 0 && d.LabelFilter.labelExclude == nil {
+		var err error
+		d.LabelFilter.labelExclude, err = filter.Compile(d.LabelExclude)
+		if err != nil {
+			return err
+		}
+	}
+
+	return nil
+}
+
 func init() {
 	inputs.Add("docker", func() telegraf.Input {
 		return &Docker{
-			PerDevice: true,
-			Timeout:   internal.Duration{Duration: time.Second * 5},
+			PerDevice:           true,
+			Timeout:             internal.Duration{Duration: time.Second * 5},
+			labelFiltersCreated: false,
 		}
 	})
 }

plugins/inputs/docker/docker_test.go

@@ -244,6 +244,57 @@ func testStats() *types.StatsJSON {
 	return stats
 }
 
+var gatherLabelsTests = []struct {
+	include     []string
+	exclude     []string
+	expected    []string
+	notexpected []string
+}{
+	{[]string{}, []string{}, []string{"label1", "label2"}, []string{}},
+	{[]string{"*"}, []string{}, []string{"label1", "label2"}, []string{}},
+	{[]string{"lab*"}, []string{}, []string{"label1", "label2"}, []string{}},
+	{[]string{"label1"}, []string{}, []string{"label1"}, []string{"label2"}},
+	{[]string{"label1*"}, []string{}, []string{"label1"}, []string{"label2"}},
+	{[]string{}, []string{"*"}, []string{}, []string{"label1", "label2"}},
+	{[]string{}, []string{"lab*"}, []string{}, []string{"label1", "label2"}},
+	{[]string{}, []string{"label1"}, []string{"label2"}, []string{"label1"}},
+	{[]string{"*"}, []string{"*"}, []string{}, []string{"label1", "label2"}},
+}
+
+func TestDockerGatherLabels(t *testing.T) {
+	for _, tt := range gatherLabelsTests {
+		var acc testutil.Accumulator
+		d := Docker{
+			client:  nil,
+			testing: true,
+		}
+
+		for _, label := range tt.include {
+			d.LabelInclude = append(d.LabelInclude, label)
+		}
+		for _, label := range tt.exclude {
+			d.LabelExclude = append(d.LabelExclude, label)
+		}
+
+		err := d.Gather(&acc)
+		require.NoError(t, err)
+
+		for _, label := range tt.expected {
+			if !acc.HasTag("docker_container_cpu", label) {
+				t.Errorf("Didn't get expected label of %s.  Test was:  Include: %s  Exclude %s",
+					label, tt.include, tt.exclude)
+			}
+		}
+
+		for _, label := range tt.notexpected {
+			if acc.HasTag("docker_container_cpu", label) {
+				t.Errorf("Got unexpected label of %s.  Test was:  Include: %s  Exclude %s",
+					label, tt.include, tt.exclude)
+			}
+		}
+	}
+}
+
 func TestDockerGatherInfo(t *testing.T) {
 	var acc testutil.Accumulator
 	d := Docker{
@@ -251,7 +302,7 @@ func TestDockerGatherInfo(t *testing.T) {
 		testing: true,
 	}
 
-	err := d.Gather(&acc)
+	err := acc.GatherError(d.Gather)
 	require.NoError(t, err)
 
 	acc.AssertContainsTaggedFields(t,
@@ -294,6 +345,8 @@ func TestDockerGatherInfo(t *testing.T) {
 			"cpu":               "cpu3",
 			"container_version": "v2.2.2",
 			"engine_host":       "absol",
+			"label1":            "test_value_1",
+			"label2":            "test_value_2",
 		},
 	)
 	acc.AssertContainsTaggedFields(t,
@@ -340,6 +393,8 @@ func TestDockerGatherInfo(t *testing.T) {
 			"container_name":    "etcd2",
 			"container_image":   "quay.io:4443/coreos/etcd",
 			"container_version": "v2.2.2",
+			"label1":            "test_value_1",
+			"label2":            "test_value_2",
 		},
 	)
 

plugins/inputs/docker/fake_client.go

@@ -92,6 +92,10 @@ func (d FakeDockerClient) ContainerList(octx context.Context, options types.Cont
 				IP:          "0.0.0.0",
 			},
 		},
+		Labels: map[string]string{
+			"label1": "test_value_1",
+			"label2": "test_value_2",
+		},
 		SizeRw:     0,
 		SizeRootFs: 0,
 	}
@@ -125,6 +129,10 @@ func (d FakeDockerClient) ContainerList(octx context.Context, options types.Cont
 				IP:          "0.0.0.0",
 			},
 		},
+		Labels: map[string]string{
+			"label1": "test_value_1",
+			"label2": "test_value_2",
+		},
 		SizeRw:     0,
 		SizeRootFs: 0,
 	}

plugins/inputs/dovecot/dovecot.go

@@ -12,7 +12,6 @@ import (
 	"time"
 
 	"github.com/influxdata/telegraf"
-	"github.com/influxdata/telegraf/internal/errchan"
 	"github.com/influxdata/telegraf/plugins/inputs"
 )
 
@@ -66,19 +65,18 @@ func (d *Dovecot) Gather(acc telegraf.Accumulator) error {
 	}
 
 	var wg sync.WaitGroup
-	errChan := errchan.New(len(d.Servers) * len(d.Filters))
 	for _, server := range d.Servers {
 		for _, filter := range d.Filters {
 			wg.Add(1)
 			go func(s string, f string) {
 				defer wg.Done()
-				errChan.C <- d.gatherServer(s, acc, d.Type, f)
+				acc.AddError(d.gatherServer(s, acc, d.Type, f))
 			}(server, filter)
 		}
 	}
 
 	wg.Wait()
-	return errChan.Error()
+	return nil
 }
 
 func (d *Dovecot) gatherServer(addr string, acc telegraf.Accumulator, qtype string, filter string) error {

plugins/inputs/elasticsearch/elasticsearch.go

@@ -10,7 +10,6 @@ import (
 
 	"github.com/influxdata/telegraf"
 	"github.com/influxdata/telegraf/internal"
-	"github.com/influxdata/telegraf/internal/errchan"
 	"github.com/influxdata/telegraf/plugins/inputs"
 	jsonparser "github.com/influxdata/telegraf/plugins/parsers/json"
 	"io/ioutil"
@@ -153,7 +152,6 @@ func (e *Elasticsearch) Gather(acc telegraf.Accumulator) error {
 		e.client = client
 	}
 
-	errChan := errchan.New(len(e.Servers) * 3)
 	var wg sync.WaitGroup
 	wg.Add(len(e.Servers))
 
@@ -176,24 +174,21 @@ func (e *Elasticsearch) Gather(acc telegraf.Accumulator) error {
 
 			// Always gather node states
 			if err := e.gatherNodeStats(url, acc); err != nil {
-				err = fmt.Errorf(mask.ReplaceAllString(err.Error(), "http(s)://XXX:XXX@"))
-				errChan.C <- err
+				acc.AddError(fmt.Errorf(mask.ReplaceAllString(err.Error(), "http(s)://XXX:XXX@")))
 				return
 			}
 
 			if e.ClusterHealth {
 				url = s + "/_cluster/health?level=indices"
 				if err := e.gatherClusterHealth(url, acc); err != nil {
-					err = fmt.Errorf(mask.ReplaceAllString(err.Error(), "http(s)://XXX:XXX@"))
-					errChan.C <- err
+					acc.AddError(fmt.Errorf(mask.ReplaceAllString(err.Error(), "http(s)://XXX:XXX@")))
 					return
 				}
 			}
 
 			if e.ClusterStats && e.isMaster {
 				if err := e.gatherClusterStats(s+"/_cluster/stats", acc); err != nil {
-					err = fmt.Errorf(mask.ReplaceAllString(err.Error(), "http(s)://XXX:XXX@"))
-					errChan.C <- err
+					acc.AddError(fmt.Errorf(mask.ReplaceAllString(err.Error(), "http(s)://XXX:XXX@")))
 					return
 				}
 			}
@@ -201,7 +196,7 @@ func (e *Elasticsearch) Gather(acc telegraf.Accumulator) error {
 	}
 
 	wg.Wait()
-	return errChan.Error()
+	return nil
 }
 
 func (e *Elasticsearch) createHttpClient() (*http.Client, error) {

plugins/inputs/elasticsearch/elasticsearch_test.go

@@ -71,7 +71,7 @@ func TestGather(t *testing.T) {
 	es.client.Transport = newTransportMock(http.StatusOK, nodeStatsResponse)
 
 	var acc testutil.Accumulator
-	if err := es.Gather(&acc); err != nil {
+	if err := acc.GatherError(es.Gather); err != nil {
 		t.Fatal(err)
 	}
 

plugins/inputs/exec/exec.go

@@ -15,7 +15,6 @@ import (
 
 	"github.com/influxdata/telegraf"
 	"github.com/influxdata/telegraf/internal"
-	"github.com/influxdata/telegraf/internal/errchan"
 	"github.com/influxdata/telegraf/plugins/inputs"
 	"github.com/influxdata/telegraf/plugins/parsers"
 	"github.com/influxdata/telegraf/plugins/parsers/nagios"
@@ -36,7 +35,7 @@ const sampleConfig = `
   name_suffix = "_mycollector"
 
   ## Data format to consume.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
   data_format = "influx"
@@ -49,8 +48,7 @@ type Exec struct {
 
 	parser parsers.Parser
 
-	runner  Runner
-	errChan chan error
+	runner Runner
 }
 
 func NewExec() *Exec {
@@ -150,13 +148,13 @@ func (e *Exec) ProcessCommand(command string, acc telegraf.Accumulator, wg *sync
 
 	out, err := e.runner.Run(e, command, acc)
 	if err != nil {
-		e.errChan <- err
+		acc.AddError(err)
 		return
 	}
 
 	metrics, err := e.parser.Parse(out)
 	if err != nil {
-		e.errChan <- err
+		acc.AddError(err)
 	} else {
 		for _, metric := range metrics {
 			acc.AddFields(metric.Name(), metric.Fields(), metric.Tags(), metric.Time())
@@ -193,7 +191,8 @@ func (e *Exec) Gather(acc telegraf.Accumulator) error {
 
 		matches, err := filepath.Glob(cmdAndArgs[0])
 		if err != nil {
-			return err
+			acc.AddError(err)
+			continue
 		}
 
 		if len(matches) == 0 {
@@ -214,15 +213,12 @@ func (e *Exec) Gather(acc telegraf.Accumulator) error {
 		}
 	}
 
-	errChan := errchan.New(len(commands))
-	e.errChan = errChan.C
-
 	wg.Add(len(commands))
 	for _, command := range commands {
 		go e.ProcessCommand(command, acc, &wg)
 	}
 	wg.Wait()
-	return errChan.Error()
+	return nil
 }
 
 func init() {

plugins/inputs/exec/exec_test.go

@@ -101,7 +101,7 @@ func TestExec(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	err := e.Gather(&acc)
+	err := acc.GatherError(e.Gather)
 	require.NoError(t, err)
 	assert.Equal(t, acc.NFields(), 8, "non-numeric measurements should be ignored")
 
@@ -127,8 +127,7 @@ func TestExecMalformed(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	err := e.Gather(&acc)
-	require.Error(t, err)
+	require.Error(t, acc.GatherError(e.Gather))
 	assert.Equal(t, acc.NFields(), 0, "No new points should have been added")
 }
 
@@ -141,8 +140,7 @@ func TestCommandError(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	err := e.Gather(&acc)
-	require.Error(t, err)
+	require.Error(t, acc.GatherError(e.Gather))
 	assert.Equal(t, acc.NFields(), 0, "No new points should have been added")
 }
 
@@ -155,8 +153,7 @@ func TestLineProtocolParse(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	err := e.Gather(&acc)
-	require.NoError(t, err)
+	require.NoError(t, acc.GatherError(e.Gather))
 
 	fields := map[string]interface{}{
 		"usage_idle": float64(99),
@@ -191,7 +188,7 @@ func TestLineProtocolShortParse(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	err := e.Gather(&acc)
+	err := acc.GatherError(e.Gather)
 	require.Error(t, err)
 	assert.Contains(t, err.Error(), "buffer too short", "A buffer too short error was expected")
 }
@@ -205,7 +202,7 @@ func TestLineProtocolParseMultiple(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	err := e.Gather(&acc)
+	err := acc.GatherError(e.Gather)
 	require.NoError(t, err)
 
 	fields := map[string]interface{}{
@@ -231,7 +228,7 @@ func TestExecCommandWithGlob(t *testing.T) {
 	e.SetParser(parser)
 
 	var acc testutil.Accumulator
-	err := e.Gather(&acc)
+	err := acc.GatherError(e.Gather)
 	require.NoError(t, err)
 
 	fields := map[string]interface{}{
@@ -247,7 +244,7 @@ func TestExecCommandWithoutGlob(t *testing.T) {
 	e.SetParser(parser)
 
 	var acc testutil.Accumulator
-	err := e.Gather(&acc)
+	err := acc.GatherError(e.Gather)
 	require.NoError(t, err)
 
 	fields := map[string]interface{}{
@@ -263,7 +260,7 @@ func TestExecCommandWithoutGlobAndPath(t *testing.T) {
 	e.SetParser(parser)
 
 	var acc testutil.Accumulator
-	err := e.Gather(&acc)
+	err := acc.GatherError(e.Gather)
 	require.NoError(t, err)
 
 	fields := map[string]interface{}{

plugins/inputs/filestat/filestat.go

@@ -48,7 +48,6 @@ func (_ *FileStat) Description() string {
 func (_ *FileStat) SampleConfig() string { return sampleConfig }
 
 func (f *FileStat) Gather(acc telegraf.Accumulator) error {
-	var errS string
 	var err error
 
 	for _, filepath := range f.Files {
@@ -56,7 +55,7 @@ func (f *FileStat) Gather(acc telegraf.Accumulator) error {
 		g, ok := f.globs[filepath]
 		if !ok {
 			if g, err = globpath.Compile(filepath); err != nil {
-				errS += err.Error() + " "
+				acc.AddError(err)
 				continue
 			}
 			f.globs[filepath] = g
@@ -92,7 +91,7 @@ func (f *FileStat) Gather(acc telegraf.Accumulator) error {
 			if f.Md5 {
 				md5, err := getMd5(fileName)
 				if err != nil {
-					errS += err.Error() + " "
+					acc.AddError(err)
 				} else {
 					fields["md5_sum"] = md5
 				}
@@ -102,9 +101,6 @@ func (f *FileStat) Gather(acc telegraf.Accumulator) error {
 		}
 	}
 
-	if errS != "" {
-		return fmt.Errorf(errS)
-	}
 	return nil
 }
 

plugins/inputs/filestat/filestat_test.go

@@ -19,7 +19,7 @@ func TestGatherNoMd5(t *testing.T) {
 	}
 
 	acc := testutil.Accumulator{}
-	fs.Gather(&acc)
+	acc.GatherError(fs.Gather)
 
 	tags1 := map[string]string{
 		"file": dir + "log1.log",
@@ -59,7 +59,7 @@ func TestGatherExplicitFiles(t *testing.T) {
 	}
 
 	acc := testutil.Accumulator{}
-	fs.Gather(&acc)
+	acc.GatherError(fs.Gather)
 
 	tags1 := map[string]string{
 		"file": dir + "log1.log",
@@ -99,7 +99,7 @@ func TestGatherGlob(t *testing.T) {
 	}
 
 	acc := testutil.Accumulator{}
-	fs.Gather(&acc)
+	acc.GatherError(fs.Gather)
 
 	tags1 := map[string]string{
 		"file": dir + "log1.log",
@@ -131,7 +131,7 @@ func TestGatherSuperAsterisk(t *testing.T) {
 	}
 
 	acc := testutil.Accumulator{}
-	fs.Gather(&acc)
+	acc.GatherError(fs.Gather)
 
 	tags1 := map[string]string{
 		"file": dir + "log1.log",

plugins/inputs/graylog/graylog.go

@@ -4,7 +4,6 @@ import (
 	"bytes"
 	"encoding/base64"
 	"encoding/json"
-	"errors"
 	"fmt"
 	"io/ioutil"
 	"net"
@@ -149,31 +148,17 @@ func (h *GrayLog) Gather(acc telegraf.Accumulator) error {
 		h.client.SetHTTPClient(client)
 	}
 
-	errorChannel := make(chan error, len(h.Servers))
-
 	for _, server := range h.Servers {
 		wg.Add(1)
 		go func(server string) {
 			defer wg.Done()
-			if err := h.gatherServer(acc, server); err != nil {
-				errorChannel <- err
-			}
+			acc.AddError(h.gatherServer(acc, server))
 		}(server)
 	}
 
 	wg.Wait()
-	close(errorChannel)
 
-	// Get all errors and return them as one giant error
-	errorStrings := []string{}
-	for err := range errorChannel {
-		errorStrings = append(errorStrings, err.Error())
-	}
-
-	if len(errorStrings) == 0 {
-		return nil
-	}
-	return errors.New(strings.Join(errorStrings, "\n"))
+	return nil
 }
 
 // Gathers data from a particular server

plugins/inputs/graylog/graylog_test.go

@@ -157,7 +157,7 @@ func TestNormalResponse(t *testing.T) {
 
 	for _, service := range graylog {
 		var acc testutil.Accumulator
-		err := service.Gather(&acc)
+		err := acc.GatherError(service.Gather)
 		require.NoError(t, err)
 		for k, v := range expectedFields {
 			acc.AssertContainsTaggedFields(t, k, v, validTags[k])
@@ -170,9 +170,9 @@ func TestHttpJson500(t *testing.T) {
 	graylog := genMockGrayLog(validJSON, 500)
 
 	var acc testutil.Accumulator
-	err := graylog[0].Gather(&acc)
+	err := acc.GatherError(graylog[0].Gather)
 
-	assert.NotNil(t, err)
+	assert.Error(t, err)
 	assert.Equal(t, 0, acc.NFields())
 }
 
@@ -181,9 +181,9 @@ func TestHttpJsonBadJson(t *testing.T) {
 	graylog := genMockGrayLog(invalidJSON, 200)
 
 	var acc testutil.Accumulator
-	err := graylog[0].Gather(&acc)
+	err := acc.GatherError(graylog[0].Gather)
 
-	assert.NotNil(t, err)
+	assert.Error(t, err)
 	assert.Equal(t, 0, acc.NFields())
 }
 
@@ -192,8 +192,8 @@ func TestHttpJsonEmptyResponse(t *testing.T) {
 	graylog := genMockGrayLog(empty, 200)
 
 	var acc testutil.Accumulator
-	err := graylog[0].Gather(&acc)
+	err := acc.GatherError(graylog[0].Gather)
 
-	assert.NotNil(t, err)
+	assert.Error(t, err)
 	assert.Equal(t, 0, acc.NFields())
 }

plugins/inputs/haproxy/README.md

@@ -7,7 +7,30 @@
 ```toml
 # SampleConfig
 [[inputs.haproxy]]
-  servers = ["http://1.2.3.4/haproxy?stats", "/var/run/haproxy*.sock"]
+  ## An array of address to gather stats about. Specify an ip on hostname
+  ## with optional port. ie localhost, 10.10.3.33:1936, etc.
+  ## Make sure you specify the complete path to the stats endpoint
+  ## including the protocol, ie http://10.10.3.33:1936/haproxy?stats
+
+  ## If no servers are specified, then default to 127.0.0.1:1936/haproxy?stats
+  servers = ["http://myhaproxy.com:1936/haproxy?stats"]
+
+  ## You can also use local socket with standard wildcard globbing.
+  ## Server address not starting with 'http' will be treated as a possible
+  ## socket, so both examples below are valid.
+  # servers = ["socket:/run/haproxy/admin.sock", "/run/haproxy/*.sock"]
+
+  ## By default, some of the fields are renamed from what haproxy calls them.
+  ## Setting this option to true results in the plugin keeping the original
+  ## field names.
+  # keep_field_names = true
+
+  ## Optional SSL Config
+  # ssl_ca = "/etc/telegraf/ca.pem"
+  # ssl_cert = "/etc/telegraf/cert.pem"
+  # ssl_key = "/etc/telegraf/key.pem"
+  ## Use SSL but skip chain & host verification
+  # insecure_skip_verify = false
 ```
 
 #### `servers`

plugins/inputs/haproxy/haproxy.go

@@ -14,6 +14,7 @@ import (
 	"time"
 
 	"github.com/influxdata/telegraf"
+	"github.com/influxdata/telegraf/internal"
 	"github.com/influxdata/telegraf/plugins/inputs"
 )
 
@@ -25,6 +26,15 @@ type haproxy struct {
 	client *http.Client
 
 	KeepFieldNames bool
+
+	// Path to CA file
+	SSLCA string `toml:"ssl_ca"`
+	// Path to host cert file
+	SSLCert string `toml:"ssl_cert"`
+	// Path to cert key file
+	SSLKey string `toml:"ssl_key"`
+	// Use SSL but skip chain & host verification
+	InsecureSkipVerify bool
 }
 
 var sampleConfig = `
@@ -32,19 +42,26 @@ var sampleConfig = `
   ## with optional port. ie localhost, 10.10.3.33:1936, etc.
   ## Make sure you specify the complete path to the stats endpoint
   ## including the protocol, ie http://10.10.3.33:1936/haproxy?stats
-  #
+
   ## If no servers are specified, then default to 127.0.0.1:1936/haproxy?stats
   servers = ["http://myhaproxy.com:1936/haproxy?stats"]
-  ##
+
   ## You can also use local socket with standard wildcard globbing.
   ## Server address not starting with 'http' will be treated as a possible
   ## socket, so both examples below are valid.
-  ## servers = ["socket:/run/haproxy/admin.sock", "/run/haproxy/*.sock"]
-  #
+  # servers = ["socket:/run/haproxy/admin.sock", "/run/haproxy/*.sock"]
+
   ## By default, some of the fields are renamed from what haproxy calls them.
   ## Setting this option to true results in the plugin keeping the original
   ## field names.
-  ## keep_field_names = true
+  # keep_field_names = true
+
+  ## Optional SSL Config
+  # ssl_ca = "/etc/telegraf/ca.pem"
+  # ssl_cert = "/etc/telegraf/cert.pem"
+  # ssl_key = "/etc/telegraf/key.pem"
+  ## Use SSL but skip chain & host verification
+  # insecure_skip_verify = false
 `
 
 func (r *haproxy) SampleConfig() string {
@@ -127,7 +144,15 @@ func (g *haproxy) gatherServer(addr string, acc telegraf.Accumulator) error {
 	}
 
 	if g.client == nil {
-		tr := &http.Transport{ResponseHeaderTimeout: time.Duration(3 * time.Second)}
+		tlsCfg, err := internal.GetTLSConfig(
+			g.SSLCert, g.SSLKey, g.SSLCA, g.InsecureSkipVerify)
+		if err != nil {
+			return err
+		}
+		tr := &http.Transport{
+			ResponseHeaderTimeout: time.Duration(3 * time.Second),
+			TLSClientConfig:       tlsCfg,
+		}
 		client := &http.Client{
 			Transport: tr,
 			Timeout:   time.Duration(4 * time.Second),

plugins/inputs/http_listener/README.md

@@ -2,11 +2,18 @@
 
 The HTTP listener is a service input plugin that listens for messages sent via HTTP POST.
 The plugin expects messages in the InfluxDB line-protocol ONLY, other Telegraf input data formats are not supported.
-The intent of the plugin is to allow Telegraf to serve as a proxy/router for the /write endpoint of the InfluxDB HTTP API.
+The intent of the plugin is to allow Telegraf to serve as a proxy/router for the `/write` endpoint of the InfluxDB HTTP API.
+
+The `/write` endpoint supports the `precision` query parameter and can be set to one of `ns`, `u`, `ms`, `s`, `m`, `h`.  All other parameters are ignored and defer to the output plugins configuration.
+
 When chaining Telegraf instances using this plugin, CREATE DATABASE requests receive a 200 OK response with message body `{"results":[]}` but they are not relayed. The output configuration of the Telegraf instance which ultimately submits data to InfluxDB determines the destination database.
 
 See: [Telegraf Input Data Formats](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md#influx).
-Example:  curl -i -XPOST 'http://localhost:8186/write' --data-binary 'cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000'
+
+**Example:**
+```
+curl -i -XPOST 'http://localhost:8186/write' --data-binary 'cpu_load_short,host=server01,region=us-west value=0.64 1434055562000000000'
+```
 
 ### Configuration:
 

plugins/inputs/http_listener/http_listener.go

@@ -35,6 +35,7 @@ type HTTPListener struct {
 	WriteTimeout   internal.Duration
 	MaxBodySize    int64
 	MaxLineSize    int
+	Port           int
 
 	mu sync.Mutex
 	wg sync.WaitGroup
@@ -124,6 +125,7 @@ func (h *HTTPListener) Start(acc telegraf.Accumulator) error {
 		return err
 	}
 	h.listener = listener
+	h.Port = listener.Addr().(*net.TCPAddr).Port
 
 	h.wg.Add(1)
 	go func() {
@@ -205,10 +207,12 @@ func (h *HTTPListener) serveWrite(res http.ResponseWriter, req *http.Request) {
 	}
 	now := time.Now()
 
+	precision := req.URL.Query().Get("precision")
+
 	// Handle gzip request bodies
 	body := req.Body
-	var err error
 	if req.Header.Get("Content-Encoding") == "gzip" {
+		var err error
 		body, err = gzip.NewReader(req.Body)
 		defer body.Close()
 		if err != nil {
@@ -261,7 +265,7 @@ func (h *HTTPListener) serveWrite(res http.ResponseWriter, req *http.Request) {
 
 		if err == io.ErrUnexpectedEOF {
 			// finished reading the request body
-			if err := h.parse(buf[:n+bufStart], now); err != nil {
+			if err := h.parse(buf[:n+bufStart], now, precision); err != nil {
 				log.Println("E! " + err.Error())
 				return400 = true
 			}
@@ -286,7 +290,7 @@ func (h *HTTPListener) serveWrite(res http.ResponseWriter, req *http.Request) {
 			bufStart = 0
 			continue
 		}
-		if err := h.parse(buf[:i+1], now); err != nil {
+		if err := h.parse(buf[:i+1], now, precision); err != nil {
 			log.Println("E! " + err.Error())
 			return400 = true
 		}
@@ -299,8 +303,8 @@ func (h *HTTPListener) serveWrite(res http.ResponseWriter, req *http.Request) {
 	}
 }
 
-func (h *HTTPListener) parse(b []byte, t time.Time) error {
-	metrics, err := h.parser.ParseWithDefaultTime(b, t)
+func (h *HTTPListener) parse(b []byte, t time.Time, precision string) error {
+	metrics, err := h.parser.ParseWithDefaultTimePrecision(b, t, precision)
 
 	for _, m := range metrics {
 		h.acc.AddFields(m.Name(), m.Fields(), m.Tags(), m.Time())

plugins/inputs/http_response/http_response_test.go

@@ -329,7 +329,7 @@ func TestTimeout(t *testing.T) {
 		Address:         ts.URL + "/twosecondnap",
 		Body:            "{ 'test': 'data'}",
 		Method:          "GET",
-		ResponseTimeout: internal.Duration{Duration: time.Second * 1},
+		ResponseTimeout: internal.Duration{Duration: time.Millisecond},
 		Headers: map[string]string{
 			"Content-Type": "application/json",
 		},

plugins/inputs/httpjson/README.md

@@ -1,128 +1,79 @@
-# HTTP JSON Plugin
+# HTTP JSON Input Plugin
 
-The httpjson plugin can collect data from remote URLs which respond with JSON. Then it flattens JSON and finds all numeric values, treating them as floats.
+The httpjson plugin collects data from HTTP URLs which respond with JSON.  It flattens the JSON and finds all numeric values, treating them as floats.
 
-For example, if you have a service called _mycollector_, which has HTTP endpoint for gathering stats at http://my.service.com/_stats, you would configure the HTTP JSON plugin like this:
+### Configuration:
 
-```
+```toml
 [[inputs.httpjson]]
-  name = "mycollector"
+  ## NOTE This plugin only reads numerical measurements, strings and booleans
+  ## will be ignored.
 
+  ## Name for the service being polled.  Will be appended to the name of the
+  ## measurement e.g. "httpjson_webserver_stats".
+  ##
+  ## Deprecated (1.3.0): Use name_override, name_suffix, name_prefix instead.
+  name = "webserver_stats"
+
+  ## URL of each server in the service's cluster
   servers = [
-    "http://my.service.com/_stats"
+    "http://localhost:9999/stats/",
+    "http://localhost:9998/stats/",
   ]
-
-  # HTTP method to use (case-sensitive)
-  method = "GET"
-
-  # Set response_timeout (default 5 seconds)
+  ## Set response_timeout (default 5 seconds)
   response_timeout = "5s"
-```
 
-`name` is used as a prefix for the measurements.
-
-`method` specifies HTTP method to use for requests.
-
-`response_timeout` specifies timeout to wait to get the response
-
-You can also specify which keys from server response should be considered tags:
-
-```
-[[inputs.httpjson]]
-  ...
-
-  tag_keys = [
-    "role",
-    "version"
-  ]
-```
-
-If the JSON response is an array of objects, then each object will be parsed with the same configuration.
-
-You can also specify additional request parameters for the service:
-
-```
-[[inputs.httpjson]]
-  ...
-
- [inputs.httpjson.parameters]
-    event_type = "cpu_spike"
-    threshold = "0.75"
-
-```
-
-You can also specify additional request header parameters for the service:
-
-```
-[[inputs.httpjson]]
-  ...
-
- [inputs.httpjson.headers]
-    X-Auth-Token = "my-xauth-token"
-    apiVersion = "v1"
-```
-
-# Example:
-
-Let's say that we have a service named "mycollector" configured like this:
-
-```
-[[inputs.httpjson]]
-  name = "mycollector"
-  servers = [
-    "http://my.service.com/_stats"
-  ]
-  # HTTP method to use (case-sensitive)
-  method = "GET"
-  tag_keys = ["service"]
-```
-
-which responds with the following JSON:
-
-```json
-{
-    "service": "service01",
-    "a": 0.5,
-    "b": {
-        "c": "some text",
-        "d": 0.1,
-        "e": 5
-    }
-}
-```
-
-The collected metrics will be:
-```
-httpjson_mycollector_a,service='service01',server='http://my.service.com/_stats' value=0.5
-httpjson_mycollector_b_d,service='service01',server='http://my.service.com/_stats' value=0.1
-httpjson_mycollector_b_e,service='service01',server='http://my.service.com/_stats' value=5
-```
-
-# Example 2, Multiple Services:
-
-There is also the option to collect JSON from multiple services, here is an example doing that.
-
-```
-[[inputs.httpjson]]
-  name = "mycollector1"
-  servers = [
-    "http://my.service1.com/_stats"
-  ]
-  # HTTP method to use (case-sensitive)
+  ## HTTP method to use: GET or POST (case-sensitive)
   method = "GET"
 
-[[inputs.httpjson]]
-  name = "mycollector2"
-  servers = [
-    "http://service.net/json/stats"
-  ]
-  # HTTP method to use (case-sensitive)
-  method = "POST"
+  ## Tags to extract from top-level of JSON server response.
+  # tag_keys = [
+  #   "my_tag_1",
+  #   "my_tag_2"
+  # ]
+
+  ## HTTP Request Parameters (all values must be strings).  For "GET" requests, data
+  ## will be included in the query.  For "POST" requests, data will be included
+  ## in the request body as "x-www-form-urlencoded".
+  # [inputs.httpjson.parameters]
+  #   event_type = "cpu_spike"
+  #   threshold = "0.75"
+
+  ## HTTP Request Headers (all values must be strings).
+  # [inputs.httpjson.headers]
+  #   X-Auth-Token = "my-xauth-token"
+  #   apiVersion = "v1"
+
+  ## Optional SSL Config
+  # ssl_ca = "/etc/telegraf/ca.pem"
+  # ssl_cert = "/etc/telegraf/cert.pem"
+  # ssl_key = "/etc/telegraf/key.pem"
+  ## Use SSL but skip chain & host verification
+  # insecure_skip_verify = false
 ```
 
-The services respond with the following JSON:
+### Measurements & Fields:
 
-mycollector1:
+- httpjson
+	- response_time (float): Response time in seconds
+
+Additional fields are dependant on the response of the remote service being polled.
+
+### Tags:
+
+- All measurements have the following tags:
+	- server: HTTP origin as defined in configuration as `servers`.
+
+Any top level keys listed under `tag_keys` in the configuration are added as tags.  Top level keys are defined as keys in the root level of the object in a single object response, or in the root level of each object within an array of objects.
+
+
+### Examples Output:
+
+This plugin understands responses containing a single JSON object, or a JSON Array of Objects.
+
+**Object Output:**
+
+Given the following response body:
 ```json
 {
     "a": 0.5,
@@ -130,45 +81,30 @@ mycollector1:
         "c": "some text",
         "d": 0.1,
         "e": 5
-    }
+    },
+    "service": "service01"
 }
 ```
+The following metric is produced:
 
-mycollector2:
-```json
-{
-    "load": 100,
-    "users": 1335
-}
-```
+`httpjson,server=http://localhost:9999/stats/ b_d=0.1,a=0.5,b_e=5,response_time=0.001`
 
-The collected metrics will be:
+Note that only numerical values are extracted and the type is float.
 
-```
-httpjson_mycollector1_a,server='http://my.service.com/_stats' value=0.5
-httpjson_mycollector1_b_d,server='http://my.service.com/_stats' value=0.1
-httpjson_mycollector1_b_e,server='http://my.service.com/_stats' value=5
+If `tag_keys` is included in the configuration:
 
-httpjson_mycollector2_load,server='http://service.net/json/stats' value=100
-httpjson_mycollector2_users,server='http://service.net/json/stats' value=1335
-```
-
-# Example 3, Multiple Metrics in Response:
-
-The response JSON can be treated as an array of data points that are all parsed with the same configuration.
-
-```
+```toml
 [[inputs.httpjson]]
-  name = "mycollector"
-  servers = [
-    "http://my.service.com/_stats"
-  ]
-  # HTTP method to use (case-sensitive)
-  method = "GET"
   tag_keys = ["service"]
 ```
 
-which responds with the following JSON:
+Then the `service` tag will also be added:
+
+`httpjson,server=http://localhost:9999/stats/,service=service01 b_d=0.1,a=0.5,b_e=5,response_time=0.001`
+
+**Array Output:**
+
+If the service returns an array of objects, one metric is be created for each object:
 
 ```json
 [
@@ -193,12 +129,5 @@ which responds with the following JSON:
 ]
 ```
 
-The collected metrics will be:
-```
-httpjson_mycollector_a,service='service01',server='http://my.service.com/_stats' value=0.5
-httpjson_mycollector_b_d,service='service01',server='http://my.service.com/_stats' value=0.1
-httpjson_mycollector_b_e,service='service01',server='http://my.service.com/_stats' value=5
-httpjson_mycollector_a,service='service02',server='http://my.service.com/_stats' value=0.6
-httpjson_mycollector_b_d,service='service02',server='http://my.service.com/_stats' value=0.2
-httpjson_mycollector_b_e,service='service02',server='http://my.service.com/_stats' value=6
-```
+`httpjson,server=http://localhost:9999/stats/,service=service01 a=0.5,b_d=0.1,b_e=5,response_time=0.003`
+`httpjson,server=http://localhost:9999/stats/,service=service02 a=0.6,b_d=0.2,b_e=6,response_time=0.003`

plugins/inputs/httpjson/httpjson.go

@@ -1,7 +1,6 @@
 package httpjson
 
 import (
-	"errors"
 	"fmt"
 	"io/ioutil"
 	"net/http"
@@ -73,7 +72,10 @@ var sampleConfig = `
   ## NOTE This plugin only reads numerical measurements, strings and booleans
   ## will be ignored.
 
-  ## a name for the service being polled
+  ## Name for the service being polled.  Will be appended to the name of the
+  ## measurement e.g. httpjson_webserver_stats
+  ##
+  ## Deprecated (1.3.0): Use name_override, name_suffix, name_prefix instead.
   name = "webserver_stats"
 
   ## URL of each server in the service's cluster
@@ -93,12 +95,14 @@ var sampleConfig = `
   #   "my_tag_2"
   # ]
 
-  ## HTTP parameters (all values must be strings)
-  [inputs.httpjson.parameters]
-    event_type = "cpu_spike"
-    threshold = "0.75"
+  ## HTTP parameters (all values must be strings).  For "GET" requests, data
+  ## will be included in the query.  For "POST" requests, data will be included
+  ## in the request body as "x-www-form-urlencoded".
+  # [inputs.httpjson.parameters]
+  #   event_type = "cpu_spike"
+  #   threshold = "0.75"
 
-  ## HTTP Header parameters (all values must be strings)
+  ## HTTP Headers (all values must be strings)
   # [inputs.httpjson.headers]
   #   X-Auth-Token = "my-xauth-token"
   #   apiVersion = "v1"
@@ -140,31 +144,17 @@ func (h *HttpJson) Gather(acc telegraf.Accumulator) error {
 		h.client.SetHTTPClient(client)
 	}
 
-	errorChannel := make(chan error, len(h.Servers))
-
 	for _, server := range h.Servers {
 		wg.Add(1)
 		go func(server string) {
 			defer wg.Done()
-			if err := h.gatherServer(acc, server); err != nil {
-				errorChannel <- err
-			}
+			acc.AddError(h.gatherServer(acc, server))
 		}(server)
 	}
 
 	wg.Wait()
-	close(errorChannel)
 
-	// Get all errors and return them as one giant error
-	errorStrings := []string{}
-	for err := range errorChannel {
-		errorStrings = append(errorStrings, err.Error())
-	}
-
-	if len(errorStrings) == 0 {
-		return nil
-	}
-	return errors.New(strings.Join(errorStrings, "\n"))
+	return nil
 }
 
 // Gathers data from a particular server

plugins/inputs/httpjson/httpjson_test.go

@@ -210,7 +210,7 @@ func TestHttpJson200(t *testing.T) {
 
 	for _, service := range httpjson {
 		var acc testutil.Accumulator
-		err := service.Gather(&acc)
+		err := acc.GatherError(service.Gather)
 		require.NoError(t, err)
 		assert.Equal(t, 12, acc.NFields())
 		// Set responsetime
@@ -245,7 +245,7 @@ func TestHttpJsonGET_URL(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	err := a.Gather(&acc)
+	err := acc.GatherError(a.Gather)
 	require.NoError(t, err)
 
 	// remove response_time from gathered fields because it's non-deterministic
@@ -318,7 +318,7 @@ func TestHttpJsonGET(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	err := a.Gather(&acc)
+	err := acc.GatherError(a.Gather)
 	require.NoError(t, err)
 
 	// remove response_time from gathered fields because it's non-deterministic
@@ -392,7 +392,7 @@ func TestHttpJsonPOST(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	err := a.Gather(&acc)
+	err := acc.GatherError(a.Gather)
 	require.NoError(t, err)
 
 	// remove response_time from gathered fields because it's non-deterministic
@@ -448,9 +448,9 @@ func TestHttpJson500(t *testing.T) {
 	httpjson := genMockHttpJson(validJSON, 500)
 
 	var acc testutil.Accumulator
-	err := httpjson[0].Gather(&acc)
+	err := acc.GatherError(httpjson[0].Gather)
 
-	assert.NotNil(t, err)
+	assert.Error(t, err)
 	assert.Equal(t, 0, acc.NFields())
 }
 
@@ -460,9 +460,9 @@ func TestHttpJsonBadMethod(t *testing.T) {
 	httpjson[0].Method = "NOT_A_REAL_METHOD"
 
 	var acc testutil.Accumulator
-	err := httpjson[0].Gather(&acc)
+	err := acc.GatherError(httpjson[0].Gather)
 
-	assert.NotNil(t, err)
+	assert.Error(t, err)
 	assert.Equal(t, 0, acc.NFields())
 }
 
@@ -471,9 +471,9 @@ func TestHttpJsonBadJson(t *testing.T) {
 	httpjson := genMockHttpJson(invalidJSON, 200)
 
 	var acc testutil.Accumulator
-	err := httpjson[0].Gather(&acc)
+	err := acc.GatherError(httpjson[0].Gather)
 
-	assert.NotNil(t, err)
+	assert.Error(t, err)
 	assert.Equal(t, 0, acc.NFields())
 }
 
@@ -482,9 +482,9 @@ func TestHttpJsonEmptyResponse(t *testing.T) {
 	httpjson := genMockHttpJson(empty, 200)
 
 	var acc testutil.Accumulator
-	err := httpjson[0].Gather(&acc)
+	err := acc.GatherError(httpjson[0].Gather)
 
-	assert.NotNil(t, err)
+	assert.Error(t, err)
 	assert.Equal(t, 0, acc.NFields())
 }
 
@@ -495,7 +495,7 @@ func TestHttpJson200Tags(t *testing.T) {
 	for _, service := range httpjson {
 		if service.Name == "other_webapp" {
 			var acc testutil.Accumulator
-			err := service.Gather(&acc)
+			err := acc.GatherError(service.Gather)
 			// Set responsetime
 			for _, p := range acc.Metrics {
 				p.Fields["response_time"] = 1.0
@@ -533,7 +533,7 @@ func TestHttpJsonArray200Tags(t *testing.T) {
 	for _, service := range httpjson {
 		if service.Name == "other_webapp" {
 			var acc testutil.Accumulator
-			err := service.Gather(&acc)
+			err := acc.GatherError(service.Gather)
 			// Set responsetime
 			for _, p := range acc.Metrics {
 				p.Fields["response_time"] = 1.0

plugins/inputs/influxdb/influxdb.go

@@ -5,7 +5,6 @@ import (
 	"errors"
 	"fmt"
 	"net/http"
-	"strings"
 	"sync"
 	"time"
 
@@ -57,34 +56,20 @@ func (i *InfluxDB) Gather(acc telegraf.Accumulator) error {
 		}
 	}
 
-	errorChannel := make(chan error, len(i.URLs))
-
 	var wg sync.WaitGroup
 	for _, u := range i.URLs {
 		wg.Add(1)
 		go func(url string) {
 			defer wg.Done()
 			if err := i.gatherURL(acc, url); err != nil {
-				errorChannel <- fmt.Errorf("[url=%s]: %s", url, err)
+				acc.AddError(fmt.Errorf("[url=%s]: %s", url, err))
 			}
 		}(u)
 	}
 
 	wg.Wait()
-	close(errorChannel)
 
-	// If there weren't any errors, we can return nil now.
-	if len(errorChannel) == 0 {
-		return nil
-	}
-
-	// There were errors, so join them all together as one big error.
-	errorStrings := make([]string, 0, len(errorChannel))
-	for err := range errorChannel {
-		errorStrings = append(errorStrings, err.Error())
-	}
-
-	return errors.New(strings.Join(errorStrings, "\n"))
+	return nil
 }
 
 type point struct {

plugins/inputs/influxdb/influxdb_test.go

@@ -25,7 +25,7 @@ func TestBasic(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	require.NoError(t, plugin.Gather(&acc))
+	require.NoError(t, acc.GatherError(plugin.Gather))
 
 	require.Len(t, acc.Metrics, 3)
 	fields := map[string]interface{}{
@@ -72,7 +72,7 @@ func TestInfluxDB(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	require.NoError(t, plugin.Gather(&acc))
+	require.NoError(t, acc.GatherError(plugin.Gather))
 
 	require.Len(t, acc.Metrics, 34)
 
@@ -132,7 +132,7 @@ func TestInfluxDB2(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	require.NoError(t, plugin.Gather(&acc))
+	require.NoError(t, acc.GatherError(plugin.Gather))
 
 	require.Len(t, acc.Metrics, 34)
 
@@ -157,7 +157,7 @@ func TestErrorHandling(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	require.Error(t, plugin.Gather(&acc))
+	require.Error(t, acc.GatherError(plugin.Gather))
 }
 
 func TestErrorHandling404(t *testing.T) {
@@ -175,7 +175,7 @@ func TestErrorHandling404(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	require.Error(t, plugin.Gather(&acc))
+	require.Error(t, acc.GatherError(plugin.Gather))
 }
 
 const basicJSON = `

plugins/inputs/internal/README.md

@@ -80,4 +80,4 @@ internal_write,output=file,host=tyrion buffer_limit=10000i,write_time_ns=636609i
 internal_gather,input=internal,host=tyrion metrics_gathered=19i,gather_time_ns=442114i 1480682800000000000
 internal_gather,input=http_listener,host=tyrion metrics_gathered=0i,gather_time_ns=167285i 1480682800000000000
 internal_http_listener,address=:8186,host=tyrion queries_received=0i,writes_received=0i,requests_received=0i,buffers_created=0i,requests_served=0i,pings_received=0i,bytes_received=0i,not_founds_served=0i,pings_served=0i,queries_served=0i,writes_served=0i 1480682800000000000
-```
+```

plugins/inputs/internal/internal.go

@@ -48,7 +48,7 @@ func (s *Self) Gather(acc telegraf.Accumulator) error {
 			"heap_idle_bytes":     m.HeapIdle,     // bytes in idle spans
 			"heap_in_use_bytes":   m.HeapInuse,    // bytes in non-idle span
 			"heap_released_bytes": m.HeapReleased, // bytes released to the OS
-			"heap_objects_bytes":  m.HeapObjects,  // total number of allocated objects
+			"heap_objects":        m.HeapObjects,  // total number of allocated objects
 			"num_gc":              m.NumGC,
 		}
 		acc.AddFields("internal_memstats", fields, map[string]string{})

plugins/inputs/interrupts/README.md

@@ -0,0 +1,35 @@
+# Interrupts Input Plugin
+
+The interrupts plugin gathers metrics about IRQs from `/proc/interrupts` and `/proc/softirqs`.
+
+### Configuration
+```
+[[inputs.interrupts]]
+  ## To filter which IRQs to collect, make use of tagpass / tagdrop, i.e.
+  # [inputs.interrupts.tagdrop]
+    # irq = [ "NET_RX", "TASKLET" ]
+```
+
+### Measurements
+There are two measurements reported by this plugin.
+- `interrupts` gathers metrics from the `/proc/interrupts` file
+- `soft_interrupts` gathers metrics from the `/proc/softirqs` file
+
+### Fields
+- CPUx: the amount of interrupts for the IRQ handled by that CPU
+- total: total amount of interrupts for all CPUs
+
+### Tags
+- irq: the IRQ
+- type: the type of interrupt
+- device: the name of the device that is located at that IRQ
+
+### Example Output
+```
+./telegraf -config ~/interrupts_config.conf -test
+* Plugin: inputs.interrupts, Collection 1
+> interrupts,irq=0,type=IO-APIC,device=2-edge\ timer,host=hostname CPU0=23i,total=23i 1489346531000000000
+> interrupts,irq=1,host=hostname,type=IO-APIC,device=1-edge\ i8042 CPU0=9i,total=9i 1489346531000000000
+> interrupts,irq=30,type=PCI-MSI,device=65537-edge\ virtio1-input.0,host=hostname CPU0=1i,total=1i 1489346531000000000
+> soft_interrupts,irq=NET_RX,host=hostname CPU0=280879i,total=280879i 1489346531000000000
+```

plugins/inputs/interrupts/interrupts.go

@@ -0,0 +1,123 @@
+package interrupts
+
+import (
+	"bufio"
+	"fmt"
+	"github.com/influxdata/telegraf"
+	"github.com/influxdata/telegraf/plugins/inputs"
+	"io"
+	"os"
+	"strconv"
+	"strings"
+)
+
+type Interrupts struct{}
+
+type IRQ struct {
+	ID     string
+	Type   string
+	Device string
+	Total  int64
+	Cpus   []int64
+}
+
+func NewIRQ(id string) *IRQ {
+	return &IRQ{ID: id, Cpus: []int64{}}
+}
+
+const sampleConfig = `
+  ## To filter which IRQs to collect, make use of tagpass / tagdrop, i.e.
+  # [inputs.interrupts.tagdrop]
+    # irq = [ "NET_RX", "TASKLET" ]
+`
+
+func (s *Interrupts) Description() string {
+	return "This plugin gathers interrupts data from /proc/interrupts and /proc/softirqs."
+}
+
+func (s *Interrupts) SampleConfig() string {
+	return sampleConfig
+}
+
+func parseInterrupts(r io.Reader) ([]IRQ, error) {
+	var irqs []IRQ
+	var cpucount int
+	scanner := bufio.NewScanner(r)
+	if scanner.Scan() {
+		cpus := strings.Fields(scanner.Text())
+		if cpus[0] != "CPU0" {
+			return nil, fmt.Errorf("Expected first line to start with CPU0, but was %s", scanner.Text())
+		}
+		cpucount = len(cpus)
+	}
+	for scanner.Scan() {
+		fields := strings.Fields(scanner.Text())
+		if !strings.HasSuffix(fields[0], ":") {
+			continue
+		}
+		irqid := strings.TrimRight(fields[0], ":")
+		irq := NewIRQ(irqid)
+		irqvals := fields[1:len(fields)]
+		for i := 0; i < cpucount; i++ {
+			if i < len(irqvals) {
+				irqval, err := strconv.ParseInt(irqvals[i], 10, 64)
+				if err != nil {
+					return irqs, fmt.Errorf("Unable to parse %q from %q: %s", irqvals[i], scanner.Text(), err)
+				}
+				irq.Cpus = append(irq.Cpus, irqval)
+			}
+		}
+		for _, irqval := range irq.Cpus {
+			irq.Total += irqval
+		}
+		_, err := strconv.ParseInt(irqid, 10, 64)
+		if err == nil && len(fields) >= cpucount+2 {
+			irq.Type = fields[cpucount+1]
+			irq.Device = strings.Join(fields[cpucount+2:], " ")
+		} else if len(fields) > cpucount {
+			irq.Type = strings.Join(fields[cpucount+1:], " ")
+		}
+		irqs = append(irqs, *irq)
+	}
+	if scanner.Err() != nil {
+		return nil, fmt.Errorf("Error scanning file: %s", scanner.Err())
+	}
+	return irqs, nil
+}
+
+func gatherTagsFields(irq IRQ) (map[string]string, map[string]interface{}) {
+	tags := map[string]string{"irq": irq.ID, "type": irq.Type, "device": irq.Device}
+	fields := map[string]interface{}{"total": irq.Total}
+	for i := 0; i < len(irq.Cpus); i++ {
+		cpu := fmt.Sprintf("CPU%d", i)
+		fields[cpu] = irq.Cpus[i]
+	}
+	return tags, fields
+}
+
+func (s *Interrupts) Gather(acc telegraf.Accumulator) error {
+	for measurement, file := range map[string]string{"interrupts": "/proc/interrupts", "soft_interrupts": "/proc/softirqs"} {
+		f, err := os.Open(file)
+		if err != nil {
+			acc.AddError(fmt.Errorf("Could not open file: %s", file))
+			continue
+		}
+		defer f.Close()
+		irqs, err := parseInterrupts(f)
+		if err != nil {
+			acc.AddError(fmt.Errorf("Parsing %s: %s", file, err))
+			continue
+		}
+		for _, irq := range irqs {
+			tags, fields := gatherTagsFields(irq)
+			acc.AddFields(measurement, fields, tags)
+		}
+	}
+	return nil
+}
+
+func init() {
+	inputs.Add("interrupts", func() telegraf.Input {
+		return &Interrupts{}
+	})
+}

plugins/inputs/interrupts/interrupts_test.go

@@ -0,0 +1,60 @@
+package interrupts
+
+import (
+	"bytes"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"testing"
+)
+
+func TestParseInterrupts(t *testing.T) {
+	interruptStr := `           CPU0       CPU1
+  0:        134          0   IO-APIC-edge      timer
+  1:          7          3   IO-APIC-edge      i8042
+NMI:          0          0   Non-maskable interrupts
+LOC: 2338608687 2334309625   Local timer interrupts
+MIS:          0
+NET_RX:     867028		225
+TASKLET:	205			0`
+	f := bytes.NewBufferString(interruptStr)
+	parsed := []IRQ{
+		IRQ{
+			ID: "0", Type: "IO-APIC-edge", Device: "timer",
+			Cpus: []int64{int64(134), int64(0)}, Total: int64(134),
+		},
+		IRQ{
+			ID: "1", Type: "IO-APIC-edge", Device: "i8042",
+			Cpus: []int64{int64(7), int64(3)}, Total: int64(10),
+		},
+		IRQ{
+			ID: "NMI", Type: "Non-maskable interrupts",
+			Cpus: []int64{int64(0), int64(0)}, Total: int64(0),
+		},
+		IRQ{
+			ID: "LOC", Type: "Local timer interrupts",
+			Cpus:  []int64{int64(2338608687), int64(2334309625)},
+			Total: int64(4672918312),
+		},
+		IRQ{
+			ID: "MIS", Cpus: []int64{int64(0)}, Total: int64(0),
+		},
+		IRQ{
+			ID: "NET_RX", Cpus: []int64{int64(867028), int64(225)},
+			Total: int64(867253),
+		},
+		IRQ{
+			ID: "TASKLET", Cpus: []int64{int64(205), int64(0)},
+			Total: int64(205),
+		},
+	}
+	got, err := parseInterrupts(f)
+	require.Equal(t, nil, err)
+	require.NotEqual(t, 0, len(got))
+	require.Equal(t, len(got), len(parsed))
+	for i := 0; i < len(parsed); i++ {
+		assert.Equal(t, parsed[i], got[i])
+		for k := 0; k < len(parsed[i].Cpus); k++ {
+			assert.Equal(t, parsed[i].Cpus[k], got[i].Cpus[k])
+		}
+	}
+}

plugins/inputs/ipmi_sensor/connection.go

@@ -18,7 +18,7 @@ type Connection struct {
 
 func NewConnection(server string) *Connection {
 	conn := &Connection{}
-	inx1 := strings.Index(server, "@")
+	inx1 := strings.LastIndex(server, "@")
 	inx2 := strings.Index(server, "(")
 	inx3 := strings.Index(server, ")")
 

plugins/inputs/ipmi_sensor/connection_test.go

@@ -0,0 +1,42 @@
+package ipmi_sensor
+
+import (
+	"testing"
+
+	"github.com/stretchr/testify/assert"
+)
+
+type conTest struct {
+	Got  string
+	Want *Connection
+}
+
+func TestNewConnection(t *testing.T) {
+	testData := []struct {
+		addr string
+		con  *Connection
+	}{
+		{
+			"USERID:PASSW0RD@lan(192.168.1.1)",
+			&Connection{
+				Hostname:  "192.168.1.1",
+				Username:  "USERID",
+				Password:  "PASSW0RD",
+				Interface: "lan",
+			},
+		},
+		{
+			"USERID:PASS:!@#$%^&*(234)_+W0RD@lan(192.168.1.1)",
+			&Connection{
+				Hostname:  "192.168.1.1",
+				Username:  "USERID",
+				Password:  "PASS:!@#$%^&*(234)_+W0RD",
+				Interface: "lan",
+			},
+		},
+	}
+
+	for _, v := range testData {
+		assert.Equal(t, v.con, NewConnection(v.addr))
+	}
+}

plugins/inputs/ipmi_sensor/ipmi.go

@@ -52,7 +52,8 @@ func (m *Ipmi) Gather(acc telegraf.Accumulator) error {
 		for _, server := range m.Servers {
 			err := m.parse(acc, server)
 			if err != nil {
-				return err
+				acc.AddError(err)
+				continue
 			}
 		}
 	} else {
@@ -152,6 +153,7 @@ func init() {
 		m.Path = path
 	}
 	inputs.Add("ipmi_sensor", func() telegraf.Input {
+		m := m
 		return &m
 	})
 }

plugins/inputs/ipmi_sensor/ipmi_test.go

@@ -20,7 +20,7 @@ func TestGather(t *testing.T) {
 	execCommand = fakeExecCommand
 	var acc testutil.Accumulator
 
-	err := i.Gather(&acc)
+	err := acc.GatherError(i.Gather)
 
 	require.NoError(t, err)
 
@@ -121,7 +121,7 @@ func TestGather(t *testing.T) {
 		Path: "ipmitool",
 	}
 
-	err = i.Gather(&acc)
+	err = acc.GatherError(i.Gather)
 
 	var testsWithoutServer = []struct {
 		fields map[string]interface{}

plugins/inputs/iptables/iptables.go

@@ -54,20 +54,19 @@ func (ipt *Iptables) Gather(acc telegraf.Accumulator) error {
 	}
 	// best effort : we continue through the chains even if an error is encountered,
 	// but we keep track of the last error.
-	var err error
 	for _, chain := range ipt.Chains {
 		data, e := ipt.lister(ipt.Table, chain)
 		if e != nil {
-			err = e
+			acc.AddError(e)
 			continue
 		}
 		e = ipt.parseAndGather(data, acc)
 		if e != nil {
-			err = e
+			acc.AddError(e)
 			continue
 		}
 	}
-	return err
+	return nil
 }
 
 func (ipt *Iptables) chainList(table, chain string) (string, error) {

plugins/inputs/iptables/iptables_test.go

@@ -141,7 +141,7 @@ func TestIptables_Gather(t *testing.T) {
 			},
 		}
 		acc := new(testutil.Accumulator)
-		err := ipt.Gather(acc)
+		err := acc.GatherError(ipt.Gather)
 		if !reflect.DeepEqual(tt.err, err) {
 			t.Errorf("%d: expected error '%#v' got '%#v'", i, tt.err, err)
 		}
@@ -199,7 +199,7 @@ func TestIptables_Gather_listerError(t *testing.T) {
 		},
 	}
 	acc := new(testutil.Accumulator)
-	err := ipt.Gather(acc)
+	err := acc.GatherError(ipt.Gather)
 	if !reflect.DeepEqual(err, errFoo) {
 		t.Errorf("Expected error %#v got\n%#v\n", errFoo, err)
 	}

plugins/inputs/jolokia/jolokia.go

@@ -3,7 +3,6 @@ package jolokia
 import (
 	"bytes"
 	"encoding/json"
-	"errors"
 	"fmt"
 	"io/ioutil"
 	"net/http"
@@ -130,7 +129,7 @@ func (j *Jolokia) Description() string {
 	return "Read JMX metrics through Jolokia"
 }
 
-func (j *Jolokia) doRequest(req *http.Request) (map[string]interface{}, error) {
+func (j *Jolokia) doRequest(req *http.Request) ([]map[string]interface{}, error) {
 	resp, err := j.jClient.MakeRequest(req)
 	if err != nil {
 		return nil, err
@@ -155,85 +154,81 @@ func (j *Jolokia) doRequest(req *http.Request) (map[string]interface{}, error) {
 	}
 
 	// Unmarshal json
-	var jsonOut map[string]interface{}
+	var jsonOut []map[string]interface{}
 	if err = json.Unmarshal([]byte(body), &jsonOut); err != nil {
-		return nil, errors.New("Error decoding JSON response")
-	}
-
-	if status, ok := jsonOut["status"]; ok {
-		if status != float64(200) {
-			return nil, fmt.Errorf("Not expected status value in response body: %3.f",
-				status)
-		}
-	} else {
-		return nil, fmt.Errorf("Missing status in response body")
+		return nil, fmt.Errorf("Error decoding JSON response: %s: %s", err, body)
 	}
 
 	return jsonOut, nil
 }
 
-func (j *Jolokia) prepareRequest(server Server, metric Metric) (*http.Request, error) {
+func (j *Jolokia) prepareRequest(server Server, metrics []Metric) (*http.Request, error) {
 	var jolokiaUrl *url.URL
 	context := j.Context // Usually "/jolokia/"
 
-	// Create bodyContent
-	bodyContent := map[string]interface{}{
-		"type":  "read",
-		"mbean": metric.Mbean,
+	var bulkBodyContent []map[string]interface{}
+	for _, metric := range metrics {
+		// Create bodyContent
+		bodyContent := map[string]interface{}{
+			"type":  "read",
+			"mbean": metric.Mbean,
+		}
+
+		if metric.Attribute != "" {
+			bodyContent["attribute"] = metric.Attribute
+			if metric.Path != "" {
+				bodyContent["path"] = metric.Path
+			}
+		}
+
+		// Add target, only in proxy mode
+		if j.Mode == "proxy" {
+			serviceUrl := fmt.Sprintf("service:jmx:rmi:///jndi/rmi://%s:%s/jmxrmi",
+				server.Host, server.Port)
+
+			target := map[string]string{
+				"url": serviceUrl,
+			}
+
+			if server.Username != "" {
+				target["user"] = server.Username
+			}
+
+			if server.Password != "" {
+				target["password"] = server.Password
+			}
+
+			bodyContent["target"] = target
+
+			proxy := j.Proxy
+
+			// Prepare ProxyURL
+			proxyUrl, err := url.Parse("http://" + proxy.Host + ":" + proxy.Port + context)
+			if err != nil {
+				return nil, err
+			}
+			if proxy.Username != "" || proxy.Password != "" {
+				proxyUrl.User = url.UserPassword(proxy.Username, proxy.Password)
+			}
+
+			jolokiaUrl = proxyUrl
+
+		} else {
+			serverUrl, err := url.Parse("http://" + server.Host + ":" + server.Port + context)
+			if err != nil {
+				return nil, err
+			}
+			if server.Username != "" || server.Password != "" {
+				serverUrl.User = url.UserPassword(server.Username, server.Password)
+			}
+
+			jolokiaUrl = serverUrl
+		}
+
+		bulkBodyContent = append(bulkBodyContent, bodyContent)
 	}
 
-	if metric.Attribute != "" {
-		bodyContent["attribute"] = metric.Attribute
-		if metric.Path != "" {
-			bodyContent["path"] = metric.Path
-		}
-	}
-
-	// Add target, only in proxy mode
-	if j.Mode == "proxy" {
-		serviceUrl := fmt.Sprintf("service:jmx:rmi:///jndi/rmi://%s:%s/jmxrmi",
-			server.Host, server.Port)
-
-		target := map[string]string{
-			"url": serviceUrl,
-		}
-
-		if server.Username != "" {
-			target["user"] = server.Username
-		}
-
-		if server.Password != "" {
-			target["password"] = server.Password
-		}
-
-		bodyContent["target"] = target
-
-		proxy := j.Proxy
-
-		// Prepare ProxyURL
-		proxyUrl, err := url.Parse("http://" + proxy.Host + ":" + proxy.Port + context)
-		if err != nil {
-			return nil, err
-		}
-		if proxy.Username != "" || proxy.Password != "" {
-			proxyUrl.User = url.UserPassword(proxy.Username, proxy.Password)
-		}
-
-		jolokiaUrl = proxyUrl
-
-	} else {
-		serverUrl, err := url.Parse("http://" + server.Host + ":" + server.Port + context)
-		if err != nil {
-			return nil, err
-		}
-		if server.Username != "" || server.Password != "" {
-			serverUrl.User = url.UserPassword(server.Username, server.Password)
-		}
-
-		jolokiaUrl = serverUrl
-	}
-
-	requestBody, err := json.Marshal(bodyContent)
+	requestBody, err := json.Marshal(bulkBodyContent)
 
 	req, err := http.NewRequest("POST", jolokiaUrl.String(), bytes.NewBuffer(requestBody))
 
@@ -276,25 +271,35 @@ func (j *Jolokia) Gather(acc telegraf.Accumulator) error {
 		tags["jolokia_host"] = server.Host
 		fields := make(map[string]interface{})
 
-		for _, metric := range metrics {
-			measurement := metric.Name
+		req, err := j.prepareRequest(server, metrics)
+		if err != nil {
+			acc.AddError(fmt.Errorf("unable to create request: %s", err))
+			continue
+		}
+		out, err := j.doRequest(req)
+		if err != nil {
+			acc.AddError(fmt.Errorf("error performing request: %s", err))
+			continue
+		}
 
-			req, err := j.prepareRequest(server, metric)
-			if err != nil {
-				return err
+		if len(out) != len(metrics) {
+			acc.AddError(fmt.Errorf("did not receive the correct number of metrics in response. expected %d, received %d", len(metrics), len(out)))
+			continue
+		}
+		for i, resp := range out {
+			if status, ok := resp["status"]; ok && status != float64(200) {
+				acc.AddError(fmt.Errorf("Not expected status value in response body (%s:%s mbean=\"%s\" attribute=\"%s\"): %3.f",
+					server.Host, server.Port, metrics[i].Mbean, metrics[i].Attribute, status))
+				continue
+			} else if !ok {
+				acc.AddError(fmt.Errorf("Missing status in response body"))
+				continue
 			}
 
-			out, err := j.doRequest(req)
-
-			if err != nil {
-				fmt.Printf("Error handling response: %s\n", err)
+			if values, ok := resp["value"]; ok {
+				j.extractValues(metrics[i].Name, values, fields)
 			} else {
-				if values, ok := out["value"]; ok {
-					j.extractValues(measurement, values, fields)
-				} else {
-					fmt.Printf("Missing key 'value' in output response\n")
-				}
-
+				acc.AddError(fmt.Errorf("Missing key 'value' in output response\n"))
 			}
 		}
 

plugins/inputs/jolokia/jolokia_test.go

@@ -13,65 +13,105 @@ import (
 )
 
 const validThreeLevelMultiValueJSON = `
-{
-  "request":{
-    "mbean":"java.lang:type=*",
-    "type":"read"
+[
+  {
+    "request":{
+      "mbean":"java.lang:type=*",
+      "type":"read"
+    },
+    "value":{
+      "java.lang:type=Memory":{
+        "ObjectPendingFinalizationCount":0,
+        "Verbose":false,
+        "HeapMemoryUsage":{
+          "init":134217728,
+          "committed":173015040,
+          "max":1908932608,
+          "used":16840016
+        },
+        "NonHeapMemoryUsage":{
+          "init":2555904,
+          "committed":51380224,
+          "max":-1,
+          "used":49944048
+        },
+        "ObjectName":{
+          "objectName":"java.lang:type=Memory"
+        }
+      }
+    },
+    "timestamp":1446129191,
+    "status":200
+  }
+]`
+
+const validBulkResponseJSON = `
+[
+  {
+    "request":{
+      "mbean":"java.lang:type=Memory",
+      "attribute":"HeapMemoryUsage",
+      "type":"read"
+    },
+    "value":{
+      "init":67108864,
+      "committed":456130560,
+      "max":477626368,
+      "used":203288528
+    },
+    "timestamp":1446129191,
+    "status":200
   },
-  "value":{
-		"java.lang:type=Memory":{
-			"ObjectPendingFinalizationCount":0,
-			"Verbose":false,
-			"HeapMemoryUsage":{
-				"init":134217728,
-				"committed":173015040,
-				"max":1908932608,
-				"used":16840016
-			},
-			"NonHeapMemoryUsage":{
-				"init":2555904,
-				"committed":51380224,
-				"max":-1,
-				"used":49944048
-			},
-			"ObjectName":{
-				"objectName":"java.lang:type=Memory"
-			}
-		}
-  },
-  "timestamp":1446129191,
-  "status":200
-}`
+  {
+    "request":{
+      "mbean":"java.lang:type=Memory",
+      "attribute":"NonHeapMemoryUsage",
+      "type":"read"
+    },
+    "value":{
+      "init":2555904,
+      "committed":51380224,
+      "max":-1,
+      "used":49944048
+    },
+    "timestamp":1446129191,
+    "status":200
+  }
+]`
 
 const validMultiValueJSON = `
-{
-  "request":{
-    "mbean":"java.lang:type=Memory",
-    "attribute":"HeapMemoryUsage",
-    "type":"read"
-  },
-  "value":{
-    "init":67108864,
-    "committed":456130560,
-    "max":477626368,
-    "used":203288528
-  },
-  "timestamp":1446129191,
-  "status":200
-}`
+[
+  {
+    "request":{
+      "mbean":"java.lang:type=Memory",
+      "attribute":"HeapMemoryUsage",
+      "type":"read"
+    },
+    "value":{
+      "init":67108864,
+      "committed":456130560,
+      "max":477626368,
+      "used":203288528
+    },
+    "timestamp":1446129191,
+    "status":200
+  }
+]`
 
 const validSingleValueJSON = `
-{
-  "request":{
-    "path":"used",
-    "mbean":"java.lang:type=Memory",
-    "attribute":"HeapMemoryUsage",
-    "type":"read"
-  },
-  "value":209274376,
-  "timestamp":1446129256,
-  "status":200
-}`
+[
+  {
+    "request":{
+      "path":"used",
+      "mbean":"java.lang:type=Memory",
+      "attribute":"HeapMemoryUsage",
+      "type":"read"
+    },
+    "value":209274376,
+    "timestamp":1446129256,
+    "status":200
+  }
+]`
 
 const invalidJSON = "I don't think this is JSON"
 
@@ -82,6 +122,8 @@ var HeapMetric = Metric{Name: "heap_memory_usage",
 	Mbean: "java.lang:type=Memory", Attribute: "HeapMemoryUsage"}
 var UsedHeapMetric = Metric{Name: "heap_memory_usage",
 	Mbean: "java.lang:type=Memory", Attribute: "HeapMemoryUsage"}
+var NonHeapMetric = Metric{Name: "non_heap_memory_usage",
+	Mbean: "java.lang:type=Memory", Attribute: "NonHeapMemoryUsage"}
 
 type jolokiaClientStub struct {
 	responseBody string
@@ -116,7 +158,7 @@ func TestHttpJsonMultiValue(t *testing.T) {
 	jolokia := genJolokiaClientStub(validMultiValueJSON, 200, Servers, []Metric{HeapMetric})
 
 	var acc testutil.Accumulator
-	err := jolokia.Gather(&acc)
+	err := acc.GatherError(jolokia.Gather)
 
 	assert.Nil(t, err)
 	assert.Equal(t, 1, len(acc.Metrics))
@@ -135,12 +177,40 @@ func TestHttpJsonMultiValue(t *testing.T) {
 	acc.AssertContainsTaggedFields(t, "jolokia", fields, tags)
 }
 
+// Test that bulk responses are handled
+func TestHttpJsonBulkResponse(t *testing.T) {
+	jolokia := genJolokiaClientStub(validBulkResponseJSON, 200, Servers, []Metric{HeapMetric, NonHeapMetric})
+
+	var acc testutil.Accumulator
+	err := jolokia.Gather(&acc)
+
+	assert.Nil(t, err)
+	assert.Equal(t, 1, len(acc.Metrics))
+
+	fields := map[string]interface{}{
+		"heap_memory_usage_init":          67108864.0,
+		"heap_memory_usage_committed":     456130560.0,
+		"heap_memory_usage_max":           477626368.0,
+		"heap_memory_usage_used":          203288528.0,
+		"non_heap_memory_usage_init":      2555904.0,
+		"non_heap_memory_usage_committed": 51380224.0,
+		"non_heap_memory_usage_max":       -1.0,
+		"non_heap_memory_usage_used":      49944048.0,
+	}
+	tags := map[string]string{
+		"jolokia_host": "127.0.0.1",
+		"jolokia_port": "8080",
+		"jolokia_name": "as1",
+	}
+	acc.AssertContainsTaggedFields(t, "jolokia", fields, tags)
+}
+
 // Test that the proper values are ignored or collected
 func TestHttpJsonThreeLevelMultiValue(t *testing.T) {
 	jolokia := genJolokiaClientStub(validThreeLevelMultiValueJSON, 200, Servers, []Metric{HeapMetric})
 
 	var acc testutil.Accumulator
-	err := jolokia.Gather(&acc)
+	err := acc.GatherError(jolokia.Gather)
 
 	assert.Nil(t, err)
 	assert.Equal(t, 1, len(acc.Metrics))
@@ -168,17 +238,18 @@ func TestHttpJsonThreeLevelMultiValue(t *testing.T) {
 }
 
 // Test that the proper values are ignored or collected
-func TestHttpJsonOn404(t *testing.T) {
+func TestHttp404(t *testing.T) {
 
-	jolokia := genJolokiaClientStub(validMultiValueJSON, 404, Servers,
+	jolokia := genJolokiaClientStub(invalidJSON, 404, Servers,
 		[]Metric{UsedHeapMetric})
 
 	var acc testutil.Accumulator
 	acc.SetDebug(true)
-	err := jolokia.Gather(&acc)
+	err := acc.GatherError(jolokia.Gather)
 
-	assert.Nil(t, err)
+	assert.Error(t, err)
 	assert.Equal(t, 0, len(acc.Metrics))
+	assert.Contains(t, err.Error(), "has status code 404")
 }
 
 // Test that the proper values are ignored or collected
@@ -189,8 +260,9 @@ func TestHttpInvalidJson(t *testing.T) {
 
 	var acc testutil.Accumulator
 	acc.SetDebug(true)
-	err := jolokia.Gather(&acc)
+	err := acc.GatherError(jolokia.Gather)
 
-	assert.Nil(t, err)
+	assert.Error(t, err)
 	assert.Equal(t, 0, len(acc.Metrics))
+	assert.Contains(t, err.Error(), "Error decoding JSON response")
 }

plugins/inputs/kafka_consumer/README.md

@@ -24,10 +24,14 @@ from the same topic in parallel.
 
   ## Data format to consume.
 
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
   data_format = "influx"
+
+  ## Maximum length of a message to consume, in bytes (default 0/unlimited);
+  ## larger messages are dropped
+  max_message_len = 65536
 ```
 
 ## Testing

plugins/inputs/kafka_consumer/kafka_consumer.go

@@ -1,6 +1,7 @@
 package kafka_consumer
 
 import (
+	"fmt"
 	"log"
 	"strings"
 	"sync"
@@ -16,6 +17,7 @@ import (
 type Kafka struct {
 	ConsumerGroup   string
 	Topics          []string
+	MaxMessageLen   int
 	ZookeeperPeers  []string
 	ZookeeperChroot string
 	Consumer        *consumergroup.ConsumerGroup
@@ -57,10 +59,14 @@ var sampleConfig = `
   offset = "oldest"
 
   ## Data format to consume.
-  ## Each data format has it's own unique set of configuration options, read
+  ## Each data format has its own unique set of configuration options, read
   ## more about them here:
   ## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
   data_format = "influx"
+
+  ## Maximum length of a message to consume, in bytes (default 0/unlimited);
+  ## larger messages are dropped
+  max_message_len = 65536
 `
 
 func (k *Kafka) SampleConfig() string {
@@ -129,17 +135,21 @@ func (k *Kafka) receiver() {
 			return
 		case err := <-k.errs:
 			if err != nil {
-				log.Printf("E! Kafka Consumer Error: %s\n", err)
+				k.acc.AddError(fmt.Errorf("Consumer Error: %s\n", err))
 			}
 		case msg := <-k.in:
-			metrics, err := k.parser.Parse(msg.Value)
-			if err != nil {
-				log.Printf("E! Kafka Message Parse Error\nmessage: %s\nerror: %s",
-					string(msg.Value), err.Error())
-			}
-
-			for _, metric := range metrics {
-				k.acc.AddFields(metric.Name(), metric.Fields(), metric.Tags(), metric.Time())
+			if k.MaxMessageLen != 0 && len(msg.Value) > k.MaxMessageLen {
+				k.acc.AddError(fmt.Errorf("Message longer than max_message_len (%d > %d)",
+					len(msg.Value), k.MaxMessageLen))
+			} else {
+				metrics, err := k.parser.Parse(msg.Value)
+				if err != nil {
+					k.acc.AddError(fmt.Errorf("Message Parse Error\nmessage: %s\nerror: %s",
+						string(msg.Value), err.Error()))
+				}
+				for _, metric := range metrics {
+					k.acc.AddFields(metric.Name(), metric.Fields(), metric.Tags(), metric.Time())
+				}
 			}
 
 			if !k.doNotCommitMsgs {
@@ -158,7 +168,7 @@ func (k *Kafka) Stop() {
 	defer k.Unlock()
 	close(k.done)
 	if err := k.Consumer.Close(); err != nil {
-		log.Printf("E! Error closing kafka consumer: %s\n", err.Error())
+		k.acc.AddError(fmt.Errorf("Error closing consumer: %s\n", err.Error()))
 	}
 }
 

plugins/inputs/kafka_consumer/kafka_consumer_integration_test.go

@@ -59,7 +59,7 @@ func TestReadsMetricsFromKafka(t *testing.T) {
 	waitForPoint(&acc, t)
 
 	// Gather points
-	err = k.Gather(&acc)
+	err = acc.GatherError(k.Gather)
 	require.NoError(t, err)
 	if len(acc.Metrics) == 1 {
 		point := acc.Metrics[0]

plugins/inputs/kafka_consumer/kafka_consumer_test.go

@@ -1,8 +1,8 @@
 package kafka_consumer
 
 import (
+	"strings"
 	"testing"
-	"time"
 
 	"github.com/influxdata/telegraf/plugins/parsers"
 	"github.com/influxdata/telegraf/testutil"
@@ -43,7 +43,7 @@ func TestRunParser(t *testing.T) {
 	k.parser, _ = parsers.NewInfluxParser()
 	go k.receiver()
 	in <- saramaMsg(testMsg)
-	time.Sleep(time.Millisecond * 5)
+	acc.Wait(1)
 
 	assert.Equal(t, acc.NFields(), 1)
 }
@@ -58,7 +58,24 @@ func TestRunParserInvalidMsg(t *testing.T) {
 	k.parser, _ = parsers.NewInfluxParser()
 	go k.receiver()
 	in <- saramaMsg(invalidMsg)
-	time.Sleep(time.Millisecond * 5)
+	acc.WaitError(1)
+
+	assert.Equal(t, acc.NFields(), 0)
+}
+
+// Test that overlong messages are dropped
+func TestDropOverlongMsg(t *testing.T) {
+	const maxMessageLen = 64 * 1024
+	k, in := newTestKafka()
+	k.MaxMessageLen = maxMessageLen
+	acc := testutil.Accumulator{}
+	k.acc = &acc
+	defer close(k.done)
+	overlongMsg := strings.Repeat("v", maxMessageLen+1)
+
+	go k.receiver()
+	in <- saramaMsg(overlongMsg)
+	acc.WaitError(1)
 
 	assert.Equal(t, acc.NFields(), 0)
 }
@@ -73,9 +90,9 @@ func TestRunParserAndGather(t *testing.T) {
 	k.parser, _ = parsers.NewInfluxParser()
 	go k.receiver()
 	in <- saramaMsg(testMsg)
-	time.Sleep(time.Millisecond * 5)
+	acc.Wait(1)
 
-	k.Gather(&acc)
+	acc.GatherError(k.Gather)
 
 	assert.Equal(t, acc.NFields(), 1)
 	acc.AssertContainsFields(t, "cpu_load_short",
@@ -92,9 +109,9 @@ func TestRunParserAndGatherGraphite(t *testing.T) {
 	k.parser, _ = parsers.NewGraphiteParser("_", []string{}, nil)
 	go k.receiver()
 	in <- saramaMsg(testMsgGraphite)
-	time.Sleep(time.Millisecond * 5)
+	acc.Wait(1)
 
-	k.Gather(&acc)
+	acc.GatherError(k.Gather)
 
 	assert.Equal(t, acc.NFields(), 1)
 	acc.AssertContainsFields(t, "cpu_load_short_graphite",
@@ -111,9 +128,9 @@ func TestRunParserAndGatherJSON(t *testing.T) {
 	k.parser, _ = parsers.NewJSONParser("kafka_json_test", []string{}, nil)
 	go k.receiver()
 	in <- saramaMsg(testMsgJSON)
-	time.Sleep(time.Millisecond * 5)
+	acc.Wait(1)
 
-	k.Gather(&acc)
+	acc.GatherError(k.Gather)
 
 	assert.Equal(t, acc.NFields(), 2)
 	acc.AssertContainsFields(t, "kafka_json_test",

plugins/inputs/kapacitor/README.md

@@ -0,0 +1,149 @@
+# Kapacitor Plugin
+
+The Kapacitor plugin will collect metrics from the given Kapacitor instances.
+
+### Configuration:
+
+```toml
+[[inputs.kapacitor]]
+  ## Multiple URLs from which to read Kapacitor-formatted JSON
+  ## Default is "http://localhost:9092/kapacitor/v1/debug/vars".
+  urls = [
+    "http://localhost:9092/kapacitor/v1/debug/vars"
+  ]
+
+  ## Time limit for http requests
+  timeout = "5s"
+```
+
+### Measurements & Fields
+
+- kapacitor
+    - num_enabled_tasks, integer
+    - num_subscriptions, integer
+    - num_tasks, integer
+- kapacitor_edges
+    - collected, integer
+    - emitted, integer
+- kapacitor_ingress
+    - points_received, integer
+- kapacitor_memstats
+    - alloc_bytes, integer
+    - buck_hash_sys_bytes, integer
+    - frees, integer
+    - gcc_pu_fraction, float
+    - gc_sys_bytes, integer
+    - heap_alloc_bytes, integer
+    - heap_idle_bytes, integer
+    - heap_inuse_bytes, integer
+    - heap_objects, integer
+    - heap_released_bytes, integer
+    - heap_sys_bytes, integer
+    - last_gc_ns, integer
+    - lookups, integer
+    - mallocs, integer
+    - mcache_in_use_bytes, integer
+    - mcache_sys_bytes, integer
+    - mspan_in_use_bytes, integer
+    - mspan_sys_bytes, integer
+    - next_gc_ns, integer
+    - num_gc, integer
+    - other_sys_bytes, integer
+    - pause_total_ns, integer
+    - stack_in_use_bytes, integer
+    - stack_sys_bytes, integer
+    - sys_bytes, integer
+    - total_alloc_bytes, integer
+- kapacitor_nodes
+    - alerts_triggered, integer
+    - avg_exec_time_ns, integer
+    - batches_queried, integer
+    - crits_triggered, integer
+    - eval_errors, integer
+    - fields_defaulted, integer
+    - infos_triggered, integer
+    - oks_triggered, integer
+    - points_queried, integer
+    - points_written, integer
+    - query_errors, integer
+    - tags_defaulted, integer
+    - warns_triggered, integer
+    - write_errors, integer
+
+*Note:* The Kapacitor variables `host`, `cluster_id`, and `server_id`
+are currently not recorded due to the potential high cardinality of
+these values.
+
+### Example Output:
+
+```
+$ telegraf -config /etc/telegraf.conf -input-filter kapacitor -test
+* Plugin: inputs.kapacitor, Collection 1
+> kapacitor_memstats,host=hostname.local,kap_version=1.1.0~rc2,url=http://localhost:9092/kapacitor/v1/debug/vars alloc_bytes=6974808i,buck_hash_sys_bytes=1452609i,frees=207281i,gc_sys_bytes=802816i,gcc_pu_fraction=0.00004693548939673313,heap_alloc_bytes=6974808i,heap_idle_bytes=6742016i,heap_in_use_bytes=9183232i,heap_objects=23216i,heap_released_bytes=0i,heap_sys_bytes=15925248i,last_gc_ns=1478791460012676997i,lookups=88i,mallocs=230497i,mcache_in_use_bytes=9600i,mcache_sys_bytes=16384i,mspan_in_use_bytes=98560i,mspan_sys_bytes=131072i,next_gc_ns=11467528i,num_gc=8i,other_sys_bytes=2236087i,pause_total_ns=2994110i,stack_in_use_bytes=1900544i,stack_sys_bytes=1900544i,sys_bytes=22464760i,total_alloc_bytes=35023600i 1478791462000000000
+> kapacitor,host=hostname.local,kap_version=1.1.0~rc2,url=http://localhost:9092/kapacitor/v1/debug/vars num_enabled_tasks=5i,num_subscriptions=5i,num_tasks=5i 1478791462000000000
+> kapacitor_edges,child=stream0,host=hostname.local,parent=stream,task=deadman-test,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_ingress,database=_internal,host=hostname.local,measurement=shard,retention_policy=monitor,task_master=main points_received=120 1478791462000000000
+> kapacitor_ingress,database=_internal,host=hostname.local,measurement=subscriber,retention_policy=monitor,task_master=main points_received=60 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=http_out,node=http_out3,task=sys-stats,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_edges,child=window6,host=hostname.local,parent=derivative5,task=deadman-test,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=from,node=from1,task=sys-stats,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=stream,node=stream0,task=test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=window,node=window6,task=deadman-test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_ingress,database=_internal,host=hostname.local,measurement=cq,retention_policy=monitor,task_master=main points_received=10 1478791462000000000
+> kapacitor_edges,child=http_out3,host=hostname.local,parent=window2,task=sys-stats,type=batch collected=0,emitted=0 1478791462000000000
+> kapacitor_edges,child=mean4,host=hostname.local,parent=log3,task=deadman-test,type=batch collected=0,emitted=0 1478791462000000000
+> kapacitor_ingress,database=_kapacitor,host=hostname.local,measurement=nodes,retention_policy=autogen,task_master=main points_received=207 1478791462000000000
+> kapacitor_edges,child=stream0,host=hostname.local,parent=stream,task=sys-stats,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_edges,child=log6,host=hostname.local,parent=sum5,task=derivative-test,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_edges,child=from1,host=hostname.local,parent=stream0,task=sys-stats,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=alert,node=alert2,task=test,type=stream alerts_triggered=0,avg_exec_time_ns=0i,crits_triggered=0,infos_triggered=0,oks_triggered=0,warns_triggered=0 1478791462000000000
+> kapacitor_edges,child=log3,host=hostname.local,parent=derivative2,task=derivative-test,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_ingress,database=_kapacitor,host=hostname.local,measurement=runtime,retention_policy=autogen,task_master=main points_received=9 1478791462000000000
+> kapacitor_ingress,database=_internal,host=hostname.local,measurement=tsm1_filestore,retention_policy=monitor,task_master=main points_received=120 1478791462000000000
+> kapacitor_edges,child=derivative2,host=hostname.local,parent=from1,task=derivative-test,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=stream,node=stream0,task=derivative-test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_ingress,database=_internal,host=hostname.local,measurement=queryExecutor,retention_policy=monitor,task_master=main points_received=10 1478791462000000000
+> kapacitor_ingress,database=_internal,host=hostname.local,measurement=tsm1_wal,retention_policy=monitor,task_master=main points_received=120 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=log,node=log6,task=derivative-test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_edges,child=stream,host=hostname.local,parent=stats,task=task_master:main,type=stream collected=598,emitted=598 1478791462000000000
+> kapacitor_ingress,database=_internal,host=hostname.local,measurement=write,retention_policy=monitor,task_master=main points_received=10 1478791462000000000
+> kapacitor_edges,child=stream0,host=hostname.local,parent=stream,task=derivative-test,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=log,node=log3,task=deadman-test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=from,node=from1,task=deadman-test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_ingress,database=_kapacitor,host=hostname.local,measurement=ingress,retention_policy=autogen,task_master=main points_received=148 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=eval,node=eval4,task=derivative-test,type=stream avg_exec_time_ns=0i,eval_errors=0 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=derivative,node=derivative2,task=derivative-test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_ingress,database=_internal,host=hostname.local,measurement=runtime,retention_policy=monitor,task_master=main points_received=10 1478791462000000000
+> kapacitor_ingress,database=_internal,host=hostname.local,measurement=httpd,retention_policy=monitor,task_master=main points_received=10 1478791462000000000
+> kapacitor_edges,child=sum5,host=hostname.local,parent=eval4,task=derivative-test,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_ingress,database=_kapacitor,host=hostname.local,measurement=kapacitor,retention_policy=autogen,task_master=main points_received=9 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=from,node=from1,task=test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_ingress,database=_internal,host=hostname.local,measurement=tsm1_engine,retention_policy=monitor,task_master=main points_received=120 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=window,node=window2,task=deadman-test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=stream,node=stream0,task=deadman-test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_edges,child=influxdb_out4,host=hostname.local,parent=http_out3,task=sys-stats,type=batch collected=0,emitted=0 1478791462000000000
+> kapacitor_edges,child=window2,host=hostname.local,parent=from1,task=deadman-test,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=from,node=from1,task=derivative-test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_edges,child=from1,host=hostname.local,parent=stream0,task=deadman-test,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_ingress,database=_internal,host=hostname.local,measurement=database,retention_policy=monitor,task_master=main points_received=40 1478791462000000000
+> kapacitor_edges,child=stream,host=hostname.local,parent=write_points,task=task_master:main,type=stream collected=750,emitted=750 1478791462000000000
+> kapacitor_edges,child=log7,host=hostname.local,parent=window6,task=deadman-test,type=batch collected=0,emitted=0 1478791462000000000
+> kapacitor_edges,child=window2,host=hostname.local,parent=from1,task=sys-stats,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=log,node=log7,task=deadman-test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_ingress,database=_kapacitor,host=hostname.local,measurement=edges,retention_policy=autogen,task_master=main points_received=225 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=derivative,node=derivative5,task=deadman-test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_edges,child=from1,host=hostname.local,parent=stream0,task=test,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_edges,child=alert2,host=hostname.local,parent=from1,task=test,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=log,node=log3,task=derivative-test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=influxdb_out,node=influxdb_out4,task=sys-stats,type=stream avg_exec_time_ns=0i,points_written=0,write_errors=0 1478791462000000000
+> kapacitor_edges,child=stream0,host=hostname.local,parent=stream,task=test,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_edges,child=log3,host=hostname.local,parent=window2,task=deadman-test,type=batch collected=0,emitted=0 1478791462000000000
+> kapacitor_edges,child=derivative5,host=hostname.local,parent=mean4,task=deadman-test,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=stream,node=stream0,task=sys-stats,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=window,node=window2,task=sys-stats,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=mean,node=mean4,task=deadman-test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_edges,child=from1,host=hostname.local,parent=stream0,task=derivative-test,type=stream collected=0,emitted=0 1478791462000000000
+> kapacitor_ingress,database=_internal,host=hostname.local,measurement=tsm1_cache,retention_policy=monitor,task_master=main points_received=120 1478791462000000000
+> kapacitor_nodes,host=hostname.local,kind=sum,node=sum5,task=derivative-test,type=stream avg_exec_time_ns=0i 1478791462000000000
+> kapacitor_edges,child=eval4,host=hostname.local,parent=log3,task=derivative-test,type=stream collected=0,emitted=0 1478791462000000000
+```

plugins/inputs/kapacitor/kapacitor.go

@@ -0,0 +1,228 @@
+package kapacitor
+
+import (
+	"encoding/json"
+	"fmt"
+	"net/http"
+	"sync"
+	"time"
+
+	"github.com/influxdata/telegraf"
+	"github.com/influxdata/telegraf/internal"
+	"github.com/influxdata/telegraf/plugins/inputs"
+)
+
+const (
+	defaultURL = "http://localhost:9092/kapacitor/v1/debug/vars"
+)
+
+type Kapacitor struct {
+	URLs []string `toml:"urls"`
+
+	Timeout internal.Duration
+
+	client *http.Client
+}
+
+func (*Kapacitor) Description() string {
+	return "Read Kapacitor-formatted JSON metrics from one or more HTTP endpoints"
+}
+
+func (*Kapacitor) SampleConfig() string {
+	return `
+  ## Multiple URLs from which to read Kapacitor-formatted JSON
+  ## Default is "http://localhost:9092/kapacitor/v1/debug/vars".
+  urls = [
+    "http://localhost:9092/kapacitor/v1/debug/vars"
+  ]
+
+  ## Time limit for http requests
+  timeout = "5s"
+`
+}
+
+func (k *Kapacitor) Gather(acc telegraf.Accumulator) error {
+	if k.client == nil {
+		k.client = &http.Client{Timeout: k.Timeout.Duration}
+	}
+
+	var wg sync.WaitGroup
+	for _, u := range k.URLs {
+		wg.Add(1)
+		go func(url string) {
+			defer wg.Done()
+			if err := k.gatherURL(acc, url); err != nil {
+				acc.AddError(fmt.Errorf("[url=%s]: %s", url, err))
+			}
+		}(u)
+	}
+
+	wg.Wait()
+	return nil
+}
+
+type object struct {
+	Name   string                 `json:"name"`
+	Values map[string]interface{} `json:"values"`
+	Tags   map[string]string      `json:"tags"`
+}
+
+type memstats struct {
+	Alloc         int64   `json:"Alloc"`
+	TotalAlloc    int64   `json:"TotalAlloc"`
+	Sys           int64   `json:"Sys"`
+	Lookups       int64   `json:"Lookups"`
+	Mallocs       int64   `json:"Mallocs"`
+	Frees         int64   `json:"Frees"`
+	HeapAlloc     int64   `json:"HeapAlloc"`
+	HeapSys       int64   `json:"HeapSys"`
+	HeapIdle      int64   `json:"HeapIdle"`
+	HeapInuse     int64   `json:"HeapInuse"`
+	HeapReleased  int64   `json:"HeapReleased"`
+	HeapObjects   int64   `json:"HeapObjects"`
+	StackInuse    int64   `json:"StackInuse"`
+	StackSys      int64   `json:"StackSys"`
+	MSpanInuse    int64   `json:"MSpanInuse"`
+	MSpanSys      int64   `json:"MSpanSys"`
+	MCacheInuse   int64   `json:"MCacheInuse"`
+	MCacheSys     int64   `json:"MCacheSys"`
+	BuckHashSys   int64   `json:"BuckHashSys"`
+	GCSys         int64   `json:"GCSys"`
+	OtherSys      int64   `json:"OtherSys"`
+	NextGC        int64   `json:"NextGC"`
+	LastGC        int64   `json:"LastGC"`
+	PauseTotalNs  int64   `json:"PauseTotalNs"`
+	NumGC         int64   `json:"NumGC"`
+	GCCPUFraction float64 `json:"GCCPUFraction"`
+}
+
+type stats struct {
+	CmdLine          []string           `json:"cmdline"`
+	ClusterID        string             `json:"cluster_id"`
+	Host             string             `json:"host"`
+	Kapacitor        *map[string]object `json:"kapacitor"`
+	MemStats         *memstats          `json:"memstats"`
+	NumEnabledTasks  int                `json:"num_enabled_tasks"`
+	NumSubscriptions int                `json:"num_subscriptions"`
+	NumTasks         int                `json:"num_tasks"`
+	Product          string             `json:"product"`
+	ServerID         string             `json:"server_id"`
+	Version          string             `json:"version"`
+}
+
+// Gathers data from a particular URL
+// Parameters:
+//     acc    : The telegraf Accumulator to use
+//     url    : endpoint to send request to
+//
+// Returns:
+//     error: Any error that may have occurred
+func (k *Kapacitor) gatherURL(
+	acc telegraf.Accumulator,
+	url string,
+) error {
+	now := time.Now()
+
+	resp, err := k.client.Get(url)
+	if err != nil {
+		return err
+	}
+	defer resp.Body.Close()
+
+	dec := json.NewDecoder(resp.Body)
+
+	var s stats
+	err = dec.Decode(&s)
+	if err != nil {
+		return err
+	}
+
+	if s.MemStats != nil {
+		acc.AddFields("kapacitor_memstats",
+			map[string]interface{}{
+				"alloc_bytes":         s.MemStats.Alloc,
+				"buck_hash_sys_bytes": s.MemStats.BuckHashSys,
+				"frees":               s.MemStats.Frees,
+				"gcc_pu_fraction":     s.MemStats.GCCPUFraction,
+				"gc_sys_bytes":        s.MemStats.GCSys,
+				"heap_alloc_bytes":    s.MemStats.HeapAlloc,
+				"heap_idle_bytes":     s.MemStats.HeapIdle,
+				"heap_in_use_bytes":   s.MemStats.HeapInuse,
+				"heap_objects":        s.MemStats.HeapObjects,
+				"heap_released_bytes": s.MemStats.HeapReleased,
+				"heap_sys_bytes":      s.MemStats.HeapSys,
+				"last_gc_ns":          s.MemStats.LastGC,
+				"lookups":             s.MemStats.Lookups,
+				"mallocs":             s.MemStats.Mallocs,
+				"mcache_in_use_bytes": s.MemStats.MCacheInuse,
+				"mcache_sys_bytes":    s.MemStats.MCacheSys,
+				"mspan_in_use_bytes":  s.MemStats.MSpanInuse,
+				"mspan_sys_bytes":     s.MemStats.MSpanSys,
+				"next_gc_ns":          s.MemStats.NextGC,
+				"num_gc":              s.MemStats.NumGC,
+				"other_sys_bytes":     s.MemStats.OtherSys,
+				"pause_total_ns":      s.MemStats.PauseTotalNs,
+				"stack_in_use_bytes":  s.MemStats.StackInuse,
+				"stack_sys_bytes":     s.MemStats.StackSys,
+				"sys_bytes":           s.MemStats.Sys,
+				"total_alloc_bytes":   s.MemStats.TotalAlloc,
+			},
+			map[string]string{
+				"kap_version": s.Version,
+				"url":         url,
+			},
+			now)
+	}
+
+	acc.AddFields("kapacitor",
+		map[string]interface{}{
+			"num_enabled_tasks": s.NumEnabledTasks,
+			"num_subscriptions": s.NumSubscriptions,
+			"num_tasks":         s.NumTasks,
+		},
+		map[string]string{
+			"kap_version": s.Version,
+			"url":         url,
+		},
+		now)
+
+	if s.Kapacitor != nil {
+		for _, obj := range *s.Kapacitor {
+
+			// Strip out high-cardinality or duplicative tags
+			excludeTags := []string{"host", "cluster_id", "server_id"}
+			for _, key := range excludeTags {
+				if _, ok := obj.Tags[key]; ok {
+					delete(obj.Tags, key)
+				}
+			}
+
+			// Convert time-related string field to int
+			if _, ok := obj.Values["avg_exec_time_ns"]; ok {
+				d, err := time.ParseDuration(obj.Values["avg_exec_time_ns"].(string))
+				if err != nil {
+					continue
+				}
+				obj.Values["avg_exec_time_ns"] = d.Nanoseconds()
+			}
+
+			acc.AddFields(
+				"kapacitor_"+obj.Name,
+				obj.Values,
+				obj.Tags,
+				now,
+			)
+		}
+	}
+
+	return nil
+}
+
+func init() {
+	inputs.Add("kapacitor", func() telegraf.Input {
+		return &Kapacitor{
+			URLs:    []string{defaultURL},
+			Timeout: internal.Duration{Duration: time.Second * 5},
+		}
+	})
+}

plugins/inputs/kubernetes/kubernetes.go

@@ -11,7 +11,6 @@ import (
 
 	"github.com/influxdata/telegraf"
 	"github.com/influxdata/telegraf/internal"
-	"github.com/influxdata/telegraf/internal/errchan"
 	"github.com/influxdata/telegraf/plugins/inputs"
 )
 
@@ -72,14 +71,13 @@ func (k *Kubernetes) Description() string {
 //Gather collects kubernetes metrics from a given URL
 func (k *Kubernetes) Gather(acc telegraf.Accumulator) error {
 	var wg sync.WaitGroup
-	errChan := errchan.New(1)
 	wg.Add(1)
 	go func(k *Kubernetes) {
 		defer wg.Done()
-		errChan.C <- k.gatherSummary(k.URL, acc)
+		acc.AddError(k.gatherSummary(k.URL, acc))
 	}(k)
 	wg.Wait()
-	return errChan.Error()
+	return nil
 }
 
 func buildURL(endpoint string, base string) (*url.URL, error) {

plugins/inputs/kubernetes/kubernetes_test.go

@@ -22,7 +22,7 @@ func TestKubernetesStats(t *testing.T) {
 	}
 
 	var acc testutil.Accumulator
-	err := k.Gather(&acc)
+	err := acc.GatherError(k.Gather)
 	require.NoError(t, err)
 
 	fields := map[string]interface{}{

plugins/inputs/leofs/leofs.go

@@ -154,15 +154,16 @@ func (l *LeoFS) Gather(acc telegraf.Accumulator) error {
 		return nil
 	}
 	var wg sync.WaitGroup
-	var outerr error
 	for _, endpoint := range l.Servers {
 		_, err := url.Parse(endpoint)
 		if err != nil {
-			return fmt.Errorf("Unable to parse the address:%s, err:%s", endpoint, err)
+			acc.AddError(fmt.Errorf("Unable to parse the address:%s, err:%s", endpoint, err))
+			continue
 		}
 		port, err := retrieveTokenAfterColon(endpoint)
 		if err != nil {
-			return err
+			acc.AddError(err)
+			continue
 		}
 		st, ok := serverTypeMapping[port]
 		if !ok {
@@ -171,11 +172,11 @@ func (l *LeoFS) Gather(acc telegraf.Accumulator) error {
 		wg.Add(1)
 		go func(endpoint string, st ServerType) {
 			defer wg.Done()
-			outerr = l.gatherServer(endpoint, st, acc)
+			acc.AddError(l.gatherServer(endpoint, st, acc))
 		}(endpoint, st)
 	}
 	wg.Wait()
-	return outerr
+	return nil
 }
 
 func (l *LeoFS) gatherServer(

plugins/inputs/leofs/leofs_test.go

@@ -146,7 +146,7 @@ func testMain(t *testing.T, code string, endpoint string, serverType ServerType)
 	var acc testutil.Accumulator
 	acc.SetDebug(true)
 
-	err := l.Gather(&acc)
+	err := acc.GatherError(l.Gather)
 	require.NoError(t, err)
 
 	floatMetrics := KeyMapping[serverType]

plugins/inputs/logparser/README.md

@@ -1,6 +1,6 @@
-# logparser Input Plugin
+# Logparser Input Plugin
 
-The logparser plugin streams and parses the given logfiles. Currently it only
+The `logparser` plugin streams and parses the given logfiles. Currently it
 has the capability of parsing "grok" patterns from logfiles, which also supports
 regex patterns.
 
@@ -37,35 +37,28 @@ regex patterns.
     '''
 ```
 
-## Grok Parser
-
-The grok parser uses a slightly modified version of logstash "grok" patterns,
-with the format
-
-```
-%{<capture_syntax>[:<semantic_name>][:<modifier>]}
-```
-
-Telegraf has many of it's own
-[built-in patterns](https://github.com/influxdata/telegraf/blob/master/plugins/inputs/logparser/grok/patterns/influx-patterns),
-as well as supporting
-[logstash's builtin patterns](https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/grok-patterns).
-
+### Grok Parser
 
 The best way to get acquainted with grok patterns is to read the logstash docs,
 which are available here:
   https://www.elastic.co/guide/en/logstash/current/plugins-filters-grok.html
 
+The Telegraf grok parser uses a slightly modified version of logstash "grok"
+patterns, with the format
 
-If you need help building patterns to match your logs,
-you will find the http://grokdebug.herokuapp.com application quite useful!
+```
+%{<capture_syntax>[:<semantic_name>][:<modifier>]}
+```
 
+The `capture_syntax` defines the grok pattern that's used to parse the input
+line and the `semantic_name` is used to name the field or tag.  The extension
+`modifier` controls the data type that the parsed item is converted to or
+other special handling.
 
 By default all named captures are converted into string fields.
-Modifiers can be used to convert captures to other types or tags.
 Timestamp modifiers can be used to convert captures to the timestamp of the
- parsed metric.
-
+parsed metric.  If no timestamp is parsed the metric will be created using the
+current time.
 
 - Available modifiers:
   - string   (default if nothing is specified)
@@ -91,7 +84,112 @@ Timestamp modifiers can be used to convert captures to the timestamp of the
   - ts-epochnano     (nanoseconds since unix epoch)
   - ts-"CUSTOM"
 
-
 CUSTOM time layouts must be within quotes and be the representation of the
 "reference time", which is `Mon Jan 2 15:04:05 -0700 MST 2006`
 See https://golang.org/pkg/time/#Parse for more details.
+
+Telegraf has many of its own
+[built-in patterns](./grok/patterns/influx-patterns),
+as well as supporting
+[logstash's builtin patterns](https://github.com/logstash-plugins/logstash-patterns-core/blob/master/patterns/grok-patterns).
+
+If you need help building patterns to match your logs,
+you will find the https://grokdebug.herokuapp.com application quite useful!
+
+#### Timestamp Examples
+
+This example input and config parses a file using a custom timestamp conversion:
+
+```
+2017-02-21 13:10:34 value=42
+```
+
+```toml
+[[inputs.logparser]]
+  [inputs.logparser.grok]
+    patterns = ['%{TIMESTAMP_ISO8601:timestamp:ts-"2006-01-02 15:04:05"} value=%{NUMBER:value:int}']
+```
+
+This example parses a file using a built-in conversion and a custom pattern:
+
+```
+Wed Apr 12 13:10:34 PST 2017 value=42
+```
+
+```toml
+[[inputs.logparser]]
+  [inputs.logparser.grok]
+	patterns = ["%{TS_UNIX:timestamp:ts-unix} value=%{NUMBER:value:int}"]
+    custom_patterns = '''
+      TS_UNIX %{DAY} %{MONTH} %{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND} %{TZ} %{YEAR}
+    '''
+```
+
+#### TOML Escaping
+
+When saving patterns to the configuration file, keep in mind the different TOML
+[string](https://github.com/toml-lang/toml#string) types and the escaping
+rules for each.  These escaping rules must be applied in addition to the
+escaping required by the grok syntax.  Using the Multi-line line literal
+syntax with `'''` may be useful.
+
+The following config examples will parse this input file:
+
+```
+|42|\uD83D\uDC2F|'telegraf'|
+```
+
+Since `|` is a special character in the grok language, we must escape it to
+get a literal `|`.  With a basic TOML string, special characters such as
+backslash must be escaped, requiring us to escape the backslash a second time.
+
+```toml
+[[inputs.logparser]]
+  [inputs.logparser.grok]
+    patterns = ["\\|%{NUMBER:value:int}\\|%{UNICODE_ESCAPE:escape}\\|'%{WORD:name}'\\|"]
+    custom_patterns = "UNICODE_ESCAPE (?:\\\\u[0-9A-F]{4})+"
+```
+
+We cannot use a literal TOML string for the pattern, because we cannot match a
+`'` within it.  However, it works well for the custom pattern.
+```toml
+[[inputs.logparser]]
+  [inputs.logparser.grok]
+    patterns = ["\\|%{NUMBER:value:int}\\|%{UNICODE_ESCAPE:escape}\\|'%{WORD:name}'\\|"]
+    custom_patterns = 'UNICODE_ESCAPE (?:\\u[0-9A-F]{4})+'
+```
+
+A multi-line literal string allows us to encode the pattern:
+```toml
+[[inputs.logparser]]
+  [inputs.logparser.grok]
+    patterns = ['''
+	  \|%{NUMBER:value:int}\|%{UNICODE_ESCAPE:escape}\|'%{WORD:name}'\|
+	''']
+    custom_patterns = 'UNICODE_ESCAPE (?:\\u[0-9A-F]{4})+'
+```
+
+### Tips for creating patterns
+
+Writing complex patterns can be difficult, here is some advice for writing a
+new pattern or testing a pattern developed [online](https://grokdebug.herokuapp.com).
+
+Create a file output that writes to stdout, and disable other outputs while
+testing.  This will allow you to see the captured metrics.  Keep in mind that
+the file output will only print once per `flush_interval`.
+
+```toml
+[[outputs.file]]
+  files = ["stdout"]
+```
+
+- Start with a file containing only a single line of your input.
+- Remove all but the first token or piece of the line.
+- Add the section of your pattern to match this piece to your configuration file.
+- Verify that the metric is parsed successfully by running Telegraf.
+- If successful, add the next token, update the pattern and retest.
+- Continue one token at a time until the entire line is successfully parsed.
+
+### Additional Resources
+
+- https://www.influxdata.com/telegraf-correlate-log-metrics-data-performance-bottlenecks/

plugins/inputs/logparser/grok/grok.go

@@ -118,11 +118,18 @@ func (p *Parser) Compile() error {
 
 	// Give Patterns fake names so that they can be treated as named
 	// "custom patterns"
-	p.namedPatterns = make([]string, len(p.Patterns))
+	p.namedPatterns = make([]string, 0, len(p.Patterns))
 	for i, pattern := range p.Patterns {
+		if pattern == "" {
+			continue
+		}
 		name := fmt.Sprintf("GROK_INTERNAL_PATTERN_%d", i)
 		p.CustomPatterns += "\n" + name + " " + pattern + "\n"
-		p.namedPatterns[i] = "%{" + name + "}"
+		p.namedPatterns = append(p.namedPatterns, "%{"+name+"}")
+	}
+
+	if len(p.namedPatterns) == 0 {
+		return fmt.Errorf("pattern required")
 	}
 
 	// Combine user-supplied CustomPatterns with DEFAULT_PATTERNS and parse
@@ -168,6 +175,7 @@ func (p *Parser) ParseLine(line string) (telegraf.Metric, error) {
 	}
 
 	if len(values) == 0 {
+		log.Printf("D! Grok no match found for: %q", line)
 		return nil, nil
 	}