Merge branch 'master' into master

CHANGELOG.md

@@ -2,6 +2,8 @@
 
 ### Release Notes
 
+- Telegraf now supports two new types of plugins: processors & aggregators.
+
 - On systemd Telegraf will no longer redirect it's stdout to /var/log/telegraf/telegraf.log.
 On most systems, the logs will be directed to the systemd journal and can be
 accessed by `journalctl -u telegraf.service`. Consult the systemd journal
@@ -11,6 +13,7 @@ continue sending logs to /var/log/telegraf/telegraf.log.
 
 ### Features
 
+- [#1726](https://github.com/influxdata/telegraf/issues/1726): Processor & Aggregator plugin support.
 - [#1861](https://github.com/influxdata/telegraf/pull/1861): adding the tags in the graylog output plugin
 - [#1732](https://github.com/influxdata/telegraf/pull/1732): Telegraf systemd service, log to journal.
 - [#1782](https://github.com/influxdata/telegraf/pull/1782): Allow numeric and non-string values for tag_keys.
@@ -58,6 +61,13 @@ continue sending logs to /var/log/telegraf/telegraf.log.
 - [#1854](https://github.com/influxdata/telegraf/pull/1853): SQL Server waitstats truncation bug.
 - [#1810](https://github.com/influxdata/telegraf/issues/1810): Fix logparser common log format: numbers in ident.
 - [#1793](https://github.com/influxdata/telegraf/pull/1793): Fix JSON Serialization in OpenTSDB output.
+- [#1731](https://github.com/influxdata/telegraf/issues/1731): Fix Graphite template ordering, use most specific.
+- [#1836](https://github.com/influxdata/telegraf/pull/1836): Fix snmp table field initialization for non-automatic table.
+- [#1724](https://github.com/influxdata/telegraf/issues/1724): cgroups path being parsed as metric.
+- [#1886](https://github.com/influxdata/telegraf/issues/1886): Fix phpfpm fcgi client panic when URL does not exist.
+- [#1344](https://github.com/influxdata/telegraf/issues/1344): Fix config file parse error logging.
+- [#1771](https://github.com/influxdata/telegraf/issues/1771): Delete nil fields in the metric maker.
+- [#870](https://github.com/influxdata/telegraf/issues/870): Fix MySQL special characters in DSN parsing.
 
 ## v1.0.1 [2016-09-26]
 

CONTRIBUTING.md

@@ -2,7 +2,7 @@
 
 1. [Sign the CLA](http://influxdb.com/community/cla.html)
 1. Make changes or write plugin (see below for details)
-1. Add your plugin to `plugins/inputs/all/all.go` or `plugins/outputs/all/all.go`
+1. Add your plugin to one of: `plugins/{inputs,outputs,aggregators,processors}/all/all.go`
 1. If your plugin requires a new Go package,
 [add it](https://github.com/influxdata/telegraf/blob/master/CONTRIBUTING.md#adding-a-dependency)
 1. Write a README for your plugin, if it's an input plugin, it should be structured
@@ -16,8 +16,8 @@ for a good example.
 
 ## GoDoc
 
-Public interfaces for inputs, outputs, metrics, and the accumulator can be found
+Public interfaces for inputs, outputs, processors, aggregators, metrics,
-on the GoDoc
+and the accumulator can be found on the GoDoc
 
 [![GoDoc](https://godoc.org/github.com/influxdata/telegraf?status.svg)](https://godoc.org/github.com/influxdata/telegraf)
 
@@ -46,7 +46,7 @@ and submit new inputs.
 
 ### Input Plugin Guidelines
 
-* A plugin must conform to the `telegraf.Input` interface.
+* A plugin must conform to the [`telegraf.Input`](https://godoc.org/github.com/influxdata/telegraf#Input) interface.
 * Input Plugins should call `inputs.Add` in their `init` function to register themselves.
 See below for a quick example.
 * Input Plugins must be added to the
@@ -177,7 +177,7 @@ similar constructs.
 
 ### Output Plugin Guidelines
 
-* An output must conform to the `outputs.Output` interface.
+* An output must conform to the [`telegraf.Output`](https://godoc.org/github.com/influxdata/telegraf#Output) interface.
 * Outputs should call `outputs.Add` in their `init` function to register themselves.
 See below for a quick example.
 * To be available within Telegraf itself, plugins must add themselves to the
@@ -275,6 +275,186 @@ and `Stop()` methods.
 * Same as the `Output` guidelines, except that they must conform to the
 `output.ServiceOutput` interface.
 
+## Processor Plugins
+
+This section is for developers who want to create a new processor plugin.
+
+### Processor Plugin Guidelines
+
+* A processor must conform to the [`telegraf.Processor`](https://godoc.org/github.com/influxdata/telegraf#Processor) interface.
+* Processors should call `processors.Add` in their `init` function to register themselves.
+See below for a quick example.
+* To be available within Telegraf itself, plugins must add themselves to the
+`github.com/influxdata/telegraf/plugins/processors/all/all.go` file.
+* The `SampleConfig` function should return valid toml that describes how the
+processor can be configured. This is include in `telegraf -sample-config`.
+* The `Description` function should say in one line what this processor does.
+
+### Processor Example
+
+```go
+package printer
+
+// printer.go
+
+import (
+	"fmt"
+
+	"github.com/influxdata/telegraf"
+	"github.com/influxdata/telegraf/plugins/processors"
+)
+
+type Printer struct {
+}
+
+var sampleConfig = `
+`
+
+func (p *Printer) SampleConfig() string {
+	return sampleConfig
+}
+
+func (p *Printer) Description() string {
+	return "Print all metrics that pass through this filter."
+}
+
+func (p *Printer) Apply(in ...telegraf.Metric) []telegraf.Metric {
+	for _, metric := range in {
+		fmt.Println(metric.String())
+	}
+	return in
+}
+
+func init() {
+	processors.Add("printer", func() telegraf.Processor {
+		return &Printer{}
+	})
+}
+```
+
+## Aggregator Plugins
+
+This section is for developers who want to create a new aggregator plugin.
+
+### Aggregator Plugin Guidelines
+
+* A aggregator must conform to the [`telegraf.Aggregator`](https://godoc.org/github.com/influxdata/telegraf#Aggregator) interface.
+* Aggregators should call `aggregators.Add` in their `init` function to register themselves.
+See below for a quick example.
+* To be available within Telegraf itself, plugins must add themselves to the
+`github.com/influxdata/telegraf/plugins/aggregators/all/all.go` file.
+* The `SampleConfig` function should return valid toml that describes how the
+aggregator can be configured. This is include in `telegraf -sample-config`.
+* The `Description` function should say in one line what this aggregator does.
+* The Aggregator plugin will need to keep caches of metrics that have passed
+through it. This should be done using the builtin `HashID()` function of each
+metric.
+* When the `Reset()` function is called, all caches should be cleared.
+
+### Aggregator Example
+
+```go
+package min
+
+// min.go
+
+import (
+	"github.com/influxdata/telegraf"
+	"github.com/influxdata/telegraf/plugins/aggregators"
+)
+
+type Min struct {
+	// caches for metric fields, names, and tags
+	fieldCache map[uint64]map[string]float64
+	nameCache  map[uint64]string
+	tagCache   map[uint64]map[string]string
+}
+
+func NewMin() telegraf.Aggregator {
+	m := &Min{}
+	m.Reset()
+	return m
+}
+
+var sampleConfig = `
+  ## period is the flush & clear interval of the aggregator.
+  period = "30s"
+  ## If true drop_original will drop the original metrics and
+  ## only send aggregates.
+  drop_original = false
+`
+
+func (m *Min) SampleConfig() string {
+	return sampleConfig
+}
+
+func (m *Min) Description() string {
+	return "Keep the aggregate min of each metric passing through."
+}
+
+func (m *Min) Add(in telegraf.Metric) {
+	id := in.HashID()
+	if _, ok := m.nameCache[id]; !ok {
+		// hit an uncached metric, create caches for first time:
+		m.nameCache[id] = in.Name()
+		m.tagCache[id] = in.Tags()
+		m.fieldCache[id] = make(map[string]float64)
+		for k, v := range in.Fields() {
+			if fv, ok := convert(v); ok {
+				m.fieldCache[id][k] = fv
+			}
+		}
+	} else {
+		for k, v := range in.Fields() {
+			if fv, ok := convert(v); ok {
+				if _, ok := m.fieldCache[id][k]; !ok {
+					// hit an uncached field of a cached metric
+					m.fieldCache[id][k] = fv
+					continue
+				}
+				if fv < m.fieldCache[id][k] {
+                    // set new minimum
+					m.fieldCache[id][k] = fv
+				}
+			}
+		}
+	}
+}
+
+func (m *Min) Push(acc telegraf.Accumulator) {
+	for id, _ := range m.nameCache {
+		fields := map[string]interface{}{}
+		for k, v := range m.fieldCache[id] {
+			fields[k+"_min"] = v
+		}
+		acc.AddFields(m.nameCache[id], fields, m.tagCache[id])
+	}
+}
+
+func (m *Min) Reset() {
+	m.fieldCache = make(map[uint64]map[string]float64)
+	m.nameCache = make(map[uint64]string)
+	m.tagCache = make(map[uint64]map[string]string)
+}
+
+func convert(in interface{}) (float64, bool) {
+	switch v := in.(type) {
+	case float64:
+		return v, true
+	case int64:
+		return float64(v), true
+	default:
+		return 0, false
+	}
+}
+
+func init() {
+	aggregators.Add("min", func() telegraf.Aggregator {
+		return NewMin()
+	})
+}
+```
+
 ## Unit Tests
 
 ### Execute short tests

Godeps

@@ -19,7 +19,7 @@ github.com/eclipse/paho.mqtt.golang 0f7a459f04f13a41b7ed752d47944528d4bf9a86
 github.com/go-sql-driver/mysql 1fca743146605a172a266e1654e01e5cd5669bee
 github.com/gobwas/glob 49571a1557cd20e6a2410adc6421f85b66c730b5
 github.com/golang/protobuf 552c7b9542c194800fd493123b3798ef0a832032
-github.com/golang/snappy 427fb6fc07997f43afa32f35e850833760e489a7
+github.com/golang/snappy d9eb7a3d35ec988b8585d4a0068e462c27d28380
 github.com/gonuts/go-shellquote e842a11b24c6abfb3dd27af69a17f482e4b483c2
 github.com/gorilla/context 1ea25387ff6f684839d82767c1733ff4d4d15d0a
 github.com/gorilla/mux c9e326e2bdec29039a3761c07bece13133863e1e
@@ -27,7 +27,7 @@ github.com/hailocab/go-hostpool e80d13ce29ede4452c43dea11e79b9bc8a15b478
 github.com/hashicorp/consul 5aa90455ce78d4d41578bafc86305e6e6b28d7d2
 github.com/hpcloud/tail b2940955ab8b26e19d43a43c4da0475dd81bdb56
 github.com/influxdata/config b79f6829346b8d6e78ba73544b1e1038f1f1c9da
-github.com/influxdata/influxdb e094138084855d444195b252314dfee9eae34cab
+github.com/influxdata/influxdb fc57c0f7c635df3873f3d64f0ed2100ddc94d5ae
 github.com/influxdata/toml af4df43894b16e3fd2b788d01bd27ad0776ef2d0
 github.com/influxdata/wlog 7c63b0a71ef8300adc255344d275e10e5c3a71ec
 github.com/kardianos/osext 29ae4ffbc9a6fe9fb2bc5029050ce6996ea1d3bc
@@ -56,7 +56,7 @@ github.com/wvanbergen/kafka 46f9a1cf3f670edec492029fadded9c2d9e18866
 github.com/wvanbergen/kazoo-go 0f768712ae6f76454f987c3356177e138df258f8
 github.com/yuin/gopher-lua bf3808abd44b1e55143a2d7f08571aaa80db1808
 github.com/zensqlmonitor/go-mssqldb ffe5510c6fa5e15e6d983210ab501c815b56b363
-golang.org/x/crypto 5dc8cb4b8a8eb076cbb5a06bc3b8682c15bdbbd3
+golang.org/x/crypto c197bcf24cde29d3f73c7b4ac6fd41f4384e8af6
 golang.org/x/net 6acef71eb69611914f7a30939ea9f6e194c78172
 golang.org/x/text a71fd10341b064c10f4a81ceac72bcf70f26ea34
 gopkg.in/dancannon/gorethink.v1 7d1af5be49cb5ecc7b177bf387d232050299d6ef

README.md

@@ -86,42 +86,43 @@ if you don't have it already. You also must build with golang version 1.5+.
 ## How to use it:
 
 ```console
-$ telegraf -help
+$ telegraf --help
 Telegraf, The plugin-driven server agent for collecting and reporting metrics.
 
 Usage:
 
-  telegraf <flags>
+  telegraf [commands|flags]
 
-The flags are:
+The commands & flags are:
 
-  -config <file>     configuration file to load
+  config             print out full sample configuration to stdout
-  -test              gather metrics once, print them to stdout, and exit
+  version            print the version to stdout
-  -sample-config     print out full sample configuration to stdout
+
-  -config-directory  directory containing additional *.conf files
+  --config <file>     configuration file to load
-  -input-filter      filter the input plugins to enable, separator is :
+  --test              gather metrics once, print them to stdout, and exit
-  -output-filter     filter the output plugins to enable, separator is :
+  --config-directory  directory containing additional *.conf files
-  -usage             print usage for a plugin, ie, 'telegraf -usage mysql'
+  --input-filter      filter the input plugins to enable, separator is :
-  -debug             print metrics as they're generated to stdout
+  --output-filter     filter the output plugins to enable, separator is :
-  -quiet             run in quiet mode
+  --usage             print usage for a plugin, ie, 'telegraf -usage mysql'
-  -version           print the version to stdout
+  --debug             print metrics as they're generated to stdout
+  --quiet             run in quiet mode
 
 Examples:
 
   # generate a telegraf config file:
-  telegraf -sample-config > telegraf.conf
+  telegraf config > telegraf.conf
 
   # generate config with only cpu input & influxdb output plugins defined
-  telegraf -sample-config -input-filter cpu -output-filter influxdb
+  telegraf config -input-filter cpu -output-filter influxdb
 
   # run a single telegraf collection, outputing metrics to stdout
-  telegraf -config telegraf.conf -test
+  telegraf --config telegraf.conf -test
 
   # run telegraf with all plugins defined in config file
-  telegraf -config telegraf.conf
+  telegraf --config telegraf.conf
 
   # run telegraf, enabling the cpu & memory input, and influxdb output plugins
-  telegraf -config telegraf.conf -input-filter cpu:mem -output-filter influxdb
+  telegraf --config telegraf.conf -input-filter cpu:mem -output-filter influxdb
 ```
 
 ## Configuration

accumulator.go

@@ -2,9 +2,8 @@ package telegraf
 
 import "time"
 
-// Accumulator is an interface for "accumulating" metrics from input plugin(s).
+// Accumulator is an interface for "accumulating" metrics from plugin(s).
-// The metrics are sent down a channel shared between all input plugins and then
+// The metrics are sent down a channel shared between all plugins.
-// flushed on the configured flush_interval.
 type Accumulator interface {
 	// AddFields adds a metric to the accumulator with the given measurement
 	// name, fields, and tags (and timestamp). If a timestamp is not provided,
@@ -29,12 +28,7 @@ type Accumulator interface {
 		tags map[string]string,
 		t ...time.Time)
 
-	AddError(err error)
-
-	Debug() bool
-	SetDebug(enabled bool)
-
 	SetPrecision(precision, interval time.Duration)
 
-	DisablePrecision()
+	AddError(err error)
 }

agent/accumulator.go

@@ -1,37 +1,40 @@
 package agent
 
 import (
-	"fmt"
 	"log"
-	"math"
 	"sync/atomic"
 	"time"
 
 	"github.com/influxdata/telegraf"
-	"github.com/influxdata/telegraf/internal/models"
 )
 
+type MetricMaker interface {
+	Name() string
+	MakeMetric(
+		measurement string,
+		fields map[string]interface{},
+		tags map[string]string,
+		mType telegraf.ValueType,
+		t time.Time,
+	) telegraf.Metric
+}
+
 func NewAccumulator(
-	inputConfig *models.InputConfig,
+	maker MetricMaker,
 	metrics chan telegraf.Metric,
 ) *accumulator {
-	acc := accumulator{}
+	acc := accumulator{
-	acc.metrics = metrics
+		maker:     maker,
-	acc.inputConfig = inputConfig
+		metrics:   metrics,
-	acc.precision = time.Nanosecond
+		precision: time.Nanosecond,
+	}
 	return &acc
 }
 
 type accumulator struct {
 	metrics chan telegraf.Metric
 
-	defaultTags map[string]string
+	maker MetricMaker
-
-	debug bool
-	// print every point added to the accumulator
-	trace bool
-
-	inputConfig *models.InputConfig
 
 	precision time.Duration
 
@@ -44,7 +47,7 @@ func (ac *accumulator) AddFields(
 	tags map[string]string,
 	t ...time.Time,
 ) {
-	if m := ac.makeMetric(measurement, fields, tags, telegraf.Untyped, t...); m != nil {
+	if m := ac.maker.MakeMetric(measurement, fields, tags, telegraf.Untyped, ac.getTime(t)); m != nil {
 		ac.metrics <- m
 	}
 }
@@ -55,7 +58,7 @@ func (ac *accumulator) AddGauge(
 	tags map[string]string,
 	t ...time.Time,
 ) {
-	if m := ac.makeMetric(measurement, fields, tags, telegraf.Gauge, t...); m != nil {
+	if m := ac.maker.MakeMetric(measurement, fields, tags, telegraf.Gauge, ac.getTime(t)); m != nil {
 		ac.metrics <- m
 	}
 }
@@ -66,114 +69,11 @@ func (ac *accumulator) AddCounter(
 	tags map[string]string,
 	t ...time.Time,
 ) {
-	if m := ac.makeMetric(measurement, fields, tags, telegraf.Counter, t...); m != nil {
+	if m := ac.maker.MakeMetric(measurement, fields, tags, telegraf.Counter, ac.getTime(t)); m != nil {
 		ac.metrics <- m
 	}
 }
 
-// makeMetric either returns a metric, or returns nil if the metric doesn't
-// need to be created (because of filtering, an error, etc.)
-func (ac *accumulator) makeMetric(
-	measurement string,
-	fields map[string]interface{},
-	tags map[string]string,
-	mType telegraf.ValueType,
-	t ...time.Time,
-) telegraf.Metric {
-	if len(fields) == 0 || len(measurement) == 0 {
-		return nil
-	}
-	if tags == nil {
-		tags = make(map[string]string)
-	}
-
-	// Override measurement name if set
-	if len(ac.inputConfig.NameOverride) != 0 {
-		measurement = ac.inputConfig.NameOverride
-	}
-	// Apply measurement prefix and suffix if set
-	if len(ac.inputConfig.MeasurementPrefix) != 0 {
-		measurement = ac.inputConfig.MeasurementPrefix + measurement
-	}
-	if len(ac.inputConfig.MeasurementSuffix) != 0 {
-		measurement = measurement + ac.inputConfig.MeasurementSuffix
-	}
-
-	// Apply plugin-wide tags if set
-	for k, v := range ac.inputConfig.Tags {
-		if _, ok := tags[k]; !ok {
-			tags[k] = v
-		}
-	}
-	// Apply daemon-wide tags if set
-	for k, v := range ac.defaultTags {
-		if _, ok := tags[k]; !ok {
-			tags[k] = v
-		}
-	}
-
-	// Apply the metric filter(s)
-	if ok := ac.inputConfig.Filter.Apply(measurement, fields, tags); !ok {
-		return nil
-	}
-
-	for k, v := range fields {
-		// Validate uint64 and float64 fields
-		switch val := v.(type) {
-		case uint64:
-			// InfluxDB does not support writing uint64
-			if val < uint64(9223372036854775808) {
-				fields[k] = int64(val)
-			} else {
-				fields[k] = int64(9223372036854775807)
-			}
-			continue
-		case float64:
-			// NaNs are invalid values in influxdb, skip measurement
-			if math.IsNaN(val) || math.IsInf(val, 0) {
-				if ac.debug {
-					log.Printf("I! Measurement [%s] field [%s] has a NaN or Inf "+
-						"field, skipping",
-						measurement, k)
-				}
-				delete(fields, k)
-				continue
-			}
-		}
-
-		fields[k] = v
-	}
-
-	var timestamp time.Time
-	if len(t) > 0 {
-		timestamp = t[0]
-	} else {
-		timestamp = time.Now()
-	}
-	timestamp = timestamp.Round(ac.precision)
-
-	var m telegraf.Metric
-	var err error
-	switch mType {
-	case telegraf.Counter:
-		m, err = telegraf.NewCounterMetric(measurement, tags, fields, timestamp)
-	case telegraf.Gauge:
-		m, err = telegraf.NewGaugeMetric(measurement, tags, fields, timestamp)
-	default:
-		m, err = telegraf.NewMetric(measurement, tags, fields, timestamp)
-	}
-	if err != nil {
-		log.Printf("E! Error adding point [%s]: %s\n", measurement, err.Error())
-		return nil
-	}
-
-	if ac.trace {
-		fmt.Println("> " + m.String())
-	}
-
-	return m
-}
-
 // AddError passes a runtime error to the accumulator.
 // The error will be tagged with the plugin name and written to the log.
 func (ac *accumulator) AddError(err error) {
@@ -182,23 +82,7 @@ func (ac *accumulator) AddError(err error) {
 	}
 	atomic.AddUint64(&ac.errCount, 1)
 	//TODO suppress/throttle consecutive duplicate errors?
-	log.Printf("E! Error in input [%s]: %s", ac.inputConfig.Name, err)
+	log.Printf("E! Error in plugin [%s]: %s", ac.maker.Name(), err)
-}
-
-func (ac *accumulator) Debug() bool {
-	return ac.debug
-}
-
-func (ac *accumulator) SetDebug(debug bool) {
-	ac.debug = debug
-}
-
-func (ac *accumulator) Trace() bool {
-	return ac.trace
-}
-
-func (ac *accumulator) SetTrace(trace bool) {
-	ac.trace = trace
 }
 
 // SetPrecision takes two time.Duration objects. If the first is non-zero,
@@ -222,17 +106,12 @@ func (ac *accumulator) SetPrecision(precision, interval time.Duration) {
 	}
 }
 
-func (ac *accumulator) DisablePrecision() {
+func (ac accumulator) getTime(t []time.Time) time.Time {
-	ac.precision = time.Nanosecond
+	var timestamp time.Time
-}
+	if len(t) > 0 {
-
+		timestamp = t[0]
-func (ac *accumulator) setDefaultTags(tags map[string]string) {
+	} else {
-	ac.defaultTags = tags
+		timestamp = time.Now()
-}
-
-func (ac *accumulator) addDefaultTag(key, value string) {
-	if ac.defaultTags == nil {
-		ac.defaultTags = make(map[string]string)
 	}
-	ac.defaultTags[key] = value
+	return timestamp.Round(ac.precision)
 }

agent/accumulator_test.go

@@ -4,24 +4,21 @@ import (
 	"bytes"
 	"fmt"
 	"log"
-	"math"
 	"os"
 	"testing"
 	"time"
 
 	"github.com/influxdata/telegraf"
-	"github.com/influxdata/telegraf/internal/models"
 
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 )
 
 func TestAdd(t *testing.T) {
-	a := accumulator{}
 	now := time.Now()
-	a.metrics = make(chan telegraf.Metric, 10)
+	metrics := make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
+	defer close(metrics)
-	a.inputConfig = &models.InputConfig{}
+	a := NewAccumulator(&TestMetricMaker{}, metrics)
 
 	a.AddFields("acctest",
 		map[string]interface{}{"value": float64(101)},
@@ -33,97 +30,142 @@ func TestAdd(t *testing.T) {
 		map[string]interface{}{"value": float64(101)},
 		map[string]string{"acc": "test"}, now)
 
-	testm := <-a.metrics
+	testm := <-metrics
 	actual := testm.String()
 	assert.Contains(t, actual, "acctest value=101")
 
-	testm = <-a.metrics
+	testm = <-metrics
 	actual = testm.String()
 	assert.Contains(t, actual, "acctest,acc=test value=101")
 
-	testm = <-a.metrics
+	testm = <-metrics
 	actual = testm.String()
 	assert.Equal(t,
 		fmt.Sprintf("acctest,acc=test value=101 %d", now.UnixNano()),
 		actual)
 }
 
-func TestAddGauge(t *testing.T) {
+func TestAddFields(t *testing.T) {
-	a := accumulator{}
 	now := time.Now()
-	a.metrics = make(chan telegraf.Metric, 10)
+	metrics := make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
+	defer close(metrics)
-	a.inputConfig = &models.InputConfig{}
+	a := NewAccumulator(&TestMetricMaker{}, metrics)
 
-	a.AddGauge("acctest",
+	fields := map[string]interface{}{
-		map[string]interface{}{"value": float64(101)},
+		"usage": float64(99),
-		map[string]string{})
+	}
-	a.AddGauge("acctest",
+	a.AddFields("acctest", fields, map[string]string{})
-		map[string]interface{}{"value": float64(101)},
+	a.AddGauge("acctest", fields, map[string]string{"acc": "test"})
-		map[string]string{"acc": "test"})
+	a.AddCounter("acctest", fields, map[string]string{"acc": "test"}, now)
-	a.AddGauge("acctest",
-		map[string]interface{}{"value": float64(101)},
-		map[string]string{"acc": "test"}, now)
 
-	testm := <-a.metrics
+	testm := <-metrics
 	actual := testm.String()
-	assert.Contains(t, actual, "acctest value=101")
+	assert.Contains(t, actual, "acctest usage=99")
-	assert.Equal(t, testm.Type(), telegraf.Gauge)
 
-	testm = <-a.metrics
+	testm = <-metrics
 	actual = testm.String()
-	assert.Contains(t, actual, "acctest,acc=test value=101")
+	assert.Contains(t, actual, "acctest,acc=test usage=99")
-	assert.Equal(t, testm.Type(), telegraf.Gauge)
 
-	testm = <-a.metrics
+	testm = <-metrics
 	actual = testm.String()
 	assert.Equal(t,
-		fmt.Sprintf("acctest,acc=test value=101 %d", now.UnixNano()),
+		fmt.Sprintf("acctest,acc=test usage=99 %d", now.UnixNano()),
 		actual)
-	assert.Equal(t, testm.Type(), telegraf.Gauge)
 }
 
-func TestAddCounter(t *testing.T) {
+func TestAccAddError(t *testing.T) {
-	a := accumulator{}
+	errBuf := bytes.NewBuffer(nil)
-	now := time.Now()
+	log.SetOutput(errBuf)
-	a.metrics = make(chan telegraf.Metric, 10)
+	defer log.SetOutput(os.Stderr)
-	defer close(a.metrics)
-	a.inputConfig = &models.InputConfig{}
 
-	a.AddCounter("acctest",
+	metrics := make(chan telegraf.Metric, 10)
+	defer close(metrics)
+	a := NewAccumulator(&TestMetricMaker{}, metrics)
+
+	a.AddError(fmt.Errorf("foo"))
+	a.AddError(fmt.Errorf("bar"))
+	a.AddError(fmt.Errorf("baz"))
+
+	errs := bytes.Split(errBuf.Bytes(), []byte{'\n'})
+	assert.EqualValues(t, 3, a.errCount)
+	require.Len(t, errs, 4) // 4 because of trailing newline
+	assert.Contains(t, string(errs[0]), "TestPlugin")
+	assert.Contains(t, string(errs[0]), "foo")
+	assert.Contains(t, string(errs[1]), "TestPlugin")
+	assert.Contains(t, string(errs[1]), "bar")
+	assert.Contains(t, string(errs[2]), "TestPlugin")
+	assert.Contains(t, string(errs[2]), "baz")
+}
+
+func TestAddNoIntervalWithPrecision(t *testing.T) {
+	now := time.Date(2006, time.February, 10, 12, 0, 0, 82912748, time.UTC)
+	metrics := make(chan telegraf.Metric, 10)
+	defer close(metrics)
+	a := NewAccumulator(&TestMetricMaker{}, metrics)
+	a.SetPrecision(0, time.Second)
+
+	a.AddFields("acctest",
 		map[string]interface{}{"value": float64(101)},
 		map[string]string{})
-	a.AddCounter("acctest",
+	a.AddFields("acctest",
 		map[string]interface{}{"value": float64(101)},
 		map[string]string{"acc": "test"})
-	a.AddCounter("acctest",
+	a.AddFields("acctest",
 		map[string]interface{}{"value": float64(101)},
 		map[string]string{"acc": "test"}, now)
 
 	testm := <-a.metrics
 	actual := testm.String()
 	assert.Contains(t, actual, "acctest value=101")
-	assert.Equal(t, testm.Type(), telegraf.Counter)
 
 	testm = <-a.metrics
 	actual = testm.String()
 	assert.Contains(t, actual, "acctest,acc=test value=101")
-	assert.Equal(t, testm.Type(), telegraf.Counter)
 
 	testm = <-a.metrics
 	actual = testm.String()
 	assert.Equal(t,
-		fmt.Sprintf("acctest,acc=test value=101 %d", now.UnixNano()),
+		fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800000000000)),
+		actual)
+}
+
+func TestAddDisablePrecision(t *testing.T) {
+	now := time.Date(2006, time.February, 10, 12, 0, 0, 82912748, time.UTC)
+	metrics := make(chan telegraf.Metric, 10)
+	defer close(metrics)
+	a := NewAccumulator(&TestMetricMaker{}, metrics)
+
+	a.SetPrecision(time.Nanosecond, 0)
+	a.AddFields("acctest",
+		map[string]interface{}{"value": float64(101)},
+		map[string]string{})
+	a.AddFields("acctest",
+		map[string]interface{}{"value": float64(101)},
+		map[string]string{"acc": "test"})
+	a.AddFields("acctest",
+		map[string]interface{}{"value": float64(101)},
+		map[string]string{"acc": "test"}, now)
+
+	testm := <-a.metrics
+	actual := testm.String()
+	assert.Contains(t, actual, "acctest value=101")
+
+	testm = <-a.metrics
+	actual = testm.String()
+	assert.Contains(t, actual, "acctest,acc=test value=101")
+
+	testm = <-a.metrics
+	actual = testm.String()
+	assert.Equal(t,
+		fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800082912748)),
 		actual)
-	assert.Equal(t, testm.Type(), telegraf.Counter)
 }
 
 func TestAddNoPrecisionWithInterval(t *testing.T) {
-	a := accumulator{}
 	now := time.Date(2006, time.February, 10, 12, 0, 0, 82912748, time.UTC)
-	a.metrics = make(chan telegraf.Metric, 10)
+	metrics := make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
+	defer close(metrics)
-	a.inputConfig = &models.InputConfig{}
+	a := NewAccumulator(&TestMetricMaker{}, metrics)
 
 	a.SetPrecision(0, time.Second)
 	a.AddFields("acctest",
@@ -151,79 +193,11 @@ func TestAddNoPrecisionWithInterval(t *testing.T) {
 		actual)
 }
 
-func TestAddNoIntervalWithPrecision(t *testing.T) {
-	a := accumulator{}
-	now := time.Date(2006, time.February, 10, 12, 0, 0, 82912748, time.UTC)
-	a.metrics = make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
-	a.inputConfig = &models.InputConfig{}
-
-	a.SetPrecision(time.Second, time.Millisecond)
-	a.AddFields("acctest",
-		map[string]interface{}{"value": float64(101)},
-		map[string]string{})
-	a.AddFields("acctest",
-		map[string]interface{}{"value": float64(101)},
-		map[string]string{"acc": "test"})
-	a.AddFields("acctest",
-		map[string]interface{}{"value": float64(101)},
-		map[string]string{"acc": "test"}, now)
-
-	testm := <-a.metrics
-	actual := testm.String()
-	assert.Contains(t, actual, "acctest value=101")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Contains(t, actual, "acctest,acc=test value=101")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Equal(t,
-		fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800000000000)),
-		actual)
-}
-
-func TestAddDisablePrecision(t *testing.T) {
-	a := accumulator{}
-	now := time.Date(2006, time.February, 10, 12, 0, 0, 82912748, time.UTC)
-	a.metrics = make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
-	a.inputConfig = &models.InputConfig{}
-
-	a.SetPrecision(time.Second, time.Millisecond)
-	a.DisablePrecision()
-	a.AddFields("acctest",
-		map[string]interface{}{"value": float64(101)},
-		map[string]string{})
-	a.AddFields("acctest",
-		map[string]interface{}{"value": float64(101)},
-		map[string]string{"acc": "test"})
-	a.AddFields("acctest",
-		map[string]interface{}{"value": float64(101)},
-		map[string]string{"acc": "test"}, now)
-
-	testm := <-a.metrics
-	actual := testm.String()
-	assert.Contains(t, actual, "acctest value=101")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Contains(t, actual, "acctest,acc=test value=101")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Equal(t,
-		fmt.Sprintf("acctest,acc=test value=101 %d", int64(1139572800082912748)),
-		actual)
-}
-
 func TestDifferentPrecisions(t *testing.T) {
-	a := accumulator{}
 	now := time.Date(2006, time.February, 10, 12, 0, 0, 82912748, time.UTC)
-	a.metrics = make(chan telegraf.Metric, 10)
+	metrics := make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
+	defer close(metrics)
-	a.inputConfig = &models.InputConfig{}
+	a := NewAccumulator(&TestMetricMaker{}, metrics)
 
 	a.SetPrecision(0, time.Second)
 	a.AddFields("acctest",
@@ -266,349 +240,100 @@ func TestDifferentPrecisions(t *testing.T) {
 		actual)
 }
 
-func TestAddDefaultTags(t *testing.T) {
+func TestAddGauge(t *testing.T) {
-	a := accumulator{}
-	a.addDefaultTag("default", "tag")
 	now := time.Now()
-	a.metrics = make(chan telegraf.Metric, 10)
+	metrics := make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
+	defer close(metrics)
-	a.inputConfig = &models.InputConfig{}
+	a := NewAccumulator(&TestMetricMaker{}, metrics)
 
-	a.AddFields("acctest",
+	a.AddGauge("acctest",
 		map[string]interface{}{"value": float64(101)},
 		map[string]string{})
-	a.AddFields("acctest",
+	a.AddGauge("acctest",
 		map[string]interface{}{"value": float64(101)},
 		map[string]string{"acc": "test"})
-	a.AddFields("acctest",
+	a.AddGauge("acctest",
 		map[string]interface{}{"value": float64(101)},
 		map[string]string{"acc": "test"}, now)
 
-	testm := <-a.metrics
+	testm := <-metrics
-	actual := testm.String()
-	assert.Contains(t, actual, "acctest,default=tag value=101")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Contains(t, actual, "acctest,acc=test,default=tag value=101")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Equal(t,
-		fmt.Sprintf("acctest,acc=test,default=tag value=101 %d", now.UnixNano()),
-		actual)
-}
-
-func TestAddFields(t *testing.T) {
-	a := accumulator{}
-	now := time.Now()
-	a.metrics = make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
-	a.inputConfig = &models.InputConfig{}
-
-	fields := map[string]interface{}{
-		"usage": float64(99),
-	}
-	a.AddFields("acctest", fields, map[string]string{})
-	a.AddFields("acctest", fields, map[string]string{"acc": "test"})
-	a.AddFields("acctest", fields, map[string]string{"acc": "test"}, now)
-
-	testm := <-a.metrics
-	actual := testm.String()
-	assert.Contains(t, actual, "acctest usage=99")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Contains(t, actual, "acctest,acc=test usage=99")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Equal(t,
-		fmt.Sprintf("acctest,acc=test usage=99 %d", now.UnixNano()),
-		actual)
-}
-
-// Test that all Inf fields get dropped, and not added to metrics channel
-func TestAddInfFields(t *testing.T) {
-	inf := math.Inf(1)
-	ninf := math.Inf(-1)
-
-	a := accumulator{}
-	now := time.Now()
-	a.metrics = make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
-	a.inputConfig = &models.InputConfig{}
-
-	fields := map[string]interface{}{
-		"usage":  inf,
-		"nusage": ninf,
-	}
-	a.AddFields("acctest", fields, map[string]string{})
-	a.AddFields("acctest", fields, map[string]string{"acc": "test"})
-	a.AddFields("acctest", fields, map[string]string{"acc": "test"}, now)
-
-	assert.Len(t, a.metrics, 0)
-
-	// test that non-inf fields are kept and not dropped
-	fields["notinf"] = float64(100)
-	a.AddFields("acctest", fields, map[string]string{})
-	testm := <-a.metrics
-	actual := testm.String()
-	assert.Contains(t, actual, "acctest notinf=100")
-}
-
-// Test that nan fields are dropped and not added
-func TestAddNaNFields(t *testing.T) {
-	nan := math.NaN()
-
-	a := accumulator{}
-	now := time.Now()
-	a.metrics = make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
-	a.inputConfig = &models.InputConfig{}
-
-	fields := map[string]interface{}{
-		"usage": nan,
-	}
-	a.AddFields("acctest", fields, map[string]string{})
-	a.AddFields("acctest", fields, map[string]string{"acc": "test"})
-	a.AddFields("acctest", fields, map[string]string{"acc": "test"}, now)
-
-	assert.Len(t, a.metrics, 0)
-
-	// test that non-nan fields are kept and not dropped
-	fields["notnan"] = float64(100)
-	a.AddFields("acctest", fields, map[string]string{})
-	testm := <-a.metrics
-	actual := testm.String()
-	assert.Contains(t, actual, "acctest notnan=100")
-}
-
-func TestAddUint64Fields(t *testing.T) {
-	a := accumulator{}
-	now := time.Now()
-	a.metrics = make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
-	a.inputConfig = &models.InputConfig{}
-
-	fields := map[string]interface{}{
-		"usage": uint64(99),
-	}
-	a.AddFields("acctest", fields, map[string]string{})
-	a.AddFields("acctest", fields, map[string]string{"acc": "test"})
-	a.AddFields("acctest", fields, map[string]string{"acc": "test"}, now)
-
-	testm := <-a.metrics
-	actual := testm.String()
-	assert.Contains(t, actual, "acctest usage=99i")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Contains(t, actual, "acctest,acc=test usage=99i")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Equal(t,
-		fmt.Sprintf("acctest,acc=test usage=99i %d", now.UnixNano()),
-		actual)
-}
-
-func TestAddUint64Overflow(t *testing.T) {
-	a := accumulator{}
-	now := time.Now()
-	a.metrics = make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
-	a.inputConfig = &models.InputConfig{}
-
-	fields := map[string]interface{}{
-		"usage": uint64(9223372036854775808),
-	}
-	a.AddFields("acctest", fields, map[string]string{})
-	a.AddFields("acctest", fields, map[string]string{"acc": "test"})
-	a.AddFields("acctest", fields, map[string]string{"acc": "test"}, now)
-
-	testm := <-a.metrics
-	actual := testm.String()
-	assert.Contains(t, actual, "acctest usage=9223372036854775807i")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Contains(t, actual, "acctest,acc=test usage=9223372036854775807i")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Equal(t,
-		fmt.Sprintf("acctest,acc=test usage=9223372036854775807i %d", now.UnixNano()),
-		actual)
-}
-
-func TestAddInts(t *testing.T) {
-	a := accumulator{}
-	a.addDefaultTag("default", "tag")
-	now := time.Now()
-	a.metrics = make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
-	a.inputConfig = &models.InputConfig{}
-
-	a.AddFields("acctest",
-		map[string]interface{}{"value": int(101)},
-		map[string]string{})
-	a.AddFields("acctest",
-		map[string]interface{}{"value": int32(101)},
-		map[string]string{"acc": "test"})
-	a.AddFields("acctest",
-		map[string]interface{}{"value": int64(101)},
-		map[string]string{"acc": "test"}, now)
-
-	testm := <-a.metrics
-	actual := testm.String()
-	assert.Contains(t, actual, "acctest,default=tag value=101i")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Contains(t, actual, "acctest,acc=test,default=tag value=101i")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Equal(t,
-		fmt.Sprintf("acctest,acc=test,default=tag value=101i %d", now.UnixNano()),
-		actual)
-}
-
-func TestAddFloats(t *testing.T) {
-	a := accumulator{}
-	a.addDefaultTag("default", "tag")
-	now := time.Now()
-	a.metrics = make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
-	a.inputConfig = &models.InputConfig{}
-
-	a.AddFields("acctest",
-		map[string]interface{}{"value": float32(101)},
-		map[string]string{"acc": "test"})
-	a.AddFields("acctest",
-		map[string]interface{}{"value": float64(101)},
-		map[string]string{"acc": "test"}, now)
-
-	testm := <-a.metrics
-	actual := testm.String()
-	assert.Contains(t, actual, "acctest,acc=test,default=tag value=101")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Equal(t,
-		fmt.Sprintf("acctest,acc=test,default=tag value=101 %d", now.UnixNano()),
-		actual)
-}
-
-func TestAddStrings(t *testing.T) {
-	a := accumulator{}
-	a.addDefaultTag("default", "tag")
-	now := time.Now()
-	a.metrics = make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
-	a.inputConfig = &models.InputConfig{}
-
-	a.AddFields("acctest",
-		map[string]interface{}{"value": "test"},
-		map[string]string{"acc": "test"})
-	a.AddFields("acctest",
-		map[string]interface{}{"value": "foo"},
-		map[string]string{"acc": "test"}, now)
-
-	testm := <-a.metrics
-	actual := testm.String()
-	assert.Contains(t, actual, "acctest,acc=test,default=tag value=\"test\"")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Equal(t,
-		fmt.Sprintf("acctest,acc=test,default=tag value=\"foo\" %d", now.UnixNano()),
-		actual)
-}
-
-func TestAddBools(t *testing.T) {
-	a := accumulator{}
-	a.addDefaultTag("default", "tag")
-	now := time.Now()
-	a.metrics = make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
-	a.inputConfig = &models.InputConfig{}
-
-	a.AddFields("acctest",
-		map[string]interface{}{"value": true}, map[string]string{"acc": "test"})
-	a.AddFields("acctest",
-		map[string]interface{}{"value": false}, map[string]string{"acc": "test"}, now)
-
-	testm := <-a.metrics
-	actual := testm.String()
-	assert.Contains(t, actual, "acctest,acc=test,default=tag value=true")
-
-	testm = <-a.metrics
-	actual = testm.String()
-	assert.Equal(t,
-		fmt.Sprintf("acctest,acc=test,default=tag value=false %d", now.UnixNano()),
-		actual)
-}
-
-// Test that tag filters get applied to metrics.
-func TestAccFilterTags(t *testing.T) {
-	a := accumulator{}
-	now := time.Now()
-	a.metrics = make(chan telegraf.Metric, 10)
-	defer close(a.metrics)
-	filter := models.Filter{
-		TagExclude: []string{"acc"},
-	}
-	assert.NoError(t, filter.Compile())
-	a.inputConfig = &models.InputConfig{}
-	a.inputConfig.Filter = filter
-
-	a.AddFields("acctest",
-		map[string]interface{}{"value": float64(101)},
-		map[string]string{})
-	a.AddFields("acctest",
-		map[string]interface{}{"value": float64(101)},
-		map[string]string{"acc": "test"})
-	a.AddFields("acctest",
-		map[string]interface{}{"value": float64(101)},
-		map[string]string{"acc": "test"}, now)
-
-	testm := <-a.metrics
 	actual := testm.String()
 	assert.Contains(t, actual, "acctest value=101")
+	assert.Equal(t, testm.Type(), telegraf.Gauge)
 
-	testm = <-a.metrics
+	testm = <-metrics
 	actual = testm.String()
-	assert.Contains(t, actual, "acctest value=101")
+	assert.Contains(t, actual, "acctest,acc=test value=101")
+	assert.Equal(t, testm.Type(), telegraf.Gauge)
 
-	testm = <-a.metrics
+	testm = <-metrics
 	actual = testm.String()
 	assert.Equal(t,
-		fmt.Sprintf("acctest value=101 %d", now.UnixNano()),
+		fmt.Sprintf("acctest,acc=test value=101 %d", now.UnixNano()),
 		actual)
+	assert.Equal(t, testm.Type(), telegraf.Gauge)
 }
 
-func TestAccAddError(t *testing.T) {
+func TestAddCounter(t *testing.T) {
-	errBuf := bytes.NewBuffer(nil)
+	now := time.Now()
-	log.SetOutput(errBuf)
+	metrics := make(chan telegraf.Metric, 10)
-	defer log.SetOutput(os.Stderr)
+	defer close(metrics)
+	a := NewAccumulator(&TestMetricMaker{}, metrics)
 
-	a := accumulator{}
+	a.AddCounter("acctest",
-	a.inputConfig = &models.InputConfig{}
+		map[string]interface{}{"value": float64(101)},
-	a.inputConfig.Name = "mock_plugin"
+		map[string]string{})
+	a.AddCounter("acctest",
+		map[string]interface{}{"value": float64(101)},
+		map[string]string{"acc": "test"})
+	a.AddCounter("acctest",
+		map[string]interface{}{"value": float64(101)},
+		map[string]string{"acc": "test"}, now)
 
-	a.AddError(fmt.Errorf("foo"))
+	testm := <-metrics
-	a.AddError(fmt.Errorf("bar"))
+	actual := testm.String()
-	a.AddError(fmt.Errorf("baz"))
+	assert.Contains(t, actual, "acctest value=101")
+	assert.Equal(t, testm.Type(), telegraf.Counter)
 
-	errs := bytes.Split(errBuf.Bytes(), []byte{'\n'})
+	testm = <-metrics
-	assert.EqualValues(t, 3, a.errCount)
+	actual = testm.String()
-	require.Len(t, errs, 4) // 4 because of trailing newline
+	assert.Contains(t, actual, "acctest,acc=test value=101")
-	assert.Contains(t, string(errs[0]), "mock_plugin")
+	assert.Equal(t, testm.Type(), telegraf.Counter)
-	assert.Contains(t, string(errs[0]), "foo")
+
-	assert.Contains(t, string(errs[1]), "mock_plugin")
+	testm = <-metrics
-	assert.Contains(t, string(errs[1]), "bar")
+	actual = testm.String()
-	assert.Contains(t, string(errs[2]), "mock_plugin")
+	assert.Equal(t,
-	assert.Contains(t, string(errs[2]), "baz")
+		fmt.Sprintf("acctest,acc=test value=101 %d", now.UnixNano()),
+		actual)
+	assert.Equal(t, testm.Type(), telegraf.Counter)
+}
+
+type TestMetricMaker struct {
+}
+
+func (tm *TestMetricMaker) Name() string {
+	return "TestPlugin"
+}
+func (tm *TestMetricMaker) MakeMetric(
+	measurement string,
+	fields map[string]interface{},
+	tags map[string]string,
+	mType telegraf.ValueType,
+	t time.Time,
+) telegraf.Metric {
+	switch mType {
+	case telegraf.Untyped:
+		if m, err := telegraf.NewMetric(measurement, tags, fields, t); err == nil {
+			return m
+		}
+	case telegraf.Counter:
+		if m, err := telegraf.NewCounterMetric(measurement, tags, fields, t); err == nil {
+			return m
+		}
+	case telegraf.Gauge:
+		if m, err := telegraf.NewGaugeMetric(measurement, tags, fields, t); err == nil {
+			return m
+		}
+	}
+	return nil
 }

agent/agent.go

@@ -89,7 +89,7 @@ func panicRecover(input *models.RunningInput) {
 		trace := make([]byte, 2048)
 		runtime.Stack(trace, true)
 		log.Printf("E! FATAL: Input [%s] panicked: %s, Stack:\n%s\n",
-			input.Name, err, trace)
+			input.Name(), err, trace)
 		log.Println("E! PLEASE REPORT THIS PANIC ON GITHUB with " +
 			"stack trace, configuration, and OS information: " +
 			"https://github.com/influxdata/telegraf/issues/new")
@@ -103,19 +103,18 @@ func (a *Agent) gatherer(
 	input *models.RunningInput,
 	interval time.Duration,
 	metricC chan telegraf.Metric,
-) error {
+) {
 	defer panicRecover(input)
 
 	ticker := time.NewTicker(interval)
 	defer ticker.Stop()
 
 	for {
-		var outerr error
+		acc := NewAccumulator(input, metricC)
-
-		acc := NewAccumulator(input.Config, metricC)
 		acc.SetPrecision(a.Config.Agent.Precision.Duration,
 			a.Config.Agent.Interval.Duration)
-		acc.setDefaultTags(a.Config.Tags)
+		input.SetDebug(a.Config.Agent.Debug)
+		input.SetDefaultTags(a.Config.Tags)
 
 		internal.RandomSleep(a.Config.Agent.CollectionJitter.Duration, shutdown)
 
@@ -123,15 +122,12 @@ func (a *Agent) gatherer(
 		gatherWithTimeout(shutdown, input, acc, interval)
 		elapsed := time.Since(start)
 
-		if outerr != nil {
-			return outerr
-		}
 		log.Printf("D! Input [%s] gathered metrics, (%s interval) in %s\n",
-			input.Name, interval, elapsed)
+			input.Name(), interval, elapsed)
 
 		select {
 		case <-shutdown:
-			return nil
+			return
 		case <-ticker.C:
 			continue
 		}
@@ -160,13 +156,13 @@ func gatherWithTimeout(
 		select {
 		case err := <-done:
 			if err != nil {
-				log.Printf("E! ERROR in input [%s]: %s", input.Name, err)
+				log.Printf("E! ERROR in input [%s]: %s", input.Name(), err)
 			}
 			return
 		case <-ticker.C:
 			log.Printf("E! ERROR: input [%s] took longer to collect than "+
 				"collection interval (%s)",
-				input.Name, timeout)
+				input.Name(), timeout)
 			continue
 		case <-shutdown:
 			return
@@ -194,13 +190,13 @@ func (a *Agent) Test() error {
 	}()
 
 	for _, input := range a.Config.Inputs {
-		acc := NewAccumulator(input.Config, metricC)
+		acc := NewAccumulator(input, metricC)
-		acc.SetTrace(true)
 		acc.SetPrecision(a.Config.Agent.Precision.Duration,
 			a.Config.Agent.Interval.Duration)
-		acc.setDefaultTags(a.Config.Tags)
+		input.SetTrace(true)
+		input.SetDefaultTags(a.Config.Tags)
 
-		fmt.Printf("* Plugin: %s, Collection 1\n", input.Name)
+		fmt.Printf("* Plugin: %s, Collection 1\n", input.Name())
 		if input.Config.Interval != 0 {
 			fmt.Printf("* Internal: %s\n", input.Config.Interval)
 		}
@@ -214,10 +210,10 @@ func (a *Agent) Test() error {
 
 		// Special instructions for some inputs. cpu, for example, needs to be
 		// run twice in order to return cpu usage percentages.
-		switch input.Name {
+		switch input.Name() {
 		case "cpu", "mongodb", "procstat":
 			time.Sleep(500 * time.Millisecond)
-			fmt.Printf("* Plugin: %s, Collection 2\n", input.Name)
+			fmt.Printf("* Plugin: %s, Collection 2\n", input.Name())
 			if err := input.Input.Gather(acc); err != nil {
 				return err
 			}
@@ -250,47 +246,73 @@ func (a *Agent) flush() {
 func (a *Agent) flusher(shutdown chan struct{}, metricC chan telegraf.Metric) error {
 	// Inelegant, but this sleep is to allow the Gather threads to run, so that
 	// the flusher will flush after metrics are collected.
-	time.Sleep(time.Millisecond * 200)
+	time.Sleep(time.Millisecond * 300)
+
+	// create an output metric channel and a gorouting that continously passes
+	// each metric onto the output plugins & aggregators.
+	outMetricC := make(chan telegraf.Metric, 100)
+	var wg sync.WaitGroup
+	wg.Add(1)
+	go func() {
+		defer wg.Done()
+		for {
+			select {
+			case <-shutdown:
+				if len(outMetricC) > 0 {
+					// keep going until outMetricC is flushed
+					continue
+				}
+				return
+			case m := <-outMetricC:
+				// if dropOriginal is set to true, then we will only send this
+				// metric to the aggregators, not the outputs.
+				var dropOriginal bool
+				if !m.IsAggregate() {
+					for _, agg := range a.Config.Aggregators {
+						if ok := agg.Add(copyMetric(m)); ok {
+							dropOriginal = true
+						}
+					}
+				}
+				if !dropOriginal {
+					for i, o := range a.Config.Outputs {
+						if i == len(a.Config.Outputs)-1 {
+							o.AddMetric(m)
+						} else {
+							o.AddMetric(copyMetric(m))
+						}
+					}
+				}
+			}
+		}
+	}()
 
 	ticker := time.NewTicker(a.Config.Agent.FlushInterval.Duration)
-
 	for {
 		select {
 		case <-shutdown:
 			log.Println("I! Hang on, flushing any cached metrics before shutdown")
+			// wait for outMetricC to get flushed before flushing outputs
+			wg.Wait()
 			a.flush()
 			return nil
 		case <-ticker.C:
 			internal.RandomSleep(a.Config.Agent.FlushJitter.Duration, shutdown)
 			a.flush()
-		case m := <-metricC:
+		case metric := <-metricC:
-			for i, o := range a.Config.Outputs {
+			// NOTE potential bottleneck here as we put each metric through the
-				if i == len(a.Config.Outputs)-1 {
+			// processors serially.
-					o.AddMetric(m)
+			mS := []telegraf.Metric{metric}
-				} else {
+			for _, processor := range a.Config.Processors {
-					o.AddMetric(copyMetric(m))
+				mS = processor.Apply(mS...)
-				}
+			}
+			for _, m := range mS {
+				outMetricC <- m
 			}
 		}
 	}
 }
 
-func copyMetric(m telegraf.Metric) telegraf.Metric {
-	t := time.Time(m.Time())
-
-	tags := make(map[string]string)
-	fields := make(map[string]interface{})
-	for k, v := range m.Tags() {
-		tags[k] = v
-	}
-	for k, v := range m.Fields() {
-		fields[k] = v
-	}
-
-	out, _ := telegraf.NewMetric(m.Name(), tags, fields, t)
-	return out
-}
-
 // Run runs the agent daemon, gathering every Interval
 func (a *Agent) Run(shutdown chan struct{}) error {
 	var wg sync.WaitGroup
@@ -301,20 +323,20 @@ func (a *Agent) Run(shutdown chan struct{}) error {
 		a.Config.Agent.Hostname, a.Config.Agent.FlushInterval.Duration)
 
 	// channel shared between all input threads for accumulating metrics
-	metricC := make(chan telegraf.Metric, 10000)
+	metricC := make(chan telegraf.Metric, 100)
 
+	// Start all ServicePlugins
 	for _, input := range a.Config.Inputs {
-		// Start service of any ServicePlugins
 		switch p := input.Input.(type) {
 		case telegraf.ServiceInput:
-			acc := NewAccumulator(input.Config, metricC)
+			acc := NewAccumulator(input, metricC)
 			// Service input plugins should set their own precision of their
 			// metrics.
-			acc.DisablePrecision()
+			acc.SetPrecision(time.Nanosecond, 0)
-			acc.setDefaultTags(a.Config.Tags)
+			input.SetDefaultTags(a.Config.Tags)
 			if err := p.Start(acc); err != nil {
 				log.Printf("E! Service for input %s failed to start, exiting\n%s\n",
-					input.Name, err.Error())
+					input.Name(), err.Error())
 				return err
 			}
 			defer p.Stop()
@@ -336,6 +358,17 @@ func (a *Agent) Run(shutdown chan struct{}) error {
 		}
 	}()
 
+	wg.Add(len(a.Config.Aggregators))
+	for _, aggregator := range a.Config.Aggregators {
+		go func(agg *models.RunningAggregator) {
+			defer wg.Done()
+			acc := NewAccumulator(agg, metricC)
+			acc.SetPrecision(a.Config.Agent.Precision.Duration,
+				a.Config.Agent.Interval.Duration)
+			agg.Run(acc, shutdown)
+		}(aggregator)
+	}
+
 	wg.Add(len(a.Config.Inputs))
 	for _, input := range a.Config.Inputs {
 		interval := a.Config.Agent.Interval.Duration
@@ -345,12 +378,26 @@ func (a *Agent) Run(shutdown chan struct{}) error {
 		}
 		go func(in *models.RunningInput, interv time.Duration) {
 			defer wg.Done()
-			if err := a.gatherer(shutdown, in, interv, metricC); err != nil {
+			a.gatherer(shutdown, in, interv, metricC)
-				log.Printf("E! " + err.Error())
-			}
 		}(input, interval)
 	}
 
 	wg.Wait()
 	return nil
 }
+
+func copyMetric(m telegraf.Metric) telegraf.Metric {
+	t := time.Time(m.Time())
+
+	tags := make(map[string]string)
+	fields := make(map[string]interface{})
+	for k, v := range m.Tags() {
+		tags[k] = v
+	}
+	for k, v := range m.Fields() {
+		fields[k] = v
+	}
+
+	out, _ := telegraf.NewMetric(m.Name(), tags, fields, t)
+	return out
+}

aggregator.go

@@ -0,0 +1,22 @@
+package telegraf
+
+// Aggregator is an interface for implementing an Aggregator plugin.
+// the RunningAggregator wraps this interface and guarantees that
+// Add, Push, and Reset can not be called concurrently, so locking is not
+// required when implementing an Aggregator plugin.
+type Aggregator interface {
+	// SampleConfig returns the default configuration of the Input.
+	SampleConfig() string
+
+	// Description returns a one-sentence description on the Input.
+	Description() string
+
+	// Add the metric to the aggregator.
+	Add(in Metric)
+
+	// Push pushes the current aggregates to the accumulator.
+	Push(acc Accumulator)
+
+	// Reset resets the aggregators caches and aggregates.
+	Reset()
+}

cmd/telegraf/telegraf.go

@@ -13,11 +13,12 @@ import (
 	"github.com/influxdata/telegraf/agent"
 	"github.com/influxdata/telegraf/internal/config"
 	"github.com/influxdata/telegraf/logger"
+	_ "github.com/influxdata/telegraf/plugins/aggregators/all"
 	"github.com/influxdata/telegraf/plugins/inputs"
 	_ "github.com/influxdata/telegraf/plugins/inputs/all"
 	"github.com/influxdata/telegraf/plugins/outputs"
 	_ "github.com/influxdata/telegraf/plugins/outputs/all"
-
+	_ "github.com/influxdata/telegraf/plugins/processors/all"
 	"github.com/kardianos/service"
 )
 
@@ -41,6 +42,10 @@ var fOutputFilters = flag.String("output-filter", "",
 	"filter the outputs to enable, separator is :")
 var fOutputList = flag.Bool("output-list", false,
 	"print available output plugins.")
+var fAggregatorFilters = flag.String("aggregator-filter", "",
+	"filter the aggregators to enable, separator is :")
+var fProcessorFilters = flag.String("processor-filter", "",
+	"filter the processors to enable, separator is :")
 var fUsage = flag.String("usage", "",
 	"print usage for a plugin, ie, 'telegraf -usage mysql'")
 var fService = flag.String("service", "",
@@ -68,47 +73,38 @@ const usage = `Telegraf, The plugin-driven server agent for collecting and repor
 
 Usage:
 
-  telegraf <flags>
+  telegraf [commands|flags]
 
-The flags are:
+The commands & flags are:
 
-  -config <file>     configuration file to load
+  config             print out full sample configuration to stdout
-  -test              gather metrics once, print them to stdout, and exit
+  version            print the version to stdout
-  -sample-config     print out full sample configuration to stdout
-  -config-directory  directory containing additional *.conf files
-  -input-filter      filter the input plugins to enable, separator is :
-  -input-list        print all the plugins inputs
-  -output-filter     filter the output plugins to enable, separator is :
-  -output-list       print all the available outputs
-  -usage             print usage for a plugin, ie, 'telegraf -usage mysql'
-  -debug             print metrics as they're generated to stdout
-  -quiet             run in quiet mode
-  -version           print the version to stdout
-  -service           Control the service, ie, 'telegraf -service install (windows only)'
 
-In addition to the -config flag, telegraf will also load the config file from
+  --config <file>     configuration file to load
-an environment variable or default location. Precedence is:
+  --test              gather metrics once, print them to stdout, and exit
-  1. -config flag
+  --config-directory  directory containing additional *.conf files
-  2. $TELEGRAF_CONFIG_PATH environment variable
+  --input-filter      filter the input plugins to enable, separator is :
-  3. $HOME/.telegraf/telegraf.conf
+  --output-filter     filter the output plugins to enable, separator is :
-  4. /etc/telegraf/telegraf.conf
+  --usage             print usage for a plugin, ie, 'telegraf --usage mysql'
+  --debug             print metrics as they're generated to stdout
+  --quiet             run in quiet mode
 
 Examples:
 
   # generate a telegraf config file:
-  telegraf -sample-config > telegraf.conf
+  telegraf config > telegraf.conf
 
   # generate config with only cpu input & influxdb output plugins defined
-  telegraf -sample-config -input-filter cpu -output-filter influxdb
+  telegraf config -input-filter cpu -output-filter influxdb
 
   # run a single telegraf collection, outputing metrics to stdout
-  telegraf -config telegraf.conf -test
+  telegraf --config telegraf.conf -test
 
   # run telegraf with all plugins defined in config file
-  telegraf -config telegraf.conf
+  telegraf --config telegraf.conf
 
   # run telegraf, enabling the cpu & memory input, and influxdb output plugins
-  telegraf -config telegraf.conf -input-filter cpu:mem -output-filter influxdb
+  telegraf --config telegraf.conf --input-filter cpu:mem --output-filter influxdb
 `
 
 var stop chan struct{}
@@ -128,7 +124,6 @@ func reloadLoop(stop chan struct{}, s service.Service) {
 	reload <- true
 	for <-reload {
 		reload <- false
-		flag.Usage = func() { usageExit(0) }
 		flag.Parse()
 		args := flag.Args()
 
@@ -142,6 +137,16 @@ func reloadLoop(stop chan struct{}, s service.Service) {
 			outputFilter := strings.TrimSpace(*fOutputFilters)
 			outputFilters = strings.Split(":"+outputFilter+":", ":")
 		}
+		var aggregatorFilters []string
+		if *fAggregatorFilters != "" {
+			aggregatorFilter := strings.TrimSpace(*fAggregatorFilters)
+			aggregatorFilters = strings.Split(":"+aggregatorFilter+":", ":")
+		}
+		var processorFilters []string
+		if *fProcessorFilters != "" {
+			processorFilter := strings.TrimSpace(*fProcessorFilters)
+			processorFilters = strings.Split(":"+processorFilter+":", ":")
+		}
 
 		if len(args) > 0 {
 			switch args[0] {
@@ -149,7 +154,12 @@ func reloadLoop(stop chan struct{}, s service.Service) {
 				fmt.Printf("Telegraf v%s (git: %s %s)\n", version, branch, commit)
 				return
 			case "config":
-				config.PrintSampleConfig(inputFilters, outputFilters)
+				config.PrintSampleConfig(
+					inputFilters,
+					outputFilters,
+					aggregatorFilters,
+					processorFilters,
+				)
 				return
 			}
 		}
@@ -172,12 +182,17 @@ func reloadLoop(stop chan struct{}, s service.Service) {
 			fmt.Printf("Telegraf v%s (git: %s %s)\n", version, branch, commit)
 			return
 		case *fSampleConfig:
-			config.PrintSampleConfig(inputFilters, outputFilters)
+			config.PrintSampleConfig(
+				inputFilters,
+				outputFilters,
+				aggregatorFilters,
+				processorFilters,
+			)
 			return
 		case *fUsage != "":
 			if err := config.PrintInputConfig(*fUsage); err != nil {
 				if err2 := config.PrintOutputConfig(*fUsage); err2 != nil {
-					log.Fatalf("%s and %s", err, err2)
+					log.Fatalf("E! %s and %s", err, err2)
 				}
 			}
 			return
@@ -189,26 +204,25 @@ func reloadLoop(stop chan struct{}, s service.Service) {
 		c.InputFilters = inputFilters
 		err := c.LoadConfig(*fConfig)
 		if err != nil {
-			fmt.Println(err)
+			log.Fatal("E! " + err.Error())
-			os.Exit(1)
 		}
 
 		if *fConfigDirectory != "" {
 			err = c.LoadDirectory(*fConfigDirectory)
 			if err != nil {
-				log.Fatal(err)
+				log.Fatal("E! " + err.Error())
 			}
 		}
 		if len(c.Outputs) == 0 {
-			log.Fatalf("Error: no outputs found, did you provide a valid config file?")
+			log.Fatalf("E! Error: no outputs found, did you provide a valid config file?")
 		}
 		if len(c.Inputs) == 0 {
-			log.Fatalf("Error: no inputs found, did you provide a valid config file?")
+			log.Fatalf("E! Error: no inputs found, did you provide a valid config file?")
 		}
 
 		ag, err := agent.NewAgent(c)
 		if err != nil {
-			log.Fatal(err)
+			log.Fatal("E! " + err.Error())
 		}
 
 		// Setup logging
@@ -221,14 +235,14 @@ func reloadLoop(stop chan struct{}, s service.Service) {
 		if *fTest {
 			err = ag.Test()
 			if err != nil {
-				log.Fatal(err)
+				log.Fatal("E! " + err.Error())
 			}
 			return
 		}
 
 		err = ag.Connect()
 		if err != nil {
-			log.Fatal(err)
+			log.Fatal("E! " + err.Error())
 		}
 
 		shutdown := make(chan struct{})
@@ -259,7 +273,7 @@ func reloadLoop(stop chan struct{}, s service.Service) {
 		if *fPidfile != "" {
 			f, err := os.Create(*fPidfile)
 			if err != nil {
-				log.Fatalf("Unable to create pidfile: %s", err)
+				log.Fatalf("E! Unable to create pidfile: %s", err)
 			}
 
 			fmt.Fprintf(f, "%d\n", os.Getpid())
@@ -291,6 +305,7 @@ func (p *program) Stop(s service.Service) error {
 }
 
 func main() {
+	flag.Usage = func() { usageExit(0) }
 	flag.Parse()
 	if runtime.GOOS == "windows" {
 		svcConfig := &service.Config{
@@ -304,7 +319,7 @@ func main() {
 		prg := &program{}
 		s, err := service.New(prg, svcConfig)
 		if err != nil {
-			log.Fatal(err)
+			log.Fatal("E! " + err.Error())
 		}
 		// Handle the -service flag here to prevent any issues with tooling that
 		// may not have an interactive session, e.g. installing from Ansible.
@@ -314,7 +329,7 @@ func main() {
 			}
 			err := service.Control(s, *fService)
 			if err != nil {
-				log.Fatal(err)
+				log.Fatal("E! " + err.Error())
 			}
 		} else {
 			err = s.Run()

docs/CONFIGURATION.md

@@ -1,38 +1,38 @@
 # Telegraf Configuration
 
+You can see the latest config file with all available plugins here:
+[telegraf.conf](https://github.com/influxdata/telegraf/blob/master/etc/telegraf.conf)
+
 ## Generating a Configuration File
 
-A default Telegraf config file can be generated using the -sample-config flag:
+A default Telegraf config file can be auto-generated by telegraf:
 
 ```
-telegraf -sample-config > telegraf.conf
+telegraf config > telegraf.conf
 ```
 
 To generate a file with specific inputs and outputs, you can use the
 -input-filter and -output-filter flags:
 
 ```
-telegraf -sample-config -input-filter cpu:mem:net:swap -output-filter influxdb:kafka
+telegraf config -input-filter cpu:mem:net:swap -output-filter influxdb:kafka
 ```
 
-You can see the latest config file with all available plugins here:
-[telegraf.conf](https://github.com/influxdata/telegraf/blob/master/etc/telegraf.conf)
-
 ## Environment Variables
 
 Environment variables can be used anywhere in the config file, simply prepend
 them with $. For strings the variable must be within quotes (ie, "$STR_VAR"),
 for numbers and booleans they should be plain (ie, $INT_VAR, $BOOL_VAR)
 
-## `[global_tags]` Configuration
+# Global Tags
 
 Global tags can be specified in the `[global_tags]` section of the config file
 in key="value" format. All metrics being gathered on this host will be tagged
 with the tags specified here.
 
-## `[agent]` Configuration
+## Agent Configuration
 
-Telegraf has a few options you can configure under the `agent` section of the
+Telegraf has a few options you can configure under the `[agent]` section of the
 config.
 
 * **interval**: Default data collection interval for all inputs
@@ -56,13 +56,63 @@ interval. Maximum flush_interval will be flush_interval + flush_jitter
 This is primarily to avoid
 large write spikes for users running a large number of telegraf instances.
 ie, a jitter of 5s and flush_interval 10s means flushes will happen every 10-15s.
+* **precision**: By default, precision will be set to the same timestamp order
+as the collection interval, with the maximum being 1s. Precision will NOT
+be used for service inputs, such as logparser and statsd. Valid values are
+"ns", "us" (or "µs"), "ms", "s".
+* **logfile**: Specify the log file name. The empty string means to log to stdout.
 * **debug**: Run telegraf in debug mode.
-* **quiet**: Run telegraf in quiet mode.
+* **quiet**: Run telegraf in quiet mode (error messages only).
 * **hostname**: Override default hostname, if empty use os.Hostname().
+* **omit_hostname**: If true, do no set the "host" tag in the telegraf agent.
+
+## Input Configuration
+
+The following config parameters are available for all inputs:
+
+* **interval**: How often to gather this metric. Normal plugins use a single
+global interval, but if one particular input should be run less or more often,
+you can configure that here.
+* **name_override**: Override the base name of the measurement.
+(Default is the name of the input).
+* **name_prefix**: Specifies a prefix to attach to the measurement name.
+* **name_suffix**: Specifies a suffix to attach to the measurement name.
+* **tags**: A map of tags to apply to a specific input's measurements.
+
+## Output Configuration
+
+There are no generic configuration options available for all outputs.
+
+## Aggregator Configuration
+
+The following config parameters are available for all aggregators:
+
+* **period**: The period on which to flush & clear each aggregator. All metrics
+that are sent with timestamps outside of this period will be ignored by the
+aggregator.
+* **delay**: The delay before each aggregator is flushed. This is to control
+how long for aggregators to wait before receiving metrics from input plugins,
+in the case that aggregators are flushing and inputs are gathering on the
+same interval.
+* **drop_original**: If true, the original metric will be dropped by the
+aggregator and will not get sent to the output plugins.
+* **name_override**: Override the base name of the measurement.
+(Default is the name of the input).
+* **name_prefix**: Specifies a prefix to attach to the measurement name.
+* **name_suffix**: Specifies a suffix to attach to the measurement name.
+* **tags**: A map of tags to apply to a specific input's measurements.
+
+## Processor Configuration
+
+The following config parameters are available for all processors:
+
+* **order**: This is the order in which the processor(s) get executed. If this
+is not specified then processor execution order will be random.
 
 #### Measurement Filtering
 
-Filters can be configured per input or output, see below for examples.
+Filters can be configured per input, output, processor, or aggregator,
+see below for examples.
 
 * **namepass**: An array of strings that is used to filter metrics generated by the
 current input. Each string in the array is tested as a glob match against
@@ -90,19 +140,6 @@ the tag keys in the final measurement.
 the plugin definition, otherwise subsequent plugin config options will be
 interpreted as part of the tagpass/tagdrop map.
 
-## Input Configuration
-
-Some configuration options are configurable per input:
-
-* **name_override**: Override the base name of the measurement.
-(Default is the name of the input).
-* **name_prefix**: Specifies a prefix to attach to the measurement name.
-* **name_suffix**: Specifies a suffix to attach to the measurement name.
-* **tags**: A map of tags to apply to a specific input's measurements.
-* **interval**: How often to gather this metric. Normal plugins use a single
-global interval, but if one particular input should be run less or more often,
-you can configure that here.
-
 #### Input Configuration Examples
 
 This is a full working config that will output CPU data to an InfluxDB instance
@@ -254,11 +291,7 @@ to avoid measurement collisions:
   fielddrop = ["cpu_time*"]
 ```
 
-## Output Configuration
+#### Output Configuration Examples:
-
-Telegraf also supports specifying multiple output sinks to send data to,
-configuring each output sink is different, but examples can be
-found by running `telegraf -sample-config`.
 
 ```toml
 [[outputs.influxdb]]
@@ -283,3 +316,39 @@ found by running `telegraf -sample-config`.
   [outputs.influxdb.tagpass]
     cpu = ["cpu0"]
 ```
+
+#### Aggregator Configuration Examples:
+
+This will collect and emit the min/max of the system load1 metric every
+30s, dropping the originals.
+
+```toml
+[[inputs.system]]
+  fieldpass = ["load1"] # collects system load1 metric.
+
+[[aggregators.minmax]]
+  period = "30s"        # send & clear the aggregate every 30s.
+  drop_original = true  # drop the original metrics.
+
+[[outputs.file]]
+  files = ["stdout"]
+```
+
+This will collect and emit the min/max of the swap metrics every
+30s, dropping the originals. The aggregator will not be applied
+to the system load metrics due to the `namepass` parameter.
+
+```toml
+[[inputs.swap]]
+
+[[inputs.system]]
+  fieldpass = ["load1"] # collects system load1 metric.
+
+[[aggregators.minmax]]
+  period = "30s"        # send & clear the aggregate every 30s.
+  drop_original = true  # drop the original metrics.
+  namepass = ["swap"]   # only "pass" swap metrics through the aggregator.
+
+[[outputs.file]]
+  files = ["stdout"]
+```

docs/DATA_FORMATS_INPUT.md

@@ -232,6 +232,16 @@ us.west.cpu.load 100
 => cpu.load,region=us.west value=100
 ```
 
+Multiple templates can also be specified, but these should be differentiated
+using _filters_ (see below for more details)
+
+```toml
+templates = [
+    "*.*.* region.region.measurement", # <- all 3-part measurements will match this one.
+    "*.*.*.* region.region.host.measurement", # <- all 4-part measurements will match this one.
+]
+```
+
 #### Field Templates:
 
 The field keyword tells Telegraf to give the metric that field name.

internal/config/config.go

@@ -11,15 +11,18 @@ import (
 	"regexp"
 	"runtime"
 	"sort"
+	"strconv"
 	"strings"
 	"time"
 
 	"github.com/influxdata/telegraf"
 	"github.com/influxdata/telegraf/internal"
 	"github.com/influxdata/telegraf/internal/models"
+	"github.com/influxdata/telegraf/plugins/aggregators"
 	"github.com/influxdata/telegraf/plugins/inputs"
 	"github.com/influxdata/telegraf/plugins/outputs"
 	"github.com/influxdata/telegraf/plugins/parsers"
+	"github.com/influxdata/telegraf/plugins/processors"
 	"github.com/influxdata/telegraf/plugins/serializers"
 
 	"github.com/influxdata/config"
@@ -47,9 +50,12 @@ type Config struct {
 	InputFilters  []string
 	OutputFilters []string
 
-	Agent   *AgentConfig
+	Agent       *AgentConfig
-	Inputs  []*models.RunningInput
+	Inputs      []*models.RunningInput
-	Outputs []*models.RunningOutput
+	Outputs     []*models.RunningOutput
+	Aggregators []*models.RunningAggregator
+	// Processors have a slice wrapper type because they need to be sorted
+	Processors models.RunningProcessors
 }
 
 func NewConfig() *Config {
@@ -64,6 +70,7 @@ func NewConfig() *Config {
 		Tags:          make(map[string]string),
 		Inputs:        make([]*models.RunningInput, 0),
 		Outputs:       make([]*models.RunningOutput, 0),
+		Processors:    make([]*models.RunningProcessor, 0),
 		InputFilters:  make([]string, 0),
 		OutputFilters: make([]string, 0),
 	}
@@ -138,7 +145,7 @@ type AgentConfig struct {
 func (c *Config) InputNames() []string {
 	var name []string
 	for _, input := range c.Inputs {
-		name = append(name, input.Name)
+		name = append(name, input.Name())
 	}
 	return name
 }
@@ -234,7 +241,7 @@ var header = `# Telegraf Configuration
   debug = false
   ## Run telegraf in quiet mode (error log messages only).
   quiet = false
-  ## Specify the log file name. The empty string means to log to stdout.
+  ## Specify the log file name. The empty string means to log to stderr.
   logfile = ""
 
   ## Override default hostname, if empty use os.Hostname()
@@ -248,6 +255,20 @@ var header = `# Telegraf Configuration
 ###############################################################################
 `
 
+var processorHeader = `
+
+###############################################################################
+#                            PROCESSOR PLUGINS                                #
+###############################################################################
+`
+
+var aggregatorHeader = `
+
+###############################################################################
+#                            AGGREGATOR PLUGINS                               #
+###############################################################################
+`
+
 var inputHeader = `
 
 ###############################################################################
@@ -263,9 +284,15 @@ var serviceInputHeader = `
 `
 
 // PrintSampleConfig prints the sample config
-func PrintSampleConfig(inputFilters []string, outputFilters []string) {
+func PrintSampleConfig(
+	inputFilters []string,
+	outputFilters []string,
+	aggregatorFilters []string,
+	processorFilters []string,
+) {
 	fmt.Printf(header)
 
+	// print output plugins
 	if len(outputFilters) != 0 {
 		printFilteredOutputs(outputFilters, false)
 	} else {
@@ -281,6 +308,33 @@ func PrintSampleConfig(inputFilters []string, outputFilters []string) {
 		printFilteredOutputs(pnames, true)
 	}
 
+	// print processor plugins
+	fmt.Printf(processorHeader)
+	if len(processorFilters) != 0 {
+		printFilteredProcessors(processorFilters, false)
+	} else {
+		pnames := []string{}
+		for pname := range processors.Processors {
+			pnames = append(pnames, pname)
+		}
+		sort.Strings(pnames)
+		printFilteredProcessors(pnames, true)
+	}
+
+	// pring aggregator plugins
+	fmt.Printf(aggregatorHeader)
+	if len(aggregatorFilters) != 0 {
+		printFilteredAggregators(aggregatorFilters, false)
+	} else {
+		pnames := []string{}
+		for pname := range aggregators.Aggregators {
+			pnames = append(pnames, pname)
+		}
+		sort.Strings(pnames)
+		printFilteredAggregators(pnames, true)
+	}
+
+	// print input plugins
 	fmt.Printf(inputHeader)
 	if len(inputFilters) != 0 {
 		printFilteredInputs(inputFilters, false)
@@ -298,6 +352,42 @@ func PrintSampleConfig(inputFilters []string, outputFilters []string) {
 	}
 }
 
+func printFilteredProcessors(processorFilters []string, commented bool) {
+	// Filter processors
+	var pnames []string
+	for pname := range processors.Processors {
+		if sliceContains(pname, processorFilters) {
+			pnames = append(pnames, pname)
+		}
+	}
+	sort.Strings(pnames)
+
+	// Print Outputs
+	for _, pname := range pnames {
+		creator := processors.Processors[pname]
+		output := creator()
+		printConfig(pname, output, "processors", commented)
+	}
+}
+
+func printFilteredAggregators(aggregatorFilters []string, commented bool) {
+	// Filter outputs
+	var anames []string
+	for aname := range aggregators.Aggregators {
+		if sliceContains(aname, aggregatorFilters) {
+			anames = append(anames, aname)
+		}
+	}
+	sort.Strings(anames)
+
+	// Print Outputs
+	for _, aname := range anames {
+		creator := aggregators.Aggregators[aname]
+		output := creator()
+		printConfig(aname, output, "aggregators", commented)
+	}
+}
+
 func printFilteredInputs(inputFilters []string, commented bool) {
 	// Filter inputs
 	var pnames []string
@@ -507,6 +597,7 @@ func (c *Config) LoadConfig(path string) error {
 		case "outputs":
 			for pluginName, pluginVal := range subTable.Fields {
 				switch pluginSubTable := pluginVal.(type) {
+				// legacy [outputs.influxdb] support
 				case *ast.Table:
 					if err = c.addOutput(pluginName, pluginSubTable); err != nil {
 						return fmt.Errorf("Error parsing %s, %s", path, err)
@@ -525,6 +616,7 @@ func (c *Config) LoadConfig(path string) error {
 		case "inputs", "plugins":
 			for pluginName, pluginVal := range subTable.Fields {
 				switch pluginSubTable := pluginVal.(type) {
+				// legacy [inputs.cpu] support
 				case *ast.Table:
 					if err = c.addInput(pluginName, pluginSubTable); err != nil {
 						return fmt.Errorf("Error parsing %s, %s", path, err)
@@ -540,6 +632,34 @@ func (c *Config) LoadConfig(path string) error {
 						pluginName, path)
 				}
 			}
+		case "processors":
+			for pluginName, pluginVal := range subTable.Fields {
+				switch pluginSubTable := pluginVal.(type) {
+				case []*ast.Table:
+					for _, t := range pluginSubTable {
+						if err = c.addProcessor(pluginName, t); err != nil {
+							return fmt.Errorf("Error parsing %s, %s", path, err)
+						}
+					}
+				default:
+					return fmt.Errorf("Unsupported config format: %s, file %s",
+						pluginName, path)
+				}
+			}
+		case "aggregators":
+			for pluginName, pluginVal := range subTable.Fields {
+				switch pluginSubTable := pluginVal.(type) {
+				case []*ast.Table:
+					for _, t := range pluginSubTable {
+						if err = c.addAggregator(pluginName, t); err != nil {
+							return fmt.Errorf("Error parsing %s, %s", path, err)
+						}
+					}
+				default:
+					return fmt.Errorf("Unsupported config format: %s, file %s",
+						pluginName, path)
+				}
+			}
 		// Assume it's an input input for legacy config file support if no other
 		// identifiers are present
 		default:
@@ -548,6 +668,10 @@ func (c *Config) LoadConfig(path string) error {
 			}
 		}
 	}
+
+	if len(c.Processors) > 1 {
+		sort.Sort(c.Processors)
+	}
 	return nil
 }
 
@@ -580,6 +704,52 @@ func parseFile(fpath string) (*ast.Table, error) {
 	return toml.Parse(contents)
 }
 
+func (c *Config) addAggregator(name string, table *ast.Table) error {
+	creator, ok := aggregators.Aggregators[name]
+	if !ok {
+		return fmt.Errorf("Undefined but requested aggregator: %s", name)
+	}
+	aggregator := creator()
+
+	conf, err := buildAggregator(name, table)
+	if err != nil {
+		return err
+	}
+
+	if err := config.UnmarshalTable(table, aggregator); err != nil {
+		return err
+	}
+
+	c.Aggregators = append(c.Aggregators, models.NewRunningAggregator(aggregator, conf))
+	return nil
+}
+
+func (c *Config) addProcessor(name string, table *ast.Table) error {
+	creator, ok := processors.Processors[name]
+	if !ok {
+		return fmt.Errorf("Undefined but requested processor: %s", name)
+	}
+	processor := creator()
+
+	processorConfig, err := buildProcessor(name, table)
+	if err != nil {
+		return err
+	}
+
+	if err := config.UnmarshalTable(table, processor); err != nil {
+		return err
+	}
+
+	rf := &models.RunningProcessor{
+		Name:      name,
+		Processor: processor,
+		Config:    processorConfig,
+	}
+
+	c.Processors = append(c.Processors, rf)
+	return nil
+}
+
 func (c *Config) addOutput(name string, table *ast.Table) error {
 	if len(c.OutputFilters) > 0 && !sliceContains(name, c.OutputFilters) {
 		return nil
@@ -652,7 +822,6 @@ func (c *Config) addInput(name string, table *ast.Table) error {
 	}
 
 	rp := &models.RunningInput{
-		Name:   name,
 		Input:  input,
 		Config: pluginConfig,
 	}
@@ -660,6 +829,144 @@ func (c *Config) addInput(name string, table *ast.Table) error {
 	return nil
 }
 
+// buildAggregator parses Aggregator specific items from the ast.Table,
+// builds the filter and returns a
+// models.AggregatorConfig to be inserted into models.RunningAggregator
+func buildAggregator(name string, tbl *ast.Table) (*models.AggregatorConfig, error) {
+	unsupportedFields := []string{"tagexclude", "taginclude"}
+	for _, field := range unsupportedFields {
+		if _, ok := tbl.Fields[field]; ok {
+			return nil, fmt.Errorf("%s is not supported for aggregator plugins (%s).",
+				field, name)
+		}
+	}
+
+	conf := &models.AggregatorConfig{
+		Name:   name,
+		Delay:  time.Millisecond * 100,
+		Period: time.Second * 30,
+	}
+
+	if node, ok := tbl.Fields["period"]; ok {
+		if kv, ok := node.(*ast.KeyValue); ok {
+			if str, ok := kv.Value.(*ast.String); ok {
+				dur, err := time.ParseDuration(str.Value)
+				if err != nil {
+					return nil, err
+				}
+
+				conf.Period = dur
+			}
+		}
+	}
+
+	if node, ok := tbl.Fields["delay"]; ok {
+		if kv, ok := node.(*ast.KeyValue); ok {
+			if str, ok := kv.Value.(*ast.String); ok {
+				dur, err := time.ParseDuration(str.Value)
+				if err != nil {
+					return nil, err
+				}
+
+				conf.Delay = dur
+			}
+		}
+	}
+
+	if node, ok := tbl.Fields["drop_original"]; ok {
+		if kv, ok := node.(*ast.KeyValue); ok {
+			if b, ok := kv.Value.(*ast.Boolean); ok {
+				var err error
+				conf.DropOriginal, err = strconv.ParseBool(b.Value)
+				if err != nil {
+					log.Printf("Error parsing boolean value for %s: %s\n", name, err)
+				}
+			}
+		}
+	}
+
+	if node, ok := tbl.Fields["name_prefix"]; ok {
+		if kv, ok := node.(*ast.KeyValue); ok {
+			if str, ok := kv.Value.(*ast.String); ok {
+				conf.MeasurementPrefix = str.Value
+			}
+		}
+	}
+
+	if node, ok := tbl.Fields["name_suffix"]; ok {
+		if kv, ok := node.(*ast.KeyValue); ok {
+			if str, ok := kv.Value.(*ast.String); ok {
+				conf.MeasurementSuffix = str.Value
+			}
+		}
+	}
+
+	if node, ok := tbl.Fields["name_override"]; ok {
+		if kv, ok := node.(*ast.KeyValue); ok {
+			if str, ok := kv.Value.(*ast.String); ok {
+				conf.NameOverride = str.Value
+			}
+		}
+	}
+
+	conf.Tags = make(map[string]string)
+	if node, ok := tbl.Fields["tags"]; ok {
+		if subtbl, ok := node.(*ast.Table); ok {
+			if err := config.UnmarshalTable(subtbl, conf.Tags); err != nil {
+				log.Printf("Could not parse tags for input %s\n", name)
+			}
+		}
+	}
+
+	delete(tbl.Fields, "period")
+	delete(tbl.Fields, "delay")
+	delete(tbl.Fields, "drop_original")
+	delete(tbl.Fields, "name_prefix")
+	delete(tbl.Fields, "name_suffix")
+	delete(tbl.Fields, "name_override")
+	delete(tbl.Fields, "tags")
+	var err error
+	conf.Filter, err = buildFilter(tbl)
+	if err != nil {
+		return conf, err
+	}
+	return conf, nil
+}
+
+// buildProcessor parses Processor specific items from the ast.Table,
+// builds the filter and returns a
+// models.ProcessorConfig to be inserted into models.RunningProcessor
+func buildProcessor(name string, tbl *ast.Table) (*models.ProcessorConfig, error) {
+	conf := &models.ProcessorConfig{Name: name}
+	unsupportedFields := []string{"tagexclude", "taginclude", "fielddrop", "fieldpass"}
+	for _, field := range unsupportedFields {
+		if _, ok := tbl.Fields[field]; ok {
+			return nil, fmt.Errorf("%s is not supported for processor plugins (%s).",
+				field, name)
+		}
+	}
+
+	if node, ok := tbl.Fields["order"]; ok {
+		if kv, ok := node.(*ast.KeyValue); ok {
+			if b, ok := kv.Value.(*ast.Integer); ok {
+				var err error
+				conf.Order, err = strconv.ParseInt(b.Value, 10, 64)
+				if err != nil {
+					log.Printf("Error parsing int value for %s: %s\n", name, err)
+				}
+			}
+		}
+	}
+
+	delete(tbl.Fields, "order")
+	var err error
+	conf.Filter, err = buildFilter(tbl)
+	if err != nil {
+		return conf, err
+	}
+	return conf, nil
+}
+
 // buildFilter builds a Filter
 // (tagpass/tagdrop/namepass/namedrop/fieldpass/fielddrop) to
 // be inserted into the models.OutputConfig/models.InputConfig

internal/models/filter.go

@@ -96,7 +96,7 @@ func (f *Filter) Compile() error {
 // Apply applies the filter to the given measurement name, fields map, and
 // tags map. It will return false if the metric should be "filtered out", and
 // true if the metric should "pass".
-// It will modify tags in-place if they need to be deleted.
+// It will modify tags & fields in-place if they need to be deleted.
 func (f *Filter) Apply(
 	measurement string,
 	fields map[string]interface{},

internal/models/makemetric.go

@@ -0,0 +1,154 @@
+package models
+
+import (
+	"log"
+	"math"
+	"time"
+
+	"github.com/influxdata/telegraf"
+)
+
+// makemetric is used by both RunningAggregator & RunningInput
+// to make metrics.
+//   nameOverride: override the name of the measurement being made.
+//   namePrefix:   add this prefix to each measurement name.
+//   nameSuffix:   add this suffix to each measurement name.
+//   pluginTags:   these are tags that are specific to this plugin.
+//   daemonTags:   these are daemon-wide global tags, and get applied after pluginTags.
+//   filter:       this is a filter to apply to each metric being made.
+//   applyFilter:  if false, the above filter is not applied to each metric.
+//                 This is used by Aggregators, because aggregators use filters
+//                 on incoming metrics instead of on created metrics.
+// TODO refactor this to not have such a huge func signature.
+func makemetric(
+	measurement string,
+	fields map[string]interface{},
+	tags map[string]string,
+	nameOverride string,
+	namePrefix string,
+	nameSuffix string,
+	pluginTags map[string]string,
+	daemonTags map[string]string,
+	filter Filter,
+	applyFilter bool,
+	debug bool,
+	mType telegraf.ValueType,
+	t time.Time,
+) telegraf.Metric {
+	if len(fields) == 0 || len(measurement) == 0 {
+		return nil
+	}
+	if tags == nil {
+		tags = make(map[string]string)
+	}
+
+	// Override measurement name if set
+	if len(nameOverride) != 0 {
+		measurement = nameOverride
+	}
+	// Apply measurement prefix and suffix if set
+	if len(namePrefix) != 0 {
+		measurement = namePrefix + measurement
+	}
+	if len(nameSuffix) != 0 {
+		measurement = measurement + nameSuffix
+	}
+
+	// Apply plugin-wide tags if set
+	for k, v := range pluginTags {
+		if _, ok := tags[k]; !ok {
+			tags[k] = v
+		}
+	}
+	// Apply daemon-wide tags if set
+	for k, v := range daemonTags {
+		if _, ok := tags[k]; !ok {
+			tags[k] = v
+		}
+	}
+
+	// Apply the metric filter(s)
+	// for aggregators, the filter does not get applied when the metric is made.
+	// instead, the filter is applied to metric incoming into the plugin.
+	//   ie, it gets applied in the RunningAggregator.Apply function.
+	if applyFilter {
+		if ok := filter.Apply(measurement, fields, tags); !ok {
+			return nil
+		}
+	}
+
+	for k, v := range fields {
+		// Validate uint64 and float64 fields
+		// convert all int & uint types to int64
+		switch val := v.(type) {
+		case nil:
+			// delete nil fields
+			delete(fields, k)
+		case uint:
+			fields[k] = int64(val)
+			continue
+		case uint8:
+			fields[k] = int64(val)
+			continue
+		case uint16:
+			fields[k] = int64(val)
+			continue
+		case uint32:
+			fields[k] = int64(val)
+			continue
+		case int:
+			fields[k] = int64(val)
+			continue
+		case int8:
+			fields[k] = int64(val)
+			continue
+		case int16:
+			fields[k] = int64(val)
+			continue
+		case int32:
+			fields[k] = int64(val)
+			continue
+		case uint64:
+			// InfluxDB does not support writing uint64
+			if val < uint64(9223372036854775808) {
+				fields[k] = int64(val)
+			} else {
+				fields[k] = int64(9223372036854775807)
+			}
+			continue
+		case float32:
+			fields[k] = float64(val)
+			continue
+		case float64:
+			// NaNs are invalid values in influxdb, skip measurement
+			if math.IsNaN(val) || math.IsInf(val, 0) {
+				if debug {
+					log.Printf("Measurement [%s] field [%s] has a NaN or Inf "+
+						"field, skipping",
+						measurement, k)
+				}
+				delete(fields, k)
+				continue
+			}
+		default:
+			fields[k] = v
+		}
+	}
+
+	var m telegraf.Metric
+	var err error
+	switch mType {
+	case telegraf.Counter:
+		m, err = telegraf.NewCounterMetric(measurement, tags, fields, t)
+	case telegraf.Gauge:
+		m, err = telegraf.NewGaugeMetric(measurement, tags, fields, t)
+	default:
+		m, err = telegraf.NewMetric(measurement, tags, fields, t)
+	}
+	if err != nil {
+		log.Printf("Error adding point [%s]: %s\n", measurement, err.Error())
+		return nil
+	}
+
+	return m
+}

internal/models/running_aggregator.go

@@ -0,0 +1,164 @@
+package models
+
+import (
+	"time"
+
+	"github.com/influxdata/telegraf"
+)
+
+type RunningAggregator struct {
+	a      telegraf.Aggregator
+	Config *AggregatorConfig
+
+	metrics chan telegraf.Metric
+
+	periodStart time.Time
+	periodEnd   time.Time
+}
+
+func NewRunningAggregator(
+	a telegraf.Aggregator,
+	conf *AggregatorConfig,
+) *RunningAggregator {
+	return &RunningAggregator{
+		a:       a,
+		Config:  conf,
+		metrics: make(chan telegraf.Metric, 100),
+	}
+}
+
+// AggregatorConfig containing configuration parameters for the running
+// aggregator plugin.
+type AggregatorConfig struct {
+	Name string
+
+	DropOriginal      bool
+	NameOverride      string
+	MeasurementPrefix string
+	MeasurementSuffix string
+	Tags              map[string]string
+	Filter            Filter
+
+	Period time.Duration
+	Delay  time.Duration
+}
+
+func (r *RunningAggregator) Name() string {
+	return "aggregators." + r.Config.Name
+}
+
+func (r *RunningAggregator) MakeMetric(
+	measurement string,
+	fields map[string]interface{},
+	tags map[string]string,
+	mType telegraf.ValueType,
+	t time.Time,
+) telegraf.Metric {
+	m := makemetric(
+		measurement,
+		fields,
+		tags,
+		r.Config.NameOverride,
+		r.Config.MeasurementPrefix,
+		r.Config.MeasurementSuffix,
+		r.Config.Tags,
+		nil,
+		r.Config.Filter,
+		false,
+		false,
+		mType,
+		t,
+	)
+
+	m.SetAggregate(true)
+
+	return m
+}
+
+// Add applies the given metric to the aggregator.
+// Before applying to the plugin, it will run any defined filters on the metric.
+// Apply returns true if the original metric should be dropped.
+func (r *RunningAggregator) Add(in telegraf.Metric) bool {
+	if r.Config.Filter.IsActive() {
+		// check if the aggregator should apply this metric
+		name := in.Name()
+		fields := in.Fields()
+		tags := in.Tags()
+		t := in.Time()
+		if ok := r.Config.Filter.Apply(name, fields, tags); !ok {
+			// aggregator should not apply this metric
+			return false
+		}
+
+		in, _ = telegraf.NewMetric(name, tags, fields, t)
+	}
+
+	r.metrics <- in
+	return r.Config.DropOriginal
+}
+func (r *RunningAggregator) add(in telegraf.Metric) {
+	r.a.Add(in)
+}
+
+func (r *RunningAggregator) push(acc telegraf.Accumulator) {
+	r.a.Push(acc)
+}
+
+func (r *RunningAggregator) reset() {
+	r.a.Reset()
+}
+
+// Run runs the running aggregator, listens for incoming metrics, and waits
+// for period ticks to tell it when to push and reset the aggregator.
+func (r *RunningAggregator) Run(
+	acc telegraf.Accumulator,
+	shutdown chan struct{},
+) {
+	// The start of the period is truncated to the nearest second.
+	//
+	// Every metric then gets it's timestamp checked and is dropped if it
+	// is not within:
+	//
+	//   start < t < end + truncation + delay
+	//
+	// So if we start at now = 00:00.2 with a 10s period and 0.3s delay:
+	//   now = 00:00.2
+	//   start = 00:00
+	//   truncation = 00:00.2
+	//   end = 00:10
+	// 1st interval: 00:00 - 00:10.5
+	// 2nd interval: 00:10 - 00:20.5
+	// etc.
+	//
+	now := time.Now()
+	r.periodStart = now.Truncate(time.Second)
+	truncation := now.Sub(r.periodStart)
+	r.periodEnd = r.periodStart.Add(r.Config.Period)
+	time.Sleep(r.Config.Delay)
+	periodT := time.NewTicker(r.Config.Period)
+	defer periodT.Stop()
+
+	for {
+		select {
+		case <-shutdown:
+			if len(r.metrics) > 0 {
+				// wait until metrics are flushed before exiting
+				continue
+			}
+			return
+		case m := <-r.metrics:
+			if m.Time().Before(r.periodStart) ||
+				m.Time().After(r.periodEnd.Add(truncation).Add(r.Config.Delay)) {
+				// the metric is outside the current aggregation period, so
+				// skip it.
+				continue
+			}
+			r.add(m)
+		case <-periodT.C:
+			r.periodStart = r.periodEnd
+			r.periodEnd = r.periodStart.Add(r.Config.Period)
+			r.push(acc)
+			r.reset()
+		}
+	}
+}

internal/models/running_aggregator_test.go

@@ -0,0 +1,256 @@
+package models
+
+import (
+	"fmt"
+	"sync"
+	"sync/atomic"
+	"testing"
+	"time"
+
+	"github.com/influxdata/telegraf"
+	"github.com/influxdata/telegraf/testutil"
+
+	"github.com/stretchr/testify/assert"
+)
+
+func TestAdd(t *testing.T) {
+	a := &TestAggregator{}
+	ra := NewRunningAggregator(a, &AggregatorConfig{
+		Name: "TestRunningAggregator",
+		Filter: Filter{
+			NamePass: []string{"*"},
+		},
+		Period: time.Millisecond * 500,
+	})
+	assert.NoError(t, ra.Config.Filter.Compile())
+	acc := testutil.Accumulator{}
+	go ra.Run(&acc, make(chan struct{}))
+
+	m := ra.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Untyped,
+		time.Now().Add(time.Millisecond*150),
+	)
+	assert.False(t, ra.Add(m))
+
+	for {
+		time.Sleep(time.Millisecond)
+		if atomic.LoadInt64(&a.sum) > 0 {
+			break
+		}
+	}
+	assert.Equal(t, int64(101), atomic.LoadInt64(&a.sum))
+}
+
+func TestAddMetricsOutsideCurrentPeriod(t *testing.T) {
+	a := &TestAggregator{}
+	ra := NewRunningAggregator(a, &AggregatorConfig{
+		Name: "TestRunningAggregator",
+		Filter: Filter{
+			NamePass: []string{"*"},
+		},
+		Period: time.Millisecond * 500,
+	})
+	assert.NoError(t, ra.Config.Filter.Compile())
+	acc := testutil.Accumulator{}
+	go ra.Run(&acc, make(chan struct{}))
+
+	// metric before current period
+	m := ra.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Untyped,
+		time.Now().Add(-time.Hour),
+	)
+	assert.False(t, ra.Add(m))
+
+	// metric after current period
+	m = ra.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Untyped,
+		time.Now().Add(time.Hour),
+	)
+	assert.False(t, ra.Add(m))
+
+	// "now" metric
+	m = ra.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Untyped,
+		time.Now().Add(time.Millisecond*50),
+	)
+	assert.False(t, ra.Add(m))
+
+	for {
+		time.Sleep(time.Millisecond)
+		if atomic.LoadInt64(&a.sum) > 0 {
+			break
+		}
+	}
+	assert.Equal(t, int64(101), atomic.LoadInt64(&a.sum))
+}
+
+func TestAddAndPushOnePeriod(t *testing.T) {
+	a := &TestAggregator{}
+	ra := NewRunningAggregator(a, &AggregatorConfig{
+		Name: "TestRunningAggregator",
+		Filter: Filter{
+			NamePass: []string{"*"},
+		},
+		Period: time.Millisecond * 500,
+	})
+	assert.NoError(t, ra.Config.Filter.Compile())
+	acc := testutil.Accumulator{}
+	shutdown := make(chan struct{})
+
+	var wg sync.WaitGroup
+	wg.Add(1)
+	go func() {
+		defer wg.Done()
+		ra.Run(&acc, shutdown)
+	}()
+
+	m := ra.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Untyped,
+		time.Now().Add(time.Millisecond*100),
+	)
+	assert.False(t, ra.Add(m))
+
+	for {
+		time.Sleep(time.Millisecond)
+		if acc.NMetrics() > 0 {
+			break
+		}
+	}
+	acc.AssertContainsFields(t, "TestMetric", map[string]interface{}{"sum": int64(101)})
+
+	close(shutdown)
+	wg.Wait()
+}
+
+func TestAddDropOriginal(t *testing.T) {
+	ra := NewRunningAggregator(&TestAggregator{}, &AggregatorConfig{
+		Name: "TestRunningAggregator",
+		Filter: Filter{
+			NamePass: []string{"RI*"},
+		},
+		DropOriginal: true,
+	})
+	assert.NoError(t, ra.Config.Filter.Compile())
+
+	m := ra.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Untyped,
+		time.Now(),
+	)
+	assert.True(t, ra.Add(m))
+
+	// this metric name doesn't match the filter, so Add will return false
+	m2 := ra.MakeMetric(
+		"foobar",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Untyped,
+		time.Now(),
+	)
+	assert.False(t, ra.Add(m2))
+}
+
+// make an untyped, counter, & gauge metric
+func TestMakeMetricA(t *testing.T) {
+	now := time.Now()
+	ra := NewRunningAggregator(&TestAggregator{}, &AggregatorConfig{
+		Name: "TestRunningAggregator",
+	})
+	assert.Equal(t, "aggregators.TestRunningAggregator", ra.Name())
+
+	m := ra.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Untyped,
+		now,
+	)
+	assert.Equal(
+		t,
+		m.String(),
+		fmt.Sprintf("RITest value=101i %d", now.UnixNano()),
+	)
+	assert.Equal(
+		t,
+		m.Type(),
+		telegraf.Untyped,
+	)
+
+	m = ra.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Counter,
+		now,
+	)
+	assert.Equal(
+		t,
+		m.String(),
+		fmt.Sprintf("RITest value=101i %d", now.UnixNano()),
+	)
+	assert.Equal(
+		t,
+		m.Type(),
+		telegraf.Counter,
+	)
+
+	m = ra.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Gauge,
+		now,
+	)
+	assert.Equal(
+		t,
+		m.String(),
+		fmt.Sprintf("RITest value=101i %d", now.UnixNano()),
+	)
+	assert.Equal(
+		t,
+		m.Type(),
+		telegraf.Gauge,
+	)
+}
+
+type TestAggregator struct {
+	sum int64
+}
+
+func (t *TestAggregator) Description() string  { return "" }
+func (t *TestAggregator) SampleConfig() string { return "" }
+func (t *TestAggregator) Reset() {
+	atomic.StoreInt64(&t.sum, 0)
+}
+
+func (t *TestAggregator) Push(acc telegraf.Accumulator) {
+	acc.AddFields("TestMetric",
+		map[string]interface{}{"sum": t.sum},
+		map[string]string{},
+	)
+}
+
+func (t *TestAggregator) Add(in telegraf.Metric) {
+	for _, v := range in.Fields() {
+		if vi, ok := v.(int64); ok {
+			atomic.AddInt64(&t.sum, vi)
+		}
+	}
+}

internal/models/running_input.go

@@ -1,15 +1,19 @@
 package models
 
 import (
+	"fmt"
 	"time"
 
 	"github.com/influxdata/telegraf"
 )
 
 type RunningInput struct {
-	Name   string
 	Input  telegraf.Input
 	Config *InputConfig
+
+	trace       bool
+	debug       bool
+	defaultTags map[string]string
 }
 
 // InputConfig containing a name, interval, and filter
@@ -22,3 +26,59 @@ type InputConfig struct {
 	Filter            Filter
 	Interval          time.Duration
 }
+
+func (r *RunningInput) Name() string {
+	return "inputs." + r.Config.Name
+}
+
+// MakeMetric either returns a metric, or returns nil if the metric doesn't
+// need to be created (because of filtering, an error, etc.)
+func (r *RunningInput) MakeMetric(
+	measurement string,
+	fields map[string]interface{},
+	tags map[string]string,
+	mType telegraf.ValueType,
+	t time.Time,
+) telegraf.Metric {
+	m := makemetric(
+		measurement,
+		fields,
+		tags,
+		r.Config.NameOverride,
+		r.Config.MeasurementPrefix,
+		r.Config.MeasurementSuffix,
+		r.Config.Tags,
+		r.defaultTags,
+		r.Config.Filter,
+		true,
+		r.debug,
+		mType,
+		t,
+	)
+
+	if r.trace && m != nil {
+		fmt.Println("> " + m.String())
+	}
+
+	return m
+}
+
+func (r *RunningInput) Debug() bool {
+	return r.debug
+}
+
+func (r *RunningInput) SetDebug(debug bool) {
+	r.debug = debug
+}
+
+func (r *RunningInput) Trace() bool {
+	return r.trace
+}
+
+func (r *RunningInput) SetTrace(trace bool) {
+	r.trace = trace
+}
+
+func (r *RunningInput) SetDefaultTags(tags map[string]string) {
+	r.defaultTags = tags
+}

internal/models/running_input_test.go

@@ -0,0 +1,352 @@
+package models
+
+import (
+	"fmt"
+	"math"
+	"testing"
+	"time"
+
+	"github.com/influxdata/telegraf"
+
+	"github.com/stretchr/testify/assert"
+)
+
+func TestMakeMetricNoFields(t *testing.T) {
+	now := time.Now()
+	ri := RunningInput{
+		Config: &InputConfig{
+			Name: "TestRunningInput",
+		},
+	}
+
+	m := ri.MakeMetric(
+		"RITest",
+		map[string]interface{}{},
+		map[string]string{},
+		telegraf.Untyped,
+		now,
+	)
+	assert.Nil(t, m)
+}
+
+// nil fields should get dropped
+func TestMakeMetricNilFields(t *testing.T) {
+	now := time.Now()
+	ri := RunningInput{
+		Config: &InputConfig{
+			Name: "TestRunningInput",
+		},
+	}
+
+	m := ri.MakeMetric(
+		"RITest",
+		map[string]interface{}{
+			"value": int(101),
+			"nil":   nil,
+		},
+		map[string]string{},
+		telegraf.Untyped,
+		now,
+	)
+	assert.Equal(
+		t,
+		fmt.Sprintf("RITest value=101i %d", now.UnixNano()),
+		m.String(),
+	)
+}
+
+// make an untyped, counter, & gauge metric
+func TestMakeMetric(t *testing.T) {
+	now := time.Now()
+	ri := RunningInput{
+		Config: &InputConfig{
+			Name: "TestRunningInput",
+		},
+	}
+	ri.SetDebug(true)
+	assert.Equal(t, true, ri.Debug())
+	ri.SetTrace(true)
+	assert.Equal(t, true, ri.Trace())
+	assert.Equal(t, "inputs.TestRunningInput", ri.Name())
+
+	m := ri.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Untyped,
+		now,
+	)
+	assert.Equal(
+		t,
+		m.String(),
+		fmt.Sprintf("RITest value=101i %d", now.UnixNano()),
+	)
+	assert.Equal(
+		t,
+		m.Type(),
+		telegraf.Untyped,
+	)
+
+	m = ri.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Counter,
+		now,
+	)
+	assert.Equal(
+		t,
+		m.String(),
+		fmt.Sprintf("RITest value=101i %d", now.UnixNano()),
+	)
+	assert.Equal(
+		t,
+		m.Type(),
+		telegraf.Counter,
+	)
+
+	m = ri.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Gauge,
+		now,
+	)
+	assert.Equal(
+		t,
+		m.String(),
+		fmt.Sprintf("RITest value=101i %d", now.UnixNano()),
+	)
+	assert.Equal(
+		t,
+		m.Type(),
+		telegraf.Gauge,
+	)
+}
+
+func TestMakeMetricWithPluginTags(t *testing.T) {
+	now := time.Now()
+	ri := RunningInput{
+		Config: &InputConfig{
+			Name: "TestRunningInput",
+			Tags: map[string]string{
+				"foo": "bar",
+			},
+		},
+	}
+	ri.SetDebug(true)
+	assert.Equal(t, true, ri.Debug())
+	ri.SetTrace(true)
+	assert.Equal(t, true, ri.Trace())
+
+	m := ri.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		nil,
+		telegraf.Untyped,
+		now,
+	)
+	assert.Equal(
+		t,
+		m.String(),
+		fmt.Sprintf("RITest,foo=bar value=101i %d", now.UnixNano()),
+	)
+}
+
+func TestMakeMetricFilteredOut(t *testing.T) {
+	now := time.Now()
+	ri := RunningInput{
+		Config: &InputConfig{
+			Name: "TestRunningInput",
+			Tags: map[string]string{
+				"foo": "bar",
+			},
+			Filter: Filter{NamePass: []string{"foobar"}},
+		},
+	}
+	ri.SetDebug(true)
+	assert.Equal(t, true, ri.Debug())
+	ri.SetTrace(true)
+	assert.Equal(t, true, ri.Trace())
+	assert.NoError(t, ri.Config.Filter.Compile())
+
+	m := ri.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		nil,
+		telegraf.Untyped,
+		now,
+	)
+	assert.Nil(t, m)
+}
+
+func TestMakeMetricWithDaemonTags(t *testing.T) {
+	now := time.Now()
+	ri := RunningInput{
+		Config: &InputConfig{
+			Name: "TestRunningInput",
+		},
+	}
+	ri.SetDefaultTags(map[string]string{
+		"foo": "bar",
+	})
+	ri.SetDebug(true)
+	assert.Equal(t, true, ri.Debug())
+	ri.SetTrace(true)
+	assert.Equal(t, true, ri.Trace())
+
+	m := ri.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Untyped,
+		now,
+	)
+	assert.Equal(
+		t,
+		m.String(),
+		fmt.Sprintf("RITest,foo=bar value=101i %d", now.UnixNano()),
+	)
+}
+
+// make an untyped, counter, & gauge metric
+func TestMakeMetricInfFields(t *testing.T) {
+	inf := math.Inf(1)
+	ninf := math.Inf(-1)
+	now := time.Now()
+	ri := RunningInput{
+		Config: &InputConfig{
+			Name: "TestRunningInput",
+		},
+	}
+	ri.SetDebug(true)
+	assert.Equal(t, true, ri.Debug())
+	ri.SetTrace(true)
+	assert.Equal(t, true, ri.Trace())
+
+	m := ri.MakeMetric(
+		"RITest",
+		map[string]interface{}{
+			"value": int(101),
+			"inf":   inf,
+			"ninf":  ninf,
+		},
+		map[string]string{},
+		telegraf.Untyped,
+		now,
+	)
+	assert.Equal(
+		t,
+		m.String(),
+		fmt.Sprintf("RITest value=101i %d", now.UnixNano()),
+	)
+}
+
+func TestMakeMetricAllFieldTypes(t *testing.T) {
+	now := time.Now()
+	ri := RunningInput{
+		Config: &InputConfig{
+			Name: "TestRunningInput",
+		},
+	}
+	ri.SetDebug(true)
+	assert.Equal(t, true, ri.Debug())
+	ri.SetTrace(true)
+	assert.Equal(t, true, ri.Trace())
+
+	m := ri.MakeMetric(
+		"RITest",
+		map[string]interface{}{
+			"a": int(10),
+			"b": int8(10),
+			"c": int16(10),
+			"d": int32(10),
+			"e": uint(10),
+			"f": uint8(10),
+			"g": uint16(10),
+			"h": uint32(10),
+			"i": uint64(10),
+			"j": float32(10),
+			"k": uint64(9223372036854775810),
+			"l": "foobar",
+			"m": true,
+		},
+		map[string]string{},
+		telegraf.Untyped,
+		now,
+	)
+	assert.Equal(
+		t,
+		fmt.Sprintf("RITest a=10i,b=10i,c=10i,d=10i,e=10i,f=10i,g=10i,h=10i,i=10i,j=10,k=9223372036854775807i,l=\"foobar\",m=true %d", now.UnixNano()),
+		m.String(),
+	)
+}
+
+func TestMakeMetricNameOverride(t *testing.T) {
+	now := time.Now()
+	ri := RunningInput{
+		Config: &InputConfig{
+			Name:         "TestRunningInput",
+			NameOverride: "foobar",
+		},
+	}
+
+	m := ri.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Untyped,
+		now,
+	)
+	assert.Equal(
+		t,
+		m.String(),
+		fmt.Sprintf("foobar value=101i %d", now.UnixNano()),
+	)
+}
+
+func TestMakeMetricNamePrefix(t *testing.T) {
+	now := time.Now()
+	ri := RunningInput{
+		Config: &InputConfig{
+			Name:              "TestRunningInput",
+			MeasurementPrefix: "foobar_",
+		},
+	}
+
+	m := ri.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Untyped,
+		now,
+	)
+	assert.Equal(
+		t,
+		m.String(),
+		fmt.Sprintf("foobar_RITest value=101i %d", now.UnixNano()),
+	)
+}
+
+func TestMakeMetricNameSuffix(t *testing.T) {
+	now := time.Now()
+	ri := RunningInput{
+		Config: &InputConfig{
+			Name:              "TestRunningInput",
+			MeasurementSuffix: "_foobar",
+		},
+	}
+
+	m := ri.MakeMetric(
+		"RITest",
+		map[string]interface{}{"value": int(101)},
+		map[string]string{},
+		telegraf.Untyped,
+		now,
+	)
+	assert.Equal(
+		t,
+		m.String(),
+		fmt.Sprintf("RITest_foobar value=101i %d", now.UnixNano()),
+	)
+}

internal/models/running_output_test.go

@@ -132,7 +132,6 @@ func TestRunningOutput_PassFilter(t *testing.T) {
 func TestRunningOutput_TagIncludeNoMatch(t *testing.T) {
 	conf := &OutputConfig{
 		Filter: Filter{
-
 			TagInclude: []string{"nothing*"},
 		},
 	}
@@ -154,7 +153,6 @@ func TestRunningOutput_TagIncludeNoMatch(t *testing.T) {
 func TestRunningOutput_TagExcludeMatch(t *testing.T) {
 	conf := &OutputConfig{
 		Filter: Filter{
-
 			TagExclude: []string{"tag*"},
 		},
 	}
@@ -176,7 +174,6 @@ func TestRunningOutput_TagExcludeMatch(t *testing.T) {
 func TestRunningOutput_TagExcludeNoMatch(t *testing.T) {
 	conf := &OutputConfig{
 		Filter: Filter{
-
 			TagExclude: []string{"nothing*"},
 		},
 	}
@@ -198,7 +195,6 @@ func TestRunningOutput_TagExcludeNoMatch(t *testing.T) {
 func TestRunningOutput_TagIncludeMatch(t *testing.T) {
 	conf := &OutputConfig{
 		Filter: Filter{
-
 			TagInclude: []string{"tag*"},
 		},
 	}

internal/models/running_processor.go

@@ -0,0 +1,44 @@
+package models
+
+import (
+	"github.com/influxdata/telegraf"
+)
+
+type RunningProcessor struct {
+	Name      string
+	Processor telegraf.Processor
+	Config    *ProcessorConfig
+}
+
+type RunningProcessors []*RunningProcessor
+
+func (rp RunningProcessors) Len() int           { return len(rp) }
+func (rp RunningProcessors) Swap(i, j int)      { rp[i], rp[j] = rp[j], rp[i] }
+func (rp RunningProcessors) Less(i, j int) bool { return rp[i].Config.Order < rp[j].Config.Order }
+
+// FilterConfig containing a name and filter
+type ProcessorConfig struct {
+	Name   string
+	Order  int64
+	Filter Filter
+}
+
+func (rp *RunningProcessor) Apply(in ...telegraf.Metric) []telegraf.Metric {
+	ret := []telegraf.Metric{}
+
+	for _, metric := range in {
+		if rp.Config.Filter.IsActive() {
+			// check if the filter should be applied to this metric
+			if ok := rp.Config.Filter.Apply(metric.Name(), metric.Fields(), metric.Tags()); !ok {
+				// this means filter should not be applied
+				ret = append(ret, metric)
+				continue
+			}
+		}
+		// This metric should pass through the filter, so call the filter Apply
+		// function and append results to the output slice.
+		ret = append(ret, rp.Processor.Apply(metric)...)
+	}
+
+	return ret
+}

internal/models/running_processor_test.go

@@ -0,0 +1,117 @@
+package models
+
+import (
+	"testing"
+
+	"github.com/influxdata/telegraf"
+	"github.com/influxdata/telegraf/testutil"
+
+	"github.com/stretchr/testify/assert"
+)
+
+type TestProcessor struct {
+}
+
+func (f *TestProcessor) SampleConfig() string { return "" }
+func (f *TestProcessor) Description() string  { return "" }
+
+// Apply renames:
+//   "foo" to "fuz"
+//   "bar" to "baz"
+// And it also drops measurements named "dropme"
+func (f *TestProcessor) Apply(in ...telegraf.Metric) []telegraf.Metric {
+	out := make([]telegraf.Metric, 0)
+	for _, m := range in {
+		switch m.Name() {
+		case "foo":
+			out = append(out, testutil.TestMetric(1, "fuz"))
+		case "bar":
+			out = append(out, testutil.TestMetric(1, "baz"))
+		case "dropme":
+			// drop the metric!
+		default:
+			out = append(out, m)
+		}
+	}
+	return out
+}
+
+func NewTestRunningProcessor() *RunningProcessor {
+	out := &RunningProcessor{
+		Name:      "test",
+		Processor: &TestProcessor{},
+		Config:    &ProcessorConfig{Filter: Filter{}},
+	}
+	return out
+}
+
+func TestRunningProcessor(t *testing.T) {
+	inmetrics := []telegraf.Metric{
+		testutil.TestMetric(1, "foo"),
+		testutil.TestMetric(1, "bar"),
+		testutil.TestMetric(1, "baz"),
+	}
+
+	expectedNames := []string{
+		"fuz",
+		"baz",
+		"baz",
+	}
+	rfp := NewTestRunningProcessor()
+	filteredMetrics := rfp.Apply(inmetrics...)
+
+	actualNames := []string{
+		filteredMetrics[0].Name(),
+		filteredMetrics[1].Name(),
+		filteredMetrics[2].Name(),
+	}
+	assert.Equal(t, expectedNames, actualNames)
+}
+
+func TestRunningProcessor_WithNameDrop(t *testing.T) {
+	inmetrics := []telegraf.Metric{
+		testutil.TestMetric(1, "foo"),
+		testutil.TestMetric(1, "bar"),
+		testutil.TestMetric(1, "baz"),
+	}
+
+	expectedNames := []string{
+		"foo",
+		"baz",
+		"baz",
+	}
+	rfp := NewTestRunningProcessor()
+
+	rfp.Config.Filter.NameDrop = []string{"foo"}
+	assert.NoError(t, rfp.Config.Filter.Compile())
+
+	filteredMetrics := rfp.Apply(inmetrics...)
+
+	actualNames := []string{
+		filteredMetrics[0].Name(),
+		filteredMetrics[1].Name(),
+		filteredMetrics[2].Name(),
+	}
+	assert.Equal(t, expectedNames, actualNames)
+}
+
+func TestRunningProcessor_DroppedMetric(t *testing.T) {
+	inmetrics := []telegraf.Metric{
+		testutil.TestMetric(1, "dropme"),
+		testutil.TestMetric(1, "foo"),
+		testutil.TestMetric(1, "bar"),
+	}
+
+	expectedNames := []string{
+		"fuz",
+		"baz",
+	}
+	rfp := NewTestRunningProcessor()
+	filteredMetrics := rfp.Apply(inmetrics...)
+
+	actualNames := []string{
+		filteredMetrics[0].Name(),
+		filteredMetrics[1].Name(),
+	}
+	assert.Equal(t, expectedNames, actualNames)
+}

logger/logger.go

@@ -27,8 +27,8 @@ func (t *telegrafLog) Write(p []byte) (n int, err error) {
 //   debug   will set the log level to DEBUG
 //   quiet   will set the log level to ERROR
 //   logfile will direct the logging output to a file. Empty string is
-//           interpreted as stdout. If there is an error opening the file the
+//           interpreted as stderr. If there is an error opening the file the
-//           logger will fallback to stdout.
+//           logger will fallback to stderr.
 func SetupLogging(debug, quiet bool, logfile string) {
 	if debug {
 		wlog.SetLevel(wlog.DEBUG)
@@ -41,17 +41,17 @@ func SetupLogging(debug, quiet bool, logfile string) {
 	if logfile != "" {
 		if _, err := os.Stat(logfile); os.IsNotExist(err) {
 			if oFile, err = os.Create(logfile); err != nil {
-				log.Printf("E! Unable to create %s (%s), using stdout", logfile, err)
+				log.Printf("E! Unable to create %s (%s), using stderr", logfile, err)
-				oFile = os.Stdout
+				oFile = os.Stderr
 			}
 		} else {
 			if oFile, err = os.OpenFile(logfile, os.O_APPEND|os.O_WRONLY, os.ModeAppend); err != nil {
-				log.Printf("E! Unable to append to %s (%s), using stdout", logfile, err)
+				log.Printf("E! Unable to append to %s (%s), using stderr", logfile, err)
-				oFile = os.Stdout
+				oFile = os.Stderr
 			}
 		}
 	} else {
-		oFile = os.Stdout
+		oFile = os.Stderr
 	}
 
 	log.SetOutput(newTelegrafWriter(oFile))

metric.go

@@ -4,6 +4,7 @@ import (
 	"time"
 
 	"github.com/influxdata/influxdb/client/v2"
+	"github.com/influxdata/influxdb/models"
 )
 
 // ValueType is an enumeration of metric types that represent a simple value.
@@ -33,6 +34,10 @@ type Metric interface {
 	// UnixNano returns the unix nano time of the metric
 	UnixNano() int64
 
+	// HashID returns a non-cryptographic hash of the metric (name + tags)
+	// NOTE: do not persist & depend on this value to disk.
+	HashID() uint64
+
 	// Fields returns the fields for the metric
 	Fields() map[string]interface{}
 
@@ -44,13 +49,28 @@ type Metric interface {
 
 	// Point returns a influxdb client.Point object
 	Point() *client.Point
+
+	// SetAggregate sets the metric's aggregate status
+	// This is so that aggregate metrics don't get re-sent to aggregator plugins
+	SetAggregate(bool)
+	// IsAggregate returns true if the metric is an aggregate
+	IsAggregate() bool
 }
 
 // metric is a wrapper of the influxdb client.Point struct
 type metric struct {
-	pt *client.Point
+	pt models.Point
 
 	mType ValueType
+
+	isaggregate bool
+}
+
+func NewMetricFromPoint(pt models.Point) Metric {
+	return &metric{
+		pt:    pt,
+		mType: Untyped,
+	}
 }
 
 // NewMetric returns an untyped metric.
@@ -60,7 +80,7 @@ func NewMetric(
 	fields map[string]interface{},
 	t time.Time,
 ) (Metric, error) {
-	pt, err := client.NewPoint(name, tags, fields, t)
+	pt, err := models.NewPoint(name, models.NewTags(tags), fields, t)
 	if err != nil {
 		return nil, err
 	}
@@ -79,7 +99,7 @@ func NewGaugeMetric(
 	fields map[string]interface{},
 	t time.Time,
 ) (Metric, error) {
-	pt, err := client.NewPoint(name, tags, fields, t)
+	pt, err := models.NewPoint(name, models.NewTags(tags), fields, t)
 	if err != nil {
 		return nil, err
 	}
@@ -98,7 +118,7 @@ func NewCounterMetric(
 	fields map[string]interface{},
 	t time.Time,
 ) (Metric, error) {
-	pt, err := client.NewPoint(name, tags, fields, t)
+	pt, err := models.NewPoint(name, models.NewTags(tags), fields, t)
 	if err != nil {
 		return nil, err
 	}
@@ -113,7 +133,7 @@ func (m *metric) Name() string {
 }
 
 func (m *metric) Tags() map[string]string {
-	return m.pt.Tags()
+	return m.pt.Tags().Map()
 }
 
 func (m *metric) Time() time.Time {
@@ -124,6 +144,10 @@ func (m *metric) Type() ValueType {
 	return m.mType
 }
 
+func (m *metric) HashID() uint64 {
+	return m.pt.HashID()
+}
+
 func (m *metric) UnixNano() int64 {
 	return m.pt.UnixNano()
 }
@@ -141,5 +165,13 @@ func (m *metric) PrecisionString(precison string) string {
 }
 
 func (m *metric) Point() *client.Point {
-	return m.pt
+	return client.NewPointFrom(m.pt)
+}
+
+func (m *metric) IsAggregate() bool {
+	return m.isaggregate
+}
+
+func (m *metric) SetAggregate(b bool) {
+	m.isaggregate = b
 }

plugins/aggregators/all/all.go

@@ -0,0 +1,5 @@
+package all
+
+import (
+	_ "github.com/influxdata/telegraf/plugins/aggregators/minmax"
+)

plugins/aggregators/minmax/minmax.go

@@ -0,0 +1,119 @@
+package minmax
+
+import (
+	"github.com/influxdata/telegraf"
+	"github.com/influxdata/telegraf/plugins/aggregators"
+)
+
+type MinMax struct {
+	cache map[uint64]aggregate
+}
+
+func NewMinMax() telegraf.Aggregator {
+	mm := &MinMax{}
+	mm.Reset()
+	return mm
+}
+
+type aggregate struct {
+	fields map[string]minmax
+	name   string
+	tags   map[string]string
+}
+
+type minmax struct {
+	min float64
+	max float64
+}
+
+var sampleConfig = `
+  ## General Aggregator Arguments:
+  ## The period on which to flush & clear the aggregator.
+  period = "30s"
+  ## If true, the original metric will be dropped by the
+  ## aggregator and will not get sent to the output plugins.
+  drop_original = false
+`
+
+func (m *MinMax) SampleConfig() string {
+	return sampleConfig
+}
+
+func (m *MinMax) Description() string {
+	return "Keep the aggregate min/max of each metric passing through."
+}
+
+func (m *MinMax) Add(in telegraf.Metric) {
+	id := in.HashID()
+	if _, ok := m.cache[id]; !ok {
+		// hit an uncached metric, create caches for first time:
+		a := aggregate{
+			name:   in.Name(),
+			tags:   in.Tags(),
+			fields: make(map[string]minmax),
+		}
+		for k, v := range in.Fields() {
+			if fv, ok := convert(v); ok {
+				a.fields[k] = minmax{
+					min: fv,
+					max: fv,
+				}
+			}
+		}
+		m.cache[id] = a
+	} else {
+		for k, v := range in.Fields() {
+			if fv, ok := convert(v); ok {
+				if _, ok := m.cache[id].fields[k]; !ok {
+					// hit an uncached field of a cached metric
+					m.cache[id].fields[k] = minmax{
+						min: fv,
+						max: fv,
+					}
+					continue
+				}
+				if fv < m.cache[id].fields[k].min {
+					tmp := m.cache[id].fields[k]
+					tmp.min = fv
+					m.cache[id].fields[k] = tmp
+				} else if fv > m.cache[id].fields[k].max {
+					tmp := m.cache[id].fields[k]
+					tmp.max = fv
+					m.cache[id].fields[k] = tmp
+				}
+			}
+		}
+	}
+}
+
+func (m *MinMax) Push(acc telegraf.Accumulator) {
+	for _, aggregate := range m.cache {
+		fields := map[string]interface{}{}
+		for k, v := range aggregate.fields {
+			fields[k+"_min"] = v.min
+			fields[k+"_max"] = v.max
+		}
+		acc.AddFields(aggregate.name, fields, aggregate.tags)
+	}
+}
+
+func (m *MinMax) Reset() {
+	m.cache = make(map[uint64]aggregate)
+}
+
+func convert(in interface{}) (float64, bool) {
+	switch v := in.(type) {
+	case float64:
+		return v, true
+	case int64:
+		return float64(v), true
+	default:
+		return 0, false
+	}
+}
+
+func init() {
+	aggregators.Add("minmax", func() telegraf.Aggregator {
+		return NewMinMax()
+	})
+}

plugins/aggregators/minmax/minmax_test.go

@@ -0,0 +1,162 @@
+package minmax
+
+import (
+	"testing"
+	"time"
+
+	"github.com/influxdata/telegraf"
+	"github.com/influxdata/telegraf/testutil"
+)
+
+var m1, _ = telegraf.NewMetric("m1",
+	map[string]string{"foo": "bar"},
+	map[string]interface{}{
+		"a": int64(1),
+		"b": int64(1),
+		"c": int64(1),
+		"d": int64(1),
+		"e": int64(1),
+		"f": float64(2),
+		"g": float64(2),
+		"h": float64(2),
+		"i": float64(2),
+		"j": float64(3),
+	},
+	time.Now(),
+)
+var m2, _ = telegraf.NewMetric("m1",
+	map[string]string{"foo": "bar"},
+	map[string]interface{}{
+		"a":        int64(1),
+		"b":        int64(3),
+		"c":        int64(3),
+		"d":        int64(3),
+		"e":        int64(3),
+		"f":        float64(1),
+		"g":        float64(1),
+		"h":        float64(1),
+		"i":        float64(1),
+		"j":        float64(1),
+		"k":        float64(200),
+		"ignoreme": "string",
+		"andme":    true,
+	},
+	time.Now(),
+)
+
+func BenchmarkApply(b *testing.B) {
+	minmax := NewMinMax()
+
+	for n := 0; n < b.N; n++ {
+		minmax.Add(m1)
+		minmax.Add(m2)
+	}
+}
+
+// Test two metrics getting added.
+func TestMinMaxWithPeriod(t *testing.T) {
+	acc := testutil.Accumulator{}
+	minmax := NewMinMax()
+
+	minmax.Add(m1)
+	minmax.Add(m2)
+	minmax.Push(&acc)
+
+	expectedFields := map[string]interface{}{
+		"a_max": float64(1),
+		"a_min": float64(1),
+		"b_max": float64(3),
+		"b_min": float64(1),
+		"c_max": float64(3),
+		"c_min": float64(1),
+		"d_max": float64(3),
+		"d_min": float64(1),
+		"e_max": float64(3),
+		"e_min": float64(1),
+		"f_max": float64(2),
+		"f_min": float64(1),
+		"g_max": float64(2),
+		"g_min": float64(1),
+		"h_max": float64(2),
+		"h_min": float64(1),
+		"i_max": float64(2),
+		"i_min": float64(1),
+		"j_max": float64(3),
+		"j_min": float64(1),
+		"k_max": float64(200),
+		"k_min": float64(200),
+	}
+	expectedTags := map[string]string{
+		"foo": "bar",
+	}
+	acc.AssertContainsTaggedFields(t, "m1", expectedFields, expectedTags)
+}
+
+// Test two metrics getting added with a push/reset in between (simulates
+// getting added in different periods.)
+func TestMinMaxDifferentPeriods(t *testing.T) {
+	acc := testutil.Accumulator{}
+	minmax := NewMinMax()
+
+	minmax.Add(m1)
+	minmax.Push(&acc)
+	expectedFields := map[string]interface{}{
+		"a_max": float64(1),
+		"a_min": float64(1),
+		"b_max": float64(1),
+		"b_min": float64(1),
+		"c_max": float64(1),
+		"c_min": float64(1),
+		"d_max": float64(1),
+		"d_min": float64(1),
+		"e_max": float64(1),
+		"e_min": float64(1),
+		"f_max": float64(2),
+		"f_min": float64(2),
+		"g_max": float64(2),
+		"g_min": float64(2),
+		"h_max": float64(2),
+		"h_min": float64(2),
+		"i_max": float64(2),
+		"i_min": float64(2),
+		"j_max": float64(3),
+		"j_min": float64(3),
+	}
+	expectedTags := map[string]string{
+		"foo": "bar",
+	}
+	acc.AssertContainsTaggedFields(t, "m1", expectedFields, expectedTags)
+
+	acc.ClearMetrics()
+	minmax.Reset()
+	minmax.Add(m2)
+	minmax.Push(&acc)
+	expectedFields = map[string]interface{}{
+		"a_max": float64(1),
+		"a_min": float64(1),
+		"b_max": float64(3),
+		"b_min": float64(3),
+		"c_max": float64(3),
+		"c_min": float64(3),
+		"d_max": float64(3),
+		"d_min": float64(3),
+		"e_max": float64(3),
+		"e_min": float64(3),
+		"f_max": float64(1),
+		"f_min": float64(1),
+		"g_max": float64(1),
+		"g_min": float64(1),
+		"h_max": float64(1),
+		"h_min": float64(1),
+		"i_max": float64(1),
+		"i_min": float64(1),
+		"j_max": float64(1),
+		"j_min": float64(1),
+		"k_max": float64(200),
+		"k_min": float64(200),
+	}
+	expectedTags = map[string]string{
+		"foo": "bar",
+	}
+	acc.AssertContainsTaggedFields(t, "m1", expectedFields, expectedTags)
+}

plugins/aggregators/registry.go

@@ -0,0 +1,11 @@
+package aggregators
+
+import "github.com/influxdata/telegraf"
+
+type Creator func() telegraf.Aggregator
+
+var Aggregators = map[string]Creator{}
+
+func Add(name string, creator Creator) {
+	Aggregators[name] = creator
+}

plugins/inputs/cgroup/README.md

@@ -2,6 +2,10 @@
 
 This input plugin will capture specific statistics per cgroup.
 
+Consider restricting paths to the set of cgroups you really
+want to monitor if you have a large number of cgroups, to avoid
+any cardinality issues.
+
 Following file formats are supported:
 
 * Single value
@@ -33,9 +37,8 @@ KEY1 VAL1\n
 
 ### Tags:
 
-Measurements don't have any specific tags unless you define them at the telegraf level (defaults). We
+All measurements have the following tags:
-used to have the path listed as a tag, but to keep cardinality in check it's easier to move this 
+  - path
-value to a field. Thanks @sebito91!
 
 
 ### Configuration:

plugins/inputs/cgroup/cgroup.go

@@ -11,15 +11,18 @@ type CGroup struct {
 }
 
 var sampleConfig = `
-    ## Directories in which to look for files, globs are supported.
+  ## Directories in which to look for files, globs are supported.
-	# paths = [
+  ## Consider restricting paths to the set of cgroups you really
-	#   "/cgroup/memory",
+  ## want to monitor if you have a large number of cgroups, to avoid
-	#   "/cgroup/memory/child1",
+  ## any cardinality issues.
-	#   "/cgroup/memory/child2/*",
+  # paths = [
-	# ]
+  #   "/cgroup/memory",
-	## cgroup stat fields, as file names, globs are supported.
+  #   "/cgroup/memory/child1",
-	## these file names are appended to each path from above.
+  #   "/cgroup/memory/child2/*",
-	# files = ["memory.*usage*", "memory.limit_in_bytes"]
+  # ]
+  ## cgroup stat fields, as file names, globs are supported.
+  ## these file names are appended to each path from above.
+  # files = ["memory.*usage*", "memory.limit_in_bytes"]
 `
 
 func (g *CGroup) SampleConfig() string {

plugins/inputs/cgroup/cgroup_linux.go

@@ -56,9 +56,10 @@ func (g *CGroup) gatherDir(dir string, acc telegraf.Accumulator) error {
 			return err
 		}
 	}
-	fields["path"] = dir
 
-	acc.AddFields(metricName, fields, nil)
+	tags := map[string]string{"path": dir}
+
+	acc.AddFields(metricName, fields, tags)
 
 	return nil
 }

plugins/inputs/cgroup/cgroup_test.go

@@ -3,13 +3,10 @@
 package cgroup
 
 import (
-	"fmt"
 	"testing"
 
 	"github.com/influxdata/telegraf/testutil"
-	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
-	"reflect"
 )
 
 var cg1 = &CGroup{
@@ -24,32 +21,15 @@ var cg1 = &CGroup{
 	},
 }
 
-func assertContainsFields(a *testutil.Accumulator, t *testing.T, measurement string, fieldSet []map[string]interface{}) {
-	a.Lock()
-	defer a.Unlock()
-
-	numEquals := 0
-	for _, p := range a.Metrics {
-		if p.Measurement == measurement {
-			for _, fields := range fieldSet {
-				if reflect.DeepEqual(fields, p.Fields) {
-					numEquals++
-				}
-			}
-		}
-	}
-
-	if numEquals != len(fieldSet) {
-		assert.Fail(t, fmt.Sprintf("only %d of %d are equal", numEquals, len(fieldSet)))
-	}
-}
-
 func TestCgroupStatistics_1(t *testing.T) {
 	var acc testutil.Accumulator
 
 	err := cg1.Gather(&acc)
 	require.NoError(t, err)
 
+	tags := map[string]string{
+		"path": "testdata/memory",
+	}
 	fields := map[string]interface{}{
 		"memory.stat.cache":           1739362304123123123,
 		"memory.stat.rss":             1775325184,
@@ -62,9 +42,8 @@ func TestCgroupStatistics_1(t *testing.T) {
 		"memory.limit_in_bytes":       223372036854771712,
 		"memory.use_hierarchy":        "12-781",
 		"notify_on_release":           0,
-		"path":                        "testdata/memory",
 	}
-	assertContainsFields(&acc, t, "cgroup", []map[string]interface{}{fields})
+	acc.AssertContainsTaggedFields(t, "cgroup", fields, tags)
 }
 
 // ======================================================================
@@ -80,14 +59,16 @@ func TestCgroupStatistics_2(t *testing.T) {
 	err := cg2.Gather(&acc)
 	require.NoError(t, err)
 
+	tags := map[string]string{
+		"path": "testdata/cpu",
+	}
 	fields := map[string]interface{}{
 		"cpuacct.usage_percpu.0": -1452543795404,
 		"cpuacct.usage_percpu.1": 1376681271659,
 		"cpuacct.usage_percpu.2": 1450950799997,
 		"cpuacct.usage_percpu.3": -1473113374257,
-		"path": "testdata/cpu",
 	}
-	assertContainsFields(&acc, t, "cgroup", []map[string]interface{}{fields})
+	acc.AssertContainsTaggedFields(t, "cgroup", fields, tags)
 }
 
 // ======================================================================
@@ -103,16 +84,18 @@ func TestCgroupStatistics_3(t *testing.T) {
 	err := cg3.Gather(&acc)
 	require.NoError(t, err)
 
+	tags := map[string]string{
+		"path": "testdata/memory/group_1",
+	}
 	fields := map[string]interface{}{
 		"memory.limit_in_bytes": 223372036854771712,
-		"path":                  "testdata/memory/group_1",
 	}
+	acc.AssertContainsTaggedFields(t, "cgroup", fields, tags)
 
-	fieldsTwo := map[string]interface{}{
+	tags = map[string]string{
-		"memory.limit_in_bytes": 223372036854771712,
+		"path": "testdata/memory/group_2",
-		"path":                  "testdata/memory/group_2",
 	}
-	assertContainsFields(&acc, t, "cgroup", []map[string]interface{}{fields, fieldsTwo})
+	acc.AssertContainsTaggedFields(t, "cgroup", fields, tags)
 }
 
 // ======================================================================
@@ -128,22 +111,23 @@ func TestCgroupStatistics_4(t *testing.T) {
 	err := cg4.Gather(&acc)
 	require.NoError(t, err)
 
+	tags := map[string]string{
+		"path": "testdata/memory/group_1/group_1_1",
+	}
 	fields := map[string]interface{}{
 		"memory.limit_in_bytes": 223372036854771712,
-		"path":                  "testdata/memory/group_1/group_1_1",
 	}
+	acc.AssertContainsTaggedFields(t, "cgroup", fields, tags)
 
-	fieldsTwo := map[string]interface{}{
+	tags = map[string]string{
-		"memory.limit_in_bytes": 223372036854771712,
+		"path": "testdata/memory/group_1/group_1_2",
-		"path":                  "testdata/memory/group_1/group_1_2",
 	}
+	acc.AssertContainsTaggedFields(t, "cgroup", fields, tags)
 
-	fieldsThree := map[string]interface{}{
+	tags = map[string]string{
-		"memory.limit_in_bytes": 223372036854771712,
+		"path": "testdata/memory/group_2",
-		"path":                  "testdata/memory/group_2",
 	}
-
+	acc.AssertContainsTaggedFields(t, "cgroup", fields, tags)
-	assertContainsFields(&acc, t, "cgroup", []map[string]interface{}{fields, fieldsTwo, fieldsThree})
 }
 
 // ======================================================================
@@ -159,16 +143,18 @@ func TestCgroupStatistics_5(t *testing.T) {
 	err := cg5.Gather(&acc)
 	require.NoError(t, err)
 
+	tags := map[string]string{
+		"path": "testdata/memory/group_1/group_1_1",
+	}
 	fields := map[string]interface{}{
 		"memory.limit_in_bytes": 223372036854771712,
-		"path":                  "testdata/memory/group_1/group_1_1",
 	}
+	acc.AssertContainsTaggedFields(t, "cgroup", fields, tags)
 
-	fieldsTwo := map[string]interface{}{
+	tags = map[string]string{
-		"memory.limit_in_bytes": 223372036854771712,
+		"path": "testdata/memory/group_2/group_1_1",
-		"path":                  "testdata/memory/group_2/group_1_1",
 	}
-	assertContainsFields(&acc, t, "cgroup", []map[string]interface{}{fields, fieldsTwo})
+	acc.AssertContainsTaggedFields(t, "cgroup", fields, tags)
 }
 
 // ======================================================================
@@ -184,11 +170,13 @@ func TestCgroupStatistics_6(t *testing.T) {
 	err := cg6.Gather(&acc)
 	require.NoError(t, err)
 
+	tags := map[string]string{
+		"path": "testdata/memory",
+	}
 	fields := map[string]interface{}{
 		"memory.usage_in_bytes":      3513667584,
 		"memory.use_hierarchy":       "12-781",
 		"memory.kmem.limit_in_bytes": 9223372036854771712,
-		"path": "testdata/memory",
 	}
-	assertContainsFields(&acc, t, "cgroup", []map[string]interface{}{fields})
+	acc.AssertContainsTaggedFields(t, "cgroup", fields, tags)
 }

plugins/inputs/http_listener/http_listener_test.go

@@ -99,14 +99,14 @@ func TestWriteHTTPHighTraffic(t *testing.T) {
 	var wg sync.WaitGroup
 	for i := 0; i < 10; i++ {
 		wg.Add(1)
-		go func() {
+		go func(innerwg *sync.WaitGroup) {
+			defer innerwg.Done()
 			for i := 0; i < 500; i++ {
 				resp, err := http.Post("http://localhost:8286/write?db=mydb", "", bytes.NewBuffer([]byte(testMsgs)))
 				require.NoError(t, err)
 				require.EqualValues(t, 204, resp.StatusCode)
 			}
-			wg.Done()
+		}(&wg)
-		}()
 	}
 
 	wg.Wait()

plugins/inputs/mysql/mysql.go

@@ -4,16 +4,16 @@ import (
 	"bytes"
 	"database/sql"
 	"fmt"
-	"net/url"
 	"strconv"
 	"strings"
 	"sync"
 	"time"
 
-	_ "github.com/go-sql-driver/mysql"
 	"github.com/influxdata/telegraf"
 	"github.com/influxdata/telegraf/internal/errchan"
 	"github.com/influxdata/telegraf/plugins/inputs"
+
+	"github.com/go-sql-driver/mysql"
 )
 
 type Mysql struct {
@@ -398,27 +398,6 @@ var (
 	}
 )
 
-func dsnAddTimeout(dsn string) (string, error) {
-
-	// DSN "?timeout=5s" is not valid, but "/?timeout=5s" is valid ("" and "/"
-	// are the same DSN)
-	if dsn == "" {
-		dsn = "/"
-	}
-	u, err := url.Parse(dsn)
-	if err != nil {
-		return "", err
-	}
-	v := u.Query()
-
-	// Only override timeout if not already defined
-	if _, ok := v["timeout"]; ok == false {
-		v.Add("timeout", defaultTimeout.String())
-		u.RawQuery = v.Encode()
-	}
-	return u.String(), nil
-}
-
 // Math constants
 const (
 	picoSeconds = 1e12
@@ -682,10 +661,7 @@ func (m *Mysql) gatherGlobalVariables(db *sql.DB, serv string, acc telegraf.Accu
 	var val sql.RawBytes
 
 	// parse DSN and save server tag
-	servtag, err := parseDSN(serv)
+	servtag := getDSNTag(serv)
-	if err != nil {
-		servtag = "localhost"
-	}
 	tags := map[string]string{"server": servtag}
 	fields := make(map[string]interface{})
 	for rows.Next() {
@@ -722,10 +698,7 @@ func (m *Mysql) gatherSlaveStatuses(db *sql.DB, serv string, acc telegraf.Accumu
 	}
 	defer rows.Close()
 
-	servtag, err := parseDSN(serv)
+	servtag := getDSNTag(serv)
-	if err != nil {
-		servtag = "localhost"
-	}
 
 	tags := map[string]string{"server": servtag}
 	fields := make(map[string]interface{})
@@ -770,11 +743,7 @@ func (m *Mysql) gatherBinaryLogs(db *sql.DB, serv string, acc telegraf.Accumulat
 	defer rows.Close()
 
 	// parse DSN and save host as a tag
-	var servtag string
+	servtag := getDSNTag(serv)
-	servtag, err = parseDSN(serv)
-	if err != nil {
-		servtag = "localhost"
-	}
 	tags := map[string]string{"server": servtag}
 	var (
 		size     uint64 = 0
@@ -817,11 +786,7 @@ func (m *Mysql) gatherGlobalStatuses(db *sql.DB, serv string, acc telegraf.Accum
 	}
 
 	// parse the DSN and save host name as a tag
-	var servtag string
+	servtag := getDSNTag(serv)
-	servtag, err = parseDSN(serv)
-	if err != nil {
-		servtag = "localhost"
-	}
 	tags := map[string]string{"server": servtag}
 	fields := make(map[string]interface{})
 	for rows.Next() {
@@ -932,10 +897,7 @@ func (m *Mysql) GatherProcessListStatuses(db *sql.DB, serv string, acc telegraf.
 
 	var servtag string
 	fields := make(map[string]interface{})
-	servtag, err = parseDSN(serv)
+	servtag = getDSNTag(serv)
-	if err != nil {
-		servtag = "localhost"
-	}
 
 	// mapping of state with its counts
 	stateCounts := make(map[string]uint32, len(generalThreadStates))
@@ -978,10 +940,7 @@ func (m *Mysql) gatherPerfTableIOWaits(db *sql.DB, serv string, acc telegraf.Acc
 		timeFetch, timeInsert, timeUpdate, timeDelete     float64
 	)
 
-	servtag, err = parseDSN(serv)
+	servtag = getDSNTag(serv)
-	if err != nil {
-		servtag = "localhost"
-	}
 
 	for rows.Next() {
 		err = rows.Scan(&objSchema, &objName,
@@ -1030,10 +989,7 @@ func (m *Mysql) gatherPerfIndexIOWaits(db *sql.DB, serv string, acc telegraf.Acc
 		timeFetch, timeInsert, timeUpdate, timeDelete     float64
 	)
 
-	servtag, err = parseDSN(serv)
+	servtag = getDSNTag(serv)
-	if err != nil {
-		servtag = "localhost"
-	}
 
 	for rows.Next() {
 		err = rows.Scan(&objSchema, &objName, &indexName,
@@ -1085,10 +1041,7 @@ func (m *Mysql) gatherInfoSchemaAutoIncStatuses(db *sql.DB, serv string, acc tel
 		incValue, maxInt      uint64
 	)
 
-	servtag, err := parseDSN(serv)
+	servtag := getDSNTag(serv)
-	if err != nil {
-		servtag = "localhost"
-	}
 
 	for rows.Next() {
 		if err := rows.Scan(&schema, &table, &column, &incValue, &maxInt); err != nil {
@@ -1132,10 +1085,7 @@ func (m *Mysql) gatherPerfTableLockWaits(db *sql.DB, serv string, acc telegraf.A
 	}
 	defer rows.Close()
 
-	servtag, err := parseDSN(serv)
+	servtag := getDSNTag(serv)
-	if err != nil {
-		servtag = "localhost"
-	}
 
 	var (
 		objectSchema               string
@@ -1257,10 +1207,7 @@ func (m *Mysql) gatherPerfEventWaits(db *sql.DB, serv string, acc telegraf.Accum
 		starCount, timeWait float64
 	)
 
-	servtag, err := parseDSN(serv)
+	servtag := getDSNTag(serv)
-	if err != nil {
-		servtag = "localhost"
-	}
 	tags := map[string]string{
 		"server": servtag,
 	}
@@ -1295,10 +1242,7 @@ func (m *Mysql) gatherPerfFileEventsStatuses(db *sql.DB, serv string, acc telegr
 		sumNumBytesRead, sumNumBytesWrite         float64
 	)
 
-	servtag, err := parseDSN(serv)
+	servtag := getDSNTag(serv)
-	if err != nil {
-		servtag = "localhost"
-	}
 	tags := map[string]string{
 		"server": servtag,
 	}
@@ -1365,10 +1309,7 @@ func (m *Mysql) gatherPerfEventsStatements(db *sql.DB, serv string, acc telegraf
 		noIndexUsed                          float64
 	)
 
-	servtag, err := parseDSN(serv)
+	servtag := getDSNTag(serv)
-	if err != nil {
-		servtag = "localhost"
-	}
 	tags := map[string]string{
 		"server": servtag,
 	}
@@ -1412,14 +1353,8 @@ func (m *Mysql) gatherPerfEventsStatements(db *sql.DB, serv string, acc telegraf
 
 // gatherTableSchema can be used to gather stats on each schema
 func (m *Mysql) gatherTableSchema(db *sql.DB, serv string, acc telegraf.Accumulator) error {
-	var (
+	var dbList []string
-		dbList  []string
+	servtag := getDSNTag(serv)
-		servtag string
-	)
-	servtag, err := parseDSN(serv)
-	if err != nil {
-		servtag = "localhost"
-	}
 
 	// if the list of databases if empty, then get all databases
 	if len(m.TableSchemaDatabases) == 0 {
@@ -1575,6 +1510,27 @@ func copyTags(in map[string]string) map[string]string {
 	return out
 }
 
+func dsnAddTimeout(dsn string) (string, error) {
+	conf, err := mysql.ParseDSN(dsn)
+	if err != nil {
+		return "", err
+	}
+
+	if conf.Timeout == 0 {
+		conf.Timeout = time.Second * 5
+	}
+
+	return conf.FormatDSN(), nil
+}
+
+func getDSNTag(dsn string) string {
+	conf, err := mysql.ParseDSN(dsn)
+	if err != nil {
+		return "127.0.0.1:3306"
+	}
+	return conf.Addr
+}
+
 func init() {
 	inputs.Add("mysql", func() telegraf.Input {
 		return &Mysql{}

plugins/inputs/mysql/mysql_test.go

@@ -26,7 +26,7 @@ func TestMysqlDefaultsToLocal(t *testing.T) {
 	assert.True(t, acc.HasMeasurement("mysql"))
 }
 
-func TestMysqlParseDSN(t *testing.T) {
+func TestMysqlGetDSNTag(t *testing.T) {
 	tests := []struct {
 		input  string
 		output string
@@ -78,9 +78,9 @@ func TestMysqlParseDSN(t *testing.T) {
 	}
 
 	for _, test := range tests {
-		output, _ := parseDSN(test.input)
+		output := getDSNTag(test.input)
 		if output != test.output {
-			t.Errorf("Expected %s, got %s\n", test.output, output)
+			t.Errorf("Input: %s Expected %s, got %s\n", test.input, test.output, output)
 		}
 	}
 }
@@ -92,7 +92,7 @@ func TestMysqlDNSAddTimeout(t *testing.T) {
 	}{
 		{
 			"",
-			"/?timeout=5s",
+			"tcp(127.0.0.1:3306)/?timeout=5s",
 		},
 		{
 			"tcp(192.168.1.1:3306)/",
@@ -104,7 +104,19 @@ func TestMysqlDNSAddTimeout(t *testing.T) {
 		},
 		{
 			"root:passwd@tcp(192.168.1.1:3306)/?tls=false&timeout=10s",
-			"root:passwd@tcp(192.168.1.1:3306)/?tls=false&timeout=10s",
+			"root:passwd@tcp(192.168.1.1:3306)/?timeout=10s&tls=false",
+		},
+		{
+			"tcp(10.150.1.123:3306)/",
+			"tcp(10.150.1.123:3306)/?timeout=5s",
+		},
+		{
+			"root:@!~(*&$#%(&@#(@&#Password@tcp(10.150.1.123:3306)/",
+			"root:@!~(*&$#%(&@#(@&#Password@tcp(10.150.1.123:3306)/?timeout=5s",
+		},
+		{
+			"root:Test3a#@!@tcp(10.150.1.123:3306)/",
+			"root:Test3a#@!@tcp(10.150.1.123:3306)/?timeout=5s",
 		},
 	}
 

plugins/inputs/mysql/parse_dsn.go

@@ -1,85 +0,0 @@
-// Copyright 2012 The Go-MySQL-Driver Authors. All rights reserved.
-//
-// This Source Code Form is subject to the terms of the Mozilla Public
-// License, v. 2.0. If a copy of the MPL was not distributed with this file,
-// You can obtain one at http://mozilla.org/MPL/2.0/.
-
-package mysql
-
-import (
-	"errors"
-	"strings"
-)
-
-// parseDSN parses the DSN string to a config
-func parseDSN(dsn string) (string, error) {
-	//var user, passwd string
-	var addr, net string
-
-	// [user[:password]@][net[(addr)]]/dbname[?param1=value1&paramN=valueN]
-	// Find the last '/' (since the password or the net addr might contain a '/')
-	for i := len(dsn) - 1; i >= 0; i-- {
-		if dsn[i] == '/' {
-			var j, k int
-
-			// left part is empty if i <= 0
-			if i > 0 {
-				// [username[:password]@][protocol[(address)]]
-				// Find the last '@' in dsn[:i]
-				for j = i; j >= 0; j-- {
-					if dsn[j] == '@' {
-						// username[:password]
-						// Find the first ':' in dsn[:j]
-						for k = 0; k < j; k++ {
-							if dsn[k] == ':' {
-								//passwd = dsn[k+1 : j]
-								break
-							}
-						}
-						//user = dsn[:k]
-
-						break
-					}
-				}
-
-				// [protocol[(address)]]
-				// Find the first '(' in dsn[j+1:i]
-				for k = j + 1; k < i; k++ {
-					if dsn[k] == '(' {
-						// dsn[i-1] must be == ')' if an address is specified
-						if dsn[i-1] != ')' {
-							if strings.ContainsRune(dsn[k+1:i], ')') {
-								return "", errors.New("Invalid DSN unescaped")
-							}
-							return "", errors.New("Invalid DSN Addr")
-						}
-						addr = dsn[k+1 : i-1]
-						break
-					}
-				}
-				net = dsn[j+1 : k]
-			}
-
-			break
-		}
-	}
-
-	// Set default network if empty
-	if net == "" {
-		net = "tcp"
-	}
-
-	// Set default address if empty
-	if addr == "" {
-		switch net {
-		case "tcp":
-			addr = "127.0.0.1:3306"
-		case "unix":
-			addr = "/tmp/mysql.sock"
-		default:
-			return "", errors.New("Default addr for network '" + net + "' unknown")
-		}
-	}
-
-	return addr, nil
-}

plugins/inputs/phpfpm/phpfpm.go

@@ -122,6 +122,9 @@ func (g *phpfpm) gatherServer(addr string, acc telegraf.Accumulator) error {
 		fcgiIp := socketAddr[0]
 		fcgiPort, _ := strconv.Atoi(socketAddr[1])
 		fcgi, err = newFcgiClient(fcgiIp, fcgiPort)
+		if err != nil {
+			return err
+		}
 		if len(u.Path) > 1 {
 			statusPath = strings.Trim(u.Path, "/")
 		} else {

plugins/inputs/postgresql_extensible/postgresql_extensible.go

@@ -29,6 +29,7 @@ type Postgresql struct {
 		Tagvalue    string
 		Measurement string
 	}
+	Debug bool
 }
 
 type query []struct {

plugins/inputs/snmp/snmp.go

@@ -178,13 +178,30 @@ type Table struct {
 	initialized bool
 }
 
-// init() populates Fields if a table OID is provided.
+// init() builds & initializes the nested fields.
 func (t *Table) init() error {
 	if t.initialized {
 		return nil
 	}
+
+	if err := t.initBuild(); err != nil {
+		return err
+	}
+
+	// initialize all the nested fields
+	for i := range t.Fields {
+		if err := t.Fields[i].init(); err != nil {
+			return err
+		}
+	}
+
+	t.initialized = true
+	return nil
+}
+
+// init() populates Fields if a table OID is provided.
+func (t *Table) initBuild() error {
 	if t.Oid == "" {
-		t.initialized = true
 		return nil
 	}
 
@@ -242,14 +259,6 @@ func (t *Table) init() error {
 		t.Fields = append(t.Fields, Field{Name: col, Oid: mibPrefix + col, IsTag: isTag})
 	}
 
-	// initialize all the nested fields
-	for i := range t.Fields {
-		if err := t.Fields[i].init(); err != nil {
-			return err
-		}
-	}
-
-	t.initialized = true
 	return nil
 }
 

plugins/inputs/snmp/snmp_mocks_generate.go

@@ -21,6 +21,7 @@ var mockedCommands = [][]string{
 	{"snmptranslate", "-Td", "-Ob", "-m", "all", "1.0.0.1.1"},
 	{"snmptranslate", "-Td", "-Ob", "-m", "all", ".1.0.0.0.1.1"},
 	{"snmptranslate", "-Td", "-Ob", "-m", "all", ".1.0.0.0.1.1.0"},
+	{"snmptranslate", "-Td", "-Ob", "-m", "all", ".1.0.0.0.1.4"},
 	{"snmptranslate", "-Td", "-Ob", "-m", "all", ".1.2.3"},
 	{"snmptranslate", "-Td", "-Ob", ".iso.2.3"},
 	{"snmptranslate", "-Td", "-Ob", "-m", "all", ".999"},

plugins/inputs/snmp/snmp_mocks_test.go

@@ -67,6 +67,7 @@ var mockedCommandResults = map[string]mockedCommandResult{
 	"snmptranslate\x00-Td\x00-Ob\x00-m\x00all\x001.0.0.1.1":                   mockedCommandResult{stdout: "TEST::hostname\nhostname OBJECT-TYPE\n  -- FROM\tTEST\n  SYNTAX\tOCTET STRING\n  MAX-ACCESS\tread-only\n  STATUS\tcurrent\n::= { iso(1) 0 testOID(0) 1 1 }\n", stderr: "", exitError: false},
 	"snmptranslate\x00-Td\x00-Ob\x00-m\x00all\x00.1.0.0.0.1.1":                mockedCommandResult{stdout: "TEST::server\nserver OBJECT-TYPE\n  -- FROM\tTEST\n  SYNTAX\tOCTET STRING\n  MAX-ACCESS\tread-only\n  STATUS\tcurrent\n::= { iso(1) 0 testOID(0) testTable(0) testTableEntry(1) 1 }\n", stderr: "", exitError: false},
 	"snmptranslate\x00-Td\x00-Ob\x00-m\x00all\x00.1.0.0.0.1.1.0":              mockedCommandResult{stdout: "TEST::server.0\nserver OBJECT-TYPE\n  -- FROM\tTEST\n  SYNTAX\tOCTET STRING\n  MAX-ACCESS\tread-only\n  STATUS\tcurrent\n::= { iso(1) 0 testOID(0) testTable(0) testTableEntry(1) server(1) 0 }\n", stderr: "", exitError: false},
+	"snmptranslate\x00-Td\x00-Ob\x00-m\x00all\x00.1.0.0.0.1.4":                mockedCommandResult{stdout: "TEST::testTableEntry.4\ntestTableEntry OBJECT-TYPE\n  -- FROM\tTEST\n  MAX-ACCESS\tnot-accessible\n  STATUS\tcurrent\n  INDEX\t\t{ server }\n::= { iso(1) 0 testOID(0) testTable(0) testTableEntry(1) 4 }\n", stderr: "", exitError: false},
 	"snmptranslate\x00-Td\x00-Ob\x00-m\x00all\x00.1.2.3":                      mockedCommandResult{stdout: "iso.2.3\niso OBJECT-TYPE\n  -- FROM\t#-1\n::= { iso(1) 2 3 }\n", stderr: "", exitError: false},
 	"snmptranslate\x00-Td\x00-Ob\x00.iso.2.3":                                 mockedCommandResult{stdout: "iso.2.3\niso OBJECT-TYPE\n  -- FROM\t#-1\n::= { iso(1) 2 3 }\n", stderr: "", exitError: false},
 	"snmptranslate\x00-Td\x00-Ob\x00-m\x00all\x00.999":                        mockedCommandResult{stdout: ".999\n [TRUNCATED]\n", stderr: "", exitError: false},

plugins/parsers/graphite/parser.go

@@ -57,38 +57,34 @@ func NewGraphiteParser(
 	defaultTemplate, _ := NewTemplate("measurement*", nil, p.Separator)
 	matcher.AddDefaultTemplate(defaultTemplate)
 
+	tmplts := parsedTemplates{}
 	for _, pattern := range p.Templates {
-		template := pattern
+		tmplt := parsedTemplate{}
-		filter := ""
+		tmplt.template = pattern
 		// Format is [filter] <template> [tag1=value1,tag2=value2]
 		parts := strings.Fields(pattern)
 		if len(parts) < 1 {
 			continue
 		} else if len(parts) >= 2 {
 			if strings.Contains(parts[1], "=") {
-				template = parts[0]
+				tmplt.template = parts[0]
+				tmplt.tagstring = parts[1]
 			} else {
-				filter = parts[0]
+				tmplt.filter = parts[0]
-				template = parts[1]
+				tmplt.template = parts[1]
+				if len(parts) > 2 {
+					tmplt.tagstring = parts[2]
+				}
 			}
 		}
+		tmplts = append(tmplts, tmplt)
+	}
 
-		// Parse out the default tags specific to this template
+	sort.Sort(tmplts)
-		tags := map[string]string{}
+	for _, tmplt := range tmplts {
-		if strings.Contains(parts[len(parts)-1], "=") {
+		if err := p.addToMatcher(tmplt); err != nil {
-			tagStrs := strings.Split(parts[len(parts)-1], ",")
+			return nil, err
-			for _, kv := range tagStrs {
-				parts := strings.Split(kv, "=")
-				tags[parts[0]] = parts[1]
-			}
 		}
-
-		tmpl, err1 := NewTemplate(template, tags, p.Separator)
-		if err1 != nil {
-			err = err1
-			break
-		}
-		matcher.Add(filter, tmpl)
 	}
 
 	if err != nil {
@@ -98,6 +94,24 @@ func NewGraphiteParser(
 	}
 }
 
+func (p *GraphiteParser) addToMatcher(tmplt parsedTemplate) error {
+	// Parse out the default tags specific to this template
+	tags := map[string]string{}
+	if tmplt.tagstring != "" {
+		for _, kv := range strings.Split(tmplt.tagstring, ",") {
+			parts := strings.Split(kv, "=")
+			tags[parts[0]] = parts[1]
+		}
+	}
+
+	tmpl, err := NewTemplate(tmplt.template, tags, p.Separator)
+	if err != nil {
+		return err
+	}
+	p.matcher.Add(tmplt.filter, tmpl)
+	return nil
+}
+
 func (p *GraphiteParser) Parse(buf []byte) ([]telegraf.Metric, error) {
 	// parse even if the buffer begins with a newline
 	buf = bytes.TrimPrefix(buf, []byte("\n"))
@@ -465,3 +479,30 @@ func (n *nodes) Less(j, k int) bool {
 
 func (n *nodes) Swap(i, j int) { (*n)[i], (*n)[j] = (*n)[j], (*n)[i] }
 func (n *nodes) Len() int      { return len(*n) }
+
+type parsedTemplate struct {
+	template  string
+	filter    string
+	tagstring string
+}
+type parsedTemplates []parsedTemplate
+
+func (e parsedTemplates) Less(j, k int) bool {
+	if len(e[j].filter) == 0 && len(e[k].filter) == 0 {
+		nj := len(strings.Split(e[j].template, "."))
+		nk := len(strings.Split(e[k].template, "."))
+		return nj < nk
+	}
+	if len(e[j].filter) == 0 {
+		return true
+	}
+	if len(e[k].filter) == 0 {
+		return false
+	}
+
+	nj := len(strings.Split(e[j].template, "."))
+	nk := len(strings.Split(e[k].template, "."))
+	return nj < nk
+}
+func (e parsedTemplates) Swap(i, j int) { e[i], e[j] = e[j], e[i] }
+func (e parsedTemplates) Len() int      { return len(e) }

plugins/parsers/graphite/parser_test.go

@@ -747,6 +747,48 @@ func TestApplyTemplateGreedyField(t *testing.T) {
 	}
 }
 
+func TestApplyTemplateOverSpecific(t *testing.T) {
+	p, err := NewGraphiteParser(
+		".",
+		[]string{
+			"measurement.host.metric.metric.metric",
+		},
+		nil,
+	)
+	assert.NoError(t, err)
+
+	measurement, tags, _, err := p.ApplyTemplate("net.server001.a.b 2")
+	assert.Equal(t, "net", measurement)
+	assert.Equal(t,
+		map[string]string{"host": "server001", "metric": "a.b"},
+		tags)
+}
+
+func TestApplyTemplateMostSpecificTemplate(t *testing.T) {
+	p, err := NewGraphiteParser(
+		".",
+		[]string{
+			"measurement.host.metric",
+			"measurement.host.metric.metric.metric",
+			"measurement.host.metric.metric",
+		},
+		nil,
+	)
+	assert.NoError(t, err)
+
+	measurement, tags, _, err := p.ApplyTemplate("net.server001.a.b.c 2")
+	assert.Equal(t, "net", measurement)
+	assert.Equal(t,
+		map[string]string{"host": "server001", "metric": "a.b.c"},
+		tags)
+
+	measurement, tags, _, err = p.ApplyTemplate("net.server001.a.b 2")
+	assert.Equal(t, "net", measurement)
+	assert.Equal(t,
+		map[string]string{"host": "server001", "metric": "a.b"},
+		tags)
+}
+
 // Test Helpers
 func errstr(err error) string {
 	if err != nil {

plugins/parsers/influx/parser.go

@@ -26,17 +26,15 @@ func (p *InfluxParser) Parse(buf []byte) ([]telegraf.Metric, error) {
 	points, err := models.ParsePoints(buf)
 	metrics := make([]telegraf.Metric, len(points))
 	for i, point := range points {
-		tags := point.Tags()
 		for k, v := range p.DefaultTags {
-			// Only set tags not in parsed metric
+			// only set the default tag if it doesn't already exist:
-			if _, ok := tags[k]; !ok {
+			if tmp := point.Tags().GetString(k); tmp == "" {
-				tags[k] = v
+				point.AddTag(k, v)
 			}
 		}
 		// Ignore error here because it's impossible that a model.Point
 		// wouldn't parse into client.Point properly
-		metrics[i], _ = telegraf.NewMetric(point.Name(), tags,
+		metrics[i] = telegraf.NewMetricFromPoint(point)
-			point.Fields(), point.Time())
 	}
 	return metrics, err
 }

plugins/processors/all/all.go

@@ -0,0 +1,5 @@
+package all
+
+import (
+	_ "github.com/influxdata/telegraf/plugins/processors/printer"
+)

plugins/processors/printer/printer.go

@@ -0,0 +1,35 @@
+package printer
+
+import (
+	"fmt"
+
+	"github.com/influxdata/telegraf"
+	"github.com/influxdata/telegraf/plugins/processors"
+)
+
+type Printer struct {
+}
+
+var sampleConfig = `
+`
+
+func (p *Printer) SampleConfig() string {
+	return sampleConfig
+}
+
+func (p *Printer) Description() string {
+	return "Print all metrics that pass through this filter."
+}
+
+func (p *Printer) Apply(in ...telegraf.Metric) []telegraf.Metric {
+	for _, metric := range in {
+		fmt.Println(metric.String())
+	}
+	return in
+}
+
+func init() {
+	processors.Add("printer", func() telegraf.Processor {
+		return &Printer{}
+	})
+}

plugins/processors/printer/printer_test.go

@@ -0,0 +1 @@
+package printer

plugins/processors/registry.go

@@ -0,0 +1,11 @@
+package processors
+
+import "github.com/influxdata/telegraf"
+
+type Creator func() telegraf.Processor
+
+var Processors = map[string]Creator{}
+
+func Add(name string, creator Creator) {
+	Processors[name] = creator
+}

processor.go

@@ -0,0 +1,12 @@
+package telegraf
+
+type Processor interface {
+	// SampleConfig returns the default configuration of the Input
+	SampleConfig() string
+
+	// Description returns a one-sentence description on the Input
+	Description() string
+
+	// Apply the filter to the given metric
+	Apply(in ...Metric) []Metric
+}

testutil/accumulator.go

@@ -39,6 +39,13 @@ func (a *Accumulator) NMetrics() uint64 {
 	return atomic.LoadUint64(&a.nMetrics)
 }
 
+func (a *Accumulator) ClearMetrics() {
+	atomic.StoreUint64(&a.nMetrics, 0)
+	a.Lock()
+	defer a.Unlock()
+	a.Metrics = make([]*Metric, 0)
+}
+
 // AddFields adds a measurement point with a specified timestamp.
 func (a *Accumulator) AddFields(
 	measurement string,