Commit Graph

208 Commits

Author SHA1 Message Date
Daniel Nelson 23a8498963
Fix locking if output is an AggregatingOutput 2018-09-10 15:14:14 -07:00
Daniel Nelson 1ca17652cd
Fix parsing and documentation for json_string_fields (#4656) 2018-09-10 11:55:08 -07:00
Greg 4ef058c120 Document all supported cli arguments (#4655) 2018-09-10 11:53:04 -07:00
Gunnar f70d6519e7 Add Azure Monitor output plugin (#4089) 2018-09-05 14:50:32 -07:00
maxunt 889745a112 Add csv parser (#4439) 2018-08-24 16:40:41 -07:00
maxunt 2729378b7f Add name, time, path and string field options to JSON parser (#4351) 2018-08-22 19:26:48 -07:00
estk edb6e1f655 Implement a lock based ring buffer for internal/buffer. (#3377) 2018-08-20 13:47:48 -07:00
Daniel Nelson 886d8cc840
Drop message batches in kafka output if too large (#4565) 2018-08-17 13:51:21 -07:00
Daniel Nelson feb75d493a
Lock buffer when adding metrics (#4514)
This function is not thread-safe but is currently used by multiple
goroutines in RunningOutput
2018-08-07 11:22:10 -07:00
Shanshi Shi e538433959 Preserve metric type when using filters in output plugins (#4481) 2018-08-01 16:39:54 -06:00
Daniel Nelson 93ed28e745 Add support for configuring an AWS endpoint_url (#4485) 2018-07-31 16:07:21 -06:00
maxunt 774a9f0492 Add file input plugin and grok parser (#4332) 2018-07-13 23:22:59 -07:00
maxunt 9ebf16636d Add parse_multivalue to collectd parser (#4403) 2018-07-11 17:29:23 -07:00
maxunt 39206677f8 Add new measurement with results of pgrep lookup to procstat input (#4307) 2018-06-19 11:47:13 -07:00
Daniel Nelson acba20af1a
Fix TLS and SSL config option parsing (#4247) 2018-06-06 18:29:59 -07:00
Piotr Popieluch dfe7b5eec2 Don't skip metrics during startup in aggregate phase (#4230) 2018-06-05 16:30:53 -07:00
Piotr Popieluch e0071f365a Print the enabled aggregator and processor plugins on startup (#4212) 2018-05-31 11:56:49 -07:00
Daniel Nelson 0d21296aed
Expose graphite_tag_support option in graphite output data format 2018-05-21 16:39:15 -07:00
Daniel Nelson de355b76d6
Simplify testing with TLS (#4095) 2018-05-04 16:33:23 -07:00
Daniel Nelson 7c0754ebe5
Move usage string to internal to fix `go run` 2018-05-04 14:16:21 -07:00
Bob Shannon 0b4f4b089f Add TLS support to socket_writer and socket_listener plugins (#4021) 2018-04-17 17:02:04 -07:00
Daniel Nelson c8d2ba2bc8
Remove RateLimiter tests due to race conditions
These tests are fundamentally racy, removing to improve reliability of
test cases.
2018-04-16 18:52:52 -07:00
jvassev 2c19d74829 Prevent loading config twice in K8S (#3999)
When config dir is mounted from configmap, filepath.Walk() finds the same
.conf file twice as 20-acme.conf is a link to ..data/20-acme.conf for example.

This patch skips all folder names starting with '..' which is pretty
uncommon and mainly used by Kubernetes mounts.
2018-04-11 16:51:19 -07:00
Daniel Nelson beeab2c509
Sort field names when running --test 2018-04-04 16:30:22 -07:00
Daniel Nelson b99cd14129
Add influx uint support as a runtime option (#3948) 2018-03-29 13:31:43 -07:00
Daniel Nelson 1c0f63a90d
Add new line protocol parser and serializer, influxdb output (#3924) 2018-03-27 17:30:51 -07:00
Daniel Nelson 8147d60973
Fix metric buffer limit in internal plugin after reload 2018-02-19 20:55:28 -08:00
Paul Myjavec 12d62e60b3 Use CircleCI 2.0 for builds (#3731) 2018-02-01 15:05:39 -08:00
Piotr Popieluch 601dc99606 Align aggregator period with internal ticker to avoid skipping metrics (#3693)
By the time the aggregator.run() was called about 600ms already passed since setting now which was skewing up the aggregation intervals and skipping metrics.
2018-01-18 17:37:53 -08:00
atzoum 317de40ac4 Add support for dropwizard input format (#2846) 2018-01-08 15:11:36 -08:00
Daniel Nelson b0c2bb870e
Escape environment variables during config toml parsing (#3637) 2018-01-04 15:28:00 -08:00
Maximilien Richer 5f215c22fe Fix typos in comments (#3415) 2017-10-31 17:00:06 -07:00
Daniel Nelson d0b690f040 Fix short tests on darwin (#3099) 2017-09-08 13:03:37 -07:00
Daniel Nelson 28d16188b3 Fix panic when handling string fields with escapes (#3188) 2017-08-30 21:16:37 -07:00
Daniel Nelson 573bd4aa32 Start first aggregator period at startup time (#3050)
Fixes issue where metrics collected immediately after startup would not
be aggregated.
2017-07-24 18:25:05 -07:00
Daniel Nelson 31ce98fa91 Don't match pattern on any error (#3040)
This prevents a pattern with no wildcards from matching in case
permissions is denied.
2017-07-21 14:28:14 -07:00
DanKans d903a9142d Fix filtering when both pass and drop match an item (#3036)
Adjust logic in functions responsible for passing metrics in order to be able
to process them correctly in case where pass and drop are defined together.
2017-07-21 10:53:57 -07:00
DanKans 34c042c7dc Fix combined tagdrop/tagpass filtering (#3031) 2017-07-19 13:08:40 -07:00
Daniel Nelson b165ce4cd5 Prevent possible deadlock when using aggregators (#3016)
Looping the metrics back through the same channel could result in a
deadlock, by using a new channel and locking the processor we can ensure
that all stages can make continual progress.
2017-07-13 15:34:21 -07:00
Daniel Nelson 5d2c093105 Prevent Write from being called concurrently (#3011) 2017-07-12 12:03:23 -07:00
Daniel Nelson 1388e2cf92 Do not allow metrics with trailing slashes (#3007)
It is not possible to encode a measurement, tag, or field whose last
character is a backslash due to it being an unescapable character.
Because the tight coupling between line protocol and the internal metric
model, prevent metrics like this from being created.

Measurements with a trailing slash are not allowed and the point will be
dropped.  Tags and fields with a trailing a slash will be dropped from
the point.
2017-07-11 15:54:38 -07:00
Seuf 2092443cd7 Add Kafka 0.9+ consumer support (#2487) 2017-06-07 18:22:28 -07:00
Daniel Nelson 052e88ad5e
Fix grammar 2017-04-27 14:59:18 -07:00
Patrick Hemmer 06baf7cf78 use AddError everywhere (#2372) 2017-04-24 11:13:26 -07:00
Daniel Nelson 360b10c4de Clarify precision documentation (#2655) 2017-04-12 10:42:11 -07:00
Daniel Nelson 2c98e5ae66 Add collectd parser (#2654) 2017-04-12 10:41:26 -07:00
Daniel Nelson 540f98e228 Fix possible deadlock when output cannot write. (#2610) 2017-03-31 12:45:28 -07:00
tjmcs fb1c7d0154 Adds a new json_timestamp_units configuration parameter (#2587) 2017-03-29 17:12:29 -07:00
Cameron Sparr 9e810ac463
Handle nil os.FileInfo in filepath.Walk
closes #2466
2017-02-28 17:51:03 +00:00
Cameron Sparr 06176ef410
Only set the buffer size once
fixes #2380
2017-02-17 14:11:15 +00:00
Cameron Sparr 21cf79163c
don't use influxdata/config, just use influxdata/toml 2017-02-10 17:27:18 +00:00
Cameron Sparr f7d551a807
Add more nested globpath tests 2017-02-01 23:44:35 +00:00
Cameron Sparr e87ce22af9
running output: Drop nil metrics
fixes #2317
2017-02-01 08:55:22 +00:00
Cameron Sparr 7279018cfe
readme fixup & test output fixup 2017-01-09 12:28:13 +00:00
Cameron Sparr 12db3b9120
Check if metric is nil before calling SetAggregate
fixes #2146
2016-12-13 12:27:10 +00:00
Cameron Sparr d71a42cd1b Implement telegraf collecting stats on itself
closes #1348
2016-12-05 18:56:54 +00:00
Cameron Sparr e5c7a71d8e Fix unit tests for new metric implementation 2016-12-01 18:17:02 +00:00
Cameron Sparr db7a4b24b6 Implement telegraf's own full metric type
main reasons behind this:
- make adding/removing tags cheap
- make adding/removing fields cheap
- make parsing cheaper
- make parse -> decorate -> write out bytes metric flow much faster

Refactor serializer to use byte buffer
2016-12-01 18:17:02 +00:00
Cameron Sparr 0ecbf9e349 Fix single quote parsing of TOML durations
closes #2023
2016-11-10 09:47:46 +00:00
Cameron Sparr f729fa990d Unit testing for internal.Duration Unmarshal
closes #1926
2016-10-25 13:11:32 +01:00
Alex Zorin 662db7a944 Fix panic in internal.Duration UnmarshalTOML 2016-10-25 18:30:01 +11:00
Cameron Sparr c849b58de9 http_listener input unit tests 2016-10-24 18:17:49 +01:00
Cameron Sparr 097b1e09db http listener refactor
in this commit:

- chunks out the http request body to avoid making very large
  allocations.
- establishes a limit for the maximum http request body size that the
  listener will accept.
- utilizes a pool of byte buffers to reduce GC pressure.
2016-10-24 18:17:49 +01:00
Cameron Sparr b00ad65b08 Log config file parsing errors properly
closes #1344
2016-10-12 16:50:22 +01:00
Cameron Sparr a84ce5d5cb drop metrics outside of the aggregators period 2016-10-12 14:56:03 +01:00
Cameron Sparr 8ca4a50c18 delete nil fields in the metric maker.
closes #1771
2016-10-12 14:50:19 +01:00
Cameron Sparr 03b2984ac2 Fixup some code based on feedback from @dgnorton 2016-10-12 14:50:19 +01:00
Cameron Sparr 9540a6532f Update influxdb dependency for new models.Tags 2016-10-12 14:50:19 +01:00
Cameron Sparr 78f544c0aa Support --aggregator-filter & --processor-filter 2016-10-12 14:50:19 +01:00
Cameron Sparr b03c1d9691 Support ordering of processor plugins 2016-10-12 14:50:19 +01:00
Cameron Sparr fead80844e Refactor handling of MinMax functionality into RunningAggregator
allows for easier addition of a sliding window at a later time.

Also makes `period` be a generic argument for all aggregator plugins.
2016-10-12 14:50:19 +01:00
Cameron Sparr ef885eda62 Change minmax aggregator to store float64 2016-10-12 14:50:19 +01:00
Cameron Sparr 64a71263a1 Support Processor & Aggregator Plugins
closes #1726
2016-10-12 14:50:19 +01:00
Cameron Sparr c7834209d2 Major Logging Overhaul
in this commit:

- centralize logging output handler.
- set global Info/Debug/Error log levels based on config file or flags.
- remove per-plugin debug arg handling.
- add a I!, D!, or E! to every log message.
- add configuration option to specify where to send logs.

closes #1786
2016-10-03 17:13:03 +01:00
Cameron Sparr e19845c202 Load config directory using filepath.Walk
closes #1137
2016-09-28 16:01:52 +01:00
Cameron Sparr 52134555d6 globpath: only walk tree if ** is defined
closes #1517
2016-09-28 15:44:29 +01:00
Cameron Sparr 2fe427b3b3 mongodb input: fix version 2.2 panic
closes #1628
2016-09-06 11:58:06 +01:00
Cameron Sparr 50ef3282b6 Refactor and code cleanup of filtering
started working on this with the idea of fixing #1623, although I
realized that this was actually just a documentation issue around
a toml eccentricity.

closes #1623
2016-09-05 16:30:18 +01:00
Cameron Sparr 8e22526756 Adding c:\program files\telegraf\telegraf.conf
this will now be the default config file location on windows, basically
it is the windows equivalent of /etc/telegraf/telegraf.conf

also updating the changelog

closes #1543
2016-08-08 23:17:27 +01:00
Cameron Sparr d3bb1e7010 Rename internal_models package to models 2016-08-08 14:41:40 +01:00
Cameron Sparr 49988b15a3 Default config typo fix 2016-08-06 07:40:28 +01:00
Jason Gardner 986735234b Fix output config typo. (#1527) 2016-07-22 16:05:53 +01:00
Cameron Sparr 2d4864e126 nil metric list panic fix 2016-06-29 12:08:36 +02:00
Cameron Sparr 755b2ec953 fixup: BOM Trim -> TrimPrefix 2016-06-24 08:47:31 +01:00
Cameron Sparr 5ddd61d2e2 Trim BOM from config file for windows support
closes #1378
2016-06-23 09:00:51 +01:00
Mike Glazer e3448153e1 Allow for TLS connections to ElasticSearch (#1398)
* Allow for TLS connections to ElasticSearch

Extremely similar implementation to the HTTP JSON module's
implementation of the same code.

* Changelog update
2016-06-22 16:23:49 +01:00
Cameron Sparr cb3c54a1ae logparser input plugin
closes #102
closes #328
2016-06-21 14:23:01 +01:00
Cameron Sparr d7efb7a71d Add precision rounding to accumulator
Adding precision rounding to the accumulator. This means that now every
input metric will get rounded at collection, rather than at write (and
only for the influxdb output).

This feature is disabled for service inputs, because service inputs
should be in control of their own timestamps & precisions.
2016-06-14 00:36:39 +01:00
Cameron Sparr 2c448e22e1 New object: ErrChan for concurrent err handling 2016-06-02 13:29:37 +01:00
Cameron Sparr bd1282eddf Don't print config with trailing whitespace 2016-05-31 16:25:02 +01:00
Cameron Sparr 892abec025 Refactor collection_jitter and flush_jitter
use a common function between collection_jitter and flush_jitter. which
creates the same behavior between the two options.

going forward, both jitters will be random sleeps that get re-evaluated
at runtime for every interval (previously only collection_jitter did
this)

also fixes behavior so that both jitters will exit in the event of a
process exit.

closes #1296
2016-05-31 14:24:32 +01:00
Cameron Sparr 9ff536d94d Limit GetMetricStatistics to 10 per second
closes #1197
2016-05-31 11:26:52 +01:00
Cameron Sparr 9bbdb2d562 Allow wildcard filtering of varnish stats
closes #1275
2016-05-26 10:42:34 +01:00
John Engelman 5f3a91bffd Consolidate AWS credentials (#1208)
* Use shared AWS credential configuration.

*  Cloudwatch dimension wilcards 

* Allow configuring cache_ttl for cloudwatch metrics.

* Allow for wildcard in dimension values to select all available metrics.

* Use internal.Duration for CacheTTL and go fmt

* Refactor to not use embedded structs for config.

* Update AWS plugin READMEs with credentials details, update Changelog.

* Fix changelog after rebasing to master and 0.13.1 release.

* Fix changelog after rebase.
2016-05-25 12:30:39 +01:00
Cameron Sparr c44ecf54a5 Utilize timeout in net_response plugin.
Also changing the net_response and http_response plugins to only accept
duration strings for their timeout parameters. This is a breaking config
file change.

closes #1214
2016-05-23 15:59:23 +01:00
Cameron Sparr 4e9798d0e6 agent and tags configs sometimes not applied
closes #1090
2016-04-29 19:44:01 -06:00
Cameron Sparr 3f807a9432 Implement timeouts for all exec command runners
First is to write an internal CombinedOutput and Run function with a
timeout.

Second, the following instances of command runners need to have timeouts:

    plugins/inputs/ping/ping.go
    125:	out, err := c.CombinedOutput()

    plugins/inputs/exec/exec.go
    91:	if err := cmd.Run(); err != nil {

    plugins/inputs/ipmi_sensor/command.go
    31:	err := cmd.Run()

    plugins/inputs/sysstat/sysstat.go
    194:	out, err := cmd.CombinedOutput()

    plugins/inputs/leofs/leofs.go
    185:	defer cmd.Wait()

    plugins/inputs/sysstat/sysstat.go
    282:	if err := cmd.Wait(); err != nil {

closes #1067
2016-04-29 12:06:22 -06:00
Cameron Sparr cbe32c7482 Support default config paths
precedence will be:

 1. --config command-line option
 2. $TELEGRAF_CONFIG_PATH
 3. $HOME/.telegraf/telegraf.conf
 4. /etc/telegraf/telegraf.conf
2016-04-28 17:48:24 -06:00
Cameron Sparr 4de75ce621 Performance refactor of running_output buffers
closes #914
closes #967
2016-04-27 13:40:05 -06:00
Cameron Sparr c7719ac365 buffers: fix bug when Write called before AddMetric 2016-04-26 10:25:04 -06:00