* Add configuration docs to Postgresql input plugin
Add configuration docs to PostgreSQL input plugin README (mostly from the source code) though I've not included the configuration example that seems to use all he connections on the database[1].
[1] https://github.com/influxdata/telegraf/issues/2410
* Fix typo in readme and sampleConfig string.
* Exporting Ipmi.Path to be set by config.
Currently "path" is not exported, giving this error when users try to
override the variable via telegraf.conf as per the sample config:
`field corresponding to `path' is not defined in `*ipmi_sensor.Ipmi'`
Exporting the variable solves the problem.
* Updating changelog.
During issue #2215 it was highlighted that the current behavior where
rules without a comment are ignored is confusing for several users.
This commit improves the documentation and adds a NOTE to the sample
config to clarify the behavior for new users.
* Procstat: don't cache PIDs
Changed the procstat input plugin to not cache PIDs. Solves #1636.
The logic of creating a process by pid was moved from `procstat.go` to
`spec_processor.go`.
* Procstat: go fmt
* procstat: modify changelog for #2206
* ceph: maps are already refs, no need to use a pointer
* ceph: pgmap_states are represented in a single metric "count", differenciated by tag
* Update CHANGELOG
* Add in support for looking for substring in response
* Add note to CHANGELOG.md
* Switch from substring match to regex match
* Requested code changes
* Make requested changes and refactor to avoid nested if-else.
* Convert tabs to space and compile regex once
* Make Logparser Plugin Check For New Files
Check in the Gather metric to see if any new files matching the glob
have appeared. If so, start tailing them from the beginning.
* changelog update for #2141
This changes the current use of the InfluxDB client to instead use a
baked-in client that uses the fasthttp library.
This allows for significantly smaller allocations, the re-use of http
body buffers, and the re-use of the actual bytes of the line-protocol
metric representations.
We were having problems with telegraf talking to
carbon-relay-ng using the graphite output. When
the carbon-relay-ng server restarted the connection
the telegraf side would go into CLOSE_WAIT but telegraf
would continue to send statistics through the connection.
Reading around it seems you need to a read from the connection
and see a EOF error. We've implemented this and added a test
that replicates roughly the error we were having.
Pair: @whpearson @joshmyers
If we write a batch of points and get a "field type conflict" error
message in return, we should drop the entire batch of points because
this indicates that one or more points have a type that doesnt match the
database.
These errors will never go away on their own, and InfluxDB will
successfully write the points that dont have a conflict.
closes#2245
* Added GatherUserStatistics, row Uptime in gatherGlobalStatuses, and version fields & tags
* Updated README file
* pulling in latest from master
* ran go fmt to fix formatting
* fix unreachable code
* few fixes
* cleaning up and applying suggestions from sparrc
I don't like this behavior, but it's what InfluxDB accepts, so the
telegraf listener should be consistent with that.
I accidentally reverted this behavior when I refactored the telegraf
metric representation earlier in this release cycle.
* Fix for broken librato output
These errors are delightful, but I'd rather avoid them:
```
Error parsing /etc/telegraf/telegraf.conf, line 2: field corresponding to `api_user' is not defined in `*librato.Librato'
```
* Fixed bad format from last commit
this basically reverts #887
at some point we might want to do some special handling of reloading
plugins and keeping their state intact, but that will need to be done at
a higher level, and in a way that is thread-safe for multiple input
plugins of the same type.
Unfortunately this is a rather large feature that will not have a quick
fix available for it.
fixes#1975fixes#2102
* plugins/input/consul: moved check_id from regular fields to tags.
When service has more than one check sending data for both would overwrite each other
resulting only in one check being written (the last one). Adding check_id as a tag
ensures we will get info for all unique checks per service.
* plugins/inputs/consul: updated tests
* fixed parsing of docker image name/version
now accounts for custom docker repo's which contain a colon for a non-default port
* 1978: modifying docker test case to have a custom repo with non-standard port
* using a temp var to store index, ran gofmt
* fixes#1987, renaming iterator to 'i'
* MongoDB input plugin: Improve state data
Adds ARB as a "member_status" (replica set arbiter).
Uses MongoDB replica set state string for "state" value.
* MongoDB input plugin: Improve state data - changelog update
put Makefile back to normal
removed comment from puppetagent.go
changed config_version to config_version_string and fixed yaml for build
changed workind from branch to environment for config_string
fixed casing and Changelog
fixed test case
closes#1917
* Fix bug: too many cloudwatch metrics
Cloudwatch metrics were being added incorrectly. The most obvious
symptom of this was that too many metrics were being added. A simple
check against the name of the metric proved to be a sufficient fix. In
order to test the fix, a metric selection function was factored out.
* Go fmt cloudwatch
* Cloudwatch isSelected checks metric name
* Move cloudwatch line in changelog to 1.2 features
* return partition stat alongside disk stat from disk usage method, and report device name (minus /dev/) as a tag in disk stats
* update system/disk tests to include new partition stat return value from disk usage method calls
* update changelog for #1807 (use device name instead of path to report disk stats)
main reasons behind this:
- make adding/removing tags cheap
- make adding/removing fields cheap
- make parsing cheaper
- make parse -> decorate -> write out bytes metric flow much faster
Refactor serializer to use byte buffer
The old gonuts fork has no License and has not seen any commits
differing from the original project, while the original has seen some
activity, even if low.
Having no license is a problem for distributors, as by default, such
projects are undistributable.
* Trim null characters in Value data format
Some producers (such as the paho embedded c mqtt client) add a null
character "\x00" to the end of a message. The Value parser would fail on
any message from such a producer.
* Trim whitespace and null in all Value data formats
* No unnecessary reassignments in Value data format parser
* Update change log for Value data format fix
* added connection Timeout parámeter, basic HTTP autentication and HTTP support with Sslskipverify option
* updated README.md
* added optional SSL config , changed timeout name and type , and other minor fixes
* added some code style improvements
* Update README.md
* NATS output plug-in now retries to reconnect forever after a lost connection.
* NATS input plug-in now retries to reconnect forever after a lost connection.
* Fixes#1953
in this commit:
- chunks out the http request body to avoid making very large
allocations.
- establishes a limit for the maximum http request body size that the
listener will accept.
- utilizes a pool of byte buffers to reduce GC pressure.
The MySQL DB driver has it's own DSN parsing function. Previously we
were using the url.Parse function, but this causes problems because a
valid MySQL DSN can be an invalid http URL, namely when using some
special characters in the password.
This change uses the MySQL DB driver's builtin ParseDSN function and
applies a timeout parameter natively via that.
Another benefit of this change is that we fail earlier if given an
invalid MySQL DSN.
closes#870closes#1842
Previously, the graphite parser would simply overwrite any template that
had an identical filter to a previous template. This included the empty
filter.
Now we will still overwrite, but first we will sort to make sure that
the most "specific" template always matches.
closes#1731
Map holding expected results was defined in multiple places, making test
cases a bit hard to read. This way we can change our expectations of
good results in one place and have them affect multiple test cases.
in this commit:
- centralize logging output handler.
- set global Info/Debug/Error log levels based on config file or flags.
- remove per-plugin debug arg handling.
- add a I!, D!, or E! to every log message.
- add configuration option to specify where to send logs.
closes#1786
Due to quite real problem of generating vast number of data series through
mesos tasks metrics this feature is disabled until better solution is found.
Also consolidated the translation code to obtain all info with just 1 command execution.
Also split test command mocks out to their own file for cleanliness.
The default is 0 so we hit a division by 0 error and crash. This checks
ensure we will not crash and `log` and continue to let telegraf run
Also we set default allow pending message number to 10000
* Allow numeric and non-string values for tag_keys.
According to the go documentation the JSON deserializer only produces these
base types in output:
- string
- bool
- float64
- nil
With this patch bool, float64 and nil values get converted to a string when
their field key is specified in tag_keys. Previously the field was simply
discarded.
* Updated handling of nil for passing tests.
The automated tests are less than trivial to reproduece locally for me,
so I hope CircleCI wonn't mind...
* Updated changelog entries with PR and issue links.
* separate hello and authenticate functions, force connection close at end of write cycle so we don't hold open idle connections, which has the benefit of mostly removing the chance of getting hopelessly connection lost
* update changelog, though this will need to be updated again to merge into telegraf master
* bump instrumental agent version
* fix test to deal with better better connect/reconnect logic and changed ident & auth handshake
* Update CHANGELOG.md
correct URL from instrumental fork to origin and put the change in the correct part of the file
* go fmt
* Split out Instrumental tests for invalid metric and value.
* Ensure nothing remains on the wire after final test.
* Force valid metric names by replacing invalid parts with underscores.
* Multiple invalid characters being joined into a single udnerscore.
* Adjust comment to what happens.
* undo split hello and auth commands, to reduce roundtrips
* Add ignored_databases option to postgresql configuration files, to enable easy filtering of system databases without needing to whitelist all the databases on the server. Add tests for database whitelist and blacklist.
* run go fmt on new postgresql database whitelist/blacklist code
* add postgresql database blacklist option to changelog
* remove a bad merge from the changelog
also remove locking around adding metrics. Instead, keep a waitgroup on
the ServeHTTP function and wait for that to finish before returning from
the Stop() function
closes#1407
fix incredibly stupid bugs
populate README
support query endpoint and change default listen port
set response headers for query endpoint
add unit tests
revert erroneous Godeps change
add plugin ref to top-level README
remove debug output and add empty post body test
fix linter errors
move stoppableListener into repo
use constants for http status codes
add CHANGELOG entry
address code review comments re. style/structure
address further code review comments
add note to README re. database creation calls per PR comments
started working on this with the idea of fixing #1623, although I
realized that this was actually just a documentation issue around
a toml eccentricity.
closes#1623
And use them in the prometheus output plugin.
Still need to test the prometheus output plugin.
Also need to actually create typed metrics in the system plugins.
closes#1683
closes#1542
Generalize event.
Add doc.
Update default config.
Add filestack to the list of plugins.
Check that video conversion event returns 400.
Update the readme.
Update the changelog.
The iptables plugin aims at monitoring bytes and packet counters
matching a given set of iptables rules.
Typically the user would set a dedicated monitoring chain into a given
iptables table, and add the rules to monitor to this chain. The plugin
will allow to focus on the counters for this particular table/chain.
closes#1471
Added the option removecr to inputs.exec to remove all carraige returns
(CR, ASCII 0x0D, Unicode codepoint \u0D, ^M). The option is boolean and
not enabled if not present in the config file.
closes#1606
Updated CHANGELOG.md with information about removecr
Ran go fmt ./...
Moved removal of CRs to internal/internal.go
Moved the code to remove carriage returns from
plugins/inputs/exec/exec.go to internal/internal.go. Additionally
changed the conditional on which it gets applied from using a
configuration file option to checking if it is running on Windows.
Moved Carriage Return check to correct place
Moved the carriage return removal back to the exec plugin. Added unit
testing for it. Fixed a bug (removing too many characters).
Ran go fmt ./...
Reverted CHANGELOG to master
Updated Changelog
* Move CloudWatch rate limit to config
Reference #1670
* make that variable a string
* ahem, apparently limiter wants an int
* add the ratelimit to the sample config
* update the test to include the rate
* set a default value of 10 for ratelimit
* Move default ratelimit to init
closes#1539
First version of http put working
Refactored code to separate http handling from opentsdb module. Added batching support.
Fixed tag cleaning in http output and refactored telnet output.
Removed useless struct.
Fixed current unittest and added a new one.
Added benchmark test to test json serialization. Made sure http client would reuse connection.
Ran go fmt on opentsdb sources.
Updated README file
Removed useHttp in favor of parsing host string to determine the right API to use for sending metrics. Also renamed BatchSize to HttpBatchSize to better convey that it is only used when using Http API.
Updated changelog
Fixed format issues.
Removed TagSet type to make it more explicit.
Fixed unittest after removing TagSet type.
Revert "Updated changelog"
This reverts commit 24dba5520008d876b5a8d266c34a53e8805cc5f5.
Added PR under 1.1 release.
add missing redis metrics
This makes sure that all redis metrics are present without having to use a hard-coded list of what metrics to pull in.
The existing ceph input plugin only has access to the local admin daemon socket
on the local host, and as such has access to a limited subset of data. This
extends the plugin to use CLI commands to get access to the full spread of Ceph
data. This patch collects global OSD map and IO statistics, PG state and per pool
IO and utilization statistics.
closes#1513
* Some improvment in mesos input plugin,
Removing uneeded statistics prefix for task's metric,
Adding framework id tags into each task's metric,
Adding state (leader/follower) tags to master's metric,
Make sure the slave's metrics are tags with slave
* typo, replacing cpus_total with elected to determine leader
* Remove remaining statistics_ from sample
* using timestamp from mesos as metric timestamp
* change mesos-tasks to mesos_tasks, measurement
* change measurement name in test
* Replace follower by standby
* separate hello and authenticate functions, force connection close at end of write cycle so we don't hold open idle connections, which has the benefit of mostly removing the chance of getting hopelessly connection lost
* update changelog, though this will need to be updated again to merge into telegraf master
* bump instrumental agent version
* fix test to deal with better better connect/reconnect logic and changed ident & auth handshake
* Update CHANGELOG.md
correct URL from instrumental fork to origin and put the change in the correct part of the file
* go fmt
* Split out Instrumental tests for invalid metric and value.
* Ensure nothing remains on the wire after final test.
* Force valid metric names by replacing invalid parts with underscores.
* Multiple invalid characters being joined into a single udnerscore.
* Adjust comment to what happens.
* undo split hello and auth commands, to reduce roundtrips
* Split out Instrumental tests for invalid metric and value.
* Ensure nothing remains on the wire after final test.
* Force valid metric names by replacing invalid parts with underscores.
* Multiple invalid characters being joined into a single udnerscore.
* add an entry to CHANGELOG for easy merging upstream
* go fmt variable alignment
* remove some bugfixes from changelog which now more properly are in a different section.
* remove headers and whitespace should should have been removed with the last commit
* Source improvement for librato output
Build the source from the list of tag instead of a configuration specified
single tag
Graphite Serializer:
* make buildTags public
* make sure not to use empty tags
Librato output:
* Improve Error handling for librato API base on error or debug flag
* Send Metric per Batch (max 300)
* use Graphite BuildTag function to generate source
The change is made that it should be retro compatible
Metric sample:
server=127.0.0.1 port=80 state=leader env=test
measurement.metric_name value
service_n.metric_x
Metric before with source tags set as "server":
source=127.0.0.1
test.80.127_0_0_1.leader.measurement.metric_name
test.80.127_0_0_1.leader.service_n.metric_x
Metric now:
source=test.80.127.0.0.1.leader
measurement.metric_name
service_n.metric_x
As you can see the source in the "new" version is much more precise
That way when filter (only from source) you can filter by env or any other tags
* Using template to specify which tagsusing for source, default concat all
tags
* revert change in graphite serializer
* better documentation, change default for template
* fmt
* test passing with new host as default tags
* use host tag in api integration test
* Limit 80 char per line, change resolution to be a int in the sample
* fmt
* remove resolution, doc for template
* fmt
* Fix problem with metrics when ping return Destination net unreachable
Add test case TestUnreachablePingGather
Add percent_reply_loss
Fix some other tests
* Add errors measurment
* fir problem with ping reply "TTL expired in transit" ( use regex for more specific condition - TTL in line but it's a not valid replay )
add test case for "TTL expired in transit" - TestTTLExpiredPingGather
this log format is likely soon going to be removed from a future
influxdb release, so we should not be recommending that users base any
of their log parsing infra on this.
* Ping for windows
* En ping output
* Code format
* Code review
* Default timeout
* Fix problem with std error when no data received ( exit status = 1 )
1. in prometheus client, do not check for invalid characters anymore,
because we are already replacing all invalid characters with regex
anyways.
2. in win_perf_counters, sanitize field name _and_ measurement name.
Also add '%' to the list of sanitized characters, because this character
is invalid for most output plugins, and can also easily cause string
formatting issues throughout the stack.
3. All '%' will now be translated to 'Percent'
closes#1430
closes#1499closes#1019
Do no try to guess HAproxy stats url, just add ";csv" at the end of the
url if not present.
Signed-off-by: tgermain <timothee.germain@corp.ovh.com>
* add initial support to allow self-signed certs
When using self-signed the metrics collection will fail, this will allow
the user to specify in the input configuration file if they want to skip
certificate verification. This is functionally identical to `curl -k`
At some point this functionality should be moved to the agent as it is
already implemented identically in several different input plugins.
* Add initial comment strings to remove noise
These should be properly fleshed out at some point to ensure
code completeness
* refactor to use generic helper function
* fix import statement against fork
* update changelog
closes#1436
This also fixes the bad behavior of waiting until runtime to return log
parsing pattern compile errors when a pattern was simply unfound.
closes#1418
Also protect against user error when the telegraf user does not have
permission to open the provided file. We will now error and exit in this
case, rather than silently waiting to get permission to open it.
* Add mandrill webhook.
* Store the id of the msg as part of event.
Signed-off-by: Cyril Duez <cyril@stormz.me>
Signed-off-by: François de Metz <francois@stormz.me>
* Decode body to get the mandrill_events.
Signed-off-by: Cyril Duez <cyril@stormz.me>
Signed-off-by: François de Metz <francois@stormz.me>
* Handle HEAD request.
Signed-off-by: Cyril Duez <cyril@stormz.me>
Signed-off-by: François de Metz <francois@stormz.me>
* Add the README.
Signed-off-by: Cyril Duez <cyril@stormz.me>
Signed-off-by: François de Metz <francois@stormz.me>
* Add mandrill_webhooks to the README.
Signed-off-by: Cyril Duez <cyril@stormz.me>
Signed-off-by: François de Metz <francois@stormz.me>
* Update changelog.
Signed-off-by: Cyril Duez <cyril@stormz.me>
Signed-off-by: François de Metz <francois@stormz.me>
* Run gofmt.
Signed-off-by: Cyril Duez <cyril@stormz.me>
Signed-off-by: François de Metz <francois@stormz.me>
closes#1412
separate hello and authenticate functions,
force connection close at end of write cycle so we don't
hold open idle connections,
which has the benefit of mostly removing
the chance of getting hopelessly connection lost
bump instrumental agent version
fix test to deal with better better connect/reconnect logic and changed ident & auth handshake
Update CHANGELOG.md
correct URL from instrumental fork to origin and put the change in the correct part of the file
go fmt
undo split hello and auth commands, to reduce roundtrips
This is for better thread-safety when running with multiple outputs,
which can cause very odd panics at very high loads
primarily this is to address #1432closes#1432
closes#1289
Signed-off-by: François de Metz <francois@stormz.me>
Signed-off-by: Cyril Duez <cyril@stormz.me>
Rename internals struct.
Signed-off-by: François de Metz <francois@stormz.me>
Signed-off-by: Cyril Duez <cyril@stormz.me>
Update changelog.
Signed-off-by: François de Metz <francois@stormz.me>
Signed-off-by: Cyril Duez <cyril@stormz.me>
Update READMEs and CHANGELOG.
Signed-off-by: François de Metz <francois@stormz.me>
Signed-off-by: Cyril Duez <cyril@stormz.me>
Update SampleConfig.
Update the config format.
Update telegraf config.
Update the webhooks README.
Update changelog.
Update the changelog with an upgrade path.
Update default ports.
Fix indent.
Check for nil value on AvailableWebhooks.
Check for CanInterface.
* Allow for TLS connections to ElasticSearch
Extremely similar implementation to the HTTP JSON module's
implementation of the same code.
* Changelog update
I added Rows/Logs max size counters for tracking databases that do not have autogrowth enabled. The counters return numbers in 8KB pages since there are a few special values (such as -1 for no max size) that can't directly be multiplied by 8192 to get size in bytes.
Also added Rows/Logs size in 8KB pages for comparison from the same system table. Even though it returns the same size as sizes from sys.dm_io_virtual_file_stats which are already collected.
Adding precision rounding to the accumulator. This means that now every
input metric will get rounded at collection, rather than at write (and
only for the influxdb output).
This feature is disabled for service inputs, because service inputs
should be in control of their own timestamps & precisions.
* WIP: Initial support for ZFS on FreeBSD
* Added build directives
* Ignore 'kstatPath' config option on FreeBSD
* Added tests for ZFS FreeBSD input plugin.
* Updated the README to confrom with the guidelines and added FreeBSD info
* Fixed indents
* Spell check
- Updated README/CHANGELOG
- Added links to further info to input README
- Reduced lines to 80 chars
Removing input declaration from SampleConfig
Moved PR to unreleased section of changelog
closes#1165
- Collects conntrack stats from the configured directories and files.
Applying PR feedback:
- Rebased onto master
- Updated README/CHANGELOG
- Limited lines to 80 chars
- Improved plugin docs and README
- added a dummy notlinux build file
Fixed up CHANGELOG and README after rebase
closes#1164
Allow using glob pattern in the command list in configuration. This enables for
example placing all commands in a single directory and using /path/to/dir/*.sh
as one of the commands to run all shell scripts in that directory.
Glob patterns are applied on every run of the commands, so matching commands can
be added without restarting telegraf.
closes#1142
* Use shared AWS credential configuration.
* Cloudwatch dimension wilcards
* Allow configuring cache_ttl for cloudwatch metrics.
* Allow for wildcard in dimension values to select all available metrics.
* Use internal.Duration for CacheTTL and go fmt
* Refactor to not use embedded structs for config.
* Update AWS plugin READMEs with credentials details, update Changelog.
* Fix changelog after rebasing to master and 0.13.1 release.
* Fix changelog after rebase.
* Report rollbar events.
Signed-off-by: Cyril Duez <cyril@stormz.me>
Signed-off-by: François de Metz <francois@stormz.me>
* Fix indent with go fmt.
* Add test for rollbar webhooks.
* Report more data from new_item event.
* Handle new deploy webhook.
Signed-off-by: Cyril Duez <cyril@stormz.me>
Signed-off-by: François de Metz <francois@stormz.me>
* Update default port.
* Add readme.
* Add rollbar_webhooks to the readme.
* Add rollbar_webhooks to plugins list.
* Add tag level for new_item event.
* Update readme.
* Update changelog.
* Adding Varnish HTTP Cache input plugin
* Applying PR feedback
- Linked to varnish in input README
- Updated README/CHANGELOG
- Cleaned up sampleConfig to remove formatting
- Shorted lines to 80 chars (except where test input requires long strings)
- Using internal.RunTimeout to wrap call to varnishtat
- Added dummy file for windows
Also changing the net_response and http_response plugins to only accept
duration strings for their timeout parameters. This is a breaking config
file change.
closes#1214
Being able to override the process_name in the procstat module
is useful for daemonized perl, ruby, erlang etc. processes. This
allows for manually setting process_name rather than it being set to
the interpreter/VM of the process.
Allow using glob pattern in the command list in configuration. This enables for
example placing all commands in a single directory and using /path/to/dir/*.sh
as one of the commands to run all shell scripts in that directory.
Glob patterns are applied on every run of the commands, so matching commands can
be added without restarting telegraf.
closes#1127
First is to write an internal CombinedOutput and Run function with a
timeout.
Second, the following instances of command runners need to have timeouts:
plugins/inputs/ping/ping.go
125: out, err := c.CombinedOutput()
plugins/inputs/exec/exec.go
91: if err := cmd.Run(); err != nil {
plugins/inputs/ipmi_sensor/command.go
31: err := cmd.Run()
plugins/inputs/sysstat/sysstat.go
194: out, err := cmd.CombinedOutput()
plugins/inputs/leofs/leofs.go
185: defer cmd.Wait()
plugins/inputs/sysstat/sysstat.go
282: if err := cmd.Wait(); err != nil {
closes#1067
Lustre Jobstats allows for RPCs to be tagged with a value, such
as a job's ID. This allows for per job statistics. This plugin
collects statistics and tags the data with the jobid.
closes#1107
Allow overriding the the metrics "server" tag with the specified
value. Can be used to give a more user-friendly value for the server
name.
closes#1093
this is so that we don't call os.Stat twice for every file matched
by Match(). Also changing the behavior to _not_ return the name of a
file that doesn't exist if it's not a glob.
Network metrics are pretty important and the block adds a couple with a link to the names for more. This adds a block with a few counters to the Generic Queries examples in plugins/inputs/win_perf_counters/README.md
- renaming cont_name and cont_image to container_name and
container_image.
- cont_id is now a field, called container_id
- docker_cpu, docker_mem, docker_net measurements have been renamed to
docker_container_cpu, docker_container_mem, and docker_container_net
closes#1014closes#1052
changes:
- -sample-config will now comment out all but a few default plugins.
- config file parse errors will output path to bad conf file.
- cleanup 80-char line-length and some other style issues.
- default package conf file will now have all plugins, but commented
out.
closes#199closes#944
The following configuration is now possible
## CompressionCodec represents the various compression codecs
recognized by Kafka in messages.
## "none" : No compression
## "gzip" : Gzip compression
## "snappy" : Snappy compression
# compression_codec = "none"
## RequiredAcks is used in Produce Requests to tell the broker how
many replica acknowledgements it must see before responding
## "none" : the producer never waits for an acknowledgement from the
broker. This option provides the lowest latency but the weakest
durability guarantees (some data will be lost when a server fails).
## "leader" : the producer gets an acknowledgement after the leader
replica has received the data. This option provides better durability
as the client waits until the server acknowledges the request as
successful (only messages that were written to the now-dead leader but
not yet replicated will be lost).
## "leader_and_replicas" : the producer gets an acknowledgement after
all in-sync replicas have received the data. This option provides the
best durability, we guarantee that no messages will be lost as long as
at least one in sync replica remains.
# required_acks = "leader_and_replicas"
## The total number of times to retry sending a message
# max_retry = "3"
if i understand the prometheus data model correctly, the current output
for this plugin is unusable
prometheus only accepts a single value per measurement. prior to this change, the range loop
causes a measurement to end up w/ a random value
for instance:
net,dc=sjc1,grp_dashboard=1,grp_home=1,grp_hwy_fetcher=1,grp_web_admin=1,host=sjc1-b4-8,hw=app,interface=docker0,state=live
bytes_recv=477596i,bytes_sent=152963303i,drop_in=0i,drop_out=0i,err_in=0i,err_out=0i,packets_recv=7231i,packets_sent=11460i
1457121990003778992
this 'net' measurent would have all it's tags copied to prometheus
labels, but any of 152963303, or 0, or 7231 as a value for
'net' depending on which field is last in the map iteration
this change expands the fields into new measurements by appending
the field name to the influxdb measurement name.
ie, the above example results with 'net' dropped and new measurements
to take it's place:
net_bytes_recv
net_bytes_sent
net_drop_in
net_err_in
net_packets_recv
net_packets_sent
i hope this can be merged, i love telegraf's composability of tags and
filtering
This plugin is intended to add an extended support of Postgresql
compared to the legacy postgres plugin.
Basically, the plugin don’t have any metrics defined and it’s up to the
user to define it in the telegraph.conf (as a toml structure).
Each query can have it’s specific tags, and can be written specifically
using a where clause in order to eventually filter per database name.
To be more generic, a minimum postgresql version has been defined per
query in case you have 2 different version of Postgresql running on the
same host.
Hi @sparrc
(Sorry for the noise - new pr)
closes#853
Please find some improvements to readability including the \n for the exec/telegraf line-protocol input.
I hope you (and others) find it easier to read.
/Mark
This is an ammend
added a plugin option zookeeper_chroot to set up the kafka endpoint in zookeeper, which may not be / (default).
This chroot is then configured in the consumergroup config.Zookeeper.Chroot
This is workaround the fact that this plugins does not handle the urls like "zookeeper_server:port/chroot"
As the peers are stored in an array, it makes no sens to have them beeing URL. Peers should all be members of the same cluster, so they all have the same chroot.
- using 8092 as the default port because it's similar to the rest of
the TICK stack (InfluxDB, for example, uses 8083, 8086, 8088, etc.).
didn't want to use 8125 because that conflicts with statsd.
closes#758
Modern Linux has a lots of boring filesystem (tmpfs on /dev, devpts on
/dev/pts, lots of cgroup on /sys/fs/cgroup/*, ...).
* Ignore filesystem with 0 bytes (this cover cgroup, devpts and other).
* Add IgnoreFS to ignore additional FS by their type. Add tmpfs and
devtmpfs as default ignored type.
- Check and return error from NewBatchPoints to prevent runtime panic if
user provides an unparsable precision time unit in config.
- Provide correct sample config precision examples.
- Update etc/telegraf.conf precision comment.
closes#715
The plugin is able to query a Mesos master and push the metrics, a
blacklist can be configured and a timeout, it's still not used.
Added unit test, might be a good idea to have system test using docker.
this includes:
- Add Accumulator to the Start() function of service inputs
- For message consumer plugins, use the Accumulator to constantly add
metrics and make Gather a dummy function
- rework unit tests to match this new behavior.
- make "flush_buffer_when_full" a config option that defaults to true
closes#666
* Customizable 'separator' option instead of hard-coded '_'
* String values are sent as "State" instead of "Metric", preventing
Riemann from rejecting them
* Riemann service name is set to an (ugly) combination of input name &
(sorted) tags' values...this allows connecting different events for
the same input together on the Riemann side
closes#642
This constitutes a large change in how we will parse different data
formats going forward (for the plugins that support it)
This is working off @henrypfhu's changes.
CouchDB Plugin - Formatted Code
closes#652
Minor fix for CouchDB Plugin
Formatted code fix for CouchDB Plugin
CouchDB Plugin - Changed hosts to full urls
CouchDB Plugin - Formatted Code
CouchDB Plugin - Fatal commit from local fix
CouchDB Plugin - Updated test case
percentage characters in sampleConfig string so that fmt.Printf
will interpret them as literal percentage characters when
running 'telegraf.exe -sample-config'
closes#620
This will basically make the root directory a place for storing the
major telegraf interfaces, which will make telegraf's godoc looks quite
a bit nicer. And make it easier for contributors to lookup the few data
types that they actually care about.
closes#564
Extra fields describing size of all message bodies in the queue.
* message_bytes
* message_bytes_ready
* message_bytes_unacknowledged
* message_bytes_ram
* message_bytes_persistent
More information about each field:
https://www.rabbitmq.com/man/rabbitmqctl.1.man.htmlcloses#577
Also modifying the internal UDP listener/parser code to make it able to
handle higher load. The udp listener will no longer do any parsing or
string conversion. It will simply read UDP packets as bytes and put them
into a channel. The parser thread will now deal with splitting the UDP
metrics into separated strings.
This could probably be made even better by leaving everything as byte
arrays.
fixes#543
With the advent of Kafka 0.9.0+ it is possible to set up TLS client
certificate based authentication to limit access to Kafka.
Four new configuration variables are specified for setting up the
authentication. If they're not set the behavior stays the same as
before the change.
closes#541
- If we detect errors when gathering stat via socket, return those error
so it canbe appear in Telegraf log
- Improve fcgi client, also upgrade it to current version of Go at
https://golang.org/src/net/http/fcgi/fcgi.go
- Add test for unix socket and fcgi
to remotely connect but only as an extra url field.
- Allow customization of fpm status path
- Document about using of `host` in case `unixsocket` that it isn't used
- Documet upgrade for new data layout
closes#499closes#502closes#538
- Polls a set of NSQD REST endpoints and collects counters for all topics, channels, and clients
Signed-off-by: Jeff Nickoloff <jeff@allingeek.com>
closes#492
This was primarily intended to consume InfluxDB-style expvars,
particularly InfluxDB's `/debug/vars` endpoint.
That endpoint follows a structure like
```json
{
"httpd::8086": {
"name": "httpd",
"tags": {
"bind": ":8086"
},
"values": {
"pointsWrittenOK": 33756,
"queryReq": 19,
"queryRespBytes": 26973,
"req": 428,
"writeReq": 205,
"writeReqBytes": 3939161
}
}
}
```
There are an arbitrary number of top-level keys in the JSON response at
the configured URLs, and this plugin will iterate through all of their
values looking for objects with keys "name", "tags", and "values"
indicating a metric to be consumed by telegraf.
Running this on current master of InfluxDB, I am able to record nearly
the same information that is normally stored in the `_internal`
database; the only measurement missing from `_internal` is `runtime`,
which is present under the "memstats" key but does not follow the format
and so is not consumed in this plugin.
```
$ influx -database=telegraf -execute 'SHOW FIELD KEYS FROM /influxdb/'
name: influxdb_influxdb_engine
----------------------------
fieldKey
blksWrite
blksWriteBytes
blksWriteBytesC
pointsWrite
pointsWriteDedupe
name: influxdb_influxdb_httpd
---------------------------
fieldKey
pingReq
pointsWrittenOK
queryReq
queryRespBytes
req
writeReq
writeReqBytes
name: influxdb_influxdb_shard
---------------------------
fieldKey
fieldsCreate
seriesCreate
writePointsOk
writeReq
name: influxdb_influxdb_subscriber
--------------------------------
fieldKey
pointsWritten
name: influxdb_influxdb_wal
-------------------------
fieldKey
autoFlush
flushDuration
idleFlush
memSize
metaFlush
pointsFlush
pointsWrite
pointsWriteReq
seriesFlush
name: influxdb_influxdb_write
---------------------------
fieldKey
pointReq
pointReqLocal
req
subWriteOk
writeOk
```
and rearrange the order to match the index order from the CSV endpoint
add test coverage. add back wretr. remove check_status from recently added column
closes#445
And unit test it using a sample response string. This will make it
easier to see what other metrics are available to the plugin for adding
future metrics.
"Devices" can be used to specify storage devices on which stats
should be reported. "SkipSerialNumber" can be used to omit
the device serial number.
Added tests to verify the new parameters.
closes#344
- Adds a client implementation using the prometheus go_client library
that exposes metrics.
- Adds a new type of output "ServiceOutput" which follows inline with
the "ServicePlugin", adding a Stop and Start method for the service
This change also requires the newer prometheus/client_golang code, so
the prometheus plugin needed to be changed.
Added the following to Godep:
- bitbucket.org/ww/goautoneg (in github.com/common/expfmt/encode.go)
- prometheus/common/expfmt (in plugins/prometheus.go)
- github.com/prometheus/common/model (in plugins/prometheus.go)
- github.com/prometheus/procfs (in github.com/client_golang/prometheus)
- github.com/beorn7/perks/quantile (in github.com/client_golang/prometheus)
X-Github-Meta: closes#306
- Does not use the aerospike client, but sends the stats command
using the aerospike required format
- Queries available namespaces and gets stats for all of them
closes#300
- HTTP status or Socket status
- Collect those metric:
accepted conn:
listen queue:
max listen queue:
listen queue len:
idle processes:
active processes:
total processes:
max active processes:
max children reached:
slow requests:
- Tag metric with: `host` and `pool` name
Closes#255
Fixed a bug in restricting Disk reporting to specific mountpoints
Added tests for the Disk.Mountpoints option
Fixed minor bug in usage of assert for the cpu tests where expected and actual values were swapped.
Currently adding with time is broken, because InfluxDB does not support
using precision for timestamp truncation both with and without
timestamps. This will be re-enabled once we fix InfluxDB to use the
precision argument for truncation in all cases, and a "unit" argument
in the line-protocol for adding points with non-nanosecond stamps
Fixes#175
/proc/meminfo reports memory in KiloBytes and so needs a multiplier of 1024 instead of 1000.
The kernel reports in terms of pages and the proc filesystem is left shifting by 2 for 4KB pages to get KB. Since this is a binary shift, Bytes will need to shift by 10 and so get multiplied by 1024.
From the kernel code. PAGE_SHIFT = 12 for 4KB pages
"MemTotal: %8lu kB\n", K(i.totalram)
Closes#131
Patrick Hemmer
Daniel Nelson
Daniel Nelson
Vladimir S
Rajaseelan Ganeswaran
Victor Yunevich
James
Shakeel Sorathia
tjmcs
djjorjinho
mgresser
Oskar
Leandro Piccilli
Antoine Augusti
jeremydenoun
Dennis Dryden
Robpol86
vvvkamper
Cameron Sparr
François de Metz
Jack Zampolin
Charles-Henri
Cameron Sparr
Carlos
Leandro Piccilli
Priyank Trivedi
Yaron de Leeuw
Matteo Cerutti
Jérôme Vizcaino
James Gregory
ldep30
Len Smith
njwhite
John Engelman
Patrick Hemmer
Fabio Berchtold
Jonas Hahnfeld
Will Pearson
Pierre Fersing
Claudius Zingerli
acezellponce
Viet Hung Nguyen
Kebus1
Mohammad Ali Alfarra
Emil Haugbergsmyr
Kurt Mackey
YKlausz
Dominik Labuda
Mark Wolfe
Matt O'Hara
Łukasz Harasimowicz
Jeff Ashton
Ken Dilley
Pierre Tessier
Steven Pall
Vincent
Tevin Jeffrey
alekseyp
Alex Sherwin
Doug Reese
Frank Stutz
Florian Klink
Jose Luis Navarro
Pieter Slabbert
Jonas Falck
Da1den
krise3k
Kishore Nallan
Anthony Arnaud
Leon Barrett
Rikaard Hosein
Patrick Hemmer
Christian Eichelmann
Nathan D Acuff
Foxlik
Guillem Jover
karech
Chris Goller
leplan73
Toni Moreno
Mike Ragalie
Sebastian Borza
Johannes Rudolph
Matteo Cerutti
Prunar
albundy83
Paulo Pires
Jonathan Chauncey
John Hu
Eric
Marko Crnic
Stian Øvrevåge
Edie Zhang
James Carr
zensqlmonitor
Łukasz Harasimowicz
lost_z
Vinh Quốc Nguyễn
Daniele Gozzi
Ross McDonald
David Moravek
David Norton
Rene Zbinden
ncohensm
aaron jheng
Phil
Denis Orlikhin
David Caldwell
Butitsnotme
Nathan Haneysmith
Yaser Alraddadi
Simon Murray
tuier
Joel "The Merciless" Meador
Robert Kánia
David Bayendor
jsvisa
Rene Zbinden
Mariusz Brzeski
Mendelson Gusmão
Torsten Rehn
Victor Garcia
Timothée GERMAIN
Matt Jones
Nathaniel Cook
Tim Allen
Mark McKinstry
ashish
Shashank Sahni
Sebastian Borza
Andrei Burd
Kostas Botsas
Thibault Cohen
Mike Glazer
Konstantin Kulikov
Iiro Uusitalo
Adrian Moisey
kodek
Tobias Schoknecht
Meng Ye
Ali Alrahahleh
Rickard von Essen
Martin Seener
vanillahsu
robinpercy-xm
Jan Shim
Martin
Björn Lichtblau
Lukasz Jagiello
Jari Sukanen
Nick
Cyril Duez
robinpercy-xm
Klaudiusz Staniek
Jared Biel
Kane Dou
Jason Roelofs
Anthony Brodard
Zdenek Styblik
Baptiste Mille-Mathias
Maksadbek
Jörg Thalheim
Lukasz Jagiello
Pierre Fersing
Victor Garcia
G-regL
Bob Zoller
Simone Aiello
Jesse Hanley
Mika Eloranta
Hannu Valtonen
Martin Gehrke
Larry Kim
Eugene Chupriyanov
Eugene Chupriyanov
chaton78
Pascal Larin
Shahzheeb Khan
subhachandrachandra
Michele Fadda
Luke Swithenbank
Luke Swithenbank
Josh Hardy
Sergio Jimenez
Martti Rannanjärvi
Armin Wolfermann
Nikhil Bafna
Rubycut
Florent Ramière
Rudenkovk Konstantin
张光权
Adam Argo
JP
david birdsong
Chris H (CruftMaster)
Vebjorn Ljosa
Thomas Menard
HUANG Wei
Balakrishnan
Marcelo Salazar
Eugene Dementiev
Thibault Cohen
marknmel
Prune Sebastien THOMAS
Matt Morrison
Arthur Deschamps
Manuel Sangoi
Andrea Leopardi
Aleksei Magusev and Andrea Leopardi
bastard
Matt Heath
Aurélien DEHAY
Marcin Jasion
Jason Coene
Dragostin Yanev (netixen)
Gabriel Levine
Andrei Burd
Vladislav Shub
Thomas Menard
Mike Tonks
Dhruv Bansal
Henry Hu
codehate
Reginaldo Sousa
Rune Darrud
Christoph Wegener
Pavel Yudin
Marcin Bunsch
Andrea Sosso
Stephen Kwong
Jeff Nickoloff
Matt Davis
Philip Silva
Mark Rushakoff
chrispeterson
Allen Petersen
Carlos J. Torres
Mischa Gresser
Regan Kuchan
Daniel Malon
Tait Clarridge
Tero Marttila
Eduard Carreras
Codeb Fan
鲁晓敏
Nicholas Katsaros
Roman Statsevich
Subhachandra Chandra
saiello
Simone Aiello
cornerot
JP
palkan
Tyler Nisonoff
Jonathan Cross
Shirou WAKAYAMA
Ranjib Dey
Josh Palay
Ruslan Islamgaliev
Oliver Buschjost
Cédric Menassa
Kevin Bouwkamp
Kevin Bouwkamp
nickscript0
Alexander Oleinik
Bruno Bigras
Simon Fraser
Alvaro Morales
mocchira
Graham Floyd
Todd Persen
Vinh
Evgeny Kulev
Evan Phoenix
Ian Unruh
Vadim Tkachenko
Loïc
Orne Brocaar