deprecate udp_listener & tcp_listener

This commit is contained in:
Cameron Sparr 2017-02-02 17:06:41 +00:00
parent 694955c87b
commit c8cc01ba6a
No known key found for this signature in database
GPG Key ID: 19E67263DCB25D0F
7 changed files with 142 additions and 147 deletions

View File

@ -27,8 +27,15 @@ The previous riemann output will still be available using
`outputs.riemann_legacy` if needed, but that will eventually be deprecated. `outputs.riemann_legacy` if needed, but that will eventually be deprecated.
It is highly recommended that all users migrate to the new riemann output plugin. It is highly recommended that all users migrate to the new riemann output plugin.
- Generic [socket_listener](./plugins/inputs/socket_listener) and
[socket_writer](./plugins/outputs/socket_writer) plugins have been implemented
for receiving and sending UDP, TCP, unix, & unix-datagram data. These plugins
will replace udp_listener and tcp_listener, which are still available but will
be deprecated eventually.
### Features ### Features
- [#2094](https://github.com/influxdata/telegraf/pull/2094): Add generic socket listener & writer.
- [#2204](https://github.com/influxdata/telegraf/pull/2204): Extend http_response to support searching for a substring in response. Return 1 if found, else 0. - [#2204](https://github.com/influxdata/telegraf/pull/2204): Extend http_response to support searching for a substring in response. Return 1 if found, else 0.
- [#2137](https://github.com/influxdata/telegraf/pull/2137): Added userstats to mysql input plugin. - [#2137](https://github.com/influxdata/telegraf/pull/2137): Added userstats to mysql input plugin.
- [#2179](https://github.com/influxdata/telegraf/pull/2179): Added more InnoDB metric to MySQL plugin. - [#2179](https://github.com/influxdata/telegraf/pull/2179): Added more InnoDB metric to MySQL plugin.
@ -112,7 +119,6 @@ plugins, not just statsd.
- [#1980](https://github.com/influxdata/telegraf/issues/1980): Hide username/password from elasticsearch error log messages. - [#1980](https://github.com/influxdata/telegraf/issues/1980): Hide username/password from elasticsearch error log messages.
- [#2097](https://github.com/influxdata/telegraf/issues/2097): Configurable HTTP timeouts in Jolokia plugin - [#2097](https://github.com/influxdata/telegraf/issues/2097): Configurable HTTP timeouts in Jolokia plugin
- [#2255](https://github.com/influxdata/telegraf/pull/2255): Allow changing jolokia attribute delimiter - [#2255](https://github.com/influxdata/telegraf/pull/2255): Allow changing jolokia attribute delimiter
- [#2094](https://github.com/influxdata/telegraf/pull/2094): Add generic socket listener & writer.
### Bugfixes ### Bugfixes

View File

@ -184,8 +184,8 @@ Telegraf can also collect metrics via the following service plugins:
* [statsd](./plugins/inputs/statsd) * [statsd](./plugins/inputs/statsd)
* [socket_listener](./plugins/inputs/socket_listener) * [socket_listener](./plugins/inputs/socket_listener)
* [tail](./plugins/inputs/tail) * [tail](./plugins/inputs/tail)
* [tcp_listener](./plugins/inputs/tcp_listener) * [tcp_listener](./plugins/inputs/socket_listener)
* [udp_listener](./plugins/inputs/udp_listener) * [udp_listener](./plugins/inputs/socket_listener)
* [webhooks](./plugins/inputs/webhooks) * [webhooks](./plugins/inputs/webhooks)
* [filestack](./plugins/inputs/webhooks/filestack) * [filestack](./plugins/inputs/webhooks/filestack)
* [github](./plugins/inputs/webhooks/github) * [github](./plugins/inputs/webhooks/github)
@ -220,9 +220,11 @@ Telegraf can also collect metrics via the following service plugins:
* [nsq](./plugins/outputs/nsq) * [nsq](./plugins/outputs/nsq)
* [opentsdb](./plugins/outputs/opentsdb) * [opentsdb](./plugins/outputs/opentsdb)
* [prometheus](./plugins/outputs/prometheus_client) * [prometheus](./plugins/outputs/prometheus_client)
* [socket_writer](./plugins/outputs/socket_writer)
* [riemann](./plugins/outputs/riemann) * [riemann](./plugins/outputs/riemann)
* [riemann_legacy](./plugins/outputs/riemann_legacy) * [riemann_legacy](./plugins/outputs/riemann_legacy)
* [socket_writer](./plugins/outputs/socket_writer)
* [tcp](./plugins/outputs/socket_writer)
* [udp](./plugins/outputs/socket_writer)
## Contributing ## Contributing

View File

@ -0,0 +1,112 @@
# socket listener service input plugin
The Socket Listener is a service input plugin that listens for messages from
streaming (tcp, unix) or datagram (udp, unixgram) protocols.
The plugin expects messages in the
[Telegraf Input Data Formats](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md).
### Configuration:
This is a sample configuration for the plugin.
```toml
# Generic socket listener capable of handling multiple socket types.
[[inputs.socket_listener]]
## URL to listen on
# service_address = "tcp://:8094"
# service_address = "tcp://127.0.0.1:http"
# service_address = "tcp4://:8094"
# service_address = "tcp6://:8094"
# service_address = "tcp6://[2001:db8::1]:8094"
# service_address = "udp://:8094"
# service_address = "udp4://:8094"
# service_address = "udp6://:8094"
# service_address = "unix:///tmp/telegraf.sock"
# service_address = "unixgram:///tmp/telegraf.sock"
## Maximum number of concurrent connections.
## Only applies to stream sockets (e.g. TCP).
## 0 (default) is unlimited.
# max_connections = 1024
## Maximum socket buffer size in bytes.
## For stream sockets, once the buffer fills up, the sender will start backing up.
## For datagram sockets, once the buffer fills up, metrics will start dropping.
## Defaults to the OS default.
# read_buffer_size = 65535
## Data format to consume.
## Each data format has it's own unique set of configuration options, read
## more about them here:
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
# data_format = "influx"
```
## A Note on UDP OS Buffer Sizes
The `read_buffer_size` config option can be used to adjust the size of the socket
buffer, but this number is limited by OS settings. On Linux, `read_buffer_size`
will default to `rmem_default` and will be capped by `rmem_max`. On BSD systems,
`read_buffer_size` is capped by `maxsockbuf`, and there is no OS default
setting.
Instructions on how to adjust these OS settings are available below.
Some OSes (most notably, Linux) place very restricive limits on the performance
of UDP protocols. It is _highly_ recommended that you increase these OS limits to
at least 8MB before trying to run large amounts of UDP traffic to your instance.
8MB is just a recommendation, and can be adjusted higher.
### Linux
Check the current UDP/IP receive buffer limit & default by typing the following
commands:
```
sysctl net.core.rmem_max
sysctl net.core.rmem_default
```
If the values are less than 8388608 bytes you should add the following lines to
the /etc/sysctl.conf file:
```
net.core.rmem_max=8388608
net.core.rmem_default=8388608
```
Changes to /etc/sysctl.conf do not take effect until reboot.
To update the values immediately, type the following commands as root:
```
sysctl -w net.core.rmem_max=8388608
sysctl -w net.core.rmem_default=8388608
```
### BSD/Darwin
On BSD/Darwin systems you need to add about a 15% padding to the kernel limit
socket buffer. Meaning if you want an 8MB buffer (8388608 bytes) you need to set
the kernel limit to `8388608*1.15 = 9646900`. This is not documented anywhere but
happens
[in the kernel here.](https://github.com/freebsd/freebsd/blob/master/sys/kern/uipc_sockbuf.c#L63-L64)
Check the current UDP/IP buffer limit by typing the following command:
```
sysctl kern.ipc.maxsockbuf
```
If the value is less than 9646900 bytes you should add the following lines
to the /etc/sysctl.conf file (create it if necessary):
```
kern.ipc.maxsockbuf=9646900
```
Changes to /etc/sysctl.conf do not take effect until reboot.
To update the values immediately, type the following command as root:
```
sysctl -w kern.ipc.maxsockbuf=9646900
```

View File

@ -1,30 +1,4 @@
# TCP listener service input plugin # TCP listener service input plugin
The TCP listener is a service input plugin that listens for messages on a TCP > DEPRECATED: As of version 1.3 the TCP listener plugin has been deprecated in favor of the
socket and adds those messages to InfluxDB. > [socket_listener plugin](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/socket_listener)
The plugin expects messages in the
[Telegraf Input Data Formats](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md).
### Configuration:
This is a sample configuration for the plugin.
```toml
# Generic TCP listener
[[inputs.tcp_listener]]
## Address and port to host TCP listener on
service_address = ":8094"
## Number of TCP messages allowed to queue up. Once filled, the
## TCP listener will start dropping packets.
allowed_pending_messages = 10000
## Maximum number of concurrent TCP connections to allow
max_tcp_connections = 250
## Data format to consume.
## Each data format has it's own unique set of configuration options, read
## more about them here:
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
data_format = "influx"
```

View File

@ -58,21 +58,9 @@ var malformedwarn = "E! tcp_listener has received %d malformed packets" +
" thus far." " thus far."
const sampleConfig = ` const sampleConfig = `
## Address and port to host TCP listener on # DEPRECATED: the TCP listener plugin has been deprecated in favor of the
# service_address = ":8094" # socket_listener plugin
# see https://github.com/influxdata/telegraf/tree/master/plugins/inputs/socket_listener
## Number of TCP messages allowed to queue up. Once filled, the
## TCP listener will start dropping packets.
# allowed_pending_messages = 10000
## Maximum number of concurrent TCP connections to allow
# max_tcp_connections = 250
## Data format to consume.
## Each data format has it's own unique set of configuration options, read
## more about them here:
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
data_format = "influx"
` `
func (t *TcpListener) SampleConfig() string { func (t *TcpListener) SampleConfig() string {
@ -98,6 +86,10 @@ func (t *TcpListener) Start(acc telegraf.Accumulator) error {
t.Lock() t.Lock()
defer t.Unlock() defer t.Unlock()
log.Println("W! DEPRECATED: the TCP listener plugin has been deprecated " +
"in favor of the socket_listener plugin " +
"(https://github.com/influxdata/telegraf/tree/master/plugins/inputs/socket_listener)")
tags := map[string]string{ tags := map[string]string{
"address": t.ServiceAddress, "address": t.ServiceAddress,
} }

View File

@ -1,86 +1,4 @@
# UDP listener service input plugin # UDP listener service input plugin
The UDP listener is a service input plugin that listens for messages on a UDP > DEPRECATED: As of version 1.3 the UDP listener plugin has been deprecated in favor of the
socket and adds those messages to InfluxDB. > [socket_listener plugin](https://github.com/influxdata/telegraf/tree/master/plugins/inputs/socket_listener)
The plugin expects messages in the
[Telegraf Input Data Formats](https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md).
### Configuration:
This is a sample configuration for the plugin.
```toml
[[inputs.udp_listener]]
## Address and port to host UDP listener on
service_address = ":8092"
## Number of UDP messages allowed to queue up. Once filled, the
## UDP listener will start dropping packets.
allowed_pending_messages = 10000
## Data format to consume.
## Each data format has it's own unique set of configuration options, read
## more about them here:
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
data_format = "influx"
```
## A Note on UDP OS Buffer Sizes
Some OSes (most notably, Linux) place very restricive limits on the performance
of UDP protocols. It is _highly_ recommended that you increase these OS limits to
at least 8MB before trying to run large amounts of UDP traffic to your instance.
8MB is just a recommendation, and can be adjusted higher.
### Linux
Check the current UDP/IP receive buffer limit & default by typing the following
commands:
```
sysctl net.core.rmem_max
sysctl net.core.rmem_default
```
If the values are less than 8388608 bytes you should add the following lines to
the /etc/sysctl.conf file:
```
net.core.rmem_max=8388608
net.core.rmem_default=8388608
```
Changes to /etc/sysctl.conf do not take effect until reboot.
To update the values immediately, type the following commands as root:
```
sysctl -w net.core.rmem_max=8388608
sysctl -w net.core.rmem_default=8388608
```
### BSD/Darwin
On BSD/Darwin systems you need to add about a 15% padding to the kernel limit
socket buffer. Meaning if you want an 8MB buffer (8388608 bytes) you need to set
the kernel limit to `8388608*1.15 = 9646900`. This is not documented anywhere but
happens
[in the kernel here.](https://github.com/freebsd/freebsd/blob/master/sys/kern/uipc_sockbuf.c#L63-L64)
Check the current UDP/IP buffer limit by typing the following command:
```
sysctl kern.ipc.maxsockbuf
```
If the value is less than 9646900 bytes you should add the following lines
to the /etc/sysctl.conf file (create it if necessary):
```
kern.ipc.maxsockbuf=9646900
```
Changes to /etc/sysctl.conf do not take effect until reboot.
To update the values immediately, type the following commands as root:
```
sysctl -w kern.ipc.maxsockbuf=9646900
```

View File

@ -66,22 +66,9 @@ var malformedwarn = "E! udp_listener has received %d malformed packets" +
" thus far." " thus far."
const sampleConfig = ` const sampleConfig = `
## Address and port to host UDP listener on # DEPRECATED: the TCP listener plugin has been deprecated in favor of the
# service_address = ":8092" # socket_listener plugin
# see https://github.com/influxdata/telegraf/tree/master/plugins/inputs/socket_listener
## Number of UDP messages allowed to queue up. Once filled, the
## UDP listener will start dropping packets.
# allowed_pending_messages = 10000
## Set the buffer size of the UDP connection outside of OS default (in bytes)
## If set to 0, take OS default
udp_buffer_size = 16777216
## Data format to consume.
## Each data format has it's own unique set of configuration options, read
## more about them here:
## https://github.com/influxdata/telegraf/blob/master/docs/DATA_FORMATS_INPUT.md
data_format = "influx"
` `
func (u *UdpListener) SampleConfig() string { func (u *UdpListener) SampleConfig() string {
@ -106,6 +93,10 @@ func (u *UdpListener) Start(acc telegraf.Accumulator) error {
u.Lock() u.Lock()
defer u.Unlock() defer u.Unlock()
log.Println("W! DEPRECATED: the UDP listener plugin has been deprecated " +
"in favor of the socket_listener plugin " +
"(https://github.com/influxdata/telegraf/tree/master/plugins/inputs/socket_listener)")
tags := map[string]string{ tags := map[string]string{
"address": u.ServiceAddress, "address": u.ServiceAddress,
} }