dom
/
telegraf
1
0
Fork 0
Code Issues Pull Requests Releases Wiki Activity

Remove references to logparser from the grok documentation

This commit is contained in:
Daniel Nelson 2018-08-10 18:11:12 -07:00
parent 2a4267ed72
commit 22b3bc4f8e
No known key found for this signature in database
GPG Key ID: CAAD59C9444F6155
1 changed files with 26 additions and 32 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

58
docs/DATA_FORMATS_INPUT.md
View File

@ -671,7 +671,7 @@ which are available here:
#### Grok Configuration: #### Grok Configuration:
```toml ```toml
[[inputs.reader]] [[inputs.file]]
## Files to parse each interval. ## Files to parse each interval.
## These accept standard unix glob matching rules, but with the addition of ## These accept standard unix glob matching rules, but with the addition of
## ** as a "super asterisk". ie: ## ** as a "super asterisk". ie:
@ -688,7 +688,7 @@ which are available here:
## This is a list of patterns to check the given log file(s) for. ## This is a list of patterns to check the given log file(s) for.
## Note that adding patterns here increases processing time. The most ## Note that adding patterns here increases processing time. The most
## efficient configuration is to have one pattern per logparser. ## efficient configuration is to have one pattern.
## Other common built-in patterns are: ## Other common built-in patterns are:
## %{COMMON_LOG_FORMAT} (plain apache & nginx access logs) ## %{COMMON_LOG_FORMAT} (plain apache & nginx access logs)
## %{COMBINED_LOG_FORMAT} (access logs + referrer & agent) ## %{COMBINED_LOG_FORMAT} (access logs + referrer & agent)
@ -713,8 +713,8 @@ which are available here:
grok_timezone = "Canada/Eastern" grok_timezone = "Canada/Eastern"
``` ```
The Telegraf grok parser uses a slightly modified version of logstash "grok" The grok parser uses a slightly modified version of logstash "grok"
patterns, with the format patterns, with the format:
``` ```
%{<capture_syntax>[:<semantic_name>][:<modifier>]} %{<capture_syntax>[:<semantic_name>][:<modifier>]}
@ -781,9 +781,8 @@ This example input and config parses a file using a custom timestamp conversion:
``` ```
```toml ```toml
[[inputs.logparser]] [[inputs.file]]
[inputs.logparser.grok] grok_patterns = ['%{TIMESTAMP_ISO8601:timestamp:ts-"2006-01-02 15:04:05"} value=%{NUMBER:value:int}']
patterns = ['%{TIMESTAMP_ISO8601:timestamp:ts-"2006-01-02 15:04:05"} value=%{NUMBER:value:int}']
``` ```
This example input and config parses a file using a timestamp in unix time: This example input and config parses a file using a timestamp in unix time:
@ -794,9 +793,8 @@ This example input and config parses a file using a timestamp in unix time:
``` ```
```toml ```toml
[[inputs.logparser]] [[inputs.file]]
[inputs.logparser.grok] grok_patterns = ['%{NUMBER:timestamp:ts-epoch} value=%{NUMBER:value:int}']
patterns = ['%{NUMBER:timestamp:ts-epoch} value=%{NUMBER:value:int}']
``` ```
This example parses a file using a built-in conversion and a custom pattern: This example parses a file using a built-in conversion and a custom pattern:
@ -806,20 +804,19 @@ Wed Apr 12 13:10:34 PST 2017 value=42
``` ```
```toml ```toml
[[inputs.logparser]] [[inputs.file]]
[inputs.logparser.grok] grok_patterns = ["%{TS_UNIX:timestamp:ts-unix} value=%{NUMBER:value:int}"]
patterns = ["%{TS_UNIX:timestamp:ts-unix} value=%{NUMBER:value:int}"] grok_custom_patterns = '''
custom_patterns = ''' TS_UNIX %{DAY} %{MONTH} %{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND} %{TZ} %{YEAR}
TS_UNIX %{DAY} %{MONTH} %{MONTHDAY} %{HOUR}:%{MINUTE}:%{SECOND} %{TZ} %{YEAR} '''
'''
``` ```
For cases where the timestamp itself is without offset, the `timezone` config var is available For cases where the timestamp itself is without offset, the `timezone` config var is available
to denote an offset. By default (with `timezone` either omit, blank or set to `"UTC"`), the times to denote an offset. By default (with `timezone` either omit, blank or set to `"UTC"`), the times
are processed as if in the UTC timezone. If specified as `timezone = "Local"`, the timestamp are processed as if in the UTC timezone. If specified as `timezone = "Local"`, the timestamp
will be processed based on the current machine timezone configuration. Lastly, if using a will be processed based on the current machine timezone configuration. Lastly, if using a
timezone from the list of Unix [timezones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones), the logparser grok will attempt to offset timezone from the list of Unix [timezones](https://en.wikipedia.org/wiki/List_of_tz_database_time_zones),
the timestamp accordingly. See test cases for more detailed examples. grok will offset the timestamp accordingly.
#### TOML Escaping #### TOML Escaping
@ -840,29 +837,26 @@ get a literal `|`. With a basic TOML string, special characters such as
backslash must be escaped, requiring us to escape the backslash a second time. backslash must be escaped, requiring us to escape the backslash a second time.
```toml ```toml
[[inputs.logparser]] [[inputs.file]]
[inputs.logparser.grok] grok_patterns = ["\\|%{NUMBER:value:int}\\|%{UNICODE_ESCAPE:escape}\\|'%{WORD:name}'\\|"]
patterns = ["\\|%{NUMBER:value:int}\\|%{UNICODE_ESCAPE:escape}\\|'%{WORD:name}'\\|"] grok_custom_patterns = "UNICODE_ESCAPE (?:\\\\u[0-9A-F]{4})+"
custom_patterns = "UNICODE_ESCAPE (?:\\\\u[0-9A-F]{4})+"
``` ```
We cannot use a literal TOML string for the pattern, because we cannot match a We cannot use a literal TOML string for the pattern, because we cannot match a
`'` within it. However, it works well for the custom pattern. `'` within it. However, it works well for the custom pattern.
```toml ```toml
[[inputs.logparser]] [[inputs.file]]
[inputs.logparser.grok] grok_patterns = ["\\|%{NUMBER:value:int}\\|%{UNICODE_ESCAPE:escape}\\|'%{WORD:name}'\\|"]
patterns = ["\\|%{NUMBER:value:int}\\|%{UNICODE_ESCAPE:escape}\\|'%{WORD:name}'\\|"] grok_custom_patterns = 'UNICODE_ESCAPE (?:\\u[0-9A-F]{4})+'
custom_patterns = 'UNICODE_ESCAPE (?:\\u[0-9A-F]{4})+'
``` ```
A multi-line literal string allows us to encode the pattern: A multi-line literal string allows us to encode the pattern:
```toml ```toml
[[inputs.logparser]] [[inputs.file]]
[inputs.logparser.grok] grok_patterns = ['''
patterns = [''' \|%{NUMBER:value:int}\|%{UNICODE_ESCAPE:escape}\|'%{WORD:name}'\|
\|%{NUMBER:value:int}\|%{UNICODE_ESCAPE:escape}\|'%{WORD:name}'\| ''']
'''] grok_custom_patterns = 'UNICODE_ESCAPE (?:\\u[0-9A-F]{4})+'
custom_patterns = 'UNICODE_ESCAPE (?:\\u[0-9A-F]{4})+'
``` ```
#### Tips for creating patterns #### Tips for creating patterns