elasticsearch - 格式化elasticsearch结果

Question

我正在使用Elasticsearch检索一些日志:

http:/ localhost:9200 / collection / _search？q = type:“log”

它给我带来了一些像这样的命中:

        {
                "_index": "collection",
                "_type": "doc",
                "_id": "UL878GMBYKUUOvfyQJWl",
                "_score": 6.487114,
                "_source": {
                    "@version": "1",
                    "type": "log",
                    "message": "64.242.88.10;[07/Mar/2004:16:11:58 -0800];"GET /twiki/bin/view/TWiki/WikiSyntax HTTP/1.1\"; 200 7352\r",
                    "@timestamp": "2018-06-11T19:03:23.163Z",
                    "host": "logstash",
                    "path": "/opt/access_log.log"
                }
            }

每个匹配都有一个“消息”，就像CSV中的一行“access_log.log”一样。

但是，每个有用的信息都只是一个大字符串，而在“消息”中。因此，我需要提取某种方式来标识服务器IP(64.242.88.10)。

如何使用“;”分割此“消息”字符串作为正则表达式，以便我只能获取所需的数据？

Answer 1

您可以为此使用grok filter plugin。

Grok is a great way to parse unstructured log data into something structured and queryable.

This tool is perfect for syslog logs, apache and other webserver logs, mysql logs, and in general, any log format that is generally written for humans and not computer consumption.

Logstash ships with about 120 patterns by default. You can find them here: https://github.com/logstash-plugins/logstash-patterns-core/tree/master/patterns. You can add your own trivially. (See the patterns_dir setting)

If you need help building patterns to match your logs, you will find the http://grokdebug.herokuapp.com and http://grokconstructor.appspot.com/ applications quite useful!