- html - 出于某种原因,IE8 对我的 Sass 文件中继承的 html5 CSS 不友好?
- JMeter 在响应断言中使用 span 标签的问题
- html - 在 :hover and :active? 上具有不同效果的 CSS 动画
- html - 相对于居中的 html 内容固定的 CSS 重复背景?
我有这样的 json:
{"date":"2018-12-14 00:00:44,292","service":"aaa","severity":"DEBUG","trace":"abb161a98c23fc04","span":"cd782a330dd3271b","parent":"abb161a98c23fc04","pid":"12691","thread":"http-nio-9080-exec-12","message":"{\"type\":\"Request\",\"lang\":\"pl\",\"method\":\"POST\",\"sessionId\":5200,\"ipAddress\":\"127.0.0.1\",\"username\":\"<a href="https://stackoverflow.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="78131908380f08560814" rel="noreferrer noopener nofollow">[email protected]</a>\",\"contentType\":\"null\",\"url\":\"/aaa/getTime\",\"queryString\":\"null\",\"payload\":\",}"}
问题是上面我们有:
"message":"{\"type\":\"Request\",\"lang\":\"pl\",\"method\":\"POST\",\"sessionId\":5200,\"ipAddress\":\"127.0.0.1\",\"username\":\"<a href="https://stackoverflow.com/cdn-cgi/l/email-protection" class="__cf_email__" data-cfemail="462d273606313668362a" rel="noreferrer noopener nofollow">[email protected]</a>\",\"contentType\":\"null\",\"url\":\"/aaa/getTime\",\"queryString\":\"null\",\"payload\":\",}
该应用程序以这种方式保存日志文件并且 filebeat 和logstash 没有按照我想要的方式解析它。我在 Kibana
中只看到一个名为 message
的字段,但我希望有单独的字段,例如:type
、lang
、 方法
等
我认为出现此问题的原因是 \
符号靠近 "
字符。我如何改变 filebeat/logstash 的行为来实现它?
该应用程序对我来说太大了,无法在项目 java 文件中添加任何地方 net.logstash.logback.encoder.LogstashEncoder
。
我有很多 logback-json.xml
文件。这些文件有:
<encoder class="net.logstash.logback.encoder.LoggingEventCompositeJsonEncoder">
<providers>
<pattern>
<pattern>
{
"date":"%date",
"severity": "%level",
"service": "${springAppName}",
"trace": "%X{X-B3-TraceId:-}",
"span": "%X{X-B3-SpanId:-}",
"parent": "%X{X-B3-ParentSpanId:-}",
"exportable": "%X{X-Span-Export:-}",
"pid": "${PID:-}",
"thread": "%thread",
"class": "%logger{26}",
"message": "%message",
"ex": "%ex"
}
</pattern>
</pattern>
</providers>
</encoder>
我尝试添加类似 "jsonMessage": "#asJson{%message}"的内容
这里提到:https://stackoverflow.com/a/45095983/4983983
但如果消息像之前提到的那样,我发现它无法解析并且我得到 "jsonMessage":null
在更简单的情况下我得到:"jsonMessage":{"type":"Response","payload":"2018-12-17T09:23:23.414"}
例如且不为空。
我的 filebeat 配置:
###################### Filebeat Configuration Example #########################
# This file is an example configuration file highlighting only the most common
# options. The filebeat.reference.yml file from the same directory contains all the
# supported options with more comments. You can use it as a reference.
#
# You can find the full configuration reference here:
# https://www.elastic.co/guide/en/beats/filebeat/index.html
# For more available modules and options, please see the filebeat.reference.yml sample
# configuration file.
#=========================== Filebeat inputs =============================
filebeat.inputs:
# Each - is an input. Most options can be set at the input level, so
# you can use different inputs for various configurations.
# Below are the input specific configurations.
- type: log
# Change to true to enable this input configuration.
enabled: true
# Paths that should be crawled and fetched. Glob based paths.
paths:
- /opt/tomcat-gw/logs/*.json
- /opt/tomcat-bo/logs/*.json
- /opt/tomcat-gw/logs/localhost_access_log*.txt
- /opt/tomcat-bo/logs/localhost_access_log*.txt
json:
message_key: event
keys_under_root: true
# - /var/log/*.log
#- c:\programdata\elasticsearch\logs\*
# Exclude lines. A list of regular expressions to match. It drops the lines that are
# matching any regular expression from the list.
#exclude_lines: ['^DBG']
# Include lines. A list of regular expressions to match. It exports the lines that are
# matching any regular expression from the list.
#include_lines: ['^ERR', '^WARN']
# Exclude files. A list of regular expressions to match. Filebeat drops the files that
# are matching any regular expression from the list. By default, no files are dropped.
#exclude_files: ['.gz$']
# Optional additional fields. These fields can be freely picked
# to add additional information to the crawled log files for filtering
#fields:
# level: debug
# review: 1
### Multiline options
multiline:
pattern: '^({|Traceback)'
negate: true
match: after
# Multiline can be used for log messages spanning multiple lines. This is common
# for Java Stack Traces or C-Line Continuation
# The regexp Pattern that has to be matched. The example pattern matches all lines starting with [
#multiline.pattern: ^\[
# Defines if the pattern set under pattern should be negated or not. Default is false.
#multiline.negate: false
# Match can be set to "after" or "before". It is used to define if lines should be append to a pattern
# that was (not) matched before or after or as long as a pattern is not matched based on negate.
# Note: After is the equivalent to previous and before is the equivalent to to next in Logstash
#multiline.match: after
#============================= Filebeat modules ===============================
filebeat.config.modules:
# Glob pattern for configuration loading
path: ${path.config}/modules.d/*.yml
# Set to true to enable config reloading
reload.enabled: false
# Period on which files under path should be checked for changes
#reload.period: 10s
#==================== Elasticsearch template setting ==========================
setup.template.settings:
index.number_of_shards: 3
#index.codec: best_compression
#_source.enabled: false
#================================ General =====================================
# The name of the shipper that publishes the network data. It can be used to group
# all the transactions sent by a single shipper in the web interface.
#name:
# The tags of the shipper are included in their own field with each
# transaction published.
#tags: ["service-X", "web-tier"]
# Optional fields that you can specify to add additional information to the
# output.
#fields:
# env: staging
#============================== Dashboards =====================================
# These settings control loading the sample dashboards to the Kibana index. Loading
# the dashboards is disabled by default and can be enabled either by setting the
# options here, or by using the `-setup` CLI flag or the `setup` command.
#setup.dashboards.enabled: false
# The URL from where to download the dashboards archive. By default this URL
# has a value which is computed based on the Beat name and version. For released
# versions, this URL points to the dashboard archive on the artifacts.elastic.co
# website.
#setup.dashboards.url:
#============================== Kibana =====================================
# Starting with Beats version 6.0.0, the dashboards are loaded via the Kibana API.
# This requires a Kibana endpoint configuration.
setup.kibana:
# Kibana Host
# Scheme and port can be left out and will be set to the default (http and 5601)
# In case you specify and additional path, the scheme is required: http://localhost:5601/path
# IPv6 addresses should always be defined as: https://[2001:db8::1]:5601
host: "hiddenIp:5602"
# Kibana Space ID
# ID of the Kibana Space into which the dashboards should be loaded. By default,
# the Default Space will be used.
#space.id:
#============================= Elastic Cloud ==================================
# These settings simplify using filebeat with the Elastic Cloud (https://cloud.elastic.co/).
# The cloud.id setting overwrites the `output.elasticsearch.hosts` and
# `setup.kibana.host` options.
# You can find the `cloud.id` in the Elastic Cloud web UI.
#cloud.id:
# The cloud.auth setting overwrites the `output.elasticsearch.username` and
# `output.elasticsearch.password` settings. The format is `<user>:<pass>`.
#cloud.auth:
#================================ Outputs =====================================
# Configure what output to use when sending the data collected by the beat.
#-------------------------- Elasticsearch output ------------------------------
#output.elasticsearch:
# Array of hosts to connect to.
# hosts: ["localhost:9200"]
# Optional protocol and basic auth credentials.
#protocol: "https"
#username: "elastic"
#password: "changeme"
#----------------------------- Logstash output --------------------------------
output.logstash:
# The Logstash hosts
hosts: ["hiddenIp:5044"]
# Optional SSL. By default is off.
# List of root certificates for HTTPS server verifications
#ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]
# Certificate for SSL client authentication
#ssl.certificate: "/etc/pki/client/cert.pem"
# Client Certificate Key
#ssl.key: "/etc/pki/client/cert.key"
#================================ Procesors =====================================
# Configure processors to enhance or manipulate events generated by the beat.
processors:
- add_host_metadata: ~
- add_cloud_metadata: ~
#================================ Logging =====================================
# Sets log level. The default log level is info.
# Available log levels are: error, warning, info, debug
#logging.level: debug
# At debug level, you can selectively enable logging only for some components.
# To enable all selectors use ["*"]. Examples of other selectors are "beat",
# "publish", "service".
#logging.selectors: ["*"]
#============================== Xpack Monitoring ===============================
# filebeat can export internal metrics to a central Elasticsearch monitoring
# cluster. This requires xpack monitoring to be enabled in Elasticsearch. The
# reporting is disabled by default.
# Set to true to enable the monitoring reporter.
#xpack.monitoring.enabled: false
# Uncomment to send the metrics to Elasticsearch. Most settings from the
# Elasticsearch output are accepted here as well. Any setting that is not set is
# automatically inherited from the Elasticsearch output configuration, so if you
# have the Elasticsearch output configured, you can simply uncomment the
# following line.
#xpack.monitoring.elasticsearch:
最佳答案
我写了下面的代码,如果我用这个文件启动logstash,那么我可以在kibana中看到正确的json。
input {
file {
path => "C:/Temp/logFile.log"
start_position => "beginning"
}
}
filter {
json{
source => "message"
target => "parsedJson"
}
}
output {
elasticsearch {
hosts => "localhost:9200"
index => "demo"
document_type => "demo"
}
stdout { }
}
请引用 Kibana 图片 引用自:Reference
关于elasticsearch - ELK 将 json 字段解析为单独的字段,我们在Stack Overflow上找到一个类似的问题: https://stackoverflow.com/questions/53810267/
在有些场景下,我们需要对我们的varchar类型的字段做修改,而修改的结果为两个字段的拼接或者一个字段+字符串的拼接。 如下所示,我们希望将xx_role表中的name修改为name+id。
SELECT incMonth as Month, SUM( IF(item_type IN('typ1', 'typ2') AND incMonth = Month, 1, 0 ) )AS
我最近读到 volatile 字段是线程安全的,因为 When we use volatile keyword with a variable, all the threads read its va
我在一些模型中添加了一个 UUID 字段,然后使用 South 进行了迁移。我创建的任何新对象都正确填充了 UUID 字段。但是,我所有旧数据的 UUID 字段为空。 有没有办法为现有数据填充 UUI
刚刚将我的网站从 mysql_ 更新为 mysqli,并破坏了之前正常运行的查询。 我试图从旋转中提取 id,因为它每次都会增加 1,但我不断获取玩家 id,有人可以告诉我我做错了什么吗?我尝试了将
我在 Mac OS X 上使用带有 Sequel Pro 的 MySQL。我想将一个表中的一个字段(即名为“GAME_DY”的列)复制到另一个名为“DAY_ID”的表的空字段中。两个表都是同一数据库的
问题: 是否有可能有一个字段被 JPA 保留但被序列化跳过? 可以实现相反的效果(JPA 跳过字段而序列化则不会),如果使用此功能,那么相反的操作肯定会很有用。 类似这样的事情: @Entity cl
假设我有一个名为“dp”的表 Year | Month | Payment| Payer_ID | Payment_Recipient | 2008/2009 | July
我将尝试通过我的 Raspberry Pi 接入点保证一些 QoS。 开始之前,我先动手:我阅读了有关 tcp、udp 和 ip header 的内容。在IP header description我看
如果你能弄清楚如何重命名这个问题,我愿意接受建议。 在 Dart 语言中,可以编写一个带有 final 字段的类。这些是只能设置的字段构造函数前 body 跑。这可以在声明中(通常用于类中的静态常量)
你怎么样? 我有两个带有两个字段的日期选择器 我希望当用户选择 (From) 时,第二个字段 (TO) 将是 next day 。比如 booking.com 例如:当用户选择From 01-01-2
我想我已经看到了这个问题的一些答案,这些答案可能与我需要的相差不远,但我对 mysql 的了解还不够确定,所以我会根据我的具体情况提出问题。 我有一个包含多个表的数据库,为此,如果“image”表上的
我在 mySQL 数据库中有 2 个表: customers ============ customer_id (1, 2 ) customer_name (john, mark) orders ==
我正在开发一个员工目标 Web 应用程序。 领导/经理在与团队成员讨论后为他们设定目标。这是一年/半年/季度,具体取决于组织遵循的评估周期。 现在的问题是添加基于时间段的字段或存档上一季度/年度数据的
我正在寻找允许内容编辑器从媒体库中选择多个文件的东西,这些文件将在渲染中列出。他们还需要能够上传文件和搜索。它必须在页面编辑器(版本 8 中称为体验编辑器)中工作。 到目前为止我所考虑的: 一堆文件字
现在,我有以下由 original.df %.% group_by(Category) %.% tally() %.% arrange(desc(n)) 创建的 data.frame。 DF 5),
我想知道是否有一些步骤/解决方案可以处理错误消息并将它们放入 Pentaho 工具中的某个字符串或字段中?例如,如果连接到数据库时发生某些错误,则将该消息从登录到字符串/字段。 最佳答案 我们在作业的
如何制作像短信应用程序一样的“收件人”字段?例如,右侧有一个“+”按钮,当添加某人时,名称将突出显示并可单击,如圆角矩形等。有没有内置的框架? 最佳答案 不,但请参阅 Three20 的 TTMess
是否可以获取记录的元素或字段的列表 通过类型信息类似于类的已发布属性的列表吗? 谢谢 ! 最佳答案 取决于您的delphi版本,如果您使用的是delphi 2010或更高版本,则可以使用“新rtti”
我正在构建一个 SQLite 数据库来保存我的房地产经纪人的列表。我已经能够使用外键来识别每个代理的列表,但我想在每个代理的记录中创建一个列表;从代理商和列表之间的一对一关系转变为一对多关系。 看这里
我是一名优秀的程序员,十分优秀!