xpath - 在Logstash上使用Xpath解析XML

Question

我是XML解析的新手，我正在尝试提取包含具有多个XML日志的多个文件的文件夹/我正在使用filebeat将日志推送到logstash中。

这是我的LOGSTASH conf：

  input {
  beats {
     port => 5044
   }
 }
 filter{
   xml{
    target => "doc"
    store_xml => false
    source => "message"
    xpath =>
    ["/E2ETraceEvent/System/EventID/@EventID", "event_id",
     "/E2ETraceEvent/System/Type/@Type", "type",
     "/E2ETraceEvent/System/SubType/@SubType", "name",
     "/E2ETraceEvent/System/Level/@Level", "level",
     "/E2ETraceEvent/System/TimeCreated/@TimeCreated", "time"]
 }
 }
 output {
 stdout { codec => rubydebug }
   elasticsearch {
     hosts => "100.101.15.181:9200"
     manage_template => false
     index => "t11-%{+YYYY.MM}"
   }
 }

这是一行XML日志的示例：

<E2ETraceEvent xmlns="http://schemas.microsoft.com/2004/06/E2ETraceEvent"><System xmlns="http://schemas.microsoft.com/2004/06/windows/eventlog/system"><EventID>0</EventID><Type>3</Type><SubType Name="Information">0</SubType><Level>8</Level><TimeCreated SystemTime="2018-04-30T14:10:24.9708868Z" /><Source Name="All Events" /><Correlation ActivityID="{00000000-0000-0000-0000-000000000000}" /><Execution ProcessName="w3wp" ProcessID="5840" ThreadID="6" /><Channel/><Computer>T-XXXX</Computer></System><ApplicationData><TraceData><DataItem>Timestamp: 4/30/2018 10:10:24 AM&#xD;&#xA;Message: T-XXX &#xD;&#xA;Method: XXX.Common.Config.Config:System.Object GetConfigFromDB(System.String, System.String, System.String) &#xD;&#xA;XXXX.Common.Config.Config:GetConfigFromDB: Done reading key:ClientAccountBaseURL for ApplicatioName:XXXX.DataLayer.AppDomainKey. value:https://test.XXXX.com/account&#xD;&#xA;&#xD;&#xA;Category: Information&#xD;&#xA;Priority: 8&#xD;&#xA;EventId: 0&#xD;&#xA;Severity: Information&#xD;&#xA;Title:T-XXXX &#xD;&#xA;Method: XXXX.Common.Config.Config:System.Object GetConfigFromDB(System.String, System.String, System.String) &#xD;&#xA;XXXX.Common.Config.Config:GetConfigFromDB: Done reading key:ClientAccountBaseURL for ApplicatioName:XXXX.DataLayer.AppDomainKey. value:https://test.XXXX.com/account&#xD;&#xA;&#xD;&#xA;Machine: T-XXXX&#xD;&#xA;App Domain: /YY/GGGG/1/ROOT/UnlistedUpgrade-1-131695710240192502&#xD;&#xA;ProcessId: 5840&#xD;&#xA;Process Name: c:\windows\system32\inetsrv\w3wp.exe&#xD;&#xA;Thread Name: &#xD;&#xA;Win32 ThreadId:4760&#xD;&#xA;Extended Properties: </DataItem></TraceData></ApplicationData></E2ETraceEvent>

在我的conf中，由于复杂性，我尝试仅解析几列
日志，我需要解析所有日志数据。

我有几天了，在logstash日志或filebeat上没有任何错误
但没有数据插入ELASTIC。

有人可以帮助我了解如何解决吗？

提前致谢。

Answer 1

尝试：

   xml{
    target => "doc"
    store_xml => false
    source => "message"
    xpath =>
    ["//EventID/text()", "event_id",
     "//Type/text()", "type",
     "//SubType/text()", "name",
     "//Level/text()", "level",
     "//TimeCreated/@SystemTime", "time"]
 }