csv - 当使用Logstash导入csv时，ElasticSearch抛出字符串到日期转换错误

Question

我正在尝试使用Logstash将简单的csv文件运行到ElasticSearch。

但是，当我运行它时，将字符串转换为日期格式(第一个Date列)时出现以下错误。

"error"=>{
"type"=>"mapper_parsing_exception", 
"reason"=>"failed to parse [Date]",
"caused_by"=>{
    "type"=>"illegal_argument_exception", 
    "reason"=>"Invalid format: \"Date\""}}}}

当我删除日期列时，一切正常。

我正在使用以下csv文件:

Date,Open,High,Low,Close,Volume,Adj Close
2015-04-02,125.03,125.56,124.19,125.32,32120700,125.32
2015-04-01,124.82,125.12,123.10,124.25,40359200,124.25
2015-03-31,126.09,126.49,124.36,124.43,41852400,124.43
2015-03-30,124.05,126.40,124.00,126.37,46906700,126.37

和以下logstash.conf:

input {
  file {
    path => "path/file.csv"
    type => "core2"
    start_position => "beginning"    
  }
}
filter {
  csv {
      separator => ","
      columns => ["Date","Open","High","Low","Close","Volume","Adj Close"]
  }
  mutate {convert => ["High", "float"]}
  mutate {convert => ["Open", "float"]}
  mutate {convert => ["Low", "float"]}
  mutate {convert => ["Close", "float"]}
  mutate {convert => ["Volume", "float"]}
  date {
    match => ["Date", "yyyy-MM-dd"]
    target => "Date"
  }
}
output {  
    elasticsearch {
        action => "index"
        hosts => "localhost"
        index => "stock15"
        workers => 1
    }
    stdout {}
}

似乎我在处理Date罚款。知道发生了什么问题吗？

谢谢!

Answer 1

问题出在文件本身。 Logstash正在读取第一行，并且无法解析:

Date,Open,High,Low,Close,Volume,Adj Close

不正确的解决方案是删除文件头:

2015-04-02,125.03,125.56,124.19,125.32,32120700,125.32
2015-04-01,124.82,125.12,123.10,124.25,40359200,124.25
2015-03-31,126.09,126.49,124.36,124.43,41852400,124.43
2015-03-30,124.05,126.40,124.00,126.37,46906700,126.37

而且应该没关系。

GitHub上有一个与此有关的问题: https://github.com/elastic/logstash/issues/2088